We waited over an hour for a table of six at the Cheesecake Factory and it only got worse later on. Lots of places to eat - just plan on waiting. Our hotel is a bit of a walk to the CC but it's walking distance to Little Italy, Fells Point, and there's a Whole Foods right there (which turned out to be our best bet for two meals a day and snacks!)
At the convention center they had limited choices: burgers and fries, pizza, chicken nuggets, hot pretzels, chips, coffee, water at $3 a bottle, some Gatorade, salads, yogurt & fruit. I heard there is a Starbucks on the 3rd floor but didn't go there.
Didn't see too many fast food places, but there is a Chipotle which we stopped at after a 8pm game.