I've been to the Hall over 20 times and I make sure I include it any time I go back home- Saratoga Springs.
Take two days to get the real experience. Lots of stuff to see but more importantly lots of stuff to read. Take a moment and read the information by the memorabilia and get the inside story on the items.
I would suggest the spring time before school gets out and visit during the week so the crowds are low. If you go during the summer it gets hot, lots of people, bad parking, long lines and it can be a real strain on a relationship unless your girl is "really" into baseball!
It's a baseball city so there is little to do other than picnic type stuff. Not sure where you are staying but the big hotel is great (it's where the players stay) with a nice golf course. The lake is nice as well but I'm not sure if you're into the boating stuff.
My first two visits were during the late 1960's when the hall was just an old building with things displayed in glass cases in the middle of the floor. You went floor to floor up/down old staircases at the end of the hallways and it was just like an old office building. It's come a long way since then and as I said before it is a must stop every time I go back home!