you can probably fly into jfk or laguardia, though laguardias closer to manhattan so it's easier a little bit...
forget about staying in nj, without a car, if you're going to yankee stadium.....it can be done but you would need public transportation to penn then the 1 train uptown for a good 30-45 mins to the stadium packed like a sardine........if you want to do nj, try hoboken or jersey city, wont be cheap but cheaper than manhattan....its only a very short ride to manhatan, than you can subway which is easy......
of course, if $ is no object find somewhere in manhattan since you're right in the action.....anywhere midtown or upper, then subway as noted (check nyc pub transp website for a map)........a 3rd option which could work too....stay in lower westchester, nice and not far from bronx, and pub transp or cab to the game......then you can take metro north train to nyc for the sightseeing also, very easy and cheap......have fun in New York, I loved visiting chicago last month in the opposide situation you're doing, saw the city and a cubs game...