I agree that the noise is more likely the AIR pump, but you might want to check the charging system if you think the whine is from the alternator.
The alternator can behave as you describe if there is a failed or failing diode in the rectifier/regulator. You ought to be able to take the car to an auto-electric shop and have the alternator tested--typically its free. What is happening is possibly that the process of starting the car draws down the battery enough that the alternator is suddenly seeing a big recharge demand, especially if there are other things being turned on at the same time--radio, heater/fan, wipers, headlights, and rear defogger. The unstable regulator is going nuts trying to make the alternator recharge the battery--in fact, OVERcharge it. This can quickly cook the battery.