Edit: since people are complaining that this answer does only answer the "Why" question, and does not provide a solution to the problem (which wasn't the question asked by the OP), please find the solution to the problem in the next answer https://superuser.com/a/343483
This is because you have a keyboard where
" are set to be a dead-key.
The behaviour of a dead key is ... well... to be dead! So no reaction when you press it.
A dead key is gonna be waiting for the next key to be pressed, to alter what that key is gonna print. For example
a might produce
á if the quote key is configured as a dead key.
Usually, when a keystroke has no meaning after a dead key, it defaults to printing the character of the dead key, followed by the new character that has been typed.
So you can follow advice of other posters on how to change your keyboard (look at which KB you have configured). You can also get the Microsoft Keyboard Layout Creator to map a new keyboard and see how you can program dead keys.
Update: with the keyboard editor, you can also load the current keyboard and modify the dead key setting