You're probably using the US International keyboard layout, rather than the basic US layout. The former uses the ', `, ", ^ and ~ keys as dead keys, allowing you to combine them with letters to produce characters with diacritics.
See the images below (courtesy of Wikipedia) for the differences between the two layouts.
As you can see, with the international layout you can use AltGr+? and AltGr+! for
Another option, if you have a numeric keypad, is to use Alt codes (holding down the Alt key and typing in a numeric sequence on the numpad. Alt+0191 produces
¿ and Alt+0161 produces
¡. See here for a full list of Alt codes.
Finally, you can download Microsoft Keyboard Layout Creator, create your own custom layout and map those symbols to any key combination you want.