The fastest option would have been to order a 2GB stick that exactly matches the 2GB stick you already have. Then your motherboard would likely be able to use dual channel mode so that not only do you (almost) double your existing RAM to bring it up to the max supported by your operating system, but the RAM now runs twice as fast. If it's not too late to return or cancel the 4GB order for a 2GB stick that matches what you have (or a new pair of 2GB sticks), I would strongly advise it.
If you absolutely cannot get to dual channel mode, than you should just install the 4GB stick and remove the 2GB entirely - you'll be able to use just as much RAM that is just as fast without the 2GB stick in the system, and removing it will save you a little bit of electricity (heat) and allow you to sell it on eBay or use it elsewhere.
A additional point: starting with Windows 7, Microsoft doesn't distinguish between a 32bit license and a 64bit license. This means you already have a 64bit license and lack only install media to be able to use it. (Note that you still have just one license total, but that the license can be applied to either a 64bit or 32bit system). Your computer manufacturer may be able to supply you with 64bit media for a very reasonable fee (as low as $10). It's probably a little more than that, but if it's too late to change your order, it's worth checking into whether you can effectively get a 2GB+ RAM upgrade for $10.