I recently switched to Mac and I had the same question you did. I was advised to upgrade and, although I didn't yet, (couldn't and can't afford it,) I recommend getting more memory if you can afford it.
If you are going to be running Windows, odds are you will be better off running it through Boot Camp. (If you really have the need to run Windows, you probably will be running a powerful program that a VM can't handle.)
The key thing to note when switching is that unlike PCs, Apple computers are not made to have the internals upgraded. (Unless you have a Mac Pro, in which case you will be worrying about upgrading more than just memory.)
On my iMac, with 4GB of RAM, I've been fine, even doing video editing. On my MacBook, 2GB or memory does lag occasionally. (But then again, I routinely run five, six, or more programs at once. Mail, iChat, Safari, Firefox, Flash, XCode and then some...)
Like others here have pointed out, it's cost-prohibitive later to upgrade the memory in the MacBook Air systems, due to the way the machine is put together. You are saving on service charges etc. Also, there is always a slight risk that the person doing the upgrade will mess up. You avoid this risk by purchasing the memory now.
On the other hand, if you are going to be running only basic programs, you may want to put that same $100 or $200 towards an AppleCare Protection plan or a nice case and screen protector.
I would go with the upgrade. (If nothing else, you won't be left wondering what life would have been like with that extra two gigabytes.)