Warning: this is a bit more complex question than you may have realized, and certainly more than most of the answers have portrayed. To try to cover at least a bit more of the story, this post is pretty long...
First of all, the situation with video memory. The entirety of the video card's memory is not (at least normally) directly visible in the CPU's address space. You usually have a "window" of about 128-256 megabytes of video memory visible to the CPU. The GPU can map different parts of its memory to that window, so you can typically write to most GPU memory from the CPU, but you need to tell the GPU ahead of time what you're going to write to before you can do so. Just for example, when you're using DirectX and you want to write directly to graphics memory, you can do that -- but to do it, you start by calling
LockRect to get the GPU to map the right memory into the window (and as soon as you're done, you call
UnlockRect so it can map other memory as needed). At least in the typical case, you can adjust the video window size in the BIOS setup.
Second, even in 32-bit mode x86 processors support what's called Physical Address Extensions (PAE), which allows the CPU to address more than 4 Gig. of memory. Microsoft does support PAE in some versions of Windows -- but only the relatively expensive Enterprise Edition of Windows Server. In addition, the program must be written specifically to use AWE. There are only a few programs (e.g., SQL Server Enterprise Edition) that have ever done so. Bottom line: it's possible, but fairly rare.
As far as reading 64 bits at a time goes: the last processor to read data only 32 bits at a time was the 486. Starting with the Pentium, all processors have had 64 bit data buses. Note that the address bus is not the same as the data bus, so expanding to a 64-bit data bus did not allow the processor to work with more memory (although, as it happens, PAE was added in the Pentium Pro, so it was almost concurrent with the expansion to a 64-bit data bus).
Even though the Pentium could read 64 bits of data in a single clock cycle, internally it (mostly) processed data 32 bits at a time. The wider bus primarily helped make up for the difference in speed between the CPU and main memory. The MMX instructions of the Pentium MMX supported processing data in 64-bit chunks, but it was rather clumsy to use and (in particular) most compilers didn't generate MMX code, so you pretty much had to write in assembly language to use it. The Pentium also supported 64-bit (and 80-bit) operations for floating point numbers, but that's used for only a fairly small minority of most programming (and was generally slower than working with integers anyway).
The x86-64 design does have twice as many general purpose registers as the 32-bit x86 design, and each of those registers is 64 bits wide. When the CPU is running in 32-bit mode, however, only the subset of the registers that existed on 32-bit CPUs is usable.
Going back to floating point for a moment, in 32-bit mode, floating point math is done using the instruction set originally designed for the Intel 8087 math coprocessor. This uses a model that makes it fairly difficult to make full use of its capabilities. Intel (and AMD) have more recently added "Streaming SIMD Extensions", or SSE. Most 64-bit compilers generate code that can/does use SSE instead of 8087-compatible floating point. The "SIMD" in SSE stands for "single instruction, multiple data". What this means is that one SSE instruction can carry out two floating point operations on two separate pieces of data. Though this doesn't necessarily double the speed, it usually improves speed at least a little bit.
Finally, I should probably point out that there's one part of the comparison that's generally a bit unfair. When you install a new OS, it's nearly always at least a little faster than an OS that's been in use for quite a while. Programs store data in the registry, on the hard drive, etc. The more data you add, the more work tends to be involved in finding the data you want at any given time. While companies selling "registry cleaners" and such frequently exaggerate this effect, it's true to some extent anyway. As such, if all you did was re-format your hard drive and re-install exactly the same OS, you could expect to see some speed improvement -- and if you've had the old OS installed and used heavily for a long time, that improvement (all by itself) can be pretty substantial. At least part of the speed improvement you're seeing may be independent of any real difference between the two OSes.
There's more than that, of course, but you start to get the idea. It isn't so much a matter of a single feature making a huge difference, as a lot of features that usually make fairly small differences individually, but many of them add (or even multiply) together to make a fairly substantial difference overall.