There seems to be a lot of confusion in the industry about what's commonly called the Windows “4GB memory limit.” When talking about performance tuning and server sizing, people are quick to mention the fact that an application on a 32-bit Windows system can only access 4GB of memory. But what exactly does this mean?
By definition, a 32-bit processor uses 32 bits to refer to the location of each byte of memory. 2^32 = 4.2 billion, which means a memory address that's 32 bits long can only refer to 4.2 billion unique locations (i.e. 4 GB).
In the 32-bit Windows world, each application has its own “virtual” 4GB memory space. (This means that each application functions as if it has a flat 4GB of memory, and the system's memory manager keeps track of memory mapping, which applications are using which memory, page file management, and so on.)
This 4GB space is evenly divided into two parts, with 2GB dedicated for kernel usage, and 2GB left for application usage. Each application gets its own 2GB, but all applications have to share the same 2GB kernel space.
SOURCE: http://www.brianmadden.com/blogs/brianmadden/archive/2004/02/19/the-4gb-windows-memory-limit-what-does-it-really-mean.aspx
[Edit RE: PAE]
x86 processor hardware-architecture is augmented with additional address lines used to select the additional memory, so physical address size increases from 32 bits to 36 bits. This, theoretically, increases maximum physical memory size from 4 GB to 64 GB. The 32-bit size of the virtual address is not changed, so regular application software continues to use instructions with 32-bit addresses and (in a flat memory model) is limited to 4 gigabytes of virtual address space. The operating system uses page tables to map this 4-GB address space into the 64 GB of physical memory. The mapping is typically applied differently for each process. In this way, the extra memory is useful even though no single regular application can access it all simultaneously.