64-bit CPUs have existed in supercomputers since the 1970s but have only recently entered the mainstream of personal computers.
The maximum memory address for 64-bit computers is 264, which is 18,446,744,073,709,551,616 bytes, or 17,179,869,184 GB. As a practical matter though, many operating systems like Windows will still allocate only 2 GB of memory to a process (or 3 GB is some cases), so it is still possible to get out of memory errors when in fact there is plenty of memory left.
The usage of 64-bit processors solves a multitude of practical issues, including the Year 2038 problem and the ability to address more than 4 GB of RAM on computers.
Keep in mind that while the OS you are using may be 64-bit, the applications you are running may still be 32-bit. This is true in Windows, Linux and OS X.