Something I never understood, why do graphics cards usually have small amounts of memory on them? I know memory isn't everything, but surely it's pretty cheap just to load 4gb of memory on any old graphics card?
It's not cheap, per se, but it could be done, yes. It'd raise the price of the card, but it could be done.
Question is, though, why bother? If your card isn't fast enough to actually need 4GB of VRAM, it's wasted money, what's the point. You can have as many textures as you like, but unless you can actually pull that 4GB of data through for every single frame, it's wasted.
The more important reason, however, is that in a 32-bit OS, VRAM gets first dibs on the 4GB addressable space. If you have 4GB of VRAM, that leaves precisely nothing for anything else, and your machine wouldn't even boot. Bummer!
Because it's usually not your run-of-the-mill RAM, it will be high-speed, multi-port and possibly in a very compact size all of which make it more expensive for the manufacturer.
In addition to RAM price, I suspect the memory controller and cache is designed to us as few bits as possible for addresses. It might prove too slow to use more memory.
There isn't a strong market need for large memory graphics cards yet. The GPU market is driven by gamers. Even the most high end games don't really need double precision. They're fine with single precision, the textures load faster 'n such, and what's more, in single precision land, they can cram twice as much in the space it would take to store a double.
This does nothing for my CUDA programs, and I for one would love a GPU with 48 GB of GDDR5 memory -- I drool at the thought -- but the market isn't there to build such a card because the gamers and the game developers just don't use that much memory.
As a simple thought experiment, if I buy a brand new game of a single density DVD, I could almost fit the entire image within a 4GB GPU memory space (You'd need 4.7GB). And of course they're not going to do that, but again perspective. This then begs the question, if they don't need the space, why would they want to use 64-bit addressing on the card (necessary to support more memory)? Why waste the hardware and penalize the performance for the vast majority of their clientele. It doesn't make good buisness sense.
Now with the recent uptick in GPGPU especially within the research, financial, and government sectors, you will start to see niche cards developed which much more memory. But they will be slow to appear and be much more expensive. Aka Tesla