But what I don't get is why standard system RAM has always been a generation behind the RAM used on video cards.
The GDDR specification, while based on the DDR standard, has its own hardware specification. The DDR specification is technically ahead of the GDDR specification, since GDDR is based on the previous DDR specification (most of the time, except when it's based on the previous GDDR specification).
One of the reasons there is a false belief that GDDR is ahead of DDR is that there have been multiple iterations of the GDDR standard that were based on DDR3. This was also the case with GDDR2, being that its specification has design elements from both DDR and DDR2.
However, it is important to note that this GDDR2 memory used on graphics cards is not DDR2 per se, but rather an early midpoint between DDR and DDR2 technologies. Using "DDR2" to refer to GDDR2 is a colloquial misnomer.
Source: DDR2 SDRAM
Likewise, GDDR4 and GDDR5 both took design elements from DDR3. GDDR5 obviously is an improved GDDR design, when compared to GDDR4.
Like its predecessor, GDDR4, GDDR5 is based on DDR3 SDRAM memory, which has double the data lines compared to DDR2 SDRAM. GDDR5 also uses 8-bit wide prefetch buffers similar to GDDR4 and DDR3 SDRAM.
Source: GDDR5 SDRAM
As I understand it, they're both just different types of SDRAM, but it seems to me that the differences could be abstracted away by the memory controller baked into CPU and GPU silicon.
The two standards are actually vastly different. The difference in the number of bits that can be transferred over a data line is one of those differences. The GDDR specification is not compatible with Intel and AMD x86 processors. The GDDR specification is able to transfer more bits, due to it being connected to an entirely different connection, mainly PCI-e (within the specification of the various revisions of this standard).
The current standard for system RAM is DDR4, but video cards were using GDDR4 for years before DDR4 became a thing for desktops.
This is due to the fact that GDDR4 is based off the DDR3 specification, not the DDR2 specification. The DDR3 standard wasn't ratified until 2005. We didn't see products until 2007 due to entirely different market needs. GDDR4 was announced in 2005 and didn't see products until 2007. So you can see that while they have different names, the actual products were released together.
Video cards are now shipping with HBM RAM (GDDR5?), which is faster than DDR4 system memory.
The current GDDR standard(s) are actually GDDR5X and GDDR6. HBM (High Bandwidth Memory) is a Hynix and Samsung DDR manufacturing process.
Why aren't we using the same kind of RAM for both?
The two standards are not compatible with one another.
What makes them different?
What makes them different is their manufacturing process and their specifications. While GDDR is based off the DDR specification, GDDR is not actually ahead of DDR, although there huge performance gaps between the two standards at this point due to the available bandwidth that GDDR has access to.