At first I was thinking to do this with some specialized motherboard form factor that uses a riser board so the video cards are parallel to the motherboard. However, it seems that no one really does that, so it might be unmaintainable or very limiting.
Another possibility is to use low-profile graphics cards, but I'm not a hardware person so I don't know what's available out there on the market (can you get a good GPU in a low-profile form factor?).
How do people usually handle this? I guess something around 3 inches would be ideal. Is that possible?
I've fully specified my problem. There is nothing "I'm trying to do" that I haven't covered above. If you want more explicit directions, then tell me an example of the best performing graphics card available in a low-profile form factor. I don't know how to shop for graphics cards very well.
It needs to be a GPU. Many algorithms these days are optimized for GPU programming. Having more CPUs isn't as helpful.
It needs to be done with commodity hardware that will be easily replaceable/upgradable. I'm obviously not a hardware person so I won't be adding any "dedicated chips". Think of it like a gaming PC that has to be slim to fit into a tight space.