Lately I learned compute shaders in Unity3D, I could suddenly do many more calculations on the GPU per second than I could on the CPU, provided that I could write them to be tasks that run with many threads in parallel doing the same operation in a parameterized way.
However, the GPU shares memory with the screen and not the CPU, making a lag time in transferring lots of data between the CPU and GPU, plus it shares a space with your screen, which is generally a lot lower than CPU memory.
My desktop here is an i7 INUC that has 16GB memory and a 1TB SSD, while the GPU is an integrated Intel 650 with much lower memory, even though it can share some memory with the CPU.
If I want to expand the GPU, I would have to buy a Thunderbolt 3 external GPU. These are fairly pricey, I saw a Sonnet developer edition one that runs about $430 that includes an RX 580 with about 8GB of RAM, which of course still has to share memory with the screen instead of the main CPU memory.
So it makes me wonder if systems can exist that have a separate Math Processing Unit for parallel computation like a GPU that could share memory with the main CPU?
Are there perhaps systems that do things like already, or ways to expand my current system?
This would essentially leave no transfer time for operations. They used to make math co processors back in the 386 and 486 days, of course it's not exactly the same.
My concern by the way is not gaming, I was a hobby game developer at one point, but then I moved on into wanting to understand quantum physics, engineering, differential equations, and other mathematical and scientific pursuits.