Lately I learned compute shaders in Unity3D, I could suddenly do many more calculations on the GPU per second than I could on the CPU, provided that I could write them to be tasks that run with many threads in parallel doing the same operation in a parameterized way.

However, the GPU shares memory with the screen and not the CPU, making a lag time in transferring lots of data between the CPU and GPU, plus it shares a space with your screen, which is generally a lot lower than CPU memory.

My desktop here is an i7 INUC that has 16GB memory and a 1TB SSD, while the GPU is an integrated Intel 650 with much lower memory, even though it can share some memory with the CPU.

If I want to expand the GPU, I would have to buy a Thunderbolt 3 external GPU. These are fairly pricey, I saw a Sonnet developer edition one that runs about $430 that includes an RX 580 with about 8GB of RAM, which of course still has to share memory with the screen instead of the main CPU memory.

So it makes me wonder if systems can exist that have a separate Math Processing Unit for parallel computation like a GPU that could share memory with the main CPU?

Are there perhaps systems that do things like already, or ways to expand my current system?

This would essentially leave no transfer time for operations. They used to make math co processors back in the 386 and 486 days, of course it's not exactly the same.

My concern by the way is not gaming, I was a hobby game developer at one point, but then I moved on into wanting to understand quantum physics, engineering, differential equations, and other mathematical and scientific pursuits.

closed as off-topic by Ramhound, music2myear, bertieb, Toto, PeterH Oct 10 '18 at 7:05

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question is not about computer hardware or software, within the scope defined in the help center." – Ramhound, music2myear, bertieb, Toto, PeterH
If this question can be reworded to fit the rules in the help center, please edit the question.

  • This question is not about computer hardware or software, within the scope defined in the help center. – Ramhound Oct 8 '18 at 16:30
  • 1
    Sorry, John, SuperUser is not a good place for discussions. Try some forums with the same topics and you'll get better results. – Christopher Hostage Oct 8 '18 at 16:34
  • Oh! What's the best one maybe for this kind of discussion? Sorry I just woke up. – John Ernest Oct 8 '18 at 16:36
  • Usually folks are pretty friendly and helpful on stack overflow and math on stack exchange. Why is SuperUser not such a good place for discussions? Just wondering what's maybe the best forum to ask such questions out there. – John Ernest Oct 8 '18 at 17:06
  • @JohnErnest A lot of forums out there would be good for discussion, but Stack Exchange is a Q&A site so across the SE network using questions for discussions is discouraged. – dsstorefile1 Oct 8 '18 at 18:20

Yes, such systems do exist, but not in the way you're probably thinking of. Most supercomputers actually work this way , they've got a handful of what you would conventionally call CPU's, usually referred to as I/O processors, and a huge number of what you're calling math processors, which are typically called application processors.

Examples of hardware used for this purpose as application processors include:

  • Intel's MIC platform, also classically known as Xeon Phi. These are x86-based devices (most of the models are full-length double-wide PCI-e cards, though some models have been released as socketed chips) with two hundred or more threads of execution. A number of big supercomputers are primarily built out of these, though you can actually get single cards from the older generations for a few thousand USD some places online.
  • NVIDIA's Tesla platform. These originated as simple hyper-specialized variants of their equivalent Quadro GPU's, modified to have no video outputs and optimized for raw throughput of FP computations instead of rendering. These days, they're mostly full featured GPU's, but still focus on raw processing power over rendering. They're pretty readily available commercially, but have similarly high price tags.
  • AMD's FireStream platform. Similar story to NVIDIA Tesla, except FireStream essentially died out almost a decade ago. You can still find the cards some places, often for pretty cheap, but they're not very powerful by today's standards.

Various other companies have similar offerings of one for or another. IBM's Watson platform for example is a complete system built on this principal, except that the individual processors are each a functionally independent system.

  • Thank you Austin for the helpful feedback. I figured this might be only available on supercomputers, but I wasn't sure. – John Ernest Oct 9 '18 at 1:06
  • It's not really an 'only supercomputers' thing. Supercomputers are the most common users of things like this, but you can get them commercially if you're willing to throw enough money at it. It's probably worth noting that the previous generation Tesla cards have about the same processing power as the top of the line current Generation GeForce cards but cost significantly more, which is a large part of why people preferentially buy the GeForce cards. – Austin Hemmelgarn Oct 9 '18 at 11:43

Not the answer you're looking for? Browse other questions tagged or ask your own question.