Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I work on parallel algorithms and want to validate certain theoretic claims by runtime experiments. As it turns out, most (?) modern multicore CPUs use NUMA which inherently breaks many models to the point of rendering experiments meaningless.

At the same time, all non-NUMA CPUs I could find have only two cores, limiting investigations regarding scalability a lot.

Are there commercially available CPUs with many (i.e. more than two) cores that have uniform memory access? Ideally, they would also have no private caches.

share|improve this question

closed as not constructive by BBlake, Breakthrough, Mokubai, TFM, Nifle Mar 15 '13 at 19:06

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

If you want to simulate using a model, you don't need a real physical CPU with the characteristics. – David Schwartz Mar 15 '13 at 12:00
@DavidSchwartz I don't want to simluate a model, I want to validate it. It's the scientific method even if -- sadly -- uncommon in computer science. – Raphael Mar 15 '13 at 12:03
While there's a difference between real tsunami and a simulated tsunami such that one can't validate, say, a way of predicting a tsunami in a computer as well as one can with a real tsunami, there is no difference between a "real CPU" and a "CPU on a computer". "Real" CPUs are built out of the same stuff as simulated CPUs and are essentially "simulated" by their gates, microcode, and so on. – David Schwartz Mar 15 '13 at 12:09
Why would I simulate the highly interdependent tangle a modern computer is if I can just use the one standing there? Well, if you can point me to a simulator for the whole stack -- hardware and operating system -- that allows me to control many parameters and performs efficiently, we can talk. If not, I reject your proposal. (Also, note that executions on simulated machines are usually not equivalent to native execution because they abstract from low-level effects such as hardware-interrupts, branch prediction, cache misses, ...) – Raphael Mar 15 '13 at 12:24
@Breakthrough I am not simulating anything, I am benchmarking parallel algorithms. Shared memory models are rendered useless because of e.g. cache syncing. Could we please stick to the question instead of discussing whether the question should be asked? – Raphael Mar 15 '13 at 13:01

AFAICT all current single-socket computers are truly UMA. Memory at physical address x in core 0 is also at physical address x in core 1.

The OS generally creates different virtual address spaces for different processes, to protect them against each other, but offer "shared memory" if you don't want this protection. Even simpler, a multi-threaded application has the exact same virtual address space for all threads, regardless of the CPU core on which the thread runs.

You may wonder about the CPU caches, but on all modern CPU's all caches within a single socket are coherent. Even if they're physically distributed, they never contain visibly conflicting values (i.e. if two caches would contain conflicting values, at least one would know its value is stale)

share|improve this answer
NUMA has nothing to do with the actual address x, but the physical location of address x (and x+1 and so on). Of course address x and x+1 are at the same location for all processors, but they might physically exist at different locations, and thus have different access times. This is the definition of NUMA. – Breakthrough Mar 15 '13 at 12:39
I am not concerned about cache consistence in itself, but with the cost of maintaining it (among others). I want to eliminate such costs from my experience, hence my question for a "simple" CPU. – Raphael Mar 15 '13 at 13:03
@Raphael I believe some computer systems allow you to actually disable some levels of the CPU's cache in the BIOS itself. – Breakthrough Mar 15 '13 at 13:14
@Breakthrough: That depends strongly on your definition of "address x". In some NUMA architectures, different CPU's may assign the same address x to distinct words in memory. I.e. write A to address 4 on core 0, write B to address 4 on core 1, and you'd still be able to read back A from address 4 on core 0. – MSalters Mar 15 '13 at 13:43
@Raphael: There is no objective measurement of that cost. Every cache uses a cache validity model such as MSI or MESI, even in the most simple single-core setup. Cache-coherency is implemented within that model. Furthermore, how would you even measure this cost? The only real measurement would be in microwatts. You can't measure it in CPU cycles, as the work done to remain cache-coherent is done independently from the CPU cores. – MSalters Mar 15 '13 at 13:49

Azul Systems' Vega might be worth considering if UMA is that important. Quoting from Cliff Click's blog:

Azul builds are gear as ‘UMA’ because our programs do not have well understood access patterns. Instead, the patterns are mostly random (after cache filtering) and so it makes sense to have uniform mediocre speeds instead of 1/16th of memory fast and 15/16ths as slow.

However, they are Java application engines, and so might not fit your targeted use.

Private--in the sense of capacity use--caches (at least L1 caches in a single chip and other levels in multiple chips--though one IBM mainframe has an off-chip L4 shared by multiple processor chips) are effectively unavoidable.

As has been noted, for a single chip, the main memory typically has "uniform" access, so if the thread count is sufficient for your test then such could be used (though I suspect cache effects would make meaningful measurements difficult).

share|improve this answer

NUMA is used in high end server class motherboards that have multiple physical processor packages. Your typical desktop 4 or 6 core cpu is not NUMA. Caches do not count, otherwise nothing would be UMA. NUMA just refers to where the physical memory is, and that there are different length paths ( or none at all ) to it depending on which CPU you are on.

In other words, if ram bank A is connected to CPU package A, which has 6 cores, and ram bank B is connected to CPU package B, and it also has 6 cores, that's NUMA since the 6 cores in package B have to go over the bus, and through package A to get to ram bank A. A single 6 core CPU is directly connected to all of the ram, so is UMA.

share|improve this answer

Any system with more than one "level" of cache or memory technically classifies as a NUMA design, so this might be a misnomer depending on your perspective. To fully leverage any modern processor's full power, one must be considerate of the cache design & size, or consider a cache-oblivious algorithm.

Indeed, the closest thing to "shared" memory in a modern computer is the concept of L2/L3 cache, or cache lines shared between all cores on a processor die. While some servers have specific memory controllers which can handle multiple reads/writes at once, this is very rare to see in a consumer environment, and they still implement a NUMA design for higher throughput. This requires significant processing power/time to ensure the integrity of the memory contents in different segments.

Uniform memory access made sense back when computer clock speeds were relatively slow, or equal to, that of DRAM cycle times. Now, however, our computer RAM is significantly slower (paying a penalty of hundreds of clock cycles for memory accesses), thus why we segment our memory map with subsequent layers of faster and faster memories (RAM, several cache levels, and processor registers).

share|improve this answer
This does not really the answer; in particular, I don't care whether the CPU is slow in the usual sense. There can be as many caches stacked on top of each other as they want, but I need that each memory access (going to the same hierarchy level) costs the same, which NUMA does not provide. Do you intend to suggest that there is no such CPU? – Raphael Mar 15 '13 at 13:02
@Raphael there is such a CPU, but they don't exist in the x86 realm. Even tiny 8-bit microcontrollers now come with data/instruction cache. You could test your algorithm on a smaller/older one without cache and a uniform memory map, but then you would be only using a single processor. – Breakthrough Mar 15 '13 at 13:10
@Raphael: with todays RAM chips (i.e. DDR3), you cannot even guarantee that the cost of two accesses at the same main-memory hierarchy level are equal. – MSalters Mar 15 '13 at 13:52
@MSalters That would be okay if there is no structural difference among cores. Uniformly spread noise can be average out over many runs, if it affects the (qualitative) interpretation of the experiment at all. – Raphael Mar 15 '13 at 15:25

Just a guess, but I know for the CPU and GPU to access video RAM at the same time the memory must be dual ported, i.e. the RAM needs more than one connection to the things using it.

I don't know why they don't design CPUs with more than one RAM port (if that is even the correct terminology). CPU is connected to a memory controller which then does external memory operations on behalf of the CPU. I know at some point "interleaved" slots started to come into play - where the memory controller would try to iterate accesses among slots (sort of like RAID striping). But the CPU doesn't have multiple physical connections to RAM, even in this case.

Maybe really high end non-x86 architectures do have multiple RAM ports.

I do know that most if not all CPUs (through MSR's if I'm not mistaken) can be set to disable the cache. I know Linux gives access to MSRs but something tells me the kernel would not like that. Maybe an MSR can disable NUMA as well. May be getting into seriously modifying the Linux kernel, though.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.