- Would I be facing significant slowdowns if something needed to transfer data from one to the other?
Answer: No. Special buses are used in multi-CPU systems to facilitate the fast transfer of data between CPUs and their respective caches. In the case of modern Intel chips, this is called the QPI Link. The only thing special you need to be aware of is that you will have to populate at least one DIMM per CPU in order to use both CPUs.
- Two E5-2640 v3's would be a much better investment than say one E5-2690.
Answer: If you can get twi 2640 v3's for the same price as a single E5-2690, color me very impressed or else you are not looking in the right spot for 2690s. My first Ebay search turned up 2690s running about $150, while I couldn't find an E5-2640 v3 (the newer chip) for under $750. In any case, these two CPUs perform similarly, so a pair of whichever is cheaper will suit you better, all other things being equal.
- For multi-threading, a multi-cpu system's performance/score should essentially scale linearly with the score for each processor, correct?
Answer: Incorrect. There are many, many factors to consider in both the construction of the hardware and the software it runs which impinge on whether or not a multicore system can scale its performance linearly. In most real-world cases, therefore, even a perfectly-built multicore system will scale somewhat less than linearly. I'm sorry I can't be more specific than that. If I were you, dropping this much cash on your hardware, I'd spend a long time talking to people who do what you do, and looking at comparative benchmarks they recommend, before making your purchase decision.