Running multiple commands (or even computers) in parallel for more efficiency.

learn more… | top users | synonyms (1)

0
votes
0answers
15 views

Do threads/processes that create/delete files/directories under a shared parent dir interfere?

I was wondering: If I have a directory /a/b/c/ and under c X threads create subdirectories and files and delete them (they are temporary directories and files as intermediate steps of processing) do ...
2
votes
0answers
103 views

Jenkins realtime console output for parallel threaded fabric command

We have a Python fabric command that runs in parallel across several hosts, something like this: $ fab --hosts=prod1.server,prod2.server,prod3.server --parallel copy_cache This will copy cache to ...
0
votes
1answer
68 views

Is Virtual Machine capable of running as normal PC would?

Let's imagine that you have a super-powerful-server at home of size of desktop (just visualisation). 2048 physical cores each 3GHz, 64TB RAM, GTX 780 Ti Black Edition (SLIx4, (four graphics card ...
0
votes
2answers
56 views

fastest RAM configuration [closed]

I am planning to buy a workstation for doing heavy image processing parallel computation (for 3-dimensional electron microscopy) under linux. It will have 64 cores (4 AMD opterons of 16 cores each). ...
1
vote
0answers
40 views

What are some good options for small embedded parallel computing systems? [closed]

I've been trying to find an affordable and small system for parallel computing. I'm looking for something similar to the raspberry pi in that its a fully functional computer system. So far, the best ...
1
vote
0answers
59 views

Linux command-line utils with built-in support of multithreading

I would like to be able to compare performance of multicore computers from a point of view of a user who performs typical day-to-day tasks in Linux. My current goal is to collect a list of compute ...
0
votes
0answers
33 views

How to parallelize crontab executions to increase user base for web app based on mongodb and mysql?

I have a symfony based web application that runs on mongodb and mysql backend. The principle of the application is that for each user there is a python script that runs 4-12 times a day on cronjobs ...
0
votes
0answers
25 views

How to accelerate a shell script for a mongoDB application?

I started shell scripting for my work, but I must admit I'm still far away from even being a rookie. Therefore I wanted to ask you for your help/advice. I build a script for a big data application ...
0
votes
0answers
63 views

How to extract multi volume cpio archive in parallel

I have a large set of files. How can I create multivolume archive using cpio in such a way that I can extract them in parallel? Currently I'm using: find mydir | cpio -ov | split -d -C 3840M - ...
0
votes
0answers
22 views

Is parallelism in computer science always directly tied to the number of processor cores?

Specific to the Java programming language, would it be accurate to say that two threads can be concurrent, but can only truly be run in parallel if there are two or more cores?
0
votes
1answer
23 views

How does multi-core architecture work with regard to parellelism?

I think I am correct in saying that scheduling is the/a main principle of the single core CPU, where the CPU is interrupted when switching from processing one set of instructions belonging to one ...
-3
votes
3answers
144 views

Quad-CPU vs Dual-CPU AMD performance difference [closed]

important note: in this text - CPU is a CPU (processor, physical object) and CORE is a CORE (invisible, inside cpu)... Quad-CPU systems are expensive, and there is not so many information about them. ...
0
votes
2answers
69 views

What does MPICH stands for? [closed]

As we know that MPI is an abbreviation of Message Passing Interface. However, I am wandering about what does MPICH stands for? Is that just a brand name or does it has some terminology behind it? I ...
0
votes
0answers
43 views

Does OpenMp supports Real Time Task Computing?

OpenMP API provides a portable environment for modeling shared memory parallel applications in multi-Core platform. Since it works on the top of kernel level threads, my concern is can we use OpenMp ...
0
votes
2answers
174 views

Supercomputer via LAN [closed]

If I decide to parallel two computers via direct LAN connection would LAN (100 Mbits/s) bus bandwidth be enough for CPUs to communicate without performance loss? Or do I have to use some other bus ...
1
vote
1answer
699 views

Does a HyperThreaded Quad Core Processor Actually Have 8 “Cores” [duplicate]

I have an Early 2011 Quad Core i7 Mac. Due to hyperthreading, a lot of programs report 8 "cores". If I were to write a program that executes in parallel, could I kick off 8 instances of this program ...
0
votes
0answers
36 views

How does niceness relate to CPU allocation on Solaris?

The wikipedia article for nice describes the relationship as follows: As a simple example, when two otherwise identical CPU-bound processes are running simultaneously on a single-CPU Linux system, ...
0
votes
1answer
77 views

How to execute a script 20 at a time with different variables at a time in parallel?

I am trying to run 20 files at a time, using warc.sh or a better way to put it is 20 instances of warc.sh at a time with directory contents as the input. I am also trying to watch the PIDs in case bad ...
3
votes
4answers
557 views

Optimum number of threads while multitasking

I know similar questions have been asked but I think my case is a little bit diffrent. Let's say I have a computer with 8 cores and infinite memory with a Linux OS. I have a calculation software ...
-2
votes
1answer
104 views

GPU nvidia architecture made with CPU? [closed]

I am doing a research about the difference between CPU and GPU parallel processing. So in bref, nvidia GPU card is able to do a massive parallel processing, by using threads. each GPU can contains at ...
0
votes
0answers
46 views

How to setup grid/clutster computers?

We have 3 labs and each lab have 30 computers having different OS's on them windows, Ubuntu mostly, so i was thinking if i can connect them using grid computing, so we can use them for our research ...
4
votes
1answer
6k views

Differentiate CUDA Cores(NVIDIA) and Stream processor(ATI/AMD)

I think the question is answered here but I'm still wondering if what is the difference between the units it is referring to. For example, a GTX 570 has 480 CUDA cores, while the ATI equivalent HD ...
0
votes
1answer
32 views

losetup Find in Parallel / Concurrency

I'm writing a makefile that at some point builds image files. I'm using losetup -f in the file to find a free loop device when building out each image. Since the images are independent of each ...
1
vote
1answer
88 views

Parallel-Processing in Integrated GPU

I have an integrated GPU, meaning it works on the same RAM as my CPU. (ATI Radeon HD 4200, integrated on M4A785TD-V EVO) Does this also mean my GPU doesn't compute in parallel? see also: ...
0
votes
1answer
42 views

The exact workflow of PVM (Parallel virtual machine)

These days I am learning PVM technology for concurrent processing. I go through so many blogs, tutorials to understand . Yes, I got some points, but no one gives me clear workflow of how it perform ...
1
vote
1answer
332 views

Is it faster to run one process that spawns N threads or to run N processes?

I need to run ~1500 jobs on my lab's server. The program I am using (blastpgp if anyone cares) is capable of running in parallel mode, the "-a" option sets how many threads it will launch. The server ...
2
votes
3answers
819 views

Running threads on a non hyper-threading Intel processor

Intel Pentium Processor E5700 (2M Cache, 3.00 GHz, 800 MHz FSB) does not have hyper-threading but it has has 2 cores so I am assuming it has 2 threads Now if I write a heavy number crunching program ...
0
votes
0answers
42 views

need advice on hardware for doing parallel processing

We are intending to buy a powerful workstation for our daily processing of some seismic data. The process consists of doing numeric processing on data acquired on different seismic stations and by its ...
0
votes
1answer
320 views

PBS batch jobs - the qalter command

I've got a giant computation running on a Scientific Linux cluster. At present I have over 600 jobs parked in the queue, waiting for processor time, while a few are running. I'm trying to use the ...
5
votes
1answer
176 views

Read non-blocking from multiple fifos in parallel

I sometimes sit with a bunch of output fifos from programs that run in parallel. I would like to merge these fifos. The naïve solution is: cat fifo* > output But this requires the first fifo to ...
3
votes
2answers
387 views

How can I run several parallel shell jobs and monitor all of them at once?

I have a large number of large files (hundreds of files, hundreds of MB each) which I need to pipe through a number of programs to filter and convert them. I'm taking advantage of multiple CPU cores ...
2
votes
1answer
744 views

How do the cores in an AMD Bulldozer module compare to Intel HTT virtual cores and two separate cores, in terms of multitasking performance?

This question is basically an extension of Hyper-Threading and Dual-Core, What's the Difference? The Bulldozer-based AMD FX processors are marketed as having four, six, or eight cores, but ...
0
votes
1answer
255 views

Kerrighed cluster does not use all nodes when running OpenMP

I've been given access to a recently built cluster that uses Kerrighed. I'm the first user of this cluster, so setup and configuration problems may be a possibility. I'm also totally new to ...
1
vote
1answer
129 views

What kind of CPU/GPU integration is offered by APUs?

I'm truly fascinated by the idea of GPGPU and using the GPU for heavy processing. I'm seeing that also APUs (Accelerated Processing Units, CPU+GPU on the same chip) are gaining a consistent ...
3
votes
1answer
395 views

Does anyone know of an antivirus that can scan multiple hard drives simultaneously?

I have 5 hard drives and am so very sick of waiting for 3 hours for the virus scanner to scan each drive one at a time. I see absolutely no reason that a virus scanner can't be threaded to scan each ...
1
vote
2answers
606 views

gunzip multiple files and concatenate in parallel?

Is it possible to gunzip multiple files and concatenate them into one big file, but do it in parallel given a multicore machine? For example, right now I do: gunzip -c file1.gz > final gunzip -c ...
7
votes
5answers
8k views

Running multiple concurrent background processes then wait for all of them to terminate, on Windows

Anybody know if there is cmd script that does that and/or a command line tool ready for the job? Thanks!
2
votes
1answer
228 views

How can I share memory for parallel computing in Mathematica?

Is it possible to use RAM from other computers to assist with Mathematica Light Grid Server 7 parallel computing? If so, how?
-1
votes
1answer
207 views

MapReduce and Hadoop: what kinds of problems are they good at solving? [closed]

As I understand it already, it seems that both have similar functions (except that MapReduce is proprietary to Google, while Hadoop is open-source). I was wondering, not "how they work", but rather ...
20
votes
7answers
16k views

Parallelizing rsync

I just moved and discovered after some trial and error that somewhere between my home and my remote server, there is some throttling going on...but the throttling is not very intelligent. It only ...
1
vote
1answer
165 views

Is it normal to overworking of just one CPU when using openMP

When I use OpenMP in FORTRAN to parallelize a program on linux, ' top' command show me just one processor working ~%800 instead of ~%100 x 8 processors. Is it a expected situation ?
5
votes
4answers
4k views

What's the best way to perform a parallel copy on Unix?

I routinely have to copy the contents of a folder on a network file system to my local computer. There are many files (1000s) on the remote folder that are all relatively small but due to network ...
14
votes
5answers
15k views

Is a higher core count or higher clock speed more beneficial to a computer's performance? [closed]

With lowering silicon costs and rising consumer needs, manufacturers seem to be pushing one of two things: clock speed and/or core count. With the way things are going, it doesn't seem that clock ...
4
votes
1answer
481 views

Parallel processing slower than sequential?

EDIT: For anyone who stumbles upon this in the future: Imagemagick uses a MP library. It's faster to use available cores if they're around, but if you have parallel jobs, it's unhelpful. Do one of ...
0
votes
1answer
40 views

How to find out whether libav is built on MPI/LAM?

Just what the title states ; I expect to be doing a touch of audio cleanup/format switching. My plan is to try to do this with libav. Apparently such processing is fairly intensive; hence I would like ...
1
vote
1answer
107 views

Channelling processing power from Linux servers over a gigabit LAN to Macintosh system

As you may know Apple offers a program called Xgrid for sharing processing power between Macs. It also allows for easy clustering. Is there a way to channel processing power from Linux nodes over a ...
2
votes
1answer
226 views

Simulink - running parallel simulations

Is it possible to run Simulink calculations in parallel mode on cluster? And if its possible then what actually Simulink do when number of processes that i need is more than cluster has nodes? Thank ...
1
vote
3answers
6k views

Have today's desktop PCs surpassed IBM's Deep Blue of 1997?

I was wondering if my PC has surpassed the processing power of Deep Blue which famously won against the human world champion Garry Kasparov. What was the processing speed of Deep blue as compared to ...
2
votes
2answers
52 views

Why does F@H not bind to more than one core on Windows?

I have been contributing to Stanford's Folding@Home project for some time with most of the computers I own. I just installed the Windows client on a new machine running Windows 7, but see that the ...
5
votes
3answers
6k views

netcat as a multithread server

I use netcat to run a simple server like this: while true; do nc -l -p 2468 -e ./my_exe; done This way, anyone is able to connect to my host on port 2468 and talk with "my_exe". Unfortunately, if ...