Running multiple commands (or even computers) in parallel for more efficiency.

learn more… | top users | synonyms (1)

0
votes
2answers
32 views

zero a large block device with parallelized dd

We have a sixty terabyte block device which we want to zero using dd. Is it possible to parallelize this operation in order to reduce the amount of time it takes? I've tried running the following ...
2
votes
1answer
59 views

Sort --parallel isn't parallelizing

I'm trying to make unique a set of lines pulled from a file with egrep with sort -u, then count them. About 10% of the lines(all 100 characters long from the alphabet [ATCG]) are duplicated. There are ...
2
votes
2answers
37 views

Will process run slower on one thread rather than one core

So many CPUs now come with 2/4 cores and 4/8 threads. There are also some CPUs that are 4 core & 4 threads. Would that mean that unparalleled process will run slower on the same clock cycle CPU ...
0
votes
0answers
20 views

How to implement Intel MPI benchmarks for mpich2?

Many people have recommended the use of Intel MPI benchmarks for mpich2, however the installation is not very simple. Could anyone provide steps for the installation and sample test case for it? Which ...
0
votes
0answers
14 views

Scientific Linux 5.5 hangs at the beginning

We are taking theses days a parallel processing course in college, we are working on scientific linux 5.5 version with MPI, I just imported .ova on the virtual box but unfortunately when am trying to ...
-1
votes
1answer
121 views

Is it possible to use GPU to boost performance of computer automatically? [duplicate]

As we all know GPU is a sleeping beast. And we are also familiar with Intel's Turbo Boost technology which is really great. But, Intel's Turbo Boost technology works only on CPU by overclocking CPU ...
0
votes
1answer
108 views

Clock speed vs. number of cores for parallelized computer simulation [duplicate]

My question is similar to some "speed vs. cores"-questions that have already been asked on this website. However, I am interested in some very specific and technical points. So I hope that my question ...
1
vote
1answer
44 views

How can I force gnuparallel to carry out a command set sequentially?

Gnu parallel is a powerful tool that I use to run many independant BASH commands as a single set in parallel. I would like to be able to run the same commands SEQUENTIALLY without significant changes ...
0
votes
0answers
30 views

Is it possible to configure Linux OpenMPI v1.6.5 to use multiple working directories on a single node?

I am running some quantum chemical calculations on a refurbished PowerEdge rack server, with dual quad-core Xeons, 16GB RAM, and a single 500GB SATA HDD. OS is vanilla Debian Wheezy. For one of the ...
0
votes
0answers
24 views

Do threads/processes that create/delete files/directories under a shared parent dir interfere?

I was wondering: If I have a directory /a/b/c/ and under c X threads create subdirectories and files and delete them (they are temporary directories and files as intermediate steps of processing) do ...
7
votes
0answers
404 views

Jenkins realtime console output for parallel threaded fabric command

We have a Python fabric command that runs in parallel across several hosts, something like this: $ fab --hosts=prod1.server,prod2.server,prod3.server --parallel copy_cache This will copy cache to ...
0
votes
1answer
143 views

Is Virtual Machine capable of running as normal PC would?

Let's imagine that you have a super-powerful-server at home of size of desktop (just visualisation). 2048 physical cores each 3GHz, 64TB RAM, GTX 780 Ti Black Edition (SLIx4, (four graphics card ...
0
votes
2answers
113 views

fastest RAM configuration [closed]

I am planning to buy a workstation for doing heavy image processing parallel computation (for 3-dimensional electron microscopy) under linux. It will have 64 cores (4 AMD opterons of 16 cores each). ...
1
vote
0answers
112 views

Linux command-line utils with built-in support of multithreading

I would like to be able to compare performance of multicore computers from a point of view of a user who performs typical day-to-day tasks in Linux. My current goal is to collect a list of compute ...
0
votes
0answers
26 views

Is parallelism in computer science always directly tied to the number of processor cores?

Specific to the Java programming language, would it be accurate to say that two threads can be concurrent, but can only truly be run in parallel if there are two or more cores?
0
votes
1answer
29 views

How does multi-core architecture work with regard to parellelism?

I think I am correct in saying that scheduling is the/a main principle of the single core CPU, where the CPU is interrupted when switching from processing one set of instructions belonging to one ...
-4
votes
3answers
232 views

Quad-CPU vs Dual-CPU AMD performance difference [closed]

important note: in this text - CPU is a CPU (processor, physical object) and CORE is a CORE (invisible, inside cpu)... Quad-CPU systems are expensive, and there is not so many information about them. ...
0
votes
2answers
246 views

Supercomputer via LAN [closed]

If I decide to parallel two computers via direct LAN connection would LAN (100 Mbits/s) bus bandwidth be enough for CPUs to communicate without performance loss? Or do I have to use some other bus ...
1
vote
1answer
1k views

Does a HyperThreaded Quad Core Processor Actually Have 8 “Cores” [duplicate]

I have an Early 2011 Quad Core i7 Mac. Due to hyperthreading, a lot of programs report 8 "cores". If I were to write a program that executes in parallel, could I kick off 8 instances of this program ...
0
votes
0answers
39 views

How does niceness relate to CPU allocation on Solaris?

The wikipedia article for nice describes the relationship as follows: As a simple example, when two otherwise identical CPU-bound processes are running simultaneously on a single-CPU Linux system, ...
0
votes
1answer
89 views

How to execute a script 20 at a time with different variables at a time in parallel?

I am trying to run 20 files at a time, using warc.sh or a better way to put it is 20 instances of warc.sh at a time with directory contents as the input. I am also trying to watch the PIDs in case bad ...
3
votes
4answers
1k views

Optimum number of threads while multitasking

I know similar questions have been asked but I think my case is a little bit diffrent. Let's say I have a computer with 8 cores and infinite memory with a Linux OS. I have a calculation software ...
-2
votes
1answer
145 views

GPU nvidia architecture made with CPU? [closed]

I am doing a research about the difference between CPU and GPU parallel processing. So in bref, nvidia GPU card is able to do a massive parallel processing, by using threads. each GPU can contains at ...
0
votes
0answers
52 views

How to setup grid/clutster computers?

We have 3 labs and each lab have 30 computers having different OS's on them windows, Ubuntu mostly, so i was thinking if i can connect them using grid computing, so we can use them for our research ...
4
votes
1answer
10k views

Differentiate CUDA Cores(NVIDIA) and Stream processor(ATI/AMD)

I think the question is answered here but I'm still wondering if what is the difference between the units it is referring to. For example, a GTX 570 has 480 CUDA cores, while the ATI equivalent HD ...
0
votes
1answer
36 views

losetup Find in Parallel / Concurrency

I'm writing a makefile that at some point builds image files. I'm using losetup -f in the file to find a free loop device when building out each image. Since the images are independent of each ...
2
votes
1answer
101 views

Parallel-Processing in Integrated GPU

I have an integrated GPU, meaning it works on the same RAM as my CPU. (ATI Radeon HD 4200, integrated on M4A785TD-V EVO) Does this also mean my GPU doesn't compute in parallel? see also: ...
0
votes
1answer
49 views

The exact workflow of PVM (Parallel virtual machine)

These days I am learning PVM technology for concurrent processing. I go through so many blogs, tutorials to understand . Yes, I got some points, but no one gives me clear workflow of how it perform ...
1
vote
1answer
439 views

Is it faster to run one process that spawns N threads or to run N processes?

I need to run ~1500 jobs on my lab's server. The program I am using (blastpgp if anyone cares) is capable of running in parallel mode, the "-a" option sets how many threads it will launch. The server ...
2
votes
3answers
1k views

Running threads on a non hyper-threading Intel processor

Intel Pentium Processor E5700 (2M Cache, 3.00 GHz, 800 MHz FSB) does not have hyper-threading but it has has 2 cores so I am assuming it has 2 threads Now if I write a heavy number crunching program ...
0
votes
1answer
459 views

PBS batch jobs - the qalter command

I've got a giant computation running on a Scientific Linux cluster. At present I have over 600 jobs parked in the queue, waiting for processor time, while a few are running. I'm trying to use the ...
5
votes
1answer
244 views

Read non-blocking from multiple fifos in parallel

I sometimes sit with a bunch of output fifos from programs that run in parallel. I would like to merge these fifos. The naïve solution is: cat fifo* > output But this requires the first fifo to ...
3
votes
2answers
456 views

How can I run several parallel shell jobs and monitor all of them at once?

I have a large number of large files (hundreds of files, hundreds of MB each) which I need to pipe through a number of programs to filter and convert them. I'm taking advantage of multiple CPU cores ...
2
votes
1answer
894 views

How do the cores in an AMD Bulldozer module compare to Intel HTT virtual cores and two separate cores, in terms of multitasking performance?

This question is basically an extension of Hyper-Threading and Dual-Core, What's the Difference? The Bulldozer-based AMD FX processors are marketed as having four, six, or eight cores, but ...
0
votes
1answer
300 views

Kerrighed cluster does not use all nodes when running OpenMP

I've been given access to a recently built cluster that uses Kerrighed. I'm the first user of this cluster, so setup and configuration problems may be a possibility. I'm also totally new to ...
1
vote
1answer
144 views

What kind of CPU/GPU integration is offered by APUs?

I'm truly fascinated by the idea of GPGPU and using the GPU for heavy processing. I'm seeing that also APUs (Accelerated Processing Units, CPU+GPU on the same chip) are gaining a consistent ...
3
votes
1answer
492 views

Does anyone know of an antivirus that can scan multiple hard drives simultaneously?

I have 5 hard drives and am so very sick of waiting for 3 hours for the virus scanner to scan each drive one at a time. I see absolutely no reason that a virus scanner can't be threaded to scan each ...
1
vote
2answers
725 views

gunzip multiple files and concatenate in parallel?

Is it possible to gunzip multiple files and concatenate them into one big file, but do it in parallel given a multicore machine? For example, right now I do: gunzip -c file1.gz > final gunzip -c ...
8
votes
5answers
11k views

Running multiple concurrent background processes then wait for all of them to terminate, on Windows

Anybody know if there is cmd script that does that and/or a command line tool ready for the job? Thanks!
2
votes
1answer
266 views

How can I share memory for parallel computing in Mathematica?

Is it possible to use RAM from other computers to assist with Mathematica Light Grid Server 7 parallel computing? If so, how?
-1
votes
1answer
220 views

MapReduce and Hadoop: what kinds of problems are they good at solving? [closed]

As I understand it already, it seems that both have similar functions (except that MapReduce is proprietary to Google, while Hadoop is open-source). I was wondering, not "how they work", but rather ...
25
votes
7answers
20k views

Parallelizing rsync

I just moved and discovered after some trial and error that somewhere between my home and my remote server, there is some throttling going on...but the throttling is not very intelligent. It only ...
1
vote
1answer
182 views

Is it normal to overworking of just one CPU when using openMP

When I use OpenMP in FORTRAN to parallelize a program on linux, ' top' command show me just one processor working ~%800 instead of ~%100 x 8 processors. Is it a expected situation ?
5
votes
4answers
6k views

What's the best way to perform a parallel copy on Unix?

I routinely have to copy the contents of a folder on a network file system to my local computer. There are many files (1000s) on the remote folder that are all relatively small but due to network ...
14
votes
5answers
20k views

Is a higher core count or higher clock speed more beneficial to a computer's performance? [closed]

With lowering silicon costs and rising consumer needs, manufacturers seem to be pushing one of two things: clock speed and/or core count. With the way things are going, it doesn't seem that clock ...
4
votes
1answer
576 views

Parallel processing slower than sequential?

EDIT: For anyone who stumbles upon this in the future: Imagemagick uses a MP library. It's faster to use available cores if they're around, but if you have parallel jobs, it's unhelpful. Do one of ...
0
votes
1answer
44 views

How to find out whether libav is built on MPI/LAM?

Just what the title states ; I expect to be doing a touch of audio cleanup/format switching. My plan is to try to do this with libav. Apparently such processing is fairly intensive; hence I would like ...
1
vote
1answer
109 views

Channelling processing power from Linux servers over a gigabit LAN to Macintosh system

As you may know Apple offers a program called Xgrid for sharing processing power between Macs. It also allows for easy clustering. Is there a way to channel processing power from Linux nodes over a ...
2
votes
1answer
252 views

Simulink - running parallel simulations

Is it possible to run Simulink calculations in parallel mode on cluster? And if its possible then what actually Simulink do when number of processes that i need is more than cluster has nodes? Thank ...
3
votes
3answers
8k views

Have today's desktop PCs surpassed IBM's Deep Blue of 1997?

I was wondering if my PC has surpassed the processing power of Deep Blue which famously won against the human world champion Garry Kasparov. What was the processing speed of Deep blue as compared to ...