Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Is it possible to gunzip multiple files and concatenate them into one big file, but do it in parallel given a multicore machine? For example, right now I do:

gunzip -c file1.gz > final
gunzip -c file2.gz >> final
gunzip -c file3.gz >> final
gunzip -c file4.gz >> final

Can I do the same so that the gunzip processing of the different files is done in different CPUs in the multicore machine and they are concatenated all into the same final file?

share|improve this question
Are you sure that it is CPU and not disk speed that is limiting? – Daniel Andersson Oct 3 '12 at 14:45

It is a little shorter to do that using GNU Parallel:

parallel gunzip -c ::: file*.gz > final

but essentially it also writes to temporary files.

Watch the introvideos to learn more:

share|improve this answer

you have to use temporary files for this:

gunzip -c file1.gz > final &
gunzip -c file2.gz > final2 &
gunzip -c file3.gz > final3 &
gunzip -c file4.gz > final4 &

wait $one && wait $two && wait $three && wait $four
cat final2 >> final
cat final3 >> final
cat final4 >> final

to decompress the parts of a bigger file into the one final file you would have to know the decompressed size of the parts. only then you could create a biiiig empty file and write the output of the decompression to the right position in the big file (with dd for example). since you don't know the decompressed size (without decompressing the parts first) this won't work.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.