Today I wanted to create a backup from a folder on a server with 16 CPUs. I started looking for an option that can utilize my hardware to archive a folder better than how tar does it. Something with multi-thread support probably. I did some research and I figured there are tools like pbzip2 and pigz but they can only compress, not archive. So do you guys have an elegant solution for that?


Archiving itself is I/O-intensive, and won't benefit from multiple cores. Feed the uncompressed output of tar to one of the programs you found.


tar -cO /directory/path | whizbang -compress --ultra-brute --cpus=16

where whizbang is replaced with your favorite compressor depending on speed vs size preferences


What Ignacio said... but -O is for extracting files to stdout...

So, my suggestion:

tar cf - /directory/path | whizbang -compress --cpus=16 > archive.tar.whizbang

The next step, of course, would be badgering the maintainers of tar to include whizbang support in their next release. ;-)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.