4

I have a large folder of RAR archives. There is a significant hierarchy of folder levels. I want to unRAR the entire collection of archives all at once.

I have the following one-liner, which will work:

find -name "*.rar" -print0 | xargs -0 -n 1 -P 4 unrar x

(Note that we're running four threads in parallel to speed up the operation. :-) )

The problem with this command is that xargs is executed in the top level directory for each RAR files. This means that all of the output is being dumped into the top level folder.

Instead, I want the output to exist in the same folder as the RAR archive.

Example:

Top level
 |--FolderA
 |----File1.rar
 |----File2.rar
 |--FolderB
 |----File1.rar
 |----File2.rar
 |----File3.rar
 |--FolderC
 |----File1.rar
 |----File2.rar

Each of the "File1.rar" files contains a file with the same name. Thus extracting them all into the top level folder causes overwrite issues.

To summarize, I want to extract all the RAR files in the above hierarchy. I want the contents of each RAR file to exist in the folder that the RAR file exists in.

It seems to me that the solution is to somehow set the working directory, and then run the unrar command form there. However, since the find command is giving me filenames, not directories, I can't do something like

| xargs -I{} -n 1 -P 4 cd {} \; unrar x {}

Short of writing a Perl or Python script that will wrap around the unrar command and handle splitting the provided path into its parts and executing the command, is there a better way to achieve this?

3

There exist commands to extract a directory name (dirname) and file name (basename) from a path. So you could do something like

find . -name '*.rar' -print0 | \
xargs -0 -I{} -n1 -P4 /bin/sh -c 'cd "$(dirname {})"; unrar x "$(basename {})"'

AFAIK, xargs doesn't support changing directories, so you would need some intermediary to do that, hence the /bin/sh. You mentioned writing a wrapper around unrar, and that is basically what this is doing, except in one-liner form.

  • This works if there are no spaces in the path names, if there are spaces it seems you need to dial up the quotes to a ludicrious level 'cd "$(dirname "{}")"; unrar x "$(basename "{}")"' (i'm using unzip not unrar but same difference) – junichiro Apr 12 '18 at 19:47
1

Using GNU Parallel it looks like this:

find . -name '*.rar' | parallel cd {//} '&&' unrar x {/}

GNU Parallel is a general parallelizer and makes is easy to run jobs in parallel on the same machine or on multiple machines you have ssh access to.

If you have 32 different jobs you want to run on 4 CPUs, a straight forward way to parallelize is to run 8 jobs on each CPU:

Simple scheduling

GNU Parallel instead spawns a new process when one finishes - keeping the CPUs active and thus saving time:

GNU Parallel scheduling

Installation

If GNU Parallel is not packaged for your distribution, you can do a personal installation, which does not require root access. It can be done in 10 seconds by doing this:

(wget -O - pi.dk/3 || curl pi.dk/3/ || fetch -o - http://pi.dk/3) | bash

For other installation options see http://git.savannah.gnu.org/cgit/parallel.git/tree/README

Learn more

See more examples: http://www.gnu.org/software/parallel/man.html

Watch the intro videos: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Walk through the tutorial: http://www.gnu.org/software/parallel/parallel_tutorial.html

Sign up for the email list to get support: https://lists.gnu.org/mailman/listinfo/parallel

0

If you don't need the -P option from xargs, then you can use the find -execdir option, which is like -exec but does a cd into the directory before executing. Example at: https://stackoverflow.com/questions/16541582/finding-multiple-files-recursively-and-renaming-in-linux/54163971#54163971

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.