2

Given many directories/sub-directories containing many different file types (csv, txt, jpg, etc.), what command will search each directory recursively and copy files out into a new top-level directory named after each extension.

Is there a way to accomplish this with a one-liner?

  • 1
    The command you need called find. You can construct it as one-liner – Alex Jan 16 '17 at 8:04
  • Why would it have to be on one line? – Seth Jan 16 '17 at 9:21
  • I'd imagine that they want a command to run via SSH or something where a script won't necessarily exist on the destination system. – stuts Jan 16 '17 at 9:23
1

Solution

The following command will list all files in the current directory (.) and move them into directories named after their extensions in ./OUTPUT:

for file in $(find . -type f) ; do EXTENSION=$(echo "$file" |sed 's/.*\.//g') ; rsync -av $file ./OUTPUT/$EXTENSION/ ; done

Before running the above command, you will need to:

  • Create the directory OUTPUT in the current directory (mkdir OUTPUT)
  • Ensure that there are no files with matching names in any of the subdirectories

Explanation

Firstly, the loop assigns the variable file with a file from the output of find . -type f. Example output of the find command is below:

./a/myfile1.zip
./a/b/adoc1.txt
./a/b/adoc2.txt
./a/b/adoc3.txt
./a/b/adoc4.txt
./a/myfile3.zip
./a/myfile4.zip
./a/myfile2.zip

The variable EXTENSION is created and its value is the file variable with everything up to the last full-stop stripped out. So ./a/myfile1.zip would become zip.

The file is then synced (with verbosity, v) to a directory within OUTPUT named after the file extension.

An example of my file tree after running this command on some test:

./a
./a/b
./a/b/adoc1.txt
./a/b/adoc2.txt
./a/b/adoc3.txt
./a/b/adoc4.txt
./a/b/c
./a/myfile1.zip
./a/myfile2.zip
./a/myfile3.zip
./a/myfile4.zip
./b
./b/c
./b/c/d
./b/c/file1.txt
./b/c/file2.txt
./b/c/file3.txt
./b/c/file4.txt
./c
./c/d
./c/d/e
./c/d/e/afile1.rar
./c/d/e/afile2.rar
./c/d/e/afile3.rar
./c/d/e/afile4.rar
./OUTPUT
./OUTPUT/rar
./OUTPUT/rar/afile1.rar
./OUTPUT/rar/afile2.rar
./OUTPUT/rar/afile3.rar
./OUTPUT/rar/afile4.rar
./OUTPUT/txt
./OUTPUT/txt/adoc1.txt
./OUTPUT/txt/adoc2.txt
./OUTPUT/txt/adoc3.txt
./OUTPUT/txt/adoc4.txt
./OUTPUT/txt/file1.txt
./OUTPUT/txt/file2.txt
./OUTPUT/txt/file3.txt
./OUTPUT/txt/file4.txt
./OUTPUT/zip
./OUTPUT/zip/myfile1.zip
./OUTPUT/zip/myfile2.zip
./OUTPUT/zip/myfile3.zip
./OUTPUT/zip/myfile4.zip
0

This can be done very simply with find, as correctly suggested by Alex: assuming the directory to scan for files is /path1/to/some/dir, and you want the files moved to /path2/to/other/dir/txt and so on,

find /path1/some/dir -type f -exec bash -c 'MYDIR=/path2/to/other/dir${0##*.}; [ -d $MYDIR ] && mv $0 $MYDIR  || mv $0 /path3/to/miscellanea' {} \; 

This command first finds all files (-type f) contained within the directory /path1/to/some/dir and all of its subdirectories, then passes the name of each such file ({}) to a small bash two-liner, which first extracts the extension from the file name (referred to as $0 inside bash) and assigns it to the local shell variable MYDIR, then tests for the existence of a suitable directory (if there is no such directory, then maybe we made a mistake), and if the directory exists (&& ...) it moves the file in the appropriate directory, otherwise (||...) it places the file in a default place, /path3/to/miscellanea.

The above assumes you already have created the necessary directories, /path2/to/other/dir/txt and so on. If instead you want to create them on the go, you will have to modify the above as:

find /path1/some/dir -type f -exec bash -c 'MYDIR=/path2/to/other/dir/${0##*.}; mkdir -p $MYDIR && mv $0 $MYDIR' {} \; 

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.