4

Is there a way to copy a list of files, that share a common base directory, preserving their relative directory structure? The files with relative paths are listed in a text file. edit I have the filenames listed in a plain text file.

I'd like to do this without using a script (no bash/batch/python/ruby/lua/etc.), similar to how wget -i works for urls. (My reason is that it'd give overhead/non-standard solution for a reasonably common task.)

I've tried cat list.txt | xargs -I % cp % new_folder from Copy list of files but it didn't preserve the relative directory path.

I'd do this to export my selection of files (that I already have in a text file) from a directory structure that has big number of files (happens from time to time).

1
  • another reason for using a stand-alone command instead of custom scripts is that programs tend to be installed already or have packages/installers that can be found on the net by remembering the program's name alone, while having to write it again each time or taking your own scripts everywhere somehow feels less of benefit (even if you have no problems with your skills).
    – n611x007
    Oct 31, 2013 at 18:25

3 Answers 3

4

tar cvf - ./filesAndDirecotries | (cd targetDirectory ; tar xvf - )

TAR (Tape Archive is old command to create an archive file) can write to stdin ( " - " parameter for the file ) and pipe and read from stdin in the second directory.

Above command first creates a tar file and writes to stdin and piped to a shell and change directory and untar the file by reading it from stdin.

2
  • I wasn't clear enough in my question that I tried to use a text file listing the filenames, and looking for an option to use that text file from a program that can copy. This solution however is very nice, and led me in the right direction because I think this can be done via the -T switch: -T, --files-from F: get names to extract or create from file F
    – n611x007
    Nov 1, 2013 at 11:32
  • 1
    I'll accept this as this came the closest to what I had in mind and set me on the right track. I've edited the question to pinpoint the existence of the text file. You can update your answer if you want. Finally I've used tar cvf - -T filelist.txt | tar xvf - -C newdir
    – n611x007
    Nov 1, 2013 at 11:50
5

The answer of unixUser set me on the right track. Turns out that tar can do both reading file and directory names from a text file with -T, and can even change directory with -C:

tar cvf - -T filelist.txt | tar xvf - -C newdir

This answers the question I had.


sources

To obtain this information from the idea unixUser gave me, I've ducked for tar files from text file, leading to trpn's (Q) and Tinkster's (A) conversion on linuxquestions.org. Simultaneously checking the tar manual page for any option that has something to do with file, both search led to the -T switch! To avoid relying on shell for changing directory, I've also searched it for any directory related options and found the -C switch. -- I see that tar is probably the command meant for grouping (copies of) files in many ways.

With this, one can use some tar binary without* relying on scripting. Just amazing how useful tar is.


(*: out of preference: I find bash/perl/python/whatever a too general tool for this quite exact task. However, as pointed out by grawity, pipes are convenient - without them one should be forced to create an intermediate file, eg. tar cvf intermediate.tar -T filelist.txt and tar xvf intermediate.tar -C newdir.)

1

Technically, your given example already uses bash. And that's exactly why the shell is scriptable in the first place – it's such a simple task that there is no need for a dedicated tool/program just for it.

cat list.txt | xargs -n1 dirname | sort | uniq | xargs -I% mkdir -p new_folder/%
cat list.txt | xargs -I% cp % new_folder/%

Or:

cat list.txt | while read path; do
    mkdir -p "new_folder/${path%/*}" && cp "$path" "new_folder/$path"
done

Or:

cat list.txt | xargs -I% install -Dm0644 % new_folder/%

Or:

cat list.txt | while read path; do
    install -Dm0644 "$path" "new_folder/$path"
done

(install is shorter but does not preserve modes, unfortunately.)

5
  • you are right about bash, but I've made the distinction on the basis that introducing control flow and variables really is an overkill for this and the more dependency you introduce the more dependent the solution on the platform is. I'm not happy with the string substitution either but that was the closest example I got. On the other hand, most platforms have understands a pipe, so it is a smaller dependency. Your first solution that fixes the example comes closest, thanks!
    – n611x007
    Oct 31, 2013 at 20:40
  • Well, there are platforms that easily run Perl but don't have bash nor xargs (Windows) nor even easily-usable pipes (VMS). Nov 1, 2013 at 9:06
  • Right again. I suppose I've tried to explain a preference of mine. On the other hand, the tar command seems to suit my needs; I wasn't aware it had a -T switch. I think it's good to have these options available just like here and your answer also explains how this is normally done.
    – n611x007
    Nov 1, 2013 at 11:47
  • On the other hand, I should make myself more familiar with VMS. I never used it but wouldn't thought it lacks pipes. I wonder what VMS can use here because these ones also use pipes. May it solely rely on scripting? (on Windows, there is unxutils, gnuwin32, cygwin, msysgit's environment and so on, to provide familiarity/features)
    – n611x007
    Nov 1, 2013 at 12:02
  • @naxa: Nah, VMS is mostly dead. Although it seems that its command line interface does have support for pipes in the later versions (the PIPE keyword), but it was most definitely not designed for it – in comparison, even the Windows CLI is more Unix-like. Nov 1, 2013 at 16:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .