I'm trying to copy some files into existing folders. This is an example of the schema

# origin 

The destination folder schema already exists, its just the origin files (jpgs) that are not there.

My question is how to do this?

I would like to end up with something like below, but this is not working. Maybe I should include regex in the query to separate asterisk into values and use as origin/destination?

 cp /home/user/myfiles/*/*/*.jpg  /home2/user/myfiles/*/*/*.jpg

My original intend is to use this copying schema on my unanswered question: s3cmd: how to copy files from local to S3 bucket with existing structure as a possible solution.

  • Are the destination folders completely empty? If so just delete them and copy files and directory structure. Also, the shell expansion of the second set of asterixes is your problem.
    – Hennes
    Jul 25 '12 at 23:34
  • They're not. They have other type of files that I still need.
    – Martin
    Jul 25 '12 at 23:42
  • Can you elaborate on your last sentence?
    – Martin
    Jul 25 '12 at 23:43
  • I can, but I am still trying to wrap my head around what your precise goal is. The cp command has the same source and destination. The s3cmd question identified local dev and remote s3 site, that I could understand. But the current question seems to be a NOP. (Granted, that might be because I am sleepy).
    – Hennes
    Jul 26 '12 at 0:04
  • @Hennes Sorry for misleading you, just fixed the question. The dest should be different than origin!
    – Martin
    Jul 26 '12 at 1:42

With GNU cp you can use the --parents option, like this:

cd /home/user/myfiles ; cp --parents */*/*.jpg /home2/user/myfiles/

You have to cd first because if you do cp --parents /home/user/myfiles/... it will create those parents too, and you'll end up with /home2/user/myfiles/home/parents/myfiles/.

If you need a non-GNUish answer, the traditional unix method would be tar over a pipe.

cd /home/user/myfiles ; tar cf - */*/*.jpg | ( cd /home2/user/myfiles ; tar xf - )

The second cd is in a subshell with the second tar, so that's where the files get extracted. The tarball itself is created, sent over the pipe, and extracted all at the same time. Add a v flag to one of the tars if you want progress reporting.


The following should do the trick:

for i in `find /home/user/myfiles/folder1 -print`; do 
  if [ ! -d $i ]; 
    then filePath=`echo $i | cut -f 4- -d"/"`; 
         cp $i /home/user/myfiles/folder2/$filePath; 

Basically, I am extracting the part of the filename that is going to change between the two folders, and then issuing a copy command.

Notice the "cut -f 4-" above - it gets the fields from the 4th one onwards. You can change the number as per your wish

Perhaps even basename or a simpler command exists... but this will do for now :)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.