Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Normally I would just do something like:

tar -czf archive.tar.gz *.csv

But when there are too many files in the directory for the shell to expand on a single line this doesn't work.

In these cases I would normally resort to using find. Something like:

find /path -name '*.csv' -exec tar -rf "./archive.tar.gz" {} +;`

But this only seems to work if I don't include the -z option because you can't append to compressed archives, and using -c instead of -r will overwrite the first archive since find runs tar multiple times.

The only other solution I could come up with is to create a .tar file with find (as above) and then use a second command to compress it. Is there a better way to handle cases like this?

I'm using Ubuntu Linux.

share|improve this question
4  
You can create a tar archive and compress it later. (As used to be the way we did things back when we walked uphill to school, both ways). –  Hennes Sep 6 '13 at 22:31
3  
Your find command suffers from the same problem with too many arguments since the glob is expanded before find sees it. You should use find -name '*.csv' instead. –  slhck Sep 7 '13 at 9:27
    
Indeed, I fixed it now. –  Code Commander Sep 7 '13 at 19:11

2 Answers 2

up vote 3 down vote accepted

As a robust solution, use find to separate filenames by a null character, and then pipe directly to tar, which reads null-delimited input:

find . -name '*.csv' -maxdepth 1 -print0 |
tar -czf archive.tgz --null -T -

This will now handle all file names correctly and is not limited by the number of files either.

Using ls to generate a list of filenames to be parsed by another program is a common antipattern that should be avoided whenever possible. find can generate null-delimited output (-print0) that most utilities can read or parse further. Since the null character is the only character that cannot appear in a filename (and the /, obviously), you'll always be safe with that.

share|improve this answer

No, you cannot append to a compressed tar file without uncompressing it first.

However, tar can accept its list of files to process from a file, so you can just do:

ls *.csv > temp.txt
tar -zcf ball.tgz -T temp.txt

@slhck points out that the above solution will not work if there are spaces (and probably other annoying characters) in your filenames. This version encloses each filename in double quotes:

ls *.csv | sed -e 's/^\(.*\)$/"\1"/' > temp.txt
tar -zcf ball.tgz -T temp.txt

(This will of course break if you have double quotes in your filenames, in which case you get what you deserve. :)

share|improve this answer
2  
If you want to do it in a single command you could use tar -zcf ball.tgz -T <( ls *.csv) or tar -zcf ball.tgz -T <( find -name '*.csv'). See tldp.org/LDP/abs/html/abs-guide.html#PROCESS-SUB –  Zoredache Sep 6 '13 at 23:23
    
That's not quite the solution I was thinking of. See my answer for a solution that doesn't choke on file names. Don't blame users for creating files with spaces or double quotes in the path – all Linux tools can handle this correctly, you just need to apply the proper options. In general, as a good tip, once you find yourself using ls to enumerate files other than just looking at them, something will probably break. –  slhck Sep 8 '13 at 11:26

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.