Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Running Linux. I have a directory of around 150 large CSV files; simply doing a zip -9 on them results in a monolithic file that is still too large. I would like it to simply zip them in maybe four or five zip files of 30-40 CSVs each; this way sequencing or spanned zip order won't be a problem, as each zip is independent. There must be a simple way to do this. Any suggestions?

(and yes, zip is the preferred format, if possible)

share|improve this question

migrated from Jun 2 '13 at 7:27

This question came from our site for professional and enthusiast programmers.

Isn't -s switch enough? You may use zip -s to split the file into files of maximum size, e.g.:

"zip -s 300m <2 gb file>" produces: (300 mb, master file) (300 mb) (300 mb) (300 mb) (300 mb) (300 mb) (200 mb)

Then "unzip" will unzip everything together.

share|improve this answer
What version of zip is this?? I get file.z01 file.z02 ... and unzip does not work directly (I would use zip -F to recombine them first). Note these are not "independent" as requested. – sourcejedi Jun 2 '13 at 10:22
@sourcejedi: In this answer ( are some more detailed explanations. – mpy Jun 2 '13 at 10:43
@mpy I know, I've just written that answer :). – sourcejedi Jun 2 '13 at 11:44
@sourcejedi: Oh yes, now you say it... ;) – mpy Jun 2 '13 at 11:52

Use split on the list of input files :-).

(Not tested, I've included rm commands for cleanup, take care).

ls *.csv > csvfiles
split -d -l30 - csvfiles < csvfiles
for i in csvfiles[0-9][0-9]; do
  zip "$" -@ < "$i"

rm csvfiles
rm csvfiles[0-9][0-9]
share|improve this answer
Why do you use split -C (--line-bytes) and not split -l (--lines)? That would be more predictable, with regard to how many CSV files are in one archive. – mpy Jun 2 '13 at 12:00
I skimmed the manpage too quickly. Thanks, I'll fix it! – sourcejedi Jun 2 '13 at 12:58

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.