Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Running Linux. I have a directory of around 150 large CSV files; simply doing a zip -9 on them results in a monolithic file that is still too large. I would like it to simply zip them in maybe four or five zip files of 30-40 CSVs each; this way sequencing or spanned zip order won't be a problem, as each zip is independent. There must be a simple way to do this. Any suggestions?

(and yes, zip is the preferred format, if possible)

share|improve this question

migrated from Jun 2 '13 at 7:27

This question came from our site for professional and enthusiast programmers.

Isn't -s switch enough? You may use zip -s to split the file into files of maximum size, e.g.:

"zip -s 300m <2 gb file>" produces: (300 mb, master file) (300 mb) (300 mb) (300 mb) (300 mb) (300 mb) (200 mb)

Then "unzip" will unzip everything together.

share|improve this answer
What version of zip is this?? I get file.z01 file.z02 ... and unzip does not work directly (I would use zip -F to recombine them first). Note these are not "independent" as requested. – sourcejedi Jun 2 '13 at 10:22
@sourcejedi: In this answer ( are some more detailed explanations. – mpy Jun 2 '13 at 10:43
@mpy I know, I've just written that answer :). – sourcejedi Jun 2 '13 at 11:44
@sourcejedi: Oh yes, now you say it... ;) – mpy Jun 2 '13 at 11:52

Use split on the list of input files :-).

(Not tested, I've included rm commands for cleanup, take care).

ls *.csv > csvfiles
split -d -l30 - csvfiles < csvfiles
for i in csvfiles[0-9][0-9]; do
  zip "$" -@ < "$i"

rm csvfiles
rm csvfiles[0-9][0-9]
share|improve this answer
Why do you use split -C (--line-bytes) and not split -l (--lines)? That would be more predictable, with regard to how many CSV files are in one archive. – mpy Jun 2 '13 at 12:00
I skimmed the manpage too quickly. Thanks, I'll fix it! – sourcejedi Jun 2 '13 at 12:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .