Running Linux. I have a directory of around 150 large CSV files; simply doing a zip -9 on them results in a monolithic file that is still too large. I would like it to simply zip them in maybe four or five zip files of 30-40 CSVs each; this way sequencing or spanned zip order won't be a problem, as each zip is independent. There must be a simple way to do this. Any suggestions?

(and yes, zip is the preferred format, if possible)


2 Answers 2


Isn't -s switch enough? You may use zip -s to split the file into files of maximum size, e.g.:

"zip -s 300m <2 gb file>" produces:

file.zip (300 mb, master file)
file.001.zip (300 mb)
file.002.zip (300 mb)
file.003.zip (300 mb)
file.004.zip (300 mb)
file.005.zip (300 mb)
file.006.zip (200 mb)

Then "unzip file.zip" will unzip everything together.

  • What version of zip is this?? I get file.z01 file.z02 ... file.zip and unzip file.zip does not work directly (I would use zip -F to recombine them first). Note these are not "independent" as requested.
    – sourcejedi
    Jun 2, 2013 at 10:22
  • 1
    @sourcejedi: In this answer (superuser.com/a/602736/195224) are some more detailed explanations.
    – mpy
    Jun 2, 2013 at 10:43
  • @mpy I know, I've just written that answer :).
    – sourcejedi
    Jun 2, 2013 at 11:44
  • @sourcejedi: Oh yes, now you say it... ;)
    – mpy
    Jun 2, 2013 at 11:52

Use split on the list of input files :-).

(Not tested, I've included rm commands for cleanup, take care).

ls *.csv > csvfiles
split -d -l30 - csvfiles < csvfiles
for i in csvfiles[0-9][0-9]; do
  zip "$i.zip" -@ < "$i"

rm csvfiles
rm csvfiles[0-9][0-9]
  • Why do you use split -C (--line-bytes) and not split -l (--lines)? That would be more predictable, with regard to how many CSV files are in one archive.
    – mpy
    Jun 2, 2013 at 12:00
  • I skimmed the manpage too quickly. Thanks, I'll fix it!
    – sourcejedi
    Jun 2, 2013 at 12:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.