10

If I have a 100 GB folder and I split ZIP it, is there a difference in how much disk space is consumed if I split it into 100 .ZIP files at 1 GB each or 10 .ZIP files at 10 GB each?

Do 100 .ZIP files at 1 GB each take up more space than 10 .ZIP files at 10 GB each?

  • And you can't find out because? – Dave Oct 28 '18 at 17:41
  • 5
    Why can't you just try it? – Peter Mortensen Oct 28 '18 at 19:02
  • 1
    Each standalone ZIP file has some overhead. However, you can chop up a ZIP file into pieces that can be reassembled. Those pieces don't have the ZIP overhead in each one, and if you split at sector or block boundaries, they don't contain wasted space. – fixer1234 Nov 1 '18 at 2:54
16

Let's find out!

100 MB files (27 pieces):

7z a -tzip -v100M ./100m/archive ./kali-linux-xfce-2018.2-amd64.iso

$ du ./100m/
2677884 ./100m/

10 MB files (262 pieces):

7z a -tzip -v10M ./10m/archive ./kali-linux-xfce-2018.2-amd64.iso

$ du ./10m/
2677908 ./10m

Results: The 10 MB split archive takes up an extra 24 KB. So yes, there is a difference, the 100 1 GB files will take up more space than the 10 10 GB files.

The difference seems to be negligible though. I would go for whichever is more convenient for you.

  • 4
    du doesn't output the size in bytes by default (unless your 270M of files turned into 2,677,908 bytes). It does display the on-disk size of files, which may be different than the actual data size (maybe applicable for uploading or storing on other filesystems) – Xen2050 Oct 28 '18 at 15:42
  • You are correct, it's actually outputting in KB. I've edited the answer to correct this discrepancy. The original file is a Kali Linux ISO, it is ~2.6GB. You have a good point about the on-disk size vs actual data size, I was specifically thinking about on-disk size because it accounts for the overhead of having additional files but you're right that it would be different depending on what you're actually doing with the archives. – Layne Bernardo Oct 28 '18 at 15:50
  • Sorry, I crossed with your largely similar answer while I was double-checking the run strings. – AFH Oct 28 '18 at 15:50
  • Zip file max size is 4GB. – pbies Oct 28 '18 at 17:38
  • 1
    Yeah, that's why I didn't bother calculating a percent. I don't think it works out to a flat percentage of original file size, especially considering differences in file systems. – Layne Bernardo Oct 29 '18 at 10:47
16

Every file has a file system overhead of unused logical sector space after the end-of-file, but this is eliminated if the split size is a multiple of the logical sector size (not necessarily true of my example below).

There may be extra bytes used by the extra directory entries, but these will not figure unless the directory now occupies an extra logical sector.

The split files are identical in content to those created by a binary splitter program with the same split size.

I verified these on Linux by using the GUI version on a 7+MB file, giving 8 split files of 1MB size with 7-Zip (File.7z.00?), then created a single, full archive (Full.7z), which I split with:-

7z -v1000000 a File;                                         # Create split volumes File.7z.00?
7z a Full File;                                              # Create full archive Full.7z
split -b 1000000 -a 3 --numeric-suffixes=1 Full.7z Full.7z.; # Split full archive into Full.7z.00?
for f in {001..008}; do cmp Full.7z.$f File.7z.$f; done;     # Compare splits with 7z volumes

To test on another OS you may need to down-load or write an appropriate splitter program.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.