Say that you have a file you want to put in a .zip archive:
zip a1.zip foo.dll
My test .dll file is ~10MB, and the archive turns out to be 3.5MB
Then you create a file with the exact same contents, and put these both into an archive:
cp foo.dll bar.dll
zip a2.zip foo.dll bar.dll
You might expect that ZIP is smart enough to figure out this is repeating data and use only one compression object inside the .zip, but this is not the case: a2.zip is 7.0MB!
Basically most such utilities behave similarly (tar.gz, tar.bz2, rar in solid mode) - only 7zip caught me and the resulting a2.7z is only marginally larger than a1.7z.
So the question is: is it possible to construct a .zip file where this space wastage is avoided? We create the .zip files with C++ code, which uses the minizip project from zlib.
Why do we need this?
We ship our software in both ".exe installer" and ".zip file" form. The software doesn't really require installation, you can just unzip and use it. The .zip option is preferred by large clients which have many workstations and use automated deployment / software update services.
We introduced three .dll files recently, that now need to be put in two different folders, to be used by different components (only one central directory for these files is impossible for technical reasons). This three .dll files are exact copies in both folders. The .exe installer figures this out, since we instruct it to use the exact same compressed blob for each of the two destinations. But that's not the case with .zip and the resulting install is 15MB larger, meaning more bandwidth usage, slower download times and searing engineer-unhappy-that-things-are-not-optimal type of rage. Also, the .zip install suddenly becomes larger than the .exe install, so we'd be asked what did we omit in the .exe install.
There are some potential solutions to this,
- Use 7-zip: the boss is strongly against this, however, since this forces the aforementioned automated deployment people to modify their scripts to accommodate 7-zip.
- Use symlinks: if you put a symlink inside a .zip, that points to another file inside the .zip, it is stored as a reference (e.g., by using the
zip). Hopefully, unarchivers under Win32 will be kind enough to support these and extract a copy of the file to the path where the symlink should be located. F.e. WinRAR does this, but there are a lot of programs that "can do .zip" and I'm not sure if all do it this way.