I have, for example, filesystem image which have some compressed files (with weak compression such as gzip), for example, manpages or archives with the same uncompressed content nearby.

How to pre-filter the data to "expand" compressed data to plain form (to re-compress it with strong compression) and then post-filter after decompression to restore original "semi-compressed" image? SHA-1 match is advices but not strictly required (but the resulting image must work, e.g. re-compressed files should not grow too much, be decompressible etc.)

Like improving compression ratio by reversing weak compression algorithms.

Are there programs for this?

  • Unrelated. My question is about preparation of information to be compressed better, not just compressed filesystems. – Vi. Nov 8 '12 at 18:50
  • I think is seems relevant enough. Reason for linking was that good answers to both will probably be interesting to all who have interest in either. BTW, what you are describing sounds like a product that someone would never do without being paid well for it. Hopefully they will notice this question. It's also alot less trouble to just get more hard drives and not worry so much about 15% more compression ratio on stored data. Besides, you lose time compressing and uncompressing them. – Ярослав Рахматуллин Nov 9 '12 at 0:55
  • "sounds like a product that someone would never do without being paid well for it" -> It is most likely to be done by ones that develop advanced compression algorithms (for competitions, for example). By explicitly undoing some zip, your can gain compression ratio advantage. For example, you can compress jar file without spoiling it's signature. – Vi. Nov 9 '12 at 19:44

Assuming you're in an environment where you can write and run shell scripts this is fairly easy to do:

when doing a store:

  1. mount the filesystem image.
  2. look for all files with "weak compression" suffixes
  3. uncompress those files
  4. save the list of uncompressed files somewhere
  5. unmount the image
  6. compress the image


  1. uncompress the image
  2. mount the system
  3. use the previously saved list to compress the files

This can be done using some of the file-detection code in lesspipe.sh from Gentoo, find and gzip, bzipt2, etc.

If you prefer to compress individual files with "strong compression" instead of the whole images, than this is certainly doable too.

  • Are there already programs (i.e. FUSE filesystems) that does this? Also I want this to be on lower than file level. Something like "looks like compressed zip content? Uncompress + store additional tips to produce exact compressed copy later" (in one big chunk of data) – Vi. Nov 7 '12 at 2:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.