8

I have a text file that is 19 bytes in size and having compressed the file using zip and 7zip, it appears to be larger. I had a read of the question on Why is a 7zipped file larger than the raw file? as well as Why doesn't ZIP Compression compress anything? but considering the file is not already compressed I would have expected further compression. Attached is a screenshot.

enter image description here

EDIT0

I took the example further by creating a file that contained random data as follows dd if=/dev/urandom of=sample.log bs=1G count=1 and attempted to compress the file using both zip and 7zip however there were no compression gains. Why is that?

enter image description here

Improve this question
11
  • 1
    And that is a plain text 1GB log file?
    – CyberSkull
    Aug 29, 2012 at 17:50
  • @CyberSkull - Yes it is.
    – PeanutsMonkey
    Aug 29, 2012 at 19:21
  • 6
    Random data from /dev/urandom does not generate a true text file; it will not compress well at all. Text bytes are limited in range, with many spaces and repeating patterns (e.g. "th" and "sp") and words. You have in fact generated a random binary file.
    – Ken
    Aug 29, 2012 at 19:49
  • 1
    Open your favorite text editor. Now get your cat or small child and induce them to play with the keyboard for 5 minutes or so. You now have a large random text file! ;)
    – CyberSkull
    Aug 30, 2012 at 5:58
  • 1
    @CyberSkull: No, you have a random stream of ASCII characters. Which is a bit more compressible than random binary, but still nowhere near as structured as text.
    – Ben Voigt
    May 28, 2013 at 19:24

2 Answers 2

Reset to default
12

As @kinokijuf said, there is a file header. But to expand upon that there are a few other things to understand about file compression.

The zip header contains all the necessary info for identifying the file type (the magic number), zip version and finally a listing of all the files included in the archive.

Your file probably wasn't compressed anyways. If you run unzip -l example.zip you will probably see that the file size is unchanged. 19 bytes would probably generate more overhead than would be saved if it were compressible at all by DEFLATE (the main compression method used by zip).

In other cases, PNG images for example, they are already compressed so zip will just store them. DEFLATE won't bother compressing anything already compressed.

If on the other hand you had a lot of text files, and their size was more than a few kilobytes each, you would get great savings by putting them all into a single zip archive.

You will get your best savings when compressing very regular, formatted data, like a text file containing a SQL dump. For example, I once had a dump of a small SQL database at around 13MB. I ran zip -9 dump.sql dump.zip on it and ended up with around a 1MB afterwards.

Another factor is your compression level. Many archivers by default will only compress at mid-level, going for speed over reduction. When compressing with zip, try the -9 flag for maximum compression (I think the 3.x manual says that compression levels are only supported by DEFLATE at this time).

TL;DR

The overhead for the archive exceeded any gains you may have gotten for compressing the file. Try putting larger text files in there and see what you get. Use the -v flag when zipping to see your savings as you go.

Improve this answer
2
  • When you say the file size is unchanged if I were to unzip it, do you mean the size of the of the archive? Secondly if I were to use a different compression method other than DEFLATE such as PPMD, will it make a difference? So when you say the -V flag do you mean when I execute the zip command?
    – PeanutsMonkey
    Aug 29, 2012 at 7:20
  • Also when you say file type do you mean the type of file the source is e.g. text, MP3, etc?
    – PeanutsMonkey
    Aug 29, 2012 at 7:49
1

Compression removes redundant information, which appears when the data is highly structured.

From this it should be apparent that already-compressed files cannot compress further, because the redundancy is already gone, but also that random data won't compress well, because it never had any structure or redundancy.

There a whole science, information theory, which deals with measuring the density of information (and mutual information) and uses redundancy and structure to perform compression, attacks on encryption, and error detection and recovery.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .