Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a text file that is 19 bytes in size and having compressed the file using zip and 7zip, it appears to be larger. I had a read of the question on Why is a 7zipped file larger than the raw file? as well as Why doesn't ZIP Compression compress anything? but considering the file is not already compressed I would have expected further compression. Attached is a screenshot.

enter image description here

EDIT0

I took the example further by creating a file that contained random data as follows dd if=/dev/urandom of=sample.log bs=1G count=1 and attempted to compress the file using both zip and 7zip however there were no compression gains. Why is that?

enter image description here

share|improve this question
    
And that is a plain text 1GB log file? –  CyberSkull Aug 29 '12 at 17:50
    
@CyberSkull - Yes it is. –  PeanutsMonkey Aug 29 '12 at 19:21
    
Can you please tell us what your zip parameters were? I would have done something like zip -9T "example.zip" sample.log (-t is just to test the integrity of the archive.). –  CyberSkull Aug 29 '12 at 19:31
    
@CyberSkull - I only ran the standard command i.e. zip sample.zip sample.log however when I ran 7zip I defined the maximum compression i.e. 7zr a -mx=9 sample.7z sample.log –  PeanutsMonkey Aug 29 '12 at 19:44
3  
Random data from /dev/urandom does not generate a true text file; it will not compress well at all. Text bytes are limited in range, with many spaces and repeating patterns (e.g. "th" and "sp") and words. You have in fact generated a random binary file. –  Ken Aug 29 '12 at 19:49
show 5 more comments

3 Answers

up vote 7 down vote accepted

As @kinokijuf said, there is a file header. But to expand upon that there are a few other things to understand about file compression.

The zip header contains all the necessary info for identifying the file type (the magic number), zip version and finally a listing of all the files included in the archive.

Your file probably wasn't compressed anyways. If you run unzip -l example.zip you will probably see that the file size is unchanged. 19 bytes would probably generate more overhead than would be saved if it were compressible at all by DEFLATE (the main compression method used by zip).

In other cases, PNG images for example, they are already compressed so zip will just store them. DEFLATE won't bother compressing anything already compressed.

If on the other hand you had a lot of text files, and their size was more than a few kilobytes each, you would get great savings by putting them all into a single zip archive.

You will get your best savings when compressing very regular, formatted data, like a text file containing a SQL dump. For example, I once had a dump of a small SQL database at around 13MB. I ran zip -9 dump.sql dump.zip on it and ended up with around a 1MB afterwards.

Another factor is your compression level. Many archivers by default will only compress at mid-level, going for speed over reduction. When compressing with zip, try the -9 flag for maximum compression (I think the 3.x manual says that compression levels are only supported by DEFLATE at this time).

TL;DR

The overhead for the archive exceeded any gains you may have gotten for compressing the file. Try putting larger text files in there and see what you get. Use the -v flag when zipping to see your savings as you go.

share|improve this answer
    
When you say the file size is unchanged if I were to unzip it, do you mean the size of the of the archive? Secondly if I were to use a different compression method other than DEFLATE such as PPMD, will it make a difference? So when you say the -V flag do you mean when I execute the zip command? –  PeanutsMonkey Aug 29 '12 at 7:20
    
Also when you say file type do you mean the type of file the source is e.g. text, MP3, etc? –  PeanutsMonkey Aug 29 '12 at 7:49
add comment

Because the overhead of .zip headers is way larger than 19 bytes.

share|improve this answer
    
How does this affect larger text files? –  PeanutsMonkey Aug 29 '12 at 7:05
add comment

Compression removes redundant information, which appears when the data is highly structured.

From this it should be apparent that already-compressed files cannot compress further, because the redundancy is already gone, but also that random data won't compress well, because it never had any structure or redundancy.

There a whole science, information theory, which deals with measuring the density of information (and mutual information) and uses redundancy and structure to perform compression, attacks on encryption, and error detection and recovery.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.