Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am working with some very large packet capture files (~150GB across 30 files) but because of how highly redundant packet captures are, these compress to ~7% (14.29x) of their originally file size using .zip files.

I was wondering if I could work with these files "natively" by creating a ZFS partation with the "compression=gzip-9" setting. I created the partition, copied the packet captures over, and the end result was a compression ratio of 20.83% (4.80x).

I am wondering why there is such a large discrepancy.

share|improve this question
up vote 2 down vote accepted

This is because each record (file block) is compressed independently (to allow reading from the middle of the file for large files). Set large record size:

zfs set recordsize=128k zpool/fsname

or create new filesystem with large record size:

zfs create -o recordsize=128k zpool/fsname

This setting affects only new files created in the filesystem fsname.

share|improve this answer
    
The stats I posted were with a 128k recordsize, however such a small block size would definitely be the root cause. – Chuu Mar 8 '13 at 0:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .