Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I am working with some very large packet capture files (~150GB across 30 files) but because of how highly redundant packet captures are, these compress to ~7% (14.29x) of their originally file size using .zip files.

I was wondering if I could work with these files "natively" by creating a ZFS partation with the "compression=gzip-9" setting. I created the partition, copied the packet captures over, and the end result was a compression ratio of 20.83% (4.80x).

I am wondering why there is such a large discrepancy.

share|improve this question

1 Answer 1

up vote 2 down vote accepted

This is because each record (file block) is compressed independently (to allow reading from the middle of the file for large files). Set large record size:

zfs set recordsize=128k zpool/fsname

or create new filesystem with large record size:

zfs create -o recordsize=128k zpool/fsname

This setting affects only new files created in the filesystem fsname.

share|improve this answer
    
The stats I posted were with a 128k recordsize, however such a small block size would definitely be the root cause. –  Chuu Mar 8 '13 at 0:36

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.