3

I am working with some very large packet capture files (~150GB across 30 files) but because of how highly redundant packet captures are, these compress to ~7% (14.29x) of their originally file size using .zip files.

I was wondering if I could work with these files "natively" by creating a ZFS partation with the "compression=gzip-9" setting. I created the partition, copied the packet captures over, and the end result was a compression ratio of 20.83% (4.80x).

I am wondering why there is such a large discrepancy.

1 Answer 1

5

This is because each record (file block) is compressed independently (to allow reading from the middle of the file for large files). Set large record size:

zfs set recordsize=128k zpool/fsname

or create new filesystem with large record size:

zfs create -o recordsize=128k zpool/fsname

This setting affects only new files created in the filesystem fsname.

1
  • The stats I posted were with a 128k recordsize, however such a small block size would definitely be the root cause.
    – Chuu
    Mar 8, 2013 at 0:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .