count lines in a compressed file

Question

if i have a .gz file on unix which has certain number of lines. How could i count the lines on unix without uncompressing it.

Answer 1

You can obviously not count newlines if the file is still compressed.

But you can decompress to a stream, and count the newlines in that stream, without ever writing the (decompressed) file to disk. That would go something like so:

zcat file.gz | wc -l

zcat for decompress & cat, wc for wordcount. See man pages for both if you want to know more.

EDIT

If you do not have zcat, zcat is just another name for gunzip -c.

Answer 2

This also seems to work - grep for the number of line-endings in the file

zgrep -Ec "$" file.gz

Answer 3

If you want to do it quickly, I recommend using 'pigz' (which IIRC stands for "Parallel Implementation of GZip"). I just had a similar situation where I wanted to count the number of lines in a bunch of gzip'ed files and here was my solution:

for x in *.gz; do unpigz -p 8 -c $x | wc -l && echo $x; done

Which gave me the number of lines and the file it counted from on alternating lines, using 8 processors. It ran quickly!