Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I often have a long-running job where an application reads through a large file (>2GB) and performs processing on each record.

Is there any UNIX command I can use to see how deep into the file the process might be? The application will have only one file handle to the input file.

share|improve this question

3 Answers 3

Does this program accept pipes/STDIN/STDOUT? If so, you're in luck. I will demonstrate with md5sum'ing a large file:

$ pv 2GB.zero | md5sum -
1.56GB 0:00:31 [60.2MB/s] [===============================>         ] 80% ETA 0:00:07
f3444be3893c1b9b6fa9dd1acde671a9  -

pv "{filename}" will read the file, pipe it to anything that supports a pipe. You could also do cat "{filename}"|pv| md5sum - but then pv would not know how many bytes the file is, thus it would only know how fast it's going, now how much is left.

The possibilities are endless. Whenever you're using a pipe or redirecting to a file, you just add pv in the pipeline to tell you how fast data is going through the pipe! :)

share|improve this answer

I've always wondered if lsof's "offset" tells you the current file-handle read position in the file but the documentation does not really spell this out clearly and I've never bothered to test or verify my assumptions. http://www.netadmintools.com/html/lsof.man.html

When writing long-running processes I tend to have them either write info to a log file I can periodically peek at (e.g. tail -f) or have them periodically report progress (e.g. a -v option that makes it report record count once a second).

share|improve this answer

Try pv, Pipe Viewer

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.