I have a relatively large log file (46M) on a vServer, of which I'm only interested in the most recent part.
How can I download only the last 10% of the file?
The server is running Debian and I'm using Ubuntu locally.
First get the size of the remote file in bytes:
$ ssh user@host 'stat -c%s FILENAME'
50000
Calculate 10% of that number, and copy the last ten percent:
$ ssh user@host 'tail -c 5000 FILENAME' > DESTINATION
ssh user@host 'f=FILENAME; tail -c $(($(stat -c%s $f)/10)) $f' > DESTINATION
stat()
syscall, which is fast, although du
also examines all mounted filesystems using stat()
before the actual file. But that's not a difference in practice. ⫽ However, if you need to get just the bytes (like in the one-liner in your last comment), then running stat
will be easier because it doesn't output the unnecessary filename, making the one-liner shorter than it would have been with du -b | sed ...
.
Aug 18, 2013 at 12:42
rsync
. Why don't you just download the whole file and keep usingrsync
to periodically update it from the server? (Log files also compress well, so withrsync -z
you'll just need to download 4-5 MB for the entire log file.)