scp: How to copy only the last 10% of a file?

Question

I have a relatively large log file (46M) on a vServer, of which I'm only interested in the most recent part.

How can I download only the last 10% of the file?

The server is running Debian and I'm using Ubuntu locally.

You tagged the post with rsync. Why don't you just download the whole file and keep using rsync to periodically update it from the server? (Log files also compress well, so with rsync -z you'll just need to download 4-5 MB for the entire log file.) — u1686_grawity, Aug 18, 2013 at 12:42

Flimm · Accepted Answer · 2013-08-18 11:12:51Z

21

First get the size of the remote file in bytes:

$ ssh user@host 'stat -c%s FILENAME'    
50000

Calculate 10% of that number, and copy the last ten percent:

$ ssh user@host 'tail -c 5000 FILENAME' > DESTINATION

answered Aug 18, 2013 at 11:12

Flimm

10.7k5 gold badges35 silver badges42 bronze badges

1

Excellent. :) Does stat -c%s have any advantage over du -b?
– n.st
Aug 18, 2013 at 11:15
2

I guess one could even combine those commands to decrease readability and save one SSH connection: ssh user@host 'f=FILENAME; tail -c $(($(stat -c%s $f)/10)) $f' > DESTINATION
– n.st
Aug 18, 2013 at 11:43
2

@n.st: They both end up using the same stat() syscall, which is fast, although du also examines all mounted filesystems using stat() before the actual file. But that's not a difference in practice. ⫽ However, if you need to get just the bytes (like in the one-liner in your last comment), then running stat will be easier because it doesn't output the unnecessary filename, making the one-liner shorter than it would have been with du -b | sed ....
– u1686_grawity
Aug 18, 2013 at 12:42
Can similar thing be done for upload ? Like upload only some bytes to server, instead of entire file ?
– Simplecode
Aug 6, 2021 at 14:41

Add a comment |

1 Answer 1