Is there a way to resume an interrupted scp of a file?

Question

I'm occasionally downloading a very large file via scp and there's a small chance each time of the connection dropping and cutting the transfer mid-way.

Is there a way to resume it?

Answer 1

You can try the following approch: instead of scp use dd to skip over the downloaded part and append the remainder to the file.

sofar=`ls -l ./destfile | awk '{print $5}'`
ssh user@host "dd if=./srcfile bs=1 skip=$sofar" >> ./destfile

Possible optimization: work with big blocks. Let's leave this as a homework.

Answer 2

With scp, no.

If both ends have it, you can use rsync -LvzP remoteserver:path/to/file localfile to transfer a single file.

Answer 3

Yes, there are ways to resume from the point of interruption, but it is not possible using scp. sftp reget filename does what you need. Yarek and Grawity have provided valid solutions that I +1 to both, but for resuming from a point of interruption, I like rsync. The example commands provided both assume you are retrieving a file from a remote server to your local workstation (downloading). Please keep in mind that the final two parameters should be considered source_file and target_file in that order. The syntax of the filename varies based on whether the source or target file is local or remote. If I were sending (uploading) [text] files, I would rewrite the examples provided as:

#From local to remote
sofar=`ssh remote_system ls -l interrupted_file | awk '{print $5}'`;
dd if=source_file bs=1 skip=$sofar | ssh remote_system "cat >> ./interrupted_file"

And to the rsync solution, I add -e ssh. You should consider whether or not you need verbosity, compression, preserve ownership, permissions, timestamp, recurse directories, etc. Check man pages and google, regarding the -L parameter. You might want symlinks to remain as links instead of referencing them.

rsync -P -e ssh local_source_file remoteserver:path/to/interrupted_target_file

Answer 4

There's another solution (besides the rsync or dd solutions stated here) which I'm surprised nobody mentioned: it's the reput command of sftp.

Answer 5

scp -o ConnectTimeout 60

Lets you specify the timeout for a connection. May keep your connection from breaking up.

Answer 6

Yes, if both ends support sftp - after scp remoteuser@remotehost:/absolute/filename . fails you can resume by doing sftp remoteuser@remotehost and then reget /absolute/filename to resume the download.

Answer 7

The accepted answer of yrk does work if you are copying from a remote to a local machine. If in your case you are trying to run scp -3 between two remotes that cannot communicate directly to eachother you can use the following solution. Assume 'source_file' is located on 'host_source' and 'dest_file' is located on 'host_dest'.

1. ssh into host_dest and check the current file size

   ls -l dest_file | awk '{print $5}'

   Write down the output, e.g. 7544684

2. From your local machine:

   ssh user@host_source "dd if=source_file bs=1 skip=7544684" | ssh user@host_dest -T "cat >> dest_file"

Answer 8

I think it should be:

scp -o ConnectTimeout=60

instead of:

scp -o ConnectTimeout 60