Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a machine a couple of hops away, and I need to set up port forwarding in order to transfer files.

Edit: To be clear, the multiple hops are needed in order to access the remote machine. From my machine, I set up a VPN, where I can access 10.255.x.x - that is the only machine I can connect to via the VPN. Once logged into .x.x, I can then connect to other machines - .y.y being one of those.

From my machine:

ssh -L 4567:localhost:4567 me@10.255.x.x

Then from that machine:

ssh -L 4567:localhost:22 me@10.255.y.y

I can then

scp -P 4567 me@localhost:/path/to/large/file.gz .

I left this running overnite, only to find that the transfer died at some point.

I've seen a couple of suggestions to use rsync over ssh to resume the transfer, but I'm unclear how to set that up. Is this possible?

share|improve this question
    
What are all these hops for? Wouldn't scp me@10.255.x.x:/path/to/large/file.gz . achieve exactly the same? Which version of scp (ssh) is installed on client and server? –  Dennis Mar 6 '13 at 13:38
    
@Dennis: scp does not support resuming a transfer - what you suggest will restart the download. –  chris Mar 6 '13 at 13:51
    
Some versions do (my laptop's scp does, my VPS's doesn't). The reason I'm asking is because the hops seems unnecessary to me, and rsyncing without hops will be a lot easier. –  Dennis Mar 6 '13 at 13:54
    
This answer might help you as well: unix.stackexchange.com/a/43097/14084 –  Bernhard Mar 6 '13 at 13:58
add comment

3 Answers 3

With some versions of scp (the version on the source computer seems to be the key), just re-executing the scp command will suffice to resume the transfer. But be careful! If your version doesn't support partial transfers, the partial file will simply be overwritten.

The following rsync switches are handy for resuming a broken transfer if scp doesn't support it:

     --append                append data onto shorter files
     --append-verify         like --append, but with old data in file checksum
 -e, --rsh=COMMAND           specify the remote shell to use
     --progress              show progress during transfer

The command

rsync --append-verify --progress --rsh="ssh -p 4567" me@localhost:/path/to/large/file.gz .

should have the desired effect. Note that the -p switch must be lowercase for ssh.

share|improve this answer
    
The hops are needed because of the way the network on the destination server is configured. The problem with using rsync over ssh is that I need to use the local port 4567, the file does not exist on the 10.255.x.x server. –  chris Mar 6 '13 at 14:03
    
My mistake. I was assuming that 10.255.y.y was the client computer. –  Dennis Mar 6 '13 at 14:10
    
I've updated my answer. –  Dennis Mar 6 '13 at 14:24
add comment

rsync uses ssh by default, you may have to specify the exact ssh command using rsync's -e switch. It also has a --partial which should keep the incomplete file around so it can resume the transfer.

share|improve this answer
add comment

I can't really see what's the point of your tunnels, so here's how I would go at solving your problem:

ssh -R $tunnelPort:localhost:$localSSHPort $remoteUser@$remoteHost -p $remoteSSHPort

this opens a reverse tunnel from the local machine (localhost) to the remote machine (remotehost). $tunnelPort is the port on which the tunnel is to be found on remotehost. $localSSHPort is the port on which the local sshd runs and $remoteSSHPort is where remotehost's sshd listens.

Now that you're inside the (reverse!) tunnel you can more or less do what Dennis proposed:

rsync --apend-verify -az -e 'ssh -p $tunnelPort' /path/to/large/file $user@localhost:/destination

the -a and -z flags are explained in rsync's man page, so I will not elaborate on them. What happens now is that when you run rsync on remotehost as above it pushes all the data into port $tunnelPort which is then forwarded to your localhost's sshd $localSSHPort.

Hope that helped.

share|improve this answer
    
@ingenous: I added an explanation for the tunnels. –  chris Mar 6 '13 at 15:04
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.