2

I intend getting a vps server real soon for some of the resource-intensive sites I manage. I live in a country where 1Mb/s internet speed is a luxury and constant uninterrupted electricity, a myth.

Here's What I'm Trying To Achieve:

  • How to get the files and directories copied from the source server to the destination server over SSH / SCP.
  • How to still have the files copying between the servers even if my internet disconnects or power supply goes out.
  • How to get the files still copying between both servers if I close my Terminal window.

I'm aware of copying over scp but it doesn't look like it keeps working when I go offline.

4
  • When you say "go offline" are you disconnecting while you're transferring the data? If so, that's your problem right there. The connection must be open to copy data. May 14, 2015 at 14:28
  • IIUC, OP doesn't want to transfer data from/to his machine but between 2 remote servers. If so, there may or may not be a problem if your machine goes down because depending on a remote server settings scp process may be killed or not. Anyway, I recommend rsync at any time of the day for copying a lot of data. May 14, 2015 at 14:30
  • @Volumetricsteve Yeah, the internet disconnects very randomly and after about 5 minutes, comes back, so I'm scared the connection might time out, making me have to restart entire process. May 14, 2015 at 14:32
  • Whatever you do, I advise putting it in a screen. This will preserve the process and allow you to handle it later on if it's taking a while. 'screen -S copyjob', start the copy, Ctrl+a then Ctrl+d to detach. 'screen -ls' will list open screens, and 'screen -r <screenname>' will reattach. May 14, 2015 at 20:19

3 Answers 3

3
rsync -azvP source_directory user@remote:remote_directory

should let you copy files between two computers while saving progress in case of interruptions.

Here's an explanation of each flag, too. http://explainshell.com/explain?cmd=rsync+-azvP

If you would like to allow this to run on a server while not being logged in, you can pass the above command to the nohup command. You can also run this in a tmux session which will allow you to monitor the progress (or restart the command) by attaching the session when you log in to your machine.

You can start a tmux session by typing tmux on your server. Then you can run the rsync command and detach from the tmux session by typing CTRL+b and then d. Logging out won't affect the copying procedure. When you log back in you can type tmux attach and you will be able to view the progress of the copy operation.

Hope this helps! Let me know if I can clarify anything.

2

I strongly suggest you take a look at using the rsync command. The rsync command is useful because it can easily resume transfers. Rsync by default operates over a SSH transport, you simply need the rsync binary on the client and server.

Rsync will detect what has already be transmitted and only transmit the files, and parts of files that have not be transferred.

2

I believe this should solve the problem:

nohup scp -r big-folder remote-machine:/home/user/ &

This command starts the copy process and will continue copying after you log off.

4
  • Wow, so all I needed was nohup, you just saved a life! May 14, 2015 at 14:51
  • hey. One more thing. how do I check the progress of the command? May 14, 2015 at 15:21
  • 1
    You simply log in back to the machine from which you started the scp copy in the nohup, change to the directory where you started it and you will find a file called nohup.out there - all output from the command running in nohup is being saved there. May 14, 2015 at 15:25
  • 1
    I really don't think scp is the best tool for this. It doesn't really do good at resumes.
    – Zoredache
    May 14, 2015 at 17:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .