Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a big gzipped file which is 2.5G in size, I tried scp it from the remote server, after the transfer has finished, I md5sum checked it, it failed. The transfer was completed and the file size(in bytes) are exactly the same.

My question is how can I transfer this file reliably with scp or with any other tool?


share|improve this question
Please improve your question with any other information you might have, i.e., if you are sure that the transfer completed, are the file sizes identical? What are the source and target operating systems and file systems? What kind of access do you have on the remote host? Can you run rsync or a simple web server there (via Python, perhaps) to then get the file with wget or curl? – roguesys Mar 9 '12 at 6:43

rsync will do its best to make sure that the file transferred is intact, provided you don't override its normal behavior.

share|improve this answer
Does -e option count as 'override normal behavoir'? – Shawn Mar 9 '12 at 6:45
Nope. rsync isn't that concerned about the transport. As long as it can talk to rsync on the other side, it's all good. – Ignacio Vazquez-Abrams Mar 9 '12 at 6:58
Do you mean an rsync server on the remote? I don't have one but it looks running fine. – Shawn Mar 9 '12 at 7:04
rsync running on the local system opens a shell on the remote system and runs rsync there, and then the two instances of rsync communicate over the link. – Ignacio Vazquez-Abrams Mar 9 '12 at 7:06
Turns out rsync isn't reliable either, I downloaded a couple of large files with rsync, some were fine some were not. – Shawn Mar 13 '12 at 1:30

Bittorent, DirectConnect clients also verify checksums on transfer and re-transfer broken blocks

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .