up vote 1 down vote favorite
share [g+] share [fb]

When I upload files to a server through HTTP or FTP, I normally download them again just to check that they were successfully uploaded (I use md5 checksums to check that).

Is this necessary? Can I be sure that file uploads through HTTP or FTP are successful, if the software used to upload them (for example the browser when using SkyDrive or similar services, or FTPZilla when using FTP, etc) did not give me any errors?

Oh, and I know about rsync, no need to suggest it. This is for cases where rsync is not an option.

link|improve this question
88% accept rate
feedback

1 Answer

up vote 2 down vote accepted

Success in terms of whether or not the packets of data actually arrived there as it was sent is usually determined by the underlying protocol, in this case TCP. Often this is plenty to assume success as most file transfer protocols use this to determine if any data corruption occurred at the network level.

I assume you mean the entire file though, in which case yes, checksumming is the best way to verify uploaded data.

However there is no reason you need to download them again to verify them, you can perform the checksumming on the server itself. Specifically with FTP there are solutions which perform checksumming of files as part of the upload process itself, for example drftpd with the zipscript plugin uses .sfv files to check uploaded files on a directory level.

link|improve this answer
So FTP natively supports checksumming (+1 for the info)? Is this enabled by default on popular clients such as FileZilla, or generally it requires downloading a plugin? What about HTTP, should I assume a browser is smart enough to check for success of the transfer at the TCP level (and that servers are smart enough to check that the data came through)? – Camilo Martin Feb 8 at 16:18
Agreed. If you have a shell account on the server, you could hash the files. Better yet, by having to hash the files locally, you can keep a record for yourself of hashes of your files which can come in handy some day. – Synetech Feb 8 at 16:20
@Synetech If I had a shell account on every server where I store files, I'd use rsync. This question is more for cases like SkyDrive, where you're forced to use a simple http ulpoad/download process. Also, I'm curious of this checksumming feature of FTP. – Camilo Martin Feb 8 at 23:12
Checksumming as part of FTP has to be something supported by the server I'm afraid. In your case you're limited by what your service provider is offering. You should ask if they have a feature to perform checksumming of uploaded files against an .sfv (basically a file uploaded with a dir containing filename and CRC of file), which could then confirm everything is OK (usually with a pseudo-dir specifying 100% of files belonging in this dir are uploaded). – deed02392 Feb 9 at 10:04
1  
Basically, TCP and the lower level protocols (OSI L2) have checksumming built in. They are technically weak compared to modern standards but this would only be a problem if you were operating on a very unreliable connection. en.wikipedia.org/wiki/… citeseer.ist.psu.edu/stone00when.html – deed02392 Feb 9 at 22:32
show 2 more comments
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.