I am in the process of creating an offsite backup, and need to do a initial load of data. Currently, that's about 400Gb, give or take 10Gb or so... The backup system is producing files which are about 4Gb each, and has some other, smaller related files also. So, i need to transfer all 400ish gigs to a remote server, but how? What is the best method?
I have full remote access to the server, so i can install anything i need to install. There are Windows, Linux and a Solaris VM running on the box itself, so any of those can be used there, and i have Windows and Linux at home. I have 2 internet connections in house, 10Mb/s uploading on each, so something that could potentially split the number of connections would be handy (kind of like GetRight, but in reverse... PutRight?).
[Update] I have tried the option below of using RSync, but there is only one connection being made between my workstation and server. When uploading large files to Amazon, there is an option of breaking them apart locally and unbreaking them on the server end. multiple files are also send over multiple connections. so, a 4Gb file could be split into 800 ish 5Mb chunks, and sent 4 or 8 at a time, using as much of your upload bandwidth as possible. So, how would i do this automatically? I could hack something together to do it and use SFTP to send the files, but is there an automagic option?