Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I need to transfer files from one CentOS server to another. I'll transfer 5MB files about every 10 minutes. I do not need encryption.

What is an easy way for a fast transfer of those files? Is there something simpler than FTP?

share|improve this question
    
you should rephrase 'simpler' in a way that describes what kind of problem you have with ftp or with your current setup. and if your files are 5MB in numbers or in bytes each (5*10^6 files or files a 5*10^6 bytes) –  akira May 21 '12 at 7:35
1  
Trivial FTP (protocol) made for its simplicity. –  Amicable May 21 '12 at 8:20
add comment

8 Answers

A distributed filesystem or use crontab to perform an scp/rsync job.

scp = secure remote copy, you will need to setup identity file and authentication key for key authentication i.e without password

crontab job:

crontab -e

insert this into the file (for ):

*/10 * * * * scp [email protected]:/remote/path/file /save/to/path

IF everything is harder than a ftp, then use a USB thumb stick!!!

share|improve this answer
    
"simpler" ... and you propose "distributed filesystem"? AND / OR "scp" and "rsync"? really? :) –  akira May 21 '12 at 7:19
2  
in terms of hassle free, it is easier. –  c2h2 May 21 '12 at 7:21
2  
crontab with scp and pre-sharedkey would be one of the simplest setup and very low maintenance –  Sibster May 21 '12 at 7:27
    
@Sibster: crontab with ftp is as simple as scp. –  akira May 21 '12 at 7:32
1  
@akira, ftp servers have a long history of vulnerabilities, and all linux servers have/need ssh, few need ftp so why install another possible attack vector –  Sibster May 21 '12 at 8:08
show 6 more comments

netcat is simpler since it is just raw bytes (ala cat) over the wire. but you would need a listening netcaton the other side as well .. "at the right time" :)

receiver> nc -l -s 0.0.0.0 -p 12345

and then you send the file via

sender> nc receiver 12345 < file

this is the most simple way of transfering files from one machine onto the other. no authentification, no encrytion, nothing but raw bytes.

but i won't recommend this in any way to reliably transfer files over the net. use ssh/scp.

share|improve this answer
add comment

I would say rsync is the way to go.

Much easier to script the xfer than ftp can be, and quite smart about how it works (resuming, diffs, etc). You can also set it up to periodically scan the directory itself, and upload the new/changed files, so you can skip a crontab.

Normally rsync is slient, so if you want to see what it is doing the you need

A fairly nice overview/tutorial: http://everythinglinux.org/rsync/

(copy recursively, keep symlinks/users, verbose, checksum)
rsync --progress -avz  
       /copy/files/*.tar.gz 
       username@other-server:/destination/folder

As for distributed filesystems, that sounds complex. Can you simply mount the same nfs volume on both machines? If only one is writing, you have nothing to worry about.

share|improve this answer
    
Just and extension to this, if you use something like openfiler.com you can have it share an NFS to each of your servers and it will handle file locking, so both (all) servers would be allowed read / write access –  sgtbeano May 21 '12 at 7:50
add comment

Do you want it to be automatically?

sshfs is a good way to go if not, just install sshfs and run

mkdir /mount/point
sshfs user@server:/remote/path /mount/point

and enter your password in the prompt, then the remote path looks like a local dir on your computer. Ofcourse you will need to have an ssh-server on the remote end. OpenSSH is a good alternative

share|improve this answer
    
Thanks for editing my typo Deltik. –  Wilhelmsen Jun 18 '12 at 17:11
add comment

Another option is to set up an NFS server on one node and mount it on the other node.

share|improve this answer
add comment

If the transfer is one direction only, you could roll out a simple HTTP server such as mini-httpd, publish the files to a directory that mini-httpd exposes, and let the destination download them with wget. mini-httpd is the first thing that comes to mind when I think of a simple HTTP server, but I'm sure there's simpler ones (can't you technically have a webserver setup with bash? isn't there like a Perl one-liner somewhere that implements a simple webserver?)

If you already have a web server on this host not exposed to the public you could reuse that.

share|improve this answer
    
There's a python one liner for a server I think. I use lighttpd –  Rob May 21 '12 at 14:08
    
python -m SimpleHTTPServer –  akira May 22 '12 at 5:53
add comment

wput is a command-line ftp-client that makes easier uploading files, with just one single command:

wput myfile.tar ftp://user:[email protected]/mydir/

Of course, you could do the same from the other side using wget/curl and a web server.

share|improve this answer
add comment

Another option is the MidnightCommander (mc). If you grew up with DOS and NortonCommander.

share|improve this answer
    
which part of mc is an option? coz it has ftp-integrated? –  akira May 21 '12 at 12:44
    
I thought of the shell connection in mc. It's true that is has ftp too. –  ott-- May 21 '12 at 13:16
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.