1

I need to transfer files from one CentOS server to another. I'll transfer 5MB files about every 10 minutes. I do not need encryption.

What is an easy way for a fast transfer of those files? Is there something simpler than FTP?

2
  • you should rephrase 'simpler' in a way that describes what kind of problem you have with ftp or with your current setup. and if your files are 5MB in numbers or in bytes each (5*10^6 files or files a 5*10^6 bytes)
    – akira
    May 21, 2012 at 7:35
  • 1
    Trivial FTP (protocol) made for its simplicity.
    – Amicable
    May 21, 2012 at 8:20

8 Answers 8

5

A distributed filesystem or use crontab to perform an scp/rsync job.

scp = secure remote copy, you will need to setup identity file and authentication key for key authentication i.e without password

crontab job:

crontab -e

insert this into the file (for ):

*/10 * * * * scp [email protected]:/remote/path/file /save/to/path

IF everything is harder than a ftp, then use a USB thumb stick!!!

11
  • "simpler" ... and you propose "distributed filesystem"? AND / OR "scp" and "rsync"? really? :)
    – akira
    May 21, 2012 at 7:19
  • 2
    in terms of hassle free, it is easier.
    – c2h2
    May 21, 2012 at 7:21
  • 2
    crontab with scp and pre-sharedkey would be one of the simplest setup and very low maintenance
    – Sibster
    May 21, 2012 at 7:27
  • @Sibster: crontab with ftp is as simple as scp.
    – akira
    May 21, 2012 at 7:32
  • 1
    @akira, ftp servers have a long history of vulnerabilities, and all linux servers have/need ssh, few need ftp so why install another possible attack vector
    – Sibster
    May 21, 2012 at 8:08
1

netcat is simpler since it is just raw bytes (ala cat) over the wire. but you would need a listening netcaton the other side as well .. "at the right time" :)

receiver> nc -l -s 0.0.0.0 -p 12345

and then you send the file via

sender> nc receiver 12345 < file

this is the most simple way of transfering files from one machine onto the other. no authentification, no encrytion, nothing but raw bytes.

but i won't recommend this in any way to reliably transfer files over the net. use ssh/scp.

1

I would say rsync is the way to go.

Much easier to script the xfer than ftp can be, and quite smart about how it works (resuming, diffs, etc). You can also set it up to periodically scan the directory itself, and upload the new/changed files, so you can skip a crontab.

Normally rsync is slient, so if you want to see what it is doing the you need

A fairly nice overview/tutorial: http://everythinglinux.org/rsync/

(copy recursively, keep symlinks/users, verbose, checksum)
rsync --progress -avz  
       /copy/files/*.tar.gz 
       username@other-server:/destination/folder

As for distributed filesystems, that sounds complex. Can you simply mount the same nfs volume on both machines? If only one is writing, you have nothing to worry about.

1
  • Just and extension to this, if you use something like openfiler.com you can have it share an NFS to each of your servers and it will handle file locking, so both (all) servers would be allowed read / write access
    – sgtbeano
    May 21, 2012 at 7:50
1

Do you want it to be automatically?

sshfs is a good way to go if not, just install sshfs and run

mkdir /mount/point
sshfs user@server:/remote/path /mount/point

and enter your password in the prompt, then the remote path looks like a local dir on your computer. Ofcourse you will need to have an ssh-server on the remote end. OpenSSH is a good alternative

0
0

Another option is to set up an NFS server on one node and mount it on the other node.

0

If the transfer is one direction only, you could roll out a simple HTTP server such as mini-httpd, publish the files to a directory that mini-httpd exposes, and let the destination download them with wget. mini-httpd is the first thing that comes to mind when I think of a simple HTTP server, but I'm sure there's simpler ones (can't you technically have a webserver setup with bash? isn't there like a Perl one-liner somewhere that implements a simple webserver?)

If you already have a web server on this host not exposed to the public you could reuse that.

2
  • There's a python one liner for a server I think. I use lighttpd
    – Rob
    May 21, 2012 at 14:08
  • python -m SimpleHTTPServer
    – akira
    May 22, 2012 at 5:53
0

wput is a command-line ftp-client that makes easier uploading files, with just one single command:

wput myfile.tar ftp://user:[email protected]/mydir/

Of course, you could do the same from the other side using wget/curl and a web server.

-1

Another option is the MidnightCommander (mc). If you grew up with DOS and NortonCommander.

2
  • which part of mc is an option? coz it has ftp-integrated?
    – akira
    May 21, 2012 at 12:44
  • I thought of the shell connection in mc. It's true that is has ftp too.
    – ott--
    May 21, 2012 at 13:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .