I need to transfer files from one CentOS server to another. I'll transfer 5MB files about every 10 minutes. I do not need encryption.
What is an easy way for a fast transfer of those files? Is there something simpler than FTP?
I need to transfer files from one CentOS server to another. I'll transfer 5MB files about every 10 minutes. I do not need encryption.
What is an easy way for a fast transfer of those files? Is there something simpler than FTP?
A distributed filesystem or use crontab
to perform an scp
/rsync
job.
scp = secure remote copy, you will need to setup identity file and authentication key for key authentication i.e without password
crontab job:
crontab -e
insert this into the file (for ):
*/10 * * * * scp [email protected]:/remote/path/file /save/to/path
IF everything is harder than a ftp, then use a USB thumb stick!!!
netcat
is simpler since it is just raw bytes (ala cat
) over the wire. but you would need a listening netcat
on the other side as well .. "at the right time" :)
receiver> nc -l -s 0.0.0.0 -p 12345
and then you send the file via
sender> nc receiver 12345 < file
this is the most simple way of transfering files from one machine onto the other. no authentification, no encrytion, nothing but raw bytes.
but i won't recommend this in any way to reliably transfer files over the net. use ssh/scp.
I would say rsync
is the way to go.
Much easier to script the xfer than ftp can be, and quite smart about how it works (resuming, diffs, etc). You can also set it up to periodically scan the directory itself, and upload the new/changed files, so you can skip a crontab.
Normally rsync is slient, so if you want to see what it is doing the you need
A fairly nice overview/tutorial: http://everythinglinux.org/rsync/
(copy recursively, keep symlinks/users, verbose, checksum)
rsync --progress -avz
/copy/files/*.tar.gz
username@other-server:/destination/folder
As for distributed filesystems, that sounds complex. Can you simply mount the same nfs volume on both machines? If only one is writing, you have nothing to worry about.
Do you want it to be automatically?
sshfs is a good way to go if not, just install sshfs and run
mkdir /mount/point
sshfs user@server:/remote/path /mount/point
and enter your password in the prompt, then the remote path looks like a local dir on your computer. Ofcourse you will need to have an ssh-server on the remote end. OpenSSH is a good alternative
Another option is to set up an NFS server on one node and mount it on the other node.
If the transfer is one direction only, you could roll out a simple HTTP server such as mini-httpd
, publish the files to a directory that mini-httpd
exposes, and let the destination download them with wget
. mini-httpd
is the first thing that comes to mind when I think of a simple HTTP server, but I'm sure there's simpler ones (can't you technically have a webserver setup with bash
? isn't there like a Perl one-liner somewhere that implements a simple webserver?)
If you already have a web server on this host not exposed to the public you could reuse that.
wput is a command-line ftp-client that makes easier uploading files, with just one single command:
wput myfile.tar ftp://user:[email protected]/mydir/
Of course, you could do the same from the other side using wget/curl and a web server.
Another option is the MidnightCommander
(mc). If you grew up with DOS and NortonCommander.