Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I want to transfer files (a music folder) between two Linux computers. After searching for the best way to do this, I've seen that there are lots of ways of doing this. I know this has been asked a lot, everywhere and all the time. The main problem with this is that there is no clear, recent consensus on one best way to do this task in 2011 for Linux beginners (even depending on some parameters).

So in the spirit of the Stack Exchange websites, I want this not to be related to my particular situation, but more of a guide to others as well on how to transfer files between two Linux computers over a local network. I think a wiki would be useful for many.

Here's what I found so far:

  • ssh
  • sshfs
  • scp
  • sftp
  • nfs
  • samba
  • giver

What is the easiest? Most flexible? Simplest? Best solution? What are the pros and cons of each? Are there other (better) options? What are the parameters in choosing the best method (solution might depend on number of files, filesize, easiness vs. flexibility, ...)?

share|improve this question
    
+1 for sftp - for it's ease of use, and small learning curve –  Alex Waters Aug 22 '11 at 6:01
1  
Could anyone explain where rsync comes to play in all of this? –  Konerak Aug 22 '11 at 8:39
    
jonallard, please don't add the answers to the question (doesn't really make sense to do this, does it?) -- if you feel some answers need additional info, you can suggest edits on them, or create your own answer which summarizes everything! –  slhck Aug 22 '11 at 10:19

8 Answers 8

up vote 32 down vote accepted

In a Linux environment, for both security and ease of use, ssh is the best way to go. SSH, SSHFS, SCP, and SFTP as you list are all just different services built on top of the SSH protocol. SCP is very easy to use, it works just like CP but you can provide user and machine names in the path. So, we might do a CP like cp ~/music/ ~/newmusic/, but we could just as easily do scp ~/music/ user@host:~/newmusic to send it to the computer named host. That's it - we don't need to set anything up. You'll be prompted for the account password on the other machine if you don't have certificate or some other authentication set up (scp shares those settings with ssh, of course).

SFTP is a tool that makes it easy to do a lot of operations on a remote file system - it works just like FTP, but it runs through SSH so it's secure and requires only an SSH server. man sftp will tell you all about how to use it. I don't use SFTP just to move a folder between two machines, it's more useful when you have a lot of operations to do, like if you're rearranging files on another computer.

SSHFS just extends SFTP in to a file system: it allows you to mount a virtual host in to your file system, so the network stuff happens totally transparently. SSHFS is for semi-permanent setups, not just a one-time file transfer. It takes some more effort to get set up, which you can read about on the project website.

If you need to work in a mixed-OS environment, Samba becomes your next best bet. Windows and OS X support Samba completely automatically, and Linux does as well although it's sometimes rough to use.

share|improve this answer
2  
Exactly the kind of answer I was wishing for: complete, exhaustive, detailed, to the point. –  jonallard Aug 22 '11 at 5:12
1  
One thing though, for scp to work, do we need to set up some kind of ssh server, listener or unblock something on the other side? I'm getting "Connection refused" errors. –  jonallard Aug 22 '11 at 5:28
1  
scp uses ssh, so it will work if SSH works. This means, of course, that you need to have an SSH server running (default in every linux distro I'm aware of) and a connection must be possible (firewalls, NAT, etc must have the appropriate exceptions). –  jcrawfordor Aug 22 '11 at 5:42
7  
Apparently openssh-server has to be installed in Ubuntu Natty. –  jonallard Aug 22 '11 at 5:51
2  
Note that ssh uses encryption, which will cause some additional overhead. If the computers involved have fairly slow CPUs, this might make a difference. In that case netcat or similar (see Caspar's answer) might be preferrable. Of course only if you don't actually need encryption (in a protected LAN). –  sleske Aug 22 '11 at 7:25

My personal favorite for cases where security doesn't matter is netcat + tar:

To send a directory, cd to inside the directory whose contents you want to send on the computer doing the sending and do:

tar -cz . | nc -q 10 -l -p 45454

On the computer receiving the contents, cd to where you want the contents to appear and do:

nc -w 10 $REMOTE_HOST 45454 | tar -xz

Replace $REMOTE_HOST with ip / hostname of computer doing the sending. You can also use a different port instead of 45454.

What's actually happening here is that the 'receiving' computer is connecting to the sending computer on port 45454 and receiving the tar'd and gzip'd contents of the directory, and is passing that directly to tar (and gzip) to extract it into the current directory.

Quick example (using localhost as a remote host)

Computer 1

caspar@jumpy:~/nctest/a/mydir$ ls
file_a.txt  file_b.log
caspar@jumpy:~/nctest/a/mydir$ tar -cz . | nc -q 10 -l -p 45454

Computer 2

caspar@jumpy:~/nctest/b$ ls
caspar@jumpy:~/nctest/b$ nc -w 10 localhost 45454 | tar -xz
caspar@jumpy:~/nctest/b$ ls
file_a.txt  file_b.log
share|improve this answer
    
kinda reminds me of sendnet and recnet from Novel's ipx services of the olden days... –  sum1stolemyname Aug 22 '11 at 12:33
2  
## netcat + bzip2 may be a little faster on slow connections ## Sending server # cat file.txt | bzip2 -c | nc -l 1234 ## Receiving server # nc $sending_ip 1234 | bzip2 -cd > file.txt –  shantanuo Aug 24 '11 at 3:20
    
@Caspar: How about editing this answer to suggest bzip2 or lzma compression? –  einpoklum Nov 23 '13 at 18:59
    
the -q option indicates that you are using openbsd-netcat, while gnu-netcat is also quite common (default in Arch Linux). Could you extend your answer to include the syntax of gnu-netcat? –  Sebastian Jan 6 at 20:48

For one time moves, scp is recommended.

But if you find that this dir may work and you need to move it many times to keep the other position updated then you can use rsync (with ssh).

Since rsync has a lot of args I usually put it in a small shell so I get it right (every time). The idea is to only send things that has changed since the last time it ran.

#!/bin/bash

user="nisse"
host="192.168.0.33"

echo "Sync: /home/media/musik/"
rsync --archive --delete -v --progress -e "ssh -l $user " /home/media/musik/ $host:/home/media/musik/

This will move a dir called "/home/media/musik/" from the local computer to the the pc called 192.168.0.33, using user "nisse". And delete anything on the target that dont exist on the local pc.

share|improve this answer
    
+ for rsync, which seems a tad faster and is very nice if you have to sync the directory later on –  Wiesław Herr Aug 23 '11 at 14:55

The fastest is probably netcat (as caspar described).

I like the combination of tar & ssh, which is secure and still fast:

On the source

tar -cf - . | ( ssh user@target && cd /target/path && tar -xf - )

Doing that as root, it preserves file permissions. Or use -p on both sides. Also -S might be considered if you have sparse files.

It's possible to reduce the encryption overhead of ssh if you use arcfour as cipher which works with openSSH:

tar -cpSf - . | ( ssh -c arcfour user@targethost && cd /target/path && tar -xpSf - )

To update the remote path, rsync is perfect:

rsync -av --sparse --delete -e "ssh -c arcfour" . root@targethost:/target/path
share|improve this answer

I would suggest rsync as it will copy files incrementally. You can set it up to copy either only modified or new files only once you have done the initial update. You can use ssh as a transport layer if you wish.

share|improve this answer

If it absolutely has to be done over the LAN, I'd use rsync, as it will pick up where it left off if it gets interrupted. It also has a few other tricks for minimizing the amount of data that gets transferred, although I doubt that many/any of them would be relevant to the case of copying a music library to a virgin location. If security is a concern, just set RSYNC_RSH=ssh first and the data will be tunneled over ssh.

If I were actually doing it, though, I probably wouldn't use the LAN at all. I'd copy the files onto, then off of, a USB hard drive. In my experience, this can easily be multiple orders of magnitude faster than going over the LAN, despite having to copy the files twice - USB 2.0 is rated for 480Mbps, which is faster than anything short of gigabit ethernet, plus it's less sensitive to conditions which will degrade the performance of a LAN. It's also completely OS-independent, provided you use a file system that all involved machines can handle - I'd recommend VFAT/FAT32, since that's pretty much universal.

share|improve this answer
    
I'm a fan of the (so-called) sneakernet as well, but it's worth pointing out that while USB 2 is supposed to be able to get 480Mbps, I've only ever seen it get 30MB/s (about 240Mbps).. maybe I just have cheap USB<->SATA hardware ;) Also, FAT32 is pretty much universal but is a no-go for copying things like DVD images due to the 4gb filesize constraint; it's worth pointing out so people don't get too frustrated by the "out of space" error message which Windows (at least) gives. –  Caspar Aug 23 '11 at 0:55
    
@Caspar: Good caveats! Thanks for mentioning them; I always forget about the FAT32 file size limit... –  Dave Sherohman Aug 23 '11 at 9:09

I use Unison, which is an awesome file synchronizer over many different protocols. You can configure it to use scp, rcp, ftp or even locally on the filesystem between two folders. I use it to synchronize my music library, as it can transfer multiple files at once over the network and is really tuneable in its configuration. I keep my music collection backed up and in sync over 2-3 computers. It will only copy changed files, and does so by keeping an index on both ends of the transfer so as to be able to tell when a client has changed the file or when the server file has changed.

Your mileage may vary, but it's certainly a lot better than scping your entire music collection every time you add a new song :)

share|improve this answer

I would recommend you trying alternatives instead going straight with SSH for moving files inside your own LAN as the overhead is IMMENSE. I would go with Caspar's solution if this one for whatever reason won't work for you:

At the source: $ python3 -m http.server {PICK_YOUR_PORT} On destination: $ wget -r {ip / hostname}:{port}/{File / Directory}

This won't be only lighter than using SSH but far faster with speeds ranging 45~65MiB on standard CAT6 UTP. If you really want to squeeze the most out of the connection try replacing wget with lftp and using pget -n20 and mirror -r commands.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.