How to synchronize the home folder between multiple computers?

Question

I have three computers at home, and would like to have the /home/ folder tree synchronized between the three.
Any files/folders (except those hidden) that are modified/added/removed in one of the three computers is updated in the other two.

Is rsync enough to do this? and how?

Answer 1

Unison might be a good candidate:

Unison is a file-synchronization tool for Unix and Windows. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other.

It already does 2-way syncs. I think you can daisy-chain the 3 machines with 2-way syncs to get what you need. e.g

machines A, B, C => (A <-> B), (B <-> C), (C <-> A)

It's not ideal, but I couldn't find anything better at the moment.

I've learnt that there are very few things rsync cannot do, and it can probably provide an equal or better solution, but you'll have to wait for an rsync expert to turn up for that solution.

Update: Yes, Unison can sync more than 2 machines. From their user manual:

Using Unison to Synchronize More Than Two Machines

Unison is designed for synchronizing pairs of replicas. However, it is possible to use it to keep larger groups of machines in sync by performing multiple pairwise synchronizations.

If you need to do this, the most reliable way to set things up is to organize the machines into a “star topology,” with one machine designated as the “hub” and the rest as “spokes,” and with each spoke machine synchronizing only with the hub. The big advantage of the star topology is that it eliminates the possibility of confusing “spurious conflicts” arising from the fact that a separate archive is maintained by Unison for every pair of hosts that it synchronizes.

Answer 2

How about to put your files in a version control system like SubVersion or git?

I don't version my entire home dir, but only a handful subdirs with some important stuff. But I am thinking about converting to git since it seem to be better for this type of job. (a quick search is going to send you in the right direction).

Good luck

---

Update: A nice side effect with git is that it is easy to have computer specific files, since you only need to have a branch for that computer (and in git you get that by default). So you don't need to have exactly the same files in all the computer, they only need to be similar, and have a common "main" branch with stuff that you want on all the different computers.

Answer 3

Your main problem when trying to do this is deciding how to merge changes, propagate deletes, and resolve conflicts. This is hard to do in a completely automated way, especially if you've got a 3 computer setup used by multiple people.

If you separate users things get much simpler. Because one user can't be in two places at once (and therefore generate conflicts) you could then setup an rsync job to run on login to "get changes" and logout to "push changes" ... to one of your computers which would be the master ... so at this level of granularity you'd be syncing /home/myuser each time rather than the entire /home/. An added refinement (in case of people not logging out) would be run the push script after a short period of inactivity.

There are lots of other potential solutions, but none that magically solve this problem as far as I'm aware. The first step is probably to think about how you use the machines and come up with a syncing policy to suit your user's behaviour.

Answer 4

This won't entirely provide a solution, but it will give you a start:

Set up a cron job every so often to rsync the files. I use a command like the following:

rsync -alhz --stats --progress --exclude-from '/etc/rsync_backup_excludes.conf' / -e ssh user@10.10.0.1:/path/to/home/

This will use rsync to only copy the needed changes, not re-copy everything every time the command is ran.

--stats and --progress are optional and should probably be excluded from your scripts

the contents of my rsync_backup_excludes.conf file are:

dev/*
proc/*
lost+found/*
mnt/*
sys/*

Answer 5

Take a look at dropbox http://www.getdropbox.com/ Cross platform Win / Linux / Mac

Answer 6

I think you can achieve what you want better by NFS mounting a common home folder. Check out this article http://www.linuxjournal.com/article/4880

Answer 7

Mount /home from one computer over to the other two. Automount works pretty well for this.

Answer 8

PowerFolder - Sync Files, Sync Folders, Remote Storage, Backup and Private File Sharing. Sync home and office PC, share holiday pictures or work together on documents. PowerFolder's secure peer-to-peer technology works over the Internet or in LAN.

Answer 9

Ubuntu One might be what you're looking for. Unfortunately it's still in beta phase, and I have no experience with it, so I'm not sure if it'd work for you.

Answer 10

If you also want to sync configs - tools mentioned above are totally helpless: configs are often changed, and many logfiles will make conflicts so they can't be merged.
My deceision is simple, and stupid :) I've got a Master workstation, and all files from "~" are just copied to the second "Slave" one. The moment i realize I need to modify something - i do it on Master, and Slave catches these changes on sync.

Also, some bash scripts should run different on these machines, so I edited my /etc/bash.bashrc:

export OOHOST=Master

Now, scripts know which host they're serving ;)

Answer 11

With gftp or filezila you can sync two folders .

Answer 12

check this out ! http://code.google.com/p/lsyncd/

Answer 13

I use SyncTwoFolders for Mac (http://tcfj.pagesperso-orange.fr/site/index.html) and Syncback for PC (http://www.2brightsparks.com/syncback/) - the first one for my 3macs at home (inclusive laptop), and the second for my PC at work. With the various back-up and sync options this is a breeze once pre-sets have been set up. Easy-peasy! I'm sure there are other versions of similar software avialable for Linux and Ubuntu.

Answer 14

The workarounds I would consider:

• Dropbox.
  Main drawbacks: it's not free for any reasonable amount of data. Otherwise it works flawlessly IMHO

• A NAS server. Put your files on a NAS server, they became reasonably affordable lately (from 150 $ or so) and a good one doesn't consume much energy. No synching issues ever.
  Main drawback: slower than a local copy

Answer 15

What about this:

-Have multiple USB HDD, each with a copy, so if one fails there are the other two. -Make changes to one of them, add, replace and/or delete some files and/or folders.

How to sync the other two or more?

Also a bit more complex: -Not all HDDs must have all, some only some part.

A very complex example with a collection of paths and an alias for each one, so better understand it: -AliasA -> ./MyDir1/MyDir2/* -AliasB -> ./MyDir1/MyDir3/* -AliasC -> ./MyDir4/MyDir5/* -AliasD -> ./MyDir6/*

Now to not be very complex, only three HDD, called HDD1, HDD2 and HDD3, each of a different size, type, etc... (NTFS or EXT4 or FAT32, etc)... -On HDD1: AliasA and AliasD -On HDD2: AliasC, AliasB and AliasD -On HDD3: AliasA, AliasC and AliasD

As you can see AliasB is only on one, so no sync needed, but for the rest sync is needed since they are on more than one.

As you can see AliasA is on two HDDs: HDD1 and HDD3, so any change made to both must populate to the other one.

As you can see AliasC is on two HDDs: HDD2 and HDD3, so any change made to both must populate to the other one.

As you can see AliasD is on all three HDDs: HDD1, HDD2 and HDD3, so any change made to any one must populate to the other two.

The idea is to have relative paths replicated on some external disk, of course, for BackUp purposes, but also to be used on Set Top Box TVs.

Files can be Photos or Videos of Holidays.

Some HDD can be the SD memory cards of each Photo camera, and also microSD of Video camera, and two USB HDD one of 1000GB and another of 750GB, so both Set Top Boxes TV can reproduce at same time different content.

I want multiply copies just in case one fails (also i save on DVD+R for an extra BackUp place).

I am a little paranoic, i know... i use more than 6 copies, but i can not afford to loose them, they can not be recorded again, they are my trips on holidays!!!

Well that was just a complex example why someone would want to sync local folder structure... and why not all root!

Answer 16

Try dvcs-autosync.

This is based on git, uses XMPP to notify conected clients of file changes and reacts on file change events via inode changes. So it always gets informed right after the file change, in case of conflicts it relies on the proven methods of git.

I've been using it for a while now as a Dropbox replacement via SSH public keys and it really does the job.