23

I'm looking for a way to sync the secondary drives on my two Macs across the internet. I have consistent VPN connectivity between the two macs, and I can access them as if they are sitting right next to each other.

I'm looking for a way to implement a Dropbox-like experience where if I modify a file on one Mac, the file would get pushed to the other Mac (and vice-versa). I also need the ability to throttle the bandwidth (so our little 1 megabit upload pipe doesn't get overloaded). Any suggestions?

Oh, and using dropbox is kind of out of the question at the moment since my drive is 500GB. Thanks!

6
  • Check for a port of "rsync" which is a Linux tool for just that. It can synchronize your files on each run; then you only need a way to trigger this whenever sg changes.
    – dkellner
    May 12, 2014 at 7:08
  • Do you want all 500GB to be synced?
    – cubuspl42
    May 12, 2014 at 9:00
  • 5
    This would be on-topic on Software Recommendations. See this question (not specific to Mac): softwarerecs.stackexchange.com/questions/129/… May 12, 2014 at 10:42
  • Do you need this too? "If I delete the file on either machine, it should be deleted from the other machine as well, instead of being re-created."
    – Patrick M
    May 12, 2014 at 20:37
  • Isn't this what iCloud is for? hints.macworld.com/article.php?story=20120803092156975 May 12, 2014 at 21:49

9 Answers 9

30

Check out Bittorrent Sync. It has the sync functionality like Dropbox, but without the cloud.

9
  • Downside: Both machines have to be online simultaneously to be able to sync. May 12, 2014 at 13:20
  • 18
    @GeraldSchneider Do you know of something that doesn't have this limitation?
    – Shelvacu
    May 12, 2014 at 13:58
  • 2
    @shelvacu, not possible. If you want it to sync without needing both machines online, you necessitate cloud storage (counter to OP request).
    – HalosGhost
    May 12, 2014 at 15:32
  • 5
    Not necessarily a cloud storage. You need one node that is always online. This can be a computer, a NAS, a cloud service, your self hosted OwnCloud instance as proposed by Lawrence or any similar solution. But as long as you don't have one node always online this limitation always applies. May 12, 2014 at 16:25
  • 3
    I use BitTorrent Sync and I can highly recommend it - I use it in conjunction with Dropbox for files that are too big and not worth taking up space there. One thing to consider though is that it does not deal well with simultaneous edits of files - there is no conflict handling like with Dropbox - the last edited file simply wins. Not necessarily a problem but it depends on your requirements.
    – zelanix
    May 12, 2014 at 23:38
16

If you want to go hardcore, you can use rsync command natively in OsX, like *nix staff.

Before everything, you must have the permission to access to the remote host by ssh. OpenSSH server is preinstalled on Mac OsX and you don't need to install any additional package. Just need to do some system settings. To enable the OpenSSH server on Mac OS X, open System Preferences and click to Sharing. And than, check the Remote Login box to enable SSH, then select the radio button labeled All Users from the Allow Access For section.

You can now access to the remote host by ssh.

Now, you must create a public access key for passwordless access from local guest machine to remote host over ssh. For that you must get your hand dirty a little bit. :)

First, determine if you already have authentication keys. In the Terminal, run:

sudo ls -la /var/root/.ssh

If you see "id_dsa" and id_dsa.pub, then you can skip the rest of this section.

On the client machine, run the following in the Terminal:

sudo ssh-keygen -t dsa -f /private/var/root/.ssh/id_dsa -C "comment about this key"

AFter created access key in local guest machine, you need to copy the guest's public key to the host's authorized_keys file. You can do this with a simple Terminal command that appends the public key to the list of authorized keys:

sudo cat /private/var/root/.ssh/id_dsa.pub | ssh root@remote_host_address 'cat - >> ~/.ssh/authorized_keys'

The command below that will perform an incremental backup of your root filesystem in local guest machine on to your remote host:

/usr/local/bin/rsync -aNHAXx --protect-args --fileflags --force-change --rsync-path="/usr/local/bin/rsync" / root@remote_host_address:/Volumes/Backup/GuestMachine

You can change parameters of rsync for change synchronization behaviors. If you add this command into crontab, synchronization can be run in any time cycles. For example:

sudo crontab -e

append line below to crontab:

*/30 * * * * /usr/local/bin/rsync -aNHAXx --protect-args --fileflags --force-change --rsync-path="/usr/local/bin/rsync" / root@remote_host_address:/Volumes/Backup/GuestMachine

This will be run synchronization in every half hour.

Here is the sample Python script:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

"""
Cron automated synchronization script.

Desc: Synchronize remote folder into local machine. Duplicate processes 
      are disallowed until running process finished.

Usage:
    :: Change required variables as _user, _password, _domain etc.
    :: Edit crontab 
    # crontab -e

    :: Append line below.
    */30 * * * * python synchronizator.py 2>&1 &

Author: Sencer HAMARAT (RecNes)
E-Mail: [email protected]
"""

import shlex
from subprocess import Popen, PIPE
import logging as log
import sys

__author__ = "Sencer HAMARAT"

_user = 'username'
_password = 'password'
_domain = 'example.com'

_expectation = "Enter passphrase for key \'/home/%s/.ssh/id_rsa\':" % _user
_rsync = '/usr/bin/rsync --partial --progress -avvz -e'
_pub_key = '/home/%s/.ssh/id_rsa.pub' % _user
_ssh = '/usr/bin/ssh -i %s' % _pub_key
_remoteDir = '/home/%s/backup/' % _user
_localDir = '/home/%s/backup/' % _user
_command = '%s %s %s@%s:%s %s' % (_rsync, _ssh, _user, _domain, _remoteDir, _localDir)
run_command = shlex.split(_command)

_logFile = "logfile.log"
_logFormat = "%(asctime)s %(levelname)s %(name)s %(process)d %(threadName)s %(module)s:%(lineno)d %(funcName)s() " \
             "%(message)s\n"
log.basicConfig(filename=_logFile, level=log.DEBUG, format=_logFormat)

log.debug(u'Command will run: %s' % _command)

try:
    running_command = Popen(run_command, stdin=PIPE, stdout=PIPE, stderr=PIPE)
    if running_command.poll():
        log.debug(repr(running_command.poll()))
        sys.exit()
    if _expectation in running_command.communicate():
        running_command.communicate(_password)
    print running_command.communicate()
except Exception as e:
    log.debug(repr(e))
finally:
    sys.exit()
4
  • I had considered using rsync, however the problem lies here: "What if the rsync isn't completely finished after the 30 minutes?" I don't want two rsyncs occurring at the same time. Is there any way around this? May 13, 2014 at 8:14
  • 1
    @muncherelli: it's relatively painless to write a small "locking" mechanism around the rsync to prevent simultaneous syncs. Basically, wrap the rsync call in a bash script and you're free to do anything you want.
    – mike3996
    May 13, 2014 at 12:06
  • I added sample script. I hope it will help.
    – Sencer H.
    May 13, 2014 at 13:52
  • This is a great script. Thanks so much for the help. I decided to go with BitTorrent Sync due to the simplicity and bandwidth throttling. (I know you can set max bandwidth usage with rsync, but BTS is way more intuitive). May 14, 2014 at 20:08
8

Owncloud is a possible solution if you want web based access.

You will need your own server to host Owncloud on, however it's more of a server based solution where your 2 clients can access what they need at will, but there will only be 1 copy of the file in question, which will be situated on the Owncloud server.

2
  • Is it still the case that you can only connect to one owncloud? E.g. I can't have my own one and share another one with my brother? May 12, 2014 at 9:59
  • At least the desktop sync clients now support multiple simultaneous cloud connections. But also within one server, you are able to share only specific files/directories, not necessarily everything.
    – ojdo
    May 12, 2014 at 10:41
4

I recently stumbled across Syncthing. Lot of cool feautures.

http://syncthing.net/

2

If Windows, there is a SyncToy tool which does it in three steps. if Mac, it is about ChronSync tool or tri-backup tool

2
  • Hi Ruslan, can you edit your answer to explain why ChronSync or Tri-Backup would be the ideal solution for the OP? Right now, it is a very low quality answer, and doesn't actually answer the how of the question. Thanks May 12, 2014 at 22:44
  • Hi, thank you for your question. Well I am not telling these programs are ideal solution, but I think it is easy enough because they have GUI, that also mean anyone can do it without deep understanding of a bunch of command lines magic-spellings. Also I believe if one has an issue of using listed programs it should be asked in a separate question, like what he's done, what's wrong\strange. It is rather than copying here documentation for them without being asked any particular questions. May 13, 2014 at 0:40
2

Consider Unison. It handles two-way file synchronization. You can set it on a schedule with cron or some other scheduling app.

http://www.cis.upenn.edu/~bcpierce/unison/

Unison is a file-synchronization tool for Unix and Windows. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other.

Unison shares a number of features with tools such as configuration management packages (CVS, PRCS, Subversion, BitKeeper, etc.), distributed filesystems (Coda, etc.), uni-directional mirroring utilities (rsync, etc.), and other synchronizers (Intellisync, Reconcile, etc). However, there are several points where it differs:

1
  • 1
    +1: When bidirectional sync is required, this clearly wins over rsync, while having all its cool features.
    – ojdo
    May 13, 2014 at 20:14
1

I've had good luck with GoodSync with two PC's on the same LAN, but I bet it would still work well in your situation.

Here is their product description:

  • GoodSync syncs files between your computer, mobile devices, FTP, SFTP, Amazon S3, Google Drive, SkyDrive, WebDAV, Azure.
  • GoodSync backs up files to portable drives, mobile devices and to FTP, S3, GDocs, SkyDrive, DAV or another computer.
  • Direct P2P Sync: GoodSync Connect syncs files directly between your computers, without slowness introduced by syncing via 3rd party servers.
  • GoodSync is very fast and syncs files on your command, periodically, by a schedule or in real time (on file change).

You only need GoodSync set up on one of your PC's; it handles two-way synchronization and the kind of sync it does is customizable. Also, it is highly recommended by CNet.

1
  • Hi Jon, can you edit your answer to explain why GoodSync would be the ideal solution for the OP? Right now, it is a very low quality answer, and doesn't actually answer the how of the question. Thanks May 12, 2014 at 22:45
1

Another possible solution is AeroFS.

AeroFS bi-directionally syncs data between your devices while only ever storing your data on your devices - your data never goes near their servers. AeroFS maintains a revision history for your files so you can revert to previous versions when necessary and it provides a tool for resolving conflicts.

AeroFS works on Windows, Mac, Linux, iOS and Android, so it should have you covered.

0

So, if you're not opposed to cloud storage on principle, there are two hardware solutions for external drives:

Space monkey uses an interesting paradigm.. it allocates part of your disk to storing small pieces of other people's data, and also stores your data on lots of other people's drives (encrypted, of course). That way, there is a lot of redundancy in case of disaster, and your machines don't need to be able to see each other or even be connected at the same time.

File transporter is a bit simpler, in that the data is only ever synced on other file transporter drives that you specifically set up to sync.

Both options are good for large amounts of data, and you just buy the hardware once without a subscription fee.

Not the answer you're looking for? Browse other questions tagged .