45

Is there any possibility to use Amazon S3 for Mac OS X time machine backups?

Or in other words: Use S3 rather an external hard disk for time machine backups?

1
  • For any online backup solution: what if you need to restore? How would you restore your system into a working state? And even when just restoring your documents and email: how much time would it take to download all? Or how much time would it take Time Machine to even only display the star field galaxy thingy? See also Best choice for a personal “online backup” in Europe for references to some test results on that.
    – Arjan
    Aug 19, 2009 at 16:04

8 Answers 8

16

I've used arq for years and love it. It is not timemachine (bummer) but does do automatic backups to either Amazon's S3 or Glacier.

Update

As of 5/4/2015, Arq supports incremental backups the following services:

  • Amazon s3 and Glacier
  • Google Drive
  • Microsoft OneDrive
  • DropBox
  • Google Cloud Storage
  • SFTP to your own server
  • DreamObjects
  • Other S3 compatible services
2
  • As I mentioned in another comment, I also used arq finally. Unfortunately if you have a lot of data (as I do with my photos and videos) then S3 gets expensive quite quickly. I'm now using Backblaze backblaze.com.
    – Marc
    Aug 28, 2013 at 9:31
  • 2
    You can set ARQ to use reduced redundancy storage, set a monthly budget, and for even better cost effectiveness, if your data is purely for back up, use Amazon Glacier via Arq to have storage for a fraction of the cost of s3. Sep 16, 2013 at 20:48
18

Using AWS S3 as the storage of Mac Time Machine definitely works now.

Time Machine can backup the data to external device, so the key is mounting S3 bucket as a POSIX FS in Mac.

There are many tools or solutions to support mounting S3 like a local device in MacOSX, such as S3fs, Goofys. Juicefs is a personal recommended SaaS which provides 1TB free quota.

Once you mount the S3 bucket as local device whatever above tools you prefer, you can follow this guide to create a sparse image in S3 mount point(actually it's created in S3 bucket) as the destination of time machine backup.

Also you are recommended to enable S3 IA or intelligent tier for saving cost. You can also use KMS to encrypt the backup data stored in S3.

1
  • Note that the service for Juicefs Free has changed since this answer was made. JuiceFS now only provides a metadata service, your files are stored at your object storage in a public cloud. Therefore you need to prepare a public cloud account.
    – mike
    Mar 9, 2023 at 22:08
4

Edit: I tried this and it didn't work. (Time Machine cannot see the mounted volume/bucket.)

You may be able to use Panic's Transmit app to [mount an S3 bucket as a local Volume][1] and then point Time Machine to that mounted volume as the destination.

I haven't tried this yet, but I plan to.

[1]: https://library.panic.com/transmit/td-install/

4
  • Why the anonymous downvote? Seems like an appropriate suggestion to this question. Thanks user38217 and welcome to superuser.com. I gave you a +1 to compensate.
    – Marc
    Apr 6, 2016 at 9:41
  • 2
    This is really a comment and not an answer to the original question. To critique or request clarification from an author, leave a comment below their post - you can always comment on your own posts, and once you have sufficient reputation you will be able to comment on any post. Please read Why do I need 50 reputation to comment? What can I do instead?
    – DavidPostill
    Apr 8, 2016 at 9:49
  • @DavidPostill I really don't see why his entry shouldn't be an answer. Reading my question followed by his entry it sure feels like an answer. And in case it works fine it could potentially also be a much better answer than the one I've accepted years ago. Sorry for the nitpicking but I sure hope you didn't downvote for that reason.
    – Marc
    Apr 11, 2016 at 11:54
  • 2
    @Marc I didn't downvote. I commented because "You may be able to" and "I haven't tried this yet" does not make for a good answer. It comes across as guesswork (and hence should be a comment).. If he has since tried it ("but I plan to") then the answer should be updated accordingly. My personal opinion is he should try first and if it works then write an answer, otherwise just leave a suggestion in a comment (he has enough rep to comment).
    – DavidPostill
    Apr 11, 2016 at 12:00
2

Haven't tried it myself, but Dolly Drive is designed to create a Time Machine volume that exists "in the cloud"

2
  • 1
    I didn't try it out as well, but this looks like the correct answer to my question. I'm now using "Arq" (www.haystacksoftware.com/arq) for my backups to S3.
    – Marc
    Feb 25, 2011 at 10:35
  • Looks interesting - you should post it as an answer to your own question instead of a comment and then accept if it is what you are using ;) Feb 26, 2011 at 22:21
1

Short answer: No.

Although Time Machine cannot use Amazon S3 as a backup medium, you could set a cron job to rsync your files and folders that you wish to backup OR you could use a GUI program that manages the backups for you, such as Jungle Disk (which is cross platform).

For an even better (and slightly more complicated), you could combine the two and use rsync and Jungle Disk together to backup your data to S3. Check out this gudie for instructions.

Here is a list of backup programs that take advantage of Amazon S3

1

Automatically backup your Mac to Amazon S3 *

There are some great tools already in existence that can do most of the heavy lifting for you. The primary tool for doing remote directory syncs is called s3sync which is a script written in Ruby. Lucky for us OS X comes with Ruby pre-installed so there isn’t much work to get it working.

Here is my step-by-step guide to getting your machine setup to do automatic daily backups to Amazon. I developed these steps on my MacBook Air running Leopard however they should work for previous versions of OS X as well.

Continue Reading...

* I cannot confirm the success of this method

1
  • 1
    Thank you, @Gray. I've redirected to a cached version on web.archive.org.
    – Sampson
    Jul 20, 2015 at 18:07
1

I believe there is an issue in that TimeMachine backups use file links, so that it doesn't need to duplicate the files for each backup version. S3 doesn't understand file links, so either copies the file, taking huge space and bandwidth, or doesn't, meaning downloading a complete version is very fiddly, depending on the --[no-]follow-symlinks parameter, default is to follow them. (I presume the backup services get round this.)

I just needed a backup of individual files, rather than a fully bootable, versioned dump, so I set up a bucket in S3, then used the following bash script:

until AWS_SHARED_CREDENTIALS_FILE="/path/to/credentials/in/mounted/encrypted/file" \
  AWS_MAX_ATTEMPTS=99999 \
    aws s3 sync /path/to/backup/ s3://mybucket/path/ \
      --storage-class DEEP_ARCHIVE \
      --region my-region-1 \
      --output json \
      --sse-c AES256 \
      --sse-c-key "fileb:///path/to/encryption/key/in/mounted/encrypted/file" \
      --no-follow-symlinks \
      --exclude "*.ssh/*" \
      --exclude "*.aws/*"
  do echo "Retrying whole backup unless you ctrl+C within 2 secs"
  sleep 2
done

The encrypted file can be binary, although I kept it unicode so I can store a copy. I put both my AWS credentials and the encryption key in encrypted drives that I can mount while I back up.

I chose deep archive - request to download can take 24 hours, but it is 0.01 cent per gigabyte month.

I timed transfer rates to each AWS location then chose the quickest, cheapest one (prices vary slightly).

This script then loops until sync returns 0 (I've a very unreliable connection).

Cost me $24 to upload the files (400,000 PutObject requests), then $6 a year to store 450GBs.

Files will not be consistent, so this isn't any good for eg, backing up application state at a point in time. But it is a good last resort back up of files, should all my offline backups fail (this was inspired by a home robbery in which laptop and disks were stolen - fortunately they missed a hidden hard disk).

0

Also remember even though it's backed by S3, S3 isn't a file system. You cant read or append to an existing object, it needs to be completely rewritten (Put) every time it changes. I'm not sure what this will translate to in terms of costs with the Macs Sparse Bundling format used by Apples Time Machine where the volume is divided into 8MB chunks.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .