I am backing up my personal photo/video library to Amazon S3 (several hundred GB).

I am currently using Cyberduck on my Mac to drag & drop, but I've already had a few fail without a clear retry/resume behavior.

I'd like to find a way to upload a directory with confidence that they are all transferred completely, and with a clear way to retry/resume any that failed.

What would you recommend? A command line approach is fine.

  • If you're on a Mac, is there a reason you can't just mount the SSH connection as a disk and run Time Machine against that disk? (I don't use a Mac myself but I don't see a reason why this wouldn't work, and AFAIK Time Machine is incremental which saves you bandwidth) – pzkpfw Nov 30 '15 at 12:43
  • woah, that's kind of a crazy idea. Never thought of using S3 with TM. However, I don't need TM and would rather do a one time full backup. I mentioned that I'm on a Mac, but I'm curious if there is something that would work with Linux too. For example, would rsync work for this? I'm not familiar with it enough to know which flags to use, but it seems like a reliable tool. – Andrew Nov 30 '15 at 23:22
  • 1
    rsync should work and is easy to use, just google some examples. There's also duplicity which wraps around rsync to create a time machine-like backup solution, works well. See easyengine.io/tutorials/backups/duplicity-amazon-s3 – pzkpfw Dec 1 '15 at 7:17

According to your problem you can try Bucket Explorer

Features of Bucket Explorer

  • You can browse buckets and the files stored at Amazon S3.
  • Upload and download files, to and from Amazon S3 buckets.
  • Bucket Explorer works on every OS where Java is supported.

Bucket Explorer comes with Multipart upload feature.This allows you to upload a single object as a set of parts. In upload process, each part is a portion of the object's data, and each part can be uploaded independently. Multipart Upload offers many benefits over simple upload, especially when object size is very large.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.