Questions tagged [amazon-s3]

Amazon S3 (Simple Storage Service) is a web service which provides storage and various interfaces for accessing data stored there.

Filter by
Sorted by
Tagged with
85 votes
7 answers
155k views

Uploading files to S3 account from Linux command line

I've got several large files sitting in my Linux hosted account that I need to upload to my S3 account. I don't want to download them first and then upload them into S3. Is there any way I can "...
siliconpi's user avatar
  • 2,767
60 votes
5 answers
27k views

How to automate regular Google Takeout backups to cloud storage

I would like to create regular Google Takeout backups (let's say every 3 months) and store them encrypted in some other cloud storage like DropBox or S3. It does not have to be a cloud-to-cloud ...
Michał Šrajer's user avatar
45 votes
8 answers
49k views

Time machine backup on S3

Is there any possibility to use Amazon S3 for Mac OS X time machine backups? Or in other words: Use S3 rather an external hard disk for time machine backups?
Marc's user avatar
  • 1,327
33 votes
5 answers
56k views

Amazon S3 Recover Deleted File

Is it possible to recover a file I deleted from my S3 bucket?
vonhogen's user avatar
  • 2,379
23 votes
7 answers
19k views

Replacing Dropbox with: Amazon S3 + SSL + GPG/TrueCrypt + Mounting on OSX ?? [closed]

So, right now we're using Dropbox to share various data files around between approximately 10 Mac OS X systems. However, we already have an S3 account and everyone on the lowest Dropbox plan of $10/...
Matt Rogish's user avatar
22 votes
5 answers
32k views

Can I use rsync for backing up at Amazon S3

I have bought some space from evbackup which provides me FTP, rsync and SSH. But I have also read that Amazon S3 can also be used to backups. So I am really confused that do Amazon give FTP access ...
Mirage's user avatar
  • 2,903
21 votes
5 answers
32k views

A decent S3 bucket manager for Ubuntu

I'm looking for a decent S3 bucket manager for Ubuntu (Gnome). I prefer it to integrate with Nautilus so it will look like just any other drive (a la WebDAV) but so far I haven't been able to find ...
Luke's user avatar
  • 2,649
19 votes
1 answer
38k views

Amazon S3 GUI client for OS X [closed]

I'm looking for a good GUI client for Amazon S3. On Windows I used Cloudberry Explorer and I would like to know if there's anything like that for Mac OS X.
Cyclonecode's user avatar
16 votes
2 answers
19k views

AWS upload folder to S3 as tar.gz without compressing locally

In AWS CLI, how do I upload a folder as a tar.gz file without creating a tar.gz locally? For example, I have a folder at /var/test and I want to upload it to /tests/test1.tar.gz How do I do that ...
Michael Samsung's user avatar
10 votes
6 answers
12k views

Alternative to Amazon's S3 service? [closed]

Just wondering if there is good alternative to Amazon's S3 service? I like S3 but the bandwidth cost is high. I looked at CouldFiles from Rackspace but the cost is even higher. I don't mind ...
Cory's user avatar
  • 169
10 votes
3 answers
11k views

How to upload big files to S3 on a flaky connection?

I have a bunch of files (between 500MB and 7GB) that I need uploaded to an S3 bucket. My connection is very flaky. I tried uploading a 500MB file via s3cmd but it timed out after it was 91% done (...
user2254's user avatar
  • 663
9 votes
2 answers
12k views

Auto-set permissions when uploading to Amazon S3 with Transmit 4 [closed]

Does anyone know how to auto-set the read permissions for "World" upon upload of a file to Amazon S3 with Transmit? It is currently a two step process in my workflow: upload the file change ...
ToddSmithSalter's user avatar
9 votes
5 answers
3k views

Let's Encrypt with Amazon S3

Will it be possible to have https on the my static Amazon S3 website with Let's Encrypt? Even I have Let's Encrypt Closed Beta Invite now. But how to use it with Amazon S3?
Dmitry Ponomarev's user avatar
7 votes
1 answer
6k views

Know the identity behind an AWS canonical ID

Today while auditing a friends security settings, I found that an Amazon Folder (s3 bucket) with his most personal files had an unrecognised ACL entry. A person (with id ending in f6995f) had "Write" ...
american-ninja-warrior's user avatar
7 votes
0 answers
17k views

Can't connect to S3 using cyberduck

I'm trying to connect to my S3 bucket using cyberduck. I select S3 from the dropdown. Input into server name: {bucket-name}.s3.amazonaws.com Access key ID and Secret Access Key into the next two ...
goose's user avatar
  • 171
6 votes
1 answer
21k views

GPG program not found on OS X

Trying to run s3cmd on my mac book air, receiving the following message. Success. Your access key and secret key worked fine :-) Now verifying that encryption works... ERROR: Test failed: GPG ...
Sorter's user avatar
  • 421
6 votes
1 answer
12k views

Why can't I connect to my Amazon S3 storage with two of my three clients?

I use Transmit, Cyberduck, and Espresso, all of which support connecting to Amazon S3. But for some reason, I can only connect to my S3 storage successfully in Transmit; Cyberduck and Espresso both ...
daGUY's user avatar
  • 135
6 votes
0 answers
2k views

AWS Client S3 mv command is slow

I am using aws cli to move the files around in S3 in the same bucket. As far as I know, AWS S3 does not maintain the folder structure. It stores the entire path as key to the file and displays it in ...
nightgaunt's user avatar
6 votes
3 answers
4k views

What free method would you recommend for mounting an Amazon S3 bucket on OS X 10.6?

It seems s3fs is broken on Snow Leopard ( http://mark.aufflick.com/blog/2007/10/28/leopard-amazon-s3-network-storage ) Does anyone know of a free (and preferably open source) way to mount a bucket so ...
GJ.'s user avatar
  • 9,843
5 votes
2 answers
9k views

How to search for nested files in my S3 bucket?

I'm on the S3 Management Console and I see the instructions to type a prefix to search: However, when I search, it only finds files in the current folder. Is there a way to search for files that ...
nachocab's user avatar
  • 157
5 votes
1 answer
4k views

How to mount Amazon S3 bucket to a directory on FreeBSD?

I can't find a solution on how to mount an Amazon S3 bucket to an existing directory on FreeBSD. I suspect, I need to install port fusefs-s3fs. But how should I configure it? I can't find any info.
heximal's user avatar
  • 175
4 votes
4 answers
2k views

Can I rely on S3 to keep my data secure?

I want to back up sensitive personal data to S3 via an rsync-style interface. I'm currently using s3cmd - a great tool - but it doesn't yet support encrypted syncs. This means that while my data is ...
Jamie Hale's user avatar
4 votes
1 answer
4k views

How can I download a copy of an S3 public data set?

i was naively assuming I could do something like s3cmd sync s3://snap-d203feb5 /var/tmp/copy but I seem to have the wrong idea of how to go about this. I cannot even get a simple thing to work; ...
tripleee's user avatar
  • 3,181
4 votes
3 answers
3k views

Transfer file to Amazon S3 from web server

How do I transfer a tarball from my web server (e.g. http://example.com/backup.tar.gz) to an Amazon S3 bucket? Do I have to download it locally and upload it?
ethicka's user avatar
  • 43
4 votes
1 answer
5k views

multiple s3 users in one bucket or per-user buckers

intelligent people, noob here! I am planning on building a multi-user apparatus on s3 for photo/object storage, and I was planning on using s3. I have the whole front-end planned out but I have a ...
BurnDownTheIgloo's user avatar
4 votes
1 answer
563 views

A company wide dropbox like system on top of Amazon S3

Is there a system that will provide dropbox like features on top of Amazon s3? It would be used company wide, mostly for documents, recordings, etc... Some users work on Microsoft Windows, others on ...
TimothyP's user avatar
  • 1,003
4 votes
5 answers
863 views

Cloud based backup solutions based on open standards?

I am looking for a solution to backup and consolidate important media from a couple Windows laptops and Mac laptop. I would like a solutions that based on open standards, so my data isn't trapped by ...
Rick's user avatar
  • 141
4 votes
1 answer
3k views

S3 based dropbox like program for mac os and windows [closed]

I use dropbox to sync file between several computers. It does the job quite well but it have several disadvantages: Storing files on S3 is much cheaper (I need about 10Gb for all my shared files) and ...
Konstantin Solomatov's user avatar
4 votes
1 answer
432 views

Automatically Transcode Video and Upload to Cloud

I'm looking for a solution to take video files from my HD Camera and automate transcoding to a smaller format and then uploading to the cloud (AWS or otherwise). The ideal workflow would be, I have a ...
Rapture's user avatar
  • 285
3 votes
5 answers
2k views

Setting up scripts in Amazon EC2 Cloud

I am currently running a few perl and python scripts on a windows pc and would like to port over to the Amazon EC2 servers running 64-bit LINUX. The scripts are basic web scrapers that go to a ...
racket99's user avatar
  • 131
3 votes
2 answers
7k views

Using S3 instead of FTP- any reason NOT to?

Whenever a client wants an FTP storage solution, my instinct it to direct them to S3 instead, as a secure and cost-effective alternative. Is there any good reason NOT to use S3 instead of FTP/SFTP in ...
Yarin's user avatar
  • 406
3 votes
2 answers
560 views

Amazon S3 - To Backup or Not to Backup

So heres a question: Should I back up my data if I store using S3 (not RRS) or does S3 already handle a good enough multiple site data replication redundancy? If backing up is desired, I'd like to ...
dlikhten's user avatar
  • 133
3 votes
1 answer
2k views

Encryption strategy for Amazon S3?

I'm running Windows and strongly considering using Amazon S3 to back up some of my more important data, but i'm having trouble coming up with a good encryption strategy. I'm a fan of Truecypt and use ...
Don's user avatar
  • 31
3 votes
1 answer
1k views

Nginx SSL Preread Sporadically Gets Requests Where No Server Name Is Extracted

I am using the nginx stream module to leverage nginx as a tcp reverse proxy in front of s3. Recently, I needed to add logic to accommodate an additional upstream. To do this, I chose to make use of ...
Howard_Roark's user avatar
3 votes
1 answer
2k views

AWS S3 cp does not download 40 GB file

I just uploaded a 40GB backup file from my linux EC2 machine to an S3 bucket. However when I want to download it the aws command ends with no message and nothing was copied. Here is my command: ...
Jörg Rech's user avatar
3 votes
1 answer
1k views

Are whole VM images backed up on Amazon EC2/S3?

I've been trying to get my head around Amazon Web Services as a VPS provider. My understanding is a EC2 instance running Windows is basically a Windows VM, very similar to renting a VPS from a more ...
Mr. Boy's user avatar
  • 5,236
3 votes
1 answer
2k views

https proxy s3 aws via nginx with pagespeed

I'm trying to setup pagespeed to proxy s3 files via nginx. I want to "save" the files on amazon s3; The story I want is: User A 1. nginx serves page to user; and rewrites all https://s3.amazonaws....
Daniel's user avatar
  • 163
3 votes
1 answer
2k views

Use personal Amazon S3 account to backup Gmail using Ubuntu

As question, I have found online that backupify offer shifting from their S3 to user's personal S3, I have a backupify account but I can't find this options, besides, I don't prefer having my email ...
lokheart's user avatar
  • 523
3 votes
0 answers
704 views

s3fs gives me an empty directory

I built s3fs, and now I'm getting an empty directory where I mounted the s3 bucket. I put a text file in the directory, and I can see it in there, and it transfers into the bucket. How can I see the ...
psion's user avatar
  • 211
3 votes
1 answer
2k views

Subversion repository on Amazon S3

Is it possible to put a subversion repository on Amazon S3. I found SvnToS3 on google code but that project seems dead.
Ahmad's user avatar
  • 41
2 votes
1 answer
1k views

Is having Linux VPS same as Amazon EC2 or S3 [closed]

I am new to Amazon services so have no idea, whenever i ask some questions on forums , few recommend Amazon instead of VPS. So i want to know is having Amazon EC2 or S3 is exact replacement of VPS. i ...
Mirage's user avatar
  • 2,903
2 votes
2 answers
782 views

Why CNAME is not working as supposed?

I have set a CNAME entry to redirect over Amazon S3 bucket, however there is something that doesnt works as supposed (by me of course) with CNAME I expect to redirect a fictional subdomain, say for ...
Ritardi.Net's user avatar
2 votes
4 answers
2k views

What desktop app can sync Amazon S3 with an FTP folder?

What I'm trying to do is migrate large files off my web site to S3.
Andrew Warner's user avatar
2 votes
2 answers
16k views

curl can not download file but browser can

URL example: https://xxxxxxxxxx.s3.amazonaws.com/xxxx-xxxx-xxxx-xxxx/xxxxxxxxxxxxx/x?AWSAccessKeyId=AKIAIFKEBR4CLVTDYSLA&Signature=Kv%2B859dJ2eUmLgHdkT%2F9nUJEeRM%3D&Expires=1922517562 I ...
Vitaly Zdanevich's user avatar
2 votes
1 answer
6k views

Determining the service that is using *.CloudFront.net

How can I determine that accessing to cloudfront.net is associated to which service, software or owner? (as far as I understand and the normal who-is lookup tells, it is related to Amazons' cloud ...
F.I.V's user avatar
  • 182
2 votes
1 answer
1k views

s3cmd tool falls over on local directory

I'm trying to use s3cmd to copy a full bucket down to my local machine like so: s3cmd -c .s3cfg-bespoke sync s3://my_bucket/ ~/local_folder/ There are three folders and a smattering of files in the ...
ghickman's user avatar
  • 385
2 votes
1 answer
864 views

Zip Multiple Text Files from Multiple Directories into One ZIP File in One Command in Bash

I have an account on AWS and I use S3 for storing data. I have commands that will enable me to zip data files like so: data.zip s3_dir/level-1/level-2/*/*/data.txt But there are, say, 50 or more ...
nicorellius's user avatar
  • 6,715
2 votes
2 answers
6k views

copy specific files from s3 bucket

I have some files in s3 bucket. aws s3 ls s3://bucketname/ file-111-100x100.jpg file-112-1400x1400.jpg file-123-250x250.jpg file-231-1400x1400.jpg file-222-700x700.jpg file-333-100x100.jpg file-131-...
Jaimin's user avatar
  • 63
2 votes
1 answer
292 views

Backup software for Ubuntu - which one? [closed]

I have spent some time testing out different backup solutions for my small home office during the last weeks, but still haven't found anything that have been working out too well yet. We can ...
Industrial's user avatar
2 votes
3 answers
1k views

Is there the equivalent of cloud computing for modems?

I asked this question on SF, and someone recommended that I ask it on SO, and on SO, it was voted to be closed, and someone suggest that I post it here - (and so, the merry go round continues) - so ...
morpheous's user avatar
  • 4,333