Amazon S3 (Simple Storage Service) is a web service which provides storage and various interfaces for accessing data stored there. Questions about Amazon S3 should be asked on webapps.stackexchange.com.

learn more… | top users | synonyms (1)

1
vote
2answers
2k views

can I create a symlink from an ec2 disk to S3 bucket?

Is it possible to create a symlink from a file on the local disk of a Windows machine in ec2 so that it is actually stored in an S3 bucket? MSMQ wants to write our queue to a local disk and this ...
1
vote
2answers
1k views

Using S3 instead of FTP- any reason NOT to?

Whenever a client wants an FTP storage solution, my instinct it to direct them to S3 instead, as a secure and cost-effective alternative. Is there any good reason NOT to use S3 instead of FTP/SFTP in ...
1
vote
1answer
2k views

Tortoise SVN on Amazon S3

I would like to setup a Tortoise SVN repository on my Amazon S3 server. I know how to do the setup on my local machine, and I know how to do it on a server (virtual or nonvirtual) running windows. I ...
1
vote
1answer
50 views

How to copy a .zip from within a .tar to a given location without extracting entire .tar using AWS S3 CLI

OSX 10.11 - python3.5 or AWS CLI (or other tool?) I have ~ 5,000 subdirectories within an Amazon S3 bucket, each subdirectory contains a single .tar. In each .tar it contains only one .zip, ~<1mb ...
1
vote
3answers
75 views

Renaming files fetched via “wget --mirror” in Bash before uploading to an Amazon S3 statically hosted area

I’m trying to archive and upload an old website to statically hosted Amazon S3 area. I was able to get the contents with wget, using the following command: wget --mirror --no-parent --html-extension ...
1
vote
1answer
55 views

How to get the last time a particular file was downloaded from S3

I was wondering if there is a way to get the last downloaded time on of an S3 object. I had a use case where I need this data to decide whether to remove a file if its not downloaded for a set period. ...
1
vote
1answer
360 views

Allow a group to upload files on an Amazon S3 bucket

After a group trip abroad, I would like to gather the photos & videos that everybody took. I have an Amazon S3 account, how can I use it to allow people to send files to it ? Most people in the ...
1
vote
1answer
581 views

Can't get instances of a video (mp4) to play in multiple tabs in Chrome

I have a mp4 that I would like to play in Chrome. The mp4 is stored on s3 and has a url like http://bucket.s3.amazonaws.com/vid.mp4 Now when I put that url into Chrome 27.0.1453.94 the video plays. ...
1
vote
1answer
617 views

Is it possible to transfer files from Amazon S3 directly to another web server? [closed]

I need to transfer some files from several Amazon S3 buckets to a different web server. The amount of data is several gigs, so downloading and re-uploading is not favorable. I have done some ...
1
vote
1answer
1k views

Using a script that uses Duplicity + S3 excluding large files

I'm trying to write an backup script that will exclude files over a certain size. If i run the script duplicity gives an error. However if I copy and paste the same command generated by the script ...
1
vote
2answers
386 views

All documents from S3 backup corrupted

Before I left for my trip I made a backup of all my important data to an Amazon S3 bucket. I also have a Citrix remote desktop to my disposal. Today I installed the citrix client and logged in to my ...
1
vote
1answer
624 views

Limited access to Amazon S3 buckets

Is it possible to somehow limit the access to an Amazon S3 account. I don't really like the idea of distributing my secret access key to all of my applications, that want to access just a single ...
1
vote
0answers
28 views

does s3cmd put command overwrite files?

I am trying to write a script that will automatically backup a drive I have to amazon s3. I have downloaded s3cmd and I am wondering what the put command does... It seems to me like it uploads the ...
1
vote
0answers
99 views

How to transcode AES encrypted files with AWS Elastic Transcoder

I want to upload my encrypted videos to S3 and use Elastic Transcoder to transcode them. This is what I'm currently doing and isn't working at all. I'm using openssl to encrypt my video files. The ...
1
vote
0answers
51 views

AWS server retired; new instance lost access to S3

My old company has a system with a node.js front end running on an AWS EC2 instance. The instance was 'retired' yesterday morning, and I got a frantic call from the CEO because they were down. I got ...
1
vote
1answer
427 views

How to download a large S3 object from the command line much faster than s3cmd

With the s3cmd command you can download files from S3 like this: s3cmd get s3://my.private.bucket/some/location/hugefile.dat But that can take quite a while if the stored files are really big. Are ...
1
vote
1answer
367 views

File permissions with s3fs and ubuntu

I followed this guide to set up an amazon s3 upload procedure for my ember-js application with grunt-s3: http://www.octolabs.com/blogs/octoblog/2014/05/24/deploying-ember-cli-to-amazon-s3-with-grunt/ ...
1
vote
0answers
2k views

Can't connect to S3 using cyberduck

I'm trying to connect to my S3 bucket using cyberduck. I select S3 from the dropdown. Input into server name: {bucket-name}.s3.amazonaws.com Access key ID and Secret Access Key into the next two ...
1
vote
0answers
1k views

How to use FFMPEG to transcode mp4 video for http stream in Amazon S3?

I want to use FFMPEG to transcode my mp4 video for http streaming on Amazon S3 without downloading. However, with reference to this post, I still cannot get it streaming: ffmpeg -i input.avi -c:v ...
1
vote
2answers
275 views

How do I backup all the photos I have in Picasa 3 to Amazon S3?

I have thousands of photos that I would like to push to Amazon S3. Picasa has them all indexed, but they are spread all over my computer and a USB drive. How do I backup all the photos I have in ...
1
vote
3answers
594 views

Initiate restore of directory hierarchy in Glacier to S3?

Using software that runs under Linux, is there a way to initiate a restore from Glacier to S3 of an entire directory hierarchy? The files are transferred to Glacier using a S3 bucket lifecycle rule. ...
1
vote
1answer
2k views

Would it be okay to use the Amazon S3 for personal use? [closed]

I understand that AWS is meant for corporate use, but I just thought about using the S3 for personal backups, and by using their calculator it seems like it would be quite cheap in comparison to other ...
1
vote
0answers
79 views

Is there a way to automate XML playlist creation for files in an S3 bucket?

I'm working on an XML-based, non-Flash music player for a client. They want to be able to upload files to a bucket and stream MP3s back to their website. I'm wondering if there's a way to ...
1
vote
0answers
166 views

s3fs gives me an empty directory

I built s3fs, and now I'm getting an empty directory where I mounted the s3 bucket. I put a text file in the directory, and I can see it in there, and it transfers into the bucket. How can I see the ...
1
vote
1answer
303 views

Can I convert a Jungle Disk 2.0 bucket to a “compatible” bucket?

I have some data in a Jungle Disk 2.0 bucket on Amazon S3. It seems like the data in this bucket cannot be accessed via Amazon's own web interface to S3 - at least not very conveniently (the data is ...
1
vote
1answer
598 views

Adding a disk to an Amazon EC2 instance

I am using Amazon EC2 instance where I have low disk space [root@ip-10-59-143-73 ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/xvde1 5.7G 3.3G 2.3G 60% / none ...
0
votes
3answers
1k views

Upload very many files to S3

I have about 1 million images (organized in directories) that I need to get into S3. I started using s3sync.rb, but since it is built for syncing it creates tons of extra files for keeping track of ...
0
votes
1answer
292 views

how to access an s3 bucket given user name and keys?

a friend gave me information on his s3 bucket, so I can look at his logs: user name access key secret key A lot of the material I have is to access your own bucket, but how do people share a ...
0
votes
1answer
41 views

How to know what are in a file and what aren't in another?

I doing a nrpe check for my nagios about the files that I backup, sending to Amazon S3. I have two files then, s3, which list all my files that are in Amazon already. And local which list all my files ...
0
votes
2answers
2k views

How sync files with s3cmd to Amazon S3, check if sent and remove locally?

I'm trying to use Amazon S3 service to store logs from my applications. Given /user/bin/s3cmd --help tells me what I need to know how send the files: s3cmd --help usage: s3cmd [options] COMMAND ...
0
votes
1answer
1k views

I can't resolve s3 dns from time-to-time

I'm using the .net SDK to talk to S3, but too often I get the following exception: The remote name could not be resolved: 's3.amazonaws.com' I'm using the two standard OpenDNS DNS servers, and ...
0
votes
1answer
242 views

s3 file access denied for large files using cyberduck

When I use a custom policy below, I get the following error when uploading large files (1G and over) using cyberduck file access denied. please contact your web hosting service provider for ...
0
votes
1answer
94 views

How to safely upload many GB of data to Amazon S3?

I am backing up my personal photo/video library to Amazon S3 (several hundred GB). I am currently using Cyberduck on my Mac to drag & drop, but I've already had a few fail without a clear ...
0
votes
1answer
469 views

S3Fox Exception: Too Many Arguments

I'm running the S3Fox add-on in FireFox 38.0.1 Getting this error when I try to download a file from Amazon S3 to my local PC: [Exception... "Not enough arguments [nsIWebBrowserPersist.saveURI]" ...
0
votes
1answer
563 views

Sync local files to S3 similar to Robocopy

I am looking for a way to synchronize an entire local folder structure to Amazon S3, similar to how one might synchronize two folders using Robocopy. Whatever solution I come up with needs to be ...
0
votes
1answer
49 views

Find out who delete the file

Hi we are using the S3 on AWS. and there are several people who have access to that box, however, everyone has to log in as the same user/ubuntu. I was doing some work, zipping a big file using ...
0
votes
1answer
1k views

s3cmd: how to copy files from local to S3 bucket with existing structure

I'm trying to copy files with s3cmd to a bucket with an existing folder structure. Both structures already exist both on local dev machine and in the bucket. The folders structure is as follows ...
0
votes
1answer
2k views

Environment Variables and Shell Rc File [duplicate]

Possible Duplicate: What are PATH and other environment variables, and how can I set or use them? I am trying to automatically establish a connection with Amazon s3 using keys. This ...
0
votes
1answer
1k views

Mount a S3 bucket on Linux without 3rd party tool?

Is there any way to mount a S3 bucket on the Linux filesystem without any 3rd party tools? Assuming I have a standard Ubuntu installation.
0
votes
1answer
856 views

Ubuntu: Backing up network files with duplicity?

I just installed Duplicity on Ubuntu and got everything working. However, when I try to backup a network samba-connected windows drive, The following occurs: Backup source directory ...
0
votes
0answers
11 views

keep amazon s3 bucket public between uploads

I've setup an Amazon CloudFront CDN to point to an Amazon S3 Bucket. It works great, I love how it's streamlined much of my process! But I have a problem, every time I do a new upload to it, the new ...
0
votes
0answers
26 views

aws linuxAMI crontab ec2 to s3 backup command not working

I am new to Aws Platform and was trying to get Hands on experience on Command Line... but when i was configuring installed wordpress site...and syncing it to S3 bucket.. and for that my instructor was ...
0
votes
0answers
10 views

How do I change all objects within a S3 bucket to reduced redundancy?

I'd like to know if there is a bucket wide (not per object) way of doing this. Bash, s3cmd, python any possible way would work for me. Thank you!
0
votes
0answers
39 views

How can I restore a MySql database dump direct from S3?

I have a gzipped database backup stored on S3 and would like to restore it to MySql without having to first download it due to disk space constraints. I tried the two commands below but got gzip: ...
0
votes
0answers
51 views

Would be great to send a large file to AWS EC2

Background: I have battled to install an rstudio server on AWS EC2 for some time (since November). It has been like the peeling back layers of an onion, always another hurdle. For a windows user ...
0
votes
1answer
55 views

Geo-balanced (static) website hosting

Abstract I'm rebuilding my website and this time around, I have decided to go with a static website generator (jekyll, sculpin.io, etc). Partially as an excercise, I want this site to be hosted on a ...
0
votes
0answers
58 views

Amazon S3 upload error: An exception occurred while uploading parts to a multipart upload

I am trying to upload a 30 GB file to Amazon S3 using the AWS PHP SDK. require('../vendor/autoload.php'); $client = new Aws\S3\S3Client([ 'version' => 'latest', 'region' => 'us-east-1' ]); ...
0
votes
0answers
158 views

Tar a very large file directly to a drive

I was running my code to download some data on the Amazon cloud. The instance that I was running had an 8GB storage that I had bought with it plus 140GB extra instance storage (sort of plugged in as a ...
0
votes
0answers
692 views

Amazon S3 usage reports per bucket

Are there any good AWS Usage Report parsers that will separate S3 usage per bucket, in a more human-readable form? Right now, you can export the usage exports into an XML or CSV file, but these are ...
0
votes
0answers
258 views

Parallel upload of directory structure with small files to Amazon S3

How can I efficiently upload a big directory structure with many files to Amazon S3 from OSX? I would like to make parallel uploads.