62
votes
Amazon S3 Recover Deleted File
If you have versioning enable, yes you can!
On a versioned bucket, a delete action on a file does not really delete it but it adds a version with a "Delete Marker". You can delete the delete marker ...
- 721
23
votes
AWS upload folder to S3 as tar.gz without compressing locally
What you're really looking for is not saving a local file. You can use pipes to send the data from tar through gzip to s3 without saving anything to disk.
tar c /var/test | gzip | aws s3 cp - "s3://...
- 60.6k
18
votes
Time machine backup on S3
Using AWS S3 as the storage of Mac Time Machine definitely works now.
Time Machine can backup the data to external device, so the key is mounting S3 bucket as a POSIX FS in Mac.
There are many tools ...
- 495
8
votes
How to automate regular Google Takeout backups to cloud storage
This is a partial answer with partial automation. It may stop working in the future if Google chooses to crack down on automated access to Google Takeout. Features currently supported in this answer:...
8
votes
AWS upload folder to S3 as tar.gz without compressing locally
tar cvfz - /var/test | aws s3 cp - s3://tests/test1.tar.gz
You don't have to separately gzip; tar does that for you with the z option.
This works both in directions.
- 81
7
votes
Let's Encrypt with Amazon S3
The answer to the question comes in two parts.
First, though, using "Let's Encrypt" is not actually relevant to the question: it's a Certificate Authority (CA). It issues SSL (TLS) certificates, ...
- 1,222
7
votes
Know the identity behind an AWS canonical ID
The AWS documentation at https://docs.aws.amazon.com/AmazonS3/latest/dev/example-walkthroughs-managing-access-example4.html says:
The canonical user ID is the Amazon S3–only concept. It is a
64-...
- 458
6
votes
Let's Encrypt with Amazon S3
You can store files of your website on S3, but to publish it you need to use CloudFront. And there you'll be able to select custom certificate which you have to upload at AWS IAM certificate store.
...
- 101
6
votes
How to automate regular Google Takeout backups to cloud storage
Google Takeout lets you schedule exports every two months, so six a year
and up to one year.
You can choose to have the file added to a cloud drive or have
a download link emailed (downloads are kept ...
- 445k
6
votes
Can I use rsync for backing up at Amazon S3
Amazon's official command line interface worked well for me. It can sync from local to an S3 bucket, and also sync from an S3 bucket back to local files.
1) install Amazon's command line interface, ...
- 179
4
votes
Time machine backup on S3
Edit: I tried this and it didn't work. (Time Machine cannot see the mounted volume/bucket.)
You may be able to use Panic's Transmit app to [mount an S3 bucket as
a local Volume][1] and then point ...
- 147
4
votes
Uploading files to S3 account from Linux command line
Alternatively you can try https://github.com/minio/mc
mc provides minimal tools to work with Amazon S3 compatible cloud storage and filesystems. It has features like resumable uploads, progress bar, ...
- 141
4
votes
Accepted
curl can not download file but browser can
Use single quotes to make sure that any potentially special characters are taken literally.
[user@localhost ~]# curl 'https://xxxxxxxxxx.s3.amazonaws.com/xxxx-xxxx-xxxx-xxxx/xxxxxxxxxxxxx/x?...
- 1,178
3
votes
multiple s3 users in one bucket or per-user buckers
You definitely do not need a bucket for each user. Never mind the fact that it seems very unlikely that AWS support would approve a request to increase your account's default total bucket limit from ...
- 1,222
3
votes
Accepted
s3 file access denied for large files using cyberduck
I kept digging around and found that Amazon recommends to use multipart uploads for all files over 100M (http://docs.aws.amazon.com/AmazonS3/latest/dev/uploadobjusingmpu.html), which I guess what ...
- 41
3
votes
Accepted
Determining the service that is using *.CloudFront.net
The IP addresses associated with *.cloudfront.net hostnames do indeed have the format you described:
iii-ppp-aaa-ddd.locNN.r.cloudfront.net
For example:
server-13-32-14-48.muc51.r.cloudfront.net
...
- 1,222
3
votes
How to search for nested files in my S3 bucket?
From command line you can use command like:
aws s3 ls s3://bucket/ --recursive "filename"
- 4,884
3
votes
How to search for nested files in my S3 bucket?
There isn't a find equivalent that I know of. I do this:
aws s3 ls s3://bucket/ --recursive | grep filename
- 31
2
votes
How to upload big files to S3 on a flaky connection?
You can use FileZilla Pro to transfer files to and from a S3 bucket. FileZilla Pro supports multipart upload and in case of failure it will resume the transfer.
For the records it comes with a lot of ...
- 156
2
votes
Can I use rsync for backing up at Amazon S3
I'm surprised nobody has mentioned rclone. It is like rsync but it connects to just about any cloud storage provider out there, and also supports full encryption. I use it regularly, it is well ...
- 121
2
votes
Accepted
Renaming files fetched via “wget --mirror” in Bash before uploading to an Amazon S3 statically hosted area
This should work:
find . -maxdepth 1 -type f -name '*\?*' |\
while read FILENAME
do
IFS='?'
SPLIT_FILENAME=(${FILENAME})
unset IFS
echo mv "${FILENAME}" "${SPLIT_FILENAME}"
# ...
- 52.5k
2
votes
Accepted
Can I make an Amazon AWS Lamda function which is triggered by the creation of an S3 bucket?
EDIT
As @michael-sqlbot has pointed out in the comments, what you are trying to achieve may be possible but with a multi-step process. CloudTrail can log S3 bucket-level events (https://docs.aws....
- 136
2
votes
copy specific files from s3 bucket
I'm not sure if awscli has a built-in way to do this, but I've always just used simple bash to do things like this. For example:
for i in $(aws s3 ls s3://bucketname/ | grep 1400x1400); do aws s3 cp ...
- 141
2
votes
Get all files (or file names) out of s3 bucket for specific date
If the filename contains the date, you can use include and exclude filters:
aws s3 cp s3:{path}/ {directoryToCopyTo} --exclude "*" --include "*2019-09-09*"
If the date is only in ...
- 309
2
votes
Is there a way we can way to minimize or eliminate warning when a recipient receives an email from AmazonSES
With quick googling it appears setting up SPF and DKIM should resolve it.
When you first set up Amazon SES to send emails, you may find there is a "Via amazonses.com" warning for the email ...
- 4,423
2
votes
make new objects in s3 bucket public by default
For details see the post
How to Make All Objects in Amazon S3 Bucket Public by Default.
To resume:
Open the Properties for the bucket you want to make public
Click "Permissions"
Click &...
- 445k
1
vote
How to encypt files in transit to S3 from local macine
"By default, the AWS CLI uses SSL when communicating with AWS services. For each SSL connection, the AWS CLI will verify SSL certificates.."
Source - AWS CLI documentation.
Note that this SSL ...
- 594
1
vote
Accepted
How to copy a .zip from within a .tar to a given location without extracting entire .tar using AWS S3 CLI
to pull out a single file called zipfile.zip from archive tarfile.tar:
tar xvf /path/to/tarfile.tar /path/to/where/you/want/zipfile.zip
You could use perl to recurse
#!/usr/bin/perl
my @...
- 347
1
vote
How to safely upload many GB of data to Amazon S3?
According to your problem you can try Bucket Explorer
Features of Bucket Explorer
You can browse buckets and the files stored at Amazon S3.
Upload and download files, to and from Amazon S3 buckets.
...
- 147
1
vote
Can I host a static website on Amazon EC2 coupled with EFS?
Important note: As of 2018-07-12, EFS allows you to purchase provisioned throughput. The answer below reflects the behavior of the service before this feature was available. Previously, small EFS ...
- 1,222
Only top scored, non community-wiki answers of a minimum length are eligible