Questions tagged [amazon-s3]
Amazon S3 (Simple Storage Service) is a web service which provides storage and various interfaces for accessing data stored there.
0
votes
0answers
18 views
Bash completion not working for 'aws cli' after installed via brew
After I use brew to install aws cli, here is the message form brew:
==> awscli
The "examples" directory has been installed to:
/usr/local/share/awscli/examples
Bash completion has been ...
0
votes
0answers
13 views
In the elastic beanstalk, I get the message 'Service: Amazon S3, Message: Forbidden'
My eb is currently on the docker platform and will read the Dockerrun.aws.json file on s3.
There is no problem when deploying from the web console, but deploying in the cli environment with the ...
0
votes
0answers
7 views
S3FS ownership and permissions
I am trying to mount a S3 bucket to my local machine using secret password key with the following command:
s3fs mybucket /mnt/mybucket -o allow_other -o umask=0000,uid=$UID -o use_cache=/tmp -o ...
0
votes
0answers
12 views
S3 file is not equal upon counting in AWS athena
I have a file stored in S3 with a count of 33,09,1073. Upon counting it via query in AWS Athena the count is already 33,196,272 and we found out the excess rows are all blanks. We're trying to make it ...
0
votes
0answers
29 views
Is it possible to remove the download link option for Amazon S3 audio files? [on hold]
I'm sharing files using Amazon S3 and I only want them to be playable, not downloadable from Amazon. Is there any way to do that?
Amazon s3 audio download option:
0
votes
0answers
12 views
Writing to frankfurt s3 bucket - Bad request Error
I am trying to write to frankfurt s3 bucket using spark submit but getting 400 errors. com.amazonaws.services.s3.model.AmazonS3Exception: Status Code: 400, AWS Service: Amazon S3. I have following ...
0
votes
0answers
4 views
Authorization Error Using AWS S3 Buckets [migrated]
node s3_listbuckets.js
Error TypeError [ERR_INVALID_CHAR]: Invalid character in header content ["Authorization"]
at ClientRequest.setHeader (_http_outgoing.js:470:3)
at new ClientRequest (...
0
votes
0answers
22 views
How can I capture URL's that result in a real but friendly 404 on my AWS S3 hosted website?
I configured my AWS S3 bucket as a static website and specified an HTML page as the error document. URLs not found on my site are redirected to the error document instead of generating a real 404 ...
0
votes
0answers
40 views
Presto hive connector S3 access denied
I have some hive external tables on my S3 buckets. To access the hive external tables from presto, I made presto hive catalogs for each S3 buckets as they have different aws_access_key and ...
0
votes
0answers
63 views
AWS s3 cross account policy allowing public access
I'm having some difficulty locking down an S3 bucket that I must share across accounts.
I have no public ACL defined. "Everyone" has no options enabled.
The console bucket list says "Bucket and ...
0
votes
1answer
37 views
AWS ACCESS KEY ISSUE
I accidentally pushed my access keys up to github and amazon contacted me with instructions on "fixing" the issue. I messed up and accidentally deleted the access key, thinking thats what i was ...
0
votes
0answers
52 views
AMI Permissions not working when using AWS CLI VM import/export
I am having some problems with AWS permissions, I am trying to use the VM Import/export to export an instance to a local device. I followed the AWS notes at https://docs.aws.amazon.com/vm-import/...
1
vote
1answer
37 views
How do I set up DNS, AWS S3, AWS Cloudfront, and AWS Certificate Manager to secure multiple sites
I am trying to secure multiple static AWS S3 websites delivered via AWS Cloudfront CDN with one certificate.
This AWS article says I should be able to do this, but it's not working.
One is a web-app,...
0
votes
0answers
30 views
Want to use windows network shared folder directly from ec2 instance (local)
I want to transfer files from network folder to S3 buckets by using ec2. For that I want to map the network folder in ec2 directory. PS : I have an ec2 instance created along with its private key.
0
votes
0answers
58 views
RequestError with aws-sdk-go for S3 upload
When I try to upload mp3 files to S3 using the s3manager package of aws-sdk-go, I get this error.
2019/03/29 11:02:07 Error occurred while uploading
/var/recordings/98a8d64ab5d01c9e00ba4ca1e292ce61....
0
votes
0answers
13 views
Move files to specific location from one s3 bucket to another bucket
I have two s3 buckets in same account. One bucket having multiple images with name with
00000-file01.jpg
11111-file90.jpg
12321-file33.jpg
12345-file80.jpg
55555-file29.jpg
Second bucket has some ...
1
vote
0answers
57 views
Nginx SSL Preread Sporadically Gets Requests Where No Server Name Is Extracted
I am using the nginx stream module to leverage nginx as a tcp reverse proxy in front of s3. Recently, I needed to add logic to accommodate an additional upstream. To do this, I chose to make use of ...
0
votes
0answers
54 views
Automatic one-way syncing of a Windows file share to an S3 bucket
Our clients have Windows file shares where they store lots of different documents.
We want to ingest files matching wildcard patterns (e.g. *.pdf, *.xlsx, etc.) from those file shares into S3 buckets ...
0
votes
0answers
39 views
decrypt a file stored by server side encryption of minio
i'm trying to use server side encryption of files by minio (https://www.minio.io/), that is similar to aws s3. Before start to use it, i would to know which inverse function i can use to be able to ...
0
votes
1answer
68 views
Know the identity behind an AWS canonical ID
Today while auditing a friends security settings, I found that an Amazon Folder (s3 bucket) with his most personal files had an unrecognised ACL entry.
A person (with id ending in f6995f) had "Write" ...
0
votes
0answers
106 views
Can i upload files directly to AWS S3 using filepond plugin?
I am using filepond plugin to upload the files in my website. I need to send that files to directly to S3 bucket. Is it possible to do that?
1
vote
0answers
62 views
AWS S3 CLI: sum bytes in all top-level prefixes with a single --query?
I have an S3 bucket with a bunch of top-level prefixes. I have a relatively inefficient way of summing the bytes across top-level prefixes:
Get the list of prefixes via aws s3 ls [bucket name], ...
0
votes
0answers
126 views
AWS S3/Athena: How to receive JSON in query result?
Let's say, I have messages in amazon-s3 with following structure:
{ "msg_type": "foo", "msg_body": { "str_field": "hello", "int_field": 1 } }
When perform following query in Athena:
SELECT * FROM ...
0
votes
1answer
53 views
How to auto upload a file to s3
How can I upload a json file to my s3 in easiest way?
Basically there is a json file in a webpage and I want to keep copy it every 5 minutes and upload to my s3.
0
votes
0answers
36 views
405 error when trying post to elastic beanstalk from s3 static hosting axios call
I'm a super newbie at AWS and I'm trying to get my full stack application, react for front and node.js for the back, to be hosted, but so far I can only get the front to work.
I have an S3 bucket ...
0
votes
0answers
52 views
Why can I download but not stream MP3 content from AWS S3?
I'm trying to set up an RSS feed for podcast-like content. I've generated my RSS XML with links to mp3's hosted in https://s3.amazonaws.com/d4hines-bible-rss/WOP-12-2Kings-001.mp3 AWS S3. On my ...
0
votes
1answer
64 views
Linux - How to disable disk cache for a specific mount point?
Is it possible to disable disk cache for a mount point on Linux system?
I have a s3fs mount on multiple systems and i need to disable disk cache for it. how can i achieve this?
0
votes
0answers
44 views
goofys mount point on multiple linux systems
I have a Minio bucket which is mounted on three different linux systems. the problem is when a file changes on one of mount points but file size doesn't changes that change doesn't get reflected on ...
0
votes
1answer
405 views
How to fix awscli s3 error “The Content-MD5 you specified did not match what we received” (BadDigest)?
Uploading a file over 10 GB to s3 fails with:
aws s3 cp 'foo.log.gz' s3://mybucket/2018/12/ --storage-class REDUCED_REDUNDANCY
upload failed: ./foo.log.gz to s3://mybucket/2018/12/foo.log.gz
An ...
0
votes
0answers
66 views
Unable to view public s3 buckets in CloudBerry?
When I enter a public Amazon s3 bucket (not in my AWS account) in the CloudBerry Explorer GUI, CloudBerry returns "403 - bucket forbidden", even though the bucket is public and can be viewed through ...
0
votes
2answers
1k views
AWS S3 downloaded files have “last modified date” of the file in S3 after download
I'm using AWS CLI to download some files from S3 (command: 'aws s3 sync'). The problem I have is, that the downloaded file has the same "last modified" date as the file in S3. What I want is that the "...
1
vote
2answers
1k views
copy specific files from s3 bucket
I have some files in s3 bucket.
aws s3 ls s3://bucketname/
file-111-100x100.jpg
file-112-1400x1400.jpg
file-123-250x250.jpg
file-231-1400x1400.jpg
file-222-700x700.jpg
file-333-100x100.jpg
file-131-...
0
votes
2answers
93 views
Why CNAME is not working as supposed?
I have set a CNAME entry to redirect over Amazon S3 bucket, however there is something that doesnt works as supposed (by me of course) with CNAME I expect to redirect a fictional subdomain, say for ...
0
votes
2answers
3k views
curl can not download file but browser can
URL example: https://xxxxxxxxxx.s3.amazonaws.com/xxxx-xxxx-xxxx-xxxx/xxxxxxxxxxxxx/x?AWSAccessKeyId=AKIAIFKEBR4CLVTDYSLA&Signature=Kv%2B859dJ2eUmLgHdkT%2F9nUJEeRM%3D&Expires=1922517562
I ...
2
votes
1answer
156 views
Jungledisk does not restore my files. How can this be fixed?
I backup my files with Jungledisk for many years now, my backup is on Amazon S3 and is ~600 GB in size. Unluckily there came the time when I need to restore some files (my email files got damaged).
...
0
votes
0answers
220 views
Amazon CloudFront buffering while streaming video to Roku
I'm looking for help on trying to troubleshoot a buffering issue using Amazon's CloudFront service. I've set up an S3 bucket and have it associated with a CloudFront service. I use the service to ...
0
votes
1answer
880 views
Determining the service that is using *.CloudFront.net
How can I determine that accessing to cloudfront.net is associated to which service, software or owner?
(as far as I understand and the normal who-is lookup tells, it is related to Amazons' cloud ...
7
votes
1answer
4k views
AWS upload folder to S3 as tar.gz without compressing locally
In AWS CLI, how do I upload a folder as a tar.gz file without creating a tar.gz locally?
For example, I have a folder at /var/test and I want to upload it to /tests/test1.tar.gz
How do I do that ...
1
vote
0answers
87 views
Meteor(Blaze) : CFS: S3 Create thumbnail Image from Video and upload both to S3
I am using CFS packages to upload videos to Amazon S3, but it takes a long time to load the videos on Amazon when they are of 10-20 MB. Till then, I want to use the videos' first frame and when ...
0
votes
2answers
550 views
How to encypt files in transit to S3 from local macine
I have installed AWS CLI in a server in my on-prem server I am able to copy files via the cli S3 commands, how can i know if the files in transit to S3 are encrypted?. There is a SSL installed on the ...
0
votes
0answers
225 views
Large files split when uploaded to cloud (B2 Backblaze)
I'm uploading some ISO files which I made from DVDs to B2 Backblaze cloud. But I see they split there:
There are also some 0 bytes size files, which seems strange.
How can this be explained?
I'm ...
0
votes
1answer
53 views
AWS S3 bucket size reduced after configuring life cycle with Transisition to Standard IA class
Few days back I tried to configure a life cycle for S3 bucket intending to move old files to Glacier.
I configured a LifeCycle for that bucket and added transistion to Standard IA class after 31 days ...
-1
votes
1answer
281 views
Can I make an Amazon AWS Lamda function which is triggered by the creation of an S3 bucket? [closed]
I would like to send a mail when a new bucket is created on our AWS account. I'd prefer this to be near-instantaneous so preferably not time scheduled, but triggered by the CreatBucket event directly.
...
0
votes
1answer
57 views
Configure root login bucket for an s3 user?
I want to configure a policy/permission for a specific s3 user so they will login to a particular sub folder under a bucket.
I've got a bucket location here: s3://foo.s3.amazonaws.com and underneath ...
-3
votes
3answers
517 views
How to implement anti-scraping mechanisms for my Amazon S3 based site?
I have a few static web pages hosted on amazon S3 that are very frequently updated. I want to implement few anti-scraping mechanisms like banning IP's which are making too many requests or making ...
2
votes
0answers
977 views
AWS Client S3 mv command is slow
I am using aws cli to move the files around in S3 in the same bucket. As far as I know, AWS S3 does not maintain the folder structure. It stores the entire path as key to the file and displays it in ...
1
vote
1answer
2k views
multiple s3 users in one bucket or per-user buckers
intelligent people, noob here!
I am planning on building a multi-user apparatus on s3 for photo/object storage, and I was planning on using s3. I have the whole front-end planned out but I have a ...
0
votes
1answer
481 views
Can I host a static website on Amazon EC2 coupled with EFS?
My website is currently hosted on EC2 with EBS volume. But very often I run a java program on the same instance (Which outputs to EBS) to generate/update files for the website. CPU & memory peaks ...
1
vote
2answers
644 views
S4cmd working only for Amazon S3. Is it possible to configure it for DreamObjects cloud storage the way S3cmd worked?
I just switched from S3cmd to S4cmd. It's working great for EC2 to Amazon S3 data transfer. But when I'm trying DreamObjects or CenturyLink Object storage in it's place, it shows the below error:
...
0
votes
2answers
133 views
Centralizing data AWS
I want to centralizer about 1TB of data in AWS for 3 on premise locations (5 users each location). We are going growing but still small. Each location has a 30/5mpbs connection. Each has its own Linux ...
