0

I have setup an AWS S3 bucket for personal use and have configured a Lambda function running a Python script which runs at 12AM EST every day.

The Python script connects to the S3 bucket using boto3, downloads 3 files, modifies them and uploads them again.

By my reasoning this should mean that every day I should generate 3 S3 PUT requests on my AWS account. However, we are on the 4th day of the month and I have already racked up 58000 PUT requests.

Obviously I want this to stop however I'm not sure whether this is my Python script causing this or not.

Is there a way that I can view all S3 PUT requests on AWS with as much detail as possible?

1
  • You could consider activating CloudTrail. It won't give you access logs, but it does record all API Requests in the account. – Tim Sep 8 '17 at 1:01
3

Just follow the official documentation to enable S3 logging: http://docs.aws.amazon.com/AmazonS3/latest/user-guide/server-access-logging.html

After enabling this option you'll be receiving log files that look like this:

79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be mybucket [06/Feb/2014:00:00:38 +0000] 192.0.2.3 79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be 3E57427F3EXAMPLE REST.GET.VERSIONING - "GET /mybucket?versioning HTTP/1.1" 200 - 113 - 7 - "-" "S3Console/0.4" -
79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be mybucket [06/Feb/2014:00:00:38 +0000] 192.0.2.3 79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be 891CE47D2EXAMPLE REST.GET.LOGGING_STATUS - "GET /mybucket?logging HTTP/1.1" 200 - 242 - 11 - "-" "S3Console/0.4" -
79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be mybucket [06/Feb/2014:00:00:38 +0000] 192.0.2.3 79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be A1206F460EXAMPLE REST.GET.BUCKETPOLICY - "GET /mybucket?policy HTTP/1.1" 404 NoSuchBucketPolicy 297 - 38 - "-" "S3Console/0.4" -
79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be mybucket [06/Feb/2014:00:01:00 +0000] 192.0.2.3 79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be 7B4A0FABBEXAMPLE REST.GET.VERSIONING - "GET /mybucket?versioning HTTP/1.1" 200 - 113 - 33 - "-" "S3Console/0.4" -
79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be mybucket [06/Feb/2014:00:01:57 +0000] 192.0.2.3 79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be DD6CC733AEXAMPLE REST.PUT.OBJECT s3-dg.pdf "PUT /mybucket/s3-dg.pdf HTTP/1.1" 200 - - 4406583 41754 28 "-" "S3Console/0.4" -
79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be mybucket [06/Feb/2014:00:03:21 +0000] 192.0.2.3 79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be BC3C074D0EXAMPLE REST.GET.VERSIONING - "GET /mybucket?versioning HTTP/1.1" 200 - 113 - 28 - "-" "S3Console/0.4" -

This should give you an idea of who does what with your bucket. You might have to wait for some time (several hours) for those logs to get delivered.

3
  • Can you also tell where to see logs themselves? I've enabled logging but I still don't know where to look after. Thanks. – mimic Jan 9 '19 at 18:59
  • 1
    @mimic When you enable logging, you specify a target bucket. The logs will be written to this bucket. – Sergey Kovalev Jan 10 '19 at 14:37
  • Sergy, thanks I see now! But it's awful, they created hundred files with a small piece of information in eash. And when I looked at them it has nothing to do with my site. It says about some problem with cors on JDK what is not about my website because it's written in Python. I'm really puzzled... – mimic Jan 10 '19 at 17:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.