OSX 10.11 - python3.5 or AWS CLI (or other tool?)
I have ~ 5,000 subdirectories within an Amazon S3 bucket, each subdirectory contains a single .tar. In each .tar it contains only one .zip, ~<1mb in size.
What I would like to do is run a script that will access each subdirectory within the S3 bucket and copy this .zip found within each .tar to either a given s3 location, or to a local destination.
Each .tar is ~10-15GB when uncompressed, so extracting the full contents is not feasible/wanted. I do believe that the .tar header can instead be read, in order to locate the .zip and copy.
Can you tell me of a way I can achieve this