65
votes
Accepted
Is there a way to list available/configured AWS CLI profiles?
You can list profiles in your aws config/credential files using list-profiles:
aws configure list-profiles
This is working for me with aws-cli version 2.0.8.
- 766
62
votes
Amazon S3 Recover Deleted File
If you have versioning enable, yes you can!
On a versioned bucket, a delete action on a file does not really delete it but it adds a version with a "Delete Marker". You can delete the delete marker ...
- 721
39
votes
permission denied on AWS Transfer on SFTP server
User Role should be:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowListingOfUserFolder",
"Action": [
"s3:ListBucket",
...
- 1,549
30
votes
parsing json in bash with pipe operators
Yes, there are several different tools that have a full JSON parser and some form of query language (along the lines of XML having XPath).
jq -r .DBClusterSnapshots[0].Status
jshon -e ...
- 408k
23
votes
Is there any option to use Amazon Cloud Drive from terminal (Linux)?
At the time of asking the answer was unfortunately No, but no more! You can now easily access Amazon Cloud Drive from the terminal using an OpenSource client called acd_cli, which is written in Python ...
- 1,178
23
votes
AWS upload folder to S3 as tar.gz without compressing locally
What you're really looking for is not saving a local file. You can use pipes to send the data from tar through gzip to s3 without saving anything to disk.
tar c /var/test | gzip | aws s3 cp - "s3://...
- 60.3k
22
votes
Accepted
parsing json in bash with pipe operators
The AWS CLI tools have a built-in --query parameter that accepts a JMESPath expression to select a subset of the JSON output.
Your example would look something like this:
aws rds describe-db-cluster-...
- 335
20
votes
Is there any option to use Amazon Cloud Drive from terminal (Linux)?
I've been using http://rclone.org/ with much success. I find it quite mature and robust. It also includes bandwidth throttling that works!
Edit: Please note that this is no longer the case. Amazon ...
- 301
17
votes
Time machine backup on S3
Using AWS S3 as the storage of Mac Time Machine definitely works now.
Time Machine can backup the data to external device, so the key is mounting S3 bucket as a POSIX FS in Mac.
There are many tools ...
- 475
17
votes
boto responds: AWS was not able to validate the provided access credentials
Had a similar issue and decided to post it as an answer given that this may help others (coming here from Google):
Make sure the time on your machine is set correctly.
My machine's time was running ...
- 270
16
votes
Accepted
Unable to run AWS CLI: "ImportError: cannot import name 'docevents' from 'botocore.docs.bcdoc'"
try running
pip3 install --upgrade awscli
or
pip install --upgrade awscli
- 176
15
votes
Is there a way to list available/configured AWS CLI profiles?
(Answering my own question.)
No, there is not.
I wrote two scripts that include the parsing I ended up using. For anyone interested, they're available in two GitHub repositories:
awscli-mfa and aws-...
- 2,722
14
votes
Is there a way to list available/configured AWS CLI profiles?
Parsing ~/.aws/credentials was simple enough for me.
$ cat ~/.aws/credentials | grep -o '\[[^]]*\]'
=> [default] [other_profile] [other_profile2]
I also aliased the command into aws-profiles by ...
- 270
14
votes
aws --version error: bash: /usr/bin/aws: No such file or directory
It turns out the problem was that bash had cached the path /usr/bin/aws from a previous install of an old version 1 of aws cli. Prior to the commands shown in my question, I had deleted the old ...
- 401
13
votes
Accepted
How to ssh from one ec2 instance to another?
Method 1 - use the same keys on the servers:
Convert the keys to openssh format and upload the private keys to the servers. When you ssh to the destination host, specify the private key file:
ssh -i ...
- 266
11
votes
Accepted
SSH via multiple hosts or bastions
Method 1 – onion (nested tunnels)
With OpenSSH 7.3 and later:
Host webserverA
ProxyJump bastionA,bastionB
The same via command line:
$ ssh -J bastionA,bastionB webserverA
Alternatively (also ...
- 408k
11
votes
AWS CLI, using `--cli-input-json` in a pipeline
Found a workaround for the time being with xargs that is quite clean:
cat ./mytask.json \
| xargs -0 aws ecs register-task-definition --cli-input-json
It only adds xargs -0 and requires --cli-...
- 1,331
10
votes
Accepted
AWS EC2 Linux headless Firefox issue: Xvfb: undefined symbol: pixman_glyph_cache_create
It is trying to load wring pixman so. Delete /usr/local/lib/libpixman-1.so.0 and try again, it will make Xvfb use distribution pixman library which has this symbol.
- 116
9
votes
Accepted
In elasticbeanstalk, can I ssh in and manually restart the (node) app?
To modify the running code, edit the files in:
/var/app/current
Then, running:
sudo pkill -f node
will cause your node app to be restarted, with the latest code running.
- 218
9
votes
Accepted
AWS allow user to call create-role
If you have root access to your account, you can just write your own policy and attach it to the user (AWS Console => IAM => Users => Add inline policy). Here is an example:
{
"Version": "2012-10-...
- 336
8
votes
Accepted
Yum Install software-properties-common on EC2
The answer is that software-properties-common is a package for managing packages installed with Apt, so since CentOS uses Yum, it's irrelevant anyway.
- 459
8
votes
Accepted
Unable to mount Elastic File System on EC2 (mount.nfs4: Connection timed out)
You probably need to create inbound policies. From the amazon documentation:
The file system must have a mount target whose network interface has a security group with a rule than enables inbound ...
- 96
8
votes
How to ssh from one ec2 instance to another?
There is a 3rd and IMHO the best solution so called ssh agent forwarding:
on local machine configure ~/.ssh/config, by adding following section:
Host <ip-or-name-of-A-server>
ForwardAgent ...
- 193
8
votes
AWS upload folder to S3 as tar.gz without compressing locally
tar cvfz - /var/test | aws s3 cp - s3://tests/test1.tar.gz
You don't have to separately gzip; tar does that for you with the z option.
This works both in directions.
- 81
8
votes
scp command silently does nothing
You are missing a colon (:) after the "target" argument:
scp -i learndeepai.pem sb_code.zip ubuntu@34.211.158.***:
Without the colon, the command is an equivalent to cp, so it copies learndeepai.pem ...
- 20.5k
8
votes
Accepted
How to solve 404 Error in AWS apg-get for debian jessie fetch?
The error has nothing to do with Amazon or with your CircleCI configuration.
Jessie's repos have moved to debian's archive and you need to update the following file:
/etc/apt/sources.list
The new ...
- 4,430
8
votes
How to import .ppk file in winscp?
Open Winscp
Go to Session window, fill the details with Host name Username and password
Click on Edit button, then it will highlight the Advanced button.
Click on that,
select SSH → Authentication on ...
- 181
7
votes
How to install let's Encrypt on Amazon Linux
Amazon's EPEL doesn't seem to have certbot for me. Try manually installing it, here's a pair of articles with full instructions:
Try https://ivopetkov.com/b/let-s-encrypt-on-ec2/ for Apache
Or ...
7
votes
Accepted
Can you recover ~/.ssh contents on AWS EC2 with an open session and keys?
You cannot recover it.
But as long as you have your session open, you can re-create the authorized_keys the same way, you would create it for the very first time on a normal *nix system.
create ~/....
- 20.5k
7
votes
Accepted
AWS CLI, using `--cli-input-json` in a pipeline
I went digging... It looks like aws will read the indicated file twice, using the second dataset for it's operation. Of course, in a pipeline, the second read() will get nothing.
I've added a pipe:// ...
- 18.7k
Only top scored, non community-wiki answers of a minimum length are eligible