Organizing Your AWS CLI Profiles

The AWS CLI is a very powerful tool. I often prefer using it over the console for repetive tasks as it's often much easier to run one command versus a couple of clicks in the console. However, once you are working out of more than one AWS account or need numerous sets of access key pairs there is some additional configuration you can do to make your life a lot easier.

Making use of AWS CLI named profiles allows you to add your access keys for numerous accounts with human readable names. These go in your ~/.aws/credentials file like below.

[account-one]
aws_access_key_id=ABCDEFGHIJKLMNOPQRSTUVWXYZ
aws_secret_access_key=1a2b3c4d5e6f7g8h9i1j2k3l

[account-two]
aws_access_key_id=ZYXWVUTSRQPONMLKJIHGFEDCBA
aws_secret_access_key=l3k2j1i9h8g7f6e5d4c3b2a1

You swap between these various credentials using an environment variable called (to no surprise) AWS_PROFILE. The bash function below is one I've added to my ~/.bash_profile to make this a little easier to do. Once added you just type setawsprof account-two to change the current profile you are using.

function setawsprof { export AWS_PROFILE=$1; }
export -f setawsprof

You need to execute source ~/.bash_profile to register these changes on the first Terminal tab you add this function. From then on it will be automatically added to your Terminal sessions for new tabs. If you're curious of the difference between ~/.bashrc and ~/.bash_profile like I was, check out this article.

It's worth noting that this environment variable is per tab in your Terminal. This means you can have numerous tabs open on numerous profiles which is also very convinient if you are jumping around a couple of different profiles at once.

Another place this can be useful is if you use different regions for different environments. For instance, you could have a account-one-prod and account-one-staging profile that have access to services in different regions on AWS.

When it comes to using Docker and, more specifically, Docker Compose with AWS profiles, that looks like the snippet below from a docker-compose.yml file. You'll want to mount your ~/.aws folder in to the Docker container so that your code is able to authenticate with AWS. At the same time, you'll also want to set the same AWS_PROFILE environment variable inside of the Docker container to ensure it uses the correct credentials.

environment:
  - AWS_PROFILE=account-one
volumes:
  - ~/.aws:/.aws

Alternatively you are able to set explicit, other environment variables such as AWS_ACCESS_KEY_ID to authenticate with the AWS CLI. However, this would then mean that those values will make it to your version control which isn't ideal for security reasons.

Named profiles coupled with the aws configure command can save a lot of time. For instance, executing aws configure get region can return the region associated with that profile. These values go in ~/.aws/config like below.

[profile account-1]
region=us-east-1
output=json

A place I've found this very useful is with ECR when developing with Fargate. The push URLs for ECR repositories change per region. With a setup as described above, I can simply execute the command below to have my Docker image pushed to the correct repository regardless of which profile I am using (assuming they are for the same AWS account).

docker push 1234567890.dkr.ecr.$(aws configure get region).amazonaws.com/image-name:latest

I hope you found this useful! Tweet @joshholat if you have any additional tips for managing AWS profiles.

Ilija Boshkov