This article is a must-read for anyone who interacts with AWS using Python, and particularly the Boto3 library.
Configuring credentials in Boto3 might seem like a straightforward task at first, but, in reality, it is a critical process that demands meticulous handling.
Why? Because the way you set up and manage your AWS credentials can have significant impacts on the security of your AWS resources and the efficiency of your development workflows.
In this guide, we will walk you through four methods of specifying credentials in Boto3, starting from the basic approaches of using environment variables and shared credential files to the more advanced and scalable solutions of AWS Config file and AWS IAM Identity Center.
Table of Contents
Before you can start using boto3 on you AWS Account, you’re required to have done the following prerequisites before you can interact with AWS Services using Boto3 with your credentials:
- Install Python3 and Boto3 on your system
- Install the AWS CLI and configure an AWS profile
1. Install Python3
Python3 installation differs based on your operating system:
Download the official Python3 installer from the Python website here.
Run the installer file and follow the prompts, make sure to check the box that says “Add Python to PATH” before clicking Install Now.
MacOS comes with Python 2.7 by default, but you can install Python3 using Homebrew.
If you don’t have Homebrew installed, you can install it by pasting the following command in your terminal:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
Once Homebrew is installed, you can install Python3 by running:
brew install python3
Open your terminal and update the package list using:
sudo apt update
Install Python3 by running:
sudo apt install python3
You can verify your Python installation by running
python --version in your terminal. You should see a response with the Python version number.
2. Installing Boto3
Now that Python3 is installed, you can install Boto3. The process is the same for all operating systems:
- Open your terminal.
- Install Boto3 using pip, which is a package manager for Python. Run the following command:
pip3 install boto3
That’s it! You’ve installed Boto3 on your system.
You can verify the installation by opening your Python interpreter with the command
python, and then try to import the boto3 module using
If you don’t see any error messages, Boto3 was installed successfully.
3. Install AWS CLI and configure an AWS profile
The AWS CLI is a command line tool that allows you to interact with AWS services in your terminal. Depending on if you’re running Linux, macOS, or Windows the installation goes like this:
# macOS install method:
brew install awscli
# Windows install method:
msiexec.exe /i https://awscli.amazonaws.com/AWSCLIV2.msi
# Linux (Ubuntu) install method:
sudo apt install awscli
In order to access your AWS account with the AWS CLI, you first need to configure an AWS Profile. There are 2 ways of configuring a profile:
- Access and secret key credentials from an IAM user
- AWS Single Sign-on (SSO) user
In this article, I’ll briefly explain how to configure the first method so that you can proceed more quickly to set up the Amazon S3 Bucket in AWS CDK.
If you wish to set up the AWS profile more securely, then I’d suggest you read and apply the steps described in setting up AWS CLI with AWS Single Sign-On (SSO).
In order to configure the AWS CLI with your IAM user’s access and secret key credentials, you need to login to the AWS Console.
Go to IAM > Users, select your IAM user and click on the Security credentials tab to create an access and secret key.
Then configure the AWS profile on the AWS CLI as follows:
➜ aws configure
AWS Access Key ID [None]: <insert_access_key>
AWS Secret Access Key [None]: <insert_secret_key>
Default region name [None]: <insert_aws_region>
Default output format [json]: json
Your was credentials are stored in ~/.aws/credentials and you can validate that your AWS profile is working by running the command:
➜ aws sts get-caller-identity
How Boto3 chooses which method to use first when multiple credential methods are available
In Boto3, configuration data is essentially bifurcated into two categories: credentials and non-credentials.
Items such as
aws_session_token fall under the umbrella of credentials. These are the keys that allow you to authenticate your requests when using Boto3.
The non-credential configurations, on the other hand, contain data that isn’t necessarily related to access permissions but are integral to how Boto3 interacts with AWS services.
These include settings like the region in which your resources reside, or the addressing style that Amazon S3 should use.
When Boto3 needs to authenticate a request, it goes on a hunt for the appropriate credentials.
This hunt involves a systematic check of several possible locations where these credentials could be stored. As soon as Boto3 finds valid credentials, it ceases its search.
The order of precedence when Boto3 searches for these credentials is as follows:
- Passing credentials as parameters in the
- Passing credentials as parameters when creating a
- Environment variables
- Shared credential file (
- AWS config file (
- Assume Role provider
- Boto2 config file (
- Instance metadata service on an Amazon EC2 instance that has an IAM role configured.
Down below you’ll find 4 methods of specifying your credentials when connecting to AWS Services using Boto3.
Each method explains the pro’s and cons of the implementation.
Method 1: Using AWS credentials file
The AWS credentials file is a text file on your local machine that stores your AWS access keys. By default, it is located in
You can create and configure the AWS credentials file manually or use the AWS CLI command
To connect to an AWS service, you can use Boto3 like this:
s3 = boto3.resource('s3')
Boto3 will automatically look for AWS credentials in your credentials file.
Method 2: Using environment variables
Environment variables are a way to store key-value pairs in the environment of the operating system.
You can set AWS credentials as environment variables like this:
Once these environment variables are set, you can connect to an AWS service using Boto3 like this:
s3 = boto3.resource('s3)
Boto3 will automatically look for AWS credentials in your environment variables.
Method 3: Using IAM Identity Center
In AWS, the IAM Identity Center, which is the successor to AWS Single Sign-On (SSO), has a unique feature that lets you manage your SSO profiles and login sessions.
This feature was introduced in version 1.14.0 of Boto3 and provides support for single sign-on (SSO) credentials.
Using profiles in your shared configuration file (
~/.aws/config). These profiles could be one of the following:
# SSO Profile
sso_start_url = https://my-sso-portal.awsapps.com/start
sso_region = eu-west-1
sso_account_id = 123456789011
sso_role_name = Administrator
# IAM Role Profile
Now, to create a Boto3 session, you can specify the profile name via the
AWS_PROFILE environment variable or use the
profile_name argument when creating a Session.
The following Python code demonstrates this:
session = boto3.Session(profile_name='my-sso-profile')
s3_client = session.client('s3')
In this code snippet, we first import the Boto3 module. We then create a new session using
boto3.Session() and specify the profile name
Finally, we use this session to create an S3 client. This S3 client will use the credentials associated with the
Method 4: Providing credentials directly in code
Disclaimer: Don’t use this method! Let me tell you why it’s risky.
Embedding credentials directly in your Python scripts might expose them to unauthorized access, especially when the scripts are committed to version control systems.
If you do choose to use this method, please make sure not to expose the script to public repositories:
s3 = boto3.client(
When it comes to setting up credentials for Boto3 to connect to AWS services, you have a multitude of options at your disposal. Each method we’ve discussed offers its unique benefits and use-cases.
However, among these methods, using the AWS IAM Identity Center (Method 4) shines as the most robust and scalable solution.
Why is it the best? By using the IAM Identity Center, you can leverage the power of AWS Single Sign-On (SSO), allowing for enhanced security, streamlined credential management, and easy profile switching.
This approach not only facilitates the efficient management of multiple AWS accounts but also strengthens the overall security by narrowing the attack surface that could be exploited due to credential mishandling.
Moreover, the ability to directly specify these profiles when creating a Boto3 session simplifies the task of maintaining different sessions for different AWS environments.