Clean up unused security groups in AWS

When you’re working with AWS, you might create security groups that are no longer needed.

These unused security groups can clutter up your account and make it difficult to manage your security configuration.

In this tutorial, we’ll show you how to use Python and the boto3 library to find and delete unused security groups in a single AWS Region.

How to delete all unused security groups in an AWS Region

Before you can start, you’re required to have done the following prerequisites before you can run the Python script on your AWS account.

1. Install the AWS CLI and configure an AWS profile
2. Setting up the Python Environment

If you’ve already done this, you can proceed to step 3.

1. Install AWS CLI and configure an AWS profile

The AWS CLI is a command line tool that allows you to interact with AWS services in your terminal.

Depending on if you’re running Linux, macOS, or Windows the installation goes like this:

# macOS install method:
brew install awscli

# Windows install method:
wget https://awscli.amazonaws.com/AWSCLIV2.msi
msiexec.exe /i https://awscli.amazonaws.com/AWSCLIV2.msi

# Linux (Ubuntu) install method:
sudo apt install awscli

In order to access your AWS account with the AWS CLI, you first need to configure an AWS Profile. There are 2 ways of configuring a profile:

• Access and secret key credentials from an IAM user
• AWS Single Sign-on (SSO) user

In this article, I’ll briefly explain how to configure the first method so that you can proceed with running the python script on your AWS account.

If you wish to set up the AWS profile more securely, then I’d suggest you read and apply the steps described in setting up AWS CLI with AWS Single Sign-On (SSO).

In order to configure the AWS CLI with your IAM user’s access and secret key credentials, you need to login to the AWS Console.

Go to IAM > Users, select your IAM user, and click on the Security credentials tab to create an access and secret key.

Then configure the AWS profile on the AWS CLI as follows:

➜ aws configure
AWS Access Key ID [None]: <insert_access_key>
AWS Secret Access Key [None]: <insert_secret_key>
Default region name [None]: <insert_aws_region>
Default output format [json]: json

Your was credentials are stored in ~/.aws/credentials and you can validate that your AWS profile is working by running the command:

➜ aws sts get-caller-identity
{
    "UserId": "AIDA5BRFSNF24CDMD7FNY",
    "Account": "012345678901",
    "Arn": "arn:aws:iam::012345678901:user/test-user"
}

2. Setting up the Python Environment

To be able to run the Python boto3 script, you will need to have Python installed on your machine.

Depending on if you’re running Linux, macOS, or Windows the installation goes like this:

# macOS install method:
brew install python

# Windows install method:
wget https://www.python.org/ftp/python/3.11.2/python-3.11.2-amd64.exe
msiexec.exe /i https://www.python.org/ftp/python/3.11.2/python-3.11.2-amd64.exe

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py

# Linux (Ubuntu) install method:
sudo apt install python3 python3-pip

Once you have installed Python, you will need to install the boto3 library.

You can install boto3 using pip, the Python package manager, by running the following command in your terminal:

pip install boto3

3. Create the Python Script to Delete Unused Security Groups in a single AWS Region

Once you have our environment set up, you can create the Python script.

Copy the following code into a new file on the desired location and name it: delete_unused_security_groups.py.

#  https://github.com/dannysteenman/aws-toolbox
#
#  License: MIT
#
# This script deletes all unused security groups in a single AWS Region

import boto3
from botocore.exceptions import ClientError

if __name__ == "__main__":
    ec2 = boto3.client("ec2")
    elb = boto3.client("elb")
    elbv2 = boto3.client("elbv2")
    rds = boto3.client("rds")

    used_SG = set()

    # Find EC2 instances security group in use.
    response = ec2.describe_instances()
    for reservation in response["Reservations"]:
        for instance in reservation["Instances"]:
            for sg in instance["SecurityGroups"]:
                used_SG.add(sg["GroupId"])

    # Find Classic load balancer security group in use
    response = elb.describe_load_balancers()
    for lb in response["LoadBalancerDescriptions"]:
        for sg in lb["SecurityGroups"]:
            used_SG.add(sg)

    # Find Application load balancer security group in use
    response = elbv2.describe_load_balancers()
    for lb in response["LoadBalancers"]:
        for sg in lb["SecurityGroups"]:
            used_SG.add(sg)

    # Find RDS db security group in use
    response = rds.describe_db_instances()
    for instance in response["DBInstances"]:
        for sg in instance["VpcSecurityGroups"]:
            used_SG.add(sg["VpcSecurityGroupId"])

    response = ec2.describe_security_groups()
    total_SG = [sg["GroupId"] for sg in response["SecurityGroups"]]
    unused_SG = set(total_SG) - used_SG

    print(f"Total Security Groups: {len(total_SG)}")
    print(f"Used Security Groups: {len(used_SG)}\n")
    print(f"Unused Security Groups: {len(unused_SG)} compiled in the following list:")
    print(f"{list(unused_SG)}\n")

    # Delete unused security groups, except those containing "default" in the name
    for sg_id in unused_SG:
        response = ec2.describe_security_groups(GroupIds=[sg_id])
        sg_name = response["SecurityGroups"][0]["GroupName"]
        if "default" in sg_name:
            print(
                f"Skipping deletion of security group '{sg_name}' (ID: {sg_id}) because it contains 'default'"
            )
        else:
            try:
                print(f"Deleting security group '{sg_name}' (ID: {sg_id})")
                ec2.delete_security_group(GroupId=sg_id)
            except ClientError as e:
                if e.response["Error"]["Code"] == "DependencyViolation":
                    print(
                        f"Skipping deletion of security group '{sg_name}' (ID: {sg_id}) because it has a dependent object."
                    )
                else:
                    raise e

The script first finds all the security groups that are currently in use by EC2 instances, load balancers, and RDS instances by making API calls to the respective AWS services.

It then compares the list of all security groups in the Region with the list of security groups in use and identifies the security groups that are not in use.

After identifying the unused security groups, the script attempts to delete them using the delete_security_group method provided by the EC2 service client in boto3.

However, the script checks if the security group contains “default” in its name, and if it does, it skips the deletion process.

This is because security groups with “default” in their name are pre-existing security groups that are essential for the functioning of AWS and should not be deleted.

The script also checks for dependency violations before attempting to delete a security group.

If a security group contains in- or outbound rules where another security group is associated, then the script skips the deletion of that security group and moves on to the next one.

4. Run the python script on your AWS account

To run the script, simply execute the following command in your terminal or command prompt:

python delete_unused_security_groups.py

The script will start running, and you should see output similar to the following:

➜ python delete_unused_security_groups.py

Total Security Groups: 3
Used Security Groups: 0

Unused Security Groups: 3 compiled in the following list:
['sg-05fb07fc61fe187ad', 'sg-0d48a3989d74bd109', 'sg-06db595a19bbd3441']

Deleting security group 'test1-sg' (ID: sg-05fb07fc61fe187ad)

Skipping deletion of security group 'default' (ID: sg-0d48a3989d74bd109) because it contains 'default'
Deleting security group 'test2-sg' (ID: sg-06db595a19bbd3441)

The output will show the total number of security groups, the number of used security groups, the number of unused security groups, and the names and IDs of the deleted security groups.

This information can be used to verify that the script has completed its task successfully.

Conclusion

In this tutorial, you’ve learned how to find and delete unused security groups in an AWS Region using a Python script based on the boto3 library.

By following the steps outlined in this tutorial, you can easily clean up your AWS environment and improve your security posture by removing unused security groups for EC2, Load balancers, and RDS resources.

Remember to always exercise caution when performing any delete actions on your AWS environment, and to follow AWS best practices to keep your infrastructure secure and well-maintained.