CloudFormation Linter (cfn-lint) is a static code analysis tool that validates CloudFormation YAML and JSON templates against the CloudFormation Resource Specification.
Engineers who work with CloudFormation normally run this tool locally on their machines to fix any issues with the templates and correct them immediately.
However… there are use cases where you work in bigger teams and need to manage hundreds of templates.
Integrating cfn-lint in a CI/CD pipeline helps to enforce rules over shared CloudFormation templates, which in turn makes standardizing rules and guidelines over bigger teams easier.
We are going to be integrating cfn-lint in AWS CodePipeline using a custom AWS CodeBuild project.
AWS CodePipeline Overview
AWS CloudFormation CodePipeline linter diagram
In this example, we have an AWS CodePipeline that contains the source code including the CloudFormation templates in an AWS CodeCommit repository.
Next up we have the Validation stage where we have an AWS CodeBuild action that contains the cfn-lint tool. The templates will be scanned and the report will be sent to an encrypted S3 bucket.
The last stage contains the actual deployment after the validation has finished.
Creating AWS CodePipeline in CloudFormation
If we're building the pipeline it looks as follows (snippet):
CodePipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
ArtifactStore:
Location:
Ref: ArtifactStore
Type: S3
Name: !Ref CloudFormationPipelineName
RoleArn: !GetAtt CodePipelineRole.Arn
RestartExecutionOnUpdate: False
Stages:
- Name: Source
Actions:
- Name: GetSource
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: CodeCommit
OutputArtifacts:
- Name: source
Configuration:
BranchName: master
RepositoryName: cloudformation-codecommit-repo
- Name: Validation
Actions:
- Name: Linter
RunOrder: 1
InputArtifacts:
- Name: source
ActionTypeId:
Category: Build
Owner: AWS
Version: 1
Provider: CodeBuild
Configuration:
ProjectName: !Ref CodeBuildLinter
- Name: Deploy
Actions:
- Name: Dev
RunOrder: 1
InputArtifacts:
- Name: source
ActionTypeId:
Category: Build
Owner: AWS
Version: 1
Provider: CodeBuild
Configuration:
ProjectName: !Ref CodeBuildDev
ArtifactStore:
Type: AWS::S3::Bucket
Properties:
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: aws:kmsThe CloudFormation template snippet contains the three stages that were mentioned before Source -> Validation -> Deploy including an artifact bucket to save the artifacts from the build.
Creating AWS CodeBuild in CloudFormation
The most important part of this pipeline is the validation stage which contains the CodeBuild step:
CodeBuildLinter:
Type: AWS::CodeBuild::Project
Properties:
Artifacts:
Type: CODEPIPELINE
Description: Linting CloudFormation templates and sending reports to S3
Environment:
ComputeType: BUILD_GENERAL1_SMALL
EnvironmentVariables:
- Name: AWS_ACCOUNT_ID
Value: !Sub ${AWS::AccountId}
- Name: CFN_LINT_BUCKET
Value: !Ref CfnLintBucket
- Name: CFN_LINT_KMS_KEY_ARN
Value: !Ref CfnLintKmsKeyId
- Name: CODE_PIPELINE_NAME
Value: !Ref ProvisioningPipelineName
Image: aws/codebuild/standard:3.0
Type: LINUX_CONTAINER
ServiceRole: !GetAtt CodeBuildRole.Arn
Source:
Type: CODEPIPELINE
BuildSpec: buildspec-linter.yml
TimeoutInMinutes: 20The cfn-lint tool is added as a CodeBuild action in the validation stage of this pipeline. We add additional environment variables like the AWS Account ID, S3 report bucket, KMS Key, and the pipeline name.
The next step is to set up the buildspec of the CodeBuild project and add the container with the commands that need to be executed:
version: 0.2
phases:
install:
runtime-versions:
python: 3.8
commands:
- apt-get update -y
- apt-get install -y uuid-runtime
- pip install cfn-lint
build:
commands:
- |
REPORT=$(uuidgen)-cfnlint-report.json
set +e; cfn-lint -f json > $REPORT
aws s3 cp $REPORT s3://$CFN_LINT_BUCKET/$AWS_ACCOUNT_ID/$CODE_PIPELINE_NAME/$(date +'%Y')/$(date +'%m')/$(date +'%d')/ --sse=aws:kms --sse-kms-key-id $CFN_LINT_KMS_KEY_ARN --acl bucket-owner-full-controlFirst, we install the required packages in the container. Next, we run cfn-lint and make sure the command won't exit (set +e) when it reports an error. In this situation, we don't want to stop exit CodeBuild if we discover an error because we use cfn-lint only for reporting.
In this setup, I've included a configuration file that specifies what templates need to be analyzed.
After the analysis, it will generate a report in JSON format which will be uploaded to an encrypted S3 bucket. As you can see the S3 CLI command is being used to transfer the report to the S3 bucket.
We've included the variables which we passed from the AWS::CodeBuild::Project resource to dynamically create the object and enable KMS Encryption.
After the report is uploaded to S3 it is ready for consumption for other applications.
Enable AWS KMS & S3 Bucket
If you enable AWS KMS, make sure to include the following policies for the CodeBuild role:
- Action:
- s3:ListBucket
- s3:ListBucketVersions
- s3:ListObjects
- s3:PutObject
- s3:PutObjectAcl
Effect: Allow
Resource:
- !GetAtt CfnLintBucket.Arn
- !Sub ${CfnLintBucket.Arn}/*
Sid: Allow S3 put CfnLintBucket
- Action:
- kms:Encrypt
- kms:ReEncrypt*
- kms:GenerateDataKey*
- kms:DescribeKey
- kms:ListKeys
Effect: Allow
Resource: !GetAtt KMSKey.Arn
Sid: Allow use of CMK in CfnLintBucketThe CodeBuild role is allowed to put the report in the destination bucket (CfnLintBucket). We also allow the role to encrypt the object using the KMSKey we created for the S3 Bucket.
Then we need to explicitly allow the CodeBuildRole on the KMS Key policy, we can do that as follows:
- Sid: Allow encrypt use of the key by the CodeBuildRole
Effect: Allow
Principal:
AWS:
- !GetAtt CodeBuildRole.Arn
Action:
- kms:Encrypt
- kms:ReEncrypt*
- kms:GenerateDataKey*
- kms:DescribeKey
- kms:ListKeys
Resource: '*'To Summarize
We've set up a CodePipeline which adds cfn-lint as a validation, stage to analyze your CloudFormation templates and send the reports to an encrypted S3 Bucket.
This allows auditors for example to check how well the templates are built according to best practices.