Running AWS CodeBuild Locally

I’ve been using AWS CodeBuild a lot in the past couple of weeks. I really like the elegant simplicity of it: take a buildspec.yml and a containerized build environment and run your build.

Such a simple architecture should be pretty easy to run locally - and as it turns out there was a locally runnable CodeBuild harness announced last year. This is useful for testing your build process locally before you commit instead of iterating with many round-trips through your git repo and to the CodeBuild service.

There are three key references you’ll need if you want to use this:

• AWS CodeBuild Local Builds - The repo README provides partial documentation on how to use local CodeBuild. It provides a complete list of switches but does not show HOW to use the tool. For that see the announcement blog post.
• Announcing Local Build Support for AWS CodeBuild - The announcement blog post for local CodeBuild. Provides a more complete explanation for how to use it.
• AWS CodeBuild curated Docker images - The home of the build environment Docker images. The README covers the clone and docker steps to build those images locally.

The announcement blog post gives a good walkthrough but there were a couple of things that didn’t work exactly the same for me, presumably due to changes since the original announcement, so here are some updates:

First get the CodeBuild local software:

docker pull amazon/aws-codebuild-local:latest --disable-content-trust=false

Get the runner script:

wget https://raw.githubusercontent.com/aws/aws-codebuild-docker-images/master/local_builds/codebuild_build.sh
chmod 700 ~/codebuild_build.sh

Get the Docker images and build them. Here is the first difference as compared to the blog post because the paths and tags for the images are different:

git clone https://github.com/aws/aws-codebuild-docker-images.git

# Each image takes about 20 minutes.  The Amazon Linux image uses around 11Gb
# and the Ubuntu image about 8Gb - ensure you have enough space!

sudo docker build -t aws/codebuild/standard:1.0 aws-codebuild-docker-images/al2/x86_64/standard/1.0/

sudo docker build -t aws/codebuild/ubuntu:2.0 aws-codebuild-docker-images/ubuntu/standard/2.0/

These two images correspond to the two environments available in CodeBuild - amazonlinux:1.0 and ubuntu:2.0. You can just build the one you need if preferred:

Continuing with the steps from the blog post, clone the sample project:

git clone https://github.com/karthiksambandam/sample-web-app.git

In the blog post the author shows you a failed build first before adding the buildspec. Let’s go ahead and add the buildspec right away - this differs slightly from the blog post in that we need to add the install section which identifies the runtimes you want to use within the image. Add the following file as buildspec.yml in sample-web-app:

version: 0.2

phases:

  install:
    runtime-versions:
      java: openjdk8

  build:
    commands:
      - echo Build started on `date`
      - mvn install

artifacts:
  files:
    - target/javawebdemo.war

Then run your build - this also differs from the blog post because of the changed image name. Also assuming you didn’t add your Linux user account to the docker group you’ll also need to sudo this:

mkdir ~/artifacts

sudo ./codebuild_build.sh -i aws/codebuild/ubuntu:2.0 -a ~/artifacts -s sample-web-app

My real build needs to call a Lambda function, so therefore CodeBuild local needs AWS credentials. There are two ways to do this:

• If running the build on an EC2 instance then just apply an instance profile linked to a role with the necessary permissions - this could be a role linked to the same policy that you will attach to your CodeBuild service role.
• Configure local credentials using aws configure. For this option if you’re sudo’ing codebuild_build.sh, then you must sudo aws configure so that the credentials are stored in the root user account. You can then add the -c switch:

sudo aws configure
...

sudo ./codebuild_build.sh -i aws/codebuild/standard:1.0 -a ~/artifacts -s ~/my-project/ -c

Ok after this little detour to speed up my dev-test cycle with CodeBuild local, its time to get back to trying to figure out how to get Maven to stop outputting megabytes of downloading messages to the log which supposedly is possible since 3.6.1.