At $job, we’ve started using AWS CodePipeline and AWS CodeBuild to build and deploy our Docker images to ECS. Once we got the pipeline working, we could git push and a few minutes later a container build would kickoff. This was mostly great, but the problem was that our builds took too long. Developers would push their changes and expect to see them live, but it took about half an hour for the build to complete and go live.

Speeding up our Docker builds on CodeBuild

Docker layer caching leads to way faster container builds, and CodeBuild supports caching, so we should be good, right?

Wrong. CodeBuild doesn’t support Docker layer caching at all (as of this writing). You have to roll your own here.

Docker pull and –cache-from

Specifically, before we run a docker build, we have our CodeBuild buildspec.yml pull in the Docker image of our last build, so we can use it as a source to cache from.

docker pull $ECR_URL:latest || true

Then when we build a container, we use docker build --cache-from $ECR_URL:latest. This, along with reorganizing our Dockerfile to make it more cache-friendly, has cut our builds time by two-thirds. Pretty cool.