Speeding Up Docker Builds

Docker caching starts with a wonderfully simple idea:

If nothing changed, don't do the work again.

On a developer laptop, that works almost effortlessly. The machine stays around, yesterday's cache remains on disk, and the next build can reuse it.

CI is more interesting.

The machine doing the build may exist for only a few minutes, disappear when the job finishes, and be replaced by a completely fresh runner for the next build. Meanwhile, multiple builds may be running in parallel.

That was the Docker cache problem we had to solve at Monk CI.

How Docker Build Cache Works

Take a simple Dockerfile:

FROM node:22

COPY package.json .
RUN npm install

COPY . .
RUN npm run build

The first build has no cache to reuse. It pulls the base image, copies files, installs dependencies, and builds the application.

But Docker and BuildKit do not treat this as one giant operation. They track reusable build steps based on their inputs.

If the next commit changes only application code, the dependency installation can still be reused:

FROM node:22       ✓ reuse
COPY package.json  ✓ reuse
RUN npm install    ✓ reuse
COPY .             ↻ changed
RUN npm run build  ↻ rebuild

That is the value of Docker build caching: remembering expensive work when its inputs have not changed.

On persistent infrastructure, this is straightforward.

In CI, the cache may disappear with the machine.

The Challenge of Docker Cache in CI

Monk CI runners are ephemeral.

A job gets a fresh runner, performs its work, and the runner can disappear when the job finishes.

That gives us clean environments and elastic infrastructure, but it creates a problem for local Docker cache storage.

Build #1 → Runner A → creates useful cache → Runner A disappears
Build #2 → Runner B → starts with no local cache

If useful Docker build state lives only on the runner, every new machine risks starting from scratch.

So the key principle became:

The runner may be disposable. The useful Docker build state should not be.

We separate the lifecycle of compute from the lifecycle of reusable build state.

A runner can start completely fresh while still benefiting from work completed by previous builds.

But ephemeral runners are only half the problem.

Then parallel CI enters the picture.

One Docker Cache, Many Parallel Builds

A repository may have multiple pull requests building at once. A workflow may fan out into several jobs, and new commits may arrive before previous builds have finished.

Instead of:

Build A → Cache

you now have:

Build A ─┐
Build B ─┤
Build C ─┼──→ shared build history
Build D ─┤
Build E ─┘

The obvious answer might be to place the Docker cache on persistent storage and let every runner access it directly.

But BuildKit cache state is not simply a collection of immutable files waiting to be downloaded.

Builds create snapshots, update metadata, reuse existing state, and eventually clean up old data. Multiple independent machines directly sharing ownership of mutable build state creates a coordination problem.

You could solve that by serializing access:

Build A → cache
Build B → wait
Build C → wait
Build D → wait

afe, perhaps.

But it defeats much of the purpose of running CI jobs in parallel.

A cache should remove work, not replace it with another queue.

Keeping Runners Ephemeral and Build State Persistent

The approach we use at Monk CI is to separate the machine running the CI job from the system responsible for reusable Docker build state.

At a high level:

Runner A ─┐
Runner B ─┼──→ BuildKit-backed build system ───→ reusable build state
Runner C ─┘

The CI runners remain temporary.

The build environment owns and manages the reusable state.

This creates an important responsibility boundary. Short-lived runners do not need direct ownership of the same mutable cache disk, while the build system remains responsible for deciding which layers and build results can actually be reused.

We are not trying to reinvent Docker caching.

We are giving Docker cache for CI a place where it can survive after the CI runner disappears.

Sharing Useful Work Across Builds

This becomes particularly valuable when parallel builds share a common history.

Imagine three pull requests that start with the same base image and dependencies:

              base image
                  │
           system packages
                  │
            dependencies
             /    |    \
          PR #41 PR #42 PR #43

The application layers may eventually differ, but much of the earlier build graph can be identical.

The goal is not to make every build the same.

The goal is to avoid paying for the same work multiple times.

That can mean fewer repeated dependency installations, fewer repeated image pulls, and less compute spent rebuilding unchanged layers.

This is where Docker caching in CI/CD becomes especially useful: parallel builds often share more reusable work than they initially appear to.

Docker Cache Needs Boundaries Too

Of course, a shared cache cannot mean share everything with everything.

Build state needs appropriate boundaries so unrelated repositories and incompatible environments do not casually inherit each other's cache.

It also needs a lifecycle.

Keeping every layer forever would eventually create a very expensive archive of historical Docker builds.

Useful state should stay warm. Old and idle state can eventually be cleaned up.

Think refrigerator, not museum.

Keep what is useful.

Eventually throw out the mysterious container from February.

The Cache Must Never Become a Requirement

One of the most important constraints is this:

Caching may make a build faster. It must not be required to make the build work.

If the cached build path is unavailable, the job should still be able to build normally.

                    ┌─ Cache available   → reuse work → faster build
Build request ──────┤
                    └─ Cache unavailable → normal build

The failure mode should be slower, not broken.

Optimizations have a habit of quietly becoming critical dependencies. We prefer the boring failure mode.

Fresh Machines Without Fresh Builds

The Docker caching problem in CI comes down to two things:

1. 1The machine holding the cache can disappear.
2. 2Multiple builds may want to reuse build state at the same time.

The Monk CI approach is to keep runners disposable while maintaining reusable Docker build state outside their lifecycle, with a BuildKit-backed build environment responsible for coordinating reuse.

In simpler terms:

Throw away the runner              ✓
Throw away useful build history    ✗
Let every runner fight over a disk ✗
Let the build system manage state  ✓

The result is the property we wanted from the beginning:

Fresh machines without fresh builds.

Your CI runner can forget everything when the job ends.

Monk CI makes sure Docker doesn't have to.

Build Faster with Monk CI

Faster CI is not just about adding more compute. It also means avoiding work that has already been done.

Connect your GitHub repository to Monk CI and run your Docker builds on infrastructure designed to keep builds fast, reliable, and ready to scale.