Docker layer caching for Rust Cargo dependencies

Contents

  1. 1. Host-mounted cache — failed
  2. 2. Image layer cache — worked

I was working through Chen Tian’s Programming Rust course and building the sample code in Docker. Every run spent time refreshing the crates.io index—even when Cargo.toml had not changed.

There has to be a way to cache that. This is the path I took (including what did not work).

Host-mounted cache — failed

One approach: point CARGO_HOME at a directory on the host so the crate index persists.

1
docker run --rm --user "$(id -u)":"$(id -g)" -v "$PWD":/usr/src/myapp -w /usr/src/myapp -e CARGO_HOME=".cargo" rust cargo run 

That run blew up:

1
2
3
4
5
6
7
8
9
10
11
12
   Compiling libc v0.2.107
Compiling autocfg v1.0.1
Compiling siphasher v0.3.7
Compiling unicode-xid v0.2.2
Compiling memchr v2.4.1
Compiling lazy_static v1.4.0
Compiling ppv-lite86 v0.2.15
Compiling pin-project-lite v0.2.7
Compiling pkg-config v0.3.22
error: failed to build archive: Input/output error

error: error writing dependencies to `/usr/src/myapp/.target/debug/deps/pkg_config-8dafe7024c916ac6.d`: Input/output error (os error 5)

Digging in, it looks like an environment mismatch: pkg-config and native libs do not line up when you mix host paths with the container userspace. References: Rust cross-compile to Darwin troubleshooting and Stack Overflow: Docker image with a Rust binary.

Image layer cache — worked

This Stack Overflow answer (cache-rust-dependencies-with-docker-build) is the trick that stuck.

Idea: dependency layers should stay valid while Cargo.toml is stable. A docker build that copies application source invalidates cache whenever any file changes.

So compile dependencies against a throwaway main, cache that layer, swap in real sources, then build again—fast iteration and warm deps.

Dockerfile:

1
2
3
4
5
6
7
8
9
10
11
12
FROM rust
WORKDIR /app
RUN echo "fn main() {}" > dummy.rs
ARG SRC_DIR
ENV SRC_DIR=${SRC_DIR:-}
COPY ${SRC_DIR}/Cargo.toml .
RUN sed -i 's#src/main.rs#dummy.rs#' Cargo.toml
RUN cargo build --release
RUN sed -i 's#dummy.rs#src/main.rs#' Cargo.toml
COPY ${SRC_DIR}/. .
RUN cargo build --release
ENTRYPOINT ["target/release/app"]

Build arg SRC_DIR lets one Dockerfile serve multiple crates.

Matching Cargo.toml bin section:

1
2
3
[[bin]]
name = "app"
path = "src/main.rs"

Build and run in one shot:

1
2
3
4
5
# run.sh
#!/usr/bin/env bash

WORKSPACE=$(cd "$(dirname "$0")" && pwd -P)
docker run --rm -it "$(docker build --build-arg SRC_DIR="$SRC_DIR" -q "$WORKSPACE" -t "rust-$SRC_DIR")" #"$@"

Example:

1
2
# eg:
# SRC_DIR=scrape_url sh run.sh https://www.rust-lang.org/ rust.md

Dependency downloads stay cached; back to studying.

Sample repo: rust-koan.

Programming Rust course

Questions? Leave a comment below or email: newbvirgil@gmail.com