feat(cargo-cache): hardlink-clone a per-ref Cargo cache from a published snapshot
Replaces the phase-0 resolution probe with the real actions, merging the two independent per-branch Cargo cache implementations on this forge into the design neither of them had. ## The merge - zemyna seeds a PR branch by `cp -al` hardlink clone (near-free: cost scales with inode count, not bytes) from the base branch's LIVE target dir — a torn read waiting for a second job slot (its own #911). - emowheel seeds from a PUBLISHED IMMUTABLE SNAPSHOT (no race by construction) but with `cp -a`, duplicating ~35 GB per branch. This ships hardlink-clone FROM a published snapshot: zemyna's cost profile, emowheel's soundness, and #911 closed structurally rather than by the runner happening to have one execution slot. ## The bug both implementations have A build inside a `cp -al` clone DOES mutate the directory it was cloned from. Cargo replaces real artifacts, but writes its metadata — and build scripts write their OUT_DIR — with a plain truncating write, straight through the shared inode. Measured set: `.fingerprint/<unit>/dep-<target>` (under CARGO_UNSTABLE_CHECKSUM_FRESHNESS), `build/<pkg>/{output,root-output,out/**}`, `deps/*.d` and `<profile>/*.d`. The checksum-freshness case is a wrong answer, not a slow build: a PR clone rewrites the base's dep-info to describe the PR's sources while the base's cache still holds the artifact built from the base's; once the PR merges, the base's next run finds the checksums match, reports `Fresh`, and links a binary built from the pre-merge code. Reproduced end to end. Fix: hardlink the artifacts (the GB), real-copy the metadata (the MB) — about 3.7% of a 6.9 GB Bevy target dir, against 100% for a full copy. ## Contents - `cargo-cache/action.yml` — consume: resolve keys, seed from the base's snapshot via staging + one atomic rename, strip Cargo lock files, unshare the mutable paths, restore mtimes from git history, lock, prune. - `cargo-cache-publish/action.yml` — publish: record the build watermark, atomically republish the snapshot on a protected branch, release the lock (`mode: release-lock` for the `if: always()` step). - `scripts/` — all logic, so it is testable standalone; the YAML is wiring. - `scripts/*selftest.sh` + `selftest.sh` — five suites, 63 assertions, every fix paired with a control that reproduces the bug. All green locally. Eviction merges emowheel's liveness pass (dead branches pruned unconditionally, not gated on disk pressure) with LRU-under-pressure, but inverts the order within the pressure pass: `target-*` before `snapshot-*`, because a snapshot is hardlinked to everything cloned from it, so evicting one frees almost no real bytes while costing every future PR its warm start. restore-mtimes.sh is ported from emowheel (the watermark variant, which closes the merge hazard zemyna's copy still has) with its provenance de-projectised. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sqh2vscfzisk83VuPVQX9L
This commit is contained in:
Claude Opus 5
Executable
+178
@@ -0,0 +1,178 @@
|
||||
#!/usr/bin/env bash
|
||||
# Regression test for the single assumption this whole caching scheme rests
|
||||
# on: that a build running inside a hardlink clone cannot mutate the directory
|
||||
# it was cloned from.
|
||||
#
|
||||
# That assumption is FALSE for a plain `cp -al`. Measured, and asserted below
|
||||
# as an explicit control: build in a raw `cp -al` clone and the source's
|
||||
# `.fingerprint/<unit>/dep-*` (under CARGO_UNSTABLE_CHECKSUM_FRESHNESS),
|
||||
# `build/<pkg>/output`, `build/<pkg>/out/**` and `deps/*.d` all change,
|
||||
# because Cargo and build scripts write those with a plain truncating write
|
||||
# rather than the write-then-rename Cargo uses for real artifacts.
|
||||
#
|
||||
# The consequence is not a slow build, it is a wrong one: a PR clone rewrites
|
||||
# the base's dep-info to describe the PR's sources while the base's cache
|
||||
# still holds the artifact built from the base's sources; once the PR merges,
|
||||
# the base's next run finds the checksums match its (now merged) sources,
|
||||
# reports `Fresh`, and links a binary built from the pre-merge code.
|
||||
#
|
||||
# cache-lib.sh's unshare_mutable_paths() is what closes that, and this test is
|
||||
# what proves it stays closed. The control matters as much as the fix: a
|
||||
# scenario that passes for both would prove nothing.
|
||||
#
|
||||
# Needs a working cargo on PATH. Everything happens under a mktemp -d scratch
|
||||
# tree. Run by hand: bash scripts/hardlink-clone-selftest.sh
|
||||
set -euo pipefail
|
||||
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
||||
. "$script_dir/cache-lib.sh"
|
||||
|
||||
command -v cargo >/dev/null || { echo "SKIP: no cargo on PATH"; exit 0; }
|
||||
|
||||
scratch=$(mktemp -d)
|
||||
trap 'rm -rf "$scratch"' EXIT
|
||||
pass_count=0
|
||||
|
||||
fail() { echo "ASSERTION FAILED: $*" >&2; exit 1; }
|
||||
ok() { pass_count=$((pass_count + 1)); echo "PASS: $*"; }
|
||||
|
||||
# Content hash of every file in a tree, keyed by relative path.
|
||||
snapshot_tree() { (cd "$1" && find . -type f -print0 | sort -z | xargs -0 -r sha1sum) 2>/dev/null; }
|
||||
|
||||
# `diff` exits 1 when the trees differ, which is the expected case here and
|
||||
# must not trip `pipefail` — the difference IS the result.
|
||||
mutated_paths() {
|
||||
{ diff <(printf '%s' "$1") <(printf '%s' "$2") || true; } 2>/dev/null \
|
||||
| awk '/^[<>]/ { print $3 }' | sort -u
|
||||
}
|
||||
|
||||
# A crate with a build script, because build-script OUT_DIR writes are one of
|
||||
# the two mutation families and are invisible without one.
|
||||
mkcrate() {
|
||||
local dir="$1"
|
||||
mkdir -p "$dir/src"
|
||||
cat > "$dir/Cargo.toml" <<'TOML'
|
||||
[package]
|
||||
name = "probe"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
[workspace]
|
||||
TOML
|
||||
cat > "$dir/build.rs" <<'RS'
|
||||
use std::{env, fs, path::PathBuf};
|
||||
fn main() {
|
||||
println!("cargo::rerun-if-changed=src/lib.rs");
|
||||
let out = PathBuf::from(env::var("OUT_DIR").unwrap());
|
||||
let src = fs::read_to_string("src/lib.rs").unwrap();
|
||||
fs::write(out.join("gen.txt"), format!("generated from {} bytes", src.len())).unwrap();
|
||||
}
|
||||
RS
|
||||
}
|
||||
|
||||
crate_dir="$scratch/probe"
|
||||
mkcrate "$crate_dir"
|
||||
cd "$crate_dir"
|
||||
|
||||
export CARGO_INCREMENTAL=0
|
||||
# Checksum freshness is where the worst failure lives (the dep-* file carries
|
||||
# per-source checksums and is rewritten in place). Only available on nightly;
|
||||
# without it the test still covers the build/ and *.d families.
|
||||
CHECKSUM_MODE="off"
|
||||
if cargo +nightly -V >/dev/null 2>&1; then
|
||||
export CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true
|
||||
CARGO_BIN=(cargo +nightly)
|
||||
CHECKSUM_MODE="on"
|
||||
else
|
||||
CARGO_BIN=(cargo)
|
||||
fi
|
||||
echo "=== checksum-freshness mode: ${CHECKSUM_MODE} ==="
|
||||
|
||||
CONTENT_A='pub fn f() -> u32 { 1 }'
|
||||
CONTENT_B='pub fn f() -> u32 { 22222 } pub fn g() -> u32 { 7 }'
|
||||
|
||||
build_base() {
|
||||
local dir="$1"
|
||||
printf '%s\n' "$CONTENT_A" > src/lib.rs
|
||||
CARGO_TARGET_DIR="$dir" "${CARGO_BIN[@]}" build -q
|
||||
}
|
||||
|
||||
echo
|
||||
echo "=== control: a raw \`cp -al\` clone DOES mutate its source ==="
|
||||
base_ctl="$scratch/base-ctl"; clone_ctl="$scratch/clone-ctl"
|
||||
build_base "$base_ctl"
|
||||
before=$(snapshot_tree "$base_ctl")
|
||||
cp -al "$base_ctl" "$clone_ctl"
|
||||
strip_cargo_locks "$clone_ctl" # locks alone are not the hazard under test
|
||||
printf '%s\n' "$CONTENT_B" > src/lib.rs
|
||||
CARGO_TARGET_DIR="$clone_ctl" "${CARGO_BIN[@]}" build -q
|
||||
after=$(snapshot_tree "$base_ctl")
|
||||
ctl_mutated=$(mutated_paths "$before" "$after")
|
||||
if [ -z "$ctl_mutated" ]; then
|
||||
fail "control produced no mutation — the test can no longer distinguish fixed from broken"
|
||||
fi
|
||||
ok "raw cp -al clone mutates the source ($(printf '%s\n' "$ctl_mutated" | wc -l) paths)"
|
||||
printf '%s\n' "$ctl_mutated" | sed 's/^/ /'
|
||||
|
||||
if [ "$CHECKSUM_MODE" = "on" ]; then
|
||||
if printf '%s' "$ctl_mutated" | grep -q '\.fingerprint/.*/dep-'; then
|
||||
ok "control confirms the checksum-freshness dep-info file is among the mutated set"
|
||||
else
|
||||
fail "expected .fingerprint/*/dep-* in the control's mutated set under checksum freshness"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "=== fix: hardlink_clone_into() leaves the source byte-identical ==="
|
||||
base_fix="$scratch/base-fix"; clone_fix="$scratch/clone-fix"
|
||||
build_base "$base_fix"
|
||||
before=$(snapshot_tree "$base_fix")
|
||||
hardlink_clone_into "$base_fix" "$clone_fix" "selftest" || fail "hardlink_clone_into reported the destination already existed"
|
||||
|
||||
# The clone's contract, asserted before anything builds in it: artifacts
|
||||
# share inodes (that is what makes the clone near-free), and every file Cargo
|
||||
# rewrites in place does not (that is what makes it sound). Checking after a
|
||||
# rebuild would prove nothing — the rebuild replaces those files anyway.
|
||||
shared=0; unshared=0
|
||||
while IFS= read -r f; do
|
||||
rel="${f#$base_fix/}"
|
||||
[ -e "$clone_fix/$rel" ] || continue
|
||||
if [ "$(stat -c '%i' "$f")" = "$(stat -c '%i' "$clone_fix/$rel")" ]; then
|
||||
case "$rel" in
|
||||
*/.fingerprint/*|*/build/*|*.d|.rustc_info.json)
|
||||
fail "mutable path still shares an inode with the source: $rel" ;;
|
||||
esac
|
||||
shared=$((shared + 1))
|
||||
else
|
||||
unshared=$((unshared + 1))
|
||||
fi
|
||||
done < <(find "$base_fix" -type f)
|
||||
[ "$shared" -gt 0 ] || fail "nothing is shared — the clone degenerated into a full copy"
|
||||
[ "$unshared" -gt 0 ] || fail "nothing was unshared — unshare_mutable_paths did not run"
|
||||
ok "fresh clone shares ${shared} artifact files and privately owns ${unshared} mutable ones"
|
||||
|
||||
printf '%s\n' "$CONTENT_B" > src/lib.rs
|
||||
CARGO_TARGET_DIR="$clone_fix" "${CARGO_BIN[@]}" build -q
|
||||
after=$(snapshot_tree "$base_fix")
|
||||
fix_mutated=$(mutated_paths "$before" "$after")
|
||||
if [ -n "$fix_mutated" ]; then
|
||||
echo " still mutated:" >&2
|
||||
printf '%s\n' "$fix_mutated" | sed 's/^/ /' >&2
|
||||
fail "a build in the clone mutated the source through a shared inode"
|
||||
fi
|
||||
ok "no file in the source changed after a full rebuild in the clone"
|
||||
|
||||
echo
|
||||
echo "=== the whole point: the source's next build is still correct ==="
|
||||
# The source's cache holds artifacts built from CONTENT_A. Advance the source
|
||||
# to CONTENT_B (as a merge would) and rebuild in it. If the clone had
|
||||
# corrupted its dep-info, Cargo would report Fresh and keep the stale rlib.
|
||||
printf '%s\n' "$CONTENT_B" > src/lib.rs
|
||||
touch -d '@1000000000' src/lib.rs
|
||||
log="$scratch/rebuild.log"
|
||||
CARGO_TARGET_DIR="$base_fix" "${CARGO_BIN[@]}" build -v > "$log" 2>&1 || { cat "$log"; fail "rebuild in the source failed"; }
|
||||
if grep -qE '^\s+Fresh probe' "$log"; then
|
||||
fail "source declared its own crate Fresh against sources it has never built — stale-artifact reuse"
|
||||
fi
|
||||
ok "source correctly rebuilt its crate after advancing to the clone's content"
|
||||
|
||||
echo
|
||||
echo "hardlink-clone-selftest: ${pass_count} assertions passed"
|
||||
Reference in New Issue
Block a user