fix(cache): separate build directories for same-ref jobs, and a CI gate #13

Merged
daniel merged 9 commits from fix/lock-contention into main 2026-08-26 21:38:35 +00:00
2 changed files with 51 additions and 24 deletions
Showing only changes of commit 6a80423bd0 - Show all commits
+1 -1
View File
@@ -562,7 +562,7 @@ change here reaches all of them at once. That is what the gate is for.
| suite | covers |
|---|---|
| `cache-root-selftest.sh` | that a lineage nests one level and nothing else moves: no lineage resolves byte-for-byte to the cache root, two lineages on one cache key get disjoint target dirs, seed/publish/prune all stay inside their own lineage, a PR layers over its own lineage's base snapshot — **and one rejection per lineage name a reader elsewhere would stop seeing**, plus the publish-side mismatch guard |
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly for its last scenario**: the source's-next-build check reasons about content rather than mtime, so it only means anything under `-Z checksum-freshness`. Without one it is skipped, loudly. The control also reports which mutation families the running Cargo exhibits — a note, not an assertion, since that set moves upstream. |
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly that actually resolves freshness by content for its last scenario**: the source's-next-build check reasons about content rather than mtime, so under mtime freshness it would assert a bug. Whether the toolchain does is settled by experiment on a throwaway crate, not by asking it — 1.100.0-nightly accepts `-Z checksum-freshness` and rebuilds on mtime anyway. Otherwise the scenario is skipped, loudly. The control also reports which mutation families the running Cargo exhibits — a note, not an assertion, since that set moves upstream. |
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published, and the publisher's log showing it waited on the consumer's own reader-lock marker before reclaiming a rotated snapshot |
| `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone |
| `prune-cache-selftest.sh` | liveness, protection, locking, eviction order, self-clear, **and that a cache a job claims *inside* the check-to-unlink window survives it** — against a real scratch `origin` |
+44 -17
View File
@@ -85,32 +85,59 @@ export CARGO_TERM_COLOR=never
# Checksum freshness is where the worst failure lives (the dep-* file carries
# per-source checksums and is rewritten in place). Only available on nightly;
# without it the test still covers the build/ and *.d families.
CHECKSUM_MODE="off"
# Probe the CAPABILITY, not the channel. `cargo +nightly -V` answers "did a
# cargo proxy called with +nightly exit 0", which is a different question from
# "will this build have checksum freshness" — `-V` short-circuits before `-Z`
# is validated at all, so that probe says yes on any cargo that resolves the
# name, including one whose nightly has since moved the flag. The scenario
# below then asserts the checksum-freshness mutation family against a build
# that never had it, and fails in the CONTROL, where a failure reads as "the
# hazard is gone" rather than "the toolchain is wrong". Observed the first
# time this suite ran on gitdan-ci.
CONTENT_A='pub fn f() -> u32 { 1 }'
CONTENT_B='pub fn f() -> u32 { 22222 } pub fn g() -> u32 { 7 }'
# Probe the BEHAVIOUR, not the channel and not the flag. Two weaker probes
# were tried against gitdan-ci's runner and each let the suite assert a
# property the toolchain did not have:
#
# `-Z <flag> locate-project` is the narrowest command that actually parses the
# flag: it rejects the stable channel and an unknown flag name alike, needs no
# network, and builds nothing.
# `cargo +nightly -V` — answers "did a proxy called with
# +nightly exit 0". `-V`
# short-circuits before `-Z` is
# even parsed.
# `cargo +nightly -Z checksum-freshness — answers "is this flag still
# locate-project` accepted". 1.100.0-nightly
# (2026-08-25) accepts it and does
# not resolve freshness by content
# anyway.
#
# The scenario at the end of this file depends on one thing and it is neither
# of those: that changed content with an OLDER mtime rebuilds. Under mtime
# freshness the correct answer is Fresh, so under mtime freshness that
# scenario asserts a bug. So the probe simply performs that experiment, on its
# own crate and its own target dir, with no clone anywhere near it — which is
# also what makes it a control rather than a restatement of the scenario: the
# probe establishes that the toolchain rebuilds on content, the scenario
# establishes that a hardlink clone did not take that away.
CHECKSUM_MODE="off"
CARGO_BIN=(cargo)
checksum_freshness_active() {
local d="$scratch/freshness-probe" t="$scratch/freshness-probe-target"
mkcrate "$d"
(
cd "$d" || exit 1
printf '%s\n' "$CONTENT_A" > src/lib.rs
CARGO_TARGET_DIR="$t" cargo +nightly build -q > /dev/null 2>&1 || exit 1
printf '%s\n' "$CONTENT_B" > src/lib.rs
touch -d '@1000000000' src/lib.rs
CARGO_TARGET_DIR="$t" cargo +nightly build -v > "$scratch/freshness-probe.log" 2>&1 || exit 1
! grep -qE '^\s+Fresh probe' "$scratch/freshness-probe.log"
)
}
if cargo +nightly -Z checksum-freshness locate-project > /dev/null 2>&1; then
export CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true
if checksum_freshness_active; then
CARGO_BIN=(cargo +nightly)
CHECKSUM_MODE="on"
else
CARGO_BIN=(cargo)
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS
echo "note: this nightly accepts -Z checksum-freshness but still resolves freshness by mtime"
fi
fi
cd "$crate_dir"
echo "=== checksum-freshness mode: ${CHECKSUM_MODE} ==="
CONTENT_A='pub fn f() -> u32 { 1 }'
CONTENT_B='pub fn f() -> u32 { 22222 } pub fn g() -> u32 { 7 }'
build_base() {
local dir="$1"
printf '%s\n' "$CONTENT_A" > src/lib.rs