fix(cache): separate build directories for same-ref jobs, and a CI gate #13

Merged
daniel merged 9 commits from fix/lock-contention into main 2026-08-26 21:38:35 +00:00
3 changed files with 30 additions and 5 deletions
Showing only changes of commit 5e8e773f78 - Show all commits
+2 -2
View File
@@ -546,7 +546,7 @@ Both run in CI — `.gitea/workflows/ci.yaml`, one job, on pushes to `main` and
on PRs that were non-draft when the run was created. It installs shellcheck
and both a stable and a nightly Rust toolchain (nightly for
`-Z checksum-freshness`, without which `hardlink-clone-selftest.sh` skips the
two assertions that cover the silent-stale-reuse hazard) and references no
scenario that covers the silent-stale-reuse hazard) and references no
credentials; the scratch
workspaces the compiler-backed suites build use path dependencies only, so
nothing reaches crates.io. It runs the full suite rather than `--fast`,
@@ -562,7 +562,7 @@ change here reaches all of them at once. That is what the gate is for.
| suite | covers |
|---|---|
| `cache-root-selftest.sh` | that a lineage nests one level and nothing else moves: no lineage resolves byte-for-byte to the cache root, two lineages on one cache key get disjoint target dirs, seed/publish/prune all stay inside their own lineage, a PR layers over its own lineage's base snapshot — **and one rejection per lineage name a reader elsewhere would stop seeing**, plus the publish-side mismatch guard |
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly for two of its five assertions**: the checksum-freshness dep-info file is the mutation that turns a hardlink clone into silent stale-artifact reuse, and it only exists under `-Z checksum-freshness`. Without one those two are skipped, loudly. |
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly for its last scenario**: the source's-next-build check reasons about content rather than mtime, so it only means anything under `-Z checksum-freshness`. Without one it is skipped, loudly. The control also reports which mutation families the running Cargo exhibits — a note, not an assertion, since that set moves upstream. |
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published, and the publisher's log showing it waited on the consumer's own reader-lock marker before reclaiming a rotated snapshot |
| `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone |
| `prune-cache-selftest.sh` | liveness, protection, locking, eviction order, self-clear, **and that a cache a job claims *inside* the check-to-unlink window survives it** — against a real scratch `origin` |
+11 -1
View File
@@ -332,7 +332,17 @@ _unshare_files() {
# <profile>/.fingerprint/<unit>/dep-<target> (only under
# CARGO_UNSTABLE_CHECKSUM_FRESHNESS,
# where this file carries the
# per-source blake3 checksums)
# per-source blake3 checksums.
# NOT reproduced on
# 1.100.0-nightly (2026-08-25),
# measured by this repo's own CI
# — upstream appears to have
# stopped writing it in place.
# Kept in the unshared set
# anyway: it costs 22 MB of a
# 6.9 GB tree, and the failure
# it guards is a wrong answer,
# not a slow one.)
# <profile>/build/<pkg>/output, root-output (Cargo build-script metadata)
# <profile>/build/<pkg>/out/** (whatever the build script
# writes into OUT_DIR — build
+17 -2
View File
@@ -134,11 +134,26 @@ fi
ok "raw cp -al clone mutates the source ($(printf '%s\n' "$ctl_mutated" | wc -l) paths)"
printf '%s\n' "$ctl_mutated" | sed 's/^/ /'
# Reported, not asserted, and the distinction is the point. The control's job
# is to prove the hazard exists at all, which the non-empty set above already
# does; this line records WHICH families a given Cargo exhibits.
#
# `.fingerprint/*/dep-*` is the worst of them — it carries the per-source
# checksums, so mutating it through a shared inode turns a hardlink clone into
# silent stale-artifact reuse rather than a slow build. It was measured on
# cargo 1.9x nightly (see unshare_mutable_paths in cache-lib.sh) and is NOT
# reproduced on 1.100.0-nightly (2026-08-25), where the control mutates only
# the build/ and *.d families. Failing on its absence would mean this suite
# goes red whenever upstream stops doing something we never wanted it to do —
# and it would go red in the CONTROL, where a failure reads as "the hazard is
# gone" rather than "upstream changed". Nothing is lost by reporting it: the
# fix scenario below asserts the source is byte-identical after a full rebuild
# in the clone, which covers every family this Cargo has, named or not.
if [ "$CHECKSUM_MODE" = "on" ]; then
if printf '%s' "$ctl_mutated" | grep -q '\.fingerprint/.*/dep-'; then
ok "control confirms the checksum-freshness dep-info file is among the mutated set"
echo " note: this cargo DOES rewrite .fingerprint/*/dep-* in place under checksum freshness"
else
fail "expected .fingerprint/*/dep-* in the control's mutated set under checksum freshness"
echo " note: this cargo does NOT rewrite .fingerprint/*/dep-* in place; only the build/ and *.d families appear above"
fi
fi