From 3f97d3d1e73a4cf7b5f94f368545551894e8db3c Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 26 Aug 2026 12:46:03 -0500 Subject: [PATCH] fix(selftest): make the two compiler-backed suites survive a CI runner MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The new gate's first run went red on both suites that drive a real Cargo. Neither failure was a defect in what they test; both were assumptions about the machine, which had only ever been a dev box with a nightly installed and no colour forcing. Fixed here rather than waived — turning a gate on is what obliges fixing what it finds. COLOUR. Every assertion in restore-mtimes-selftest.sh, and one in hardlink-clone-selftest.sh, reads cargo's own words out of a build log (`Compiling libdep`, `Fresh probe`). gitdan-ci's runner image forces colour, so cargo wrote `Compiling\e[0m libdep` into the log and `grep -q "Compiling libdep"` stopped matching. The suite then reported the opposite of what happened — the failure printed the log, and the log plainly said `Compiling libdep`. Both suites now pin CARGO_TERM_COLOR=never, which is the format their assertions are written against. Red-proven: with the export removed and CARGO_TERM_COLOR=always, restore-mtimes-selftest reproduces the CI failure verbatim ("expected to find: Compiling libdep"); with it, 14/14 pass under the same forced colour. THE NIGHTLY PROBE ASKED THE WRONG QUESTION. `cargo +nightly -V` answers "did a cargo proxy called with +nightly exit 0", which is not "will this build have checksum freshness": `-V` short-circuits before `-Z` is validated at all. So the probe said "on" on a runner where the flag was not in effect, and the suite asserted the checksum-freshness mutation family against a build that never had it — failing in the CONTROL, where a failure reads as "the hazard is gone" rather than "the toolchain is wrong". It now probes the capability: `cargo +nightly -Z checksum-freshness locate-project`, the narrowest command that actually parses the flag. It rejects the stable channel and an unknown flag name alike, needs no network and builds nothing. Red-proven: the old channel probe, pointed at a cargo without the flag, reproduces the CI failure exactly. AND THE OFF PATH DID NOT WORK EITHER. The suite's header claimed that without checksum freshness "the test still covers the build/ and *.d families". It did not: the final scenario backdates the source to 2001 and asserts the rebuild is not Fresh, which is checksum-freshness-only reasoning. Under Cargo's ordinary mtime freshness a 2001 source IS older than the artifact and Fresh is the correct answer, so the scenario asserted a bug. It is now gated on the mode and skipped loudly, like the control's dep-* assertion already was: 5 assertions with a nightly, 3 without. CI installs a nightly as well as stable (nightly first, so stable stays the default) — that hazard is the one this whole scheme exists to close, and a CI that skips it is checking the cheap half. --- .gitea/workflows/ci.yaml | 12 ++++++ README.md | 7 +++- scripts/hardlink-clone-selftest.sh | 60 ++++++++++++++++++++++++------ scripts/restore-mtimes-selftest.sh | 9 +++++ 4 files changed, 74 insertions(+), 14 deletions(-) diff --git a/.gitea/workflows/ci.yaml b/.gitea/workflows/ci.yaml index 89331bd..4bb9ea8 100644 --- a/.gitea/workflows/ci.yaml +++ b/.gitea/workflows/ci.yaml @@ -66,6 +66,18 @@ jobs: # # The scratch workspaces use path dependencies only, so nothing here # reaches crates.io. + # Nightly first, stable second, so stable ends up the default and + # nightly is only reachable through an explicit `+nightly`. + # + # Nightly is not optional here. `hardlink-clone-selftest.sh` gates its + # two strongest assertions on `-Z checksum-freshness` — the mode where + # Cargo's dep-info file carries per-source checksums and is rewritten in + # place, which is the mutation that turns a hardlink clone into SILENT + # stale-artifact reuse rather than a slow build. Without a nightly the + # suite still runs, and skips exactly the hazard this whole scheme exists + # to close. + - name: Install Rust nightly + uses: dtolnay/rust-toolchain@nightly - name: Install Rust toolchain uses: dtolnay/rust-toolchain@stable diff --git a/README.md b/README.md index 671f9f5..3f4d847 100644 --- a/README.md +++ b/README.md @@ -544,7 +544,10 @@ bash scripts/selftest.sh --fast # fixture-only suites, no compiler Both run in CI — `.gitea/workflows/ci.yaml`, one job, on pushes to `main` and on PRs that were non-draft when the run was created. It installs shellcheck -and a stable Rust toolchain and references no credentials; the scratch +and both a stable and a nightly Rust toolchain (nightly for +`-Z checksum-freshness`, without which `hardlink-clone-selftest.sh` skips the +two assertions that cover the silent-stale-reuse hazard) and references no +credentials; the scratch workspaces the compiler-backed suites build use path dependencies only, so nothing reaches crates.io. It runs the full suite rather than `--fast`, because the two compiler-backed suites are the ones that check this scheme @@ -559,7 +562,7 @@ change here reaches all of them at once. That is what the gate is for. | suite | covers | |---|---| | `cache-root-selftest.sh` | that a lineage nests one level and nothing else moves: no lineage resolves byte-for-byte to the cache root, two lineages on one cache key get disjoint target dirs, seed/publish/prune all stay inside their own lineage, a PR layers over its own lineage's base snapshot — **and one rejection per lineage name a reader elsewhere would stop seeing**, plus the publish-side mismatch guard | -| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler. | +| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly for two of its five assertions**: the checksum-freshness dep-info file is the mutation that turns a hardlink clone into silent stale-artifact reuse, and it only exists under `-Z checksum-freshness`. Without one those two are skipped, loudly. | | `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published, and the publisher's log showing it waited on the consumer's own reader-lock marker before reclaiming a rotated snapshot | | `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone | | `prune-cache-selftest.sh` | liveness, protection, locking, eviction order, self-clear, **and that a cache a job claims *inside* the check-to-unlink window survives it** — against a real scratch `origin` | diff --git a/scripts/hardlink-clone-selftest.sh b/scripts/hardlink-clone-selftest.sh index 7364a49..818032f 100755 --- a/scripts/hardlink-clone-selftest.sh +++ b/scripts/hardlink-clone-selftest.sh @@ -73,11 +73,33 @@ mkcrate "$crate_dir" cd "$crate_dir" export CARGO_INCREMENTAL=0 + +# Every assertion below reads cargo's own words out of a build log +# (`Compiling libdep`, `Fresh probe`). A CI image that forces colour splices an +# ANSI reset between the status word and the crate name, at which point every +# one of those greps silently stops matching and the suite reports the +# opposite of what happened — observed on gitdan-ci's runner image, where +# scenario 2 failed while the log it printed plainly showed `Compiling libdep`. +# Pin the format the assertions are written against. +export CARGO_TERM_COLOR=never # Checksum freshness is where the worst failure lives (the dep-* file carries # per-source checksums and is rewritten in place). Only available on nightly; # without it the test still covers the build/ and *.d families. CHECKSUM_MODE="off" -if cargo +nightly -V >/dev/null 2>&1; then +# Probe the CAPABILITY, not the channel. `cargo +nightly -V` answers "did a +# cargo proxy called with +nightly exit 0", which is a different question from +# "will this build have checksum freshness" — `-V` short-circuits before `-Z` +# is validated at all, so that probe says yes on any cargo that resolves the +# name, including one whose nightly has since moved the flag. The scenario +# below then asserts the checksum-freshness mutation family against a build +# that never had it, and fails in the CONTROL, where a failure reads as "the +# hazard is gone" rather than "the toolchain is wrong". Observed the first +# time this suite ran on gitdan-ci. +# +# `-Z locate-project` is the narrowest command that actually parses the +# flag: it rejects the stable channel and an unknown flag name alike, needs no +# network, and builds nothing. +if cargo +nightly -Z checksum-freshness locate-project >/dev/null 2>&1; then export CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true CARGO_BIN=(cargo +nightly) CHECKSUM_MODE="on" @@ -161,18 +183,32 @@ fi ok "no file in the source changed after a full rebuild in the clone" echo -echo "=== the whole point: the source's next build is still correct ===" -# The source's cache holds artifacts built from CONTENT_A. Advance the source -# to CONTENT_B (as a merge would) and rebuild in it. If the clone had -# corrupted its dep-info, Cargo would report Fresh and keep the stale rlib. -printf '%s\n' "$CONTENT_B" > src/lib.rs -touch -d '@1000000000' src/lib.rs -log="$scratch/rebuild.log" -CARGO_TARGET_DIR="$base_fix" "${CARGO_BIN[@]}" build -v > "$log" 2>&1 || { cat "$log"; fail "rebuild in the source failed"; } -if grep -qE '^\s+Fresh probe' "$log"; then - fail "source declared its own crate Fresh against sources it has never built — stale-artifact reuse" +if [ "$CHECKSUM_MODE" = "on" ]; then + echo "=== the whole point: the source's next build is still correct ===" + # The source's cache holds artifacts built from CONTENT_A. Advance the + # source to CONTENT_B (as a merge would) and rebuild in it. If the clone had + # corrupted its dep-info, Cargo would report Fresh and keep the stale rlib. + # + # CHECKSUM-FRESHNESS ONLY, and the backdated mtime is why. Under checksum + # freshness the dep-info file's per-source checksums decide, so a 2001 + # timestamp on changed content must still rebuild — the assertion below. + # Under Cargo's ordinary MTIME freshness the same timestamp means the source + # is older than the artifact, and reporting Fresh is the correct answer; + # asserting otherwise asserts a bug. This scenario was written against a + # machine with a nightly installed and, run without one, failed on that + # correct answer. + printf '%s\n' "$CONTENT_B" > src/lib.rs + touch -d '@1000000000' src/lib.rs + log="$scratch/rebuild.log" + CARGO_TARGET_DIR="$base_fix" "${CARGO_BIN[@]}" build -v > "$log" 2>&1 || { cat "$log"; fail "rebuild in the source failed"; } + if grep -qE '^\s+Fresh probe' "$log"; then + fail "source declared its own crate Fresh against sources it has never built — stale-artifact reuse" + fi + ok "source correctly rebuilt its crate after advancing to the clone's content" +else + echo "=== skipped: the source's-next-build scenario needs checksum freshness ===" + echo " (a nightly cargo accepting -Z checksum-freshness; see the probe above)" fi -ok "source correctly rebuilt its crate after advancing to the clone's content" echo echo "hardlink-clone-selftest: ${pass_count} assertions passed" diff --git a/scripts/restore-mtimes-selftest.sh b/scripts/restore-mtimes-selftest.sh index 66db0ea..020c9c1 100755 --- a/scripts/restore-mtimes-selftest.sh +++ b/scripts/restore-mtimes-selftest.sh @@ -75,6 +75,15 @@ # Asserts BOTH jobs correctly recompile the dependency and succeed. set -euo pipefail +# Every assertion below reads cargo's own words out of a build log +# (`Compiling libdep`, `Fresh probe`). A CI image that forces colour splices an +# ANSI reset between the status word and the crate name, at which point every +# one of those greps silently stops matching and the suite reports the +# opposite of what happened — observed on gitdan-ci's runner image, where +# scenario 2 failed while the log it printed plainly showed `Compiling libdep`. +# Pin the format the assertions are written against. +export CARGO_TERM_COLOR=never + script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) restore_mtimes="$script_dir/restore-mtimes.sh"