fix(selftest): make the two compiler-backed suites survive a CI runner
CI / shellcheck + selftests (pull_request) Failing after 1m36s

The new gate's first run went red on both suites that drive a real Cargo.
Neither failure was a defect in what they test; both were assumptions about
the machine, which had only ever been a dev box with a nightly installed and
no colour forcing. Fixed here rather than waived — turning a gate on is what
obliges fixing what it finds.

COLOUR. Every assertion in restore-mtimes-selftest.sh, and one in
hardlink-clone-selftest.sh, reads cargo's own words out of a build log
(`Compiling libdep`, `Fresh probe`). gitdan-ci's runner image forces colour, so
cargo wrote `Compiling\e[0m libdep` into the log and `grep -q "Compiling
libdep"` stopped matching. The suite then reported the opposite of what
happened — the failure printed the log, and the log plainly said `Compiling
libdep`. Both suites now pin CARGO_TERM_COLOR=never, which is the format their
assertions are written against.

Red-proven: with the export removed and CARGO_TERM_COLOR=always,
restore-mtimes-selftest reproduces the CI failure verbatim ("expected to find:
Compiling libdep"); with it, 14/14 pass under the same forced colour.

THE NIGHTLY PROBE ASKED THE WRONG QUESTION. `cargo +nightly -V` answers "did a
cargo proxy called with +nightly exit 0", which is not "will this build have
checksum freshness": `-V` short-circuits before `-Z` is validated at all. So
the probe said "on" on a runner where the flag was not in effect, and the
suite asserted the checksum-freshness mutation family against a build that
never had it — failing in the CONTROL, where a failure reads as "the hazard is
gone" rather than "the toolchain is wrong". It now probes the capability:
`cargo +nightly -Z checksum-freshness locate-project`, the narrowest command
that actually parses the flag. It rejects the stable channel and an unknown
flag name alike, needs no network and builds nothing.

Red-proven: the old channel probe, pointed at a cargo without the flag,
reproduces the CI failure exactly.

AND THE OFF PATH DID NOT WORK EITHER. The suite's header claimed that without
checksum freshness "the test still covers the build/ and *.d families". It did
not: the final scenario backdates the source to 2001 and asserts the rebuild
is not Fresh, which is checksum-freshness-only reasoning. Under Cargo's
ordinary mtime freshness a 2001 source IS older than the artifact and Fresh is
the correct answer, so the scenario asserted a bug. It is now gated on the
mode and skipped loudly, like the control's dep-* assertion already was: 5
assertions with a nightly, 3 without.

CI installs a nightly as well as stable (nightly first, so stable stays the
default) — that hazard is the one this whole scheme exists to close, and a CI
that skips it is checking the cheap half.
This commit is contained in:
2026-08-26 12:46:03 -05:00
parent fb7a788c90
commit 3f97d3d1e7
4 changed files with 74 additions and 14 deletions
+48 -12
View File
@@ -73,11 +73,33 @@ mkcrate "$crate_dir"
cd "$crate_dir"
export CARGO_INCREMENTAL=0
# Every assertion below reads cargo's own words out of a build log
# (`Compiling libdep`, `Fresh probe`). A CI image that forces colour splices an
# ANSI reset between the status word and the crate name, at which point every
# one of those greps silently stops matching and the suite reports the
# opposite of what happened — observed on gitdan-ci's runner image, where
# scenario 2 failed while the log it printed plainly showed `Compiling libdep`.
# Pin the format the assertions are written against.
export CARGO_TERM_COLOR=never
# Checksum freshness is where the worst failure lives (the dep-* file carries
# per-source checksums and is rewritten in place). Only available on nightly;
# without it the test still covers the build/ and *.d families.
CHECKSUM_MODE="off"
if cargo +nightly -V >/dev/null 2>&1; then
# Probe the CAPABILITY, not the channel. `cargo +nightly -V` answers "did a
# cargo proxy called with +nightly exit 0", which is a different question from
# "will this build have checksum freshness" — `-V` short-circuits before `-Z`
# is validated at all, so that probe says yes on any cargo that resolves the
# name, including one whose nightly has since moved the flag. The scenario
# below then asserts the checksum-freshness mutation family against a build
# that never had it, and fails in the CONTROL, where a failure reads as "the
# hazard is gone" rather than "the toolchain is wrong". Observed the first
# time this suite ran on gitdan-ci.
#
# `-Z <flag> locate-project` is the narrowest command that actually parses the
# flag: it rejects the stable channel and an unknown flag name alike, needs no
# network, and builds nothing.
if cargo +nightly -Z checksum-freshness locate-project >/dev/null 2>&1; then
export CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true
CARGO_BIN=(cargo +nightly)
CHECKSUM_MODE="on"
@@ -161,18 +183,32 @@ fi
ok "no file in the source changed after a full rebuild in the clone"
echo
echo "=== the whole point: the source's next build is still correct ==="
# The source's cache holds artifacts built from CONTENT_A. Advance the source
# to CONTENT_B (as a merge would) and rebuild in it. If the clone had
# corrupted its dep-info, Cargo would report Fresh and keep the stale rlib.
printf '%s\n' "$CONTENT_B" > src/lib.rs
touch -d '@1000000000' src/lib.rs
log="$scratch/rebuild.log"
CARGO_TARGET_DIR="$base_fix" "${CARGO_BIN[@]}" build -v > "$log" 2>&1 || { cat "$log"; fail "rebuild in the source failed"; }
if grep -qE '^\s+Fresh probe' "$log"; then
fail "source declared its own crate Fresh against sources it has never built — stale-artifact reuse"
if [ "$CHECKSUM_MODE" = "on" ]; then
echo "=== the whole point: the source's next build is still correct ==="
# The source's cache holds artifacts built from CONTENT_A. Advance the
# source to CONTENT_B (as a merge would) and rebuild in it. If the clone had
# corrupted its dep-info, Cargo would report Fresh and keep the stale rlib.
#
# CHECKSUM-FRESHNESS ONLY, and the backdated mtime is why. Under checksum
# freshness the dep-info file's per-source checksums decide, so a 2001
# timestamp on changed content must still rebuild — the assertion below.
# Under Cargo's ordinary MTIME freshness the same timestamp means the source
# is older than the artifact, and reporting Fresh is the correct answer;
# asserting otherwise asserts a bug. This scenario was written against a
# machine with a nightly installed and, run without one, failed on that
# correct answer.
printf '%s\n' "$CONTENT_B" > src/lib.rs
touch -d '@1000000000' src/lib.rs
log="$scratch/rebuild.log"
CARGO_TARGET_DIR="$base_fix" "${CARGO_BIN[@]}" build -v > "$log" 2>&1 || { cat "$log"; fail "rebuild in the source failed"; }
if grep -qE '^\s+Fresh probe' "$log"; then
fail "source declared its own crate Fresh against sources it has never built — stale-artifact reuse"
fi
ok "source correctly rebuilt its crate after advancing to the clone's content"
else
echo "=== skipped: the source's-next-build scenario needs checksum freshness ==="
echo " (a nightly cargo accepting -Z checksum-freshness; see the probe above)"
fi
ok "source correctly rebuilt its crate after advancing to the clone's content"
echo
echo "hardlink-clone-selftest: ${pass_count} assertions passed"
+9
View File
@@ -75,6 +75,15 @@
# Asserts BOTH jobs correctly recompile the dependency and succeed.
set -euo pipefail
# Every assertion below reads cargo's own words out of a build log
# (`Compiling libdep`, `Fresh probe`). A CI image that forces colour splices an
# ANSI reset between the status word and the crate name, at which point every
# one of those greps silently stops matching and the suite reports the
# opposite of what happened — observed on gitdan-ci's runner image, where
# scenario 2 failed while the log it printed plainly showed `Compiling libdep`.
# Pin the format the assertions are written against.
export CARGO_TERM_COLOR=never
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
restore_mtimes="$script_dir/restore-mtimes.sh"