test(hardlink): move the probe's INACTIVE answer off bash's error codes
CI / shellcheck + selftests (pull_request) Successful in 1m21s
CI / shellcheck + selftests (pull_request) Successful in 1m21s
Review finding on #13, and the last change to this function. A typo'd variable name on a line that HAS its `|| exit 2` guard — `$CONTNET_B` for `$CONTENT_B` — fails in EXPANSION, before the command runs, so neither the guard nor the ERR trap ever sees it. Under `set -u` that exits 1, which was the code meaning "measured, and the answer is mtime". Not a live defect: the probe references only set variables today, and the 1 gitdan-ci reports is a genuine measurement. The defect is that the answer codes and the error codes overlapped at all. Three rounds on this function each found a narrower way for a shell-generated 1 to be read as a measurement — an unguarded command, then a guarded line whose guard could not fire — and each was closed by narrowing the failure surface, which is a game with no last move. INACTIVE is now 3, and anything that is not 0 or 3 is NOT MEASURED. Bash generates 1, 2, 126, 127 and 128+n for its own errors and never 3, so "not an answer" is decided by a property of the shell rather than by an enumeration of the ways a step can go wrong. A step added later without a guard, or with a guard that cannot fire, lands on NOT MEASURED by construction. The guards and the trap stay, with their job restated: reporting rather than correctness. They put a failed step on 2 with both probe logs printed instead of on some incidental status — nicer to debug, and the same destination either way. The `set +e` bracket at the call site keeps its own note, since a `set -e` that cannot fire is the kind of thing a reader assumes works. The not-measured message now quotes the actual exit code, so the two ways to reach it are distinguishable in a log without reading this file. Red-proven with the reviewer's own mutation, A/B: with INACTIVE at 1 the typo reports `mode: off — resolves freshness by mtime`, a false measurement; at 3 it reports `mode: unmeasured — the probe exited 1`. All four paths re-verified — unmodified 4 assertions; env stripped inside the probe, measured INACTIVE with the mtime reason; a broken probe build, unmeasured via a guard; an unguarded `false`, unmeasured via the trap.
This commit is contained in:
@@ -562,7 +562,7 @@ change here reaches all of them at once. That is what the gate is for.
|
||||
| suite | covers |
|
||||
|---|---|
|
||||
| `cache-root-selftest.sh` | that a lineage nests one level and nothing else moves: no lineage resolves byte-for-byte to the cache root, two lineages on one cache key get disjoint target dirs, seed/publish/prune all stay inside their own lineage, a PR layers over its own lineage's base snapshot — **and one rejection per lineage name a reader elsewhere would stop seeing**, plus the publish-side mismatch guard |
|
||||
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly that actually resolves freshness by content for its last scenario**: the source's-next-build check reasons about content rather than mtime, so under mtime freshness it would assert a bug. Whether the toolchain does is settled by experiment on a throwaway crate, not by asking it — 1.100.0-nightly accepts `-Z checksum-freshness` and rebuilds on mtime anyway. The experiment reports **three** outcomes, not two: active, measured-inactive, and *not measured* when a probe build failed. The scenario is skipped for the last two alike, but a failure to measure is never reported as a measurement. The control also reports which mutation families the running Cargo exhibits — a note, not an assertion, since that set moves upstream. |
|
||||
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly that actually resolves freshness by content for its last scenario**: the source's-next-build check reasons about content rather than mtime, so under mtime freshness it would assert a bug. Whether the toolchain does is settled by experiment on a throwaway crate, not by asking it — 1.100.0-nightly accepts `-Z checksum-freshness` and rebuilds on mtime anyway. The experiment reports **three** outcomes, not two: active, measured-inactive, and *not measured*. Its answer codes are `0` and `3`, deliberately clear of every status bash generates for its own errors — so nothing that goes wrong inside the probe, including an expansion failure no guard can catch, can be read as an answer. The scenario is skipped for the last two alike, but a failure to measure is never reported as a measurement. The control also reports which mutation families the running Cargo exhibits — a note, not an assertion, since that set moves upstream. |
|
||||
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published, and the publisher's log showing it waited on the consumer's own reader-lock marker before reclaiming a rotated snapshot |
|
||||
| `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone |
|
||||
| `prune-cache-selftest.sh` | liveness, protection, locking, eviction order, self-clear, **and that a cache a job claims *inside* the check-to-unlink window survives it** — against a real scratch `origin` |
|
||||
|
||||
@@ -121,43 +121,57 @@ CONTENT_B='pub fn f() -> u32 { 22222 } pub fn g() -> u32 { 7 }'
|
||||
#
|
||||
# 0 content freshness measured ACTIVE — both builds ran, the backdated
|
||||
# rebuild recompiled
|
||||
# 1 measured INACTIVE — both builds ran, the backdated
|
||||
# 3 measured INACTIVE — both builds ran, the backdated
|
||||
# rebuild reported Fresh
|
||||
# 2 NOT MEASURED — a probe step failed; nothing was
|
||||
# learned about the toolchain
|
||||
# anything else NOT MEASURED — nothing was learned about the
|
||||
# toolchain
|
||||
#
|
||||
# Two mechanisms, and BOTH are needed. Every step that could fail for a reason
|
||||
# other than the experiment's own outcome exits 2 explicitly; the subshell also
|
||||
# arms `set -e` with an ERR trap that maps any unguarded failure onto 2, so a
|
||||
# step added later without a guard lands on "not measured" rather than on an
|
||||
# answer.
|
||||
# THE ANSWER CODES ARE 0 AND 3, AND THE GAP IS THE MECHANISM. Bash produces 1
|
||||
# for an ordinary command failure, 2 for a usage error, 126/127 for a command
|
||||
# it could not run, 128+n for a signal, and — this is the one that matters —
|
||||
# 1 for an unbound-variable or other EXPANSION failure, which happens before
|
||||
# the command runs and is therefore invisible to a `||` guard and to an ERR
|
||||
# trap alike. It never produces 3. So "not an answer code" is decided by a
|
||||
# property of the shell rather than by an enumeration of the ways a step can
|
||||
# go wrong, and a step added later without a guard, or with a guard that
|
||||
# cannot fire, lands on NOT MEASURED by construction.
|
||||
#
|
||||
# The `set +e` around the call site is what makes the second mechanism work,
|
||||
# and it is not decoration. A command on the left of `||` — or in an `if`
|
||||
# condition — runs with errexit suppressed, and that suppression propagates
|
||||
# into a subshell and is NOT undone by a `set -e` inside it (verified on bash
|
||||
# 5.3: an unguarded `false` there falls through to `exit 0` and reports
|
||||
# ACTIVE). Calling with errexit disarmed at the site is the only form that
|
||||
# lets the subshell re-arm it. The ERR trap is then required on top, because a
|
||||
# bare `set -e` abort exits with the FAILING COMMAND's status — `false` gives
|
||||
# 1, which is precisely the value that means "measured, and the answer is
|
||||
# mtime". Belt and braces here buys a wrong answer; belt, braces and a trap
|
||||
# buys "not measured".
|
||||
# That is the whole reason INACTIVE is not 1. It was, and three review rounds
|
||||
# on this function each found a narrower way for a shell-generated 1 to be read
|
||||
# as a measurement — an unguarded command, then a typo'd variable name on a
|
||||
# line that HAS its guard. Each was closed by narrowing the failure surface,
|
||||
# which is a game with no last move. Moving the answer off the codes bash can
|
||||
# generate ends it instead: there is no longer a mutation that turns an error
|
||||
# into an answer, only mutations that turn an error into a different error.
|
||||
#
|
||||
# WHY 2 SKIPS RATHER THAN FAILS. The scenario it gates is the only thing in
|
||||
# The guards below stay, and so does the trap, but their job is now reporting
|
||||
# rather than correctness: they make a failed step land on 2 with its logs
|
||||
# printed instead of on some incidental status, which is nicer to debug and
|
||||
# lands in the same place either way.
|
||||
#
|
||||
# One piece of that reporting layer is load-bearing and not obvious. A command
|
||||
# on the left of `||` — or in an `if` condition — runs with errexit suppressed,
|
||||
# and that suppression propagates into a subshell and is NOT undone by a
|
||||
# `set -e` inside it (measured on bash 5.3: an unguarded `false` there falls
|
||||
# through to `exit 0`). Calling with errexit disarmed at the site is the only
|
||||
# form that lets the subshell re-arm it; hence the `set +e` bracket. The ERR
|
||||
# trap is then required on top, because a bare `set -e` abort exits with the
|
||||
# FAILING COMMAND's status, and `false` gives 1.
|
||||
#
|
||||
# WHY NOT-MEASURED SKIPS RATHER THAN FAILS. The scenario it gates is the only thing in
|
||||
# this suite that depends on freshness mode; everything else still runs and
|
||||
# still catches real regressions. Failing instead would turn a statement about
|
||||
# one machine's toolchain into a red gate reading "the hardlink scheme is
|
||||
# broken" across the three repos consuming this action — the same category
|
||||
# error the three-state split exists to prevent, one level up. What would
|
||||
# change the answer is 2 becoming the everyday CI outcome; it is not (gitdan-ci
|
||||
# reports 1, by measurement).
|
||||
# change the answer is not-measured becoming the everyday CI outcome; it is not
|
||||
# (gitdan-ci reports a measured INACTIVE, by measurement).
|
||||
CHECKSUM_MODE="off"
|
||||
CHECKSUM_REASON="no nightly on PATH accepting -Z checksum-freshness"
|
||||
CARGO_BIN=(cargo)
|
||||
checksum_freshness_probe() {
|
||||
local d="$scratch/freshness-probe" t="$scratch/freshness-probe-target"
|
||||
mkcrate "$d" || return 2
|
||||
mkcrate "$d" || return 2 # 2 is simply "not 0 and not 3"; see the header
|
||||
(
|
||||
set -e
|
||||
trap 'exit 2' ERR
|
||||
@@ -167,7 +181,9 @@ checksum_freshness_probe() {
|
||||
printf '%s\n' "$CONTENT_B" > src/lib.rs || exit 2
|
||||
touch -d '@1000000000' src/lib.rs || exit 2
|
||||
CARGO_TARGET_DIR="$t" cargo +nightly build -v > "$scratch/freshness-probe.log" 2>&1 || exit 2
|
||||
if grep -qE '^\s+Fresh probe' "$scratch/freshness-probe.log"; then exit 1; fi
|
||||
# 3, not 1: see the header. This is the only statement in the subshell that
|
||||
# may report a measurement, and it is the only one that may exit 3.
|
||||
if grep -qE '^\s+Fresh probe' "$scratch/freshness-probe.log"; then exit 3; fi
|
||||
exit 0
|
||||
)
|
||||
}
|
||||
@@ -186,18 +202,18 @@ if cargo +nightly -Z checksum-freshness locate-project > /dev/null 2>&1; then
|
||||
CHECKSUM_MODE="on"
|
||||
CHECKSUM_REASON=""
|
||||
;;
|
||||
1)
|
||||
3)
|
||||
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS
|
||||
CHECKSUM_REASON="this nightly accepts -Z checksum-freshness but resolves freshness by mtime"
|
||||
;;
|
||||
*)
|
||||
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS
|
||||
CHECKSUM_MODE="unmeasured"
|
||||
CHECKSUM_REASON="a probe build failed, so this was NOT MEASURED — this toolchain may or may not resolve freshness by content"
|
||||
CHECKSUM_REASON="the probe exited ${probe_rc}, which is not one of its answer codes, so this was NOT MEASURED — this toolchain may or may not resolve freshness by content"
|
||||
# Loud, because the cost is silently lost coverage on a machine that
|
||||
# might have had it. The suite continues: everything else it asserts is
|
||||
# independent of freshness mode.
|
||||
echo "::warning::hardlink-clone-selftest: could not measure whether this toolchain resolves freshness by content — a probe build failed. This is a failure to measure, not a finding about Cargo."
|
||||
echo "::warning::hardlink-clone-selftest: could not measure whether this toolchain resolves freshness by content — the probe exited ${probe_rc}. This is a failure to measure, not a finding about Cargo."
|
||||
tail -n 15 "$scratch/freshness-probe-warm.log" "$scratch/freshness-probe.log" 2>/dev/null | sed 's/^/ /' >&2 || true
|
||||
;;
|
||||
esac
|
||||
|
||||
Reference in New Issue
Block a user