test(hardlink): let the freshness probe report that it could not measure
CI / shellcheck + selftests (pull_request) Successful in 1m21s
CI / shellcheck + selftests (pull_request) Successful in 1m21s
Review finding on #13. `checksum_freshness_active()` returned non-zero for any reason — including its own `cargo build` failing — and the caller's `else` branch then announced "this nightly accepts -Z checksum-freshness but still resolves freshness by mtime" regardless. On a machine where content freshness IS live, breaking only the probe's build produced that sentence, which is false, and silently dropped the scenario, which is lost coverage. Reachable, not theoretical: it needs a half-installed toolchain, which is exactly the state a probe exists to notice. The whole point of the preceding commit was to settle this by experiment rather than by asking, and an experiment that cannot tell a negative result from a failed measurement is not settling it. "This toolchain resolves freshness by mtime" is a statement about Cargo; "a probe build failed" is a statement about this machine. They are not interchangeable. Three outcomes now, carried in an exit code: 0 measured ACTIVE both builds ran, the backdated rebuild recompiled 1 measured INACTIVE both builds ran, the backdated rebuild was Fresh 2 NOT MEASURED a probe step failed; nothing was learned Every step that could fail for a reason other than the experiment's own outcome exits 2 explicitly, so a `set -e` abort cannot be mistaken for the 1 that means "measured, and the answer is mtime". Outcome 2 emits a ::warning:: saying in those words that this is a failure to measure and not a finding about Cargo, and prints the tail of both probe logs. The scenario still skips for 1 and 2 alike — everything else the suite asserts is independent of freshness mode — but the reason is now carried to the skip line, so the two are distinguishable at a glance. Verified all three states on a machine where freshness is live: unmodified, mode `on` and 4 assertions; probe build broken with a bogus flag, mode `unmeasured` with the warning and 3 assertions, and no claim about mtime; CARGO_UNSTABLE_CHECKSUM_FRESHNESS stripped inside the probe — CI's condition — mode `off` naming mtime, and 3 assertions.
This commit is contained in:
@@ -110,33 +110,70 @@ CONTENT_B='pub fn f() -> u32 { 22222 } pub fn g() -> u32 { 7 }'
|
||||
# also what makes it a control rather than a restatement of the scenario: the
|
||||
# probe establishes that the toolchain rebuilds on content, the scenario
|
||||
# establishes that a hardlink clone did not take that away.
|
||||
# THREE OUTCOMES, NOT TWO. An experiment that cannot tell a negative result
|
||||
# from a failed measurement is not settling the question, and the two are not
|
||||
# interchangeable here: "this toolchain resolves freshness by mtime" is a
|
||||
# statement about Cargo, while "a probe build failed" is a statement about this
|
||||
# machine. Collapsing them — which an earlier cut of this did, by returning
|
||||
# non-zero for both — makes a half-installed toolchain print a confident and
|
||||
# wrong explanation and quietly drop a scenario. The scenario still has to be
|
||||
# skipped in either case; what must not happen is the log claiming to know why.
|
||||
#
|
||||
# 0 content freshness measured ACTIVE — both builds ran, the backdated
|
||||
# rebuild recompiled
|
||||
# 1 measured INACTIVE — both builds ran, the backdated
|
||||
# rebuild reported Fresh
|
||||
# 2 NOT MEASURED — a probe step failed; nothing was
|
||||
# learned about the toolchain
|
||||
#
|
||||
# Every step that could fail for a reason other than the experiment's own
|
||||
# outcome exits 2 explicitly, so a `set -e` abort can never be mistaken for the
|
||||
# `1` that means "measured, and the answer is mtime".
|
||||
CHECKSUM_MODE="off"
|
||||
CHECKSUM_REASON="no nightly on PATH accepting -Z checksum-freshness"
|
||||
CARGO_BIN=(cargo)
|
||||
checksum_freshness_active() {
|
||||
checksum_freshness_probe() {
|
||||
local d="$scratch/freshness-probe" t="$scratch/freshness-probe-target"
|
||||
mkcrate "$d"
|
||||
mkcrate "$d" || return 2
|
||||
(
|
||||
cd "$d" || exit 1
|
||||
printf '%s\n' "$CONTENT_A" > src/lib.rs
|
||||
CARGO_TARGET_DIR="$t" cargo +nightly build -q > /dev/null 2>&1 || exit 1
|
||||
printf '%s\n' "$CONTENT_B" > src/lib.rs
|
||||
touch -d '@1000000000' src/lib.rs
|
||||
CARGO_TARGET_DIR="$t" cargo +nightly build -v > "$scratch/freshness-probe.log" 2>&1 || exit 1
|
||||
! grep -qE '^\s+Fresh probe' "$scratch/freshness-probe.log"
|
||||
cd "$d" || exit 2
|
||||
printf '%s\n' "$CONTENT_A" > src/lib.rs || exit 2
|
||||
CARGO_TARGET_DIR="$t" cargo +nightly build -q > "$scratch/freshness-probe-warm.log" 2>&1 || exit 2
|
||||
printf '%s\n' "$CONTENT_B" > src/lib.rs || exit 2
|
||||
touch -d '@1000000000' src/lib.rs || exit 2
|
||||
CARGO_TARGET_DIR="$t" cargo +nightly build -v > "$scratch/freshness-probe.log" 2>&1 || exit 2
|
||||
if grep -qE '^\s+Fresh probe' "$scratch/freshness-probe.log"; then exit 1; fi
|
||||
exit 0
|
||||
)
|
||||
}
|
||||
if cargo +nightly -Z checksum-freshness locate-project > /dev/null 2>&1; then
|
||||
export CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true
|
||||
if checksum_freshness_active; then
|
||||
CARGO_BIN=(cargo +nightly)
|
||||
CHECKSUM_MODE="on"
|
||||
else
|
||||
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS
|
||||
echo "note: this nightly accepts -Z checksum-freshness but still resolves freshness by mtime"
|
||||
fi
|
||||
probe_rc=0
|
||||
checksum_freshness_probe || probe_rc=$?
|
||||
case "$probe_rc" in
|
||||
0)
|
||||
CARGO_BIN=(cargo +nightly)
|
||||
CHECKSUM_MODE="on"
|
||||
CHECKSUM_REASON=""
|
||||
;;
|
||||
1)
|
||||
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS
|
||||
CHECKSUM_REASON="this nightly accepts -Z checksum-freshness but resolves freshness by mtime"
|
||||
;;
|
||||
*)
|
||||
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS
|
||||
CHECKSUM_MODE="unmeasured"
|
||||
CHECKSUM_REASON="a probe build failed, so this was NOT MEASURED — this toolchain may or may not resolve freshness by content"
|
||||
# Loud, because the cost is silently lost coverage on a machine that
|
||||
# might have had it. The suite continues: everything else it asserts is
|
||||
# independent of freshness mode.
|
||||
echo "::warning::hardlink-clone-selftest: could not measure whether this toolchain resolves freshness by content — a probe build failed. This is a failure to measure, not a finding about Cargo."
|
||||
tail -n 15 "$scratch/freshness-probe-warm.log" "$scratch/freshness-probe.log" 2>/dev/null | sed 's/^/ /' >&2 || true
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
cd "$crate_dir"
|
||||
echo "=== checksum-freshness mode: ${CHECKSUM_MODE} ==="
|
||||
echo "=== checksum-freshness mode: ${CHECKSUM_MODE}${CHECKSUM_REASON:+ — ${CHECKSUM_REASON}} ==="
|
||||
|
||||
build_base() {
|
||||
local dir="$1"
|
||||
@@ -248,8 +285,8 @@ if [ "$CHECKSUM_MODE" = "on" ]; then
|
||||
fi
|
||||
ok "source correctly rebuilt its crate after advancing to the clone's content"
|
||||
else
|
||||
echo "=== skipped: the source's-next-build scenario needs checksum freshness ==="
|
||||
echo " (a nightly cargo accepting -Z checksum-freshness; see the probe above)"
|
||||
echo "=== skipped: the source's-next-build scenario needs content-based freshness ==="
|
||||
echo " reason: ${CHECKSUM_REASON}"
|
||||
fi
|
||||
|
||||
echo
|
||||
|
||||
Reference in New Issue
Block a user