test(hardlink): let the freshness probe report that it could not measure
CI / shellcheck + selftests (pull_request) Successful in 1m21s
CI / shellcheck + selftests (pull_request) Successful in 1m21s
Review finding on #13. `checksum_freshness_active()` returned non-zero for any reason — including its own `cargo build` failing — and the caller's `else` branch then announced "this nightly accepts -Z checksum-freshness but still resolves freshness by mtime" regardless. On a machine where content freshness IS live, breaking only the probe's build produced that sentence, which is false, and silently dropped the scenario, which is lost coverage. Reachable, not theoretical: it needs a half-installed toolchain, which is exactly the state a probe exists to notice. The whole point of the preceding commit was to settle this by experiment rather than by asking, and an experiment that cannot tell a negative result from a failed measurement is not settling it. "This toolchain resolves freshness by mtime" is a statement about Cargo; "a probe build failed" is a statement about this machine. They are not interchangeable. Three outcomes now, carried in an exit code: 0 measured ACTIVE both builds ran, the backdated rebuild recompiled 1 measured INACTIVE both builds ran, the backdated rebuild was Fresh 2 NOT MEASURED a probe step failed; nothing was learned Every step that could fail for a reason other than the experiment's own outcome exits 2 explicitly, so a `set -e` abort cannot be mistaken for the 1 that means "measured, and the answer is mtime". Outcome 2 emits a ::warning:: saying in those words that this is a failure to measure and not a finding about Cargo, and prints the tail of both probe logs. The scenario still skips for 1 and 2 alike — everything else the suite asserts is independent of freshness mode — but the reason is now carried to the skip line, so the two are distinguishable at a glance. Verified all three states on a machine where freshness is live: unmodified, mode `on` and 4 assertions; probe build broken with a bogus flag, mode `unmeasured` with the warning and 3 assertions, and no claim about mtime; CARGO_UNSTABLE_CHECKSUM_FRESHNESS stripped inside the probe — CI's condition — mode `off` naming mtime, and 3 assertions.
This commit is contained in:
@@ -562,7 +562,7 @@ change here reaches all of them at once. That is what the gate is for.
|
||||
| suite | covers |
|
||||
|---|---|
|
||||
| `cache-root-selftest.sh` | that a lineage nests one level and nothing else moves: no lineage resolves byte-for-byte to the cache root, two lineages on one cache key get disjoint target dirs, seed/publish/prune all stay inside their own lineage, a PR layers over its own lineage's base snapshot — **and one rejection per lineage name a reader elsewhere would stop seeing**, plus the publish-side mismatch guard |
|
||||
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly that actually resolves freshness by content for its last scenario**: the source's-next-build check reasons about content rather than mtime, so under mtime freshness it would assert a bug. Whether the toolchain does is settled by experiment on a throwaway crate, not by asking it — 1.100.0-nightly accepts `-Z checksum-freshness` and rebuilds on mtime anyway. Otherwise the scenario is skipped, loudly. The control also reports which mutation families the running Cargo exhibits — a note, not an assertion, since that set moves upstream. |
|
||||
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly that actually resolves freshness by content for its last scenario**: the source's-next-build check reasons about content rather than mtime, so under mtime freshness it would assert a bug. Whether the toolchain does is settled by experiment on a throwaway crate, not by asking it — 1.100.0-nightly accepts `-Z checksum-freshness` and rebuilds on mtime anyway. The experiment reports **three** outcomes, not two: active, measured-inactive, and *not measured* when a probe build failed. The scenario is skipped for the last two alike, but a failure to measure is never reported as a measurement. The control also reports which mutation families the running Cargo exhibits — a note, not an assertion, since that set moves upstream. |
|
||||
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published, and the publisher's log showing it waited on the consumer's own reader-lock marker before reclaiming a rotated snapshot |
|
||||
| `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone |
|
||||
| `prune-cache-selftest.sh` | liveness, protection, locking, eviction order, self-clear, **and that a cache a job claims *inside* the check-to-unlink window survives it** — against a real scratch `origin` |
|
||||
|
||||
@@ -110,33 +110,70 @@ CONTENT_B='pub fn f() -> u32 { 22222 } pub fn g() -> u32 { 7 }'
|
||||
# also what makes it a control rather than a restatement of the scenario: the
|
||||
# probe establishes that the toolchain rebuilds on content, the scenario
|
||||
# establishes that a hardlink clone did not take that away.
|
||||
# THREE OUTCOMES, NOT TWO. An experiment that cannot tell a negative result
|
||||
# from a failed measurement is not settling the question, and the two are not
|
||||
# interchangeable here: "this toolchain resolves freshness by mtime" is a
|
||||
# statement about Cargo, while "a probe build failed" is a statement about this
|
||||
# machine. Collapsing them — which an earlier cut of this did, by returning
|
||||
# non-zero for both — makes a half-installed toolchain print a confident and
|
||||
# wrong explanation and quietly drop a scenario. The scenario still has to be
|
||||
# skipped in either case; what must not happen is the log claiming to know why.
|
||||
#
|
||||
# 0 content freshness measured ACTIVE — both builds ran, the backdated
|
||||
# rebuild recompiled
|
||||
# 1 measured INACTIVE — both builds ran, the backdated
|
||||
# rebuild reported Fresh
|
||||
# 2 NOT MEASURED — a probe step failed; nothing was
|
||||
# learned about the toolchain
|
||||
#
|
||||
# Every step that could fail for a reason other than the experiment's own
|
||||
# outcome exits 2 explicitly, so a `set -e` abort can never be mistaken for the
|
||||
# `1` that means "measured, and the answer is mtime".
|
||||
CHECKSUM_MODE="off"
|
||||
CHECKSUM_REASON="no nightly on PATH accepting -Z checksum-freshness"
|
||||
CARGO_BIN=(cargo)
|
||||
checksum_freshness_active() {
|
||||
checksum_freshness_probe() {
|
||||
local d="$scratch/freshness-probe" t="$scratch/freshness-probe-target"
|
||||
mkcrate "$d"
|
||||
mkcrate "$d" || return 2
|
||||
(
|
||||
cd "$d" || exit 1
|
||||
printf '%s\n' "$CONTENT_A" > src/lib.rs
|
||||
CARGO_TARGET_DIR="$t" cargo +nightly build -q > /dev/null 2>&1 || exit 1
|
||||
printf '%s\n' "$CONTENT_B" > src/lib.rs
|
||||
touch -d '@1000000000' src/lib.rs
|
||||
CARGO_TARGET_DIR="$t" cargo +nightly build -v > "$scratch/freshness-probe.log" 2>&1 || exit 1
|
||||
! grep -qE '^\s+Fresh probe' "$scratch/freshness-probe.log"
|
||||
cd "$d" || exit 2
|
||||
printf '%s\n' "$CONTENT_A" > src/lib.rs || exit 2
|
||||
CARGO_TARGET_DIR="$t" cargo +nightly build -q > "$scratch/freshness-probe-warm.log" 2>&1 || exit 2
|
||||
printf '%s\n' "$CONTENT_B" > src/lib.rs || exit 2
|
||||
touch -d '@1000000000' src/lib.rs || exit 2
|
||||
CARGO_TARGET_DIR="$t" cargo +nightly build -v > "$scratch/freshness-probe.log" 2>&1 || exit 2
|
||||
if grep -qE '^\s+Fresh probe' "$scratch/freshness-probe.log"; then exit 1; fi
|
||||
exit 0
|
||||
)
|
||||
}
|
||||
if cargo +nightly -Z checksum-freshness locate-project > /dev/null 2>&1; then
|
||||
export CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true
|
||||
if checksum_freshness_active; then
|
||||
CARGO_BIN=(cargo +nightly)
|
||||
CHECKSUM_MODE="on"
|
||||
else
|
||||
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS
|
||||
echo "note: this nightly accepts -Z checksum-freshness but still resolves freshness by mtime"
|
||||
fi
|
||||
probe_rc=0
|
||||
checksum_freshness_probe || probe_rc=$?
|
||||
case "$probe_rc" in
|
||||
0)
|
||||
CARGO_BIN=(cargo +nightly)
|
||||
CHECKSUM_MODE="on"
|
||||
CHECKSUM_REASON=""
|
||||
;;
|
||||
1)
|
||||
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS
|
||||
CHECKSUM_REASON="this nightly accepts -Z checksum-freshness but resolves freshness by mtime"
|
||||
;;
|
||||
*)
|
||||
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS
|
||||
CHECKSUM_MODE="unmeasured"
|
||||
CHECKSUM_REASON="a probe build failed, so this was NOT MEASURED — this toolchain may or may not resolve freshness by content"
|
||||
# Loud, because the cost is silently lost coverage on a machine that
|
||||
# might have had it. The suite continues: everything else it asserts is
|
||||
# independent of freshness mode.
|
||||
echo "::warning::hardlink-clone-selftest: could not measure whether this toolchain resolves freshness by content — a probe build failed. This is a failure to measure, not a finding about Cargo."
|
||||
tail -n 15 "$scratch/freshness-probe-warm.log" "$scratch/freshness-probe.log" 2>/dev/null | sed 's/^/ /' >&2 || true
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
cd "$crate_dir"
|
||||
echo "=== checksum-freshness mode: ${CHECKSUM_MODE} ==="
|
||||
echo "=== checksum-freshness mode: ${CHECKSUM_MODE}${CHECKSUM_REASON:+ — ${CHECKSUM_REASON}} ==="
|
||||
|
||||
build_base() {
|
||||
local dir="$1"
|
||||
@@ -248,8 +285,8 @@ if [ "$CHECKSUM_MODE" = "on" ]; then
|
||||
fi
|
||||
ok "source correctly rebuilt its crate after advancing to the clone's content"
|
||||
else
|
||||
echo "=== skipped: the source's-next-build scenario needs checksum freshness ==="
|
||||
echo " (a nightly cargo accepting -Z checksum-freshness; see the probe above)"
|
||||
echo "=== skipped: the source's-next-build scenario needs content-based freshness ==="
|
||||
echo " reason: ${CHECKSUM_REASON}"
|
||||
fi
|
||||
|
||||
echo
|
||||
|
||||
Reference in New Issue
Block a user