test(hardlink): let the freshness probe report that it could not measure
CI / shellcheck + selftests (pull_request) Successful in 1m21s

Review finding on #13. `checksum_freshness_active()` returned non-zero for any
reason — including its own `cargo build` failing — and the caller's `else`
branch then announced "this nightly accepts -Z checksum-freshness but still
resolves freshness by mtime" regardless. On a machine where content freshness
IS live, breaking only the probe's build produced that sentence, which is
false, and silently dropped the scenario, which is lost coverage. Reachable,
not theoretical: it needs a half-installed toolchain, which is exactly the
state a probe exists to notice.

The whole point of the preceding commit was to settle this by experiment
rather than by asking, and an experiment that cannot tell a negative result
from a failed measurement is not settling it. "This toolchain resolves
freshness by mtime" is a statement about Cargo; "a probe build failed" is a
statement about this machine. They are not interchangeable.

Three outcomes now, carried in an exit code:

  0  measured ACTIVE     both builds ran, the backdated rebuild recompiled
  1  measured INACTIVE   both builds ran, the backdated rebuild was Fresh
  2  NOT MEASURED        a probe step failed; nothing was learned

Every step that could fail for a reason other than the experiment's own
outcome exits 2 explicitly, so a `set -e` abort cannot be mistaken for the 1
that means "measured, and the answer is mtime". Outcome 2 emits a ::warning::
saying in those words that this is a failure to measure and not a finding
about Cargo, and prints the tail of both probe logs. The scenario still skips
for 1 and 2 alike — everything else the suite asserts is independent of
freshness mode — but the reason is now carried to the skip line, so the two
are distinguishable at a glance.

Verified all three states on a machine where freshness is live: unmodified,
mode `on` and 4 assertions; probe build broken with a bogus flag, mode
`unmeasured` with the warning and 3 assertions, and no claim about mtime;
CARGO_UNSTABLE_CHECKSUM_FRESHNESS stripped inside the probe — CI's condition —
mode `off` naming mtime, and 3 assertions.
This commit is contained in:
2026-08-26 13:34:42 -05:00
parent eb3f0bd09b
commit 5a90d5c829
2 changed files with 57 additions and 20 deletions
+1 -1
View File
@@ -562,7 +562,7 @@ change here reaches all of them at once. That is what the gate is for.
| suite | covers |
|---|---|
| `cache-root-selftest.sh` | that a lineage nests one level and nothing else moves: no lineage resolves byte-for-byte to the cache root, two lineages on one cache key get disjoint target dirs, seed/publish/prune all stay inside their own lineage, a PR layers over its own lineage's base snapshot — **and one rejection per lineage name a reader elsewhere would stop seeing**, plus the publish-side mismatch guard |
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly that actually resolves freshness by content for its last scenario**: the source's-next-build check reasons about content rather than mtime, so under mtime freshness it would assert a bug. Whether the toolchain does is settled by experiment on a throwaway crate, not by asking it — 1.100.0-nightly accepts `-Z checksum-freshness` and rebuilds on mtime anyway. Otherwise the scenario is skipped, loudly. The control also reports which mutation families the running Cargo exhibits — a note, not an assertion, since that set moves upstream. |
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly that actually resolves freshness by content for its last scenario**: the source's-next-build check reasons about content rather than mtime, so under mtime freshness it would assert a bug. Whether the toolchain does is settled by experiment on a throwaway crate, not by asking it — 1.100.0-nightly accepts `-Z checksum-freshness` and rebuilds on mtime anyway. The experiment reports **three** outcomes, not two: active, measured-inactive, and *not measured* when a probe build failed. The scenario is skipped for the last two alike, but a failure to measure is never reported as a measurement. The control also reports which mutation families the running Cargo exhibits — a note, not an assertion, since that set moves upstream. |
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published, and the publisher's log showing it waited on the consumer's own reader-lock marker before reclaiming a rotated snapshot |
| `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone |
| `prune-cache-selftest.sh` | liveness, protection, locking, eviction order, self-clear, **and that a cache a job claims *inside* the check-to-unlink window survives it** — against a real scratch `origin` |