fix(selftest): make the two compiler-backed suites survive a CI runner
CI / shellcheck + selftests (pull_request) Failing after 1m36s
CI / shellcheck + selftests (pull_request) Failing after 1m36s
The new gate's first run went red on both suites that drive a real Cargo.
Neither failure was a defect in what they test; both were assumptions about
the machine, which had only ever been a dev box with a nightly installed and
no colour forcing. Fixed here rather than waived — turning a gate on is what
obliges fixing what it finds.
COLOUR. Every assertion in restore-mtimes-selftest.sh, and one in
hardlink-clone-selftest.sh, reads cargo's own words out of a build log
(`Compiling libdep`, `Fresh probe`). gitdan-ci's runner image forces colour, so
cargo wrote `Compiling\e[0m libdep` into the log and `grep -q "Compiling
libdep"` stopped matching. The suite then reported the opposite of what
happened — the failure printed the log, and the log plainly said `Compiling
libdep`. Both suites now pin CARGO_TERM_COLOR=never, which is the format their
assertions are written against.
Red-proven: with the export removed and CARGO_TERM_COLOR=always,
restore-mtimes-selftest reproduces the CI failure verbatim ("expected to find:
Compiling libdep"); with it, 14/14 pass under the same forced colour.
THE NIGHTLY PROBE ASKED THE WRONG QUESTION. `cargo +nightly -V` answers "did a
cargo proxy called with +nightly exit 0", which is not "will this build have
checksum freshness": `-V` short-circuits before `-Z` is validated at all. So
the probe said "on" on a runner where the flag was not in effect, and the
suite asserted the checksum-freshness mutation family against a build that
never had it — failing in the CONTROL, where a failure reads as "the hazard is
gone" rather than "the toolchain is wrong". It now probes the capability:
`cargo +nightly -Z checksum-freshness locate-project`, the narrowest command
that actually parses the flag. It rejects the stable channel and an unknown
flag name alike, needs no network and builds nothing.
Red-proven: the old channel probe, pointed at a cargo without the flag,
reproduces the CI failure exactly.
AND THE OFF PATH DID NOT WORK EITHER. The suite's header claimed that without
checksum freshness "the test still covers the build/ and *.d families". It did
not: the final scenario backdates the source to 2001 and asserts the rebuild
is not Fresh, which is checksum-freshness-only reasoning. Under Cargo's
ordinary mtime freshness a 2001 source IS older than the artifact and Fresh is
the correct answer, so the scenario asserted a bug. It is now gated on the
mode and skipped loudly, like the control's dep-* assertion already was: 5
assertions with a nightly, 3 without.
CI installs a nightly as well as stable (nightly first, so stable stays the
default) — that hazard is the one this whole scheme exists to close, and a CI
that skips it is checking the cheap half.
This commit is contained in:
@@ -66,6 +66,18 @@ jobs:
|
|||||||
#
|
#
|
||||||
# The scratch workspaces use path dependencies only, so nothing here
|
# The scratch workspaces use path dependencies only, so nothing here
|
||||||
# reaches crates.io.
|
# reaches crates.io.
|
||||||
|
# Nightly first, stable second, so stable ends up the default and
|
||||||
|
# nightly is only reachable through an explicit `+nightly`.
|
||||||
|
#
|
||||||
|
# Nightly is not optional here. `hardlink-clone-selftest.sh` gates its
|
||||||
|
# two strongest assertions on `-Z checksum-freshness` — the mode where
|
||||||
|
# Cargo's dep-info file carries per-source checksums and is rewritten in
|
||||||
|
# place, which is the mutation that turns a hardlink clone into SILENT
|
||||||
|
# stale-artifact reuse rather than a slow build. Without a nightly the
|
||||||
|
# suite still runs, and skips exactly the hazard this whole scheme exists
|
||||||
|
# to close.
|
||||||
|
- name: Install Rust nightly
|
||||||
|
uses: dtolnay/rust-toolchain@nightly
|
||||||
- name: Install Rust toolchain
|
- name: Install Rust toolchain
|
||||||
uses: dtolnay/rust-toolchain@stable
|
uses: dtolnay/rust-toolchain@stable
|
||||||
|
|
||||||
|
|||||||
@@ -544,7 +544,10 @@ bash scripts/selftest.sh --fast # fixture-only suites, no compiler
|
|||||||
|
|
||||||
Both run in CI — `.gitea/workflows/ci.yaml`, one job, on pushes to `main` and
|
Both run in CI — `.gitea/workflows/ci.yaml`, one job, on pushes to `main` and
|
||||||
on PRs that were non-draft when the run was created. It installs shellcheck
|
on PRs that were non-draft when the run was created. It installs shellcheck
|
||||||
and a stable Rust toolchain and references no credentials; the scratch
|
and both a stable and a nightly Rust toolchain (nightly for
|
||||||
|
`-Z checksum-freshness`, without which `hardlink-clone-selftest.sh` skips the
|
||||||
|
two assertions that cover the silent-stale-reuse hazard) and references no
|
||||||
|
credentials; the scratch
|
||||||
workspaces the compiler-backed suites build use path dependencies only, so
|
workspaces the compiler-backed suites build use path dependencies only, so
|
||||||
nothing reaches crates.io. It runs the full suite rather than `--fast`,
|
nothing reaches crates.io. It runs the full suite rather than `--fast`,
|
||||||
because the two compiler-backed suites are the ones that check this scheme
|
because the two compiler-backed suites are the ones that check this scheme
|
||||||
@@ -559,7 +562,7 @@ change here reaches all of them at once. That is what the gate is for.
|
|||||||
| suite | covers |
|
| suite | covers |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `cache-root-selftest.sh` | that a lineage nests one level and nothing else moves: no lineage resolves byte-for-byte to the cache root, two lineages on one cache key get disjoint target dirs, seed/publish/prune all stay inside their own lineage, a PR layers over its own lineage's base snapshot — **and one rejection per lineage name a reader elsewhere would stop seeing**, plus the publish-side mismatch guard |
|
| `cache-root-selftest.sh` | that a lineage nests one level and nothing else moves: no lineage resolves byte-for-byte to the cache root, two lineages on one cache key get disjoint target dirs, seed/publish/prune all stay inside their own lineage, a PR layers over its own lineage's base snapshot — **and one rejection per lineage name a reader elsewhere would stop seeing**, plus the publish-side mismatch guard |
|
||||||
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler. |
|
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly for two of its five assertions**: the checksum-freshness dep-info file is the mutation that turns a hardlink clone into silent stale-artifact reuse, and it only exists under `-Z checksum-freshness`. Without one those two are skipped, loudly. |
|
||||||
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published, and the publisher's log showing it waited on the consumer's own reader-lock marker before reclaiming a rotated snapshot |
|
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published, and the publisher's log showing it waited on the consumer's own reader-lock marker before reclaiming a rotated snapshot |
|
||||||
| `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone |
|
| `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone |
|
||||||
| `prune-cache-selftest.sh` | liveness, protection, locking, eviction order, self-clear, **and that a cache a job claims *inside* the check-to-unlink window survives it** — against a real scratch `origin` |
|
| `prune-cache-selftest.sh` | liveness, protection, locking, eviction order, self-clear, **and that a cache a job claims *inside* the check-to-unlink window survives it** — against a real scratch `origin` |
|
||||||
|
|||||||
@@ -73,11 +73,33 @@ mkcrate "$crate_dir"
|
|||||||
cd "$crate_dir"
|
cd "$crate_dir"
|
||||||
|
|
||||||
export CARGO_INCREMENTAL=0
|
export CARGO_INCREMENTAL=0
|
||||||
|
|
||||||
|
# Every assertion below reads cargo's own words out of a build log
|
||||||
|
# (`Compiling libdep`, `Fresh probe`). A CI image that forces colour splices an
|
||||||
|
# ANSI reset between the status word and the crate name, at which point every
|
||||||
|
# one of those greps silently stops matching and the suite reports the
|
||||||
|
# opposite of what happened — observed on gitdan-ci's runner image, where
|
||||||
|
# scenario 2 failed while the log it printed plainly showed `Compiling libdep`.
|
||||||
|
# Pin the format the assertions are written against.
|
||||||
|
export CARGO_TERM_COLOR=never
|
||||||
# Checksum freshness is where the worst failure lives (the dep-* file carries
|
# Checksum freshness is where the worst failure lives (the dep-* file carries
|
||||||
# per-source checksums and is rewritten in place). Only available on nightly;
|
# per-source checksums and is rewritten in place). Only available on nightly;
|
||||||
# without it the test still covers the build/ and *.d families.
|
# without it the test still covers the build/ and *.d families.
|
||||||
CHECKSUM_MODE="off"
|
CHECKSUM_MODE="off"
|
||||||
if cargo +nightly -V >/dev/null 2>&1; then
|
# Probe the CAPABILITY, not the channel. `cargo +nightly -V` answers "did a
|
||||||
|
# cargo proxy called with +nightly exit 0", which is a different question from
|
||||||
|
# "will this build have checksum freshness" — `-V` short-circuits before `-Z`
|
||||||
|
# is validated at all, so that probe says yes on any cargo that resolves the
|
||||||
|
# name, including one whose nightly has since moved the flag. The scenario
|
||||||
|
# below then asserts the checksum-freshness mutation family against a build
|
||||||
|
# that never had it, and fails in the CONTROL, where a failure reads as "the
|
||||||
|
# hazard is gone" rather than "the toolchain is wrong". Observed the first
|
||||||
|
# time this suite ran on gitdan-ci.
|
||||||
|
#
|
||||||
|
# `-Z <flag> locate-project` is the narrowest command that actually parses the
|
||||||
|
# flag: it rejects the stable channel and an unknown flag name alike, needs no
|
||||||
|
# network, and builds nothing.
|
||||||
|
if cargo +nightly -Z checksum-freshness locate-project >/dev/null 2>&1; then
|
||||||
export CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true
|
export CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true
|
||||||
CARGO_BIN=(cargo +nightly)
|
CARGO_BIN=(cargo +nightly)
|
||||||
CHECKSUM_MODE="on"
|
CHECKSUM_MODE="on"
|
||||||
@@ -161,10 +183,20 @@ fi
|
|||||||
ok "no file in the source changed after a full rebuild in the clone"
|
ok "no file in the source changed after a full rebuild in the clone"
|
||||||
|
|
||||||
echo
|
echo
|
||||||
|
if [ "$CHECKSUM_MODE" = "on" ]; then
|
||||||
echo "=== the whole point: the source's next build is still correct ==="
|
echo "=== the whole point: the source's next build is still correct ==="
|
||||||
# The source's cache holds artifacts built from CONTENT_A. Advance the source
|
# The source's cache holds artifacts built from CONTENT_A. Advance the
|
||||||
# to CONTENT_B (as a merge would) and rebuild in it. If the clone had
|
# source to CONTENT_B (as a merge would) and rebuild in it. If the clone had
|
||||||
# corrupted its dep-info, Cargo would report Fresh and keep the stale rlib.
|
# corrupted its dep-info, Cargo would report Fresh and keep the stale rlib.
|
||||||
|
#
|
||||||
|
# CHECKSUM-FRESHNESS ONLY, and the backdated mtime is why. Under checksum
|
||||||
|
# freshness the dep-info file's per-source checksums decide, so a 2001
|
||||||
|
# timestamp on changed content must still rebuild — the assertion below.
|
||||||
|
# Under Cargo's ordinary MTIME freshness the same timestamp means the source
|
||||||
|
# is older than the artifact, and reporting Fresh is the correct answer;
|
||||||
|
# asserting otherwise asserts a bug. This scenario was written against a
|
||||||
|
# machine with a nightly installed and, run without one, failed on that
|
||||||
|
# correct answer.
|
||||||
printf '%s\n' "$CONTENT_B" > src/lib.rs
|
printf '%s\n' "$CONTENT_B" > src/lib.rs
|
||||||
touch -d '@1000000000' src/lib.rs
|
touch -d '@1000000000' src/lib.rs
|
||||||
log="$scratch/rebuild.log"
|
log="$scratch/rebuild.log"
|
||||||
@@ -173,6 +205,10 @@ if grep -qE '^\s+Fresh probe' "$log"; then
|
|||||||
fail "source declared its own crate Fresh against sources it has never built — stale-artifact reuse"
|
fail "source declared its own crate Fresh against sources it has never built — stale-artifact reuse"
|
||||||
fi
|
fi
|
||||||
ok "source correctly rebuilt its crate after advancing to the clone's content"
|
ok "source correctly rebuilt its crate after advancing to the clone's content"
|
||||||
|
else
|
||||||
|
echo "=== skipped: the source's-next-build scenario needs checksum freshness ==="
|
||||||
|
echo " (a nightly cargo accepting -Z checksum-freshness; see the probe above)"
|
||||||
|
fi
|
||||||
|
|
||||||
echo
|
echo
|
||||||
echo "hardlink-clone-selftest: ${pass_count} assertions passed"
|
echo "hardlink-clone-selftest: ${pass_count} assertions passed"
|
||||||
|
|||||||
@@ -75,6 +75,15 @@
|
|||||||
# Asserts BOTH jobs correctly recompile the dependency and succeed.
|
# Asserts BOTH jobs correctly recompile the dependency and succeed.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
|
# Every assertion below reads cargo's own words out of a build log
|
||||||
|
# (`Compiling libdep`, `Fresh probe`). A CI image that forces colour splices an
|
||||||
|
# ANSI reset between the status word and the crate name, at which point every
|
||||||
|
# one of those greps silently stops matching and the suite reports the
|
||||||
|
# opposite of what happened — observed on gitdan-ci's runner image, where
|
||||||
|
# scenario 2 failed while the log it printed plainly showed `Compiling libdep`.
|
||||||
|
# Pin the format the assertions are written against.
|
||||||
|
export CARGO_TERM_COLOR=never
|
||||||
|
|
||||||
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
||||||
restore_mtimes="$script_dir/restore-mtimes.sh"
|
restore_mtimes="$script_dir/restore-mtimes.sh"
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user