Compare commits
7
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e94c46f2ec
|
||
|
|
5b46986cd7
|
||
|
|
5a90d5c829
|
||
|
|
eb3f0bd09b
|
||
|
|
6a80423bd0
|
||
|
|
5e8e773f78
|
||
|
|
3f97d3d1e7
|
@@ -66,6 +66,21 @@ jobs:
|
||||
#
|
||||
# The scratch workspaces use path dependencies only, so nothing here
|
||||
# reaches crates.io.
|
||||
# Nightly first, stable second, so stable ends up the default and
|
||||
# nightly is only reachable through an explicit `+nightly`.
|
||||
#
|
||||
# `hardlink-clone-selftest.sh`'s last scenario needs a Cargo that
|
||||
# resolves freshness by CONTENT — the mode where the dep-info file
|
||||
# carries per-source checksums, which is the mutation that turns a
|
||||
# hardlink clone into silent stale-artifact reuse rather than a slow
|
||||
# build. As of 1.100.0-nightly (2026-08-25) no nightly provides it:
|
||||
# `-Z checksum-freshness` is still accepted and freshness is still
|
||||
# resolved by mtime, so the suite's probe reports that by name and skips
|
||||
# the scenario. This step therefore buys nothing today and is kept
|
||||
# anyway — it costs about twenty seconds, and the day upstream restores
|
||||
# the behaviour the coverage comes back with no edit here. See daniel/gitdan#62.
|
||||
- name: Install Rust nightly
|
||||
uses: dtolnay/rust-toolchain@nightly
|
||||
- name: Install Rust toolchain
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
|
||||
|
||||
@@ -544,7 +544,10 @@ bash scripts/selftest.sh --fast # fixture-only suites, no compiler
|
||||
|
||||
Both run in CI — `.gitea/workflows/ci.yaml`, one job, on pushes to `main` and
|
||||
on PRs that were non-draft when the run was created. It installs shellcheck
|
||||
and a stable Rust toolchain and references no credentials; the scratch
|
||||
and both a stable and a nightly Rust toolchain (nightly so
|
||||
`hardlink-clone-selftest.sh` can run its content-freshness scenario — which no
|
||||
nightly currently enables, so it is skipped and the step is kept only against
|
||||
the day upstream restores it) and references no credentials; the scratch
|
||||
workspaces the compiler-backed suites build use path dependencies only, so
|
||||
nothing reaches crates.io. It runs the full suite rather than `--fast`,
|
||||
because the two compiler-backed suites are the ones that check this scheme
|
||||
@@ -559,7 +562,7 @@ change here reaches all of them at once. That is what the gate is for.
|
||||
| suite | covers |
|
||||
|---|---|
|
||||
| `cache-root-selftest.sh` | that a lineage nests one level and nothing else moves: no lineage resolves byte-for-byte to the cache root, two lineages on one cache key get disjoint target dirs, seed/publish/prune all stay inside their own lineage, a PR layers over its own lineage's base snapshot — **and one rejection per lineage name a reader elsewhere would stop seeing**, plus the publish-side mismatch guard |
|
||||
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler. |
|
||||
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly that actually resolves freshness by content for its last scenario**: the source's-next-build check reasons about content rather than mtime, so under mtime freshness it would assert a bug. Whether the toolchain does is settled by experiment on a throwaway crate, not by asking it — 1.100.0-nightly accepts `-Z checksum-freshness` and rebuilds on mtime anyway. The experiment reports **three** outcomes, not two: active, measured-inactive, and *not measured*. Its answer codes are `0` and `3`, deliberately clear of every status bash generates for its own errors — so nothing that goes wrong inside the probe, including an expansion failure no guard can catch, can be read as an answer. The scenario is skipped for the last two alike, but a failure to measure is never reported as a measurement. The control also reports which mutation families the running Cargo exhibits — a note, not an assertion, since that set moves upstream. |
|
||||
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published, and the publisher's log showing it waited on the consumer's own reader-lock marker before reclaiming a rotated snapshot |
|
||||
| `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone |
|
||||
| `prune-cache-selftest.sh` | liveness, protection, locking, eviction order, self-clear, **and that a cache a job claims *inside* the check-to-unlink window survives it** — against a real scratch `origin` |
|
||||
|
||||
+11
-1
@@ -332,7 +332,17 @@ _unshare_files() {
|
||||
# <profile>/.fingerprint/<unit>/dep-<target> (only under
|
||||
# CARGO_UNSTABLE_CHECKSUM_FRESHNESS,
|
||||
# where this file carries the
|
||||
# per-source blake3 checksums)
|
||||
# per-source blake3 checksums.
|
||||
# NOT reproduced on
|
||||
# 1.100.0-nightly (2026-08-25),
|
||||
# measured by this repo's own CI
|
||||
# — upstream appears to have
|
||||
# stopped writing it in place.
|
||||
# Kept in the unshared set
|
||||
# anyway: it costs 22 MB of a
|
||||
# 6.9 GB tree, and the failure
|
||||
# it guards is a wrong answer,
|
||||
# not a slow one.)
|
||||
# <profile>/build/<pkg>/output, root-output (Cargo build-script metadata)
|
||||
# <profile>/build/<pkg>/out/** (whatever the build script
|
||||
# writes into OUT_DIR — build
|
||||
|
||||
@@ -73,22 +73,154 @@ mkcrate "$crate_dir"
|
||||
cd "$crate_dir"
|
||||
|
||||
export CARGO_INCREMENTAL=0
|
||||
|
||||
# Every assertion below reads cargo's own words out of a build log
|
||||
# (`Compiling libdep`, `Fresh probe`). A CI image that forces colour splices an
|
||||
# ANSI reset between the status word and the crate name, at which point every
|
||||
# one of those greps silently stops matching and the suite reports the
|
||||
# opposite of what happened — observed on gitdan-ci's runner image, where
|
||||
# scenario 2 failed while the log it printed plainly showed `Compiling libdep`.
|
||||
# Pin the format the assertions are written against.
|
||||
export CARGO_TERM_COLOR=never
|
||||
# Checksum freshness is where the worst failure lives (the dep-* file carries
|
||||
# per-source checksums and is rewritten in place). Only available on nightly;
|
||||
# without it the test still covers the build/ and *.d families.
|
||||
CHECKSUM_MODE="off"
|
||||
if cargo +nightly -V >/dev/null 2>&1; then
|
||||
export CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true
|
||||
CARGO_BIN=(cargo +nightly)
|
||||
CHECKSUM_MODE="on"
|
||||
else
|
||||
CARGO_BIN=(cargo)
|
||||
fi
|
||||
echo "=== checksum-freshness mode: ${CHECKSUM_MODE} ==="
|
||||
|
||||
CONTENT_A='pub fn f() -> u32 { 1 }'
|
||||
CONTENT_B='pub fn f() -> u32 { 22222 } pub fn g() -> u32 { 7 }'
|
||||
|
||||
# Probe the BEHAVIOUR, not the channel and not the flag. Two weaker probes
|
||||
# were tried against gitdan-ci's runner and each let the suite assert a
|
||||
# property the toolchain did not have:
|
||||
#
|
||||
# `cargo +nightly -V` — answers "did a proxy called with
|
||||
# +nightly exit 0". `-V`
|
||||
# short-circuits before `-Z` is
|
||||
# even parsed.
|
||||
# `cargo +nightly -Z checksum-freshness — answers "is this flag still
|
||||
# locate-project` accepted". 1.100.0-nightly
|
||||
# (2026-08-25) accepts it and does
|
||||
# not resolve freshness by content
|
||||
# anyway.
|
||||
#
|
||||
# The scenario at the end of this file depends on one thing and it is neither
|
||||
# of those: that changed content with an OLDER mtime rebuilds. Under mtime
|
||||
# freshness the correct answer is Fresh, so under mtime freshness that
|
||||
# scenario asserts a bug. So the probe simply performs that experiment, on its
|
||||
# own crate and its own target dir, with no clone anywhere near it — which is
|
||||
# also what makes it a control rather than a restatement of the scenario: the
|
||||
# probe establishes that the toolchain rebuilds on content, the scenario
|
||||
# establishes that a hardlink clone did not take that away.
|
||||
# THREE OUTCOMES, NOT TWO. An experiment that cannot tell a negative result
|
||||
# from a failed measurement is not settling the question, and the two are not
|
||||
# interchangeable here: "this toolchain resolves freshness by mtime" is a
|
||||
# statement about Cargo, while "a probe build failed" is a statement about this
|
||||
# machine. Collapsing them — which an earlier cut of this did, by returning
|
||||
# non-zero for both — makes a half-installed toolchain print a confident and
|
||||
# wrong explanation and quietly drop a scenario. The scenario still has to be
|
||||
# skipped in either case; what must not happen is the log claiming to know why.
|
||||
#
|
||||
# 0 content freshness measured ACTIVE — both builds ran, the backdated
|
||||
# rebuild recompiled
|
||||
# 3 measured INACTIVE — both builds ran, the backdated
|
||||
# rebuild reported Fresh
|
||||
# anything else NOT MEASURED — nothing was learned about the
|
||||
# toolchain
|
||||
#
|
||||
# THE ANSWER CODES ARE 0 AND 3, AND THE GAP IS THE MECHANISM. Bash produces 1
|
||||
# for an ordinary command failure, 2 for a usage error, 126/127 for a command
|
||||
# it could not run, 128+n for a signal, and — this is the one that matters —
|
||||
# 1 for an unbound-variable or other EXPANSION failure, which happens before
|
||||
# the command runs and is therefore invisible to a `||` guard and to an ERR
|
||||
# trap alike. It never produces 3. So "not an answer code" is decided by a
|
||||
# property of the shell rather than by an enumeration of the ways a step can
|
||||
# go wrong, and a step added later without a guard, or with a guard that
|
||||
# cannot fire, lands on NOT MEASURED by construction.
|
||||
#
|
||||
# That is the whole reason INACTIVE is not 1. It was, and three review rounds
|
||||
# on this function each found a narrower way for a shell-generated 1 to be read
|
||||
# as a measurement — an unguarded command, then a typo'd variable name on a
|
||||
# line that HAS its guard. Each was closed by narrowing the failure surface,
|
||||
# which is a game with no last move. Moving the answer off the codes bash can
|
||||
# generate ends it instead: there is no longer a mutation that turns an error
|
||||
# into an answer, only mutations that turn an error into a different error.
|
||||
#
|
||||
# The guards below stay, and so does the trap, but their job is now reporting
|
||||
# rather than correctness: they make a failed step land on 2 with its logs
|
||||
# printed instead of on some incidental status, which is nicer to debug and
|
||||
# lands in the same place either way.
|
||||
#
|
||||
# One piece of that reporting layer is load-bearing and not obvious. A command
|
||||
# on the left of `||` — or in an `if` condition — runs with errexit suppressed,
|
||||
# and that suppression propagates into a subshell and is NOT undone by a
|
||||
# `set -e` inside it (measured on bash 5.3: an unguarded `false` there falls
|
||||
# through to `exit 0`). Calling with errexit disarmed at the site is the only
|
||||
# form that lets the subshell re-arm it; hence the `set +e` bracket. The ERR
|
||||
# trap is then required on top, because a bare `set -e` abort exits with the
|
||||
# FAILING COMMAND's status, and `false` gives 1.
|
||||
#
|
||||
# WHY NOT-MEASURED SKIPS RATHER THAN FAILS. The scenario it gates is the only thing in
|
||||
# this suite that depends on freshness mode; everything else still runs and
|
||||
# still catches real regressions. Failing instead would turn a statement about
|
||||
# one machine's toolchain into a red gate reading "the hardlink scheme is
|
||||
# broken" across the three repos consuming this action — the same category
|
||||
# error the three-state split exists to prevent, one level up. What would
|
||||
# change the answer is not-measured becoming the everyday CI outcome; it is not
|
||||
# (gitdan-ci reports a measured INACTIVE, by measurement).
|
||||
CHECKSUM_MODE="off"
|
||||
CHECKSUM_REASON="no nightly on PATH accepting -Z checksum-freshness"
|
||||
CARGO_BIN=(cargo)
|
||||
checksum_freshness_probe() {
|
||||
local d="$scratch/freshness-probe" t="$scratch/freshness-probe-target"
|
||||
mkcrate "$d" || return 2 # 2 is simply "not 0 and not 3"; see the header
|
||||
(
|
||||
set -e
|
||||
trap 'exit 2' ERR
|
||||
cd "$d" || exit 2
|
||||
printf '%s\n' "$CONTENT_A" > src/lib.rs || exit 2
|
||||
CARGO_TARGET_DIR="$t" cargo +nightly build -q > "$scratch/freshness-probe-warm.log" 2>&1 || exit 2
|
||||
printf '%s\n' "$CONTENT_B" > src/lib.rs || exit 2
|
||||
touch -d '@1000000000' src/lib.rs || exit 2
|
||||
CARGO_TARGET_DIR="$t" cargo +nightly build -v > "$scratch/freshness-probe.log" 2>&1 || exit 2
|
||||
# 3, not 1: see the header. This is the only statement in the subshell that
|
||||
# may report a measurement, and it is the only one that may exit 3.
|
||||
if grep -qE '^\s+Fresh probe' "$scratch/freshness-probe.log"; then exit 3; fi
|
||||
exit 0
|
||||
)
|
||||
}
|
||||
if cargo +nightly -Z checksum-freshness locate-project > /dev/null 2>&1; then
|
||||
export CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true
|
||||
# Errexit off across the call, so the subshell can arm its own — see the
|
||||
# header. `probe_rc` is read before it is restored.
|
||||
probe_rc=0
|
||||
set +e
|
||||
checksum_freshness_probe
|
||||
probe_rc=$?
|
||||
set -e
|
||||
case "$probe_rc" in
|
||||
0)
|
||||
CARGO_BIN=(cargo +nightly)
|
||||
CHECKSUM_MODE="on"
|
||||
CHECKSUM_REASON=""
|
||||
;;
|
||||
3)
|
||||
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS
|
||||
CHECKSUM_REASON="this nightly accepts -Z checksum-freshness but resolves freshness by mtime"
|
||||
;;
|
||||
*)
|
||||
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS
|
||||
CHECKSUM_MODE="unmeasured"
|
||||
CHECKSUM_REASON="the probe exited ${probe_rc}, which is not one of its answer codes, so this was NOT MEASURED — this toolchain may or may not resolve freshness by content"
|
||||
# Loud, because the cost is silently lost coverage on a machine that
|
||||
# might have had it. The suite continues: everything else it asserts is
|
||||
# independent of freshness mode.
|
||||
echo "::warning::hardlink-clone-selftest: could not measure whether this toolchain resolves freshness by content — the probe exited ${probe_rc}. This is a failure to measure, not a finding about Cargo."
|
||||
tail -n 15 "$scratch/freshness-probe-warm.log" "$scratch/freshness-probe.log" 2>/dev/null | sed 's/^/ /' >&2 || true
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
cd "$crate_dir"
|
||||
echo "=== checksum-freshness mode: ${CHECKSUM_MODE}${CHECKSUM_REASON:+ — ${CHECKSUM_REASON}} ==="
|
||||
|
||||
build_base() {
|
||||
local dir="$1"
|
||||
printf '%s\n' "$CONTENT_A" > src/lib.rs
|
||||
@@ -112,11 +244,26 @@ fi
|
||||
ok "raw cp -al clone mutates the source ($(printf '%s\n' "$ctl_mutated" | wc -l) paths)"
|
||||
printf '%s\n' "$ctl_mutated" | sed 's/^/ /'
|
||||
|
||||
# Reported, not asserted, and the distinction is the point. The control's job
|
||||
# is to prove the hazard exists at all, which the non-empty set above already
|
||||
# does; this line records WHICH families a given Cargo exhibits.
|
||||
#
|
||||
# `.fingerprint/*/dep-*` is the worst of them — it carries the per-source
|
||||
# checksums, so mutating it through a shared inode turns a hardlink clone into
|
||||
# silent stale-artifact reuse rather than a slow build. It was measured on
|
||||
# cargo 1.9x nightly (see unshare_mutable_paths in cache-lib.sh) and is NOT
|
||||
# reproduced on 1.100.0-nightly (2026-08-25), where the control mutates only
|
||||
# the build/ and *.d families. Failing on its absence would mean this suite
|
||||
# goes red whenever upstream stops doing something we never wanted it to do —
|
||||
# and it would go red in the CONTROL, where a failure reads as "the hazard is
|
||||
# gone" rather than "upstream changed". Nothing is lost by reporting it: the
|
||||
# fix scenario below asserts the source is byte-identical after a full rebuild
|
||||
# in the clone, which covers every family this Cargo has, named or not.
|
||||
if [ "$CHECKSUM_MODE" = "on" ]; then
|
||||
if printf '%s' "$ctl_mutated" | grep -q '\.fingerprint/.*/dep-'; then
|
||||
ok "control confirms the checksum-freshness dep-info file is among the mutated set"
|
||||
echo " note: this cargo DOES rewrite .fingerprint/*/dep-* in place under checksum freshness"
|
||||
else
|
||||
fail "expected .fingerprint/*/dep-* in the control's mutated set under checksum freshness"
|
||||
echo " note: this cargo does NOT rewrite .fingerprint/*/dep-* in place; only the build/ and *.d families appear above"
|
||||
fi
|
||||
fi
|
||||
|
||||
@@ -161,10 +308,20 @@ fi
|
||||
ok "no file in the source changed after a full rebuild in the clone"
|
||||
|
||||
echo
|
||||
if [ "$CHECKSUM_MODE" = "on" ]; then
|
||||
echo "=== the whole point: the source's next build is still correct ==="
|
||||
# The source's cache holds artifacts built from CONTENT_A. Advance the source
|
||||
# to CONTENT_B (as a merge would) and rebuild in it. If the clone had
|
||||
# The source's cache holds artifacts built from CONTENT_A. Advance the
|
||||
# source to CONTENT_B (as a merge would) and rebuild in it. If the clone had
|
||||
# corrupted its dep-info, Cargo would report Fresh and keep the stale rlib.
|
||||
#
|
||||
# CHECKSUM-FRESHNESS ONLY, and the backdated mtime is why. Under checksum
|
||||
# freshness the dep-info file's per-source checksums decide, so a 2001
|
||||
# timestamp on changed content must still rebuild — the assertion below.
|
||||
# Under Cargo's ordinary MTIME freshness the same timestamp means the source
|
||||
# is older than the artifact, and reporting Fresh is the correct answer;
|
||||
# asserting otherwise asserts a bug. This scenario was written against a
|
||||
# machine with a nightly installed and, run without one, failed on that
|
||||
# correct answer.
|
||||
printf '%s\n' "$CONTENT_B" > src/lib.rs
|
||||
touch -d '@1000000000' src/lib.rs
|
||||
log="$scratch/rebuild.log"
|
||||
@@ -173,6 +330,10 @@ if grep -qE '^\s+Fresh probe' "$log"; then
|
||||
fail "source declared its own crate Fresh against sources it has never built — stale-artifact reuse"
|
||||
fi
|
||||
ok "source correctly rebuilt its crate after advancing to the clone's content"
|
||||
else
|
||||
echo "=== skipped: the source's-next-build scenario needs content-based freshness ==="
|
||||
echo " reason: ${CHECKSUM_REASON}"
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "hardlink-clone-selftest: ${pass_count} assertions passed"
|
||||
|
||||
@@ -65,10 +65,10 @@ origin="$scratch/origin.git"; git init -q --bare "$origin"
|
||||
work="$scratch/work"; git init -q "$work"
|
||||
(
|
||||
cd "$work"
|
||||
git -c user.email=t@t -c user.name=t commit -q --allow-empty -m init
|
||||
git -c user.email=t@t -c user.name=t -c commit.gpgsign=false commit -q --allow-empty -m init
|
||||
git branch -M main
|
||||
git checkout -q -b dev; git -c user.email=t@t -c user.name=t commit -q --allow-empty -m dev
|
||||
git checkout -q -b feat/live; git -c user.email=t@t -c user.name=t commit -q --allow-empty -m live
|
||||
git checkout -q -b dev; git -c user.email=t@t -c user.name=t -c commit.gpgsign=false commit -q --allow-empty -m dev
|
||||
git checkout -q -b feat/live; git -c user.email=t@t -c user.name=t -c commit.gpgsign=false commit -q --allow-empty -m live
|
||||
git remote add origin "$origin"
|
||||
git push -q origin main dev feat/live
|
||||
)
|
||||
|
||||
@@ -75,6 +75,15 @@
|
||||
# Asserts BOTH jobs correctly recompile the dependency and succeed.
|
||||
set -euo pipefail
|
||||
|
||||
# Every assertion below reads cargo's own words out of a build log
|
||||
# (`Compiling libdep`, `Fresh probe`). A CI image that forces colour splices an
|
||||
# ANSI reset between the status word and the crate name, at which point every
|
||||
# one of those greps silently stops matching and the suite reports the
|
||||
# opposite of what happened — observed on gitdan-ci's runner image, where
|
||||
# scenario 2 failed while the log it printed plainly showed `Compiling libdep`.
|
||||
# Pin the format the assertions are written against.
|
||||
export CARGO_TERM_COLOR=never
|
||||
|
||||
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
||||
restore_mtimes="$script_dir/restore-mtimes.sh"
|
||||
|
||||
@@ -131,6 +140,11 @@ cd "$repo"
|
||||
git init -q
|
||||
git config user.email test@example.com
|
||||
git config user.name "restore-mtimes-selftest"
|
||||
# Local to this mktemp'd throwaway repo. Without it the eight commits below
|
||||
# inherit the developer's GLOBAL commit.gpgsign, which makes whether this gate
|
||||
# passes depend on their gpg agent — observed as a red run caused by a full
|
||||
# disk breaking gpg, in a suite that has nothing to say about either.
|
||||
git config commit.gpgsign false
|
||||
|
||||
cat > Cargo.toml <<'EOF'
|
||||
[workspace]
|
||||
|
||||
Reference in New Issue
Block a user