fix(cache): separate build directories for same-ref jobs, and a CI gate #13
@@ -66,6 +66,18 @@ jobs:
|
||||
#
|
||||
# The scratch workspaces use path dependencies only, so nothing here
|
||||
# reaches crates.io.
|
||||
# Nightly first, stable second, so stable ends up the default and
|
||||
# nightly is only reachable through an explicit `+nightly`.
|
||||
#
|
||||
# Nightly is not optional here. `hardlink-clone-selftest.sh` gates its
|
||||
# two strongest assertions on `-Z checksum-freshness` — the mode where
|
||||
# Cargo's dep-info file carries per-source checksums and is rewritten in
|
||||
# place, which is the mutation that turns a hardlink clone into SILENT
|
||||
# stale-artifact reuse rather than a slow build. Without a nightly the
|
||||
# suite still runs, and skips exactly the hazard this whole scheme exists
|
||||
# to close.
|
||||
- name: Install Rust nightly
|
||||
uses: dtolnay/rust-toolchain@nightly
|
||||
- name: Install Rust toolchain
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
|
||||
|
||||
@@ -544,7 +544,10 @@ bash scripts/selftest.sh --fast # fixture-only suites, no compiler
|
||||
|
||||
Both run in CI — `.gitea/workflows/ci.yaml`, one job, on pushes to `main` and
|
||||
on PRs that were non-draft when the run was created. It installs shellcheck
|
||||
and a stable Rust toolchain and references no credentials; the scratch
|
||||
and both a stable and a nightly Rust toolchain (nightly for
|
||||
`-Z checksum-freshness`, without which `hardlink-clone-selftest.sh` skips the
|
||||
two assertions that cover the silent-stale-reuse hazard) and references no
|
||||
credentials; the scratch
|
||||
workspaces the compiler-backed suites build use path dependencies only, so
|
||||
nothing reaches crates.io. It runs the full suite rather than `--fast`,
|
||||
because the two compiler-backed suites are the ones that check this scheme
|
||||
@@ -559,7 +562,7 @@ change here reaches all of them at once. That is what the gate is for.
|
||||
| suite | covers |
|
||||
|---|---|
|
||||
| `cache-root-selftest.sh` | that a lineage nests one level and nothing else moves: no lineage resolves byte-for-byte to the cache root, two lineages on one cache key get disjoint target dirs, seed/publish/prune all stay inside their own lineage, a PR layers over its own lineage's base snapshot — **and one rejection per lineage name a reader elsewhere would stop seeing**, plus the publish-side mismatch guard |
|
||||
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler. |
|
||||
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly for two of its five assertions**: the checksum-freshness dep-info file is the mutation that turns a hardlink clone into silent stale-artifact reuse, and it only exists under `-Z checksum-freshness`. Without one those two are skipped, loudly. |
|
||||
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published, and the publisher's log showing it waited on the consumer's own reader-lock marker before reclaiming a rotated snapshot |
|
||||
| `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone |
|
||||
| `prune-cache-selftest.sh` | liveness, protection, locking, eviction order, self-clear, **and that a cache a job claims *inside* the check-to-unlink window survives it** — against a real scratch `origin` |
|
||||
|
||||
@@ -73,11 +73,33 @@ mkcrate "$crate_dir"
|
||||
cd "$crate_dir"
|
||||
|
||||
export CARGO_INCREMENTAL=0
|
||||
|
||||
# Every assertion below reads cargo's own words out of a build log
|
||||
# (`Compiling libdep`, `Fresh probe`). A CI image that forces colour splices an
|
||||
# ANSI reset between the status word and the crate name, at which point every
|
||||
# one of those greps silently stops matching and the suite reports the
|
||||
# opposite of what happened — observed on gitdan-ci's runner image, where
|
||||
# scenario 2 failed while the log it printed plainly showed `Compiling libdep`.
|
||||
# Pin the format the assertions are written against.
|
||||
export CARGO_TERM_COLOR=never
|
||||
# Checksum freshness is where the worst failure lives (the dep-* file carries
|
||||
# per-source checksums and is rewritten in place). Only available on nightly;
|
||||
# without it the test still covers the build/ and *.d families.
|
||||
CHECKSUM_MODE="off"
|
||||
if cargo +nightly -V >/dev/null 2>&1; then
|
||||
# Probe the CAPABILITY, not the channel. `cargo +nightly -V` answers "did a
|
||||
# cargo proxy called with +nightly exit 0", which is a different question from
|
||||
# "will this build have checksum freshness" — `-V` short-circuits before `-Z`
|
||||
# is validated at all, so that probe says yes on any cargo that resolves the
|
||||
# name, including one whose nightly has since moved the flag. The scenario
|
||||
# below then asserts the checksum-freshness mutation family against a build
|
||||
# that never had it, and fails in the CONTROL, where a failure reads as "the
|
||||
# hazard is gone" rather than "the toolchain is wrong". Observed the first
|
||||
# time this suite ran on gitdan-ci.
|
||||
#
|
||||
# `-Z <flag> locate-project` is the narrowest command that actually parses the
|
||||
# flag: it rejects the stable channel and an unknown flag name alike, needs no
|
||||
# network, and builds nothing.
|
||||
if cargo +nightly -Z checksum-freshness locate-project >/dev/null 2>&1; then
|
||||
export CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true
|
||||
CARGO_BIN=(cargo +nightly)
|
||||
CHECKSUM_MODE="on"
|
||||
@@ -161,10 +183,20 @@ fi
|
||||
ok "no file in the source changed after a full rebuild in the clone"
|
||||
|
||||
echo
|
||||
if [ "$CHECKSUM_MODE" = "on" ]; then
|
||||
echo "=== the whole point: the source's next build is still correct ==="
|
||||
# The source's cache holds artifacts built from CONTENT_A. Advance the source
|
||||
# to CONTENT_B (as a merge would) and rebuild in it. If the clone had
|
||||
# The source's cache holds artifacts built from CONTENT_A. Advance the
|
||||
# source to CONTENT_B (as a merge would) and rebuild in it. If the clone had
|
||||
# corrupted its dep-info, Cargo would report Fresh and keep the stale rlib.
|
||||
#
|
||||
# CHECKSUM-FRESHNESS ONLY, and the backdated mtime is why. Under checksum
|
||||
# freshness the dep-info file's per-source checksums decide, so a 2001
|
||||
# timestamp on changed content must still rebuild — the assertion below.
|
||||
# Under Cargo's ordinary MTIME freshness the same timestamp means the source
|
||||
# is older than the artifact, and reporting Fresh is the correct answer;
|
||||
# asserting otherwise asserts a bug. This scenario was written against a
|
||||
# machine with a nightly installed and, run without one, failed on that
|
||||
# correct answer.
|
||||
printf '%s\n' "$CONTENT_B" > src/lib.rs
|
||||
touch -d '@1000000000' src/lib.rs
|
||||
log="$scratch/rebuild.log"
|
||||
@@ -173,6 +205,10 @@ if grep -qE '^\s+Fresh probe' "$log"; then
|
||||
fail "source declared its own crate Fresh against sources it has never built — stale-artifact reuse"
|
||||
fi
|
||||
ok "source correctly rebuilt its crate after advancing to the clone's content"
|
||||
else
|
||||
echo "=== skipped: the source's-next-build scenario needs checksum freshness ==="
|
||||
echo " (a nightly cargo accepting -Z checksum-freshness; see the probe above)"
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "hardlink-clone-selftest: ${pass_count} assertions passed"
|
||||
|
||||
@@ -75,6 +75,15 @@
|
||||
# Asserts BOTH jobs correctly recompile the dependency and succeed.
|
||||
set -euo pipefail
|
||||
|
||||
# Every assertion below reads cargo's own words out of a build log
|
||||
# (`Compiling libdep`, `Fresh probe`). A CI image that forces colour splices an
|
||||
# ANSI reset between the status word and the crate name, at which point every
|
||||
# one of those greps silently stops matching and the suite reports the
|
||||
# opposite of what happened — observed on gitdan-ci's runner image, where
|
||||
# scenario 2 failed while the log it printed plainly showed `Compiling libdep`.
|
||||
# Pin the format the assertions are written against.
|
||||
export CARGO_TERM_COLOR=never
|
||||
|
||||
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
||||
restore_mtimes="$script_dir/restore-mtimes.sh"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user