claude
fb3c72aa28
CI / shellcheck + selftests (pull_request) Skipped
The comments said hardlink count at link time was ruled out as the discriminator between a rewritten executable and an intact one, citing a lib+bin crate whose two test binaries were both `nlink == 1` and appeared to behave differently. Re-checked on review: the intact one had not been rebuilt at all — same content, same inode — so it demonstrated nothing, and forcing both to rebuild rewrote both. What was actually observed is narrower and now says so: every executable measured intact had an uplift hardlink twin Cargo must re-create anyway, every one measured rewritten had none, and whether the twin is the mechanism or a correlate was not determined. The rule does not rest on the answer — exempting twinned executables would recover none of the bytes this change newly copies. This PR exists because a claim outlived its evidence; it should not ship one.
691 lines
36 KiB
Bash
Executable File
691 lines
36 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Regression test for the single assumption this whole caching scheme rests
|
|
# on: that a build running inside a hardlink clone cannot mutate the directory
|
|
# it was cloned from.
|
|
#
|
|
# That assumption is FALSE for a plain `cp -al`. Measured, and asserted below
|
|
# as an explicit control: build in a raw `cp -al` clone and the source's
|
|
# dep-info file (`.fingerprint/<unit>/dep-*` under Cargo's build-dir layout
|
|
# v1, `build/<pkg>/<hash>/fingerprint/dep-*` under v2 — and under content
|
|
# freshness only, see the probe below), `build/<pkg>/output`,
|
|
# `build/<pkg>/out/**` and `deps/*.d` all change,
|
|
# because Cargo and build scripts write those with a plain truncating write
|
|
# rather than the write-then-rename Cargo uses for real artifacts.
|
|
#
|
|
# The consequence is not a slow build, it is a wrong one: a PR clone rewrites
|
|
# the base's dep-info to describe the PR's sources while the base's cache
|
|
# still holds the artifact built from the base's sources; once the PR merges,
|
|
# the base's next run finds the checksums match its (now merged) sources,
|
|
# reports `Fresh`, and links a binary built from the pre-merge code.
|
|
#
|
|
# cache-lib.sh's unshare_mutable_paths() is what closes that, and this test is
|
|
# what proves it stays closed. The control matters as much as the fix: a
|
|
# scenario that passes for both would prove nothing.
|
|
#
|
|
# Needs a working cargo on PATH. Everything happens under a mktemp -d scratch
|
|
# tree. Run by hand: bash scripts/hardlink-clone-selftest.sh
|
|
set -euo pipefail
|
|
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
|
. "$script_dir/cache-lib.sh"
|
|
|
|
scratch=$(mktemp -d)
|
|
trap 'rm -rf "$scratch"' EXIT
|
|
pass_count=0
|
|
|
|
fail() { echo "ASSERTION FAILED: $*" >&2; exit 1; }
|
|
ok() { pass_count=$((pass_count + 1)); echo "PASS: $*"; }
|
|
|
|
# Content hash of every file in a tree, keyed by relative path.
|
|
snapshot_tree() { (cd "$1" && find . -type f -print0 | sort -z | xargs -0 -r sha1sum) 2>/dev/null; }
|
|
|
|
# `diff` exits 1 when the trees differ, which is the expected case here and
|
|
# must not trip `pipefail` — the difference IS the result.
|
|
mutated_paths() {
|
|
{ diff <(printf '%s' "$1") <(printf '%s' "$2") || true; } 2>/dev/null \
|
|
| awk '/^[<>]/ { print $3 }' | sort -u
|
|
}
|
|
|
|
# A crate with a build script, because build-script OUT_DIR writes are one of
|
|
# the two mutation families and are invisible without one.
|
|
mkcrate() {
|
|
local dir="$1"
|
|
mkdir -p "$dir/src"
|
|
cat > "$dir/Cargo.toml" <<'TOML'
|
|
[package]
|
|
name = "probe"
|
|
version = "0.1.0"
|
|
edition = "2021"
|
|
[workspace]
|
|
TOML
|
|
# A binary as well as a library, because the two are written differently and
|
|
# only one of them is safe to share: rustc writes an rlib to a temporary and
|
|
# renames it into place, while the LINKER writes an executable through the
|
|
# existing inode. Without a bin target this suite never relinks anything and
|
|
# cannot see that difference.
|
|
cat > "$dir/src/main.rs" <<'RS'
|
|
fn main() { println!("{}", probe::f()); }
|
|
RS
|
|
cat > "$dir/build.rs" <<'RS'
|
|
use std::{env, fs, path::PathBuf};
|
|
fn main() {
|
|
println!("cargo::rerun-if-changed=src/lib.rs");
|
|
let out = PathBuf::from(env::var("OUT_DIR").unwrap());
|
|
let src = fs::read_to_string("src/lib.rs").unwrap();
|
|
fs::write(out.join("gen.txt"), format!("generated from {} bytes", src.len())).unwrap();
|
|
}
|
|
RS
|
|
}
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Both build-dir layouts, without a compiler
|
|
# ---------------------------------------------------------------------------
|
|
#
|
|
# Every other scenario in this file runs whichever layout the installed Cargo
|
|
# happens to write, so on any one machine it exercises exactly ONE of the two —
|
|
# and gitdan-ci's is v1. The two fixtures below reproduce both directory shapes
|
|
# from files alone, clone them through the real `hardlink_clone_into()`, and
|
|
# assert file by file which side of the partition each one lands on.
|
|
#
|
|
# Shapes taken from a scratch crate (serde + serde_json + regex, plus a build
|
|
# script) built on 2026-08-27: cargo 1.98.0-nightly (a335d47ff 2026-06-26)
|
|
# writes v1, cargo 1.100.0-nightly (e8cb624d5 2026-08-22) writes v2.
|
|
#
|
|
# v2 is where the partition is easy to get wrong, and the fixtures are built to
|
|
# say so: a build script's OUT_DIR and a compile unit's rlib are BOTH a
|
|
# directory called `out`, one directory apart, and they need opposite
|
|
# treatment.
|
|
mkfile() { mkdir -p "$(dirname "$1")"; printf '%s' "$2" > "$1"; }
|
|
|
|
# Big enough that the byte-fraction assertion below measures something.
|
|
artifact_bytes=$(head -c 4096 /dev/zero | tr '\0' 'A')
|
|
|
|
# Spec lines are `<shared|private>|<path relative to the tree root>`.
|
|
assert_partition() {
|
|
local label="$1" src="$2" spec="$3"
|
|
local clone="${src}-clone" want rel si ci total=0 copied=0 sz
|
|
hardlink_clone_into "$src" "$clone" "selftest-$label" \
|
|
|| fail "$label: hardlink_clone_into refused the destination"
|
|
while IFS='|' read -r want rel; do
|
|
[ -n "${rel:-}" ] || continue
|
|
[ -e "$clone/$rel" ] || fail "$label: $rel is missing from the clone"
|
|
si=$(stat -c '%i' "$src/$rel"); ci=$(stat -c '%i' "$clone/$rel")
|
|
case "$want" in
|
|
shared)
|
|
[ "$si" = "$ci" ] \
|
|
|| fail "$label: $rel was real-copied, but it is an artifact and must stay shared" ;;
|
|
private)
|
|
[ "$si" != "$ci" ] \
|
|
|| fail "$label: $rel still shares an inode with the source, so a build in the clone can rewrite it" ;;
|
|
*) fail "$label: unknown spec verb '$want'" ;;
|
|
esac
|
|
done <<< "$spec"
|
|
ok "$label: every file landed on the right side of the partition"
|
|
|
|
# The cost model, asserted rather than assumed. Selecting too much is not a
|
|
# correctness bug, which is exactly why nothing caught layout v2 taking the
|
|
# selection from 39.3% of one scratch crate's tree to 99.996% of it
|
|
# (gitdan-actions#14):
|
|
# a hardlink clone that real-copies everything is a `cp -a` with extra steps.
|
|
# The bound is loose on purpose. It is not a budget — the honest figure moves
|
|
# with how much of a tree is linker output, and these fixtures are mostly
|
|
# that by construction — it is a floor under "still a hardlink clone at all".
|
|
while IFS= read -r rel; do
|
|
# `.cargo-*lock*` is stripped from every clone by design, so it has no
|
|
# counterpart to compare against.
|
|
[ -e "$clone/$rel" ] || continue
|
|
sz=$(stat -c '%s' "$src/$rel")
|
|
total=$((total + sz))
|
|
[ "$(stat -c '%i' "$src/$rel")" = "$(stat -c '%i' "$clone/$rel")" ] || copied=$((copied + sz))
|
|
done < <(cd "$src" && find . -type f -printf '%P\n')
|
|
[ "$total" -gt 0 ] || fail "$label: fixture has no bytes to measure"
|
|
[ $((copied * 100 / total)) -lt 90 ] \
|
|
|| fail "$label: the clone real-copied $((copied * 100 / total))% of its source's bytes — the hardlink saving is gone"
|
|
ok "$label: clone real-copies $((copied * 100 / total))% of ${total} B (${copied} B), the rest is shared"
|
|
}
|
|
|
|
# Layout v2: no `.fingerprint`, no `deps`. Everything regroups per build unit
|
|
# under `build/<pkg>/<hash>/{fingerprint,out,run}`, artifacts included — which
|
|
# is what took `-name build` from "the metadata" to "the whole tree".
|
|
#
|
|
# THE THREE UNIT KINDS ARE THE POINT. `1bf...` is a compile unit: its `out`
|
|
# holds the rlib. `08c...` is the build script's own compile unit: its `out`
|
|
# holds the build-script binary. `da9...` is the build-script RUN unit: its
|
|
# `out` IS the OUT_DIR, and it is the only one of the three whose `out` is
|
|
# mutable. The `run/` directory beside it is the structural difference.
|
|
v2="$scratch/layout-v2"
|
|
mkfile "$v2/CACHEDIR.TAG" 'Signature: 8a477f597d28d172'
|
|
mkfile "$v2/.rustc_info.json" '{"rustc_fingerprint":1}'
|
|
mkfile "$v2/debug/.cargo-lock" ''
|
|
mkfile "$v2/debug/libprobe.rlib" "$artifact_bytes"
|
|
mkfile "$v2/debug/libprobe.d" '/probe/src/lib.rs:'
|
|
for f in dep-lib-probe lib-probe lib-probe.json invoked.timestamp; do
|
|
mkfile "$v2/debug/build/probe/1bf5493368dce3cd/fingerprint/$f" "$f"
|
|
done
|
|
mkfile "$v2/debug/build/probe/1bf5493368dce3cd/out/libprobe-1bf5493368dce3cd.rlib" "$artifact_bytes"
|
|
mkfile "$v2/debug/build/probe/1bf5493368dce3cd/out/libprobe-1bf5493368dce3cd.rmeta" "$artifact_bytes"
|
|
mkfile "$v2/debug/build/probe/1bf5493368dce3cd/out/probe-1bf5493368dce3cd.d" '/probe/src/lib.rs:'
|
|
for f in build-script-build-script-build build-script-build-script-build.json \
|
|
dep-build-script-build-script-build invoked.timestamp; do
|
|
mkfile "$v2/debug/build/probe/08c7dda6eacd6dca/fingerprint/$f" "$f"
|
|
done
|
|
mkfile "$v2/debug/build/probe/08c7dda6eacd6dca/out/build_script_build" "$artifact_bytes"
|
|
chmod +x "$v2/debug/build/probe/08c7dda6eacd6dca/out/build_script_build"
|
|
mkfile "$v2/debug/build/probe/08c7dda6eacd6dca/out/build_script_build.d" '/probe/build.rs:'
|
|
# A test binary: the same `out` directory as the rlib above, and the largest
|
|
# thing in a real tree that a linker writes.
|
|
for f in dep-test-lib-probe test-lib-probe test-lib-probe.json invoked.timestamp; do
|
|
mkfile "$v2/debug/build/probe/6a091d813b2be60d/fingerprint/$f" "$f"
|
|
done
|
|
mkfile "$v2/debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d" "$artifact_bytes"
|
|
chmod +x "$v2/debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d"
|
|
mkfile "$v2/debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d.d" '/probe/src/lib.rs:'
|
|
for f in run-build-script-build-script-build run-build-script-build-script-build.json; do
|
|
mkfile "$v2/debug/build/probe/da96cf45111f80dd/fingerprint/$f" "$f"
|
|
done
|
|
mkfile "$v2/debug/build/probe/da96cf45111f80dd/out/gen.txt" 'generated from 24 bytes'
|
|
for f in invoked.timestamp root-output stdout stderr; do
|
|
mkfile "$v2/debug/build/probe/da96cf45111f80dd/run/$f" "$f"
|
|
done
|
|
# A build script that wrote into OUT_DIR and then FAILED: Cargo records the
|
|
# run only on success, so this unit has `out/` populated and no `run/` at all.
|
|
# Reading "no execution record" as "compile unit" left this shared, which is
|
|
# the one state the old `-name build` selection covered and the first cut of
|
|
# this one did not.
|
|
mkfile "$v2/debug/build/probe/f00ded00f00ded00/out/gen.txt" 'half-written'
|
|
mkfile "$v2/debug/incremental/probe-abc/s-xyz/dep-graph.bin" "$artifact_bytes"
|
|
|
|
assert_partition "layout v2" "$v2" "$(cat <<'SPEC'
|
|
private|.rustc_info.json
|
|
private|debug/libprobe.d
|
|
private|debug/build/probe/1bf5493368dce3cd/fingerprint/dep-lib-probe
|
|
private|debug/build/probe/1bf5493368dce3cd/fingerprint/lib-probe
|
|
private|debug/build/probe/1bf5493368dce3cd/fingerprint/lib-probe.json
|
|
private|debug/build/probe/1bf5493368dce3cd/fingerprint/invoked.timestamp
|
|
private|debug/build/probe/1bf5493368dce3cd/out/probe-1bf5493368dce3cd.d
|
|
private|debug/build/probe/08c7dda6eacd6dca/fingerprint/dep-build-script-build-script-build
|
|
private|debug/build/probe/08c7dda6eacd6dca/fingerprint/invoked.timestamp
|
|
private|debug/build/probe/08c7dda6eacd6dca/out/build_script_build.d
|
|
private|debug/build/probe/da96cf45111f80dd/fingerprint/run-build-script-build-script-build
|
|
private|debug/build/probe/da96cf45111f80dd/out/gen.txt
|
|
private|debug/build/probe/da96cf45111f80dd/run/root-output
|
|
private|debug/build/probe/da96cf45111f80dd/run/stdout
|
|
private|debug/build/probe/da96cf45111f80dd/run/stderr
|
|
private|debug/build/probe/da96cf45111f80dd/run/invoked.timestamp
|
|
private|debug/build/probe/f00ded00f00ded00/out/gen.txt
|
|
shared|debug/libprobe.rlib
|
|
shared|debug/build/probe/1bf5493368dce3cd/out/libprobe-1bf5493368dce3cd.rlib
|
|
shared|debug/build/probe/1bf5493368dce3cd/out/libprobe-1bf5493368dce3cd.rmeta
|
|
private|debug/build/probe/08c7dda6eacd6dca/out/build_script_build
|
|
private|debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d
|
|
private|debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d.d
|
|
private|debug/build/probe/6a091d813b2be60d/fingerprint/dep-test-lib-probe
|
|
shared|debug/incremental/probe-abc/s-xyz/dep-graph.bin
|
|
SPEC
|
|
)"
|
|
# Layout v1: one `.fingerprint` and one `deps` per profile; `build/<pkg>-<hash>`
|
|
# holds the build script's compiled binary in one unit directory and its run
|
|
# metadata plus OUT_DIR in another.
|
|
v1="$scratch/layout-v1"
|
|
mkfile "$v1/CACHEDIR.TAG" 'Signature: 8a477f597d28d172'
|
|
mkfile "$v1/.rustc_info.json" '{"rustc_fingerprint":1}'
|
|
mkfile "$v1/debug/.cargo-lock" ''
|
|
mkfile "$v1/debug/libprobe.rlib" "$artifact_bytes"
|
|
mkfile "$v1/debug/libprobe.d" '/probe/src/lib.rs:'
|
|
mkfile "$v1/debug/deps/libprobe-1bf5493368dce3cd.rlib" "$artifact_bytes"
|
|
mkfile "$v1/debug/deps/libprobe-1bf5493368dce3cd.rmeta" "$artifact_bytes"
|
|
mkfile "$v1/debug/deps/probe-1bf5493368dce3cd.d" '/probe/src/lib.rs:'
|
|
# A test binary, which layout v1 leaves in `deps/` beside the rlibs.
|
|
mkfile "$v1/debug/deps/probe-6a091d813b2be60d" "$artifact_bytes"
|
|
chmod +x "$v1/debug/deps/probe-6a091d813b2be60d"
|
|
mkfile "$v1/debug/deps/probe-6a091d813b2be60d.d" '/probe/src/lib.rs:'
|
|
for f in dep-lib-probe lib-probe lib-probe.json invoked.timestamp; do
|
|
mkfile "$v1/debug/.fingerprint/probe-1bf5493368dce3cd/$f" "$f"
|
|
done
|
|
mkfile "$v1/debug/build/probe-08c7dda6eacd6dca/build-script-build" "$artifact_bytes"
|
|
mkfile "$v1/debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca" "$artifact_bytes"
|
|
chmod +x "$v1/debug/build/probe-08c7dda6eacd6dca/build-script-build" \
|
|
"$v1/debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca"
|
|
mkfile "$v1/debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca.d" '/probe/build.rs:'
|
|
for f in invoked.timestamp output root-output stderr; do
|
|
mkfile "$v1/debug/build/probe-da96cf45111f80dd/$f" "$f"
|
|
done
|
|
mkfile "$v1/debug/build/probe-da96cf45111f80dd/out/gen.txt" 'generated from 24 bytes'
|
|
# The same never-succeeded build script under layout v1.
|
|
mkfile "$v1/debug/build/probe-f00ded00f00ded00/out/gen.txt" 'half-written'
|
|
mkfile "$v1/debug/incremental/probe-abc/s-xyz/dep-graph.bin" "$artifact_bytes"
|
|
|
|
assert_partition "layout v1" "$v1" "$(cat <<'SPEC'
|
|
private|.rustc_info.json
|
|
private|debug/libprobe.d
|
|
private|debug/deps/probe-1bf5493368dce3cd.d
|
|
private|debug/.fingerprint/probe-1bf5493368dce3cd/dep-lib-probe
|
|
private|debug/.fingerprint/probe-1bf5493368dce3cd/lib-probe
|
|
private|debug/.fingerprint/probe-1bf5493368dce3cd/lib-probe.json
|
|
private|debug/.fingerprint/probe-1bf5493368dce3cd/invoked.timestamp
|
|
private|debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca.d
|
|
private|debug/deps/probe-6a091d813b2be60d
|
|
private|debug/deps/probe-6a091d813b2be60d.d
|
|
private|debug/build/probe-da96cf45111f80dd/invoked.timestamp
|
|
private|debug/build/probe-da96cf45111f80dd/output
|
|
private|debug/build/probe-da96cf45111f80dd/root-output
|
|
private|debug/build/probe-da96cf45111f80dd/stderr
|
|
private|debug/build/probe-da96cf45111f80dd/out/gen.txt
|
|
private|debug/build/probe-f00ded00f00ded00/out/gen.txt
|
|
shared|debug/libprobe.rlib
|
|
shared|debug/deps/libprobe-1bf5493368dce3cd.rlib
|
|
shared|debug/deps/libprobe-1bf5493368dce3cd.rmeta
|
|
private|debug/build/probe-08c7dda6eacd6dca/build-script-build
|
|
private|debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca
|
|
shared|debug/incremental/probe-abc/s-xyz/dep-graph.bin
|
|
SPEC
|
|
)"
|
|
|
|
|
|
echo
|
|
command -v cargo >/dev/null || {
|
|
echo "SKIP: no cargo on PATH — the fixture scenarios above ran, the live-Cargo ones cannot"
|
|
echo "hardlink-clone-selftest: ${pass_count} assertions passed"
|
|
exit 0
|
|
}
|
|
|
|
crate_dir="$scratch/probe"
|
|
mkcrate "$crate_dir"
|
|
cd "$crate_dir"
|
|
|
|
export CARGO_INCREMENTAL=0
|
|
|
|
# Every assertion below reads cargo's own words out of a build log
|
|
# (`Compiling libdep`, `Fresh probe`). A CI image that forces colour splices an
|
|
# ANSI reset between the status word and the crate name, at which point every
|
|
# one of those greps silently stops matching and the suite reports the
|
|
# opposite of what happened — observed on gitdan-ci's runner image, where
|
|
# scenario 2 failed while the log it printed plainly showed `Compiling libdep`.
|
|
# Pin the format the assertions are written against.
|
|
export CARGO_TERM_COLOR=never
|
|
# Checksum freshness is where the worst failure lives (the dep-* file carries
|
|
# per-source checksums and is rewritten in place). Only available on nightly;
|
|
# without it the test still covers the build/ and *.d families.
|
|
CONTENT_A='pub fn f() -> u32 { 1 }'
|
|
CONTENT_B='pub fn f() -> u32 { 22222 } pub fn g() -> u32 { 7 }'
|
|
|
|
# Probe the BEHAVIOUR, not the channel and not the flag. Two weaker probes
|
|
# were tried against gitdan-ci's runner and each let the suite assert a
|
|
# property the toolchain did not have:
|
|
#
|
|
# `cargo +nightly -V` — answers "did a proxy called with
|
|
# +nightly exit 0". `-V`
|
|
# short-circuits before `-Z` is
|
|
# even parsed.
|
|
# `cargo +nightly -Z checksum-freshness — answers "is this flag still
|
|
# locate-project` accepted", which since cargo PR
|
|
# #17382 (2026-08-22) is a
|
|
# different question from "is
|
|
# content freshness on". That PR
|
|
# demoted the flag to a gate and
|
|
# gave `build.fingerprint` the
|
|
# choice, defaulting to `mtime` —
|
|
# so 1.100.0-nightly accepts the
|
|
# flag and resolves freshness by
|
|
# mtime unless
|
|
# CARGO_BUILD_FINGERPRINT=content
|
|
# is set too. Measured 2026-08-26;
|
|
# see daniel/gitdan#62.
|
|
#
|
|
# The scenario at the end of this file depends on one thing and it is neither
|
|
# of those: that changed content with an OLDER mtime rebuilds. Under mtime
|
|
# freshness the correct answer is Fresh, so under mtime freshness that
|
|
# scenario asserts a bug. So the probe simply performs that experiment, on its
|
|
# own crate and its own target dir, with no clone anywhere near it — which is
|
|
# also what makes it a control rather than a restatement of the scenario: the
|
|
# probe establishes that the toolchain rebuilds on content, the scenario
|
|
# establishes that a hardlink clone did not take that away.
|
|
# THREE OUTCOMES, NOT TWO. An experiment that cannot tell a negative result
|
|
# from a failed measurement is not settling the question, and the two are not
|
|
# interchangeable here: "this toolchain resolves freshness by mtime" is a
|
|
# statement about Cargo, while "a probe build failed" is a statement about this
|
|
# machine. Collapsing them — which an earlier cut of this did, by returning
|
|
# non-zero for both — makes a half-installed toolchain print a confident and
|
|
# wrong explanation and quietly drop a scenario. The scenario still has to be
|
|
# skipped in either case; what must not happen is the log claiming to know why.
|
|
#
|
|
# 0 content freshness measured ACTIVE — both builds ran, the backdated
|
|
# rebuild recompiled
|
|
# 3 measured INACTIVE — both builds ran, the backdated
|
|
# rebuild reported Fresh
|
|
# anything else NOT MEASURED — nothing was learned about the
|
|
# toolchain
|
|
#
|
|
# THE ANSWER CODES ARE 0 AND 3, AND THE GAP IS THE MECHANISM. Bash produces 1
|
|
# for an ordinary command failure, 2 for a usage error, 126/127 for a command
|
|
# it could not run, 128+n for a signal, and — this is the one that matters —
|
|
# 1 for an unbound-variable or other EXPANSION failure, which happens before
|
|
# the command runs and is therefore invisible to a `||` guard and to an ERR
|
|
# trap alike. It never produces 3. So "not an answer code" is decided by a
|
|
# property of the shell rather than by an enumeration of the ways a step can
|
|
# go wrong, and a step added later without a guard, or with a guard that
|
|
# cannot fire, lands on NOT MEASURED by construction.
|
|
#
|
|
# That is the whole reason INACTIVE is not 1. It was, and three review rounds
|
|
# on this function each found a narrower way for a shell-generated 1 to be read
|
|
# as a measurement — an unguarded command, then a typo'd variable name on a
|
|
# line that HAS its guard. Each was closed by narrowing the failure surface,
|
|
# which is a game with no last move. Moving the answer off the codes bash can
|
|
# generate ends it instead: there is no longer a mutation that turns an error
|
|
# into an answer, only mutations that turn an error into a different error.
|
|
#
|
|
# The guards below stay, and so does the trap, but their job is now reporting
|
|
# rather than correctness: they make a failed step land on 2 with its logs
|
|
# printed instead of on some incidental status, which is nicer to debug and
|
|
# lands in the same place either way.
|
|
#
|
|
# One piece of that reporting layer is load-bearing and not obvious. A command
|
|
# on the left of `||` — or in an `if` condition — runs with errexit suppressed,
|
|
# and that suppression propagates into a subshell and is NOT undone by a
|
|
# `set -e` inside it (measured on bash 5.3: an unguarded `false` there falls
|
|
# through to `exit 0`). Calling with errexit disarmed at the site is the only
|
|
# form that lets the subshell re-arm it; hence the `set +e` bracket. The ERR
|
|
# trap is then required on top, because a bare `set -e` abort exits with the
|
|
# FAILING COMMAND's status, and `false` gives 1.
|
|
#
|
|
# WHY NOT-MEASURED SKIPS RATHER THAN FAILS. The scenario it gates is the only thing in
|
|
# this suite that depends on freshness mode; everything else still runs and
|
|
# still catches real regressions. Failing instead would turn a statement about
|
|
# one machine's toolchain into a red gate reading "the hardlink scheme is
|
|
# broken" across the three repos consuming this action — the same category
|
|
# error the three-state split exists to prevent, one level up. What would
|
|
# change the answer is not-measured becoming the everyday CI outcome; it is not
|
|
# — gitdan-ci's outcome is a measurement either way. It reported a measured
|
|
# INACTIVE until 2026-08-26, for the reason recorded at the `export` below, and
|
|
# an ACTIVE once both switches were set.
|
|
CHECKSUM_MODE="off"
|
|
CHECKSUM_REASON="no nightly on PATH accepting -Z checksum-freshness"
|
|
CARGO_BIN=(cargo)
|
|
checksum_freshness_probe() {
|
|
local d="$scratch/freshness-probe" t="$scratch/freshness-probe-target"
|
|
mkcrate "$d" || return 2 # 2 is simply "not 0 and not 3"; see the header
|
|
(
|
|
set -e
|
|
trap 'exit 2' ERR
|
|
cd "$d" || exit 2
|
|
printf '%s\n' "$CONTENT_A" > src/lib.rs || exit 2
|
|
CARGO_TARGET_DIR="$t" cargo +nightly build -q > "$scratch/freshness-probe-warm.log" 2>&1 || exit 2
|
|
printf '%s\n' "$CONTENT_B" > src/lib.rs || exit 2
|
|
touch -d '@1000000000' src/lib.rs || exit 2
|
|
CARGO_TARGET_DIR="$t" cargo +nightly build -v > "$scratch/freshness-probe.log" 2>&1 || exit 2
|
|
# 3, not 1: see the header. This is the only statement in the subshell that
|
|
# may report a measurement, and it is the only one that may exit 3.
|
|
if grep -qE '^\s+Fresh probe' "$scratch/freshness-probe.log"; then exit 3; fi
|
|
exit 0
|
|
)
|
|
}
|
|
if cargo +nightly -Z checksum-freshness locate-project > /dev/null 2>&1; then
|
|
export CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true
|
|
# BOTH, since cargo PR #17382 (2026-08-22): the -Z flag only unlocks the
|
|
# feature and `build.fingerprint` selects it, defaulting to `mtime`. Setting
|
|
# the gate alone is what made this suite report a measured INACTIVE on
|
|
# 1.100.0-nightly and skip its strongest scenario (daniel/gitdan#62). Safe to
|
|
# export unconditionally — a Cargo that does not know the key ignores it
|
|
# silently, verified 2026-08-26 on 1.93.1 stable and 1.96.0-nightly, both of
|
|
# which still measure ACTIVE from the gate alone.
|
|
export CARGO_BUILD_FINGERPRINT=content
|
|
# Errexit off across the call, so the subshell can arm its own — see the
|
|
# header. `probe_rc` is read before it is restored.
|
|
probe_rc=0
|
|
set +e
|
|
checksum_freshness_probe
|
|
probe_rc=$?
|
|
set -e
|
|
case "$probe_rc" in
|
|
0)
|
|
CARGO_BIN=(cargo +nightly)
|
|
CHECKSUM_MODE="on"
|
|
CHECKSUM_REASON=""
|
|
;;
|
|
3)
|
|
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS CARGO_BUILD_FINGERPRINT
|
|
CHECKSUM_REASON="this nightly accepts -Z checksum-freshness and build.fingerprint=content but still resolves freshness by mtime"
|
|
;;
|
|
*)
|
|
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS CARGO_BUILD_FINGERPRINT
|
|
CHECKSUM_MODE="unmeasured"
|
|
CHECKSUM_REASON="the probe exited ${probe_rc}, which is not one of its answer codes, so this was NOT MEASURED — this toolchain may or may not resolve freshness by content"
|
|
# Loud, because the cost is silently lost coverage on a machine that
|
|
# might have had it. The suite continues: everything else it asserts is
|
|
# independent of freshness mode.
|
|
echo "::warning::hardlink-clone-selftest: could not measure whether this toolchain resolves freshness by content — the probe exited ${probe_rc}. This is a failure to measure, not a finding about Cargo."
|
|
tail -n 15 "$scratch/freshness-probe-warm.log" "$scratch/freshness-probe.log" 2>/dev/null | sed 's/^/ /' >&2 || true
|
|
;;
|
|
esac
|
|
fi
|
|
cd "$crate_dir"
|
|
echo "=== checksum-freshness mode: ${CHECKSUM_MODE}${CHECKSUM_REASON:+ — ${CHECKSUM_REASON}} ==="
|
|
|
|
build_base() {
|
|
local dir="$1"
|
|
printf '%s\n' "$CONTENT_A" > src/lib.rs
|
|
CARGO_TARGET_DIR="$dir" "${CARGO_BIN[@]}" build -q
|
|
}
|
|
|
|
echo
|
|
echo "=== control: a raw \`cp -al\` clone DOES mutate its source ==="
|
|
base_ctl="$scratch/base-ctl"; clone_ctl="$scratch/clone-ctl"
|
|
build_base "$base_ctl"
|
|
before=$(snapshot_tree "$base_ctl")
|
|
cp -al "$base_ctl" "$clone_ctl"
|
|
strip_cargo_locks "$clone_ctl" # locks alone are not the hazard under test
|
|
printf '%s\n' "$CONTENT_B" > src/lib.rs
|
|
CARGO_TARGET_DIR="$clone_ctl" "${CARGO_BIN[@]}" build -q
|
|
after=$(snapshot_tree "$base_ctl")
|
|
ctl_mutated=$(mutated_paths "$before" "$after")
|
|
if [ -z "$ctl_mutated" ]; then
|
|
fail "control produced no mutation — the test can no longer distinguish fixed from broken"
|
|
fi
|
|
ok "raw cp -al clone mutates the source ($(printf '%s\n' "$ctl_mutated" | wc -l) paths)"
|
|
printf '%s\n' "$ctl_mutated" | sed 's/^/ /'
|
|
|
|
# Reported, not asserted, and the distinction is the point. The control's job
|
|
# is to prove the hazard exists at all, which the non-empty set above already
|
|
# does; this line records WHICH families a given Cargo exhibits.
|
|
#
|
|
# The dep-info file is the worst of them — it carries the per-source
|
|
# checksums, so mutating it through a shared inode turns a hardlink clone into
|
|
# silent stale-artifact reuse rather than a slow build. Failing on its absence
|
|
# would mean this suite goes red whenever upstream stops doing something we
|
|
# never wanted it to do — and it would go red in the CONTROL, where a failure
|
|
# reads as "the hazard is gone" rather than "upstream changed". Nothing is lost
|
|
# by reporting it: the fix scenario below asserts the source is byte-identical
|
|
# after a full rebuild in the clone, which covers every family this Cargo has,
|
|
# named or not.
|
|
#
|
|
# THE PATTERN MUST MATCH BOTH LAYOUTS, and that is not a detail. Cargo's
|
|
# build-dir layout v2 moved the file from `<profile>/.fingerprint/<unit>/dep-*`
|
|
# to `<profile>/build/<pkg>/<hash>/fingerprint/dep-*` (stabilised by cargo PR
|
|
# #17354, cargo 1.100.0, stable 2026-11-12; nightly default since 1.99). An
|
|
# earlier cut of this line looked for the v1 path only, so on 2026-08-26,
|
|
# against 1.100.0-nightly with content freshness genuinely on, it printed
|
|
# "does NOT rewrite ... in place" directly beneath a control listing that
|
|
# showed the rewrite. A reporting line that can contradict the data three
|
|
# lines above it is worse than no line at all. `fingerprint/.*dep-` matches
|
|
# either layout and neither `.d` family.
|
|
if [ "$CHECKSUM_MODE" = "on" ]; then
|
|
if printf '%s' "$ctl_mutated" | grep -q 'fingerprint/.*dep-'; then
|
|
echo " note: this cargo DOES rewrite its dep-info fingerprint file in place under content freshness"
|
|
else
|
|
echo " note: this cargo does NOT rewrite its dep-info fingerprint file in place; only the build/ and *.d families appear above"
|
|
fi
|
|
fi
|
|
|
|
echo
|
|
echo "=== fix: hardlink_clone_into() leaves the source byte-identical ==="
|
|
base_fix="$scratch/base-fix"; clone_fix="$scratch/clone-fix"
|
|
build_base "$base_fix"
|
|
before=$(snapshot_tree "$base_fix")
|
|
hardlink_clone_into "$base_fix" "$clone_fix" "selftest" || fail "hardlink_clone_into reported the destination already existed"
|
|
|
|
# The clone's contract, asserted before anything builds in it and asserted in
|
|
# BOTH directions: every file Cargo rewrites in place is privately owned (that
|
|
# is what makes the clone sound), and every artifact still shares its inode
|
|
# (that is what makes it near-free). Checking after a rebuild would prove
|
|
# nothing — the rebuild replaces those files anyway.
|
|
#
|
|
# The mutable-family patterns cover both layouts: `.fingerprint/` is v1's,
|
|
# `fingerprint/` and `run/` are v2's, and `gen.txt` is this crate's build
|
|
# script's OUT_DIR product, which under v2 sits in a directory called `out`
|
|
# beside sibling units whose `out` holds artifacts.
|
|
shared=0; unshared=0; shared_bytes=0; copied_bytes=0
|
|
while IFS= read -r f; do
|
|
rel="${f#"$base_fix"/}"
|
|
[ -e "$clone_fix/$rel" ] || continue
|
|
sz=$(stat -c '%s' "$f")
|
|
if [ "$(stat -c '%i' "$f")" = "$(stat -c '%i' "$clone_fix/$rel")" ]; then
|
|
case "$rel" in
|
|
*/.fingerprint/*|*/fingerprint/*|*/run/*|*/out/gen.txt|*/output|*/root-output|*/stderr|*/invoked.timestamp|*.d|.rustc_info.json)
|
|
fail "mutable path still shares an inode with the source: $rel" ;;
|
|
esac
|
|
shared=$((shared + 1)); shared_bytes=$((shared_bytes + sz))
|
|
else
|
|
case "$rel" in
|
|
*.rlib|*.rmeta)
|
|
fail "artifact was real-copied rather than shared: $rel" ;;
|
|
esac
|
|
unshared=$((unshared + 1)); copied_bytes=$((copied_bytes + sz))
|
|
fi
|
|
done < <(find "$base_fix" -type f)
|
|
[ "$shared" -gt 0 ] || fail "nothing is shared — the clone degenerated into a full copy"
|
|
[ "$unshared" -gt 0 ] || fail "nothing was unshared — unshare_mutable_paths did not run"
|
|
total_bytes=$((shared_bytes + copied_bytes))
|
|
ok "fresh clone shares ${shared} artifact files and privately owns ${unshared} mutable ones"
|
|
# Reported, not asserted. This crate has no dependencies, so almost all of its
|
|
# bytes are the two executables — a ratio that says nothing about a real tree.
|
|
# The fixtures above are where the cost model is gated, because there the
|
|
# composition is fixed.
|
|
echo " note: this clone real-copies $((copied_bytes * 100 / total_bytes))% of ${total_bytes} B"
|
|
|
|
printf '%s\n' "$CONTENT_B" > src/lib.rs
|
|
CARGO_TARGET_DIR="$clone_fix" "${CARGO_BIN[@]}" build -q
|
|
after=$(snapshot_tree "$base_fix")
|
|
fix_mutated=$(mutated_paths "$before" "$after")
|
|
if [ -n "$fix_mutated" ]; then
|
|
echo " still mutated:" >&2
|
|
printf '%s\n' "$fix_mutated" | sed 's/^/ /' >&2
|
|
fail "a build in the clone mutated the source through a shared inode"
|
|
fi
|
|
ok "no file in the source changed after a full rebuild in the clone"
|
|
|
|
echo
|
|
echo "=== a linked TEST binary, which nothing uplifts and nothing replaces ==="
|
|
# The one artifact family that is NOT safe to share, and the reason
|
|
# `unshare_mutable_paths` privately owns every executable. rustc writes an
|
|
# rlib to a temporary and renames it in; the LINKER writes an executable
|
|
# through whatever inode is already at the path.
|
|
#
|
|
# THE CRATE SHAPE IS LOAD-BEARING AND WAS WRONG ONCE. An earlier cut of this
|
|
# scenario reused the lib+bin probe crate above, whose test binaries relink to
|
|
# a FRESH inode — a shape gitdan-actions#17 records as measured safe. Both
|
|
# halves then passed green against the unfixed selection, on the strength of
|
|
# dep-info mutations the previous scenario already covers, and the scenario
|
|
# pinned nothing. A bin-only crate with a unit test does exhibit the rewrite,
|
|
# on cargo 1.93.1 stable and on 1.98.0-nightly and 1.100.0-nightly, so that is
|
|
# what this builds. The shape is chosen by measurement rather than derived:
|
|
# what separates a rewritten executable from an intact one is not established,
|
|
# so the only crate shape this scenario may rest on is one observed to exhibit
|
|
# the rewrite.
|
|
mkbincrate() {
|
|
local dir="$1" marker="$2"
|
|
mkdir -p "$dir/src"
|
|
cat > "$dir/Cargo.toml" <<'TOML'
|
|
[package]
|
|
name = "binprobe"
|
|
version = "0.1.0"
|
|
edition = "2021"
|
|
[workspace]
|
|
TOML
|
|
cat > "$dir/src/main.rs" <<RS
|
|
fn main() { println!("${marker}"); }
|
|
#[cfg(test)]
|
|
mod t { #[test] fn a() { assert_eq!("${marker}".len() > 0, true); } }
|
|
RS
|
|
}
|
|
|
|
bin_dir="$scratch/binprobe"
|
|
mkbincrate "$bin_dir" MARKER_AAAA
|
|
cd "$bin_dir"
|
|
base_exe="$scratch/base-exe"; clone_exe_ctl="$scratch/clone-exe-ctl"; clone_exe="$scratch/clone-exe"
|
|
|
|
# Only the executables are read here. The families the other scenarios cover
|
|
# would satisfy a "something changed" assertion on their own, which is exactly
|
|
# how the earlier cut of this passed while pinning nothing.
|
|
source_exe_digest() {
|
|
(cd "$1" && find . -type f -executable -print0 | sort -z | xargs -0 -r sha1sum) 2>/dev/null
|
|
}
|
|
|
|
CARGO_TARGET_DIR="$base_exe" "${CARGO_BIN[@]}" test --no-run -q > /dev/null 2>&1 \
|
|
|| fail "the bin-only probe crate failed to build"
|
|
before=$(source_exe_digest "$base_exe")
|
|
cp -al "$base_exe" "$clone_exe_ctl"
|
|
strip_cargo_locks "$clone_exe_ctl"
|
|
mkbincrate "$bin_dir" MARKER_BBBB
|
|
CARGO_TARGET_DIR="$clone_exe_ctl" "${CARGO_BIN[@]}" test --no-run -q > /dev/null 2>&1
|
|
exe_ctl_mutated=$(mutated_paths "$before" "$(source_exe_digest "$base_exe")")
|
|
|
|
# THREE OUTCOMES, as the freshness probe above has, and for the same reason: a
|
|
# scenario that cannot tell "the fix works" from "the hazard never fired" is
|
|
# not a gate. If this Cargo does not rewrite the source's test binary, the
|
|
# assertion below would pass for a toolchain reason rather than a code one, so
|
|
# it is skipped LOUDLY instead of passing quietly.
|
|
if [ -z "$exe_ctl_mutated" ]; then
|
|
echo "::warning::hardlink-clone-selftest: this toolchain did not rewrite the source's test binary through a raw cp -al clone, so the linked-output scenario proves nothing here and was SKIPPED. That is a statement about this Cargo, not about unshare_mutable_paths."
|
|
else
|
|
ok "control: a raw cp -al clone rewrites the source's own linked test binary"
|
|
printf '%s\n' "$exe_ctl_mutated" | sed 's/^/ /'
|
|
|
|
# Rebuild the base from the original marker so it is warm and consistent
|
|
# again, then do the same thing through the real clone.
|
|
mkbincrate "$bin_dir" MARKER_AAAA
|
|
CARGO_TARGET_DIR="$base_exe" "${CARGO_BIN[@]}" test --no-run -q > /dev/null 2>&1
|
|
before=$(snapshot_tree "$base_exe")
|
|
hardlink_clone_into "$base_exe" "$clone_exe" "selftest-exe" \
|
|
|| fail "hardlink_clone_into refused the destination"
|
|
mkbincrate "$bin_dir" MARKER_BBBB
|
|
CARGO_TARGET_DIR="$clone_exe" "${CARGO_BIN[@]}" test --no-run -q > /dev/null 2>&1
|
|
exe_mutated=$(mutated_paths "$before" "$(snapshot_tree "$base_exe")")
|
|
if [ -n "$exe_mutated" ]; then
|
|
printf '%s\n' "$exe_mutated" | sed 's/^/ /' >&2
|
|
fail "a test build in the clone mutated the source through a shared inode"
|
|
fi
|
|
ok "no file in the source changed after a full test build in the clone"
|
|
fi
|
|
cd "$crate_dir"
|
|
|
|
echo
|
|
if [ "$CHECKSUM_MODE" = "on" ]; then
|
|
echo "=== the whole point: the source's next build is still correct ==="
|
|
# The source's cache holds artifacts built from CONTENT_A. Advance the
|
|
# source to CONTENT_B (as a merge would) and rebuild in it. If the clone had
|
|
# corrupted its dep-info, Cargo would report Fresh and keep the stale rlib.
|
|
#
|
|
# CHECKSUM-FRESHNESS ONLY, and the backdated mtime is why. Under checksum
|
|
# freshness the dep-info file's per-source checksums decide, so a 2001
|
|
# timestamp on changed content must still rebuild — the assertion below.
|
|
# Under Cargo's ordinary MTIME freshness the same timestamp means the source
|
|
# is older than the artifact, and reporting Fresh is the correct answer;
|
|
# asserting otherwise asserts a bug. This scenario was written against a
|
|
# machine with a nightly installed and, run without one, failed on that
|
|
# correct answer.
|
|
printf '%s\n' "$CONTENT_B" > src/lib.rs
|
|
touch -d '@1000000000' src/lib.rs
|
|
log="$scratch/rebuild.log"
|
|
CARGO_TARGET_DIR="$base_fix" "${CARGO_BIN[@]}" build -v > "$log" 2>&1 || { cat "$log"; fail "rebuild in the source failed"; }
|
|
if grep -qE '^\s+Fresh probe' "$log"; then
|
|
fail "source declared its own crate Fresh against sources it has never built — stale-artifact reuse"
|
|
fi
|
|
ok "source correctly rebuilt its crate after advancing to the clone's content"
|
|
else
|
|
echo "=== skipped: the source's-next-build scenario needs content-based freshness ==="
|
|
echo " reason: ${CHECKSUM_REASON}"
|
|
fi
|
|
|
|
echo
|
|
echo "hardlink-clone-selftest: ${pass_count} assertions passed"
|