fix(hardlink): name the mutable set directly, under both build-dir layouts
CI / shellcheck + selftests (pull_request) Skipped

`unshare_mutable_paths` selected `.fingerprint` and `build` directories. Under
Cargo's build-dir layout v2 the first clause matches nothing and the second
matches the whole tree, because v2 regroups artifacts under `build/` alongside
the metadata. Measured on one scratch crate: 39.3% of the tree real-copied
under v1, 99.996% under v2.

The selection now names the mutable set rather than the container it used to
live in: fingerprint directories under either spelling, layout v2's `run/`
directories, layout v1's loose build-script run metadata, and the `out`
directories that are a build script's OUT_DIR rather than a compile unit's
artifact directory. The two are told apart structurally, by Cargo's record of
the build-script execution sitting beside the OUT_DIR and nowhere else.

Verifying that turned up a second, layout-independent hazard: a linked
executable is written through whatever inode is already at its path, so a
`cargo test --no-run` inside a `cp -al` clone rewrites the source's own test
binary. Reproduced on cargo 1.93.1 stable, 1.96.0-nightly, 1.98.0-nightly and
1.100.0-nightly, under both layouts. Every executable is now real-copied;
`.rlib`, `.rmeta` and `incremental/` are what stay shared.

hardlink-clone-selftest.sh gains two file-only layout fixtures that pin the
partition in both directions without a compiler, and a live scenario that
relinks a test binary.
This commit is contained in:
2026-08-27 14:12:01 -05:00
parent 4114996954
commit e5b26a9368
4 changed files with 398 additions and 52 deletions
+86 -28
View File
@@ -320,14 +320,13 @@ _unshare_files() {
# THE load-bearing function of this whole design.
#
# A hardlink clone is only safe if every write the clone's build performs
# lands on a NEW inode, leaving the source's data untouched. That is true for
# compilation artifacts — rustc and the linker replace `deps/*.rlib`,
# `*.rmeta`, and binaries rather than truncating them in place — and it is
# NOT true for the metadata Cargo and build scripts write with a plain
# truncating write. Measured directly (Linux, ext4, cargo 1.9x nightly:
# `cp -al` a warm target dir, change a source file, build in the clone, diff
# the source) the following files in the SOURCE were mutated through the
# shared inode:
# lands on a NEW inode, leaving the source's data untouched. That is true of
# rustc's own outputs — it writes an `.rlib` or `.rmeta` to a temporary and
# renames it into place — and it is NOT true of the metadata Cargo and build
# scripts write with a plain truncating write, nor of anything the LINKER
# produces. Measured directly (Linux, ext4: `cp -al` a warm target dir, change
# a source file, build in the clone, diff the source) the following files in
# the SOURCE were mutated through the shared inode:
#
# <profile>/.fingerprint/<unit>/dep-<target> (build-dir layout v1) — or,
# <profile>/build/<pkg>/<hash>/fingerprint/dep-<target>
@@ -347,6 +346,20 @@ _unshare_files() {
# plain fs::write)
# <profile>/deps/*.d, <profile>/*.d (Cargo's post-processed
# dep-info)
# <profile>/deps/<test>-<hash> (a linked TEST binary; under
# layout v2,
# `build/<pkg>/<hash>/out/`)
#
# THE LINKED-OUTPUT CASE IS NOT LAYOUT-SPECIFIC AND WAS NOT PART OF THIS
# FUNCTION UNTIL 2026-08-27 (gitdan-actions#14). A `cargo test --no-run` inside
# a raw `cp -al` clone rewrote the source's own test binary in place on cargo
# 1.93.1 stable, 1.96.0-nightly, 1.98.0-nightly (layout v1) and 1.100.0-nightly
# (layout v2) alike. Cargo re-creates the path first when it also has to uplift
# the result — a bin target's `deps/<bin>-<hash>` has a hardlink twin at
# `<profile>/<bin>` — and other crate shapes relinked to a fresh inode for
# reasons this measurement did not pin down. Since the safe cases could not be
# enumerated, every executable is treated as mutable; `.rlib`, `.rmeta` and
# `incremental/` are what stay shared, and they are the bytes worth sharing.
#
# The checksum-freshness case is not a cosmetic one. Reproduced end to end:
# branch B clones base's cache, builds its own content, and thereby rewrites
@@ -390,40 +403,61 @@ _unshare_files() {
# included. Bracketed locally: cargo 1.97.1 and 1.98.0-nightly write v1,
# 1.100.0-nightly writes v2.
#
# The `-name .fingerprint` clause below therefore matches nothing under a v2
# Cargo, and the dep-info file is covered only because the `-name build` clause
# happens to swallow its new home. That is belt-and-braces by accident, not by
# design — and the same accident makes this function real-copy essentially the
# whole tree, because the artifacts moved under `build/` too. Measured on one
# scratch crate (serde + serde_json + regex), same sources both ways:
# Until 2026-08-27 the selection below was `-name .fingerprint -o -name build`,
# which under a v2 Cargo matched nothing on its first clause and the entire
# tree on its second, because the artifacts moved under `build/` too. The guard
# held by accident and the saving did not: on one scratch crate (serde +
# serde_json + regex plus a build script), same sources both ways —
#
# cargo 1.97.1 (layout v1) 27.0 MB unshared of 126.7 MB 21.3%
# 1.100.0-nightly (layout v2) 105.9 MB unshared of 105.9 MB — 99.998%
# cargo 1.98.0-nightly (layout v1) 64.9 MB unshared of 165.1 MB — 39.3%
# 1.100.0-nightly (layout v2) 110.4 MB unshared of 110.4 MB — 99.996%
#
# So the guard still holds and the saving does not. Deliberately NOT fixed
# here: adjusting the selection is a change to what gets hardlinked on every
# consumer, which wants its own change and its own review, and the deadline is
# cargo 1.100.0 stable on 2026-11-12. Tracked as gitdan-actions#14.
#
# The historical v1 figure this block used to quote stands as measured: on a
# 6.9 GB Bevy workspace target dir the unshared set was .fingerprint 22 MB +
# build/ 237 MB + a handful of dep-info files — about 3.7% of the tree, against
# 100% for a plain `cp -a`. It describes layout v1 only.
# The selection now names the mutable set directly rather than by the container
# it used to live in, so it holds under both layouts; the same crate measures
# 45.3% (v1) and 38.4% (v2), both dominated by the linked-output rule above
# rather than by the layout. On a real 5.5 GB Bevy target dir the whole change
# moves the real-copied share from 9.0% to 14.1%.
#
# `incremental/` is deliberately left shared: rustc writes each incremental
# session to a fresh `s-*-working` directory and finalises it with a rename,
# and garbage-collects old sessions by unlinking directory entries — neither
# of which mutates a shared inode. CI should still set CARGO_INCREMENTAL=0,
# for size rather than correctness.
# The directories `unshare_mutable_paths` replaces, under either layout.
#
# All four names are pruned, so nothing selected here can contain anything else
# selected here and the caller never unshares a subtree twice.
#
# `out` is the one that needs deciding rather than naming, and it is the whole
# difficulty of layout v2: a compile unit's rlib and a build script's OUT_DIR
# are both a directory called `out`, one directory apart, and they need
# opposite treatment. The discriminator is structural — Cargo records a build
# script's execution beside its OUT_DIR and nowhere else, as `run/root-output`
# under v2 and as a loose `root-output` under v1 — so a unit directory holding
# one of those is a RUN unit and its `out` is the OUT_DIR. A unit directory
# without one is a compile unit, and its `out` holds the artifact whose sharing
# is the entire point of the clone.
_mutable_dirs() {
local root="$1" d unit
while IFS= read -r d; do
if [ "${d##*/}" = out ]; then
unit="${d%/out}"
[ -d "$unit/run" ] || [ -e "$unit/root-output" ] || continue
fi
printf '%s\n' "$d"
done < <(find "$root" -type d \
\( -name .fingerprint -o -name fingerprint -o -name run -o -name out \) \
-prune -print 2>/dev/null)
}
unshare_mutable_paths() {
local root="$1" d
[ -d "$root" ] || return 0
# The list is materialised in full before anything is replaced: each
# replacement deletes and recreates a directory, and a live `find` walk over
# a tree being mutated underneath it is a needless hazard. `-prune` keeps a
# match's own contents out of the list.
# a tree being mutated underneath it is a needless hazard.
local -a dirs=()
mapfile -t dirs < <(find "$root" -type d \( -name .fingerprint -o -name build \) -prune -print 2>/dev/null)
mapfile -t dirs < <(_mutable_dirs "$root")
for d in "${dirs[@]}"; do
[ -n "$d" ] || continue
unshare_subtree "$d" || {
@@ -435,6 +469,30 @@ unshare_mutable_paths() {
echo "::error::unshare_mutable_paths: failed to unshare dep-info files under ${root}" >&2
return 1
}
# Layout v1's build-script run metadata, which v2 groups under `run/` and v1
# leaves loose in the run unit's directory. `invoked.timestamp` is empty and
# carries its meaning in its mtime, which a shared inode carries too.
_unshare_files "$root" -type f \
\( -name output -o -name root-output -o -name stderr -o -name invoked.timestamp \) || {
echo "::error::unshare_mutable_paths: failed to unshare build-script run metadata under ${root}" >&2
return 1
}
# Linked outputs. Unlike an rlib or an rmeta — which rustc writes to a
# temporary and renames into place — an executable or shared object is
# written by the LINKER, and the linker writes THROUGH an existing inode
# whenever the path it is given has no other hard link. Measured 2026-08-27
# on cargo 1.98.0-nightly (a335d47ff, layout v1) and 1.100.0-nightly
# (e8cb624d5, layout v2), mold and the default linker alike: a `cargo test`
# binary in a `cp -al` clone rewrote the SOURCE's copy of itself in place,
# under both layouts.
#
# The executable bit is the discriminator because it is the linker's own
# output that is at risk, not the directory it happens to land in — `.rlib`,
# `.rmeta` and `incremental/` stay shared and they are the bytes that matter.
_unshare_files "$root" -type f -perm -u+x || {
echo "::error::unshare_mutable_paths: failed to unshare linked outputs under ${root}" >&2
return 1
}
_unshare_files "$root" -maxdepth 3 -type f -name '.rustc_info.json' || {
echo "::error::unshare_mutable_paths: failed to unshare .rustc_info.json under ${root}" >&2
return 1
+279 -10
View File
@@ -28,8 +28,6 @@ set -euo pipefail
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
. "$script_dir/cache-lib.sh"
command -v cargo >/dev/null || { echo "SKIP: no cargo on PATH"; exit 0; }
scratch=$(mktemp -d)
trap 'rm -rf "$scratch"' EXIT
pass_count=0
@@ -59,6 +57,14 @@ version = "0.1.0"
edition = "2021"
[workspace]
TOML
# A binary as well as a library, because the two are written differently and
# only one of them is safe to share: rustc writes an rlib to a temporary and
# renames it into place, while the LINKER writes an executable through the
# existing inode. Without a bin target this suite never relinks anything and
# cannot see that difference.
cat > "$dir/src/main.rs" <<'RS'
fn main() { println!("{}", probe::f()); }
RS
cat > "$dir/build.rs" <<'RS'
use std::{env, fs, path::PathBuf};
fn main() {
@@ -70,6 +76,208 @@ fn main() {
RS
}
# ---------------------------------------------------------------------------
# Both build-dir layouts, without a compiler
# ---------------------------------------------------------------------------
#
# Every other scenario in this file runs whichever layout the installed Cargo
# happens to write, so on any one machine it exercises exactly ONE of the two —
# and gitdan-ci's is v1. The two fixtures below reproduce both directory shapes
# from files alone, clone them through the real `hardlink_clone_into()`, and
# assert file by file which side of the partition each one lands on.
#
# Shapes taken from a scratch crate (serde + serde_json + regex, plus a build
# script) built on 2026-08-27: cargo 1.98.0-nightly (a335d47ff 2026-06-26)
# writes v1, cargo 1.100.0-nightly (e8cb624d5 2026-08-22) writes v2.
#
# v2 is where the partition is easy to get wrong, and the fixtures are built to
# say so: a build script's OUT_DIR and a compile unit's rlib are BOTH a
# directory called `out`, one directory apart, and they need opposite
# treatment.
mkfile() { mkdir -p "$(dirname "$1")"; printf '%s' "$2" > "$1"; }
# Big enough that the byte-fraction assertion below measures something.
artifact_bytes=$(head -c 4096 /dev/zero | tr '\0' 'A')
# Spec lines are `<shared|private>|<path relative to the tree root>`.
assert_partition() {
local label="$1" src="$2" spec="$3"
local clone="${src}-clone" want rel si ci total=0 copied=0 sz
hardlink_clone_into "$src" "$clone" "selftest-$label" \
|| fail "$label: hardlink_clone_into refused the destination"
while IFS='|' read -r want rel; do
[ -n "${rel:-}" ] || continue
[ -e "$clone/$rel" ] || fail "$label: $rel is missing from the clone"
si=$(stat -c '%i' "$src/$rel"); ci=$(stat -c '%i' "$clone/$rel")
case "$want" in
shared)
[ "$si" = "$ci" ] \
|| fail "$label: $rel was real-copied, but it is an artifact and must stay shared" ;;
private)
[ "$si" != "$ci" ] \
|| fail "$label: $rel still shares an inode with the source, so a build in the clone can rewrite it" ;;
*) fail "$label: unknown spec verb '$want'" ;;
esac
done <<< "$spec"
ok "$label: every file landed on the right side of the partition"
# The cost model, asserted rather than assumed. Selecting too much is not a
# correctness bug, which is exactly why nothing caught layout v2 taking the
# selection from a fifth of the tree to 99.998% of it (gitdan-actions#14):
# a hardlink clone that real-copies everything is a `cp -a` with extra steps.
# The bound is loose on purpose. It is not a budget — the honest figure moves
# with how much of a tree is linker output, and these fixtures are mostly
# that by construction — it is a floor under "still a hardlink clone at all".
while IFS= read -r rel; do
# `.cargo-*lock*` is stripped from every clone by design, so it has no
# counterpart to compare against.
[ -e "$clone/$rel" ] || continue
sz=$(stat -c '%s' "$src/$rel")
total=$((total + sz))
[ "$(stat -c '%i' "$src/$rel")" = "$(stat -c '%i' "$clone/$rel")" ] || copied=$((copied + sz))
done < <(cd "$src" && find . -type f -printf '%P\n')
[ "$total" -gt 0 ] || fail "$label: fixture has no bytes to measure"
[ $((copied * 100 / total)) -lt 90 ] \
|| fail "$label: the clone real-copied $((copied * 100 / total))% of its source's bytes — the hardlink saving is gone"
ok "$label: clone real-copies $((copied * 100 / total))% of ${total} B (${copied} B), the rest is shared"
}
# Layout v2: no `.fingerprint`, no `deps`. Everything regroups per build unit
# under `build/<pkg>/<hash>/{fingerprint,out,run}`, artifacts included — which
# is what took `-name build` from "the metadata" to "the whole tree".
#
# THE THREE UNIT KINDS ARE THE POINT. `1bf...` is a compile unit: its `out`
# holds the rlib. `08c...` is the build script's own compile unit: its `out`
# holds the build-script binary. `da9...` is the build-script RUN unit: its
# `out` IS the OUT_DIR, and it is the only one of the three whose `out` is
# mutable. The `run/` directory beside it is the structural difference.
v2="$scratch/layout-v2"
mkfile "$v2/CACHEDIR.TAG" 'Signature: 8a477f597d28d172'
mkfile "$v2/.rustc_info.json" '{"rustc_fingerprint":1}'
mkfile "$v2/debug/.cargo-lock" ''
mkfile "$v2/debug/libprobe.rlib" "$artifact_bytes"
mkfile "$v2/debug/libprobe.d" '/probe/src/lib.rs:'
for f in dep-lib-probe lib-probe lib-probe.json invoked.timestamp; do
mkfile "$v2/debug/build/probe/1bf5493368dce3cd/fingerprint/$f" "$f"
done
mkfile "$v2/debug/build/probe/1bf5493368dce3cd/out/libprobe-1bf5493368dce3cd.rlib" "$artifact_bytes"
mkfile "$v2/debug/build/probe/1bf5493368dce3cd/out/libprobe-1bf5493368dce3cd.rmeta" "$artifact_bytes"
mkfile "$v2/debug/build/probe/1bf5493368dce3cd/out/probe-1bf5493368dce3cd.d" '/probe/src/lib.rs:'
for f in build-script-build-script-build build-script-build-script-build.json \
dep-build-script-build-script-build invoked.timestamp; do
mkfile "$v2/debug/build/probe/08c7dda6eacd6dca/fingerprint/$f" "$f"
done
mkfile "$v2/debug/build/probe/08c7dda6eacd6dca/out/build_script_build" "$artifact_bytes"
chmod +x "$v2/debug/build/probe/08c7dda6eacd6dca/out/build_script_build"
mkfile "$v2/debug/build/probe/08c7dda6eacd6dca/out/build_script_build.d" '/probe/build.rs:'
# A test binary: the same `out` directory as the rlib above, and the largest
# thing in a real tree that a linker writes.
for f in dep-test-lib-probe test-lib-probe test-lib-probe.json invoked.timestamp; do
mkfile "$v2/debug/build/probe/6a091d813b2be60d/fingerprint/$f" "$f"
done
mkfile "$v2/debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d" "$artifact_bytes"
chmod +x "$v2/debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d"
mkfile "$v2/debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d.d" '/probe/src/lib.rs:'
for f in run-build-script-build-script-build run-build-script-build-script-build.json; do
mkfile "$v2/debug/build/probe/da96cf45111f80dd/fingerprint/$f" "$f"
done
mkfile "$v2/debug/build/probe/da96cf45111f80dd/out/gen.txt" 'generated from 24 bytes'
for f in invoked.timestamp root-output stdout stderr; do
mkfile "$v2/debug/build/probe/da96cf45111f80dd/run/$f" "$f"
done
mkfile "$v2/debug/incremental/probe-abc/s-xyz/dep-graph.bin" "$artifact_bytes"
assert_partition "layout v2" "$v2" "$(cat <<'SPEC'
private|.rustc_info.json
private|debug/libprobe.d
private|debug/build/probe/1bf5493368dce3cd/fingerprint/dep-lib-probe
private|debug/build/probe/1bf5493368dce3cd/fingerprint/lib-probe
private|debug/build/probe/1bf5493368dce3cd/fingerprint/lib-probe.json
private|debug/build/probe/1bf5493368dce3cd/fingerprint/invoked.timestamp
private|debug/build/probe/1bf5493368dce3cd/out/probe-1bf5493368dce3cd.d
private|debug/build/probe/08c7dda6eacd6dca/fingerprint/dep-build-script-build-script-build
private|debug/build/probe/08c7dda6eacd6dca/fingerprint/invoked.timestamp
private|debug/build/probe/08c7dda6eacd6dca/out/build_script_build.d
private|debug/build/probe/da96cf45111f80dd/fingerprint/run-build-script-build-script-build
private|debug/build/probe/da96cf45111f80dd/out/gen.txt
private|debug/build/probe/da96cf45111f80dd/run/root-output
private|debug/build/probe/da96cf45111f80dd/run/stdout
private|debug/build/probe/da96cf45111f80dd/run/stderr
private|debug/build/probe/da96cf45111f80dd/run/invoked.timestamp
shared|debug/libprobe.rlib
shared|debug/build/probe/1bf5493368dce3cd/out/libprobe-1bf5493368dce3cd.rlib
shared|debug/build/probe/1bf5493368dce3cd/out/libprobe-1bf5493368dce3cd.rmeta
private|debug/build/probe/08c7dda6eacd6dca/out/build_script_build
private|debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d
private|debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d.d
private|debug/build/probe/6a091d813b2be60d/fingerprint/dep-test-lib-probe
shared|debug/incremental/probe-abc/s-xyz/dep-graph.bin
SPEC
)"
# Layout v1: one `.fingerprint` and one `deps` per profile; `build/<pkg>-<hash>`
# holds the build script's compiled binary in one unit directory and its run
# metadata plus OUT_DIR in another.
v1="$scratch/layout-v1"
mkfile "$v1/CACHEDIR.TAG" 'Signature: 8a477f597d28d172'
mkfile "$v1/.rustc_info.json" '{"rustc_fingerprint":1}'
mkfile "$v1/debug/.cargo-lock" ''
mkfile "$v1/debug/libprobe.rlib" "$artifact_bytes"
mkfile "$v1/debug/libprobe.d" '/probe/src/lib.rs:'
mkfile "$v1/debug/deps/libprobe-1bf5493368dce3cd.rlib" "$artifact_bytes"
mkfile "$v1/debug/deps/libprobe-1bf5493368dce3cd.rmeta" "$artifact_bytes"
mkfile "$v1/debug/deps/probe-1bf5493368dce3cd.d" '/probe/src/lib.rs:'
# A test binary, which layout v1 leaves in `deps/` beside the rlibs.
mkfile "$v1/debug/deps/probe-6a091d813b2be60d" "$artifact_bytes"
chmod +x "$v1/debug/deps/probe-6a091d813b2be60d"
mkfile "$v1/debug/deps/probe-6a091d813b2be60d.d" '/probe/src/lib.rs:'
for f in dep-lib-probe lib-probe lib-probe.json invoked.timestamp; do
mkfile "$v1/debug/.fingerprint/probe-1bf5493368dce3cd/$f" "$f"
done
mkfile "$v1/debug/build/probe-08c7dda6eacd6dca/build-script-build" "$artifact_bytes"
mkfile "$v1/debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca" "$artifact_bytes"
chmod +x "$v1/debug/build/probe-08c7dda6eacd6dca/build-script-build" \
"$v1/debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca"
mkfile "$v1/debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca.d" '/probe/build.rs:'
for f in invoked.timestamp output root-output stderr; do
mkfile "$v1/debug/build/probe-da96cf45111f80dd/$f" "$f"
done
mkfile "$v1/debug/build/probe-da96cf45111f80dd/out/gen.txt" 'generated from 24 bytes'
mkfile "$v1/debug/incremental/probe-abc/s-xyz/dep-graph.bin" "$artifact_bytes"
assert_partition "layout v1" "$v1" "$(cat <<'SPEC'
private|.rustc_info.json
private|debug/libprobe.d
private|debug/deps/probe-1bf5493368dce3cd.d
private|debug/.fingerprint/probe-1bf5493368dce3cd/dep-lib-probe
private|debug/.fingerprint/probe-1bf5493368dce3cd/lib-probe
private|debug/.fingerprint/probe-1bf5493368dce3cd/lib-probe.json
private|debug/.fingerprint/probe-1bf5493368dce3cd/invoked.timestamp
private|debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca.d
private|debug/deps/probe-6a091d813b2be60d
private|debug/deps/probe-6a091d813b2be60d.d
private|debug/build/probe-da96cf45111f80dd/invoked.timestamp
private|debug/build/probe-da96cf45111f80dd/output
private|debug/build/probe-da96cf45111f80dd/root-output
private|debug/build/probe-da96cf45111f80dd/stderr
private|debug/build/probe-da96cf45111f80dd/out/gen.txt
shared|debug/libprobe.rlib
shared|debug/deps/libprobe-1bf5493368dce3cd.rlib
shared|debug/deps/libprobe-1bf5493368dce3cd.rmeta
private|debug/build/probe-08c7dda6eacd6dca/build-script-build
private|debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca
shared|debug/incremental/probe-abc/s-xyz/dep-graph.bin
SPEC
)"
echo
command -v cargo >/dev/null || {
echo "SKIP: no cargo on PATH — the fixture scenarios above ran, the live-Cargo ones cannot"
echo "hardlink-clone-selftest: ${pass_count} assertions passed"
exit 0
}
crate_dir="$scratch/probe"
mkcrate "$crate_dir"
cd "$crate_dir"
@@ -304,27 +512,44 @@ build_base "$base_fix"
before=$(snapshot_tree "$base_fix")
hardlink_clone_into "$base_fix" "$clone_fix" "selftest" || fail "hardlink_clone_into reported the destination already existed"
# The clone's contract, asserted before anything builds in it: artifacts
# share inodes (that is what makes the clone near-free), and every file Cargo
# rewrites in place does not (that is what makes it sound). Checking after a
# rebuild would prove nothing — the rebuild replaces those files anyway.
shared=0; unshared=0
# The clone's contract, asserted before anything builds in it and asserted in
# BOTH directions: every file Cargo rewrites in place is privately owned (that
# is what makes the clone sound), and every artifact still shares its inode
# (that is what makes it near-free). Checking after a rebuild would prove
# nothing — the rebuild replaces those files anyway.
#
# The mutable-family patterns cover both layouts: `.fingerprint/` is v1's,
# `fingerprint/` and `run/` are v2's, and `gen.txt` is this crate's build
# script's OUT_DIR product, which under v2 sits in a directory called `out`
# beside sibling units whose `out` holds artifacts.
shared=0; unshared=0; shared_bytes=0; copied_bytes=0
while IFS= read -r f; do
rel="${f#"$base_fix"/}"
[ -e "$clone_fix/$rel" ] || continue
sz=$(stat -c '%s' "$f")
if [ "$(stat -c '%i' "$f")" = "$(stat -c '%i' "$clone_fix/$rel")" ]; then
case "$rel" in
*/.fingerprint/*|*/build/*|*.d|.rustc_info.json)
*/.fingerprint/*|*/fingerprint/*|*/run/*|*/out/gen.txt|*/output|*/root-output|*/stderr|*/invoked.timestamp|*.d|.rustc_info.json)
fail "mutable path still shares an inode with the source: $rel" ;;
esac
shared=$((shared + 1))
shared=$((shared + 1)); shared_bytes=$((shared_bytes + sz))
else
unshared=$((unshared + 1))
case "$rel" in
*.rlib|*.rmeta)
fail "artifact was real-copied rather than shared: $rel" ;;
esac
unshared=$((unshared + 1)); copied_bytes=$((copied_bytes + sz))
fi
done < <(find "$base_fix" -type f)
[ "$shared" -gt 0 ] || fail "nothing is shared — the clone degenerated into a full copy"
[ "$unshared" -gt 0 ] || fail "nothing was unshared — unshare_mutable_paths did not run"
total_bytes=$((shared_bytes + copied_bytes))
ok "fresh clone shares ${shared} artifact files and privately owns ${unshared} mutable ones"
# Reported, not asserted. This crate has no dependencies, so almost all of its
# bytes are the two executables — a ratio that says nothing about a real tree.
# The fixtures above are where the cost model is gated, because there the
# composition is fixed.
echo " note: this clone real-copies $((copied_bytes * 100 / total_bytes))% of ${total_bytes} B"
printf '%s\n' "$CONTENT_B" > src/lib.rs
CARGO_TARGET_DIR="$clone_fix" "${CARGO_BIN[@]}" build -q
@@ -337,6 +562,50 @@ if [ -n "$fix_mutated" ]; then
fi
ok "no file in the source changed after a full rebuild in the clone"
echo
echo "=== a linked TEST binary, which nothing uplifts and nothing replaces ==="
# The one artifact family that is NOT safe to share, and the reason
# `unshare_mutable_paths` privately owns every executable. rustc writes an
# rlib to a temporary and renames it in; the LINKER writes an executable
# through whatever inode is already at the path. Cargo re-creates the path
# first when it also has to uplift the result — a bin target's
# `deps/<bin>-<hash>` has a hardlink twin at `<profile>/<bin>` — which is why
# bin targets look safe. A test binary has no twin, nothing re-creates it, and
# the write lands on the source's inode.
#
# Measured 2026-08-27 on cargo 1.98.0-nightly (a335d47ff, layout v1) and
# 1.100.0-nightly (e8cb624d5, layout v2): under BOTH layouts a `cargo test
# --no-run` inside a raw `cp -al` clone rewrote the SOURCE's own test binary.
base_exe="$scratch/base-exe"; clone_exe_ctl="$scratch/clone-exe-ctl"; clone_exe="$scratch/clone-exe"
printf '%s\n' "$CONTENT_A" > src/lib.rs
CARGO_TARGET_DIR="$base_exe" "${CARGO_BIN[@]}" test --no-run -q > /dev/null 2>&1
before=$(snapshot_tree "$base_exe")
cp -al "$base_exe" "$clone_exe_ctl"
strip_cargo_locks "$clone_exe_ctl"
printf '%s\n' "$CONTENT_B" > src/lib.rs
CARGO_TARGET_DIR="$clone_exe_ctl" "${CARGO_BIN[@]}" test --no-run -q > /dev/null 2>&1
exe_ctl_mutated=$(mutated_paths "$before" "$(snapshot_tree "$base_exe")")
[ -n "$exe_ctl_mutated" ] \
|| fail "control: a raw cp -al clone of a test build no longer mutates the source — the test can no longer tell fixed from broken"
ok "raw cp -al clone of a test build mutates the source ($(printf '%s\n' "$exe_ctl_mutated" | wc -l) paths)"
printf '%s\n' "$exe_ctl_mutated" | sed 's/^/ /'
# Rebuild the base from CONTENT_A so it is warm and consistent again, then do
# the same thing through the real clone.
printf '%s\n' "$CONTENT_A" > src/lib.rs
CARGO_TARGET_DIR="$base_exe" "${CARGO_BIN[@]}" test --no-run -q > /dev/null 2>&1
before=$(snapshot_tree "$base_exe")
hardlink_clone_into "$base_exe" "$clone_exe" "selftest-exe" \
|| fail "hardlink_clone_into refused the destination"
printf '%s\n' "$CONTENT_B" > src/lib.rs
CARGO_TARGET_DIR="$clone_exe" "${CARGO_BIN[@]}" test --no-run -q > /dev/null 2>&1
exe_mutated=$(mutated_paths "$before" "$(snapshot_tree "$base_exe")")
if [ -n "$exe_mutated" ]; then
printf '%s\n' "$exe_mutated" | sed 's/^/ /' >&2
fail "a test build in the clone mutated the source through a shared inode"
fi
ok "no file in the source changed after a full test build in the clone"
echo
if [ "$CHECKSUM_MODE" = "on" ]; then
echo "=== the whole point: the source's next build is still correct ==="
+4 -2
View File
@@ -53,9 +53,11 @@ pass_count=0
fail() { echo "ASSERTION FAILED: $*" >&2; exit 1; }
ok() { pass_count=$((pass_count + 1)); echo "PASS: $*"; }
# Cargo and rustc REPLACE an artifact (write elsewhere, rename over the path)
# rustc REPLACES an `.rlib`/`.rmeta` (writes elsewhere, renames over the path)
# rather than truncating it in place, which is exactly why a snapshot may
# share artifact inodes with the live target dir it was cloned from. The
# share those inodes with the live target dir it was cloned from. Linker
# outputs are the exception and are real-copied instead — see
# `unshare_mutable_paths` in cache-lib.sh. The
# fixtures here have to model that faithfully — a plain `>` redirect truncates
# in place and would write straight through the shared inode into the
# snapshot and every consumer, which is a property of the test fixture, not of