fix(hardlink): name the mutable set directly, under both build-dir layouts
CI / shellcheck + selftests (pull_request) Skipped
CI / shellcheck + selftests (pull_request) Skipped
`unshare_mutable_paths` selected `.fingerprint` and `build` directories. Under Cargo's build-dir layout v2 the first clause matches nothing and the second matches the whole tree, because v2 regroups artifacts under `build/` alongside the metadata. Measured on one scratch crate: 39.3% of the tree real-copied under v1, 99.996% under v2. The selection now names the mutable set rather than the container it used to live in: fingerprint directories under either spelling, layout v2's `run/` directories, layout v1's loose build-script run metadata, and the `out` directories that are a build script's OUT_DIR rather than a compile unit's artifact directory. The two are told apart structurally, by Cargo's record of the build-script execution sitting beside the OUT_DIR and nowhere else. Verifying that turned up a second, layout-independent hazard: a linked executable is written through whatever inode is already at its path, so a `cargo test --no-run` inside a `cp -al` clone rewrites the source's own test binary. Reproduced on cargo 1.93.1 stable, 1.96.0-nightly, 1.98.0-nightly and 1.100.0-nightly, under both layouts. Every executable is now real-copied; `.rlib`, `.rmeta` and `incremental/` are what stay shared. hardlink-clone-selftest.sh gains two file-only layout fixtures that pin the partition in both directions without a compiler, and a live scenario that relinks a test binary.
This commit is contained in:
+86
-28
@@ -320,14 +320,13 @@ _unshare_files() {
|
||||
# THE load-bearing function of this whole design.
|
||||
#
|
||||
# A hardlink clone is only safe if every write the clone's build performs
|
||||
# lands on a NEW inode, leaving the source's data untouched. That is true for
|
||||
# compilation artifacts — rustc and the linker replace `deps/*.rlib`,
|
||||
# `*.rmeta`, and binaries rather than truncating them in place — and it is
|
||||
# NOT true for the metadata Cargo and build scripts write with a plain
|
||||
# truncating write. Measured directly (Linux, ext4, cargo 1.9x nightly:
|
||||
# `cp -al` a warm target dir, change a source file, build in the clone, diff
|
||||
# the source) the following files in the SOURCE were mutated through the
|
||||
# shared inode:
|
||||
# lands on a NEW inode, leaving the source's data untouched. That is true of
|
||||
# rustc's own outputs — it writes an `.rlib` or `.rmeta` to a temporary and
|
||||
# renames it into place — and it is NOT true of the metadata Cargo and build
|
||||
# scripts write with a plain truncating write, nor of anything the LINKER
|
||||
# produces. Measured directly (Linux, ext4: `cp -al` a warm target dir, change
|
||||
# a source file, build in the clone, diff the source) the following files in
|
||||
# the SOURCE were mutated through the shared inode:
|
||||
#
|
||||
# <profile>/.fingerprint/<unit>/dep-<target> (build-dir layout v1) — or,
|
||||
# <profile>/build/<pkg>/<hash>/fingerprint/dep-<target>
|
||||
@@ -347,6 +346,20 @@ _unshare_files() {
|
||||
# plain fs::write)
|
||||
# <profile>/deps/*.d, <profile>/*.d (Cargo's post-processed
|
||||
# dep-info)
|
||||
# <profile>/deps/<test>-<hash> (a linked TEST binary; under
|
||||
# layout v2,
|
||||
# `build/<pkg>/<hash>/out/`)
|
||||
#
|
||||
# THE LINKED-OUTPUT CASE IS NOT LAYOUT-SPECIFIC AND WAS NOT PART OF THIS
|
||||
# FUNCTION UNTIL 2026-08-27 (gitdan-actions#14). A `cargo test --no-run` inside
|
||||
# a raw `cp -al` clone rewrote the source's own test binary in place on cargo
|
||||
# 1.93.1 stable, 1.96.0-nightly, 1.98.0-nightly (layout v1) and 1.100.0-nightly
|
||||
# (layout v2) alike. Cargo re-creates the path first when it also has to uplift
|
||||
# the result — a bin target's `deps/<bin>-<hash>` has a hardlink twin at
|
||||
# `<profile>/<bin>` — and other crate shapes relinked to a fresh inode for
|
||||
# reasons this measurement did not pin down. Since the safe cases could not be
|
||||
# enumerated, every executable is treated as mutable; `.rlib`, `.rmeta` and
|
||||
# `incremental/` are what stay shared, and they are the bytes worth sharing.
|
||||
#
|
||||
# The checksum-freshness case is not a cosmetic one. Reproduced end to end:
|
||||
# branch B clones base's cache, builds its own content, and thereby rewrites
|
||||
@@ -390,40 +403,61 @@ _unshare_files() {
|
||||
# included. Bracketed locally: cargo 1.97.1 and 1.98.0-nightly write v1,
|
||||
# 1.100.0-nightly writes v2.
|
||||
#
|
||||
# The `-name .fingerprint` clause below therefore matches nothing under a v2
|
||||
# Cargo, and the dep-info file is covered only because the `-name build` clause
|
||||
# happens to swallow its new home. That is belt-and-braces by accident, not by
|
||||
# design — and the same accident makes this function real-copy essentially the
|
||||
# whole tree, because the artifacts moved under `build/` too. Measured on one
|
||||
# scratch crate (serde + serde_json + regex), same sources both ways:
|
||||
# Until 2026-08-27 the selection below was `-name .fingerprint -o -name build`,
|
||||
# which under a v2 Cargo matched nothing on its first clause and the entire
|
||||
# tree on its second, because the artifacts moved under `build/` too. The guard
|
||||
# held by accident and the saving did not: on one scratch crate (serde +
|
||||
# serde_json + regex plus a build script), same sources both ways —
|
||||
#
|
||||
# cargo 1.97.1 (layout v1) 27.0 MB unshared of 126.7 MB — 21.3%
|
||||
# 1.100.0-nightly (layout v2) 105.9 MB unshared of 105.9 MB — 99.998%
|
||||
# cargo 1.98.0-nightly (layout v1) 64.9 MB unshared of 165.1 MB — 39.3%
|
||||
# 1.100.0-nightly (layout v2) 110.4 MB unshared of 110.4 MB — 99.996%
|
||||
#
|
||||
# So the guard still holds and the saving does not. Deliberately NOT fixed
|
||||
# here: adjusting the selection is a change to what gets hardlinked on every
|
||||
# consumer, which wants its own change and its own review, and the deadline is
|
||||
# cargo 1.100.0 stable on 2026-11-12. Tracked as gitdan-actions#14.
|
||||
#
|
||||
# The historical v1 figure this block used to quote stands as measured: on a
|
||||
# 6.9 GB Bevy workspace target dir the unshared set was .fingerprint 22 MB +
|
||||
# build/ 237 MB + a handful of dep-info files — about 3.7% of the tree, against
|
||||
# 100% for a plain `cp -a`. It describes layout v1 only.
|
||||
# The selection now names the mutable set directly rather than by the container
|
||||
# it used to live in, so it holds under both layouts; the same crate measures
|
||||
# 45.3% (v1) and 38.4% (v2), both dominated by the linked-output rule above
|
||||
# rather than by the layout. On a real 5.5 GB Bevy target dir the whole change
|
||||
# moves the real-copied share from 9.0% to 14.1%.
|
||||
#
|
||||
# `incremental/` is deliberately left shared: rustc writes each incremental
|
||||
# session to a fresh `s-*-working` directory and finalises it with a rename,
|
||||
# and garbage-collects old sessions by unlinking directory entries — neither
|
||||
# of which mutates a shared inode. CI should still set CARGO_INCREMENTAL=0,
|
||||
# for size rather than correctness.
|
||||
# The directories `unshare_mutable_paths` replaces, under either layout.
|
||||
#
|
||||
# All four names are pruned, so nothing selected here can contain anything else
|
||||
# selected here and the caller never unshares a subtree twice.
|
||||
#
|
||||
# `out` is the one that needs deciding rather than naming, and it is the whole
|
||||
# difficulty of layout v2: a compile unit's rlib and a build script's OUT_DIR
|
||||
# are both a directory called `out`, one directory apart, and they need
|
||||
# opposite treatment. The discriminator is structural — Cargo records a build
|
||||
# script's execution beside its OUT_DIR and nowhere else, as `run/root-output`
|
||||
# under v2 and as a loose `root-output` under v1 — so a unit directory holding
|
||||
# one of those is a RUN unit and its `out` is the OUT_DIR. A unit directory
|
||||
# without one is a compile unit, and its `out` holds the artifact whose sharing
|
||||
# is the entire point of the clone.
|
||||
_mutable_dirs() {
|
||||
local root="$1" d unit
|
||||
while IFS= read -r d; do
|
||||
if [ "${d##*/}" = out ]; then
|
||||
unit="${d%/out}"
|
||||
[ -d "$unit/run" ] || [ -e "$unit/root-output" ] || continue
|
||||
fi
|
||||
printf '%s\n' "$d"
|
||||
done < <(find "$root" -type d \
|
||||
\( -name .fingerprint -o -name fingerprint -o -name run -o -name out \) \
|
||||
-prune -print 2>/dev/null)
|
||||
}
|
||||
|
||||
unshare_mutable_paths() {
|
||||
local root="$1" d
|
||||
[ -d "$root" ] || return 0
|
||||
# The list is materialised in full before anything is replaced: each
|
||||
# replacement deletes and recreates a directory, and a live `find` walk over
|
||||
# a tree being mutated underneath it is a needless hazard. `-prune` keeps a
|
||||
# match's own contents out of the list.
|
||||
# a tree being mutated underneath it is a needless hazard.
|
||||
local -a dirs=()
|
||||
mapfile -t dirs < <(find "$root" -type d \( -name .fingerprint -o -name build \) -prune -print 2>/dev/null)
|
||||
mapfile -t dirs < <(_mutable_dirs "$root")
|
||||
for d in "${dirs[@]}"; do
|
||||
[ -n "$d" ] || continue
|
||||
unshare_subtree "$d" || {
|
||||
@@ -435,6 +469,30 @@ unshare_mutable_paths() {
|
||||
echo "::error::unshare_mutable_paths: failed to unshare dep-info files under ${root}" >&2
|
||||
return 1
|
||||
}
|
||||
# Layout v1's build-script run metadata, which v2 groups under `run/` and v1
|
||||
# leaves loose in the run unit's directory. `invoked.timestamp` is empty and
|
||||
# carries its meaning in its mtime, which a shared inode carries too.
|
||||
_unshare_files "$root" -type f \
|
||||
\( -name output -o -name root-output -o -name stderr -o -name invoked.timestamp \) || {
|
||||
echo "::error::unshare_mutable_paths: failed to unshare build-script run metadata under ${root}" >&2
|
||||
return 1
|
||||
}
|
||||
# Linked outputs. Unlike an rlib or an rmeta — which rustc writes to a
|
||||
# temporary and renames into place — an executable or shared object is
|
||||
# written by the LINKER, and the linker writes THROUGH an existing inode
|
||||
# whenever the path it is given has no other hard link. Measured 2026-08-27
|
||||
# on cargo 1.98.0-nightly (a335d47ff, layout v1) and 1.100.0-nightly
|
||||
# (e8cb624d5, layout v2), mold and the default linker alike: a `cargo test`
|
||||
# binary in a `cp -al` clone rewrote the SOURCE's copy of itself in place,
|
||||
# under both layouts.
|
||||
#
|
||||
# The executable bit is the discriminator because it is the linker's own
|
||||
# output that is at risk, not the directory it happens to land in — `.rlib`,
|
||||
# `.rmeta` and `incremental/` stay shared and they are the bytes that matter.
|
||||
_unshare_files "$root" -type f -perm -u+x || {
|
||||
echo "::error::unshare_mutable_paths: failed to unshare linked outputs under ${root}" >&2
|
||||
return 1
|
||||
}
|
||||
_unshare_files "$root" -maxdepth 3 -type f -name '.rustc_info.json' || {
|
||||
echo "::error::unshare_mutable_paths: failed to unshare .rustc_info.json under ${root}" >&2
|
||||
return 1
|
||||
|
||||
@@ -28,8 +28,6 @@ set -euo pipefail
|
||||
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
||||
. "$script_dir/cache-lib.sh"
|
||||
|
||||
command -v cargo >/dev/null || { echo "SKIP: no cargo on PATH"; exit 0; }
|
||||
|
||||
scratch=$(mktemp -d)
|
||||
trap 'rm -rf "$scratch"' EXIT
|
||||
pass_count=0
|
||||
@@ -59,6 +57,14 @@ version = "0.1.0"
|
||||
edition = "2021"
|
||||
[workspace]
|
||||
TOML
|
||||
# A binary as well as a library, because the two are written differently and
|
||||
# only one of them is safe to share: rustc writes an rlib to a temporary and
|
||||
# renames it into place, while the LINKER writes an executable through the
|
||||
# existing inode. Without a bin target this suite never relinks anything and
|
||||
# cannot see that difference.
|
||||
cat > "$dir/src/main.rs" <<'RS'
|
||||
fn main() { println!("{}", probe::f()); }
|
||||
RS
|
||||
cat > "$dir/build.rs" <<'RS'
|
||||
use std::{env, fs, path::PathBuf};
|
||||
fn main() {
|
||||
@@ -70,6 +76,208 @@ fn main() {
|
||||
RS
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Both build-dir layouts, without a compiler
|
||||
# ---------------------------------------------------------------------------
|
||||
#
|
||||
# Every other scenario in this file runs whichever layout the installed Cargo
|
||||
# happens to write, so on any one machine it exercises exactly ONE of the two —
|
||||
# and gitdan-ci's is v1. The two fixtures below reproduce both directory shapes
|
||||
# from files alone, clone them through the real `hardlink_clone_into()`, and
|
||||
# assert file by file which side of the partition each one lands on.
|
||||
#
|
||||
# Shapes taken from a scratch crate (serde + serde_json + regex, plus a build
|
||||
# script) built on 2026-08-27: cargo 1.98.0-nightly (a335d47ff 2026-06-26)
|
||||
# writes v1, cargo 1.100.0-nightly (e8cb624d5 2026-08-22) writes v2.
|
||||
#
|
||||
# v2 is where the partition is easy to get wrong, and the fixtures are built to
|
||||
# say so: a build script's OUT_DIR and a compile unit's rlib are BOTH a
|
||||
# directory called `out`, one directory apart, and they need opposite
|
||||
# treatment.
|
||||
mkfile() { mkdir -p "$(dirname "$1")"; printf '%s' "$2" > "$1"; }
|
||||
|
||||
# Big enough that the byte-fraction assertion below measures something.
|
||||
artifact_bytes=$(head -c 4096 /dev/zero | tr '\0' 'A')
|
||||
|
||||
# Spec lines are `<shared|private>|<path relative to the tree root>`.
|
||||
assert_partition() {
|
||||
local label="$1" src="$2" spec="$3"
|
||||
local clone="${src}-clone" want rel si ci total=0 copied=0 sz
|
||||
hardlink_clone_into "$src" "$clone" "selftest-$label" \
|
||||
|| fail "$label: hardlink_clone_into refused the destination"
|
||||
while IFS='|' read -r want rel; do
|
||||
[ -n "${rel:-}" ] || continue
|
||||
[ -e "$clone/$rel" ] || fail "$label: $rel is missing from the clone"
|
||||
si=$(stat -c '%i' "$src/$rel"); ci=$(stat -c '%i' "$clone/$rel")
|
||||
case "$want" in
|
||||
shared)
|
||||
[ "$si" = "$ci" ] \
|
||||
|| fail "$label: $rel was real-copied, but it is an artifact and must stay shared" ;;
|
||||
private)
|
||||
[ "$si" != "$ci" ] \
|
||||
|| fail "$label: $rel still shares an inode with the source, so a build in the clone can rewrite it" ;;
|
||||
*) fail "$label: unknown spec verb '$want'" ;;
|
||||
esac
|
||||
done <<< "$spec"
|
||||
ok "$label: every file landed on the right side of the partition"
|
||||
|
||||
# The cost model, asserted rather than assumed. Selecting too much is not a
|
||||
# correctness bug, which is exactly why nothing caught layout v2 taking the
|
||||
# selection from a fifth of the tree to 99.998% of it (gitdan-actions#14):
|
||||
# a hardlink clone that real-copies everything is a `cp -a` with extra steps.
|
||||
# The bound is loose on purpose. It is not a budget — the honest figure moves
|
||||
# with how much of a tree is linker output, and these fixtures are mostly
|
||||
# that by construction — it is a floor under "still a hardlink clone at all".
|
||||
while IFS= read -r rel; do
|
||||
# `.cargo-*lock*` is stripped from every clone by design, so it has no
|
||||
# counterpart to compare against.
|
||||
[ -e "$clone/$rel" ] || continue
|
||||
sz=$(stat -c '%s' "$src/$rel")
|
||||
total=$((total + sz))
|
||||
[ "$(stat -c '%i' "$src/$rel")" = "$(stat -c '%i' "$clone/$rel")" ] || copied=$((copied + sz))
|
||||
done < <(cd "$src" && find . -type f -printf '%P\n')
|
||||
[ "$total" -gt 0 ] || fail "$label: fixture has no bytes to measure"
|
||||
[ $((copied * 100 / total)) -lt 90 ] \
|
||||
|| fail "$label: the clone real-copied $((copied * 100 / total))% of its source's bytes — the hardlink saving is gone"
|
||||
ok "$label: clone real-copies $((copied * 100 / total))% of ${total} B (${copied} B), the rest is shared"
|
||||
}
|
||||
|
||||
# Layout v2: no `.fingerprint`, no `deps`. Everything regroups per build unit
|
||||
# under `build/<pkg>/<hash>/{fingerprint,out,run}`, artifacts included — which
|
||||
# is what took `-name build` from "the metadata" to "the whole tree".
|
||||
#
|
||||
# THE THREE UNIT KINDS ARE THE POINT. `1bf...` is a compile unit: its `out`
|
||||
# holds the rlib. `08c...` is the build script's own compile unit: its `out`
|
||||
# holds the build-script binary. `da9...` is the build-script RUN unit: its
|
||||
# `out` IS the OUT_DIR, and it is the only one of the three whose `out` is
|
||||
# mutable. The `run/` directory beside it is the structural difference.
|
||||
v2="$scratch/layout-v2"
|
||||
mkfile "$v2/CACHEDIR.TAG" 'Signature: 8a477f597d28d172'
|
||||
mkfile "$v2/.rustc_info.json" '{"rustc_fingerprint":1}'
|
||||
mkfile "$v2/debug/.cargo-lock" ''
|
||||
mkfile "$v2/debug/libprobe.rlib" "$artifact_bytes"
|
||||
mkfile "$v2/debug/libprobe.d" '/probe/src/lib.rs:'
|
||||
for f in dep-lib-probe lib-probe lib-probe.json invoked.timestamp; do
|
||||
mkfile "$v2/debug/build/probe/1bf5493368dce3cd/fingerprint/$f" "$f"
|
||||
done
|
||||
mkfile "$v2/debug/build/probe/1bf5493368dce3cd/out/libprobe-1bf5493368dce3cd.rlib" "$artifact_bytes"
|
||||
mkfile "$v2/debug/build/probe/1bf5493368dce3cd/out/libprobe-1bf5493368dce3cd.rmeta" "$artifact_bytes"
|
||||
mkfile "$v2/debug/build/probe/1bf5493368dce3cd/out/probe-1bf5493368dce3cd.d" '/probe/src/lib.rs:'
|
||||
for f in build-script-build-script-build build-script-build-script-build.json \
|
||||
dep-build-script-build-script-build invoked.timestamp; do
|
||||
mkfile "$v2/debug/build/probe/08c7dda6eacd6dca/fingerprint/$f" "$f"
|
||||
done
|
||||
mkfile "$v2/debug/build/probe/08c7dda6eacd6dca/out/build_script_build" "$artifact_bytes"
|
||||
chmod +x "$v2/debug/build/probe/08c7dda6eacd6dca/out/build_script_build"
|
||||
mkfile "$v2/debug/build/probe/08c7dda6eacd6dca/out/build_script_build.d" '/probe/build.rs:'
|
||||
# A test binary: the same `out` directory as the rlib above, and the largest
|
||||
# thing in a real tree that a linker writes.
|
||||
for f in dep-test-lib-probe test-lib-probe test-lib-probe.json invoked.timestamp; do
|
||||
mkfile "$v2/debug/build/probe/6a091d813b2be60d/fingerprint/$f" "$f"
|
||||
done
|
||||
mkfile "$v2/debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d" "$artifact_bytes"
|
||||
chmod +x "$v2/debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d"
|
||||
mkfile "$v2/debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d.d" '/probe/src/lib.rs:'
|
||||
for f in run-build-script-build-script-build run-build-script-build-script-build.json; do
|
||||
mkfile "$v2/debug/build/probe/da96cf45111f80dd/fingerprint/$f" "$f"
|
||||
done
|
||||
mkfile "$v2/debug/build/probe/da96cf45111f80dd/out/gen.txt" 'generated from 24 bytes'
|
||||
for f in invoked.timestamp root-output stdout stderr; do
|
||||
mkfile "$v2/debug/build/probe/da96cf45111f80dd/run/$f" "$f"
|
||||
done
|
||||
mkfile "$v2/debug/incremental/probe-abc/s-xyz/dep-graph.bin" "$artifact_bytes"
|
||||
|
||||
assert_partition "layout v2" "$v2" "$(cat <<'SPEC'
|
||||
private|.rustc_info.json
|
||||
private|debug/libprobe.d
|
||||
private|debug/build/probe/1bf5493368dce3cd/fingerprint/dep-lib-probe
|
||||
private|debug/build/probe/1bf5493368dce3cd/fingerprint/lib-probe
|
||||
private|debug/build/probe/1bf5493368dce3cd/fingerprint/lib-probe.json
|
||||
private|debug/build/probe/1bf5493368dce3cd/fingerprint/invoked.timestamp
|
||||
private|debug/build/probe/1bf5493368dce3cd/out/probe-1bf5493368dce3cd.d
|
||||
private|debug/build/probe/08c7dda6eacd6dca/fingerprint/dep-build-script-build-script-build
|
||||
private|debug/build/probe/08c7dda6eacd6dca/fingerprint/invoked.timestamp
|
||||
private|debug/build/probe/08c7dda6eacd6dca/out/build_script_build.d
|
||||
private|debug/build/probe/da96cf45111f80dd/fingerprint/run-build-script-build-script-build
|
||||
private|debug/build/probe/da96cf45111f80dd/out/gen.txt
|
||||
private|debug/build/probe/da96cf45111f80dd/run/root-output
|
||||
private|debug/build/probe/da96cf45111f80dd/run/stdout
|
||||
private|debug/build/probe/da96cf45111f80dd/run/stderr
|
||||
private|debug/build/probe/da96cf45111f80dd/run/invoked.timestamp
|
||||
shared|debug/libprobe.rlib
|
||||
shared|debug/build/probe/1bf5493368dce3cd/out/libprobe-1bf5493368dce3cd.rlib
|
||||
shared|debug/build/probe/1bf5493368dce3cd/out/libprobe-1bf5493368dce3cd.rmeta
|
||||
private|debug/build/probe/08c7dda6eacd6dca/out/build_script_build
|
||||
private|debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d
|
||||
private|debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d.d
|
||||
private|debug/build/probe/6a091d813b2be60d/fingerprint/dep-test-lib-probe
|
||||
shared|debug/incremental/probe-abc/s-xyz/dep-graph.bin
|
||||
SPEC
|
||||
)"
|
||||
# Layout v1: one `.fingerprint` and one `deps` per profile; `build/<pkg>-<hash>`
|
||||
# holds the build script's compiled binary in one unit directory and its run
|
||||
# metadata plus OUT_DIR in another.
|
||||
v1="$scratch/layout-v1"
|
||||
mkfile "$v1/CACHEDIR.TAG" 'Signature: 8a477f597d28d172'
|
||||
mkfile "$v1/.rustc_info.json" '{"rustc_fingerprint":1}'
|
||||
mkfile "$v1/debug/.cargo-lock" ''
|
||||
mkfile "$v1/debug/libprobe.rlib" "$artifact_bytes"
|
||||
mkfile "$v1/debug/libprobe.d" '/probe/src/lib.rs:'
|
||||
mkfile "$v1/debug/deps/libprobe-1bf5493368dce3cd.rlib" "$artifact_bytes"
|
||||
mkfile "$v1/debug/deps/libprobe-1bf5493368dce3cd.rmeta" "$artifact_bytes"
|
||||
mkfile "$v1/debug/deps/probe-1bf5493368dce3cd.d" '/probe/src/lib.rs:'
|
||||
# A test binary, which layout v1 leaves in `deps/` beside the rlibs.
|
||||
mkfile "$v1/debug/deps/probe-6a091d813b2be60d" "$artifact_bytes"
|
||||
chmod +x "$v1/debug/deps/probe-6a091d813b2be60d"
|
||||
mkfile "$v1/debug/deps/probe-6a091d813b2be60d.d" '/probe/src/lib.rs:'
|
||||
for f in dep-lib-probe lib-probe lib-probe.json invoked.timestamp; do
|
||||
mkfile "$v1/debug/.fingerprint/probe-1bf5493368dce3cd/$f" "$f"
|
||||
done
|
||||
mkfile "$v1/debug/build/probe-08c7dda6eacd6dca/build-script-build" "$artifact_bytes"
|
||||
mkfile "$v1/debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca" "$artifact_bytes"
|
||||
chmod +x "$v1/debug/build/probe-08c7dda6eacd6dca/build-script-build" \
|
||||
"$v1/debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca"
|
||||
mkfile "$v1/debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca.d" '/probe/build.rs:'
|
||||
for f in invoked.timestamp output root-output stderr; do
|
||||
mkfile "$v1/debug/build/probe-da96cf45111f80dd/$f" "$f"
|
||||
done
|
||||
mkfile "$v1/debug/build/probe-da96cf45111f80dd/out/gen.txt" 'generated from 24 bytes'
|
||||
mkfile "$v1/debug/incremental/probe-abc/s-xyz/dep-graph.bin" "$artifact_bytes"
|
||||
|
||||
assert_partition "layout v1" "$v1" "$(cat <<'SPEC'
|
||||
private|.rustc_info.json
|
||||
private|debug/libprobe.d
|
||||
private|debug/deps/probe-1bf5493368dce3cd.d
|
||||
private|debug/.fingerprint/probe-1bf5493368dce3cd/dep-lib-probe
|
||||
private|debug/.fingerprint/probe-1bf5493368dce3cd/lib-probe
|
||||
private|debug/.fingerprint/probe-1bf5493368dce3cd/lib-probe.json
|
||||
private|debug/.fingerprint/probe-1bf5493368dce3cd/invoked.timestamp
|
||||
private|debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca.d
|
||||
private|debug/deps/probe-6a091d813b2be60d
|
||||
private|debug/deps/probe-6a091d813b2be60d.d
|
||||
private|debug/build/probe-da96cf45111f80dd/invoked.timestamp
|
||||
private|debug/build/probe-da96cf45111f80dd/output
|
||||
private|debug/build/probe-da96cf45111f80dd/root-output
|
||||
private|debug/build/probe-da96cf45111f80dd/stderr
|
||||
private|debug/build/probe-da96cf45111f80dd/out/gen.txt
|
||||
shared|debug/libprobe.rlib
|
||||
shared|debug/deps/libprobe-1bf5493368dce3cd.rlib
|
||||
shared|debug/deps/libprobe-1bf5493368dce3cd.rmeta
|
||||
private|debug/build/probe-08c7dda6eacd6dca/build-script-build
|
||||
private|debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca
|
||||
shared|debug/incremental/probe-abc/s-xyz/dep-graph.bin
|
||||
SPEC
|
||||
)"
|
||||
|
||||
|
||||
echo
|
||||
command -v cargo >/dev/null || {
|
||||
echo "SKIP: no cargo on PATH — the fixture scenarios above ran, the live-Cargo ones cannot"
|
||||
echo "hardlink-clone-selftest: ${pass_count} assertions passed"
|
||||
exit 0
|
||||
}
|
||||
|
||||
crate_dir="$scratch/probe"
|
||||
mkcrate "$crate_dir"
|
||||
cd "$crate_dir"
|
||||
@@ -304,27 +512,44 @@ build_base "$base_fix"
|
||||
before=$(snapshot_tree "$base_fix")
|
||||
hardlink_clone_into "$base_fix" "$clone_fix" "selftest" || fail "hardlink_clone_into reported the destination already existed"
|
||||
|
||||
# The clone's contract, asserted before anything builds in it: artifacts
|
||||
# share inodes (that is what makes the clone near-free), and every file Cargo
|
||||
# rewrites in place does not (that is what makes it sound). Checking after a
|
||||
# rebuild would prove nothing — the rebuild replaces those files anyway.
|
||||
shared=0; unshared=0
|
||||
# The clone's contract, asserted before anything builds in it and asserted in
|
||||
# BOTH directions: every file Cargo rewrites in place is privately owned (that
|
||||
# is what makes the clone sound), and every artifact still shares its inode
|
||||
# (that is what makes it near-free). Checking after a rebuild would prove
|
||||
# nothing — the rebuild replaces those files anyway.
|
||||
#
|
||||
# The mutable-family patterns cover both layouts: `.fingerprint/` is v1's,
|
||||
# `fingerprint/` and `run/` are v2's, and `gen.txt` is this crate's build
|
||||
# script's OUT_DIR product, which under v2 sits in a directory called `out`
|
||||
# beside sibling units whose `out` holds artifacts.
|
||||
shared=0; unshared=0; shared_bytes=0; copied_bytes=0
|
||||
while IFS= read -r f; do
|
||||
rel="${f#"$base_fix"/}"
|
||||
[ -e "$clone_fix/$rel" ] || continue
|
||||
sz=$(stat -c '%s' "$f")
|
||||
if [ "$(stat -c '%i' "$f")" = "$(stat -c '%i' "$clone_fix/$rel")" ]; then
|
||||
case "$rel" in
|
||||
*/.fingerprint/*|*/build/*|*.d|.rustc_info.json)
|
||||
*/.fingerprint/*|*/fingerprint/*|*/run/*|*/out/gen.txt|*/output|*/root-output|*/stderr|*/invoked.timestamp|*.d|.rustc_info.json)
|
||||
fail "mutable path still shares an inode with the source: $rel" ;;
|
||||
esac
|
||||
shared=$((shared + 1))
|
||||
shared=$((shared + 1)); shared_bytes=$((shared_bytes + sz))
|
||||
else
|
||||
unshared=$((unshared + 1))
|
||||
case "$rel" in
|
||||
*.rlib|*.rmeta)
|
||||
fail "artifact was real-copied rather than shared: $rel" ;;
|
||||
esac
|
||||
unshared=$((unshared + 1)); copied_bytes=$((copied_bytes + sz))
|
||||
fi
|
||||
done < <(find "$base_fix" -type f)
|
||||
[ "$shared" -gt 0 ] || fail "nothing is shared — the clone degenerated into a full copy"
|
||||
[ "$unshared" -gt 0 ] || fail "nothing was unshared — unshare_mutable_paths did not run"
|
||||
total_bytes=$((shared_bytes + copied_bytes))
|
||||
ok "fresh clone shares ${shared} artifact files and privately owns ${unshared} mutable ones"
|
||||
# Reported, not asserted. This crate has no dependencies, so almost all of its
|
||||
# bytes are the two executables — a ratio that says nothing about a real tree.
|
||||
# The fixtures above are where the cost model is gated, because there the
|
||||
# composition is fixed.
|
||||
echo " note: this clone real-copies $((copied_bytes * 100 / total_bytes))% of ${total_bytes} B"
|
||||
|
||||
printf '%s\n' "$CONTENT_B" > src/lib.rs
|
||||
CARGO_TARGET_DIR="$clone_fix" "${CARGO_BIN[@]}" build -q
|
||||
@@ -337,6 +562,50 @@ if [ -n "$fix_mutated" ]; then
|
||||
fi
|
||||
ok "no file in the source changed after a full rebuild in the clone"
|
||||
|
||||
echo
|
||||
echo "=== a linked TEST binary, which nothing uplifts and nothing replaces ==="
|
||||
# The one artifact family that is NOT safe to share, and the reason
|
||||
# `unshare_mutable_paths` privately owns every executable. rustc writes an
|
||||
# rlib to a temporary and renames it in; the LINKER writes an executable
|
||||
# through whatever inode is already at the path. Cargo re-creates the path
|
||||
# first when it also has to uplift the result — a bin target's
|
||||
# `deps/<bin>-<hash>` has a hardlink twin at `<profile>/<bin>` — which is why
|
||||
# bin targets look safe. A test binary has no twin, nothing re-creates it, and
|
||||
# the write lands on the source's inode.
|
||||
#
|
||||
# Measured 2026-08-27 on cargo 1.98.0-nightly (a335d47ff, layout v1) and
|
||||
# 1.100.0-nightly (e8cb624d5, layout v2): under BOTH layouts a `cargo test
|
||||
# --no-run` inside a raw `cp -al` clone rewrote the SOURCE's own test binary.
|
||||
base_exe="$scratch/base-exe"; clone_exe_ctl="$scratch/clone-exe-ctl"; clone_exe="$scratch/clone-exe"
|
||||
printf '%s\n' "$CONTENT_A" > src/lib.rs
|
||||
CARGO_TARGET_DIR="$base_exe" "${CARGO_BIN[@]}" test --no-run -q > /dev/null 2>&1
|
||||
before=$(snapshot_tree "$base_exe")
|
||||
cp -al "$base_exe" "$clone_exe_ctl"
|
||||
strip_cargo_locks "$clone_exe_ctl"
|
||||
printf '%s\n' "$CONTENT_B" > src/lib.rs
|
||||
CARGO_TARGET_DIR="$clone_exe_ctl" "${CARGO_BIN[@]}" test --no-run -q > /dev/null 2>&1
|
||||
exe_ctl_mutated=$(mutated_paths "$before" "$(snapshot_tree "$base_exe")")
|
||||
[ -n "$exe_ctl_mutated" ] \
|
||||
|| fail "control: a raw cp -al clone of a test build no longer mutates the source — the test can no longer tell fixed from broken"
|
||||
ok "raw cp -al clone of a test build mutates the source ($(printf '%s\n' "$exe_ctl_mutated" | wc -l) paths)"
|
||||
printf '%s\n' "$exe_ctl_mutated" | sed 's/^/ /'
|
||||
|
||||
# Rebuild the base from CONTENT_A so it is warm and consistent again, then do
|
||||
# the same thing through the real clone.
|
||||
printf '%s\n' "$CONTENT_A" > src/lib.rs
|
||||
CARGO_TARGET_DIR="$base_exe" "${CARGO_BIN[@]}" test --no-run -q > /dev/null 2>&1
|
||||
before=$(snapshot_tree "$base_exe")
|
||||
hardlink_clone_into "$base_exe" "$clone_exe" "selftest-exe" \
|
||||
|| fail "hardlink_clone_into refused the destination"
|
||||
printf '%s\n' "$CONTENT_B" > src/lib.rs
|
||||
CARGO_TARGET_DIR="$clone_exe" "${CARGO_BIN[@]}" test --no-run -q > /dev/null 2>&1
|
||||
exe_mutated=$(mutated_paths "$before" "$(snapshot_tree "$base_exe")")
|
||||
if [ -n "$exe_mutated" ]; then
|
||||
printf '%s\n' "$exe_mutated" | sed 's/^/ /' >&2
|
||||
fail "a test build in the clone mutated the source through a shared inode"
|
||||
fi
|
||||
ok "no file in the source changed after a full test build in the clone"
|
||||
|
||||
echo
|
||||
if [ "$CHECKSUM_MODE" = "on" ]; then
|
||||
echo "=== the whole point: the source's next build is still correct ==="
|
||||
|
||||
@@ -53,9 +53,11 @@ pass_count=0
|
||||
fail() { echo "ASSERTION FAILED: $*" >&2; exit 1; }
|
||||
ok() { pass_count=$((pass_count + 1)); echo "PASS: $*"; }
|
||||
|
||||
# Cargo and rustc REPLACE an artifact (write elsewhere, rename over the path)
|
||||
# rustc REPLACES an `.rlib`/`.rmeta` (writes elsewhere, renames over the path)
|
||||
# rather than truncating it in place, which is exactly why a snapshot may
|
||||
# share artifact inodes with the live target dir it was cloned from. The
|
||||
# share those inodes with the live target dir it was cloned from. Linker
|
||||
# outputs are the exception and are real-copied instead — see
|
||||
# `unshare_mutable_paths` in cache-lib.sh. The
|
||||
# fixtures here have to model that faithfully — a plain `>` redirect truncates
|
||||
# in place and would write straight through the shared inode into the
|
||||
# snapshot and every consumer, which is a property of the test fixture, not of
|
||||
|
||||
Reference in New Issue
Block a user