fix(hardlink): name the mutable set directly, under both build-dir layouts
CI / shellcheck + selftests (pull_request) Skipped
CI / shellcheck + selftests (pull_request) Skipped
`unshare_mutable_paths` selected `.fingerprint` and `build` directories. Under Cargo's build-dir layout v2 the first clause matches nothing and the second matches the whole tree, because v2 regroups artifacts under `build/` alongside the metadata. Measured on one scratch crate: 39.3% of the tree real-copied under v1, 99.996% under v2. The selection now names the mutable set rather than the container it used to live in: fingerprint directories under either spelling, layout v2's `run/` directories, layout v1's loose build-script run metadata, and the `out` directories that are a build script's OUT_DIR rather than a compile unit's artifact directory. The two are told apart structurally, by Cargo's record of the build-script execution sitting beside the OUT_DIR and nowhere else. Verifying that turned up a second, layout-independent hazard: a linked executable is written through whatever inode is already at its path, so a `cargo test --no-run` inside a `cp -al` clone rewrites the source's own test binary. Reproduced on cargo 1.93.1 stable, 1.96.0-nightly, 1.98.0-nightly and 1.100.0-nightly, under both layouts. Every executable is now real-copied; `.rlib`, `.rmeta` and `incremental/` are what stay shared. hardlink-clone-selftest.sh gains two file-only layout fixtures that pin the partition in both directions without a compiler, and a live scenario that relinks a test binary.
This commit is contained in:
@@ -29,7 +29,7 @@ on the runner having a single execution slot.
|
||||
|
||||
One thing neither project had, and the reason the clone is not a plain
|
||||
`cp -al`: **a build inside a hardlink clone does mutate the directory it was
|
||||
cloned from.** Cargo writes real artifacts by replacing them, but writes its
|
||||
cloned from.** rustc renames its own outputs into place, but Cargo writes its
|
||||
metadata — and build scripts write their `OUT_DIR` — with a plain truncating
|
||||
write, straight through the shared inode. When Cargo resolves freshness by
|
||||
content the file that gets corrupted is its dep-info fingerprint —
|
||||
@@ -37,17 +37,34 @@ content the file that gets corrupted is its dep-info fingerprint —
|
||||
`build/<pkg>/<hash>/fingerprint/dep-<target>` under Cargo's build-dir layout v2
|
||||
— which holds the per-source checksums that decide freshness, and the failure
|
||||
is silent stale-artifact reuse rather than a slow build.
|
||||
`scripts/hardlink-clone-selftest.sh` reproduces it as an explicit control and
|
||||
asserts the fix. The fix is to hardlink the artifacts (the GB) and real-copy
|
||||
the metadata (the MB) — about 3.7% of a Bevy-sized target directory, against
|
||||
100% for a full copy.
|
||||
|
||||
That ratio is a **layout-v1** figure and does not survive layout v2, which
|
||||
regroups artifacts under `build/` alongside the metadata and so drags nearly
|
||||
the whole tree into the real-copy set. Measured 2026-08-26 at 99.998% on a
|
||||
scratch crate. v2 is the nightly default and stabilises in cargo 1.100.0 on
|
||||
2026-11-12; the correctness guarantee above is unaffected, the saving is not.
|
||||
Tracked as [#14](https://gitdan.com/daniel/gitdan-actions/issues/14).
|
||||
One more family joins them, and it is not metadata: **anything the linker
|
||||
writes**. rustc writes an `.rlib` or `.rmeta` to a temporary and renames it
|
||||
into place, but a linked executable is written through whatever inode is
|
||||
already at its path — so a `cargo test --no-run` in a raw `cp -al` clone
|
||||
rewrites the source's own test binary. Measured 2026-08-27 on cargo 1.93.1
|
||||
stable, 1.96.0-nightly, 1.98.0-nightly and 1.100.0-nightly, under both
|
||||
build-dir layouts.
|
||||
|
||||
`scripts/hardlink-clone-selftest.sh` reproduces both as explicit controls and
|
||||
asserts the fix. The fix is to hardlink what rustc renames into place — the
|
||||
`.rlib`, `.rmeta` and `incremental/` bulk — and real-copy the metadata and the
|
||||
linker outputs.
|
||||
|
||||
**The selection names that set directly, under either build-dir layout.**
|
||||
Layout v2 regroups everything per build unit under
|
||||
`build/<pkg>/<hash>/{fingerprint,out,run}/`, artifacts included, so there is no
|
||||
`.fingerprint` and no `deps` to key off and `build/` is no longer a proxy for
|
||||
"metadata" — it is the whole tree. The one place the two layouts genuinely
|
||||
differ is that under v2 a build script's `OUT_DIR` and a compile unit's rlib
|
||||
are both a directory called `out`; the run unit is told apart by Cargo's record
|
||||
of the execution beside it (`run/root-output` under v2, a loose `root-output`
|
||||
under v1). v2 is the nightly default and stabilises in cargo 1.100.0 on
|
||||
2026-11-12.
|
||||
|
||||
Measured real-copied share of a 5.5 GB Bevy target directory: **9.0% before,
|
||||
14.1% after** — the increase is the linker-output rule, not the layout work.
|
||||
On a scratch crate the layout fix alone takes v2 from 99.996% to 0.2%.
|
||||
|
||||
---
|
||||
|
||||
@@ -578,7 +595,7 @@ change here reaches all of them at once. That is what the gate is for.
|
||||
| suite | covers |
|
||||
|---|---|
|
||||
| `cache-root-selftest.sh` | that a lineage nests one level and nothing else moves: no lineage resolves byte-for-byte to the cache root, two lineages on one cache key get disjoint target dirs, seed/publish/prune all stay inside their own lineage, a PR layers over its own lineage's base snapshot — **and one rejection per lineage name a reader elsewhere would stop seeing**, plus the publish-side mismatch guard |
|
||||
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly that actually resolves freshness by content for its last scenario**: the source's-next-build check reasons about content rather than mtime, so under mtime freshness it would assert a bug. Whether the toolchain does is settled by experiment on a throwaway crate, not by asking it — accepting `-Z checksum-freshness` stopped implying it on 2026-08-22, when cargo PR #17382 demoted the flag to a gate and gave `build.fingerprint` (default `mtime`) the choice; the suite now exports both and 1.100.0-nightly measures ACTIVE again. The experiment reports **three** outcomes, not two: active, measured-inactive, and *not measured*. Its answer codes are `0` and `3`, deliberately clear of every status bash generates for its own errors — so nothing that goes wrong inside the probe, including an expansion failure no guard can catch, can be read as an answer. The scenario is skipped for the last two alike, but a failure to measure is never reported as a measurement. The control also reports which mutation families the running Cargo exhibits — a note, not an assertion, since that set moves upstream. |
|
||||
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly that actually resolves freshness by content for its last scenario**: the source's-next-build check reasons about content rather than mtime, so under mtime freshness it would assert a bug. Whether the toolchain does is settled by experiment on a throwaway crate, not by asking it — accepting `-Z checksum-freshness` stopped implying it on 2026-08-22, when cargo PR #17382 demoted the flag to a gate and gave `build.fingerprint` (default `mtime`) the choice; the suite now exports both and 1.100.0-nightly measures ACTIVE again. The experiment reports **three** outcomes, not two: active, measured-inactive, and *not measured*. Its answer codes are `0` and `3`, deliberately clear of every status bash generates for its own errors — so nothing that goes wrong inside the probe, including an expansion failure no guard can catch, can be read as an answer. The scenario is skipped for the last two alike, but a failure to measure is never reported as a measurement. The control also reports which mutation families the running Cargo exhibits — a note, not an assertion, since that set moves upstream. It also pins the SELECTION itself against both of Cargo's build-dir layouts, from two file-only fixtures that need no compiler — so the layout the installed Cargo does not happen to write is still covered — and asserts the partition in both directions: every file Cargo rewrites in place is privately owned, and every `.rlib`/`.rmeta` still shares its inode. The second half is what the old suite never checked beyond `shared > 0`, and it is what a layout change silently inverts. |
|
||||
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published, and the publisher's log showing it waited on the consumer's own reader-lock marker before reclaiming a rotated snapshot |
|
||||
| `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone |
|
||||
| `prune-cache-selftest.sh` | liveness, protection, locking, eviction order, self-clear, **and that a cache a job claims *inside* the check-to-unlink window survives it** — against a real scratch `origin` |
|
||||
|
||||
Reference in New Issue
Block a user