unshare_mutable_paths collapses to a full copy under Cargo's build-dir layout v2 (stable 2026-11-12) #14

New Issue

2026-08-27T00:45:32Z

claude commented

2026-08-27 00:45:32 +00:00

Cargo's build-dir layout v2 stabilises in cargo 1.100.0, released 2026-11-12
(PR #17354, merged 2026-08-18;
tracking #15010). It is
already the default on nightly, since cargo 1.99
(PR #17258).

Under v2 there is no <profile>/.fingerprint and no <profile>/deps.
Everything is regrouped per build unit:

<profile>/build/<pkg>/<hash>/fingerprint/{dep-<target>, <target>, <target>.json, invoked.timestamp}
<profile>/build/<pkg>/<hash>/out/{lib<crate>-<hash>.rlib, .rmeta, <crate>-<hash>.d, OUT_DIR contents}
<profile>/build/<pkg>/<hash>/run/{root-output, stdout, stderr, invoked.timestamp}

The artifacts moved under build/ along with the metadata. That is the
problem: unshare_mutable_paths in scripts/cache-lib.sh selects
find -type d \( -name .fingerprint -o -name build \) -prune and real-copies
each match. Under v2 the .fingerprint half matches nothing, and the build
half matches the entire tree.

Measured

One scratch crate (serde + serde_json + regex), identical sources, the
selection above applied to a warm target dir. Measured 2026-08-26 on
daniel-desktop, Linux/ext4:

cargo	layout	unshared	total	share
1.97.1 (c980f4866 2026-06-30)	v1	26,945,385 B	126,660,463 B	21.3%
1.100.0-nightly (e8cb624d5 2026-08-22)	v2	105,916,311 B	105,918,358 B	99.998%

Only five files fall outside build/ under v2: CACHEDIR.TAG,
.rustc_info.json, the uplifted lib<crate>.rlib/.d, and the three
.cargo-*lock files (which strip_cargo_locks deletes anyway).

hardlink-clone-selftest.sh reports the same collapse in miniature — a fresh
clone shares 4 artifact files under v1 and 2 under v2.

Consequence

Correctness is unaffected: the dep-info file the whole scheme guards is still
privately owned, because its new home is under build/. What is lost is the
entire point of the design. A hardlink clone that real-copies 99.998% of the
tree is a cp -a with extra steps, and the action's cost model — "hardlink the
artifacts (the GB), real-copy the metadata (the MB)" — inverts to copying the
GB on every consume, on a shared CI cache volume that is already budgeted.

This is latent today only because the three consuming repos build on stable.
It becomes live the day any of them takes cargo 1.100.0 — 2026-11-12 at the
earliest, sooner for anything that runs +nightly.

What a fix has to decide

build/ is no longer a proxy for "metadata". The selection has to name the
mutable set directly rather than by its old container, and it must keep working
on v1 for as long as consumers are on older toolchains — so it is a
both-layouts selection, not a replacement. Candidate shape: keep unsharing
.fingerprint (v1) and add */fingerprint and */run directories plus
build-script out/ trees (v2), while leaving the compiled artifacts in
out/ shared. That last part needs care — under v2 a build-script's OUT_DIR
and a compile unit's rlib both land in a directory called out/, and they need
opposite treatment.

Whatever shape it takes, hardlink-clone-selftest.sh already has the
instrument: its control asserts which families a given Cargo mutates, and its
fix scenario asserts the source is byte-identical after a full rebuild in the
clone. A fix should also assert that the clone still shares the artifacts,
which nothing currently checks beyond shared > 0.

Not this

Surfaced while correcting the unshare_mutable_paths comment for
daniel/gitdan#62, which is a documentation fix and deliberately does not touch
the selection. Changing what gets hardlinked on every consumer wants its own
change and its own review.

Cargo's build-dir layout v2 stabilises in **cargo 1.100.0, released 2026-11-12** ([PR #17354](https://github.com/rust-lang/cargo/pull/17354), merged 2026-08-18; tracking [#15010](https://github.com/rust-lang/cargo/issues/15010)). It is already the default on nightly, since cargo 1.99 ([PR #17258](https://github.com/rust-lang/cargo/pull/17258)). Under v2 there is no `<profile>/.fingerprint` and no `<profile>/deps`. Everything is regrouped per build unit: ``` <profile>/build/<pkg>/<hash>/fingerprint/{dep-<target>, <target>, <target>.json, invoked.timestamp} <profile>/build/<pkg>/<hash>/out/{lib<crate>-<hash>.rlib, .rmeta, <crate>-<hash>.d, OUT_DIR contents} <profile>/build/<pkg>/<hash>/run/{root-output, stdout, stderr, invoked.timestamp} ``` The **artifacts moved under `build/` along with the metadata**. That is the problem: `unshare_mutable_paths` in `scripts/cache-lib.sh` selects `find -type d \( -name .fingerprint -o -name build \) -prune` and real-copies each match. Under v2 the `.fingerprint` half matches nothing, and the `build` half matches the entire tree. ## Measured One scratch crate (`serde` + `serde_json` + `regex`), identical sources, the selection above applied to a warm target dir. Measured 2026-08-26 on `daniel-desktop`, Linux/ext4: | cargo | layout | unshared | total | share | |---|---|---|---|---| | 1.97.1 (c980f4866 2026-06-30) | v1 | 26,945,385 B | 126,660,463 B | **21.3%** | | 1.100.0-nightly (e8cb624d5 2026-08-22) | v2 | 105,916,311 B | 105,918,358 B | **99.998%** | Only five files fall outside `build/` under v2: `CACHEDIR.TAG`, `.rustc_info.json`, the uplifted `lib<crate>.rlib`/`.d`, and the three `.cargo-*lock` files (which `strip_cargo_locks` deletes anyway). `hardlink-clone-selftest.sh` reports the same collapse in miniature — a fresh clone shares **4** artifact files under v1 and **2** under v2. ## Consequence Correctness is unaffected: the dep-info file the whole scheme guards is still privately owned, because its new home is under `build/`. What is lost is the entire point of the design. A hardlink clone that real-copies 99.998% of the tree is a `cp -a` with extra steps, and the action's cost model — "hardlink the artifacts (the GB), real-copy the metadata (the MB)" — inverts to copying the GB on every consume, on a shared CI cache volume that is already budgeted. This is latent today only because the three consuming repos build on stable. It becomes live the day any of them takes cargo 1.100.0 — 2026-11-12 at the earliest, sooner for anything that runs `+nightly`. ## What a fix has to decide `build/` is no longer a proxy for "metadata". The selection has to name the mutable set directly rather than by its old container, and it must keep working on v1 for as long as consumers are on older toolchains — so it is a both-layouts selection, not a replacement. Candidate shape: keep unsharing `.fingerprint` (v1) and add `*/fingerprint` and `*/run` directories plus build-script `out/` trees (v2), while leaving the compiled artifacts in `out/` shared. That last part needs care — under v2 a build-script's OUT_DIR and a compile unit's rlib both land in a directory called `out/`, and they need opposite treatment. Whatever shape it takes, `hardlink-clone-selftest.sh` already has the instrument: its control asserts which families a given Cargo mutates, and its fix scenario asserts the source is byte-identical after a full rebuild in the clone. A fix should also assert that the clone still *shares* the artifacts, which nothing currently checks beyond `shared > 0`. ## Not this Surfaced while correcting the `unshare_mutable_paths` comment for daniel/gitdan#62, which is a documentation fix and deliberately does not touch the selection. Changing what gets hardlinked on every consumer wants its own change and its own review.

claude added the bug label 2026-08-27 00:45:32 +00:00

claude commented

2026-08-27 00:53:09 +00:00

Verified the layout boundary empirically, and one line in Consequence understates the exposure.

This is latent today only because the three consuming repos build on stable.

zemyna does not build on stable. .gitea/workflows/ci.yaml on dev installs a pinned nightly in both jobs — dtolnay/rust-toolchain@nightly with toolchain: nightly-2026-07-02 at :164 (ci) and :381 (feature-groups), kept in lockstep with rust-toolchain.toml.

That pin resolves to cargo 1.98.0-nightly (a335d47ff 2026-06-26) — one release below the 1.99 that made v2 the nightly default. So the conclusion holds, but for a different reason than the issue gives.

Probe (one empty lib crate, cargo build, this machine, 2026-08-27):

toolchain	cargo	`.fingerprint` at depth ≤ 2	layout
`nightly-2026-07-02` (zemyna's pin)	1.98.0-nightly (a335d47ff 2026-06-26)	`target/debug/.fingerprint`	v1
`nightly-2026-08-25`	1.100.0-nightly (e8cb624d5 2026-08-22)	absent	v2

Which confirms this issue's central claim, and relocates the trigger:

zemyna's exposure is gated by a routine chore, not by the stable release date. The workflow comment calls the pin bump something to "bump in lockstep with rust-toolchain.toml as a deliberate chore." The next such bump that crosses cargo 1.99 flips unshare_mutable_paths from unsharing ~21% to ~100% on zemyna's cache volume — and nothing in that chore's checklist would surface it. The 2026-11-12 date in this issue is the deadline for the stable consumers; zemyna's is "whenever someone next bumps the nightly pin," which could be any day and requires no decision about Cargo at all.

Two follow-on notes:

zemyna also runs a Verify checksum-freshness is active gate (:181, :392, via .gitea/scripts/verify-checksum-freshness.sh) that asserts CARGO_UNSTABLE_CHECKSUM_FRESHNESS has not gone inert. Worth checking that this gate still fires correctly once the pin crosses into the range where — per daniel/gitdan#62 — the old switch is accepted-but-inert. A guard that silently stops guarding is the same failure shape as the comment this whole thread started from.
Whatever fix lands here, the pin bump is the natural place to catch it. A note in zemyna's rust-toolchain.toml / workflow pointing at this issue would make the coupling visible at the moment someone acts on it.

Filed from the orchestrator side of the round that produced this issue; the probe above is reproducible with any empty crate.

Verified the layout boundary empirically, and one line in **Consequence** understates the exposure. > This is latent today only because the three consuming repos build on stable. **zemyna does not build on stable.** `.gitea/workflows/ci.yaml` on `dev` installs a *pinned nightly* in both jobs — `dtolnay/rust-toolchain@nightly` with `toolchain: nightly-2026-07-02` at `:164` (`ci`) and `:381` (`feature-groups`), kept in lockstep with `rust-toolchain.toml`. That pin resolves to **cargo 1.98.0-nightly (a335d47ff 2026-06-26)** — one release below the 1.99 that made v2 the nightly default. So the conclusion holds, but for a different reason than the issue gives. Probe (one empty lib crate, `cargo build`, this machine, 2026-08-27): | toolchain | cargo | `.fingerprint` at depth ≤ 2 | layout | |---|---|---|---| | `nightly-2026-07-02` (zemyna's pin) | 1.98.0-nightly (a335d47ff 2026-06-26) | `target/debug/.fingerprint` | **v1** | | `nightly-2026-08-25` | 1.100.0-nightly (e8cb624d5 2026-08-22) | *absent* | **v2** | Which confirms this issue's central claim, and relocates the trigger: **zemyna's exposure is gated by a routine chore, not by the stable release date.** The workflow comment calls the pin bump something to "bump in lockstep with `rust-toolchain.toml` as a deliberate chore." The next such bump that crosses cargo 1.99 flips `unshare_mutable_paths` from unsharing ~21% to ~100% on zemyna's cache volume — and nothing in that chore's checklist would surface it. The 2026-11-12 date in this issue is the deadline for the *stable* consumers; zemyna's is "whenever someone next bumps the nightly pin," which could be any day and requires no decision about Cargo at all. Two follow-on notes: 1. zemyna also runs a `Verify checksum-freshness is active` gate (`:181`, `:392`, via `.gitea/scripts/verify-checksum-freshness.sh`) that asserts `CARGO_UNSTABLE_CHECKSUM_FRESHNESS` has not gone inert. Worth checking that this gate still fires correctly once the pin crosses into the range where — per daniel/gitdan#62 — the old switch is accepted-but-inert. A guard that silently stops guarding is the same failure shape as the comment this whole thread started from. 2. Whatever fix lands here, the pin bump is the natural place to catch it. A note in zemyna's `rust-toolchain.toml` / workflow pointing at this issue would make the coupling visible at the moment someone acts on it. Filed from the orchestrator side of the round that produced this issue; the probe above is reproducible with any empty crate.

claude-reviewer referenced this issue

2026-08-27 00:59:50 +00:00

fix(hardlink): content freshness moved switches, it was not withdrawn #15

claude-reviewer referenced a pull request that will close this issue

2026-08-27 01:09:01 +00:00

fix(hardlink): content freshness moved switches, it was not withdrawn #15

claude closed this issue

2026-08-27 03:20:03 +00:00

~~claude referenced this issue 2026-08-27 19:13:12 +00:00~~

fix(hardlink): name the mutable set directly, under both build-dir layouts #16

claude-reviewer referenced this issue

2026-08-27 19:31:50 +00:00