Author SHA1 Message Date
claude 4bb880b7a7 chore(ci): trigger CI after un-WIP
CI / shellcheck + selftests (pull_request) Successful in 1m21s
2026-08-27 14:52:11 -05:00
claude fb3c72aa28 docs(hardlink): withdraw the nlink claim, which asserted more than was measured
CI / shellcheck + selftests (pull_request) Skipped
The comments said hardlink count at link time was ruled out as the
discriminator between a rewritten executable and an intact one, citing a
lib+bin crate whose two test binaries were both `nlink == 1` and appeared to
behave differently. Re-checked on review: the intact one had not been rebuilt
at all — same content, same inode — so it demonstrated nothing, and forcing
both to rebuild rewrote both.

What was actually observed is narrower and now says so: every executable
measured intact had an uplift hardlink twin Cargo must re-create anyway, every
one measured rewritten had none, and whether the twin is the mechanism or a
correlate was not determined. The rule does not rest on the answer — exempting
twinned executables would recover none of the bytes this change newly copies.

This PR exists because a claim outlived its evidence; it should not ship one.
2026-08-27 14:50:53 -05:00
claude 0553a6b956 fix(hardlink): resolve an ambiguous out/ toward unsharing, and pin the linked-output scenario on a shape that exhibits it
CI / shellcheck + selftests (pull_request) Skipped
Two review findings on #16.

The `out/` discriminator keyed on Cargo's record of a build-script execution,
which Cargo writes only AFTER the script exits successfully. A build script
that populates OUT_DIR and then fails leaves a unit with no record at all, so
its OUT_DIR read as a compile unit's artifact directory and stayed shared —
a regression against the old `-name build` selection, which real-copied that
state by construction. Reproduced on cargo 1.93.1 stable.

An `out` directory now stays shared only when two independent signals agree:
it holds an `.rlib`/`.rmeta` of its own, and its unit carries no execution
record. Either one missing real-copies it. The cost is unchanged to the byte —
the newly-unshared directories hold only executables and `*.d`, both already
privately owned by the file rules.

The live linked-test-binary scenario was built on the lib+bin probe crate,
whose test binaries relink to a fresh inode — a shape gitdan-actions#17 records
as measured safe. Both halves passed green against the unfixed selection on
dep-info mutations the previous scenario already covers. It now builds a
bin-only crate with a unit test, reads only executables, and skips loudly with
a warning rather than passing quietly if the toolchain does not exhibit the
rewrite at all.

Also: drop a clause asserting the linker writes in place "whenever the path has
no other hard link", which this change's own evidence denies; move the
load-bearing comment block back above `unshare_mutable_paths`; correct a
superseded 99.998% figure; and record both cost rows in the README rather than
only the flattering whole-tree one.
2026-08-27 14:39:29 -05:00
claude e5b26a9368 fix(hardlink): name the mutable set directly, under both build-dir layouts
CI / shellcheck + selftests (pull_request) Skipped
`unshare_mutable_paths` selected `.fingerprint` and `build` directories. Under
Cargo's build-dir layout v2 the first clause matches nothing and the second
matches the whole tree, because v2 regroups artifacts under `build/` alongside
the metadata. Measured on one scratch crate: 39.3% of the tree real-copied
under v1, 99.996% under v2.

The selection now names the mutable set rather than the container it used to
live in: fingerprint directories under either spelling, layout v2's `run/`
directories, layout v1's loose build-script run metadata, and the `out`
directories that are a build script's OUT_DIR rather than a compile unit's
artifact directory. The two are told apart structurally, by Cargo's record of
the build-script execution sitting beside the OUT_DIR and nowhere else.

Verifying that turned up a second, layout-independent hazard: a linked
executable is written through whatever inode is already at its path, so a
`cargo test --no-run` inside a `cp -al` clone rewrites the source's own test
binary. Reproduced on cargo 1.93.1 stable, 1.96.0-nightly, 1.98.0-nightly and
1.100.0-nightly, under both layouts. Every executable is now real-copied;
`.rlib`, `.rmeta` and `incremental/` are what stay shared.

hardlink-clone-selftest.sh gains two file-only layout fixtures that pin the
partition in both directions without a compiler, and a live scenario that
relinks a test binary.
2026-08-27 14:12:01 -05:00
claude 4114996954 Merge pull request 'fix(hardlink): content freshness moved switches, it was not withdrawn' (#15) from chore/checksum-freshness into main
CI / shellcheck + selftests (push) Successful in 1m19s
2026-08-27 03:20:03 +00:00
claude fa3cef53e0 docs(ci): the nightly does enable content freshness, as of 2026-08-26
CI / shellcheck + selftests (pull_request) Successful in 1m36s
The README's CI section still described the nightly toolchain step as
buying nothing: "which no nightly currently enables, so it is skipped and
the step is kept only against the day upstream restores it". That
sentence predates this branch and states as fact the exact reading the
rest of the PR retracts.

Three things in this PR falsify it. The corrected `env:` block at
README.md:110-117 records that since cargo PR #17382 (2026-08-22) the
`-Z` gate only unlocks the feature and `build.fingerprint` selects it, so
setting both turns it on. The corrected workflow comment in
.gitea/workflows/ci.yaml says "as of 2026-08-26 it does". And this
branch's own CI run printed `=== checksum-freshness mode: on ===` and
`hardlink-clone-selftest: 4 assertions passed` on 1.100.0-nightly
(787af2b8c 2026-08-25) — the scenario is not skipped, it runs.

The paragraph also carried no date, which is the failure mode every other
block this PR touched was rewritten to prevent. The replacement is dated
and names the toolchain, matching the corrected blocks elsewhere.

It deliberately stops short of "the scenario always runs": the suite
still settles the question by experiment on every run and still skips
loudly when it cannot measure, so the step is not unconditionally
exercised. Saying otherwise would trade one overclaim for its mirror.

Docs-only; no behaviour change.
2026-08-26 20:02:02 -05:00
claude 554310186f fix(hardlink): content freshness moved switches, it was not withdrawn
CI / shellcheck + selftests (pull_request) Successful in 1m17s
`unshare_mutable_paths`' comment recorded that upstream had stopped
rewriting `dep-<target>` in place, on a measurement taken against
1.100.0-nightly. It had not. Two unrelated cargo changes landed within
four days of each other and between them moved the switch that turns the
behaviour on and the path it writes to:

  - cargo PR #17382 (2026-08-22) demoted `-Z checksum-freshness` to a
    gate and gave `build.fingerprint` the choice, defaulting to `mtime`.
    Setting only the gate is accepted and does nothing, which is exactly
    the result that was read as a withdrawal.
  - build-dir layout v2 (cargo PR #17354, stable 1.100.0 on 2026-11-12,
    nightly default since 1.99) moved the file from
    `.fingerprint/<unit>/dep-*` to `build/<pkg>/<hash>/fingerprint/dep-*`.

Measured 2026-08-26 on 1.100.0-nightly (e8cb624d5): same toolchain, same
clone procedure, one env var apart — with the gate alone a `cp -al` clone
mutates only the build/ and *.d families; add
`CARGO_BUILD_FINGERPRINT=content` and the source's dep-info file is
mutated through the shared inode again. The hazard is intact.

So the suite now exports both switches, and its strongest scenario runs
again on current nightlies — verified passing against both layouts. Its
control note learned the v2 path too: it looked for the v1 path only, and
so printed "does NOT rewrite ... in place" three lines beneath a listing
that showed the rewrite.

Every claim these comments make is now dated and cited, because the
defect being fixed is a comment that cited one measurement and silently
stopped reproducing.

Layout v2 also drags the artifacts under `build/`, which collapses this
function's real-copy set from 21.3% of a target dir to 99.998%. That is a
live cost, not a correctness problem, and it is filed as gitdan-actions#14
rather than fixed here.

Part of daniel/gitdan#62.
2026-08-26 19:48:07 -05:00
daniel 50e430f4de Merge pull request 'fix(cache): separate build directories for same-ref jobs, and a CI gate' (#13) from fix/lock-contention into main
CI / shellcheck + selftests (push) Successful in 1m22s
Reviewed-on: #13
Reviewed-by: claude-reviewer <3113+claude-reviewer@noreply.gitdan.com>
2026-08-26 21:38:35 +00:00
claude e94c46f2ec test(hardlink): move the probe's INACTIVE answer off bash's error codes
CI / shellcheck + selftests (pull_request) Successful in 1m21s
Review finding on #13, and the last change to this function. A typo'd variable
name on a line that HAS its `|| exit 2` guard — `$CONTNET_B` for `$CONTENT_B`
— fails in EXPANSION, before the command runs, so neither the guard nor the
ERR trap ever sees it. Under `set -u` that exits 1, which was the code meaning
"measured, and the answer is mtime". Not a live defect: the probe references
only set variables today, and the 1 gitdan-ci reports is a genuine measurement.

The defect is that the answer codes and the error codes overlapped at all.
Three rounds on this function each found a narrower way for a shell-generated
1 to be read as a measurement — an unguarded command, then a guarded line
whose guard could not fire — and each was closed by narrowing the failure
surface, which is a game with no last move.

INACTIVE is now 3, and anything that is not 0 or 3 is NOT MEASURED. Bash
generates 1, 2, 126, 127 and 128+n for its own errors and never 3, so "not an
answer" is decided by a property of the shell rather than by an enumeration of
the ways a step can go wrong. A step added later without a guard, or with a
guard that cannot fire, lands on NOT MEASURED by construction.

The guards and the trap stay, with their job restated: reporting rather than
correctness. They put a failed step on 2 with both probe logs printed instead
of on some incidental status — nicer to debug, and the same destination either
way. The `set +e` bracket at the call site keeps its own note, since a `set -e`
that cannot fire is the kind of thing a reader assumes works.

The not-measured message now quotes the actual exit code, so the two ways to
reach it are distinguishable in a log without reading this file.

Red-proven with the reviewer's own mutation, A/B: with INACTIVE at 1 the typo
reports `mode: off — resolves freshness by mtime`, a false measurement; at 3 it
reports `mode: unmeasured — the probe exited 1`. All four paths re-verified —
unmodified 4 assertions; env stripped inside the probe, measured INACTIVE with
the mtime reason; a broken probe build, unmeasured via a guard; an unguarded
`false`, unmeasured via the trap.
2026-08-26 14:40:27 -05:00
claude 5b46986cd7 test(hardlink): make the errexit invariant real, and stop gpg deciding a gate
CI / shellcheck + selftests (pull_request) Successful in 1m23s
Two review findings on #13.

THE STATED INVARIANT WAS NOT IN FORCE. The header claimed a `set -e` abort
could never be mistaken for the `1` that means "measured, and the answer is
mtime". It could not fire at all: `checksum_freshness_probe || probe_rc=$?`
runs the left side with errexit suppressed, and that suppression propagates
into the subshell. An unguarded step there fell through to `exit 0` and
reported ACTIVE — a fourth outcome the header said was impossible. The guard
audit was complete, so nothing was broken; the wrong mechanism had the credit,
in a comment inviting the next editor to add an unguarded step and rely on it.

The suggested fix — `set -e` as the first line of the subshell — does not work,
and measured on bash 5.3 it makes things worse rather than not-better:

  set -e inside a subshell called via `||` or `if`   still falls through: rc=0
  set -e inside, called with errexit off at the site aborts, but with the
                                                     FAILING COMMAND's status —
                                                     `false` gives 1, which is
                                                     exactly the value that
                                                     means "mtime"

So both halves are needed and neither is decoration: `set +e` around the call
site, so the subshell can arm its own errexit at all, and `trap 'exit 2' ERR`
inside it, so an abort lands on "not measured" instead of on an answer. The
explicit `|| exit 2` guards stay as the first line of defence. All of that is
now written down where the false claim was.

Red-proven with one unguarded `false` between the `touch` and the second
build, on a box where freshness is live: without the fix, mode `on` and 4
assertions — the reviewer's fourth outcome, reproduced; with it, mode
`unmeasured`, the warning, 3 assertions; unmodified, mode `on` and 4.

Recorded in the same block, since the next person to hit a `2` will ask: why it
skips rather than fails. The gated scenario is the only thing here that depends
on freshness mode, and failing would turn a statement about one machine into a
red gate reading "the hardlink scheme is broken" across three consuming repos.
What would change the answer is `2` becoming the everyday CI outcome, and run
2583 measures that it is not — gitdan-ci reports `1`.

GPG. The two suites that commit in scratch repos inherited the developer's
GLOBAL `commit.gpgsign`, so whether this gate passes could depend on their gpg
agent — seen as a red `restore-mtimes-selftest.sh` caused by a full disk
breaking gpg, in a suite with nothing to say about either. Pinned off locally,
in the throwaway repos only: `git config commit.gpgsign false` beside the
identity the scratch repo already sets, and `-c commit.gpgsign=false` on
prune-cache-selftest's three commits, matching its existing `-c` style.
Red-proven under a GIT_CONFIG_GLOBAL with gpgsign on and a nonexistent gpg
program: without it `fatal: failed to write commit object`, with it both
suites pass.
2026-08-26 14:28:22 -05:00
claude 5a90d5c829 test(hardlink): let the freshness probe report that it could not measure
CI / shellcheck + selftests (pull_request) Successful in 1m21s
Review finding on #13. `checksum_freshness_active()` returned non-zero for any
reason — including its own `cargo build` failing — and the caller's `else`
branch then announced "this nightly accepts -Z checksum-freshness but still
resolves freshness by mtime" regardless. On a machine where content freshness
IS live, breaking only the probe's build produced that sentence, which is
false, and silently dropped the scenario, which is lost coverage. Reachable,
not theoretical: it needs a half-installed toolchain, which is exactly the
state a probe exists to notice.

The whole point of the preceding commit was to settle this by experiment
rather than by asking, and an experiment that cannot tell a negative result
from a failed measurement is not settling it. "This toolchain resolves
freshness by mtime" is a statement about Cargo; "a probe build failed" is a
statement about this machine. They are not interchangeable.

Three outcomes now, carried in an exit code:

  0  measured ACTIVE     both builds ran, the backdated rebuild recompiled
  1  measured INACTIVE   both builds ran, the backdated rebuild was Fresh
  2  NOT MEASURED        a probe step failed; nothing was learned

Every step that could fail for a reason other than the experiment's own
outcome exits 2 explicitly, so a `set -e` abort cannot be mistaken for the 1
that means "measured, and the answer is mtime". Outcome 2 emits a ::warning::
saying in those words that this is a failure to measure and not a finding
about Cargo, and prints the tail of both probe logs. The scenario still skips
for 1 and 2 alike — everything else the suite asserts is independent of
freshness mode — but the reason is now carried to the skip line, so the two
are distinguishable at a glance.

Verified all three states on a machine where freshness is live: unmodified,
mode `on` and 4 assertions; probe build broken with a bogus flag, mode
`unmeasured` with the warning and 3 assertions, and no claim about mtime;
CARGO_UNSTABLE_CHECKSUM_FRESHNESS stripped inside the probe — CI's condition —
mode `off` naming mtime, and 3 assertions.
2026-08-26 13:34:42 -05:00
claude eb3f0bd09b docs(ci): say what the nightly step actually buys today
CI / shellcheck + selftests (pull_request) Successful in 1m24s
The step's comment claimed nightly was not optional. CI's own first green run
falsified that: 1.100.0-nightly accepts -Z checksum-freshness and resolves
freshness by mtime regardless, so the suite's probe reports it and skips the
scenario either way.

The step stays — twenty seconds, and the coverage returns by itself the day
upstream restores the behaviour — but the comment now says that rather than
the opposite. The open question about what upstream actually did is
daniel/gitdan#62.
2026-08-26 12:57:15 -05:00
claude 6a80423bd0 test(hardlink): settle checksum freshness by experiment, not by asking
CI / shellcheck + selftests (pull_request) Successful in 1m17s
Third toolchain probe in this branch, and the first one that asks the
question the suite actually depends on. The two before it each let the suite
assert a property the toolchain did not have:

  cargo +nightly -V                    answers "did a proxy called with
                                       +nightly exit 0". `-V` short-circuits
                                       before `-Z` is parsed at all.
  cargo +nightly -Z checksum-freshness answers "is the flag still accepted".
   locate-project                      1.100.0-nightly (2026-08-25) accepts it
                                       and resolves freshness by mtime anyway,
                                       which is how CI reached the final
                                       scenario and failed there.

The final scenario depends on exactly one property: that changed content with
an OLDER mtime rebuilds. Under mtime freshness the correct answer is Fresh, so
under mtime freshness that scenario asserts a bug — which is what CI reported.

So the probe performs that experiment, on its own crate and its own target
dir, with no clone anywhere near it. That separation is also what keeps it a
control rather than a restatement: the probe establishes that the toolchain
rebuilds on content, the scenario establishes that a hardlink clone did not
take that away.

Verified both ways locally: with a real content-freshness nightly, mode on and
4 assertions; with the env var stripped inside the probe — the runner's
condition, faithfully — the probe says so by name, mode goes off, the scenario
is skipped loudly and the remaining 3 assertions pass.
2026-08-26 12:54:24 -05:00
claude 5e8e773f78 test(hardlink): report the mutation families, don't assert one of them
CI / shellcheck + selftests (pull_request) Failing after 1m21s
CI's nightly is 1.100.0-nightly (2026-08-25); the machine this suite was
written on had 1.96.0-nightly (2026-02-24). On the newer one the control's
`cp -al` clone mutates only the build/ and *.d families — upstream appears to
have stopped rewriting `.fingerprint/*/dep-*` in place under checksum
freshness — so the suite went red on the *absence* of a hazard.

That is the wrong shape for a gate. The control's job is to prove the hazard
exists at all, which a non-empty mutated set already does; naming one family
as mandatory makes the suite red whenever upstream stops doing something we
never wanted it to do, and red in the CONTROL, where a failure reads as "the
hazard is gone" rather than "upstream changed". It is now a note either way.

Nothing is given up. The fix scenario asserts the source is byte-identical
after a full rebuild in the clone, which covers every family the running Cargo
has, named or not — and the checksum-freshness scenario after it tests the
stale-reuse hazard directly. The dep-* line only ever documented which family
was in play.

`unshare_mutable_paths` keeps unsharing dep-* regardless, and its measurement
block now records both observations with their versions: 22 MB of a 6.9 GB
tree against a failure mode that is a wrong answer rather than a slow one.
2026-08-26 12:50:42 -05:00
claude 3f97d3d1e7 fix(selftest): make the two compiler-backed suites survive a CI runner
CI / shellcheck + selftests (pull_request) Failing after 1m36s
The new gate's first run went red on both suites that drive a real Cargo.
Neither failure was a defect in what they test; both were assumptions about
the machine, which had only ever been a dev box with a nightly installed and
no colour forcing. Fixed here rather than waived — turning a gate on is what
obliges fixing what it finds.

COLOUR. Every assertion in restore-mtimes-selftest.sh, and one in
hardlink-clone-selftest.sh, reads cargo's own words out of a build log
(`Compiling libdep`, `Fresh probe`). gitdan-ci's runner image forces colour, so
cargo wrote `Compiling\e[0m libdep` into the log and `grep -q "Compiling
libdep"` stopped matching. The suite then reported the opposite of what
happened — the failure printed the log, and the log plainly said `Compiling
libdep`. Both suites now pin CARGO_TERM_COLOR=never, which is the format their
assertions are written against.

Red-proven: with the export removed and CARGO_TERM_COLOR=always,
restore-mtimes-selftest reproduces the CI failure verbatim ("expected to find:
Compiling libdep"); with it, 14/14 pass under the same forced colour.

THE NIGHTLY PROBE ASKED THE WRONG QUESTION. `cargo +nightly -V` answers "did a
cargo proxy called with +nightly exit 0", which is not "will this build have
checksum freshness": `-V` short-circuits before `-Z` is validated at all. So
the probe said "on" on a runner where the flag was not in effect, and the
suite asserted the checksum-freshness mutation family against a build that
never had it — failing in the CONTROL, where a failure reads as "the hazard is
gone" rather than "the toolchain is wrong". It now probes the capability:
`cargo +nightly -Z checksum-freshness locate-project`, the narrowest command
that actually parses the flag. It rejects the stable channel and an unknown
flag name alike, needs no network and builds nothing.

Red-proven: the old channel probe, pointed at a cargo without the flag,
reproduces the CI failure exactly.

AND THE OFF PATH DID NOT WORK EITHER. The suite's header claimed that without
checksum freshness "the test still covers the build/ and *.d families". It did
not: the final scenario backdates the source to 2001 and asserts the rebuild
is not Fresh, which is checksum-freshness-only reasoning. Under Cargo's
ordinary mtime freshness a 2001 source IS older than the artifact and Fresh is
the correct answer, so the scenario asserted a bug. It is now gated on the
mode and skipped loudly, like the control's dep-* assertion already was: 5
assertions with a nightly, 3 without.

CI installs a nightly as well as stable (nightly first, so stable stays the
default) — that hazard is the one this whole scheme exists to close, and a CI
that skips it is checking the cheap half.
2026-08-26 12:46:03 -05:00
8 changed files with 845 additions and 78 deletions
+20
View File
@@ -66,6 +66,26 @@ jobs:
#
# The scratch workspaces use path dependencies only, so nothing here
# reaches crates.io.
# Nightly first, stable second, so stable ends up the default and
# nightly is only reachable through an explicit `+nightly`.
#
# `hardlink-clone-selftest.sh`'s last scenario needs a Cargo that
# resolves freshness by CONTENT — the mode where the dep-info file
# carries per-source checksums, which is the mutation that turns a
# hardlink clone into silent stale-artifact reuse rather than a slow
# build. This step is what supplies it, and as of 2026-08-26 it does:
# the scenario ran and passed against 1.100.0-nightly.
#
# It briefly did not. Cargo PR #17382 (2026-08-22) demoted
# `-Z checksum-freshness` to a gate and gave `build.fingerprint` the
# choice, defaulting to `mtime`, so the suite — which set only the gate —
# measured a genuine INACTIVE and skipped its strongest scenario. That
# read as "upstream withdrew content freshness" and was written up here
# as this step buying nothing. It was a moved switch, not a withdrawal;
# the suite now exports both and the coverage is back. See
# daniel/gitdan#62 for the investigation.
- name: Install Rust nightly
uses: dtolnay/rust-toolchain@nightly
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
+73 -16
View File
@@ -29,16 +29,63 @@ on the runner having a single execution slot.
One thing neither project had, and the reason the clone is not a plain
`cp -al`: **a build inside a hardlink clone does mutate the directory it was
cloned from.** Cargo writes real artifacts by replacing them, but writes its
cloned from.** rustc renames its own outputs into place, but Cargo writes its
metadata — and build scripts write their `OUT_DIR` — with a plain truncating
write, straight through the shared inode. Under
`CARGO_UNSTABLE_CHECKSUM_FRESHNESS` the file that gets corrupted is
`.fingerprint/<unit>/dep-<target>`, which holds the per-source checksums that
decide freshness, and the failure is silent stale-artifact reuse rather than a
slow build. `scripts/hardlink-clone-selftest.sh` reproduces it as an explicit
control and asserts the fix. The fix is to hardlink the artifacts (the GB) and
real-copy the metadata (the MB) — about 3.7% of a Bevy-sized target directory,
against 100% for a full copy.
write, straight through the shared inode. When Cargo resolves freshness by
content the file that gets corrupted is its dep-info fingerprint —
`.fingerprint/<unit>/dep-<target>`, or
`build/<pkg>/<hash>/fingerprint/dep-<target>` under Cargo's build-dir layout v2
— which holds the per-source checksums that decide freshness, and the failure
is silent stale-artifact reuse rather than a slow build.
One more family joins them, and it is not metadata: **anything the linker
writes**. rustc writes an `.rlib` or `.rmeta` to a temporary and renames it
into place, but a linked executable is written through whatever inode is
already at its path — so a `cargo test --no-run` in a raw `cp -al` clone
rewrites the source's own test binary. Measured 2026-08-27 on cargo 1.93.1
stable, 1.96.0-nightly, 1.98.0-nightly and 1.100.0-nightly, under both
build-dir layouts.
`scripts/hardlink-clone-selftest.sh` reproduces both as explicit controls and
asserts the fix. The fix is to hardlink what rustc renames into place — the
`.rlib`, `.rmeta` and `incremental/` bulk — and real-copy the metadata and the
linker outputs.
**The selection names that set directly, under either build-dir layout.**
Layout v2 regroups everything per build unit under
`build/<pkg>/<hash>/{fingerprint,out,run}/`, artifacts included, so there is no
`.fingerprint` and no `deps` to key off and `build/` is no longer a proxy for
"metadata" — it is the whole tree. The one place the two layouts genuinely
differ is that under v2 a build script's `OUT_DIR` and a compile unit's rlib
are both a directory called `out`.
**Ambiguity there resolves toward unsharing**, because over-unsharing costs
bytes and under-unsharing costs corruption. An `out` directory stays shared
only when two independent signals agree it is a compile unit's: it holds an
`.rlib`/`.rmeta` of its own, *and* its unit carries no record of a build-script
execution beside it (`run/` under v2, a loose `root-output` under v1). The
execution record alone is not enough — Cargo writes it only after the script
succeeds, so a build script that populates `OUT_DIR` and then fails leaves a
unit that reads as a compile unit. v2 is the nightly default and stabilises in
cargo 1.100.0 on 2026-11-12.
### What it costs
Real-copied share of a 5.5 GB Bevy target directory, before and after the
linker-output rule landed:
| tree | before | after |
|---|---|---|
| **excluding `incremental/`** — the figure to plan against, since the quick-start below sets `CARGO_INCREMENTAL: 0` | **36.4%** | **57.0%** |
| whole tree, `incremental/` included (a local dev checkout, not CI) | 9.0% | 14.1% |
The first row is the one a CI consumer gets. The increase is the linker-output
rule, not the layout work: on a scratch crate the layout fix alone takes v2
from 99.996% to 0.2%.
The copy is paid per clone and does not amortise — a fresh `cp -al` leaves
every file with `nlink >= 2`, so the `-links +1` filter cannot skip anything —
and a clone happens twice per job, once seeding and once publishing.
---
@@ -98,10 +145,14 @@ env:
CARGO_INCREMENTAL: 0 # per-run bloat on a persistent volume
CARGO_PROFILE_DEV_DEBUG: line-tables-only
CARGO_PROFILE_TEST_DEBUG: line-tables-only
# Nightly only. Content-addressed freshness instead of mtime-based — a
# strictly stronger guarantee, complementary to the mtime restore (which
# still covers directory-form `rerun-if-changed` build-script watches).
# Nightly only, and BOTH are needed. Content-addressed freshness instead of
# mtime-based — a strictly stronger guarantee, complementary to the mtime
# restore (which still covers directory-form `rerun-if-changed` build-script
# watches). Since cargo PR #17382 (2026-08-22) the `-Z` gate below only
# unlocks the feature; `build.fingerprint` selects it and defaults to
# `mtime`, so the gate on its own is accepted and does nothing.
CARGO_UNSTABLE_CHECKSUM_FRESHNESS: "true"
CARGO_BUILD_FINGERPRINT: "content"
```
---
@@ -544,9 +595,15 @@ bash scripts/selftest.sh --fast # fixture-only suites, no compiler
Both run in CI — `.gitea/workflows/ci.yaml`, one job, on pushes to `main` and
on PRs that were non-draft when the run was created. It installs shellcheck
and a stable Rust toolchain and references no credentials; the scratch
workspaces the compiler-backed suites build use path dependencies only, so
nothing reaches crates.io. It runs the full suite rather than `--fast`,
and both a stable and a nightly Rust toolchain (nightly so
`hardlink-clone-selftest.sh` can run its content-freshness scenario, which as
of 2026-08-26 a nightly does enable — 1.100.0-nightly (787af2b8c 2026-08-25)
resolves freshness by content given both `CARGO_UNSTABLE_CHECKSUM_FRESHNESS`
and `CARGO_BUILD_FINGERPRINT: content`, per cargo PR #17382; the suite still
settles that by experiment on every run and skips the scenario loudly when it
cannot measure) and references no credentials; the scratch workspaces the
compiler-backed suites build use path dependencies only, so nothing reaches
crates.io. It runs the full suite rather than `--fast`,
because the two compiler-backed suites are the ones that check this scheme
against real Cargo instead of against a fixture. Draft (`WIP:`-titled) PRs
skip it, and un-drafting does **not** un-skip them — the guard is evaluated
@@ -559,7 +616,7 @@ change here reaches all of them at once. That is what the gate is for.
| suite | covers |
|---|---|
| `cache-root-selftest.sh` | that a lineage nests one level and nothing else moves: no lineage resolves byte-for-byte to the cache root, two lineages on one cache key get disjoint target dirs, seed/publish/prune all stay inside their own lineage, a PR layers over its own lineage's base snapshot — **and one rejection per lineage name a reader elsewhere would stop seeing**, plus the publish-side mismatch guard |
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler. |
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly that actually resolves freshness by content for its last scenario**: the source's-next-build check reasons about content rather than mtime, so under mtime freshness it would assert a bug. Whether the toolchain does is settled by experiment on a throwaway crate, not by asking it — accepting `-Z checksum-freshness` stopped implying it on 2026-08-22, when cargo PR #17382 demoted the flag to a gate and gave `build.fingerprint` (default `mtime`) the choice; the suite now exports both and 1.100.0-nightly measures ACTIVE again. The experiment reports **three** outcomes, not two: active, measured-inactive, and *not measured*. Its answer codes are `0` and `3`, deliberately clear of every status bash generates for its own errors — so nothing that goes wrong inside the probe, including an expansion failure no guard can catch, can be read as an answer. The scenario is skipped for the last two alike, but a failure to measure is never reported as a measurement. The control also reports which mutation families the running Cargo exhibits — a note, not an assertion, since that set moves upstream. It also pins the SELECTION itself against both of Cargo's build-dir layouts, from two file-only fixtures that need no compiler — so the layout the installed Cargo does not happen to write is still covered — and asserts the partition in both directions: every file Cargo rewrites in place is privately owned, and every `.rlib`/`.rmeta` still shares its inode. The second half is what the old suite never checked beyond `shared > 0`, and it is what a layout change silently inverts. |
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published, and the publisher's log showing it waited on the consumer's own reader-lock marker before reclaiming a rotated snapshot |
| `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone |
| `prune-cache-selftest.sh` | liveness, protection, locking, eviction order, self-clear, **and that a cache a job claims *inside* the check-to-unlink window survives it** — against a real scratch `origin` |
+181 -20
View File
@@ -317,29 +317,116 @@ _unshare_files() {
xargs -0 -r -n 64 bash -c 'rc=0; for f; do cp -p -- "$f" "$f.unshare.$$" && mv -f -- "$f.unshare.$$" "$f" || rc=1; done; exit $rc' _
}
# True when a directory holds a compiled library artifact of its own.
#
# The glob is left unquoted and unmatched-glob-safe on purpose: with nullglob
# off an unmatched pattern stays literal and the `-e` test fails, which is the
# answer wanted.
_holds_compiled_artifact() {
local f
for f in "$1"/*.rlib "$1"/*.rmeta; do
[ -e "$f" ] && return 0
done
return 1
}
# The directories `unshare_mutable_paths` replaces, under either layout.
#
# All four names are pruned, so nothing selected here can contain anything else
# selected here and the caller never unshares a subtree twice.
#
# `out` is the one that needs deciding rather than naming, and it is the whole
# difficulty of layout v2: a compile unit's rlib and a build script's OUT_DIR
# are both a directory called `out`, one directory apart, and they need
# opposite treatment.
#
# AMBIGUITY RESOLVES TOWARD UNSHARING, and that direction is the rule rather
# than a default: over-unsharing costs bytes, under-unsharing costs corruption.
# So an `out` directory is left shared only when TWO independent signals agree
# it is a compile unit's artifact directory, and either one missing is enough
# to real-copy it:
#
# 1. it holds an `.rlib`/`.rmeta` of its own — the artifact whose sharing is
# the entire point of the clone; and
# 2. its unit directory has no record of a build-script execution beside it
# (`run/` under layout v2, a loose `root-output` under v1).
#
# Signal 2 alone was the first cut of this and it is NOT sufficient, because
# Cargo writes `root-output` only AFTER the script exits successfully. A build
# script that populates `OUT_DIR` and then FAILS leaves a unit with no record
# at all, which reads as "compile unit" — and the old `-name build` selection
# covered that state by real-copying `build/` wholesale, so trusting signal 2
# alone was a regression against it. Reproduced on cargo 1.93.1 stable: the
# clone's build wrote through the shared inode into the source's OUT_DIR.
# Signal 1 closes it, because a failed build script's OUT_DIR holds no rlib.
#
# The residual is a build script that writes a file NAMED `*.rlib`/`*.rmeta`
# into `OUT_DIR` and has never once succeeded. Nothing bounds that away; it is
# simply far narrower than what it replaces.
#
# Requiring signal 1 also means a bin, test or build-script COMPILE unit's
# `out` is real-copied rather than shared — at no cost in bytes, since
# everything in one is an executable or a `*.d`, and both are privately owned
# by the file rules below either way.
_mutable_dirs() {
local root="$1" d unit
while IFS= read -r d; do
if [ "${d##*/}" = out ]; then
unit="${d%/out}"
if ! [ -d "$unit/run" ] && ! [ -e "$unit/root-output" ] \
&& _holds_compiled_artifact "$d"; then
continue
fi
fi
printf '%s\n' "$d"
done < <(find "$root" -type d \
\( -name .fingerprint -o -name fingerprint -o -name run -o -name out \) \
-prune -print 2>/dev/null)
}
# THE load-bearing function of this whole design.
#
# A hardlink clone is only safe if every write the clone's build performs
# lands on a NEW inode, leaving the source's data untouched. That is true for
# compilation artifacts — rustc and the linker replace `deps/*.rlib`,
# `*.rmeta`, and binaries rather than truncating them in place — and it is
# NOT true for the metadata Cargo and build scripts write with a plain
# truncating write. Measured directly (Linux, ext4, cargo 1.9x nightly:
# `cp -al` a warm target dir, change a source file, build in the clone, diff
# the source) the following files in the SOURCE were mutated through the
# shared inode:
# lands on a NEW inode, leaving the source's data untouched. That is true of
# rustc's own outputs — it writes an `.rlib` or `.rmeta` to a temporary and
# renames it into place — and it is NOT true of the metadata Cargo and build
# scripts write with a plain truncating write, nor of anything the LINKER
# produces. Measured directly (Linux, ext4: `cp -al` a warm target dir, change
# a source file, build in the clone, diff the source) the following files in
# the SOURCE were mutated through the shared inode:
#
# <profile>/.fingerprint/<unit>/dep-<target> (only under
# CARGO_UNSTABLE_CHECKSUM_FRESHNESS,
# where this file carries the
# per-source blake3 checksums)
# <profile>/build/<pkg>/output, root-output (Cargo build-script metadata)
# <profile>/.fingerprint/<unit>/dep-<target> (build-dir layout v1) — or,
# <profile>/build/<pkg>/<hash>/fingerprint/dep-<target>
# (build-dir layout v2; see the
# dated note below for which
# Cargo writes which). Only
# when Cargo resolves freshness
# by CONTENT, where this file
# carries the per-source blake3
# checksums.
# <profile>/build/<pkg>/output, root-output (Cargo build-script metadata;
# `<pkg>/<hash>/run/root-output`
# under layout v2)
# <profile>/build/<pkg>/out/** (whatever the build script
# writes into OUT_DIR — build
# scripts overwhelmingly use a
# plain fs::write)
# <profile>/deps/*.d, <profile>/*.d (Cargo's post-processed
# dep-info)
# <profile>/deps/<test>-<hash> (a linked TEST binary; under
# layout v2,
# `build/<pkg>/<hash>/out/`)
#
# THE LINKED-OUTPUT CASE IS NOT LAYOUT-SPECIFIC AND WAS NOT PART OF THIS
# FUNCTION UNTIL 2026-08-27 (gitdan-actions#14). A `cargo test --no-run` inside
# a raw `cp -al` clone rewrote the source's own test binary in place on cargo
# 1.93.1 stable, 1.96.0-nightly, 1.98.0-nightly (layout v1) and 1.100.0-nightly
# (layout v2) alike. Cargo re-creates the path first when it also has to uplift
# the result — a bin target's `deps/<bin>-<hash>` has a hardlink twin at
# `<profile>/<bin>` — and other crate shapes relinked to a fresh inode for
# reasons this measurement did not pin down. Since the safe cases could not be
# enumerated, every executable is treated as mutable; `.rlib`, `.rmeta` and
# `incremental/` are what stay shared, and they are the bytes worth sharing.
#
# The checksum-freshness case is not a cosmetic one. Reproduced end to end:
# branch B clones base's cache, builds its own content, and thereby rewrites
@@ -349,10 +436,54 @@ _unshare_files() {
# sources, reports `Fresh`, and reuses a binary built from the PRE-merge code.
# That is silent stale-artifact reuse — a wrong answer, not a slow one.
#
# So: hardlink the artifacts (the GB), real-copy the metadata (the MB).
# Measured on a 6.9 GB Bevy workspace target dir, the unshared set is
# .fingerprint 22 MB + build/ 237 MB + a handful of dep-info files — about
# 3.7% of the tree, against 100% for a plain `cp -a`.
# WHAT UPSTREAM CHANGED, AND WHAT IT DID NOT (measured 2026-08-26, daniel/gitdan#62).
#
# An earlier revision of this comment recorded that the dep-info write was
# "NOT reproduced on 1.100.0-nightly (2026-08-25) — upstream appears to have
# stopped writing it in place". That reading was wrong, and the way it was
# wrong is the reason this paragraph is dated. Two unrelated upstream changes
# landed within days of each other, and between them they moved both the
# switch that turns the behaviour on and the path it writes to:
#
# 1. The ON-SWITCH MOVED. cargo PR #17382 `feat(config): Add build.fingerprint`
# (merged 2026-08-22) demoted `-Z checksum-freshness` to a gate: it now
# only UNLOCKS the feature, and `build.fingerprint` SELECTS it, defaulting
# to `"mtime"`. So `CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true` on its own is
# accepted and does nothing, which is exactly the "flag accepted, mtime
# anyway" result that was mistaken for a withdrawal. Content freshness
# needs BOTH, and with both it is entirely intact:
#
# CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true CARGO_BUILD_FINGERPRINT=content
#
# Measured on cargo 1.100.0-nightly (e8cb624d5 2026-08-22): with the gate
# alone a `cp -al` clone mutates only the build/ and *.d families; add
# `CARGO_BUILD_FINGERPRINT=content` and the source's dep-info file is
# mutated through the shared inode again. Same toolchain, same clone, one
# env var apart. The hazard was never removed — it was switched off.
#
# 2. THE PATH MOVED. Build-dir layout v2 (`-Z build-dir-new-layout`, cargo
# 1.91) became the nightly default in cargo 1.99 (PR #17258) and was
# stabilised by PR #17354, merged 2026-08-18, shipping in cargo 1.100.0
# stable on 2026-11-12. Under v2 there is no `<profile>/.fingerprint` and
# no `<profile>/deps` at all: everything is regrouped per build unit under
# `<profile>/build/<pkg>/<hash>/{fingerprint,out,run}/`, artifacts
# included. Bracketed locally: cargo 1.97.1 and 1.98.0-nightly write v1,
# 1.100.0-nightly writes v2.
#
# Until 2026-08-27 the selection was `-name .fingerprint -o -name build`,
# which under a v2 Cargo matched nothing on its first clause and the entire
# tree on its second, because the artifacts moved under `build/` too. The guard
# held by accident and the saving did not: on one scratch crate (serde +
# serde_json + regex plus a build script), same sources both ways —
#
# cargo 1.98.0-nightly (layout v1) 64.9 MB unshared of 165.1 MB — 39.3%
# 1.100.0-nightly (layout v2) 110.4 MB unshared of 110.4 MB — 99.996%
#
# The selection now names the mutable set directly rather than by the container
# it used to live in, so it holds under both layouts; the same crate measures
# 45.3% (v1) and 38.4% (v2), both dominated by the linked-output rule above
# rather than by the layout. On a real 5.5 GB Bevy target dir the whole change
# moves the real-copied share from 9.0% to 14.1%.
#
# `incremental/` is deliberately left shared: rustc writes each incremental
# session to a fresh `s-*-working` directory and finalises it with a rename,
@@ -364,10 +495,9 @@ unshare_mutable_paths() {
[ -d "$root" ] || return 0
# The list is materialised in full before anything is replaced: each
# replacement deletes and recreates a directory, and a live `find` walk over
# a tree being mutated underneath it is a needless hazard. `-prune` keeps a
# match's own contents out of the list.
# a tree being mutated underneath it is a needless hazard.
local -a dirs=()
mapfile -t dirs < <(find "$root" -type d \( -name .fingerprint -o -name build \) -prune -print 2>/dev/null)
mapfile -t dirs < <(_mutable_dirs "$root")
for d in "${dirs[@]}"; do
[ -n "$d" ] || continue
unshare_subtree "$d" || {
@@ -379,6 +509,37 @@ unshare_mutable_paths() {
echo "::error::unshare_mutable_paths: failed to unshare dep-info files under ${root}" >&2
return 1
}
# Layout v1's build-script run metadata, which v2 groups under `run/` and v1
# leaves loose in the run unit's directory. `invoked.timestamp` is empty and
# carries its meaning in its mtime, which a shared inode carries too.
_unshare_files "$root" -type f \
\( -name output -o -name root-output -o -name stderr -o -name invoked.timestamp \) || {
echo "::error::unshare_mutable_paths: failed to unshare build-script run metadata under ${root}" >&2
return 1
}
# Linked outputs. Unlike an rlib or an rmeta — which rustc writes to a
# temporary and renames into place — an executable or shared object is
# written by the LINKER, and the linker writes THROUGH an existing inode.
# Measured 2026-08-27 on cargo 1.93.1 stable, 1.96.0-nightly, 1.98.0-nightly
# (layout v1) and 1.100.0-nightly (e8cb624d5, layout v2), mold and the
# default linker alike: a `cargo test --no-run` binary in a `cp -al` clone
# rewrote the SOURCE's copy of itself in place, under both layouts.
#
# What separates that from the executables measured INTACT is not
# established. Every intact case observed was one Cargo has to re-create
# anyway to maintain an uplift hardlink — a bin target's
# `deps/<bin>-<hash>`, twinned at `<profile>/<bin>`. Whether the twin is the
# mechanism or a correlate of it was not determined, and the rule below does
# not depend on the answer: exempting twinned executables would recover no
# bytes this function newly copies. See gitdan-actions#17.
#
# The executable bit is the discriminator because it is the linker's own
# output that is at risk, not the directory it happens to land in — `.rlib`,
# `.rmeta` and `incremental/` stay shared and they are the bytes that matter.
_unshare_files "$root" -type f -perm -u+x || {
echo "::error::unshare_mutable_paths: failed to unshare linked outputs under ${root}" >&2
return 1
}
_unshare_files "$root" -maxdepth 3 -type f -name '.rustc_info.json' || {
echo "::error::unshare_mutable_paths: failed to unshare .rustc_info.json under ${root}" >&2
return 1
+548 -36
View File
@@ -5,8 +5,10 @@
#
# That assumption is FALSE for a plain `cp -al`. Measured, and asserted below
# as an explicit control: build in a raw `cp -al` clone and the source's
# `.fingerprint/<unit>/dep-*` (under CARGO_UNSTABLE_CHECKSUM_FRESHNESS),
# `build/<pkg>/output`, `build/<pkg>/out/**` and `deps/*.d` all change,
# dep-info file (`.fingerprint/<unit>/dep-*` under Cargo's build-dir layout
# v1, `build/<pkg>/<hash>/fingerprint/dep-*` under v2 — and under content
# freshness only, see the probe below), `build/<pkg>/output`,
# `build/<pkg>/out/**` and `deps/*.d` all change,
# because Cargo and build scripts write those with a plain truncating write
# rather than the write-then-rename Cargo uses for real artifacts.
#
@@ -26,8 +28,6 @@ set -euo pipefail
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
. "$script_dir/cache-lib.sh"
command -v cargo >/dev/null || { echo "SKIP: no cargo on PATH"; exit 0; }
scratch=$(mktemp -d)
trap 'rm -rf "$scratch"' EXIT
pass_count=0
@@ -57,6 +57,14 @@ version = "0.1.0"
edition = "2021"
[workspace]
TOML
# A binary as well as a library, because the two are written differently and
# only one of them is safe to share: rustc writes an rlib to a temporary and
# renames it into place, while the LINKER writes an executable through the
# existing inode. Without a bin target this suite never relinks anything and
# cannot see that difference.
cat > "$dir/src/main.rs" <<'RS'
fn main() { println!("{}", probe::f()); }
RS
cat > "$dir/build.rs" <<'RS'
use std::{env, fs, path::PathBuf};
fn main() {
@@ -68,27 +76,391 @@ fn main() {
RS
}
# ---------------------------------------------------------------------------
# Both build-dir layouts, without a compiler
# ---------------------------------------------------------------------------
#
# Every other scenario in this file runs whichever layout the installed Cargo
# happens to write, so on any one machine it exercises exactly ONE of the two —
# and gitdan-ci's is v1. The two fixtures below reproduce both directory shapes
# from files alone, clone them through the real `hardlink_clone_into()`, and
# assert file by file which side of the partition each one lands on.
#
# Shapes taken from a scratch crate (serde + serde_json + regex, plus a build
# script) built on 2026-08-27: cargo 1.98.0-nightly (a335d47ff 2026-06-26)
# writes v1, cargo 1.100.0-nightly (e8cb624d5 2026-08-22) writes v2.
#
# v2 is where the partition is easy to get wrong, and the fixtures are built to
# say so: a build script's OUT_DIR and a compile unit's rlib are BOTH a
# directory called `out`, one directory apart, and they need opposite
# treatment.
mkfile() { mkdir -p "$(dirname "$1")"; printf '%s' "$2" > "$1"; }
# Big enough that the byte-fraction assertion below measures something.
artifact_bytes=$(head -c 4096 /dev/zero | tr '\0' 'A')
# Spec lines are `<shared|private>|<path relative to the tree root>`.
assert_partition() {
local label="$1" src="$2" spec="$3"
local clone="${src}-clone" want rel si ci total=0 copied=0 sz
hardlink_clone_into "$src" "$clone" "selftest-$label" \
|| fail "$label: hardlink_clone_into refused the destination"
while IFS='|' read -r want rel; do
[ -n "${rel:-}" ] || continue
[ -e "$clone/$rel" ] || fail "$label: $rel is missing from the clone"
si=$(stat -c '%i' "$src/$rel"); ci=$(stat -c '%i' "$clone/$rel")
case "$want" in
shared)
[ "$si" = "$ci" ] \
|| fail "$label: $rel was real-copied, but it is an artifact and must stay shared" ;;
private)
[ "$si" != "$ci" ] \
|| fail "$label: $rel still shares an inode with the source, so a build in the clone can rewrite it" ;;
*) fail "$label: unknown spec verb '$want'" ;;
esac
done <<< "$spec"
ok "$label: every file landed on the right side of the partition"
# The cost model, asserted rather than assumed. Selecting too much is not a
# correctness bug, which is exactly why nothing caught layout v2 taking the
# selection from 39.3% of one scratch crate's tree to 99.996% of it
# (gitdan-actions#14):
# a hardlink clone that real-copies everything is a `cp -a` with extra steps.
# The bound is loose on purpose. It is not a budget — the honest figure moves
# with how much of a tree is linker output, and these fixtures are mostly
# that by construction — it is a floor under "still a hardlink clone at all".
while IFS= read -r rel; do
# `.cargo-*lock*` is stripped from every clone by design, so it has no
# counterpart to compare against.
[ -e "$clone/$rel" ] || continue
sz=$(stat -c '%s' "$src/$rel")
total=$((total + sz))
[ "$(stat -c '%i' "$src/$rel")" = "$(stat -c '%i' "$clone/$rel")" ] || copied=$((copied + sz))
done < <(cd "$src" && find . -type f -printf '%P\n')
[ "$total" -gt 0 ] || fail "$label: fixture has no bytes to measure"
[ $((copied * 100 / total)) -lt 90 ] \
|| fail "$label: the clone real-copied $((copied * 100 / total))% of its source's bytes — the hardlink saving is gone"
ok "$label: clone real-copies $((copied * 100 / total))% of ${total} B (${copied} B), the rest is shared"
}
# Layout v2: no `.fingerprint`, no `deps`. Everything regroups per build unit
# under `build/<pkg>/<hash>/{fingerprint,out,run}`, artifacts included — which
# is what took `-name build` from "the metadata" to "the whole tree".
#
# THE THREE UNIT KINDS ARE THE POINT. `1bf...` is a compile unit: its `out`
# holds the rlib. `08c...` is the build script's own compile unit: its `out`
# holds the build-script binary. `da9...` is the build-script RUN unit: its
# `out` IS the OUT_DIR, and it is the only one of the three whose `out` is
# mutable. The `run/` directory beside it is the structural difference.
v2="$scratch/layout-v2"
mkfile "$v2/CACHEDIR.TAG" 'Signature: 8a477f597d28d172'
mkfile "$v2/.rustc_info.json" '{"rustc_fingerprint":1}'
mkfile "$v2/debug/.cargo-lock" ''
mkfile "$v2/debug/libprobe.rlib" "$artifact_bytes"
mkfile "$v2/debug/libprobe.d" '/probe/src/lib.rs:'
for f in dep-lib-probe lib-probe lib-probe.json invoked.timestamp; do
mkfile "$v2/debug/build/probe/1bf5493368dce3cd/fingerprint/$f" "$f"
done
mkfile "$v2/debug/build/probe/1bf5493368dce3cd/out/libprobe-1bf5493368dce3cd.rlib" "$artifact_bytes"
mkfile "$v2/debug/build/probe/1bf5493368dce3cd/out/libprobe-1bf5493368dce3cd.rmeta" "$artifact_bytes"
mkfile "$v2/debug/build/probe/1bf5493368dce3cd/out/probe-1bf5493368dce3cd.d" '/probe/src/lib.rs:'
for f in build-script-build-script-build build-script-build-script-build.json \
dep-build-script-build-script-build invoked.timestamp; do
mkfile "$v2/debug/build/probe/08c7dda6eacd6dca/fingerprint/$f" "$f"
done
mkfile "$v2/debug/build/probe/08c7dda6eacd6dca/out/build_script_build" "$artifact_bytes"
chmod +x "$v2/debug/build/probe/08c7dda6eacd6dca/out/build_script_build"
mkfile "$v2/debug/build/probe/08c7dda6eacd6dca/out/build_script_build.d" '/probe/build.rs:'
# A test binary: the same `out` directory as the rlib above, and the largest
# thing in a real tree that a linker writes.
for f in dep-test-lib-probe test-lib-probe test-lib-probe.json invoked.timestamp; do
mkfile "$v2/debug/build/probe/6a091d813b2be60d/fingerprint/$f" "$f"
done
mkfile "$v2/debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d" "$artifact_bytes"
chmod +x "$v2/debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d"
mkfile "$v2/debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d.d" '/probe/src/lib.rs:'
for f in run-build-script-build-script-build run-build-script-build-script-build.json; do
mkfile "$v2/debug/build/probe/da96cf45111f80dd/fingerprint/$f" "$f"
done
mkfile "$v2/debug/build/probe/da96cf45111f80dd/out/gen.txt" 'generated from 24 bytes'
for f in invoked.timestamp root-output stdout stderr; do
mkfile "$v2/debug/build/probe/da96cf45111f80dd/run/$f" "$f"
done
# A build script that wrote into OUT_DIR and then FAILED: Cargo records the
# run only on success, so this unit has `out/` populated and no `run/` at all.
# Reading "no execution record" as "compile unit" left this shared, which is
# the one state the old `-name build` selection covered and the first cut of
# this one did not.
mkfile "$v2/debug/build/probe/f00ded00f00ded00/out/gen.txt" 'half-written'
mkfile "$v2/debug/incremental/probe-abc/s-xyz/dep-graph.bin" "$artifact_bytes"
assert_partition "layout v2" "$v2" "$(cat <<'SPEC'
private|.rustc_info.json
private|debug/libprobe.d
private|debug/build/probe/1bf5493368dce3cd/fingerprint/dep-lib-probe
private|debug/build/probe/1bf5493368dce3cd/fingerprint/lib-probe
private|debug/build/probe/1bf5493368dce3cd/fingerprint/lib-probe.json
private|debug/build/probe/1bf5493368dce3cd/fingerprint/invoked.timestamp
private|debug/build/probe/1bf5493368dce3cd/out/probe-1bf5493368dce3cd.d
private|debug/build/probe/08c7dda6eacd6dca/fingerprint/dep-build-script-build-script-build
private|debug/build/probe/08c7dda6eacd6dca/fingerprint/invoked.timestamp
private|debug/build/probe/08c7dda6eacd6dca/out/build_script_build.d
private|debug/build/probe/da96cf45111f80dd/fingerprint/run-build-script-build-script-build
private|debug/build/probe/da96cf45111f80dd/out/gen.txt
private|debug/build/probe/da96cf45111f80dd/run/root-output
private|debug/build/probe/da96cf45111f80dd/run/stdout
private|debug/build/probe/da96cf45111f80dd/run/stderr
private|debug/build/probe/da96cf45111f80dd/run/invoked.timestamp
private|debug/build/probe/f00ded00f00ded00/out/gen.txt
shared|debug/libprobe.rlib
shared|debug/build/probe/1bf5493368dce3cd/out/libprobe-1bf5493368dce3cd.rlib
shared|debug/build/probe/1bf5493368dce3cd/out/libprobe-1bf5493368dce3cd.rmeta
private|debug/build/probe/08c7dda6eacd6dca/out/build_script_build
private|debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d
private|debug/build/probe/6a091d813b2be60d/out/probe-6a091d813b2be60d.d
private|debug/build/probe/6a091d813b2be60d/fingerprint/dep-test-lib-probe
shared|debug/incremental/probe-abc/s-xyz/dep-graph.bin
SPEC
)"
# Layout v1: one `.fingerprint` and one `deps` per profile; `build/<pkg>-<hash>`
# holds the build script's compiled binary in one unit directory and its run
# metadata plus OUT_DIR in another.
v1="$scratch/layout-v1"
mkfile "$v1/CACHEDIR.TAG" 'Signature: 8a477f597d28d172'
mkfile "$v1/.rustc_info.json" '{"rustc_fingerprint":1}'
mkfile "$v1/debug/.cargo-lock" ''
mkfile "$v1/debug/libprobe.rlib" "$artifact_bytes"
mkfile "$v1/debug/libprobe.d" '/probe/src/lib.rs:'
mkfile "$v1/debug/deps/libprobe-1bf5493368dce3cd.rlib" "$artifact_bytes"
mkfile "$v1/debug/deps/libprobe-1bf5493368dce3cd.rmeta" "$artifact_bytes"
mkfile "$v1/debug/deps/probe-1bf5493368dce3cd.d" '/probe/src/lib.rs:'
# A test binary, which layout v1 leaves in `deps/` beside the rlibs.
mkfile "$v1/debug/deps/probe-6a091d813b2be60d" "$artifact_bytes"
chmod +x "$v1/debug/deps/probe-6a091d813b2be60d"
mkfile "$v1/debug/deps/probe-6a091d813b2be60d.d" '/probe/src/lib.rs:'
for f in dep-lib-probe lib-probe lib-probe.json invoked.timestamp; do
mkfile "$v1/debug/.fingerprint/probe-1bf5493368dce3cd/$f" "$f"
done
mkfile "$v1/debug/build/probe-08c7dda6eacd6dca/build-script-build" "$artifact_bytes"
mkfile "$v1/debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca" "$artifact_bytes"
chmod +x "$v1/debug/build/probe-08c7dda6eacd6dca/build-script-build" \
"$v1/debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca"
mkfile "$v1/debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca.d" '/probe/build.rs:'
for f in invoked.timestamp output root-output stderr; do
mkfile "$v1/debug/build/probe-da96cf45111f80dd/$f" "$f"
done
mkfile "$v1/debug/build/probe-da96cf45111f80dd/out/gen.txt" 'generated from 24 bytes'
# The same never-succeeded build script under layout v1.
mkfile "$v1/debug/build/probe-f00ded00f00ded00/out/gen.txt" 'half-written'
mkfile "$v1/debug/incremental/probe-abc/s-xyz/dep-graph.bin" "$artifact_bytes"
assert_partition "layout v1" "$v1" "$(cat <<'SPEC'
private|.rustc_info.json
private|debug/libprobe.d
private|debug/deps/probe-1bf5493368dce3cd.d
private|debug/.fingerprint/probe-1bf5493368dce3cd/dep-lib-probe
private|debug/.fingerprint/probe-1bf5493368dce3cd/lib-probe
private|debug/.fingerprint/probe-1bf5493368dce3cd/lib-probe.json
private|debug/.fingerprint/probe-1bf5493368dce3cd/invoked.timestamp
private|debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca.d
private|debug/deps/probe-6a091d813b2be60d
private|debug/deps/probe-6a091d813b2be60d.d
private|debug/build/probe-da96cf45111f80dd/invoked.timestamp
private|debug/build/probe-da96cf45111f80dd/output
private|debug/build/probe-da96cf45111f80dd/root-output
private|debug/build/probe-da96cf45111f80dd/stderr
private|debug/build/probe-da96cf45111f80dd/out/gen.txt
private|debug/build/probe-f00ded00f00ded00/out/gen.txt
shared|debug/libprobe.rlib
shared|debug/deps/libprobe-1bf5493368dce3cd.rlib
shared|debug/deps/libprobe-1bf5493368dce3cd.rmeta
private|debug/build/probe-08c7dda6eacd6dca/build-script-build
private|debug/build/probe-08c7dda6eacd6dca/build_script_build-08c7dda6eacd6dca
shared|debug/incremental/probe-abc/s-xyz/dep-graph.bin
SPEC
)"
echo
command -v cargo >/dev/null || {
echo "SKIP: no cargo on PATH — the fixture scenarios above ran, the live-Cargo ones cannot"
echo "hardlink-clone-selftest: ${pass_count} assertions passed"
exit 0
}
crate_dir="$scratch/probe"
mkcrate "$crate_dir"
cd "$crate_dir"
export CARGO_INCREMENTAL=0
# Every assertion below reads cargo's own words out of a build log
# (`Compiling libdep`, `Fresh probe`). A CI image that forces colour splices an
# ANSI reset between the status word and the crate name, at which point every
# one of those greps silently stops matching and the suite reports the
# opposite of what happened — observed on gitdan-ci's runner image, where
# scenario 2 failed while the log it printed plainly showed `Compiling libdep`.
# Pin the format the assertions are written against.
export CARGO_TERM_COLOR=never
# Checksum freshness is where the worst failure lives (the dep-* file carries
# per-source checksums and is rewritten in place). Only available on nightly;
# without it the test still covers the build/ and *.d families.
CHECKSUM_MODE="off"
if cargo +nightly -V >/dev/null 2>&1; then
export CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true
CARGO_BIN=(cargo +nightly)
CHECKSUM_MODE="on"
else
CARGO_BIN=(cargo)
fi
echo "=== checksum-freshness mode: ${CHECKSUM_MODE} ==="
CONTENT_A='pub fn f() -> u32 { 1 }'
CONTENT_B='pub fn f() -> u32 { 22222 } pub fn g() -> u32 { 7 }'
# Probe the BEHAVIOUR, not the channel and not the flag. Two weaker probes
# were tried against gitdan-ci's runner and each let the suite assert a
# property the toolchain did not have:
#
# `cargo +nightly -V` — answers "did a proxy called with
# +nightly exit 0". `-V`
# short-circuits before `-Z` is
# even parsed.
# `cargo +nightly -Z checksum-freshness — answers "is this flag still
# locate-project` accepted", which since cargo PR
# #17382 (2026-08-22) is a
# different question from "is
# content freshness on". That PR
# demoted the flag to a gate and
# gave `build.fingerprint` the
# choice, defaulting to `mtime` —
# so 1.100.0-nightly accepts the
# flag and resolves freshness by
# mtime unless
# CARGO_BUILD_FINGERPRINT=content
# is set too. Measured 2026-08-26;
# see daniel/gitdan#62.
#
# The scenario at the end of this file depends on one thing and it is neither
# of those: that changed content with an OLDER mtime rebuilds. Under mtime
# freshness the correct answer is Fresh, so under mtime freshness that
# scenario asserts a bug. So the probe simply performs that experiment, on its
# own crate and its own target dir, with no clone anywhere near it — which is
# also what makes it a control rather than a restatement of the scenario: the
# probe establishes that the toolchain rebuilds on content, the scenario
# establishes that a hardlink clone did not take that away.
# THREE OUTCOMES, NOT TWO. An experiment that cannot tell a negative result
# from a failed measurement is not settling the question, and the two are not
# interchangeable here: "this toolchain resolves freshness by mtime" is a
# statement about Cargo, while "a probe build failed" is a statement about this
# machine. Collapsing them — which an earlier cut of this did, by returning
# non-zero for both — makes a half-installed toolchain print a confident and
# wrong explanation and quietly drop a scenario. The scenario still has to be
# skipped in either case; what must not happen is the log claiming to know why.
#
# 0 content freshness measured ACTIVE — both builds ran, the backdated
# rebuild recompiled
# 3 measured INACTIVE — both builds ran, the backdated
# rebuild reported Fresh
# anything else NOT MEASURED — nothing was learned about the
# toolchain
#
# THE ANSWER CODES ARE 0 AND 3, AND THE GAP IS THE MECHANISM. Bash produces 1
# for an ordinary command failure, 2 for a usage error, 126/127 for a command
# it could not run, 128+n for a signal, and — this is the one that matters —
# 1 for an unbound-variable or other EXPANSION failure, which happens before
# the command runs and is therefore invisible to a `||` guard and to an ERR
# trap alike. It never produces 3. So "not an answer code" is decided by a
# property of the shell rather than by an enumeration of the ways a step can
# go wrong, and a step added later without a guard, or with a guard that
# cannot fire, lands on NOT MEASURED by construction.
#
# That is the whole reason INACTIVE is not 1. It was, and three review rounds
# on this function each found a narrower way for a shell-generated 1 to be read
# as a measurement — an unguarded command, then a typo'd variable name on a
# line that HAS its guard. Each was closed by narrowing the failure surface,
# which is a game with no last move. Moving the answer off the codes bash can
# generate ends it instead: there is no longer a mutation that turns an error
# into an answer, only mutations that turn an error into a different error.
#
# The guards below stay, and so does the trap, but their job is now reporting
# rather than correctness: they make a failed step land on 2 with its logs
# printed instead of on some incidental status, which is nicer to debug and
# lands in the same place either way.
#
# One piece of that reporting layer is load-bearing and not obvious. A command
# on the left of `||` — or in an `if` condition — runs with errexit suppressed,
# and that suppression propagates into a subshell and is NOT undone by a
# `set -e` inside it (measured on bash 5.3: an unguarded `false` there falls
# through to `exit 0`). Calling with errexit disarmed at the site is the only
# form that lets the subshell re-arm it; hence the `set +e` bracket. The ERR
# trap is then required on top, because a bare `set -e` abort exits with the
# FAILING COMMAND's status, and `false` gives 1.
#
# WHY NOT-MEASURED SKIPS RATHER THAN FAILS. The scenario it gates is the only thing in
# this suite that depends on freshness mode; everything else still runs and
# still catches real regressions. Failing instead would turn a statement about
# one machine's toolchain into a red gate reading "the hardlink scheme is
# broken" across the three repos consuming this action — the same category
# error the three-state split exists to prevent, one level up. What would
# change the answer is not-measured becoming the everyday CI outcome; it is not
# — gitdan-ci's outcome is a measurement either way. It reported a measured
# INACTIVE until 2026-08-26, for the reason recorded at the `export` below, and
# an ACTIVE once both switches were set.
CHECKSUM_MODE="off"
CHECKSUM_REASON="no nightly on PATH accepting -Z checksum-freshness"
CARGO_BIN=(cargo)
checksum_freshness_probe() {
local d="$scratch/freshness-probe" t="$scratch/freshness-probe-target"
mkcrate "$d" || return 2 # 2 is simply "not 0 and not 3"; see the header
(
set -e
trap 'exit 2' ERR
cd "$d" || exit 2
printf '%s\n' "$CONTENT_A" > src/lib.rs || exit 2
CARGO_TARGET_DIR="$t" cargo +nightly build -q > "$scratch/freshness-probe-warm.log" 2>&1 || exit 2
printf '%s\n' "$CONTENT_B" > src/lib.rs || exit 2
touch -d '@1000000000' src/lib.rs || exit 2
CARGO_TARGET_DIR="$t" cargo +nightly build -v > "$scratch/freshness-probe.log" 2>&1 || exit 2
# 3, not 1: see the header. This is the only statement in the subshell that
# may report a measurement, and it is the only one that may exit 3.
if grep -qE '^\s+Fresh probe' "$scratch/freshness-probe.log"; then exit 3; fi
exit 0
)
}
if cargo +nightly -Z checksum-freshness locate-project > /dev/null 2>&1; then
export CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true
# BOTH, since cargo PR #17382 (2026-08-22): the -Z flag only unlocks the
# feature and `build.fingerprint` selects it, defaulting to `mtime`. Setting
# the gate alone is what made this suite report a measured INACTIVE on
# 1.100.0-nightly and skip its strongest scenario (daniel/gitdan#62). Safe to
# export unconditionally — a Cargo that does not know the key ignores it
# silently, verified 2026-08-26 on 1.93.1 stable and 1.96.0-nightly, both of
# which still measure ACTIVE from the gate alone.
export CARGO_BUILD_FINGERPRINT=content
# Errexit off across the call, so the subshell can arm its own — see the
# header. `probe_rc` is read before it is restored.
probe_rc=0
set +e
checksum_freshness_probe
probe_rc=$?
set -e
case "$probe_rc" in
0)
CARGO_BIN=(cargo +nightly)
CHECKSUM_MODE="on"
CHECKSUM_REASON=""
;;
3)
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS CARGO_BUILD_FINGERPRINT
CHECKSUM_REASON="this nightly accepts -Z checksum-freshness and build.fingerprint=content but still resolves freshness by mtime"
;;
*)
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS CARGO_BUILD_FINGERPRINT
CHECKSUM_MODE="unmeasured"
CHECKSUM_REASON="the probe exited ${probe_rc}, which is not one of its answer codes, so this was NOT MEASURED — this toolchain may or may not resolve freshness by content"
# Loud, because the cost is silently lost coverage on a machine that
# might have had it. The suite continues: everything else it asserts is
# independent of freshness mode.
echo "::warning::hardlink-clone-selftest: could not measure whether this toolchain resolves freshness by content — the probe exited ${probe_rc}. This is a failure to measure, not a finding about Cargo."
tail -n 15 "$scratch/freshness-probe-warm.log" "$scratch/freshness-probe.log" 2>/dev/null | sed 's/^/ /' >&2 || true
;;
esac
fi
cd "$crate_dir"
echo "=== checksum-freshness mode: ${CHECKSUM_MODE}${CHECKSUM_REASON:+ — ${CHECKSUM_REASON}} ==="
build_base() {
local dir="$1"
printf '%s\n' "$CONTENT_A" > src/lib.rs
@@ -112,11 +484,35 @@ fi
ok "raw cp -al clone mutates the source ($(printf '%s\n' "$ctl_mutated" | wc -l) paths)"
printf '%s\n' "$ctl_mutated" | sed 's/^/ /'
# Reported, not asserted, and the distinction is the point. The control's job
# is to prove the hazard exists at all, which the non-empty set above already
# does; this line records WHICH families a given Cargo exhibits.
#
# The dep-info file is the worst of them — it carries the per-source
# checksums, so mutating it through a shared inode turns a hardlink clone into
# silent stale-artifact reuse rather than a slow build. Failing on its absence
# would mean this suite goes red whenever upstream stops doing something we
# never wanted it to do — and it would go red in the CONTROL, where a failure
# reads as "the hazard is gone" rather than "upstream changed". Nothing is lost
# by reporting it: the fix scenario below asserts the source is byte-identical
# after a full rebuild in the clone, which covers every family this Cargo has,
# named or not.
#
# THE PATTERN MUST MATCH BOTH LAYOUTS, and that is not a detail. Cargo's
# build-dir layout v2 moved the file from `<profile>/.fingerprint/<unit>/dep-*`
# to `<profile>/build/<pkg>/<hash>/fingerprint/dep-*` (stabilised by cargo PR
# #17354, cargo 1.100.0, stable 2026-11-12; nightly default since 1.99). An
# earlier cut of this line looked for the v1 path only, so on 2026-08-26,
# against 1.100.0-nightly with content freshness genuinely on, it printed
# "does NOT rewrite ... in place" directly beneath a control listing that
# showed the rewrite. A reporting line that can contradict the data three
# lines above it is worse than no line at all. `fingerprint/.*dep-` matches
# either layout and neither `.d` family.
if [ "$CHECKSUM_MODE" = "on" ]; then
if printf '%s' "$ctl_mutated" | grep -q '\.fingerprint/.*/dep-'; then
ok "control confirms the checksum-freshness dep-info file is among the mutated set"
if printf '%s' "$ctl_mutated" | grep -q 'fingerprint/.*dep-'; then
echo " note: this cargo DOES rewrite its dep-info fingerprint file in place under content freshness"
else
fail "expected .fingerprint/*/dep-* in the control's mutated set under checksum freshness"
echo " note: this cargo does NOT rewrite its dep-info fingerprint file in place; only the build/ and *.d families appear above"
fi
fi
@@ -127,27 +523,44 @@ build_base "$base_fix"
before=$(snapshot_tree "$base_fix")
hardlink_clone_into "$base_fix" "$clone_fix" "selftest" || fail "hardlink_clone_into reported the destination already existed"
# The clone's contract, asserted before anything builds in it: artifacts
# share inodes (that is what makes the clone near-free), and every file Cargo
# rewrites in place does not (that is what makes it sound). Checking after a
# rebuild would prove nothing — the rebuild replaces those files anyway.
shared=0; unshared=0
# The clone's contract, asserted before anything builds in it and asserted in
# BOTH directions: every file Cargo rewrites in place is privately owned (that
# is what makes the clone sound), and every artifact still shares its inode
# (that is what makes it near-free). Checking after a rebuild would prove
# nothing — the rebuild replaces those files anyway.
#
# The mutable-family patterns cover both layouts: `.fingerprint/` is v1's,
# `fingerprint/` and `run/` are v2's, and `gen.txt` is this crate's build
# script's OUT_DIR product, which under v2 sits in a directory called `out`
# beside sibling units whose `out` holds artifacts.
shared=0; unshared=0; shared_bytes=0; copied_bytes=0
while IFS= read -r f; do
rel="${f#"$base_fix"/}"
[ -e "$clone_fix/$rel" ] || continue
sz=$(stat -c '%s' "$f")
if [ "$(stat -c '%i' "$f")" = "$(stat -c '%i' "$clone_fix/$rel")" ]; then
case "$rel" in
*/.fingerprint/*|*/build/*|*.d|.rustc_info.json)
*/.fingerprint/*|*/fingerprint/*|*/run/*|*/out/gen.txt|*/output|*/root-output|*/stderr|*/invoked.timestamp|*.d|.rustc_info.json)
fail "mutable path still shares an inode with the source: $rel" ;;
esac
shared=$((shared + 1))
shared=$((shared + 1)); shared_bytes=$((shared_bytes + sz))
else
unshared=$((unshared + 1))
case "$rel" in
*.rlib|*.rmeta)
fail "artifact was real-copied rather than shared: $rel" ;;
esac
unshared=$((unshared + 1)); copied_bytes=$((copied_bytes + sz))
fi
done < <(find "$base_fix" -type f)
[ "$shared" -gt 0 ] || fail "nothing is shared — the clone degenerated into a full copy"
[ "$unshared" -gt 0 ] || fail "nothing was unshared — unshare_mutable_paths did not run"
total_bytes=$((shared_bytes + copied_bytes))
ok "fresh clone shares ${shared} artifact files and privately owns ${unshared} mutable ones"
# Reported, not asserted. This crate has no dependencies, so almost all of its
# bytes are the two executables — a ratio that says nothing about a real tree.
# The fixtures above are where the cost model is gated, because there the
# composition is fixed.
echo " note: this clone real-copies $((copied_bytes * 100 / total_bytes))% of ${total_bytes} B"
printf '%s\n' "$CONTENT_B" > src/lib.rs
CARGO_TARGET_DIR="$clone_fix" "${CARGO_BIN[@]}" build -q
@@ -161,18 +574,117 @@ fi
ok "no file in the source changed after a full rebuild in the clone"
echo
echo "=== the whole point: the source's next build is still correct ==="
# The source's cache holds artifacts built from CONTENT_A. Advance the source
# to CONTENT_B (as a merge would) and rebuild in it. If the clone had
# corrupted its dep-info, Cargo would report Fresh and keep the stale rlib.
printf '%s\n' "$CONTENT_B" > src/lib.rs
touch -d '@1000000000' src/lib.rs
log="$scratch/rebuild.log"
CARGO_TARGET_DIR="$base_fix" "${CARGO_BIN[@]}" build -v > "$log" 2>&1 || { cat "$log"; fail "rebuild in the source failed"; }
if grep -qE '^\s+Fresh probe' "$log"; then
fail "source declared its own crate Fresh against sources it has never built — stale-artifact reuse"
echo "=== a linked TEST binary, which nothing uplifts and nothing replaces ==="
# The one artifact family that is NOT safe to share, and the reason
# `unshare_mutable_paths` privately owns every executable. rustc writes an
# rlib to a temporary and renames it in; the LINKER writes an executable
# through whatever inode is already at the path.
#
# THE CRATE SHAPE IS LOAD-BEARING AND WAS WRONG ONCE. An earlier cut of this
# scenario reused the lib+bin probe crate above, whose test binaries relink to
# a FRESH inode — a shape gitdan-actions#17 records as measured safe. Both
# halves then passed green against the unfixed selection, on the strength of
# dep-info mutations the previous scenario already covers, and the scenario
# pinned nothing. A bin-only crate with a unit test does exhibit the rewrite,
# on cargo 1.93.1 stable and on 1.98.0-nightly and 1.100.0-nightly, so that is
# what this builds. The shape is chosen by measurement rather than derived:
# what separates a rewritten executable from an intact one is not established,
# so the only crate shape this scenario may rest on is one observed to exhibit
# the rewrite.
mkbincrate() {
local dir="$1" marker="$2"
mkdir -p "$dir/src"
cat > "$dir/Cargo.toml" <<'TOML'
[package]
name = "binprobe"
version = "0.1.0"
edition = "2021"
[workspace]
TOML
cat > "$dir/src/main.rs" <<RS
fn main() { println!("${marker}"); }
#[cfg(test)]
mod t { #[test] fn a() { assert_eq!("${marker}".len() > 0, true); } }
RS
}
bin_dir="$scratch/binprobe"
mkbincrate "$bin_dir" MARKER_AAAA
cd "$bin_dir"
base_exe="$scratch/base-exe"; clone_exe_ctl="$scratch/clone-exe-ctl"; clone_exe="$scratch/clone-exe"
# Only the executables are read here. The families the other scenarios cover
# would satisfy a "something changed" assertion on their own, which is exactly
# how the earlier cut of this passed while pinning nothing.
source_exe_digest() {
(cd "$1" && find . -type f -executable -print0 | sort -z | xargs -0 -r sha1sum) 2>/dev/null
}
CARGO_TARGET_DIR="$base_exe" "${CARGO_BIN[@]}" test --no-run -q > /dev/null 2>&1 \
|| fail "the bin-only probe crate failed to build"
before=$(source_exe_digest "$base_exe")
cp -al "$base_exe" "$clone_exe_ctl"
strip_cargo_locks "$clone_exe_ctl"
mkbincrate "$bin_dir" MARKER_BBBB
CARGO_TARGET_DIR="$clone_exe_ctl" "${CARGO_BIN[@]}" test --no-run -q > /dev/null 2>&1
exe_ctl_mutated=$(mutated_paths "$before" "$(source_exe_digest "$base_exe")")
# THREE OUTCOMES, as the freshness probe above has, and for the same reason: a
# scenario that cannot tell "the fix works" from "the hazard never fired" is
# not a gate. If this Cargo does not rewrite the source's test binary, the
# assertion below would pass for a toolchain reason rather than a code one, so
# it is skipped LOUDLY instead of passing quietly.
if [ -z "$exe_ctl_mutated" ]; then
echo "::warning::hardlink-clone-selftest: this toolchain did not rewrite the source's test binary through a raw cp -al clone, so the linked-output scenario proves nothing here and was SKIPPED. That is a statement about this Cargo, not about unshare_mutable_paths."
else
ok "control: a raw cp -al clone rewrites the source's own linked test binary"
printf '%s\n' "$exe_ctl_mutated" | sed 's/^/ /'
# Rebuild the base from the original marker so it is warm and consistent
# again, then do the same thing through the real clone.
mkbincrate "$bin_dir" MARKER_AAAA
CARGO_TARGET_DIR="$base_exe" "${CARGO_BIN[@]}" test --no-run -q > /dev/null 2>&1
before=$(snapshot_tree "$base_exe")
hardlink_clone_into "$base_exe" "$clone_exe" "selftest-exe" \
|| fail "hardlink_clone_into refused the destination"
mkbincrate "$bin_dir" MARKER_BBBB
CARGO_TARGET_DIR="$clone_exe" "${CARGO_BIN[@]}" test --no-run -q > /dev/null 2>&1
exe_mutated=$(mutated_paths "$before" "$(snapshot_tree "$base_exe")")
if [ -n "$exe_mutated" ]; then
printf '%s\n' "$exe_mutated" | sed 's/^/ /' >&2
fail "a test build in the clone mutated the source through a shared inode"
fi
ok "no file in the source changed after a full test build in the clone"
fi
cd "$crate_dir"
echo
if [ "$CHECKSUM_MODE" = "on" ]; then
echo "=== the whole point: the source's next build is still correct ==="
# The source's cache holds artifacts built from CONTENT_A. Advance the
# source to CONTENT_B (as a merge would) and rebuild in it. If the clone had
# corrupted its dep-info, Cargo would report Fresh and keep the stale rlib.
#
# CHECKSUM-FRESHNESS ONLY, and the backdated mtime is why. Under checksum
# freshness the dep-info file's per-source checksums decide, so a 2001
# timestamp on changed content must still rebuild — the assertion below.
# Under Cargo's ordinary MTIME freshness the same timestamp means the source
# is older than the artifact, and reporting Fresh is the correct answer;
# asserting otherwise asserts a bug. This scenario was written against a
# machine with a nightly installed and, run without one, failed on that
# correct answer.
printf '%s\n' "$CONTENT_B" > src/lib.rs
touch -d '@1000000000' src/lib.rs
log="$scratch/rebuild.log"
CARGO_TARGET_DIR="$base_fix" "${CARGO_BIN[@]}" build -v > "$log" 2>&1 || { cat "$log"; fail "rebuild in the source failed"; }
if grep -qE '^\s+Fresh probe' "$log"; then
fail "source declared its own crate Fresh against sources it has never built — stale-artifact reuse"
fi
ok "source correctly rebuilt its crate after advancing to the clone's content"
else
echo "=== skipped: the source's-next-build scenario needs content-based freshness ==="
echo " reason: ${CHECKSUM_REASON}"
fi
ok "source correctly rebuilt its crate after advancing to the clone's content"
echo
echo "hardlink-clone-selftest: ${pass_count} assertions passed"
+3 -3
View File
@@ -65,10 +65,10 @@ origin="$scratch/origin.git"; git init -q --bare "$origin"
work="$scratch/work"; git init -q "$work"
(
cd "$work"
git -c user.email=t@t -c user.name=t commit -q --allow-empty -m init
git -c user.email=t@t -c user.name=t -c commit.gpgsign=false commit -q --allow-empty -m init
git branch -M main
git checkout -q -b dev; git -c user.email=t@t -c user.name=t commit -q --allow-empty -m dev
git checkout -q -b feat/live; git -c user.email=t@t -c user.name=t commit -q --allow-empty -m live
git checkout -q -b dev; git -c user.email=t@t -c user.name=t -c commit.gpgsign=false commit -q --allow-empty -m dev
git checkout -q -b feat/live; git -c user.email=t@t -c user.name=t -c commit.gpgsign=false commit -q --allow-empty -m live
git remote add origin "$origin"
git push -q origin main dev feat/live
)
+4 -2
View File
@@ -53,9 +53,11 @@ pass_count=0
fail() { echo "ASSERTION FAILED: $*" >&2; exit 1; }
ok() { pass_count=$((pass_count + 1)); echo "PASS: $*"; }
# Cargo and rustc REPLACE an artifact (write elsewhere, rename over the path)
# rustc REPLACES an `.rlib`/`.rmeta` (writes elsewhere, renames over the path)
# rather than truncating it in place, which is exactly why a snapshot may
# share artifact inodes with the live target dir it was cloned from. The
# share those inodes with the live target dir it was cloned from. Linker
# outputs are the exception and are real-copied instead — see
# `unshare_mutable_paths` in cache-lib.sh. The
# fixtures here have to model that faithfully — a plain `>` redirect truncates
# in place and would write straight through the shared inode into the
# snapshot and every consumer, which is a property of the test fixture, not of
+14
View File
@@ -75,6 +75,15 @@
# Asserts BOTH jobs correctly recompile the dependency and succeed.
set -euo pipefail
# Every assertion below reads cargo's own words out of a build log
# (`Compiling libdep`, `Fresh probe`). A CI image that forces colour splices an
# ANSI reset between the status word and the crate name, at which point every
# one of those greps silently stops matching and the suite reports the
# opposite of what happened — observed on gitdan-ci's runner image, where
# scenario 2 failed while the log it printed plainly showed `Compiling libdep`.
# Pin the format the assertions are written against.
export CARGO_TERM_COLOR=never
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
restore_mtimes="$script_dir/restore-mtimes.sh"
@@ -131,6 +140,11 @@ cd "$repo"
git init -q
git config user.email test@example.com
git config user.name "restore-mtimes-selftest"
# Local to this mktemp'd throwaway repo. Without it the eight commits below
# inherit the developer's GLOBAL commit.gpgsign, which makes whether this gate
# passes depend on their gpg agent — observed as a red run caused by a full
# disk breaking gpg, in a suite that has nothing to say about either.
git config commit.gpgsign false
cat > Cargo.toml <<'EOF'
[workspace]
+2 -1
View File
@@ -64,7 +64,8 @@
# two refs' fingerprints ever share a directory and this script never has to
# arbitrate freshness across refs — only within one ref's own history, which
# is exactly what it is built to do soundly. On a nightly toolchain,
# CARGO_UNSTABLE_CHECKSUM_FRESHNESS is a complementary, stronger guarantee
# CARGO_UNSTABLE_CHECKSUM_FRESHNESS plus CARGO_BUILD_FINGERPRINT=content (both,
# since cargo PR #17382 on 2026-08-22) is a complementary, stronger guarantee
# (content-addressed rather than mtime-based freshness); this script is not
# made redundant by it, because directory-form `rerun-if-changed` build-script
# watches are not covered by it and stable historical mtimes stay cheap