12 Commits

Include renames

Author	SHA1	Message	Date
claude	fb3c72aa28	docs(hardlink): withdraw the nlink claim, which asserted more than was measured ... The comments said hardlink count at link time was ruled out as the discriminator between a rewritten executable and an intact one, citing a lib+bin crate whose two test binaries were both `nlink == 1` and appeared to behave differently. Re-checked on review: the intact one had not been rebuilt at all — same content, same inode — so it demonstrated nothing, and forcing both to rebuild rewrote both. What was actually observed is narrower and now says so: every executable measured intact had an uplift hardlink twin Cargo must re-create anyway, every one measured rewritten had none, and whether the twin is the mechanism or a correlate was not determined. The rule does not rest on the answer — exempting twinned executables would recover none of the bytes this change newly copies. This PR exists because a claim outlived its evidence; it should not ship one.	2026-08-27 14:50:53 -05:00
claude	0553a6b956	fix(hardlink): resolve an ambiguous out/ toward unsharing, and pin the linked-output scenario on a shape that exhibits it ... Two review findings on #16. The `out/` discriminator keyed on Cargo's record of a build-script execution, which Cargo writes only AFTER the script exits successfully. A build script that populates OUT_DIR and then fails leaves a unit with no record at all, so its OUT_DIR read as a compile unit's artifact directory and stayed shared — a regression against the old `-name build` selection, which real-copied that state by construction. Reproduced on cargo 1.93.1 stable. An `out` directory now stays shared only when two independent signals agree: it holds an `.rlib`/`.rmeta` of its own, and its unit carries no execution record. Either one missing real-copies it. The cost is unchanged to the byte — the newly-unshared directories hold only executables and `*.d`, both already privately owned by the file rules. The live linked-test-binary scenario was built on the lib+bin probe crate, whose test binaries relink to a fresh inode — a shape gitdan-actions#17 records as measured safe. Both halves passed green against the unfixed selection on dep-info mutations the previous scenario already covers. It now builds a bin-only crate with a unit test, reads only executables, and skips loudly with a warning rather than passing quietly if the toolchain does not exhibit the rewrite at all. Also: drop a clause asserting the linker writes in place "whenever the path has no other hard link", which this change's own evidence denies; move the load-bearing comment block back above `unshare_mutable_paths`; correct a superseded 99.998% figure; and record both cost rows in the README rather than only the flattering whole-tree one.	2026-08-27 14:39:29 -05:00
claude	e5b26a9368	fix(hardlink): name the mutable set directly, under both build-dir layouts ... `unshare_mutable_paths` selected `.fingerprint` and `build` directories. Under Cargo's build-dir layout v2 the first clause matches nothing and the second matches the whole tree, because v2 regroups artifacts under `build/` alongside the metadata. Measured on one scratch crate: 39.3% of the tree real-copied under v1, 99.996% under v2. The selection now names the mutable set rather than the container it used to live in: fingerprint directories under either spelling, layout v2's `run/` directories, layout v1's loose build-script run metadata, and the `out` directories that are a build script's OUT_DIR rather than a compile unit's artifact directory. The two are told apart structurally, by Cargo's record of the build-script execution sitting beside the OUT_DIR and nowhere else. Verifying that turned up a second, layout-independent hazard: a linked executable is written through whatever inode is already at its path, so a `cargo test --no-run` inside a `cp -al` clone rewrites the source's own test binary. Reproduced on cargo 1.93.1 stable, 1.96.0-nightly, 1.98.0-nightly and 1.100.0-nightly, under both layouts. Every executable is now real-copied; `.rlib`, `.rmeta` and `incremental/` are what stay shared. hardlink-clone-selftest.sh gains two file-only layout fixtures that pin the partition in both directions without a compiler, and a live scenario that relinks a test binary.	2026-08-27 14:12:01 -05:00
claude	554310186f	fix(hardlink): content freshness moved switches, it was not withdrawn ... `unshare_mutable_paths`' comment recorded that upstream had stopped rewriting `dep-<target>` in place, on a measurement taken against 1.100.0-nightly. It had not. Two unrelated cargo changes landed within four days of each other and between them moved the switch that turns the behaviour on and the path it writes to: - cargo PR #17382 (2026-08-22) demoted `-Z checksum-freshness` to a gate and gave `build.fingerprint` the choice, defaulting to `mtime`. Setting only the gate is accepted and does nothing, which is exactly the result that was read as a withdrawal. - build-dir layout v2 (cargo PR #17354, stable 1.100.0 on 2026-11-12, nightly default since 1.99) moved the file from `.fingerprint/<unit>/dep-*` to `build/<pkg>/<hash>/fingerprint/dep-*`. Measured 2026-08-26 on 1.100.0-nightly (e8cb624d5): same toolchain, same clone procedure, one env var apart — with the gate alone a `cp -al` clone mutates only the build/ and *.d families; add `CARGO_BUILD_FINGERPRINT=content` and the source's dep-info file is mutated through the shared inode again. The hazard is intact. So the suite now exports both switches, and its strongest scenario runs again on current nightlies — verified passing against both layouts. Its control note learned the v2 path too: it looked for the v1 path only, and so printed "does NOT rewrite ... in place" three lines beneath a listing that showed the rewrite. Every claim these comments make is now dated and cited, because the defect being fixed is a comment that cited one measurement and silently stopped reproducing. Layout v2 also drags the artifacts under `build/`, which collapses this function's real-copy set from 21.3% of a target dir to 99.998%. That is a live cost, not a correctness problem, and it is filed as gitdan-actions#14 rather than fixed here. Part of daniel/gitdan#62.	2026-08-26 19:48:07 -05:00
claude	e94c46f2ec	test(hardlink): move the probe's INACTIVE answer off bash's error codes ... Review finding on #13, and the last change to this function. A typo'd variable name on a line that HAS its `|| exit 2` guard — `$CONTNET_B` for `$CONTENT_B` — fails in EXPANSION, before the command runs, so neither the guard nor the ERR trap ever sees it. Under `set -u` that exits 1, which was the code meaning "measured, and the answer is mtime". Not a live defect: the probe references only set variables today, and the 1 gitdan-ci reports is a genuine measurement. The defect is that the answer codes and the error codes overlapped at all. Three rounds on this function each found a narrower way for a shell-generated 1 to be read as a measurement — an unguarded command, then a guarded line whose guard could not fire — and each was closed by narrowing the failure surface, which is a game with no last move. INACTIVE is now 3, and anything that is not 0 or 3 is NOT MEASURED. Bash generates 1, 2, 126, 127 and 128+n for its own errors and never 3, so "not an answer" is decided by a property of the shell rather than by an enumeration of the ways a step can go wrong. A step added later without a guard, or with a guard that cannot fire, lands on NOT MEASURED by construction. The guards and the trap stay, with their job restated: reporting rather than correctness. They put a failed step on 2 with both probe logs printed instead of on some incidental status — nicer to debug, and the same destination either way. The `set +e` bracket at the call site keeps its own note, since a `set -e` that cannot fire is the kind of thing a reader assumes works. The not-measured message now quotes the actual exit code, so the two ways to reach it are distinguishable in a log without reading this file. Red-proven with the reviewer's own mutation, A/B: with INACTIVE at 1 the typo reports `mode: off — resolves freshness by mtime`, a false measurement; at 3 it reports `mode: unmeasured — the probe exited 1`. All four paths re-verified — unmodified 4 assertions; env stripped inside the probe, measured INACTIVE with the mtime reason; a broken probe build, unmeasured via a guard; an unguarded `false`, unmeasured via the trap.	2026-08-26 14:40:27 -05:00
claude	5b46986cd7	test(hardlink): make the errexit invariant real, and stop gpg deciding a gate ... Two review findings on #13. THE STATED INVARIANT WAS NOT IN FORCE. The header claimed a `set -e` abort could never be mistaken for the `1` that means "measured, and the answer is mtime". It could not fire at all: `checksum_freshness_probe || probe_rc=$?` runs the left side with errexit suppressed, and that suppression propagates into the subshell. An unguarded step there fell through to `exit 0` and reported ACTIVE — a fourth outcome the header said was impossible. The guard audit was complete, so nothing was broken; the wrong mechanism had the credit, in a comment inviting the next editor to add an unguarded step and rely on it. The suggested fix — `set -e` as the first line of the subshell — does not work, and measured on bash 5.3 it makes things worse rather than not-better: set -e inside a subshell called via `||` or `if` still falls through: rc=0 set -e inside, called with errexit off at the site aborts, but with the FAILING COMMAND's status — `false` gives 1, which is exactly the value that means "mtime" So both halves are needed and neither is decoration: `set +e` around the call site, so the subshell can arm its own errexit at all, and `trap 'exit 2' ERR` inside it, so an abort lands on "not measured" instead of on an answer. The explicit `|| exit 2` guards stay as the first line of defence. All of that is now written down where the false claim was. Red-proven with one unguarded `false` between the `touch` and the second build, on a box where freshness is live: without the fix, mode `on` and 4 assertions — the reviewer's fourth outcome, reproduced; with it, mode `unmeasured`, the warning, 3 assertions; unmodified, mode `on` and 4. Recorded in the same block, since the next person to hit a `2` will ask: why it skips rather than fails. The gated scenario is the only thing here that depends on freshness mode, and failing would turn a statement about one machine into a red gate reading "the hardlink scheme is broken" across three consuming repos. What would change the answer is `2` becoming the everyday CI outcome, and run 2583 measures that it is not — gitdan-ci reports `1`. GPG. The two suites that commit in scratch repos inherited the developer's GLOBAL `commit.gpgsign`, so whether this gate passes could depend on their gpg agent — seen as a red `restore-mtimes-selftest.sh` caused by a full disk breaking gpg, in a suite with nothing to say about either. Pinned off locally, in the throwaway repos only: `git config commit.gpgsign false` beside the identity the scratch repo already sets, and `-c commit.gpgsign=false` on prune-cache-selftest's three commits, matching its existing `-c` style. Red-proven under a GIT_CONFIG_GLOBAL with gpgsign on and a nonexistent gpg program: without it `fatal: failed to write commit object`, with it both suites pass.	2026-08-26 14:28:22 -05:00
claude	5a90d5c829	test(hardlink): let the freshness probe report that it could not measure ... Review finding on #13. `checksum_freshness_active()` returned non-zero for any reason — including its own `cargo build` failing — and the caller's `else` branch then announced "this nightly accepts -Z checksum-freshness but still resolves freshness by mtime" regardless. On a machine where content freshness IS live, breaking only the probe's build produced that sentence, which is false, and silently dropped the scenario, which is lost coverage. Reachable, not theoretical: it needs a half-installed toolchain, which is exactly the state a probe exists to notice. The whole point of the preceding commit was to settle this by experiment rather than by asking, and an experiment that cannot tell a negative result from a failed measurement is not settling it. "This toolchain resolves freshness by mtime" is a statement about Cargo; "a probe build failed" is a statement about this machine. They are not interchangeable. Three outcomes now, carried in an exit code: 0 measured ACTIVE both builds ran, the backdated rebuild recompiled 1 measured INACTIVE both builds ran, the backdated rebuild was Fresh 2 NOT MEASURED a probe step failed; nothing was learned Every step that could fail for a reason other than the experiment's own outcome exits 2 explicitly, so a `set -e` abort cannot be mistaken for the 1 that means "measured, and the answer is mtime". Outcome 2 emits a ::warning:: saying in those words that this is a failure to measure and not a finding about Cargo, and prints the tail of both probe logs. The scenario still skips for 1 and 2 alike — everything else the suite asserts is independent of freshness mode — but the reason is now carried to the skip line, so the two are distinguishable at a glance. Verified all three states on a machine where freshness is live: unmodified, mode `on` and 4 assertions; probe build broken with a bogus flag, mode `unmeasured` with the warning and 3 assertions, and no claim about mtime; CARGO_UNSTABLE_CHECKSUM_FRESHNESS stripped inside the probe — CI's condition — mode `off` naming mtime, and 3 assertions.	2026-08-26 13:34:42 -05:00
claude	6a80423bd0	test(hardlink): settle checksum freshness by experiment, not by asking ... Third toolchain probe in this branch, and the first one that asks the question the suite actually depends on. The two before it each let the suite assert a property the toolchain did not have: cargo +nightly -V answers "did a proxy called with +nightly exit 0". `-V` short-circuits before `-Z` is parsed at all. cargo +nightly -Z checksum-freshness answers "is the flag still accepted". locate-project 1.100.0-nightly (2026-08-25) accepts it and resolves freshness by mtime anyway, which is how CI reached the final scenario and failed there. The final scenario depends on exactly one property: that changed content with an OLDER mtime rebuilds. Under mtime freshness the correct answer is Fresh, so under mtime freshness that scenario asserts a bug — which is what CI reported. So the probe performs that experiment, on its own crate and its own target dir, with no clone anywhere near it. That separation is also what keeps it a control rather than a restatement: the probe establishes that the toolchain rebuilds on content, the scenario establishes that a hardlink clone did not take that away. Verified both ways locally: with a real content-freshness nightly, mode on and 4 assertions; with the env var stripped inside the probe — the runner's condition, faithfully — the probe says so by name, mode goes off, the scenario is skipped loudly and the remaining 3 assertions pass.	2026-08-26 12:54:24 -05:00
claude	5e8e773f78	test(hardlink): report the mutation families, don't assert one of them ... CI's nightly is 1.100.0-nightly (2026-08-25); the machine this suite was written on had 1.96.0-nightly (2026-02-24). On the newer one the control's `cp -al` clone mutates only the build/ and *.d families — upstream appears to have stopped rewriting `.fingerprint/*/dep-*` in place under checksum freshness — so the suite went red on the *absence* of a hazard. That is the wrong shape for a gate. The control's job is to prove the hazard exists at all, which a non-empty mutated set already does; naming one family as mandatory makes the suite red whenever upstream stops doing something we never wanted it to do, and red in the CONTROL, where a failure reads as "the hazard is gone" rather than "upstream changed". It is now a note either way. Nothing is given up. The fix scenario asserts the source is byte-identical after a full rebuild in the clone, which covers every family the running Cargo has, named or not — and the checksum-freshness scenario after it tests the stale-reuse hazard directly. The dep-* line only ever documented which family was in play. `unshare_mutable_paths` keeps unsharing dep-* regardless, and its measurement block now records both observations with their versions: 22 MB of a 6.9 GB tree against a failure mode that is a wrong answer rather than a slow one.	2026-08-26 12:50:42 -05:00
claude	3f97d3d1e7	fix(selftest): make the two compiler-backed suites survive a CI runner ... The new gate's first run went red on both suites that drive a real Cargo. Neither failure was a defect in what they test; both were assumptions about the machine, which had only ever been a dev box with a nightly installed and no colour forcing. Fixed here rather than waived — turning a gate on is what obliges fixing what it finds. COLOUR. Every assertion in restore-mtimes-selftest.sh, and one in hardlink-clone-selftest.sh, reads cargo's own words out of a build log (`Compiling libdep`, `Fresh probe`). gitdan-ci's runner image forces colour, so cargo wrote `Compiling\e[0m libdep` into the log and `grep -q "Compiling libdep"` stopped matching. The suite then reported the opposite of what happened — the failure printed the log, and the log plainly said `Compiling libdep`. Both suites now pin CARGO_TERM_COLOR=never, which is the format their assertions are written against. Red-proven: with the export removed and CARGO_TERM_COLOR=always, restore-mtimes-selftest reproduces the CI failure verbatim ("expected to find: Compiling libdep"); with it, 14/14 pass under the same forced colour. THE NIGHTLY PROBE ASKED THE WRONG QUESTION. `cargo +nightly -V` answers "did a cargo proxy called with +nightly exit 0", which is not "will this build have checksum freshness": `-V` short-circuits before `-Z` is validated at all. So the probe said "on" on a runner where the flag was not in effect, and the suite asserted the checksum-freshness mutation family against a build that never had it — failing in the CONTROL, where a failure reads as "the hazard is gone" rather than "the toolchain is wrong". It now probes the capability: `cargo +nightly -Z checksum-freshness locate-project`, the narrowest command that actually parses the flag. It rejects the stable channel and an unknown flag name alike, needs no network and builds nothing. Red-proven: the old channel probe, pointed at a cargo without the flag, reproduces the CI failure exactly. AND THE OFF PATH DID NOT WORK EITHER. The suite's header claimed that without checksum freshness "the test still covers the build/ and *.d families". It did not: the final scenario backdates the source to 2001 and asserts the rebuild is not Fresh, which is checksum-freshness-only reasoning. Under Cargo's ordinary mtime freshness a 2001 source IS older than the artifact and Fresh is the correct answer, so the scenario asserted a bug. It is now gated on the mode and skipped loudly, like the control's dep-* assertion already was: 5 assertions with a nightly, 3 without. CI installs a nightly as well as stable (nightly first, so stable stays the default) — that hazard is the one this whole scheme exists to close, and a CI that skips it is checking the cheap half.	2026-08-26 12:46:03 -05:00
claude	f76789358d	ci(gitea): gate scripts/ with shellcheck and the selftests ... This repo had eight scripts, six selftest suites and nothing that ran any of them. It is a composite action consumed at `@v1` — a moving tag — by three repos' CI, so a bad edit here reaches zemyna, emowheel and lublub at once and is discovered by whichever of them builds next. One job: `shellcheck -x --source-path=scripts scripts/*.sh`, then `bash scripts/selftest.sh`. `-x` follows the `. cache-lib.sh` every script sources, which is where most of the logic being checked lives; without it shellcheck reports SC1091 and analyses each file with a hole in it. The full suite, not `--fast`: `hardlink-clone-selftest.sh` and `restore-mtimes-selftest.sh` are the two that drive a real Cargo rather than a fixture, and selftest.sh's own header says `--fast` is for iterating, not for signing off a change. So the job installs a stable toolchain. The scratch workspaces they build use path dependencies only, so nothing reaches crates.io, and the job references no credentials at all. Modelled on daniel/gitdan's workflow, including the two clauses of the draft-skip guard (`github.event_name != 'pull_request' || !github.event.pull_request.draft` — the first is what stops the second from also skipping pushes to `main`, where there is no `pull_request` context) and the per-commit concurrency group for `push`. No `container.volumes:` entry, deliberately: the suites want a cold target dir every run, since "did this rebuild?" is exactly what restore-mtimes-selftest asks. This job takes no share of the shared CI cache disk budget. Turning the gate on surfaced three pre-existing findings, all fixed here rather than suppressed or waived: - `restore-mtimes-selftest.sh` SC2038: `find | xargs touch` -> `-print0 | xargs -0`. - `hardlink-clone-selftest.sh` SC2295: `${f#$base_fix/}` -> `${f#"$base_fix"/}`. - `cache-lib.sh` SC2016: a per-line disable with the rationale. The single quotes are load-bearing — the program is for the inner shell, where `$f` is its loop variable, `$rc` its accumulator and `$$` its pid — so this is the documented-false-positive case, not a stopgap.	2026-08-26 12:31:50 -05:00
claudeandClaude Opus 5	248af3061e	feat(cargo-cache): hardlink-clone a per-ref Cargo cache from a published snapshot ... Replaces the phase-0 resolution probe with the real actions, merging the two independent per-branch Cargo cache implementations on this forge into the design neither of them had. ## The merge - zemyna seeds a PR branch by `cp -al` hardlink clone (near-free: cost scales with inode count, not bytes) from the base branch's LIVE target dir — a torn read waiting for a second job slot (its own #911). - emowheel seeds from a PUBLISHED IMMUTABLE SNAPSHOT (no race by construction) but with `cp -a`, duplicating ~35 GB per branch. This ships hardlink-clone FROM a published snapshot: zemyna's cost profile, emowheel's soundness, and #911 closed structurally rather than by the runner happening to have one execution slot. ## The bug both implementations have A build inside a `cp -al` clone DOES mutate the directory it was cloned from. Cargo replaces real artifacts, but writes its metadata — and build scripts write their OUT_DIR — with a plain truncating write, straight through the shared inode. Measured set: `.fingerprint/<unit>/dep-<target>` (under CARGO_UNSTABLE_CHECKSUM_FRESHNESS), `build/<pkg>/{output,root-output,out/**}`, `deps/*.d` and `<profile>/*.d`. The checksum-freshness case is a wrong answer, not a slow build: a PR clone rewrites the base's dep-info to describe the PR's sources while the base's cache still holds the artifact built from the base's; once the PR merges, the base's next run finds the checksums match, reports `Fresh`, and links a binary built from the pre-merge code. Reproduced end to end. Fix: hardlink the artifacts (the GB), real-copy the metadata (the MB) — about 3.7% of a 6.9 GB Bevy target dir, against 100% for a full copy. ## Contents - `cargo-cache/action.yml` — consume: resolve keys, seed from the base's snapshot via staging + one atomic rename, strip Cargo lock files, unshare the mutable paths, restore mtimes from git history, lock, prune. - `cargo-cache-publish/action.yml` — publish: record the build watermark, atomically republish the snapshot on a protected branch, release the lock (`mode: release-lock` for the `if: always()` step). - `scripts/` — all logic, so it is testable standalone; the YAML is wiring. - `scripts/*selftest.sh` + `selftest.sh` — five suites, 63 assertions, every fix paired with a control that reproduces the bug. All green locally. Eviction merges emowheel's liveness pass (dead branches pruned unconditionally, not gated on disk pressure) with LRU-under-pressure, but inverts the order within the pressure pass: `target-*` before `snapshot-*`, because a snapshot is hardlinked to everything cloned from it, so evicting one frees almost no real bytes while costing every future PR its warm start. restore-mtimes.sh is ported from emowheel (the watermark variant, which closes the merge hazard zemyna's copy still has) with its provenance de-projectised. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sqh2vscfzisk83VuPVQX9L	2026-08-23 14:18:01 -05:00