fix(hardlink): content freshness moved switches, it was not withdrawn
CI / shellcheck + selftests (pull_request) Successful in 1m17s
CI / shellcheck + selftests (pull_request) Successful in 1m17s
`unshare_mutable_paths`' comment recorded that upstream had stopped rewriting `dep-<target>` in place, on a measurement taken against 1.100.0-nightly. It had not. Two unrelated cargo changes landed within four days of each other and between them moved the switch that turns the behaviour on and the path it writes to: - cargo PR #17382 (2026-08-22) demoted `-Z checksum-freshness` to a gate and gave `build.fingerprint` the choice, defaulting to `mtime`. Setting only the gate is accepted and does nothing, which is exactly the result that was read as a withdrawal. - build-dir layout v2 (cargo PR #17354, stable 1.100.0 on 2026-11-12, nightly default since 1.99) moved the file from `.fingerprint/<unit>/dep-*` to `build/<pkg>/<hash>/fingerprint/dep-*`. Measured 2026-08-26 on 1.100.0-nightly (e8cb624d5): same toolchain, same clone procedure, one env var apart — with the gate alone a `cp -al` clone mutates only the build/ and *.d families; add `CARGO_BUILD_FINGERPRINT=content` and the source's dep-info file is mutated through the shared inode again. The hazard is intact. So the suite now exports both switches, and its strongest scenario runs again on current nightlies — verified passing against both layouts. Its control note learned the v2 path too: it looked for the v1 path only, and so printed "does NOT rewrite ... in place" three lines beneath a listing that showed the rewrite. Every claim these comments make is now dated and cited, because the defect being fixed is a comment that cited one measurement and silently stopped reproducing. Layout v2 also drags the artifacts under `build/`, which collapses this function's real-copy set from 21.3% of a target dir to 99.998%. That is a live cost, not a correctness problem, and it is filed as gitdan-actions#14 rather than fixed here. Part of daniel/gitdan#62.
This commit is contained in:
+65
-19
@@ -329,21 +329,18 @@ _unshare_files() {
|
||||
# the source) the following files in the SOURCE were mutated through the
|
||||
# shared inode:
|
||||
#
|
||||
# <profile>/.fingerprint/<unit>/dep-<target> (only under
|
||||
# CARGO_UNSTABLE_CHECKSUM_FRESHNESS,
|
||||
# where this file carries the
|
||||
# per-source blake3 checksums.
|
||||
# NOT reproduced on
|
||||
# 1.100.0-nightly (2026-08-25),
|
||||
# measured by this repo's own CI
|
||||
# — upstream appears to have
|
||||
# stopped writing it in place.
|
||||
# Kept in the unshared set
|
||||
# anyway: it costs 22 MB of a
|
||||
# 6.9 GB tree, and the failure
|
||||
# it guards is a wrong answer,
|
||||
# not a slow one.)
|
||||
# <profile>/build/<pkg>/output, root-output (Cargo build-script metadata)
|
||||
# <profile>/.fingerprint/<unit>/dep-<target> (build-dir layout v1) — or,
|
||||
# <profile>/build/<pkg>/<hash>/fingerprint/dep-<target>
|
||||
# (build-dir layout v2; see the
|
||||
# dated note below for which
|
||||
# Cargo writes which). Only
|
||||
# when Cargo resolves freshness
|
||||
# by CONTENT, where this file
|
||||
# carries the per-source blake3
|
||||
# checksums.
|
||||
# <profile>/build/<pkg>/output, root-output (Cargo build-script metadata;
|
||||
# `<pkg>/<hash>/run/root-output`
|
||||
# under layout v2)
|
||||
# <profile>/build/<pkg>/out/** (whatever the build script
|
||||
# writes into OUT_DIR — build
|
||||
# scripts overwhelmingly use a
|
||||
@@ -359,10 +356,59 @@ _unshare_files() {
|
||||
# sources, reports `Fresh`, and reuses a binary built from the PRE-merge code.
|
||||
# That is silent stale-artifact reuse — a wrong answer, not a slow one.
|
||||
#
|
||||
# So: hardlink the artifacts (the GB), real-copy the metadata (the MB).
|
||||
# Measured on a 6.9 GB Bevy workspace target dir, the unshared set is
|
||||
# .fingerprint 22 MB + build/ 237 MB + a handful of dep-info files — about
|
||||
# 3.7% of the tree, against 100% for a plain `cp -a`.
|
||||
# WHAT UPSTREAM CHANGED, AND WHAT IT DID NOT (measured 2026-08-26, daniel/gitdan#62).
|
||||
#
|
||||
# An earlier revision of this comment recorded that the dep-info write was
|
||||
# "NOT reproduced on 1.100.0-nightly (2026-08-25) — upstream appears to have
|
||||
# stopped writing it in place". That reading was wrong, and the way it was
|
||||
# wrong is the reason this paragraph is dated. Two unrelated upstream changes
|
||||
# landed within days of each other, and between them they moved both the
|
||||
# switch that turns the behaviour on and the path it writes to:
|
||||
#
|
||||
# 1. The ON-SWITCH MOVED. cargo PR #17382 `feat(config): Add build.fingerprint`
|
||||
# (merged 2026-08-22) demoted `-Z checksum-freshness` to a gate: it now
|
||||
# only UNLOCKS the feature, and `build.fingerprint` SELECTS it, defaulting
|
||||
# to `"mtime"`. So `CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true` on its own is
|
||||
# accepted and does nothing, which is exactly the "flag accepted, mtime
|
||||
# anyway" result that was mistaken for a withdrawal. Content freshness
|
||||
# needs BOTH, and with both it is entirely intact:
|
||||
#
|
||||
# CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true CARGO_BUILD_FINGERPRINT=content
|
||||
#
|
||||
# Measured on cargo 1.100.0-nightly (e8cb624d5 2026-08-22): with the gate
|
||||
# alone a `cp -al` clone mutates only the build/ and *.d families; add
|
||||
# `CARGO_BUILD_FINGERPRINT=content` and the source's dep-info file is
|
||||
# mutated through the shared inode again. Same toolchain, same clone, one
|
||||
# env var apart. The hazard was never removed — it was switched off.
|
||||
#
|
||||
# 2. THE PATH MOVED. Build-dir layout v2 (`-Z build-dir-new-layout`, cargo
|
||||
# 1.91) became the nightly default in cargo 1.99 (PR #17258) and was
|
||||
# stabilised by PR #17354, merged 2026-08-18, shipping in cargo 1.100.0
|
||||
# stable on 2026-11-12. Under v2 there is no `<profile>/.fingerprint` and
|
||||
# no `<profile>/deps` at all: everything is regrouped per build unit under
|
||||
# `<profile>/build/<pkg>/<hash>/{fingerprint,out,run}/`, artifacts
|
||||
# included. Bracketed locally: cargo 1.97.1 and 1.98.0-nightly write v1,
|
||||
# 1.100.0-nightly writes v2.
|
||||
#
|
||||
# The `-name .fingerprint` clause below therefore matches nothing under a v2
|
||||
# Cargo, and the dep-info file is covered only because the `-name build` clause
|
||||
# happens to swallow its new home. That is belt-and-braces by accident, not by
|
||||
# design — and the same accident makes this function real-copy essentially the
|
||||
# whole tree, because the artifacts moved under `build/` too. Measured on one
|
||||
# scratch crate (serde + serde_json + regex), same sources both ways:
|
||||
#
|
||||
# cargo 1.97.1 (layout v1) 27.0 MB unshared of 126.7 MB — 21.3%
|
||||
# 1.100.0-nightly (layout v2) 105.9 MB unshared of 105.9 MB — 99.998%
|
||||
#
|
||||
# So the guard still holds and the saving does not. Deliberately NOT fixed
|
||||
# here: adjusting the selection is a change to what gets hardlinked on every
|
||||
# consumer, which wants its own change and its own review, and the deadline is
|
||||
# cargo 1.100.0 stable on 2026-11-12. Tracked as gitdan-actions#14.
|
||||
#
|
||||
# The historical v1 figure this block used to quote stands as measured: on a
|
||||
# 6.9 GB Bevy workspace target dir the unshared set was .fingerprint 22 MB +
|
||||
# build/ 237 MB + a handful of dep-info files — about 3.7% of the tree, against
|
||||
# 100% for a plain `cp -a`. It describes layout v1 only.
|
||||
#
|
||||
# `incremental/` is deliberately left shared: rustc writes each incremental
|
||||
# session to a fresh `s-*-working` directory and finalises it with a rename,
|
||||
|
||||
@@ -5,8 +5,10 @@
|
||||
#
|
||||
# That assumption is FALSE for a plain `cp -al`. Measured, and asserted below
|
||||
# as an explicit control: build in a raw `cp -al` clone and the source's
|
||||
# `.fingerprint/<unit>/dep-*` (under CARGO_UNSTABLE_CHECKSUM_FRESHNESS),
|
||||
# `build/<pkg>/output`, `build/<pkg>/out/**` and `deps/*.d` all change,
|
||||
# dep-info file (`.fingerprint/<unit>/dep-*` under Cargo's build-dir layout
|
||||
# v1, `build/<pkg>/<hash>/fingerprint/dep-*` under v2 — and under content
|
||||
# freshness only, see the probe below), `build/<pkg>/output`,
|
||||
# `build/<pkg>/out/**` and `deps/*.d` all change,
|
||||
# because Cargo and build scripts write those with a plain truncating write
|
||||
# rather than the write-then-rename Cargo uses for real artifacts.
|
||||
#
|
||||
@@ -97,10 +99,19 @@ CONTENT_B='pub fn f() -> u32 { 22222 } pub fn g() -> u32 { 7 }'
|
||||
# short-circuits before `-Z` is
|
||||
# even parsed.
|
||||
# `cargo +nightly -Z checksum-freshness — answers "is this flag still
|
||||
# locate-project` accepted". 1.100.0-nightly
|
||||
# (2026-08-25) accepts it and does
|
||||
# not resolve freshness by content
|
||||
# anyway.
|
||||
# locate-project` accepted", which since cargo PR
|
||||
# #17382 (2026-08-22) is a
|
||||
# different question from "is
|
||||
# content freshness on". That PR
|
||||
# demoted the flag to a gate and
|
||||
# gave `build.fingerprint` the
|
||||
# choice, defaulting to `mtime` —
|
||||
# so 1.100.0-nightly accepts the
|
||||
# flag and resolves freshness by
|
||||
# mtime unless
|
||||
# CARGO_BUILD_FINGERPRINT=content
|
||||
# is set too. Measured 2026-08-26;
|
||||
# see daniel/gitdan#62.
|
||||
#
|
||||
# The scenario at the end of this file depends on one thing and it is neither
|
||||
# of those: that changed content with an OLDER mtime rebuilds. Under mtime
|
||||
@@ -165,7 +176,9 @@ CONTENT_B='pub fn f() -> u32 { 22222 } pub fn g() -> u32 { 7 }'
|
||||
# broken" across the three repos consuming this action — the same category
|
||||
# error the three-state split exists to prevent, one level up. What would
|
||||
# change the answer is not-measured becoming the everyday CI outcome; it is not
|
||||
# (gitdan-ci reports a measured INACTIVE, by measurement).
|
||||
# — gitdan-ci's outcome is a measurement either way. It reported a measured
|
||||
# INACTIVE until 2026-08-26, for the reason recorded at the `export` below, and
|
||||
# an ACTIVE once both switches were set.
|
||||
CHECKSUM_MODE="off"
|
||||
CHECKSUM_REASON="no nightly on PATH accepting -Z checksum-freshness"
|
||||
CARGO_BIN=(cargo)
|
||||
@@ -189,6 +202,14 @@ checksum_freshness_probe() {
|
||||
}
|
||||
if cargo +nightly -Z checksum-freshness locate-project > /dev/null 2>&1; then
|
||||
export CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true
|
||||
# BOTH, since cargo PR #17382 (2026-08-22): the -Z flag only unlocks the
|
||||
# feature and `build.fingerprint` selects it, defaulting to `mtime`. Setting
|
||||
# the gate alone is what made this suite report a measured INACTIVE on
|
||||
# 1.100.0-nightly and skip its strongest scenario (daniel/gitdan#62). Safe to
|
||||
# export unconditionally — a Cargo that does not know the key ignores it
|
||||
# silently, verified 2026-08-26 on 1.93.1 stable and 1.96.0-nightly, both of
|
||||
# which still measure ACTIVE from the gate alone.
|
||||
export CARGO_BUILD_FINGERPRINT=content
|
||||
# Errexit off across the call, so the subshell can arm its own — see the
|
||||
# header. `probe_rc` is read before it is restored.
|
||||
probe_rc=0
|
||||
@@ -203,11 +224,11 @@ if cargo +nightly -Z checksum-freshness locate-project > /dev/null 2>&1; then
|
||||
CHECKSUM_REASON=""
|
||||
;;
|
||||
3)
|
||||
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS
|
||||
CHECKSUM_REASON="this nightly accepts -Z checksum-freshness but resolves freshness by mtime"
|
||||
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS CARGO_BUILD_FINGERPRINT
|
||||
CHECKSUM_REASON="this nightly accepts -Z checksum-freshness and build.fingerprint=content but still resolves freshness by mtime"
|
||||
;;
|
||||
*)
|
||||
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS
|
||||
unset CARGO_UNSTABLE_CHECKSUM_FRESHNESS CARGO_BUILD_FINGERPRINT
|
||||
CHECKSUM_MODE="unmeasured"
|
||||
CHECKSUM_REASON="the probe exited ${probe_rc}, which is not one of its answer codes, so this was NOT MEASURED — this toolchain may or may not resolve freshness by content"
|
||||
# Loud, because the cost is silently lost coverage on a machine that
|
||||
@@ -248,22 +269,31 @@ printf '%s\n' "$ctl_mutated" | sed 's/^/ /'
|
||||
# is to prove the hazard exists at all, which the non-empty set above already
|
||||
# does; this line records WHICH families a given Cargo exhibits.
|
||||
#
|
||||
# `.fingerprint/*/dep-*` is the worst of them — it carries the per-source
|
||||
# The dep-info file is the worst of them — it carries the per-source
|
||||
# checksums, so mutating it through a shared inode turns a hardlink clone into
|
||||
# silent stale-artifact reuse rather than a slow build. It was measured on
|
||||
# cargo 1.9x nightly (see unshare_mutable_paths in cache-lib.sh) and is NOT
|
||||
# reproduced on 1.100.0-nightly (2026-08-25), where the control mutates only
|
||||
# the build/ and *.d families. Failing on its absence would mean this suite
|
||||
# goes red whenever upstream stops doing something we never wanted it to do —
|
||||
# and it would go red in the CONTROL, where a failure reads as "the hazard is
|
||||
# gone" rather than "upstream changed". Nothing is lost by reporting it: the
|
||||
# fix scenario below asserts the source is byte-identical after a full rebuild
|
||||
# in the clone, which covers every family this Cargo has, named or not.
|
||||
# silent stale-artifact reuse rather than a slow build. Failing on its absence
|
||||
# would mean this suite goes red whenever upstream stops doing something we
|
||||
# never wanted it to do — and it would go red in the CONTROL, where a failure
|
||||
# reads as "the hazard is gone" rather than "upstream changed". Nothing is lost
|
||||
# by reporting it: the fix scenario below asserts the source is byte-identical
|
||||
# after a full rebuild in the clone, which covers every family this Cargo has,
|
||||
# named or not.
|
||||
#
|
||||
# THE PATTERN MUST MATCH BOTH LAYOUTS, and that is not a detail. Cargo's
|
||||
# build-dir layout v2 moved the file from `<profile>/.fingerprint/<unit>/dep-*`
|
||||
# to `<profile>/build/<pkg>/<hash>/fingerprint/dep-*` (stabilised by cargo PR
|
||||
# #17354, cargo 1.100.0, stable 2026-11-12; nightly default since 1.99). An
|
||||
# earlier cut of this line looked for the v1 path only, so on 2026-08-26,
|
||||
# against 1.100.0-nightly with content freshness genuinely on, it printed
|
||||
# "does NOT rewrite ... in place" directly beneath a control listing that
|
||||
# showed the rewrite. A reporting line that can contradict the data three
|
||||
# lines above it is worse than no line at all. `fingerprint/.*dep-` matches
|
||||
# either layout and neither `.d` family.
|
||||
if [ "$CHECKSUM_MODE" = "on" ]; then
|
||||
if printf '%s' "$ctl_mutated" | grep -q '\.fingerprint/.*/dep-'; then
|
||||
echo " note: this cargo DOES rewrite .fingerprint/*/dep-* in place under checksum freshness"
|
||||
if printf '%s' "$ctl_mutated" | grep -q 'fingerprint/.*dep-'; then
|
||||
echo " note: this cargo DOES rewrite its dep-info fingerprint file in place under content freshness"
|
||||
else
|
||||
echo " note: this cargo does NOT rewrite .fingerprint/*/dep-* in place; only the build/ and *.d families appear above"
|
||||
echo " note: this cargo does NOT rewrite its dep-info fingerprint file in place; only the build/ and *.d families appear above"
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
@@ -64,7 +64,8 @@
|
||||
# two refs' fingerprints ever share a directory and this script never has to
|
||||
# arbitrate freshness across refs — only within one ref's own history, which
|
||||
# is exactly what it is built to do soundly. On a nightly toolchain,
|
||||
# CARGO_UNSTABLE_CHECKSUM_FRESHNESS is a complementary, stronger guarantee
|
||||
# CARGO_UNSTABLE_CHECKSUM_FRESHNESS plus CARGO_BUILD_FINGERPRINT=content (both,
|
||||
# since cargo PR #17382 on 2026-08-22) is a complementary, stronger guarantee
|
||||
# (content-addressed rather than mtime-based freshness); this script is not
|
||||
# made redundant by it, because directory-form `rerun-if-changed` build-script
|
||||
# watches are not covered by it and stable historical mtimes stay cheap
|
||||
|
||||
Reference in New Issue
Block a user