From 3b2dec6a50c524a87784852b06904e884af9c114 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 23 Aug 2026 16:52:53 -0500 Subject: [PATCH] test(cargo-cache): cover the stale-reader-marker sweep and its bound MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The staleness path decides whether a publisher may reclaim disk, so getting it wrong means an abandoned marker pins a snapshot generation forever — the exact outcome the bound exists to prevent. It was previously covered only by analogy to prune-cache.sh's .ci-lock-* staleness, which is not the bar. New publish-snapshot-selftest.sh scenario 8 asserts both directions against the SAME backdated marker, which is what separates "honours the bound" from "ignores anything that looks old": * under CACHE_READ_STALE_SECONDS=86400 a three-hour-old marker is left alone and still defers reclamation, exactly as a live reader does; * under the 7200s default the same marker is swept, reported as swept, and the generation it was pinning — plus the one deferred by the first half — is reclaimed. Backdated with `touch -d`, not slept for; the suite stays fast. Red-proven by mutation rather than against the pre-fix scripts, since the whole mechanism is new there and "it does not exist yet" proves nothing about the threshold logic. Mutating live_reader_count's bound test to `true` (never sweep) fails scenario 8: ASSERTION FAILED: the stale marker was not reported as swept and to `false` (sweep everything, bound ignored) fails scenario 6 instead, which is the right blast radius — ignoring the bound means unlinking under a LIVE reader: ASSERTION FAILED: the previous generation was unlinked while a reader still held it Also documents the entry-count check's measured cost in the README: on ext4 with a warm cache over 78,554 entries, 44 ms per metadata walk against 3,126 ms for the `cp -al` it guards — about 2.8%. Not a perf-claiming change; the number is there so the next reader does not have to wonder. The reviewer's other nit — scenarios 8/9 of the seed suite exercising publish-snapshot.sh's interlock — was already covered by the cross-reference in this file's header, so no move. Verification: `bash scripts/selftest.sh` — 5 suites, exit 0, 88 assertions (82 before this commit, 63 at baseline). shellcheck: no new findings. Refs: daniel/gitdan#11, zemyna#911 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Sqh2vscfzisk83VuPVQX9L --- README.md | 4 ++- scripts/publish-snapshot-selftest.sh | 46 ++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index e8a98f4..63bd0a7 100644 --- a/README.md +++ b/README.md @@ -161,7 +161,9 @@ mechanisms, both required: count (the only signal for a subtree unlinked before its parent was listed — there is no error to read). A tree that fails any of the three is deleted and the clone retried; one that fails the last attempt fails the job. A - partial tree never reaches the final name. + partial tree never reaches the final name. The two counts cost one metadata + walk each: measured on ext4 with a warm cache over a 78,554-entry tree, 44 ms + per walk against 3,126 ms for the `cp -al` they guard — about 2.8%. **What this does and does not guarantee.** Three separate claims, deliberately not collapsed into one: diff --git a/scripts/publish-snapshot-selftest.sh b/scripts/publish-snapshot-selftest.sh index 0bbe56d..ef726e5 100755 --- a/scripts/publish-snapshot-selftest.sh +++ b/scripts/publish-snapshot-selftest.sh @@ -31,6 +31,14 @@ # of that snapshot reclaims it once no reader holds it. Without this the # "we defer instead of forcing" answer would just be a disk leak with # better manners. +# 8. AN ABANDONED MARKER DOES NOT PIN A GENERATION FOREVER — and the +# staleness bound is genuinely consulted rather than old markers being +# unconditionally ignored. Scenario 6 covers a marker that is cleanly +# released; this is the other exit from a reader's lifetime, the one a +# killed job takes. Both halves are asserted against the SAME backdated +# marker, which is what separates "honours the bound" from "ignores +# anything that looks old": under a bound wide enough to still cover it, +# that marker must still defer. # # The consumer's half of the same race — a seed catching a rotation mid-clone # — is in seed-target-dir-selftest.sh scenario 8. @@ -160,5 +168,43 @@ ok "the deferred generation is reclaimed once no reader holds it" || fail "scratch left behind: $(find "$root" -maxdepth 1 \( -name '.stage-*' -o -name '.reading-*' \) -print)" ok "no staging or reader-marker scratch left behind" +echo +echo "=== 8: an abandoned reader marker is swept, and the bound is honoured ===" +# A reader whose job the runner killed never reaches its release. Backdated +# rather than slept for: the default bound is two hours, and a suite that +# waited it out would not be a suite anyone runs. +CRASHED="$root/.reading-${SNAP_NAME}-crashed" +date +%s > "$CRASHED" +touch -d '3 hours ago' "$CRASHED" + +# First, the negative control. The same three-hour-old marker, under a bound +# wide enough to still cover it, must defer exactly as a live one does — if +# this passed only because the marker looked old, the sweep below would prove +# nothing about the bound. +replace_file "$TGT/debug/deps/libx.rlib" gen5 +CACHE_READ_STALE_SECONDS=86400 CACHE_READ_GRACE_SECONDS=1 \ + bash "$script_dir/publish-snapshot.sh" "$KEY" "$root" jobWideBound > "$scratch/log" 2>&1 \ + || { cat "$scratch/log"; fail "publish-snapshot.sh exited non-zero"; } +[ -e "$CRASHED" ] || fail "a marker inside the staleness bound was swept anyway" +ok "a marker inside the staleness bound is left alone" +[ -n "$(find "$root" -maxdepth 1 -name ".publish-old-${KEY}-*" -print -quit)" ] \ + || fail "the generation was reclaimed despite a marker inside the bound" +ok "and still defers reclamation, exactly as a live reader does" + +# Now the same marker against the default bound it is genuinely past. +replace_file "$TGT/debug/deps/libx.rlib" gen6 +CACHE_READ_GRACE_SECONDS=1 \ + bash "$script_dir/publish-snapshot.sh" "$KEY" "$root" jobStaleSweep > "$scratch/log" 2>&1 \ + || { cat "$scratch/log"; fail "publish-snapshot.sh exited non-zero"; } +grep -q 'sweeping stale marker' "$scratch/log" || { cat "$scratch/log"; fail "the stale marker was not reported as swept"; } +ok "a marker past the bound is swept, and says so" +[ -e "$CRASHED" ] && fail "the stale marker survived the sweep" +ok "the abandoned marker is gone" +[ -z "$(find "$root" -maxdepth 1 -name '.publish-old-*' -print -quit)" ] \ + || fail "an abandoned marker pinned a generation past its staleness bound" +ok "the generation it was pinning — and the one deferred earlier — are reclaimed" +[ "$(cat "$SNAP/debug/deps/libx.rlib")" = "gen6" ] || fail "the current generation is wrong" +ok "publishing is otherwise unaffected" + echo echo "publish-snapshot-selftest: ${pass_count} assertions passed"