fix(prune-cache): close the reader-marker check-then-delete window

Both eviction sites checked for a consumer's `.reading-` marker and then,
seconds later — `usage_gb` runs `du -sk` over a multi-GB tree between the
two — unlinked the directory. A consumer that started a clone inside that
gap had its source removed mid-walk, which `cp -al` does not report: a
subtree unlinked before its parent is listed is silently omitted.

Unreachable today, and only by policy: snapshots belong to protected refs
and protected refs never reach the marker check. `cargo-cache` and
`cargo-cache-publish` take that ref list as two independent inputs, so a
workflow listing a publisher in one and not the other arms this with no
code change at all.

Closed structurally, with publish-snapshot.sh's rotation rather than a new
mechanism: the candidate is renamed aside and only then re-examined, so the
scan the unlink rests on happens strictly after the rename. A consumer that
resolved the directory published its marker before that scan and cannot be
missed; one arriving after cannot resolve the path and starts cold, the same
degrade the publisher's swap window already produces. Renaming disturbs no
clone in flight — no entry is unlinked and the inode is unchanged — so a
declined eviction costs a deferred eviction and nothing else. A reprieved
cache is put back under its own name; one whose name a concurrent seed has
retaken is left aside and swept by a later pass once its readers drain,
since nothing else globs a dotted name.

prune-cache-selftest gains three scenarios (21 -> 31 assertions). Scenario
12 is the one that bites: the `du` the pass runs on its candidate publishes
the marker, placing it strictly after the check and strictly before the
unlink. Against check-then-delete, 1-11 pass and 12 fails; breaking only
the second look and leaving the rename fails it too.

scripts/prune-cache-selftest.sh

@@ -29,6 +29,15 @@
 #   9. OWN CACHE NEVER EVICTED by a sibling pass.
 #  10. SCOPED TO THE CACHE ROOT — a decoy outside it (standing in for another
 #      project's volume) is never touched.
+#  11. A LIVE READER MARKER PROTECTS A CACHE the same way a lock file does — a
+#      directory somebody is hardlink-cloning this instant is not a candidate,
+#      however dead and however tight the disk.
+#  12. AND SO DOES ONE PUBLISHED INSIDE THE CHECK-TO-UNLINK WINDOW, which is
+#      the property a check-then-delete eviction does NOT have. This is the
+#      one that fails against the pre-fix script.
+#  13. A DEFERRED EVICTION IS RECLAIMED, but not while its reader is live.
+#      Nothing else globs a dotted name, so an unswept one is disk lost for
+#      good on the volume whose whole problem is disk.
 set -euo pipefail
 script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
 . "$script_dir/cache-lib.sh"
@@ -112,7 +121,7 @@ mk "snapshot-$LIVE" '2020-01-02'
 # evictable — which is what makes the ORDER the observable property here, not
 # what survives. Assert the eviction order directly from the log.
 run_prune "1000000 1000"
-order=$(grep -o "evicting \(target\|snapshot\)-$LIVE" "$scratch/log" | sed "s/evicting //")
+order=$(grep -o "evicted \(target\|snapshot\)-$LIVE" "$scratch/log" | sed "s/evicted //")
 [ "$(printf '%s\n' "$order" | head -1)" = "target-$LIVE" ] \
   || fail "expected target-$LIVE to be evicted before snapshot-$LIVE, got: $order"
 ok "target dirs are evicted before snapshots"
@@ -170,5 +179,59 @@ decoy="$scratch/other-project"; mkdir -p "$decoy/target-$DEAD"; touch "$decoy/ta
 run_prune "1000000 1000"
 assert_kept "$decoy/target-$DEAD" "a cache outside the cache root is never touched"
 
+echo
+echo "=== 11: a live reader marker protects a cache, like a lock file does ==="
+reset_cache
+date +%s > "$root/.reading-target-$DEAD-job1"
+run_prune "1000000 1000"
+assert_kept "$root/target-$DEAD" "a cache being hardlink-cloned right now survives both passes"
+assert_log "currently cloning it" "the reader is named in the log, not silently honoured"
+rm -f "$root/.reading-target-$DEAD-job1"
+
+echo
+echo "=== 12: a reader marker published INSIDE the check-to-unlink window ==="
+reset_cache
+# A consumer publishes its marker whenever it starts a clone, which can be at
+# any instant — including after the pass has checked for markers and before it
+# unlinks. That window is real time, not a theoretical interleaving: `du -sk`
+# on a multi-GB cache runs for seconds, and the pass runs one on every
+# candidate. It is reproduced deterministically here by making that very `du`
+# publish the marker, which places it strictly after the check and strictly
+# before the unlink — exactly where a check-then-delete eviction cannot see
+# it. The candidate must still be standing afterwards, with its contents
+# intact and nothing left renamed aside.
+mkdir -p "$scratch/bin"
+real_du=$(command -v du)
+cat > "$scratch/bin/du" <<EOF
+#!/usr/bin/env bash
+for arg; do
+  case "\$arg" in */target-$DEAD) date +%s > "$root/.reading-target-$DEAD-racer" ;; esac
+done
+exec "$real_du" "\$@"
+EOF
+chmod +x "$scratch/bin/du"
+( PATH="$scratch/bin:$PATH"; run_prune "1000000 900000" )
+[ -e "$root/.reading-target-$DEAD-racer" ] || fail "the racing marker was never published — scenario 12 proves nothing"
+assert_kept "$root/target-$DEAD" "a cache claimed inside the eviction window is not unlinked"
+assert_kept "$root/target-$DEAD/blob" "the reprieved cache still has its contents"
+assert_log "restored, not evicted" "the reprieve is reported, not silent"
+[ -z "$(ls -d "$root"/.evicting-* 2>/dev/null)" ] || fail "an aside directory was left behind after the reprieve"
+ok "nothing left renamed aside once the eviction is declined"
+rm -f "$root/.reading-target-$DEAD-racer"
+
+echo
+echo "=== 13: a deferred eviction is reclaimed, but not under a live reader ==="
+reset_cache
+aside="$root/.evicting-target-$DEAD-9999"
+mkdir -p "$aside"; head -c 4096 /dev/zero > "$aside/blob"
+date +%s > "$root/.reading-target-$DEAD-job1"
+run_prune "1000000 900000"
+assert_kept "$aside" "a deferred eviction is not reclaimed while a job is still reading it"
+assert_log "deferring its reclamation again" "the continued deferral is reported"
+rm -f "$root/.reading-target-$DEAD-job1"
+run_prune "1000000 900000"
+assert_gone "$aside" "a deferred eviction is reclaimed once its reader is gone"
+assert_log "reclaiming deferred eviction" "the reclamation is reported"
+
 echo
 echo "prune-cache-selftest: ${pass_count} assertions passed"