README.md

@@ -115,6 +115,10 @@ env:
                      the only thing a consumer ever clones from.
 ```
 
+Transient dot-prefixed entries appear alongside those two — staging trees,
+eviction asides, reader markers. Their names are a contract with the host that
+owns the volume; see [Scratch names in a cache root](#scratch-names-in-a-cache-root-are-a-cross-repo-contract).
+
 `<key>` is the ref sanitised to a safe path component, capped at 48
 characters, plus an 8-hex SHA-1 prefix of the *raw* ref. The hash is not
 decoration: `feat/foo` and `feat-foo` sanitise identically and would otherwise
@@ -228,6 +232,63 @@ last build, where the historically-correct mtime is exactly the wrong answer.
 
 ---
 
+## Scratch names in a cache root are a cross-repo contract
+
+Read this before adding a dot-prefixed directory under a cache root.
+
+The volume is not swept by these scripts alone. A host-level arbiter —
+daniel/gitdan's `scripts/ci-cache-reclaim.sh`, which runs outside any job —
+reclaims the scratch trees a killed job strands here, and reads the reader
+markers to decide whether a tree is still live. Its `LEFTOVER NAMING CONTRACT`
+block is the canonical description; this side owns the names.
+
+| name | produced by | if the job dies holding it |
+|---|---|---|
+| `.stage-<tag>` | `cache-lib.sh`, `hardlink_clone_into()` | stranded; only the arbiter reclaims it |
+| `.publish-new-<tag>` | `publish-snapshot.sh` | stranded; see `daniel/gitdan#30` |
+| `.publish-old-<key>-<tag>` | `publish-snapshot.sh` | swept by the next publish of that key, if there is one |
+| `.evicting-<name>-<pid>` | `prune-cache.sh`, `evict_dir()` | swept at the start of the next prune pass |
+| `.reading-<source>-<tag>` | `cache-lib.sh`, `reader_lock_acquire()` | not garbage — see below |
+
+A `.reading-*` marker is protective, not scratch: it is how both this repo's
+prune pass and the arbiter tell an in-flight clone from an abandoned one, and
+the arbiter never deletes one. Delete a live marker and the tree it covers
+becomes eligible for an unlink underneath the walk that is reading it, which is
+the silent truncation the whole interlock exists to prevent.
+
+**The rule: no new dot-prefixed entry under a cache root without a matching
+prefix in `ci-cache-reclaim.sh`.** Adding a shape counts exactly as much as
+renaming one. That script enumerates by explicit prefix rather than by dotglob
+— deliberately, because a dotglob would pull reader markers into the candidate
+stream alongside the trees they protect — so a name it has not been told about
+is not handled conservatively, it is invisible, and an unreclaimed staging tree
+is a full clone of a multi-GB target dir on the one volume whose entire problem
+is disk. `.publish-new-` is the worked example: the two `.publish-*` names were
+outside the contract when it was written, and `.publish-new-` strands exactly
+as `.stage-` does, so it is the one that needed catching. Bringing both under
+the arbiter's enumeration is tracked as `daniel/gitdan#30`.
+
+The two lists are meant to be the same length — the five names above, and the
+prefix constants `ci-cache-reclaim.sh` *declares*. Not the subset it enumerates
+as reclaim candidates: that one is smaller, because `.reading-` is read and
+never swept. A mismatch means one side gained a shape without telling the
+other, which is the drift the rule exists to catch and the cheapest thing to
+check.
+
+**The staleness constants are part of the same contract, and that half has a
+direction.** `CACHE_READ_STALE_SECONDS` (`cache-lib.sh`) and
+`STALE_LOCK_SECONDS` (`prune-cache.sh`) are mirrored there, and the arbiter's
+copies must be **greater than or equal to** these. Raising one here for longer
+jobs, without raising its mirror first, makes the arbiter treat a marker whose
+owner still considers it live as stale and delete a tree under an in-flight
+clone — and its own minimum-age guard does not back-stop that, since a clone
+holding a three-hour-old marker has a roughly three-hour-old staging tree.
+Lowering either here needs no coordination: the arbiter then only defers a
+reclamation this side would already have permitted, which costs disk rather
+than correctness.
+
+---
+
 ## Inputs
 
 ### `cargo-cache`

scripts/cache-lib.sh

@@ -8,6 +8,73 @@
 # needs as an argument, so the selftests can drive them against scratch
 # directories without a CI context.
 
+# ---------------------------------------------------------------------------
+# CROSS-REPO CONTRACT: the dot-prefixed names left in a cache root
+# ---------------------------------------------------------------------------
+#
+# The volume these scripts write into is also swept by a host-level arbiter
+# that runs outside any job and outside this repo: daniel/gitdan's
+# `scripts/ci-cache-reclaim.sh`. It reclaims the dot-prefixed trees a killed
+# job strands here, and it reads the reader markers below to decide whether one
+# of those trees is still live. Its `LEFTOVER NAMING CONTRACT` block is the
+# canonical description of the arrangement; what follows is the producing
+# side's half — which names this repo creates, and what changing one obliges.
+#
+# Created directly under a cache root:
+#
+#   .stage-<tag>              cache-lib.sh, hardlink_clone_into(): the tree a
+#                             clone is built in before the atomic rename that
+#                             gives it its real name. <tag> is unique per job
+#                             per run, so a job killed before the rename
+#                             strands a whole hardlink clone under a name no
+#                             later run reuses. Nothing in this repo sweeps it.
+#   .publish-new-<tag>        publish-snapshot.sh: the staged snapshot, between
+#                             its clone and the swap. Strands the same way.
+#   .publish-old-<key>-<tag>  publish-snapshot.sh: the rotated-away generation,
+#                             kept while a reader still holds it and swept by
+#                             the next publish of the same key — if there ever
+#                             is one. A merged or renamed branch never
+#                             publishes again, and its last one stays.
+#   .evicting-<name>-<pid>    prune-cache.sh, evict_dir(): a cache renamed
+#                             aside so the decision to unlink it can be retaken
+#                             after the rename. Swept at the start of every
+#                             prune pass, so it only strands when this repo's
+#                             workflow stops running at all.
+#   .reading-<source>-<tag>   cache-lib.sh, reader_lock_acquire(): NOT garbage.
+#                             It is the live-reader signal the arbiter reads,
+#                             and the one shape it must never delete — removing
+#                             one clears the way to unlink a tree out from
+#                             under an in-flight walk, which is the silent
+#                             truncation this whole interlock exists to
+#                             prevent.
+#
+# THE RULE, which the arbiter states as its own: no new dot-prefixed entry
+# under a cache root without a matching prefix in that script. ADDING a shape
+# counts exactly as much as renaming one, because that script enumerates by
+# explicit prefix rather than by dotglob — deliberately, since a dotglob would
+# pull reader markers into the candidate stream alongside the trees they
+# protect. A shape it has not been told about is not handled conservatively, it
+# is invisible: an unreclaimed staging tree is a full clone of a multi-GB
+# target dir on the one volume whose entire problem is disk.
+#
+# The two `.publish-*` names predate the contract and were outside it when this
+# block was written — the rule catching an uncovered shape on its first
+# application. `.publish-new-` is the one that mattered: it is tagged per job
+# per run exactly as `.stage-` is, so a publisher killed before the swap
+# strands a tree under a name no later run of that script matches, which is
+# precisely the shape only the arbiter can reach. Bringing both in is tracked
+# as daniel/gitdan#30.
+#
+# The two lists are meant to be the same length: the five names above, and the
+# prefix constants that script DECLARES — not the subset it enumerates as
+# reclaim candidates, which is smaller because `.reading-` is read and never
+# swept. A shape here without a constant there is one side having changed
+# without telling the other, and it is cheapest to notice by counting.
+#
+# The two staleness constants the arbiter mirrors are part of the same
+# contract, and that half has a direction to it — see CACHE_READ_STALE_SECONDS
+# below and STALE_LOCK_SECONDS in prune-cache.sh.
+
 # ---------------------------------------------------------------------------
 # Cache keys
 # ---------------------------------------------------------------------------
@@ -249,8 +316,24 @@ CACHE_READ_GRACE_SECONDS="${CACHE_READ_GRACE_SECONDS:-300}"
 # A marker older than this belongs to a job the runner killed before it could
 # clean up. Honouring one forever would let a crashed job pin an entire
 # snapshot generation on disk permanently.
+#
+# RAISING THIS IS A CROSS-REPO CHANGE, and the drift is not symmetric.
+# daniel/gitdan's ci-cache-reclaim.sh mirrors this value as
+# CI_CACHE_READER_STALE_SECONDS (and CI_CACHE_LEFTOVER_MIN_AGE_SECONDS beside
+# it), and its copies must be GREATER THAN OR EQUAL TO this one. Raise this for
+# longer jobs while that one stays at 7200 and the arbiter reads a marker whose
+# owner still considers it live as stale, then deletes the tree under an
+# in-flight clone; its minimum-age guard does not back-stop that, because a
+# clone holding a three-hour-old marker has a staging tree roughly three hours
+# old too, so both of its guards pass. Raise theirs first. Lowering this one
+# needs no coordination at all: the arbiter then defers a reclamation this side
+# would already have permitted, which costs disk and not correctness.
 CACHE_READ_STALE_SECONDS="${CACHE_READ_STALE_SECONDS:-7200}"
 
+# `.reading-<source>-<tag>` is a contract name, not a private one: the
+# host-level arbiter reads these to tell a live clone from an abandoned one,
+# and never deletes one. See the cross-repo contract at the top of this file
+# before changing the spelling.
 reader_marker_path() { printf '%s/.reading-%s-%s' "$1" "$2" "$3"; }
 
 reader_lock_acquire() {
@@ -388,6 +471,10 @@ hardlink_clone_into() {
 
   parent=$(dirname "$dst")
   src_name=$(basename "$src")
+  # `.stage-<tag>` is a contract name (see the top of this file): a job killed
+  # between the copy below and the rename at the end strands this tree, and the
+  # only thing that ever reclaims one is the host-level arbiter, by this exact
+  # prefix.
   tmp="${parent}/.stage-${tag}"
 
   attempt=1

scripts/prune-cache.sh

@@ -85,6 +85,10 @@ ROOT="${1:?usage: prune-cache.sh <cache-root> <own-target-dir> <protected-branch
 OWN_DIR="${2:?}"
 PROTECTED_REFS="${3:-}"
 MIN_FREE_PCT="${4:-10}"
+# Mirrored by daniel/gitdan's ci-cache-reclaim.sh, whose copy must be >= this
+# one — raising this without raising theirs first lets that script treat a lock
+# this side still honours as abandoned. Same direction and same reasoning as
+# CACHE_READ_STALE_SECONDS; the argument is written out in cache-lib.sh.
 STALE_LOCK_SECONDS="${STALE_LOCK_SECONDS:-7200}"
 # An aside directory is in flight for one rename plus one marker glob —
 # milliseconds. Anything older belongs to a pass that died between the two, so
@@ -174,6 +178,12 @@ is_locked() {
 # leaves the source inode unchanged, which is precisely why the publish side
 # can rotate a snapshot out from under a live reader. A declined eviction
 # therefore costs a deferred eviction and nothing else.
+#
+# `.evicting-<name>-<pid>` is a contract name shared with daniel/gitdan's
+# host-level arbiter (see the cross-repo contract at the top of cache-lib.sh).
+# The sweep at the top of this script reclaims these on every pass, so the only
+# one that reaches the arbiter belongs to a repo whose workflow has stopped
+# running — which is exactly the case no in-workflow pass can reach.
 evict_dir() {
   local dir="$1" name aside
   name=$(basename "$dir")

scripts/publish-snapshot.sh

@@ -12,9 +12,11 @@
 # The swap is two renames, not one, because POSIX rename() can only replace an
 # EMPTY directory and a snapshot from a prior publish is not one:
 #
-#   1. stage the new snapshot at .stage-<tag>   (copy time is off every
+#   1. stage the new snapshot at .publish-new-<tag>, which hardlink_clone_into
-#      consumer's hot path — nothing reads a staging path);
+#      builds at its own .stage-<tag> and renames there (copy time is off
-#   2. rename the current snapshot aside to .publish-old-<tag>, if present;
+#      every consumer's hot path — nothing reads either path);
+#   2. rename the current snapshot aside to .publish-old-<key>-<tag>, if
+#      present;
 #   3. rename the staged snapshot into place.
 #
 # Step 3 is a single atomic rename onto a path now guaranteed absent, so it
@@ -29,8 +31,9 @@
 # consumer still WALKING the old generation has its entries unlinked out from
 # under it, and `cp -al` does not report a subtree that was removed before it
 # read the parent's listing. That is a silently truncated clone — the failure
-# mode this script's own selftest (scenario 8) reproduces against the
+# mode seed-target-dir-selftest.sh's scenario 8b reproduces from the consumer's
-# unguarded version.
+# side, against the unguarded version. (This script's own selftest also has a
+# scenario 8; that one is the abandoned-reader-marker case, not this.)
 #
 # So the unlink is interlocked with the consume side rather than
 # unconditional: after the swap, this script waits for every in-flight reader
@@ -50,6 +53,19 @@ SRC=$(target_dir_for "$ROOT" "$OWN_KEY")
 DST=$(snapshot_dir_for "$ROOT" "$OWN_KEY")
 # Keyed by cache key as well as tag, so a deferred generation can be matched
 # back to the snapshot whose readers must drain before it is safe to reclaim.
+#
+# `.publish-old-` and `.publish-new-` are dot-prefixed names under a cache
+# root, so the cross-repo contract at the top of cache-lib.sh covers them:
+# renaming either, or adding a third, requires a matching prefix in
+# daniel/gitdan's ci-cache-reclaim.sh. Both were outside its list when that
+# contract was written, and `.publish-new-` is why it mattered: it is tagged
+# per job per run, so a publisher killed between staging its snapshot and the
+# swap below strands a full hardlink clone that no later run of this script
+# will ever match. `.publish-old-` is milder — the sweep further down reclaims
+# it on the next publish of the same key, if there ever is one; a key whose
+# branch is merged, deleted or renamed is never published again, and its
+# rotated generation stays until something outside this repo takes it.
+# Bringing both under the arbiter's enumeration is tracked as daniel/gitdan#30.
 OLD="${ROOT}/.publish-old-${OWN_KEY}-${TAG}"
 SNAP_NAME=$(basename "$DST")
 GRACE="${CACHE_READ_GRACE_SECONDS}"
@@ -85,6 +101,8 @@ start=$(date +%s)
 # the snapshot it just published — the same aliasing hazard the consume side
 # closes, pointing the other way. The staging path is not the final name, so
 # `hardlink_clone_into`'s rename lands on DST only after OLD is out of the way.
+# `.publish-new-` is a contract name — see the note at OLD above before
+# changing it.
 TMP_DST="${ROOT}/.publish-new-${TAG}"
 rm -rf "$TMP_DST"
 hardlink_clone_into "$SRC" "$TMP_DST" "$TAG" || {