docs(cache): write down the producer half of the leftover naming contract

daniel/gitdan's host-level arbiter (scripts/ci-cache-reclaim.sh) reclaims the
dot-prefixed trees this repo's scripts strand inside the cache volumes it
scans, and reads this repo's reader markers to decide whether one is still
live. That arrangement was documented only on the consuming side: a
contributor here could add or rename a dot-prefixed shape with no local signal
that anything outside the repo depended on the spelling, and the arbiter
enumerates by explicit prefix — deliberately, so it never sees a .reading-*
marker as a candidate — which makes an unannounced shape invisible to it
rather than conservatively handled.

Adds the producing side's half at the sites someone changing a name will
actually be looking at, pointing at gitdan's LEFTOVER NAMING CONTRACT block as
canonical rather than restating it:

- cache-lib.sh gains a header block naming every shape this repo creates under
  a cache root, its producing function, and how each strands; plus the rule
  that adding a shape obliges the same matching prefix over there as renaming
  one does.
- Site notes at .stage-'s and .reading-'s producing lines, and at
  .evicting-'s in prune-cache.sh.
- publish-snapshot.sh's .publish-old- / .publish-new- pair is documented as
  the shapes that are NOT in the arbiter's list today, with .publish-new-
  called out as the one that strands exactly as .stage- does and that neither
  side reclaims.
- The staleness direction: CACHE_READ_STALE_SECONDS and STALE_LOCK_SECONDS are
  mirrored there and the mirrors must be >= ours, because raising ours alone
  makes the arbiter delete a tree under an in-flight clone (its minimum-age
  guard does not back-stop that case). Lowering ours is safe in any order.
- README gains a short section a newcomer meets before adding a scratch
  directory under a cache root, cross-linked from the cache-layout block.

Comments and docs only; no behaviour change.

Closes #7
This commit is contained in:
2026-08-24 12:50:07 -05:00
parent 1f42064d20
commit 0cf6cc5973
4 changed files with 150 additions and 0 deletions
+52
View File
@@ -115,6 +115,10 @@ env:
the only thing a consumer ever clones from.
```
Transient dot-prefixed entries appear alongside those two — staging trees,
eviction asides, reader markers. Their names are a contract with the host that
owns the volume; see [Scratch names in a cache root](#scratch-names-in-a-cache-root-are-a-cross-repo-contract).
`<key>` is the ref sanitised to a safe path component, capped at 48
characters, plus an 8-hex SHA-1 prefix of the *raw* ref. The hash is not
decoration: `feat/foo` and `feat-foo` sanitise identically and would otherwise
@@ -228,6 +232,54 @@ last build, where the historically-correct mtime is exactly the wrong answer.
---
## Scratch names in a cache root are a cross-repo contract
Read this before adding a dot-prefixed directory under a cache root.
The volume is not swept by these scripts alone. A host-level arbiter —
daniel/gitdan's `scripts/ci-cache-reclaim.sh`, which runs outside any job —
reclaims the scratch trees a killed job strands here, and reads the reader
markers to decide whether a tree is still live. Its `LEFTOVER NAMING CONTRACT`
block is the canonical description; this side owns the names.
| name | produced by | if the job dies holding it |
|---|---|---|
| `.stage-<tag>` | `cache-lib.sh`, `hardlink_clone_into()` | stranded; only the arbiter reclaims it |
| `.publish-new-<tag>` | `publish-snapshot.sh` | stranded, and not in the arbiter's list |
| `.publish-old-<key>-<tag>` | `publish-snapshot.sh` | swept by the next publish of that key |
| `.evicting-<name>-<pid>` | `prune-cache.sh`, `evict_dir()` | swept at the start of the next prune pass |
| `.reading-<source>-<tag>` | `cache-lib.sh`, `reader_lock_acquire()` | not garbage — see below |
A `.reading-*` marker is protective, not scratch: it is how both this repo's
prune pass and the arbiter tell an in-flight clone from an abandoned one, and
the arbiter never deletes one. Delete a live marker and the tree it covers
becomes eligible for an unlink underneath the walk that is reading it, which is
the silent truncation the whole interlock exists to prevent.
**The rule: no new dot-prefixed entry under a cache root without a matching
prefix in `ci-cache-reclaim.sh`.** Adding a shape counts exactly as much as
renaming one. That script enumerates by explicit prefix rather than by dotglob
— deliberately, because a dotglob would pull reader markers into the candidate
stream alongside the trees they protect — so a name it has not been told about
is not handled conservatively, it is invisible, and an unreclaimed staging tree
is a full clone of a multi-GB target dir on the one volume whose entire problem
is disk. `.publish-new-` is the standing example: it strands exactly as
`.stage-` does and is in neither side's sweep.
**The staleness constants are part of the same contract, and that half has a
direction.** `CACHE_READ_STALE_SECONDS` (`cache-lib.sh`) and
`STALE_LOCK_SECONDS` (`prune-cache.sh`) are mirrored there, and the arbiter's
copies must be **greater than or equal to** these. Raising one here for longer
jobs, without raising its mirror first, makes the arbiter treat a marker whose
owner still considers it live as stale and delete a tree under an in-flight
clone — and its own minimum-age guard does not back-stop that, since a clone
holding a three-hour-old marker has a roughly three-hour-old staging tree.
Lowering either here needs no coordination: the arbiter then only defers a
reclamation this side would already have permitted, which costs disk rather
than correctness.
---
## Inputs
### `cargo-cache`