docs(cache): write down the producer half of the leftover naming contract
daniel/gitdan's host-level arbiter (scripts/ci-cache-reclaim.sh) reclaims the dot-prefixed trees this repo's scripts strand inside the cache volumes it scans, and reads this repo's reader markers to decide whether one is still live. That arrangement was documented only on the consuming side: a contributor here could add or rename a dot-prefixed shape with no local signal that anything outside the repo depended on the spelling, and the arbiter enumerates by explicit prefix — deliberately, so it never sees a .reading-* marker as a candidate — which makes an unannounced shape invisible to it rather than conservatively handled. Adds the producing side's half at the sites someone changing a name will actually be looking at, pointing at gitdan's LEFTOVER NAMING CONTRACT block as canonical rather than restating it: - cache-lib.sh gains a header block naming every shape this repo creates under a cache root, its producing function, and how each strands; plus the rule that adding a shape obliges the same matching prefix over there as renaming one does. - Site notes at .stage-'s and .reading-'s producing lines, and at .evicting-'s in prune-cache.sh. - publish-snapshot.sh's .publish-old- / .publish-new- pair is documented as the shapes that are NOT in the arbiter's list today, with .publish-new- called out as the one that strands exactly as .stage- does and that neither side reclaims. - The staleness direction: CACHE_READ_STALE_SECONDS and STALE_LOCK_SECONDS are mirrored there and the mirrors must be >= ours, because raising ours alone makes the arbiter delete a tree under an in-flight clone (its minimum-age guard does not back-stop that case). Lowering ours is safe in any order. - README gains a short section a newcomer meets before adding a scratch directory under a cache root, cross-linked from the cache-layout block. Comments and docs only; no behaviour change. Closes #7
This commit is contained in:
@@ -115,6 +115,10 @@ env:
|
||||
the only thing a consumer ever clones from.
|
||||
```
|
||||
|
||||
Transient dot-prefixed entries appear alongside those two — staging trees,
|
||||
eviction asides, reader markers. Their names are a contract with the host that
|
||||
owns the volume; see [Scratch names in a cache root](#scratch-names-in-a-cache-root-are-a-cross-repo-contract).
|
||||
|
||||
`<key>` is the ref sanitised to a safe path component, capped at 48
|
||||
characters, plus an 8-hex SHA-1 prefix of the *raw* ref. The hash is not
|
||||
decoration: `feat/foo` and `feat-foo` sanitise identically and would otherwise
|
||||
@@ -228,6 +232,54 @@ last build, where the historically-correct mtime is exactly the wrong answer.
|
||||
|
||||
---
|
||||
|
||||
## Scratch names in a cache root are a cross-repo contract
|
||||
|
||||
Read this before adding a dot-prefixed directory under a cache root.
|
||||
|
||||
The volume is not swept by these scripts alone. A host-level arbiter —
|
||||
daniel/gitdan's `scripts/ci-cache-reclaim.sh`, which runs outside any job —
|
||||
reclaims the scratch trees a killed job strands here, and reads the reader
|
||||
markers to decide whether a tree is still live. Its `LEFTOVER NAMING CONTRACT`
|
||||
block is the canonical description; this side owns the names.
|
||||
|
||||
| name | produced by | if the job dies holding it |
|
||||
|---|---|---|
|
||||
| `.stage-<tag>` | `cache-lib.sh`, `hardlink_clone_into()` | stranded; only the arbiter reclaims it |
|
||||
| `.publish-new-<tag>` | `publish-snapshot.sh` | stranded, and not in the arbiter's list |
|
||||
| `.publish-old-<key>-<tag>` | `publish-snapshot.sh` | swept by the next publish of that key |
|
||||
| `.evicting-<name>-<pid>` | `prune-cache.sh`, `evict_dir()` | swept at the start of the next prune pass |
|
||||
| `.reading-<source>-<tag>` | `cache-lib.sh`, `reader_lock_acquire()` | not garbage — see below |
|
||||
|
||||
A `.reading-*` marker is protective, not scratch: it is how both this repo's
|
||||
prune pass and the arbiter tell an in-flight clone from an abandoned one, and
|
||||
the arbiter never deletes one. Delete a live marker and the tree it covers
|
||||
becomes eligible for an unlink underneath the walk that is reading it, which is
|
||||
the silent truncation the whole interlock exists to prevent.
|
||||
|
||||
**The rule: no new dot-prefixed entry under a cache root without a matching
|
||||
prefix in `ci-cache-reclaim.sh`.** Adding a shape counts exactly as much as
|
||||
renaming one. That script enumerates by explicit prefix rather than by dotglob
|
||||
— deliberately, because a dotglob would pull reader markers into the candidate
|
||||
stream alongside the trees they protect — so a name it has not been told about
|
||||
is not handled conservatively, it is invisible, and an unreclaimed staging tree
|
||||
is a full clone of a multi-GB target dir on the one volume whose entire problem
|
||||
is disk. `.publish-new-` is the standing example: it strands exactly as
|
||||
`.stage-` does and is in neither side's sweep.
|
||||
|
||||
**The staleness constants are part of the same contract, and that half has a
|
||||
direction.** `CACHE_READ_STALE_SECONDS` (`cache-lib.sh`) and
|
||||
`STALE_LOCK_SECONDS` (`prune-cache.sh`) are mirrored there, and the arbiter's
|
||||
copies must be **greater than or equal to** these. Raising one here for longer
|
||||
jobs, without raising its mirror first, makes the arbiter treat a marker whose
|
||||
owner still considers it live as stale and delete a tree under an in-flight
|
||||
clone — and its own minimum-age guard does not back-stop that, since a clone
|
||||
holding a three-hour-old marker has a roughly three-hour-old staging tree.
|
||||
Lowering either here needs no coordination: the arbiter then only defers a
|
||||
reclamation this side would already have permitted, which costs disk rather
|
||||
than correctness.
|
||||
|
||||
---
|
||||
|
||||
## Inputs
|
||||
|
||||
### `cargo-cache`
|
||||
|
||||
Reference in New Issue
Block a user