Author SHA1 Message Date
claudeandClaude Opus 5 4da6e8d30f docs(cache): write down the producer half of the depended-upon names contract
daniel/gitdan's ci-cache-reclaim.sh hard-codes four dot-prefixed names it
reads to make correctness decisions but never reclaims as leftovers —
.ci-lock-*, .cache-last-used, .gitea-last-used, .ci-keep — none of which
carried a matching note here (gitdan#32, the follow-up shape to #28/#30
that #7/PR #8 covered for the leftover names specifically).

Of the four, this repo actually produces two:

- .ci-lock-<id>, written by scripts/cache-lock.sh (acquire/release) and
  cache-lib.sh's write_cache_lock(). Documents the contract at
  cache-lock.sh's header, the site someone renaming the marker would
  most likely be editing.
- .cache-last-used, stamped every run by cargo-cache/action.yml. Documents
  the contract at the exact line that writes it.

The other two are read by gitdan's script but produced by nothing in this
repo: .gitea-last-used is a legacy naming convention individual repos
used before adopting the shared cargo-cache action (nothing here writes
it today), and .ci-keep is a per-repo, hand-placed opt-out any consuming
repo's own workflow may drop directly into a cache directory, with no
single owner. Both get a paragraph in README.md's new subsection
explaining why no producer-side counterpart exists for them, rather than
inventing an owner this repo doesn't have.

README.md's "Scratch names in a cache root are a cross-repo contract"
section gains a new subsection, "Names this repo doesn't reclaim, but
the arbiter depends on", covering all four and pointing at gitdan's
DEPENDED-UPON NAMES CONTRACT block as the canonical description.

No behaviour change: comments and docs only.

Ref: gitdan#32. Consumer-side counterpart: daniel/gitdan (this branch's
sibling PR).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJHKJFJVUjnvVsemVBnvrd
2026-08-24 17:08:54 -05:00
4 changed files with 56 additions and 109 deletions
+35 -19
View File
@@ -287,6 +287,36 @@ Lowering either here needs no coordination: the arbiter then only defers a
reclamation this side would already have permitted, which costs disk rather
than correctness.
### Names this repo doesn't reclaim, but the arbiter depends on
The five names above are leftovers — dead trees gitdan's arbiter finds and
deletes. Four other dot-prefixed names change that arbiter's *behaviour*
without ever being a leftover: it reads them to make a correctness decision
and never reclaims them. They are a real cross-repo dependency too, on the
name's spelling rather than on the tree's lifetime — gitdan's
`DEPENDED-UPON NAMES CONTRACT` block (in `scripts/ci-cache-reclaim.sh`) is the
canonical description; this is the producer's half for the two we own.
| Name | Produced here by | Consequence of an unannounced rename |
|---|---|---|
| `.ci-lock-<id>` | `scripts/cache-lock.sh` (acquire/release) and `cache-lib.sh`'s `write_cache_lock()` | The arbiter's lock check silently stops matching — a live job's staging tree loses its liveness guard and becomes an ordinary reclaim candidate while still in use |
| `.cache-last-used` | `cargo-cache/action.yml`, stamped every run | The arbiter falls back to directory mtime — silently reordering its eviction order, and possibly failing to recognise the directory as a cache dir at all |
Two more names are in gitdan's list — `.gitea-last-used` and `.ci-keep` — and
neither is produced by anything in this repo, so there is no producer-side
half to write here. `.gitea-last-used` is a naming convention individual
repos used before adopting this shared action; nothing here writes it, and
gitdan's script reads it only for compatibility with directories created
under that older scheme. `.ci-keep` is a per-repo, hand-placed opt-out a
consuming repo's own workflow drops directly into a cache directory it wants
exempted from eviction — never something this action or its scripts write.
**Adding a fifth depended-upon name counts as much as renaming one of the two
above.** If a future change here makes gitdan's arbiter start depending on the
spelling of some new dot-prefixed name — a name it reads for a decision but
never reclaims — that is exactly this category, and it needs the matching
`DEPEND_*` entry on gitdan's side before it ships, not after.
---
## Inputs
@@ -422,7 +452,7 @@ bash scripts/selftest.sh --fast # fixture-only suites, no compiler
| suite | covers |
|---|---|
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler. |
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published, and the publisher's log showing it waited on the consumer's own reader-lock marker before reclaiming a rotated snapshot |
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published |
| `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone |
| `prune-cache-selftest.sh` | liveness, protection, locking, eviction order, self-clear, **and that a cache a job claims *inside* the check-to-unlink window survives it** — against a real scratch `origin` |
| `restore-mtimes-selftest.sh` | the merge hazard and the watermark that closes it, including the two-jobs-one-namespace case. Needs a real compiler. |
@@ -442,7 +472,7 @@ key and asserts only what must be true whichever of them wins the rename.
which places the interference inside the window rather than hoping it lands
there. `prune-cache-selftest.sh` scenario 12 stubs `du`, so the pass's own
measurement publishes a reader marker strictly between its check and its
unlink. `seed-target-dir-selftest.sh` uses the shape five times over, on two
unlink. `seed-target-dir-selftest.sh` uses the shape four times over, on two
different commands. Scenarios 8a, 8b and 8c stub `cp`, so the consumer's own
clone is what rotates the snapshot underneath it, loses a subtree of its own
source, or reports a failure over a tree that is in fact whole — each strictly
@@ -453,14 +483,9 @@ could not be READ rather than on one that changed, and the only way to make
that the sole witness is to fail the identity reads while the copy between
them succeeds. 8a also starts a second real process — the actual
`publish-snapshot.sh` — but the stub is what fixes where its swap lands; the
concurrency is incidental to the determinism. Scenario 11 reuses 8a's exact
stub and the same forced rotation, but reads a different witness: not the
consumer's own checks, but a line in the *publisher's* log reporting that it
waited on the reader marker this consumer's clone wrote — `reader_lock_acquire`
is exercised by 8a already, but nothing asserts it actually fired until this
scenario reads that line back. Every stub asserts that it fired, because a
scenario whose interference silently did not happen passes for the wrong
reason.
concurrency is incidental to the determinism. Every stub asserts that it
fired, because a scenario whose interference silently did not happen passes
for the wrong reason.
**A synthetic stand-in for the other side, where that artefact *is* the
contract.** `publish-snapshot-selftest.sh` scenarios 6 to 8 hold a
@@ -492,15 +517,6 @@ how both `[ "$cp_rc" -eq 0 ]` and `[ "$i_before" != missing ]` sat unpinned
(issue #5) while looking well covered. Isolating a check means constructing
the state only it can see, not the state that trips several at once.
Scenario 11 applies the same discipline to a witness outside the clone
entirely: not which of several checks inside `hardlink_clone_into` caught a
fault, but whether `reader_lock_acquire`'s marker was observed by anything
outside it at all. The consumer's own log and exit status are silent either
way — a run with the marker deleted still succeeds — so what is asserted is
one line in the *publisher's* log reporting that it waited. Deleting
`reader_lock_acquire` (issue #10) leaves that line unwritten without failing
anything else in the suite.
The action YAML holds no logic beyond wiring; everything testable lives in
`scripts/`. A composite action needs `shell: bash` on every `run:` step, and
the actions reach their shared scripts through
+8
View File
@@ -165,6 +165,14 @@ runs:
# touched unconditionally every run: a run that hits the cache for every
# crate may write nothing at all inside the tree, which would make a
# just-used directory look stale to the eviction pass.
#
# `.cache-last-used` IS A CROSS-REPO CONTRACT NAME with daniel/gitdan's
# `scripts/ci-cache-reclaim.sh`, which reads this exact marker for its own
# LRU ordering and to recognise a directory as a cache dir at all. Rename
# it here without a matching change there and that script silently falls
# back to directory mtime for both — see README.md's "Scratch names in a
# cache root" section and the `DEPENDED-UPON NAMES CONTRACT` block in
# gitdan's script.
- shell: bash
run: |
set -euo pipefail
+13
View File
@@ -14,6 +14,19 @@
# until STALE_LOCK_SECONDS, after which it is treated as abandoned by a job
# the runner killed before it reached its own release step. Honouring a lock
# forever would let one crashed job pin a directory permanently.
#
# `.ci-lock-<id>` IS A CROSS-REPO CONTRACT NAME with daniel/gitdan's
# `scripts/ci-cache-reclaim.sh`: that host-level arbiter reads this same
# marker (its `leftover_is_locked()`) to decide whether a leftover it is about
# to reclaim is still held open by a live job. `cache-lib.sh`'s
# `write_cache_lock()` writes the identical shape independently of this
# script. Rename or reshape this marker here without a matching change on
# gitdan's side and the arbiter's lock check silently stops matching — a live
# job's staging tree loses its liveness guard and becomes an ordinary reclaim
# candidate while still in use. See the "Scratch names in a cache root" section
# in README.md, and daniel/gitdan's `DEPENDED-UPON NAMES CONTRACT` block in
# `scripts/ci-cache-reclaim.sh`, for the full arrangement — this repo's half
# is: don't rename `.ci-lock-` without telling that script.
set -euo pipefail
MODE="${1:?usage: cache-lock.sh acquire|release <dir> <lock-id>}"
-90
View File
@@ -74,17 +74,6 @@
# onto one set of fingerprints. That is the silent stale-reuse bug the
# whole scheme exists to prevent, so the failure has to abort the clone
# rather than be swallowed.
# 11. THE CONSUMER'S MARKER ACTUALLY STOPS THE PUBLISHER — reader_lock_acquire
# is exercised (not faked, unlike publish-snapshot-selftest.sh's scenario
# 6) by a real hardlink_clone_into racing a real, concurrent
# publish-snapshot.sh republish of the exact snapshot being cloned. Pins
# that the publisher OBSERVABLY WAITS on this consumer's marker — its own
# log reports entering the drain wait — rather than only that the run
# succeeds, which stayed green with the marker call deleted (issue #10).
# Kept as its own scenario, not folded into 8a, because 8a already pins
# exactly one property (the identity check) for exactly one mutant, and
# the suite's one-scenario-one-mutant diagonal across 8a to 8d and 10 is
# deliberate.
set -euo pipefail
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
. "$script_dir/cache-lib.sh"
@@ -646,84 +635,5 @@ leftovers=$(find "$root" -maxdepth 1 \( -name '.stage-*' -o -name '.reading-*' \
ok "no staging or reader-marker scratch left behind"
rm -f "$scratch/bin/cp"
echo
echo "=== 11: the consumer's marker actually stops the publisher ==="
# hardlink_clone_into's reader_lock_acquire call is exercised here, not
# faked. publish-snapshot-selftest.sh's scenario 6 stands a hand-written
# marker file in for "a consumer whose clone outlasts the grace period" — a
# deliberate simplification that does not need this consumer's clone code at
# all, so it cannot tell reader_lock_acquire apart from no marker existing.
# This scenario forces the two real scripts to race on the same snapshot: a
# consumer hardlink-cloning it, and a publisher republishing it out from
# under that clone, exactly as 8a does to force the identity check — but what
# is pinned here is not the consumer's response to the rotation (8a's
# property), it is that the PUBLISHER, reading the SAME marker this
# consumer's clone wrote, is observed to have entered its drain wait. Deleting
# reader_lock_acquire (issue #10) leaves the marker never written: the
# publisher's live_reader_count sees zero readers on its first check and
# proceeds straight to reclaiming the rotated generation, silently — the run
# still succeeds, and nothing about its own outcome says so, only the absence
# of a line in the publisher's log.
INTLK=$(cache_key feat/interlock-consumer)
INTLK_BASE=$(cache_key release/5)
TAG_INTLK=jobIntlk
rm -rf "$root/snapshot-$INTLK_BASE" "$root/target-$INTLK_BASE"
make_tree "$root/snapshot-$INTLK_BASE" intlk-gen1
make_tree "$root/target-$INTLK_BASE" intlk-gen2
intlk_gen1_inode=$(stat -c '%i' "$root/snapshot-$INTLK_BASE")
cat > "$scratch/bin/cp" <<EOF
#!/usr/bin/env bash
# Fires once, only on the consumer's own top-level hardlink clone — identified
# by its destination, the seed's private staging path. Every other cp in the
# process tree (the unshare copies, and the publisher's own staging clone)
# falls through to the real one.
if [ "\${@: -1}" = "$root/.stage-$TAG_INTLK" ] && [ ! -e "$scratch/firedIntlk" ]; then
: > "$scratch/firedIntlk"
rc=0; "$real_cp" "\$@" || rc=\$?
# Concurrently: a real, second publish of the exact snapshot this consumer
# is cloning — the republish that, without the marker this consumer's clone
# holds, would reclaim the generation out from under it.
( CACHE_READ_GRACE_SECONDS=10 bash "$script_dir/publish-snapshot.sh" "$INTLK_BASE" "$root" pubIntlk \
> "$scratch/logPubIntlk" 2>&1
echo \$? > "$scratch/rcPubIntlk" ) &
# Hand control back only once the swap is on disk, so the identity read
# immediately after this cp is guaranteed to resolve to the new generation
# — the same technique 8a uses to force the interleaving rather than hope
# for it. reader_lock_release does not run until AFTER this script exits,
# so the marker stays live for the publisher's whole swap-and-scan.
deadline=\$(( \$(date +%s) + 60 ))
while [ "\$(stat -c '%i' "$root/snapshot-$INTLK_BASE" 2>/dev/null)" = "$intlk_gen1_inode" ]; do
[ "\$(date +%s)" -lt "\$deadline" ] || { echo "stub cp: the publisher never swapped the snapshot" >&2; exit 92; }
sleep 0.05
done
exit \$rc
fi
exec "$real_cp" "\$@"
EOF
chmod +x "$scratch/bin/cp"
rcIntlk=0
seed_with_stub "$INTLK" "$INTLK_BASE" "$root" "$TAG_INTLK" > "$scratch/logIntlk" 2>&1 || rcIntlk=$?
[ -e "$scratch/firedIntlk" ] \
|| fail "the stubbed cp never fired: the consumer never raced the publisher, so this scenario proves nothing"
ok "the consumer's clone raced a real, concurrent republish of its own source"
wait_for_file "$scratch/rcPubIntlk" "the publisher never finished"
[ "$(cat "$scratch/rcPubIntlk")" = "0" ] || { tail -40 "$scratch/logPubIntlk"; fail "publish-snapshot.sh exited non-zero"; }
[ "$rcIntlk" = "0" ] || { tail -40 "$scratch/logIntlk"; fail "the seed exited non-zero"; }
ok "both sides of the race completed"
# The property under test: not that the run succeeded, but that the publisher
# itself reports having found a live reader and waited on it. This is silent
# in the consumer's own log and in the run's exit status alike — only the
# publisher's log carries it.
grep -q "readers: waiting for 1 in-flight clone(s) of snapshot-${INTLK_BASE}" "$scratch/logPubIntlk" \
|| { tail -40 "$scratch/logPubIntlk"; fail "the publisher never reported waiting on the consumer's reader marker — the interlock did not observably engage"; }
ok "the publisher observably waited on the consumer's own reader marker before reclaiming the rotated snapshot generation"
leftovers=$(find "$root" -maxdepth 1 \( -name '.stage-*' -o -name '.reading-*' -o -name '.publish-*' \) -print)
[ -z "$leftovers" ] || fail "scratch left behind: ${leftovers}"
ok "no staging, reader-marker or deferred-generation scratch left behind"
rm -f "$scratch/bin/cp"
echo
echo "seed-target-dir-selftest: ${pass_count} assertions passed"