fix(cargo-cache): scope the safety claim to what the code actually prevents
Follow-up tof57e2a6, addressing the reviewer's sharpest question: is the race "closed by construction", or merely detected and retried? The honest answer is "both, on different paths", and the README said only the first half. * README now states three claims separately instead of collapsing them: a publisher rotating a snapshot cannot tear a clone of it (by construction — the marker ordering prevents the unlink, and the consumer's verification is a redundant second check on that path); every OTHER way the source can change mid-clone is detected, not prevented (the eviction pass's reader check is check-then-delete, and a seed-fallback-dir has no interlock at all — there, verification plus a bounded retry and a loud failure is the whole guard); and disk reclamation is bounded rather than immediate. Overclaiming this property once was the finding; overclaiming it twice would be worse. * publish-snapshot-selftest.sh now covers the PUBLISHER's half of the race, where a reader of the swap belongs. Scenario 3 only ever covered a consumer that had already FINISHED cloning — safe for free, since its own hardlinks keep the inodes alive. New scenario 6 covers a reader still in flight past the grace period (the generation is left on disk, the deferral is warned about, and an earlier consumer is still unaffected); scenario 7 covers the sweep, so "we defer instead of forcing" cannot quietly become a disk leak. Red-proven against 248af306's scripts: ASSERTION FAILED: the previous generation was unlinked while a reader still held it The consumer's half stays in seed-target-dir-selftest.sh scenario 8, which still red-proves at 16693 of 48805 entries against the same scripts. * seed-target-dir-selftest.sh now asserts what happens when the retries are EXHAUSTED, not just what hardlink_clone_into returns: an unreadable source makes the seed script exit non-zero, name the reason, leave no target dir, and — the one that matters — not fall through to its cold-start branch. A corrupt-cache bug degrading into an invisible 4x-slower CI job is the failure mode worth pinning down. Skipped when running as root, where mode bits deny nothing. * usage_kb: a directory we cannot read measured as the empty string, which was then spliced into usage_gb's awk program and made it a syntax error at the exact moment something was already going wrong. Now measures 0. Verification: `bash scripts/selftest.sh` — 5 suites, exit 0, 82 assertions (was 75 afterf57e2a6, 63 before). shellcheck over scripts/: no new findings. Measured the cost the reviewer asked about, on ext4, warm cache, 78,554 entries: `cp -al` 3126 ms against 44 ms for one `find | wc -l`. Two counts per attempt is ~2.8% on top of the clone. Not measured on the CI runner's volume. Refs: daniel/gitdan#11, zemyna#911 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sqh2vscfzisk83VuPVQX9L
This commit is contained in:
@@ -163,16 +163,32 @@ mechanisms, both required:
|
||||
and the clone retried; one that fails the last attempt fails the job. A
|
||||
partial tree never reaches the final name.
|
||||
|
||||
**What this does and does not guarantee.** *Correctness* is closed by
|
||||
construction: no combination of publish and seed timing produces a target
|
||||
directory holding part of one generation, and a source that cannot be read
|
||||
consistently fails the job loudly instead of seeding a truncated cache.
|
||||
*Disk reclamation* is bounded, not immediate: a consumer slower than the grace
|
||||
period leaves one extra snapshot generation of directory entries on the volume
|
||||
until the next publish of that snapshot sweeps it. That residual is capped at
|
||||
one deferred generation per publisher ref, and its real cost is close to the
|
||||
inode count rather than the byte count, since the artifacts are hardlinked to
|
||||
whatever cloned them.
|
||||
**What this does and does not guarantee.** Three separate claims, deliberately
|
||||
not collapsed into one:
|
||||
|
||||
- **A publisher rotating a snapshot cannot tear a clone of it — by
|
||||
construction.** This is the case zemyna #911 is about, and the marker
|
||||
ordering above is what closes it: the publisher's scan cannot miss a
|
||||
consumer that resolved the old generation, and on timeout it defers the
|
||||
unlink rather than forcing it. On this path the consumer's own verification
|
||||
is a redundant second check, not the thing holding the guarantee up.
|
||||
- **Every other way the source can change mid-clone is detected, not
|
||||
prevented.** The eviction pass's reader check is check-then-delete, so a
|
||||
consumer publishing its marker inside that gap is narrowed but not excluded
|
||||
— unreachable today only because snapshots belong to protected refs and
|
||||
protected refs are never eviction candidates, which is policy rather than
|
||||
structure. A `seed-fallback-dir` pointing at a directory something else
|
||||
writes has no interlock at all. There, the per-attempt verification is what
|
||||
stands between a torn read and a corrupt cache: the clone is retried
|
||||
(`CACHE_CLONE_ATTEMPTS`, default 4) and then **fails the job loudly** —
|
||||
never seeded partially, and never degraded to a silent cold build.
|
||||
- **Disk reclamation is bounded, not immediate.** A consumer slower than the
|
||||
grace period leaves one extra snapshot generation of directory entries on
|
||||
the volume until a later publish sweeps it; a consumer whose job was killed
|
||||
outright holds it until its marker passes `reader-stale-seconds`. The
|
||||
residual is capped at one deferred generation per publisher ref, and its
|
||||
real cost is close to inode count rather than byte count, since the
|
||||
artifacts are hardlinked to whatever cloned them.
|
||||
|
||||
**Eviction** runs three passes: caches for branches that no longer exist on
|
||||
origin are removed unconditionally; then, only if free space is under the
|
||||
@@ -289,7 +305,7 @@ bash scripts/selftest.sh --fast # fixture-only suites, no compiler
|
||||
|---|---|
|
||||
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler. |
|
||||
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and a seed racing a publisher's rotation of the source it is reading** — the race that actually truncates a tree |
|
||||
| `publish-snapshot-selftest.sh` | the atomic swap, and that a live consumer survives a republish |
|
||||
| `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone |
|
||||
| `prune-cache-selftest.sh` | liveness, protection, locking, eviction order, self-clear — against a real scratch `origin` |
|
||||
| `restore-mtimes-selftest.sh` | the merge hazard and the watermark that closes it, including the two-jobs-one-namespace case. Needs a real compiler. |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user