diff --git a/README.md b/README.md index 273fea4..4b55110 100644 --- a/README.md +++ b/README.md @@ -111,10 +111,14 @@ jobs: steps: - uses: actions/checkout@v4 with: - # REQUIRED. The mtime restore walks every commit that ever touched a - # tracked file; a depth-1 checkout makes every file resolve to the - # tip commit and the cache stops working. The action fails loudly - # rather than silently degrading if this is missing. + # REQUIRED, for two things. The mtime restore walks every commit + # that ever touched a tracked file; a depth-1 checkout makes every + # file resolve to the tip commit and the cache stops working, and + # the action fails loudly rather than silently degrading. The prune + # also decides whether a branch has been merged by asking this + # checkout for ancestry, which a shallow one cannot answer — there + # it withholds that half of the pass and says so, so merged-but- + # undeleted branches keep their caches. fetch-depth: 0 - uses: https://gitdan.com/daniel/gitdan-actions/cargo-cache@v1 @@ -263,14 +267,59 @@ not collapsed into one: disk, normally as the cache restored under its own name and otherwise as one set aside for a later pass to reclaim. -**Eviction** runs three passes: caches for branches that no longer exist on -origin are removed unconditionally; then, only if free space is under the -threshold, live caches are evicted oldest-first; then, as a last resort, this -run's own cache. Protected refs and any cache held open by a running job are -never candidates. Within the pressure pass, `target-*` directories are evicted -before `snapshot-*` ones — the reverse of the obvious order, because a -snapshot is hardlinked to everything cloned from it, so removing one frees -almost no real bytes while costing every future PR its warm start. +**Eviction** runs three passes, ahead of the seed so that what it frees is +available to the clone that follows. Caches for branches that are DEAD are +removed unconditionally; then, only if free space is under the requirement, +live caches are evicted oldest-first; then, as a last resort, this run's own +cache. Protected refs, the source this run is about to clone, and any cache +held open by a running job are never candidates. Within the pressure pass, +`target-*` directories are evicted before `snapshot-*` ones — the reverse of +the obvious order, because a snapshot is hardlinked to everything cloned from +it, so removing one frees almost no real bytes while costing every future PR +its warm start. + +**A branch is dead in two ways, and the second is the one that reclaims +anything here.** Gitea keeps a PR's branch after the merge unless the repo +opts into delete-on-merge, so `git ls-remote` reports merged branches forever +and "gone from origin" fires for almost nothing. The second signal is +ancestry: a branch still on origin whose tip is an ancestor of a protected +branch's tip holds no commit that branch does not, so its cache will never be +read again and goes in the same pass. On zemyna's volume that is the +difference between reclaiming nothing and reclaiming a 40 GB directory per +merged PR (gitdan-actions#20). + +Ancestry is answered from the commits in the job's own checkout, so it is only +answered where they are there to answer it — and "cannot tell" is never folded +into "dead", at either granularity. A shallow checkout withholds the signal +entirely, because a missing object is its normal case rather than evidence; a +single branch whose tip is not in the checkout is kept, with a warning naming +it. A squash or rebase merge leaves no ancestry at all, so its branch reads as +live until it is deleted. All three are missed reclamations, which cost disk; +the alternative direction costs a branch its cache while it is still being +built on. + +**How much free space is enough is measured, not chosen.** What the seed is +about to do is hardlink-clone a snapshot and then real-copy that clone's +*mutable set* — the dep-info, build-script metadata and linked outputs that a +build would otherwise write through a shared inode. The rest stays hardlinked +and costs nothing. So the requirement is derived per run, from that set, +measured off the very snapshot the seed will read using the same enumeration +`unshare_mutable_paths` copies from; the pass evicts oldest-first until it is +met and then stops. A percentage of the volume cannot express this: how much a +clone needs is a property of the snapshot, and a threshold sized for a +different failure is one that never fires before the seed refuses. `cp -al` +still materialises every directory for real, and the unshare stages each +subtree through a sibling copy, so the measurement carries a margin — +`CACHE_CLONE_HEADROOM_PERCENT` and `CACHE_CLONE_HEADROOM_FLOOR_KB`, both +hand-written defaults, both erring toward asking for more. + +A run that cannot reach the derived requirement after evicting everything +eligible **fails, naming the shortfall and every directory it kept instead**. +The seed would otherwise fail seconds later, reporting a staging path and +nothing about which cache was holding the space — which is the failure this +pass now pre-empts. `min-free-percent` is an additional floor on top and +nothing more: it defaults to `0`, it only ever raises the requirement, and +falling short of it is still a warning and a self-clear rather than a failure. **File mtimes.** `actions/checkout` stamps every file with "now", which makes every crate look changed to Cargo's mtime-based freshness check — a persistent @@ -412,10 +461,10 @@ directory; the line just doesn't say which. | `cache-root` | `/cache` | mount point of the persistent volume inside the job container | | `cache-lineage` | *(empty)* | one directory level under `cache-root`, for a second job building the same ref for a different target or profile — see [Multiple jobs in one workflow](#multiple-jobs-in-one-workflow) | | `protected-branches` | `dev main` | refs that publish snapshots and are never evicted | -| `min-free-percent` | `10` | prune when free space drops below this | +| `min-free-percent` | `0` | an ADDITIONAL free-space floor, as a percentage of the volume. The gate is derived per run from what the seed is about to clone; this only ever raises it | | `restore-mtimes` | `true` | restore tracked-file mtimes from git history | -| `prune` | `true` | run the eviction pass | -| `liveness-prune` | `true` | within eviction, remove caches for branches gone from origin | +| `prune` | `true` | run the eviction pass — before the seed, so what it frees is available to the clone | +| `liveness-prune` | `true` | within eviction, remove caches for branches that are dead: gone from origin, or merged into a protected branch | | `own-ref` | *(auto)* | override; defaults to `github.head_ref`, else `github.ref_name` | | `base-ref` | *(auto)* | override; defaults to `github.base_ref` (empty on push) | | `seed-fallback-dir` | *(empty)* | absolute path to seed from when no snapshot exists — for migrating off an existing flat cache | @@ -652,7 +701,7 @@ change here reaches all of them at once. That is what the gate is for. | `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler, **and a nightly that actually resolves freshness by content for its last scenario**: the source's-next-build check reasons about content rather than mtime, so under mtime freshness it would assert a bug. Whether the toolchain does is settled by experiment on a throwaway crate, not by asking it — accepting `-Z checksum-freshness` stopped implying it on 2026-08-22, when cargo PR #17382 demoted the flag to a gate and gave `build.fingerprint` (default `mtime`) the choice; the suite now exports both and 1.100.0-nightly measures ACTIVE again. The experiment reports **three** outcomes, not two: active, measured-inactive, and *not measured*. Its answer codes are `0` and `3`, deliberately clear of every status bash generates for its own errors — so nothing that goes wrong inside the probe, including an expansion failure no guard can catch, can be read as an answer. The scenario is skipped for the last two alike, but a failure to measure is never reported as a measurement. The control also reports which mutation families the running Cargo exhibits — a note, not an assertion, since that set moves upstream. It also pins the SELECTION itself against both of Cargo's build-dir layouts, from two file-only fixtures that need no compiler — so the layout the installed Cargo does not happen to write is still covered — and asserts the partition in both directions: every file Cargo rewrites in place is privately owned, and every `.rlib`/`.rmeta` still shares its inode. The second half is what the old suite never checked beyond `shared > 0`, and it is what a layout change silently inverts. | | `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published, and the publisher's log showing it waited on the consumer's own reader-lock marker before reclaiming a rotated snapshot | | `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone | -| `prune-cache-selftest.sh` | liveness, protection, locking, eviction order, self-clear, **and that a cache a job claims *inside* the check-to-unlink window survives it** — against a real scratch `origin` | +| `prune-cache-selftest.sh` | liveness in both its forms — a branch deleted from origin, and one still on it whose tip is already merged — plus protection, locking, eviction order, self-clear, **that a cache a job claims *inside* the check-to-unlink window survives it**, and that a requirement derived from the clone's mutable set evicts exactly enough and then fails rather than under-delivering. Against a real scratch `origin`, including a genuinely shallow clone of it and a `df` that answers from the fixture's own size, since a fixed one cannot show a pass stopping | | `restore-mtimes-selftest.sh` | the merge hazard and the watermark that closes it, including the two-jobs-one-namespace case. Needs a real compiler. | Every suite runs the actual script, not a reimplementation of its logic, and