3 Commits

Include renames

Author	SHA1	Message	Date
claudeandClaude Opus 5.5	ca0ee132d9	fix(ci): add a scheduled v1 sweep and lease every v1 push ... The release guard in 17d87b0 was safe but not live. Gitea 1.27.2 calls CancelPreviousJobsByJobConcurrency whenever a job's `needs` resolve (services/actions/clear_tasks.go:91, models/actions/run_job.go:641), so a job's place in the `release-tag-v1` group followed when its own selftest finished, not merge order. A newer merge C2 finishing selftest first queued behind the older C1, C1 cancelled it, saw tip = C2, and deferred: nobody pushed, and if merges then stopped v1 stayed stale indefinitely behind a Skipped and a Cancelled job. The "always catches up once merges pause" claim in ci.yaml and README was false. What now holds: - release-sweep.yaml runs on `schedule` every 15 minutes, in its own workflow and concurrency group, so nothing in ci.yaml can cancel it. When v1 already covers main's tip it stops after a checkout and one merge-base. Otherwise it checks out the tip, runs the same shellcheck and selftest.sh as ci.yaml's selftest job, and tags the tip only if they pass; a failing main therefore turns the sweep red on every tick while v1 lags, which is #27's AC1 loud-failure half. It reads the tip itself because a scheduled run's github.sha is the CommitSHA recorded when the schedule was registered on the last push to main (services/actions/notifier_helper.go:569-580, services/actions/schedule_tasks.go:126-141), and ref is the default branch: schedules are registered only from it (notifier_helper.go:120, :531, :603-604). event_name is "schedule" (context.go:71 reads TriggerEvent, set at schedule_tasks.go:136). Cron is 5-field robfig in UTC (models/actions/schedule_spec.go:38-41). - 15 minutes, not 10: the sweep is the fallback, not the release path, and every tick is a run on gitdan-ci's shared slots and a row in the Actions list. 96 no-op runs a day of a few seconds each is the cost; the lag bound it buys is one interval plus one selftest run. - Both writers go through scripts/release-v1.sh and push with --force-with-lease=refs/tags/v1:<v1 as read>, so v1 cannot move backwards when the sweep and a merge job race. A lost lease re-reads v1: at or ahead of this run's gated commit is a clean skip (the other writer released something at least as new); still behind it is a retry leased on the new value, up to three attempts, since the other writer may have tagged an older commit and giving up there would leave v1 short of a commit this run did gate; anything else goes red. A rejection with v1 unmoved is diagnosed as a non-lease failure and goes red at once. - release-tag loses its job-level concurrency group. The lease already gives the ordering the group was there for, and the group was what cancelled the one job that could have released the newest merge. Without it each merge's job runs, and the one whose commit is still the tip when it checks releases it. The shell moves out of ci.yaml into scripts/release-v1.sh so shellcheck and selftest.sh cover it. release-v1-selftest.sh runs it against a scratch bare origin: sweep no-op at and ahead of the tip, tag on a green gate, no tag and a failing sweep on a red one, the stranded trace above followed by a catching-up sweep, and each lost-lease outcome, with a control showing an unleased push does step v1 back. Red-proved by seven mutations of release-v1.sh, each failing a named assertion: plain --force, accepting any lost lease, a merge job that never defers, ancestry reduced to equality, no non-lease diagnosis, a sweep that never needs to run, and a retry that does not re-lease. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UkSjXXtU6JYcN2vPntWhfb	2026-09-22 15:01:05 -05:00
claude	fb7a788c90	feat(cache): give same-ref jobs separate build directories via cache-lineage ... A cache key names a REF. What a target directory holds is the product of a ref and a build configuration, and emowheel builds the same ref twice on every push — once for the host, once for wasm32, in two jobs that start together. Keyed on the ref alone, `cargo-cache@v1` handed both the same CARGO_TARGET_DIR, and Cargo's target-directory lock is exclusive: the second job sat on "Blocking waiting for file lock on build directory" for the length of the first while holding a runner capacity slot, so a third repo's queued job waited behind a job doing nothing. `cache-lineage` is that second dimension. It names ONE directory level under the cache root: <cache-root>/target-<key> no lineage (unchanged) <cache-root>/<lineage>/target-<key> a lineage Nesting, not a suffix on the key, and that is the whole design decision. `prune-cache.sh`'s liveness pass classifies a directory by recomputing `target-<cache_key(branch)>` for every branch on origin and evicting whatever does not match — a `target-<key>-wasm32` matches nothing, so it would be classified dead and evicted unconditionally on every run. daniel/gitdan's host-level arbiter reads the same shape (BRANCH_DIR_RE); a suffixed name falls out of that too, so those caches would never be reclaim candidates and a whole lineage would go missing from the shared disk budget. Nesting leaves both matchers reading exactly the names they already read, one level down — which is a layout that arbiter already walks (CI_CACHE_MAX_DEPTH is 2, and its own suite pins the depth-2 case). Every interacting part, checked rather than assumed: - SEED: `seed-target-dir.sh` takes the root as an argument, so a PR branch in a lineage layers over THAT lineage's base snapshot. Asserted. - PUBLISH: `publish-snapshot.sh` derives both ends of the swap from the root. The publish action now takes the root from the `CARGO_CACHE_ROOT` the consume step exported, and CHECKS its own inputs against it — a publish step left at the default while its consume step nested would otherwise republish a different lineage's live target dir over that lineage's snapshot, on every push, silently. `mode: release-lock` is exempt: it releases a lock on `$CARGO_TARGET_DIR` and never touches a root. - WATERMARK: per target dir, so it follows the lineage. Unchanged. - PRUNE and LIVENESS: scoped to the root they are given, so a pass in one lineage neither evicts nor sees a sibling's caches, or the flat layout's. Liveness keeps resolving real branch names, which is what a key suffix would have broken. - ci_cache_reclaim: verified by dry-run against a fixture in this layout — all six nested and flat dirs collected as candidates, protection resolved correctly on the nested ones, and a `.stage-` stranded inside the lineage found by the leftover sweep. Refused lineage names are refused at resolve time, each rejection naming the reader that imposes it: a path separator (the arbiter's depth budget), a Cargo profile name (its no-descend list), a `target-`/`snapshot-` prefix (this repo's own prune globs), a hex suffix (its per-branch-dir shape), a dot prefix (the leftover-naming contract). None of these fails visibly on its own — each produces a working directory that some pass silently stops seeing. Setting no lineage resolves to the cache root byte for byte, so lublub, zemyna and emowheel's `ci` job keep the exact directories they have on the volume. New suite `cache-root-selftest.sh` (19 assertions), red-proven against three deliberate breakages: a `cache_root_for` that ignores the lineage, a disabled validator, and a `verify` that never rejects a mismatch.	2026-08-26 12:32:21 -05:00
claudeandClaude Opus 5	248af3061e	feat(cargo-cache): hardlink-clone a per-ref Cargo cache from a published snapshot ... Replaces the phase-0 resolution probe with the real actions, merging the two independent per-branch Cargo cache implementations on this forge into the design neither of them had. ## The merge - zemyna seeds a PR branch by `cp -al` hardlink clone (near-free: cost scales with inode count, not bytes) from the base branch's LIVE target dir — a torn read waiting for a second job slot (its own #911). - emowheel seeds from a PUBLISHED IMMUTABLE SNAPSHOT (no race by construction) but with `cp -a`, duplicating ~35 GB per branch. This ships hardlink-clone FROM a published snapshot: zemyna's cost profile, emowheel's soundness, and #911 closed structurally rather than by the runner happening to have one execution slot. ## The bug both implementations have A build inside a `cp -al` clone DOES mutate the directory it was cloned from. Cargo replaces real artifacts, but writes its metadata — and build scripts write their OUT_DIR — with a plain truncating write, straight through the shared inode. Measured set: `.fingerprint/<unit>/dep-<target>` (under CARGO_UNSTABLE_CHECKSUM_FRESHNESS), `build/<pkg>/{output,root-output,out/**}`, `deps/*.d` and `<profile>/*.d`. The checksum-freshness case is a wrong answer, not a slow build: a PR clone rewrites the base's dep-info to describe the PR's sources while the base's cache still holds the artifact built from the base's; once the PR merges, the base's next run finds the checksums match, reports `Fresh`, and links a binary built from the pre-merge code. Reproduced end to end. Fix: hardlink the artifacts (the GB), real-copy the metadata (the MB) — about 3.7% of a 6.9 GB Bevy target dir, against 100% for a full copy. ## Contents - `cargo-cache/action.yml` — consume: resolve keys, seed from the base's snapshot via staging + one atomic rename, strip Cargo lock files, unshare the mutable paths, restore mtimes from git history, lock, prune. - `cargo-cache-publish/action.yml` — publish: record the build watermark, atomically republish the snapshot on a protected branch, release the lock (`mode: release-lock` for the `if: always()` step). - `scripts/` — all logic, so it is testable standalone; the YAML is wiring. - `scripts/*selftest.sh` + `selftest.sh` — five suites, 63 assertions, every fix paired with a control that reproduces the bug. All green locally. Eviction merges emowheel's liveness pass (dead branches pruned unconditionally, not gated on disk pressure) with LRU-under-pressure, but inverts the order within the pressure pass: `target-*` before `snapshot-*`, because a snapshot is hardlinked to everything cloned from it, so evicting one frees almost no real bytes while costing every future PR its warm start. restore-mtimes.sh is ported from emowheel (the watermark variant, which closes the merge hazard zemyna's copy still has) with its provenance de-projectised. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sqh2vscfzisk83VuPVQX9L	2026-08-23 14:18:01 -05:00