Files
gitdan-actions/cargo-cache-publish/action.yml
T
claudeandClaude Opus 5 248af3061e feat(cargo-cache): hardlink-clone a per-ref Cargo cache from a published snapshot
Replaces the phase-0 resolution probe with the real actions, merging the two
independent per-branch Cargo cache implementations on this forge into the
design neither of them had.

## The merge

- zemyna seeds a PR branch by `cp -al` hardlink clone (near-free: cost scales
  with inode count, not bytes) from the base branch's LIVE target dir — a
  torn read waiting for a second job slot (its own #911).
- emowheel seeds from a PUBLISHED IMMUTABLE SNAPSHOT (no race by
  construction) but with `cp -a`, duplicating ~35 GB per branch.

This ships hardlink-clone FROM a published snapshot: zemyna's cost profile,
emowheel's soundness, and #911 closed structurally rather than by the runner
happening to have one execution slot.

## The bug both implementations have

A build inside a `cp -al` clone DOES mutate the directory it was cloned from.
Cargo replaces real artifacts, but writes its metadata — and build scripts
write their OUT_DIR — with a plain truncating write, straight through the
shared inode. Measured set: `.fingerprint/<unit>/dep-<target>` (under
CARGO_UNSTABLE_CHECKSUM_FRESHNESS), `build/<pkg>/{output,root-output,out/**}`,
`deps/*.d` and `<profile>/*.d`.

The checksum-freshness case is a wrong answer, not a slow build: a PR clone
rewrites the base's dep-info to describe the PR's sources while the base's
cache still holds the artifact built from the base's; once the PR merges, the
base's next run finds the checksums match, reports `Fresh`, and links a binary
built from the pre-merge code. Reproduced end to end.

Fix: hardlink the artifacts (the GB), real-copy the metadata (the MB) — about
3.7% of a 6.9 GB Bevy target dir, against 100% for a full copy.

## Contents

- `cargo-cache/action.yml` — consume: resolve keys, seed from the base's
  snapshot via staging + one atomic rename, strip Cargo lock files, unshare
  the mutable paths, restore mtimes from git history, lock, prune.
- `cargo-cache-publish/action.yml` — publish: record the build watermark,
  atomically republish the snapshot on a protected branch, release the lock
  (`mode: release-lock` for the `if: always()` step).
- `scripts/` — all logic, so it is testable standalone; the YAML is wiring.
- `scripts/*selftest.sh` + `selftest.sh` — five suites, 63 assertions, every
  fix paired with a control that reproduces the bug. All green locally.

Eviction merges emowheel's liveness pass (dead branches pruned
unconditionally, not gated on disk pressure) with LRU-under-pressure, but
inverts the order within the pressure pass: `target-*` before `snapshot-*`,
because a snapshot is hardlinked to everything cloned from it, so evicting one
frees almost no real bytes while costing every future PR its warm start.

restore-mtimes.sh is ported from emowheel (the watermark variant, which closes
the merge hazard zemyna's copy still has) with its provenance de-projectised.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sqh2vscfzisk83VuPVQX9L
2026-08-23 14:18:01 -05:00

132 lines
5.8 KiB
YAML

name: 'Cargo cache (publish)'
description: >-
Records this run''s build watermark and, on a publisher branch, atomically
republishes its target directory as the immutable snapshot that other
branches'' caches are hardlink-cloned from.
author: 'gitdan'
inputs:
cache-root:
description: 'Mount point of the persistent cache volume. Must match the consume action.'
required: false
default: '/cache'
protected-branches:
description: >-
Space-separated refs that publish snapshots. A run whose own ref is not
in this list records its watermark and skips publishing.
required: false
default: 'dev main'
mode:
description: >-
publish — record the watermark, publish a snapshot if eligible,
release this job''s cache lock (the normal call, after a
green build).
release-lock — release this job''s cache lock and do nothing else. Use
in a final `if: always()` step so a failed run does not
leave a lock behind for the staleness grace period.
required: false
default: 'publish'
own-ref:
description: 'Override this run''s ref. Defaults to github.head_ref, else github.ref_name.'
required: false
default: ''
publish-on-events:
description: >-
Space-separated event names on which a publisher branch actually
publishes. Defaults to `push` — a pull_request run never publishes,
because its ref is not the reference branch even when it targets one.
required: false
default: 'push'
record-watermark:
description: >-
Record this run''s HEAD as the build watermark for this target dir.
True for PR runs too, not just publishers: a feature branch accumulates
its own build history across several pushes and needs its own watermark.
required: false
default: 'true'
runs:
using: 'composite'
steps:
# Everything here reads the environment the consume action exported, so a
# workflow that forgets to run cargo-cache first fails loudly here rather
# than silently publishing a snapshot of the wrong directory.
- id: resolve
shell: bash
env:
PROTECTED_BRANCHES: ${{ inputs.protected-branches }}
PUBLISH_ON_EVENTS: ${{ inputs.publish-on-events }}
run: |
set -euo pipefail
# In release-lock mode this action is called from an `if: always()`
# step, which can run after a failure that happened before the
# cargo-cache action ever executed. Missing environment there means
# "there is no lock to release", not an error worth failing the job a
# second time over.
if [ -z "${CARGO_CACHE_SCRIPTS:-}" ] || [ -z "${CARGO_TARGET_DIR:-}" ]; then
if [ "${{ inputs.mode }}" = "release-lock" ]; then
echo "cargo-cache-publish: no cache environment in this job — nothing to release"
echo "publish=no" >> "$GITHUB_OUTPUT"
echo "active=no" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "::error::cargo-cache-publish: run the cargo-cache action earlier in this job"
exit 1
fi
: "${CARGO_CACHE_KEY:?cargo-cache-publish: CARGO_CACHE_KEY not set by the cargo-cache action}"
echo "active=yes" >> "$GITHUB_OUTPUT"
OWN_REF="${{ inputs.own-ref }}"
[ -n "$OWN_REF" ] || OWN_REF="${{ github.head_ref || github.ref_name }}"
# A publisher is a branch other branches layer over. Two conditions,
# both required: its ref is in protected-branches, AND the event is one
# where this ref really is the reference branch. A pull_request run
# from `dev` into `main` has own-ref `dev` and would otherwise publish
# a snapshot of a merge-preview build — which is not what `dev` is.
PUBLISH=no
for ref in $PROTECTED_BRANCHES; do
[ "$ref" = "$OWN_REF" ] || continue
for ev in $PUBLISH_ON_EVENTS; do
[ "$ev" = "${{ github.event_name }}" ] && PUBLISH=yes
done
done
echo "publish=${PUBLISH}" >> "$GITHUB_OUTPUT"
echo "own-ref=${OWN_REF}" >> "$GITHUB_OUTPUT"
echo "publisher check: ref '${OWN_REF}', event '${{ github.event_name }}' -> publish=${PUBLISH}"
# Ordered before the snapshot publish so a snapshot always carries a
# watermark at least as new as the build it holds. Both steps sit after
# the consuming job's build steps, so a run that fails an earlier gate
# never reaches either: the watermark stays at the last GREEN build and a
# red build can never overwrite a known-good snapshot.
- if: ${{ steps.resolve.outputs.active == 'yes' && inputs.mode == 'publish' && inputs.record-watermark == 'true' }}
shell: bash
run: |
set -euo pipefail
bash "${CARGO_CACHE_SCRIPTS}/record-watermark.sh" \
"$CARGO_TARGET_DIR" "${CI_WATERMARK_FILE:-.ci-watermark-sha}"
- if: ${{ inputs.mode == 'publish' && steps.resolve.outputs.publish == 'yes' }}
shell: bash
run: |
set -euo pipefail
bash "${CARGO_CACHE_SCRIPTS}/publish-snapshot.sh" \
"$CARGO_CACHE_KEY" "${{ inputs.cache-root }}" \
"${{ github.job }}-${{ github.run_id }}-$$"
# Released in both modes. In `publish` mode this is the normal end-of-job
# release; the separate `release-lock` call exists for `if: always()`, so a
# failed run does not leave its lock sitting until the staleness grace
# period expires.
- if: ${{ steps.resolve.outputs.active == 'yes' }}
shell: bash
run: |
set -euo pipefail
if [ -z "${CARGO_CACHE_LOCK_ID:-}" ]; then
echo "cargo-cache-publish: no lock id in the environment — nothing to release"
exit 0
fi
bash "${CARGO_CACHE_SCRIPTS}/cache-lock.sh" release \
"$CARGO_TARGET_DIR" "${CARGO_CACHE_LOCK_ID}"