Files
gitdan-actions/cargo-cache-publish/action.yml
claudeandClaude Opus 5 f57e2a6013 fix(cargo-cache): close the seed-vs-republish race the design claimed to close
The shared action's justification over zemyna's and emowheel's schemes was
that hardlink-cloning from a published snapshot closes gitdan #911 "by
construction, not by the single job slot". Review disproved that. This makes
the claim true, and corrects the README where it could only be bounded.

Finding 1 (verdict-level) — silent partial clone
------------------------------------------------
`hardlink_clone_into` ran `cp -al` with no exit-status check, and both call
sites invoked it as a condition, which suppresses `set -e` for the whole call.
A publisher's `rm -rf` of the generation it rotated away therefore unlinked
entries beneath an in-flight consumer walk, and the truncated tree was renamed
into place and reported as success.

Both layers are fixed:

* The consumer verifies its own clone. Every attempt checks `cp -al`'s status
  explicitly, the source directory's inode before and after (a wholesale
  replacement mid-walk splices two generations), and the entry count — the
  only signal for a subtree unlinked before its parent was listed, since
  `cp -al` reports no error for one it never saw. Any failure discards the
  staging tree and retries; exhausting the attempts returns a distinct status
  2 and fails the job rather than seeding a partial cache. `unshare_subtree` /
  `_unshare_files` now propagate failure too — a swallowed unshare leaves the
  clone aliasing its source, the exact corruption that step exists to prevent.
* The publisher does not unlink under a reader. A consumer publishes a
  `.reading-<snapshot>-<tag>` marker before it resolves the snapshot path; the
  publisher scans for markers after its first rename. A consumer holding the
  old generation therefore published its marker before that scan and cannot be
  missed; one arriving after the scan necessarily resolves to the new
  generation. The publisher waits for readers to drain and, on timeout,
  DEFERS reclamation rather than forcing it — the old generation is left as
  `.publish-old-<key>-<tag>` and swept by a later publish.

So correctness is closed by construction; disk reclamation is bounded, not
immediate. The residual is capped at one deferred generation per publisher
ref, and the README now says exactly that instead of the disproved claim.

Finding 2 — restore-mtimes.sh ran with no errexit
-------------------------------------------------
`set -euo pipefail` was glued to the end of a comment (`# soundness.set -euo
pipefail`), so it was entirely commented out: a partial failure of the
`git log | awk` pipeline would have produced wrong mtimes across the whole
restore instead of failing loudly. Moved to its own line. Audited every other
script for the same defect — this was the only instance. Independent
confirmation: shellcheck's two SC2164 warnings on this file's `cd "$repo_root"`
disappear now that errexit is actually in effect.

Finding 3 — lock-acquire window
-------------------------------
A just-seeded directory was unlocked until a later action step, so a
concurrent job's prune pass could evict it. `seed-target-dir.sh` now takes an
optional lock-id and writes the lock marker on every path out of the script,
including into the staging tree before its rename, so the directory carries a
lock the instant it appears under its final name. The action's acquire step
stays (it is idempotent and stamps the LRU marker).

Also hardened `prune-cache.sh` to treat a directory with live reader markers
as locked. Today no reachable configuration prunes a snapshot — only protected
refs publish them and protected refs are excluded from every pass — so this is
redundant by policy; it is here so that stops being the reason it is safe.

Verification
------------
New selftest scenario 8 races a real seed against a real publish rotation,
gating the rotation on the seed's *observed* clone progress so the window is
hit deterministically rather than on a fast machine's coin flip. Red-proven
against the unguarded scripts, three consecutive runs:

  ASSERTION FAILED: the seeded tree is truncated: 15443 entries against the
  snapshot's 493 (was 48805 before the rotation)      (15443 / 16986 / 16498)

Green after the fix, six consecutive runs, catching the clone mid-walk at
~10.5k of 48805 entries each time. Scenario 9 covers deferred reclamation and
its later sweep; scenario 10 covers an unreadable source failing loudly.

`bash scripts/selftest.sh`: 5 suites, exit 0, 75 assertions (was 63).
shellcheck over `scripts/`: no new findings, two SC2164 warnings resolved.

Docs: README's republish-safety paragraph replaced with what the code now
guarantees, including the bounded disk residual stated explicitly; new
`read-grace-seconds` / `reader-stale-seconds` inputs documented in the
`cargo-cache-publish` table; the selftest table names the new race.

Refs: daniel/gitdan#11, zemyna#911

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sqh2vscfzisk83VuPVQX9L
2026-08-23 16:40:21 -05:00

151 lines
6.6 KiB
YAML

name: 'Cargo cache (publish)'
description: >-
Records this run's build watermark and, on a publisher branch, atomically
republishes its target directory as the immutable snapshot that other
branches' caches are hardlink-cloned from.
author: 'gitdan'
inputs:
cache-root:
description: 'Mount point of the persistent cache volume. Must match the consume action.'
required: false
default: '/cache'
protected-branches:
description: >-
Space-separated refs that publish snapshots. A run whose own ref is not
in this list records its watermark and skips publishing.
required: false
default: 'dev main'
mode:
description: >-
publish — record the watermark, publish a snapshot if eligible,
release this job's cache lock (the normal call, after a
green build).
release-lock — release this job's cache lock and do nothing else. Use
in a final `if: always()` step so a failed run does not
leave a lock behind for the staleness grace period.
required: false
default: 'publish'
own-ref:
description: 'Override this run''s ref. Defaults to github.head_ref, else github.ref_name.'
required: false
default: ''
publish-on-events:
description: >-
Space-separated event names on which a publisher branch actually
publishes. Defaults to `push` — a pull_request run never publishes,
because its ref is not the reference branch even when it targets one.
required: false
default: 'push'
read-grace-seconds:
description: >-
How long the snapshot swap waits for in-flight consumers to finish
cloning the generation it is replacing before reclaiming it. On timeout
the old generation is LEFT ON DISK and swept by a later publish — the
unlink is never forced, because unlinking a tree a consumer is walking
is what silently truncates that consumer's clone.
required: false
default: '300'
reader-stale-seconds:
description: >-
Age past which a consumer's read marker is treated as abandoned by a
job the runner killed. Without it one crashed job would pin a snapshot
generation on disk permanently.
required: false
default: '7200'
record-watermark:
description: >-
Record this run's HEAD as the build watermark for this target dir.
True for PR runs too, not just publishers: a feature branch accumulates
its own build history across several pushes and needs its own watermark.
required: false
default: 'true'
runs:
using: 'composite'
steps:
# Everything here reads the environment the consume action exported, so a
# workflow that forgets to run cargo-cache first fails loudly here rather
# than silently publishing a snapshot of the wrong directory.
- id: resolve
shell: bash
env:
PROTECTED_BRANCHES: ${{ inputs.protected-branches }}
PUBLISH_ON_EVENTS: ${{ inputs.publish-on-events }}
run: |
set -euo pipefail
# In release-lock mode this action is called from an `if: always()`
# step, which can run after a failure that happened before the
# cargo-cache action ever executed. Missing environment there means
# "there is no lock to release", not an error worth failing the job a
# second time over.
if [ -z "${CARGO_CACHE_SCRIPTS:-}" ] || [ -z "${CARGO_TARGET_DIR:-}" ]; then
if [ "${{ inputs.mode }}" = "release-lock" ]; then
echo "cargo-cache-publish: no cache environment in this job — nothing to release"
echo "publish=no" >> "$GITHUB_OUTPUT"
echo "active=no" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "::error::cargo-cache-publish: run the cargo-cache action earlier in this job"
exit 1
fi
: "${CARGO_CACHE_KEY:?cargo-cache-publish: CARGO_CACHE_KEY not set by the cargo-cache action}"
echo "active=yes" >> "$GITHUB_OUTPUT"
OWN_REF="${{ inputs.own-ref }}"
[ -n "$OWN_REF" ] || OWN_REF="${{ github.head_ref || github.ref_name }}"
# A publisher is a branch other branches layer over. Two conditions,
# both required: its ref is in protected-branches, AND the event is one
# where this ref really is the reference branch. A pull_request run
# from `dev` into `main` has own-ref `dev` and would otherwise publish
# a snapshot of a merge-preview build — which is not what `dev` is.
PUBLISH=no
for ref in $PROTECTED_BRANCHES; do
[ "$ref" = "$OWN_REF" ] || continue
for ev in $PUBLISH_ON_EVENTS; do
[ "$ev" = "${{ github.event_name }}" ] && PUBLISH=yes
done
done
echo "publish=${PUBLISH}" >> "$GITHUB_OUTPUT"
echo "own-ref=${OWN_REF}" >> "$GITHUB_OUTPUT"
echo "publisher check: ref '${OWN_REF}', event '${{ github.event_name }}' -> publish=${PUBLISH}"
# Ordered before the snapshot publish so a snapshot always carries a
# watermark at least as new as the build it holds. Both steps sit after
# the consuming job's build steps, so a run that fails an earlier gate
# never reaches either: the watermark stays at the last GREEN build and a
# red build can never overwrite a known-good snapshot.
- if: ${{ steps.resolve.outputs.active == 'yes' && inputs.mode == 'publish' && inputs.record-watermark == 'true' }}
shell: bash
run: |
set -euo pipefail
bash "${CARGO_CACHE_SCRIPTS}/record-watermark.sh" \
"$CARGO_TARGET_DIR" "${CI_WATERMARK_FILE:-.ci-watermark-sha}"
- if: ${{ inputs.mode == 'publish' && steps.resolve.outputs.publish == 'yes' }}
shell: bash
env:
CACHE_READ_GRACE_SECONDS: ${{ inputs.read-grace-seconds }}
CACHE_READ_STALE_SECONDS: ${{ inputs.reader-stale-seconds }}
run: |
set -euo pipefail
bash "${CARGO_CACHE_SCRIPTS}/publish-snapshot.sh" \
"$CARGO_CACHE_KEY" "${{ inputs.cache-root }}" \
"${{ github.job }}-${{ github.run_id }}-$$"
# Released in both modes. In `publish` mode this is the normal end-of-job
# release; the separate `release-lock` call exists for `if: always()`, so a
# failed run does not leave its lock sitting until the staleness grace
# period expires.
- if: ${{ steps.resolve.outputs.active == 'yes' }}
shell: bash
run: |
set -euo pipefail
if [ -z "${CARGO_CACHE_LOCK_ID:-}" ]; then
echo "cargo-cache-publish: no lock id in the environment — nothing to release"
exit 0
fi
bash "${CARGO_CACHE_SCRIPTS}/cache-lock.sh" release \
"$CARGO_TARGET_DIR" "${CARGO_CACHE_LOCK_ID}"