claudeandClaude Opus 5 248af3061e feat(cargo-cache): hardlink-clone a per-ref Cargo cache from a published snapshot
Replaces the phase-0 resolution probe with the real actions, merging the two
independent per-branch Cargo cache implementations on this forge into the
design neither of them had.

## The merge

- zemyna seeds a PR branch by `cp -al` hardlink clone (near-free: cost scales
  with inode count, not bytes) from the base branch's LIVE target dir — a
  torn read waiting for a second job slot (its own #911).
- emowheel seeds from a PUBLISHED IMMUTABLE SNAPSHOT (no race by
  construction) but with `cp -a`, duplicating ~35 GB per branch.

This ships hardlink-clone FROM a published snapshot: zemyna's cost profile,
emowheel's soundness, and #911 closed structurally rather than by the runner
happening to have one execution slot.

## The bug both implementations have

A build inside a `cp -al` clone DOES mutate the directory it was cloned from.
Cargo replaces real artifacts, but writes its metadata — and build scripts
write their OUT_DIR — with a plain truncating write, straight through the
shared inode. Measured set: `.fingerprint/<unit>/dep-<target>` (under
CARGO_UNSTABLE_CHECKSUM_FRESHNESS), `build/<pkg>/{output,root-output,out/**}`,
`deps/*.d` and `<profile>/*.d`.

The checksum-freshness case is a wrong answer, not a slow build: a PR clone
rewrites the base's dep-info to describe the PR's sources while the base's
cache still holds the artifact built from the base's; once the PR merges, the
base's next run finds the checksums match, reports `Fresh`, and links a binary
built from the pre-merge code. Reproduced end to end.

Fix: hardlink the artifacts (the GB), real-copy the metadata (the MB) — about
3.7% of a 6.9 GB Bevy target dir, against 100% for a full copy.

## Contents

- `cargo-cache/action.yml` — consume: resolve keys, seed from the base's
  snapshot via staging + one atomic rename, strip Cargo lock files, unshare
  the mutable paths, restore mtimes from git history, lock, prune.
- `cargo-cache-publish/action.yml` — publish: record the build watermark,
  atomically republish the snapshot on a protected branch, release the lock
  (`mode: release-lock` for the `if: always()` step).
- `scripts/` — all logic, so it is testable standalone; the YAML is wiring.
- `scripts/*selftest.sh` + `selftest.sh` — five suites, 63 assertions, every
  fix paired with a control that reproduces the bug. All green locally.

Eviction merges emowheel's liveness pass (dead branches pruned
unconditionally, not gated on disk pressure) with LRU-under-pressure, but
inverts the order within the pressure pass: `target-*` before `snapshot-*`,
because a snapshot is hardlinked to everything cloned from it, so evicting one
frees almost no real bytes while costing every future PR its warm start.

restore-mtimes.sh is ported from emowheel (the watermark variant, which closes
the merge hazard zemyna's copy still has) with its provenance de-projectised.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sqh2vscfzisk83VuPVQX9L
2026-08-23 14:18:01 -05:00

gitdan-actions

Shared Gitea Actions composite actions for the gitdan forge.

Currently one thing, done properly: cargo-cache — a persistent, per-branch Cargo build cache for self-hosted Gitea runners, where a pull request's cache is a near-free hardlink clone of an immutable snapshot its base branch published.

Final home: this repository will live at daniel/gitdan-actions. Pin that path in uses: once the transfer completes.


Why this exists

Two of this forge's Rust projects independently built the same idea and each got one half right.

seeding mechanism seed source
project A cp -al hardlink clone — near-free, cost scales with inode count, not bytes the base branch's live target dir — races a build that is still writing
project B cp -a full copy — sound, but ~35 GB duplicated per branch a published immutable snapshot — nothing ever writes it while it is read

This action is the diagonal: hardlink-clone from a published snapshot. Cheap like A, sound like B. It also closes a latent race in A by construction (the seed is staged and swapped in with one atomic rename) rather than relying on the runner having a single execution slot.

One thing neither project had, and the reason the clone is not a plain cp -al: a build inside a hardlink clone does mutate the directory it was cloned from. Cargo writes real artifacts by replacing them, but writes its metadata — and build scripts write their OUT_DIR — with a plain truncating write, straight through the shared inode. Under CARGO_UNSTABLE_CHECKSUM_FRESHNESS the file that gets corrupted is .fingerprint/<unit>/dep-<target>, which holds the per-source checksums that decide freshness, and the failure is silent stale-artifact reuse rather than a slow build. scripts/hardlink-clone-selftest.sh reproduces it as an explicit control and asserts the fix. The fix is to hardlink the artifacts (the GB) and real-copy the metadata (the MB) — about 3.7% of a Bevy-sized target directory, against 100% for a full copy.


Quick start

name: CI
on:
  push:
    branches: [main, dev]
  pull_request:
    branches: [main, dev]

jobs:
  ci:
    runs-on: ubuntu-latest
    # REQUIRED, and it cannot come from the action: `container.volumes` is a
    # job-level property, so the persistent cache volume must be declared
    # here. Use a volume name unique to this repository.
    container:
      volumes:
        - myrepo-ci-target:/cache
    steps:
      - uses: actions/checkout@v4
        with:
          # REQUIRED. The mtime restore walks every commit that ever touched a
          # tracked file; a depth-1 checkout makes every file resolve to the
          # tip commit and the cache stops working. The action fails loudly
          # rather than silently degrading if this is missing.
          fetch-depth: 0

      - uses: https://gitdan.com/daniel/gitdan-actions/cargo-cache@v1
        with:
          protected-branches: 'dev main'

      # ... toolchain, system deps, and the build itself. CARGO_TARGET_DIR is
      # already exported to the job environment by the step above.
      - run: cargo clippy --workspace --all-targets -- -D warnings
      - run: cargo test --workspace

      # After the build succeeds: record the watermark, and publish a snapshot
      # if this run is a push to a protected branch.
      - uses: https://gitdan.com/daniel/gitdan-actions/cargo-cache-publish@v1

      # Release this job's cache lock even when the build failed, so the
      # eviction pass does not have to wait out the staleness grace period.
      - if: always()
        uses: https://gitdan.com/daniel/gitdan-actions/cargo-cache-publish@v1
        with:
          mode: release-lock

Recommended alongside it, in the workflow's env: block:

env:
  CARGO_INCREMENTAL: 0                  # per-run bloat on a persistent volume
  CARGO_PROFILE_DEV_DEBUG: line-tables-only
  CARGO_PROFILE_TEST_DEBUG: line-tables-only
  # Nightly only. Content-addressed freshness instead of mtime-based — a
  # strictly stronger guarantee, complementary to the mtime restore (which
  # still covers directory-form `rerun-if-changed` build-script watches).
  CARGO_UNSTABLE_CHECKSUM_FRESHNESS: "true"

How it works

/cache/
  target-<key>       one per ref. Where a build actually runs.
  snapshot-<key>     one per publisher ref. Immutable between publishes;
                     the only thing a consumer ever clones from.

<key> is the ref sanitised to a safe path component, capped at 48 characters, plus an 8-hex SHA-1 prefix of the raw ref. The hash is not decoration: feat/foo and feat-foo sanitise identically and would otherwise share one directory.

A pull request run resolves its own key from github.head_ref (not ref_name, which on a pull_request event is a synthetic merge ref that changes on every push) and its base key from github.base_ref. If it has no directory yet, it hardlink-clones snapshot-<base> into a staging path, strips Cargo's lock files, real-copies everything Cargo writes in place, and renames the staging path into target-<own>.

A push to a protected branch has no base to layer over. It builds in its own directory and, if the build goes green, republishes it as snapshot-<own>: stage a clone, rename the old snapshot aside, rename the new one in. Consumers only ever observe a complete snapshot or none at all.

Concurrency. Two jobs sharing one cache key each stage under their own tag and race on one atomic rename; the loser discards its staging copy. There is no window in which a partially-populated directory is visible under the final name — which means this does not depend on the runner having a single execution slot. Two jobs then building in the same directory is Cargo's own .cargo-lock territory, which is what that lock is for.

Eviction runs three passes: caches for branches that no longer exist on origin are removed unconditionally; then, only if free space is under the threshold, live caches are evicted oldest-first; then, as a last resort, this run's own cache. Protected refs and any cache held open by a running job are never candidates. Within the pressure pass, target-* directories are evicted before snapshot-* ones — the reverse of the obvious order, because a snapshot is hardlinked to everything cloned from it, so removing one frees almost no real bytes while costing every future PR its warm start.

File mtimes. actions/checkout stamps every file with "now", which makes every crate look changed to Cargo's mtime-based freshness check — a persistent target directory buys nothing without fixing that. Each tracked file is restored to the timestamp of the most recent commit that touched it, plus a watermark override: for any file that changed since this cache's own last successful build, "now" is stamped instead. That override is what makes a merge safe, since a merge can introduce a commit authored before this cache's last build, where the historically-correct mtime is exactly the wrong answer.


Inputs

cargo-cache

input default meaning
cache-root /cache mount point of the persistent volume inside the job container
protected-branches dev main refs that publish snapshots and are never evicted
min-free-percent 10 prune when free space drops below this
restore-mtimes true restore tracked-file mtimes from git history
prune true run the eviction pass
liveness-prune true within eviction, remove caches for branches gone from origin
own-ref (auto) override; defaults to github.head_ref, else github.ref_name
base-ref (auto) override; defaults to github.base_ref (empty on push)
seed-fallback-dir (empty) absolute path to seed from when no snapshot exists — for migrating off an existing flat cache
watermark-file .ci-watermark-<job>-sha must differ per job when two jobs share one cache key
lock-id <job>-<run_id> identifies this job's cache lock
stale-lock-seconds 7200 age past which another job's lock is treated as abandoned

Outputs: target-dir, cache-key, seeded-from (own | base-snapshot | own-snapshot | fallback-dir | concurrent-peer | cold).

Exports to the job environment: CARGO_TARGET_DIR, CARGO_CACHE_ROOT, CARGO_CACHE_KEY, CARGO_CACHE_LOCK_ID, CARGO_CACHE_SCRIPTS, CI_WATERMARK_FILE.

cargo-cache-publish

input default meaning
cache-root /cache must match the consume action
protected-branches dev main refs that publish snapshots
mode publish publish, or release-lock for the if: always() step
own-ref (auto) override; defaults to github.head_ref, else github.ref_name
publish-on-events push events on which a protected ref actually publishes
record-watermark true record HEAD as this cache's watermark (PR runs too)

publish-on-events defaults to push on purpose: a pull_request run from dev into main has own-ref dev and would otherwise publish a snapshot of a merge-preview build, which is not what dev is.


Multiple jobs in one workflow

Jobs sharing a cache key (a ci job and a wasm job on the same branch, say) each need their own watermark file. A shared one breaks the moment two jobs run in sequence within one trigger: job A advances the watermark to HEAD, and job B then reads that just-advanced value, computes an empty diff, and loses the merge protection entirely. The default (.ci-watermark-<job>-sha) already gives each job its own; only override watermark-file if you also override lock-id, and then keep both distinct per job.


Constraints of this runner

  • The repository must be public. act_runner fetches actions by anonymous git clone and has no credentialed-fetch option, so a private action repository simply fails to resolve. Nothing secret goes in here.
  • uses: needs the absolute URL. A bare owner/repo resolves against github.com, because Gitea's DEFAULT_ACTIONS_URL is unset — and it has to stay unset, or actions/checkout, dtolnay/rust-toolchain and taiki-e/install-action stop resolving.
  • The runner pre-fetches every referenced action before running any step, so a bad action reference fails the job at step 0 rather than where it is used.
  • container.volumes is job-level and cannot be set from inside a composite action. The consuming workflow declares it; see the quick start.
  • The cache volume is ext4 — no reflink support, which is precisely why hardlinks are the mechanism that makes cloning cheap.

Versioning

Pin @v1. It is a moving major tag: fixes and backward-compatible inputs move it forward, and anything that would break an existing consumer gets v2 instead. Pin a commit SHA if you want a frozen version.


Development

bash scripts/selftest.sh          # everything (~1 min; needs cargo)
bash scripts/selftest.sh --fast   # fixture-only suites, no compiler
suite covers
hardlink-clone-selftest.sh that a build in a clone cannot mutate its source — with a control proving a raw cp -al does. Needs a real compiler.
seed-target-dir-selftest.sh seed-source preference, lock-file stripping, and two jobs racing on one cache key
publish-snapshot-selftest.sh the atomic swap, and that a live consumer survives a republish
prune-cache-selftest.sh liveness, protection, locking, eviction order, self-clear — against a real scratch origin
restore-mtimes-selftest.sh the merge hazard and the watermark that closes it, including the two-jobs-one-namespace case. Needs a real compiler.

Every suite runs the actual script, not a reimplementation of its logic, and every fix scenario is paired with a control that reproduces the bug — a scenario that passes either way proves nothing.

The action YAML holds no logic beyond wiring; everything testable lives in scripts/. A composite action needs shell: bash on every run: step, and the actions reach their shared scripts through ${{ github.action_path }}/../scripts, which works because the runner clones the whole repository when it fetches an action.

S
Description
Shared Gitea Actions composite actions for gitdan repos (cargo-cache et al)
Readme
1.2 MiB
Languages
Shell 100%