feat(cache): give same-ref jobs separate build directories via cache-lineage
CI / shellcheck + selftests (pull_request) Failing after 1m19s

A cache key names a REF. What a target directory holds is the product of a ref
and a build configuration, and emowheel builds the same ref twice on every
push — once for the host, once for wasm32, in two jobs that start together.
Keyed on the ref alone, `cargo-cache@v1` handed both the same
CARGO_TARGET_DIR, and Cargo's target-directory lock is exclusive: the second
job sat on "Blocking waiting for file lock on build directory" for the length
of the first while holding a runner capacity slot, so a third repo's queued
job waited behind a job doing nothing.

`cache-lineage` is that second dimension. It names ONE directory level under
the cache root:

    <cache-root>/target-<key>              no lineage (unchanged)
    <cache-root>/<lineage>/target-<key>    a lineage

Nesting, not a suffix on the key, and that is the whole design decision.
`prune-cache.sh`'s liveness pass classifies a directory by recomputing
`target-<cache_key(branch)>` for every branch on origin and evicting whatever
does not match — a `target-<key>-wasm32` matches nothing, so it would be
classified dead and evicted unconditionally on every run. daniel/gitdan's
host-level arbiter reads the same shape (BRANCH_DIR_RE); a suffixed name falls
out of that too, so those caches would never be reclaim candidates and a whole
lineage would go missing from the shared disk budget. Nesting leaves both
matchers reading exactly the names they already read, one level down — which
is a layout that arbiter already walks (CI_CACHE_MAX_DEPTH is 2, and its own
suite pins the depth-2 case).

Every interacting part, checked rather than assumed:

- SEED: `seed-target-dir.sh` takes the root as an argument, so a PR branch in
  a lineage layers over THAT lineage's base snapshot. Asserted.
- PUBLISH: `publish-snapshot.sh` derives both ends of the swap from the root.
  The publish action now takes the root from the `CARGO_CACHE_ROOT` the
  consume step exported, and CHECKS its own inputs against it — a publish step
  left at the default while its consume step nested would otherwise republish
  a different lineage's live target dir over that lineage's snapshot, on every
  push, silently. `mode: release-lock` is exempt: it releases a lock on
  `$CARGO_TARGET_DIR` and never touches a root.
- WATERMARK: per target dir, so it follows the lineage. Unchanged.
- PRUNE and LIVENESS: scoped to the root they are given, so a pass in one
  lineage neither evicts nor sees a sibling's caches, or the flat layout's.
  Liveness keeps resolving real branch names, which is what a key suffix would
  have broken.
- ci_cache_reclaim: verified by dry-run against a fixture in this layout —
  all six nested and flat dirs collected as candidates, protection resolved
  correctly on the nested ones, and a `.stage-` stranded inside the lineage
  found by the leftover sweep.

Refused lineage names are refused at resolve time, each rejection naming the
reader that imposes it: a path separator (the arbiter's depth budget), a
Cargo profile name (its no-descend list), a `target-`/`snapshot-` prefix (this
repo's own prune globs), a hex suffix (its per-branch-dir shape), a dot prefix
(the leftover-naming contract). None of these fails visibly on its own — each
produces a working directory that some pass silently stops seeing.

Setting no lineage resolves to the cache root byte for byte, so lublub, zemyna
and emowheel's `ci` job keep the exact directories they have on the volume.

New suite `cache-root-selftest.sh` (19 assertions), red-proven against three
deliberate breakages: a `cache_root_for` that ignores the lineage, a disabled
validator, and a `verify` that never rejects a mismatch.
This commit is contained in:
2026-08-26 12:32:21 -05:00
parent f76789358d
commit fb7a788c90
7 changed files with 522 additions and 16 deletions
+101 -8
View File
@@ -317,6 +317,39 @@ spelling of some new dot-prefixed name — a name it reads for a decision but
never reclaims — that is exactly this category, and it needs the matching
`DEPEND_*` entry on gitdan's side before it ships, not after.
### The directory LAYOUT is part of that contract as well
Names are one half; where they sit is the other. gitdan's arbiter walks a
volume's `_data` tree to `CI_CACHE_MAX_DEPTH`, which is **2** — deliberately
tight, because a deeper walk starts meeting Cargo's own
`incremental/<crate>-<hash>` directories, which match the same name shape it
uses to recognise a cache dir and must never be evicted individually. So:
```
_data/target-<key> depth 1 — no lineage
_data/<lineage>/target-<key> depth 2 — a lineage
_data/<a>/<b>/target-<key> depth 3 — INVISIBLE to the arbiter
```
That budget is the whole reason `cache-lineage` is one path component and not
a path. Nesting deeper is not an error anywhere: the caches work, the
in-workflow prune pass keeps managing them, and the one script whose job is
the shared disk budget across every repo simply never sees them again.
It is also why the fix for daniel/gitdan#60 nests rather than suffixing the
cache key. A `target-<key>-<lineage>` name would be read as dead by
`prune-cache.sh`'s liveness pass — which classifies by recomputing
`target-<cache_key(branch)>` for every branch on origin — and evicted
unconditionally on every run; and it falls out of the arbiter's own
`BRANCH_DIR_RE` too, so the same directories would never be candidates there
either. Nesting leaves both matchers reading exactly the names they already
read, one level down.
One known rough edge, on gitdan's side and cosmetic: that script logs an
eviction as `<volume>/<basename>`, so a nested `target-<key>` and a flat one
of the same key are indistinguishable in its output. It evicts the right
directory; the line just doesn't say which.
---
## Inputs
@@ -326,6 +359,7 @@ never reclaims — that is exactly this category, and it needs the matching
| input | default | meaning |
|---|---|---|
| `cache-root` | `/cache` | mount point of the persistent volume inside the job container |
| `cache-lineage` | *(empty)* | one directory level under `cache-root`, for a second job building the same ref for a different target or profile — see [Multiple jobs in one workflow](#multiple-jobs-in-one-workflow) |
| `protected-branches` | `dev main` | refs that publish snapshots and are never evicted |
| `min-free-percent` | `10` | prune when free space drops below this |
| `restore-mtimes` | `true` | restore tracked-file mtimes from git history |
@@ -334,7 +368,7 @@ never reclaims — that is exactly this category, and it needs the matching
| `own-ref` | *(auto)* | override; defaults to `github.head_ref`, else `github.ref_name` |
| `base-ref` | *(auto)* | override; defaults to `github.base_ref` (empty on push) |
| `seed-fallback-dir` | *(empty)* | absolute path to seed from when no snapshot exists — for migrating off an existing flat cache |
| `watermark-file` | `.ci-watermark-<job>-sha` | must differ per job when two jobs share one cache key |
| `watermark-file` | `.ci-watermark-<job>-sha` | must differ per job when two jobs share one target directory; the default already does |
| `lock-id` | `<job>-<run_id>` | identifies this job's cache lock |
| `stale-lock-seconds` | `7200` | age past which another job's lock is treated as abandoned |
@@ -350,6 +384,7 @@ Exports to the job environment: `CARGO_TARGET_DIR`, `CARGO_CACHE_ROOT`,
| input | default | meaning |
|---|---|---|
| `cache-root` | `/cache` | must match the consume action |
| `cache-lineage` | *(empty)* | must match the consume action; a mismatch fails the step rather than publishing the wrong tree |
| `protected-branches` | `dev main` | refs that publish snapshots |
| `mode` | `publish` | `publish`, or `release-lock` for the `if: always()` step |
| `own-ref` | *(auto)* | override; defaults to `github.head_ref`, else `github.ref_name` |
@@ -366,13 +401,70 @@ of a merge-preview build, which is not what `dev` is.
## Multiple jobs in one workflow
Jobs sharing a cache key (a `ci` job and a `wasm` job on the same branch, say)
each need their **own** watermark file. A shared one breaks the moment two
jobs run in sequence within one trigger: job A advances the watermark to HEAD,
and job B then reads that just-advanced value, computes an empty diff, and
loses the merge protection entirely. The default (`.ci-watermark-<job>-sha`)
already gives each job its own; only override `watermark-file` if you also
override `lock-id`, and then keep both distinct per job.
Two jobs building the same ref — a `ci` job and a `wasm` job, say — are two
consumers of one cache key, and the cache key alone is not enough to keep them
apart.
**Give each its own lineage.** A cache key names a *ref*; what a target
directory holds is the product of a ref and a build configuration. Left to the
key alone, both jobs export the same `CARGO_TARGET_DIR`, and Cargo's
build-directory lock is exclusive — so on a runner with more than one slot the
second job sits on `Blocking waiting for file lock on build directory` for the
length of the first, occupying a capacity slot while doing nothing
(daniel/gitdan#60). `cache-lineage` is that second dimension:
```yaml
- name: Restore the Cargo cache
uses: https://gitdan.com/daniel/gitdan-actions/cargo-cache@v1
with:
cache-lineage: wasm32 # the `ci` job sets none
# ... build steps ...
- name: Record watermark, publish cache snapshot
uses: https://gitdan.com/daniel/gitdan-actions/cargo-cache-publish@v1
with:
cache-lineage: wasm32 # the SAME value, or the step fails
```
A lineage nests one directory level under the cache root
(`<cache-root>/<lineage>/target-<key>`), so each lineage gets its own target
dirs, its own snapshots, and its own prune pass. Everything else works as it
already did, one level down: a PR branch in a lineage layers over **that
lineage's** base snapshot, the publisher branch publishes into it, and a prune
pass run inside it never sees a sibling lineage's caches.
Setting no lineage resolves to the cache root unchanged, byte for byte, so a
workflow that does not use one keeps the exact directories it already has on
the volume.
**Both actions need the same value.** `cargo-cache-publish` derives both ends
of the snapshot swap from its own `cache-root`, so a publish step left at the
default while its consume step nested would republish a *different* lineage's
live target dir over that lineage's snapshot, on every push, with nothing in
the log to say so. The publish action therefore compares its own inputs
against the `CARGO_CACHE_ROOT` the consume step exported and fails the step on
a mismatch. (The `mode: release-lock` call is exempt: it releases a lock on
`$CARGO_TARGET_DIR` and never touches a cache root, so it takes no lineage.)
**Some lineage names are refused.** A lineage is one path component, drawn
from `[A-Za-z0-9._-]`, and several otherwise-reasonable names are rejected at
resolve time because a *reader elsewhere* would stop seeing the caches
underneath them: a Cargo profile name (`debug`, `release`, `doc`, …) is one
gitdan's arbiter never descends into, a `target-`/`snapshot-` prefix makes the
lineage directory itself an eviction candidate for this repo's own prune pass,
and a hex-suffixed name is read by that arbiter as a per-branch cache dir in
its own right. `validate_cache_lineage()` in `scripts/cache-lib.sh` states each
rejection with the reader that imposes it.
**Watermarks are still per job.** Two jobs in one lineage — or one job before
lineages were introduced — each need their **own** watermark file. A shared one
breaks the moment two jobs run in sequence within one trigger: job A advances
the watermark to HEAD, and job B then reads that just-advanced value, computes
an empty diff, and loses the merge protection entirely. The default
(`.ci-watermark-<job>-sha`) already gives each job its own; only override
`watermark-file` if you also override `lock-id`, and then keep both distinct
per job.
---
@@ -466,6 +558,7 @@ change here reaches all of them at once. That is what the gate is for.
| suite | covers |
|---|---|
| `cache-root-selftest.sh` | that a lineage nests one level and nothing else moves: no lineage resolves byte-for-byte to the cache root, two lineages on one cache key get disjoint target dirs, seed/publish/prune all stay inside their own lineage, a PR layers over its own lineage's base snapshot — **and one rejection per lineage name a reader elsewhere would stop seeing**, plus the publish-side mismatch guard |
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler. |
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published, and the publisher's log showing it waited on the consumer's own reader-lock marker before reclaiming a rotated snapshot |
| `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone |
+27 -1
View File
@@ -10,6 +10,15 @@ inputs:
description: 'Mount point of the persistent cache volume. Must match the consume action.'
required: false
default: '/cache'
cache-lineage:
description: >-
Must match the cargo-cache step in this job. Checked rather than
assumed: this action derives both ends of the snapshot swap from its own
cache-root, so a publish step left at the default while its consume step
nested would republish a DIFFERENT lineage's live target dir over that
lineage's snapshot, on every push, silently. A mismatch fails the step.
required: false
default: ''
protected-branches:
description: >-
Space-separated refs that publish snapshots. A run whose own ref is not
@@ -67,6 +76,11 @@ runs:
# Everything here reads the environment the consume action exported, so a
# workflow that forgets to run cargo-cache first fails loudly here rather
# than silently publishing a snapshot of the wrong directory.
#
# The cache root is taken from that same environment for the same reason,
# and this action's own cache-root/cache-lineage inputs are checked against
# it rather than used. The two have always had to agree; until a lineage
# existed they always did, because nobody overrode the default.
- id: resolve
shell: bash
env:
@@ -92,6 +106,18 @@ runs:
: "${CARGO_CACHE_KEY:?cargo-cache-publish: CARGO_CACHE_KEY not set by the cargo-cache action}"
echo "active=yes" >> "$GITHUB_OUTPUT"
# Only in publish mode. The `release-lock` call is an `if: always()`
# step that consuming workflows invoke with `mode:` and nothing else —
# it releases $CARGO_CACHE_LOCK_ID on $CARGO_TARGET_DIR and never
# touches a cache root at all, so holding its default inputs to the
# consume step's would fail the cleanup step of every job that sets a
# lineage, for a value it does not use.
if [ "${{ inputs.mode }}" = "publish" ]; then
: "${CARGO_CACHE_ROOT:?cargo-cache-publish: CARGO_CACHE_ROOT not set by the cargo-cache action}"
bash "${CARGO_CACHE_SCRIPTS}/cache-root.sh" verify \
"${{ inputs.cache-root }}" "${{ inputs.cache-lineage }}" "$CARGO_CACHE_ROOT"
fi
OWN_REF="${{ inputs.own-ref }}"
[ -n "$OWN_REF" ] || OWN_REF="${{ github.head_ref || github.ref_name }}"
@@ -131,7 +157,7 @@ runs:
run: |
set -euo pipefail
bash "${CARGO_CACHE_SCRIPTS}/publish-snapshot.sh" \
"$CARGO_CACHE_KEY" "${{ inputs.cache-root }}" \
"$CARGO_CACHE_KEY" "$CARGO_CACHE_ROOT" \
"${{ github.job }}-${{ github.run_id }}-$$"
# Released in both modes. In `publish` mode this is the normal end-of-job
+38 -6
View File
@@ -10,6 +10,18 @@ inputs:
description: 'Mount point of the persistent cache volume inside the job container.'
required: false
default: '/cache'
cache-lineage:
description: >-
Distinguishes two jobs that build the SAME ref for different targets or
profiles (a host build and a wasm32 build, say) and would otherwise
resolve to one CARGO_TARGET_DIR and serialise on Cargo's exclusive
build-directory lock. Names one directory level under cache-root:
<cache-root>/<lineage>/target-<key>. Must be a single path component;
several names are refused outright because a reader elsewhere would stop
seeing the caches under them (see validate_cache_lineage in
scripts/cache-lib.sh). Pass the same value to cargo-cache-publish.
required: false
default: ''
protected-branches:
description: >-
Space-separated refs that publish snapshots and are never evicted.
@@ -54,8 +66,9 @@ inputs:
watermark-file:
description: >-
Name of this job's build-watermark file inside the target dir. MUST be
distinct per job when two jobs share one cache key. Defaults to
.ci-watermark-<job>-sha.
distinct per job when two jobs share one target directory — which two
jobs no longer need to do; cache-lineage gives them separate ones.
Defaults to .ci-watermark-<job>-sha, already distinct per job.
required: false
default: ''
lock-id:
@@ -74,6 +87,12 @@ outputs:
cache-key:
description: 'Sanitized cache key for this run''s own ref.'
value: ${{ steps.resolve.outputs.cache-key }}
cache-root:
description: >-
Resolved cache root — cache-root, plus the lineage directory when one is
set. Every directory this action reads or writes is under it. Also
exported as CARGO_CACHE_ROOT.
value: ${{ steps.resolve.outputs.cache-root }}
seeded-from:
description: 'Where the target dir came from: own | base-snapshot | own-snapshot | fallback-dir | concurrent-peer | cold.'
value: ${{ steps.seed.outputs.seeded-from }}
@@ -95,6 +114,16 @@ runs:
# `base_ref` is populated only for pull_request events. A push run has
# nothing to layer over: its own ref IS the reference branch. It
# publishes, it does not consume.
#
# The cache root is resolved first because every path below hangs off it.
# A lineage nests one directory level (`<root>/<lineage>/target-<key>`),
# which is what lets two jobs on ONE ref hold two build directories and so
# not serialise on Cargo's exclusive lock. It is resolved through
# cache-root.sh rather than interpolated here so the name is validated —
# several otherwise-reasonable lineage names put their whole subtree out of
# reach of a pass that has to see it. Everything downstream reads the
# resolved value, and it is exported as CARGO_CACHE_ROOT so the publish
# action can check it agrees with its own inputs.
- id: resolve
shell: bash
run: |
@@ -102,6 +131,8 @@ runs:
SCRIPTS=$(cd "${{ github.action_path }}/.." && pwd)/scripts
[ -d "$SCRIPTS" ] || { echo "::error::cargo-cache: scripts/ not found at $SCRIPTS"; exit 1; }
echo "CARGO_CACHE_SCRIPTS=${SCRIPTS}" >> "$GITHUB_ENV"
CACHE_ROOT=$(bash "${SCRIPTS}/cache-root.sh" resolve \
"${{ inputs.cache-root }}" "${{ inputs.cache-lineage }}")
OWN_REF="${{ inputs.own-ref }}"
[ -n "$OWN_REF" ] || OWN_REF="${{ github.head_ref || github.ref_name }}"
BASE_REF="${{ inputs.base-ref }}"
@@ -116,7 +147,7 @@ runs:
echo "cache: own ref '${OWN_REF}' -> ${OWN_KEY} (no base ref — this ref publishes, it does not consume)"
fi
TARGET_DIR="${{ inputs.cache-root }}/target-${OWN_KEY}"
TARGET_DIR="${CACHE_ROOT}/target-${OWN_KEY}"
WATERMARK="${{ inputs.watermark-file }}"
[ -n "$WATERMARK" ] || WATERMARK=".ci-watermark-${{ github.job }}-sha"
LOCK_ID="${{ inputs.lock-id }}"
@@ -128,10 +159,11 @@ runs:
echo "base-key=${BASE_KEY}"
echo "lock-id=${LOCK_ID}"
echo "watermark-file=${WATERMARK}"
echo "cache-root=${CACHE_ROOT}"
} >> "$GITHUB_OUTPUT"
{
echo "CARGO_TARGET_DIR=${TARGET_DIR}"
echo "CARGO_CACHE_ROOT=${{ inputs.cache-root }}"
echo "CARGO_CACHE_ROOT=${CACHE_ROOT}"
echo "CARGO_CACHE_KEY=${OWN_KEY}"
echo "CARGO_CACHE_LOCK_ID=${LOCK_ID}"
echo "CI_WATERMARK_FILE=${WATERMARK}"
@@ -155,7 +187,7 @@ runs:
bash "${SCRIPTS}/seed-target-dir.sh" \
"${{ steps.resolve.outputs.cache-key }}" \
"${{ steps.resolve.outputs.base-key }}" \
"${{ inputs.cache-root }}" \
"${{ steps.resolve.outputs.cache-root }}" \
"${{ github.job }}-${{ github.run_id }}-$$" \
"${{ inputs.seed-fallback-dir }}" \
"${{ steps.resolve.outputs.lock-id }}"
@@ -206,7 +238,7 @@ runs:
set -euo pipefail
SCRIPTS=$(cd "${{ github.action_path }}/.." && pwd)/scripts
bash "${SCRIPTS}/prune-cache.sh" \
"${{ inputs.cache-root }}" \
"${{ steps.resolve.outputs.cache-root }}" \
"${{ steps.resolve.outputs.target-dir }}" \
"${{ inputs.protected-branches }}" \
"${{ inputs.min-free-percent }}"
+100
View File
@@ -114,6 +114,106 @@ cache_key() {
target_dir_for() { printf '%s/target-%s' "$1" "$2"; }
snapshot_dir_for() { printf '%s/snapshot-%s' "$1" "$2"; }
# ---------------------------------------------------------------------------
# Cache lineages
# ---------------------------------------------------------------------------
#
# A cache key names a REF. What a target directory holds is the product of a
# ref and a BUILD CONFIGURATION, and the two are not the same thing: emowheel
# builds the same ref twice on every push, once for the host and once for
# wasm32, in two jobs that run concurrently. Keyed on the ref alone both
# resolve to one CARGO_TARGET_DIR, and Cargo's build-directory lock is
# exclusive — so the second job sits on `Blocking waiting for file lock on
# build directory` for the length of the first, holding a runner capacity slot
# while doing nothing (daniel/gitdan#60).
#
# A lineage is that second dimension, and it is expressed as ONE DIRECTORY
# LEVEL above the per-ref directories rather than as a suffix on the key:
#
# <cache-root>/target-<key> no lineage (the flat layout)
# <cache-root>/<lineage>/target-<key> a lineage
#
# Nesting rather than suffixing is what keeps every existing reader correct
# without teaching any of them a new name shape. `prune-cache.sh` resolves
# liveness by recomputing `target-<cache_key(branch)>` for every branch on
# origin and evicting whatever does not match — a suffixed `target-<key>-wasm32`
# matches nothing, so it would be classified dead and unconditionally evicted
# on every single run. The host-level arbiter in daniel/gitdan reads the same
# shape (its BRANCH_DIR_RE), and a suffixed name falls out of it too: not
# evicted there, but never a candidate either, so a whole lineage becomes
# invisible to the global disk budget. Nesting leaves both matchers reading
# exactly the names they already read, one directory deeper.
#
# ONE LEVEL, AND NOT TWO. The arbiter walks a volume to CI_CACHE_MAX_DEPTH,
# which is 2 — `_data/target-<key>` and `_data/<lineage>/target-<key>`. It is
# kept tight there on purpose (a deeper walk starts meeting Cargo's own
# `incremental/<crate>-<hash>` directories, which match the same name shape and
# must never be evicted individually), so a lineage is a single path component
# and validate_cache_lineage refuses one containing a slash.
# Directory names daniel/gitdan's ci-cache-reclaim.sh refuses to descend into
# (its CI_CACHE_NODESCEND_NAMES). A lineage named one of these puts its whole
# subtree outside the global arbiter's reach: the caches accumulate and the one
# script whose job is the shared disk budget cannot see them.
CACHE_LINEAGE_RESERVED_NAMES="debug release deps incremental build .fingerprint tmp examples doc"
# The shape that same script reads as a per-branch cache directory (its
# BRANCH_DIR_RE). A lineage matching it is taken for a cache dir in its own
# right — never descended into, and an eviction candidate whole, which is the
# entire lineage rather than one ref's share of it.
CACHE_LINEAGE_BRANCH_DIR_RE='^.+-[0-9a-f]{7,40}$'
# Every rejection below names the reader that imposes it, because that is the
# only way the constraint survives: none of these is a filesystem limit, and a
# name that trips one produces no error anywhere — it produces a lineage that
# silently stops being pruned, or silently stops being reclaimed.
validate_cache_lineage() {
local lineage="$1" reserved
[ -n "$lineage" ] || return 0
case "$lineage" in
*/*)
echo "::error::cache lineage '${lineage}' must be a single path component: the host-level arbiter walks a cache volume to depth 2, so <cache-root>/<lineage>/target-<key> is as deep as a cache directory may sit and still be reclaimable" >&2
return 1
;;
.*)
echo "::error::cache lineage '${lineage}' must not start with a dot: every dot-prefixed entry under a cache root belongs to the leftover-naming contract (see the top of this file), and a lineage is not garbage to be reclaimed" >&2
return 1
;;
target-* | snapshot-*)
echo "::error::cache lineage '${lineage}' must not start with 'target-' or 'snapshot-': prune-cache.sh globs both prefixes at the cache root, so the lineage directory itself would become an eviction candidate" >&2
return 1
;;
*[!A-Za-z0-9._-]*)
echo "::error::cache lineage '${lineage}' may contain only [A-Za-z0-9._-] — the same charset cache_key() sanitises a ref down to" >&2
return 1
;;
esac
for reserved in $CACHE_LINEAGE_RESERVED_NAMES; do
if [ "$lineage" = "$reserved" ]; then
echo "::error::cache lineage '${lineage}' is one of the Cargo directory names daniel/gitdan's ci-cache-reclaim.sh never descends into (CI_CACHE_NODESCEND_NAMES) — every cache under it would be invisible to the host-level disk budget" >&2
return 1
fi
done
if [[ $lineage =~ $CACHE_LINEAGE_BRANCH_DIR_RE ]]; then
echo "::error::cache lineage '${lineage}' ends in a hex suffix, which is the shape daniel/gitdan's ci-cache-reclaim.sh reads as a per-branch cache directory — it would treat the lineage directory as one cache and evict the whole thing" >&2
return 1
fi
return 0
}
# The cache root a lineage's directories actually live under. An empty lineage
# resolves to the cache root unchanged, byte for byte: that is what makes this
# a no-op for every consumer that does not set one, rather than a migration.
cache_root_for() {
local root="$1" lineage="${2:-}"
validate_cache_lineage "$lineage" || return 1
printf '%s%s' "$root" "${lineage:+/$lineage}"
}
# ---------------------------------------------------------------------------
# Disk accounting
# ---------------------------------------------------------------------------
+194
View File
@@ -0,0 +1,194 @@
#!/usr/bin/env bash
# Regression test for cache-root.sh and the lineage rules in cache-lib.sh —
# the fix for daniel/gitdan#60, where two jobs building the same ref for
# different targets resolved to one CARGO_TARGET_DIR and serialised on Cargo's
# exclusive build-directory lock.
#
# 1. NO LINEAGE CHANGES NOTHING — the effective root is the cache root byte
# for byte, so every consumer that does not set a lineage keeps the exact
# directories it already has on the volume. This is the whole of the
# migration story for lublub, zemyna and emowheel's `ci` job, so it is
# asserted rather than assumed.
# 2. TWO LINEAGES, ONE REF, TWO TARGET DIRS — the bug itself. The two jobs
# keep one cache key (they are the same ref) and still get directories
# that are neither equal nor nested one inside the other, which is what
# Cargo's per-directory lock needs in order not to serialise them.
# 3. THE WHOLE PIPELINE MOVES TOGETHER — seed, publish and prune all operate
# inside the lineage root. A pass run in one lineage must not evict, or
# even see, a sibling lineage's caches or the flat layout's.
# 4. BASE SEEDING IS PER LINEAGE — a PR branch layers over ITS OWN lineage's
# base snapshot, not over whatever the flat root happens to hold. This is
# the property that keeps a warm start for both jobs rather than one.
# 5. A NAME NO READER CAN HANDLE IS REFUSED AT RESOLVE TIME — one assertion
# per constraint, each named for the reader that imposes it. None of these
# is a filesystem limit: every one of them produces a working directory
# that some pass silently stops seeing, which is the failure mode this
# whole scheme exists to avoid rather than to relocate.
# 6. A PUBLISH THAT DISAGREES WITH ITS CONSUME STEP FAILS LOUDLY — the
# footgun the lineage input introduces. cargo-cache-publish derives both
# ends of the snapshot swap from its own `cache-root`, so a publish step
# left at the default while its consume step nested would republish the
# OTHER lineage's live target dir over that lineage's snapshot, silently.
set -euo pipefail
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
. "$script_dir/cache-lib.sh"
scratch=$(mktemp -d)
trap 'rm -rf "$scratch"' EXIT
root="$scratch/cache"; mkdir -p "$root"
pass_count=0
fail() { echo "ASSERTION FAILED: $*" >&2; exit 1; }
ok() { pass_count=$((pass_count + 1)); echo "PASS: $*"; }
resolve() { bash "$script_dir/cache-root.sh" resolve "$@"; }
# A tree that looks enough like a Cargo target dir for the pipeline scripts,
# with a marker naming which lineage produced it — scenario 4 turns on reading
# that marker back out of a seeded directory.
make_tree() {
local d="$1" marker="$2"
mkdir -p "$d/debug/deps" "$d/debug/.fingerprint/x"
echo "$marker" > "$d/debug/deps/libx.rlib"
echo "$marker" > "$d/lineage-marker"
echo "$marker" > "$d/debug/.fingerprint/x/dep-lib-x"
}
echo "=== 1. no lineage changes nothing ==="
[ "$(resolve /cache)" = /cache ] || fail "an omitted lineage changed the root"
[ "$(resolve /cache '')" = /cache ] || fail "an empty lineage changed the root"
ok "no lineage resolves to the cache root unchanged"
KEY=$(cache_key dev)
[ "$(target_dir_for "$(resolve /cache '')" "$KEY")" = "/cache/target-$KEY" ] \
|| fail "the flat target-dir name moved"
[ "$(snapshot_dir_for "$(resolve /cache '')" "$KEY")" = "/cache/snapshot-$KEY" ] \
|| fail "the flat snapshot name moved"
ok "the flat layout's directory names are untouched"
echo
echo "=== 2. two lineages, one ref, two target dirs ==="
HOST_ROOT=$(resolve /cache)
WASM_ROOT=$(resolve /cache wasm32)
[ "$WASM_ROOT" = /cache/wasm32 ] || fail "lineage root resolved to '$WASM_ROOT'"
HOST_DIR=$(target_dir_for "$HOST_ROOT" "$KEY")
WASM_DIR=$(target_dir_for "$WASM_ROOT" "$KEY")
[ "$HOST_DIR" != "$WASM_DIR" ] || fail "both lineages resolved to $HOST_DIR"
case "$WASM_DIR" in "$HOST_DIR"/*) fail "the wasm target dir sits inside the host one" ;; esac
case "$HOST_DIR" in "$WASM_DIR"/*) fail "the host target dir sits inside the wasm one" ;; esac
ok "one cache key ($KEY), two disjoint target dirs: $HOST_DIR and $WASM_DIR"
echo
echo "=== 3. the whole pipeline moves together ==="
DEAD=$(cache_key feat/dead)
lin="$root/wasm32"
mkdir -p "$lin"
make_tree "$root/target-$DEAD" flat
make_tree "$root/native/target-$DEAD" native
make_tree "$lin/target-$DEAD" wasm32
bash "$script_dir/seed-target-dir.sh" "$KEY" "" "$lin" tag-3 > "$scratch/seed3.log" 2>&1 \
|| { cat "$scratch/seed3.log"; fail "seed inside a lineage root failed"; }
[ -d "$lin/target-$KEY" ] || fail "seed did not create $lin/target-$KEY"
[ -d "$root/target-$KEY" ] && fail "seed created a directory at the flat root as well"
ok "seed creates its directory under the lineage root and nowhere else"
make_tree "$lin/target-$KEY" wasm32
bash "$script_dir/publish-snapshot.sh" "$KEY" "$lin" tag-3 > "$scratch/pub3.log" 2>&1 \
|| { cat "$scratch/pub3.log"; fail "publish inside a lineage root failed"; }
[ -d "$lin/snapshot-$KEY" ] || fail "publish did not create $lin/snapshot-$KEY"
[ -d "$root/snapshot-$KEY" ] && fail "publish created a snapshot at the flat root as well"
ok "publish writes its snapshot under the lineage root and nowhere else"
# Free space far below the threshold, so pass 2 evicts every eligible
# directory it can see. What it can see is the point of the scenario.
CACHE_LIVENESS=false CACHE_DF_OVERRIDE="1000000 1000" \
bash "$script_dir/prune-cache.sh" "$lin" "$lin/target-$KEY" 'dev main' 10 \
> "$scratch/prune3.log" 2>&1 || { cat "$scratch/prune3.log"; fail "prune inside a lineage root failed"; }
[ -d "$lin/target-$DEAD" ] && { cat "$scratch/prune3.log"; fail "prune left its own lineage's evictable cache in place"; }
[ -d "$root/target-$DEAD" ] || fail "prune reached out of its lineage and evicted the flat root's cache"
[ -d "$root/native/target-$DEAD" ] || fail "prune reached into a sibling lineage and evicted its cache"
ok "prune under disk pressure evicts inside its own lineage only"
echo
echo "=== 4. base seeding is per lineage ==="
BASE=$(cache_key dev)
PR=$(cache_key feat/pr)
rm -rf "$lin" "$root/snapshot-$BASE"
mkdir -p "$lin"
make_tree "$root/snapshot-$BASE" flat-base
make_tree "$lin/snapshot-$BASE" wasm32-base
bash "$script_dir/seed-target-dir.sh" "$PR" "$BASE" "$lin" tag-4 > "$scratch/seed4.log" 2>&1 \
|| { cat "$scratch/seed4.log"; fail "seeding a PR branch inside a lineage failed"; }
[ -d "$lin/target-$PR" ] || fail "the PR branch's lineage target dir was not created"
got=$(cat "$lin/target-$PR/lineage-marker")
[ "$got" = wasm32-base ] || fail "the PR branch layered over '$got', not its own lineage's base snapshot"
grep -q 'base snapshot' "$scratch/seed4.log" || { cat "$scratch/seed4.log"; fail "seed did not report a base-snapshot clone"; }
ok "a PR branch layers over its own lineage's base snapshot ($got)"
echo
echo "=== 5. a name no reader can handle is refused ==="
reject() {
local lineage="$1" want="$2" desc="$3" out
if out=$(resolve /cache "$lineage" 2>&1); then
fail "lineage '$lineage' was accepted (resolved to '$out') — $desc"
fi
case "$out" in
*"$want"*) ;;
*) fail "lineage '$lineage' was rejected without naming '$want': $out" ;;
esac
ok "rejected '$lineage' — $desc"
}
reject 'a/b' 'single path component' "the arbiter walks a volume to depth 2"
reject '.hidden' 'dot' "every dot-prefixed name under a cache root is leftover-contract territory"
reject 'target-x' 'prune-cache.sh' "prune-cache.sh globs target-* at the cache root"
reject 'snapshot-x' 'prune-cache.sh' "prune-cache.sh globs snapshot-* at the cache root"
reject 'wasm 32' 'A-Za-z0-9._-' "a cache key is sanitised to that charset and a lineage sits beside one"
reject 'release' 'CI_CACHE_NODESCEND_NAMES' "the arbiter never descends into a Cargo profile name"
reject 'doc' 'CI_CACHE_NODESCEND_NAMES' "same, for the docs profile directory"
reject 'lineage-deadbeef' 'per-branch cache directory' "the arbiter reads a hex-suffixed name as one cache dir"
for good in wasm32 web android host wasm32.release lineage_2; do
out=$(resolve /cache "$good") || fail "lineage '$good' was rejected: $out"
[ "$out" = "/cache/$good" ] || fail "lineage '$good' resolved to '$out'"
done
ok "ordinary lineage names still resolve"
echo
echo "=== 6. a publish that disagrees with its consume step fails loudly ==="
verify() { bash "$script_dir/cache-root.sh" verify "$@"; }
verify /cache wasm32 /cache/wasm32 > /dev/null 2>&1 \
|| fail "verify rejected a publish step that agrees with its consume step"
verify /cache '' /cache > /dev/null 2>&1 \
|| fail "verify rejected an unmigrated consumer's matching default pair"
ok "verify accepts a publish step whose inputs match what the consume step exported"
# The exact shape of the mistake: the consume step nested, the publish step
# kept the default. Left unchecked this republishes the host lineage's live
# target dir over the host lineage's snapshot, from the wasm job.
if out=$(verify /cache '' /cache/wasm32 2>&1); then
fail "verify accepted a publish step that resolved to /cache while the job exported /cache/wasm32"
fi
case "$out" in
*'/cache/wasm32'*) ;;
*) fail "the mismatch error does not quote what the consume step exported: $out" ;;
esac
case "$out" in
*'SAME cache-root and cache-lineage'*) ;;
*) fail "the mismatch error does not say what to do about it: $out" ;;
esac
ok "verify rejects a publish step that forgot the lineage, and says so"
if verify /cache 'a/b' /cache/a/b > /dev/null 2>&1; then
fail "verify accepted an invalid lineage as long as both sides agreed on it"
fi
ok "verify validates the lineage as well as comparing it"
echo
echo "cache-root-selftest: ${pass_count} assertions passed"
+61
View File
@@ -0,0 +1,61 @@
#!/usr/bin/env bash
# Resolves — and cross-checks — the cache root a job's directories live under.
#
# cache-root.sh resolve <cache-root> [lineage]
# cache-root.sh verify <cache-root> <lineage> <exported-root>
#
# `resolve` prints the effective root: the cache root unchanged when no lineage
# is given, or `<cache-root>/<lineage>` when one is. Invalid lineage names are
# rejected here rather than downstream — see validate_cache_lineage() in
# cache-lib.sh, where every rejection names the reader that imposes it.
#
# `verify` is the publish side's guard. cargo-cache-publish resolves the same
# two inputs the consume action was given and compares the result against the
# CARGO_CACHE_ROOT the consume step exported into the job environment. The two
# actions have always had to agree — `cache-root`'s description in the publish
# action says "must match the consume action" — and until a lineage existed
# they always did, because nobody overrode the default. A disagreement is not
# a harmless no-op: publish-snapshot.sh takes the root as an argument and
# derives BOTH ends of the swap from it, so a publish step that kept the
# default while its consume step nested would read `<root>/target-<key>` — the
# OTHER lineage's live target dir — and republish it over `<root>/snapshot-<key>`,
# which is that lineage's snapshot. Two jobs would then be publishing one
# snapshot from one tree on every push, and nothing in either action would say
# so. Hence: fail the job, loudly, rather than resolve the ambiguity in
# either direction.
#
# A thin CLI over cache-lib.sh, kept as its own entry point for the same
# reason branch-cache-key.sh is: an out-of-band job that needs to find a
# lineage's directories should resolve the path the way the action does
# instead of reimplementing the rule.
set -euo pipefail
. "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/cache-lib.sh"
MODE="${1:-}"
case "$MODE" in
resolve)
[ $# -ge 2 ] && [ $# -le 3 ] || {
echo "::error::cache-root.sh resolve: expected <cache-root> [lineage]" >&2
exit 1
}
[ -n "$2" ] || { echo "::error::cache-root.sh: cache-root must not be empty" >&2; exit 1; }
cache_root_for "$2" "${3:-}"
;;
verify)
[ $# -eq 4 ] || {
echo "::error::cache-root.sh verify: expected <cache-root> <lineage> <exported-root>" >&2
exit 1
}
[ -n "$2" ] || { echo "::error::cache-root.sh: cache-root must not be empty" >&2; exit 1; }
expected=$(cache_root_for "$2" "$3")
if [ "$expected" != "$4" ]; then
echo "::error::cache-root.sh: this step resolves its cache root to '${expected}' (cache-root '$2', cache-lineage '$3') but the cargo-cache step in this job exported '$4'. Pass the SAME cache-root and cache-lineage to both actions." >&2
exit 1
fi
echo "cache root: ${expected} (agrees with the cargo-cache step in this job)"
;;
*)
echo "::error::cache-root.sh: unknown mode '${MODE}' (expected resolve or verify)" >&2
exit 1
;;
esac
+1 -1
View File
@@ -13,7 +13,7 @@ script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
FAST=0
[ "${1:-}" = "--fast" ] && FAST=1
FIXTURE_TESTS=(seed-target-dir-selftest.sh publish-snapshot-selftest.sh prune-cache-selftest.sh)
FIXTURE_TESTS=(cache-root-selftest.sh seed-target-dir-selftest.sh publish-snapshot-selftest.sh prune-cache-selftest.sh)
CARGO_TESTS=(hardlink-clone-selftest.sh restore-mtimes-selftest.sh)
TESTS=("${FIXTURE_TESTS[@]}")