claude
fb7a788c90
CI / shellcheck + selftests (pull_request) Failing after 1m19s
A cache key names a REF. What a target directory holds is the product of a ref
and a build configuration, and emowheel builds the same ref twice on every
push — once for the host, once for wasm32, in two jobs that start together.
Keyed on the ref alone, `cargo-cache@v1` handed both the same
CARGO_TARGET_DIR, and Cargo's target-directory lock is exclusive: the second
job sat on "Blocking waiting for file lock on build directory" for the length
of the first while holding a runner capacity slot, so a third repo's queued
job waited behind a job doing nothing.
`cache-lineage` is that second dimension. It names ONE directory level under
the cache root:
<cache-root>/target-<key> no lineage (unchanged)
<cache-root>/<lineage>/target-<key> a lineage
Nesting, not a suffix on the key, and that is the whole design decision.
`prune-cache.sh`'s liveness pass classifies a directory by recomputing
`target-<cache_key(branch)>` for every branch on origin and evicting whatever
does not match — a `target-<key>-wasm32` matches nothing, so it would be
classified dead and evicted unconditionally on every run. daniel/gitdan's
host-level arbiter reads the same shape (BRANCH_DIR_RE); a suffixed name falls
out of that too, so those caches would never be reclaim candidates and a whole
lineage would go missing from the shared disk budget. Nesting leaves both
matchers reading exactly the names they already read, one level down — which
is a layout that arbiter already walks (CI_CACHE_MAX_DEPTH is 2, and its own
suite pins the depth-2 case).
Every interacting part, checked rather than assumed:
- SEED: `seed-target-dir.sh` takes the root as an argument, so a PR branch in
a lineage layers over THAT lineage's base snapshot. Asserted.
- PUBLISH: `publish-snapshot.sh` derives both ends of the swap from the root.
The publish action now takes the root from the `CARGO_CACHE_ROOT` the
consume step exported, and CHECKS its own inputs against it — a publish step
left at the default while its consume step nested would otherwise republish
a different lineage's live target dir over that lineage's snapshot, on every
push, silently. `mode: release-lock` is exempt: it releases a lock on
`$CARGO_TARGET_DIR` and never touches a root.
- WATERMARK: per target dir, so it follows the lineage. Unchanged.
- PRUNE and LIVENESS: scoped to the root they are given, so a pass in one
lineage neither evicts nor sees a sibling's caches, or the flat layout's.
Liveness keeps resolving real branch names, which is what a key suffix would
have broken.
- ci_cache_reclaim: verified by dry-run against a fixture in this layout —
all six nested and flat dirs collected as candidates, protection resolved
correctly on the nested ones, and a `.stage-` stranded inside the lineage
found by the leftover sweep.
Refused lineage names are refused at resolve time, each rejection naming the
reader that imposes it: a path separator (the arbiter's depth budget), a
Cargo profile name (its no-descend list), a `target-`/`snapshot-` prefix (this
repo's own prune globs), a hex suffix (its per-branch-dir shape), a dot prefix
(the leftover-naming contract). None of these fails visibly on its own — each
produces a working directory that some pass silently stops seeing.
Setting no lineage resolves to the cache root byte for byte, so lublub, zemyna
and emowheel's `ci` job keep the exact directories they have on the volume.
New suite `cache-root-selftest.sh` (19 assertions), red-proven against three
deliberate breakages: a `cache_root_for` that ignores the lineage, a disabled
validator, and a `verify` that never rejects a mismatch.
62 lines
2.9 KiB
Bash
Executable File
62 lines
2.9 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Resolves — and cross-checks — the cache root a job's directories live under.
|
|
#
|
|
# cache-root.sh resolve <cache-root> [lineage]
|
|
# cache-root.sh verify <cache-root> <lineage> <exported-root>
|
|
#
|
|
# `resolve` prints the effective root: the cache root unchanged when no lineage
|
|
# is given, or `<cache-root>/<lineage>` when one is. Invalid lineage names are
|
|
# rejected here rather than downstream — see validate_cache_lineage() in
|
|
# cache-lib.sh, where every rejection names the reader that imposes it.
|
|
#
|
|
# `verify` is the publish side's guard. cargo-cache-publish resolves the same
|
|
# two inputs the consume action was given and compares the result against the
|
|
# CARGO_CACHE_ROOT the consume step exported into the job environment. The two
|
|
# actions have always had to agree — `cache-root`'s description in the publish
|
|
# action says "must match the consume action" — and until a lineage existed
|
|
# they always did, because nobody overrode the default. A disagreement is not
|
|
# a harmless no-op: publish-snapshot.sh takes the root as an argument and
|
|
# derives BOTH ends of the swap from it, so a publish step that kept the
|
|
# default while its consume step nested would read `<root>/target-<key>` — the
|
|
# OTHER lineage's live target dir — and republish it over `<root>/snapshot-<key>`,
|
|
# which is that lineage's snapshot. Two jobs would then be publishing one
|
|
# snapshot from one tree on every push, and nothing in either action would say
|
|
# so. Hence: fail the job, loudly, rather than resolve the ambiguity in
|
|
# either direction.
|
|
#
|
|
# A thin CLI over cache-lib.sh, kept as its own entry point for the same
|
|
# reason branch-cache-key.sh is: an out-of-band job that needs to find a
|
|
# lineage's directories should resolve the path the way the action does
|
|
# instead of reimplementing the rule.
|
|
set -euo pipefail
|
|
. "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/cache-lib.sh"
|
|
|
|
MODE="${1:-}"
|
|
case "$MODE" in
|
|
resolve)
|
|
[ $# -ge 2 ] && [ $# -le 3 ] || {
|
|
echo "::error::cache-root.sh resolve: expected <cache-root> [lineage]" >&2
|
|
exit 1
|
|
}
|
|
[ -n "$2" ] || { echo "::error::cache-root.sh: cache-root must not be empty" >&2; exit 1; }
|
|
cache_root_for "$2" "${3:-}"
|
|
;;
|
|
verify)
|
|
[ $# -eq 4 ] || {
|
|
echo "::error::cache-root.sh verify: expected <cache-root> <lineage> <exported-root>" >&2
|
|
exit 1
|
|
}
|
|
[ -n "$2" ] || { echo "::error::cache-root.sh: cache-root must not be empty" >&2; exit 1; }
|
|
expected=$(cache_root_for "$2" "$3")
|
|
if [ "$expected" != "$4" ]; then
|
|
echo "::error::cache-root.sh: this step resolves its cache root to '${expected}' (cache-root '$2', cache-lineage '$3') but the cargo-cache step in this job exported '$4'. Pass the SAME cache-root and cache-lineage to both actions." >&2
|
|
exit 1
|
|
fi
|
|
echo "cache root: ${expected} (agrees with the cargo-cache step in this job)"
|
|
;;
|
|
*)
|
|
echo "::error::cache-root.sh: unknown mode '${MODE}' (expected resolve or verify)" >&2
|
|
exit 1
|
|
;;
|
|
esac
|