Files
gitdan-actions/scripts/prune-cache.sh
T
claudeandClaude Fable 5.1 38a6387936
CI / shellcheck + selftests (pull_request) Successful in 1m27s
docs(cache): recommend delete-on-merge, and say what ancestry still covers
`daniel/zemyna` enabled `default_delete_branch_after_merge` after this
branch was written, so the deleted-branch signal will fire there on
future merges. That makes the setting worth recommending — it is the
cheapest case for this scheme, decidable from `ls-remote` with no
checkout, no objects and no walk — and it does not make the ancestry
signal redundant.

Three things it leaves behind, now enumerated in the README's eviction
section rather than implied: every branch merged before the setting was
turned on, of which zemyna carried 48 and which nothing retroactively
deletes; every merge whose deletion the forge declines or is never asked
to make, since it is best-effort and silent and an API merge without the
flag never asks; and every repo that has not enabled it, which is the
default.

Two present-tense claims about one repo's configuration are reworded
into the conditions they were standing in for, in prune-cache.sh's
header and beside is_merged_dead, plus the two in the selftest that
asserted the forge keeps branches rather than describing the fixture.
No behaviour change; the suite is green and unchanged at 61 assertions.

Refs #20.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JXMQCJ5Eg5f9G9cfYzyh4Z
2026-09-07 00:27:11 -05:00

613 lines
30 KiB
Bash
Executable File

#!/usr/bin/env bash
# Eviction for the per-ref cache directories on the persistent volume.
#
# Usage: prune-cache.sh <cache-root> <own-target-dir> <protected-branches> \
# <min-free-percent> [own-key] [base-key] [fallback-dir]
# protected-branches space-separated raw refs (e.g. "dev main")
# min-free-percent a FLOOR, not the gate — 0 to rely on the derived
# requirement alone (the default)
# own-key, base-key, fallback-dir
# the same three the seed step resolves its source from.
# Given them, this pass sizes the volume for the clone
# that step is about to make; without them it has only
# the percentage floor, and says so.
#
# Optional environment:
# STALE_LOCK_SECONDS age past which a .ci-lock-* marker is treated as
# abandoned (default 7200)
# CACHE_LIVENESS "false"/"0" to skip the liveness pass entirely
# CACHE_DF_OVERRIDE "<total_kb> <free_kb>", for the selftest
# EVICTION_ASIDE_SETTLE_SECONDS
# how long a directory renamed aside for eviction is
# left alone before another pass may reclaim it
# (default 60)
#
# Three passes, in order:
#
# 1. LIVENESS — every target-*/snapshot-* directory whose branch is DEAD is
# removed UNCONDITIONALLY, not gated on free space. A directory for a
# branch nothing will build again is pure loss; waiting for disk pressure
# to notice means paying for it until then. Two signals make a branch
# dead, and the second exists because the first alone is inert wherever
# a merged branch stays on origin — the default, and still the outcome
# whenever delete-on-merge declines or is not asked (gitdan-actions#20):
#
# DELETED — the branch is no longer on origin at all.
# MERGED — the branch is still on origin, but its tip is an ancestor
# of a protected branch's tip, so every commit it holds is
# already on the branch its cache would be re-cloned from.
#
# Skipped entirely, loudly, if the signal itself is unavailable —
# "couldn't determine" is never folded into "dead", for either signal
# and at either granularity: a checkout that cannot answer ancestry at
# all skips the merged half, and a single branch whose tip is not in the
# checkout is kept with a warning naming it.
# 2. PRESSURE — if free space is under the requirement, evict remaining
# (now necessarily live) directories oldest-first until it clears. The
# requirement is what the seed step is about to need to clone its source,
# MEASURED off that source (see clone_headroom_kb in cache-lib.sh), and a
# percentage floor only if one is configured. A percentage cannot express
# this: the failure it has to prevent is a clone running out of disk
# part-way through unsharing its mutable paths, and how much that needs
# is a property of the snapshot, not of the volume.
# 3. SELF-CLEAR — if pass 2 still isn't enough for the percentage floor,
# wipe this run's own target dir and pay a cold rebuild, reported to the
# job summary as well as the log, because a warning on a green run is
# what lets a silently 4x-slower job go unnoticed. It is not a way out of
# the derived requirement: a run that has an own target dir to wipe is a
# run whose seed reuses it and clones nothing, so that requirement is
# zero. Falling short of a NON-ZERO derived requirement fails the job
# here, naming the shortfall and what was kept instead of it — the seed
# would otherwise fail seconds later against a half-unshared staging
# tree, which is the failure this pass exists to pre-empt.
#
# Reactive-only, with no hard cap on cache size: a workspace's natural working
# set is what it is, and bounding the footprint preemptively means wiping
# useful content before it is actually causing host pressure. The bound is
# physics, not an arbitrary GB number.
#
# EVICTION ORDER, and why it is the reverse of the obvious one: within the
# pressure pass, `target-*` directories are evicted BEFORE `snapshot-*` ones.
# A snapshot is a hardlink clone of a live target dir and of every consumer
# cloned from it, so removing it frees almost no real bytes — its inodes stay
# alive through those other links — while costing every future PR its warm
# start. Evicting snapshots first would be nearly pure loss. Target
# directories are where a branch's own divergent artifacts actually live, so
# they are what freeing space means.
#
# Two exclusions every pass respects:
#
# PROTECTED — the publisher branches' target and snapshot directories, and
# this run's own target dir, are never candidates in any pass. Evicting a
# publisher's snapshot doesn't free real disk (every open PR's clone keeps
# the data alive) but does force every subsequent PR to start cold, which is
# the entire benefit this scheme exists to deliver.
#
# LOCKED — a directory carrying a .ci-lock-* marker younger than
# STALE_LOCK_SECONDS is held open by a running job, or named by a live
# .reading-<dir>-* marker (a job is hardlink-cloning it this instant), is
# skipped by every pass, however dead and however tight the disk. This is
# what makes eviction safe on a runner with more than one execution slot.
# An older marker is treated as abandoned and logged as such, so an
# actually-still-running job that somehow exceeds the threshold is visible
# in the log rather than silently losing its cache mid-build.
#
# That exclusion is decided TWICE per eviction — once as the cheap filter
# that keeps a held directory out of the pass at all, and once after the
# directory has been renamed aside, which is the decision the unlink
# actually rests on. See evict_dir.
#
# Liveness is resolved by `git ls-remote --heads origin`, wrapped in a
# timeout. A directory name cannot be inverted back to a branch name (the
# sanitiser is lossy and the disambiguating suffix is a one-way hash), so this
# goes the other direction: it recomputes the expected directory names for
# every branch origin reports, using cache-lib.sh's OWN cache_key function —
# the same one the seed step used to create them. Reusing that function rather
# than reimplementing a lookalike is what makes the classification sound; any
# drift between two spellings would silently misclassify every directory.
#
# The merged half reads the TIP SHA out of that same `ls-remote` output and
# asks `git merge-base --is-ancestor` against each protected branch's tip,
# using the objects in this job's own checkout. Ancestry is only decidable
# where the objects are there to decide it, so the answer "I cannot tell"
# exists and is distinct from "not merged" everywhere it can arise:
#
# - a shallow checkout makes a MISSING object prove nothing, so the whole
# signal is withheld rather than read as "no branch is merged";
# - a protected tip that is not in the checkout is not used as an anchor;
# - a branch tip that is not in the checkout is kept, loudly.
#
# A squash or rebase merge leaves no ancestor relationship at all, so its
# branch reads as live here. That is a missed reclamation, not a wrong one,
# and the deleted-branch signal still covers it once the branch is removed.
#
# Only ever globs inside <cache-root>. Another project's volume is a different
# Docker named volume and is not mounted in this container at all, so "stays
# scoped to this repo's cache" holds structurally, not by convention.
set -euo pipefail
. "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/cache-lib.sh"
ROOT="${1:?usage: prune-cache.sh <cache-root> <own-target-dir> <protected-branches> <min-free-percent> [own-key] [base-key] [fallback-dir]}"
OWN_DIR="${2:?}"
PROTECTED_REFS="${3:-}"
MIN_FREE_PCT="${4:-0}"
OWN_KEY="${5:-}"
BASE_KEY="${6:-}"
FALLBACK="${7:-}"
# Mirrored by daniel/gitdan's ci-cache-reclaim.sh, whose copy must be >= this
# one — raising this without raising theirs first lets that script treat a lock
# this side still honours as abandoned. Same direction and same reasoning as
# CACHE_READ_STALE_SECONDS; the argument is written out in cache-lib.sh.
STALE_LOCK_SECONDS="${STALE_LOCK_SECONDS:-7200}"
# An aside directory is in flight for one rename plus one marker glob —
# milliseconds. Anything older belongs to a pass that died between the two, so
# an age is what separates "another pass is mid-eviction" from "a leftover",
# and it separates them without having to identify the pass that created it.
# The `$$` in an aside's name was that pass's PID inside its own job
# container, so testing it with `kill -0` from a different one is not
# unreliable, it is meaningless — and PIDs recycle besides. Three orders of
# magnitude of headroom over the operation it covers, and short enough that a
# genuine leftover is reclaimed by the next run rather than lingering while
# the volume is under pressure.
EVICTION_ASIDE_SETTLE_SECONDS="${EVICTION_ASIDE_SETTLE_SECONDS:-60}"
# What the seed step is about to do, resolved through the same function that
# step resolves it with (cache-lib.sh's seed_source_candidates). Empty when it
# will clone nothing at all: its own target dir already exists and it reuses
# it, or no source exists and it starts cold. Either way the derived
# requirement is zero, because nothing is about to be copied.
#
# gb() is for reporting only. Every comparison below is in KB, because
# `read_df` reports KB and rounding a threshold to a tenth of a GB either
# passes a run that cannot fit or evicts a cache the run did not need.
gb() { awk -v k="${1:-0}" 'BEGIN { printf "%.1f", k / 1048576 }'; }
SEED_SRC=""
CLONE_KB=0
if [ -n "$OWN_KEY" ]; then
SEED_SRC=$(seed_clone_source "$ROOT" "$OWN_KEY" "$BASE_KEY" "$FALLBACK")
if [ -n "$SEED_SRC" ]; then
# A full walk of the source, and the reason this pass moved ahead of the
# seed rather than staying where it was: the number is only useful before
# the clone it describes.
CLONE_KB=$(clone_headroom_kb "$SEED_SRC")
echo "clone requirement: seeding from $(basename "$SEED_SRC") needs $(gb "$CLONE_KB") GB free (measured from its mutable set)"
else
echo "clone requirement: none — this run reuses its own cache or starts cold, so nothing will be copied"
fi
else
echo "clone requirement: not derivable — no cache key was passed to this pass; the ${MIN_FREE_PCT}% floor is the only gate"
fi
declare -A protected_ns=()
for ref in $PROTECTED_REFS; do
suffix=$(cache_key "$ref")
protected_ns["target-${suffix}"]=1
protected_ns["snapshot-${suffix}"]=1
done
# Prints why <dir> is off limits to every pass, or nothing when it is a
# candidate. The reason is not decoration: it is what the failure report at
# the bottom lists against each directory it kept while running out of space.
#
# SEED_SRC is the third exclusion and the one this script did not used to need.
# The prune ran after the seed, so the source had already been cloned and the
# reader marker over it was gone; running BEFORE the seed puts the directory
# this run is about to read squarely in the candidate set, and pass 1 would
# take it the moment its branch merged.
protected_reason() {
local dir="$1" name
name=$(basename "$dir")
[ "$dir" = "$OWN_DIR" ] && { printf 'this run own cache'; return 0; }
[ -n "$SEED_SRC" ] && [ "$dir" = "$SEED_SRC" ] && { printf 'the source this run is about to clone'; return 0; }
[ -n "${protected_ns[$name]:-}" ] && { printf 'a protected branch cache'; return 0; }
return 1
}
is_protected() { protected_reason "$1" >/dev/null; }
# is_locked <dir> [name]
#
# `name` is the directory's own name for reporting and for the reader-marker
# lookup, which matters when <dir> has been renamed aside for eviction: the
# markers a consumer publishes are keyed on the name it resolved, not on
# whatever the eviction pass has since called the directory.
is_locked() {
local dir="$1" name="${2:-$(basename "$1")}" now lock_file lock_age locked=1 readers
now=$(date +%s)
# A directory being hardlink-cloned right now carries no .ci-lock-* of its
# own — a snapshot has its locks stripped by construction — so the reader
# markers are the only signal that unlinking it would truncate somebody's
# in-flight clone. Today no reachable configuration prunes a snapshot (only
# protected refs publish them, and protected refs are excluded from every
# pass), which makes this guard redundant *by policy* — evict_dir's second
# look is what stops policy being the reason it is safe.
readers=$(live_reader_count "$ROOT" "$name")
if [ "$readers" -gt 0 ]; then
echo " ${name}: ${readers} job(s) currently cloning it — not a candidate"
locked=0
fi
for lock_file in "$dir"/.ci-lock-*; do
[ -e "$lock_file" ] || continue
lock_age=$(( now - $(stat -c '%Y' "$lock_file") ))
if [ "$lock_age" -lt "$STALE_LOCK_SECONDS" ]; then
echo " ${name}: held open by $(basename "$lock_file") (${lock_age}s old)"
locked=0
else
echo " ${name}: ignoring stale lock $(basename "$lock_file") (${lock_age}s old > ${STALE_LOCK_SECONDS}s) — treating as abandoned"
fi
done
return "$locked"
}
# evict_dir <dir>
#
# Unlinks <dir>, or declines to and says why. Returns 0 only if the directory
# is actually gone.
#
# The caller has already established that <dir> is a candidate, which is not
# the same as establishing that it is still one at the instant of the unlink:
# a consumer publishes its reader marker whenever it starts a clone, and the
# `du` that measures the directory between those two points runs for seconds
# on a multi-GB tree. Unlinking under a live clone truncates it silently —
# `cp -al` never reports a subtree that was removed before it read the
# parent's listing (see cache-lib.sh's reader-marker section).
#
# So the directory is renamed aside first and only then re-examined, which is
# what makes the second look conclusive rather than merely closer to the
# unlink. It is publish-snapshot.sh's rotation, and it rests on the same
# ordering proof: a consumer publishes its marker BEFORE it resolves the
# source path, so one that resolved this directory did so before the rename
# and therefore published its marker before the scan below, which happens
# strictly after that rename. A consumer arriving after the rename cannot
# resolve the path at all and falls through to its own cold-start path — the
# same safe degrade the publisher's swap window already produces.
#
# The rename disturbs nothing already in flight: it unlinks no entry and
# leaves the source inode unchanged, which is precisely why the publish side
# can rotate a snapshot out from under a live reader. A declined eviction
# therefore costs a deferred eviction and nothing else.
#
# `.evicting-<name>-<pid>` is a contract name shared with daniel/gitdan's
# host-level arbiter (see the cross-repo contract at the top of cache-lib.sh).
# The sweep at the top of this script reclaims these on every pass, so the only
# one that reaches the arbiter belongs to a repo whose workflow has stopped
# running — which is exactly the case no in-workflow pass can reach.
evict_dir() {
local dir="$1" name aside
name=$(basename "$dir")
aside="${ROOT}/.evicting-${name}-$$"
mv -T "$dir" "$aside" 2>/dev/null || {
echo " ${name}: could not be set aside for eviction — skipped this pass"
return 1
}
if is_locked "$aside" "$name"; then
if mv -T "$aside" "$dir" 2>/dev/null; then
echo " ${name}: claimed by a job while its eviction was in flight — restored, not evicted"
elif [ -d "$aside" ]; then
echo "::warning::prune: ${name} was claimed mid-eviction and its own name is taken again — leaving $(basename "$aside") for a later pass to reclaim once its readers drain"
else
echo "::warning::prune: ${name} was claimed mid-eviction and is already gone — another pass reclaimed it after its readers drained"
fi
return 1
fi
rm -rf "$aside"
return 0
}
# Directories oldest-first, target-* before snapshot-* (see the header).
# The sort key is a rank digit followed by a zero-padded mtime, so the two
# groups sort as blocks rather than interleaving by age. `.cache-last-used`
# is the marker every run touches; a run that hits the cache for every crate
# may write nothing at all inside the tree, which would make a
# just-used directory look stale without it. Fall back to the directory's
# own mtime when the marker is missing (a partially-written directory from an
# interrupted run is still a valid, if less precise, "last touched" signal).
list_by_lru() {
local rank d ts
for rank in 0:target 1:snapshot; do
for d in "$ROOT"/"${rank#*:}"-*; do
[ -d "$d" ] || continue
if [ -e "$d/.cache-last-used" ]; then ts=$(stat -c '%Y' "$d/.cache-last-used" 2>/dev/null || echo 0)
else ts=$(stat -c '%Y' "$d" 2>/dev/null || echo 0); fi
printf '%s%012d\t%s\n' "${rank%%:*}" "$ts" "$d"
done
done | sort | cut -f2-
}
# Deferred reclamations from an earlier pass: a directory renamed aside for
# eviction that could not be unlinked, because a job claimed it inside the
# window and its own name was taken again before it could be restored — or
# whose run was killed between the rename and the unlink. Nothing below would
# ever see one: every pass globs target-*/snapshot-*, which a dotted name does
# not match. Left unswept it is permanently unreclaimable disk on the one
# volume whose entire problem is disk.
#
# TWO DISTINCT PROPERTIES HOLD HERE, and neither implies the other.
#
# No clone can be truncated by the unlink below, by construction: the aside
# name only comes into existence after the evicting pass's rename, so a
# consumer that resolved the directory published its marker strictly before
# that rename and therefore before the count below — it cannot be missed. A
# consumer that arrives later cannot resolve the path at all. This holds under
# any interleaving and needs no settle window.
#
# No pass mid-eviction is mistaken for a leftover, by bound rather than by
# construction, and this is what the settle window is for. Without it a
# concurrent pass could unlink an aside its owner is still deciding about, and
# `rm -rf` traverses fd-relative: the owner's restore can then republish a
# half-emptied tree under a live cache name. What the window guarantees is that
# the two cannot be confused within it. What it does not guarantee is the
# pathological case beyond it — an evicting pass suspended past the window and
# then resumed finds its aside reclaimed and fails its restore, saying so.
now=$(date +%s)
for aside in "$ROOT"/.evicting-*; do
[ -d "$aside" ] || continue
# Change time, not modification time. `mv` leaves a directory's mtime alone
# (a cache last written days ago keeps a days-old mtime, which is what
# list_by_lru wants and exactly the wrong signal here) but rename(2) does
# update ctime, so %Z is when this directory was set aside and %Y is not.
aside_age=$(( now - $(stat -c '%Z' "$aside" 2>/dev/null || echo "$now") ))
if [ "$aside_age" -lt "$EVICTION_ASIDE_SETTLE_SECONDS" ]; then
echo "prune: $(basename "$aside") was set aside ${aside_age}s ago — another pass may still be evicting it, leaving it alone"
continue
fi
aside_name=$(basename "$aside"); aside_name="${aside_name#.evicting-}"; aside_name="${aside_name%-*}"
if [ "$(live_reader_count "$ROOT" "$aside_name")" -gt 0 ]; then
echo "prune: $(basename "$aside") is still being read — deferring its reclamation again"
continue
fi
echo "prune: reclaiming deferred eviction $(basename "$aside")"
rm -rf "$aside"
done
echo "=== pass 1: liveness (unconditional, not gated on free space) ==="
declare -A live_ns=()
# The tip SHA origin reports for the branch each directory name belongs to.
# Same output, same loop, one field over — reading it from a second `git` call
# would be reading a different instant.
declare -A live_tip=()
declare -A remote_tip_of=()
LIVENESS_AVAILABLE=0
LIVENESS_REASON=""
if [ "${CACHE_LIVENESS:-true}" = "0" ] || [ "${CACHE_LIVENESS:-true}" = "false" ]; then
LIVENESS_REASON="disabled via CACHE_LIVENESS=${CACHE_LIVENESS}"
elif remote_heads=$(timeout 20 git ls-remote --heads origin 2>&1); then
LIVENESS_AVAILABLE=1
branch_count=0
while IFS= read -r line; do
[ -n "$line" ] || continue
case "$line" in *refs/heads/*) ;; *) continue ;; esac
branch="${line#*refs/heads/}"
[ -n "$branch" ] || continue
sha="${line%%[[:space:]]*}"
suffix=$(cache_key "$branch")
live_ns["target-${suffix}"]=1
live_ns["snapshot-${suffix}"]=1
live_tip["target-${suffix}"]="$sha"
live_tip["snapshot-${suffix}"]="$sha"
remote_tip_of["$branch"]="$sha"
branch_count=$((branch_count + 1))
done <<< "$remote_heads"
echo "liveness: ${branch_count} live branches on origin"
else
LIVENESS_REASON="git ls-remote --heads origin failed or timed out"
fi
# The merged half of pass 1, and whether this checkout can answer it at all.
# Every branch of this decision that ends in "no" ends in the signal being
# WITHHELD, never in a directory being classified dead by default.
MERGED_AVAILABLE=0
MERGED_REASON=""
PROTECTED_TIPS=()
declare -A merged_verdict=()
declare -A merged_into=()
MERGED_INTO=""
if [ "$LIVENESS_AVAILABLE" = "1" ]; then
if ! git rev-parse --git-dir >/dev/null 2>&1; then
MERGED_REASON="not inside a git checkout"
elif [ "$(git rev-parse --is-shallow-repository 2>/dev/null || echo unknown)" != "false" ]; then
# In a shallow clone an absent commit is the normal case, so `--is-ancestor`
# answers about the graph that was fetched rather than the one that exists.
MERGED_REASON="the checkout is shallow, so a commit missing from it says nothing about ancestry"
else
for ref in $PROTECTED_REFS; do
tip="${remote_tip_of[$ref]:-}"
[ -n "$tip" ] || continue
git cat-file -e "${tip}^{commit}" 2>/dev/null || continue
PROTECTED_TIPS+=("${ref}:${tip}")
done
if [ "${#PROTECTED_TIPS[@]}" -gt 0 ]; then
MERGED_AVAILABLE=1
echo "liveness: merged-branch detection anchored on ${PROTECTED_TIPS[*]%%:*}"
else
MERGED_REASON="none of the protected branch tips (${PROTECTED_REFS:-none configured}) is present in this checkout"
fi
fi
[ "$MERGED_AVAILABLE" = "1" ] || \
echo "::warning::liveness: ${MERGED_REASON} — merged-but-undeleted branches keep their caches this run"
fi
# is_merged_dead <dir-name>
#
# True when the branch this directory belongs to is still on origin but every
# commit it holds is already on a protected branch — a merged PR whose branch
# the forge did not delete, which the deleted-branch signal above can never
# see. Enabling delete-on-merge narrows this to the branches merged before it
# was enabled, the ones its deletion declines (protected, or used by another
# open PR), and the merges that never ask (an API merge without the flag);
# see README's eviction section.
#
# Memoised per tip because target-<key> and snapshot-<key> share one branch,
# and because the "cannot tell" warning belongs to the branch rather than to
# each of its directories.
is_merged_dead() {
local name="$1" tip="${live_tip[$1]:-}" entry ref psha
MERGED_INTO=""
[ "$MERGED_AVAILABLE" = "1" ] || return 1
[ -n "$tip" ] || return 1
case "${merged_verdict[$tip]:-}" in
dead) MERGED_INTO="${merged_into[$tip]}"; return 0 ;;
live|unknown) return 1 ;;
esac
if ! git cat-file -e "${tip}^{commit}" 2>/dev/null; then
merged_verdict["$tip"]=unknown
echo "::warning::prune: ${name}: its branch tip ${tip} is not in this checkout — cannot tell merged from live, keeping it"
return 1
fi
for entry in "${PROTECTED_TIPS[@]}"; do
ref="${entry%%:*}"; psha="${entry#*:}"
if git merge-base --is-ancestor "$tip" "$psha" 2>/dev/null; then
merged_verdict["$tip"]=dead
merged_into["$tip"]="$ref"
MERGED_INTO="$ref"
return 0
fi
done
merged_verdict["$tip"]=live
return 1
}
if [ "$LIVENESS_AVAILABLE" = "1" ]; then
pruned_any=0
# Tracked separately so the line below cannot contradict the decline lines
# above it: "none pruned" and "none found" are different outcomes, and a
# pass that declined every dead cache it found has not found none.
spared_any=0
for dir in "$ROOT"/target-* "$ROOT"/snapshot-*; do
[ -d "$dir" ] || continue
name=$(basename "$dir")
is_protected "$dir" && continue
if [ -z "${live_ns[$name]:-}" ]; then
why="no matching branch on origin"
why_summary="branch no longer exists on origin"
elif is_merged_dead "$name"; then
why="merged into ${MERGED_INTO}, whose tip already contains its every commit"
why_summary="merged into \`${MERGED_INTO}\`"
else
continue
fi
is_locked "$dir" && { spared_any=1; continue; }
dir_gb=$(usage_gb "$dir")
evict_dir "$dir" || { spared_any=1; continue; }
echo "::warning::pruned dead-branch cache ${name} (${dir_gb} GB) — ${why}"
summary_line "- pruned dead-branch cache \`${name}\` (${dir_gb} GB) — ${why_summary}"
pruned_any=1
done
if [ "$pruned_any" = "0" ]; then
if [ "$spared_any" = "1" ]; then
echo "every dead-branch cache found is still in use — none pruned this pass"
else
echo "no dead-branch caches found"
fi
fi
else
echo "liveness: ${LIVENESS_REASON} — treating as UNAVAILABLE (not as \"no branches\"); pass 1 skipped"
fi
echo
echo "=== pass 2/3: disk pressure ==="
read -r TOTAL_KB FREE_KB <<< "$(read_df "$ROOT")"
PCT_KB=$(( TOTAL_KB * MIN_FREE_PCT / 100 ))
# The two floors, and which of them governs. They are kept apart all the way
# down rather than collapsed here, because falling short of them means
# different things: the derived one predicts that the very next step cannot
# finish, and the percentage one is a hygiene target for the volume.
REQUIRED_KB="$CLONE_KB"
GOVERNS="the clone this run is about to make"
if [ "$PCT_KB" -gt "$REQUIRED_KB" ]; then
REQUIRED_KB="$PCT_KB"
GOVERNS="the ${MIN_FREE_PCT}% floor"
fi
echo "required: $(gb "$REQUIRED_KB") GB free — ${GOVERNS} (clone $(gb "$CLONE_KB") GB, floor $(gb "$PCT_KB") GB)"
own_report() {
if [ -d "$OWN_DIR" ]; then
echo "cache: $(basename "$OWN_DIR") $(usage_gb "$OWN_DIR") GB | $(report_df host "$1" "$2")"
else
# Ordinary now that this pass runs ahead of the seed: on a branch's first
# run of the day the directory does not exist yet, and reporting 0.0 GB
# for it would read as an emptied cache.
echo "cache: $(basename "$OWN_DIR") not seeded yet | $(report_df host "$1" "$2")"
fi
}
if [ "$FREE_KB" -ge "$REQUIRED_KB" ]; then
own_report "$FREE_KB" "$TOTAL_KB"
exit 0
fi
echo "::warning::$(report_df disk "$FREE_KB" "$TOTAL_KB") < the $(gb "$REQUIRED_KB") GB this run requires"
# What survived the pass, and why, in the order the pass considered them. Read
# only by the failure report at the bottom: a run that cannot fit its clone is
# a run whose log has to answer "then what is all that space?" without anyone
# having to reconstruct the pass by hand.
KEPT=()
# A plain loop over a pre-materialised, pre-sorted list rather than a live
# `find | while` pipeline, so `rm -rf` inside the loop cannot make a running
# directory walk re-observe its own deletions.
mapfile -t LRU < <(list_by_lru)
for dir in "${LRU[@]}"; do
[ -d "$dir" ] || continue
if reason=$(protected_reason "$dir"); then
KEPT+=("$(basename "$dir")${reason}")
continue
fi
read -r TOTAL_KB FREE_KB <<< "$(read_df "$ROOT")"
[ "$FREE_KB" -ge "$REQUIRED_KB" ] && break
if is_locked "$dir"; then
KEPT+=("$(basename "$dir") — held open by a running job")
continue
fi
dir_gb=$(usage_gb "$dir")
if ! evict_dir "$dir"; then
KEPT+=("$(basename "$dir") — claimed by a job while its eviction was in flight")
continue
fi
echo "::warning::evicted $(basename "$dir") (${dir_gb} GB, LRU under disk pressure)"
summary_line "- evicted \`$(basename "$dir")\` (${dir_gb} GB, LRU under disk pressure)"
done
read -r TOTAL_KB FREE_KB <<< "$(read_df "$ROOT")"
# Falling short of the DERIVED requirement is a failure, not a warning. The
# seed step is next, it will clone that source, and it will run out of disk
# part-way through unsharing the clone's mutable paths — reported against a
# staging path, with nothing in the message about which cache was holding the
# space. Failing here says that instead.
#
# There is nothing to self-clear on this path and it is not skipped in error:
# a non-zero requirement means the seed is about to CLONE, which means this
# run has no own target dir to wipe (seed_clone_source returns nothing when it
# does), so pass 3 has no candidate. See the header.
if [ "$FREE_KB" -lt "$CLONE_KB" ]; then
echo "::error::prune: $(gb "$FREE_KB") GB free after evicting every eligible cache, but seeding from $(basename "$SEED_SRC") needs $(gb "$CLONE_KB") GB — short by $(gb "$(( CLONE_KB - FREE_KB ))") GB"
summary_line "- **out of disk**: seeding from \`$(basename "$SEED_SRC")\` needs $(gb "$CLONE_KB") GB, $(gb "$FREE_KB") GB free"
echo "prune: kept, and why:"
for entry in ${KEPT[@]+"${KEPT[@]}"}; do echo " ${entry}"; done
[ "${#KEPT[@]}" -gt 0 ] || echo " (nothing — the volume holds no cache directories at all)"
exit 1
fi
if [ "$FREE_KB" -lt "$PCT_KB" ]; then
OWN_GB=$(usage_gb "$OWN_DIR")
echo "::warning::still under threshold after evicting every eligible sibling; clearing own $(basename "$OWN_DIR") (was ${OWN_GB} GB) — this run pays a cold rebuild"
summary_line "- **self-clear**: \`$(basename "$OWN_DIR")\` (was ${OWN_GB} GB) wiped — this run pays a cold rebuild"
# Recreated empty rather than left absent, and that is what keeps this path
# out of the requirement above: the seed reuses an own target dir that
# exists, whatever is in it, so a self-cleared run clones nothing and needs
# no headroom. Leaving it absent would send that run to the base snapshot
# instead, needing a clone this pass has just spent its last eligible bytes
# not sizing for.
rm -rf "$OWN_DIR"
mkdir -p "$OWN_DIR"
else
echo "$(report_df post-eviction "$FREE_KB" "$TOTAL_KB") — sibling eviction recovered enough space"
fi