fix(ci): add a scheduled v1 sweep and lease every v1 push
CI / shellcheck + selftests (pull_request) Successful in 1m46s
CI / move v1 to main (pull_request) Skipped

The release guard in 17d87b0 was safe but not live. Gitea 1.27.2 calls
CancelPreviousJobsByJobConcurrency whenever a job's `needs` resolve
(services/actions/clear_tasks.go:91, models/actions/run_job.go:641), so
a job's place in the `release-tag-v1` group followed when its own
selftest finished, not merge order. A newer merge C2 finishing selftest
first queued behind the older C1, C1 cancelled it, saw tip = C2, and
deferred: nobody pushed, and if merges then stopped v1 stayed stale
indefinitely behind a Skipped and a Cancelled job. The "always catches
up once merges pause" claim in ci.yaml and README was false.

What now holds:

- release-sweep.yaml runs on `schedule` every 15 minutes, in its own
  workflow and concurrency group, so nothing in ci.yaml can cancel it.
  When v1 already covers main's tip it stops after a checkout and one
  merge-base. Otherwise it checks out the tip, runs the same shellcheck
  and selftest.sh as ci.yaml's selftest job, and tags the tip only if
  they pass; a failing main therefore turns the sweep red on every tick
  while v1 lags, which is #27's AC1 loud-failure half. It reads the tip
  itself because a scheduled run's github.sha is the CommitSHA recorded
  when the schedule was registered on the last push to main
  (services/actions/notifier_helper.go:569-580,
  services/actions/schedule_tasks.go:126-141), and ref is the default
  branch: schedules are registered only from it
  (notifier_helper.go:120, :531, :603-604). event_name is "schedule"
  (context.go:71 reads TriggerEvent, set at schedule_tasks.go:136).
  Cron is 5-field robfig in UTC (models/actions/schedule_spec.go:38-41).

- 15 minutes, not 10: the sweep is the fallback, not the release path,
  and every tick is a run on gitdan-ci's shared slots and a row in the
  Actions list. 96 no-op runs a day of a few seconds each is the cost;
  the lag bound it buys is one interval plus one selftest run.

- Both writers go through scripts/release-v1.sh and push with
  --force-with-lease=refs/tags/v1:<v1 as read>, so v1 cannot move
  backwards when the sweep and a merge job race. A lost lease re-reads
  v1: at or ahead of this run's gated commit is a clean skip (the other
  writer released something at least as new); still behind it is a
  retry leased on the new value, up to three attempts, since the other
  writer may have tagged an older commit and giving up there would leave
  v1 short of a commit this run did gate; anything else goes red. A
  rejection with v1 unmoved is diagnosed as a non-lease failure and goes
  red at once.

- release-tag loses its job-level concurrency group. The lease already
  gives the ordering the group was there for, and the group was what
  cancelled the one job that could have released the newest merge.
  Without it each merge's job runs, and the one whose commit is still
  the tip when it checks releases it.

The shell moves out of ci.yaml into scripts/release-v1.sh so shellcheck
and selftest.sh cover it. release-v1-selftest.sh runs it against a
scratch bare origin: sweep no-op at and ahead of the tip, tag on a
green gate, no tag and a failing sweep on a red one, the stranded trace
above followed by a catching-up sweep, and each lost-lease outcome, with
a control showing an unleased push does step v1 back. Red-proved by
seven mutations of release-v1.sh, each failing a named assertion: plain
--force, accepting any lost lease, a merge job that never defers,
ancestry reduced to equality, no non-lease diagnosis, a sweep that never
needs to run, and a retry that does not re-lease.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UkSjXXtU6JYcN2vPntWhfb
This commit is contained in:
2026-09-22 15:01:05 -05:00
co-authored by Claude Opus 5.5
parent 17d87b0647
commit ca0ee132d9
6 changed files with 371 additions and 106 deletions
+165
View File
@@ -0,0 +1,165 @@
#!/usr/bin/env bash
# Regression test for release-v1.sh against a scratch bare origin: v1 reaches
# main's tip once it has been gated, never lands on an ungated commit, and
# never moves backwards when two writers race (gitdan-actions#27).
#
# run_sweep mirrors release-sweep.yaml's step order -- check, gate only when
# needed, push the gated tip leased on the v1 the check read -- with the gate
# stood in for by a command, so a failing gate is a failing sweep.
set -euo pipefail
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
release="$script_dir/release-v1.sh"
scratch=$(mktemp -d)
trap 'rm -rf "$scratch"' EXIT
pass_count=0
fail() { echo "ASSERTION FAILED: $*" >&2; exit 1; }
ok() { pass_count=$((pass_count + 1)); echo "PASS: $*"; }
export GIT_AUTHOR_NAME=t GIT_AUTHOR_EMAIL=t@t GIT_COMMITTER_NAME=t GIT_COMMITTER_EMAIL=t@t
unset GITHUB_OUTPUT
# A fresh origin with main at one commit and v1 on it; dev pushes to main,
# ci and ci2 are the runners' clones.
fresh() {
rm -rf "$scratch/w"; mkdir -p "$scratch/w"
git init -q --bare "$scratch/w/origin.git"
git clone -q "$scratch/w/origin.git" "$scratch/w/dev" 2>/dev/null
commit_to_main >/dev/null
git -C "$scratch/w/dev" push -q origin HEAD:refs/tags/v1
git clone -q "$scratch/w/origin.git" "$scratch/w/ci"
git clone -q "$scratch/w/origin.git" "$scratch/w/ci2"
}
commit_to_main() {
git -C "$scratch/w/dev" commit -q --allow-empty -m "c$RANDOM"
git -C "$scratch/w/dev" push -q origin HEAD:refs/heads/main
git -C "$scratch/w/dev" rev-parse HEAD
}
origin_v1() { git -C "$scratch/w/origin.git" rev-parse -q --verify 'refs/tags/v1^{commit}' || true; }
origin_tip() { git -C "$scratch/w/origin.git" rev-parse refs/heads/main; }
in_ci() { (cd "$scratch/w/${CLONE:-ci}" && bash "$release" "$@"); }
field() { sed -n "s/^$1=//p"; }
run_sweep() {
local gate="$1" out tip v1
out=$(in_ci sweep-check)
[ "$(field needed <<<"$out")" = true ] || return 0
tip=$(field tip <<<"$out"); v1=$(field v1 <<<"$out")
"$gate" || return 1
in_ci push "$tip" "$v1"
}
echo "=== 1. sweep with v1 at the tip is a no-op ==="
fresh
before=$(origin_v1)
out=$(in_ci sweep-check)
[ "$(field needed <<<"$out")" = false ] || fail "a current v1 was reported as lagging"
run_sweep false || fail "a current v1 ran the gate"
[ "$(origin_v1)" = "$before" ] || fail "a no-op sweep moved v1"
ok "v1 == tip: needed=false, gate not run, v1 unchanged"
echo
echo "=== 2. sweep with v1 ahead of the tip is a no-op ==="
fresh
ahead=$(git -C "$scratch/w/dev" commit-tree -p HEAD -m ahead 'HEAD^{tree}')
git -C "$scratch/w/dev" push -q -f origin "$ahead:refs/tags/v1"
out=$(in_ci sweep-check)
[ "$(field needed <<<"$out")" = false ] || fail "a v1 descending from the tip was reported as lagging"
ok "v1 descends from tip: needed=false"
echo
echo "=== 3. sweep with v1 behind and a passing gate tags the tip ==="
fresh
tip=$(commit_to_main)
run_sweep true || fail "a passing sweep failed"
[ "$(origin_v1)" = "$tip" ] || fail "v1 is $(origin_v1), not the gated tip $tip"
ok "v1 behind, gate green: v1 -> tip"
echo
echo "=== 4. sweep with v1 behind and a failing gate goes red and tags nothing ==="
fresh
before=$(origin_v1)
commit_to_main >/dev/null
if run_sweep false; then fail "a sweep over a failing gate succeeded"; fi
[ "$(origin_v1)" = "$before" ] || fail "a failing gate still moved v1"
ok "v1 behind, gate red: sweep red, v1 unchanged"
echo
echo "=== 5. a lost lease to a newer writer is a clean skip, never a step back ==="
fresh
t1=$(commit_to_main)
out=$(in_ci sweep-check); v1_read=$(field v1 <<<"$out")
t2=$(commit_to_main)
CLONE=ci2 in_ci merge "$t2" >/dev/null
[ "$(origin_v1)" = "$t2" ] || fail "the merge job did not release its own tip"
in_ci push "$t1" "$v1_read" || fail "a lease lost to a newer v1 went red"
[ "$(origin_v1)" = "$t2" ] || fail "v1 went backwards from $t2 to $(origin_v1)"
ok "older writer lost the lease: exit 0, v1 stays at the newer $t2"
git -C "$scratch/w/ci" push -q -f origin "$t1:refs/tags/v1"
[ "$(origin_v1)" = "$t1" ] || fail "control: an unleased push did not step v1 back"
ok "control: the same push without the lease steps v1 back to $t1"
echo
echo "=== 6. a lease lost to an older writer retries and lands the newer commit ==="
fresh
t1=$(commit_to_main)
t2=$(commit_to_main)
out=$(in_ci sweep-check); v1_read=$(field v1 <<<"$out")
git -C "$scratch/w/dev" push -q -f origin "$t1:refs/tags/v1"
in_ci push "$t2" "$v1_read" || fail "a lease lost to an older v1 went red"
[ "$(origin_v1)" = "$t2" ] || fail "v1 is $(origin_v1), not $t2"
ok "v1 moved to an ancestor under us: retried, v1 -> $t2"
echo
echo "=== 7. a lease lost to an unrelated commit goes red ==="
fresh
tip=$(commit_to_main)
out=$(in_ci sweep-check); v1_read=$(field v1 <<<"$out")
stray=$(git -C "$scratch/w/dev" commit-tree -m stray 'HEAD^{tree}')
git -C "$scratch/w/dev" push -q -f origin "$stray:refs/tags/v1"
if in_ci push "$tip" "$v1_read" 2>/dev/null; then fail "a v1 moved sideways was accepted"; fi
[ "$(origin_v1)" = "$stray" ] || fail "the stray v1 was overwritten"
ok "v1 moved to a commit neither ahead nor behind: red, v1 untouched"
echo
echo "=== 8. a push rejected for another reason goes red ==="
fresh
tip=$(commit_to_main)
mkdir -p "$scratch/w/origin.git/hooks"
printf '#!/bin/sh\nexit 1\n' > "$scratch/w/origin.git/hooks/pre-receive"
chmod +x "$scratch/w/origin.git/hooks/pre-receive"
before=$(origin_v1)
if in_ci merge "$tip" 2>"$scratch/err"; then fail "a rejected push reported success"; fi
[ "$(origin_v1)" = "$before" ] || fail "v1 moved despite the rejection"
grep -q 'not a lost lease' "$scratch/err" || fail "the rejection was not diagnosed as one: $(cat "$scratch/err")"
ok "server rejection with v1 unmoved: red, not a lost lease"
echo
echo "=== 9. the merge job defers on a moved tip; the next sweep catches up ==="
# The stranded trace: C1's job runs after C2 merged and defers, C2's job was
# cancelled in the concurrency group, and merges stop.
fresh
before=$(origin_v1)
c1=$(commit_to_main)
c2=$(commit_to_main)
in_ci merge "$c1" >/dev/null || fail "the deferring merge job went red"
[ "$(origin_v1)" = "$before" ] || fail "the merge job released a commit that was not the tip"
run_sweep true || fail "the catch-up sweep failed"
[ "$(origin_v1)" = "$c2" ] || fail "v1 is $(origin_v1), not the tip $c2"
ok "C1 deferred, C2 never ran: the sweep moved v1 to $c2"
echo
echo "=== 10. the merge job releases its own tip, and creates a missing v1 ==="
fresh
tip=$(commit_to_main)
in_ci merge "$tip" >/dev/null
[ "$(origin_v1)" = "$tip" ] || fail "the merge job did not release the tip"
git -C "$scratch/w/dev" push -q origin :refs/tags/v1
tip=$(commit_to_main)
in_ci merge "$tip" >/dev/null
[ "$(origin_v1)" = "$tip" ] || fail "the merge job did not create an absent v1"
[ "$(origin_tip)" = "$tip" ] || fail "main moved"
ok "tip == gated sha: released, including onto an absent v1"
echo
echo "release-v1-selftest: all $pass_count assertions passed"