daniel/gitdan-actions
1
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request 'test(seed): pin the two unguarded terms of the torn-clone condition' (#9) from test/pin-clone-guards into main

Browse Source
This commit was merged in pull request #9.
This commit is contained in:
claude
2026-08-24 19:49:20 +00:00
parent 68da3e69a3 31b4113a26
commit a9e9190e5a
2 changed files with 267 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+29 -11
View File
@@ -422,7 +422,7 @@ bash scripts/selftest.sh --fast # fixture-only suites, no compiler
| suite | covers |
|---|---|
| `hardlink-clone-selftest.sh` | that a build in a clone cannot mutate its source — with a control proving a raw `cp -al` does. Needs a real compiler. |
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and a seed whose source is rotated — or silently loses a subtree — underneath its clone**: the two ways a hardlink clone tears |
| `seed-target-dir-selftest.sh` | seed-source preference, lock-file stripping, two jobs racing on one cache key, **and one scenario per check a hardlink clone is validated against**: a source rotated wholesale, a subtree silently lost from the walk, a copy that reports failure over a tree both other checks read as whole, and a source identity that resolved at neither end — plus a staging tree that could not be privately owned being discarded rather than published |
| `publish-snapshot-selftest.sh` | the atomic swap, that a live consumer survives a republish, and the publisher's side of the rotation race: deferred reclamation under a live reader, and its sweep once the reader is gone |
| `prune-cache-selftest.sh` | liveness, protection, locking, eviction order, self-clear, **and that a cache a job claims *inside* the check-to-unlink window survives it** — against a real scratch `origin` |
| `restore-mtimes-selftest.sh` | the merge hazard and the watermark that closes it, including the two-jobs-one-namespace case. Needs a real compiler. |
@@ -442,13 +442,20 @@ key and asserts only what must be true whichever of them wins the rename.
which places the interference inside the window rather than hoping it lands
there. `prune-cache-selftest.sh` scenario 12 stubs `du`, so the pass's own
measurement publishes a reader marker strictly between its check and its
unlink; `seed-target-dir-selftest.sh` scenarios 8a and 8b stub `cp`, so the
consumer's own clone is what rotates the snapshot underneath it, or what
loses a subtree of its own source, strictly inside the identity window. 8a
does start a second real process — the actual `publish-snapshot.sh` — but the
stub is what fixes where its swap lands; the concurrency is incidental to the
determinism. Every stub asserts that it fired, because a scenario whose
interference silently did not happen passes for the wrong reason.
unlink. `seed-target-dir-selftest.sh` uses the shape four times over, on two
different commands. Scenarios 8a, 8b and 8c stub `cp`, so the consumer's own
clone is what rotates the snapshot underneath it, loses a subtree of its own
source, or reports a failure over a tree that is in fact whole — each strictly
inside the window that clone's checks cover; scenario 10 stubs `cp` one level
down instead, refusing the per-file copies that unshare the mutable paths.
Scenario 8d stubs `stat`, because the check it pins fires on an identity that
could not be READ rather than on one that changed, and the only way to make
that the sole witness is to fail the identity reads while the copy between
them succeeds. 8a also starts a second real process — the actual
`publish-snapshot.sh` — but the stub is what fixes where its swap lands; the
concurrency is incidental to the determinism. Every stub asserts that it
fired, because a scenario whose interference silently did not happen passes
for the wrong reason.
**A synthetic stand-in for the other side, where that artefact *is* the
contract.** `publish-snapshot-selftest.sh` scenarios 6 to 8 hold a
@@ -465,9 +472,20 @@ does it any more.
Where more than one guard could catch a fault, a scenario should assert
*which* one did — otherwise deleting the guard under test leaves the suite
green because a sibling fires in its place. Scenarios 8a and 8b of the seed
suite do; scenario 9 of the same suite does not yet, which is why a mutation
survives it (issue #5).
green because a sibling fires in its place. Scenarios 8a to 8d and 10 of the
seed suite do, and each is reddened by exactly one mutation of the clone's
checks. Scenario 9 does not, and a mutation still survives it: with its source
unreadable, `cp -al` leaves the staging directory at mode `000`, so deleting
the exit-status check does not change the outcome — the unshare pass aborts
the clone instead, and the assertion is satisfied down a path it was not
written for.
That is the standing hazard here, and it is not hypothetical. Two guards that
can each catch the same fault mask each other, so **neither** is individually
necessary and no fixture built around that fault can pin either one — which is
how both `[ "$cp_rc" -eq 0 ]` and `[ "$i_before" != missing ]` sat unpinned
(issue #5) while looking well covered. Isolating a check means constructing
the state only it can see, not the state that trips several at once.
The action YAML holds no logic beyond wiring; everything testable lives in
`scripts/`. A composite action needs `shell: bash` on every `run:` step, and
scripts/seed-target-dir-selftest.sh
+238 -11
View File
@@ -44,8 +44,19 @@
# never changes. Against the unguarded version this is a partial tree
# renamed into place and reported as a success — 20,328 of 48,805
# entries, `seed: cloned in 1s`, exit 0, seeded-from=base-snapshot.
# Both scenarios force their interleaving rather than racing for it, and
# each asserts WHICH check caught the tear, so neither stays green if
# 8c. AND A COPY THAT SAYS SO ITSELF — the third witness, and the only one
# the tool volunteers: `cp -al` exiting non-zero over a tree that both
# other checks read as whole. Pins that a copy's own failure report is
# never overruled by two inferences that saw nothing.
# 8d. AND AN IDENTITY THAT COULD NOT BE READ AT ALL — the fourth. A failed
# identity read is reported as the string `missing`, so two of them
# compare equal to each other; without the term that rejects the
# sentinel, a clone whose source could not be identified at either end
# is published on the strength of two errors. A regression guard on a
# defensive term rather than a reproduction of a reachable state — see
# the scenario's own comment.
# All four scenarios force their interleaving rather than racing for it,
# and each asserts WHICH check caught the tear, so none stays green if
# the check it exercises is removed.
# 9. AN UNREADABLE SOURCE FAILS LOUDLY — the clone reports a distinct status
# instead of renaming whatever it managed to produce into place, and the
@@ -56,6 +67,13 @@
# (The publisher's half of the rotation race — deferring reclamation
# while a reader is still in flight — lives in
# publish-snapshot-selftest.sh, next to the swap it modifies.)
# 10. A TREE THAT CANNOT BE PRIVATELY OWNED IS DISCARDED — the other way a
# clone must refuse to publish, and the one that is not about tearing at
# all: if unsharing the mutable paths fails, the staging tree still
# aliases its source, so renaming it into place would wire two branches
# onto one set of fingerprints. That is the silent stale-reuse bug the
# whole scheme exists to prevent, so the failure has to abort the clone
# rather than be swallowed.
set -euo pipefail
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
. "$script_dir/cache-lib.sh"
@@ -92,6 +110,7 @@ seed() { bash "$script_dir/seed-target-dir.sh" "$@" > "$scratch/log" 2>&1 || { t
# that needs them, so either scenario can be run, moved or mutated alone.
mkdir -p "$scratch/bin"
real_cp=$(command -v cp)
real_stat=$(command -v stat)
seed_with_stub() { PATH="$scratch/bin:$PATH" bash "$script_dir/seed-target-dir.sh" "$@"; }
# Always succeeds and always prints a number: a directory that does not exist
@@ -115,17 +134,20 @@ wait_for_file() {
done
}
# Reads the clone's own torn-read report back and asserts WHICH of its three
# checks fired: the copy's exit status, the entry count, or the source's
# identity. Scenarios 8a and 8b each force exactly one of the three, and a
# scenario that only asserted "some tear was reported" would stay green if
# the check it exercises were deleted and one of the others happened to fire
# in its place.
# Reads the clone's own torn-read report back and asserts WHICH of its four
# checks fired: the copy's exit status, the entry count, the source's identity,
# or that identity being unreadable at all. Scenarios 8a to 8d each force
# exactly one of the four, and a scenario that only asserted "some tear was
# reported" would stay green if the check it exercises were deleted and one of
# the others happened to fire in its place.
#
# assert_tear <log> <cp-status> <same|short> <same|differs> <message>
# assert_tear <log> <cp-status> <same|short> <same|differs|unreadable> <msg>
assert_tear() {
local log="$1" want_rc="$2" want_count="$3" want_inode="$4" msg="$5" line
local re='cp rc=([0-9]+), ([0-9]+)/([0-9]+) entries, source inode ([0-9]+) -> ([0-9]+)'
# The identity fields are matched as `<digits>|missing` rather than as
# "anything up to the space", so a report whose inode field is neither is a
# loud parse failure instead of a silently-compared string.
local re='cp rc=([0-9]+), ([0-9]+)/([0-9]+) entries, source inode ([0-9]+|missing) -> ([0-9]+|missing)'
# `|| line=""` rather than a bare assignment: no match makes grep exit 1,
# which under `set -e` would abort the suite with no message at all — the
# exact case this assertion exists to report.
@@ -139,8 +161,14 @@ assert_tear() {
*) fail "assert_tear: bad entry-count expectation '${want_count}'" ;;
esac
case "$want_inode" in
same) [ "${BASH_REMATCH[4]}" = "${BASH_REMATCH[5]}" ] || fail "expected the source's identity to hold: ${line}" ;;
# `same` demands a READ identity that held, not merely two equal strings:
# two failed reads are both `missing` and would otherwise satisfy it, which
# is the exact confusion scenario 8d exists to pin.
same) [ "${BASH_REMATCH[4]}" != missing ] || fail "expected the source's identity to be readable: ${line}"
[ "${BASH_REMATCH[4]}" = "${BASH_REMATCH[5]}" ] || fail "expected the source's identity to hold: ${line}" ;;
differs) [ "${BASH_REMATCH[4]}" != "${BASH_REMATCH[5]}" ] || fail "expected the source's identity to change: ${line}" ;;
unreadable) [ "${BASH_REMATCH[4]}" = missing ] && [ "${BASH_REMATCH[5]}" = missing ] \
|| fail "expected neither identity read to have resolved: ${line}" ;;
*) fail "assert_tear: bad identity expectation '${want_inode}'" ;;
esac
ok "$msg (${line})"
@@ -373,6 +401,143 @@ leftovers=$(find "$root" -maxdepth 1 \( -name '.stage-*' -o -name '.reading-*' \
[ -z "$leftovers" ] || fail "scratch left behind: ${leftovers}"
ok "no staging or reader-marker scratch left behind"
rm -f "$scratch/bin/cp"
echo
echo "=== 8c: a copy that reports failure over a tree that looks whole ==="
# The third witness, and the only one the tool volunteers rather than leaving
# to be inferred: `cp -al` exiting non-zero. What this pins is that a copy's
# own failure report is never overruled by the other two checks agreeing that
# the tree looks intact.
#
# Both of those are inferences with blind spots. The entry count is
# `find | wc -l` over two trees, so a failure below a directory NEITHER walk
# could enumerate moves it not at all, and an entry lost against one gained
# cancels out. The inode comparison sees only a source replaced wholesale. A
# `cp -a` that links every entry and still fails — it could not preserve a
# directory's ownership, say — is invisible to both, and that is the shape
# forced here: the report reads 0 entries short and one unchanged inode, so
# the exit status is the only witness there is.
#
# Forced with the PATH stub the suite already uses for 8a and 8b, on the
# consumer's own top-level clone: the whole tree really is copied, and then
# the failure is reported, once. The retry finds nothing wrong, so the
# scenario also pins that the seed RECOVERS rather than merely refusing —
# a status that is honoured but not survivable would fail every job whose
# runner hiccuped once.
CPSTAT=$(cache_key feat/cp-status)
CPSTAT_BASE=$(cache_key release/2)
make_wide_tree "$root/snapshot-$CPSTAT_BASE" cpstatgen 4
cpstat_entries=$(tree_entries "$root/snapshot-$CPSTAT_BASE")
cat > "$scratch/bin/cp" <<EOF
#!/usr/bin/env bash
# Fires once, and only on the consumer's own top-level hardlink clone. The
# real copy runs to completion first: the point is a status nothing else can
# see, so the staging tree it reports on must be whole.
if [ "\${@: -1}" = "$root/.stage-jobCpStat" ] && [ ! -e "$scratch/cp_reported_failure" ]; then
: > "$scratch/cp_reported_failure"
"$real_cp" "\$@"
echo "cp: failed to preserve ownership for '\${@: -1}': Operation not permitted" >&2
exit 1
fi
exec "$real_cp" "\$@"
EOF
chmod +x "$scratch/bin/cp"
rcCpStat=0
seed_with_stub "$CPSTAT" "$CPSTAT_BASE" "$root" jobCpStat > "$scratch/logCpStat" 2>&1 || rcCpStat=$?
[ -e "$scratch/cp_reported_failure" ] \
|| fail "the stubbed cp never fired: no copy reported failure, so this scenario proves nothing"
ok "a copy reported failure over a staging tree that was in fact complete"
[ "$rcCpStat" = "0" ] || { tail -40 "$scratch/logCpStat"; fail "the seed exited non-zero"; }
ok "the seed completed"
# `same` on both inferences is the whole point: neither of them saw anything,
# so deleting the exit-status check leaves nothing to report and this
# assertion is the one that goes red.
assert_tear "$scratch/logCpStat" 1 same same "the failure was caught by the copy's exit status alone"
cpstat_dir="$root/target-$CPSTAT"
[ "$(tree_entries "$cpstat_dir")" -eq "$cpstat_entries" ] \
|| fail "the seeded tree is short: $(tree_entries "$cpstat_dir") entries against the source's ${cpstat_entries}"
ok "the seed re-cloned and holds every entry the source has (${cpstat_entries})"
assert_content "$cpstat_dir/debug/.fingerprint/x/dep-lib-x" cpstatgen "the seeded tree is the source's content"
leftovers=$(find "$root" -maxdepth 1 \( -name '.stage-*' -o -name '.reading-*' \) -print)
[ -z "$leftovers" ] || fail "scratch left behind: ${leftovers}"
ok "no staging or reader-marker scratch left behind"
rm -f "$scratch/bin/cp"
echo
echo "=== 8d: an identity that could not be read at either end ==="
# The fourth check, `[ "$i_before" != missing ]`, and the one whose absence is
# hardest to see. `_dir_inode` folds every stat failure into the string
# `missing`, so two FAILED identity reads compare equal TO EACH OTHER: drop
# this check and a clone whose source could not be identified before or after
# the walk satisfies `i_before = i_after` on the strength of two errors, and
# is renamed into place having proven nothing about the tree it holds.
#
# It cannot be forced by taking the source away. A source that is really gone
# fails `cp -al` too, so the exit status would fire in this check's place and
# the scenario would pin 8c's property over again. What isolates it is the
# identity read failing while the copy SUCCEEDS — a transient stat error over
# a source that is otherwise perfectly readable — so the stub goes on `stat`
# rather than on the tree, and narrowly: only the `%i` reads of this clone's
# own source, which is the sole caller of `stat -c '%i'` on this path. The
# `%Y` reads the reader markers do are left alone.
#
# BOTH reads of the attempt have to fail. If only one did, the surviving one
# would differ from `missing` and the identity COMPARISON would become the
# witness instead — 8a's property, not this one. The assertion that the stub
# fired is therefore a count rather than a flag.
#
# So be clear about what this scenario is. It is NOT a reproduction of a state
# a CI job reaches: every route to it is closed off — a rotation hands the
# witness to 8a, and a source that is genuinely gone fails `cp -al` and hands
# it to 8c. It is a REGRESSION GUARD ON A DEFENSIVE TERM, and the thing it
# defends against is a sentinel comparing equal to itself, which is a property
# of the code rather than of the filesystem. That is worth pinning precisely
# because nothing else can reach it: a term no fixture exercises is the one a
# refactor drops without argument.
IDENT=$(cache_key feat/identity-unreadable)
IDENT_BASE=$(cache_key release/3)
make_wide_tree "$root/snapshot-$IDENT_BASE" identgen 4
ident_src="$root/snapshot-$IDENT_BASE"
ident_entries=$(tree_entries "$ident_src")
cat > "$scratch/bin/stat" <<EOF
#!/usr/bin/env bash
# Fails the first two identity reads of this clone's source — the pair that
# opens and closes one attempt's identity window — and passes everything else
# through, including the reader markers' own \`%Y\` reads.
if [ "\$1" = "-c" ] && [ "\$2" = "%i" ] && [ "\$3" = "$ident_src" ] \
&& [ "\$(wc -c < "$scratch/identity_reads_failed" 2>/dev/null || echo 0)" -lt 2 ]; then
printf 'x' >> "$scratch/identity_reads_failed"
echo "stat: cannot statx '\$3': Input/output error" >&2
exit 1
fi
exec "$real_stat" "\$@"
EOF
chmod +x "$scratch/bin/stat"
rcIdent=0
seed_with_stub "$IDENT" "$IDENT_BASE" "$root" jobIdent > "$scratch/logIdent" 2>&1 || rcIdent=$?
ident_failures=$(wc -c < "$scratch/identity_reads_failed" 2>/dev/null || echo 0)
[ "$ident_failures" -eq 2 ] \
|| fail "expected both identity reads of one attempt to fail, got ${ident_failures} — a single failure would make the identity comparison the witness instead, which is scenario 8a's property"
ok "neither identity read of the first attempt resolved"
[ "$rcIdent" = "0" ] || { tail -40 "$scratch/logIdent"; fail "the seed exited non-zero"; }
ok "the seed completed"
assert_tear "$scratch/logIdent" 0 same unreadable "the unreadable identity was caught by the sentinel check alone"
ident_dir="$root/target-$IDENT"
[ "$(tree_entries "$ident_dir")" -eq "$ident_entries" ] \
|| fail "the seeded tree is short: $(tree_entries "$ident_dir") entries against the source's ${ident_entries}"
ok "the seed re-cloned once the identity was readable again (${ident_entries} entries)"
assert_content "$ident_dir/debug/.fingerprint/x/dep-lib-x" identgen "the seeded tree is the source's content"
leftovers=$(find "$root" -maxdepth 1 \( -name '.stage-*' -o -name '.reading-*' \) -print)
[ -z "$leftovers" ] || fail "scratch left behind: ${leftovers}"
ok "no staging or reader-marker scratch left behind"
rm -f "$scratch/bin/stat"
echo
echo "=== 9: a source that cannot be read fails loudly ==="
rc=0
@@ -408,5 +573,67 @@ else
assert_absent "$root/target-$VICTIM" "no target dir was left behind by the failed seed"
fi
echo
echo "=== 10: a staging tree that cannot be privately owned is never published ==="
# unshare_mutable_paths turns the hardlinked copies of Cargo's mutable
# metadata back into private inodes, and its failure status is load-bearing in
# a way no other check covers: a staging tree whose dep-info files still point
# at the SOURCE's inodes is not torn — every entry is present and the source
# never moved — so all four torn-clone checks pass it. Publish it anyway and
# this branch's build rewrites the base branch's dep-info, which is the silent
# stale-artifact reuse documented at length in cache-lib.sh. The only thing
# standing between that tree and DST is the status being propagated.
#
# Forced with the same PATH stub shape as 8a to 8d, on the narrowest possible
# target: `_unshare_files` copies each shared file as `cp -p -- <f>
# <f>.unshare.<pid>`, so refusing exactly the `.d` copies leaves the directory
# unshares (`cp -a`) and the clone itself (`cp -al`) untouched, and the
# failure that reaches the clone is unambiguously this one. The dep-info file
# has to live outside .fingerprint/ and build/, or it would be inside a
# subtree already replaced wholesale and never reach the per-file pass.
ALIAS=$(cache_key feat/unshare-fails)
ALIAS_BASE=$(cache_key release/4)
make_tree "$root/snapshot-$ALIAS_BASE" aliasgen
alias_dep="$root/snapshot-$ALIAS_BASE/debug/deps/libx.d"
echo aliasgen > "$alias_dep"
cat > "$scratch/bin/cp" <<EOF
#!/usr/bin/env bash
# Refuses the dep-info unshares and nothing else. Modelled on the failure that
# actually happens here — the cache volume filling up mid-clone.
if [ "\$1" = "-p" ] && [ "\$2" = "--" ] && [ "\${3%.d}" != "\$3" ]; then
: > "$scratch/depinfo_unshare_refused"
echo "cp: cannot create regular file '\$4': No space left on device" >&2
exit 1
fi
exec "$real_cp" "\$@"
EOF
chmod +x "$scratch/bin/cp"
rcAlias=0
seed_with_stub "$ALIAS" "$ALIAS_BASE" "$root" jobAlias > "$scratch/logAlias" 2>&1 || rcAlias=$?
[ -e "$scratch/depinfo_unshare_refused" ] \
|| fail "the stubbed cp never fired: no unshare was refused, so this scenario proves nothing"
ok "a dep-info unshare was refused inside the clone"
[ "$rcAlias" != "0" ] || { tail -40 "$scratch/logAlias"; fail "the seed reported success over a staging tree that still aliased its source"; }
ok "the seed exits non-zero when the staging tree cannot be privately owned"
# Which failure aborted the clone is asserted, not assumed: without this the
# scenario would stay green if the clone had failed for any other reason —
# the same wrong-reason pass that let a mutation survive scenario 9 (issue #5).
grep -q 'failed to unshare dep-info files' "$scratch/logAlias" \
|| { tail -20 "$scratch/logAlias"; fail "the dep-info unshare failure was not the one reported"; }
ok "the refused unshare is what was reported"
grep -q 'could not privately own the mutable paths' "$scratch/logAlias" \
|| { tail -20 "$scratch/logAlias"; fail "the clone did not report why it refused to publish"; }
ok "the clone names aliasing as its reason for refusing"
assert_absent "$root/target-$ALIAS" "nothing was renamed into place"
[ "$(stat -c '%h' "$alias_dep")" = "1" ] \
|| fail "the source's dep-info file is still hardlinked from somewhere: $(stat -c '%h' "$alias_dep") links"
ok "the discarded staging tree took its hardlinks to the source with it"
leftovers=$(find "$root" -maxdepth 1 \( -name '.stage-*' -o -name '.reading-*' \) -print)
[ -z "$leftovers" ] || fail "scratch left behind: ${leftovers}"
ok "no staging or reader-marker scratch left behind"
rm -f "$scratch/bin/cp"
echo
echo "seed-target-dir-selftest: ${pass_count} assertions passed"