docs(cache): write down the producer half of the leftover naming contract #8
@@ -246,7 +246,7 @@ block is the canonical description; this side owns the names.
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `.stage-<tag>` | `cache-lib.sh`, `hardlink_clone_into()` | stranded; only the arbiter reclaims it |
|
| `.stage-<tag>` | `cache-lib.sh`, `hardlink_clone_into()` | stranded; only the arbiter reclaims it |
|
||||||
| `.publish-new-<tag>` | `publish-snapshot.sh` | stranded; see `daniel/gitdan#30` |
|
| `.publish-new-<tag>` | `publish-snapshot.sh` | stranded; see `daniel/gitdan#30` |
|
||||||
| `.publish-old-<key>-<tag>` | `publish-snapshot.sh` | swept by the next publish of that key |
|
| `.publish-old-<key>-<tag>` | `publish-snapshot.sh` | swept by the next publish of that key, if there is one |
|
||||||
| `.evicting-<name>-<pid>` | `prune-cache.sh`, `evict_dir()` | swept at the start of the next prune pass |
|
| `.evicting-<name>-<pid>` | `prune-cache.sh`, `evict_dir()` | swept at the start of the next prune pass |
|
||||||
| `.reading-<source>-<tag>` | `cache-lib.sh`, `reader_lock_acquire()` | not garbage — see below |
|
| `.reading-<source>-<tag>` | `cache-lib.sh`, `reader_lock_acquire()` | not garbage — see below |
|
||||||
|
|
||||||
@@ -269,9 +269,11 @@ as `.stage-` does, so it is the one that needed catching. Bringing both under
|
|||||||
the arbiter's enumeration is tracked as `daniel/gitdan#30`.
|
the arbiter's enumeration is tracked as `daniel/gitdan#30`.
|
||||||
|
|
||||||
The two lists are meant to be the same length — the five names above, and the
|
The two lists are meant to be the same length — the five names above, and the
|
||||||
prefixes `ci-cache-reclaim.sh` enumerates. A mismatch means one side gained a
|
prefix constants `ci-cache-reclaim.sh` *declares*. Not the subset it enumerates
|
||||||
shape without telling the other, which is the drift the rule exists to catch
|
as reclaim candidates: that one is smaller, because `.reading-` is read and
|
||||||
and the cheapest thing to check.
|
never swept. A mismatch means one side gained a shape without telling the
|
||||||
|
other, which is the drift the rule exists to catch and the cheapest thing to
|
||||||
|
check.
|
||||||
|
|
||||||
**The staleness constants are part of the same contract, and that half has a
|
**The staleness constants are part of the same contract, and that half has a
|
||||||
direction.** `CACHE_READ_STALE_SECONDS` (`cache-lib.sh`) and
|
direction.** `CACHE_READ_STALE_SECONDS` (`cache-lib.sh`) and
|
||||||
|
|||||||
@@ -32,7 +32,9 @@
|
|||||||
# its clone and the swap. Strands the same way.
|
# its clone and the swap. Strands the same way.
|
||||||
# .publish-old-<key>-<tag> publish-snapshot.sh: the rotated-away generation,
|
# .publish-old-<key>-<tag> publish-snapshot.sh: the rotated-away generation,
|
||||||
# kept while a reader still holds it and swept by
|
# kept while a reader still holds it and swept by
|
||||||
# the next publish of the same key.
|
# the next publish of the same key — if there ever
|
||||||
|
# is one. A merged or renamed branch never
|
||||||
|
# publishes again, and its last one stays.
|
||||||
# .evicting-<name>-<pid> prune-cache.sh, evict_dir(): a cache renamed
|
# .evicting-<name>-<pid> prune-cache.sh, evict_dir(): a cache renamed
|
||||||
# aside so the decision to unlink it can be retaken
|
# aside so the decision to unlink it can be retaken
|
||||||
# after the rename. Swept at the start of every
|
# after the rename. Swept at the start of every
|
||||||
@@ -63,9 +65,11 @@
|
|||||||
# precisely the shape only the arbiter can reach. Bringing both in is tracked
|
# precisely the shape only the arbiter can reach. Bringing both in is tracked
|
||||||
# as daniel/gitdan#30.
|
# as daniel/gitdan#30.
|
||||||
#
|
#
|
||||||
# The two lists are meant to be the same length. A shape here without a prefix
|
# The two lists are meant to be the same length: the five names above, and the
|
||||||
# there is one side having changed without telling the other, and it is
|
# prefix constants that script DECLARES — not the subset it enumerates as
|
||||||
# cheapest to notice by counting.
|
# reclaim candidates, which is smaller because `.reading-` is read and never
|
||||||
|
# swept. A shape here without a constant there is one side having changed
|
||||||
|
# without telling the other, and it is cheapest to notice by counting.
|
||||||
#
|
#
|
||||||
# The two staleness constants the arbiter mirrors are part of the same
|
# The two staleness constants the arbiter mirrors are part of the same
|
||||||
# contract, and that half has a direction to it — see CACHE_READ_STALE_SECONDS
|
# contract, and that half has a direction to it — see CACHE_READ_STALE_SECONDS
|
||||||
|
|||||||
@@ -12,9 +12,11 @@
|
|||||||
# The swap is two renames, not one, because POSIX rename() can only replace an
|
# The swap is two renames, not one, because POSIX rename() can only replace an
|
||||||
# EMPTY directory and a snapshot from a prior publish is not one:
|
# EMPTY directory and a snapshot from a prior publish is not one:
|
||||||
#
|
#
|
||||||
# 1. stage the new snapshot at .stage-<tag> (copy time is off every
|
# 1. stage the new snapshot at .publish-new-<tag>, which hardlink_clone_into
|
||||||
# consumer's hot path — nothing reads a staging path);
|
# builds at its own .stage-<tag> and renames there (copy time is off
|
||||||
# 2. rename the current snapshot aside to .publish-old-<tag>, if present;
|
# every consumer's hot path — nothing reads either path);
|
||||||
|
# 2. rename the current snapshot aside to .publish-old-<key>-<tag>, if
|
||||||
|
# present;
|
||||||
# 3. rename the staged snapshot into place.
|
# 3. rename the staged snapshot into place.
|
||||||
#
|
#
|
||||||
# Step 3 is a single atomic rename onto a path now guaranteed absent, so it
|
# Step 3 is a single atomic rename onto a path now guaranteed absent, so it
|
||||||
@@ -60,8 +62,10 @@ DST=$(snapshot_dir_for "$ROOT" "$OWN_KEY")
|
|||||||
# per job per run, so a publisher killed between staging its snapshot and the
|
# per job per run, so a publisher killed between staging its snapshot and the
|
||||||
# swap below strands a full hardlink clone that no later run of this script
|
# swap below strands a full hardlink clone that no later run of this script
|
||||||
# will ever match. `.publish-old-` is milder — the sweep further down reclaims
|
# will ever match. `.publish-old-` is milder — the sweep further down reclaims
|
||||||
# it on the next publish of the same key. Bringing both under the arbiter's
|
# it on the next publish of the same key, if there ever is one; a key whose
|
||||||
# enumeration is tracked as daniel/gitdan#30.
|
# branch is merged, deleted or renamed is never published again, and its
|
||||||
|
# rotated generation stays until something outside this repo takes it.
|
||||||
|
# Bringing both under the arbiter's enumeration is tracked as daniel/gitdan#30.
|
||||||
OLD="${ROOT}/.publish-old-${OWN_KEY}-${TAG}"
|
OLD="${ROOT}/.publish-old-${OWN_KEY}-${TAG}"
|
||||||
SNAP_NAME=$(basename "$DST")
|
SNAP_NAME=$(basename "$DST")
|
||||||
GRACE="${CACHE_READ_GRACE_SECONDS}"
|
GRACE="${CACHE_READ_GRACE_SECONDS}"
|
||||||
@@ -97,6 +101,8 @@ start=$(date +%s)
|
|||||||
# the snapshot it just published — the same aliasing hazard the consume side
|
# the snapshot it just published — the same aliasing hazard the consume side
|
||||||
# closes, pointing the other way. The staging path is not the final name, so
|
# closes, pointing the other way. The staging path is not the final name, so
|
||||||
# `hardlink_clone_into`'s rename lands on DST only after OLD is out of the way.
|
# `hardlink_clone_into`'s rename lands on DST only after OLD is out of the way.
|
||||||
|
# `.publish-new-` is a contract name — see the note at OLD above before
|
||||||
|
# changing it.
|
||||||
TMP_DST="${ROOT}/.publish-new-${TAG}"
|
TMP_DST="${ROOT}/.publish-new-${TAG}"
|
||||||
rm -rf "$TMP_DST"
|
rm -rf "$TMP_DST"
|
||||||
hardlink_clone_into "$SRC" "$TMP_DST" "$TAG" || {
|
hardlink_clone_into "$SRC" "$TMP_DST" "$TAG" || {
|
||||||
|
|||||||
Reference in New Issue
Block a user