From 001e9002f085a7315966934b2a4e8cd9fd952d3a Mon Sep 17 00:00:00 2001 From: igneum-labs <337424239+igneum-labs@users.noreply.github.com> Date: Thu, 8 Oct 2026 16:25:54 +0000 Subject: [PATCH] Rule 24: a landing that touches a crate checks and tests it on the box before the merge tools/ci/rule24-crate-gate.sh maps every .rs, Cargo.toml, Cargo.lock, build.rs or .cargo/config in the diff to its crate (the nearest [package] Cargo.toml; the workspace root when none nearer; the core pair igneum-pow and app/igneum-app for a root config; a vendor crate named and left to its lane) and runs cargo check then the crate suite on the box at gate priority; docs/ and site/ alone (docs-only-check.sh) run nothing; any other diff takes the full gate alone. merge-to-master.sh runs it against the merge base before its merge loop; the hook runs it on a push of master or release-* against the remote tip. Self-test with a fixture repo and a recording runner: the crate map for each file class, documents-only runs nothing, check then test per crate, a red suite fails the gate. Class (main through the coordinator, 8 October 2026, 17:2x UK): master's igneum-pow stopped compiling at 17:07 under landings that never built it. Co-Authored-By: Claude Fable 5.1 --- tools/ci/README.md | 1 + tools/ci/checks.txt | 1 + tools/ci/merge-to-master.sh | 4 ++ tools/ci/pre-push.sh | 9 +++ tools/ci/rule24-crate-gate.sh | 102 ++++++++++++++++++++++++++++++++++ 5 files changed, 117 insertions(+) create mode 100755 tools/ci/rule24-crate-gate.sh diff --git a/tools/ci/README.md b/tools/ci/README.md index 4316d6241..37b0f4d3b 100644 --- a/tools/ci/README.md +++ b/tools/ci/README.md @@ -10,6 +10,7 @@ | the red watcher fires on cancelled and timed-out runs too (`ci-red.yml`, `red-watch.mjs`) | The watcher's `if` missing any of failure, cancelled, timed_out, or the conclusion not handed to the record step (the self-test reads the workflow file); the line names the kind: CI red, CI cancelled, CI timed out. | 7 October 2026 | | gh's active account is the stored Igneum entry (`gh-account-check.sh`, in Igneum's own gh directory `~/.config/gh-igneum` through `gh-env.sh`, never the founder's) | A push or a landing from this Mac while Igneum's gh directory names any other account as active, or none (the refusal names the one step: the founder or main stores the Igneum token there with `GH_CONFIG_DIR=~/.config/gh-igneum gh auth login --with-token`; no lane does); skipped with a line while `github-suspended` stands. RULE: no lane switches gh accounts on this Mac, ever; the second owner's login belongs to other projects and must never touch Igneum; the stored entry's name is in ~/.config/igneum/gh-user, never in the repository. | 7 October 2026, 21:41 UK: a lane switched gh to the other login during the suspension; nobody could say which | +| rule 24: a landing that touches a .rs, Cargo.toml, Cargo.lock, build.rs or .cargo/config checks and tests its crates on the box first (`rule24-crate-gate.sh`, called by `merge-to-master.sh` before the merge and by the hook on a push of master or release-*) | A touched crate that does not `cargo check` or whose suite is red on the box at gate priority; a file under no crate (the root .cargo/config) runs the core pair igneum-pow and app/igneum-app; a vendor crate is named and left to its lane; docs/ and site/ alone (docs-only-check.sh) run nothing; any other diff takes the full gate alone. Class: master's igneum-pow stopped compiling at 17:07 UK under landings that never built it | 8 Oct 2026 | | no landing on the public host while the marker stands (`pre-push.sh` `forgejo_master_frozen`, `merge-to-master.sh` `forgejo_master_refusal`); the hook binds every remote rule to the remote's URL | A push of master to git.igneum.network, or a `--remote` naming it, while `tools/ci/github-suspended` stands: its master is a rewritten copy replaced at cut-over, so the landing would be lost (a branch pushed there for safekeeping passes). Before the fix the hook matched the remote NAME, so `git push origin master` bound neither the GitHub refusal nor the CI rule; the self-test now drives the hook by name through a fixture repo. Also: `gate-manifest-check.sh` and four other pipefail checks no longer pipe a file-sized producer into `grep -q` (GNU sed took SIGPIPE on an early match and the check read it as a missing run line on the Linux runners and boxes); `mirror_master` fast-forwards every box with a build-server file after a landing on ANY remote (before, only a GitHub landing fanned out, so a box landing left build-3 and build-4 at a tip 23 hours old), as a `--no-verify` copy of the master the gate already passed (a stale mirror had re-run the full gate for six minutes per box) | 8 Oct 2026 | | kill by exact command or pid file (owed as a check) | 6 October 2026, 21:09Z: a Mac-side `pkill -f ` matched nothing (the log name was a redirect, not part of the command line), the roll-everything script lived on and wiped a box it had been told to hold. Rule: a job is stopped by its pid file (`tools/fleet/fleet-bg.sh start|stop `) or by a pattern anchored on its exact command line (`^python3 -u /root/fleet/in/box-prover.py`), never by a word that may or may not appear in it. The check that flags a `pkill -f`/`pgrep -f` whose literal is a path or a name that never starts a command line is owed to the CI lane | diff --git a/tools/ci/checks.txt b/tools/ci/checks.txt index 55da9c617..044ceb452 100644 --- a/tools/ci/checks.txt +++ b/tools/ci/checks.txt @@ -66,6 +66,7 @@ income per tier: the public table equals its inputs, the schedule arithmetic hash-origin report: a known-finished day and a known-failed day harness summaries never carry a raw 64-hex key (the writer's own redaction and check) docs-only pushes skip the compile-or-compute CI jobs (the changes job's classifier) +rule 24: a landing that touches a .rs, Cargo.toml, build.rs or .cargo file checks and tests its crates on the box first; docs and site alone skip it (self-test) the public ledger (docs/ledger-public.md) is what docs/fud-ledger.md generates: one row per item, no commit ids, times or team names (self-test first) the ledger page reads both entry heading forms (M1 and AP-F8-1) so no in-house pass row is dropped from /ledger (known-failed first) every workflow job carries timeout-minutes (site 15, changes 10, pow 60, sims 45; the hung-job class of 7 October 2026) diff --git a/tools/ci/merge-to-master.sh b/tools/ci/merge-to-master.sh index f85a70913..9da970893 100755 --- a/tools/ci/merge-to-master.sh +++ b/tools/ci/merge-to-master.sh @@ -203,6 +203,10 @@ else echo "merge-to-master: the remote $REMOTE is not GitHub (a mirror); the box gate stamp on ${SHA:0:8} is the verdict${IGNEUM_MASTER_EXCEPTION:+ (exception: $IGNEUM_MASTER_EXCEPTION)}" VERDICT="gate: green on ${SHA:0:8}, recorded by tools/ci/pre-push.sh; landed on the $REMOTE mirror${IGNEUM_MASTER_EXCEPTION:+ under the exception declared by main: $IGNEUM_MASTER_EXCEPTION}" fi +# rule 24 (8 October 2026, 17:2x UK): a diff that touches a .rs, Cargo.toml, Cargo.lock, build.rs or .cargo/config runs cargo check +# and the crate suite on the box at gate priority for every crate it touches before the merge; docs/ and site/ alone skip it +BASE=$(git merge-base "$SHA" "$REMOTE/master" 2>/dev/null || git rev-parse "$REMOTE/master") +bash tools/ci/rule24-crate-gate.sh "$BASE" "$SHA" || { echo "merge-to-master: REFUSED by rule 24: a touched crate does not check or its suite is red (above); fix on the branch and retry" >&2; exit 1; } for i in $(seq 1 "$TRIES"); do git fetch -q "$REMOTE" master; TIP=$(git rev-parse "$REMOTE/master") if git merge-base --is-ancestor "$SHA" "$TIP"; then echo "merge-to-master: ${SHA:0:8} is already on $REMOTE/master $(git log -1 --format=%h "$REMOTE/master")"; exit 0; fi diff --git a/tools/ci/pre-push.sh b/tools/ci/pre-push.sh index cfd968da5..e5ca10697 100755 --- a/tools/ci/pre-push.sh +++ b/tools/ci/pre-push.sh @@ -165,6 +165,7 @@ tree_checks() { run "hash-origin report: a known-finished day and a known-failed day" node --test tools/observer/hash-origin.test.mjs run "harness summaries never carry a raw 64-hex key (the writer's own redaction and check)" node infra/fast-time/lib/redact-keys.mjs --self-test run "docs-only pushes skip the compile-or-compute CI jobs (the changes job's classifier)" bash tools/ci/docs-only-check.sh --self-test + run "rule 24: a landing that touches a .rs, Cargo.toml, build.rs or .cargo file checks and tests its crates on the box first; docs and site alone skip it (self-test)" bash tools/ci/rule24-crate-gate.sh --self-test run "the public ledger (docs/ledger-public.md) is what docs/fud-ledger.md generates: one row per item, no commit ids, times or team names (self-test first)" bash -c 'node tools/ledger/export-public.mjs --self-test && node tools/ledger/export-public.mjs --check' run "the ledger page reads both entry heading forms (M1 and AP-F8-1) so no in-house pass row is dropped from /ledger (known-failed first)" node tools/ledger-page.mjs --self-test run "every workflow job carries timeout-minutes (site 15, changes 10, pow 60, sims 45; the hung-job class of 7 October 2026)" bash tools/ci/workflow-timeouts-check.sh --self-test @@ -397,6 +398,14 @@ FAKEGH defer*) echo "pre-push gate: a merge of green-stamped ${verdict#defer } onto the remote tip: the light gate here, the full gate in CI on landing:" structural_checks; never_push_checks; finish "merge of a green branch (full gate deferred to CI)" ;; *) echo "pre-push gate: a push to master or release-*, the full gate (the same checks CI runs)${verdict:+ ($verdict)}:" + # rule 24: the crates the push touches check and pass their suites on the box before the push (merge-to-master.sh runs the + # same before its merge; here for a direct push of master or a release line) + while read -r lref lsha rref rsha; do + case "$rref" in refs/heads/master|refs/heads/release-*) ;; *) continue ;; esac + [ "$lsha" != 0000000000000000000000000000000000000000 ] && [ "$rsha" != 0000000000000000000000000000000000000000 ] || continue + git cat-file -e "$rsha" 2>/dev/null || continue + bash "$GATE_ROOT/tools/ci/rule24-crate-gate.sh" "$rsha" "$lsha" || { echo "pre-push gate: REFUSED by rule 24: a touched crate does not check or its suite is red (above)" >&2; exit 1; } + done <<<"$REFS" STAMP=1; structural_checks; tree_checks; finish "push to master or release-*" ;; esac else diff --git a/tools/ci/rule24-crate-gate.sh b/tools/ci/rule24-crate-gate.sh new file mode 100755 index 000000000..cf96a948e --- /dev/null +++ b/tools/ci/rule24-crate-gate.sh @@ -0,0 +1,102 @@ +#!/usr/bin/env bash +# Rule 24 (main through the coordinator, 8 October 2026, 17:2x UK): a landing whose diff touches a .rs file, a Cargo.toml, a +# Cargo.lock, a build.rs or a .cargo/config runs `cargo check` and the crate's suite ON THE BOX (tools/build-remote.sh at gate +# priority) for every crate it touches, before the merge; the documents-only gate (docs-only-check.sh: every path under docs/ or +# site/) is the only diff that skips it; any other diff (tools, infra, workflows) takes the full gate alone, as before. +# The class behind it: master's igneum-pow stopped compiling at 17:07 UK under landings that never built it. +# +# tools/ci/rule24-crate-gate.sh [--dry] the crates the diff base..head touches, then check + test each on the box +# (--dry: print the plan, run nothing); exit 0 green or nothing to run, +# 1 on a red crate, 2 on a bad argument +# tools/ci/rule24-crate-gate.sh --self-test +# A file maps to the nearest ancestor directory holding a Cargo.toml with [package]; a file under a workspace root with no nearer +# package maps to that root (cargo runs the workspace). A crate under vendor/ is a fork crate with its own lane: named, never run +# here. RULE24_RUNNER swaps the runner (the self-test records calls); it receives . +set -euo pipefail +HERE="$(cd "$(dirname "$0")" && pwd -P)"; ME="$HERE/$(basename "$0")" # absolute: the self-test calls it from a fixture directory +code_file() { case "$1" in *.rs|*/Cargo.toml|Cargo.toml|*/Cargo.lock|Cargo.lock|*/build.rs|build.rs|*/.cargo/config|*/.cargo/config.toml|.cargo/config|.cargo/config.toml) return 0 ;; esac; return 1; } +crate_of() { # -> the crate dir ("." for the root) or "" when none + local d; d=$(dirname "$1"); local ws="" + while :; do + if [ -f "$d/Cargo.toml" ]; then + if grep -q '^\[package\]' "$d/Cargo.toml"; then printf '%s' "$d"; return; fi + [ -n "$ws" ] || grep -q '^\[workspace\]' "$d/Cargo.toml" && ws="${ws:-$d}" + fi + [ "$d" = . ] && break; d=$(dirname "$d") + done + printf '%s' "$ws" +} +plan() { # -> lines "crate " / "vendor " / "documents-only" / "no-crate" + local base="$1" head="$2" files f c seen="" any_code=0 + files=$(git diff --name-only "$base" "$head" --) || { echo "rule24: cannot diff $base..$head" >&2; return 2; } + if [ -z "$files" ]; then echo "no-crate"; return 0; fi + if [ "$(printf '%s\n' "$files" | bash "$HERE/docs-only-check.sh")" = "code=false" ]; then echo "documents-only"; return 0; fi + while IFS= read -r f; do + [ -n "$f" ] || continue; code_file "$f" || continue; any_code=1 + c=$(crate_of "$f") + if [ -z "$c" ]; then # a code-class file under no crate (the root .cargo/config): the core pair every cut pairs with + for c in ${RULE24_CORE_CRATES:-igneum-pow app/igneum-app}; do case " $seen " in *" $c "*) continue ;; esac; seen="$seen $c"; echo "crate $c"; done; continue + fi + case " $seen " in *" $c "*) continue ;; esac; seen="$seen $c" + case "$c" in vendor/*) echo "vendor $c" ;; *) echo "crate $c" ;; esac + done <<<"$files" + [ "$any_code" = 1 ] || echo "no-crate" +} +run_crate() { # + local dir="$1" what="$2" + if [ -n "${RULE24_RUNNER:-}" ]; then "$RULE24_RUNNER" "$dir" "$what"; return; fi + case "$what" in + check) ( cd "$dir" && IGNEUM_AGENT="${IGNEUM_AGENT:-rule24}" bash "$HERE/../build-remote.sh" --no-fetch --priority gate -- check ) ;; + test) ( cd "$dir" && IGNEUM_AGENT="${IGNEUM_AGENT:-rule24}" bash "$HERE/../build-remote.sh" --no-fetch --priority gate -- test --release ) ;; + esac +} +if [ "${1:-}" = --self-test ]; then + d=$(mktemp -d); trap 'rm -rf "$d"' EXIT; fails=0; calls="$d/calls"; : > "$calls" + runner="$d/runner.sh"; printf '#!/usr/bin/env bash\necho "$1 $2" >> %q\n[ "$1" = bad ] && [ "$2" = test ] && exit 1\nexit 0\n' "$calls" > "$runner"; chmod +x "$runner" + ( cd "$d" && git init -q -b master . && mkdir -p a/src bad/src docs site tools/ci vendor/f/src ws/m/src .cargo + printf '[package]\nname = "a"\nversion = "0.1.0"\n' > a/Cargo.toml; : > a/src/lib.rs + printf '[package]\nname = "bad"\nversion = "0.1.0"\n' > bad/Cargo.toml; : > bad/src/lib.rs + printf '[workspace]\nmembers = ["m"]\n' > ws/Cargo.toml; printf '[package]\nname = "m"\nversion = "0.1.0"\n' > ws/m/Cargo.toml; : > ws/m/src/lib.rs + printf '[package]\nname = "f"\nversion = "0.1.0"\n' > vendor/f/Cargo.toml; : > vendor/f/src/lib.rs + echo x > docs/x.md; echo y > site/y.html; echo z > tools/ci/z.sh; printf '[build]\n' > .cargo/config.toml + git add -A && git -c user.name=t -c user.email=t@t commit -q -m base && git tag base + echo 1 >> a/src/lib.rs && echo d >> docs/x.md && git -c user.name=t -c user.email=t@t commit -qam "a and a doc" && git tag t1 + echo d >> docs/x.md && echo s >> site/y.html && git -c user.name=t -c user.email=t@t commit -qam "docs and site" && git tag t2 + echo t >> tools/ci/z.sh && git -c user.name=t -c user.email=t@t commit -qam "a tool" && git tag t3 + echo 1 >> vendor/f/src/lib.rs && echo 1 >> bad/src/lib.rs && git -c user.name=t -c user.email=t@t commit -qam "a fork crate and a bad crate" && git tag t4 + printf 'x = 1\n' >> ws/Cargo.toml && git -c user.name=t -c user.email=t@t commit -qam "a workspace root file" && git tag t5 + printf '[build]\njobs = 2\n' > .cargo/config.toml && git -c user.name=t -c user.email=t@t commit -qam "cargo config" && git tag t6 ) >/dev/null 2>&1 + cd "$d" + [ "$(bash "$ME" base t1 --dry)" = "crate a" ] || { echo "self-test failed: a .rs change beside a doc did not name its crate: $(bash "$ME" base t1 --dry)"; fails=1; } + [ "$(bash "$ME" t1 t2 --dry)" = "documents-only" ] || { echo "self-test failed: docs and site alone were not documents-only: $(bash "$ME" t1 t2 --dry)"; fails=1; } + [ "$(bash "$ME" t2 t3 --dry)" = "no-crate" ] || { echo "self-test failed: a tools change was read as a crate: $(bash "$ME" t2 t3 --dry)"; fails=1; } + [ "$(bash "$ME" t3 t4 --dry | sort | tr '\n' ';')" = "crate bad;vendor vendor/f;" ] || { echo "self-test failed: the fork crate was not set aside or the bad crate not named: $(bash "$ME" t3 t4 --dry | tr '\n' ';')"; fails=1; } + [ "$(bash "$ME" t4 t5 --dry)" = "crate ws" ] || { echo "self-test failed: a workspace root file did not map to the workspace: $(bash "$ME" t4 t5 --dry)"; fails=1; } + [ "$(bash "$ME" t5 t6 --dry | tr '\n' ';')" = "crate igneum-pow;crate app/igneum-app;" ] || { echo "self-test failed: a root .cargo/config change did not map to the core pair: $(bash "$ME" t5 t6 --dry | tr '\n' ';')"; fails=1; } + : > "$calls"; RULE24_RUNNER="$runner" bash "$ME" base t1 >/dev/null 2>&1 || { echo "self-test failed: a green crate was red"; fails=1; } + [ "$(tr '\n' ';' < "$calls")" = "a check;a test;" ] || { echo "self-test failed: check then test were not run on the crate: $(tr '\n' ';' < "$calls")"; fails=1; } + : > "$calls"; RULE24_RUNNER="$runner" bash "$ME" t3 t4 >/dev/null 2>&1 && { echo "self-test failed: a red crate suite passed the gate"; fails=1; } + grep -q '^bad test$' "$calls" || { echo "self-test failed: the red crate's suite never ran"; fails=1; } + grep -q '^vendor' "$calls" && { echo "self-test failed: a fork crate was run here"; fails=1; } + : > "$calls"; RULE24_RUNNER="$runner" bash "$ME" t1 t2 >/dev/null 2>&1 || { echo "self-test failed: documents-only was red"; fails=1; } + [ ! -s "$calls" ] || { echo "self-test failed: documents-only ran a crate"; fails=1; } + [ "$fails" = 0 ] && echo "self-test passed: a .rs, Cargo.toml, build.rs or .cargo change names its crate (workspace root when no nearer package; the core pair for a root config; a vendor crate is set aside); docs and site alone are documents-only and run nothing; a tools change runs nothing; a touched crate gets cargo check then its suite on the box and a red suite fails the gate" + exit $fails +fi +[ $# -ge 2 ] || { echo "usage: rule24-crate-gate.sh [--dry] | --self-test" >&2; exit 2; } +BASE="$1"; HEAD="$2"; DRY=0; [ "${3:-}" = --dry ] && DRY=1 +rc=0; out=$(plan "$BASE" "$HEAD") || rc=$?; [ "$rc" = 0 ] || exit 2 +if [ "$DRY" = 1 ]; then printf '%s\n' "$out"; exit 0; fi +case "$out" in + documents-only) echo "rule24: documents-only (docs/ and site/ alone): no crate suite"; exit 0 ;; + no-crate) echo "rule24: no .rs, Cargo.toml, build.rs or .cargo change: the full gate alone"; exit 0 ;; +esac +red=0 +while read -r kind dir; do + [ -n "$kind" ] || continue + if [ "$kind" = vendor ]; then echo "rule24: $dir is a fork crate (its own lane's suites); not run here"; continue; fi + echo "rule24: $dir: cargo check on the box at gate priority"; run_crate "$dir" check || { echo "rule24: RED: $dir does not check" >&2; red=1; continue; } + echo "rule24: $dir: the crate suite on the box at gate priority"; run_crate "$dir" test || { echo "rule24: RED: $dir's suite" >&2; red=1; } +done <<<"$out" +[ "$red" = 0 ] && echo "rule24: every touched crate checks and its suite is green" +exit $red