The class (7 October 2026, 11:26 to 12:47 UK): three fork-gate summaries on ca3-v4-node carried the miners' vote-key hashes under "key" and eight CI runs went red on "no secret file names and no 64-hex secrets in the tree" while the pushing lanes saw nothing: the light gate ran only conflict markers and Windows paths.
- tools/ci/pre-push.sh: never_push_checks() (identity grep, no-secrets) runs on every ref from --hook, and inside the full gate where the identity grep already sat; the self-test asserts the wiring; the light gate is about 20 s on the Mac.
- infra/fast-time/lib/redact-keys.mjs: writeSummary() shortens every 64-hex value under a key-shaped field to 8 hex and an ellipsis and refuses a text the no-secrets rule would flag (line named); --self-test and --check; the gate runs the self-test. fork-gate.mjs adopts it on ca3-v4-node.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
the project lead, 7 October 2026 15:02 UK: build-1 at load 139 / 114 / 90 with both slots held and a 1 h 40 min queue while build-2 read 4.5 with
free slots, because the class router pinned each class to its box. Now lib.sh bs_route_spill reads the preferred box (free slots,
1-minute load) with one ssh and hands the job to the other box when the preferred one has no free slot or sits above load 64 and the
other qualifies; neither qualifying queues on the class's own box. The decision travels as BR_ROUTE_* into the JSONL "route" object
for the dashboard. build-2's slots file reads 3; everything on box 2 runs at nice 10 / 32 cores / -j 32, and a bounded run takes
the band its slot owns so three never share a core. Self-test tools/ci/route-spill-check.sh (thirteen cases) in the gate.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The dashboard lane, 7 October 2026 10:39Z: both slots of build-1 flock-held and EMPTY while two suites ran. Cause: a run from a
worktree without last night's append-mode fix opens a busy sibling's slot file with > on every probe. The holder now keeps its own
line: a keeper re-writes it within BR_KEEP_S (20 s) whenever the file is empty, until release; remote-run.sh --self-test-keeper
(in the gate) truncates a held line and sees it return, and sees nothing written after release; live on build-1 at 11:19Z (the
line came back in 25 s). The first version deadlocked the runner's bare wait with the keeper (build-2's first run hung 15 min
after its test passed): the keeper stops before the wait. The two running suites' -j 90 came from explicit --jobs 90: a bounded
class now clamps it to its cap with a log line (pass --priority gate for the full set). run-from-mac.sh --box N: the host file
was suffixed twice (build-server-2-2) and every box's mirror would have shared one remote name; one remote per box (build-N).
build-2's first green run: a suite at nice 10 on 32 cores, jobs 32, 986 s cold.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The build-server agent has igneum-build-2 and -3 on order (suites and benches; proving and the fast-time nodes). The
installer takes a host argument (N reads ~/.config/igneum/build-server-N, or build@<ip>); the pusher drops the Mac and
PC facts on every box it has a host file for, pulls each box's file and publishes boxes[] (box stays the first for the
old shape); the page draws one server section and one crew card per box, reads every box's Caddy feed in parallel
(build, build-2, build-3.igneum.network) with the edge copy filling any box that does not answer, and merges every
box's builds into the lanes, the timeline and the analytics. Nothing changes for build-1 until the host files exist.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Main's order of 7 October 2026 (igneum-build-2, AX162-1, and igneum-build-3, AX102-1, on order). lib.sh: bs_box_file N and
bs_route <class>; a class whose box has no host file yet falls back to box 1 and says so. run-from-mac.sh --box N <ip> provisions
igneum-build-N and writes build-server-N. tools/build-remote.sh --box N overrides the route; --priority gate always runs on box 1;
the proving crate routes to box 3. README.md: the kind map and the project lead's rule, no mining on any Hetzner box, ever (nodes, builds, tests,
benchmarks and CPU proving only; the pool's fast-time network mines on rented GPU pods, never on build-3). capacity/run.sh refuses a
job that would start igneum-miner mine or a GPU worker, whatever SEQUENCE says.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Main's order of 7 October 2026 after a load of 190 on 96 threads (a release join bench and the 0.3.19 app gate starving each
other, 'builds' of 16 minutes). tools/build-remote.sh resolves a class from the cargo subcommand and --priority: test and bench are
the bounded class (nice 10, the last 32 cores, -j 32) unless --priority gate (nice 0, the full set, the box's own jobs rule);
builds and checks are unchanged. remote-run.sh applies renice and taskset to the command's subshell, caps the jobs, lets a queued
gate (gate-pending-<pid>) take the next slot ahead of suites and benches, and prints nice and cores in the RESULT line and the
JSONL line (nice, cores, priority). The slot label carries '; kind=<k> nice=<n> cores=<c>' before '; agent=', so the dashboard's
job card shows why a job is slow. --plan prints the resolved class without the box; tools/ci/build-kind-default-check.sh (in the
gate) holds the five shapes. Smoke on the box: a suite at jobs=32 nice=10 cores=32, a gate at nice=0 cores=96.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
remote-run.sh checkout_tree ran `git clean -fd` on the box mirror before every build from any agent, so the attack rows
lost attack-f3/, attack-f1-venv/ and tools/attack/*/target to each other's builds (7 October 2026, 09:2x UK). The clean now
also spares the fixed prefixes attack-*, scratch-*, target-attack-*, .build-remote.log and every glob in the mirror-local
.igneum-scratch-spare (one per line, # comments, the file itself spared), keeps the target and stamp excludes and still runs
without -x. The clean-tree test asks `git clean -nd` with the same excludes instead of filtering the status list, so a spared
dir is not "not clean". --self-test: a fixed-prefix dir at the root and nested, a declared dir and the spare file survive, an
undeclared dir is removed. tools/ci/scratch-spare-check.sh in the pre-push gate fails when the clean line loses the spare
arguments, spare_args stops reading the file, a fixed prefix goes, or -x appears. docs/plans/build-server.md R4a says how a
lane declares its prefix.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
the project lead added the read-only deploy key on 7 October 2026 (SHA256:51ice3W8...). The sync still said 'mirror' because ssh -T to GitHub
exits 1 after its greeting and observer-sync.sh runs under pipefail, so su ... | grep -q reported failure although grep had
matched (the same line by hand, without pipefail, said authenticated; shown under the unit's environment with systemd-run -v).
The probe's output is captured first. First github pass 07:30:54 UTC: the clone moved from the mirror's d79f4a6 to origin/master
8b878ba4, the observer restarted on it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The shipper's read-back for 0.3.18 (7 October 2026): igneum_getNodeInfo exists again and must answer powEngine igneum-pow (a stub
is a FAIL) with a blockrate object, which the mover prints; a tree before 0.3.18 answers -32601 and the engine is still read from
the binary's igneum-pow source paths. The parser is checked against the three answer shapes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Node A mines past the pruning depth at the fast-time profile (finality 120, pruning 4,600), a fresh node B joins
by headers proof; PASS = B takes A's sink with the 1026 header version. --window sets the sampled difficulty
window in samples: 150 (the minimum, 600 DAA) by default; 661 (the devnet's, 2,644 DAA) is the known-failed
shape found on the first run (7 October 2026, 04:45 to 05:55 UK): B's walk through the sampled trusted blocks
ran out before genesis and "DAA window data has only 180 entries" ended the IBD, then the N6 guard banned the
syncer on each retry. The node fix is e3798a17 on ca3-v4-0318 (a young chain's short window is legal when the
walk reached the chain's start); the devnet and the testnet never meet the shape (pruning samples at multiples
of the 43,200 finality depth).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The b3c228fa builds under the 0.3.16 app tree 5d118f58 (no rust-toolchain.toml yet) died right after the pairing line with no
message: sed on the missing file failed, pipefail carried its status into the assignment, set -e ended the script. A guard and a
tolerant pipeline; checked alive against a tree without the file.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Three classes from the 0.3.17 night. (1) bash reads a script incrementally: tools/build-remote.sh was edited while a four-minute remote
build ran, the running copy continued at shifted bytes and died with a syntax error after the build had succeeded on the box; the
four long-running tools (build-remote, cross-remote, workers-remote, move-hand) now keep their body in one brace block ending in exit,
parsed whole before a line runs; tools/ci/whole-body-check.sh (in the gate, self-test with a block-less copy) holds the shape.
(2) A fork build pairs with the igneum-pow of the igneum worktree it sits in: a fork at 12153428 under a master worktree failed in
kaspa-pow four minutes in (no chain_program_shadow; master's igneum-pow predates release-0.3.17's); build-remote.sh says the
pairing on its first line ('pairs with igneum 6f8d7a7e (detached): igneum-pow 0.2.0') and the JSONL line carries pairs_with.
The first version of that line used '[ -n ... ] && echo' inside an assignment's $( ) and set -e ended the script on the false
status; fixed. (3) move-hand.sh restart <hand> [--digest <hex>] [--go]: after binary installed a release, restart ONE unit and read
it back (first exec line, commit string in the running binary, digest against the wanted one, igneum_getNodeInfo powEngine over the
node's loopback EVM RPC); the digest readers tolerate a missing line.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The same harness at 10, 20 and 30 s of propagation at 1 bps: at 10 s one shared DAG (254 blocks via relay, mergeset
3.4, red 36 percent, the controller within 1.1 percent); at 20 s under-merging (mergeset 1.6, the rate 19 percent
under, the difficulty 40 percent under); at 30 s nothing merges (mergeset 1.0, every node its own chain). At k = 18
the blocks in flight pass k between 10 and 20 s, k's own derivation. Fix shape, not built: an IBD trigger on a stale
sink (a node whose relayed blocks stop merging syncs the heavier chain instead of mining alone; the 30-s hub never
entered IBD) and k with the merge depth sized to the measured delay bound at the block rate. Owed: the long run's
decay (3 via relay in 40 minutes) with a kept log; the harness now takes --loglevel.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
7.2: the pipelined relay (fork bb397f99) at 10 bps and a 3-s propagation delay holds 10.01 blocks/s with the
difficulty 7.7 percent over the hash-implied value, the hub taking 7,746 blocks via relay against 4,015 via submit;
the same run at a batch of 1 (the 0.3.17 flow) takes 445 against 801 and reads 0.351. N3 closed in the ledger,
network.md 5.11 and CLAUDE.md's gate line. The 1 bps, 30-s case did not merge on either flow (3 blocks via relay in
40 minutes, no IBD, no error, one reconnect per spoke): at a 30-s delay the blocks in flight exceed the k bound, and
its cause is owed its own reading (7.3).
7.7: the N4 node half's gate (peer-drop.mjs): a still sink dropped its silent peer at 720 s and re-dialled at 750 s;
a moving sink kept the same peer through 155 sinks; the harness's failed shape fails.
7.8 and ledger N6: the stale-block nuisance guard (fork 319da644, f2b25fdf, a22cb261), the shipper's record (310 to
551 connections in an hour from four stale datadirs), the two halves and the knob; the gate (nuisance-peer.mjs, a
node mining stub-engine blocks with proof of work off and restarting on the shared rule with its datadir): the
sender half flat with the skip line, the known-failed shape climbing (3 connections, 2 refusals in 300 s), its
mirror failing, the receiver half flat with one refusal and the ten-minute ban. Also recorded: the vote-or-burn
harness's devnet finality object (--finality devnet) for the replay running to about 05:00Z.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Main's order of 7 October 2026 after RunPod's Ubuntu 22.04 canaries (glibc 2.35) refused the box's GLIBC_2.38 binaries and HiveOS
turned out Ubuntu 20.04 based (2.31). The table lives once, in tools/ci/glibc-ceiling-check.sh (--class, --ceiling-of; self-test
covers the table, an unknown class and a 2.34 need against hive); tools/build-remote.sh --ship hive|rig|seed|linux (default seed)
and tools/workers-remote.sh --class (default rig) build with zig at the class's glibc and check against it. provision.sh installs
docker.io (user build in the docker group) for the proof. Proof: fork 3bfe346f at class hive, igneumd and igneum-miner need
GLIBC_2.30; the workers at class rig need GLIBC_2.17; all four run in ubuntu:20.04 (ldd 2.31) and ubuntu:22.04 (ldd 2.35) and
print their version or usage lines (the OpenCL worker its own no-libOpenCL message, the binary running in a GPU-less container).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Section 7.5: five runs recorded. The first idled nothing and never asked the gate; the second found the depth gap
(fixed 3301cf32); the third found the extension shortcut (fixed aa0182aa: a candidate on the sink's chain is not let
through, the depth along its own chain decides); the fourth, on aa0182aa, PASSES (B 2,750 bps at the fork, 183 DAA
deep, heavier by 183, A kept its chain for 150 s and learned 177 of B's blocks, 268 refusal lines, one per refused
block); the known-failed case PASSES (gate off, A reorged onto a 2,083 bps key's 162-DAA chain 25 s after the heal);
the harness's failed shape FAILS as it must. Three harness faults fixed on the way (shared ports, an RPC call before
the socket opened, a reorg test by key instead of by chain) and one build fault (ledger N5, the stub-engine node).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The previous commit's plan edit aborted on a changed anchor, so section 5c was missing; written now with the public half
(ssh-ed25519 ... igneum-build-1 observer read-only) and the CI-runner row closed. observer-sync.sh ran git rev-parse as root
against the build-owned clone (safe.directory refused it, 'up to date at' with no commit); every git call runs as build now.
Verified on the box: one sync pass prints 'source: mirror (the deploy key is not accepted by GitHub yet ...)' and the commit.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Main's order of 7 October 2026. install-hands.sh creates /srv/observer/.ssh/deploy_igneum (ed25519, user build, mode 600; the private
half never leaves the box and nothing prints it) and the ssh alias github-igneum-observer; observer-sync.sh tests the key with
ssh -T on every pass, pulls from GitHub when it is accepted and from the mirror otherwise, saying which. docs/plans/build-server.md
5c: the four steps for the project lead (print the public half, Settings > Deploy keys > Add, read-only, start one sync pass) and the public
half itself. Verified on the box: key created, alias written, one sync pass reads 'source: mirror (the deploy key is not accepted by
GitHub yet)'. The CI-runner row is closed (the box-work agent's runner service).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Main's order of 7 October 2026. The repo root carries rust-toolchain.toml (channel 1.99.0, targets x86_64-pc-windows-gnu and
x86_64-unknown-linux-gnu); rustup resolves the nearest file walking up from the crate, so the fork worktrees under vendor/ are
covered, and the fork's master carries its own copy (vendor/igneum-node 37f1206b). lib.sh bs_toolchain_check reads the pin
and refuses a build when the pin, the Mac's rustc as resolved in the crate dir, or the box's rustc differ (the message says
the three commands that align them); run-from-mac.sh passes the pin to provision.sh as RUST_TOOLCHAIN. The 1.99.0 toolchain
with both targets is installed on the Mac so no agent's build stalls on rustup's auto-install. Verified: the Mac resolves
1.99.0 in a fork worktree, the box runs 1.99.0, the check prints 'pinned 1.99.0 by rust-toolchain.toml'.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Ledger N5: cargo build -p kaspad alone leaves the igneum-pow feature off (kaspad/Cargo.toml, default kHeavyHash
stub); igneum-miner turns kaspa-pow/igneum-pow on, and feature unification carries it into kaspad only when the two
are built together. A node built alone refused every real block (two fork-gate chains, 0 blocks), the same miner
mined 118 and 115 blocks against nodes built with both packages. Recommendation: igneum-pow a default feature of
kaspad for 0.3.17, the engine named in the version string (igneum_getNodeInfo's powEngine carries it now).
Node document 7.5 and 7.6: vote-or-burn none PASS, the known-failed case FAILS, burn and bonus PASS on the payout
(0.8000 and 0.9000, the pool 0.25 and 0.35) and FAIL on the replay (a node synced from genesis across the span
disqualified 240 blocks as UTXO-invalid: its silence reading differs from a present node's; both switches stay
never until the reading is a pure function of the block's past); the replay gate's own failed shape (another
digest) refuses at the handshake. Fork gate: the rule's known-failed case PASSES (a 32.5 percent key's 171-DAA deep
fork took A's sink 25 s after the heal under the rule as written); the gate-on case found the depth gap (fixed on
the fork, 3301cf32) and then PASSED on the rebuilt pair (B 2,810 bps, 193 DAA deep, heavier by 193, A held, A learned
188 of B's blocks) with a log flood fixed next (9a44fcb8); the harness now waits for each node's RPC and clears
leftover nodes before a case. The chain on the final pair runs; its three lines land in the table when it closes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Main's order of 7 October 2026 after a seed took 14 restarts and three minutes down on a glibc 2.39 binary. provision.sh:
step_zig (zig 0.17.0 from ziglang.org, sha256 from the official download index) and cargo-zigbuild 0.23.4 in the cargo tools.
tools/build-remote.sh --ship [--glibc 2.36]: cargo zigbuild --target x86_64-unknown-linux-gnu.2.36 with zig as the C/C++
toolchain (the Mac's infra/cross/build-linux.sh recipe), artefacts from the target-triple dir, each checked by
tools/ci/glibc-ceiling-check.sh (need at most the ceiling; self-test fires on 2.38 against 2.36, passes 2.34 and 2.36;
--symbols reads a saved objdump -T text so CI needs no ELF tools). tools/workers-remote.sh builds the two GPU workers with
zig at 2.36 by default (GLIBC=native for clang). Proof on the box: fork 3bfe346f igneumd needs GLIBC_2.34 (47,023,120 B,
sha256 345dfb95...), igneum-miner GLIBC_2.34 (9,248,808 B, d09dc27b...), 3 min 17 s cold through zig; the workers at 2.36.
Rule: anything that ships to a seed or a HiveOS rig is built with --ship; a plain build (glibc 2.39) is for the box and
Ubuntu 24.04 hosts only.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
docs/plans/hands-on-build-1.md section 6: the move of 6 October 2026 23:06 to 23:22 UTC, observer node, observer, node 1, unload,
each hand's first executing line, digest eada4bda MATCH, IBD and acceptance, the open readback (the Mac's Miner app has not
started its own node since 26610/26611 were freed). lib.sh: the shipper's class from the 0.3.17 tree, the fork's igneum-exec
embeds proving/igneum-prove/elf/*.vk by include_bytes! five levels up, which is no path dependency; every .rs in the trees that
travel is scanned for include_bytes!/include_str! paths leaving the crate's repository and their directories join the overlay;
proving/igneum-prove/elf is the fixed fallback for a fork build. move-hand.sh: igneumd --version exits 1 (tolerated; it ended
the binary step before the override was written), and the launchd pid lookup used \s in macOS awk (printed 'pid none').
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Main's hold of 6 October 2026 (23:xx UK): the hands move only on the shipper's 'publish 2 live: digest <x>' line, each hand read back by
commit string and consensus digest. move-hand.sh binary now derives --node from the fork tree behind the Mac's node1 launchd agent
(igneum-wt-ship0315/vendor/igneum-node-0315 at f1ea7a38 tonight; the box's igneumd rebuilt from it is sha256 7f0bde70...), warns when the
Mac's override still has 13 fields (publish 2 writes the sixteen-field object when it restarts the hands), and after each hand's first
executing line prints the box binary's commit string and the hand's digest against the Mac hand's last digest (MATCH or DIFFER, DIFFER
stops the sequence). Dry run clean against the live Mac.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
network.md 5.11 and proposal 1: Kaspa's relay requests one round per batch, so at 10 bps and a 2-s RTT the hub took
one spoke block in three (every block a chain block, two thirds of the network's hash merged nowhere); at 400 ms it
took all of them. The fourth gate for any rate step: relay delivers the full block rate at the measured inter-region
RTT with margin. CLAUDE.md's block-rate line carries it. The live 1 bps devnet offers under one block per RTT and is
nowhere near it. Controller rows (node document 7.2): rule v3 at 600 ms holds 9.93 blocks/s with the difficulty 4.5
percent over the hash-implied value; the live rule v2 reads 22 percent under on the same DAG shape.
Ledger N4: the vote weigh-up's silent desktop key, upload by upload from miner_logs and the observer: the voter never
died and --no-vote was never set; the node fell off the network (a digest-mismatch refusal, then a same-digest peer
that relayed nothing), its tip froze for 53 minutes, and the miner hashed and voted on the frozen template with
synced=true throughout while the node accepted 1,750 blocks into a dead branch. The fix pair on the fork (miner
e6e1fbe2, node 96619b7d); infra/fast-time/miner-stall.mjs: stall PASS (STALLED at 30 s, exit 45 at 31.5 s), moving
PASS, today's miner FAILS as it must.
infra/fast-time/fork-gate.mjs and vote-or-burn.mjs with the replay gate (--replay). First lines: vote-or-burn none
PASS, the known-failed case FAILS, burn PASS on the payout (ratio 0.8000, pool 0.25) but the REPLAY FAILS: a node
synced from genesis across the span disqualified 240 blocks as UTXO-invalid (its silence reading differs from a
node that was present), so both vote switches stay never until the reading is a pure function of the block's past;
fork gate: the first run idled the honest side and the depth read 0 (fixed on the fork, 3301cf32: depth along both
chains), the re-run's lines follow. The tree ships as 0.3.17 (node document section 7).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The fleet's 22:09 UK incident (a Mac-side pkill -f <log file name> matched nothing, the roll-everything script lived on and wiped a held box) and the day's two pgrep self-matches are one class. The check flags pgrep -f / pkill -f with a plain literal (every one on a line), any pgrep/pkill on a file-name shape, and ps | grep with a literal; it allows the bracket form, -x, -F pidfile, kill $(cat pidfile), a variable and a full path; 11 banned and 16 allowed shapes in its self-test; 0.15 s over the tree. The 25 pkill -f sp1-gpu-server inside bash -c bodies (which matched the calling bash) are pkill -x; the other 11 literals take the bracket form; prover-socket-check accepts both. Row R in the record; the CLAUDE.md rule names the check and covers pkill and file names.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The box's 34 red rows of 6 October classified (docs/analysis/ci-failures-2026-10-06.md section 6): 22 iterations, 12 in three real classes (instant deaths with nothing kept, a shared worktree directory, an unread dry run). remote-run.sh now: pre-flight (subcommand, manifest, -p package, --features) refuses in a second with exit 3 and a class; the last 400 lines of every run kept in /srv/builds/_log/runs; a class on every row (compile-error, link-error, test-failure, instant, no-test-matched, slot-timeout, no-dir, preflight-*); a cargo test whose filter matched no test exits 3; a per-worktree lock in checkout and run mode; every red row appended to /srv/ci-red/red.jsonl as source box. red-watch.mjs never posts a box row alone and sends one digest a day (counts per class with each class's guard); the timer runs tick. Shared group cired on the box so the runner and build append to one file. Shown in a sandbox on the box: pass, failing test, empty filter, bad package, bad feature, missing subcommand, compile error, broken manifest, two concurrent runs of one worktree.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
