Commit graph

332 commits

Author SHA1 Message Date
igneum-labs
0cb0e413b8 Merge branch 'gpu-fleet' into HEAD 2026-10-06 23:25:53 +00:00
igneum-labs
c704c8263c Fleet: the last three process patterns in the bracket-first form
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 23:25:53 +00:00
igneum-labs
b8b807ca15 Merge branch 'gpu-fleet' into HEAD 2026-10-06 23:25:10 +00:00
igneum-labs
0e10fcfe30 Fleet scripts: process patterns in master's bracket-first form (the kill-by-name check), anchors dropped
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 23:25:10 +00:00
igneum-labs
859b79b2d4 Merge branch 'gpu-fleet' into HEAD 2026-10-06 23:23:46 +00:00
igneum-labs
4d5b80f791 Merge gpu-fleet: the rented fleet's tooling (box library, standing supervisor and library, the 10 percent and 75 percent gates, the deploy gate, publish and canary scripts, Devnet 2 scripts and gate reader, the wind-down) and the night's analyses (prover tiers, block rate, the fleet night rows 1 to 18, the bench-log entries)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 23:23:46 +00:00
igneum-labs
ddaeaf076a lib/box.py: the node path pattern in the bracket form; the pgrep check allows the variable it is read through
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 23:17:48 +00:00
igneum-labs
25d2fe8275 Fleet scripts: every pgrep/pkill -f literal in master's bracket form (the kill-by-name check); the rehearsal chain's end script
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 23:14:41 +00:00
igneum-labs
4573b6f3ed Fleet: the deploy gate (syntax + master's defaults-line check + the comment-with-code class) in lib.box.put; the 75 percent table gate (standing.table_gate, the wind-down holds under it)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 23:04:29 +00:00
igneum-labs
7b3594ab46 box-dn2.sh: the UNSYNCED comment moved above its line (it had swallowed the port variables; three Devnet 2 nodes failed to start under set -u)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 23:02:49 +00:00
igneum-labs
c5ebf5cf7c Merge site-ui-3: chip headline range (X35) and the latency ladder on the site (M34)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 23:02:46 +00:00
igneum-labs
6e5e35c8d0 site: the class v4 chip headline is a range, 2.1x to 3.9x with k stated (ledger X35); the latency ladder described (M34)
The X9 made the k = 0.33 column of the chip model a product class, so
every public sentence that stated 2.1x alone now states 2.1x (k = 1) to
3.9x (k about 0.33), with the ladder's second rung taking the X9 bracket
to about 2.8x. The ladder lane's litepaper paragraph 'The work that waits
can grow' and ledger row M34 are taken from branch ladder (7003f9f, those
two hunks only) so the ladder is on the site before the code ships.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 23:02:43 +00:00
igneum-labs
b180ac0403 dn2-check: the reorg read ignores the exec follower's replay of old history after a restart (lines far below the exec tip)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 22:52:30 +00:00
igneum-labs
5e4dc171e9 box-dn2.sh: unsynced mining is opt-in (a fresh genesis only); the standing Devnet 2 nodes restart synced-first
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 22:51:54 +00:00
igneum-labs
b53b217e56 Fleet: the publish-2 object move for the standing boxes (file swap, node restart by the supervisor, digest table)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 22:50:03 +00:00
igneum-labs
0e6f5bce6b Stale-overlay class closes with its check: tools/ci/mirror-reset-check.sh in the pre-push gate
The reset itself has been master's behaviour since 9df9688 (remote-run.sh checkout_tree: git checkout -- . and git clean of the
overlay files, target dirs and stamps kept, before the branch checkout); the UI lane met the class again from worktrees whose
tools predate it (remote-run.sh is piped to the box from each worktree per build). The check reads checkout_tree() and fails
when the reset and the clean do not both come before the branch checkout; self-test: the real script passes, a copy without the
reset fails, the same lines moved after the checkout fail. cargo-audit: already owned by provision.sh step_cargo_tools
(b2262e5), confirmed ok (0.22.2) on the box; nothing added. Gate GREEN, 33 checks.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 22:47:16 +00:00
igneum-labs
0b52c1b3da Standing supervisor: the miner runs only on a synced node past block 1,000 and is stopped while the node syncs (a box never mines from a genesis-only view)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 22:39:29 +00:00
igneum-labs
f20bda7804 dn2-check: the since stamp follows each log's own UTC offset (the build-server seed logs +02:00)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 22:29:23 +00:00
igneum-labs
7a025825f2 Standing check: every box's handshake digest against the live object (~/Desktop/fleet/expected-digest) every five minutes, one #incidents line per box per hour on a mismatch
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 22:19:23 +00:00
igneum-labs
165b0ff588 box-dn2.sh: the program-ids comment moved off the node line (it had swallowed the flags and the backgrounding, the same class as 17:18Z)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 22:15:29 +00:00
igneum-labs
a86b532736 Standing supervisor: a crash loop on a corrupt datadir moves it aside and rejoins fresh (p2-4090-1b, 24 restarts on a missing sst)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 22:13:09 +00:00
igneum-labs
552ef811f9 Devnet 2 nodes carry IGNEUM_PROOF_PROGRAM_IDS (a zero shard program id refused every segment record)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 22:10:32 +00:00
igneum-labs
11c4426a96 Kill by exact command line or pid file, never by a name: tools/ci/kill-by-name-check.sh in the gate; the 36 pgrep/pkill literals in the tree fixed
The fleet's 22:09 UK incident (a Mac-side pkill -f <log file name> matched nothing, the roll-everything script lived on and wiped a held box) and the day's two pgrep self-matches are one class. The check flags pgrep -f / pkill -f with a plain literal (every one on a line), any pgrep/pkill on a file-name shape, and ps | grep with a literal; it allows the bracket form, -x, -F pidfile, kill $(cat pidfile), a variable and a full path; 11 banned and 16 allowed shapes in its self-test; 0.15 s over the tree. The 25 pkill -f sp1-gpu-server inside bash -c bodies (which matched the calling bash) are pkill -x; the other 11 literals take the bracket form; prover-socket-check accepts both. Row R in the record; the CLAUDE.md rule names the check and covers pkill and file names.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 22:10:31 +00:00
igneum-labs
a54d58dce2 Devnet 2 prover restart as a file (one instance; an inline pkill with the host's name killed its own ssh shell)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 22:01:29 +00:00
igneum-labs
d79f4a6174 Merge site-ui-3: RandomX chip correction (X34)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:59:01 +00:00
igneum-labs
380d1b681f site: RandomX has a shipping chip; correct every sentence that said otherwise (ledger X34)
Bitmain's Antminer X9 (1 MH/s at 2,472 W, about USD 5,600) ships from
July 2026 and RandomX 2.0 shipped on 25 March 2026. Five sentences on
the home page and the litepaper that said or implied RandomX is
chip-free now state the X9 and its date; sentences that only name the
technique stand. Ledger row X34, C2 closed by the fact, the text check
carries the new sentences.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:58:59 +00:00
igneum-labs
76723b3264 Fleet: the wave's wind-down by the 10 percent rule (hourly slices sized by weight share, weight_check gated, held under two thirds signing, the lock read after each); the poisoned-peer confirmation script
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:58:04 +00:00
igneum-labs
29076448f2 Standing supervisor: one miner per GPU (the miner loop kills any other miner before starting its own)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:55:36 +00:00
igneum-labs
34fc638266 box-prover.py: one instance per box (pid file) and a per-process state tmp (two instances raced and killed each other's GPU server on Devnet 2); dn2-check.py: the Devnet 2 gate reads for the build-server seed
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:53:51 +00:00
igneum-labs
c1b6ab1971 fleet-bg.sh: the macOS note moved off the case line (a trailing comment swallowed the ;;)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:43:37 +00:00
igneum-labs
27ce42aab5 Merge ci-hardening b2262e5: one gate script for the hook and CI, the research exclusion list, the Windows paths check, the red watcher and the box's red-row classes with pre-flight
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:43:37 +00:00
igneum-labs
608bfa4a11 fleet-bg.sh: setsid optional (macOS has none)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:43:14 +00:00
igneum-labs
5b1531c15a Fleet: pid-file job helper (kill by pid, never by a name), the supervisor's P2P_LISTEN knob for inbound-port boxes, publish 1 on f1ea7a38; CI README row for the kill rule
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:42:59 +00:00
igneum-labs
80c5618eb5 Merge site-ui-3 13fafe3338: the home page restored and polished for the miner (the step scene as the hero, the network strip, the measured card picker, the Discord line). The owner, 6 October 2026: "Ok new page approved".
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:42:16 +00:00
igneum-labs
03287bcc78 box-prover.py: a statement mismatch names the hex offset and both tails (the 34-char prefix hid the difference on Devnet 2)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:40:28 +00:00
igneum-labs
b2262e5dc6 Build box: every red run classified and kept, pre-flight before the slot, one run per worktree, box reds in the red-run file, a 09:00 UK digest
The box's 34 red rows of 6 October classified (docs/analysis/ci-failures-2026-10-06.md section 6): 22 iterations, 12 in three real classes (instant deaths with nothing kept, a shared worktree directory, an unread dry run). remote-run.sh now: pre-flight (subcommand, manifest, -p package, --features) refuses in a second with exit 3 and a class; the last 400 lines of every run kept in /srv/builds/_log/runs; a class on every row (compile-error, link-error, test-failure, instant, no-test-matched, slot-timeout, no-dir, preflight-*); a cargo test whose filter matched no test exits 3; a per-worktree lock in checkout and run mode; every red row appended to /srv/ci-red/red.jsonl as source box. red-watch.mjs never posts a box row alone and sends one digest a day (counts per class with each class's guard); the timer runs tick. Shared group cired on the box so the runner and build append to one file. Shown in a sandbox on the box: pass, failing test, empty filter, bad package, bad feature, missing subcommand, compile error, broken manifest, two concurrent runs of one worktree.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:40:07 +00:00
igneum-labs
13fafe3338 Home page restored and polished for the miner: the step scene with the shards, the glow and the final line brought up to date; live network strip; measured card picker; Discord; no calendar month
The owner, 6 October 2026: "Im not sure about the site can we revert it but polish it? minimal, simple but everything needed
for the dopamine and emotional triggers of the gpu miner", then "cant we have a dag animation like we had before? with the
shards making up the block, then the final line and the other bits that looked really cool but brought upto date?"

Restored (a new commit, no history rewrite): site/index.html as it stood at 14da2b8, the step-view page with its hero, facts,
scene, three things to check, downloads, build, wallet, journey, economics and the ledger band; nothing from tonight's other
work is undone (the litepaper, /live, the roadmap and benchmark wording, ledger X31 to X33, the scroll-zoom fix). Where the
page named a month it now carries tonight's sentences: the proofs card reads "Live rows arrive with the public testnet. The
public testnet is weeks away: three seed nodes and the public RPC are up, and it opens when the go checklist closes."; the
journey lead reads "Six phases, four public gates ... No calendar dates: each phase closes at its gate." and its inlined
phases are the gate-worded ones. No "August 2027", no month anywhere on the page.

The scene (site/live-steps.js): the original bits, every one driven by the live feed. A real block arrives from the right
with a glow and its chain number under it; its outline is grey while pending and ember once included; as its shards are
verified or paid, particles fly in from the edges and its quarter cells fill molten; when every shard is proven the block
turns ember and the caption says how many provers were paid; a locked checkpoint takes a ring and a ripple and the dashed
"final" line sweeps in from the right to it, drawn only while finality is active (the legend no longer says "final to its
left"; the wallet card's state word "final, checkpoint 5" is the wallet's own vocabulary and stays). A proof or a lock that
lands on a block already off screen re-enters from the right as its own event, so every step is seen when it happens. The
counters gain blocks per second. The caption follows the newest block that changed step.

The miner's triggers, each one element with live or measured numbers, none invented: a network strip under the facts
(hash rate, blocks in the last ten minutes, the latest block's age ticking every second with its chain number and word,
shards proven and paid with the last ten minutes), each value flashing molten when it changes; a card picker from the bench
table with the time a card alone takes to find a block at the live hash rate (RTX 5090 127.7 MH/s at 227 W and RTX 4070
28.8 MH/s at 76 W from the 6 October Ember Tune on the three-card Windows rig, RX 9070 XT 17.8 MH/s from the 5 October eGPU
entry, Apple M5 Max 26.7 MH/s from the first live swap; the RTX 4090 and RX 7900 XTX shown as not yet measured with the
bench table linked) with the pool note; the fairness line (graphics cards only, a new program every hour, your card proves
the blocks, 80% to the miner and 20% to the provers, nothing to anyone else); the download buttons with sizes; a Discord join
line (the standing invite, recorded in docs/community/discord-hooks.md so nobody asks again); the testnet sentence.

docs/fud-ledger.md: G4, C2 and X8 each gain a status line saying the restored home page carries their sentence again; the
ledger-text check guards them on the home page again (55 sentences, 0 missing).

Measured headless over 40 s against the live feed: 19 captioned transitions, the strip reading 2.3 GH/s, 562 blocks in ten
minutes, 8 s ago, 2,970 shards paid, the picker's times 18 s (5090), 79 s (4070), 2 min (9070 XT), 86 s (M5 Max) at that
hash rate, 0.92 blocks/s; no console errors at 1440 or 390 in either theme; no horizontal overflow. Captures in
docs/plans/site-ui-3-shots/after-v2/: home-{1440,390}-{dark,light}-fold.jpg, home-{1440,390}-{dark,light}.jpg and
home-steps-1440-dark.webm (ten seconds of the scene). Checks: identity grep 0 hits on served files, link check 934 links
across 16 pages 0 broken, ledger text 55 of 55, contrast 32 pairs 0 under 4.5:1, api tests 5 pass. Not deployed: the owner
sees it first.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:37:38 +00:00
igneum-labs
89391d0052 Merge box-capacity 58dacf6: the builder's background capacity layer (fuzz, sims, sweeps, clippy; yields to builds)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:35:54 +00:00
igneum-labs
b7144f5dc6 Poisoned-peer canary: the already-running check uses a marker (the Vast proxy's banner on stdout defeated a bare yes)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:07:53 +00:00
igneum-labs
487cea3844 pre-push gate: clear git's hook environment (GIT_DIR and friends) so the self-tests' nested git never acts on this repository
The first master push through the gate died inside remote-run.sh's self-test: its mirror push ran this repository's hook against the fixture tree. Shown fixed by a push to a local bare repository (the real hook, GREEN).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:06:16 +00:00
igneum-labs
ee2391c9a4 Poisoned-peer canary: idempotent on a box already running the binary; a 45-minute sync wait for a fresh datadir (IBD of 128,000 blocks takes about 12 minutes)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 21:03:29 +00:00
igneum-labs
60eb06ec24 CI hardening: one gate script for the hook and CI, the research exclusion list, the Windows paths check, the red watcher, the box runner switch, the failure classification
168 non-green runs since the first workflow run were classified (docs/analysis/ci-failures-2026-10-06.md): 102 were tree checks that finish in under 25 s on the pushing machine, 40 were GitHub-side refusals nobody saw.

tools/ci/pre-push.sh is the one list of fast checks; ci.yml's site job calls it with --ci and the pre-push hook with --hook (full gate for master and release-*, structural checks for other refs; never writes into the worktree). tools/ci/export-exclude.txt lists research documents outside the public export list, pruned by identity-check.sh and by the mirror's sync.sh (self-test: an excluded path may quote the patterns, an exported one may not); polish.md and this record are its first entries, which makes master green. tools/ci/windows-paths-check.sh (colon, trailing dot or space, reserved names, over 240 characters) runs as the pre-commit hook on staged paths and in the gate. tools/ci/red-watch.mjs plus the red job on the box's runner record one line per failed master or release-* run to /srv/ci-red/red.jsonl; igneum-ci-red.timer posts each once to the updates channel. pow and sims read IGNEUM_CI_RUNNER for the box. no-foreign-tree-writes.sh no longer exits silently on its warning pipeline under pipefail.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 20:58:54 +00:00
igneum-labs
6836115d65 Capacity layer: idle-core background workload on igneum-build-1 that yields to builds
infra/build-server/capacity: igneum-capacity.service (user build, Nice 19, SCHED_IDLE,
CPUQuota leaving 8 threads free) runs run.sh, a controller over a priority queue of jobs
that pauses (SIGSTOP) the instant a build slot or the measure hold is taken (5 s poll of
/srv/builds/_locks) and resumes after. Jobs, each with a dry-run and a 10-min smoke:
pow fuzz (continuous, seed base advances), sync-request fuzz against a throwaway pruned
node on loopback (the Horizon 28-unwrap gate, igneum-p2p-probe sync-fuzz), GHOSTDAG and
finality sweeps seeds 1 to 1,000, model.py sweeps cached, clippy+audit per recent branch.
Summaries to /srv/workers/capacity.json; the worker dashboard gains a Background lane
(collect.mjs doc.background, page renderBackground). Night battery stops and restarts the
layer. docs/plans/build-server.md section 8. igneum-pow fuzz tests and ghostdag_sim.py /
attacks.py gain a seed-base knob for the continuous sweeps.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 20:58:42 +00:00
igneum-labs
86c9854863 box-prover.py: the export directory owns its disk (20 GB and 7-day caps, export deleted on record accepted or paid, no export under 10 percent free)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 20:56:19 +00:00
igneum-labs
6cc1bc9d57 cross-remote.sh: the default cross-build runs again (a trailing comment had swallowed the defaults line); the class gets a CI check
The shipper's report: with no arguments, cross-remote.sh died under the Mac's bash 3.2 with 'CARGO_ARGS: unbound variable' and
its chain kept the previous exes. Cause: a comment appended to the defaults line in the box-work merge turned OUT, COMPARE,
TARGET_DIR and CARGO_ARGS=() into comment text, so the array was never declared (the same shape build-remote.sh had at 19:5x).
Fix: the line split, and the default tests use [ -n "${CARGO_ARGS[*]:-}" ] in both tools, safe whether or not the array
exists. Proof: the default cross-build from a fork worktree under /bin/bash 3.2.57 built both exes (igneumd.exe 8f7e2ae3...,
igneum-miner.exe 6f8b49d8...). tools/ci/defaults-line-check.sh fails CI on a line where a comment start is followed by an
assignment list (quoted strings, ${...}, $# and [^#] dropped first); self-test fires on the swallowed shape and passes
${a#b}, sed s#x#y#, $# loops and prose mentions; the tree is clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 20:55:44 +00:00
igneum-labs
a99617bd9f Poisoned-peer canary: anchor the poisoned node's pgrep (the bare pattern matched its own shell and the peer never started)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 20:54:08 +00:00
igneum-labs
853c0217d9 Fleet: disk sweep every five minutes (disk per box on the page, one #incidents line per box per hour at 85 percent); fleet night row 15 (the hub's three deaths)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 20:49:41 +00:00
igneum-labs
fe5e481528 Standing supervisor: prunes the prover's segment exports and trims the node log (the hub died on a full disk at 20:42Z); disk in the status line
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 20:45:32 +00:00
igneum-labs
498370d07f Fleet: the 0.3.15 canary in the poisoned-peer form (new node beside the hub with a 1026-block peer, mid-window restart, the hub's count by peer)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 20:42:21 +00:00
igneum-labs
09cf92db48 Fleet page publish: the Fleet | Workers nav and its CSS are re-inserted on every publish when a regeneration dropped them
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 20:38:29 +00:00