History

igneum-labs e48a512c3f lease pool: four priority classes (main, 7 Oct 2026 20:37 BST: the class v5 census sat behind eleven adversarial waiters): release > v5 > measure > adv, from --class or the owner and label; a higher class takes the next freed cores before any lower class whatever the arrival order (lower classes yield while a higher one waits); a lower-class holder above 32 threads is pre-empted (TERM to its command, the lane re-queues) when a higher class has waited 120 s; the class in every line; self-test: the known-failed class-order case (a sweep must not take cores a waiting v5 gate asked for) and the pre-emption case. Also: provision.sh ROLE=sweep (a rented cloud sweep worker, minus runner, caddy, cuda, night, chromium, zig) and infra/build-server/cloud-sweep.sh (Hetzner Cloud CCX workers up/list/cost/down with the lease tool) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>		2026-10-07 19:40:32 +00:00
..
capacity	Build boxes: the measure file is retired; a pinned measurement leases its cores only (lease.sh cores), a whole-box quiet measurement is its own class (refused beside a slot or a lease, 20-minute cap, named owner), bounded suites never take it, every run keeps off leased cores, stopped holders are reaped after 5 minutes by every keeper, every waiter has a label file; the box-side self-tests in the gate; provision installs the lease tool and the headless Chromium libraries	2026-10-07 13:30:40 +00:00
ci-red	Build box: every red run classified and kept, pre-flight before the slot, one run per worktree, box reds in the red-run file, a 09:00 UK digest	2026-10-06 21:40:07 +00:00
devnet3	Devnet 3: devnet3.sh passes the argument line base64-encoded over ssh (ssh flattens its argument list: the first --go started the seed on the OLD devnet, digest c562d70e, port 26611, 7 Oct 2026 18:22 BST) and refuses a line without --devnet-suffix=3	2026-10-07 17:26:14 +00:00
discord-hooks	Discord webhooks: install.sh finishes on a partial credentials file (the check step exits 1 by design)	2026-10-06 19:07:26 +00:00
hands	Observer sync follows GitHub: the deploy-key probe captured before grep (ssh -T exits 1 under pipefail); plan 5c done	2026-10-07 07:31:42 +00:00
night	Capacity layer: idle-core background workload on igneum-build-1 that yields to builds	2026-10-06 20:58:42 +00:00
prover	Box: GitHub Actions runner as user runner, two build slots with 90 or 45 jobs, the measure hold, the CPU prover trial	2026-10-06 19:32:43 +00:00
repro	Repro check: sccache really off (RUSTC_WRAPPER=/usr/bin/env, an empty value falls back to the box's config), the author-time epoch of lib.sh bs_sde, and the re-stamp line the copied-sources check reads	2026-10-06 20:13:43 +00:00
runner	CI queue: a second self-hosted runner (igneum-build-2 joins the pool label), docs-only pushes skip the compile jobs, the red watcher pinned to the box that posts	2026-10-07 13:20:18 +00:00
workers	Worker dashboard: one server section per box, per-box live feeds, installer and pusher take build-2 and build-3	2026-10-07 10:37:33 +00:00
cloud-sweep.sh	lease pool: four priority classes (main, 7 Oct 2026 20:37 BST: the class v5 census sat behind eleven adversarial waiters): release > v5 > measure > adv, from --class or the owner and label; a higher class takes the next freed cores before any lower class whatever the arrival order (lower classes yield while a higher one waits); a lower-class holder above 32 threads is pre-empted (TERM to its command, the lane re-queues) when a higher class has waited 120 s; the class in every line; self-test: the known-failed class-order case (a sweep must not take cores a waiting v5 gate asked for) and the pre-emption case. Also: provision.sh ROLE=sweep (a rented cloud sweep worker, minus runner, caddy, cuda, night, chromium, zig) and infra/build-server/cloud-sweep.sh (Hetzner Cloud CCX workers up/list/cost/down with the lease tool)	2026-10-07 19:40:32 +00:00
lease.sh	lease pool: four priority classes (main, 7 Oct 2026 20:37 BST: the class v5 census sat behind eleven adversarial waiters): release > v5 > measure > adv, from --class or the owner and label; a higher class takes the next freed cores before any lower class whatever the arrival order (lower classes yield while a higher one waits); a lower-class holder above 32 threads is pre-empted (TERM to its command, the lane re-queues) when a higher class has waited 120 s; the class in every line; self-test: the known-failed class-order case (a sweep must not take cores a waiting v5 gate asked for) and the pre-emption case. Also: provision.sh ROLE=sweep (a rented cloud sweep worker, minus runner, caddy, cuda, night, chromium, zig) and infra/build-server/cloud-sweep.sh (Hetzner Cloud CCX workers up/list/cost/down with the lease tool)	2026-10-07 19:40:32 +00:00
lib.sh	Build boxes to near max (the project lead, 7 Oct 2026 19:4x BST): the bounded class is 88 of 96 cores with -j 88 (8 reserved for release builds, the seed and the observers), the jobs rule 88 alone / 44 shared, the spill line 80; checks updated. testnet-go.md: LG-2 waived by the owner, the new gate, the seeds held armed	2026-10-07 18:23:14 +00:00
overlap-browser.sh	Text-overlap sweep (tools/ci/overlap-check.mjs) and the first fixes it found: the home hero's step pill no longer sits on the caption (pill at the corners' baseline, caption above it, at every width), the address page title wraps, the ledger's status badges wrap, the litepaper's mobile contents bar bleeds by the real gutter, the verify harness table scrolls (7 October 2026, 15:5x UK)	2026-10-07 13:55:13 +00:00
provision.sh	lease pool: four priority classes (main, 7 Oct 2026 20:37 BST: the class v5 census sat behind eleven adversarial waiters): release > v5 > measure > adv, from --class or the owner and label; a higher class takes the next freed cores before any lower class whatever the arrival order (lower classes yield while a higher one waits); a lower-class holder above 32 threads is pre-empted (TERM to its command, the lane re-queues) when a higher class has waited 120 s; the class in every line; self-test: the known-failed class-order case (a sweep must not take cores a waiting v5 gate asked for) and the pre-emption case. Also: provision.sh ROLE=sweep (a rented cloud sweep worker, minus runner, caddy, cuda, night, chromium, zig) and infra/build-server/cloud-sweep.sh (Hetzner Cloud CCX workers up/list/cost/down with the lease tool)	2026-10-07 19:40:32 +00:00
README.md	Build boxes: one host file per box and a route by class (suites and benches to build-2, proving to build-3, the rest to build-1); the no-mining rule in a box README and as a guard in the capacity layer	2026-10-07 10:34:16 +00:00
remote-run.sh	Build boxes: the bounded POOL (main, 7 Oct 2026 20:0x BST: every bounded run took 88 threads and the boxes read load 280 to 527): cores 8 to 95 form one pool of 88 per box; a bounded run leases free cores from it (one flock per core, shared with the measurement leases), up to its class cap or --jobs and never fewer than 16, waiting in the queue when fewer are free; CARGO_BUILD_JOBS and taskset follow the cores taken, so the sum of bounded threads on a box never passes 88; self-test updated	2026-10-07 19:03:35 +00:00
run-from-mac.sh	Build boxes: the slot holder keeps its own line (a keeper, with its self-test in the gate); an explicit --jobs is clamped for bounded classes; one git remote per box; the host-file double suffix	2026-10-07 11:22:12 +00:00
wave1-0320.sh	Gate: the green stamp and the fast path for merges (a full gate that ends GREEN over a clean tree records its commit; the hook lets a merge of a stamped branch onto the exact remote tip through on the light gate, CI runs the full one on landing); tools/ci/merge-to-master.sh; the wave script's seeds mode takes --wipe-genesis with the genesis and base-unit read-backs for the testnet go	2026-10-07 13:54:10 +00:00

README.md

The build boxes (infra/build-server)

Three Hetzner dedicated servers in Falkenstein run everything the Mac must not: builds, test suites, benchmarks, CPU proving, the devnet hands and the observer. The Mac keeps macOS binaries, the DMG and Metal tests (CLAUDE.md, "Running agents on this Mac").

No mining on any Hetzner box, ever

the project lead's rule through main, 7 October 2026: Hetzner's policies forbid crypto mining. The boxes run nodes, builds, tests, benchmarks and CPU proving only. The pool's fast-time network runs its miners on rented GPU pods (tools/fleet), never on a box; a box may run the network's nodes. The capacity layer refuses a job that would start igneum-miner mine or a GPU worker (capacity/run.sh), and no hands unit carries a miner. A node started with --enable-unsynced-mining is a node flag, not a miner; nothing feeds it blocks here.

The boxes and the kind map

Box	Host file on the Mac	Takes	Never
igneum-build-1 (188.40.146.49, AX162-1-LTD)	`~/.config/igneum/build-server`	release gates (`--priority gate`), builds and cross-builds, checks, the GPU workers' host side, the devnet hands (node 1, the observer node, the observer), the Devnet 2 seed, the CI runner, the dashboard feed	suites and benches once box 2 exists
igneum-build-2 (AX162-1, on order)	`~/.config/igneum/build-server-2`	suites (`cargo test`), benches (`cargo bench`), the attack rows (`--box 2`)	gates, hands
igneum-build-3 (AX102-1, on order)	`~/.config/igneum/build-server-3`	proving and aggregation CPU work (proving/igneum-prove builds and suites), the second prover's shadow runner, the pool's fast-time NETWORK (nodes only, `--box 3`)	miners of any kind

tools/build-remote.sh routes by class (lib.sh bs_route): suite and bench to box 2, the proving crate to box 3, everything else to box 1; --box N overrides; a class whose box has no host file yet falls back to box 1 and says so. --priority gate always runs on box 1. Each box has its own mirrors, slots, locks and JSONL log under /srv; run-from-mac.sh --box N <ip> provisions a box and writes its host file; the dashboard collector reads every box it is told about.

Files

File	What
`provision.sh`	the box itself: install mode (rescue system, Ubuntu 24.04, RAID 1, no swap) and provision mode (user build, toolchains, the pin, sccache, zig, CUDA headers, docker, Caddy, mirrors, slots, sshd, ufw)
`run-from-mac.sh`	ships provision.sh, writes the host file, wires the `build` remotes and pushes every branch
`lib.sh`, `remote-run.sh`	the Mac and box halves of a remote run: sync, checkout, slots, scheduling classes, the JSONL line
`hands/`	the devnet hands' units, the mover and the restart read-backs
`capacity/`	the capacity layer (the box-work lane's): background jobs under the build slots, never a miner
`repro/`, `night/`, `prover/`, `runner/`, `workers/`	other lanes' pieces that live on the boxes

Plan, numbers and the gotchas: docs/plans/build-server.md; the hands: docs/plans/hands-on-build-1.md.