- infra/build-server/repro/rebuild-on-box.sh: one target path per target with each pass's artefacts copied aside (prost's
protowire.rs embeds its OUT_DIR path, so a pass in a dir of another name differs), SOURCE_DATE_EPOCH from the node commit
and TZ=UTC (libmimalloc-sys compiles mimalloc's C with __DATE__ and __TIME__), --reuse for a re-report, reason text
for a shipped file that is not on hand (innoextract 1.9 cannot open the Inno Setup 6 installer; the 0.3.14 HiveOS
tarball left dl/public when 0.3.15 published).
- tools/repro/rebuild-release.sh: the plan's hashes always travel (the miners' 8-hex prefixes too), bash 3.2 empty-array
fix, the box half's exit code is the script's.
- docs/evidence/reproduced/0.3.14.md: igneumd 03f35e05..., igneum-miner 900c1f0b..., igneumd.exe 166e604e...,
igneum-miner.exe fefd266c... identical across two clean passes; DIFFER against the shipped 934f393c... (zig, glibc 2.36)
and 44fa74c0... (Homebrew mingw before the timestamp fix, from a worktree).
- docs/plans/build-server.md 7.2 (night battery: timer, dry run 3 min 54 s, 10 pass, the fork's 22 cargo-audit
advisories as the one FAIL and what follows) and 7.3 (repro: results, the two classes, SOURCE_DATE_EPOCH proposed for
every build script).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- infra/build-server/night/night-battery.sh: checkout of master and the newest release-*-node fork branch under
/srv/builds/_night, cargo test --release --no-fail-fast per crate (repo and fork), igneum-pow fuzz at 10x, the three
simulators in full, the fast-time harnesses (finality-attacks, harness s3 s4, exec-sync reorg) on binaries built into
target-integration, clippy per crate dir, cargo audit per Cargo.lock; docs/benchmarks/night/<date>.md with a pass/fail
table and "new since last night"; committed as igneum-labs on night-battery and pushed to the mirror, never master.
NIGHT_SUBSET=1 is the dry-run subset. The unit runs it through remote-run.sh so the slot spans the invocation.
- igneum-night-battery.{service,timer}: 02:00 Europe/London, User build, Nice 19, idle IO, 8 h limit, not Persistent.
- provision.sh: step_cargo_tools (cargo-audit), step_night (files from the mirror at NIGHT_REF, timer enabled), innoextract.
- tools/repro/rebuild-release.sh + infra/build-server/repro/rebuild-on-box.sh: pins from docs/plans/release-<v>.md
("(node <sha>, app <sha>)"), shipped hashes from the public downloads (the HiveOS tarball, the installer via
innoextract) or --shipped, a clean clone of repo and fork on the box, two passes per target without sccache under build
slots, MATCH or DIFFER per artefact against the shipped bytes and against the other pass, with the reason read off the
binaries (glibc version needed, PE timestamp, commit string); evidence into docs/evidence/reproduced/<version>.md.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- provision.sh step_runner: actions/runner 2.338.0 (sha256 checked) at /opt/actions-runner under a dedicated user `runner`
(no sudo, not in build's group), rustup 1.99.0 pinned with both targets, sccache against /srv/sccache in READ_ONLY mode
on its own server port, Node 22 and mingw from the system, GitHub's svc.sh unit with a Nice 10 drop-in; registered on
igneum-network/igneum as igneum-build-1 (labels self-hosted, linux, x64, igneum-build-1) through
infra/build-server/runner/register.sh (gh as igneum-labs, the token on ssh stdin, never logged). Idempotent after
the env files moved behind svc.sh install (its env.sh rewrites them). libicu74 and python3-numpy added to APT.
- main's slots ruling: SLOTS default 2; remote-run.sh sets CARGO_BUILD_JOBS 90 when it holds the only taken slot and 45
when both are held, BR_MEASURE=1 takes the `measure` file exclusively and excludes builds (builds hold it shared),
lock files open in append mode (the old `exec {fd}>` truncated a busy slot's holder line on every probe), env
IGNEUM_BUILD_SLOTS_DIR and IGNEUM_BUILD_LOG_DIR win over the profile, `--self-test-slots` with five cases (the old
script fails it with JOBS=none); build-remote.sh and cross-remote.sh pass -j only when --jobs is given.
- infra/build-server/prover/cpu-trial.sh: the SP1 CPU prover on one fixture shard under the measure hold with a VmHWM
poller; 6 Oct 2026 run: core 34.2 s, compressed 85.9 s, peak RSS 28.2 GB on 96 threads, so no standing CPU prover.
- docs/plans/ci-self-hosted.md: the proposed runs-on change for ci.yml behind the repository variable IGNEUM_CI_RUNNER
(GitHub-hosted is the fallback), and why windows.yml cannot move to a Linux box. Workflows untouched.
- docs/plans/build-server.md section 7.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Copy only, from origin/master c077caa (the fleet's record: docs/analysis/prover-tiers-real-cards.md and the bench-log
entry "Prover tiers on real cards"). The miner hero lead, the proving feat and the homepage lead each carry two
sentences: today's app with the stock SP1 server (a 24 GB NVIDIA card proves the full shard, RTX 4090 5.6 s and
RTX A5000 6.4 s; a 32 GB card mines and proves, RTX 5090 8.4 s; the stock server refuses 16 GB and under), and the
6 October 2026 measurement on eleven rented cards with the patched server (every NVIDIA card from 8 GB proves; 10 GB
and up mine and prove), linked to the bench-log entry and marked "ships when the packaging row lands".
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Audit fixes, copy only. (1) "every NVIDIA card from 8 GB proves" cited docs/analysis/prover-tiers-real-cards.md, which
is not in the repository, and contradicted evidence row 16; the hero lead, the proving feat, the three meta descriptions
and the homepage lead and h2 now say what row 16 and docs/analysis/amd-proving.md state: measured on an RTX 5090 with the
shipped SP1 server (13.9 GB floor), a 32 GB card mines and proves, a 24 GB card proves the full shard alone (from the
5090's allocation, not yet run on a 24 GB card), 16 GB empty shards only, 12 GB and under nothing on this build; the
eleven rented cards of 6 October are "measured, record pending". Each links /evidence#claims. (2) The h1 was 96
characters and seven lines on a phone; it is "Install. Start. The card mines and proves." (42) with the tiers in the lead.
(3) Lever row 4 said "Ships in 0.3.6" at 0.3.14; it now says shipped in 0.3.6 (the miner-latency merge, release-0.3.6.md
section 2), the app at 0.3.14.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
provision.sh step_cuda: NVIDIA's ubuntu2404 apt repository, cuda-nvrtc-dev-12-8, cuda-cudart-dev-12-8, cuda-driver-dev-12-8
(the libcuda stub) and opencl-c-headers; no nvcc (no build file calls it), no driver. tools/workers-remote.sh builds
igneum-worker-cuda and igneum-worker-opencl on the box from proto-cuda and proto-opencl with clang++ (static libstdc++),
prints sha256 and the glibc ceiling (2.38: fine for Ubuntu 24.04 hosts, not for 22.04 containers or HiveOS, where the Mac's
zig build stays). Proof: igneum-worker-cuda 1,613,992 B sha256 6db8a9ad..., igneum-worker-opencl 124,904 B sha256 0dea75bb...
remote-run.sh evaluates the command string in a subshell (a leaked set -e killed the runner after a successful build).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The public /ledger page (site/ledger.html) carried ~/.config/igneum paths five times, "pid 33114", --rpclisten=0.0.0.0:26610,
"PC 2" fifteen times, "the Mac" nineteen times and 202 repository file paths, because tools/ledger-page.mjs scrubbed with its
own short list and never ran the forbidden-strings hard stop (found by the site audit of 6 October 2026, docs/plans/site-ui-3-audit.md).
Now: the generator's scrub replaces every config or home-directory path with "a config file" or "a home-directory file", a pid with
"the process", a listen flag or 0.0.0.0 address with its plain words, "PC 1" and "PC 2" with "the Windows machine", "the Mac" with
"the Apple M5 Max" (the bench log's rule), and every repository file path with "a repository file"; the page's own source note
names no path. After rendering, the page is grepped with tools/ci/forbidden-strings.txt plus the leak classes and the render
exits 1 on a hit. The pattern list gains ~/.config, pid N, 0.0.0.0:port, PC 1 and PC 2 and --rpclisten=, and the identity grep
now covers site/ledger.html (html added to its file types). Regenerated: 167 entries, 0 leaks, identity grep 0 hits over 231
files, link check 0 broken.
Consequence per reader: the ledger keeps every criticism, status and answer; what a reader loses is the exact repository path
of a fix, which meant nothing outside the private repository. Nothing else on the site changes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Main's ruling (6 October 2026, 20:1x UK): the box installs with the NUMBERS key alone so the 21:00 UK pulse comes from
it. install.sh accepts at least one key and names the missing ones; every tick's log line ends with "missing <keys>"; the
watcher without an incidents webhook logs its open or resolve and still advances its state, so the key landing later
does not flood the channel. Test added (31 passing).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
tools/community/discord-hooks.mjs (Node 22, standard library): one ember embed per post to #numbers, #announcements and
#incidents. Network pulse every 6 h (03/09/15/21 UK) from /api/stats, /api/live, /api/supply with the 6 h window from its
own snapshot and a Devnet 2 line that uses the fleet file's numbers and gate word only; a 24 h digest and the reddit kit's
weekly template at 09:00 UK; a release subcommand for the shipper (three downloads with sha256, node commit, digest, up to
five plain "what changed" lines read from the release plan); incident open and resolve by hand; a watcher that opens one
incident per condition (finality paused 5 min, median proof lag 15 min, observer silent 3 min) and resolves after 2 clear
minutes, and never opens on a condition already firing when it first looks (the pulse says "finality paused since" instead).
Guards before every post: the founder's name and logins, hosts, machine ids, paths, IP addresses, standalone 32-hex
tokens, dl.igneum.network outside /public/, webhook URLs, mentions, the tools/ci/forbidden-strings.txt patterns; Discord's
limits refused rather than cut; idempotency keys per slot in a state file; exponential backoff on 429; dry run by default
with a Discord-like preview in tools/community/out/preview.html. 30 tests on fixtures cut from the live API.
infra/build-server/discord-hooks: a tick every minute on igneum-build-1 (the Mac sleeps). install.sh copies the webhook file
to /srv/discord-hooks/env (mode 600, over ssh stdin) and refuses without IGNEUM_SECRET_ON_BOX_OK=1; the recorded exception
to rule R6 is in docs/plans/build-server.md (main's ruling, 6 October 2026).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
the project lead's decision of 6 October 2026: the Mac runs nothing the network depends on. docs/plans/hands-on-build-1.md plans node 1
and the observer (its node and tools/observer) as systemd units on the box, one hand at a time, with ports, DNS names,
exec recovery, rollback and the decisions for main; infra/build-server/hands/install-hands.sh writes users, dirs, run
scripts and units (inert, run on the box); move-hand.sh (dry run by default) builds 0.3.15 on the box, copies the override
and snapshot, rsyncs each data dir hot then stopped, starts the unit and prints the hand's first executing line, copies the
observer env by scp (mode 600), stops the Mac's observer first, unloads the launchd agents last. CLAUDE.md's running-agents
rule carries the decision. lib.sh: a per-worktree lock on the box across sync, build and fetch (the shipper's collision at
18:48:56Z). remote-run.sh: the clean-tree test excuses stamps and target dirs at any depth with --untracked-files=all (a
nested stamp failed every checkout of /srv/builds/igneum for twenty minutes) and removes a stale .git/index.lock; the
self-test covers both. Plan rows for the three and the cuda prover pair built for the shipper.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The second build of every repo-kind crate in a subdirectory failed (6 October 2026, 18:51 UTC, the pool build: "tree not
clean after reset: ?? pool/.build-remote-sha-target"): checkout_tree keeps the stamps with `git clean -e` at any depth but
its status check matched them at the root only. The check now reads `git status --porcelain --untracked-files=all` (an
all-untracked directory is listed file by file, not as "?? sub/") and accepts target dirs, sccache and both stamps under
any path. The self-test carries the subdirectory case (stamp and target dir kept, overlay file removed) and the known-failed
case (a stray untracked file still fails the check).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The fleet's 10-member load run (6 October 2026, 18:14Z to 18:24Z) accepted 0 shares: the pool fork's miner re-checked GPU
shares with a fixed class v2 program while the 0.3.14 workers hashed class v3 (docs/plans/pool.md section 9).
pool: node.rs builds the share verifier's seeds from the template's pow_epoch with the class and the era, installs the
node's genesis day, dataset size and class v3 activation beside the schedule, and sends program_class, next_program_class,
era_seed, era_index, genesis_day_index, genesis_dataset_log2 and program_class_v3_activation_daa in `seeds` and
program_class and era_seed in `job` (protocol.rs, additive fields); verify.rs tests through EpochSeeds::v2. Protocol,
vardiff, PPLNS, stats API and page otherwise unchanged. Suite 22 of 22 on igneum-build-1.
igneum-pow tests/recheck.rs: for class v3 (packs-ca2-mixer/mx8-devnet-epoch0) and class v4 (packs-ca3-v4/v4-devnet-epoch0)
the pack's program reproduces the pack's 96 vectors (the worker's reference), the chain seam the node engine calls
(Epoch::chain_program, chain_dataset_day) builds the same program, one known nonce hashes equal through both paths, and
the fixed-v2 program of the same seed disagrees; the pinned v3 and v4 program ids differ. 2 of 2 on igneum-build-1.
packaging/hive: the pool:// mode merged with master's per-card IDENTITIES=auto and OVERRIDE handling (selftest passes).
pool/README.md and pool.md: building on igneum-build-1 (the vendor/igneum-node symlink on the box).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
proving/igneum-prove depends on vendor/igneum-node-exec/igneum/evm-types, a member of the fork's workspace that inherits
from the fork's root manifest; syncing that one directory left cargo without a workspace root on the box. lib.sh now groups
path dependencies by git top level: a repository under vendor/ is pushed to its mirror (a fork worktree to
/srv/igneum-node.git, a repository of its own to /srv/<name>.git, created on first use), checked out whole at
/srv/builds/<worktree>/vendor/<name> and overlaid whole; run-from-mac.sh wires every vendor repository the Cargo.toml files
reach. libprotobuf-dev added (sp1-prover-types imports google/protobuf/empty.proto). build-remote.sh no longer fails on a
default artefact the caller's own -p selection did not build. Proof on the box: igneum-prove-host 71,943,192 B, sha256
e9213e3a6c979512d7859f6d8e848105bab53f4355e99fb0d30fb4a72c2d5714, 1 min 05 s warm. Plan: gotcha rows for the case, the
protoc miss and one lost ssh session (collector cleared by test).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Mine, Earnings and Settings on igneum.network/miner are now the installed 0.3.14 app on PC 1 (RTX 5090 at 122 MH/s,
222 W, 0.55 MH/W, 56 C; RTX 4070 at 28.7 MH/s, 76 W; RX 9070 XT at 18.9 MH/s, 198 W; 169 MH/s at 532 W; Ember Tune
and Power control on), shot read only by the signed run job site-capture-pc1-1 with Edge headless and brought back
over the relay. Masked in pixels: the job's own strip cut out of the content column, the rail foot and the Settings
name field (hostname, address) painted over, the Earnings address box painted over. Captions say PC 1, the time, that
no electricity price is set and that the tune lines are stopped tunes from before that afternoon's Power Helper fix.
Prove, light and the phone width stay the Mac's (PC 1's Prove shows an exporter error line that belongs in a bug
report; headless Edge followed the Windows theme so its light set is dark; its 390 px window clipped the right edge).
The contact sheet is refreshed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
On 6 October 2026 the pre-push hook's site build fetched the live downloads index (0.3.14 since 17:47Z) and rewrote site/downloads.json and the stamped pages in five worktrees that had nothing to do with the release, as uncommitted edits to tracked files. A plain build and the hook now read live and warn when the snapshot lags; the ship step refreshes it with the flag and commits it. tools/ci/no-foreign-tree-writes.sh fails a script that builds a path from a worktree name, a glob over Projects or a worktree-list loop that writes; the eight absolute defaults into the shared checkout are listed as warnings until they move to env-only defaults. The journey, bench and index rebuilt from the merged tree.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The node pin moves to 4c6b129d (release-0.3.14-node), which is what the live payload inputs carry, so windows-ci's payload-inputs step is green again (red since e9eca23 on master's 0.3.13 pin against the 0.3.14 inputs).
Conflicts: infra/fast-time/override-60x.json keeps master's side (the fresh-rule field was already there at u64::MAX; the release line would be a duplicate key); tools/ci/playbook-quit-check.sh keeps master's rule 2 and pre-rule list with the release side's Ember allow entry and the 0.3.15 expiry check.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
remote-run.sh gains a checkout mode used by lib.sh: git checkout -- . and git clean -fd (target dirs, the sha stamps and
ignored files kept), fetch, branch at the commit, then a clean-tree check; the overlay follows. Before this, the rsync of
uncommitted files stayed in the box's tree and the next commit's git checkout -B refused with 'local changes would be
overwritten'. remote-run.sh --self-test reproduces the dirty tree (edited tracked file, untracked file, target dir, sha
stamp), shows the plain checkout refusing and the mode landing clean on the new commit; it runs in ci.yml and passed on the
Mac and the box; a live dirty-then-clean pair on the fork worktree passed too. Plan: section 5, gotchas of the first day.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The "LIVE DEVNET" scene on the home page showed blocks 21, proven 0, locked 3 and grey nodes while the devnet stood past
chain block 140,000 with 2,090 shards paid and a checkpoint locking every 30 s. Four causes, all in site/index.html:
- "blocks" was the scene's own spawn counter, never the chain;
- "proven" counted blocks of the 90 s window whose shards were all paid, and proofs land a median 380 s behind the tip
(median_proof_lag_s), so the window can never hold one;
- "locked" counted locked checkpoint blocks inside the same 90 s window (always about 3);
- the live path skipped the colour state machine, so every real block stayed grey, and a failed first fetch left the
simulation running under a label that looked live, with no further polls.
Now: /api/live carries state.height (the chain block number, the same field /api/stats reports, one indexed subquery);
the scene's counters read chain block (state.height), shards proven (proving.paid_shards_total, the node's count of paid
shard records, with the 10-minute count and the median lag in the note) and last lock (finality.latest_locked_index);
live blocks take the observer's own words (pending, included, excluded, proven, locked checkpoint, final to its left),
refreshed on every poll, with a legend in those words; the hero tile shows the chain block instead of the node's
held-block count; polling never stops (2 s while fresh, 10 s while off) and the off state is labelled "simulated preview ·
live feed unavailable" or "observer offline, last update N ago" with a note that the counters count simulated blocks.
Also found while verifying: the scene advanced a fixed 16 ms per frame, so it ran at double speed on 120 Hz displays;
it now uses the real frame time, capped at 50 ms.
/live had the same window bug in its "proven 0/16" counter; it now reads the observer's 10-minute truth (chain blocks
with every shard paid, of the chain blocks planned in 10 min) with the lag and the chain total in its title.
Verified against the live API through tools/site-serve.mjs in the built-in browser and headless Chromium:
chain block 140,489, shards proven 2,130, last lock #6,784; the failed-fetch state shown by blocking /api/live.
Screenshots: docs/plans/site-ui-3-shots/item0/ (1440 and 390, live and failed). Checks: identity grep 0 hits,
link check 708 links 0 broken, site/api tests 14 pass, site build clean.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Real screenshots of the live 0.3.14 app on the Apple M5 Max (paused, Ember Tune off, as the project lead left it), shot headless
at 1440x900 and 390 px, dark and light, the rail foot, the address and the machine id masked: miner-dashboard.webp
(the hero, same name), miner-earnings, miner-prove, miner-settings, miner-dashboard-light, miner-phone.
miner.html: a page-by-page section (the card row cell by cell, three page shots, light and the phone), the lead and
the meta copy name the four pages and pounds a day at the user's price, the Ember Tune feat carries the app's one
sentence and the measured PC 1 rows from the 6 Oct log (5090: 311.0 to 226.8 W for 0.15% of rate; 4070: 106.0 to
75.6 W for none; the floor, not the optimum), the Power control feat (one approval, then the helper task, no prompts),
the dev fee moved to Earnings as in the app. index.html: the same hero shot, two pills. build.mjs: the caption no
longer says 'until 0.3.6' (0.3.14 still says Igneum Miner). The 9070 XT row, the market price for pounds earned and
the bench anchor (the run 6 entry is on the ship0314 branch) are marked as owed or coming.
relay/playbooks/site-capture-pc1.ps1: a read-only run job that shoots the installed app's UI on PC 1 with Edge
headless and drops the PNGs on the relay (no quit, pause, resume or settings change; passes the playbook checks).
docs/plans/site-miner-2026-10-06.png: before and after.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
tools/workers/collect.mjs runs every 30 s on the box (infra/build-server/workers/igneum-workers.{service,timer},
install.sh) and writes /srv/workers/workers.json from the machine itself: /proc/stat deltas per core, meminfo, df on
/srv, net bytes, hwmon temperatures, the flock state of /srv/builds/_locks, cargo processes with worktree, target and
start time, flock waiters, /srv/builds/_log/builds.jsonl (the format agreed with the build-server agent), sccache
--show-stats, headline.json. tools/workers/push.mjs (launchd every 60 s) adds the Mac's with-lock slots and waiters
and the two PCs' relay job states from the intake, drops them on the box so its file is whole, merges the box's file
and writes the fleet folder's workers.json; it deploys only when the live copy is over 6 min old, otherwise the
fleet orchestrator's 5-minute deploy carries it. The page (tools/workers/page/workers.html, shape.js) tries
https://build.igneum.network/workers.json first and falls back to the folder copy; core strip, arcs, now building,
queue, recently done with closing lines, headline timings, analytics; UK time with UTC tooltips; phone width.
node --test tools/workers/test: 13 tests over /proc/stat, lock dir, JSONL and sccache fixtures and the page shaping.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
