The sweep (main's item 1): 199 tracked text files, 783 lines. The founder's full name, first name and possessive become "the founder" (sentence starts capitalised); the lowercase operating-system user name in WSL paths and commands becomes <user>; the second owner login becomes "the second owner login"; the three earlier businesses and the two other brands become "the other business", "the earlier entity", "the earlier business" and "another brand"; the Chrome profile rule names the igneum.network profile, not the profile's label. The standing commit login igneum-labs is not a founder term here: the fresh-repository step renames it in the history (docs/plans/history-rewrite.md, tools/repo/fresh-repo.sh). The patterns never appear in plain text in the tree (a plaintext list would be the hit): tools/ci/founder-strings.b64 (perl regex, tab, a sample per row) is read by tools/ci/founder-strings-check.sh (every tracked text file, perl, known-failed first: the self-test plants each row's sample in a fixture and the hit must name the file), by tools/community/discord-hooks.mjs (the guard's founder and business rows; the test takes its fixtures from the samples) and by tools/repo/fresh-repo.sh (the business names of the rewrite rules). site/forbidden-strings.txt carries the same patterns as b64: lines, decoded case-insensitive by site/scrub.mjs and tools/ci/launch-gates-check.mjs (whose fixture now plants an encoded made-up name). The check runs in the gate's tree checks on every merge. Not in this commit, by main's word: the 105 commit messages and 40 personal-identity commits that need the history rewrite (listed, not run), and the secrets found by gitleaks over the history (reported with owners). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
81 KiB
Igneum Miner 0.3.10: the certificate-driven reorg, transaction gossip, the pack loader fix, the six-section miner and the PC-built Windows node, 5 October 2026
Shipped: manifest 0.3.10 published 21:32:40Z, every reachable machine on 0.3.10 by 21:49:41Z, the hand nodes and the seed on 21d4c73c by 21:50:15Z, one digest (1f4b4425...) on every node that restarted; PC 37ba0461 (the owner's laptop) mid-install and Sam's Mac quit at the time of writing. The cut ran from 16:39Z to 21:5xZ, two hours of it GitHub's hosted-runner outage.
Release engineer, from 16:39 UTC. Worktree /Users/joshm/Projects/igneum-wt-ship0310, branch release-0.3.10 from master
after the 0.3.9 merge. Fork worktree vendor/igneum-node-0310, branch release-0.3.10 from a24ab01a (the 0.3.9 fork tip).
Every Mac build through /Users/joshm/Projects/igneum/tools/lock/with-lock.sh build at nice -n 19 with -j 4; every PC
job through the main checkout's tools/build-job.mjs and packaging/ota/publish-jobs.sh (the signed envelope). Times are
UTC. release-0.3.9.md and release-0.3.8.md are the template; release-0.3.6.md section 10 for the Windows acceptance.
The rollout waited for 0.3.9's (shipped about 17:00Z, fee switch published about 17:50Z): nothing of 0.3.10 moved on the
network before every node's digest was ab8847da...
1. What 0.3.10 carries
| Change | Where | Fork commit |
|---|---|---|
A. The windows-gnu node links the C++ runtime statically (database/build.rs, rocks-probe), so the PC-built Windows node runs (release-0.3.6 plan section 10) |
fork housekeeping 4fb32865 |
merged 2f88a82f |
B. TestConsensus sets the unbounded PoW cache build queue, so the five node suites pass in parallel on PC 2 |
fork housekeeping 7003055b |
merged 2f88a82f |
C. The certificate-driven reorg (ledger C4, measured in FUD round 6, 75c6658: a certificate over an off-chain block stayed pending and the heavier side locked alone): a verified certificate over a block off the node's selected chain now locks it there and moves the virtual to the heaviest tip through it (spec 3.5 F1 and F2); the block relay takes a lighter block a pending certificate names. A consensus-behaviour change with NO digest change: it converges when every node is new (the c4 agent's rollout note, section 10) |
fork c4-fix e18f1e0e (its message records PC 2 job build-20261005-180827: kaspa-consensus 97 passed, kaspa-consensus-core 101 passed, three new tests), main c4-fix 68f14b9 (spec 3.5, 3.2, 3.10, 3.11.7, ledger, bench-log, c4.mjs v2 mode; the signer-pipe-check CI script and per-job build-inputs zip names) |
fork: merged 21d4c73c; main: 0cbaf3a and the tip merge |
Changelog line for C (the coordinator's words): "A node that comes back from a partition now follows the certified checkpoints, even when its own chain is heavier."
| D. EVM transaction relay between nodes: three p2p messages after Kaspa's transaction inv/request/answer, PROTOCOL_VERSION 13 to 14 (13 peers still connect: a v14 node never sends the new messages to an older peer), the mempool caps and faults, the block-added hold instead of the 4-s hand-out cooldown; the digest is unchanged, so no activation height and no fresh chain | fork tx-gossip e242acd0 (its message records PC 2: igneum-exec 15 of 15, kaspa-p2p-flows 33 of 33), main tx-gossip 0166e94 (the relay-net harness tools/txgen/relay-net.mjs, the design note); acceptance here: node tools/txgen/relay-net.mjs --rate 2 --duration 120 --wallets 16 --fund 2 on the 0.3.10 Mac node, expected 240 of 240 included | fork: merged 21babaa7; main: (pending the worktree) |
Changelog line for D (the coordinator's words): "Transactions now travel between nodes; a miner on an older node still carries only what was sent to it directly."
| E. GPU hot-plug (coordinator, 17:5xZ): cards re-enumerated every 60 s and on WM_DEVICECHANGE, new cards mine by default, Code 43 cards marked unusable, vanished cards dropped without shifting slots, the Ryzen gfx1036 iGPU classified integrated and off by default, a cards: line the relay parses (machines API gpus/hotplug). 14 files: app/igneum-app/src/{detect,engine,hotplug,main,state}.rs, ui/{app.css,app.js,notices.test.mjs}, app/windows/host.cpp, relay/{api/console.mjs,lib/parse.mjs,test/parse.test.mjs,ui.html}, tools/console.mjs; no proving, packaging or workflow file. Its Windows-only paths (detect::adapters, host.cpp WM_DEVICECHANGE) were never compiled on the Mac: PC 1's Windows app build and the GitHub runner's host build are their first compile and are read for errors | main gpu-hotplug 4517004 (igneum-wt-hotplug, on master 22ffc02); its tests on the branch: igneum-app 91 pass, notices, update-card and parse node tests pass | merged after C and D |
| F. Elevated jobs that never started (coordinator, 18:0xZ): an elevated job whose UAC prompt is refused, cancelled or times out fails with exit 251 and the summary "did not start: the administrator prompt was refused, cancelled or timed out"; before, Start-Process threw, $p stayed null and exit $p.ExitCode exited 0, so the PC 1 driver job at 17:58Z read as done. One file, app/igneum-app/src/jobrun.rs, plus its unit test | main elevated-exit 9816d58 (igneum-wt-elevated) | merged after E |
| G. The miner UI redesign (the founder, through the coordinator at 19:0xZ: into 0.3.10, not 0.3.11): six sections (Mine, Prove, Rewards, Node, Updates, Settings), one switch per card, one line of help per setting, the node's next switch shown with its height; three engine additions with tests (state.rs node.consensus_digest and node.consensus_switches, engine.rs, prover.rs program ids via --mode id), a Mac window minimum of 900 x 600, one WM_GETMINMAXINFO case in app/windows/host.cpp, view.test.mjs in CI | main miner-ui-2 (igneum-wt-miner-ui, 83a293f and 364feef, then the commit its agent reports after resolving it against gpu-hotplug's card-row states) | merged LAST, after F, at the agent's final commit; the node stays 21d4c73c, so the app, the DMG, CI and the ship files are rebuilt (section 7a) |
| H. The pack loader fix (the outage of 18:31Z, root cause found by the coordinator's agent): the generator retries a rejected candidate program with seed || k (epoch 34's attempt 0 was rejected on the saturation rule, attempt 1 accepted) and the workers' pack loader (proto-cuda/nvrtc/packfile.h, shared by the OpenCL worker) derived the expected seed words from attempt 0, so every worker started after the boundary refused a correct pack. Changes packfile.h, worker.cpp, proto-opencl/host.c, the relay's parse and tests, an emu test. It changes the Windows WORKER binaries: since 0.3.6 the payload has carried only the worker sources (host.cu, host.c, build.bat) and the PCs built the workers themselves, so the fix reaches the PCs only as exes: both workers cross-built on this Mac (proto-cuda/nvrtc/build-windows.sh, mingw, the staged redist) and carried by push-inputs.sh into the installer; a hot-fix copy is already on both PCs under packs\\workers-fix. The CI builds no worker (windows.yml copies the two worker exes and the nvrtc*.dll files from the signed inputs only), so the rebuilt pair reaches the installer through push-inputs.sh and nowhere else; the engine takes a bundled worker before its own PC-built one (detect.rs Bins, engine.rs build_worker_from_source only "when no prebuilt worker ships"), so the fix applies at the first start after the update | main pack-loop af983a7 (igneum-wt-pack-loop; the tip at merge time) | merged after G |
| I. The export lock: engine.rs serialises igneum-miner export-pack behind EXPORT_LOCK around both call sites (two per-card export threads interleaved into one folder across an epoch change on PC 1, a second way to a wrong pack) | main opencl-rdna4 a08c371: the lock hunk is the part that must ship; the rest (host.c's duplicate-platform fold, --readback select, --memprobe, detect.rs parse_opencl_list) only if it merges cleanly with hotplug and the OpenCL worker cross-builds without error, else the hunk alone (coordinator, 19:2xZ) | (decided at the merge, section 2) |
| NOT in 0.3.10 unless the coordinator says so: job-console d33a266 (one hidden-console builder for every elevated launch, the spawn check in CI, PC 1 console watchers; its agent's message at 18:5xZ) and opencl-rdna4 a08c371 (its agent's message at 19:1xZ: the OpenCL worker folds the duplicate AMD platform out of --list, a GPU select pass, --memprobe, detect.rs parsing with tests, and engine.rs serialises export-pack behind EXPORT_LOCK because PC 1's packs\devnet has held a mismatched program.h and seeds.txt since 19:07Z: the pack-export class the rollout is held for; passed to the coordinator as a candidate) | the coordinator's rule: a late branch goes in only if it lands before the final tree; the final tree b958493 was pushed at 18:36Z with CI green at 18:42Z. job-console also moves elevated-exit's exit-251 launcher text to platform.rs, so its include_str! test is repointed at its own merge to master after 0.3.10 | next cut |
| The app engine otherwise | unchanged except the version (0.3.10 in the six files); the signed jobs envelope client is master's (housekeeping C, 318a2de) | |
Changelog line for G (the coordinator's words): "The miner is now six sections: Mine, Prove, Rewards, Node, Updates, Settings. One switch per card, every setting with one line of help, the node's next switch shown with its height."
Changelog line for F (the coordinator's words): "A remote job that needs administrator rights now reports when the prompt was not accepted instead of claiming success."
Changelog line for E (the coordinator's words): "New cards are picked up while the miner runs; a card that goes away is released; integrated GPUs stay off unless you switch them on.
| The pinned guest | UNCHANGED unless C or D touches proving/igneum-prove/core or elf/ (checked at the merge: section 2) | |
2. The branch
| Commit | What |
|---|---|
9268b64 |
origin/master at the worktree's creation (18:00Z: the 0.3.9 merge, fast-forwarded) |
2e943a8 |
Merge tx-gossip (0166e94): the relay design note, rows 463 and 9, the 3-node relay harness and its evidence; no conflict |
824059b |
Igneum Miner 0.3.10: the six version files (node tools/ship-app.mjs --check: 0.3.10 in all 6) |
9997ef1 |
packaging/mac/packaged-config.sh: the four-field override object in the packaged line, the self-test fix (section 3) |
0cbaf3a |
Merge c4-fix (3072919, 18:20Z). One conflict, docs/bench-log.md: both entries kept (the fee-table entry, then the C4 entry). Brings tools/ci/signer-pipe-check.sh (igneum-ota-sign ... | head -1 under pipefail panicked the signer with SIGPIPE on a loaded Mac and killed a publish; now sed -n 1p in publish-jobs.sh, publish-manifest.sh, push-inputs.sh) and one build-inputs-<time>-<pid>.zip per build job (build-job.mjs, push-build-inputs.sh --name: with the shared name a job pinned another agent's sources three times tonight). From here every PC job and publish runs from THIS worktree's tools: they carry master's signed envelope (housekeeping C) and these two fixes, which the main checkout lacks until this branch merges |
4d10c20 |
Merge gpu-hotplug at its tip at merge time, 4d122e1 (the agent's second commit: one row per physical GPU across OpenCL platforms, Windows names on the rows, index-free card keys, the OpenCL worker's --list prints the PCI address in proto-opencl/host.c); no conflict, 15 files |
5520fb1 |
Merge elevated-exit (9816d58); no conflict |
d1f4923 |
Merge origin/master a93199a (docs and site only) |
e0a5fd1 |
infra/cross/build-linux.sh: the three paths made absolute before the cd (section 3) |
065c67c |
packaging/windows/node-source.pin = 21d4c73c (push-inputs, section 5c) |
b958493, ccd2b98 |
the site as the pre-push hook builds it; the plan so far. The FIRST final tree: pushed 18:36:44Z, CI green 18:42Z, installer and DMG built (section 7). Then the founder's scope change reopened it for G |
7f9618a |
Merge miner-ui-2 at its agent's final commit a3d9f2d (19:2xZ; it already carries gpu-hotplug 4d122e1 merged and resolved onto the new UI). One conflict, packaging/windows/push-build-inputs.sh: the usage comment only (c4's --name text against miner-ui-2's --no-node text; both options were already in the code), both kept |
5abc709 |
Merge pack-loop (af983a7, its tip at merge time). One conflict, relay/test/parse.test.mjs: the import line (the union: parseAppTail, parseCardsLine from hotplug, PACK_MISMATCH from pack-loop) and two tests that both landed at the file's end (hotplug's cards line, pack-loop's pack mismatch), both kept; node --test relay/test/parse.test.mjs: 7 pass |
854f9a8 |
engine: one pack export at a time: the EXPORT_LOCK hunk of opencl-rdna4 a08c371 (git diff origin/master...a08c371 -- app/igneum-app/src/engine.rs, applied clean: 9 lines, a static mutex and a guard at both export-pack call sites). The rest of that branch conflicts with hotplug in detect.rs and proto-opencl/host.c (a dry-run merge at 19:1xZ), so by the coordinator's rule it waits for the next cut |
854f9a8 |
the fast checks on the SECOND final tree, 19:24Z: identity 0 hits over 213 files, copied-sources, pinned-guests, signer-pipe-check ok, no-conflict-markers ok, workflow shell 0 findings, --check 0.3.10 in all 6, relay tests 17 pass, UI tests 23 pass (notices 12, update-card 6, view 5; view.test.mjs in ci.yml line 85) |
5520fb1 |
the fast checks on the first final tree: identity 0 hits over 212 files, copied-sources, pinned-guests, signer-pipe-check ok, no-conflict-markers ok, workflow shell 0 findings, --check 0.3.10 in all 6, relay tests 16 pass (hotplug's parse test added), UI tests 12 pass; 18:21Z |
A vendor symlink to the main checkout's vendor/ (untracked) makes the relative node paths resolve, as in the earlier cuts.
The app and prover target dirs were cloned by APFS from the 0.3.9 worktree (18:01Z).
The fork branch release-0.3.10 in vendor/igneum-node-0310, from a24ab01a: 2f88a82f (housekeeping 4fb32865), 21babaa7 (tx-gossip e242acd0), 21d4c73c (c4-fix e18f1e0e: finality.rs, blockrelay/flow.rs and five small files, 601 insertions), all three without conflict. Neither c4-fix nor tx-gossip touches a params file, proving/igneum-prove/core or elf/: the pinned guest and the prover rollout are untouched by 0.3.10.
3. Tests and checks, with the command
| Tip | Command | Result |
|---|---|---|
| fork 2f88a82f (a24ab01a + housekeeping) | PC 2 job build-20261005-164604 (node tools/build-job.mjs run --node vendor/igneum-node-0310 --target 1ccfe586 --targets linux --node-tests "kaspa-consensus kaspa-consensus-core igneum-exec kaspa-pow igneum-miner" --no-app) |
started 16:46Z, done 16:48:43Z (129 s): Linux node built, RESULT test node [the five packages] exit 0 39 s: the parallel five-package run is green on the housekeeping tree (housekeeping B); the short report carries the per-package exit, not the per-test names |
| fork 2f88a82f | PC 1 job build-20261005-164512 (--targets linux,windows --no-tests --no-app --no-place) |
started 16:45Z, done 16:50:26Z (292 s), every stage ok, 7 files, all sha256 and PE checks ok: igneumd.exe 37d0b1045043bc86... (50,889,728), igneum-miner.exe bc8cb3a12111e7bb... (10,725,888), Linux igneumd d0331c109babe77e... (48,733,736, glibc 2.39: HiveOS, not the seed). strings igneumd.exe: msvcrt.dll is the only C runtime named; no libstdc++-6, libgcc_s_seh-1 or libwinpthread-1 (housekeeping A) |
| fork 2f88a82f, PC 1 exe | run job run-0310-accept-hk (the housekeeping agent's acceptance script: rocks-probe, igneumd 60 s on a scratch appdir with no mingw DLL beside it, igneum-miner.exe key-hash probe, Application event 1000 check), published 16:51:01Z (the apps woken) |
ran 16:57:15 to 16:58:21Z (66 s), exit 0: rocks-probe exit 0 (21 files in its db), RESULT run node60: still running after 60 s (alive for the whole wait); killing it with 14 files written under the scratch appdir, igneum-miner.exe key-hash probe exit 0, RESULT event 1000: none; the exe's imports on the PC (objdump): KERNEL32, advapi32, bcrypt, iphlpapi, msvcrt, ntdll, ole32, ws2_32 and the api-ms-win-core set, no mingw DLL. The one warning, cannot bind the eth_ JSON-RPC server on 127.0.0.1:26790, is PC 1's live node holding the port, as in the housekeeping run. So the PC-built Windows node is the 0.3.10 Windows node (no Mac cross-build needed) |
| fork 2f88a82f, Mac arm64 | CARGO_TARGET_DIR=vendor/igneum-node/target-0310 cargo build --release -j 4 -p kaspad -p igneum-miner --features kaspad/igneum-pow from vendor/igneum-node-0310 under the lock (target dir cloned by APFS from target-036; a new worktree path rebuilds every crate) |
16:44:52 to 16:57:28Z (12 min 36 s): igneumd 2a7df6245ffe047a... (40,968,368, 2f88a82f in its strings), igneum-miner 322472ae95a316d1... (8,514,928) |
| fork 2f88a82f, Mac arm64 | the digest check: that igneumd on a scratch node (ports 60975/60976, 22 s, 16:57:58Z) with {"difficulty_v2_activation_daa":33000,"proving_v0_activation_daa":84100,"fees_v1_activation_daa":210000} |
Calibrated v1 fees from the override file: ... from DAA score 210000, digest ab8847da538dead1dc10e046dfaadab3c1c35928e3748810c4e050d4a886087a, igneumd/2.1.0-2f88a82f: housekeeping moves no parameter |
| fork 2f88a82f, Linux x86-64 for the seed | NODE_SRC=vendor/igneum-node-0310 TARGET_DIR=vendor/igneum-node/target-0310-linux infra/cross/build-linux.sh (zig, glibc 2.36 target) under the lock | queued 16:57Z behind two other agents' builds, built 17:23 to 17:35:49Z (754 s, 564 crates), and its output was WRONG: igneumd 7e26e374... BYTE-IDENTICAL to the 0.3.9 build of a24ab01a, embedding a24ab01a. Cause (found at the second run, 18:25Z): build-linux.sh does cd "$NODE_SRC" and runs cargo with CARGO_TARGET_DIR="$TARGET_DIR", so a RELATIVE target dir resolved inside the node worktree (vendor/igneum-node-0310/vendor/igneum-node/target-0310-linux, the untracked vendor/ that appeared there), while the copy step read the repository-relative clone that still held the a24ab01a binary. Every crate had compiled, into the wrong place. Fixed on this branch (e0a5fd1: the three paths made absolute before the cd) and the build rerun with an absolute target (section 5). My first reading of it, a stale kaspa-build-info output in the cloned target, was wrong and is withdrawn |
| fork 21babaa7 (2f88a82f + tx-gossip e242acd0), Mac arm64 | the same cargo build, incremental on target-0310 | 17:49:52 to 17:52:00Z (2 min 08 s): igneumd 5d7f1b576029b462... (41,118,944, 21babaa7 in its strings) |
| fork 21babaa7, Mac arm64 | the digest check as above (ports 60975/60976, 22 s, 17:52:01Z) | Calibrated v1 fees ... from DAA score 210000, digest ab8847da538dead1dc10e046dfaadab3c1c35928e3748810c4e050d4a886087a, igneumd/2.1.0-21babaa7: tx-gossip moves no parameter (its commit message says the protocol version is not in the digest; measured here) |
| fork 21babaa7 | PC 2 job build-20261005-175022 (--node-tests "kaspa-consensus kaspa-consensus-core igneum-exec kaspa-pow igneum-miner kaspa-p2p-flows" --no-app) | started 17:50Z, done 17:53:10Z (145 s): RESULT test node [the six packages] exit 0 53 s (kaspa-p2p-flows added: tx-gossip's relay tests live there and in igneum-exec) |
| 2e943a8 + bump | tools/ci/identity-check.sh (0 hits over 212 files), copied-sources-check.sh, pinned-guests-check.sh (elf/ matches its manifest), node tools/ci/check-workflow-shell.mjs (12 run blocks, 25 .ps1, 0 findings), the relay tests (15 pass), the UI tests (10 pass) | all green, 18:00Z |
| fork 21babaa7, Mac arm64 | the digest check with the FOUR-field object {"difficulty_v2_activation_daa":33000,"proving_v0_activation_daa":84100,"fees_v1_activation_daa":210000,"finality_v3_activation_daa":135200} (N3 = 135,200 from the live 0.3.9 manifest republished 17:59:14Z, deadline note "finality v3"; the founder: "deploy N3 now", the 0.3.9 agent publishing it) | 18:01:59Z, 22 s: Finality rule v3 from the override file: active from checkpoint DAA score 135200, Calibrated v1 fees ... 210000, digest 1f4b44255fcd2ea8f75664ed47200f409186ddd2292960c9e2cf95bbbdc11505, igneumd/2.1.0-21babaa7. Equal to the live hand nodes' lines at 18:02Z (observer-v4.out, node1.out: digest 1f4b4425..., igneumd/2.1.0-a24ab01a, the four-field /tmp/igneum-devnet/override-v3.json): the 0.3.9 agent's N3 rollout had reached them. This is the digest every node must show; the final 0.3.10 node is read again against it |
| 9997ef1 | packaging/mac/packaged-config.sh: NODE_OVERRIDE_PARAMS = that four-field object (it was empty on master; the manifest's consensus.override is what the apps write, the packaged line covers a first start before any manifest). Its self-test's "json has no override line" check read the file's shipped default and failed on a non-empty line (master's empty line passed it), so that check now passes NODE_OVERRIDE_PARAMS='' itself, as the "override params land" check already passes its own value; bash packaging/mac/packaged-config.sh --test: all checks passed |
| fork 21d4c73c (+ c4-fix), app 5520fb1 | PC 2 job build-20261005-182244 (--node-tests "kaspa-consensus kaspa-consensus-core igneum-exec kaspa-pow igneum-miner kaspa-p2p-flows" --app-tests igneum-app) | 18:23:15 to 18:25:41Z: the node stage FAILED, exit 101 after 44 s: kaspa-consensus 96 passed, 1 failed, 3 ignored: processes::finality::tests::ban_is_decided_by_the_carrying_block_so_nodes_agree_on_every_voter_list panicked at finality.rs:1883:82 (the mine_on_all helper's validate_and_insert_block(...).unwrap()) with UnexpectedDifficulty(<bits>, 487112096, 487129281): a block built on one TestConsensus was refused by another for a difficulty a few hundred bits off. cargo stopped at the first failing package, so the other five were not run in this job. The app tests passed: igneum-app exit 0 in 6 s (hotplug's and elevated-exit's tests included; 96 + the signer and wrapper suites) |
| fork 21d4c73c | PC 2 job build-20261005-182804 (--node-tests kaspa-consensus, the suite ALONE) | 18:28:3x to 18:30:15Z: kaspa-consensus 97 passed, 0 failed, 3 ignored in 1.99 s, ban_is_decided_by_the_carrying_block_so_nodes_agree_on_every_voter_list ... ok. So the test passes alone (as in the c4 agent's own run, build-20261005-180827, 97 passed) and failed once under the six-package parallel run: the 0.3.6 isolation class (release-0.3.6 plan 8f), now with a difficulty expectation that depends on wall-clock timing rather than the PoW cache queue. Not a consensus regression by the same reasoning as then; recorded in section 11 for the c4 agent (the test or the helper should pin its clock) |
| fork 21d4c73c | PC 2 job build-20261005-183131 (--node-tests "kaspa-consensus-core igneum-exec kaspa-pow igneum-miner kaspa-p2p-flows") | 18:31:31 to 18:33:59Z: exit 0 in 37 s: kaspa-consensus-core 101 passed (2 ignored), igneum-exec 15 passed, igneum-miner 15 passed, kaspa-p2p-flows 33 passed (the target compiles and passes on the merged tree: the nine epoch_seed_headers errors of release-0.3.6 are gone with tx-gossip's ibd/proof.rs update, as the c4 agent said), kaspa-pow 13 passed (both queue tests of housekeeping B). With build-20261005-182804 (consensus alone, 97 passed) every package of the six is green on 21d4c73c; the app suite passed in build-20261005-182244 |
| tree 9a51e32+ (before pack-loop) | proto-cuda/nvrtc/build-windows.sh under the lock (mingw-w64 from Homebrew, the redist dir of the main checkout symlinked into the worktree: NVRTC 12.8.93 DLLs and headers, Khronos CL headers, nothing downloaded) | 19:17 to 19:18:17Z, a trial of the script before the pack-loop merge: igneum-worker-cuda.exe 196082e2... (1,509,376) and igneum-worker-opencl.exe 0981c77d... (444,928), both with the Igneum resource block (whose version string is 0.3.0: the .rc files are not among the six version files, section 11); the real pair is rebuilt from the final tree (section 5d) |
4. The state of the network before the cut
Both of the 0.3.9 agent's rollouts finished before anything of 0.3.10 moved: the fee switch (H = 210,000, every node on
ab8847da..., release-0.3.9.md 10e, 17:55Z) and then N3 (the founder: "deploy N3 now"; finality_v3_activation_daa 135,200,
manifest republished 17:59:14Z, docs/plans/finality-v3-devnet-publish.md): the sweep there lists every live node on
1f4b44255fcd2ea8f75664ed47200f409186ddd2292960c9e2cf95bbbdc11505 (the observer 17:58:13Z, node 1 17:58:25Z, the seed
17:58:43Z, PC 2's app node 18:04:57Z, PC 1's app node 18:06:28Z). The override object every node runs, and the one the
0.3.10 manifest and packaged line carry:
{"difficulty_v2_activation_daa":33000,"proving_v0_activation_daa":84100,"fees_v1_activation_daa":210000,"finality_v3_activation_daa":135200}
| Node | Binary at 18:03Z | Override | Digest |
|---|---|---|---|
| Mac d937c69d | app 0.3.9; its engine has run NO node of its own since its hourly restart at 17:45:16Z: it found node 1 answering on 127.0.0.1:26610 and uses it (the N3 record, "the Mac app's external-node finding"), so the Mac card's node line is node 1's | node 1's | node 1's |
| PC 1 ae432dc7, PC 2 1ccfe586 | app 0.3.9, node a24ab01a (the Mac cross-build) | the four-field object | 1f4b4425... |
| Sam's Mac 3a9bf309 | app 0.3.9, node a24ab01a, 0.0 MH/s at 18:03Z | (its app writes the manifest's object) | not read |
| The observer, node 1 (hand nodes, this Mac) | vendor/igneum-node-036/target-integration/release/igneumd a24ab01a |
/tmp/igneum-devnet/override-v3.json, the four-field object |
1f4b4425... (read 18:02Z) |
| The seed 188.245.5.161 | /opt/igneum/v4/bin/igneumd 7e26e374... (the a24ab01a zig cross-build, glibc 2.34) |
/etc/igneum/override-v3.json, the same |
1f4b4425... |
| PC 37ba0461 | silent (0.3.7 at the 0.3.8 cut) |
Master moved under the branch while it was cut (fef98a2/a850aef CLAUDE.md, 900bba7 the site's 0.3.9 download cards, 46a6a4e and
bf8d1ba the N3 record, a93199a); merged as d1f4923 (docs and site only, 5 files, no conflict). It moved again at 19:4xZ (the coordinator's
explorer merge, origin/master 9746391: the observer's explorer detail, site/api/stats.mjs and supply.mjs, the explorer pages, three node
tests and tools/ci/public-api-check.mjs in ci.yml; no app, node or packaging file): the final merge to master lands on it (section 7b).
5. The node binaries and the DMG (fork 21d4c73c, app 5520fb1 and later)
| Platform | Build | sha256 | Size | Notes |
|---|---|---|---|---|
| Mac arm64 igneumd | CARGO_TARGET_DIR=vendor/igneum-node/target-0310 cargo build --release -j 4 -p kaspad -p igneum-miner --features kaspad/igneum-pow from vendor/igneum-node-0310, under the lock, 18:21:54 to 18:24:2xZ (incremental) |
4bb356f492a52154c932ee6f802cd59dc5c917840ad46ba2133905f52b838b3c | 41,152,144 | 21d4c73c in its strings; copied into vendor/igneum-node-0310/target-integration/release/ |
| Mac arm64 igneum-miner | same | 322472ae95a316d12281fed99b2112cdb5f7b6caf9f25ea24019219b7e907948 | 8,514,928 | byte-identical to the 2f88a82f build: the miner is untouched by gossip and c4 |
| The digest check | that igneumd on a scratch node, ports 60975/60976, 22 s, 18:24:23Z, the four-field object | Finality rule v3 ... 135200, Calibrated v1 fees ... 210000, digest 1f4b44255fcd2ea8f75664ed47200f409186ddd2292960c9e2cf95bbbdc11505, igneumd/2.1.0-21d4c73c |
equal to every live node's (section 4): the three fork branches move no parameter; the chain script stops on any other value | |
| Linux x86-64 igneumd (the seed, glibc 2.36) | NODE_SRC=<abs> TARGET_DIR=<abs> OUT_DIR=<abs> infra/cross/build-linux.sh (zig), under the lock, 18:25:43 to 18:27:29Z (105 s incremental) |
40be0e142e30f6de89ccb61f6dc13e8748ed2a41988753604dbdbb9da1466b4b | 47,638,184 | GLIBC_2.34 at most, 21d4c73c in its strings; version.txt from 21d4c73c (release-0.3.10). Kept in the scratchpad r0310/cross-final2/ and copied to infra/cross/out-0310/ of the main checkout for restart-seed.sh |
| Linux x86-64 igneum-miner | same | 6ec3536717536fd505b63eef5d01ec9fa19d1fe608ba86398135c5776222ec73 | 9,631,280 | |
| Windows x86-64 igneumd.exe | PC 1 job build-20261005-182405 (IGNEUM_WIN_RELEASE=<fork>/target-integration/x86_64-pc-windows-gnu/release node tools/build-job.mjs run --node vendor/igneum-node-0310 --target ae432dc7 --targets linux,windows --no-tests from this worktree: its own build-inputs-20261005182334-74461.zip, the per-job name of 68f14b9), published 18:24:05Z, started 18:26:1xZ after another agent's job on PC 1, every stage ok 18:31:43Z (linux 136 s, windows 162 s, pack 3 s; 9 files, 45 MB, all sha256 and PE checks ok, placed) |
3adb01a712fba678706ed5eeb1fa9788895735e4fab5e81dbefe03bd675a5e04 | 50,978,816 | the PC build (housekeeping A); no commit string inside (section 11); acceptance: section 5b |
| Windows x86-64 igneum-miner.exe | same | 1105151c91738a4e177b8f327625c6e22a90423fe66a71a5d045130618281052 | 10,725,888 | |
| Windows x86-64 igneum-app.exe (the PC's build; the installer's engine is the GitHub runner's) | same | 4ead820eb4c502de18f8c122d319543452615ba96b8471a501eb2237e76835b7 | 2,925,056 | hotplug's first Windows compile (detect::adapters, the WM_DEVICECHANGE path is host.cpp, built by the runner): warnings only; among them hotplug's own function adapter_for is never used (for its agent), the rest pre-existing (94 trailing semicolon in macro, p2p-flows 21, dead keep/wsl_path) |
| Linux x86-64 igneumd (HiveOS, glibc 2.39, not the seed) | same, infra/cross/out/ of this worktree |
38e69412b66a9ee183f3a28f25198f31ce29f83554f7a7f7dce9bdc4f61970be | 48,915,112 | |
| Linux x86-64 igneum-miner, igneum-app | same | 39efcb1773dc215d..., cd8680e5b8c07795... | 9,607,448; 2,370,544 |
5b. The acceptance of the PC-built Windows node (housekeeping A)
Run job run-0310-accept-final (the housekeeping agent's run-accept-pc1.ps1 unchanged: copies the three exes from PC 1's
/root/igneum-build/target/x86_64-pc-windows-gnu/release into a scratch folder with NO mingw DLL, runs rocks-probe, then
igneumd.exe --devnet --nodnsseed --disable-upnp --rpclisten=127.0.0.1:26710 --listen=127.0.0.1:26711 --nologfiles --yes on a scratch
appdir for 60 s, then igneum-miner.exe key-hash probe, then reads Application event 1000), published 18:32:37Z from this worktree:
ran 18:33:08 to 18:34:15Z (67 s), exit 0: rocks-probe exit 0, RESULT run node60: still running after 60 s (alive for the whole wait); killing it, igneum-miner.exe key-hash probe exit 0, RESULT event 1000: none since 18:32:11Z. The same verdict as the housekeeping run (run-hk-accept-1) and my warm-up run on 2f88a82f (run-0310-accept-hk, 16:57Z, the exe 37d0b104...). So the PC-built Windows node ships; no Mac cross-build was needed. The payload inputs: section 5c.
5a. The DMG
The chain under the lock (the 0.3.9 script's shape): the app (cargo build --release in app/igneum-app, 18:24:47Z, target cloned from the
0.3.9 worktree), the prover host and export (18:24:54Z, no Succinct toolchain: the host embeds elf/), igneum-prove-host --mode id
on this tree: shard program id 0x2b1a81cb413236cf063077b46ed3111628f6c41036bcf6e23ee4cbbf5679ef7a (2,832,504 bytes, sha256 0x150f4c05a2951fc5),
aggregator 0x474678f35f7545db28055d5e5bbc308231d84a5a072202087a2a8d5b09123896: the 0.3.9 pin, unchanged (no prover rollout). The
nine real fixtures ran --mode native with the new host (338, 341, 344, 56 x2, 58927, 72803, 72854, 78: all ok); the first chain
stopped on block-72803-skipped-copies.json.node-plan.json, which is a node-plan SIDECAR of the 72803 fixture (txgen 49796b0), not a
fixture, so the glob now excludes node-plan sidecars. Then NODE=<fork>/target-integration/release/igneumd MINER=... packaging/mac/build-dmg.sh
(18:26:04 to 18:26:34Z): prover: ... from <worktree>/proving/igneum-prove/target/release, the same --mode id line, fingerprints 477bb0ef
(intake key) and ed9c4d2e (folder).
| Artefact | sha256 | Size |
|---|---|---|
packaging/mac/dist/Igneum-Miner-0.3.10.dmg (engine 0.3.10, node 21d4c73c Mac arm64, the 0.3.9 prover host and export rebuilt from this tree) |
e8f1f9f620cb598fd21ff2b6f6a2dea3d9e58efc0550c6454dc65f0a3d791b97 | 41,377,608 |
Read back from the DMG (mounted read-only, 18:28Z): Contents/Resources/igneum-app.json carries node_override_params =
{"difficulty_v2_activation_daa": 33000, "fees_v1_activation_daa": 210000, "finality_v3_activation_daa": 135200, "proving_v0_activation_daa": 84100}
(the packaged line of 9997ef1) beside update_manifest, log_intake_url, log_intake_key, live_page, download_page; bin/ holds
igneumd, igneum-miner, igneum-bench, igneum-prove-host, igneum-prove-export.
A note on my own logs: the chained scripts print <time> ... exit $? lines where the time is a command substitution evaluated first, so
those lines always say 0. Every outcome in this plan is read from a content line (Finished, a sha256, a RESULT, a final:), never
from them; tools/lock/with-lock.sh itself propagates the exit code (checked: run bash -c 'exit 3' returns 3).
6. The harness runs (C and D)
| Run | Node | Command | Result |
|---|---|---|---|
| D, early (before c4) | fork 21babaa7 Mac arm64 (target-0310) |
IGNEUMD=... IGNEUM_MINER=... IGNEUM_HARNESS_BASE_PORT=29750 IGNEUM_HARNESS_TMP=/tmp/igneum-txrelay-0310 tools/lock/with-lock.sh run node tools/txgen/relay-net.mjs --rate 2 --duration 120 --wallets 16 --fund 2 from the tx-gossip worktree (the harness was not yet on this branch) |
17:54:26 to 18:01:01Z: final: sent 240, included 240, pending 0, failures 0, 1.951/s, latency p50 2,017 ms, p90 3,561 ms, max 6,560 ms: the coordinator's expected 240 of 240 |
| D, final | fork 21d4c73c Mac arm64 (igneumd 4bb356f4...) | the same command from the tx-gossip worktree (this worktree has no node_modules: the harness imports viem, the first attempt from here died at import; tools/txgen/ needs an install note), ports 29750+, data /tmp/igneum-txrelay-0310b |
18:26 to 18:31:05Z: final: sent 240, included 240, pending 0, failures 0, 1.975/s, latency p50 1,537 ms, p90 3,027 ms, max 5,025 ms; report in the scratchpad r0310/relay-final2/ (by miner: B and C only, A none, as the harness requires) |
| C, first run | fork 21d4c73c Mac arm64 (igneumd 4bb356f4...) | node tools/finality-attacks/c4.mjs on with the script's DEFAULTS (split 150 s, weight window 120 DAA; ports 29900+, suffix 990, /tmp/igneum-fin-c4-0310), 18:24:50 to 18:34:43Z |
[FAIL]: B locked 7 and 8 during the split, n0 reconnected 3 s after the heal, the three sinks DISAGREE, 0 locks adopted from an off-chain certificate, 9 / 4 / 4 conflicting certificates. The same shape as the bench-log's "on, split 150 s" row on the fix: a 150-s split is past the 120-DAA frozen table, so by the heal A's chain has outrun the table and the certified chain cannot be adopted; the c4 agent's measured PASS rows use WINDOW=240 (a 140-s split stays inside the 126-DAA bound, as any partition under an hour does on the live devnet) |
| C, final | the same binary | WINDOW=240 SPLIT=140 node tools/finality-attacks/c4.mjs on (the bench-log's PASS row's knobs, /tmp/igneum-fin-c4-0310b), 18:36:3x to 18:46:20Z |
[PASS] weight-vs-work-on: B locked index 9 during the split (138 s after the cut), A's sink blue score 315 against B's 292 (A the heavier by work), n0 reconnected 3 s after the heal, the three sinks AGREE (0424542743) on B's chain, A's split tip abandoned, n0 adopted 1 lock from a certificate off its chain (the C4 fix) and re-determined 1 line, 0 conflicting certificates, 0 reorg lines, max locked index 15 on all three. The coordinator's rollout note holds on the shipped binary: the certified chain wins over the heavier one when every node is new |
5c. The Windows payload inputs (18:35:33 to 18:36Z)
IGNEUM_WIN_RELEASE=<fork>/target-integration/x86_64-pc-windows-gnu/release IGNEUM_NODE_SRC=<fork> packaging/windows/push-inputs.sh from this
worktree (the sed -n 1p signer read of 68f14b9): igneumd.exe 3adb01a7... (50,978,816) and igneum-miner.exe 1105151c... (10,725,888), the PC 1
build of 21d4c73c, with the PC's three mingw DLLs (libstdc++-6 26,347,027, libgcc_s_seh-1 774,200, libwinpthread-1 324,451; the exes import
none of them since housekeeping A, the DLL gate and the copy stay for an older fork). Signed inputs: zip
c175446e33ccb020e2f5fdaf99100d5bd46138c7693b3e16948a82f7c252eaa4 (33,996,479 bytes, 5 files), node_source_commit
21d4c73c6ce32fcbd68391968e85827339a511c0 (release-0.3.10), repo_commit 9a51e32, built 18:35:37Z, deployed, payload-inputs.json HTTP 200;
node-source.pin = 21d4c73c, committed as 065c67c.
7. The ship
git push -u origin release-0.3.10 at b958493 (18:36:44Z, the credential helper; gh auth switch --user igneum-labs before every gh call,
the login renamed igneum-labs at 18:00Z, the token still under that name), gh workflow run windows.yml --ref release-0.3.10 -> run
37357266092 (queued 18:36:49Z on b958493).
The pre-push hook (fb076de: no conflict markers, cd site && node build.mjs) left site/index.html and site/journey.json modified after the
push. Measured with three builds of one tree at 18:38Z: the label of one bench entry ("RTX 5090 first run" against "RTX 5090, memory-hard
dataset") ALTERNATES on every run, whatever the cwd: build.mjs reads site/journey.json back (line 248) and writes it, so each run
transforms the previous output. Recorded in section 11. The hook's output (master's form) is committed as ccd2b98 and the next push's
hook flipped it again; the tree is restored with git checkout -- site/ after every push, so the ship's preflight sees it clean. Those commits touch no
app or packaging file, so the installer built at b958493 is the release; the ship state file (~/.cache/igneum/ship/0.3.10.json) got sha
= b958493 and runId = 37357266092, what the tool's own steps would have recorded, and the run resumes with --from ci.
Run 37357266092: green at 18:42:18Z (parse checks 52 s; engine, window host, payload, installer, smoke run 4 min 24 s; the G13 inputs step
against the 21d4c73c inputs of 5c and the pin of 065c67c). The dry run of the ship command (18:38:54Z) read everything: tree ccd2b98 clean,
0.3.10 in all 6, fork 21d4c73c, both exes, both Mac binaries, gh igneum-labs, live inputs 21d4c73c built 18:35:37Z, live manifest 0.3.9.
HOLD (coordinator, 18:4xZ): both PCs' NVIDIA workers have looped since 18:31Z (PC 1) and 18:35Z (PC 2) on the epoch seed bytes do not give the pack's IGNEUM_SEEDW_INIT (the exported GPU pack is the previous epoch's and the restart loop never re-exports it; the network fell to
about 88 MH/s); the coordinator published re-export jobs and holds every rollout until the fleet mines again. The ship steps that deploy
nothing ran by hand and the manifest step waits: OTA_SKIP=1 CONSOLE_SKIP=1 packaging/windows/fetch-ci-artifacts.sh 37357266092 (18:43:26Z)
and the DMG copied into the downloads folder.
| File (in the downloads folder, not yet deployed) | sha256 | Size |
|---|---|---|
| Igneum-Miner-0.3.10.dmg | e8f1f9f620cb598fd21ff2b6f6a2dea3d9e58efc0550c6454dc65f0a3d791b97 | 41,377,608 |
| Igneum-Miner-Setup-0.3.10.exe | 7ecf109f3867b554ecb97be3542f23c8d76d9752d6bdc7ece7a6f95fc2f9bfd4 | 24,994,420 |
| igneum-windows-app.zip | ebf0deca0e99de4085af1af5c3349028d8a8386f5016ee357ecf806df9a329c9 | 34,778,147 |
The installer is 5.2 MB larger than 0.3.9's (19,836,912) and the zip 7.1 MB larger: the PC's libstdc++-6.dll is 26,347,027 bytes
(Ubuntu's GCC 13 build, unstripped) against the Mac toolchain's, and it rides in the payload although the exes import no mingw DLL since
housekeeping A (section 11).
The ship command, to run when the hold lifts (every step before manifest skips itself):
node tools/ship-app.mjs 0.3.10 --node vendor/igneum-node-0310 --branch release-0.3.10 --public \
--activation-height 135200 --deadline-note "finality v3" --notes "<the five changelog lines, section 1; node 21d4c73c>" --from ci
consensus.override is carried over from the folder's 0.3.9 manifest (the four-field object of section 4) and --activation-height 135200 --deadline-note "finality v3" keep the consensus object identical field by field to the live one.
7a. The second final tree (the founder's scope change, 19:0xZ): the app rebuilt
The node stays 21d4c73c (its binaries, the digest, the suites, the harness runs and the Windows node acceptance all stand). What rebuilds:
the app (G, H, I: the UI, the engine additions, the export lock), the two Windows WORKER exes (H changes packfile.h, worker.cpp and
host.c), the payload inputs (now carrying the workers and the NVRTC DLLs), the DMG, the installer (CI), and the ship files.
| What | Result |
|---|---|
The two Windows workers, proto-cuda/nvrtc/build-windows.sh under the lock, 19:24Z, from 854f9a8 |
igneum-worker-cuda.exe 85cc357bb62bda36634ff4c1d80aca575aad71204ff2ef4bf620bdf183ab9236 (1,510,912), igneum-worker-opencl.exe afa73a324b9bfc3d957d0d5d3b6453770046721ac40cf7455ff3d49eb8b5774f (445,952); both differ from 0.3.9's (f50e19f2..., 1abc673e...) and from the hot-fix pair on the PCs (8dcef61c..., af7f12f5...), as the coordinator's check requires; the resource block on both |
push-inputs.sh again, 19:24:41Z |
12 files: the node pair and three DLLs of 5c, the two workers, nvrtc64_120_0.dll (86,728,192) and nvrtc-builtins64_128.dll (6,356,480), the three licence texts; zip ec1af603cb047471f8e6d6d95adbad192e0b77095e031fd4e39a807e37e8bc5c (71,940,402 bytes), node 21d4c73c, repo 854f9a8, signed, deployed, HTTP 200; the pin unchanged |
cargo build --release in app/igneum-app, then cargo test --release -p igneum-app, under the lock on this Mac (the coordinator's ask) |
19:24Z: ok, 103 (lib) + 27 (ota-sign) + 8 (prove-verify), 0 failed; igneum-app 0.3.10 |
--mode id on the worktree's host (unchanged prover) |
shard 0x2b1a81cb..., aggregator 0x474678f3... |
The DMG, packaging/mac/build-dmg.sh under the lock, 19:24:57 to 19:25:18Z |
Igneum-Miner-0.3.10.dmg 151687c5672553c571af7224a8e4228348135b8a313fc89e6641db7ae21c85b3, 41,383,375 bytes (engine 0.3.10 with G, H, I; node 21d4c73c; the 0.3.9 prover), fingerprints 477bb0ef and ed9c4d2e; it replaces the e8f1f9f6... DMG of the first tree |
PC 1 app-only job (node tools/build-job.mjs run --target ae432dc7 --targets windows --no-node --no-tests, the branch's new --no-node) |
published 19:24:50Z, done 19:26:47Z in 14 s (the engine alone, 6 s on PC 1's warm cache): igneum-app.exe 4564f8502bf40fdfb61a979ff19543838ca66be95b6b9f0beeaee53330c4d284 (2,962,944), warnings only (the dead-code set of 5a plus count is never used); host.cpp (WM_GETMINMAXINFO, WM_DEVICECHANGE) compiles on the runner only: CI run 37363381420 (section 7b) |
Not in this tree (they arrived after it; next cut): job-console 3562f26 (its second offer, 19:2xZ: power control as a setting, default off), and
the FORK-side pack-loop 05ef0fa3 (vendor/igneum-node: the miner checks every pack and rebuilds a refused one, exit 44; a node change: the
node stays 21d4c73c). The app side of af983a7 (watchdog.rs PackRebuilds, the engine re-exporting the pack on a refusal, capped per
epoch) is keyed on the miner's exit 44, which only the fork-side miner emits (the pack-loop agent's correction, 19:3xZ): with the 21d4c73c miner a worker refusal line shows "program pack out of date, rebuilding" on the strip, the card and the log but does NOT re-export; with the attempt-aware workers of this release a refusal means a genuinely broken pack, so the gap is small, and the self-healing half lands with the next node cut (section 11).
7b. The second push and CI
git push origin release-0.3.10 at 5b0d54f (19:26:13Z; the pre-push hook flipped the two site files again, restored with git checkout -- site/),
gh workflow run windows.yml --ref release-0.3.10 -> run 37363381420 (queued 19:26:19Z on 5b0d54f). The ship state file now names 5b0d54f
and that run. At 19:46Z the run was still QUEUED with no runner assigned, as were the repo's two ci runs (19:26Z, 19:39Z): GitHub's status
API reported Actions "degraded_performance" with an unresolved "Incident with Actions" (investigating since 19:15:17Z). Nothing in this
repository or on this Mac can shorten that: the installer comes only from the hosted windows-latest runner. The rollout waits for the
verdict; every other step is staged (the DMG, the signed inputs, the runbook r0310/rollout.sh with the exact commands).
Run 37363381420 ended at 19:41:24Z as failure with its first job CANCELLED by GitHub after 15 minutes queued ("The job was not acquired by
Runner of type hosted even after multiple attempts", the job's annotation) and the build job skipped: nothing of the tree ran. The workflow
was dispatched again under a watcher that dispatches again on that same annotation and stops on a green or on a failure of the tree itself:
try 2 run 37365130137 (19:42:43Z), try 3 run 37366744501 (19:57:52Z), try 4 run 37368355454 (20:13:21Z), try 5 run 37369931084
(20:28:32Z), every one cancelled by GitHub the same way after about 15 minutes queued; a follow-on watcher took over at 20:54:38Z and
dispatched try 6, run 37374158235 (20:54:55Z), which a runner acquired at 21:24:04Z and which went green at 21:30:29Z (section 7d).
Six dispatches, 2 h 04 min from the first to the green; GitHub's incident ("major outage" at 20:4xZ) was the whole of it.
7c. The fallback, prepared at 20:3xZ (the coordinator: a switch at 21:00Z, not a scramble; nothing built yet)
What the runner does for the installer, and what PC 1 has for each step (probe job probe-installer-pc1-0310, read-only, ran 20:33:5xZ
in 3 s; its R helper collided with PowerShell's r alias so every line came back inside an error message, the facts intact):
| Runner step | Needs | PC 1 (ae432dc7) | Fallback |
|---|---|---|---|
engine (cargo build --release --locked, MSVC target) |
Rust on Windows | not probed (the PC's build jobs build the engine in WSL on the GNU target: igneum-app.exe 4564f850..., 5a) | the GNU-target engine from job build-20261005-192450 unless cargo exists on the Windows side (checked at the start of job B); recorded either way |
| packaged configuration | the intake key and the folder token as files | the installed 0.3.9 app's igneum-app.json at C:\Users\Admin\AppData\Local\Programs\Igneum Miner\ carries the same manifest URL and key (override=False: 0.3.9 shipped no packaged override) |
copy that file and add node_override_params = the four-field object (not a secret); no secret leaves the Mac |
payload inputs (payload-inputs.zip, signature, hashes, node commit) |
the dl folder URL | the URL's folder is in the packaged json | download, verify the sha256s against payload-inputs.json (the signature is verified by the runner's step with the key compiled into the app; on the PC the app's own igneum-ota-sign is not installed: recorded as a gap, the hashes stand) |
window host (app\windows\BUILD-APP.bat, MSVC v143 cl.exe and rc.exe) |
Visual Studio with MSVC | vcvarsall.bat under C:\Program Files\Microsoft Visual Studio\..., cl.exe 19.51.36260 |
runs as on the runner |
payload (make-payload.sh, bash) |
Git Bash | C:\Program Files\Git\bin\bash.exe (and WSL) |
runs as on the runner |
installer (build-installer.ps1: Inno Setup 6 ISCC.exe, rcedit) |
Inno Setup 6, rcedit-x64 | ISCC NOT FOUND; rcedit not on PATH; winget present | build-installer.ps1 installs Inno Setup 6 through winget (a tool install on PC 1: the coordinator's call) and downloads rcedit-x64 from GitHub (-NoRcedit skips it: the exes then ship without the coin icon and version block, cosmetic, recorded) |
| smoke run, launcher dry run | the exes | the same PowerShell lines in job B | |
| the files back to the Mac | relay/clients/send.ps1 (a file up to 50 MB straight to Blob): the installer is 25 MB, the payload zip 35 MB |
two send.ps1 <file> calls with the sha256s in the report; on the Mac node tools/relay.mjs read <id>, sha256 compared, packaging/windows/check-runtime-dlls.sh on the payload folder |
|
| code signing | none on the runner either | none |
The jobs, in order (neither published until the word; both staged in the scratchpad r0310/fb/: the tree zip fb-tree-0310.zip, 709,814 bytes,
98 files from 5b0d54f by git archive, sha256 012c8a52c27631e8539704d703bf13eb68de0afd9cef094e68d1e6fc398f2cfc; the script fb-installer-pc1.ps1,
which stops on any sha mismatch, pins the inputs' node commit against node-source.pin as the runner's G13 step does, and takes -NoRcedit as its one argument):
fetchjobfb-tree-0310to ae432dc7: a zip of the tree at5b0d54f(git archive:app/windows,brand/icons,packaging/windows,proto-cuda/windows-app,proto-cuda/{host.cu,build.bat,README.md},proto-opencl/{host.c,build.bat,README.md},wsl2, the two worker.rcfiles),--dir jobs --extract --extract-dir fb-0310 --fresh.runjobfb-installer-0310to ae432dc7 (PowerShell, 20 min):cargo --versionif any; the packaged json copied and extended; the inputs zip downloaded and hash-checked;BUILD-APP.bat;make-payload.shunder Git Bash withIGNEUM_APP_EXE= the WSL engine (\\wsl$\Ubuntu-24.04\root\igneum-build\...\igneum-app.exe, its sha256 checked against 4564f850...);build-installer.ps1 -Payload <folder> -Version 0.3.10(with or without-NoRceditper the word);igneum-app.exe --version,Igneum Miner.exe --version;send.ps1the installer and the zip; every version and sha256 in the report.- On the Mac: the two files into the downloads folder, hashes equal to the report, the DLL gate, then
node tools/ship-app.mjs 0.3.10 ... --from dmg(preflight, then dmg already, copy, manifest, deploy, verify, console; the fetch step is skipped by--from, the console item carries no run id), then the rollout as planned.
What the gate records in this plan if the fallback ships: the installer marked "PC-built on ae432dc7, GitHub run owed"; non-reproducible (the runner's engine is the MSVC target, this one the GNU target from WSL; Inno Setup's version from winget; cl.exe 19.51.36260; Git Bash's version; Windows 11 build 26200); the sha256 and size of the engine, the host, the payload zip and the installer; the GitHub run's id and its own installer sha256 when it lands (they differ by construction); and the next cut goes back through the runner.
The coordinator asked at 21:3xZ whether the Mac mingw path was now faster and safer; the answer stands as below (there is no such path to an installer), so PC 1's queue position was kept for attempt 4.
What does not exist: a Mac mingw build of host.cpp (the coordinator's "how 0.3.7 shipped"): 0.3.7's node exes were cross-built on the Mac
and its installer still came from the runner (release-0.3.6 plan, section 9); the host has only ever been built by BUILD-APP.bat with MSVC,
on the runner or on a PC. So if PC 1 lacked MSVC the next fallback would be new work, not a known path; PC 1 has MSVC, so it is not needed.
7d. The fallback, run (the coordinator's word at 21:00Z: GitHub's status page "Actions: major outage", the sixth queued run)
The PC 1 window: asked of the Counter ASIC coordinator (it schedules PC 1 tonight) at 21:0xZ; granted at 21:09:24Z when its reproducible
benchmark released the machine (every card restored; the RX 9070 XT back on the bus). fb-tree-0310 (fetch) published 21:10:59Z, ran
21:11:32 to 21:11:33Z: the zip (709,814 bytes, sha256 ok) extracted into %LOCALAPPDATA%\igneum\app\jobs\fb-tree-0310\fb-0310 (the
script's expected path was wrong and it now finds the tree by search). fb-installer-pc1 (run, 25 min cap) published 21:19:28Z, FAILED at 21:20:05Z with exit 2 after 2 s: the script, not the build.
Its own lines: the tree found (82 files), cargo on Windows: none (so the engine is the WSL GNU-target build), Git Bash 5.3.15, Windows
10.0.26200, and then engine not found at \\wsl$\Ubuntu-24.04\root\igneum-build\...\igneum-app.exe with PowerShell's
ItemExistsUnauthorizedAccessError: the WSL tree belongs to root and the \\wsl$ share refuses it to the app's non-elevated session. The
build jobs read that tree with wsl -u root, so the script now copies the engine out with
wsl.exe -d Ubuntu-24.04 -u root -- bash -c "cp ... /mnt/c/.../fb-0310-work/igneum-app.exe". Republished as fb-installer-pc1-2 at 21:2xZ
(the Counter ASIC coordinator kept PC 1 for it: "a 2-second exit 2 is the script, not the build", 5-minute reply window met).
fb-installer-pc1-2 (21:23:08Z) failed at 21:23:58Z, exit 2 in 4 s: /root/igneum-build/app/igneum-app/target/... does not exist (the build
job keeps ONE target dir for the whole job; the acceptance script had read the node exes from /root/igneum-build/target/...): the script
now finds igneum-app.exe under /root/igneum-build with find and prints its sha256 from inside WSL. fb-installer-pc1-3 (21:26:52Z)
failed at 21:27:20Z, exit 2 in 4 s: the inline bash -c "..." string lost a quote on its way through PowerShell (unexpected EOF while looking for matching quote), the class the 0.3.6 cut closed for the app's own WSL calls (3811d8e, "every WSL script runs from a file") and
which this scratch script had reopened: the bash part is now written to engine.sh on the PC (LF, no BOM, the acceptance script's own
pattern) and run as bash <file> <arg>. Three 4-second failures of the script, none of the build; by the Counter ASIC coordinator's rule
its 10-minute measurement takes PC 1 first, then a read-only path probe (every path the script needs, found and printed), then attempt 4.
Before attempt 4 one more change, after reading the fixed block: find ... | tail -1 would take the NEWEST igneum-app.exe under
/root/igneum-build, and other agents' build jobs have run on PC 1 since mine (job-console's, pack-loop's), so the sha check could stop
attempt 4 on another tree's engine. The engine this plan already holds and verified (4564f850..., 2,962,944 bytes, from PC 1's own job
build-20261005-192450) now travels INSIDE the tree zip (fb-tree-0310b.zip, with it under app/igneum-app/target/x86_64-pc-windows-gnu/release/),
and the script reads nothing from WSL at all; the path probe checks that file too.
Attempt 4 was never published: at 21:30:29Z GitHub's runner acquired try 6 (run 37374158235, dispatched 21:10:04Z on 5b0d54f) and it went
GREEN (the parse job 21:24:04 to 21:24:57Z; engine, window host, payload, installer, smoke run 21:25:18 to 21:30:29Z; the G13 inputs step
against the 21d4c73c inputs of 7a). The recipe's own installer therefore ships; the fallback stops here with nothing built on PC 1, PC 1
released to the Counter ASIC coordinator at 21:31Z, and the prepared pieces (the tree zip with the engine, the installer script, the two
probes, the runbook steps) kept in the scratchpad r0310/fb/ as the rehearsed path for the next outage. Cost of the detour: three
4-second script failures on PC 1 and about 70 minutes of the shipper's attention; the gate record "PC-built, non-reproducible, GitHub run
owed" is not needed.
7e. The ship (21:31 to 21:40Z)
OTA_SKIP=1 CONSOLE_SKIP=1 packaging/windows/fetch-ci-artifacts.sh 37374158235 (21:31:21Z): the installer and the payload zip into the
downloads folder; the DMG copied beside them. The payload zip holds the rebuilt workers (igneum-worker-cuda.exe 85cc357b..., 1,510,912;
igneum-worker-opencl.exe afa73a32..., 445,952: both differ from 0.3.9's f50e19f2.../1abc673e... and from the PCs' hot-fix pair), the PC-built
node 3adb01a7..., and the runner's MSVC engine 496c4883... (3,361,792). The first ship run (21:32:02Z) stopped in preflight: the tree was 4
commits behind origin/master (the coordinator's explorer merge, 9746391: docs, site, observer, ci.yml; no app, node or packaging file), so
origin/master was merged as ff873f6 (37 files, no conflict), pushed 21:32:27Z (the hook's site flip restored), and the state file kept
sha = 5b0d54f, the CI commit, as the 0.3.6 and 0.3.9 cuts did.
node tools/ship-app.mjs 0.3.10 --node vendor/igneum-node-0310 --branch release-0.3.10 --public \
--activation-height 135200 --deadline-note "finality v3" --notes "<the seven changelog lines; node 21d4c73c>" --from ci
| Step | Result |
|---|---|
| preflight | ok: tree ff873f6 clean, 0.3.10 in all 6, fork 21d4c73c, gh igneum-labs, live inputs 21d4c73c built 19:24:41Z |
| ci | already: run 37374158235 green |
| fetch, dmg, copy | already (above) |
| manifest | 0.3.10 mac+windows, signed (key 8f186e37...), verified locally; consensus carried over from the folder's 0.3.9 manifest: activation_height 135200, deadline_note "finality v3", override the four-field object; and in dl/public/ (with the wallet 0.1.4 manifest) |
| deploy | one deploy, 21:33Z |
| verify | the token folder: igneum-app-latest.json 0.3.10, signature ok, mac 151687c5..., windows 24e58849...; the public folder: every file HEAD 200 with the local size (the DMG, the installer, the 0.3.9 HiveOS package, the wallet DMG, the four /public/ aliases, the two manifests and their signatures, igneum-downloads.json), but the byte comparison of a json file against the edge still failed after 12 tries at 21:37Z and again on a resume from verify: the edge cache, the class of the 0.3.6 and 0.3.9 verifies (section 11 if it does not clear) |
| console | (pending: --from console after the verify clears) |
| File | sha256 | Size |
|---|---|---|
| Igneum-Miner-0.3.10.dmg | 151687c5672553c571af7224a8e4228348135b8a313fc89e6641db7ae21c85b3 | 41,383,375 |
| Igneum-Miner-Setup-0.3.10.exe | 24e58849f2b517e8c5579455e8c9d8376ff7dd27052f458ef91913cbc48abc88 | 49,858,273 |
| igneum-windows-app.zip | 16b68b7bf625a946ac62a22413982b38c6649a124d799b08424a070f1b34810e | 71,809,486 |
The installer is 30 MB larger than 0.3.9's (19,836,912): the two NVRTC DLLs (93 MB unpacked) ride with the rebuilt CUDA worker now.
update-now-0310 to all, published 21:39:59Z (apps woken, stamp a23f594a). Timing with the Counter ASIC coordinator (it schedules PC 1 tonight): PC 1
was released by its hot-table job at 21:35:39Z, so one update-now went to every machine; its era measurement starts on PC 1's 0.3.10 STATUS line.
8. The machines after the publish (manifest live 21:33Z, update-now 21:39:59Z)
Baseline 21:40:31Z: Mac d937c69d app 0.3.9 (its card reads node 1, a24ab01a), PC 1 ae432dc7 0.3.9 node a24ab01a DAA 133,903 141.5 MH/s,
PC 2 1ccfe586 0.3.9 node a24ab01a DAA 133,945 0.0 MH/s (its 5090 worker off since a job's /api/resume at 21:25:11Z that the 0.3.9 app
answered and never acted on, section 11), Sam's Mac 3a9bf309 quit 53 min earlier, PC 37ba0461 0.3.9 node 2.1.0 2.0 MH/s. A machine
counts as updated when its engine logs the 0.3.10 header, its node reports a DAA score and its miner a hash rate on 0.3.10. The two checks
asked by the coordinator: (1) the workers start without the seed-words error on the first try, on the rebuilt pair; (2) the Mac's card
shows node 1's digest, by design. C5: the provers' "stopped after" lines (shards aborted by the restart).
| Machine | On 0.3.10 | Its log |
|---|---|---|
| PC 1 ae432dc7 (Windows) | engine restart 21:40:41Z (run win-ae432dc7-20261005-214041), 42 s after the job; [ok] updated to Igneum Miner 0.3.10 from 0.3.9 21:40:42Z; igneumd started 21:40:46Z (the PC-built node from the new install path); node proof verifier: command and reported: command +6 s; cards: NVIDIA GeForce RTX 5090 [discrete, off] | AMD Radeon(TM) Graphics [integrated, off] | AMD Radeon RX 9070 XT [discrete, off] (hotplug's line: the iGPU integrated and off, the two discrete cards then started); miners started 21:40:47Z (nvidia-ae432dc7-1, the bundled igneum-worker-cuda.exe) and 21:40:48Z (amd-ae432dc7-3, the bundled OpenCL worker); update to 0.3.10 complete 21:42:12Z. Check 1 PASS: the NVIDIA miner's epoch seed 66b26013... (daa 133967): CPU program and cache ready in 176 ms 21:40:47Z, then worker: info first pack packs\devnet: nvrtc 170 cache 4 dataset 23 check 249 race 37802 ms variant base; self-test PASS and worker: ready cuda NVIDIA_GeForce_RTX_5090 ... prepare 1 path nvrtc 12.8 at 21:41:25Z, no epoch seed bytes do not give line; STATUS 45.1 MH/s wall at 60 s (124.1 inside jobs), 124.3 MH/s inside jobs from 90 s on; the console 141.4 MH/s at 21:45:17Z (both cards). The node log: igneumd/2.1.0 (no commit: the PC build, section 11), Calibrated v1 fees ... 210000, digest 1f4b44255fcd2ea8f75664ed47200f409186ddd2292960c9e2cf95bbbdc11505, peers node 1 (192.168.68.64) outbound and inbound and the seed 188.245.5.161, flows registered at THEIR protocol version 13 (a v14 node beside v13 peers, the mixed-fleet case of section 10, measured) |
|
| Mac d937c69d | engine restart 21:40:51Z (run mac-d937c69d-20261005-214051), 52 s after the job; [ok] updated to Igneum Miner 0.3.10 from 0.3.9 21:40:52Z; a node already answers on 127.0.0.1:26610; using it (it is not stopped by this app): the app attaches to node 1 as it has since 17:45Z, so its card shows node 1's version and digest (check 2, by design); cards: Apple M5 Max [apple, off] (its miner was off before the update too); update to 0.3.10 complete 21:42:22Z |
|
| PC 2 1ccfe586 (Windows) | its 0.3.9 app fetched the woken jobs file at 21:40:33Z (1 new for this machine, 1 queued) and queued the update behind the aggregation-cost agent's running job agg-cost-pc2-2 (one job at a time), so the update ran at 21:49:24Z when that job ended: installer downloaded and verified 21:49:27Z, quit: stopping the miners, then the node 21:49:29Z (no prover "stopped after" line: no shard in flight), engine restart 21:49:41Z (run win-1ccfe586-20261005-214940), 9 min 41 s after the job; igneumd started 21:49:42Z; cards: NVIDIA GeForce RTX 5090 [discrete, off] | AMD Radeon(TM) Graphics [integrated, off]; miners started 21:49:44Z. Check 1 PASS: epoch seed 66b26013... (daa 134395): CPU program and cache ready in 169 ms, worker: ready cuda NVIDIA_GeForce_RTX_5090 ... first pack ... self-test PASS at 21:50:21Z, no seed-words line; STATUS 120.7 MH/s inside jobs at 60 s, 120.5 at 90 s; the console 123.2 MH/s at 21:53:04Z. This also ended the /api/resume no-op (its 5090 had been off since 21:25:11Z). The node log: igneumd/2.1.0, digest 1f4b4425..., flows at version 13 with node 1 and the seed (still a24ab01a for 10 s more) and at 14 with PC 1. The prover: prover: host /opt/igneum/igneum-prove-host (WSL2), CUDA, the pinned ids, then three assigned shards (22126, 51922, 84099) each failed with Failed to create the CUDA prover impl: CudaClientError: Connect(Os { code: 13, kind: PermissionDenied }) at 21:49:56, 21:50:12 and 21:50:32Z: PC 2's prover is dark after the update (section 11) |
|
| PC 37ba0461 (Windows, the US laptop) | its 0.3.9 app (run win-37ba0461-20261005-202247) downloaded and verified the installer at 21:40:52Z, started it at 21:40:53Z (per-user install, no administrator prompt), stopped its miners and node at 21:41:16Z, and no 0.3.10 engine run had reported by 21:56Z (the console: STOPPED (update) for 14 min): the install is in progress or stuck on the owner's machine; nothing to drive from here (section 11) |
|
| Sam's Mac 3a9bf309 | quit since 20:47Z (0.3.9); takes 0.3.10 when it is started |
C5, the shards aborted by the restart: PC 2's engine logged no prover "stopped after" line at its quit (21:49:29Z) and PC 1 runs no prover; the Mac's prover was off. Observed count: 0. The coverage numbers measured across the restart carry no abort from it.
The ship's last step, --from console (21:55:08Z): item #364 "Igneum Miner 0.3.10 shipped (mac+windows)"; the tool's closing line
"manifest 0.3.10 published 2026-10-05T21:32:40Z".
8. The machines after the publish
(pending)
9. The hand nodes and the seed (21:49 to 21:50Z)
Moved while PC 2 and PC 37ba0461 were still on 0.3.9 (their updates gated by another agent's job and a slow download): the digest does not change with 0.3.10 and a v14 node beside v13 peers is the measured mixed-fleet case (section 10), so moving them shortened the mixed window.
| Node | Command | Result |
|---|---|---|
| The observer, then node 1 | IGNEUMD=<fork>/target-integration/release/igneumd IGNEUMD_COMMIT=21d4c73c infra/devnet/restart-hand-nodes.sh '<the four-field object>' (the branch's script: grep -c for the commit string, the object written to /tmp/igneum-devnet/override-v3.json, the previous file kept with a stamp) |
observer restarted 21:49:38Z (pid 67770), node 1 21:49:50Z (pid 67963, caffeinate 67965); both print Calibrated v1 fees ... from DAA score 210000 and digest 1f4b44255fcd2ea8f75664ed47200f409186ddd2292960c9e2cf95bbbdc11505; the Mac app's card follows node 1 from here (its node line reads 21d4c73c) |
| The seed 188.245.5.161 | IGNEUMD_LINUX=infra/cross/out-0310/igneumd IGNEUMD_LINUX_SHA256=40be0e14... infra/devnet/restart-seed.sh '<the same object>' (the zig cross-build of 21d4c73c, glibc 2.36 target; the script checks the sha on both sides, keeps the previous binary as igneumd.prev-035 and the previous override file with a stamp) |
restart 21:50:15Z, unit active, MainPID 125195, igneumd/2.1.0-21d4c73c, Calibrated v1 fees ... 210000, digest 1f4b4425... |
Peers after the restarts: the observer and node 1 connected to the seed within 20 s and register flows at protocol version 14 with each other and the seed (node 1's inbound from the observer 21:50:08Z, node 1 to the seed 21:50:20Z, the observer to the seed 21:50:38Z; the seed's three inbound peers from this Mac's address at 21:50:20, 21:50:27 and 21:50:38Z, all at 14). PC 1's node (started 21:40:46Z, before the hand nodes moved) registered flows at version 13 with node 1 and the seed (then a24ab01a) and keeps them; PC 2's node (21:49:42Z) at 13 with node 1 and the seed (10 s before their restarts) and at 14 with PC 1: the mixed fleet of section 10, measured on the live network, with no refusal and no drop.
9a. The digest sweep (21:50Z)
| Node | Binary | Digest | Read from |
|---|---|---|---|
| PC 1 app node ae432dc7 | the PC-built 3adb01a7... (igneumd/2.1.0, no commit string) |
1f4b44255fcd2ea8f75664ed47200f409186ddd2292960c9e2cf95bbbdc11505 | its node log through the log intake (run 214041) |
| PC 2 app node 1ccfe586 | the same | 1f4b4425... | its node log (run 214940) |
| The Mac app d937c69d | node 1's (attached; its card's commit string is the app's own reading from its start at 21:40:51Z, before node 1 restarted, and refreshes on the app's schedule; its status line reads node 1's height and peers) | node 1's | |
| Node 1 | 21d4c73c Mac arm64 4bb356f4... | 1f4b4425... | /tmp/igneum-devnet/node1.out |
| The observer | the same | 1f4b4425... | /tmp/igneum-devnet/observer-v4.out |
| The seed | 21d4c73c Linux 40be0e14... | 1f4b4425... | the journal through restart-seed.sh |
| PC 37ba0461 | (its update in progress) | (pending) | |
| Sam's Mac 3a9bf309 | quit since 20:47Z | (pending) |
One digest, equal to the N3 publish's (finality-v3-devnet-publish.md) and to this cut's three readings on the fork (section 3): 0.3.10
moved no parameter, as measured.
10. The mixed fleet during the window
Between the manifest publish and the last app's update, old (a24ab01a) and new (21d4c73c) nodes share the devnet. What the two agents' reports say about whether they can disagree:
| Change | Old and new nodes together | Source |
|---|---|---|
| D, transaction relay (PROTOCOL_VERSION 13 to 14) | They connect: a v14 node sends the three new messages only to peers at version 14 or later; a 13 peer never sees them (a node drops a connection on an unknown payload, which is why the version moved). Blocks, headers and the handshake are unchanged, the digest is unchanged. Only the mempools differ: a transaction sent to an old node is included only by that node's own templates, as before; a new node's pool converges with other new nodes'. No disagreement about the chain | the tx-gossip commit message e242acd0, the coordinator's line |
| C, the certificate-driven reorg | A consensus-behaviour change with no digest change: an old node keeps the shipped behaviour (a certificate over a block off its chain stays pending and it never reorgs to it), a new node locks that block and moves to the heaviest tip through it. The two CAN disagree on the selected tip after a partition heals while the fleet is mixed (exactly the C4 shape); the c4 agent's rollout note: it converges when every node is new. On the devnet tonight there is no partition in progress and one network, so the window carries no live split; the sweep in section 9 checks every node's DAA within a few blocks of the others | the c4 agent's rollout note (coordinator, 18:2xZ), the bench-log C4 rows |
| A, B, E, F | Windows link settings, a test switch, the app's card handling and job exit codes: nothing on the wire | |
| The override object | The same four fields on every node before and after; the manifest carries it verbatim (section 7), so no node's digest moves | sections 4 and 7 |
So the window is safe for ordering (no digest change, no protocol break) and the only behavioural difference needs a partition to show; the hand nodes and the seed move last (section 9), after every app node is on 21d4c73c, so the fleet is fully new within minutes of the publish.
11. Open after the cut
The next cut (0.3.11), decided by the coordinator on 5 October 2026 night:
| Branch | What | Why not 0.3.10 |
|---|---|---|
fork pack-loop 05ef0fa3 (vendor/igneum-node) |
write_pack_checked, export-pack exit 3, the force-prepare at the epoch boundary, the miner's exit 44 that unlocks the app's capped re-export |
a node change after the node was frozen at 21d4c73c with its suites, harness runs and Windows acceptance done; the app side (af983a7) with the attempt-aware workers is the fix that matters tonight and it is in |
job-console 13755b9, then 3562f26 and whatever follows |
one hidden-console builder for every elevated launch, the spawn check in CI, PC 1 console watchers; then power control as a setting, default off (the founder: "if we don't have to ask then don't ask") | arrived after the tree closed (both times); repoints elevated-exit's include_str! test at platform.rs at its own merge |
opencl-rdna4-telemetry 7adcd4c (igneum-wt-rdna4-telemetry, not pushed: master + a08c371 as 38c9eec + 7adcd4c) |
igneum-gpu-telemetry.exe (ADLX, SetupAPI bus, PDH fallback; amdgpu sysfs on Linux) built by build-windows.sh, shipped by make-payload.sh and push-inputs.sh; the engine runs it at -l 5 and fills power, temperature, fan, memory clock and utilisation on the AMD card; 82 app tests pass. The 9070 XT measured on PC 1: 198.9 W, 64 C, 17.73 MH/s, 0.089 MH/W against the 5090's 0.398 MH/W (job tele-measure-1) |
arrived after the tree closed; a new shipped exe and an engine source |
ember-tune 54ff1bc (+38ef711, igneum-wt-ember-tune; its agent's message at 21:2xZ) |
Ember Tune: ember.rs replaces the NVIDIA power-only run in sweep.rs and the AMD single-lever sweep of 720b369; sits on cherry-picks of 7adcd4c and 13755b9+3562f26, so those merge first; 93 app tests, relay/test/ember.test.mjs and ui/tune-line.test.mjs in ci.yml; design in docs/plans/ember-tune.md |
arrived after the tree closed |
the rest of opencl-rdna4 a08c371 |
the OpenCL worker's duplicate-platform fold, --readback select, --memprobe, detect.rs parse_opencl_list, the 9070 XT bench-log entry |
conflicts with gpu-hotplug in detect.rs and host.c; only its EXPORT_LOCK hunk shipped (854f9a8) |
| Item | State |
|---|---|
The per-job build-inputs zip (68f14b9, c4's tooling commit; the coordinator's check after two agents' PC jobs ran on another agent's sources through the shared build-inputs.zip tonight, jobs build-20261005-191656 and -192537). Checked on this tree's own jobs in the live jobs file rather than a dry run (a dry publish would leave a stray entry in the file the ship deploys): build-20261005-182405 (PC 1) pins params.zip_url = .../build-inputs-20261005182334-74461.zip, params.sha256 628e6dae..., size 8,266,818; build-20261005-183131 (PC 2) pins build-inputs-20261005183051-83796.zip, c83ebb2a..., 7,271,492; both shas equal the local zips of those names, and 15 per-job zips sit beside the folder default. So a job published through tools/build-job.mjs pins the zip it just pushed. The residual: packaging/ota/publish-jobs.sh add --kind build WITHOUT --zip still defaults to the shared $DEST/build-inputs.zip (line 307); nothing refuses that name. A hand-added build job can therefore still pin whatever the folder default holds |
build-job.mjs path closed; the hand path is the first item of the next cut: refuse --kind build without --zip, or default to the newest per-job zip |
infra/cross/build-linux.sh resolved a relative TARGET_DIR inside the node source after its cd, so the copy step shipped the previous build's bytes (twice tonight, caught by the commit string in strings). Fixed here (e0a5fd1). The class: any script that takes a directory argument and changes directory before using it; proto-cuda/windows-node/cross-build.sh takes the node worktree as $1 and CARGO_TARGET_DIR from the environment (the 0.3.9 cut passed a relative one and it worked only because that script does not cd); a CI check for the shape is owed |
fixed on the branch; the check owed |
hotplug's proto-opencl/host.c change (the worker's --list prints the PCI address, the key to one row per physical card): no prebuilt OpenCL worker is in the payload inputs (the live zip holds igneumd.exe, igneum-miner.exe and three DLLs), the payload carries host.c and build.bat and the app builds the worker on the PC from them (engine.rs build_worker_from_source), so the change ships inside the 0.3.10 installer; whether an app that already has a worker exe rebuilds it from the newer source was not read here. Check on the console after the update: the PCs' card rows carry the PCI address (and one row per AMD card) only if the worker was rebuilt; else a rebuild is the hotplug agent's follow-up |
open: told the hotplug agent |
kaspa-consensus test ban_is_decided_by_the_carrying_block_so_nodes_agree_on_every_voter_list (c4-fix) failed once under the six-package parallel run on PC 2 with UnexpectedDifficulty(487112096 vs 487129281) in mine_on_all (section 3) and passes alone, twice. The helper builds a block on one TestConsensus and inserts it into others; the expected difficulty of the receiving node differs when the run is slow, which points at a wall-clock dependence (difficulty v2's sanitised clock per header). For the c4 agent: pin the clock or the timestamps in that helper, as the finality tests do for the cache queue. Until then a parallel six-package run can fail this test under load; the suites are run as consensus alone plus the other five |
open |
site/build.mjs is not idempotent: one bench entry's label alternates between "RTX 5090 first run" and "RTX 5090, memory-hard dataset" on every run of the same tree (three runs at 18:38Z: first-run, memory-hard, first-run), because the build reads its own site/journey.json back (line 248) and the label table at lines 203 and 204 matches against what the previous run wrote. Every push flips the two files through the pre-push hook, which is why the 0.3.9 and 0.3.10 cuts both met a modified tree after the push |
open: build the journey from the sources only (never from the previous output), then have the hook refuse a push whose build differs from the committed site |
The Windows payload carries the PC's three mingw DLLs (34 MB of inputs, libstdc++-6.dll alone 26.3 MB unstripped) although the exes import none of them since housekeeping A; the installer grew 5.2 MB |
open: drop the DLLs from push-inputs.sh and jobbuild.rs's copy once no shipped fork needs them, or strip them |
The two worker exes' version blocks say 0.3.0 (proto-cuda/nvrtc/igneum-worker-cuda.rc, igneum-worker-opencl.rc are not among the six files ship-app.mjs bumps) |
open: add the two .rc files to VERSION_FILES |
The app's re-export of a refused pack (af983a7's watchdog.rs PackRebuilds, 3 per epoch) waits for the miner's exit 44, which the 21d4c73c miner never emits: a refusal in 0.3.10 shows the notice and restarts the worker, no re-export. The fork-side pack-loop 05ef0fa3 (the miner checks every pack, rebuilds a refused one, exit 44) is the other half |
next node cut |
C1 (the consequences reviewer, 20:0xZ): the 0.3.10 node's igneum_exportSegments (fork igneum/exec/src/rpc.rs) writes no daaScore and no feesV1ActivationDaa per segment, which the 0.3.9 exporter needs to replay both sides of the fee switch (export/src/main.rs: "a dump without daaScore is accepted only when the switch is never or 0"). Measured here: the handler (90 lines from rpc.rs 849 in vendor/igneum-node-0310) carries neither key (daaScore appears in the fork only in other RPCs, lines 239, 423, 819); the reviewer names vendor/igneum-node-pv1 at eb32c645 (on 21d4c73c) as the carrier: its rpc.rs writes daaScore per segment (line 961) and fees and feesV1ActivationDaa at the top level (line 982); confirmed here at 20:4xZ (git -C vendor/igneum-node-pv1 grep -n feesV1ActivationDaa -- igneum/exec/src/rpc.rs: line 982); my first read of that path at 20:1xZ was wrong. So at H = 210,000 (about 19:50Z on 6 October) every 0.3.10 prover's export of a post-H block fails or meters with the wrong table and the fleet's provers go dark, unless the next node cut carries that RPC onto every prover before H, or H is republished later. The mechanism (the proving agent, 20:1xZ): the app's prover calls the exporter with no fee flags, so from H every app prover on 0.3.10 cuts with the prototype table and every statement is vetoed. The two closes: (a) the proving-v1 fork (eb32c645, on 21d4c73c) on every prover before H through 0.3.11; (b) republish H = tip + 86,400 by the fee-switch plan's rule. Decision due 16:00Z on 6 October; the coordinator, the proving agent and the 0.3.11 shipper hold the same line |
open, dated: (a) or (b) by 16:00Z on 6 October |
C5 (the same reviewer): the app kills its prover child on quit (prover.rs 266 to 272), so the update-now of this rollout aborts whichever shard each prover has in flight (up to 37 s each, no payout). Accepted as the cost of the restart; the count is read after the rollout from the provers' "stopped after" lines (section 8) so the coverage numbers measured across the restart are read with it |
recorded at the rollout |
/api/resume answered ok on the 0.3.9 app and never restarted the miners (PC 2's 5090 worker "off" with the card holding 1.7 GB from a job's resume at 21:25:11Z until its 0.3.10 restart, the iGPU miner too; the Counter ASIC coordinator, 21:4xZ). The class: a resume that reports success without a miner restart. The app should re-check the miner processes after a resume and report a failure |
open: next cut |
PC 2's prover after the 0.3.10 restart: every assigned shard failed at once with CudaClientError: Connect(PermissionDenied) (section 8) from 21:49:56Z. RESOLVED at 22:01Z by a socket fix on PC 2 (the Counter ASIC coordinator's agents; the SP1 CUDA prover's client socket permission), after which PC 2 proved and was paid every 40 s. The proving agent is adding a log line that names the cause on the app branch for the next cut |
closed 22:01Z |
| PC 37ba0461 (the US laptop) started the 0.3.10 install at 21:40:53Z, stopped its miners and node at 21:41:16Z and had not come back by 21:56Z: the per-user installer on the owner's machine, nothing to drive remotely | open: the console shows when it returns; its 0.3.10 line and worker start are read then |
dl/public/igneum-downloads.json (the unsigned index the site's download page reads) alternates at the edge between the new bytes and the previous ones for over 20 minutes after the deploy (one fetch byte-identical at 21:52Z, the next three not): different edge nodes behind one hostname. The two signed manifests were byte-identical and verified from the first check. The ship's verify step counts it as a failure and refuses to post the console item, so the item was posted with --from console |
open: the verify should accept the index after the signed manifests pass, or retry it for longer; the site serves the previous version's buttons from a stale edge until it settles |
The console's machine card keeps a machine's LAST non-empty node commit string: PC 1's card read node 2.1.0-a24ab01a for 17 minutes after its node had restarted as the PC build (igneumd/2.1.0, no commit), while PC 2's card read 2.1.0 at once; the Mac's card kept node 1's a24ab01a after node 1 moved to 21d4c73c. A card's commit string is therefore not a fact about the running node until the app re-reads it; the node log is |
open: the card should show the string the app last READ, with its time, or nothing |
The site build, run on a tree with conflict markers in site/journey.json, fails silently and leaves the markers (it reads that file back, the non-idempotence above): the first merge commit of this cut to master carried markers in two site files for one minute and was amended (the pre-push hook would have refused it, but the check must not depend on the hook) |
fixed by hand here; the build should refuse a journey.json that does not parse and say so |
The PC-built node binaries embed no commit (the build inputs zip has no git dir, build-info falls back to the bare version): the console shows PC 1 and PC 2 as node 2.1.0 with no commit after 0.3.10, as the 0.3.6 PC build did. The build inputs manifest records the fork commit (2f88a82f and later) |
open: a commit stamp through the build job (jobbuild.rs) is an app change, not tonight |