67 KiB
Igneum Miner 0.3.17: the 0.3.17 tree as cut on 7 October 2026
The hourly cut after 0.3.16, under the coordinator's rule: every new switch at never on the devnet so the digest does not move; cut what is green and list what waited. Worktree igneum-wt-ship0317 (release-0.3.17 from release-0.3.15 a9eb58f1), node release-0.3.17-node in vendor/igneum-node-0317 (from f1ea7a38).
Renumbered 7 October 2026, 03:3xZ (main): the tree this plan describes is 0.3.18. Its canary on 12153428 failed on two further faults (the idle-peer guard closes every peer mid headers-proof IBD; a window-below-epoch log flood at 1,900 lines a minute), and tonight's 0.3.17 became the node-only hotfix on the 0.3.16 app tree (f1ea7a38 plus 90aaf38e = b3c228fa; see docs/plans/release-0.3.17.md on release-0.3.17). The version scheme is three-part everywhere, so "0.3.16.1" was not available. Rows below keep "0.3.17" where they were written; read it as this tree. Branches: release-0.3.18 (app), release-0.3.18-node (fork, 12153428). Decimals is 0.3.19.
Renumbered again 7 October 2026, 08:2xZ (main): this feature tree is 0.3.19; 0.3.18 is the app-only cut on the 0.3.17 tree (the engine's clock sample gated on igneum_getExecStatus, ledger N7; docs/plans/release-0.3.18.md on release-0.3.18-app); decimals is 0.3.20. Branches now: release-0.3.19 (app), release-0.3.19-node (fork, the mirror). Rows below keep the "0.3.18" they were written with; read it as this tree.
Renumbered a third time, 7 October 2026 10:4x UK (main): this feature tree is 0.3.20 (the dc141409 line plus the class v4 amendment, option A on AP-F8-1, and the proof archive; its canary gate as ordered). 0.3.19 is an app-only cut on the 0.3.18 app tree plus miner-ui-5 on the node pin 5899f603 (docs/plans/release-0.3.19.md on release-0.3.19). Branches: release-0.3.20 (app), release-0.3.20-node (fork, the mirror). Rows below keep the numbers they were written with.
1. The node tree
| Branch | Tip | In 0.3.17 | Why |
|---|---|---|---|
| ca3-v4-0316 (node lane) | 9a44fcb8 (df787727 the last functional commit: igneum_getNodeInfo; the miner stall guard e6e1fbe2 with exit 45, the idle-peer drop 96619b7d) | yes | the unwrap class, the seven-window rule, the finality cause fields, igneum_getRecentBlocks, difficulty rule v3, the fork gate, vote-or-burn, the signing bonus, the miner's stall exit (every switch absent from the live object = never) |
| ladder-node (ladder lane) | 1591ee1d | yes | the latency ladder behind latency_ladder_activation_daa (never); the receive gate reads header_signals_active (the lane's merge note, applied as c6e12005's parent commit) |
| exec-sync-0313 (proving lane) | 40fd8d8c | yes (merged last, 02d15a87; params.rs both sides kept) | consensus proof verification behind proving_consensus_verify_daa (never), the override's program ids as the statement's ids, sp1 as cfg(not(windows)) dependencies with the host-backed verify on Windows (the first merge of a344fea3 broke the Windows cross-build in sp1-jit; 40fd8d8c fixed it) |
| finality-pause-node-0317 (finality lane) | fdc71bb5, rebased | yes (cherry-picked as d8d1de1a onto the exec-free base) | W7 the departure announcement behind finality_leave_activation_daa (never on the devnet; 0 and leave_delay 3600 in the testnet genesis per the project lead); protocol 17; the devnet digest unchanged by test |
| tail-emission-node-0317 (economy lane) | 5647533f, rebased | yes (cherry-picked as 996b592f, its params hunks without exec-sync's proving fields) | its coinbase table replaces the subsidy table the silence rules build on; the rebase resolves it; no digest move while CURRENT |
| kaspad igneum-pow default feature | 75467244 | yes | the node lane's N5 finding: a bare cargo build -p kaspad validated PoW with the kHeavyHash stub; every box reads igneum_getNodeInfo's powEngine "igneum-pow" in the table, "stub" is a FAIL |
| kaspa-testing-integration | cc78a21f | fixed here | compiles again (the block store's evm argument, Header's vote_key_hash, the Params const as a let, the four finality ops in the RPC sanity match); the box suite is to compile every test crate from here |
Final node tree 02d15a87 (75467244 + exec-sync-0313 40fd8d8c). Digests on the exec-free tree c6e12005 (Mac binary, direct run): the live sixteen-field object eada4bda (unchanged), the thirteen-field file b18ed271, no file c562d70e. A harness note: scratchpad digest.sh printed the no-file digest for a file that reads eada4bda while other nodes held its ports; it now says UNTRUSTED when the override lines are missing.
2. The app tree
| Branch | Tip | In 0.3.17 |
|---|---|---|
| ladder (repo side) | 7003f9f | yes (igneum-pow's chain_program_shadow, which the fork's kaspa-pow needs) |
| exec-app-0314 | 032f03c | yes (clean; docs as the union) |
| relay | 28c028b | yes (ci.yml on the release side) |
| ember-tune-0317 (rebased on a9eb58f1) | bb37c993 | yes: the release lines kept (job_active, install_asked, the stale-exporter line), live.rs one merged module, publish-manifest.sh Ember's guard |
| miner-ui-4 (rebased on a9eb58f1) | afc331bd | yes: ui/ whole with Ember's finality words, extnode.rs on igneum_getNodeInfo, N4 stall exits, the port-collision check, POST /api/shot, an external node that goes away hands the ports to the app after 60 s; src/live.rs stays Ember's |
| proving-v2 | d8055a7 | NO (its exec content is in c00e608; its extra is the proof-system v2 slot, its own cut, per its owner) |
App tests on the merged tree: 175 + 28 + 8; UI 41. A whole-file take of a lane's branch on an older base dropped the release tree's own fixes (seen and reverted): the lanes rebase onto the release tree and resolve there; the shipper merges, never substitutes files.
2a. Linux binaries by glibc (main's rule, 7 October 2026, 01:3x UK)
HiveOS images are Ubuntu 20.04 (glibc 2.31); the canary pods on 22.04 (2.35) refused the box's binaries. From now: the HiveOS package carries the 2.31 zig build (the build-server lane produces and container-tests it; the 0.3.17 HiveOS row stays at 0.3.16 in the index until that package lands, then joins with its own read-back); seeds and generic Linux binaries are the 2.35 build; the fleet's own boxes (Ubuntu 24.04) take the box's native build. The Mac and Windows rows publish on their gates.
2b. Checklist line for every cut (the node lane, 7 October 2026)
cargo check -p kaspa-testing-integration --tests runs in the box suite of every cut and in the fork's CI (the crate had not compiled since the finality fields landed; no integration test ran on any cut until this one). The 0.3.17 box chain carries it as its last step (rebuild-on-fix.sh step_box / the node chain's "integration check"). 0.3.18 node notes: relay pipelining bb397f99 and the sync-request fuzz gate 6cf33d36 on ca3-v4-0318; aa0182aa and 812c3ac2 from ca3-v4-0316.
2c. A kill matches a pid, never a name (main's row, 7 October 2026, 01:xx UK)
My pkill -f build-remote.sh at 00:17 UK (to stop my own chained box builds before a rerun) killed the decimals lane's box run on the same Mac: the pattern matched every build-remote.sh, not mine. Rule: every kill in the ship tooling matches a pid file or the exact command line (the run's log path, the worktree path), never a tool's name; the ship scripts run through tools/ci/pre-push.sh's kill-by-name check before the next cut. Tonight's scratch runbooks hold no pkill by tool name any more (the chains are started with their log path in the command and stopped by that path).
2d. The canary's FAIL (01:37Z): the IBD guard refuses signal-version relay blocks
c17-1 (02d15a87, the live sixteen-field object) could not join: "IBD with headers proof from was unsuccessful (peer relayed block ... header version mismatch: got 1026, expected 2 at DAA score 237583)", 483 lines in 15 minutes against all four peers, peers 0, blocks 0. Cause: protocol/flows/src/ibd/flow.rs:390 (upstream's Toccata guard, f94053a0) compares the syncer's relay-block version with the plain block version before the pruning proof; under the open window the sink carries legal 1026 signal blocks. The line is on f1ea7a38 too, so the LIVE devnet has refused every fresh join (the headers-proof IBD path) since publish 2 opened the window at 22:43Z; tonight's moves passed because every node had a datadir (the relay path) or began its IBD before the hub moved. Fix (node lane): the comparison gated like the pre-ghostdag rule (block_version_of under header_signals_active), known-failed test first; main decides between 0.3.17 carrying it and a 0.3.16.1 node hotfix. Checklist line from here: the canary's target is a FRESH node joining the LIVE object's chain, which carries signal headers once a window is open; the pre-cut harness (node-compat.mjs) gets a chain with signal-version headers under an open window.
2e. The rebuild on the fix (7 October 2026, 02:3x to 02:4x Z)
The node lane's IBD-guard fix (ca3-v4-0317-fix 90aaf38e) merged into release-0.3.17-node as 12153428. Every binary rebuilt on it:
| piece | commit string | sha256 (first 8) | bytes |
|---|---|---|---|
| Linux igneumd, box native (glibc 2.39, the fleet's canary sha) | 12153428 | 5a4a0d68 | 57,347,232 |
| Windows igneumd.exe (cross, box) | 12153428 | 1e0b49ef | 52,367,360 |
| Windows igneum-miner.exe | 12153428 | 7ca9ca01 | |
| Mac igneumd (Apple Silicon) | 12153428 | 7416d4a5 | 47,837,248 |
| Igneum-Miner-0.3.17.dmg (packaged object = the live sixteen-field object, prover pair aboard) | 916248b8 | 44,244,489 | |
| HiveOS igneumd (zig, glibc 2.31) | 12153428 | 378340e4 | 56,022,096 |
| igneum-hive-0.3.17.tar.gz (2.31 node pair + the box's CUDA/OpenCL workers) | 04fb7d45 | 27,193,392 |
Suite on 12153428: green (two load flakes pass alone), cargo check -p kaspa-testing-integration --tests rc 0. Digests read directly from the rebuilt node: sixteen-field eada4bda (igneum_getNodeInfo powEngine igneum-pow), thirteen-field b18ed271, no file c562d70e: unchanged from 0.3.16.
Windows inputs pushed 02:37Z (igneumd.exe 1e0b49ef, workers d7a413c7, 6f4bb57f, 0d68d06e, the Linux prover pair), pin b5a16f5b on release-0.3.17, windows.yml run 37562948420 dispatched 02:38Z.
HiveOS 2.31 smoke in an ubuntu:20.04 container on igneum-build-1 (ldd 2.31): igneumd, igneum-miner, igneum-worker-cuda and igneum-worker-opencl all load and answer. Found on the way: GNU tar materialises the Mac's extended headers as ._ files beside every file in the package (the live 0.3.16 tar has twelve of them; harmless, HiveOS ran it). Fixed in make-hive-package.sh (COPYFILE_DISABLE, no Mac metadata, c5187a70); the 0.3.17 tar extracts clean (10 files, 0 ._).
The scratch staging copy (r0317/dlsite-stage) had the superseded DMG d25ab372 and installer d1065ac0 removed; the manifest is re-written there once the Windows run's installer is fetched. The live downloads folder holds no 0.3.17 file until the deploy step, which waits on the fleet's second canary line.
2f. Ready at the line (7 October 2026, 02:5x Z)
- Windows: run 37562948420 green (engine, window host, payload, installer, smoke run), Igneum-Miner-Setup-0.3.17.exe 93e29580, 62,814,273 bytes, fetched into the scratch copy. The scratch manifests (token folder and public) read: version 0.3.17, mac 916248b8, windows 93e29580, consensus.override = the live sixteen-field object (floor 831,600, window 86,400), HiveOS alias held at 0.3.16 until its own row.
- The deploy runbook is
scratchpad/r0317/deploy.sh(step_preflight, step_manifest = the live folder written and deployed in one step with the same inputs, step_update_now for d937c69d, ae432dc7 and 1ccfe586, step_hive, step_readback). It runs only on the fleet's second canary line. - Hands and seed: the build-server lane builds the seed's 2.35 pair and the hands' native pair on 12153428 and restarts them LAST on my line (observer, node 1, then the seed), each read back by commit string, digest and powEngine.
- The Discord card is dry-run at
tools/community/out/release_0.3.17.json(four reader-facing change lines, no em dash); it posts only once the HiveOS row is live, since the card carries every platform. - The 0.3.16.1 fallback (main's rule: if the retry fails off the IBD-guard fix) is prepared and not pushed: scratch worktree
r0317/fallback-03161-node, branch release-0.3.16.1-node at b3c228fa = f1ea7a38 plus a hand-port of 90aaf38e (the helper gates on class_signal_active() alone, since f1ea7a38 has no ladder; the ladder assert dropped from the test).cargo checkon consensus-core, consensus and p2p-flows clean, the unit test green. The node lane confirms it is the same adaptation it made for its 0318 tree (f2b25fdf). - The node lane's two-daemon window test (10226194 on the fix branch, separable, touches only testing/integration) is NOT in 0.3.17's pin; run on the box against 12153428 plus the commit (worktree vendor/igneum-node-twodaemon, so the path igneum-pow resolves):
a_fresh_node_joins_a_chain_of_signalling_headers_and_no_window_refuses_them ... ok, 11.46 s, 03:01Z. It rides the next tree. - Pairing rule (the build-server lane, 02:5xZ): a fork build takes igneum-pow by path from the igneum worktree it sits in; a fork worktree under a master checkout fails in kaspa-pow (
chain_program_shadowmissing). 0.3.17's artefacts paired with release-0.3.17 at a73e400c to 6f8d7a7e, whose igneum-pow is unchanged since 6b30e855. The tool logs the pairing from the next master.
3. Owed
- exec-sync-0313's consensus verification behind a feature off for Windows (the proving lane), then its two commits.
- finality-pause-node rebased onto the 0.3.17 node (the finality lane).
- decimals (B10 gate), vote-weigh and tail-emission docs: the next cut.
- The fork gate's fast-time line is GREEN on aa0182aa (gate on: a 27.5 percent key's private chain 183 DAA deep refused, 268 refusal lines, the honest node kept its chain; gate off, the known-failed case reorgs), one commit past the 9a44fcb8 this tree carries; the switch is at never on every network, so 9a44fcb8 stands for 0.3.17 (the chains were building) and aa0182aa rides 0.3.18. vote-or-burn and the signing bonus: the replay gate FAILED on 9a44fcb8 (this tree) and PASSES on 812c3ac2 (the silence read as a function of the block's own past), which rides 0.3.18 with aa0182aa; both switches are at never here, so a devnet node behaves the same; the signing bonus is settable at the testnet genesis on the project lead's word, the devnet not a candidate until the longer-span replay passes.
- The canary in the full form before publish (a new node mining beside an old one with a poisoned peer, ten minutes, a mid-window re-sync), the staged-in-scratch rule, every new switch at never.
4. For the 0.3.18 tree (not in 0.3.17)
- miner-ui-4 past afc331bd: 74c12665 (the merge check: a node whose blocks never merge reads "behind" and the miner is held; src/merge.rs, observer poll, one UI line; 172 box + 42 UI tests green). The UI lane's word, 03:2xZ.
- ca3-v4-0317-fix 10226194: the two-daemon window test (green on 12153428 plus the commit, 03:01Z).
- The node lane's ca3-v4-0318 tree (98c78d9a, f2b25fdf).
- The build-server lane's pairing log line in build-remote.sh (which igneum-pow a fork build paired with).
5. The 0.3.18 cut after the hotfix (main's order, 7 October 2026, 03:5xZ)
Once "0.3.17 live" (the hotfix) is read back: rebuild release-0.3.18-node on 12153428 plus the node lane's 4f4bb9c9 (ca3-v4-0317-fix: the idle-peer drop counts headers, proof, trusted data and UTXO chunks as deliveries and never fires during a sync; the class-signal missing-history warning once per epoch per process; kaspa-p2p-lib 20, kaspa-p2p-flows 35, class_signal 1 green on the box; the same patch onto ca3-v4-0318 by 05:15 UK), every platform, digests unchanged, suites, the integration check, the window test again on the merged tip; merge the UI lane's list (afc331bd in, then 74c12665, 89501ce8, b7d33e8e, the doc-only d29a8663), ember-tune-0317 c9b31edc; re-stage 0.3.18 in scratch; the fleet's full canary on its pods with the headers-proof fresh join decisive; publish 0.3.18 on its line, whenever that falls, morning included. Ember's PC 1 window ("PC 1 on 0.3.18") and the UI lane's Mac check follow its update-nows.
Tips (node lane, 03:5xZ): release tree 4f4bb9c9 on ca3-v4-0317-fix (cherry-pick alone onto 12153428); feature tree a3fe9f67 on ca3-v4-0318 (on e1197983, a lockfile-only commit). A follow-up on both is due (the ping flow's syncing flag IBD-only; "sink at genesis" broke the peer-drop gate's --case still and is redundant for the canary): take it WITH 4f4bb9c9. The hotfix canary (0.3.17, node d712b498) started 03:58Z on c17-1; its clock in UTC: headers through about 04:17, synced 04:40, reads 04:50, cases 05:25.
Final node inputs (main, 04:0xZ): the 0.3.18 node tree is ca3-v4-0318 at 6e4ace3f (a3fe9f67 the fix, b21659e4 the IBD-only follow-up, 6e4ace3f igneum_getNodeInfo's blockrate object for the merge check; exec-only, digest untouched; all suites green). The release-branch form (4f4bb9c9 + ddda7d52 on 12153428) is the equivalent and not used. After "0.3.17 live": release-0.3.18-node = 6e4ace3f, every platform rebuilt, then the canary.
Dated fault for this tree (the node lane, 05:05Z, found by the Mac's headers-proof join gate, not by a canary): when the devnet's pruning point first leaves genesis (chain DAA 185,799, about eight hours from 05:00Z at 1 bps), every fresh join by headers proof fails for the next 2,644 DAA (about 45 minutes) with "IBD with headers proof ... unsuccessful (DAA window data has only N entries)": upstream's sampled-window walk (661 samples at rate 4) runs out at a gap in the trusted blocks instead of reaching genesis. Nodes already on the chain are untouched. Fix: joiner-side on the trusted path only, digest-neutral (the walk accepts a window that runs out at origin when the block is younger than the span), coming on ca3-v4-0318 (hash to follow); 0.3.18 carries it. Any fresh-join canary timed inside that 45-minute window FAILS from this, not from the idle-peer fix: time 0.3.18's canary outside it (before about 12:50Z or after about 13:40Z, to be firmed from the live DAA).
Main's rule for the 0.3.18 cut (05:1xZ): it carries the node lane's joiner-side fix for the pruning-point fault (on the 0.3.18 tree the N6 guard would otherwise ban the syncer for ten minutes per retry), and 0.3.18 is live on every node before 13:30 UK (12:30Z), ahead of the devnet's first pruning (DAA 185,799, about 14:00 UK). If the full canary cannot close by then, publish 1 goes on the clean form plus the node lane's harness for that case, and the cases follow as confirmation, as with the hotfix. Every new chain, the testnet genesis included, meets the same fault once at its first pruning: the testnet go checklist (docs/plans/testnet-go.md) gets the line.
Correction (the node lane, 05:1xZ): the devnet does NOT meet the young-window fault, and the 185,799 clock was wrong (the canary's 155,700 headers were taken for the chain's DAA; live DAA was 250,809 at 05:00Z, the hub's 39th pruning move was at 03:46Z and the 03:50Z fresh join synced clean). Pruning samples sit at multiples of the finality depth (pruning.rs is_pruning_sample); the devnet's finality depth is 43,200 blocks, so its first non-genesis pruning point was at blue score 43,200, sixteen window spans past genesis, and the walk from any devnet pruning point fills its window. The fault needs a pruning point within 2,644 DAA of genesis, which only a chain with finality depth under 2,644 can have: the fast-time profile (120) where the gate found it. The testnet at the compiled numbers never meets it either. So: no fresh-join window to avoid, the 45-minute rule is dropped, main's "before 13:30 UK" rule for 0.3.18 no longer rests on this fault (main to confirm), and the testnet go line is softened to a note. The fix still rides 0.3.18 (joiner-side, digest-neutral, relaxes the rule only when the walk ended within one sample of genesis, impossible on the devnet; unit test green on the box; hash to follow).
Main, 05:2xZ: the 13:30 UK deadline is withdrawn. 0.3.18 ships on its normal gates with the full canary, no shortcut; the young-window fix stays in the tree as a harness-profile correctness fix. ca3-v4-0318 tip 06:04 UK: e3798a17 (69647bd6 per-path finality timers plus the join bench, test-only plus accounting; e3798a17 the young-window fix, unit test green). The fresh-join cost fix is next on the branch with its own hash and before/after numbers; its "before" figure comes from the 0.3.18 canary's IBD-end line (the new "finality time: bodies ..., virtual ..., weight tables ..., signatures ..., persists ..." line), which I relay verbatim (route (b); a read-only joiner from the box, route (a), is main's word).
ca3-v4-0318 tip 06:06 UK: aa49613f (on e3798a17; carries the IBD-end "finality time" line; p2p-flows and kaspad check clean on the box). The cut takes aa49613f or the later tip the node lane names; the canary's fresh-join IBD-end line goes to the node lane verbatim.
6. The 0.3.18 inputs as of 05:2xZ (main's list)
- Node: the ca3-v4-0318 tip the node lane names at the cut; now 8220c944 (06:10 UK; on aa49613f): a block's votes BLS-checked across cores before the finality lock, exact by the two-path test. Box, 500-checkpoint chain of 22,221 blocks: join 218 s to 104 s, finality 130 s to 30 s, signatures 97 s to 9 s. Suites at 8220c944 on igneum-build-1: kaspa-consensus lib 110 passed, 0 failed, 3 ignored (the two new finality tests inside); kaspa-p2p-flows 37 passed; kaspad check clean at aa49613f with nothing in kaspad changed since. The real split of the canary's 8 s per checkpoint comes from the IBD-end timers line of the 0.3.18 canary's fresh join, relayed verbatim to the node lane.
- App: miner-ui-4 afc331bd (in), 74c12665, 89501ce8, b7d33e8e, bac43ac4 (main's row from PC 1's 04:51Z relaunch: the card watchdog judges a miner only while the node is synced; a silence fault from the sync is released when the node syncs and the card starts again with an Activity line; a worker silent from its start is faulted at 60 s; one Activity line per card at its first start; known-failed test from the PC 1 row in release-0.3.17.md; 173 box + 42 UI tests green) plus the doc commits (d29a8663, 0e2e8a31); ember-tune bb37c993 (c9b31edc is job-script only); exec-app 032f03c; relay 28c028b.
- Gates: the same as every cut; the full canary with the fresh join decisive; publish on its line.
7. The node tree is a merge (05:5xZ)
ca3-v4-0318 (8220c944) does not contain release-0.3.18-node 12153428: their merge-base is 9a44fcb8 (ca3-v4-0316), so the branch lacks the ladder 1591ee1d, exec-sync 40fd8d8c, finality d8d1de1a, tail-emission 996b592f, the kaspad igneum-pow default 75467244, cc78a21f and 90aaf38e (it carries f2b25fdf, its own adaptation of the canary fix). The 0.3.18 node is the merge of 8220c944 (or later) into 12153428. A no-commit test merge conflicts in six files (consensus/core/src/igneum.rs, pre_ghostdag_validation.rs, ibd/flow.rs, v10/blockrelay/flow.rs, the two integration test files): the node lane resolves it, pushes release-0.3.18-node to the mirror, runs the suites and the integration check, and names the tip. The branch stays at 12153428 until then.
PC 1's 0.3.17 relaunch (section 7 of release-0.3.17.md) adds a node row for this tree: the template RPC blocks under the finality catch-up after a restart on a synced node (every getBlockTemplate timed out at 5 s for 90 s); the app row (bac43ac4) needs its rule changed to "template timeouts while the node reports synced are not a card fault".
8. The app tree assembled (05:4xZ)
release-0.3.18 at da5c40ba: miner-ui-4 afc331bd (in), 74c12665, 89501ce8, b7d33e8e, d29a8663, 0e2e8a31, bac43ac4, 5d9c55a2 (the UI lane's rule from PC 1's read-back: "template fetch timed out" is the miner's heartbeat, the card reads "waiting for the node to answer block templates", judged again from its last line; 173 box + 42 UI tests on its branch), ember-tune c9b31edc (job script; bb37c993 in), exec-app 032f03c (in), relay 28c028b (in). One merge fix by the shipper: 74c12665's merge view called the three-argument live::fetch; this tree's is the four-argument form (e5336dc7 gave it the engine's Shared), so engine.rs:3200 passes &shared as server.rs does (da5c40ba). App tests on the Mac: 179 + 28 + 8 passed. The node pin, binaries, inputs and staging wait on the node lane's merged release-0.3.18-node tip.
UI lane on the merge fix (05:5xZ): correct; the four-argument form wins. Note: Ember's fetch clamps the window to 300 s, so the merge view sees 300 s rather than 600; the check needs a block of ours inside 120 s, so it holds. The merge view, TemplateTimeout and the synced gate are present on origin/release-0.3.18.
The node merge, the node lane, 06:0xZ (not yet pushed): 8220c944 into 12153428 resolved: five conflicts taken as the release side (the ladder's header_signals_active reading, the test fixes), the sixth the relay flow (the pipelined handle_block carries the 0.3.16 IgneumProofMissing retry arm ahead of the N6 arm); one copy of header_version_acceptable kept. Box at the merged tree: kaspad check clean, integration check clean, consensus-core 123, p2p-flows 37, p2p-lib 20. A real finding: kaspa-consensus failed 1 of 112 then 4 of 112 under a box load of 178, every panic WrongBlockVersion(1026 or 32770, 2) in a plain-block test: a test-order race both parents carry (the signal and ladder tables are process-wide statics; two tests install and restore them; a block-building test inside that window reads the installed version; the two load flakes of the 12153428 suite were this). Fix: an install lock the two installers hold; the suite runs twice on the quiet box; if green the merge is pushed as release-0.3.18-node with the lock commit on top (tip about 06:20Z); if a reader still races, the two installing tests run serially as a second pass, said before the push.
Suite rule change with the merged node (the node lane, 06:1xZ): the install lock did not close the row (2 of 112 still WrongBlockVersion(32770, 2) on the quiet box), so the two installing tests (block_template_uses_current_block_version, cheap_checks_run_before_the_pow_engine) move out of the lib unit-test binary into their own target consensus/tests/igneum_installed_signals. From the 0.3.18 node on, the suite runs cargo test -p kaspa-consensus without --lib (all targets) so both run; a --lib run silently skips them. Push with the merged tree once the lib suite is green twice and the new target once, about 06:35Z.
9. The node tree: release-0.3.18-node = 1cf43254 (the node lane, 06:09Z)
The merge of ca3-v4-0318 (8220c944) into 12153428 with the six resolutions, plus the race fix in the same commit (the two installing tests moved to consensus/tests/igneum_installed_signals.rs and igneum_order_tests.rs; INSTALL_TEST_LOCK for installers sharing a binary). Box at 1cf43254: kaspad check clean, integration check clean, kaspa-consensus lib 110 twice on the quiet box, the two new targets 1 each, consensus-core 123, p2p-flows 37, p2p-lib 20. A second merge may follow within the half hour (PC 1's two node rules: template answers under catch-up, synced reads behind), else the cut is at 1cf43254. The shipper's box and Mac chains started on 1cf43254 at 06:1xZ (scratch only; the pin, inputs and staging wait for the node lane's cut word).
Main, 06:1xZ: the chain runs on 1cf43254 now. The node lane's two PC 1 rules (getBlockTemplate answers inside its timeout during the finality catch-up; isSynced false while the catch-up is behind the sink) land as a second merge with a node rebuild only if they reach the release branch before the inputs push; otherwise they are 0.3.19, said here. Full canary with the fresh join decisive, the IBD-end timers line to the node lane, publish on its line.
10. Builds on 1cf43254 (06:10Z on)
| piece | commit string | sha256 (first 8) | bytes | note |
|---|---|---|---|---|
| Linux igneumd, box native (the fleet's canary sha) | 1cf43254 | 96858a88 | 57,408,800 | 06:11Z; the fleet has it with GO for the full form |
| Linux igneum-miner, box native | fac45489 | 10,213,112 | ||
| Windows igneumd.exe (cross) | 1cf43254 | c29ee9e5 | 52,340,224 | 06:12Z |
| Windows igneum-miner.exe | 7ca9ca01 | 11,219,456 | unchanged from the 12153428 build | |
| Mac igneumd | 1cf43254 | 25b464b4 | 47,888,704 | |
| Igneum-Miner-0.3.18.dmg | a3822c4d | 44,248,348 | packaged object = the live sixteen; prover pair aboard; version 0.3.18 |
Digests on the Mac binary: sixteen eada4bda (igneum_getNodeInfo: powEngine igneum-pow, blockrate {bps 1, finalityDepth 43200, ghostdagK 18, mergeDepth 3600, pruningDepth 108000}), thirteen b18ed271, no file c562d70e (re-read on free ports, 06:15Z). All three unchanged from 0.3.16/0.3.17.
(suite, integration check, seed, hive, inputs, Windows run, staging, canary: rows as they land)
11. The cut: release-0.3.18-node = ae17ad00 (the node lane's word, 06:14Z)
1cf43254 plus the second merge of ca3-v4-0318 (e3a00bf0: 4f7c56a0 PC 1's two node rules, getBlockTemplate answers inside its timeout during the finality catch-up and isSynced reads false while the catch-up is behind; e3a00bf0 the installing-tests move on that branch too). Box at ae17ad00: cargo test -p kaspa-consensus 111 in the lib plus the two moved targets 1 each, p2p-flows 37, cargo check -p kaspad -p kaspa-rpc-service -p kaspa-testing-integration --tests clean. One hand fix in the merge: the W7 leaves block in template_section rides the template snapshot like the other items (newest 32, emitted last under the leave_active gate). The 1cf43254 chain was stopped at its seed step (its suite: see r0318/ae17/box-1cf43254.log) and both chains restarted on ae17ad00 at 06:2xZ. The canary form gains two reads for PC 1's rules: on a restart with a kept datadir, every getBlockTemplate inside 5 s through the catch-up (no "template fetch timed out" lines), and isSynced false while the catch-up holds the finality state, true after.
Canary on ae17ad00 started 06:16:47Z (c18-1, RTX 3070 pod, wiped datadir, node b06c1a97, 57,447,712 bytes, miner fac45489; the 1cf43254 early run stopped by pid; it had read vline 1cf43254, digest eada4bda, igneum_getNodeInfo with blockrate, IBD from 4 peers). Clock: headers through about 06:52Z, synced 07:12Z, mining reads to 07:22Z, the restart reads (isSynced false then true every 10 s; max template_ms, timeout count, template count) to about 07:40Z, then the relay and poison cases with c18-1 as the target.
12. Builds on ae17ad00 (06:15Z on)
| piece | commit string | sha256 (first 8) | bytes | note |
|---|---|---|---|---|
| Linux igneumd, box native (the canary sha) | ae17ad00 | b06c1a97 | 57,447,712 | 06:16Z; the canary runs on it from 06:16:47Z |
| Linux igneum-miner | fac45489 | 10,213,112 | unchanged from 1cf43254 | |
| Windows igneumd.exe (cross) | ae17ad00 | e4979672 | 52,391,424 | pow link 9 |
| Mac igneumd | ae17ad00 | a56469d7 | 47,905,856 | |
| Igneum-Miner-0.3.18.dmg | f617b63b | 44,241,681 | packaged object = the live sixteen; prover pair aboard | |
| Windows inputs | ae17ad00 | pushed 06:19Z (workers d7a413c7 etc, the Linux prover pair), pin 1c8fb76a on release-0.3.18, windows.yml run 37580969266 dispatched 06:20Z |
Digests on the Mac binary: sixteen eada4bda (igneum_getNodeInfo powEngine igneum-pow, blockrate as before), thirteen b18ed271, no file c562d70e (free ports). All three unchanged. Box suite on ae17ad00 rc 0 (kaspa-consensus without --lib, so the two moved targets ran). Identity grep on the payload UI clean. The 1cf43254 builds (96858a88, c29ee9e5, 25b464b4, DMG a3822c4d) are superseded and not staged.
| generic Linux igneumd (class seed, glibc 2.35) | ae17ad00 | 99809615 | 56,141,136 | pow link 6; the seed's own build comes from the build-server lane |
|---|---|---|---|---|
| HiveOS igneumd (class hive, glibc 2.31) | ae17ad00 | a2e734d1 | 56,141,776 | pow link 6 |
| igneum-hive-0.3.18.tar.gz | 21ea06eb | 27,234,062 | 2.31 node pair + the box's workers 193ec36f/7a35ff2a; no ._ entries; ubuntu:20.04 container smoke on the box: all four load (06:23Z) |
| hands igneumd (box native, the build-server lane, pairs with igneum 1c8fb76a) | ae17ad00 | 17209647 | 57,448,096 | held for the line; OUT_DIR apart from b06c1a97 | | seed igneumd (class seed, the build-server lane) | ae17ad00 | fa9c2f12 | 56,139,920 | GLIBC_2.34; held for the line | | hands / seed igneum-miner | | 9adcb707 / 2ebaee58 | 10,213,112 / 10,234,128 | |
Integration check on ae17ad00 rc 0 (06:18Z). The hands and seed go last on the shipper's line after the miners (observer, node 1, seed), each read back by commit string, digest and igneum_getNodeInfo powEngine with its blockrate object.
13. Staged at the line (06:29Z)
Windows run 37580969266 green (06:27Z): Igneum-Miner-Setup-0.3.18.exe 2fcaf093, 62,822,510 bytes. Scratch copy r0318/dlsite-stage (fresh from the live folder): manifests at 0.3.18, mac f617b63b, windows 2fcaf093, consensus.override = the live sixteen-field object, HiveOS alias held at 0.3.17 until its own row; the live folder holds no 0.3.18 file. Runbook r0318/deploy.sh (preflight passes; step_manifest, step_update_now, step_hive with the 0.3.18 tar, step_readback with igneum_getNodeInfo). Discord card dry-run (four reader-facing lines, no em dash). Everything waits on the canary's line.
14. The canary (c18-1, ae17ad00 / b06c1a97)
- 06:34:54Z: headers 47 percent (59,425) in one IBD session since 06:17:32Z, 17 minutes in and 5 past the old guard mark; SendPingsFlow 0, idle-drop lines 0, "completed with error" 0; the class-signal warning is ONE line (49,844 on the hotfix). Headers through about 06:55Z on the 3070, synced about 07:15Z.
15. HOLD on ae17ad00: the exec RPC panic (the node lane, 06:4xZ)
The Mac's headers-proof join gate killed its joining node mid-IBD on the canary-fix binaries: a panic at igneum/exec/src/rpc.rs ("index out of bounds: the len is 0 but the index is 0") on a tokio worker, and the node's panic hook exits the process. The site: eth_getBlockByNumber reading state.records[n] after resolve_block mapped "latest" to tip_number() = 0 on an exec state with no records (the follower still "waiting for consensus to sync"); a local client polled 127.0.0.1:26790 during the IBD and the node died at 66 percent of the headers stage. Every records index in that file is unchecked on both trees (0.3.17's 5899f603: rpc.rs lines 474, 550, 591 to 629; ae17ad00: 575, 651, 692 to 730), so any fresh or restarting node with the exec RPC bound and a client asking eth_getBlockByNumber("latest") before the follower has a record dies. The shipped app itself calls only eth_blockNumber and igneum_* methods (none index records by block number), so the live 0.3.17 risk is a wallet or third-party client on a fresh node mid-IBD; the fix (every index bounds-checked; "latest" on an empty state answers null; a test on an empty state) goes onto ca3-v4-0318 and into release-0.3.18-node as a third merge. The pin stays at ae17ad00 on origin and nothing is staged further until the third tip; the ae17ad00 canary runs on (nothing attached to its eth_ port) as an early read of the other rows.
The 0.3.17.1 question settled (07:0xZ, main's rule: a hotfix only if the shipped app sends the panic class): no. The dead harness node's site is eth_getBlockByNumber; the shipped app sends eth_blockNumber and eleven igneum_* methods only (grep of the whole 0.3.17 app tree); its two records-indexing polls (igneum_getAssignedShards, igneum_getProofRecords) run only after the node reads synced, when a devnet follower already holds records from the packaged restart block; the two it sends while unsynced index nothing. The wallet app sends eth_getBlockByNumber to its own node. Ledger N7 carries the method map. The public testnet filter now blocks ten eth_ and six igneum_ records-indexing methods (seed1, verified; tree copy committed).
16. The cut moves to release-0.3.18-node = e69e8a39 (the node lane, 06:53Z; the pin released)
ae17ad00 plus the third merge of ca3-v4-0318 (500ddd66): c6a62e00 (every chain-block read in the exec JSON-RPC bounds-checked; an exec state with no record answers an error instead of indexing; test on the empty state) and 500ddd66 (GetBlockTemplate stage timers: a debug line per request, an info line once per 10 s past 500 ms; the epoch-seed walk and the live sink tally memoised). One hand resolution: the payout parameter on igneum_getProofRecords beside the bounds-checked read. Box at e69e8a39: igneum-exec 26, kaspa-consensus 111 plus the two moved targets, p2p-flows 37, kaspad / rpc-service / testing-integration checks clean. Both chains restarted on it 06:55Z (the ae17ad00 logs kept in r0318/ae17/). The canary form gains two reads: an eth_ client polling the joining node's exec port through the IBD (the node survives), and the first slow "GetBlockTemplate N ms" line on the pool's loaded node after the publish.
17. Builds on e69e8a39, the pin (06:55Z on)
| piece | commit string | sha256 (first 8) | bytes | note |
|---|---|---|---|---|
| Linux igneumd, box native (the canary sha) | e69e8a39 | 252c8dad | 57,461,600 | the form started from the wipe 07:02:54Z |
| Linux igneum-miner | fac45489 | 10,213,112 | unchanged | |
| Windows igneumd.exe (cross) | e69e8a39 | c1a42970 | 52,379,136 | pow link 9; inputs pushed 07:03Z, pin 0911b00f, windows.yml run 37585128393 dispatched 07:04Z |
| Mac igneumd | e69e8a39 | 066807ae | 47,921,280 | |
| Igneum-Miner-0.3.18.dmg | cb30080e | 44,247,374 | packaged object = the live sixteen; prover pair aboard; version 0.3.18 | |
| generic Linux igneumd (class seed, 2.35) | e69e8a39 | 7d67cb51 | 56,152,400 | |
| HiveOS igneumd (class hive, 2.31) | e69e8a39 | 6293b443 | 56,153,104 | |
| igneum-hive-0.3.18.tar.gz | 72699158 | 27,236,213 | 20.04 container smoke clean, no ._ entries (07:08Z) |
|
| hands igneumd / seed igneumd (the build-server lane, pairs with igneum e9ccb3a1) | e69e8a39 | 37610a77 / 63cd49be | 57,461,984 / 56,152,720 | held for the line |
Digests on the Mac binary (explicit arguments; a zsh loop had not split them on the first pass): sixteen eada4bda, thirteen b18ed271, no file c562d70e. igneum_getNodeInfo: powEngine igneum-pow, blockrate {bps 1, finalityDepth 43200, ghostdagK 18, mergeDepth 3600, pruningDepth 108000}. The N7 class on the Mac binary: eth_getBlockByNumber ["latest", false] on an empty exec state answers {"result": null} and the node lives (the node lane: null is Ethereum's shape for a block that is not there; every other read on an empty or short state answers an error object). The branch tip moved to 2a014bf1 (test-only: all 46 exec RPC methods through the real dispatcher on empty and one-record states, no panic); the pin stays at e69e8a39, whose shipped bytes it equals.
The chain's own log was lost mid-run (the build-server lane removed scratchpad/r0317 and r0318 whole while dropping its superseded pairs; rule now: no lane removes a scratch directory it did not create; this lane's scratch is r0318-ship); the suite and the integration check are re-run on the box for the record (r0318-ship/suite-e69e8a39.log).
Suite on e69e8a39 (the box, 07:08Z, re-run for the record): igneum-exec 26, kaspa-pow 19, kaspa-consensus 111 (lib) + the two moved targets 1 each, consensus-core 123 + 7, kaspa-p2p-flows 37, igneum-miner 16; 0 failed; rc 0. cargo check -p kaspa-testing-integration --tests rc 0. Together with the node lane's own runs at e69e8a39 and 2a014bf1 this is the gate's suite line.
18. Staged at the line (07:1xZ)
Windows run 37585128393 green (07:10Z): Igneum-Miner-Setup-0.3.18.exe 669d5676, 62832534 bytes. Scratch copy r0318-ship/dlsite-stage (fresh from the live folder): manifests at 0.3.18, mac cb30080e, windows 669d5676, consensus.override = the live sixteen-field object, HiveOS alias held at 0.3.17 until its own row; the live folder holds no 0.3.18 file. Runbook r0318-ship/deploy.sh (preflight passes). Discord card dry-run (five reader-facing lines, no em dash). Everything waits on the canary's cases line.
For 0.3.19 (not the pin), the node lane 07:1xZ: ca3-v4-0318 = fd7de1b4 (the live class-signal tally refreshed off the request path; kaspad check clean, consensus 109). e69e8a39 memoises the tally for 10 s per sink, so the pool's canary should show the "pow epoch" stage near zero on nine requests in ten and the full walk (1 to 1.6 s) on the tenth; if the stage line shows it high on most requests the reading is wrong. fd7de1b4 is the first 0.3.19 node commit (main, 07:1xZ: no four-part numbers, the parser refuses them); 0.3.18 ships as pinned at e69e8a39 on its canary's line.
19. The canary on the pin (c18-1, e69e8a39 / 252c8dad, from the wipe 07:02:54Z)
- 07:21Z: 17 minutes into one IBD session, headers 37 percent (48,160 at 07:16:45Z), past the old guard mark; SendPingsFlow 0, idle-drop 0, IBD errors 0, the class-signal warning 1 line, node alive; the eth_getBlockByNumber poller at 208 polls, 0 error answers, all null. Headers through about 07:45Z, synced about 08:05Z.
20. PC 1 and PC 2 after the project lead reopened the apps (07:28Z reads, read-only)
- PC 2 (1ccfe586): back online; its app came back on 0.3.16 and applied 0.3.17 at reopen (update: version 0.3.17, current, updated_from 0.3.16); node 2.1.0 synced at DAA 262,124 with 5 peers; the RTX 5090 mining at 100 MH/s (pid 19620, 11 accepted in its first minutes), no fault lines. The update-now takes it to 0.3.18 directly.
- PC 1 (ae432dc7): app 0.3.17 (reopened, uptime 531 s at the read), node binary 5899f603 (3 marks, pow link 9), digest eada4bda; but the node is in a restart loop: state "restarting", starts 16, igneumd not running at the read; every card "waiting for the node to sync" (mining "waiting", not faulted this time). Cause being read from the engine's node lines and the node logs (read-only job). Rule for the publish: PC 1 and PC 2 take their update-nows first and their workers are read back at their rates as the per-box table's first line.
Latency ladder rung 3 re-measured (the node lane, 07:46Z, in the shipper's box window under the measure hold): reps 88, cold alone 9.04 ms, cold with the SMT sibling loaded 10.85 ms, averages of 50 at 5.95 and 10.11 ms; over the 10 ms gate by 0.85, so rung 3 stays inadmissible and the testnet genesis freezes with 88 false. Rung 2 as the control on the same run: 9.25 ms cold loaded, admissible.
PC 1's restart loop IS the N7 class on the shipped kit (07:5xZ, the engine log app-20733-072141.log): every node start ends 2 to 5 s later with "panicked at igneum/exec/src/rpc.rs:591:41: index out of bounds: the len is 0 but the index is 0" (eth_getBlockByNumber's records[n] in 0.3.17's tree), the app restarts it every 40 s (starts 16 at 07:28Z), the miners are stopped each time. CORRECTED 08:0xZ: the client is the shipped engine itself: latest_block_time (update.rs:46) POSTs eth_getBlockByNumber ["latest", false] through curl to the exec port every 9 s as the app's clock sample once the node reports blocks > 0 and peers > 0 (engine.rs:3752); export-pack's lines were its own failure after the node died (igneum-miner's export-pack speaks gRPC only). So every 0.3.17 node with the app attached dies within 9 s of blocks arriving whenever its exec follower has no record: PC 1's loop, and any fresh install's IBD (the hotfix canary had no app attached). Proving-off would change nothing and was not run. c6a62e00 (0.3.18) ends it; the 0.3.18 canary's eth_ poller is the app's own call and the node lives on it. PC 1 takes the 0.3.18 update-now first on the publish line.
Main's publish rule (08:0xZ): 0.3.18 publishes on the canary's synced line plus the poller summary (the decisive reads for the N7 fault, the app's own 9-second call surviving the whole IBD), not the cases line. Order: PC 1's update-now first (it is crash-looping), then PC 2, the Mac, the fleet one box at a time under the lock rule, the hands and the seed by their owner. The restart reads and the cases follow on the same pod as confirmation; a FAIL there is a rebuild, not a rollback of this fix. Then "0.3.18 live" with the per-box table and the card.
21. HOLD on e69e8a39 (the fleet, 08:2xZ): the block stage cannot complete against any peer
The canary passed its headers proof at 08:00Z (one session, zero guard lines, the app's eth_ poll alive at 918 polls) and froze at 2,728 blocks: every IBD attempt (110 by 08:2xZ, every peer) ends "block f52e64f4... carries 1 proof records whose proofs this peer did not deliver in 20 s". The node lane's reading: exec-sync's daemon installs the proof oracle whatever proving_consensus_verify_daa says (daemon.rs, after the keys branch), so the IBD flow (flow.rs, the 0.3.16 "carried proofs come from the syncer" rule) demands the proof bytes of every record-carrying block before queuing it while the serve flow answers only from a peer's bounded recent pool; block 2,728 is months old, so a fresh node of this tree can join NO network, and a synced one would verify every relayed record natively and refuse blocks its 0.3.17 peers accept (a split in waiting). Fix, joiner-side: the oracle carries its switch (active_from); the body rule, the IBD fetch and the relay retry apply to blocks at or above it only, so at never the node is 0.3.17 on this path. A direct commit on release-0.3.19-node; the pin moves to it; the canary re-runs from the wipe.
22. For the 0.3.19 node: ledger N8 (the project lead's word, 08:5x UK)
The execution layer credits merged blocks at the chain block's DAA (the UTXO coinbase pays each its own): d840537b on ca3-v4-0318, field subsidy_per_block_activation_daa (u64::MAX = never as compiled on every network; in the digest once set). The devnet takes it at an upgrade height carried by 0.3.19 (a DAA score past the rollout's last box under the lock rule, set in DEVNET_PARAMS at the cut; every node must carry it before that height or its EVM state diverges); the testnet from genesis. The node lane names the value when the 0.3.19 cut line is named (its canary's pass).
App tree (08:3xZ): miner-ui-4 taken through 3661dc9e (a2670ace the two site captures; 3661dc9e carries 0e6c1248's exec-record clock gate without its version bumps), on top of the earlier chain; app tests 180 + 28 + 8 green. Ember: c870c383 (job-script only, equals bb37c993 on the app) at the rebuild.
The exec lane agrees (08:4xZ): exec-sync-0313 HEAD carries the same shape (ProofOracle::active_from() = proving_consensus_verify_daa, set by the daemon before the oracle is handed out; the IBD pre-fetch gated on it; the body rule and the relay retry already gated); igneum-exec 23 green. The node lane's direct commit on release-0.3.19-node is the one the pin takes; the shapes reconcile at the next merge; the switch must live on the oracle (one source). Exec lane's 0.3.19 tips: node exec-sync-0313 HEAD, app exec-app-0314 731268a1. Its "0.3.19.1" (the finality-horizon skip) is a four-part number and so 0.3.20 material.
23. The pin: release-0.3.19-node = dc141409 (the node lane, 08:36Z)
e69e8a39 plus the oracle switch: ProofOracle::active_from carries proving_consensus_verify_daa; the body rule, the IBD proof fetch and the relay retry apply to blocks at or above it only (one source, on the oracle: igneum/exec/src/proving.rs:1499, read at flow.rs:1024 and body_validation_in_context.rs:47), so at never a node demands no proof of any block; the sink takes the switch from Params through proof_sink (daemon.rs:1018). Box at dc141409: igneum-exec 27, consensus-core 123, kaspa-consensus 111 plus the two moved targets, p2p-flows 37, kaspad and testing-integration checks clean. The exec lane's own commit ac6e32cd on exec-sync-0313 has the same shape and stays as the record (reconciled at the next merge). Both chains restarted on dc141409 at 08:4xZ (scratch r0319-ship); the pairs rebuild by the build-server lane. Canary reads for this tip: the block stage passes 2,728 and every record-carrying block with no "Proofs: asking" line, the IBD-end line with its finality time, the synced line, the app's eth_ poll alive throughout, the restart reads, the cases, and on pool-1 after the publish the first slow "GetBlockTemplate N ms" line. The 0.3.19 cut line is that canary's pass; the node lane then names the devnet DAA for the per-block subsidy height (N8).
Hands and seed pairs on dc141409 (the build-server lane, 08:4xZ, pairs with igneum 5010536a): hands igneumd 8be8a5a5 (57,460,512 B), igneum-miner 92740aea; seed-class igneumd 4910352b (56,154,320 B, GLIBC_2.34), igneum-miner 9d7c4b8a; held for the line after the miners.
24. Builds on dc141409 (08:43Z on)
| piece | commit string | sha256 (first 8) | bytes | note |
|---|---|---|---|---|
| Linux igneumd, box native (the canary sha) | dc141409 | 3f9aca09 | 57,461,792 | GO sent to the fleet 08:5xZ; the form from the wipe on c18-1 |
| Linux igneum-miner | fac45489 | 10,213,112 | unchanged |
(cross, suite, integration check, seed, hive, Mac, DMG, inputs, Windows run, staging: rows as they land; the first chain start on this tip at 08:37Z had picked up the hotfix tree's script copy and was stopped at 08:39Z; its one native build, d712b498, is the 0.3.17 binary again and is not used)
25. The canary on dc141409 (c18-1, from the wipe 08:51:50Z)
Node 3f9aca09, POLL_EXEC=1 and the exec-status fields; the e69e8a39 node stopped by its kill file (frozen at 2,728 blocks since 08:00Z). The form prints the count of "Proofs: asking" lines and of "carries N proof record" IBD errors beside the IBD-end line. Clock: headers through about 09:27Z, the block stage past 2,728 about 09:35Z, synced about 09:50Z, mining reads to 10:00Z, the restart reads to about 10:15Z, then the relay and poison cases; pool-1's "GetBlockTemplate N ms" line after the publish.
26. Owed to the 0.4.0 cut (the launch-pack lane, 08:5xZ; not this tree)
- The signing step (docs/plans/launch-pack.md section 2.4 on master
9b98b837, gate LG-3 in docs/plans/testnet-go.md): implemented only once the certificates exist (Windows EV Authenticode and an Apple Developer organisation account, both under Igneum Labs LTD, enrolled the day the entity exists; the owner approved both 7 October 2026). Until then 0.4.0 ships unsigned and the download page says so. The step keeps ship-app.mjs's list and adds: windows.yml signtool through the CA's cloud KSP (two secrets; unsigned and marked so without them), Inno SignTool=; ship-app.mjs fetch osslsigncode verify (CN Igneum Labs LTD, SHA-256, timestamp; unsigned fails); build-dmg.sh codesign Developer ID with runtime and timestamp, notarytool --wait Accepted, stapler on the app and the DMG, no quarantine strip; publish-manifest.sh signed_by and notarized per platform, --public refused without; ship-app.mjs verify osslsigncode and spctl --assess on the live files; the download page and the Discord card say "Signed by Igneum Labs LTD" only when the manifest does; tools/ci/signed-release-check.sh in pre-push. Keys never on igneum-build-1. - docs/analysis/income-tiers.md regenerated from tools/launch/income-tiers.json at the 0.4.0 cut, the rows re-measured on class v4 (node tools/launch/income-tiers.mjs; --check in pre-push). Names to build against (the launch-pack lane): GitHub secrets IGNEUM_CODESIGN_ACCOUNT and IGNEUM_CODESIGN_KEY (TOTP seed or API key by the CA; absent on forks and PRs, so unsigned and marked); notary keychain profile igneum-notary (xcrun notarytool store-credentials, once on the Mac); the codesign identity "Developer ID Application: Igneum Labs LTD (TEAMID)" read from one place (an env or a packaged-config field), the team id dropping in without a code change. Values reach the shipper by relay, never the repository (launch-pack.md section 5, items 3 and 4).
Branch tip past the pin (the node lane, 09:04Z): release-0.3.19-node = aea0ca5c on the mirror, on dc141409: the proof archive (ledger N9's second half; every carried record's proof kept under the exec db dir's proofs/ for the pruning window, served to a joiner's IgneumRequestProofRecords past the pool's 600-block window, dropped below the pruning point; no digest field, no behaviour on the devnet at never). Box: igneum-exec 28, p2p-flows 37, kaspad and testing-integration checks clean. The pin stays at dc141409 (the canary running on it since 08:51Z; a re-pin only on a FAIL); aea0ca5c is 0.3.20's first node commit otherwise.
Suite coverage note (the node lane, 09:1xZ): processes::pruning_proof::igneum_m20_tests::witnesses_are_checked_in_epoch_order_under_their_own_seeds fails deterministically (MissingEpochSeed(1, _, 109) not matched) on an untouched e69e8a39 worktree with --features igneum-pow at any box load; it sits behind #[cfg(feature = "igneum-pow")], so no lane's suite line (cargo test -p kaspa-consensus without the feature) compiles it, which is why 111 reads green. Owner: the M20 pruning-proof witness work (the epoch seed table of a proof-synced node). Not a publish blocker by itself (the gated path is what the live binaries run; the red is a test or seed-table reading), but the suite line must either carry the feature or the red must be fixed before "all green" covers it: a row for the 0.3.20 suite rule.
27. miner-ui-5 is the 0.3.19 app (the project lead's word at 13:1x UK: "deploy the miner look")
Taken onto release-0.3.19 as five cherry-picks on the miner-ui-4 chain: 2b4775b4 (the ladder's engine half: ladder.json, api/ladder chain facts, the block card), 449fb207 (the ladder on every page, the count-up, the block card, Earnings in IGN), b49db57d (the plan, the captures, the first-share runbook, the Discord rules), 3973ff8b (live-dag.js 2.0.1, the site lane's real-data options), e9fe1106 (u04 and u14 reshot); tip a199160d. Gate: app 197 + 28 + 8, UI 47 (notices 8, tune-line 5, update-card 4, view 30), all green. Next: push, windows.yml, the DMG (after the dc141409 Mac chain), staging; the read-back line adds "app miner-ui-5" and the pack's two shas. If the canary slips past 15:00 UK, main decides whether the app ships alone as 0.3.19 on the 5899f603 pin and the feature node becomes 0.3.20.
28. The prover pair (main's order, 09:0xZ): handed over, verified, the CI check in
The fleet's 14 standing provers had proved nothing since the hotfix: their harness (tools/fleet/box-prover.py) exported igneum_exportSegments from the restart block 27276 to the tip on every claim and replayed 133,000 blocks through 72142, where every port (the 6 October floor exporter, master's 6c3cc8b9 core, the kit's 263bf4ce) computes state root 0x8e1bef4b against the nodes' 0x9851d7e2 (hub-1, p1-5090 and PC 2's 0.3.17 node agree: 27276 0xed27bb2d, 72141 0x103190f4, 72142 0x9851d7e2, 72143 0x212e7703). The shipped prover never replays that far: the app exports [first-1, last] with the account dump (prover.rs, the 0.3.14 rule). On p1-5090 the kit's pair (host 71bc2438, export 263bf4ce; the same bytes in the 0.3.17 and 0.3.19 kits; the proving crate and the exec types identical across both trees) fed the app's form reads "account dump: 83 accounts after chain block 160829, state root equals the node's; replayed 8 segments, every state root equals the node's", so the pair is right; the fleet rolls it one box at a time with box-prover.py exporting the app's way, the two pairing lines read after one segment. The from-27276 divergence at 72142 is with the node and exec lanes (not on any proof path). "e809e396" is not a prover on the box (the hands run igneumd only). CI: tools/ci/prover-pair-check.sh (the prover's evm-types tree equals the pinned node's; --self-test; in pre-push, 32 checks) reads ok on the pin. One oddity for the node lane: igneum_exportSegments ["0x2747d","0x27485"] answered with segments 160829 to 160837, 64 blocks below the ask.
29. The 0.3.20 tree as of 10:2xZ (after the split and the Mac's reboot)
- Numbers: 0.3.19 = the app-only miner-ui-5 cut on pin 5899f603 (release-0.3.19); this tree = 0.3.20 (release-0.3.20 app, release-0.3.20-node = dc141409 on the mirror, the canary on it running as 0.3.20's gate); decimals 0.3.21 unless main says otherwise.
- Node side to come on release-0.3.20-node (the node lane): aea0ca5c the proof archive; the amended class v4 (CLASS_SIGNAL_V4 = 5; byte-4 signals never count; the kaspa-pow vector test; the daemon's window line naming object and sub-version; igneum-pow at the hash lane's
a0aaca92with the seven packs under the sub-version-1 stamp); the exec RPC listener watchdog (a dead listener rebound once with a log line, a second death within a minute exits; gated on the every-method test and a kill-the-listener unit test); N8's subsidy height (DEVNET_PARAMS at the cut). The flip arithmetic (the Counter ASIC lane, plan 6.6 on ca3-v4-node fa5bc9e6): the two v4 fields publish only after the one-sweep rollout and every worker on the 0.3.20 tree; the floor moves to the publish DAA + 604,800 rounded up to the epoch boundary (882,000 for a 12:00 UK publish on 7 October; recomputed from the live DAA at the publish), the window 86,400 unchanged; the earliest flip about 6 days 10 hours after the publish, never before every node has had the sweep plus a week. The rollout clock for that: 32 minutes for the 14 standing boxes under the lock rule, the hands and the seed about 3 minutes after, the Mac and PCs within minutes. - App side: ember-heat (worktree igneum-wt-ember-heat, 7c779035, cd1034a2, 0d5fc6d2 on e9fe1106: heat mode in the engine and Ember, the region and price step at first run, the cost-against-rent row, two gate checks; app 198, UI 56, gate 32 green), main's word; its PC 1 4-hour hold is a runbook, not a gate. The ui-ota channel (the same lane): a "ui" object inside the signed manifest body, one signature; the bundle under dl/ as the installers; ui.version three-part; publish-manifest.sh gains --ui (the one writer); the kill switch is the object's absence plus a fallback on any mismatch; a CI check that the manifest's ui.sha256 equals the public tar. The 0.3.19 gate module (execrpc) and version carry over at the merge.
- Ledger: N10 (eadae138): the 72142 port-versus-node root is the thin-record export after a snapshot cut at FULL_RECORDS 1,200, not a rule; the exec lane owes the one-time re-execution that fattens thin records and an exporter that refuses a thin segment; "accounts" in an export is the tip state.
- The Mac rule (main, after the 10:5x UK crash): the Mac builds only the macOS binaries and the DMG, one at a time under the build lock; every other build, suite and the Windows cross-build on the box or a PC; the app gate runs on the box.
Node lane, 10:3xZ: (1) the exec RPC listener watchdog is written for the node line (rpc::serve_watched: the listener task polled every 10 s, rebound once on a death with "exec RPC listener died: {reason}; rebound on {listen}", a second death within a minute exits 3 "so the app sees it"; gated by the every-method test and a tokio kill-the-task test). (2) The igneum-pow of the 0.3.20 cut is the hash lane's 8c728ca3 (ca3-v4-amend), not a0aaca92: a0aaca92 keyed the load-source rule on the whole class with the shadow's pass count inside, so the base program moved with the ladder rung (caught by the fork's ladder test on the box 10:06Z) and did not compile against dc141409 (no chain_program_shadow); 8c728ca3 keys it on the class with the pass count set aside and carries the ladder igneum-pow underneath; the pinned ids do not move. (3) On the node line for the observer: igneum_claimSegment, igneum_getProofClaims and claims on igneum_getProofRecords (the claim posted before a prove), plus the app lane's four finality methods. Tips as each lands.
30. The dc141409 canary: synced 10:29:19Z, every decisive read clean (the fleet)
c18-1 (RTX 3070 pod, wiped datadir, from 08:51:50Z; the Mac's reboot cut the form's shell at 10:5x UK and it re-attached): synced at 141,357 blocks (headers 141,617, 3 peers). IBD-end line: "10:29:13.927+00:00 [INFO ] IBD with peer 213.173.107.74:16516 completed successfully; finality time: bodies 16592238 ms over 141699 blocks, virtual 1061858 ms over 36183 changes, weight tables 968243 ms over 359709 tables (3832285840 blocks walked), signatures 711416 ms over 439307, persists 40537 ms over 3324"; the relay catch-ups after it a second each. Counts: "Proofs: asking" 0, "carries N proof record" 0 (the oracle switch holds), SendPingsFlow 0, idle-drop 0, the class-signal warning 1 line, 6 IBD sessions, 2 "completed with error" before the re-attach (lines owed with the restart reads). The app's poller: 4,217 eth_getBlockByNumber calls, 0 errors, 0 non-null, the node alive. At the synced line the exec layer read "exec not synced: this node's consensus starts at pruning point 36a7ba0d" (executedTipHash null). Next: ten minutes of mining, the hub read about 10:40Z, the restart reads to about 10:55Z, the cases on the fresh pods. Prover roll 7 of 13 paid; hub-1's prover moved to the pair and the [first-1, last] export.
App side taken for 0.3.20: ui-ota 0247b065, c337f768, 10c881dc (on miner-ui-5 b322e9fa: the "ui" object inside the signed body plus the entry's own signature, kept; "size"; src/uiota.rs; Settings > Interface; publish-manifest.sh --ui/--no-ui; tools/ui-ota/publish.mjs with --verify as the post-deploy step; ui/VERSION 1.0.0 embedded, 1.0.1 the first bundle, min_engine 0.3.20 so a 0.3.19 engine ignores it) and ember-heat 7c779035, cd1034a2, 0d5fc6d2; both green on the box. The miner-reliability branch (workers start only when READY = synced and an executed tip; the node watchdog never counts the catch-up; the restart ladder with no permanent fault; fault lines to the intake) is main's call: a 0.3.19 follow-up on the same pin, or this tree's app.
Fleet kit rule (main, 10:3xZ): one prover identity per box; the kit never ships an identity file; box-prover generates its own at first start from the box's label and keeps it in the registry row; a shipped or duplicated identity is refused at start with a line. Migration on the shared boxes (9e4ba6b0 on three, faa34a1a on one) one at a time, prover only; prover identity keys are not vote keys (no weight, no signal), so the lock rule does not apply.
Main (10:4xZ): miner-reliability rides 0.3.20's app with ember-heat and ui-ota (ee09ae8b, docs only, goes with its three); no app-only follow-up and no renumber. One exception: if the reliability lane's watchdog-clock and export-lock changes land small and green before the node line is ready, main decides an app-only 0.3.20 then (the feature node would become 0.3.21). PC 1 today: the Arc held off and the packs-ahead restart on 0.3.19.
31. Rows from PC 1 on 0.3.19 (11:3x to 11:4x UK)
- The template path on the 0.3.17 node: on PC 1 (24 identities, 3 cards x 8) every getBlockTemplate times out at 5 s on every identity ("template fetch timed out (5 s) for identity N", 59 + 43 + 28 lines in two minutes), no STATUS line; the 0.3.19 watchdog treats it as the miner's heartbeat ("waiting for the node to answer block templates") so the cards wait rather than fault. The node logs no per-request time on this tree (the timers are 500ddd66, 0.3.20). Mitigation today: identities down to 2 per card by the project lead's tap (no signed job can set identities: a script may not POST /api/cards, and the runner's --cards-off only toggles enabled; a runner option for identities is a small jobrun.rs change, main's word). The fix is the 0.3.20 node on PC 1 first (the memoised tally 500ddd66; fd7de1b4 the off-path refresh is 0.3.21 material unless main moves it).
- Two app faults for the reliability lane's rules, both 0.3.20's app (main): (1) engine.rs:3032, the runner's --stop-miners hold is not released while a following job runs (the resume fires only when job_hold is set and no job holds the miners), so a read-only watch job kept both cards off for three minutes; (2) orphan igneum-miner.exe processes the app no longer tracks (two alive under --stop-miners with their rows at pid 0, one after) hammer the node's template RPC beside the tracked miners, part of why PC 1's template calls ran past 5 s. Sizing of the orphan-kill for an app-only 0.3.20: below.
- The fleet's prover-identity finding was withdrawn (no box shares a key; the "shared" hashes were a carrier block's record list); no migration. The kit rule stands on another ground:
igneum-miner key-hash <label>is a pure function of the label, so the 0.3.20 kit seeds the identity from the box, keeps the hash on the registry row, refuses a duplicate. - The node lane's IBD-end read (ca3-v4-node d49c71d0): the weight-table cache (64 tables, cleared past that) made every 2,000-body IBD batch walk the window again (359,709 tables for 5,335 checkpoints, 968 s); fix on release-0.3.20-node: WEIGHT_TABLE_CACHE = 1,024, oldest-first eviction, never a clear, a unit test; bodies 16,592 s is thread time behind the finality state lock, not CPU; the next fresh join on the pod should read 60 to 70 minutes.
Main (11:5x UK): a one-off exception to the 6 October rule, the project lead's word ("you will have to fix it"): exactly one POST to /api/cards on PC 1 from the Intel lane's scratch script (not in the tree), the 5090 and the 9070 XT to 2 identities and the Arc off; the engine restarts the workers on the change. The permanent path is the reliability lane's signed cards job kind (per-card enabled, identities, power_pct, through the app's own card path, read back), which replaces any runner option: 0.3.20.
29. The app side assembled on release-0.3.20 (11:5x UK)
Picked by hash onto ca1742cd, in order: miner-ui-5 810bf5a1 and b322e9fa; ui-ota 0247b065, c337f768, 10c881dc, ee09ae8b; ember-heat 7c779035, cd1034a2, 0d5fc6d2. A merge of ui-ota was tried first and aborted (14 files, its base is miner-ui-5's lineage, which this tree carries as picks). Five conflicts, each resolved by union: publish-manifest.sh keeps Ember's activation_passed form and takes ui-ota's --ui/--no-ui arms, UI block and the "${UI:-}" argument (one python call, sys.argv[1:13]); the Settings literal in config.rs and the SettingsState literal in engine.rs carry both heat and ui_builtin; app.js's export carries both lanes' names; pre-push.sh runs the ui-ota self-test and the heat gate reader. The Mac's igneum-ota-sign was rebuilt under the build lock (the stale binary lacked sign-ui, the pre-push self-test read RED). UI tests on the box: heat-region 8, notices 8, tune-line 5, ui-ota 4, update-card 4, view 30, all pass. The app gate (cargo test --release) runs on the box; the push follows its green.
Held in the 0.3.20 verdict (the fleet, 11:4x UK): isSynced reads false at the tip on dc141409 (7 of 12 reads false with blocks equal to headers and no IBD session; a 17-read sample 13 false; the 0.3.17 control reads true 12 of 12). Mining, acceptance and the exec follower unaffected. Relayed to the node lane: a fix on the node line or a ruling that the kit reads the tip. The publish waits on that word.
Main (12:0x UK), the Arc B580 for 0.3.20's worker kit: Intel's OpenCL compiler folds the kernel's rotr_var helper rotate(x, (0u - n) & 31u) into a left rotate by n, so every variable right-rotate is wrong on Intel. The fix is worker-side only: on an Intel platform the worker rewrites that one helper line to the shift form before clBuildProgram, behind the vendor check; no consensus or pack change; the vectors self-test already refuses the wrong hash. When the Intel lane reports bench d green on the Arc (vectors pass, a rate row), its host.c patch goes into the shipped igneum-worker-opencl for 0.3.20 (Windows and Linux) with a unit test that the Intel path builds the shift form and the AMD path is unchanged; the Intel vendor badge and card row from intel-arc ride the same cut. Asked of the Intel lane: the patch as one commit, the test and its command, the badge and card-row hashes. The 0.3.20 worker build holds on that; nothing else in the cut waits on it.
The fleet (12:0x UK): the 0.3.20 cases (relay and poison against c18-1 on dc141409) start at about 11:20Z after a re-rent (relay on a 3090, poison on an A5000); the ten-member pool window keys on CASES END; prover roll 10 of 13 paired.