igneum/docs/plans/release-0.3.18.md
2026-10-07 05:45:24 +00:00

22 KiB

Igneum Miner 0.3.17: the 0.3.17 tree as cut on 7 October 2026

The hourly cut after 0.3.16, under the coordinator's rule: every new switch at never on the devnet so the digest does not move; cut what is green and list what waited. Worktree igneum-wt-ship0317 (release-0.3.17 from release-0.3.15 a9eb58f1), node release-0.3.17-node in vendor/igneum-node-0317 (from f1ea7a38).

Renumbered 7 October 2026, 03:3xZ (main): the tree this plan describes is 0.3.18. Its canary on 12153428 failed on two further faults (the idle-peer guard closes every peer mid headers-proof IBD; a window-below-epoch log flood at 1,900 lines a minute), and tonight's 0.3.17 became the node-only hotfix on the 0.3.16 app tree (f1ea7a38 plus 90aaf38e = b3c228fa; see docs/plans/release-0.3.17.md on release-0.3.17). The version scheme is three-part everywhere, so "0.3.16.1" was not available. Rows below keep "0.3.17" where they were written; read it as this tree. Branches: release-0.3.18 (app), release-0.3.18-node (fork, 12153428). Decimals is 0.3.19.

1. The node tree

Branch Tip In 0.3.17 Why
ca3-v4-0316 (node lane) 9a44fcb8 (df787727 the last functional commit: igneum_getNodeInfo; the miner stall guard e6e1fbe2 with exit 45, the idle-peer drop 96619b7d) yes the unwrap class, the seven-window rule, the finality cause fields, igneum_getRecentBlocks, difficulty rule v3, the fork gate, vote-or-burn, the signing bonus, the miner's stall exit (every switch absent from the live object = never)
ladder-node (ladder lane) 1591ee1d yes the latency ladder behind latency_ladder_activation_daa (never); the receive gate reads header_signals_active (the lane's merge note, applied as c6e12005's parent commit)
exec-sync-0313 (proving lane) 40fd8d8c yes (merged last, 02d15a87; params.rs both sides kept) consensus proof verification behind proving_consensus_verify_daa (never), the override's program ids as the statement's ids, sp1 as cfg(not(windows)) dependencies with the host-backed verify on Windows (the first merge of a344fea3 broke the Windows cross-build in sp1-jit; 40fd8d8c fixed it)
finality-pause-node-0317 (finality lane) fdc71bb5, rebased yes (cherry-picked as d8d1de1a onto the exec-free base) W7 the departure announcement behind finality_leave_activation_daa (never on the devnet; 0 and leave_delay 3600 in the testnet genesis per the project lead); protocol 17; the devnet digest unchanged by test
tail-emission-node-0317 (economy lane) 5647533f, rebased yes (cherry-picked as 996b592f, its params hunks without exec-sync's proving fields) its coinbase table replaces the subsidy table the silence rules build on; the rebase resolves it; no digest move while CURRENT
kaspad igneum-pow default feature 75467244 yes the node lane's N5 finding: a bare cargo build -p kaspad validated PoW with the kHeavyHash stub; every box reads igneum_getNodeInfo's powEngine "igneum-pow" in the table, "stub" is a FAIL
kaspa-testing-integration cc78a21f fixed here compiles again (the block store's evm argument, Header's vote_key_hash, the Params const as a let, the four finality ops in the RPC sanity match); the box suite is to compile every test crate from here

Final node tree 02d15a87 (75467244 + exec-sync-0313 40fd8d8c). Digests on the exec-free tree c6e12005 (Mac binary, direct run): the live sixteen-field object eada4bda (unchanged), the thirteen-field file b18ed271, no file c562d70e. A harness note: scratchpad digest.sh printed the no-file digest for a file that reads eada4bda while other nodes held its ports; it now says UNTRUSTED when the override lines are missing.

2. The app tree

Branch Tip In 0.3.17
ladder (repo side) 7003f9f yes (igneum-pow's chain_program_shadow, which the fork's kaspa-pow needs)
exec-app-0314 032f03c yes (clean; docs as the union)
relay 28c028b yes (ci.yml on the release side)
ember-tune-0317 (rebased on a9eb58f1) bb37c993 yes: the release lines kept (job_active, install_asked, the stale-exporter line), live.rs one merged module, publish-manifest.sh Ember's guard
miner-ui-4 (rebased on a9eb58f1) afc331bd yes: ui/ whole with Ember's finality words, extnode.rs on igneum_getNodeInfo, N4 stall exits, the port-collision check, POST /api/shot, an external node that goes away hands the ports to the app after 60 s; src/live.rs stays Ember's
proving-v2 d8055a7 NO (its exec content is in c00e608; its extra is the proof-system v2 slot, its own cut, per its owner)

App tests on the merged tree: 175 + 28 + 8; UI 41. A whole-file take of a lane's branch on an older base dropped the release tree's own fixes (seen and reverted): the lanes rebase onto the release tree and resolve there; the shipper merges, never substitutes files.

2a. Linux binaries by glibc (main's rule, 7 October 2026, 01:3x UK)

HiveOS images are Ubuntu 20.04 (glibc 2.31); the canary pods on 22.04 (2.35) refused the box's binaries. From now: the HiveOS package carries the 2.31 zig build (the build-server lane produces and container-tests it; the 0.3.17 HiveOS row stays at 0.3.16 in the index until that package lands, then joins with its own read-back); seeds and generic Linux binaries are the 2.35 build; the fleet's own boxes (Ubuntu 24.04) take the box's native build. The Mac and Windows rows publish on their gates.

2b. Checklist line for every cut (the node lane, 7 October 2026)

cargo check -p kaspa-testing-integration --tests runs in the box suite of every cut and in the fork's CI (the crate had not compiled since the finality fields landed; no integration test ran on any cut until this one). The 0.3.17 box chain carries it as its last step (rebuild-on-fix.sh step_box / the node chain's "integration check"). 0.3.18 node notes: relay pipelining bb397f99 and the sync-request fuzz gate 6cf33d36 on ca3-v4-0318; aa0182aa and 812c3ac2 from ca3-v4-0316.

2c. A kill matches a pid, never a name (main's row, 7 October 2026, 01:xx UK)

My pkill -f build-remote.sh at 00:17 UK (to stop my own chained box builds before a rerun) killed the decimals lane's box run on the same Mac: the pattern matched every build-remote.sh, not mine. Rule: every kill in the ship tooling matches a pid file or the exact command line (the run's log path, the worktree path), never a tool's name; the ship scripts run through tools/ci/pre-push.sh's kill-by-name check before the next cut. Tonight's scratch runbooks hold no pkill by tool name any more (the chains are started with their log path in the command and stopped by that path).

2d. The canary's FAIL (01:37Z): the IBD guard refuses signal-version relay blocks

c17-1 (02d15a87, the live sixteen-field object) could not join: "IBD with headers proof from was unsuccessful (peer relayed block ... header version mismatch: got 1026, expected 2 at DAA score 237583)", 483 lines in 15 minutes against all four peers, peers 0, blocks 0. Cause: protocol/flows/src/ibd/flow.rs:390 (upstream's Toccata guard, f94053a0) compares the syncer's relay-block version with the plain block version before the pruning proof; under the open window the sink carries legal 1026 signal blocks. The line is on f1ea7a38 too, so the LIVE devnet has refused every fresh join (the headers-proof IBD path) since publish 2 opened the window at 22:43Z; tonight's moves passed because every node had a datadir (the relay path) or began its IBD before the hub moved. Fix (node lane): the comparison gated like the pre-ghostdag rule (block_version_of under header_signals_active), known-failed test first; main decides between 0.3.17 carrying it and a 0.3.16.1 node hotfix. Checklist line from here: the canary's target is a FRESH node joining the LIVE object's chain, which carries signal headers once a window is open; the pre-cut harness (node-compat.mjs) gets a chain with signal-version headers under an open window.

2e. The rebuild on the fix (7 October 2026, 02:3x to 02:4x Z)

The node lane's IBD-guard fix (ca3-v4-0317-fix 90aaf38e) merged into release-0.3.17-node as 12153428. Every binary rebuilt on it:

piece commit string sha256 (first 8) bytes
Linux igneumd, box native (glibc 2.39, the fleet's canary sha) 12153428 5a4a0d68 57,347,232
Windows igneumd.exe (cross, box) 12153428 1e0b49ef 52,367,360
Windows igneum-miner.exe 12153428 7ca9ca01
Mac igneumd (Apple Silicon) 12153428 7416d4a5 47,837,248
Igneum-Miner-0.3.17.dmg (packaged object = the live sixteen-field object, prover pair aboard) 916248b8 44,244,489
HiveOS igneumd (zig, glibc 2.31) 12153428 378340e4 56,022,096
igneum-hive-0.3.17.tar.gz (2.31 node pair + the box's CUDA/OpenCL workers) 04fb7d45 27,193,392

Suite on 12153428: green (two load flakes pass alone), cargo check -p kaspa-testing-integration --tests rc 0. Digests read directly from the rebuilt node: sixteen-field eada4bda (igneum_getNodeInfo powEngine igneum-pow), thirteen-field b18ed271, no file c562d70e: unchanged from 0.3.16.

Windows inputs pushed 02:37Z (igneumd.exe 1e0b49ef, workers d7a413c7, 6f4bb57f, 0d68d06e, the Linux prover pair), pin b5a16f5b on release-0.3.17, windows.yml run 37562948420 dispatched 02:38Z.

HiveOS 2.31 smoke in an ubuntu:20.04 container on igneum-build-1 (ldd 2.31): igneumd, igneum-miner, igneum-worker-cuda and igneum-worker-opencl all load and answer. Found on the way: GNU tar materialises the Mac's extended headers as ._ files beside every file in the package (the live 0.3.16 tar has twelve of them; harmless, HiveOS ran it). Fixed in make-hive-package.sh (COPYFILE_DISABLE, no Mac metadata, c5187a70); the 0.3.17 tar extracts clean (10 files, 0 ._).

The scratch staging copy (r0317/dlsite-stage) had the superseded DMG d25ab372 and installer d1065ac0 removed; the manifest is re-written there once the Windows run's installer is fetched. The live downloads folder holds no 0.3.17 file until the deploy step, which waits on the fleet's second canary line.

2f. Ready at the line (7 October 2026, 02:5x Z)

  • Windows: run 37562948420 green (engine, window host, payload, installer, smoke run), Igneum-Miner-Setup-0.3.17.exe 93e29580, 62,814,273 bytes, fetched into the scratch copy. The scratch manifests (token folder and public) read: version 0.3.17, mac 916248b8, windows 93e29580, consensus.override = the live sixteen-field object (floor 831,600, window 86,400), HiveOS alias held at 0.3.16 until its own row.
  • The deploy runbook is scratchpad/r0317/deploy.sh (step_preflight, step_manifest = the live folder written and deployed in one step with the same inputs, step_update_now for d937c69d, ae432dc7 and 1ccfe586, step_hive, step_readback). It runs only on the fleet's second canary line.
  • Hands and seed: the build-server lane builds the seed's 2.35 pair and the hands' native pair on 12153428 and restarts them LAST on my line (observer, node 1, then the seed), each read back by commit string, digest and powEngine.
  • The Discord card is dry-run at tools/community/out/release_0.3.17.json (four reader-facing change lines, no em dash); it posts only once the HiveOS row is live, since the card carries every platform.
  • The 0.3.16.1 fallback (main's rule: if the retry fails off the IBD-guard fix) is prepared and not pushed: scratch worktree r0317/fallback-03161-node, branch release-0.3.16.1-node at b3c228fa = f1ea7a38 plus a hand-port of 90aaf38e (the helper gates on class_signal_active() alone, since f1ea7a38 has no ladder; the ladder assert dropped from the test). cargo check on consensus-core, consensus and p2p-flows clean, the unit test green. The node lane confirms it is the same adaptation it made for its 0318 tree (f2b25fdf).
  • The node lane's two-daemon window test (10226194 on the fix branch, separable, touches only testing/integration) is NOT in 0.3.17's pin; run on the box against 12153428 plus the commit (worktree vendor/igneum-node-twodaemon, so the path igneum-pow resolves): a_fresh_node_joins_a_chain_of_signalling_headers_and_no_window_refuses_them ... ok, 11.46 s, 03:01Z. It rides the next tree.
  • Pairing rule (the build-server lane, 02:5xZ): a fork build takes igneum-pow by path from the igneum worktree it sits in; a fork worktree under a master checkout fails in kaspa-pow (chain_program_shadow missing). 0.3.17's artefacts paired with release-0.3.17 at a73e400c to 6f8d7a7e, whose igneum-pow is unchanged since 6b30e855. The tool logs the pairing from the next master.

3. Owed

  • exec-sync-0313's consensus verification behind a feature off for Windows (the proving lane), then its two commits.
  • finality-pause-node rebased onto the 0.3.17 node (the finality lane).
  • decimals (B10 gate), vote-weigh and tail-emission docs: the next cut.
  • The fork gate's fast-time line is GREEN on aa0182aa (gate on: a 27.5 percent key's private chain 183 DAA deep refused, 268 refusal lines, the honest node kept its chain; gate off, the known-failed case reorgs), one commit past the 9a44fcb8 this tree carries; the switch is at never on every network, so 9a44fcb8 stands for 0.3.17 (the chains were building) and aa0182aa rides 0.3.18. vote-or-burn and the signing bonus: the replay gate FAILED on 9a44fcb8 (this tree) and PASSES on 812c3ac2 (the silence read as a function of the block's own past), which rides 0.3.18 with aa0182aa; both switches are at never here, so a devnet node behaves the same; the signing bonus is settable at the testnet genesis on the project lead's word, the devnet not a candidate until the longer-span replay passes.
  • The canary in the full form before publish (a new node mining beside an old one with a poisoned peer, ten minutes, a mid-window re-sync), the staged-in-scratch rule, every new switch at never.

4. For the 0.3.18 tree (not in 0.3.17)

  • miner-ui-4 past afc331bd: 74c12665 (the merge check: a node whose blocks never merge reads "behind" and the miner is held; src/merge.rs, observer poll, one UI line; 172 box + 42 UI tests green). The UI lane's word, 03:2xZ.
  • ca3-v4-0317-fix 10226194: the two-daemon window test (green on 12153428 plus the commit, 03:01Z).
  • The node lane's ca3-v4-0318 tree (98c78d9a, f2b25fdf).
  • The build-server lane's pairing log line in build-remote.sh (which igneum-pow a fork build paired with).

5. The 0.3.18 cut after the hotfix (main's order, 7 October 2026, 03:5xZ)

Once "0.3.17 live" (the hotfix) is read back: rebuild release-0.3.18-node on 12153428 plus the node lane's 4f4bb9c9 (ca3-v4-0317-fix: the idle-peer drop counts headers, proof, trusted data and UTXO chunks as deliveries and never fires during a sync; the class-signal missing-history warning once per epoch per process; kaspa-p2p-lib 20, kaspa-p2p-flows 35, class_signal 1 green on the box; the same patch onto ca3-v4-0318 by 05:15 UK), every platform, digests unchanged, suites, the integration check, the window test again on the merged tip; merge the UI lane's list (afc331bd in, then 74c12665, 89501ce8, b7d33e8e, the doc-only d29a8663), ember-tune-0317 c9b31edc; re-stage 0.3.18 in scratch; the fleet's full canary on its pods with the headers-proof fresh join decisive; publish 0.3.18 on its line, whenever that falls, morning included. Ember's PC 1 window ("PC 1 on 0.3.18") and the UI lane's Mac check follow its update-nows.

Tips (node lane, 03:5xZ): release tree 4f4bb9c9 on ca3-v4-0317-fix (cherry-pick alone onto 12153428); feature tree a3fe9f67 on ca3-v4-0318 (on e1197983, a lockfile-only commit). A follow-up on both is due (the ping flow's syncing flag IBD-only; "sink at genesis" broke the peer-drop gate's --case still and is redundant for the canary): take it WITH 4f4bb9c9. The hotfix canary (0.3.17, node d712b498) started 03:58Z on c17-1; its clock in UTC: headers through about 04:17, synced 04:40, reads 04:50, cases 05:25.

Final node inputs (main, 04:0xZ): the 0.3.18 node tree is ca3-v4-0318 at 6e4ace3f (a3fe9f67 the fix, b21659e4 the IBD-only follow-up, 6e4ace3f igneum_getNodeInfo's blockrate object for the merge check; exec-only, digest untouched; all suites green). The release-branch form (4f4bb9c9 + ddda7d52 on 12153428) is the equivalent and not used. After "0.3.17 live": release-0.3.18-node = 6e4ace3f, every platform rebuilt, then the canary.

Dated fault for this tree (the node lane, 05:05Z, found by the Mac's headers-proof join gate, not by a canary): when the devnet's pruning point first leaves genesis (chain DAA 185,799, about eight hours from 05:00Z at 1 bps), every fresh join by headers proof fails for the next 2,644 DAA (about 45 minutes) with "IBD with headers proof ... unsuccessful (DAA window data has only N entries)": upstream's sampled-window walk (661 samples at rate 4) runs out at a gap in the trusted blocks instead of reaching genesis. Nodes already on the chain are untouched. Fix: joiner-side on the trusted path only, digest-neutral (the walk accepts a window that runs out at origin when the block is younger than the span), coming on ca3-v4-0318 (hash to follow); 0.3.18 carries it. Any fresh-join canary timed inside that 45-minute window FAILS from this, not from the idle-peer fix: time 0.3.18's canary outside it (before about 12:50Z or after about 13:40Z, to be firmed from the live DAA).

Main's rule for the 0.3.18 cut (05:1xZ): it carries the node lane's joiner-side fix for the pruning-point fault (on the 0.3.18 tree the N6 guard would otherwise ban the syncer for ten minutes per retry), and 0.3.18 is live on every node before 13:30 UK (12:30Z), ahead of the devnet's first pruning (DAA 185,799, about 14:00 UK). If the full canary cannot close by then, publish 1 goes on the clean form plus the node lane's harness for that case, and the cases follow as confirmation, as with the hotfix. Every new chain, the testnet genesis included, meets the same fault once at its first pruning: the testnet go checklist (docs/plans/testnet-go.md) gets the line.

Correction (the node lane, 05:1xZ): the devnet does NOT meet the young-window fault, and the 185,799 clock was wrong (the canary's 155,700 headers were taken for the chain's DAA; live DAA was 250,809 at 05:00Z, the hub's 39th pruning move was at 03:46Z and the 03:50Z fresh join synced clean). Pruning samples sit at multiples of the finality depth (pruning.rs is_pruning_sample); the devnet's finality depth is 43,200 blocks, so its first non-genesis pruning point was at blue score 43,200, sixteen window spans past genesis, and the walk from any devnet pruning point fills its window. The fault needs a pruning point within 2,644 DAA of genesis, which only a chain with finality depth under 2,644 can have: the fast-time profile (120) where the gate found it. The testnet at the compiled numbers never meets it either. So: no fresh-join window to avoid, the 45-minute rule is dropped, main's "before 13:30 UK" rule for 0.3.18 no longer rests on this fault (main to confirm), and the testnet go line is softened to a note. The fix still rides 0.3.18 (joiner-side, digest-neutral, relaxes the rule only when the walk ended within one sample of genesis, impossible on the devnet; unit test green on the box; hash to follow).

Main, 05:2xZ: the 13:30 UK deadline is withdrawn. 0.3.18 ships on its normal gates with the full canary, no shortcut; the young-window fix stays in the tree as a harness-profile correctness fix. ca3-v4-0318 tip 06:04 UK: e3798a17 (69647bd6 per-path finality timers plus the join bench, test-only plus accounting; e3798a17 the young-window fix, unit test green). The fresh-join cost fix is next on the branch with its own hash and before/after numbers; its "before" figure comes from the 0.3.18 canary's IBD-end line (the new "finality time: bodies ..., virtual ..., weight tables ..., signatures ..., persists ..." line), which I relay verbatim (route (b); a read-only joiner from the box, route (a), is main's word).

ca3-v4-0318 tip 06:06 UK: aa49613f (on e3798a17; carries the IBD-end "finality time" line; p2p-flows and kaspad check clean on the box). The cut takes aa49613f or the later tip the node lane names; the canary's fresh-join IBD-end line goes to the node lane verbatim.

6. The 0.3.18 inputs as of 05:2xZ (main's list)

  • Node: the ca3-v4-0318 tip the node lane names at the cut; now 8220c944 (06:10 UK; on aa49613f): a block's votes BLS-checked across cores before the finality lock, exact by the two-path test. Box, 500-checkpoint chain of 22,221 blocks: join 218 s to 104 s, finality 130 s to 30 s, signatures 97 s to 9 s. Suites at 8220c944 on igneum-build-1: kaspa-consensus lib 110 passed, 0 failed, 3 ignored (the two new finality tests inside); kaspa-p2p-flows 37 passed; kaspad check clean at aa49613f with nothing in kaspad changed since. The real split of the canary's 8 s per checkpoint comes from the IBD-end timers line of the 0.3.18 canary's fresh join, relayed verbatim to the node lane.
  • App: miner-ui-4 afc331bd (in), 74c12665, 89501ce8, b7d33e8e, bac43ac4 (main's row from PC 1's 04:51Z relaunch: the card watchdog judges a miner only while the node is synced; a silence fault from the sync is released when the node syncs and the card starts again with an Activity line; a worker silent from its start is faulted at 60 s; one Activity line per card at its first start; known-failed test from the PC 1 row in release-0.3.17.md; 173 box + 42 UI tests green) plus the doc commits (d29a8663, 0e2e8a31); ember-tune bb37c993 (c9b31edc is job-script only); exec-app 032f03c; relay 28c028b.
  • Gates: the same as every cut; the full canary with the fresh join decisive; publish on its line.

7. The node tree is a merge (05:5xZ)

ca3-v4-0318 (8220c944) does not contain release-0.3.18-node 12153428: their merge-base is 9a44fcb8 (ca3-v4-0316), so the branch lacks the ladder 1591ee1d, exec-sync 40fd8d8c, finality d8d1de1a, tail-emission 996b592f, the kaspad igneum-pow default 75467244, cc78a21f and 90aaf38e (it carries f2b25fdf, its own adaptation of the canary fix). The 0.3.18 node is the merge of 8220c944 (or later) into 12153428. A no-commit test merge conflicts in six files (consensus/core/src/igneum.rs, pre_ghostdag_validation.rs, ibd/flow.rs, v10/blockrelay/flow.rs, the two integration test files): the node lane resolves it, pushes release-0.3.18-node to the mirror, runs the suites and the integration check, and names the tip. The branch stays at 12153428 until then.

PC 1's 0.3.17 relaunch (section 7 of release-0.3.17.md) adds a node row for this tree: the template RPC blocks under the finality catch-up after a restart on a synced node (every getBlockTemplate timed out at 5 s for 90 s); the app row (bac43ac4) needs its rule changed to "template timeouts while the node reports synced are not a card fault".