489 lines
81 KiB
Markdown
489 lines
81 KiB
Markdown
# Igneum Miner 0.3.10: the certificate-driven reorg, transaction gossip, the pack loader fix, the six-section miner and the PC-built Windows node, 5 October 2026
|
|
|
|
Shipped: manifest 0.3.10 published 21:32:40Z, every reachable machine on 0.3.10 by 21:49:41Z, the hand nodes and the seed on 21d4c73c by
|
|
21:50:15Z, one digest (1f4b4425...) on every node that restarted; PC 37ba0461 (the owner's laptop) mid-install and Sam's Mac quit at the
|
|
time of writing. The cut ran from 16:39Z to 21:5xZ, two hours of it GitHub's hosted-runner outage.
|
|
|
|
Release engineer, from 16:39 UTC. Worktree `/Users/joshm/Projects/igneum-wt-ship0310`, branch `release-0.3.10` from master
|
|
after the 0.3.9 merge. Fork worktree `vendor/igneum-node-0310`, branch `release-0.3.10` from a24ab01a (the 0.3.9 fork tip).
|
|
Every Mac build through `/Users/joshm/Projects/igneum/tools/lock/with-lock.sh build` at `nice -n 19` with `-j 4`; every PC
|
|
job through the main checkout's `tools/build-job.mjs` and `packaging/ota/publish-jobs.sh` (the signed envelope). Times are
|
|
UTC. `release-0.3.9.md` and `release-0.3.8.md` are the template; `release-0.3.6.md` section 10 for the Windows acceptance.
|
|
The rollout waited for 0.3.9's (shipped about 17:00Z, fee switch published about 17:50Z): nothing of 0.3.10 moved on the
|
|
network before every node's digest was ab8847da...
|
|
|
|
## 1. What 0.3.10 carries
|
|
|
|
| Change | Where | Fork commit |
|
|
|---|---|---|
|
|
| A. The windows-gnu node links the C++ runtime statically (`database/build.rs`, `rocks-probe`), so the PC-built Windows node runs (release-0.3.6 plan section 10) | fork `housekeeping` 4fb32865 | merged 2f88a82f |
|
|
| B. `TestConsensus` sets the unbounded PoW cache build queue, so the five node suites pass in parallel on PC 2 | fork `housekeeping` 7003055b | merged 2f88a82f |
|
|
| C. The certificate-driven reorg (ledger C4, measured in FUD round 6, 75c6658: a certificate over an off-chain block stayed pending and the heavier side locked alone): a verified certificate over a block off the node's selected chain now locks it there and moves the virtual to the heaviest tip through it (spec 3.5 F1 and F2); the block relay takes a lighter block a pending certificate names. A consensus-behaviour change with NO digest change: it converges when every node is new (the c4 agent's rollout note, section 10) | fork `c4-fix` e18f1e0e (its message records PC 2 job build-20261005-180827: kaspa-consensus 97 passed, kaspa-consensus-core 101 passed, three new tests), main `c4-fix` 68f14b9 (spec 3.5, 3.2, 3.10, 3.11.7, ledger, bench-log, c4.mjs v2 mode; the signer-pipe-check CI script and per-job build-inputs zip names) | fork: merged 21d4c73c; main: 0cbaf3a and the tip merge |
|
|
|
|
Changelog line for C (the coordinator's words): "A node that comes back from a partition now follows the certified checkpoints, even when its own chain is heavier."
|
|
| D. EVM transaction relay between nodes: three p2p messages after Kaspa's transaction inv/request/answer, PROTOCOL_VERSION 13 to 14 (13 peers still connect: a v14 node never sends the new messages to an older peer), the mempool caps and faults, the block-added hold instead of the 4-s hand-out cooldown; the digest is unchanged, so no activation height and no fresh chain | fork `tx-gossip` e242acd0 (its message records PC 2: igneum-exec 15 of 15, kaspa-p2p-flows 33 of 33), main `tx-gossip` 0166e94 (the relay-net harness `tools/txgen/relay-net.mjs`, the design note); acceptance here: `node tools/txgen/relay-net.mjs --rate 2 --duration 120 --wallets 16 --fund 2` on the 0.3.10 Mac node, expected 240 of 240 included | fork: merged 21babaa7; main: (pending the worktree) |
|
|
|
|
Changelog line for D (the coordinator's words): "Transactions now travel between nodes; a miner on an older node still carries only what was sent to it directly."
|
|
| E. GPU hot-plug (coordinator, 17:5xZ): cards re-enumerated every 60 s and on `WM_DEVICECHANGE`, new cards mine by default, Code 43 cards marked unusable, vanished cards dropped without shifting slots, the Ryzen gfx1036 iGPU classified integrated and off by default, a `cards:` line the relay parses (machines API `gpus`/`hotplug`). 14 files: `app/igneum-app/src/{detect,engine,hotplug,main,state}.rs`, `ui/{app.css,app.js,notices.test.mjs}`, `app/windows/host.cpp`, `relay/{api/console.mjs,lib/parse.mjs,test/parse.test.mjs,ui.html}`, `tools/console.mjs`; no proving, packaging or workflow file. Its Windows-only paths (`detect::adapters`, `host.cpp` WM_DEVICECHANGE) were never compiled on the Mac: PC 1's Windows app build and the GitHub runner's host build are their first compile and are read for errors | main `gpu-hotplug` 4517004 (`igneum-wt-hotplug`, on master 22ffc02); its tests on the branch: igneum-app 91 pass, notices, update-card and parse node tests pass | merged after C and D |
|
|
| F. Elevated jobs that never started (coordinator, 18:0xZ): an elevated job whose UAC prompt is refused, cancelled or times out fails with exit 251 and the summary "did not start: the administrator prompt was refused, cancelled or timed out"; before, `Start-Process` threw, `$p` stayed null and `exit $p.ExitCode` exited 0, so the PC 1 driver job at 17:58Z read as done. One file, `app/igneum-app/src/jobrun.rs`, plus its unit test | main `elevated-exit` 9816d58 (`igneum-wt-elevated`) | merged after E |
|
|
| G. The miner UI redesign (the project lead, through the coordinator at 19:0xZ: into 0.3.10, not 0.3.11): six sections (Mine, Prove, Rewards, Node, Updates, Settings), one switch per card, one line of help per setting, the node's next switch shown with its height; three engine additions with tests (`state.rs` `node.consensus_digest` and `node.consensus_switches`, `engine.rs`, `prover.rs` program ids via `--mode id`), a Mac window minimum of 900 x 600, one `WM_GETMINMAXINFO` case in `app/windows/host.cpp`, `view.test.mjs` in CI | main `miner-ui-2` (`igneum-wt-miner-ui`, 83a293f and 364feef, then the commit its agent reports after resolving it against gpu-hotplug's card-row states) | merged LAST, after F, at the agent's final commit; the node stays 21d4c73c, so the app, the DMG, CI and the ship files are rebuilt (section 7a) |
|
|
| H. The pack loader fix (the outage of 18:31Z, root cause found by the coordinator's agent): the generator retries a rejected candidate program with `seed || k` (epoch 34's attempt 0 was rejected on the saturation rule, attempt 1 accepted) and the workers' pack loader (`proto-cuda/nvrtc/packfile.h`, shared by the OpenCL worker) derived the expected seed words from attempt 0, so every worker started after the boundary refused a correct pack. Changes `packfile.h`, `worker.cpp`, `proto-opencl/host.c`, the relay's parse and tests, an emu test. It changes the Windows WORKER binaries: since 0.3.6 the payload has carried only the worker sources (`host.cu`, `host.c`, `build.bat`) and the PCs built the workers themselves, so the fix reaches the PCs only as exes: both workers cross-built on this Mac (`proto-cuda/nvrtc/build-windows.sh`, mingw, the staged redist) and carried by `push-inputs.sh` into the installer; a hot-fix copy is already on both PCs under `packs\\workers-fix`. The CI builds no worker (`windows.yml` copies the two worker exes and the `nvrtc*.dll` files from the signed inputs only), so the rebuilt pair reaches the installer through `push-inputs.sh` and nowhere else; the engine takes a bundled worker before its own PC-built one (`detect.rs` `Bins`, `engine.rs` `build_worker_from_source` only "when no prebuilt worker ships"), so the fix applies at the first start after the update | main `pack-loop` af983a7 (`igneum-wt-pack-loop`; the tip at merge time) | merged after G |
|
|
| I. The export lock: `engine.rs` serialises `igneum-miner export-pack` behind `EXPORT_LOCK` around both call sites (two per-card export threads interleaved into one folder across an epoch change on PC 1, a second way to a wrong pack) | main `opencl-rdna4` a08c371: the lock hunk is the part that must ship; the rest (host.c's duplicate-platform fold, `--readback select`, `--memprobe`, `detect.rs parse_opencl_list`) only if it merges cleanly with hotplug and the OpenCL worker cross-builds without error, else the hunk alone (coordinator, 19:2xZ) | (decided at the merge, section 2) |
|
|
| NOT in 0.3.10 unless the coordinator says so: `job-console` d33a266 (one hidden-console builder for every elevated launch, the spawn check in CI, PC 1 console watchers; its agent's message at 18:5xZ) and `opencl-rdna4` a08c371 (its agent's message at 19:1xZ: the OpenCL worker folds the duplicate AMD platform out of `--list`, a GPU select pass, `--memprobe`, `detect.rs` parsing with tests, and `engine.rs` serialises `export-pack` behind `EXPORT_LOCK` because PC 1's `packs\devnet` has held a mismatched `program.h` and `seeds.txt` since 19:07Z: the pack-export class the rollout is held for; passed to the coordinator as a candidate) | the coordinator's rule: a late branch goes in only if it lands before the final tree; the final tree b958493 was pushed at 18:36Z with CI green at 18:42Z. job-console also moves elevated-exit's exit-251 launcher text to `platform.rs`, so its `include_str!` test is repointed at its own merge to master after 0.3.10 | next cut |
|
|
| The app engine otherwise | unchanged except the version (0.3.10 in the six files); the signed jobs envelope client is master's (housekeeping C, 318a2de) | |
|
|
|
|
Changelog line for G (the coordinator's words): "The miner is now six sections: Mine, Prove, Rewards, Node, Updates, Settings. One switch per card, every setting with one line of help, the node's next switch shown with its height."
|
|
|
|
Changelog line for F (the coordinator's words): "A remote job that needs administrator rights now reports when the prompt was not accepted instead of claiming success."
|
|
|
|
Changelog line for E (the coordinator's words): "New cards are picked up while the miner runs; a card that goes away is released; integrated GPUs stay off unless you switch them on.
|
|
| The pinned guest | UNCHANGED unless C or D touches `proving/igneum-prove/core` or `elf/` (checked at the merge: section 2) | |
|
|
|
|
## 2. The branch
|
|
|
|
| Commit | What |
|
|
|---|---|
|
|
| 9268b64 | `origin/master` at the worktree's creation (18:00Z: the 0.3.9 merge, fast-forwarded) |
|
|
| 2e943a8 | Merge `tx-gossip` (0166e94): the relay design note, rows 463 and 9, the 3-node relay harness and its evidence; no conflict |
|
|
| 824059b | `Igneum Miner 0.3.10: the six version files` (`node tools/ship-app.mjs --check`: 0.3.10 in all 6) |
|
|
| 9997ef1 | `packaging/mac/packaged-config.sh`: the four-field override object in the packaged line, the self-test fix (section 3) |
|
|
| 0cbaf3a | Merge `c4-fix` (3072919, 18:20Z). One conflict, `docs/bench-log.md`: both entries kept (the fee-table entry, then the C4 entry). Brings `tools/ci/signer-pipe-check.sh` (`igneum-ota-sign ... \| head -1` under pipefail panicked the signer with SIGPIPE on a loaded Mac and killed a publish; now `sed -n 1p` in `publish-jobs.sh`, `publish-manifest.sh`, `push-inputs.sh`) and one `build-inputs-<time>-<pid>.zip` per build job (`build-job.mjs`, `push-build-inputs.sh --name`: with the shared name a job pinned another agent's sources three times tonight). From here every PC job and publish runs from THIS worktree's tools: they carry master's signed envelope (housekeeping C) and these two fixes, which the main checkout lacks until this branch merges |
|
|
| 4d10c20 | Merge `gpu-hotplug` at its tip at merge time, 4d122e1 (the agent's second commit: one row per physical GPU across OpenCL platforms, Windows names on the rows, index-free card keys, the OpenCL worker's `--list` prints the PCI address in `proto-opencl/host.c`); no conflict, 15 files |
|
|
| 5520fb1 | Merge `elevated-exit` (9816d58); no conflict |
|
|
| d1f4923 | Merge `origin/master` a93199a (docs and site only) |
|
|
| e0a5fd1 | `infra/cross/build-linux.sh`: the three paths made absolute before the cd (section 3) |
|
|
| 065c67c | `packaging/windows/node-source.pin` = 21d4c73c (push-inputs, section 5c) |
|
|
| b958493, ccd2b98 | the site as the pre-push hook builds it; the plan so far. The FIRST final tree: pushed 18:36:44Z, CI green 18:42Z, installer and DMG built (section 7). Then the project lead's scope change reopened it for G |
|
|
| 7f9618a | Merge `miner-ui-2` at its agent's final commit a3d9f2d (19:2xZ; it already carries gpu-hotplug 4d122e1 merged and resolved onto the new UI). One conflict, `packaging/windows/push-build-inputs.sh`: the usage comment only (c4's `--name` text against miner-ui-2's `--no-node` text; both options were already in the code), both kept |
|
|
| 5abc709 | Merge `pack-loop` (af983a7, its tip at merge time). One conflict, `relay/test/parse.test.mjs`: the import line (the union: `parseAppTail`, `parseCardsLine` from hotplug, `PACK_MISMATCH` from pack-loop) and two tests that both landed at the file's end (hotplug's cards line, pack-loop's pack mismatch), both kept; `node --test relay/test/parse.test.mjs`: 7 pass |
|
|
| 854f9a8 | `engine: one pack export at a time`: the `EXPORT_LOCK` hunk of opencl-rdna4 a08c371 (`git diff origin/master...a08c371 -- app/igneum-app/src/engine.rs`, applied clean: 9 lines, a static mutex and a guard at both export-pack call sites). The rest of that branch conflicts with hotplug in `detect.rs` and `proto-opencl/host.c` (a dry-run merge at 19:1xZ), so by the coordinator's rule it waits for the next cut |
|
|
| 854f9a8 | the fast checks on the SECOND final tree, 19:24Z: identity 0 hits over 213 files, copied-sources, pinned-guests, signer-pipe-check ok, no-conflict-markers ok, workflow shell 0 findings, `--check` 0.3.10 in all 6, relay tests 17 pass, UI tests 23 pass (notices 12, update-card 6, view 5; `view.test.mjs` in ci.yml line 85) |
|
|
| 5520fb1 | the fast checks on the first final tree: identity 0 hits over 212 files, copied-sources, pinned-guests, signer-pipe-check ok, no-conflict-markers ok, workflow shell 0 findings, `--check` 0.3.10 in all 6, relay tests 16 pass (hotplug's parse test added), UI tests 12 pass; 18:21Z |
|
|
|
|
A `vendor` symlink to the main checkout's `vendor/` (untracked) makes the relative node paths resolve, as in the earlier cuts.
|
|
The app and prover target dirs were cloned by APFS from the 0.3.9 worktree (18:01Z).
|
|
|
|
The fork branch `release-0.3.10` in `vendor/igneum-node-0310`, from a24ab01a: 2f88a82f (housekeeping 4fb32865), 21babaa7 (tx-gossip e242acd0), 21d4c73c (c4-fix e18f1e0e: `finality.rs`, `blockrelay/flow.rs` and five small files, 601 insertions), all three without conflict. Neither c4-fix nor tx-gossip touches a params file, `proving/igneum-prove/core` or `elf/`: the pinned guest and the prover rollout are untouched by 0.3.10.
|
|
|
|
## 3. Tests and checks, with the command
|
|
|
|
| Tip | Command | Result |
|
|
|---|---|---|
|
|
| fork 2f88a82f (a24ab01a + housekeeping) | PC 2 job `build-20261005-164604` (`node tools/build-job.mjs run --node vendor/igneum-node-0310 --target 1ccfe586 --targets linux --node-tests "kaspa-consensus kaspa-consensus-core igneum-exec kaspa-pow igneum-miner" --no-app`) | started 16:46Z, done 16:48:43Z (129 s): Linux node built, `RESULT test node [the five packages] exit 0 39 s`: the parallel five-package run is green on the housekeeping tree (housekeeping B); the short report carries the per-package exit, not the per-test names |
|
|
| fork 2f88a82f | PC 1 job `build-20261005-164512` (`--targets linux,windows --no-tests --no-app --no-place`) | started 16:45Z, done 16:50:26Z (292 s), every stage ok, 7 files, all sha256 and PE checks ok: igneumd.exe 37d0b1045043bc86... (50,889,728), igneum-miner.exe bc8cb3a12111e7bb... (10,725,888), Linux igneumd d0331c109babe77e... (48,733,736, glibc 2.39: HiveOS, not the seed). `strings igneumd.exe`: msvcrt.dll is the only C runtime named; no libstdc++-6, libgcc_s_seh-1 or libwinpthread-1 (housekeeping A) |
|
|
| fork 2f88a82f, PC 1 exe | run job `run-0310-accept-hk` (the housekeeping agent's acceptance script: rocks-probe, igneumd 60 s on a scratch appdir with no mingw DLL beside it, `igneum-miner.exe key-hash probe`, Application event 1000 check), published 16:51:01Z (the apps woken) | ran 16:57:15 to 16:58:21Z (66 s), exit 0: `rocks-probe` exit 0 (21 files in its db), `RESULT run node60: still running after 60 s (alive for the whole wait); killing it` with 14 files written under the scratch appdir, `igneum-miner.exe key-hash probe` exit 0, `RESULT event 1000: none`; the exe's imports on the PC (objdump): KERNEL32, advapi32, bcrypt, iphlpapi, msvcrt, ntdll, ole32, ws2_32 and the api-ms-win-core set, no mingw DLL. The one warning, `cannot bind the eth_ JSON-RPC server on 127.0.0.1:26790`, is PC 1's live node holding the port, as in the housekeeping run. So the PC-built Windows node is the 0.3.10 Windows node (no Mac cross-build needed) |
|
|
| fork 2f88a82f, Mac arm64 | `CARGO_TARGET_DIR=vendor/igneum-node/target-0310 cargo build --release -j 4 -p kaspad -p igneum-miner --features kaspad/igneum-pow` from `vendor/igneum-node-0310` under the lock (target dir cloned by APFS from `target-036`; a new worktree path rebuilds every crate) | 16:44:52 to 16:57:28Z (12 min 36 s): igneumd 2a7df6245ffe047a... (40,968,368, `2f88a82f` in its strings), igneum-miner 322472ae95a316d1... (8,514,928) |
|
|
| fork 2f88a82f, Mac arm64 | the digest check: that igneumd on a scratch node (ports 60975/60976, 22 s, 16:57:58Z) with `{"difficulty_v2_activation_daa":33000,"proving_v0_activation_daa":84100,"fees_v1_activation_daa":210000}` | `Calibrated v1 fees from the override file: ... from DAA score 210000`, digest ab8847da538dead1dc10e046dfaadab3c1c35928e3748810c4e050d4a886087a, `igneumd/2.1.0-2f88a82f`: housekeeping moves no parameter |
|
|
|
|
| fork 2f88a82f, Linux x86-64 for the seed | `NODE_SRC=vendor/igneum-node-0310 TARGET_DIR=vendor/igneum-node/target-0310-linux infra/cross/build-linux.sh` (zig, glibc 2.36 target) under the lock | queued 16:57Z behind two other agents' builds, built 17:23 to 17:35:49Z (754 s, 564 crates), and its output was WRONG: igneumd 7e26e374... BYTE-IDENTICAL to the 0.3.9 build of a24ab01a, embedding `a24ab01a`. Cause (found at the second run, 18:25Z): `build-linux.sh` does `cd "$NODE_SRC"` and runs cargo with `CARGO_TARGET_DIR="$TARGET_DIR"`, so a RELATIVE target dir resolved inside the node worktree (`vendor/igneum-node-0310/vendor/igneum-node/target-0310-linux`, the untracked `vendor/` that appeared there), while the copy step read the repository-relative clone that still held the a24ab01a binary. Every crate had compiled, into the wrong place. Fixed on this branch (e0a5fd1: the three paths made absolute before the cd) and the build rerun with an absolute target (section 5). My first reading of it, a stale `kaspa-build-info` output in the cloned target, was wrong and is withdrawn |
|
|
| fork 21babaa7 (2f88a82f + tx-gossip e242acd0), Mac arm64 | the same cargo build, incremental on `target-0310` | 17:49:52 to 17:52:00Z (2 min 08 s): igneumd 5d7f1b576029b462... (41,118,944, `21babaa7` in its strings) |
|
|
| fork 21babaa7, Mac arm64 | the digest check as above (ports 60975/60976, 22 s, 17:52:01Z) | `Calibrated v1 fees ... from DAA score 210000`, digest ab8847da538dead1dc10e046dfaadab3c1c35928e3748810c4e050d4a886087a, `igneumd/2.1.0-21babaa7`: tx-gossip moves no parameter (its commit message says the protocol version is not in the digest; measured here) |
|
|
| fork 21babaa7 | PC 2 job `build-20261005-175022` (`--node-tests "kaspa-consensus kaspa-consensus-core igneum-exec kaspa-pow igneum-miner kaspa-p2p-flows" --no-app`) | started 17:50Z, done 17:53:10Z (145 s): `RESULT test node [the six packages] exit 0 53 s` (kaspa-p2p-flows added: tx-gossip's relay tests live there and in igneum-exec) |
|
|
| 2e943a8 + bump | `tools/ci/identity-check.sh` (0 hits over 212 files), `copied-sources-check.sh`, `pinned-guests-check.sh` (elf/ matches its manifest), `node tools/ci/check-workflow-shell.mjs` (12 run blocks, 25 .ps1, 0 findings), the relay tests (15 pass), the UI tests (10 pass) | all green, 18:00Z |
|
|
| fork 21babaa7, Mac arm64 | the digest check with the FOUR-field object `{"difficulty_v2_activation_daa":33000,"proving_v0_activation_daa":84100,"fees_v1_activation_daa":210000,"finality_v3_activation_daa":135200}` (N3 = 135,200 from the live 0.3.9 manifest republished 17:59:14Z, deadline note "finality v3"; the project lead: "deploy N3 now", the 0.3.9 agent publishing it) | 18:01:59Z, 22 s: `Finality rule v3 from the override file: active from checkpoint DAA score 135200`, `Calibrated v1 fees ... 210000`, digest 1f4b44255fcd2ea8f75664ed47200f409186ddd2292960c9e2cf95bbbdc11505, `igneumd/2.1.0-21babaa7`. Equal to the live hand nodes' lines at 18:02Z (`observer-v4.out`, `node1.out`: digest 1f4b4425..., `igneumd/2.1.0-a24ab01a`, the four-field `/tmp/igneum-devnet/override-v3.json`): the 0.3.9 agent's N3 rollout had reached them. This is the digest every node must show; the final 0.3.10 node is read again against it |
|
|
| 9997ef1 | `packaging/mac/packaged-config.sh`: `NODE_OVERRIDE_PARAMS` = that four-field object (it was empty on master; the manifest's `consensus.override` is what the apps write, the packaged line covers a first start before any manifest). Its self-test's "json has no override line" check read the file's shipped default and failed on a non-empty line (master's empty line passed it), so that check now passes `NODE_OVERRIDE_PARAMS=''` itself, as the "override params land" check already passes its own value; `bash packaging/mac/packaged-config.sh --test`: all checks passed |
|
|
| fork 21d4c73c (+ c4-fix), app 5520fb1 | PC 2 job `build-20261005-182244` (`--node-tests "kaspa-consensus kaspa-consensus-core igneum-exec kaspa-pow igneum-miner kaspa-p2p-flows" --app-tests igneum-app`) | 18:23:15 to 18:25:41Z: the node stage FAILED, exit 101 after 44 s: `kaspa-consensus` 96 passed, 1 failed, 3 ignored: `processes::finality::tests::ban_is_decided_by_the_carrying_block_so_nodes_agree_on_every_voter_list` panicked at `finality.rs:1883:82` (the `mine_on_all` helper's `validate_and_insert_block(...).unwrap()`) with `UnexpectedDifficulty(<bits>, 487112096, 487129281)`: a block built on one `TestConsensus` was refused by another for a difficulty a few hundred bits off. cargo stopped at the first failing package, so the other five were not run in this job. The app tests passed: `igneum-app` exit 0 in 6 s (hotplug's and elevated-exit's tests included; 96 + the signer and wrapper suites) |
|
|
| fork 21d4c73c | PC 2 job `build-20261005-182804` (`--node-tests kaspa-consensus`, the suite ALONE) | 18:28:3x to 18:30:15Z: `kaspa-consensus` 97 passed, 0 failed, 3 ignored in 1.99 s, `ban_is_decided_by_the_carrying_block_so_nodes_agree_on_every_voter_list ... ok`. So the test passes alone (as in the c4 agent's own run, build-20261005-180827, 97 passed) and failed once under the six-package parallel run: the 0.3.6 isolation class (release-0.3.6 plan 8f), now with a difficulty expectation that depends on wall-clock timing rather than the PoW cache queue. Not a consensus regression by the same reasoning as then; recorded in section 11 for the c4 agent (the test or the helper should pin its clock) |
|
|
| fork 21d4c73c | PC 2 job `build-20261005-183131` (`--node-tests "kaspa-consensus-core igneum-exec kaspa-pow igneum-miner kaspa-p2p-flows"`) | 18:31:31 to 18:33:59Z: exit 0 in 37 s: `kaspa-consensus-core` 101 passed (2 ignored), `igneum-exec` 15 passed, `igneum-miner` 15 passed, `kaspa-p2p-flows` 33 passed (the target compiles and passes on the merged tree: the nine `epoch_seed_headers` errors of release-0.3.6 are gone with tx-gossip's `ibd/proof.rs` update, as the c4 agent said), `kaspa-pow` 13 passed (both queue tests of housekeeping B). With `build-20261005-182804` (consensus alone, 97 passed) every package of the six is green on 21d4c73c; the app suite passed in `build-20261005-182244` |
|
|
| tree 9a51e32+ (before pack-loop) | `proto-cuda/nvrtc/build-windows.sh` under the lock (mingw-w64 from Homebrew, the redist dir of the main checkout symlinked into the worktree: NVRTC 12.8.93 DLLs and headers, Khronos CL headers, nothing downloaded) | 19:17 to 19:18:17Z, a trial of the script before the pack-loop merge: igneum-worker-cuda.exe 196082e2... (1,509,376) and igneum-worker-opencl.exe 0981c77d... (444,928), both with the Igneum resource block (whose version string is 0.3.0: the `.rc` files are not among the six version files, section 11); the real pair is rebuilt from the final tree (section 5d) |
|
|
|
|
## 4. The state of the network before the cut
|
|
|
|
Both of the 0.3.9 agent's rollouts finished before anything of 0.3.10 moved: the fee switch (H = 210,000, every node on
|
|
ab8847da..., `release-0.3.9.md` 10e, 17:55Z) and then N3 (the project lead: "deploy N3 now"; `finality_v3_activation_daa` 135,200,
|
|
manifest republished 17:59:14Z, `docs/plans/finality-v3-devnet-publish.md`): the sweep there lists every live node on
|
|
`1f4b44255fcd2ea8f75664ed47200f409186ddd2292960c9e2cf95bbbdc11505` (the observer 17:58:13Z, node 1 17:58:25Z, the seed
|
|
17:58:43Z, PC 2's app node 18:04:57Z, PC 1's app node 18:06:28Z). The override object every node runs, and the one the
|
|
0.3.10 manifest and packaged line carry:
|
|
|
|
```
|
|
{"difficulty_v2_activation_daa":33000,"proving_v0_activation_daa":84100,"fees_v1_activation_daa":210000,"finality_v3_activation_daa":135200}
|
|
```
|
|
|
|
| Node | Binary at 18:03Z | Override | Digest |
|
|
|---|---|---|---|
|
|
| Mac d937c69d | app 0.3.9; its engine has run NO node of its own since its hourly restart at 17:45:16Z: it found node 1 answering on 127.0.0.1:26610 and uses it (the N3 record, "the Mac app's external-node finding"), so the Mac card's node line is node 1's | node 1's | node 1's |
|
|
| PC 1 ae432dc7, PC 2 1ccfe586 | app 0.3.9, node a24ab01a (the Mac cross-build) | the four-field object | 1f4b4425... |
|
|
| Sam's Mac 3a9bf309 | app 0.3.9, node a24ab01a, 0.0 MH/s at 18:03Z | (its app writes the manifest's object) | not read |
|
|
| The observer, node 1 (hand nodes, this Mac) | `vendor/igneum-node-036/target-integration/release/igneumd` a24ab01a | `/tmp/igneum-devnet/override-v3.json`, the four-field object | 1f4b4425... (read 18:02Z) |
|
|
| The seed 188.245.5.161 | `/opt/igneum/v4/bin/igneumd` 7e26e374... (the a24ab01a zig cross-build, glibc 2.34) | `/etc/igneum/override-v3.json`, the same | 1f4b4425... |
|
|
| PC 37ba0461 | silent (0.3.7 at the 0.3.8 cut) | | |
|
|
|
|
Master moved under the branch while it was cut (fef98a2/a850aef CLAUDE.md, 900bba7 the site's 0.3.9 download cards, 46a6a4e and
|
|
bf8d1ba the N3 record, a93199a); merged as d1f4923 (docs and site only, 5 files, no conflict). It moved again at 19:4xZ (the coordinator's
|
|
`explorer` merge, origin/master 9746391: the observer's explorer detail, `site/api/stats.mjs` and `supply.mjs`, the explorer pages, three node
|
|
tests and `tools/ci/public-api-check.mjs` in ci.yml; no app, node or packaging file): the final merge to master lands on it (section 7b).
|
|
|
|
## 5. The node binaries and the DMG (fork 21d4c73c, app 5520fb1 and later)
|
|
|
|
| Platform | Build | sha256 | Size | Notes |
|
|
|---|---|---|---|---|
|
|
| Mac arm64 igneumd | `CARGO_TARGET_DIR=vendor/igneum-node/target-0310 cargo build --release -j 4 -p kaspad -p igneum-miner --features kaspad/igneum-pow` from `vendor/igneum-node-0310`, under the lock, 18:21:54 to 18:24:2xZ (incremental) | 4bb356f492a52154c932ee6f802cd59dc5c917840ad46ba2133905f52b838b3c | 41,152,144 | `21d4c73c` in its strings; copied into `vendor/igneum-node-0310/target-integration/release/` |
|
|
| Mac arm64 igneum-miner | same | 322472ae95a316d12281fed99b2112cdb5f7b6caf9f25ea24019219b7e907948 | 8,514,928 | byte-identical to the 2f88a82f build: the miner is untouched by gossip and c4 |
|
|
| The digest check | that igneumd on a scratch node, ports 60975/60976, 22 s, 18:24:23Z, the four-field object | `Finality rule v3 ... 135200`, `Calibrated v1 fees ... 210000`, digest 1f4b44255fcd2ea8f75664ed47200f409186ddd2292960c9e2cf95bbbdc11505, `igneumd/2.1.0-21d4c73c` | | equal to every live node's (section 4): the three fork branches move no parameter; the chain script stops on any other value |
|
|
| Linux x86-64 igneumd (the seed, glibc 2.36) | `NODE_SRC=<abs> TARGET_DIR=<abs> OUT_DIR=<abs> infra/cross/build-linux.sh` (zig), under the lock, 18:25:43 to 18:27:29Z (105 s incremental) | 40be0e142e30f6de89ccb61f6dc13e8748ed2a41988753604dbdbb9da1466b4b | 47,638,184 | `GLIBC_2.34` at most, `21d4c73c` in its strings; `version.txt` from 21d4c73c (release-0.3.10). Kept in the scratchpad `r0310/cross-final2/` and copied to `infra/cross/out-0310/` of the main checkout for `restart-seed.sh` |
|
|
| Linux x86-64 igneum-miner | same | 6ec3536717536fd505b63eef5d01ec9fa19d1fe608ba86398135c5776222ec73 | 9,631,280 | |
|
|
| Windows x86-64 igneumd.exe | PC 1 job `build-20261005-182405` (`IGNEUM_WIN_RELEASE=<fork>/target-integration/x86_64-pc-windows-gnu/release node tools/build-job.mjs run --node vendor/igneum-node-0310 --target ae432dc7 --targets linux,windows --no-tests` from this worktree: its own `build-inputs-20261005182334-74461.zip`, the per-job name of 68f14b9), published 18:24:05Z, started 18:26:1xZ after another agent's job on PC 1, every stage ok 18:31:43Z (linux 136 s, windows 162 s, pack 3 s; 9 files, 45 MB, all sha256 and PE checks ok, placed) | 3adb01a712fba678706ed5eeb1fa9788895735e4fab5e81dbefe03bd675a5e04 | 50,978,816 | the PC build (housekeeping A); no commit string inside (section 11); acceptance: section 5b |
|
|
| Windows x86-64 igneum-miner.exe | same | 1105151c91738a4e177b8f327625c6e22a90423fe66a71a5d045130618281052 | 10,725,888 | |
|
|
| Windows x86-64 igneum-app.exe (the PC's build; the installer's engine is the GitHub runner's) | same | 4ead820eb4c502de18f8c122d319543452615ba96b8471a501eb2237e76835b7 | 2,925,056 | hotplug's first Windows compile (`detect::adapters`, the WM_DEVICECHANGE path is `host.cpp`, built by the runner): warnings only; among them hotplug's own `function adapter_for is never used` (for its agent), the rest pre-existing (94 `trailing semicolon in macro`, p2p-flows 21, dead `keep`/`wsl_path`) |
|
|
| Linux x86-64 igneumd (HiveOS, glibc 2.39, not the seed) | same, `infra/cross/out/` of this worktree | 38e69412b66a9ee183f3a28f25198f31ce29f83554f7a7f7dce9bdc4f61970be | 48,915,112 | |
|
|
| Linux x86-64 igneum-miner, igneum-app | same | 39efcb1773dc215d..., cd8680e5b8c07795... | 9,607,448; 2,370,544 | |
|
|
|
|
### 5b. The acceptance of the PC-built Windows node (housekeeping A)
|
|
|
|
Run job `run-0310-accept-final` (the housekeeping agent's `run-accept-pc1.ps1` unchanged: copies the three exes from PC 1's
|
|
`/root/igneum-build/target/x86_64-pc-windows-gnu/release` into a scratch folder with NO mingw DLL, runs `rocks-probe`, then
|
|
`igneumd.exe --devnet --nodnsseed --disable-upnp --rpclisten=127.0.0.1:26710 --listen=127.0.0.1:26711 --nologfiles --yes` on a scratch
|
|
appdir for 60 s, then `igneum-miner.exe key-hash probe`, then reads Application event 1000), published 18:32:37Z from this worktree:
|
|
ran 18:33:08 to 18:34:15Z (67 s), exit 0: `rocks-probe` exit 0, `RESULT run node60: still running after 60 s (alive for the whole wait); killing it`, `igneum-miner.exe key-hash probe` exit 0, `RESULT event 1000: none since 18:32:11Z`. The same verdict as the housekeeping run (`run-hk-accept-1`) and my warm-up run on 2f88a82f (`run-0310-accept-hk`, 16:57Z, the exe 37d0b104...). So the PC-built Windows node ships; no Mac cross-build was needed. The payload inputs: section 5c.
|
|
|
|
### 5a. The DMG
|
|
|
|
The chain under the lock (the 0.3.9 script's shape): the app (`cargo build --release` in `app/igneum-app`, 18:24:47Z, target cloned from the
|
|
0.3.9 worktree), the prover host and export (18:24:54Z, no Succinct toolchain: the host embeds `elf/`), `igneum-prove-host --mode id`
|
|
on this tree: shard program id `0x2b1a81cb413236cf063077b46ed3111628f6c41036bcf6e23ee4cbbf5679ef7a` (2,832,504 bytes, sha256 0x150f4c05a2951fc5),
|
|
aggregator `0x474678f35f7545db28055d5e5bbc308231d84a5a072202087a2a8d5b09123896`: the 0.3.9 pin, unchanged (no prover rollout). The
|
|
nine real fixtures ran `--mode native` with the new host (338, 341, 344, 56 x2, 58927, 72803, 72854, 78: all ok); the first chain
|
|
stopped on `block-72803-skipped-copies.json.node-plan.json`, which is a node-plan SIDECAR of the 72803 fixture (txgen 49796b0), not a
|
|
fixture, so the glob now excludes `node-plan` sidecars. Then `NODE=<fork>/target-integration/release/igneumd MINER=... packaging/mac/build-dmg.sh`
|
|
(18:26:04 to 18:26:34Z): `prover: ... from <worktree>/proving/igneum-prove/target/release`, the same `--mode id` line, fingerprints 477bb0ef
|
|
(intake key) and ed9c4d2e (folder).
|
|
|
|
| Artefact | sha256 | Size |
|
|
|---|---|---|
|
|
| `packaging/mac/dist/Igneum-Miner-0.3.10.dmg` (engine 0.3.10, node 21d4c73c Mac arm64, the 0.3.9 prover host and export rebuilt from this tree) | e8f1f9f620cb598fd21ff2b6f6a2dea3d9e58efc0550c6454dc65f0a3d791b97 | 41,377,608 |
|
|
|
|
Read back from the DMG (mounted read-only, 18:28Z): `Contents/Resources/igneum-app.json` carries `node_override_params` =
|
|
`{"difficulty_v2_activation_daa": 33000, "fees_v1_activation_daa": 210000, "finality_v3_activation_daa": 135200, "proving_v0_activation_daa": 84100}`
|
|
(the packaged line of 9997ef1) beside `update_manifest`, `log_intake_url`, `log_intake_key`, `live_page`, `download_page`; `bin/` holds
|
|
igneumd, igneum-miner, igneum-bench, igneum-prove-host, igneum-prove-export.
|
|
|
|
A note on my own logs: the chained scripts print `<time> ... exit $?` lines where the time is a command substitution evaluated first, so
|
|
those lines always say 0. Every outcome in this plan is read from a content line (`Finished`, a sha256, a `RESULT`, a `final:`), never
|
|
from them; `tools/lock/with-lock.sh` itself propagates the exit code (checked: `run bash -c 'exit 3'` returns 3).
|
|
|
|
## 6. The harness runs (C and D)
|
|
|
|
| Run | Node | Command | Result |
|
|
|---|---|---|---|
|
|
| D, early (before c4) | fork 21babaa7 Mac arm64 (`target-0310`) | `IGNEUMD=... IGNEUM_MINER=... IGNEUM_HARNESS_BASE_PORT=29750 IGNEUM_HARNESS_TMP=/tmp/igneum-txrelay-0310 tools/lock/with-lock.sh run node tools/txgen/relay-net.mjs --rate 2 --duration 120 --wallets 16 --fund 2` from the `tx-gossip` worktree (the harness was not yet on this branch) | 17:54:26 to 18:01:01Z: `final: sent 240, included 240, pending 0, failures 0`, 1.951/s, latency p50 2,017 ms, p90 3,561 ms, max 6,560 ms: the coordinator's expected 240 of 240 |
|
|
| D, final | fork 21d4c73c Mac arm64 (igneumd 4bb356f4...) | the same command from the `tx-gossip` worktree (this worktree has no `node_modules`: the harness imports `viem`, the first attempt from here died at import; `tools/txgen/` needs an install note), ports 29750+, data `/tmp/igneum-txrelay-0310b` | 18:26 to 18:31:05Z: `final: sent 240, included 240, pending 0, failures 0`, 1.975/s, latency p50 1,537 ms, p90 3,027 ms, max 5,025 ms; report in the scratchpad `r0310/relay-final2/` (by miner: B and C only, A none, as the harness requires) |
|
|
| C, first run | fork 21d4c73c Mac arm64 (igneumd 4bb356f4...) | `node tools/finality-attacks/c4.mjs on` with the script's DEFAULTS (split 150 s, weight window 120 DAA; ports 29900+, suffix 990, `/tmp/igneum-fin-c4-0310`), 18:24:50 to 18:34:43Z | `[FAIL]`: B locked 7 and 8 during the split, n0 reconnected 3 s after the heal, the three sinks DISAGREE, 0 locks adopted from an off-chain certificate, 9 / 4 / 4 conflicting certificates. The same shape as the bench-log's "on, split 150 s" row on the fix: a 150-s split is past the 120-DAA frozen table, so by the heal A's chain has outrun the table and the certified chain cannot be adopted; the c4 agent's measured PASS rows use `WINDOW=240` (a 140-s split stays inside the 126-DAA bound, as any partition under an hour does on the live devnet) |
|
|
| C, final | the same binary | `WINDOW=240 SPLIT=140 node tools/finality-attacks/c4.mjs on` (the bench-log's PASS row's knobs, `/tmp/igneum-fin-c4-0310b`), 18:36:3x to 18:46:20Z | `[PASS] weight-vs-work-on`: B locked index 9 during the split (138 s after the cut), A's sink blue score 315 against B's 292 (A the heavier by work), n0 reconnected 3 s after the heal, the three sinks AGREE (0424542743) on B's chain, A's split tip abandoned, n0 adopted 1 lock from a certificate off its chain (the C4 fix) and re-determined 1 line, 0 conflicting certificates, 0 reorg lines, max locked index 15 on all three. The coordinator's rollout note holds on the shipped binary: the certified chain wins over the heavier one when every node is new |
|
|
|
|
### 5c. The Windows payload inputs (18:35:33 to 18:36Z)
|
|
|
|
`IGNEUM_WIN_RELEASE=<fork>/target-integration/x86_64-pc-windows-gnu/release IGNEUM_NODE_SRC=<fork> packaging/windows/push-inputs.sh` from this
|
|
worktree (the `sed -n 1p` signer read of 68f14b9): igneumd.exe 3adb01a7... (50,978,816) and igneum-miner.exe 1105151c... (10,725,888), the PC 1
|
|
build of 21d4c73c, with the PC's three mingw DLLs (libstdc++-6 26,347,027, libgcc_s_seh-1 774,200, libwinpthread-1 324,451; the exes import
|
|
none of them since housekeeping A, the DLL gate and the copy stay for an older fork). Signed inputs: zip
|
|
c175446e33ccb020e2f5fdaf99100d5bd46138c7693b3e16948a82f7c252eaa4 (33,996,479 bytes, 5 files), `node_source_commit`
|
|
21d4c73c6ce32fcbd68391968e85827339a511c0 (release-0.3.10), `repo_commit` 9a51e32, built 18:35:37Z, deployed, `payload-inputs.json` HTTP 200;
|
|
`node-source.pin` = 21d4c73c, committed as 065c67c.
|
|
|
|
## 7. The ship
|
|
|
|
`git push -u origin release-0.3.10` at b958493 (18:36:44Z, the credential helper; `gh auth switch --user igneum-labs` before every gh call,
|
|
the login renamed igneum-labs at 18:00Z, the token still under that name), `gh workflow run windows.yml --ref release-0.3.10` -> run
|
|
37357266092 (queued 18:36:49Z on b958493).
|
|
|
|
The pre-push hook (fb076de: no conflict markers, `cd site && node build.mjs`) left `site/index.html` and `site/journey.json` modified after the
|
|
push. Measured with three builds of one tree at 18:38Z: the label of one bench entry ("RTX 5090 first run" against "RTX 5090, memory-hard
|
|
dataset") ALTERNATES on every run, whatever the cwd: `build.mjs` reads `site/journey.json` back (line 248) and writes it, so each run
|
|
transforms the previous output. Recorded in section 11. The hook's output (master's form) is committed as ccd2b98 and the next push's
|
|
hook flipped it again; the tree is restored with `git checkout -- site/` after every push, so the ship's preflight sees it clean. Those commits touch no
|
|
app or packaging file, so the installer built at b958493 is the release; the ship state file (`~/.cache/igneum/ship/0.3.10.json`) got `sha`
|
|
= b958493 and `runId` = 37357266092, what the tool's own steps would have recorded, and the run resumes with `--from ci`.
|
|
|
|
Run 37357266092: green at 18:42:18Z (parse checks 52 s; engine, window host, payload, installer, smoke run 4 min 24 s; the G13 inputs step
|
|
against the 21d4c73c inputs of 5c and the pin of 065c67c). The dry run of the ship command (18:38:54Z) read everything: tree ccd2b98 clean,
|
|
0.3.10 in all 6, fork 21d4c73c, both exes, both Mac binaries, gh igneum-labs, live inputs 21d4c73c built 18:35:37Z, live manifest 0.3.9.
|
|
|
|
HOLD (coordinator, 18:4xZ): both PCs' NVIDIA workers have looped since 18:31Z (PC 1) and 18:35Z (PC 2) on `the epoch seed bytes do not give
|
|
the pack's IGNEUM_SEEDW_INIT` (the exported GPU pack is the previous epoch's and the restart loop never re-exports it; the network fell to
|
|
about 88 MH/s); the coordinator published re-export jobs and holds every rollout until the fleet mines again. The ship steps that deploy
|
|
nothing ran by hand and the manifest step waits: `OTA_SKIP=1 CONSOLE_SKIP=1 packaging/windows/fetch-ci-artifacts.sh 37357266092` (18:43:26Z)
|
|
and the DMG copied into the downloads folder.
|
|
|
|
| File (in the downloads folder, not yet deployed) | sha256 | Size |
|
|
|---|---|---|
|
|
| Igneum-Miner-0.3.10.dmg | e8f1f9f620cb598fd21ff2b6f6a2dea3d9e58efc0550c6454dc65f0a3d791b97 | 41,377,608 |
|
|
| Igneum-Miner-Setup-0.3.10.exe | 7ecf109f3867b554ecb97be3542f23c8d76d9752d6bdc7ece7a6f95fc2f9bfd4 | 24,994,420 |
|
|
| igneum-windows-app.zip | ebf0deca0e99de4085af1af5c3349028d8a8386f5016ee357ecf806df9a329c9 | 34,778,147 |
|
|
|
|
The installer is 5.2 MB larger than 0.3.9's (19,836,912) and the zip 7.1 MB larger: the PC's `libstdc++-6.dll` is 26,347,027 bytes
|
|
(Ubuntu's GCC 13 build, unstripped) against the Mac toolchain's, and it rides in the payload although the exes import no mingw DLL since
|
|
housekeeping A (section 11).
|
|
|
|
The ship command, to run when the hold lifts (every step before `manifest` skips itself):
|
|
|
|
```
|
|
node tools/ship-app.mjs 0.3.10 --node vendor/igneum-node-0310 --branch release-0.3.10 --public \
|
|
--activation-height 135200 --deadline-note "finality v3" --notes "<the five changelog lines, section 1; node 21d4c73c>" --from ci
|
|
```
|
|
|
|
`consensus.override` is carried over from the folder's 0.3.9 manifest (the four-field object of section 4) and `--activation-height 135200
|
|
--deadline-note "finality v3"` keep the consensus object identical field by field to the live one.
|
|
|
|
### 7a. The second final tree (the project lead's scope change, 19:0xZ): the app rebuilt
|
|
|
|
The node stays 21d4c73c (its binaries, the digest, the suites, the harness runs and the Windows node acceptance all stand). What rebuilds:
|
|
the app (G, H, I: the UI, the engine additions, the export lock), the two Windows WORKER exes (H changes `packfile.h`, `worker.cpp` and
|
|
`host.c`), the payload inputs (now carrying the workers and the NVRTC DLLs), the DMG, the installer (CI), and the ship files.
|
|
|
|
| What | Result |
|
|
|---|---|
|
|
| The two Windows workers, `proto-cuda/nvrtc/build-windows.sh` under the lock, 19:24Z, from 854f9a8 | igneum-worker-cuda.exe 85cc357bb62bda36634ff4c1d80aca575aad71204ff2ef4bf620bdf183ab9236 (1,510,912), igneum-worker-opencl.exe afa73a324b9bfc3d957d0d5d3b6453770046721ac40cf7455ff3d49eb8b5774f (445,952); both differ from 0.3.9's (f50e19f2..., 1abc673e...) and from the hot-fix pair on the PCs (8dcef61c..., af7f12f5...), as the coordinator's check requires; the resource block on both |
|
|
| `push-inputs.sh` again, 19:24:41Z | 12 files: the node pair and three DLLs of 5c, the two workers, nvrtc64_120_0.dll (86,728,192) and nvrtc-builtins64_128.dll (6,356,480), the three licence texts; zip ec1af603cb047471f8e6d6d95adbad192e0b77095e031fd4e39a807e37e8bc5c (71,940,402 bytes), node 21d4c73c, repo 854f9a8, signed, deployed, HTTP 200; the pin unchanged |
|
|
| `cargo build --release` in `app/igneum-app`, then `cargo test --release -p igneum-app`, under the lock on this Mac (the coordinator's ask) | 19:24Z: ok, 103 (lib) + 27 (ota-sign) + 8 (prove-verify), 0 failed; `igneum-app 0.3.10` |
|
|
| `--mode id` on the worktree's host (unchanged prover) | shard `0x2b1a81cb...`, aggregator `0x474678f3...` |
|
|
| The DMG, `packaging/mac/build-dmg.sh` under the lock, 19:24:57 to 19:25:18Z | `Igneum-Miner-0.3.10.dmg` 151687c5672553c571af7224a8e4228348135b8a313fc89e6641db7ae21c85b3, 41,383,375 bytes (engine 0.3.10 with G, H, I; node 21d4c73c; the 0.3.9 prover), fingerprints 477bb0ef and ed9c4d2e; it replaces the e8f1f9f6... DMG of the first tree |
|
|
| PC 1 app-only job (`node tools/build-job.mjs run --target ae432dc7 --targets windows --no-node --no-tests`, the branch's new `--no-node`) | published 19:24:50Z, done 19:26:47Z in 14 s (the engine alone, 6 s on PC 1's warm cache): igneum-app.exe 4564f8502bf40fdfb61a979ff19543838ca66be95b6b9f0beeaee53330c4d284 (2,962,944), warnings only (the dead-code set of 5a plus `count` is never used); `host.cpp` (WM_GETMINMAXINFO, WM_DEVICECHANGE) compiles on the runner only: CI run 37363381420 (section 7b) |
|
|
|
|
Not in this tree (they arrived after it; next cut): `job-console` 3562f26 (its second offer, 19:2xZ: power control as a setting, default off), and
|
|
the FORK-side `pack-loop` 05ef0fa3 (`vendor/igneum-node`: the miner checks every pack and rebuilds a refused one, exit 44; a node change: the
|
|
node stays 21d4c73c). The app side of af983a7 (`watchdog.rs` `PackRebuilds`, the engine re-exporting the pack on a refusal, capped per
|
|
epoch) is keyed on the miner's exit 44, which only the fork-side miner emits (the pack-loop agent's correction, 19:3xZ): with the 21d4c73c miner a worker refusal line shows "program pack out of date, rebuilding" on the strip, the card and the log but does NOT re-export; with the attempt-aware workers of this release a refusal means a genuinely broken pack, so the gap is small, and the self-healing half lands with the next node cut (section 11).
|
|
|
|
### 7b. The second push and CI
|
|
|
|
`git push origin release-0.3.10` at 5b0d54f (19:26:13Z; the pre-push hook flipped the two site files again, restored with `git checkout -- site/`),
|
|
`gh workflow run windows.yml --ref release-0.3.10` -> run 37363381420 (queued 19:26:19Z on 5b0d54f). The ship state file now names 5b0d54f
|
|
and that run. At 19:46Z the run was still QUEUED with no runner assigned, as were the repo's two `ci` runs (19:26Z, 19:39Z): GitHub's status
|
|
API reported Actions "degraded_performance" with an unresolved "Incident with Actions" (investigating since 19:15:17Z). Nothing in this
|
|
repository or on this Mac can shorten that: the installer comes only from the hosted `windows-latest` runner. The rollout waits for the
|
|
verdict; every other step is staged (the DMG, the signed inputs, the runbook `r0310/rollout.sh` with the exact commands).
|
|
|
|
Run 37363381420 ended at 19:41:24Z as `failure` with its first job CANCELLED by GitHub after 15 minutes queued ("The job was not acquired by
|
|
Runner of type hosted even after multiple attempts", the job's annotation) and the build job skipped: nothing of the tree ran. The workflow
|
|
was dispatched again under a watcher that dispatches again on that same annotation and stops on a green or on a failure of the tree itself:
|
|
try 2 run 37365130137 (19:42:43Z), try 3 run 37366744501 (19:57:52Z), try 4 run 37368355454 (20:13:21Z), try 5 run 37369931084
|
|
(20:28:32Z), every one cancelled by GitHub the same way after about 15 minutes queued; a follow-on watcher took over at 20:54:38Z and
|
|
dispatched try 6, run 37374158235 (20:54:55Z), which a runner acquired at 21:24:04Z and which went green at 21:30:29Z (section 7d).
|
|
Six dispatches, 2 h 04 min from the first to the green; GitHub's incident ("major outage" at 20:4xZ) was the whole of it.
|
|
|
|
### 7c. The fallback, prepared at 20:3xZ (the coordinator: a switch at 21:00Z, not a scramble; nothing built yet)
|
|
|
|
What the runner does for the installer, and what PC 1 has for each step (probe job `probe-installer-pc1-0310`, read-only, ran 20:33:5xZ
|
|
in 3 s; its `R` helper collided with PowerShell's `r` alias so every line came back inside an error message, the facts intact):
|
|
|
|
| Runner step | Needs | PC 1 (ae432dc7) | Fallback |
|
|
|---|---|---|---|
|
|
| engine (`cargo build --release --locked`, MSVC target) | Rust on Windows | not probed (the PC's build jobs build the engine in WSL on the GNU target: igneum-app.exe 4564f850..., 5a) | the GNU-target engine from job `build-20261005-192450` unless `cargo` exists on the Windows side (checked at the start of job B); recorded either way |
|
|
| packaged configuration | the intake key and the folder token as files | the installed 0.3.9 app's `igneum-app.json` at `C:\Users\Admin\AppData\Local\Programs\Igneum Miner\` carries the same manifest URL and key (`override=False`: 0.3.9 shipped no packaged override) | copy that file and add `node_override_params` = the four-field object (not a secret); no secret leaves the Mac |
|
|
| payload inputs (`payload-inputs.zip`, signature, hashes, node commit) | the dl folder URL | the URL's folder is in the packaged json | download, verify the sha256s against `payload-inputs.json` (the signature is verified by the runner's step with the key compiled into the app; on the PC the app's own `igneum-ota-sign` is not installed: recorded as a gap, the hashes stand) |
|
|
| window host (`app\windows\BUILD-APP.bat`, MSVC v143 cl.exe and rc.exe) | Visual Studio with MSVC | `vcvarsall.bat` under `C:\Program Files\Microsoft Visual Studio\...`, cl.exe 19.51.36260 | runs as on the runner |
|
|
| payload (`make-payload.sh`, bash) | Git Bash | `C:\Program Files\Git\bin\bash.exe` (and WSL) | runs as on the runner |
|
|
| installer (`build-installer.ps1`: Inno Setup 6 `ISCC.exe`, rcedit) | Inno Setup 6, rcedit-x64 | ISCC NOT FOUND; rcedit not on PATH; winget present | `build-installer.ps1` installs Inno Setup 6 through winget (a tool install on PC 1: the coordinator's call) and downloads rcedit-x64 from GitHub (`-NoRcedit` skips it: the exes then ship without the coin icon and version block, cosmetic, recorded) |
|
|
| smoke run, launcher dry run | the exes | | the same PowerShell lines in job B |
|
|
| the files back to the Mac | | `relay/clients/send.ps1` (a file up to 50 MB straight to Blob): the installer is 25 MB, the payload zip 35 MB | two `send.ps1 <file>` calls with the sha256s in the report; on the Mac `node tools/relay.mjs read <id>`, sha256 compared, `packaging/windows/check-runtime-dlls.sh` on the payload folder |
|
|
| code signing | none on the runner either | | none |
|
|
|
|
The jobs, in order (neither published until the word; both staged in the scratchpad `r0310/fb/`: the tree zip `fb-tree-0310.zip`, 709,814 bytes,
|
|
98 files from 5b0d54f by `git archive`, sha256 012c8a52c27631e8539704d703bf13eb68de0afd9cef094e68d1e6fc398f2cfc; the script `fb-installer-pc1.ps1`,
|
|
which stops on any sha mismatch, pins the inputs' node commit against `node-source.pin` as the runner's G13 step does, and takes `-NoRcedit` as its one argument):
|
|
|
|
1. `fetch` job `fb-tree-0310` to ae432dc7: a zip of the tree at 5b0d54f (`git archive`: `app/windows`, `brand/icons`, `packaging/windows`,
|
|
`proto-cuda/windows-app`, `proto-cuda/{host.cu,build.bat,README.md}`, `proto-opencl/{host.c,build.bat,README.md}`, `wsl2`, the two
|
|
worker `.rc` files), `--dir jobs --extract --extract-dir fb-0310 --fresh`.
|
|
2. `run` job `fb-installer-0310` to ae432dc7 (PowerShell, 20 min): `cargo --version` if any; the packaged json copied and extended; the
|
|
inputs zip downloaded and hash-checked; `BUILD-APP.bat`; `make-payload.sh` under Git Bash with `IGNEUM_APP_EXE` = the WSL engine
|
|
(`\\wsl$\Ubuntu-24.04\root\igneum-build\...\igneum-app.exe`, its sha256 checked against 4564f850...); `build-installer.ps1 -Payload <folder>
|
|
-Version 0.3.10` (with or without `-NoRcedit` per the word); `igneum-app.exe --version`, `Igneum Miner.exe --version`; `send.ps1` the
|
|
installer and the zip; every version and sha256 in the report.
|
|
3. On the Mac: the two files into the downloads folder, hashes equal to the report, the DLL gate, then
|
|
`node tools/ship-app.mjs 0.3.10 ... --from dmg` (preflight, then dmg already, copy, manifest, deploy, verify, console; the fetch step is
|
|
skipped by `--from`, the console item carries no run id), then the rollout as planned.
|
|
|
|
What the gate records in this plan if the fallback ships: the installer marked "PC-built on ae432dc7, GitHub run owed"; non-reproducible
|
|
(the runner's engine is the MSVC target, this one the GNU target from WSL; Inno Setup's version from winget; cl.exe 19.51.36260; Git Bash's
|
|
version; Windows 11 build 26200); the sha256 and size of the engine, the host, the payload zip and the installer; the GitHub run's id and its
|
|
own installer sha256 when it lands (they differ by construction); and the next cut goes back through the runner.
|
|
|
|
The coordinator asked at 21:3xZ whether the Mac mingw path was now faster and safer; the answer stands as below (there is no such path to an
|
|
installer), so PC 1's queue position was kept for attempt 4.
|
|
|
|
What does not exist: a Mac mingw build of `host.cpp` (the coordinator's "how 0.3.7 shipped"): 0.3.7's node exes were cross-built on the Mac
|
|
and its installer still came from the runner (release-0.3.6 plan, section 9); the host has only ever been built by `BUILD-APP.bat` with MSVC,
|
|
on the runner or on a PC. So if PC 1 lacked MSVC the next fallback would be new work, not a known path; PC 1 has MSVC, so it is not needed.
|
|
|
|
### 7d. The fallback, run (the coordinator's word at 21:00Z: GitHub's status page "Actions: major outage", the sixth queued run)
|
|
|
|
The PC 1 window: asked of the Counter ASIC coordinator (it schedules PC 1 tonight) at 21:0xZ; granted at 21:09:24Z when its reproducible
|
|
benchmark released the machine (every card restored; the RX 9070 XT back on the bus). `fb-tree-0310` (fetch) published 21:10:59Z, ran
|
|
21:11:32 to 21:11:33Z: the zip (709,814 bytes, sha256 ok) extracted into `%LOCALAPPDATA%\igneum\app\jobs\fb-tree-0310\fb-0310` (the
|
|
script's expected path was wrong and it now finds the tree by search). `fb-installer-pc1` (run, 25 min cap) published 21:19:28Z, FAILED at 21:20:05Z with exit 2 after 2 s: the script, not the build.
|
|
Its own lines: the tree found (82 files), `cargo on Windows: none` (so the engine is the WSL GNU-target build), Git Bash 5.3.15, Windows
|
|
10.0.26200, and then `engine not found at \\wsl$\Ubuntu-24.04\root\igneum-build\...\igneum-app.exe` with PowerShell's
|
|
`ItemExistsUnauthorizedAccessError`: the WSL tree belongs to root and the `\\wsl$` share refuses it to the app's non-elevated session. The
|
|
build jobs read that tree with `wsl -u root`, so the script now copies the engine out with
|
|
`wsl.exe -d Ubuntu-24.04 -u root -- bash -c "cp ... /mnt/c/.../fb-0310-work/igneum-app.exe"`. Republished as `fb-installer-pc1-2` at 21:2xZ
|
|
(the Counter ASIC coordinator kept PC 1 for it: "a 2-second exit 2 is the script, not the build", 5-minute reply window met).
|
|
|
|
`fb-installer-pc1-2` (21:23:08Z) failed at 21:23:58Z, exit 2 in 4 s: `/root/igneum-build/app/igneum-app/target/...` does not exist (the build
|
|
job keeps ONE target dir for the whole job; the acceptance script had read the node exes from `/root/igneum-build/target/...`): the script
|
|
now finds `igneum-app.exe` under `/root/igneum-build` with `find` and prints its sha256 from inside WSL. `fb-installer-pc1-3` (21:26:52Z)
|
|
failed at 21:27:20Z, exit 2 in 4 s: the inline `bash -c "..."` string lost a quote on its way through PowerShell (`unexpected EOF while
|
|
looking for matching quote`), the class the 0.3.6 cut closed for the app's own WSL calls (3811d8e, "every WSL script runs from a file") and
|
|
which this scratch script had reopened: the bash part is now written to `engine.sh` on the PC (LF, no BOM, the acceptance script's own
|
|
pattern) and run as `bash <file> <arg>`. Three 4-second failures of the script, none of the build; by the Counter ASIC coordinator's rule
|
|
its 10-minute measurement takes PC 1 first, then a read-only path probe (every path the script needs, found and printed), then attempt 4.
|
|
|
|
Before attempt 4 one more change, after reading the fixed block: `find ... | tail -1` would take the NEWEST `igneum-app.exe` under
|
|
`/root/igneum-build`, and other agents' build jobs have run on PC 1 since mine (job-console's, pack-loop's), so the sha check could stop
|
|
attempt 4 on another tree's engine. The engine this plan already holds and verified (4564f850..., 2,962,944 bytes, from PC 1's own job
|
|
`build-20261005-192450`) now travels INSIDE the tree zip (`fb-tree-0310b.zip`, with it under `app/igneum-app/target/x86_64-pc-windows-gnu/release/`),
|
|
and the script reads nothing from WSL at all; the path probe checks that file too.
|
|
|
|
Attempt 4 was never published: at 21:30:29Z GitHub's runner acquired try 6 (run 37374158235, dispatched 21:10:04Z on 5b0d54f) and it went
|
|
GREEN (the parse job 21:24:04 to 21:24:57Z; engine, window host, payload, installer, smoke run 21:25:18 to 21:30:29Z; the G13 inputs step
|
|
against the 21d4c73c inputs of 7a). The recipe's own installer therefore ships; the fallback stops here with nothing built on PC 1, PC 1
|
|
released to the Counter ASIC coordinator at 21:31Z, and the prepared pieces (the tree zip with the engine, the installer script, the two
|
|
probes, the runbook steps) kept in the scratchpad `r0310/fb/` as the rehearsed path for the next outage. Cost of the detour: three
|
|
4-second script failures on PC 1 and about 70 minutes of the shipper's attention; the gate record "PC-built, non-reproducible, GitHub run
|
|
owed" is not needed.
|
|
|
|
### 7e. The ship (21:31 to 21:40Z)
|
|
|
|
`OTA_SKIP=1 CONSOLE_SKIP=1 packaging/windows/fetch-ci-artifacts.sh 37374158235` (21:31:21Z): the installer and the payload zip into the
|
|
downloads folder; the DMG copied beside them. The payload zip holds the rebuilt workers (igneum-worker-cuda.exe 85cc357b..., 1,510,912;
|
|
igneum-worker-opencl.exe afa73a32..., 445,952: both differ from 0.3.9's f50e19f2.../1abc673e... and from the PCs' hot-fix pair), the PC-built
|
|
node 3adb01a7..., and the runner's MSVC engine 496c4883... (3,361,792). The first ship run (21:32:02Z) stopped in preflight: the tree was 4
|
|
commits behind origin/master (the coordinator's explorer merge, 9746391: docs, site, observer, ci.yml; no app, node or packaging file), so
|
|
`origin/master` was merged as ff873f6 (37 files, no conflict), pushed 21:32:27Z (the hook's site flip restored), and the state file kept
|
|
`sha` = 5b0d54f, the CI commit, as the 0.3.6 and 0.3.9 cuts did.
|
|
|
|
```
|
|
node tools/ship-app.mjs 0.3.10 --node vendor/igneum-node-0310 --branch release-0.3.10 --public \
|
|
--activation-height 135200 --deadline-note "finality v3" --notes "<the seven changelog lines; node 21d4c73c>" --from ci
|
|
```
|
|
|
|
| Step | Result |
|
|
|---|---|
|
|
| preflight | ok: tree ff873f6 clean, 0.3.10 in all 6, fork 21d4c73c, gh igneum-labs, live inputs 21d4c73c built 19:24:41Z |
|
|
| ci | already: run 37374158235 green |
|
|
| fetch, dmg, copy | already (above) |
|
|
| manifest | 0.3.10 mac+windows, signed (key 8f186e37...), verified locally; `consensus` carried over from the folder's 0.3.9 manifest: `activation_height` 135200, `deadline_note` "finality v3", `override` the four-field object; and in `dl/public/` (with the wallet 0.1.4 manifest) |
|
|
| deploy | one deploy, 21:33Z |
|
|
| verify | the token folder: `igneum-app-latest.json` 0.3.10, signature ok, mac 151687c5..., windows 24e58849...; the public folder: every file HEAD 200 with the local size (the DMG, the installer, the 0.3.9 HiveOS package, the wallet DMG, the four `/public/` aliases, the two manifests and their signatures, `igneum-downloads.json`), but the byte comparison of a json file against the edge still failed after 12 tries at 21:37Z and again on a resume from `verify`: the edge cache, the class of the 0.3.6 and 0.3.9 verifies (section 11 if it does not clear) |
|
|
| console | (pending: `--from console` after the verify clears) |
|
|
|
|
| File | sha256 | Size |
|
|
|---|---|---|
|
|
| Igneum-Miner-0.3.10.dmg | 151687c5672553c571af7224a8e4228348135b8a313fc89e6641db7ae21c85b3 | 41,383,375 |
|
|
| Igneum-Miner-Setup-0.3.10.exe | 24e58849f2b517e8c5579455e8c9d8376ff7dd27052f458ef91913cbc48abc88 | 49,858,273 |
|
|
| igneum-windows-app.zip | 16b68b7bf625a946ac62a22413982b38c6649a124d799b08424a070f1b34810e | 71,809,486 |
|
|
|
|
The installer is 30 MB larger than 0.3.9's (19,836,912): the two NVRTC DLLs (93 MB unpacked) ride with the rebuilt CUDA worker now.
|
|
|
|
`update-now-0310` to all, published 21:39:59Z (apps woken, stamp a23f594a). Timing with the Counter ASIC coordinator (it schedules PC 1 tonight): PC 1
|
|
was released by its hot-table job at 21:35:39Z, so one update-now went to every machine; its era measurement starts on PC 1's 0.3.10 STATUS line.
|
|
|
|
## 8. The machines after the publish (manifest live 21:33Z, update-now 21:39:59Z)
|
|
|
|
Baseline 21:40:31Z: Mac d937c69d app 0.3.9 (its card reads node 1, a24ab01a), PC 1 ae432dc7 0.3.9 node a24ab01a DAA 133,903 141.5 MH/s,
|
|
PC 2 1ccfe586 0.3.9 node a24ab01a DAA 133,945 0.0 MH/s (its 5090 worker off since a job's `/api/resume` at 21:25:11Z that the 0.3.9 app
|
|
answered and never acted on, section 11), Sam's Mac 3a9bf309 quit 53 min earlier, PC 37ba0461 0.3.9 node `2.1.0` 2.0 MH/s. A machine
|
|
counts as updated when its engine logs the 0.3.10 header, its node reports a DAA score and its miner a hash rate on 0.3.10. The two checks
|
|
asked by the coordinator: (1) the workers start without the seed-words error on the first try, on the rebuilt pair; (2) the Mac's card
|
|
shows node 1's digest, by design. C5: the provers' "stopped after" lines (shards aborted by the restart).
|
|
|
|
| Machine | On 0.3.10 | Its log |
|
|
|---|---|---|
|
|
| PC 1 ae432dc7 (Windows) | engine restart 21:40:41Z (run `win-ae432dc7-20261005-214041`), 42 s after the job; `[ok] updated to Igneum Miner 0.3.10 from 0.3.9` 21:40:42Z; `igneumd started` 21:40:46Z (the PC-built node from the new install path); `node proof verifier: command` and `reported: command` +6 s; `cards: NVIDIA GeForce RTX 5090 [discrete, off] \| AMD Radeon(TM) Graphics [integrated, off] \| AMD Radeon RX 9070 XT [discrete, off]` (hotplug's line: the iGPU integrated and off, the two discrete cards then started); miners started 21:40:47Z (nvidia-ae432dc7-1, the bundled `igneum-worker-cuda.exe`) and 21:40:48Z (amd-ae432dc7-3, the bundled OpenCL worker); `update to 0.3.10 complete` 21:42:12Z. Check 1 PASS: the NVIDIA miner's `epoch seed 66b26013... (daa 133967): CPU program and cache ready in 176 ms` 21:40:47Z, then `worker: info first pack packs\devnet: nvrtc 170 cache 4 dataset 23 check 249 race 37802 ms variant base; self-test PASS` and `worker: ready cuda NVIDIA_GeForce_RTX_5090 ... prepare 1 path nvrtc 12.8` at 21:41:25Z, no `epoch seed bytes do not give` line; STATUS 45.1 MH/s wall at 60 s (124.1 inside jobs), 124.3 MH/s inside jobs from 90 s on; the console 141.4 MH/s at 21:45:17Z (both cards). The node log: `igneumd/2.1.0` (no commit: the PC build, section 11), `Calibrated v1 fees ... 210000`, digest 1f4b44255fcd2ea8f75664ed47200f409186ddd2292960c9e2cf95bbbdc11505, peers node 1 (192.168.68.64) outbound and inbound and the seed 188.245.5.161, flows registered at THEIR protocol version 13 (a v14 node beside v13 peers, the mixed-fleet case of section 10, measured) | |
|
|
| Mac d937c69d | engine restart 21:40:51Z (run `mac-d937c69d-20261005-214051`), 52 s after the job; `[ok] updated to Igneum Miner 0.3.10 from 0.3.9` 21:40:52Z; `a node already answers on 127.0.0.1:26610; using it (it is not stopped by this app)`: the app attaches to node 1 as it has since 17:45Z, so its card shows node 1's version and digest (check 2, by design); `cards: Apple M5 Max [apple, off]` (its miner was off before the update too); `update to 0.3.10 complete` 21:42:22Z | |
|
|
| PC 2 1ccfe586 (Windows) | its 0.3.9 app fetched the woken jobs file at 21:40:33Z (`1 new for this machine, 1 queued`) and queued the update behind the aggregation-cost agent's running job `agg-cost-pc2-2` (one job at a time), so the update ran at 21:49:24Z when that job ended: installer downloaded and verified 21:49:27Z, `quit: stopping the miners, then the node` 21:49:29Z (no prover "stopped after" line: no shard in flight), engine restart 21:49:41Z (run `win-1ccfe586-20261005-214940`), 9 min 41 s after the job; `igneumd started` 21:49:42Z; `cards: NVIDIA GeForce RTX 5090 [discrete, off] \| AMD Radeon(TM) Graphics [integrated, off]`; miners started 21:49:44Z. Check 1 PASS: `epoch seed 66b26013... (daa 134395): CPU program and cache ready in 169 ms`, `worker: ready cuda NVIDIA_GeForce_RTX_5090 ... first pack ... self-test PASS` at 21:50:21Z, no seed-words line; STATUS 120.7 MH/s inside jobs at 60 s, 120.5 at 90 s; the console 123.2 MH/s at 21:53:04Z. This also ended the `/api/resume` no-op (its 5090 had been off since 21:25:11Z). The node log: `igneumd/2.1.0`, digest 1f4b4425..., flows at version 13 with node 1 and the seed (still a24ab01a for 10 s more) and at 14 with PC 1. The prover: `prover: host /opt/igneum/igneum-prove-host (WSL2), CUDA`, the pinned ids, then three assigned shards (22126, 51922, 84099) each failed with `Failed to create the CUDA prover impl: CudaClientError: Connect(Os { code: 13, kind: PermissionDenied })` at 21:49:56, 21:50:12 and 21:50:32Z: PC 2's prover is dark after the update (section 11) | |
|
|
| PC 37ba0461 (Windows, the US laptop) | its 0.3.9 app (run `win-37ba0461-20261005-202247`) downloaded and verified the installer at 21:40:52Z, started it at 21:40:53Z (`per-user install, no administrator prompt`), stopped its miners and node at 21:41:16Z, and no 0.3.10 engine run had reported by 21:56Z (the console: `STOPPED (update)` for 14 min): the install is in progress or stuck on the owner's machine; nothing to drive from here (section 11) | |
|
|
| Sam's Mac 3a9bf309 | quit since 20:47Z (0.3.9); takes 0.3.10 when it is started | |
|
|
|
|
C5, the shards aborted by the restart: PC 2's engine logged no prover "stopped after" line at its quit (21:49:29Z) and PC 1 runs no prover;
|
|
the Mac's prover was off. Observed count: 0. The coverage numbers measured across the restart carry no abort from it.
|
|
|
|
The ship's last step, `--from console` (21:55:08Z): item #364 "Igneum Miner 0.3.10 shipped (mac+windows)"; the tool's closing line
|
|
"manifest 0.3.10 published 2026-10-05T21:32:40Z".
|
|
|
|
## 8. The machines after the publish
|
|
|
|
(pending)
|
|
|
|
## 9. The hand nodes and the seed (21:49 to 21:50Z)
|
|
|
|
Moved while PC 2 and PC 37ba0461 were still on 0.3.9 (their updates gated by another agent's job and a slow download): the digest does
|
|
not change with 0.3.10 and a v14 node beside v13 peers is the measured mixed-fleet case (section 10), so moving them shortened the mixed
|
|
window.
|
|
|
|
| Node | Command | Result |
|
|
|---|---|---|
|
|
| The observer, then node 1 | `IGNEUMD=<fork>/target-integration/release/igneumd IGNEUMD_COMMIT=21d4c73c infra/devnet/restart-hand-nodes.sh '<the four-field object>'` (the branch's script: `grep -c` for the commit string, the object written to `/tmp/igneum-devnet/override-v3.json`, the previous file kept with a stamp) | observer restarted 21:49:38Z (pid 67770), node 1 21:49:50Z (pid 67963, caffeinate 67965); both print `Calibrated v1 fees ... from DAA score 210000` and digest 1f4b44255fcd2ea8f75664ed47200f409186ddd2292960c9e2cf95bbbdc11505; the Mac app's card follows node 1 from here (its node line reads 21d4c73c) |
|
|
| The seed 188.245.5.161 | `IGNEUMD_LINUX=infra/cross/out-0310/igneumd IGNEUMD_LINUX_SHA256=40be0e14... infra/devnet/restart-seed.sh '<the same object>'` (the zig cross-build of 21d4c73c, glibc 2.36 target; the script checks the sha on both sides, keeps the previous binary as `igneumd.prev-035` and the previous override file with a stamp) | restart 21:50:15Z, unit active, MainPID 125195, `igneumd/2.1.0-21d4c73c`, `Calibrated v1 fees ... 210000`, digest 1f4b4425... |
|
|
|
|
Peers after the restarts: the observer and node 1 connected to the seed within 20 s and register flows at protocol version 14 with each
|
|
other and the seed (node 1's inbound from the observer 21:50:08Z, node 1 to the seed 21:50:20Z, the observer to the seed 21:50:38Z; the
|
|
seed's three inbound peers from this Mac's address at 21:50:20, 21:50:27 and 21:50:38Z, all at 14). PC 1's node (started 21:40:46Z, before
|
|
the hand nodes moved) registered flows at version 13 with node 1 and the seed (then a24ab01a) and keeps them; PC 2's node (21:49:42Z) at 13
|
|
with node 1 and the seed (10 s before their restarts) and at 14 with PC 1: the mixed fleet of section 10, measured on the live network, with
|
|
no refusal and no drop.
|
|
|
|
### 9a. The digest sweep (21:50Z)
|
|
|
|
| Node | Binary | Digest | Read from |
|
|
|---|---|---|---|
|
|
| PC 1 app node ae432dc7 | the PC-built 3adb01a7... (`igneumd/2.1.0`, no commit string) | 1f4b44255fcd2ea8f75664ed47200f409186ddd2292960c9e2cf95bbbdc11505 | its node log through the log intake (run 214041) |
|
|
| PC 2 app node 1ccfe586 | the same | 1f4b4425... | its node log (run 214940) |
|
|
| The Mac app d937c69d | node 1's (attached; its card's commit string is the app's own reading from its start at 21:40:51Z, before node 1 restarted, and refreshes on the app's schedule; its status line reads node 1's height and peers) | node 1's | |
|
|
| Node 1 | 21d4c73c Mac arm64 4bb356f4... | 1f4b4425... | `/tmp/igneum-devnet/node1.out` |
|
|
| The observer | the same | 1f4b4425... | `/tmp/igneum-devnet/observer-v4.out` |
|
|
| The seed | 21d4c73c Linux 40be0e14... | 1f4b4425... | the journal through `restart-seed.sh` |
|
|
| PC 37ba0461 | (its update in progress) | (pending) | |
|
|
| Sam's Mac 3a9bf309 | quit since 20:47Z | (pending) | |
|
|
|
|
One digest, equal to the N3 publish's (`finality-v3-devnet-publish.md`) and to this cut's three readings on the fork (section 3): 0.3.10
|
|
moved no parameter, as measured.
|
|
|
|
## 10. The mixed fleet during the window
|
|
|
|
Between the manifest publish and the last app's update, old (a24ab01a) and new (21d4c73c) nodes share the devnet. What the two
|
|
agents' reports say about whether they can disagree:
|
|
|
|
| Change | Old and new nodes together | Source |
|
|
|---|---|---|
|
|
| D, transaction relay (PROTOCOL_VERSION 13 to 14) | They connect: a v14 node sends the three new messages only to peers at version 14 or later; a 13 peer never sees them (a node drops a connection on an unknown payload, which is why the version moved). Blocks, headers and the handshake are unchanged, the digest is unchanged. Only the mempools differ: a transaction sent to an old node is included only by that node's own templates, as before; a new node's pool converges with other new nodes'. No disagreement about the chain | the tx-gossip commit message e242acd0, the coordinator's line |
|
|
| C, the certificate-driven reorg | A consensus-behaviour change with no digest change: an old node keeps the shipped behaviour (a certificate over a block off its chain stays pending and it never reorgs to it), a new node locks that block and moves to the heaviest tip through it. The two CAN disagree on the selected tip after a partition heals while the fleet is mixed (exactly the C4 shape); the c4 agent's rollout note: it converges when every node is new. On the devnet tonight there is no partition in progress and one network, so the window carries no live split; the sweep in section 9 checks every node's DAA within a few blocks of the others | the c4 agent's rollout note (coordinator, 18:2xZ), the bench-log C4 rows |
|
|
| A, B, E, F | Windows link settings, a test switch, the app's card handling and job exit codes: nothing on the wire | |
|
|
| The override object | The same four fields on every node before and after; the manifest carries it verbatim (section 7), so no node's digest moves | sections 4 and 7 |
|
|
|
|
So the window is safe for ordering (no digest change, no protocol break) and the only behavioural difference needs a partition to show;
|
|
the hand nodes and the seed move last (section 9), after every app node is on 21d4c73c, so the fleet is fully new within minutes of the
|
|
publish.
|
|
|
|
## 11. Open after the cut
|
|
|
|
The next cut (0.3.11), decided by the coordinator on 5 October 2026 night:
|
|
|
|
| Branch | What | Why not 0.3.10 |
|
|
|---|---|---|
|
|
| fork `pack-loop` 05ef0fa3 (`vendor/igneum-node`) | `write_pack_checked`, `export-pack` exit 3, the force-prepare at the epoch boundary, the miner's exit 44 that unlocks the app's capped re-export | a node change after the node was frozen at 21d4c73c with its suites, harness runs and Windows acceptance done; the app side (af983a7) with the attempt-aware workers is the fix that matters tonight and it is in |
|
|
| `job-console` 13755b9, then 3562f26 and whatever follows | one hidden-console builder for every elevated launch, the spawn check in CI, PC 1 console watchers; then power control as a setting, default off (the project lead: "if we don't have to ask then don't ask") | arrived after the tree closed (both times); repoints elevated-exit's `include_str!` test at `platform.rs` at its own merge |
|
|
| `opencl-rdna4-telemetry` 7adcd4c (`igneum-wt-rdna4-telemetry`, not pushed: master + a08c371 as 38c9eec + 7adcd4c) | `igneum-gpu-telemetry.exe` (ADLX, SetupAPI bus, PDH fallback; amdgpu sysfs on Linux) built by `build-windows.sh`, shipped by `make-payload.sh` and `push-inputs.sh`; the engine runs it at `-l 5` and fills power, temperature, fan, memory clock and utilisation on the AMD card; 82 app tests pass. The 9070 XT measured on PC 1: 198.9 W, 64 C, 17.73 MH/s, 0.089 MH/W against the 5090's 0.398 MH/W (job `tele-measure-1`) | arrived after the tree closed; a new shipped exe and an engine source |
|
|
| `ember-tune` 54ff1bc (+38ef711, `igneum-wt-ember-tune`; its agent's message at 21:2xZ) | Ember Tune: `ember.rs` replaces the NVIDIA power-only run in `sweep.rs` and the AMD single-lever sweep of 720b369; sits on cherry-picks of 7adcd4c and 13755b9+3562f26, so those merge first; 93 app tests, `relay/test/ember.test.mjs` and `ui/tune-line.test.mjs` in ci.yml; design in `docs/plans/ember-tune.md` | arrived after the tree closed |
|
|
| the rest of `opencl-rdna4` a08c371 | the OpenCL worker's duplicate-platform fold, `--readback select`, `--memprobe`, `detect.rs parse_opencl_list`, the 9070 XT bench-log entry | conflicts with gpu-hotplug in `detect.rs` and `host.c`; only its EXPORT_LOCK hunk shipped (854f9a8) |
|
|
|
|
| Item | State |
|
|
|---|---|
|
|
| The per-job build-inputs zip (68f14b9, c4's tooling commit; the coordinator's check after two agents' PC jobs ran on another agent's sources through the shared `build-inputs.zip` tonight, jobs build-20261005-191656 and -192537). Checked on this tree's own jobs in the live jobs file rather than a dry run (a dry publish would leave a stray entry in the file the ship deploys): `build-20261005-182405` (PC 1) pins `params.zip_url` = `.../build-inputs-20261005182334-74461.zip`, `params.sha256` 628e6dae..., size 8,266,818; `build-20261005-183131` (PC 2) pins `build-inputs-20261005183051-83796.zip`, c83ebb2a..., 7,271,492; both shas equal the local zips of those names, and 15 per-job zips sit beside the folder default. So a job published through `tools/build-job.mjs` pins the zip it just pushed. The residual: `packaging/ota/publish-jobs.sh add --kind build` WITHOUT `--zip` still defaults to the shared `$DEST/build-inputs.zip` (line 307); nothing refuses that name. A hand-added build job can therefore still pin whatever the folder default holds | build-job.mjs path closed; the hand path is the first item of the next cut: refuse `--kind build` without `--zip`, or default to the newest per-job zip |
|
|
| `infra/cross/build-linux.sh` resolved a relative `TARGET_DIR` inside the node source after its `cd`, so the copy step shipped the previous build's bytes (twice tonight, caught by the commit string in `strings`). Fixed here (e0a5fd1). The class: any script that takes a directory argument and changes directory before using it; `proto-cuda/windows-node/cross-build.sh` takes the node worktree as `$1` and `CARGO_TARGET_DIR` from the environment (the 0.3.9 cut passed a relative one and it worked only because that script does not cd); a CI check for the shape is owed | fixed on the branch; the check owed |
|
|
| hotplug's `proto-opencl/host.c` change (the worker's `--list` prints the PCI address, the key to one row per physical card): no prebuilt OpenCL worker is in the payload inputs (the live zip holds igneumd.exe, igneum-miner.exe and three DLLs), the payload carries `host.c` and `build.bat` and the app builds the worker on the PC from them (`engine.rs build_worker_from_source`), so the change ships inside the 0.3.10 installer; whether an app that already has a worker exe rebuilds it from the newer source was not read here. Check on the console after the update: the PCs' card rows carry the PCI address (and one row per AMD card) only if the worker was rebuilt; else a rebuild is the hotplug agent's follow-up | open: told the hotplug agent |
|
|
| `kaspa-consensus` test `ban_is_decided_by_the_carrying_block_so_nodes_agree_on_every_voter_list` (c4-fix) failed once under the six-package parallel run on PC 2 with `UnexpectedDifficulty(487112096 vs 487129281)` in `mine_on_all` (section 3) and passes alone, twice. The helper builds a block on one `TestConsensus` and inserts it into others; the expected difficulty of the receiving node differs when the run is slow, which points at a wall-clock dependence (difficulty v2's sanitised clock per header). For the c4 agent: pin the clock or the timestamps in that helper, as the finality tests do for the cache queue. Until then a parallel six-package run can fail this test under load; the suites are run as consensus alone plus the other five | open |
|
|
| `site/build.mjs` is not idempotent: one bench entry's label alternates between "RTX 5090 first run" and "RTX 5090, memory-hard dataset" on every run of the same tree (three runs at 18:38Z: first-run, memory-hard, first-run), because the build reads its own `site/journey.json` back (line 248) and the label table at lines 203 and 204 matches against what the previous run wrote. Every push flips the two files through the pre-push hook, which is why the 0.3.9 and 0.3.10 cuts both met a modified tree after the push | open: build the journey from the sources only (never from the previous output), then have the hook refuse a push whose build differs from the committed site |
|
|
| The Windows payload carries the PC's three mingw DLLs (34 MB of inputs, `libstdc++-6.dll` alone 26.3 MB unstripped) although the exes import none of them since housekeeping A; the installer grew 5.2 MB | open: drop the DLLs from `push-inputs.sh` and `jobbuild.rs`'s copy once no shipped fork needs them, or strip them |
|
|
| The two worker exes' version blocks say 0.3.0 (`proto-cuda/nvrtc/igneum-worker-cuda.rc`, `igneum-worker-opencl.rc` are not among the six files `ship-app.mjs` bumps) | open: add the two `.rc` files to `VERSION_FILES` |
|
|
| The app's re-export of a refused pack (af983a7's `watchdog.rs` `PackRebuilds`, 3 per epoch) waits for the miner's exit 44, which the 21d4c73c miner never emits: a refusal in 0.3.10 shows the notice and restarts the worker, no re-export. The fork-side `pack-loop` 05ef0fa3 (the miner checks every pack, rebuilds a refused one, exit 44) is the other half | next node cut |
|
|
| C1 (the consequences reviewer, 20:0xZ): the 0.3.10 node's `igneum_exportSegments` (fork `igneum/exec/src/rpc.rs`) writes no `daaScore` and no `feesV1ActivationDaa` per segment, which the 0.3.9 exporter needs to replay both sides of the fee switch (`export/src/main.rs`: "a dump without `daaScore` is accepted only when the switch is never or 0"). Measured here: the handler (90 lines from `rpc.rs` 849 in `vendor/igneum-node-0310`) carries neither key (`daaScore` appears in the fork only in other RPCs, lines 239, 423, 819); the reviewer names `vendor/igneum-node-pv1` at eb32c645 (on 21d4c73c) as the carrier: its `rpc.rs` writes `daaScore` per segment (line 961) and `fees` and `feesV1ActivationDaa` at the top level (line 982); confirmed here at 20:4xZ (`git -C vendor/igneum-node-pv1 grep -n feesV1ActivationDaa -- igneum/exec/src/rpc.rs`: line 982); my first read of that path at 20:1xZ was wrong. So at H = 210,000 (about 19:50Z on 6 October) every 0.3.10 prover's export of a post-H block fails or meters with the wrong table and the fleet's provers go dark, unless the next node cut carries that RPC onto every prover before H, or H is republished later. The mechanism (the proving agent, 20:1xZ): the app's prover calls the exporter with no fee flags, so from H every app prover on 0.3.10 cuts with the prototype table and every statement is vetoed. The two closes: (a) the proving-v1 fork (eb32c645, on 21d4c73c) on every prover before H through 0.3.11; (b) republish H = tip + 86,400 by the fee-switch plan's rule. Decision due 16:00Z on 6 October; the coordinator, the proving agent and the 0.3.11 shipper hold the same line | open, dated: (a) or (b) by 16:00Z on 6 October |
|
|
| C5 (the same reviewer): the app kills its prover child on quit (`prover.rs` 266 to 272), so the `update-now` of this rollout aborts whichever shard each prover has in flight (up to 37 s each, no payout). Accepted as the cost of the restart; the count is read after the rollout from the provers' "stopped after" lines (section 8) so the coverage numbers measured across the restart are read with it | recorded at the rollout |
|
|
| `/api/resume` answered ok on the 0.3.9 app and never restarted the miners (PC 2's 5090 worker "off" with the card holding 1.7 GB from a job's resume at 21:25:11Z until its 0.3.10 restart, the iGPU miner too; the Counter ASIC coordinator, 21:4xZ). The class: a resume that reports success without a miner restart. The app should re-check the miner processes after a resume and report a failure | open: next cut |
|
|
| PC 2's prover after the 0.3.10 restart: every assigned shard failed at once with `CudaClientError: Connect(PermissionDenied)` (section 8) from 21:49:56Z. RESOLVED at 22:01Z by a socket fix on PC 2 (the Counter ASIC coordinator's agents; the SP1 CUDA prover's client socket permission), after which PC 2 proved and was paid every 40 s. The proving agent is adding a log line that names the cause on the app branch for the next cut | closed 22:01Z |
|
|
| PC 37ba0461 (the US laptop) started the 0.3.10 install at 21:40:53Z, stopped its miners and node at 21:41:16Z and had not come back by 21:56Z: the per-user installer on the owner's machine, nothing to drive remotely | open: the console shows when it returns; its 0.3.10 line and worker start are read then |
|
|
| `dl/public/igneum-downloads.json` (the unsigned index the site's download page reads) alternates at the edge between the new bytes and the previous ones for over 20 minutes after the deploy (one fetch byte-identical at 21:52Z, the next three not): different edge nodes behind one hostname. The two signed manifests were byte-identical and verified from the first check. The ship's verify step counts it as a failure and refuses to post the console item, so the item was posted with `--from console` | open: the verify should accept the index after the signed manifests pass, or retry it for longer; the site serves the previous version's buttons from a stale edge until it settles |
|
|
| The console's machine card keeps a machine's LAST non-empty node commit string: PC 1's card read `node 2.1.0-a24ab01a` for 17 minutes after its node had restarted as the PC build (`igneumd/2.1.0`, no commit), while PC 2's card read `2.1.0` at once; the Mac's card kept node 1's a24ab01a after node 1 moved to 21d4c73c. A card's commit string is therefore not a fact about the running node until the app re-reads it; the node log is | open: the card should show the string the app last READ, with its time, or nothing |
|
|
| The site build, run on a tree with conflict markers in `site/journey.json`, fails silently and leaves the markers (it reads that file back, the non-idempotence above): the first merge commit of this cut to master carried markers in two site files for one minute and was amended (the pre-push hook would have refused it, but the check must not depend on the hook) | fixed by hand here; the build should refuse a journey.json that does not parse and say so |
|
|
| The PC-built node binaries embed no commit (the build inputs zip has no git dir, `build-info` falls back to the bare version): the console shows PC 1 and PC 2 as node `2.1.0` with no commit after 0.3.10, as the 0.3.6 PC build did. The build inputs manifest records the fork commit (2f88a82f and later) | open: a commit stamp through the build job (`jobbuild.rs`) is an app change, not tonight |
|