From 53dae0ddf14e0062c57250367ddeadabe1d478e4 Mon Sep 17 00:00:00 2001 From: igneum-labs <337424239+igneum-labs@users.noreply.github.com> Date: Mon, 5 Oct 2026 15:59:59 +0000 Subject: [PATCH] release-0.3.6 plan: the two finality tests under the parallel suite answered (fork 7003055b); build-job.mjs forwards --node-tests, --app-tests, --no-app Co-Authored-By: Claude Fable 5.1 --- docs/plans/release-0.3.6.md | 2 +- tools/build-job.mjs | 6 +++++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/plans/release-0.3.6.md b/docs/plans/release-0.3.6.md index a60f083f4..813325332 100644 --- a/docs/plans/release-0.3.6.md +++ b/docs/plans/release-0.3.6.md @@ -654,7 +654,7 @@ of which the update-now job's reach on the 0.3.5 PCs (10-minute poll) was 9 min | Why the PC-built Windows node dies at start even with matching DLLs (section 9) | 0.3.8: reproduce under WSL (wine) or on PC 2 in a scratch run; GCC 13 posix vs GCC 16; `-static` and rocksdb's thread model. Until then the Windows node is the Mac cross-build | | `publish-jobs.sh --deploy` writes one folder; PC 2's 0.3.7 app refused one jobs file (`jobs file signature does not verify`, 13:19:41Z, release-0.3.8 plan section 11) | ANSWERED (housekeeping, 5 October 2026, commit on branch `housekeeping`). Cause: the app fetched `igneum-jobs.json` and then `igneum-jobs.json.sig` in two requests (`jobrun.rs` `fetch_jobs`), and the edge serves the previous deployment for some seconds after a deploy, per object (the same afternoon a publish needed 4 live-check tries, 15 s, before the edge served the new file). Two requests a moment apart can therefore return a file from one deployment and a signature from the other: a pair that does not belong together, which the key correctly refuses. The mirror step (bdde87a) was not the cause: it copies after the signer's read-back and one deploy ships both folders. Fix: ONE object, `igneum-jobs.signed.json` (`{"file":"","format":"igneum-jobs-signed-1","sig":""}`), made by the signer (`igneum-ota-sign envelope-jobs`, which refuses a pair that does not verify) and read back by it (`verify-signed-jobs`) before anything moves into place; the app fetches that one object (0.3.9 `fetch_jobs`, `Cache-Control: no-cache`, the pair only when no envelope is published); `publish-jobs.sh` writes all three files, mirrors all three, and after a deploy verifies the envelope and the pair in EVERY folder it wrote (`verify_live` walks `folders()`); `tools/jobs.mjs` reads the envelope; `ship-app.mjs` carries it. Tests: `jobs.rs` `signed_envelope_binds_file_and_signature` (round trip, a file with another publish's signature refused at wrapping and at reading with the words the app logs, a tampered inner byte, another key, each shape error named; 27 signer tests pass), `packaging/ota/test-publish-jobs.sh` (24 checks with the real key in a `--dest` folder: the three files, the envelope's text IS the plain file byte for byte, the stale pair refused by the signer and as a hand-made envelope, `sign` rewrites all three). Until the 0.3.9 apps are out, a refusal of this kind is harmless: the next poll (2 minutes) or wake fetches a consistent pair | | The app marks "update complete" on its own health | it should wait for the node's first DAA score, so a dead node rolls back (the 0.3.6 PCs looped for 20 minutes) | -| Two finality tests under the five-package parallel suite | test isolation (per-process PoW cache directory); a clean `kaspa-consensus`-alone run on 2b6d23ef still owed | +| Two finality tests under the five-package parallel suite | ANSWERED (housekeeping, 5 October 2026, fork commit 7003055b on `housekeeping` from 2b6d23ef). Cause: the PoW engine is one per process (`kaspa_pow::igneum::engine()`, a `OnceLock`), its cache build queue is capped at 2 building + 4 waiting (M15/M30, a node property: what peers can make one node do), and `cargo test -p kaspa-consensus` runs its 94 tests in parallel in ONE process, each mining on its own day; more than six cold 256 MiB builds at once had the header processor refuse two finality tests with `PowCacheQueueFull`. The engine is in play only when `igneum-miner` is in the same cargo invocation (its feature unifies `kaspa-pow/igneum-pow` onto `kaspa-consensus`), which is why the suite alone passed and the five-package run did not. Not a per-process cache directory: the caches are in memory, there is no directory. Fix: `kaspa_pow::igneum::set_unbounded_build_queue(true)`, a process-wide switch that `TestConsensus::new` and `with_db` set, so every caller in a test process waits for a build slot (the miner's own path) and nobody is refused; the node never sets it. Tests: `build_queue_is_bounded` (the default, unchanged) and `build_queue_is_unbounded_for_test_processes` (the same race with the switch on: 0 refused, every caller built), serialised on one mutex so they never read each other's setting. Evidence: PC 2 job `build-hk-tests-1` (15:53:42Z, `--node-tests "kaspa-consensus-core igneum-exec kaspa-pow kaspa-consensus igneum-miner" --app-tests igneum-app`, `tools/build-job.mjs` now forwards both flags): `RESULT test node [kaspa-consensus-core igneum-exec kaspa-pow kaspa-consensus igneum-miner] exit 0 41 s`, `kaspa-consensus` 94 passed 0 failed 3 ignored with `frozen_table_holds_a_side_without_the_other_keys_for_one_window ... ok` and `reorg_past_an_unlocked_checkpoint_re_determines_it_and_verifies_the_pending_certificate ... ok`, `kaspa-pow` 13 passed including both queue tests, `RESULT test app/igneum-app [igneum-app] exit 0 5 s` (75 + 26 + 8). The owed `kaspa-consensus`-alone run is covered: the whole five-package set is green on the fixed tree | | igneumd not reproducible across PC 1 and PC 2 (8f) | unverified why | | Sam's Mac | silent since 09:41Z; takes 0.3.7 on its next check | | The 0.3.6 app-side fixes carried into 0.3.7 and not yet exercised on a PC | the PC's windows stage shipping its DLLs (`jobbuild.rs`), the stamp after extract, the WSL scripts from files on a real probe (the Mac verifier path is proven; the Windows wrapper path only by its unit tests and the node's `verifier reported: command` on both PCs, which means the wrapper's probe answered) | diff --git a/tools/build-job.mjs b/tools/build-job.mjs index 32ce14c04..a77ad0d9b 100755 --- a/tools/build-job.mjs +++ b/tools/build-job.mjs @@ -5,6 +5,7 @@ // and puts them where the packaging scripts look. // node tools/build-job.mjs run [--node vendor/igneum-node-v4] [--target ae432dc7] [--budget-minutes 40] // [--stage-minutes '{"linux":20}'] [--targets linux,windows] [--no-tests] +// [--node-tests "kaspa-pow kaspa-consensus"] [--app-tests igneum-app] [--no-app] // [--title "..."] [--no-place] [--out dir] pack + publish + watch + fetch // node tools/build-job.mjs publish [the same flags] pack + publish, prints the id // node tools/build-job.mjs watch STAGE and RESULT lines as they land @@ -31,7 +32,7 @@ const RELAY_BASE = (cfg('relay-url') || 'https://relay.igneum.network').replace( const argv = process.argv.slice(2); const flags = {}; const pos = []; -const BOOL = new Set(['no-tests', 'no-place', 'no-deploy', 'help']); +const BOOL = new Set(['no-tests', 'no-place', 'no-deploy', 'no-app', 'help']); for (let i = 0; i < argv.length; i++) { const a = argv[i]; if (a.startsWith('--')) { const k = a.slice(2); const next = argv[i + 1]; if (!BOOL.has(k) && next !== undefined && !next.startsWith('--')) { flags[k] = next; i++; } else flags[k] = true; } @@ -198,6 +199,9 @@ function placeFor(o) { function publish() { const pack = ['packaging/windows/push-build-inputs.sh']; if (flags.node) pack.push('--node', flags.node); + if (flags['node-tests']) pack.push('--node-tests', String(flags['node-tests'])); + if (flags['app-tests']) pack.push('--app-tests', String(flags['app-tests'])); + if (flags['no-app']) pack.push('--no-app'); if (flags['no-deploy']) pack.push('--no-deploy'); console.log(`$ ${pack.join(' ')}`); const r = spawnSync('bash', pack, { cwd: ROOT, stdio: 'inherit' });