diff --git a/docs/bench-log.md b/docs/bench-log.md index 584ce9150..e17716cf1 100644 --- a/docs/bench-log.md +++ b/docs/bench-log.md @@ -1262,3 +1262,103 @@ side and is the unit test (`dev_fee_tests`: 100 of 10,000 at positions 99, 199, 100%); the chain test shows the fee blocks reach the chain and are counted the same by the miner and by both nodes. Not measured: worker (GPU) mode, where the fee template is one of the templates the background fetcher rotates through once a second per identity, so the share is 1 in 100 templates by time, not by job; and nothing on Windows. + +## 4 October 2026 (night), ledger M30: the block and transaction floods grew the node by 256 MiB per epoch roll, fixed by sharing the PoW cache across the epochs of a day (memory engineer) + +Machine: Apple M5 Max, 64 GB, load averages 126 to 146 for the whole session (ten or more agents building and running at once). Every count here (blocks accepted, cache builds, RSS before and after) is valid under that load; every latency is an upper bound and is not a number. Private test network of two igneumd on 127.0.0.1 ports 29500+ (node A on 29500/29501/29502, node B on 29510/29511/29512), data under `/tmp/igneum-fud-mem/{baseline,after,after2}` (the second is pass 1, the third the committed build), the 60x fast-time profile (`infra/fast-time/override-60x.json`, `skip_proof_of_work` on, `pow_epoch_blocks` 60, `pow_day_ms` 1,440,000) exactly as the red team ran it. The live devnet and other agents' ports were not touched. Every run went through `tools/lock/with-lock.sh run`, every build and the unit tests through `with-lock.sh build`. + +What the red team saw (`docs/review/redteam-2026-10-04.md` rows 5 and 8, ledger M30): on the 0.3.4 build the s6 submit flood grew RSS by +269 MB, the mempool flood by +270 MB, and the s7 block flood took both nodes from 302 to 1,082 MB. The s6 figures were cumulative from one baseline taken before all three loads (`rss_peak - rss_baseline` in `s6-exhaustion.mjs`), so the mempool flood's "+270 MB" was the submit flood's growth carried forward; its own cost was 1 MB. The red team's guess (execution-layer records, rejected transactions retained) did not hold: the mempool flood retains nothing measurable. + +Cause, measured: every RSS step is one `PoW cache built` line in the node log (`consensus/src/pipeline/header_processor/pre_ghostdag_validation.rs:136`), 256 MiB each. The lottery engine (`consensus/pow/src/igneum.rs`, `IgneumEngine::epoch_for_impl`) keyed its resident entries by `(epoch seed, day)` and built a full `igneum_pow::Epoch` (program plus the 256 MiB ChaCha12 cache) per entry, keeping `KEEP = 4` of them, although the cache is a function of the day seed alone (`igneum_pow::Epoch::from_seed_bytes`: `seed_words_from_bytes(day_bytes)`; the epoch seed only feeds the program). On the 60x profile an epoch is 60 DAA, so the honest chain rolls an epoch every minute and the 50x block flood every 10 to 20 s; each roll cost a cache build and 256 MiB until the fourth entry, then evictions. The engine runs under `skip_proof_of_work` too (`check_pow_and_calc_block_level` always calls it and forces the pass afterwards), so the harness exercised it. The 3 October harness run (this file, "2026-10-03, consensus attack harness") was on the plain devnet profile (3,600-DAA epochs), where no roll falls inside a 60 s flood, which is why it grew 11 to 33 MB; the profile change and the build change were conflated in M30. On the live devnet the same rule means 256 MiB per hour until 1 GiB resident, and a 50x fast miner reaches that in minutes. + +Fix (node fork branch `fud-memory`, from `finality-fixes` 6aa69a45): the engine now holds the 256 MiB caches keyed by day (`IgneumEngine::KEEP_DAYS = 3`: the chain's current day, the next day, one slot for a late block of the previous day or an off-day header; the live pair is never evicted while another day's cache exists; bound 3 x 256 MiB = 768 MiB resident plus `MAX_INFLIGHT_BUILDS = 2` x 256 MiB while builds run) and the programs keyed by `(epoch seed, day)` (`IgneumEngine::KEEP = 8`, a few KB each, LRU). An epoch roll on the same day generates a program and builds no cache; a `BuildReport` (the p2p off-day strike and the "PoW cache built" line) is produced only for a cache build. `EpochRef { program, dataset: Arc }` replaces `Arc` for the node and the miner and computes the identical hash (`interpret_warp_init` on `block_init_words`, as `Epoch::pow_bound` does); the unit test `epoch_rolls_share_the_day_cache` checks the engine's pow against a standalone `igneum_pow::Epoch` for two epochs of one day. Programs whose day cache was evicted are dropped with it, so no entry pins a cache. No consensus rule changed: the hash, the seeds and the acceptance are as before. Miner cost: the GPU prepare path (`--prepare-packs`) exports a pack per epoch roll from a standalone epoch (its own transient fill), because `igneum_pow::emit::export_pack` wants the crate's own `Epoch` and `DatasetSource` is not shareable without a change to the `igneum-pow` crate. + +Commands (run from `igneum-wt-fud-memory`; the "before" binary is the shipping `vendor/igneum-node/target-finality/release/igneumd`, the "after" binary is `vendor/igneum-node-fud-mem/target/release/igneumd` from the commit below): + +``` +IGNEUMD= IGNEUM_HARNESS_BASE_PORT=29500 IGNEUM_HARNESS_TMP=/tmp/igneum-fud-mem/ \ + tools/lock/with-lock.sh run node tools/harness/run.mjs s6 s7 --quick --fast-time --live-only --no-bench-log +cd vendor/igneum-node-fud-mem && tools/lock/with-lock.sh build nice -n 19 cargo build --release -j 4 -p kaspad -p igneum-miner --features kaspad/igneum-pow +cd vendor/igneum-node-fud-mem && tools/lock/with-lock.sh build nice -n 19 cargo test --release -j 4 -p kaspa-pow --features kaspa-pow/igneum-pow -p igneum-exec +``` + +RSS per load, node A / node B, MB (start of the load to its peak; "builds" = `PoW cache built` lines during the load). s6 loads are 30 s each in `--quick`; s7 is the vmine miner at 50x for 60 s. + +| Load | Before: start to peak A / B | Before: builds A / B | After (pass 1, build without the insert guard): start to peak A / B | After: builds A / B | +|---|---|---|---|---| +| s6 warm-up baseline (RSS before any load) | 303 / 304 | 1 / 1 (startup) | 305 / 307 | 1 / 1 (startup) | 304 / 304 | +| s6 template flood, 500/s, 14,995 and 14,999 sent, all answered | 304 to 309 / 304 to 309 (+5 / +5) | 0 / 0 | 305 to 311 / 307 to 310 (+6 / +3) | 0 / 0 | 304 to 310 / 304 to 308 (+6 / +4) | +| s6 submit flood, 50/s, 1,499 known-block submits, all answered; the 60-DAA epoch rolled during it | 309 to 572 / 309 to 566 (+263 / +257) | 1 / 1 | 311 to 314 / 310 to 312 (+3 / +2) | 0 / 0 | 310 to 319 / 308 to 310 (+9 / +2) | +| s6 mempool flood, 500/s, 14,993 and 14,995 unknown-outpoint transactions, all rejected | 572 to 573 / 566 to 567 (+1 / +1) | 0 / 0 | 315 to 316 / 312 to 313 (+1 / +1) | 0 / 0 | 319 to 320 / 310 to 310 (+1 / +0) | +| s7 block flood, vmine at 50x for 60 s: 198 and 202 blocks accepted (3.3 and 3.4 per s), epochs 0 to 3 rolled in every run | 302 to 1,085 / 303 to 1,083 (+783 / +780); steps at 10 s 569, 20 s 827, 40 s 1,084 | 3 / 3 | 302 to 318 / 302 to 315 (+16 / +13) | 0 / 0 | 300 to 315 / 302 to 315 (+15 / +13) | + +Honest template p95 (upper bounds; the before run and pass 1 ran at load over 100, the committed-build run at load under 5 for s6 and about 10 for s7): before 130.8 / 86.7 / 104.7 ms per s6 load against a baseline of 84.7, s7 91.5 against 92.8; committed build 0.5 / 0.6 / 78.7 against 0.7, s7 81.3 against 104.6. Both nodes alive in every run. The s6 row's one-instant sink check failed in the before run (120 against 121 blocks) and in the committed-build run (121 against 122) while the honest miner was mid-submit, and passed in pass 1; the s7 sinks agreed in every run. That check reads the two sinks once without waiting (`waitSameSink` exists in `lib/net.mjs` and s6 does not use it), so it is a harness flake, not a node finding; left as is tonight. The red team's harness criterion (baseline + 512 MB) still passes before and after; the 50 MB-per-load target holds after. + +Harness changes (this repo): `IGNEUM_HARNESS_BASE_PORT` and `IGNEUM_HARNESS_TMP` (ports and data directory, so two agents can run the harness at once), the u64 sentinel round-trip fixed with the BigInt reviver from `tools/finality-attacks/lib/net.mjs` (ledger F25), s6 records `rss_start`, `rss_delta` and `cache_builds` per load and its row reports per-load growth, s7 records `cache_builds` beside every RSS sample, and `--live-only` skips the s7 simulator part. Result JSON: `docs/benchmarks/memory-floods-2026-10-04/{before,after,after-pass1}-{s6-exhaustion,s7-flood}.json` (`after` is the committed build, `after-pass1` the build before its last three-line guard: `insert_program` skips a program whose day cache was evicted during its generation, which cannot fire in these single-day runs). Data directories `/tmp/igneum-fud-mem/{baseline,after-pass1 as after,after2}`. + +Not covered tonight: the execution layer's `ExecState.records` (`igneum/exec/src/service.rs:31`, pushed per chain block, never truncated) is a slow growth, not a flood effect: 197 chain blocks cost under 1 MB in these runs, and a record on an empty devnet is roughly 1 to 2 KB (approximate, from the struct), so about 100 to 170 MB per day at 1 block/s; bounding it needs a window at least as long as the proving sortition window (`proving.rs:200`) plus the RPC's by-number history, which is a design choice, not a cache. The snapshot ring (`SNAPSHOT_RING = 64` full `IgneumDb` clones) is bounded in count but scales with the state size. The finality store trims votes, checkpoints, certificates and locks every index (`processes/finality.rs:531`); its `keys` and `stripped` maps grow with distinct vote keys (about 150 bytes per key, approximate). The proof pool keeps `RECORD_WINDOW_CHAIN_BLOCKS` of entries. None of these moved in these floods. + +Steady state, no flood (5 October 2026, 01:20 to 01:48 UTC, asked for after the live app node on this Mac was reported at 1,081 MB at 27 min, 1,193 MB at 79 min and 2,258 MB at 4 h 14 min on the devnet profile): the new scenario `tools/harness/scenarios/s8-steady.mjs`, two nodes on the 60x profile (60-DAA epochs, a 24-minute day), one honest vmine miner at 1 block/s on node A, node B following, 1,500 blocks, RSS and the `PoW cache built` count every 60 s, `vmmap -summary` of both nodes at 0, 500, 1,000 and 1,500 blocks. Both builds ran at the same time on their own run slots (the lock script's three run slots) and port ranges. The first 8 minutes ran at load 60 to 85, the rest at load 2 to 8; the counts do not depend on it. + +``` +IGNEUMD= IGNEUM_FAST_TIME=1 IGNEUM_HARNESS_BASE_PORT=<29500|29600> IGNEUM_HARNESS_TMP=/tmp/igneum-fud-mem/steady- \ + IGNEUM_STEADY_BLOCKS=1500 IGNEUM_STEADY_MAX_MIN=40 tools/lock/with-lock.sh run node tools/harness/scenarios/s8-steady.mjs +``` + +| Blocks (node A / B, both equal) | Before (shipping igneumd, 29500+): RSS A / B MB, cache builds | After (796f758d, 29600+): RSS A / B MB, cache builds | +|---|---|---| +| 0 (nodes up, miner not started) | 41 / 42, 0 | 41 / 42, 0 | +| 514 and 510 | 1,342 / 1,342, 9 | 319 / 317, 1 | +| 1,008 and 1,029 | 1,355 / 1,355, 18 | 589 / 588, 2 | +| 1,529 and 1,526 (end, 1,527 and 1,521 s) | 1,371 / 1,372, 27 | 603 / 602, 2 | +| Slope from 1,000 blocks to the end | 30.7 MB per 1,000 blocks | 30.2 MB per 1,000 blocks | + +Reading. Before: one cache build per epoch roll (25 rolls in 1,529 blocks, plus the two days) and the RSS steps with them up to five chunks: `KEEP = 4` plus one evicted 256 MiB chunk the allocator keeps and never returns to the OS (vmmap at 500, 1,000 and 1,500 blocks: 1.3 G resident in the region vmmap labels IOAccelerator, which held exactly the cache chunks; the malloc zones hold 9 to 22 MB). It stays at five: the before node is flat at 1,34x to 1,37x from 514 blocks on. After: one cache for the first day (319 MB at 510 blocks), a second when the fast-time day rolled at 01:36 UTC (both runs; the day cache is per day by design, `KEEP_DAYS = 3`), 2 of 2 builds in 1,526 blocks against 27 of 27 before; on the devnet profile (24-hour day, 1-hour epochs) that is 256 MiB flat, 512 MiB around midnight UTC, 768 MiB worst case, against 256 MiB per hour up to 1.3 GB before. So per 1,000 blocks: before, 1,300 MB in the first 500 blocks (the four caches plus the kept chunk) then 31 MB; after, 30 MB, plus 256 MiB once per day. The 30 MB per 1,000 blocks is the same on both builds and is not the PoW cache: at 1,526 blocks the after node's non-cache footprint is 584.0 M physical minus 2 x 256 MiB = 72 MB against 23 MB at 0 blocks, and the malloc zones account for 22 MB of it, so most of it is in large `vm_allocate` regions, which on this node means the consensus database's write buffers and block cache and the consensus in-memory caches filling toward their fixed sizes (rusty-kaspa sizes them in entries for mainnet), not the execution layer (1,526 chain-block records on an empty chain are about 2 to 3 MB, approximate) and not the finality key maps (1 voter here). That is a reading, not a measurement: it needs a longer run to see the plateau (the live node's own figures fit it: 1,081 to 1,193 MB over 52 minutes with no epoch roll is 36 MB per 1,000 blocks). The live app node's 2,258 MB at 4 h 14 min is more than the before build can hold on its own (five chunks plus 30 MB per 1,000 blocks gives about 1.7 GB at 15,000 blocks); the app runs a GPU worker beside the node (Metal buffers and a kernel per epoch), which this harness does not cover, so that node wants its own `vmmap -summary`. Result JSON `docs/benchmarks/memory-floods-2026-10-04/{before,after}-s8-steady.json`, vmmap summaries under `.../vmmap/`. + +## 4 October 2026 (night), round-4 consensus items F23, F24, G12, X18 and M31: unit tests and fast-time 3-node runs against the control (consensus engineer) + +Branches: node fork `fud-consensus` (worktree `vendor/igneum-node-fud`, from `finality-fixes` 6aa69a45, no remote; commits 9f738e2e the four items, ae9df8a3 pending certificates at fresh determinations, b755f43d merge of `fud-memory`, then M31 and the un-determination rule), main repo `fud-consensus`. Machine shared with the red team, the release builds and the memory runs all night (load average 100 to 146 until about 01:00 BST, under 20 after); every number here is a count, a lock or an index, not a timing. Runner `tools/finality-attacks/fud.mjs` (scenarios `digest`, `ban`, `reorg`; fast-time 60x profile with `finality_v3_activation_daa` 0 merged by the BigInt-safe `overrideParams`, ports 29400+, suffix 940, `/tmp/igneum-fin-fud`, node logs kept per scenario); the control is the shipping finality-fixes build `vendor/igneum-node/target-finality/release/igneumd` driven by the same miner. Raw result files in `docs/benchmarks/round4-consensus-2026-10-04/`. + +**Builds.** `with-lock.sh build nice -n 19 cargo build --release -j 4 -p kaspad -p igneum-miner --features kaspad/igneum-pow` on a target directory cloned from `target-finality` with `cp -Rc` (APFS clonefile, 1 min, no disk): 17 min 53 s the first time under load 140, 8 min 58 s the second. Unit tests in the release profile (`cargo test --release -j 4 -p --lib`): consensus-core `config::params::tests` and `igneum` 20 of 20 (new: `consensus_digest_covers_every_consensus_field_and_nothing_else`, `env_pow_schedule_is_devnet_and_simnet_only`), consensus `processes::finality` 6 of 6 (new: `ban_is_decided_by_the_carrying_block_so_nodes_agree_on_every_voter_list` with three `TestConsensus` nodes on one chain, `reorg_past_an_unlocked_checkpoint_re_determines_it_and_verifies_the_pending_certificate`, `a_locked_checkpoint_pins_the_chain_and_a_certificate_against_it_conflicts`). After M31 and the un-determination rule (fork 977db931, with the `fud-memory` merge): consensus-core params and igneum tests 28 of 28 then params 12 of 12 (new: `largest_coinbase_fits_on_every_network`), consensus `processes::finality` 7 of 7 (new: `a_shallower_sink_un_determines_the_indices_it_cannot_reach`), kaspa-pow with `igneum-pow` 12 of 12; the second rebuild took 15 min 18 s under load 110 to 134. + +**X18, the params digest** (`fud.mjs digest`: n1 listens on the shared override, n0 dials it with `finality.weight_window` 121 instead of 120, then n2 dials with the shared override). + +| Build | n0's digest | n1's digest | mismatch lines n0 / n1 / n2 | peers on n1 after 25 s, then after n2 dialled | n2 connected | +|---|---|---|---|---|---| +| fud-consensus (9f738e2e and the final pass) | 4bf763ba... | 7a40cc3b... | 1 to 2 / 2 / 0 | 0, then 1 | after 1 s | +| control, finality-fixes 6aa69a45 | none printed | none printed | 0 / 0 / 0 | 1, then 2 | after 1 s | + +The listener's line: `Refusing peer 127.0.0.1:...: consensus params digest mismatch, local 7a40cc3b... remote 4bf763ba... (the peer's override file, environment or build differs)`; the dialler sees the reject message with both digests. Two lines on the listener per pass because the dialler redials once within 25 s. The control connects the mismatched node and says nothing. + +**F23, the ban decided by the carrier** (`fud.mjs ban`: six voters at 1/6 of 1 block/s, two per node; `a0` on n0 equivocates once at index 9 (`vmine --equivocate-at 9`, the second vote reaches n0 over RPC only); P2 cut when n0's next index reaches 9 (252 to 259 s) and healed 45 s later, so n2 learns the evidence from the carrier block after the heal; 480 s). + +| Build, pass | EQUIVOCATION lines n0 / n1 / n2 (carried by block) | refused "names N voters" | CONFLICTING | indices with 5 voters, per node | voter counts agree / differ (indices with lines on 2+ nodes) | disagreeing locked indices | max locked | +|---|---|---|---|---|---|---|---| +| fud-consensus, first pass (9f738e2e) | 2 (1) / 1 (1) / 1 (1) | 0 / 0 / 0 | 0 / 0 / 0 | 10..13 on all three | 11 / 0 | 0 | 14 / 14 / 14 | +| fud-consensus, final pass (ae9df8a3) | 2 (1) / 1 (1) / 1 (1) | 0 / 0 / 0 | 0 / 0 / 0 | 10..12 on all three | 10 / 0 | 0 | 15 / 15 / 15 | +| control, finality-fixes | 2 (0) / 1 (0) / 1 (0) | 2 / 0 / 0 | 0 / 0 / 0 | 10..13 on all three | 9 / 0 | 0 | 14 / 14 / 14 | + +On the new build n2's one EQUIVOCATION line is the "carried by block" variety (it never saw the vote), and the stripped range is the same on the node that detected over RPC, the node that saw the carrier at once and the node that saw it 45 s late. The control refused two of the other nodes' certificates on n0 with "names N voters, this node counts M"; the red team's stock s1 (two keys equivocating at every index) gave 9 / 3 / 4 refusals on the same build. Locks still agreed on the control because each node could build its own certificate from the votes it held; the refusal is the defect, the disagreement would follow on a network where one node depends on another's certificate. + +**F24, re-determination after a deep reorg** (`fud.mjs reorg`: n0 holds `q0`, `q1` at 0.15 each, n1 and n2 hold `p0` to `p3` at 0.175 each, so the n1/n2 side has 70% of the weight; 230 s warm, P0 cut 180 s, healed, 150 s heal window). + +| Build, pass | n0 determined on its own chain during the split | re-determined lines on n0 | pending kept / verified on n0 | CONFLICTING | refused "is for X, this node's checkpoint is Y" (pre-F24 wording) | indices the majority locked that n0 did not | disagreeing locked indices | max locked at the end | +|---|---|---|---|---|---|---|---|---| +| fud-consensus, first pass (9f738e2e) | 1 (index 8) | 2 | 4 / not re-read at fresh determinations (the gap fixed in ae9df8a3) | 0 / 0 / 0 | 0 | 8 and 9 (no certificate was verified at them) | 0 | 15 / 15 / 15 | +| fud-consensus, final pass (ae9df8a3) | 2 (8, 9) | 2 | 2 / 2 | 0 / 0 / 0 | 0 | none (the majority locked 10 and 11 during the split, not 8 and 9: 70% nominal is Poisson noise away from the floor) | 0 | 16 / 16 / 16 | +| control, finality-fixes, two passes | 2 then 1 | 0 | 0 / 0 | 0 / 0 / 0 | 3 (second pass) | 8 and 9 (second pass): the permanent hole | 0 | 17 then 15 | +| fud-consensus after the un-determination rule (977db931, `reorg-final2`) | 1 (index 9) | 1 | 1 / 1 | 0 / 0 / 0 | 0 | none (the majority locked nothing during the split this time: 8 at the cut, 8 at the heal, 16 at the end on all three) | 0 | 16 / 16 / 16; no record below its target | + +What the final pass found: n0's index 9 was re-determined at a sink of blue score 263 to a block of blue score 263, below the index's target 270, because the majority chain became the sink by blue work (its difficulty drifted less than n0's during the split) before it had reached index 9's depth; a record never names a block below its target, so the rule now un-determines an index the new chain has not reached and determines it again when it has (fork commit after ae9df8a3, unit test `a_shallower_sink_un_determines_the_indices_it_cannot_reach`). The control's first pass logged no CONFLICTING because that build's refusal used other words ("certificate at index 8 is for X, this node's checkpoint is Y"), counted in the second pass. + +**The red team's own reproductions** (`tools/finality-attacks/redteam/rtfin.mjs`, fin-attacks miner at 6 blocks/s, run on the fud-consensus build ae9df8a3 through the main worktree's env-aware harness lib on ports 29550+; result files in `docs/benchmarks/round4-consensus-2026-10-04/redteam-repro/`): + +| Scenario | Build | Result | +|---|---|---| +| f23: two keys equivocating at every index, four honest voters, three nodes, 105 s (the evening's 9 / 3 / 4 refusals) | ae9df8a3 | PASS: equivocation detections 14 / 7 / 7, voter-count refusals 0 / 0 / 0, CONFLICTING 0 / 0 / 0, disagreeing locked indices 0, max locked 61 / 61 / 61 | +| f24c: 3/3 split, 16 s cut (96 DAA at 6 blocks/s), heal | ae9df8a3 | PASS: n1 determined 31..32 during the cut; after the heal 0 refusals, 0 CONFLICTING, 0 stuck indices, 0 disagreeing, max locked 61 / 61 | +| f24b: 4/2 split, 24 s cut, heal | ae9df8a3 and 977db931 | FAIL on both, outside F24: 24 s at 6 blocks/s is 144 DAA, longer than the 120-DAA window and the 60-DAA merge depth, so the chains never merge and each side locks its own chain alone ("100.0% of total, 100.0% of the table frozen at lock 39" on n1): the partition longer than a window of spec 3.7 item 9 (F21), mis-scaled by the scenario's assumption of 1 DAA a second. The 1,668 and 2,025 "PoW rejected" lines are the INFO line of `pre_ghostdag_validation.rs:158` for nonce-1 blocks, which `skip_proof_of_work` then accepts; no block was refused for them | + +**G12** is covered by the digest run (the environment's schedule is part of the digest, so a devnet node with `IGNEUM_POW_EPOCH_BLOCKS` set cannot connect to one without it) and by `env_pow_schedule_is_devnet_and_simnet_only`; no mainnet node was started tonight. **M31** is covered by `largest_coinbase_fits_on_every_network`; no simnet network was started tonight (the red team's `tools/exec-attacks/net.sh` is the run that would show templates on simnet without an override). + +**Uncertain.** (1) Every run is fast time (W = 120 DAA, ban 120, depth 20) on three nodes with 100-ms links; the mainnet values are 30 days, 30 days and 60 blocks. (2) The ban run shows one equivocation at one index; the red team's s1 (equivocation at every index, two keys) was re-run on the new build only through the red team's f23 above (0 refusals where the evening had 9 / 3 / 4). (3) The digest-less allowance on devnet and simnet is deliberate for the rollout and is a hole until removed. (4) The reorg run's final pass had the majority lock no index during the first 60 s of the split, so the re-determination at 8 and 9 was exercised, the pending-certificate path only at 10 and 11; the first pass exercised the opposite. (5) The un-determination rule has a unit test and one network pass (`reorg-final2`) in which the shallow-sink case did not recur, so the rule is exercised by the test, not by a run; the case needs a split whose difficulty drifts enough for blue work to overtake blue score, which happened once in four runs. (6) The red team's f24b is a window-length partition at 6 blocks/s, so it measures F21's stated limit, not F24; a 4/2 cut under 20 s at that rate would be the F24 case. diff --git a/docs/benchmarks/memory-floods-2026-10-04/after-pass1-s6-exhaustion.json b/docs/benchmarks/memory-floods-2026-10-04/after-pass1-s6-exhaustion.json new file mode 100644 index 000000000..7819733dd --- /dev/null +++ b/docs/benchmarks/memory-floods-2026-10-04/after-pass1-s6-exhaustion.json @@ -0,0 +1,413 @@ +{ + "rows": [ + { + "scenario": "6 resource exhaustion (50x template, submit and mempool floods from one peer)", + "criterion": "honest template p95 < 200 ms and both nodes under baseline RSS + 512 MB, alive, one sink", + "result": "honest template p95 worst 101.7 ms across loads (baseline 89.5 ms); template 500ps 500/s, submit 50ps 50/s, mempool 500ps 500/s; RSS growth per load template +6MB (0/0 cache builds), submit +9MB (0/0 cache builds), mempool +1MB (0/0 cache builds), cumulative +16MB over baseline 304/304; alive true; same sink true", + "pass": true + } + ], + "data": { + "baseline_template_ms": { + "a": { + "n": 75, + "p50": 6.5, + "p95": 89.6, + "p99": 96.4, + "max": 96.4, + "mean": 26.8 + }, + "b": { + "n": 76, + "p50": 10.1, + "p95": 89.5, + "p99": 107.5, + "max": 107.5, + "mean": 25.2 + } + }, + "rss_baseline": { + "a": 304, + "b": 304 + }, + "loads": { + "template_flood_500ps": { + "requests_sent": 14993, + "accepted": 14993, + "rejected_or_error": 0, + "rate_per_s": 500, + "request_latency_ms": { + "n": 14993, + "p50": 23.9, + "p95": 168.2, + "p99": 670.8, + "max": 807.1, + "mean": 54.4 + }, + "honest_template_ms": { + "n": 170, + "p50": 3.6, + "p95": 101.7, + "p99": 698.7, + "max": 700.7, + "mean": 44.8 + }, + "attacked_node_template_ms": { + "n": 167, + "p50": 4.3, + "p95": 93.7, + "p99": 698.4, + "max": 701, + "mean": 43.5 + }, + "rss_series": [ + { + "t_s": 2.673, + "a": 305, + "b": 304 + }, + { + "t_s": 4.69, + "a": 306, + "b": 305 + }, + { + "t_s": 6.706, + "a": 306, + "b": 305 + }, + { + "t_s": 8.707, + "a": 306, + "b": 305 + }, + { + "t_s": 10.71, + "a": 306, + "b": 306 + }, + { + "t_s": 12.723, + "a": 307, + "b": 306 + }, + { + "t_s": 14.727, + "a": 307, + "b": 306 + }, + { + "t_s": 16.728, + "a": 308, + "b": 306 + }, + { + "t_s": 18.748, + "a": 308, + "b": 307 + }, + { + "t_s": 20.749, + "a": 308, + "b": 307 + }, + { + "t_s": 22.752, + "a": 308, + "b": 307 + }, + { + "t_s": 24.753, + "a": 308, + "b": 307 + }, + { + "t_s": 26.754, + "a": 309, + "b": 307 + }, + { + "t_s": 28.754, + "a": 310, + "b": 308 + } + ], + "rss_start": { + "a": 304, + "b": 304 + }, + "rss_peak": { + "a": 310, + "b": 308 + }, + "both_alive": true, + "rss_delta": { + "a": 6, + "b": 4 + }, + "cache_builds": { + "a": 0, + "b": 0 + } + }, + "submit_flood_50ps": { + "requests_sent": 1499, + "accepted": 1499, + "rejected_or_error": 0, + "rate_per_s": 50, + "request_latency_ms": { + "n": 1499, + "p50": 387.7, + "p95": 1435.5, + "p99": 1887.7, + "max": 2166.4, + "mean": 481.3 + }, + "honest_template_ms": { + "n": 175, + "p50": 5, + "p95": 66.1, + "p99": 91.4, + "max": 94.1, + "mean": 19.9 + }, + "attacked_node_template_ms": { + "n": 174, + "p50": 6.2, + "p95": 67.4, + "p99": 90.2, + "max": 91.5, + "mean": 21.3 + }, + "rss_series": [ + { + "t_s": 2.49, + "a": 312, + "b": 309 + }, + { + "t_s": 4.49, + "a": 312, + "b": 309 + }, + { + "t_s": 6.491, + "a": 314, + "b": 309 + }, + { + "t_s": 8.493, + "a": 315, + "b": 309 + }, + { + "t_s": 10.497, + "a": 315, + "b": 309 + }, + { + "t_s": 12.505, + "a": 316, + "b": 309 + }, + { + "t_s": 14.513, + "a": 316, + "b": 309 + }, + { + "t_s": 16.524, + "a": 317, + "b": 309 + }, + { + "t_s": 18.525, + "a": 318, + "b": 309 + }, + { + "t_s": 20.525, + "a": 318, + "b": 309 + }, + { + "t_s": 22.525, + "a": 318, + "b": 309 + }, + { + "t_s": 24.544, + "a": 318, + "b": 309 + }, + { + "t_s": 26.544, + "a": 318, + "b": 310 + }, + { + "t_s": 28.544, + "a": 319, + "b": 310 + }, + { + "t_s": 30.545, + "a": 319, + "b": 310 + } + ], + "rss_start": { + "a": 310, + "b": 308 + }, + "rss_peak": { + "a": 319, + "b": 310 + }, + "both_alive": true, + "rss_delta": { + "a": 9, + "b": 2 + }, + "cache_builds": { + "a": 0, + "b": 0 + } + }, + "mempool_flood_500ps": { + "requests_sent": 14999, + "accepted": 0, + "rejected_or_error": 14999, + "rate_per_s": 500, + "request_latency_ms": { + "n": 14999, + "p50": 6.9, + "p95": 30.6, + "p99": 101.6, + "max": 823, + "mean": 13.2 + }, + "honest_template_ms": { + "n": 171, + "p50": 6.2, + "p95": 84.4, + "p99": 557, + "max": 665.4, + "mean": 32 + }, + "attacked_node_template_ms": { + "n": 171, + "p50": 7.4, + "p95": 74.2, + "p99": 103, + "max": 488.7, + "mean": 25.2 + }, + "rss_series": [ + { + "t_s": 2.649, + "a": 319, + "b": 310 + }, + { + "t_s": 4.65, + "a": 319, + "b": 310 + }, + { + "t_s": 6.65, + "a": 319, + "b": 310 + }, + { + "t_s": 8.649, + "a": 319, + "b": 310 + }, + { + "t_s": 10.65, + "a": 319, + "b": 310 + }, + { + "t_s": 12.662, + "a": 319, + "b": 310 + }, + { + "t_s": 14.662, + "a": 319, + "b": 310 + }, + { + "t_s": 16.662, + "a": 320, + "b": 310 + }, + { + "t_s": 18.663, + "a": 320, + "b": 310 + }, + { + "t_s": 20.736, + "a": 320, + "b": 310 + }, + { + "t_s": 22.748, + "a": 320, + "b": 310 + }, + { + "t_s": 24.749, + "a": 320, + "b": 310 + }, + { + "t_s": 26.75, + "a": 320, + "b": 310 + }, + { + "t_s": 28.749, + "a": 320, + "b": 310 + } + ], + "rss_start": { + "a": 319, + "b": 310 + }, + "rss_peak": { + "a": 320, + "b": 310 + }, + "both_alive": true, + "rss_delta": { + "a": 1, + "b": 0 + }, + "cache_builds": { + "a": 0, + "b": 0 + } + } + }, + "recovery_template_ms": { + "n": 210, + "p50": 6.2, + "p95": 82.5, + "p99": 488.6, + "max": 665.4, + "mean": 29.9 + }, + "final": { + "blocks_a": 121, + "blocks_b": 121, + "same_sink": true + }, + "alive": true, + "mem_bound_mb": 512 + } +} \ No newline at end of file diff --git a/docs/benchmarks/memory-floods-2026-10-04/after-pass1-s7-flood.json b/docs/benchmarks/memory-floods-2026-10-04/after-pass1-s7-flood.json new file mode 100644 index 000000000..807740080 --- /dev/null +++ b/docs/benchmarks/memory-floods-2026-10-04/after-pass1-s7-flood.json @@ -0,0 +1,163 @@ +{ + "rows": [ + { + "scenario": "7 fast-miner flood, controller trajectory (sim)", + "criterion": "trajectory recorded", + "result": "skipped (--live-only)", + "pass": null + }, + { + "scenario": "7 fast-miner flood, live (50 blocks/s from one peer)", + "criterion": "node stays responsive: honest template p95 < 200 ms, both nodes alive, same sink", + "result": "flood accepted 203 blocks in 60 s (3.4/s); honest template p50/p95/max 12.4/75.4/895.1 ms under flood (baseline 7/69.4/107.3); rss a 300->315 MB, b 302->315 MB (cache builds 0/0); alive true; same sink true", + "pass": true + } + ], + "data": { + "live": { + "baseline_template_ms": { + "n": 79, + "p50": 7, + "p95": 69.4, + "p99": 107.3, + "max": 107.3, + "mean": 24.1 + }, + "under_flood_template_ms": { + "n": 465, + "p50": 12.4, + "p95": 75.4, + "p99": 123.8, + "max": 895.1, + "mean": 27 + }, + "after_flood_template_ms": { + "n": 39, + "p50": 1.7, + "p95": 88.3, + "p99": 100.2, + "max": 100.2, + "mean": 23.1 + }, + "samples": [ + { + "t_s": 10, + "blocks_a": 110, + "blocks_b": 110, + "difficulty_a": 616132003.4293169, + "sink_same": true, + "rss_a": 312, + "rss_b": 311, + "cache_builds_a": 0, + "cache_builds_b": 0, + "flood_accepted": 100, + "flood_rejected": 0, + "honest_accepted": 10 + }, + { + "t_s": 20, + "blocks_a": 151, + "blocks_b": 150, + "difficulty_a": 1634372941.381798, + "sink_same": false, + "rss_a": 313, + "rss_b": 313, + "cache_builds_a": 0, + "cache_builds_b": 0, + "flood_accepted": 139, + "flood_rejected": 0, + "honest_accepted": 12 + }, + { + "t_s": 30, + "blocks_a": 176, + "blocks_b": 176, + "difficulty_a": 3422770765.9742208, + "sink_same": true, + "rss_a": 314, + "rss_b": 313, + "cache_builds_a": 0, + "cache_builds_b": 0, + "flood_accepted": 164, + "flood_rejected": 0, + "honest_accepted": 12 + }, + { + "t_s": 40, + "blocks_a": 197, + "blocks_b": 196, + "difficulty_a": 5026862033.766903, + "sink_same": false, + "rss_a": 314, + "rss_b": 314, + "cache_builds_a": 0, + "cache_builds_b": 0, + "flood_accepted": 184, + "flood_rejected": 0, + "honest_accepted": 13 + }, + { + "t_s": 50, + "blocks_a": 207, + "blocks_b": 207, + "difficulty_a": 6515322825.578815, + "sink_same": true, + "rss_a": 315, + "rss_b": 314, + "cache_builds_a": 0, + "cache_builds_b": 0, + "flood_accepted": 194, + "flood_rejected": 0, + "honest_accepted": 13 + }, + { + "t_s": 60, + "blocks_a": 216, + "blocks_b": 216, + "difficulty_a": 6949902898.525323, + "sink_same": true, + "rss_a": 315, + "rss_b": 315, + "cache_builds_a": 0, + "cache_builds_b": 0, + "flood_accepted": 203, + "flood_rejected": 0, + "honest_accepted": 13 + } + ], + "rss_before": { + "a": 300, + "b": 302 + }, + "rss_peak": { + "a": 315, + "b": 315 + }, + "cache_builds": { + "a": 0, + "b": 0, + "before_flood": { + "a": 1, + "b": 1 + } + }, + "flood": { + "accepted": 203, + "rejected": 0, + "errors": 0 + }, + "honest": { + "accepted": 13, + "rejected": 0 + }, + "final": { + "blocks_a": 216, + "blocks_b": 216, + "same_sink": true, + "difficulty_a": 6949902898.525323, + "ratio": null + }, + "alive": true + } + } +} \ No newline at end of file diff --git a/docs/benchmarks/memory-floods-2026-10-04/after-s6-exhaustion.json b/docs/benchmarks/memory-floods-2026-10-04/after-s6-exhaustion.json new file mode 100644 index 000000000..d4d3170c3 --- /dev/null +++ b/docs/benchmarks/memory-floods-2026-10-04/after-s6-exhaustion.json @@ -0,0 +1,408 @@ +{ + "rows": [ + { + "scenario": "6 resource exhaustion (50x template, submit and mempool floods from one peer)", + "criterion": "honest template p95 < 200 ms and both nodes under baseline RSS + 512 MB, alive, one sink", + "result": "honest template p95 worst 78.7 ms across loads (baseline 0.7 ms); template 500ps 500/s, submit 50ps 50/s, mempool 500ps 500/s; RSS growth per load template +6MB (0/0 cache builds), submit +3MB (0/0 cache builds), mempool +1MB (0/0 cache builds), cumulative +11MB over baseline 305/307; alive true; same sink false", + "pass": false + } + ], + "data": { + "baseline_template_ms": { + "a": { + "n": 98, + "p50": 0.4, + "p95": 0.6, + "p99": 0.9, + "max": 0.9, + "mean": 0.4 + }, + "b": { + "n": 98, + "p50": 0.5, + "p95": 0.7, + "p99": 1.5, + "max": 1.5, + "mean": 0.5 + } + }, + "rss_baseline": { + "a": 305, + "b": 307 + }, + "loads": { + "template_flood_500ps": { + "requests_sent": 14999, + "accepted": 14999, + "rejected_or_error": 0, + "rate_per_s": 500, + "request_latency_ms": { + "n": 14999, + "p50": 0.2, + "p95": 0.7, + "p99": 3.7, + "max": 102.4, + "mean": 0.4 + }, + "honest_template_ms": { + "n": 285, + "p50": 0.3, + "p95": 0.5, + "p99": 1.4, + "max": 1.9, + "mean": 0.3 + }, + "attacked_node_template_ms": { + "n": 285, + "p50": 0.3, + "p95": 0.4, + "p99": 0.9, + "max": 1.7, + "mean": 0.3 + }, + "rss_series": [ + { + "t_s": 2.102, + "a": 305, + "b": 308 + }, + { + "t_s": 4.102, + "a": 306, + "b": 308 + }, + { + "t_s": 6.101, + "a": 306, + "b": 309 + }, + { + "t_s": 8.102, + "a": 307, + "b": 309 + }, + { + "t_s": 10.102, + "a": 307, + "b": 310 + }, + { + "t_s": 12.102, + "a": 308, + "b": 310 + }, + { + "t_s": 14.103, + "a": 309, + "b": 310 + }, + { + "t_s": 16.103, + "a": 309, + "b": 310 + }, + { + "t_s": 18.104, + "a": 310, + "b": 310 + }, + { + "t_s": 20.104, + "a": 310, + "b": 310 + }, + { + "t_s": 22.104, + "a": 310, + "b": 310 + }, + { + "t_s": 24.104, + "a": 310, + "b": 310 + }, + { + "t_s": 26.104, + "a": 311, + "b": 310 + }, + { + "t_s": 28.104, + "a": 311, + "b": 310 + } + ], + "rss_start": { + "a": 305, + "b": 307 + }, + "rss_peak": { + "a": 311, + "b": 310 + }, + "both_alive": true, + "rss_delta": { + "a": 6, + "b": 3 + }, + "cache_builds": { + "a": 0, + "b": 0 + } + }, + "submit_flood_50ps": { + "requests_sent": 1499, + "accepted": 1499, + "rejected_or_error": 0, + "rate_per_s": 50, + "request_latency_ms": { + "n": 1499, + "p50": 0.7, + "p95": 1.4, + "p99": 3.6, + "max": 5.4, + "mean": 0.8 + }, + "honest_template_ms": { + "n": 285, + "p50": 0.3, + "p95": 0.6, + "p99": 0.8, + "max": 0.8, + "mean": 0.4 + }, + "attacked_node_template_ms": { + "n": 285, + "p50": 0.4, + "p95": 0.6, + "p99": 0.8, + "max": 0.8, + "mean": 0.4 + }, + "rss_series": [ + { + "t_s": 2.114, + "a": 311, + "b": 310 + }, + { + "t_s": 4.114, + "a": 312, + "b": 311 + }, + { + "t_s": 6.114, + "a": 312, + "b": 311 + }, + { + "t_s": 8.114, + "a": 312, + "b": 311 + }, + { + "t_s": 10.114, + "a": 313, + "b": 311 + }, + { + "t_s": 12.114, + "a": 313, + "b": 311 + }, + { + "t_s": 14.115, + "a": 313, + "b": 311 + }, + { + "t_s": 16.115, + "a": 313, + "b": 311 + }, + { + "t_s": 18.116, + "a": 313, + "b": 311 + }, + { + "t_s": 20.117, + "a": 314, + "b": 311 + }, + { + "t_s": 22.117, + "a": 314, + "b": 311 + }, + { + "t_s": 24.117, + "a": 314, + "b": 312 + }, + { + "t_s": 26.117, + "a": 314, + "b": 312 + }, + { + "t_s": 28.118, + "a": 314, + "b": 312 + } + ], + "rss_start": { + "a": 311, + "b": 310 + }, + "rss_peak": { + "a": 314, + "b": 312 + }, + "both_alive": true, + "rss_delta": { + "a": 3, + "b": 2 + }, + "cache_builds": { + "a": 0, + "b": 0 + } + }, + "mempool_flood_500ps": { + "requests_sent": 14995, + "accepted": 0, + "rejected_or_error": 14995, + "rate_per_s": 500, + "request_latency_ms": { + "n": 14995, + "p50": 4.5, + "p95": 23.2, + "p99": 42.3, + "max": 812.8, + "mean": 8.7 + }, + "honest_template_ms": { + "n": 187, + "p50": 5.3, + "p95": 78.7, + "p99": 629.7, + "max": 832.9, + "mean": 31 + }, + "attacked_node_template_ms": { + "n": 194, + "p50": 4.9, + "p95": 65.4, + "p99": 528.9, + "max": 833, + "mean": 26.4 + }, + "rss_series": [ + { + "t_s": 2.228, + "a": 315, + "b": 312 + }, + { + "t_s": 4.229, + "a": 315, + "b": 312 + }, + { + "t_s": 6.229, + "a": 315, + "b": 312 + }, + { + "t_s": 8.231, + "a": 315, + "b": 312 + }, + { + "t_s": 10.242, + "a": 315, + "b": 312 + }, + { + "t_s": 12.241, + "a": 316, + "b": 312 + }, + { + "t_s": 14.242, + "a": 316, + "b": 312 + }, + { + "t_s": 16.243, + "a": 316, + "b": 312 + }, + { + "t_s": 18.243, + "a": 316, + "b": 312 + }, + { + "t_s": 20.251, + "a": 316, + "b": 313 + }, + { + "t_s": 22.251, + "a": 316, + "b": 313 + }, + { + "t_s": 24.269, + "a": 316, + "b": 313 + }, + { + "t_s": 26.269, + "a": 316, + "b": 313 + }, + { + "t_s": 28.275, + "a": 316, + "b": 313 + } + ], + "rss_start": { + "a": 315, + "b": 312 + }, + "rss_peak": { + "a": 316, + "b": 313 + }, + "both_alive": true, + "rss_delta": { + "a": 1, + "b": 1 + }, + "cache_builds": { + "a": 0, + "b": 0 + } + } + }, + "recovery_template_ms": { + "n": 226, + "p50": 6.1, + "p95": 78.7, + "p99": 543.6, + "max": 832.9, + "mean": 29.8 + }, + "final": { + "blocks_a": 121, + "blocks_b": 122, + "same_sink": false + }, + "alive": true, + "mem_bound_mb": 512 + } +} \ No newline at end of file diff --git a/docs/benchmarks/memory-floods-2026-10-04/after-s7-flood.json b/docs/benchmarks/memory-floods-2026-10-04/after-s7-flood.json new file mode 100644 index 000000000..53279e5bd --- /dev/null +++ b/docs/benchmarks/memory-floods-2026-10-04/after-s7-flood.json @@ -0,0 +1,163 @@ +{ + "rows": [ + { + "scenario": "7 fast-miner flood, controller trajectory (sim)", + "criterion": "trajectory recorded", + "result": "skipped (--live-only)", + "pass": null + }, + { + "scenario": "7 fast-miner flood, live (50 blocks/s from one peer)", + "criterion": "node stays responsive: honest template p95 < 200 ms, both nodes alive, same sink", + "result": "flood accepted 202 blocks in 60 s (3.4/s); honest template p50/p95/max 5.9/81.3/1252.2 ms under flood (baseline 12.6/104.6/120.5); rss a 302->318 MB, b 302->315 MB (cache builds 0/0); alive true; same sink true", + "pass": true + } + ], + "data": { + "live": { + "baseline_template_ms": { + "n": 76, + "p50": 12.6, + "p95": 104.6, + "p99": 120.5, + "max": 120.5, + "mean": 27.8 + }, + "under_flood_template_ms": { + "n": 469, + "p50": 5.9, + "p95": 81.3, + "p99": 150, + "max": 1252.2, + "mean": 30 + }, + "after_flood_template_ms": { + "n": 39, + "p50": 2.8, + "p95": 121.2, + "p99": 1106.2, + "max": 1106.2, + "mean": 50 + }, + "samples": [ + { + "t_s": 10, + "blocks_a": 124, + "blocks_b": 124, + "difficulty_a": 687251814.6297691, + "sink_same": true, + "rss_a": 313, + "rss_b": 311, + "cache_builds_a": 0, + "cache_builds_b": 0, + "flood_accepted": 103, + "flood_rejected": 0, + "honest_accepted": 21 + }, + { + "t_s": 20, + "blocks_a": 166, + "blocks_b": 166, + "difficulty_a": 2176913973.013581, + "sink_same": true, + "rss_a": 314, + "rss_b": 313, + "cache_builds_a": 0, + "cache_builds_b": 0, + "flood_accepted": 144, + "flood_rejected": 0, + "honest_accepted": 22 + }, + { + "t_s": 30, + "blocks_a": 187, + "blocks_b": 187, + "difficulty_a": 3197271306.1776547, + "sink_same": true, + "rss_a": 316, + "rss_b": 314, + "cache_builds_a": 0, + "cache_builds_b": 0, + "flood_accepted": 164, + "flood_rejected": 0, + "honest_accepted": 23 + }, + { + "t_s": 40, + "blocks_a": 208, + "blocks_b": 208, + "difficulty_a": 5948600103.681134, + "sink_same": true, + "rss_a": 317, + "rss_b": 314, + "cache_builds_a": 0, + "cache_builds_b": 0, + "flood_accepted": 185, + "flood_rejected": 0, + "honest_accepted": 23 + }, + { + "t_s": 50, + "blocks_a": 218, + "blocks_b": 218, + "difficulty_a": 6763165668.73272, + "sink_same": true, + "rss_a": 317, + "rss_b": 314, + "cache_builds_a": 0, + "cache_builds_b": 0, + "flood_accepted": 193, + "flood_rejected": 0, + "honest_accepted": 25 + }, + { + "t_s": 60, + "blocks_a": 228, + "blocks_b": 228, + "difficulty_a": 6313391779.974011, + "sink_same": true, + "rss_a": 318, + "rss_b": 315, + "cache_builds_a": 0, + "cache_builds_b": 0, + "flood_accepted": 202, + "flood_rejected": 0, + "honest_accepted": 26 + } + ], + "rss_before": { + "a": 302, + "b": 302 + }, + "rss_peak": { + "a": 318, + "b": 315 + }, + "cache_builds": { + "a": 0, + "b": 0, + "before_flood": { + "a": 1, + "b": 1 + } + }, + "flood": { + "accepted": 202, + "rejected": 0, + "errors": 1 + }, + "honest": { + "accepted": 26, + "rejected": 0 + }, + "final": { + "blocks_a": 228, + "blocks_b": 228, + "same_sink": true, + "difficulty_a": 6313391779.974011, + "ratio": null + }, + "alive": true + } + } +} \ No newline at end of file diff --git a/docs/benchmarks/memory-floods-2026-10-04/after-s8-steady.json b/docs/benchmarks/memory-floods-2026-10-04/after-s8-steady.json new file mode 100644 index 000000000..ae182933d --- /dev/null +++ b/docs/benchmarks/memory-floods-2026-10-04/after-s8-steady.json @@ -0,0 +1,506 @@ +{ + "rows": [ + { + "scenario": "8 steady state, one honest miner at 1 block/s, no flood (RSS per 1,000 blocks)", + "criterion": "recorded: RSS of the mining node / the follower at 0, 500, 1,000 and 1,500 blocks, cache builds", + "result": "0: 41/42 MB at 0 blocks (0/0 builds); 500: 319/317 MB at 510 blocks (1/1 builds); 1000: 589/588 MB at 1029 blocks (2/2 builds); 1500: 603/602 MB at 1526 blocks (2/2 builds); end 1526 blocks in 1521 s: 604/602 MB; honest accepted 1526 rejected 0 errors 0; alive true", + "pass": null + } + ], + "data": { + "samples": [ + { + "t_s": 0, + "blocks_a": 0, + "blocks_b": 0, + "daa_a": 0, + "rss_a": 41, + "rss_b": 42, + "cache_builds_a": 0, + "cache_builds_b": 0, + "same_sink": true, + "load": "70.63 67.18 58.38", + "vmmap_a": { + "physical_footprint": "22.8M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8a-0.txt" + }, + "vmmap_b": { + "physical_footprint": "23.0M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8b-0.txt" + } + }, + { + "t_s": 69, + "blocks_a": 80, + "blocks_b": 80, + "daa_a": 80, + "rss_a": 305, + "rss_b": 305, + "cache_builds_a": 1, + "cache_builds_b": 1, + "same_sink": true, + "load": "83.54 72.66 61.20" + }, + { + "t_s": 130, + "blocks_a": 143, + "blocks_b": 143, + "daa_a": 143, + "rss_a": 307, + "rss_b": 307, + "cache_builds_a": 1, + "cache_builds_b": 1, + "same_sink": true, + "load": "85.99 75.42 63.01" + }, + { + "t_s": 191, + "blocks_a": 204, + "blocks_b": 203, + "daa_a": 204, + "rss_a": 311, + "rss_b": 309, + "cache_builds_a": 1, + "cache_builds_b": 1, + "same_sink": false, + "load": "78.72 75.80 64.03" + }, + { + "t_s": 251, + "blocks_a": 269, + "blocks_b": 269, + "daa_a": 269, + "rss_a": 313, + "rss_b": 311, + "cache_builds_a": 1, + "cache_builds_b": 1, + "same_sink": true, + "load": "32.71 63.22 60.12" + }, + { + "t_s": 311, + "blocks_a": 332, + "blocks_b": 332, + "daa_a": 332, + "rss_a": 314, + "rss_b": 313, + "cache_builds_a": 1, + "cache_builds_b": 1, + "same_sink": true, + "load": "13.79 52.21 56.21" + }, + { + "t_s": 371, + "blocks_a": 392, + "blocks_b": 392, + "daa_a": 392, + "rss_a": 316, + "rss_b": 314, + "cache_builds_a": 1, + "cache_builds_b": 1, + "same_sink": true, + "load": "6.30 43.08 52.52" + }, + { + "t_s": 431, + "blocks_a": 456, + "blocks_b": 456, + "daa_a": 456, + "rss_a": 317, + "rss_b": 316, + "cache_builds_a": 1, + "cache_builds_b": 1, + "same_sink": true, + "load": "4.83 35.97 49.22" + }, + { + "t_s": 492, + "blocks_a": 510, + "blocks_b": 510, + "daa_a": 510, + "rss_a": 319, + "rss_b": 317, + "cache_builds_a": 1, + "cache_builds_b": 1, + "same_sink": true, + "load": "3.65 29.96 46.07", + "vmmap_a": { + "physical_footprint": "299.2M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8a-500.txt" + }, + "vmmap_b": { + "physical_footprint": "298.4M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8b-500.txt" + } + }, + { + "t_s": 554, + "blocks_a": 571, + "blocks_b": 571, + "daa_a": 571, + "rss_a": 320, + "rss_b": 319, + "cache_builds_a": 1, + "cache_builds_b": 1, + "same_sink": true, + "load": "3.15 24.72 42.91" + }, + { + "t_s": 614, + "blocks_a": 635, + "blocks_b": 635, + "daa_a": 635, + "rss_a": 322, + "rss_b": 320, + "cache_builds_a": 1, + "cache_builds_b": 1, + "same_sink": true, + "load": "3.39 20.87 40.23" + }, + { + "t_s": 674, + "blocks_a": 687, + "blocks_b": 687, + "daa_a": 687, + "rss_a": 323, + "rss_b": 322, + "cache_builds_a": 1, + "cache_builds_b": 1, + "same_sink": true, + "load": "2.77 17.54 37.67" + }, + { + "t_s": 734, + "blocks_a": 746, + "blocks_b": 746, + "daa_a": 746, + "rss_a": 325, + "rss_b": 323, + "cache_builds_a": 1, + "cache_builds_b": 1, + "same_sink": true, + "load": "4.99 15.57 35.56" + }, + { + "t_s": 795, + "blocks_a": 797, + "blocks_b": 797, + "daa_a": 797, + "rss_a": 326, + "rss_b": 325, + "cache_builds_a": 1, + "cache_builds_b": 1, + "same_sink": true, + "load": "4.22 13.41 33.39" + }, + { + "t_s": 855, + "blocks_a": 864, + "blocks_b": 864, + "daa_a": 864, + "rss_a": 328, + "rss_b": 326, + "cache_builds_a": 1, + "cache_builds_b": 1, + "same_sink": true, + "load": "5.90 12.34 31.64" + }, + { + "t_s": 915, + "blocks_a": 920, + "blocks_b": 920, + "daa_a": 920, + "rss_a": 330, + "rss_b": 328, + "cache_builds_a": 1, + "cache_builds_b": 1, + "same_sink": true, + "load": "4.99 10.91 29.79" + }, + { + "t_s": 975, + "blocks_a": 981, + "blocks_b": 981, + "daa_a": 981, + "rss_a": 588, + "rss_b": 586, + "cache_builds_a": 2, + "cache_builds_b": 2, + "same_sink": true, + "load": "4.12 9.61 28.02" + }, + { + "t_s": 1035, + "blocks_a": 1029, + "blocks_b": 1029, + "daa_a": 1029, + "rss_a": 589, + "rss_b": 588, + "cache_builds_a": 2, + "cache_builds_b": 2, + "same_sink": true, + "load": "3.05 8.28 26.26", + "vmmap_a": { + "physical_footprint": "569.6M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8a-1000.txt" + }, + "vmmap_b": { + "physical_footprint": "568.6M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8b-1000.txt" + } + }, + { + "t_s": 1098, + "blocks_a": 1095, + "blocks_b": 1095, + "daa_a": 1095, + "rss_a": 591, + "rss_b": 589, + "cache_builds_a": 2, + "cache_builds_b": 2, + "same_sink": true, + "load": "2.31 7.01 24.46" + }, + { + "t_s": 1158, + "blocks_a": 1163, + "blocks_b": 1163, + "daa_a": 1163, + "rss_a": 592, + "rss_b": 591, + "cache_builds_a": 2, + "cache_builds_b": 2, + "same_sink": true, + "load": "2.19 6.11 22.93" + }, + { + "t_s": 1218, + "blocks_a": 1219, + "blocks_b": 1219, + "daa_a": 1219, + "rss_a": 594, + "rss_b": 592, + "cache_builds_a": 2, + "cache_builds_b": 2, + "same_sink": true, + "load": "1.84 5.33 21.50" + }, + { + "t_s": 1279, + "blocks_a": 1281, + "blocks_b": 1281, + "daa_a": 1281, + "rss_a": 596, + "rss_b": 594, + "cache_builds_a": 2, + "cache_builds_b": 2, + "same_sink": true, + "load": "1.90 4.76 20.18" + }, + { + "t_s": 1339, + "blocks_a": 1341, + "blocks_b": 1341, + "daa_a": 1341, + "rss_a": 597, + "rss_b": 596, + "cache_builds_a": 2, + "cache_builds_b": 2, + "same_sink": true, + "load": "1.25 4.04 18.86" + }, + { + "t_s": 1399, + "blocks_a": 1406, + "blocks_b": 1406, + "daa_a": 1406, + "rss_a": 599, + "rss_b": 598, + "cache_builds_a": 2, + "cache_builds_b": 2, + "same_sink": true, + "load": "1.50 3.58 17.68" + }, + { + "t_s": 1459, + "blocks_a": 1465, + "blocks_b": 1465, + "daa_a": 1465, + "rss_a": 601, + "rss_b": 600, + "cache_builds_a": 2, + "cache_builds_b": 2, + "same_sink": true, + "load": "2.19 3.41 16.65" + }, + { + "t_s": 1519, + "blocks_a": 1526, + "blocks_b": 1526, + "daa_a": 1526, + "rss_a": 603, + "rss_b": 602, + "cache_builds_a": 2, + "cache_builds_b": 2, + "same_sink": true, + "load": "2.04 3.14 15.65", + "vmmap_a": { + "physical_footprint": "584.0M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8a-1500.txt" + }, + "vmmap_b": { + "physical_footprint": "582.8M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8b-1500.txt" + } + }, + { + "t_s": 1521, + "blocks_a": 1526, + "blocks_b": 1526, + "daa_a": 1526, + "rss_a": 604, + "rss_b": 602, + "cache_builds_a": 2, + "cache_builds_b": 2, + "same_sink": true, + "load": "2.04 3.14 15.65", + "vmmap_a": { + "physical_footprint": "584.0M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8a-end.txt" + }, + "vmmap_b": { + "physical_footprint": "582.8M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8b-end.txt" + } + } + ], + "milestones": { + "0": { + "t_s": 0, + "blocks_a": 0, + "blocks_b": 0, + "daa_a": 0, + "rss_a": 41, + "rss_b": 42, + "cache_builds_a": 0, + "cache_builds_b": 0, + "same_sink": true, + "load": "70.63 67.18 58.38", + "vmmap_a": { + "physical_footprint": "22.8M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8a-0.txt" + }, + "vmmap_b": { + "physical_footprint": "23.0M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8b-0.txt" + } + }, + "500": { + "t_s": 492, + "blocks_a": 510, + "blocks_b": 510, + "daa_a": 510, + "rss_a": 319, + "rss_b": 317, + "cache_builds_a": 1, + "cache_builds_b": 1, + "same_sink": true, + "load": "3.65 29.96 46.07", + "vmmap_a": { + "physical_footprint": "299.2M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8a-500.txt" + }, + "vmmap_b": { + "physical_footprint": "298.4M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8b-500.txt" + } + }, + "1000": { + "t_s": 1035, + "blocks_a": 1029, + "blocks_b": 1029, + "daa_a": 1029, + "rss_a": 589, + "rss_b": 588, + "cache_builds_a": 2, + "cache_builds_b": 2, + "same_sink": true, + "load": "3.05 8.28 26.26", + "vmmap_a": { + "physical_footprint": "569.6M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8a-1000.txt" + }, + "vmmap_b": { + "physical_footprint": "568.6M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8b-1000.txt" + } + }, + "1500": { + "t_s": 1519, + "blocks_a": 1526, + "blocks_b": 1526, + "daa_a": 1526, + "rss_a": 603, + "rss_b": 602, + "cache_builds_a": 2, + "cache_builds_b": 2, + "same_sink": true, + "load": "2.04 3.14 15.65", + "vmmap_a": { + "physical_footprint": "584.0M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8a-1500.txt" + }, + "vmmap_b": { + "physical_footprint": "582.8M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8b-1500.txt" + } + }, + "end": { + "t_s": 1521, + "blocks_a": 1526, + "blocks_b": 1526, + "daa_a": 1526, + "rss_a": 604, + "rss_b": 602, + "cache_builds_a": 2, + "cache_builds_b": 2, + "same_sink": true, + "load": "2.04 3.14 15.65", + "vmmap_a": { + "physical_footprint": "584.0M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8a-end.txt" + }, + "vmmap_b": { + "physical_footprint": "582.8M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-after/vmmap/s8b-end.txt" + } + } + }, + "honest": { + "accepted": 1526, + "rejected": 0, + "errors": 0 + }, + "alive": true + } +} \ No newline at end of file diff --git a/docs/benchmarks/memory-floods-2026-10-04/before-s6-exhaustion.json b/docs/benchmarks/memory-floods-2026-10-04/before-s6-exhaustion.json new file mode 100644 index 000000000..5e4e88133 --- /dev/null +++ b/docs/benchmarks/memory-floods-2026-10-04/before-s6-exhaustion.json @@ -0,0 +1,408 @@ +{ + "rows": [ + { + "scenario": "6 resource exhaustion (50x template, submit and mempool floods from one peer)", + "criterion": "honest template p95 < 200 ms and both nodes under baseline RSS + 512 MB, alive, one sink", + "result": "honest template p95 worst 130.8 ms across loads (baseline 84.7 ms); template 500ps 500/s, submit 50ps 50/s, mempool 500ps 500/s; RSS growth per load template +5MB (0/0 cache builds), submit +263MB (1/1 cache builds), mempool +1MB (0/0 cache builds), cumulative +270MB over baseline 303/304; alive true; same sink false", + "pass": false + } + ], + "data": { + "baseline_template_ms": { + "a": { + "n": 77, + "p50": 3.8, + "p95": 81.5, + "p99": 84.5, + "max": 84.5, + "mean": 25.6 + }, + "b": { + "n": 76, + "p50": 5.5, + "p95": 84.7, + "p99": 104.2, + "max": 104.2, + "mean": 27.1 + } + }, + "rss_baseline": { + "a": 303, + "b": 304 + }, + "loads": { + "template_flood_500ps": { + "requests_sent": 14995, + "accepted": 14995, + "rejected_or_error": 0, + "rate_per_s": 500, + "request_latency_ms": { + "n": 14995, + "p50": 36.7, + "p95": 439.3, + "p99": 1215.9, + "max": 1501.2, + "mean": 109.1 + }, + "honest_template_ms": { + "n": 159, + "p50": 11, + "p95": 130.8, + "p99": 754, + "max": 858.2, + "mean": 47 + }, + "attacked_node_template_ms": { + "n": 160, + "p50": 15.7, + "p95": 130.3, + "p99": 760.6, + "max": 858.1, + "mean": 50.2 + }, + "rss_series": [ + { + "t_s": 2.695, + "a": 305, + "b": 306 + }, + { + "t_s": 4.705, + "a": 306, + "b": 306 + }, + { + "t_s": 6.706, + "a": 307, + "b": 307 + }, + { + "t_s": 8.706, + "a": 307, + "b": 307 + }, + { + "t_s": 10.706, + "a": 307, + "b": 307 + }, + { + "t_s": 12.708, + "a": 307, + "b": 307 + }, + { + "t_s": 14.714, + "a": 308, + "b": 308 + }, + { + "t_s": 16.723, + "a": 308, + "b": 308 + }, + { + "t_s": 18.724, + "a": 308, + "b": 308 + }, + { + "t_s": 20.735, + "a": 309, + "b": 308 + }, + { + "t_s": 22.734, + "a": 309, + "b": 308 + }, + { + "t_s": 24.736, + "a": 309, + "b": 308 + }, + { + "t_s": 26.736, + "a": 309, + "b": 308 + }, + { + "t_s": 28.736, + "a": 309, + "b": 309 + } + ], + "rss_start": { + "a": 304, + "b": 304 + }, + "rss_peak": { + "a": 309, + "b": 309 + }, + "both_alive": true, + "rss_delta": { + "a": 5, + "b": 5 + }, + "cache_builds": { + "a": 0, + "b": 0 + } + }, + "submit_flood_50ps": { + "requests_sent": 1499, + "accepted": 1499, + "rejected_or_error": 0, + "rate_per_s": 50, + "request_latency_ms": { + "n": 1499, + "p50": 401.9, + "p95": 1120.6, + "p99": 1405.9, + "max": 1610.1, + "mean": 453.6 + }, + "honest_template_ms": { + "n": 166, + "p50": 5.9, + "p95": 86.7, + "p99": 600.8, + "max": 689.4, + "mean": 36.7 + }, + "attacked_node_template_ms": { + "n": 166, + "p50": 6.6, + "p95": 84.9, + "p99": 689.3, + "max": 743.1, + "mean": 38.1 + }, + "rss_series": [ + { + "t_s": 2.743, + "a": 310, + "b": 309 + }, + { + "t_s": 4.752, + "a": 311, + "b": 309 + }, + { + "t_s": 6.802, + "a": 311, + "b": 565 + }, + { + "t_s": 8.804, + "a": 311, + "b": 565 + }, + { + "t_s": 10.804, + "a": 568, + "b": 565 + }, + { + "t_s": 12.805, + "a": 568, + "b": 565 + }, + { + "t_s": 14.812, + "a": 569, + "b": 566 + }, + { + "t_s": 16.817, + "a": 569, + "b": 566 + }, + { + "t_s": 18.823, + "a": 569, + "b": 566 + }, + { + "t_s": 20.823, + "a": 570, + "b": 566 + }, + { + "t_s": 22.829, + "a": 571, + "b": 566 + }, + { + "t_s": 24.829, + "a": 571, + "b": 566 + }, + { + "t_s": 26.829, + "a": 571, + "b": 566 + }, + { + "t_s": 28.845, + "a": 572, + "b": 566 + } + ], + "rss_start": { + "a": 309, + "b": 309 + }, + "rss_peak": { + "a": 572, + "b": 566 + }, + "both_alive": true, + "rss_delta": { + "a": 263, + "b": 257 + }, + "cache_builds": { + "a": 1, + "b": 1 + } + }, + "mempool_flood_500ps": { + "requests_sent": 14993, + "accepted": 0, + "rejected_or_error": 14993, + "rate_per_s": 500, + "request_latency_ms": { + "n": 14993, + "p50": 7.1, + "p95": 49.9, + "p99": 126.6, + "max": 815.9, + "mean": 15.8 + }, + "honest_template_ms": { + "n": 164, + "p50": 7.9, + "p95": 104.7, + "p99": 735.5, + "max": 815.8, + "mean": 44.5 + }, + "attacked_node_template_ms": { + "n": 166, + "p50": 6.8, + "p95": 90.8, + "p99": 745.9, + "max": 815.9, + "mean": 37.3 + }, + "rss_series": [ + { + "t_s": 2.533, + "a": 572, + "b": 566 + }, + { + "t_s": 4.534, + "a": 572, + "b": 566 + }, + { + "t_s": 6.537, + "a": 572, + "b": 566 + }, + { + "t_s": 8.546, + "a": 572, + "b": 566 + }, + { + "t_s": 10.546, + "a": 572, + "b": 566 + }, + { + "t_s": 12.547, + "a": 572, + "b": 566 + }, + { + "t_s": 14.548, + "a": 572, + "b": 566 + }, + { + "t_s": 16.549, + "a": 573, + "b": 566 + }, + { + "t_s": 18.548, + "a": 573, + "b": 566 + }, + { + "t_s": 20.559, + "a": 573, + "b": 567 + }, + { + "t_s": 22.57, + "a": 573, + "b": 567 + }, + { + "t_s": 24.57, + "a": 573, + "b": 567 + }, + { + "t_s": 26.57, + "a": 573, + "b": 567 + }, + { + "t_s": 28.57, + "a": 573, + "b": 567 + } + ], + "rss_start": { + "a": 572, + "b": 566 + }, + "rss_peak": { + "a": 573, + "b": 567 + }, + "both_alive": true, + "rss_delta": { + "a": 1, + "b": 1 + }, + "cache_builds": { + "a": 0, + "b": 0 + } + } + }, + "recovery_template_ms": { + "n": 203, + "p50": 7.9, + "p95": 96.1, + "p99": 731.6, + "max": 815.8, + "mean": 40.4 + }, + "final": { + "blocks_a": 120, + "blocks_b": 121, + "same_sink": false + }, + "alive": true, + "mem_bound_mb": 512 + } +} \ No newline at end of file diff --git a/docs/benchmarks/memory-floods-2026-10-04/before-s7-flood.json b/docs/benchmarks/memory-floods-2026-10-04/before-s7-flood.json new file mode 100644 index 000000000..301e5dacd --- /dev/null +++ b/docs/benchmarks/memory-floods-2026-10-04/before-s7-flood.json @@ -0,0 +1,163 @@ +{ + "rows": [ + { + "scenario": "7 fast-miner flood, controller trajectory (sim)", + "criterion": "trajectory recorded", + "result": "skipped (--live-only)", + "pass": null + }, + { + "scenario": "7 fast-miner flood, live (50 blocks/s from one peer)", + "criterion": "node stays responsive: honest template p95 < 200 ms, both nodes alive, same sink", + "result": "flood accepted 198 blocks in 60 s (3.3/s); honest template p50/p95/max 7.7/91.5/1454.3 ms under flood (baseline 23.5/92.8/188.8); rss a 302->1085 MB, b 303->1083 MB (cache builds 3/3); alive true; same sink true", + "pass": true + } + ], + "data": { + "live": { + "baseline_template_ms": { + "n": 76, + "p50": 23.5, + "p95": 92.8, + "p99": 188.8, + "max": 188.8, + "mean": 27.7 + }, + "under_flood_template_ms": { + "n": 458, + "p50": 7.7, + "p95": 91.5, + "p99": 116.7, + "max": 1454.3, + "mean": 31.9 + }, + "after_flood_template_ms": { + "n": 39, + "p50": 7.9, + "p95": 87.6, + "p99": 91.2, + "max": 91.2, + "mean": 24.8 + }, + "samples": [ + { + "t_s": 10, + "blocks_a": 94, + "blocks_b": 93, + "difficulty_a": 457438921.53559726, + "sink_same": false, + "rss_a": 569, + "rss_b": 567, + "cache_builds_a": 1, + "cache_builds_b": 1, + "flood_accepted": 84, + "flood_rejected": 0, + "honest_accepted": 10 + }, + { + "t_s": 20, + "blocks_a": 137, + "blocks_b": 137, + "difficulty_a": 1287295920.1241035, + "sink_same": true, + "rss_a": 827, + "rss_b": 825, + "cache_builds_a": 2, + "cache_builds_b": 2, + "flood_accepted": 126, + "flood_rejected": 0, + "honest_accepted": 11 + }, + { + "t_s": 30, + "blocks_a": 167, + "blocks_b": 167, + "difficulty_a": 3125416130.4758606, + "sink_same": true, + "rss_a": 828, + "rss_b": 826, + "cache_builds_a": 2, + "cache_builds_b": 2, + "flood_accepted": 155, + "flood_rejected": 0, + "honest_accepted": 12 + }, + { + "t_s": 40, + "blocks_a": 183, + "blocks_b": 180, + "difficulty_a": 4327152934.829311, + "sink_same": false, + "rss_a": 1084, + "rss_b": 1082, + "cache_builds_a": 3, + "cache_builds_b": 3, + "flood_accepted": 169, + "flood_rejected": 0, + "honest_accepted": 14 + }, + { + "t_s": 50, + "blocks_a": 196, + "blocks_b": 196, + "difficulty_a": 4435640076.843163, + "sink_same": true, + "rss_a": 1084, + "rss_b": 1083, + "cache_builds_a": 3, + "cache_builds_b": 3, + "flood_accepted": 181, + "flood_rejected": 0, + "honest_accepted": 15 + }, + { + "t_s": 60, + "blocks_a": 213, + "blocks_b": 213, + "difficulty_a": 6255943304.461032, + "sink_same": true, + "rss_a": 1085, + "rss_b": 1083, + "cache_builds_a": 3, + "cache_builds_b": 3, + "flood_accepted": 198, + "flood_rejected": 0, + "honest_accepted": 15 + } + ], + "rss_before": { + "a": 302, + "b": 303 + }, + "rss_peak": { + "a": 1085, + "b": 1083 + }, + "cache_builds": { + "a": 3, + "b": 3, + "before_flood": { + "a": 1, + "b": 1 + } + }, + "flood": { + "accepted": 198, + "rejected": 0, + "errors": 0 + }, + "honest": { + "accepted": 16, + "rejected": 0 + }, + "final": { + "blocks_a": 214, + "blocks_b": 214, + "same_sink": true, + "difficulty_a": 5868605790.480026, + "ratio": null + }, + "alive": true + } + } +} \ No newline at end of file diff --git a/docs/benchmarks/memory-floods-2026-10-04/before-s8-steady.json b/docs/benchmarks/memory-floods-2026-10-04/before-s8-steady.json new file mode 100644 index 000000000..150eafe8d --- /dev/null +++ b/docs/benchmarks/memory-floods-2026-10-04/before-s8-steady.json @@ -0,0 +1,506 @@ +{ + "rows": [ + { + "scenario": "8 steady state, one honest miner at 1 block/s, no flood (RSS per 1,000 blocks)", + "criterion": "recorded: RSS of the mining node / the follower at 0, 500, 1,000 and 1,500 blocks, cache builds", + "result": "0: 41/42 MB at 0 blocks (0/0 builds); 500: 1342/1342 MB at 514 blocks (9/9 builds); 1000: 1355/1355 MB at 1008 blocks (18/18 builds); 1500: 1371/1372 MB at 1529 blocks (27/27 builds); end 1529 blocks in 1527 s: 1371/1372 MB; honest accepted 1529 rejected 0 errors 0; alive true", + "pass": null + } + ], + "data": { + "samples": [ + { + "t_s": 0, + "blocks_a": 0, + "blocks_b": 0, + "daa_a": 0, + "rss_a": 41, + "rss_b": 42, + "cache_builds_a": 0, + "cache_builds_b": 0, + "same_sink": true, + "load": "80.88 75.23 63.38", + "vmmap_a": { + "physical_footprint": "22.5M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8a-0.txt" + }, + "vmmap_b": { + "physical_footprint": "23.7M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8b-0.txt" + } + }, + { + "t_s": 67, + "blocks_a": 83, + "blocks_b": 83, + "daa_a": 83, + "rss_a": 564, + "rss_b": 563, + "cache_builds_a": 2, + "cache_builds_b": 2, + "same_sink": true, + "load": "48.05 68.48 61.83" + }, + { + "t_s": 127, + "blocks_a": 143, + "blocks_b": 143, + "daa_a": 143, + "rss_a": 821, + "rss_b": 820, + "cache_builds_a": 3, + "cache_builds_b": 3, + "same_sink": true, + "load": "19.61 56.55 57.81" + }, + { + "t_s": 187, + "blocks_a": 203, + "blocks_b": 203, + "daa_a": 203, + "rss_a": 1079, + "rss_b": 1078, + "cache_builds_a": 4, + "cache_builds_b": 4, + "same_sink": true, + "load": "8.59 46.69 54.04" + }, + { + "t_s": 248, + "blocks_a": 268, + "blocks_b": 268, + "daa_a": 268, + "rss_a": 1080, + "rss_b": 1079, + "cache_builds_a": 5, + "cache_builds_b": 5, + "same_sink": true, + "load": "5.31 38.76 50.57" + }, + { + "t_s": 308, + "blocks_a": 329, + "blocks_b": 329, + "daa_a": 329, + "rss_a": 1081, + "rss_b": 1081, + "cache_builds_a": 6, + "cache_builds_b": 6, + "same_sink": true, + "load": "4.13 32.34 47.36" + }, + { + "t_s": 368, + "blocks_a": 398, + "blocks_b": 398, + "daa_a": 398, + "rss_a": 1083, + "rss_b": 1083, + "cache_builds_a": 7, + "cache_builds_b": 7, + "same_sink": true, + "load": "4.16 26.78 44.16" + }, + { + "t_s": 428, + "blocks_a": 461, + "blocks_b": 461, + "daa_a": 461, + "rss_a": 1085, + "rss_b": 1084, + "cache_builds_a": 8, + "cache_builds_b": 8, + "same_sink": true, + "load": "3.37 22.41 41.34" + }, + { + "t_s": 488, + "blocks_a": 514, + "blocks_b": 514, + "daa_a": 514, + "rss_a": 1342, + "rss_b": 1342, + "cache_builds_a": 9, + "cache_builds_b": 9, + "same_sink": true, + "load": "3.05 18.88 38.73", + "vmmap_a": { + "physical_footprint": "1.3G", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8a-500.txt" + }, + "vmmap_b": { + "physical_footprint": "1.3G", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8b-500.txt" + } + }, + { + "t_s": 551, + "blocks_a": 569, + "blocks_b": 569, + "daa_a": 569, + "rss_a": 1343, + "rss_b": 1343, + "cache_builds_a": 10, + "cache_builds_b": 10, + "same_sink": true, + "load": "5.64 16.62 36.52" + }, + { + "t_s": 611, + "blocks_a": 633, + "blocks_b": 633, + "daa_a": 633, + "rss_a": 1345, + "rss_b": 1345, + "cache_builds_a": 11, + "cache_builds_b": 11, + "same_sink": true, + "load": "4.37 14.24 34.27" + }, + { + "t_s": 671, + "blocks_a": 699, + "blocks_b": 699, + "daa_a": 699, + "rss_a": 1347, + "rss_b": 1346, + "cache_builds_a": 12, + "cache_builds_b": 12, + "same_sink": true, + "load": "6.78 13.04 32.45" + }, + { + "t_s": 731, + "blocks_a": 761, + "blocks_b": 761, + "daa_a": 761, + "rss_a": 1348, + "rss_b": 1348, + "cache_builds_a": 13, + "cache_builds_b": 13, + "same_sink": true, + "load": "5.68 11.55 30.57" + }, + { + "t_s": 791, + "blocks_a": 823, + "blocks_b": 823, + "daa_a": 823, + "rss_a": 1350, + "rss_b": 1350, + "cache_builds_a": 14, + "cache_builds_b": 14, + "same_sink": true, + "load": "4.46 10.14 28.75" + }, + { + "t_s": 852, + "blocks_a": 894, + "blocks_b": 894, + "daa_a": 894, + "rss_a": 1352, + "rss_b": 1352, + "cache_builds_a": 16, + "cache_builds_b": 16, + "same_sink": true, + "load": "3.31 8.79 26.97" + }, + { + "t_s": 912, + "blocks_a": 958, + "blocks_b": 958, + "daa_a": 958, + "rss_a": 1354, + "rss_b": 1354, + "cache_builds_a": 17, + "cache_builds_b": 17, + "same_sink": true, + "load": "2.38 7.53 25.26" + }, + { + "t_s": 972, + "blocks_a": 1008, + "blocks_b": 1008, + "daa_a": 1008, + "rss_a": 1355, + "rss_b": 1355, + "cache_builds_a": 18, + "cache_builds_b": 18, + "same_sink": true, + "load": "2.17 6.53 23.68", + "vmmap_a": { + "physical_footprint": "1.3G", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8a-1000.txt" + }, + "vmmap_b": { + "physical_footprint": "1.3G", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8b-1000.txt" + } + }, + { + "t_s": 1034, + "blocks_a": 1069, + "blocks_b": 1069, + "daa_a": 1069, + "rss_a": 1357, + "rss_b": 1357, + "cache_builds_a": 19, + "cache_builds_b": 19, + "same_sink": true, + "load": "2.37 5.73 22.11" + }, + { + "t_s": 1095, + "blocks_a": 1139, + "blocks_b": 1139, + "daa_a": 1139, + "rss_a": 1359, + "rss_b": 1358, + "cache_builds_a": 20, + "cache_builds_b": 20, + "same_sink": true, + "load": "2.46 5.10 20.76" + }, + { + "t_s": 1155, + "blocks_a": 1192, + "blocks_b": 1192, + "daa_a": 1192, + "rss_a": 1360, + "rss_b": 1360, + "cache_builds_a": 21, + "cache_builds_b": 21, + "same_sink": true, + "load": "1.41 4.31 19.39" + }, + { + "t_s": 1215, + "blocks_a": 1247, + "blocks_b": 1247, + "daa_a": 1247, + "rss_a": 1362, + "rss_b": 1362, + "cache_builds_a": 22, + "cache_builds_b": 22, + "same_sink": true, + "load": "1.19 3.72 18.14" + }, + { + "t_s": 1275, + "blocks_a": 1294, + "blocks_b": 1294, + "daa_a": 1294, + "rss_a": 1363, + "rss_b": 1363, + "cache_builds_a": 23, + "cache_builds_b": 23, + "same_sink": true, + "load": "1.59 3.40 17.04" + }, + { + "t_s": 1335, + "blocks_a": 1352, + "blocks_b": 1352, + "daa_a": 1352, + "rss_a": 1365, + "rss_b": 1364, + "cache_builds_a": 24, + "cache_builds_b": 24, + "same_sink": true, + "load": "1.87 3.20 16.04" + }, + { + "t_s": 1395, + "blocks_a": 1401, + "blocks_b": 1401, + "daa_a": 1401, + "rss_a": 1367, + "rss_b": 1366, + "cache_builds_a": 25, + "cache_builds_b": 25, + "same_sink": true, + "load": "7.11 4.19 15.52" + }, + { + "t_s": 1456, + "blocks_a": 1468, + "blocks_b": 1468, + "daa_a": 1468, + "rss_a": 1369, + "rss_b": 1369, + "cache_builds_a": 26, + "cache_builds_b": 26, + "same_sink": true, + "load": "43.45 13.68 18.21" + }, + { + "t_s": 1517, + "blocks_a": 1529, + "blocks_b": 1529, + "daa_a": 1529, + "rss_a": 1371, + "rss_b": 1372, + "cache_builds_a": 27, + "cache_builds_b": 27, + "same_sink": true, + "load": "92.08 33.71 25.42", + "vmmap_a": { + "physical_footprint": "1.3G", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8a-1500.txt" + }, + "vmmap_b": { + "physical_footprint": "1.3G", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8b-1500.txt" + } + }, + { + "t_s": 1527, + "blocks_a": 1529, + "blocks_b": 1529, + "daa_a": 1529, + "rss_a": 1371, + "rss_b": 1372, + "cache_builds_a": 27, + "cache_builds_b": 27, + "same_sink": true, + "load": "90.06 35.17 26.04", + "vmmap_a": { + "physical_footprint": "1.3G", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8a-end.txt" + }, + "vmmap_b": { + "physical_footprint": "1.3G", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8b-end.txt" + } + } + ], + "milestones": { + "0": { + "t_s": 0, + "blocks_a": 0, + "blocks_b": 0, + "daa_a": 0, + "rss_a": 41, + "rss_b": 42, + "cache_builds_a": 0, + "cache_builds_b": 0, + "same_sink": true, + "load": "80.88 75.23 63.38", + "vmmap_a": { + "physical_footprint": "22.5M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8a-0.txt" + }, + "vmmap_b": { + "physical_footprint": "23.7M", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8b-0.txt" + } + }, + "500": { + "t_s": 488, + "blocks_a": 514, + "blocks_b": 514, + "daa_a": 514, + "rss_a": 1342, + "rss_b": 1342, + "cache_builds_a": 9, + "cache_builds_b": 9, + "same_sink": true, + "load": "3.05 18.88 38.73", + "vmmap_a": { + "physical_footprint": "1.3G", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8a-500.txt" + }, + "vmmap_b": { + "physical_footprint": "1.3G", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8b-500.txt" + } + }, + "1000": { + "t_s": 972, + "blocks_a": 1008, + "blocks_b": 1008, + "daa_a": 1008, + "rss_a": 1355, + "rss_b": 1355, + "cache_builds_a": 18, + "cache_builds_b": 18, + "same_sink": true, + "load": "2.17 6.53 23.68", + "vmmap_a": { + "physical_footprint": "1.3G", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8a-1000.txt" + }, + "vmmap_b": { + "physical_footprint": "1.3G", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8b-1000.txt" + } + }, + "1500": { + "t_s": 1517, + "blocks_a": 1529, + "blocks_b": 1529, + "daa_a": 1529, + "rss_a": 1371, + "rss_b": 1372, + "cache_builds_a": 27, + "cache_builds_b": 27, + "same_sink": true, + "load": "92.08 33.71 25.42", + "vmmap_a": { + "physical_footprint": "1.3G", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8a-1500.txt" + }, + "vmmap_b": { + "physical_footprint": "1.3G", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8b-1500.txt" + } + }, + "end": { + "t_s": 1527, + "blocks_a": 1529, + "blocks_b": 1529, + "daa_a": 1529, + "rss_a": 1371, + "rss_b": 1372, + "cache_builds_a": 27, + "cache_builds_b": 27, + "same_sink": true, + "load": "90.06 35.17 26.04", + "vmmap_a": { + "physical_footprint": "1.3G", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8a-end.txt" + }, + "vmmap_b": { + "physical_footprint": "1.3G", + "malloc_total": "0K", + "file": "/tmp/igneum-fud-mem/steady-before/vmmap/s8b-end.txt" + } + } + }, + "honest": { + "accepted": 1529, + "rejected": 0, + "errors": 0 + }, + "alive": true + } +} \ No newline at end of file diff --git a/docs/benchmarks/memory-floods-2026-10-04/vmmap/after-s8a-0.txt b/docs/benchmarks/memory-floods-2026-10-04/vmmap/after-s8a-0.txt new file mode 100644 index 000000000..ececdfd9f --- /dev/null +++ b/docs/benchmarks/memory-floods-2026-10-04/vmmap/after-s8a-0.txt @@ -0,0 +1,65 @@ +Process: igneumd [55063] +Path: /Users/USER/*/igneumd +Load Address: 0x100a4c000 +Identifier: igneumd +Version: 0 +Code Type: ARM64 +Platform: macOS +Parent Process: node [53092] +Target Type: live task + +Date/Time: 2026-10-05 02:20:03.953 +0100 +Launch Time: 2026-10-05 02:19:55.547 +0100 +OS Version: macOS 26.6.2 (25G83) +Report Version: 7 +Analysis Tool: /usr/bin/vmmap + +Physical footprint: 22.8M +Physical footprint (peak): 22.8M +Idle exit: untracked +---- + +ReadOnly portion of Libraries: Total=898.4M resident=211.2M(24%) swapped_out_or_unallocated=687.2M(76%) +Writable regions: Total=3.3G written=20.8M(1%) resident=20.8M(1%) swapped_out=0K(0%) unallocated=3.2G(99%) + + VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION +REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced) +=========== ======= ======== ===== ======= ======== ====== ===== ======= +IOAccelerator 2.1G 14.2M 14.2M 0K 0K 0K 0K 80 +IOAccelerator (reserved) 896.0M 0K 0K 0K 0K 0K 0K 1 reserved VM address space (unallocated) +Kernel Alloc Once 32K 16K 16K 0K 0K 0K 0K 1 +MALLOC guard page 3632K 0K 0K 0K 0K 0K 0K 4 +MALLOC metadata 880K 480K 480K 0K 0K 0K 0K 4 +MALLOC_SMALL 32.0M 2624K 2624K 0K 0K 0K 0K 8 see MALLOC ZONE table below +MALLOC_SMALL (empty) 4096K 32K 32K 0K 0K 0K 0K 1 see MALLOC ZONE table below +MALLOC_TINY 4096K 208K 208K 0K 0K 0K 0K 1 see MALLOC ZONE table below +Memory Tag 22 64.0M 16K 16K 0K 0K 0K 0K 1 +STACK GUARD 1424K 0K 0K 0K 0K 0K 0K 89 +Stack 154.2M 1760K 1760K 0K 0K 0K 0K 90 +Stack Guard 56.0M 0K 0K 0K 0K 0K 0K 1 +VM_ALLOCATE 1088K 0K 0K 0K 0K 0K 0K 68 +VM_ALLOCATE (reserved) 8576K 0K 0K 0K 0K 0K 0K 67 reserved VM address space (unallocated) +__AUTH 1321K 925K 0K 0K 0K 0K 0K 149 +__AUTH_CONST 17.7M 11.5M 0K 0K 0K 0K 0K 340 +__CTF 824 824 0K 0K 0K 0K 0K 1 +__DATA 4453K 2379K 346K 0K 0K 0K 0K 297 +__DATA_CONST 17.6M 12.3M 1344K 0K 0K 0K 0K 339 +__DATA_DIRTY 1320K 1056K 351K 0K 0K 0K 0K 284 +__FONT_DATA 2352 2352 0K 0K 0K 0K 0K 1 +__LINKEDIT 573.7M 32.9M 0K 0K 0K 0K 0K 2 +__OBJC_RO 79.2M 59.8M 0K 0K 0K 0K 0K 1 +__OBJC_RW 2599K 2375K 39K 0K 0K 0K 0K 1 +__TEXT 324.7M 178.3M 0K 0K 0K 0K 0K 348 +__TPRO_CONST 128K 112K 48K 0K 0K 0K 0K 2 +page table in kernel 1474K 1474K 1474K 0K 0K 0K 0K 1 +shared memory 48K 48K 48K 0K 0K 0K 0K 2 +unused but dirty shlib __DATA 79K 79K 79K 0K 0K 0K 0K 39 +=========== ======= ======== ===== ======= ======== ====== ===== ======= +TOTAL 4.3G 322.2M 22.8M 0K 0K 0K 0K 2223 +TOTAL, minus reserved VM space 3.4G 322.2M 22.8M 0K 0K 0K 0K 2223 + + VIRTUAL RESIDENT DIRTY SWAPPED ALLOCATION BYTES DIRTY+SWAP REGION +MALLOC ZONE SIZE SIZE SIZE SIZE COUNT ALLOCATED FRAG SIZE % FRAG COUNT +=========== ======= ========= ========= ========= ========= ========= ========= ====== ====== +DefaultMallocZone_0x10345c000 40.8M 3312K 3312K 0K 5443 3008K 304K 10% 12 + diff --git a/docs/benchmarks/memory-floods-2026-10-04/vmmap/after-s8a-1000.txt b/docs/benchmarks/memory-floods-2026-10-04/vmmap/after-s8a-1000.txt new file mode 100644 index 000000000..f13ab9ab7 --- /dev/null +++ b/docs/benchmarks/memory-floods-2026-10-04/vmmap/after-s8a-1000.txt @@ -0,0 +1,65 @@ +Process: igneumd [55063] +Path: /Users/USER/*/igneumd +Load Address: 0x100a4c000 +Identifier: igneumd +Version: 0 +Code Type: ARM64 +Platform: macOS +Parent Process: node [53092] +Target Type: live task + +Date/Time: 2026-10-05 02:37:18.131 +0100 +Launch Time: 2026-10-05 02:19:55.547 +0100 +OS Version: macOS 26.6.2 (25G83) +Report Version: 7 +Analysis Tool: /usr/bin/vmmap + +Physical footprint: 569.6M +Physical footprint (peak): 569.6M +Idle exit: untracked +---- + +ReadOnly portion of Libraries: Total=898.4M resident=212.5M(24%) swapped_out_or_unallocated=685.8M(76%) +Writable regions: Total=5.8G written=567.6M(10%) resident=567.6M(10%) swapped_out=0K(0%) unallocated=5.2G(90%) + + VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION +REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced) +=========== ======= ======== ===== ======= ======== ====== ===== ======= +IOAccelerator 2.6G 548.1M 548.1M 0K 0K 0K 0K 87 +IOAccelerator (reserved) 2.9G 0K 0K 0K 0K 0K 0K 2 reserved VM address space (unallocated) +Kernel Alloc Once 32K 16K 16K 0K 0K 0K 0K 1 +MALLOC guard page 3632K 0K 0K 0K 0K 0K 0K 4 +MALLOC metadata 880K 544K 544K 0K 0K 0K 0K 4 +MALLOC_SMALL 44.0M 14.7M 14.7M 0K 0K 0K 0K 11 see MALLOC ZONE table below +MALLOC_SMALL (empty) 8192K 64K 64K 0K 0K 0K 0K 2 see MALLOC ZONE table below +MALLOC_TINY 4096K 208K 208K 0K 0K 0K 0K 1 see MALLOC ZONE table below +Memory Tag 22 64.0M 16K 16K 0K 0K 0K 0K 1 +STACK GUARD 1424K 0K 0K 0K 0K 0K 0K 89 +Stack 154.2M 2128K 2128K 0K 0K 0K 0K 90 +Stack Guard 56.0M 0K 0K 0K 0K 0K 0K 1 +VM_ALLOCATE 1088K 0K 0K 0K 0K 0K 0K 68 +VM_ALLOCATE (reserved) 8576K 0K 0K 0K 0K 0K 0K 67 reserved VM address space (unallocated) +__AUTH 1321K 918K 0K 0K 0K 0K 0K 149 +__AUTH_CONST 17.7M 11.5M 0K 0K 0K 0K 0K 340 +__CTF 824 824 0K 0K 0K 0K 0K 1 +__DATA 4453K 2379K 346K 0K 0K 0K 0K 297 +__DATA_CONST 17.6M 12.3M 1344K 0K 0K 0K 0K 339 +__DATA_DIRTY 1320K 1040K 351K 0K 0K 0K 0K 284 +__FONT_DATA 2352 2352 0K 0K 0K 0K 0K 1 +__LINKEDIT 573.7M 32.9M 0K 0K 0K 0K 0K 2 +__OBJC_RO 79.2M 59.9M 0K 0K 0K 0K 0K 1 +__OBJC_RW 2599K 2375K 39K 0K 0K 0K 0K 1 +__TEXT 324.7M 179.6M 0K 0K 0K 0K 0K 348 +__TPRO_CONST 128K 96K 48K 0K 0K 0K 0K 2 +page table in kernel 1779K 1779K 1779K 0K 0K 0K 0K 1 +shared memory 48K 48K 48K 0K 0K 0K 0K 2 +unused but dirty shlib __DATA 79K 79K 79K 0K 0K 0K 0K 39 +=========== ======= ======== ===== ======= ======== ====== ===== ======= +TOTAL 6.8G 870.4M 569.6M 0K 0K 0K 0K 2235 +TOTAL, minus reserved VM space 4.0G 870.4M 569.6M 0K 0K 0K 0K 2235 + + VIRTUAL RESIDENT DIRTY SWAPPED ALLOCATION BYTES DIRTY+SWAP REGION +MALLOC ZONE SIZE SIZE SIZE SIZE COUNT ALLOCATED FRAG SIZE % FRAG COUNT +=========== ======= ========= ========= ========= ========= ========= ========= ====== ====== +DefaultMallocZone_0x10345c000 56.8M 15.5M 15.5M 0K 5469 14.9M 523K 4% 16 + diff --git a/docs/benchmarks/memory-floods-2026-10-04/vmmap/after-s8a-1500.txt b/docs/benchmarks/memory-floods-2026-10-04/vmmap/after-s8a-1500.txt new file mode 100644 index 000000000..433c93315 --- /dev/null +++ b/docs/benchmarks/memory-floods-2026-10-04/vmmap/after-s8a-1500.txt @@ -0,0 +1,65 @@ +Process: igneumd [55063] +Path: /Users/USER/*/igneumd +Load Address: 0x100a4c000 +Identifier: igneumd +Version: 0 +Code Type: ARM64 +Platform: macOS +Parent Process: node [53092] +Target Type: live task + +Date/Time: 2026-10-05 02:45:21.762 +0100 +Launch Time: 2026-10-05 02:19:55.547 +0100 +OS Version: macOS 26.6.2 (25G83) +Report Version: 7 +Analysis Tool: /usr/bin/vmmap + +Physical footprint: 584.0M +Physical footprint (peak): 584.0M +Idle exit: untracked +---- + +ReadOnly portion of Libraries: Total=898.4M resident=212.6M(24%) swapped_out_or_unallocated=685.8M(76%) +Writable regions: Total=5.8G written=582.0M(10%) resident=582.0M(10%) swapped_out=0K(0%) unallocated=5.2G(90%) + + VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION +REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced) +=========== ======= ======== ===== ======= ======== ====== ===== ======= +IOAccelerator 2.6G 555.7M 555.7M 0K 0K 0K 0K 87 +IOAccelerator (reserved) 2.9G 0K 0K 0K 0K 0K 0K 2 reserved VM address space (unallocated) +Kernel Alloc Once 32K 16K 16K 0K 0K 0K 0K 1 +MALLOC guard page 3632K 0K 0K 0K 0K 0K 0K 4 +MALLOC metadata 880K 592K 592K 0K 0K 0K 0K 4 +MALLOC_SMALL 56.0M 21.5M 21.5M 0K 0K 0K 0K 14 see MALLOC ZONE table below +MALLOC_SMALL (empty) 8192K 48K 48K 0K 0K 0K 0K 2 see MALLOC ZONE table below +MALLOC_TINY 4096K 208K 208K 0K 0K 0K 0K 1 see MALLOC ZONE table below +Memory Tag 22 64.0M 16K 16K 0K 0K 0K 0K 1 +STACK GUARD 1424K 0K 0K 0K 0K 0K 0K 89 +Stack 154.2M 2128K 2128K 0K 0K 0K 0K 90 +Stack Guard 56.0M 0K 0K 0K 0K 0K 0K 1 +VM_ALLOCATE 1088K 0K 0K 0K 0K 0K 0K 68 +VM_ALLOCATE (reserved) 8576K 0K 0K 0K 0K 0K 0K 67 reserved VM address space (unallocated) +__AUTH 1321K 925K 0K 0K 0K 0K 0K 149 +__AUTH_CONST 17.7M 11.5M 0K 0K 0K 0K 0K 340 +__CTF 824 824 0K 0K 0K 0K 0K 1 +__DATA 4453K 2379K 346K 0K 0K 0K 0K 297 +__DATA_CONST 17.6M 12.3M 1344K 0K 0K 0K 0K 339 +__DATA_DIRTY 1320K 1056K 351K 0K 0K 0K 0K 284 +__FONT_DATA 2352 2352 0K 0K 0K 0K 0K 1 +__LINKEDIT 573.7M 32.9M 0K 0K 0K 0K 0K 2 +__OBJC_RO 79.2M 59.9M 0K 0K 0K 0K 0K 1 +__OBJC_RW 2599K 2375K 39K 0K 0K 0K 0K 1 +__TEXT 324.7M 179.6M 0K 0K 0K 0K 0K 348 +__TPRO_CONST 128K 112K 48K 0K 0K 0K 0K 2 +page table in kernel 1779K 1779K 1779K 0K 0K 0K 0K 1 +shared memory 48K 48K 48K 0K 0K 0K 0K 2 +unused but dirty shlib __DATA 79K 79K 79K 0K 0K 0K 0K 39 +=========== ======= ======== ===== ======= ======== ====== ===== ======= +TOTAL 6.9G 884.9M 584.0M 0K 0K 0K 0K 2238 +TOTAL, minus reserved VM space 4.0G 884.9M 584.0M 0K 0K 0K 0K 2238 + + VIRTUAL RESIDENT DIRTY SWAPPED ALLOCATION BYTES DIRTY+SWAP REGION +MALLOC ZONE SIZE SIZE SIZE SIZE COUNT ALLOCATED FRAG SIZE % FRAG COUNT +=========== ======= ========= ========= ========= ========= ========= ========= ====== ====== +DefaultMallocZone_0x10345c000 68.8M 22.3M 22.3M 0K 5530 22.0M 269K 2% 19 + diff --git a/docs/benchmarks/memory-floods-2026-10-04/vmmap/after-s8a-500.txt b/docs/benchmarks/memory-floods-2026-10-04/vmmap/after-s8a-500.txt new file mode 100644 index 000000000..2a779f940 --- /dev/null +++ b/docs/benchmarks/memory-floods-2026-10-04/vmmap/after-s8a-500.txt @@ -0,0 +1,65 @@ +Process: igneumd [55063] +Path: /Users/USER/*/igneumd +Load Address: 0x100a4c000 +Identifier: igneumd +Version: 0 +Code Type: ARM64 +Platform: macOS +Parent Process: node [53092] +Target Type: live task + +Date/Time: 2026-10-05 02:28:14.170 +0100 +Launch Time: 2026-10-05 02:19:55.547 +0100 +OS Version: macOS 26.6.2 (25G83) +Report Version: 7 +Analysis Tool: /usr/bin/vmmap + +Physical footprint: 299.2M +Physical footprint (peak): 299.2M +Idle exit: untracked +---- + +ReadOnly portion of Libraries: Total=898.4M resident=212.3M(24%) swapped_out_or_unallocated=686.1M(76%) +Writable regions: Total=4.5G written=297.2M(6%) resident=297.2M(6%) swapped_out=0K(0%) unallocated=4.2G(94%) + + VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION +REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced) +=========== ======= ======== ===== ======= ======== ====== ===== ======= +IOAccelerator 2.4G 284.5M 284.5M 0K 0K 0K 0K 84 +IOAccelerator (reserved) 1.9G 0K 0K 0K 0K 0K 0K 1 reserved VM address space (unallocated) +Kernel Alloc Once 32K 16K 16K 0K 0K 0K 0K 1 +MALLOC guard page 3632K 0K 0K 0K 0K 0K 0K 4 +MALLOC metadata 880K 496K 496K 0K 0K 0K 0K 4 +MALLOC_SMALL 36.0M 8320K 8320K 0K 0K 0K 0K 9 see MALLOC ZONE table below +MALLOC_SMALL (empty) 4096K 32K 32K 0K 0K 0K 0K 1 see MALLOC ZONE table below +MALLOC_TINY 4096K 208K 208K 0K 0K 0K 0K 1 see MALLOC ZONE table below +Memory Tag 22 64.0M 16K 16K 0K 0K 0K 0K 1 +STACK GUARD 1424K 0K 0K 0K 0K 0K 0K 89 +Stack 154.2M 2128K 2128K 0K 0K 0K 0K 90 +Stack Guard 56.0M 0K 0K 0K 0K 0K 0K 1 +VM_ALLOCATE 1088K 0K 0K 0K 0K 0K 0K 68 +VM_ALLOCATE (reserved) 8576K 0K 0K 0K 0K 0K 0K 67 reserved VM address space (unallocated) +__AUTH 1321K 925K 0K 0K 0K 0K 0K 149 +__AUTH_CONST 17.7M 11.5M 0K 0K 0K 0K 0K 340 +__CTF 824 824 0K 0K 0K 0K 0K 1 +__DATA 4453K 2379K 346K 0K 0K 0K 0K 297 +__DATA_CONST 17.6M 12.3M 1344K 0K 0K 0K 0K 339 +__DATA_DIRTY 1320K 1072K 351K 0K 0K 0K 0K 284 +__FONT_DATA 2352 2352 0K 0K 0K 0K 0K 1 +__LINKEDIT 573.7M 32.9M 0K 0K 0K 0K 0K 2 +__OBJC_RO 79.2M 59.8M 0K 0K 0K 0K 0K 1 +__OBJC_RW 2599K 2375K 39K 0K 0K 0K 0K 1 +__TEXT 324.7M 179.4M 0K 0K 0K 0K 0K 348 +__TPRO_CONST 128K 112K 48K 0K 0K 0K 0K 2 +page table in kernel 1634K 1634K 1634K 0K 0K 0K 0K 1 +shared memory 48K 48K 48K 0K 0K 0K 0K 2 +unused but dirty shlib __DATA 79K 79K 79K 0K 0K 0K 0K 39 +=========== ======= ======== ===== ======= ======== ====== ===== ======= +TOTAL 5.6G 599.7M 299.2M 0K 0K 0K 0K 2228 +TOTAL, minus reserved VM space 3.7G 599.7M 299.2M 0K 0K 0K 0K 2228 + + VIRTUAL RESIDENT DIRTY SWAPPED ALLOCATION BYTES DIRTY+SWAP REGION +MALLOC ZONE SIZE SIZE SIZE SIZE COUNT ALLOCATED FRAG SIZE % FRAG COUNT +=========== ======= ========= ========= ========= ========= ========= ========= ====== ====== +DefaultMallocZone_0x10345c000 44.8M 9024K 9024K 0K 5463 9157K 0K 0% 13 + diff --git a/docs/benchmarks/memory-floods-2026-10-04/vmmap/before-s8a-0.txt b/docs/benchmarks/memory-floods-2026-10-04/vmmap/before-s8a-0.txt new file mode 100644 index 000000000..600a13ef7 --- /dev/null +++ b/docs/benchmarks/memory-floods-2026-10-04/vmmap/before-s8a-0.txt @@ -0,0 +1,65 @@ +Process: igneumd [55718] +Path: /Users/USER/*/igneumd +Load Address: 0x102804000 +Identifier: igneumd +Version: 0 +Code Type: ARM64 +Platform: macOS +Parent Process: node [53051] +Target Type: live task + +Date/Time: 2026-10-05 02:22:42.382 +0100 +Launch Time: 2026-10-05 02:22:34.566 +0100 +OS Version: macOS 26.6.2 (25G83) +Report Version: 7 +Analysis Tool: /usr/bin/vmmap + +Physical footprint: 22.5M +Physical footprint (peak): 22.5M +Idle exit: untracked +---- + +ReadOnly portion of Libraries: Total=898.4M resident=210.9M(23%) swapped_out_or_unallocated=687.4M(77%) +Writable regions: Total=4.3G written=20.5M(0%) resident=20.5M(0%) swapped_out=0K(0%) unallocated=4.2G(100%) + + VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION +REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced) +=========== ======= ======== ===== ======= ======== ====== ===== ======= +IOAccelerator 2.3G 14.0M 14.0M 0K 0K 0K 0K 71 +IOAccelerator (reserved) 1.8G 0K 0K 0K 0K 0K 0K 2 reserved VM address space (unallocated) +Kernel Alloc Once 32K 16K 16K 0K 0K 0K 0K 1 +MALLOC guard page 3984K 0K 0K 0K 0K 0K 0K 4 +MALLOC metadata 880K 480K 480K 0K 0K 0K 0K 4 +MALLOC_SMALL 32.0M 2560K 2560K 0K 0K 0K 0K 8 see MALLOC ZONE table below +MALLOC_SMALL (empty) 4096K 32K 32K 0K 0K 0K 0K 1 see MALLOC ZONE table below +MALLOC_TINY 4096K 192K 192K 0K 0K 0K 0K 1 see MALLOC ZONE table below +Memory Tag 22 64.0M 16K 16K 0K 0K 0K 0K 1 +STACK GUARD 1424K 0K 0K 0K 0K 0K 0K 89 +Stack 154.2M 1728K 1728K 0K 0K 0K 0K 90 +Stack Guard 56.0M 0K 0K 0K 0K 0K 0K 1 +VM_ALLOCATE 1088K 0K 0K 0K 0K 0K 0K 68 +VM_ALLOCATE (reserved) 8576K 0K 0K 0K 0K 0K 0K 67 reserved VM address space (unallocated) +__AUTH 1321K 925K 0K 0K 0K 0K 0K 149 +__AUTH_CONST 17.7M 11.5M 0K 0K 0K 0K 0K 340 +__CTF 824 824 0K 0K 0K 0K 0K 1 +__DATA 4453K 2379K 346K 0K 0K 0K 0K 297 +__DATA_CONST 17.6M 12.3M 1344K 0K 0K 0K 0K 339 +__DATA_DIRTY 1320K 1072K 351K 0K 0K 0K 0K 284 +__FONT_DATA 2352 2352 0K 0K 0K 0K 0K 1 +__LINKEDIT 573.7M 32.9M 0K 0K 0K 0K 0K 2 +__OBJC_RO 79.2M 59.8M 0K 0K 0K 0K 0K 1 +__OBJC_RW 2599K 2375K 39K 0K 0K 0K 0K 1 +__TEXT 324.7M 178.0M 0K 0K 0K 0K 0K 348 +__TPRO_CONST 128K 112K 48K 0K 0K 0K 0K 2 +page table in kernel 1474K 1474K 1474K 0K 0K 0K 0K 1 +shared memory 48K 48K 48K 0K 0K 0K 0K 2 +unused but dirty shlib __DATA 79K 79K 79K 0K 0K 0K 0K 39 +=========== ======= ======== ===== ======= ======== ====== ===== ======= +TOTAL 5.3G 321.7M 22.6M 0K 0K 0K 0K 2215 +TOTAL, minus reserved VM space 3.6G 321.7M 22.6M 0K 0K 0K 0K 2215 + + VIRTUAL RESIDENT DIRTY SWAPPED ALLOCATION BYTES DIRTY+SWAP REGION +MALLOC ZONE SIZE SIZE SIZE SIZE COUNT ALLOCATED FRAG SIZE % FRAG COUNT +=========== ======= ========= ========= ========= ========= ========= ========= ====== ====== +DefaultMallocZone_0x1053e0000 40.8M 3232K 3232K 0K 5445 3009K 223K 7% 12 + diff --git a/docs/benchmarks/memory-floods-2026-10-04/vmmap/before-s8a-1000.txt b/docs/benchmarks/memory-floods-2026-10-04/vmmap/before-s8a-1000.txt new file mode 100644 index 000000000..72be9d8c2 --- /dev/null +++ b/docs/benchmarks/memory-floods-2026-10-04/vmmap/before-s8a-1000.txt @@ -0,0 +1,65 @@ +Process: igneumd [55718] +Path: /Users/USER/*/igneumd +Load Address: 0x102804000 +Identifier: igneumd +Version: 0 +Code Type: ARM64 +Platform: macOS +Parent Process: node [53051] +Target Type: live task + +Date/Time: 2026-10-05 02:38:53.727 +0100 +Launch Time: 2026-10-05 02:22:34.566 +0100 +OS Version: macOS 26.6.2 (25G83) +Report Version: 7 +Analysis Tool: /usr/bin/vmmap + +Physical footprint: 1.3G +Physical footprint (peak): 1.3G +Idle exit: untracked +---- + +ReadOnly portion of Libraries: Total=898.4M resident=212.3M(24%) swapped_out_or_unallocated=686.0M(76%) +Writable regions: Total=10.3G written=1.3G(13%) resident=1.3G(13%) swapped_out=0K(0%) unallocated=9.0G(87%) + + VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION +REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced) +=========== ======= ======== ===== ======= ======== ====== ===== ======= +IOAccelerator 3.8G 1.3G 1.3G 0K 0K 0K 0K 83 +IOAccelerator (reserved) 6.2G 0K 0K 0K 0K 0K 0K 6 reserved VM address space (unallocated) +Kernel Alloc Once 32K 16K 16K 0K 0K 0K 0K 1 +MALLOC guard page 3984K 0K 0K 0K 0K 0K 0K 4 +MALLOC metadata 880K 560K 560K 0K 0K 0K 0K 4 +MALLOC_SMALL 44.0M 14.3M 14.3M 0K 0K 0K 0K 11 see MALLOC ZONE table below +MALLOC_SMALL (empty) 12.0M 96K 96K 0K 0K 0K 0K 3 see MALLOC ZONE table below +MALLOC_TINY 4096K 192K 192K 0K 0K 0K 0K 1 see MALLOC ZONE table below +Memory Tag 22 64.0M 16K 16K 0K 0K 0K 0K 1 +STACK GUARD 1424K 0K 0K 0K 0K 0K 0K 89 +Stack 154.2M 2080K 2080K 0K 0K 0K 0K 90 +Stack Guard 56.0M 0K 0K 0K 0K 0K 0K 1 +VM_ALLOCATE 1088K 0K 0K 0K 0K 0K 0K 68 +VM_ALLOCATE (reserved) 8576K 0K 0K 0K 0K 0K 0K 67 reserved VM address space (unallocated) +__AUTH 1321K 925K 0K 0K 0K 0K 0K 149 +__AUTH_CONST 17.7M 11.5M 0K 0K 0K 0K 0K 340 +__CTF 824 824 0K 0K 0K 0K 0K 1 +__DATA 4453K 2379K 346K 0K 0K 0K 0K 297 +__DATA_CONST 17.6M 12.3M 1344K 0K 0K 0K 0K 339 +__DATA_DIRTY 1320K 1056K 351K 0K 0K 0K 0K 284 +__FONT_DATA 2352 2352 0K 0K 0K 0K 0K 1 +__LINKEDIT 573.7M 32.9M 0K 0K 0K 0K 0K 2 +__OBJC_RO 79.2M 59.9M 0K 0K 0K 0K 0K 1 +__OBJC_RW 2599K 2375K 39K 0K 0K 0K 0K 1 +__TEXT 324.7M 179.4M 0K 0K 0K 0K 0K 348 +__TPRO_CONST 128K 112K 48K 0K 0K 0K 0K 2 +page table in kernel 2211K 2211K 2211K 0K 0K 0K 0K 1 +shared memory 48K 48K 48K 0K 0K 0K 0K 2 +unused but dirty shlib __DATA 79K 79K 79K 0K 0K 0K 0K 39 +=========== ======= ======== ===== ======= ======== ====== ===== ======= +TOTAL 11.3G 1.6G 1.3G 0K 0K 0K 0K 2236 +TOTAL, minus reserved VM space 5.1G 1.6G 1.3G 0K 0K 0K 0K 2236 + + VIRTUAL RESIDENT DIRTY SWAPPED ALLOCATION BYTES DIRTY+SWAP REGION +MALLOC ZONE SIZE SIZE SIZE SIZE COUNT ALLOCATED FRAG SIZE % FRAG COUNT +=========== ======= ========= ========= ========= ========= ========= ========= ====== ====== +DefaultMallocZone_0x1053e0000 60.8M 15.1M 15.1M 0K 5467 14.9M 203K 2% 17 + diff --git a/docs/benchmarks/memory-floods-2026-10-04/vmmap/before-s8a-1500.txt b/docs/benchmarks/memory-floods-2026-10-04/vmmap/before-s8a-1500.txt new file mode 100644 index 000000000..9aed3c87c --- /dev/null +++ b/docs/benchmarks/memory-floods-2026-10-04/vmmap/before-s8a-1500.txt @@ -0,0 +1,65 @@ +Process: igneumd [55718] +Path: /Users/USER/*/igneumd +Load Address: 0x102804000 +Identifier: igneumd +Version: 0 +Code Type: ARM64 +Platform: macOS +Parent Process: node [53051] +Target Type: live task + +Date/Time: 2026-10-05 02:47:59.295 +0100 +Launch Time: 2026-10-05 02:22:34.566 +0100 +OS Version: macOS 26.6.2 (25G83) +Report Version: 7 +Analysis Tool: /usr/bin/vmmap + +Physical footprint: 1.3G +Physical footprint (peak): 1.3G +Idle exit: untracked +---- + +ReadOnly portion of Libraries: Total=898.4M resident=212.3M(24%) swapped_out_or_unallocated=686.0M(76%) +Writable regions: Total=10.3G written=1.3G(13%) resident=1.3G(13%) swapped_out=0K(0%) unallocated=9.0G(87%) + + VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION +REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced) +=========== ======= ======== ===== ======= ======== ====== ===== ======= +IOAccelerator 3.8G 1.3G 1.3G 0K 0K 0K 0K 83 +IOAccelerator (reserved) 6.2G 0K 0K 0K 0K 0K 0K 6 reserved VM address space (unallocated) +Kernel Alloc Once 32K 16K 16K 0K 0K 0K 0K 1 +MALLOC guard page 3984K 0K 0K 0K 0K 0K 0K 4 +MALLOC metadata 880K 592K 592K 0K 0K 0K 0K 4 +MALLOC_SMALL 52.0M 21.5M 21.5M 0K 0K 0K 0K 13 see MALLOC ZONE table below +MALLOC_SMALL (empty) 12.0M 96K 96K 0K 0K 0K 0K 3 see MALLOC ZONE table below +MALLOC_TINY 4096K 192K 192K 0K 0K 0K 0K 1 see MALLOC ZONE table below +Memory Tag 22 64.0M 16K 16K 0K 0K 0K 0K 1 +STACK GUARD 1424K 0K 0K 0K 0K 0K 0K 89 +Stack 154.2M 2144K 2144K 0K 0K 0K 0K 90 +Stack Guard 56.0M 0K 0K 0K 0K 0K 0K 1 +VM_ALLOCATE 1088K 0K 0K 0K 0K 0K 0K 68 +VM_ALLOCATE (reserved) 8576K 0K 0K 0K 0K 0K 0K 67 reserved VM address space (unallocated) +__AUTH 1321K 925K 0K 0K 0K 0K 0K 149 +__AUTH_CONST 17.7M 11.5M 0K 0K 0K 0K 0K 340 +__CTF 824 824 0K 0K 0K 0K 0K 1 +__DATA 4453K 2379K 346K 0K 0K 0K 0K 297 +__DATA_CONST 17.6M 12.3M 1344K 0K 0K 0K 0K 339 +__DATA_DIRTY 1320K 1056K 351K 0K 0K 0K 0K 284 +__FONT_DATA 2352 2352 0K 0K 0K 0K 0K 1 +__LINKEDIT 573.7M 32.9M 0K 0K 0K 0K 0K 2 +__OBJC_RO 79.2M 59.9M 0K 0K 0K 0K 0K 1 +__OBJC_RW 2599K 2375K 39K 0K 0K 0K 0K 1 +__TEXT 324.7M 179.4M 0K 0K 0K 0K 0K 348 +__TPRO_CONST 128K 112K 48K 0K 0K 0K 0K 2 +page table in kernel 2211K 2211K 2211K 0K 0K 0K 0K 1 +shared memory 48K 48K 48K 0K 0K 0K 0K 2 +unused but dirty shlib __DATA 79K 79K 79K 0K 0K 0K 0K 39 +=========== ======= ======== ===== ======= ======== ====== ===== ======= +TOTAL 11.4G 1.6G 1.3G 0K 0K 0K 0K 2238 +TOTAL, minus reserved VM space 5.1G 1.6G 1.3G 0K 0K 0K 0K 2238 + + VIRTUAL RESIDENT DIRTY SWAPPED ALLOCATION BYTES DIRTY+SWAP REGION +MALLOC ZONE SIZE SIZE SIZE SIZE COUNT ALLOCATED FRAG SIZE % FRAG COUNT +=========== ======= ========= ========= ========= ========= ========= ========= ====== ====== +DefaultMallocZone_0x1053e0000 68.8M 22.4M 22.4M 0K 5528 22.0M 333K 2% 19 + diff --git a/docs/benchmarks/memory-floods-2026-10-04/vmmap/before-s8a-500.txt b/docs/benchmarks/memory-floods-2026-10-04/vmmap/before-s8a-500.txt new file mode 100644 index 000000000..3e8fb1cd2 --- /dev/null +++ b/docs/benchmarks/memory-floods-2026-10-04/vmmap/before-s8a-500.txt @@ -0,0 +1,65 @@ +Process: igneumd [55718] +Path: /Users/USER/*/igneumd +Load Address: 0x102804000 +Identifier: igneumd +Version: 0 +Code Type: ARM64 +Platform: macOS +Parent Process: node [53051] +Target Type: live task + +Date/Time: 2026-10-05 02:30:49.931 +0100 +Launch Time: 2026-10-05 02:22:34.566 +0100 +OS Version: macOS 26.6.2 (25G83) +Report Version: 7 +Analysis Tool: /usr/bin/vmmap + +Physical footprint: 1.3G +Physical footprint (peak): 1.3G +Idle exit: untracked +---- + +ReadOnly portion of Libraries: Total=898.4M resident=212.1M(24%) swapped_out_or_unallocated=686.3M(76%) +Writable regions: Total=10.3G written=1.3G(13%) resident=1.3G(13%) swapped_out=0K(0%) unallocated=9.0G(87%) + + VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION +REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced) +=========== ======= ======== ===== ======= ======== ====== ===== ======= +IOAccelerator 3.8G 1.3G 1.3G 0K 0K 0K 0K 83 +IOAccelerator (reserved) 6.2G 0K 0K 0K 0K 0K 0K 6 reserved VM address space (unallocated) +Kernel Alloc Once 32K 16K 16K 0K 0K 0K 0K 1 +MALLOC guard page 3984K 0K 0K 0K 0K 0K 0K 4 +MALLOC metadata 880K 528K 528K 0K 0K 0K 0K 4 +MALLOC_SMALL 36.0M 8304K 8304K 0K 0K 0K 0K 9 see MALLOC ZONE table below +MALLOC_SMALL (empty) 12.0M 80K 80K 0K 0K 0K 0K 3 see MALLOC ZONE table below +MALLOC_TINY 4096K 192K 192K 0K 0K 0K 0K 1 see MALLOC ZONE table below +Memory Tag 22 64.0M 16K 16K 0K 0K 0K 0K 1 +STACK GUARD 1424K 0K 0K 0K 0K 0K 0K 89 +Stack 154.2M 2048K 2048K 0K 0K 0K 0K 90 +Stack Guard 56.0M 0K 0K 0K 0K 0K 0K 1 +VM_ALLOCATE 1088K 0K 0K 0K 0K 0K 0K 68 +VM_ALLOCATE (reserved) 8576K 0K 0K 0K 0K 0K 0K 67 reserved VM address space (unallocated) +__AUTH 1321K 925K 0K 0K 0K 0K 0K 149 +__AUTH_CONST 17.7M 11.5M 0K 0K 0K 0K 0K 340 +__CTF 824 824 0K 0K 0K 0K 0K 1 +__DATA 4453K 2379K 346K 0K 0K 0K 0K 297 +__DATA_CONST 17.6M 12.3M 1344K 0K 0K 0K 0K 339 +__DATA_DIRTY 1320K 1072K 351K 0K 0K 0K 0K 284 +__FONT_DATA 2352 2352 0K 0K 0K 0K 0K 1 +__LINKEDIT 573.7M 32.9M 0K 0K 0K 0K 0K 2 +__OBJC_RO 79.2M 59.8M 0K 0K 0K 0K 0K 1 +__OBJC_RW 2599K 2375K 39K 0K 0K 0K 0K 1 +__TEXT 324.7M 179.2M 0K 0K 0K 0K 0K 348 +__TPRO_CONST 128K 112K 48K 0K 0K 0K 0K 2 +page table in kernel 2211K 2211K 2211K 0K 0K 0K 0K 1 +shared memory 48K 48K 48K 0K 0K 0K 0K 2 +unused but dirty shlib __DATA 79K 79K 79K 0K 0K 0K 0K 39 +=========== ======= ======== ===== ======= ======== ====== ===== ======= +TOTAL 11.3G 1.6G 1.3G 0K 0K 0K 0K 2234 +TOTAL, minus reserved VM space 5.1G 1.6G 1.3G 0K 0K 0K 0K 2234 + + VIRTUAL RESIDENT DIRTY SWAPPED ALLOCATION BYTES DIRTY+SWAP REGION +MALLOC ZONE SIZE SIZE SIZE SIZE COUNT ALLOCATED FRAG SIZE % FRAG COUNT +=========== ======= ========= ========= ========= ========= ========= ========= ====== ====== +DefaultMallocZone_0x1053e0000 52.8M 9072K 9072K 0K 5461 9157K 0K 0% 15 + diff --git a/docs/benchmarks/round4-consensus-2026-10-04/redteam-repro/rt-f23.json b/docs/benchmarks/round4-consensus-2026-10-04/redteam-repro/rt-f23.json new file mode 100644 index 000000000..f9405ec7e --- /dev/null +++ b/docs/benchmarks/round4-consensus-2026-10-04/redteam-repro/rt-f23.json @@ -0,0 +1,8 @@ +[ + { + "scenario": "F23 (equivocation ban node-local; honest nodes refuse each other's certs)", + "expected": "ban expiry identical across honest nodes; 0 voter-count refusals; 0 CONFLICTING; 0 disagreeing locked indices", + "observed": "equiv detections 14/7/7; stripped-until per node - | - | - (distinct values 0); voter-count-mismatch refusals 0/0/0; CONFLICTING 0/0/0; disagreeing locked indices 0; maxLocked 61/61/61", + "pass": true + } +] \ No newline at end of file diff --git a/docs/benchmarks/round4-consensus-2026-10-04/redteam-repro/rt-f24b.json b/docs/benchmarks/round4-consensus-2026-10-04/redteam-repro/rt-f24b.json new file mode 100644 index 000000000..92df1947e --- /dev/null +++ b/docs/benchmarks/round4-consensus-2026-10-04/redteam-repro/rt-f24b.json @@ -0,0 +1,8 @@ +[ + { + "scenario": "F24b (deep reorg under merge depth; determination never revisited)", + "expected": "after a reorg deeper than checkpoint_depth the losing node re-determines the moved indices and accepts the network certificates; 0 false CONFLICTING, 0 stuck indices, 0 disagreeing locks", + "observed": "n1 determined 32..33 during the 24s cut; after heal: cert-for-other-block refusals 0/26, CONFLICTING 7/3, equivocation 0/0, PoW-rejected 1668/2025, sinks equal false, disagreeing locked indices 9, n1 indices stuck unlocked that n0 locked [33], maxLocked 54/43", + "pass": false + } +] \ No newline at end of file diff --git a/docs/benchmarks/round4-consensus-2026-10-04/redteam-repro/rt-f24c.json b/docs/benchmarks/round4-consensus-2026-10-04/redteam-repro/rt-f24c.json new file mode 100644 index 000000000..810298e55 --- /dev/null +++ b/docs/benchmarks/round4-consensus-2026-10-04/redteam-repro/rt-f24c.json @@ -0,0 +1,8 @@ +[ + { + "scenario": "F24c (3/3 split, 16 s cut under merge depth; determination never revisited)", + "expected": "after a reorg deeper than checkpoint_depth the losing node re-determines the moved indices and accepts the network certificates; 0 false CONFLICTING, 0 stuck indices, 0 disagreeing locks", + "observed": "n1 determined 31..32 during the 16s cut; after heal: cert-for-other-block refusals 0/0, CONFLICTING 0/0, equivocation 0/0, PoW-rejected 1935/1930, sinks equal false, disagreeing locked indices 0, n1 indices stuck unlocked that n0 locked [], maxLocked 61/61", + "pass": true + } +] \ No newline at end of file diff --git a/docs/benchmarks/round4-consensus-2026-10-04/results-control-finality-fixes-6aa69a45.md b/docs/benchmarks/round4-consensus-2026-10-04/results-control-finality-fixes-6aa69a45.md new file mode 100644 index 000000000..5c3e37410 --- /dev/null +++ b/docs/benchmarks/round4-consensus-2026-10-04/results-control-finality-fixes-6aa69a45.md @@ -0,0 +1,37 @@ + +### digest-old: n1 on the shared fast-time override, n0 with finality.weight_window 121 (one DAA second more), n2 on the shared override + +| measure | n0 (mismatched) | n1 (listener) | n2 (matching) | +|---|---|---|---| +| params digest printed at start | none | none | none | +| "consensus params digest mismatch" lines | 0 | 0 | 0 | +| peers after 25 s (n0, n1) and after n2 dialled (n1) | 1 | 1 then 2 | connected after 1 s | + +### ban-old: 480 s, 1 blocks/s in all, 6 voters, delay 100 ms, a0 equivocates once at index 9; P2 cut at 259 s, healed at 304 s; n0 detected the equivocation at 291 s + +| measure | n0 (saw it over RPC) | n1 (from the block at once) | n2 (from the block, after the heal) | +|---|---|---|---| +| EQUIVOCATION lines (of which "carried by block") | 2 (0) | 1 (0) | 1 (0) | +| certificates refused "names N voters, this node counts M" | 2 | 0 | 0 | +| CONFLICTING certificate lines | 0 | 0 | 0 | +| indices whose certificates name 5 voters (a0 stripped) | 10..13 (4) | 10..13 (4) | 10..13 (4) | +| max locked index at the end | 14 | 14 | 14 | + +indices with certificate lines on at least two nodes: voter counts agree at 9, differ at 0; locked indices disagreeing across the three nodes: 0 + +### reorg-old: warm 230 s, split 180 s (n0 alone with 30% of the weight), heal window 150 s, 1 blocks/s in all, delay 100 ms + +| measure | n0 (cut off, 30%) | n1 (70% side) | n2 (70% side) | +|---|---|---|---| +| max locked index at the cut | 7 | 7 | 7 | +| max locked index at the heal | 7 | 12 | 12 | +| max locked index at the end | 17 | 17 | 17 | +| "re-determined" lines | 0 | 0 | 0 | +| certificates kept pending | 0 | 0 | 0 | +| CONFLICTING certificate lines | 0 | 0 | 0 | + +n0 determined 2 checkpoint(s) on its own chain during the split (indices 8, 9); after the heal n0 holds the same locked block as n1 at 1 of them; locked indices disagreeing across the three nodes: 0; n0 reconnected 10 s after the gate reopened + +[FAIL] digest-old +[FAIL] ban-old +[FAIL] reorg-old diff --git a/docs/benchmarks/round4-consensus-2026-10-04/results-final.md b/docs/benchmarks/round4-consensus-2026-10-04/results-final.md new file mode 100644 index 000000000..3dc65625e --- /dev/null +++ b/docs/benchmarks/round4-consensus-2026-10-04/results-final.md @@ -0,0 +1,44 @@ + +### digest-final: n1 on the shared fast-time override, n0 with finality.weight_window 121 (one DAA second more), n2 on the shared override + +| measure | n0 (mismatched) | n1 (listener) | n2 (matching) | +|---|---|---|---| +| params digest printed at start | 4bf763ba5b78c6ac88463932f522679186cb641f631671eb25fc17b01071aea9 | 7a40cc3b90c7726b9813113448510bd0597856baf9f0e6f245e3cf5cac494852 | 7a40cc3b90c7726b9813113448510bd0597856baf9f0e6f245e3cf5cac494852 | +| "consensus params digest mismatch" lines | 2 | 2 | 0 | +| peers after 25 s (n0, n1) and after n2 dialled (n1) | 0 | 0 then 1 | connected after 1 s | + +n0's line: `WARN ] P2P, got reject message: consensus params digest mismatch - local: 7a40cc3b90c7726b9813113448510bd0597856baf9f0e6f245e3cf5cac494852, remote: 4bf763ba5b78c6ac88463932f522679186cb641f631671eb25fc17b01071aea9: the peer's override file, environment or build differs from peer: 127.0.0.1:29411` + +### ban-final: 480 s, 1 blocks/s in all, 6 voters, delay 100 ms, a0 equivocates once at index 9; P2 cut at 252 s, healed at 297 s; n0 detected the equivocation at 288 s + +| measure | n0 (saw it over RPC) | n1 (from the block at once) | n2 (from the block, after the heal) | +|---|---|---|---| +| EQUIVOCATION lines (of which "carried by block") | 2 (1) | 1 (1) | 1 (1) | +| certificates refused "names N voters, this node counts M" | 0 | 0 | 0 | +| CONFLICTING certificate lines | 0 | 0 | 0 | +| indices whose certificates name 5 voters (a0 stripped) | 10..12 (3) | 10..12 (3) | 10..12 (3) | +| max locked index at the end | 15 | 15 | 15 | + +indices with certificate lines on at least two nodes: voter counts agree at 10, differ at 0; locked indices disagreeing across the three nodes: 0 + +### reorg-final: warm 230 s, split 180 s (n0 alone with 30% of the weight), heal window 150 s, 1 blocks/s in all, delay 100 ms + +| measure | n0 (cut off, 30%) | n1 (70% side) | n2 (70% side) | +|---|---|---|---| +| max locked index at the cut | 7 | 7 | 7 | +| max locked index at the heal | 7 | 11 | 11 | +| max locked index at the end | 16 | 16 | 16 | +| "re-determined" lines | 2 | 0 | 0 | +| certificates kept pending | 2 | 0 | 0 | +| CONFLICTING certificate lines | 0 | 0 | 0 | +| certificates refused over another block, pre-F24 wording | 0 | 0 | 0 | +| pending certificates verified at determination (did not verify) | 2 (0) | 0 (0) | 0 (0) | +| indices n1 locked that this node did not lock | none | | | + +n0 determined 2 checkpoint(s) on its own chain during the split (indices 8, 9); after the heal n0 holds the same locked block as n1 at 0 of them; locked indices disagreeing across the three nodes: 0; n0 reconnected 10 s after the gate reopened + Finality: checkpoint 8 re-determined: block a806eb0744d0e12c09c1de08bd6afc2445cd4c27edff2f78d50ed296c33fbfb7 (blue score 240, daa 239), was 46cfb6b02eb5728cba01cd729d87463fb3bd4603abbbf13921b4067b3f1b5895: the selected chain moved past it + Finality: checkpoint 9 re-determined: block 8c1b51dd691441543fcb76809c67058d1c161b432ad091d949e1879270225174 (blue score 263, daa 262), was 88138fd98a72ac2ec4a5778f97c3f0dd913a2dfc70335ff6fb487649e8ffc422: the selected chain moved past it + +[PASS] digest-final +[PASS] ban-final +[FAIL] reorg-final diff --git a/docs/benchmarks/round4-consensus-2026-10-04/results-final2.md b/docs/benchmarks/round4-consensus-2026-10-04/results-final2.md new file mode 100644 index 000000000..2096ee321 --- /dev/null +++ b/docs/benchmarks/round4-consensus-2026-10-04/results-final2.md @@ -0,0 +1,20 @@ + +### reorg-final2: warm 230 s, split 180 s (n0 alone with 30% of the weight), heal window 150 s, 1 blocks/s in all, delay 100 ms + +| measure | n0 (cut off, 30%) | n1 (70% side) | n2 (70% side) | +|---|---|---|---| +| max locked index at the cut | 8 | 8 | 8 | +| max locked index at the heal | 8 | 8 | 8 | +| max locked index at the end | 16 | 16 | 16 | +| "re-determined" lines | 1 | 0 | 0 | +| certificates kept pending | 1 | 0 | 0 | +| CONFLICTING certificate lines | 0 | 0 | 0 | +| certificates refused over another block, pre-F24 wording | 0 | 0 | 0 | +| pending certificates verified at determination (did not verify) | 1 (0) | 0 (0) | 0 (0) | +| indices n1 locked that this node did not lock | none | | | +| records whose block is below the index's target blue score | none | | | + +n0 determined 1 checkpoint(s) on its own chain during the split (indices 9); after the heal n0 holds the same locked block as n1 at 0 of them; locked indices disagreeing across the three nodes: 0; n0 reconnected 10 s after the gate reopened + Finality: checkpoint 9 re-determined: block dd8f57d7a5bc1a80eab4d87704310396c614d7cdb5ea21595694b9453b21d1bf (blue score 270, daa 269), was c3379892186e5e96e679bdfb3ec6991a9cb41ebe953e84baf3f1cf0ba07388d6: the selected chain moved past it + +[PASS] reorg-final2 diff --git a/docs/benchmarks/round4-consensus-2026-10-04/results-first-pass-9f738e2e.md b/docs/benchmarks/round4-consensus-2026-10-04/results-first-pass-9f738e2e.md new file mode 100644 index 000000000..10dd52c50 --- /dev/null +++ b/docs/benchmarks/round4-consensus-2026-10-04/results-first-pass-9f738e2e.md @@ -0,0 +1,42 @@ + +### digest-fud: n1 on the shared fast-time override, n0 with finality.weight_window 121 (one DAA second more), n2 on the shared override + +| measure | n0 (mismatched) | n1 (listener) | n2 (matching) | +|---|---|---|---| +| params digest printed at start | 4bf763ba5b78c6ac88463932f522679186cb641f631671eb25fc17b01071aea9 | 7a40cc3b90c7726b9813113448510bd0597856baf9f0e6f245e3cf5cac494852 | 7a40cc3b90c7726b9813113448510bd0597856baf9f0e6f245e3cf5cac494852 | +| "consensus params digest mismatch" lines | 1 | 2 | 0 | +| peers after 25 s (n0, n1) and after n2 dialled (n1) | 0 | 0 then 1 | connected after 1 s | + +n0's line: `WARN ] Refusing peer 127.0.0.1:29411: consensus params digest mismatch, local 4bf763ba5b78c6ac88463932f522679186cb641f631671eb25fc17b01071aea9 remote 7a40cc3b90c7726b9813113448510bd0597856baf9f0e6f245e3cf5cac494852 (the peer's override file, environment or build differs)` + +[PASS] digest-fud + +### ban-fud: 480 s, 1 blocks/s in all, 6 voters, delay 100 ms, a0 equivocates once at index 9; P2 cut at 259 s, healed at 304 s; n0 detected the equivocation at 301 s + +| measure | n0 (saw it over RPC) | n1 (from the block at once) | n2 (from the block, after the heal) | +|---|---|---|---| +| EQUIVOCATION lines (of which "carried by block") | 2 (1) | 1 (1) | 1 (1) | +| certificates refused "names N voters, this node counts M" | 0 | 0 | 0 | +| CONFLICTING certificate lines | 0 | 0 | 0 | +| indices whose certificates name 5 voters (a0 stripped) | 10..13 (4) | 10..13 (4) | 10..13 (4) | +| max locked index at the end | 14 | 14 | 14 | + +indices with certificate lines on at least two nodes: voter counts agree at 11, differ at 0; locked indices disagreeing across the three nodes: 0 + +### reorg-fud: warm 230 s, split 180 s (n0 alone with 30% of the weight), heal window 150 s, 1 blocks/s in all, delay 100 ms + +| measure | n0 (cut off, 30%) | n1 (70% side) | n2 (70% side) | +|---|---|---|---| +| max locked index at the cut | 7 | 7 | 7 | +| max locked index at the heal | 7 | 11 | 11 | +| max locked index at the end | 15 | 15 | 15 | +| "re-determined" lines | 2 | 0 | 0 | +| certificates kept pending | 4 | 0 | 0 | +| CONFLICTING certificate lines | 0 | 0 | 0 | + +n0 determined 1 checkpoint(s) on its own chain during the split (indices 8); after the heal n0 holds the same locked block as n1 at 0 of them; locked indices disagreeing across the three nodes: 0; n0 reconnected 40 s after the gate reopened + Finality: checkpoint 8 re-determined: block ce42c8457e85d754f842851e685aa141f3f46fe6de4e5f707c8239cdef7f72e5 (blue score 240, daa 239), was c3cc4e6bbf573e2b24a763728f76783ca2cd41c62194a00a028f3b28cfb1ee12: the selected chain moved past it + Finality: checkpoint 9 re-determined: block eed5a7f19d11600695f5027327fd053ecf911453955c97d906bb24601601b4fb (blue score 261, daa 260), was e2d7874fd1e059996eb2ee36aca1bbda09dcbb168ec8030315ca24df1a8601b0: the selected chain moved past it + +[PASS] ban-fud +[PASS] reorg-fud diff --git a/docs/benchmarks/round4-consensus-2026-10-04/results-old2.md b/docs/benchmarks/round4-consensus-2026-10-04/results-old2.md new file mode 100644 index 000000000..8c00f2459 --- /dev/null +++ b/docs/benchmarks/round4-consensus-2026-10-04/results-old2.md @@ -0,0 +1,18 @@ + +### reorg-old2: warm 230 s, split 180 s (n0 alone with 30% of the weight), heal window 150 s, 1 blocks/s in all, delay 100 ms + +| measure | n0 (cut off, 30%) | n1 (70% side) | n2 (70% side) | +|---|---|---|---| +| max locked index at the cut | 7 | 7 | 7 | +| max locked index at the heal | 7 | 11 | 11 | +| max locked index at the end | 15 | 15 | 15 | +| "re-determined" lines | 0 | 0 | 0 | +| certificates kept pending | 0 | 0 | 0 | +| CONFLICTING certificate lines | 0 | 0 | 0 | +| certificates refused over another block, pre-F24 wording | 3 | 0 | 0 | +| pending certificates verified at determination (did not verify) | 0 (0) | 0 (0) | 0 (0) | +| indices n1 locked that this node did not lock | 8 9 | | | + +n0 determined 1 checkpoint(s) on its own chain during the split (indices 8); after the heal n0 holds the same locked block as n1 at 0 of them; locked indices disagreeing across the three nodes: 0; n0 reconnected 40 s after the gate reopened + +[FAIL] reorg-old2 diff --git a/docs/fud-ledger.md b/docs/fud-ledger.md index 56971c1fb..12788b36b 100644 --- a/docs/fud-ledger.md +++ b/docs/fud-ledger.md @@ -1649,9 +1649,11 @@ Evidence: the files above. ### G12. The PoW schedule comes from the environment on every network, including mainnet "Your mainnet gate refuses the override file. It does not refuse `IGNEUM_POW_EPOCH_BLOCKS`. A node without a file installs the schedule from the environment and `Params.pow_epoch_blocks` is never consulted." -Status: Open (4 October 2026). +Status: Fix built, pending rollout (4 October 2026, night; node branch `fud-consensus`, local worktree `vendor/igneum-node-fud`, no remote; main repo branch `fud-consensus`). Was: Open (4 October 2026). -Answer: Correct. `daemon.rs:339-340` installs the file's schedule only when the file names one; otherwise `PowSchedule::from_env()` (`consensus/core/src/igneum.rs:137-145`) installs on first read; the difficulty manager takes the global (`services.rs:113`); `daemon.rs:316-319` gates the file only. Fix: delete the environment fallback and install `Params.pow_epoch_blocks` from the network params on every start. Review id R4.1.1. +The fix: `PowSchedule::from_env()` is gone. `PowSchedule::from_env_for(network, base)` applies the three variables on devnet and simnet only and returns `None` elsewhere; the daemon calls `Params::apply_env_pow_schedule()` after the override file, prints "PoW schedule from the environment (...)" when applied and "Ignoring IGNEUM_POW_... on igneum-mainnet: the environment never sets a consensus parameter outside devnet and simnet" when not, then installs the network's schedule from `Params` on every network (`install_pow_schedule`), so `Params.pow_epoch_blocks` is what runs; the lazy fallback in `pow_schedule()` installs the devnet constants, never the environment. The miner takes all three schedule values from the template (`pow_epoch.day_ms` joined the two epoch fields in `PowEpochInfo`, the RPC model and the gRPC proto), so `IGNEUM_POW_DAY_MS` has no reader left in the miner (R4.1.9's day split is closed with it). The effective schedule is part of the params digest of X18, so a devnet node with the variable set cannot connect to one without it. Unit test `env_pow_schedule_is_devnet_and_simnet_only` (consensus-core, `config::params::tests`): the variable moves the devnet and simnet schedule and digest, leaves mainnet's and testnet's untouched, and the caller can tell "ignored" from "nothing set". Spec 2.8 and 8.7 state the rule. + +Answer (as found): Correct. `daemon.rs:339-340` installs the file's schedule only when the file names one; otherwise `PowSchedule::from_env()` (`consensus/core/src/igneum.rs:137-145`) installs on first read; the difficulty manager takes the global (`services.rs:113`); `daemon.rs:316-319` gates the file only. Fix: delete the environment fallback and install `Params.pow_epoch_blocks` from the network params on every start. Review id R4.1.1. Evidence: the files above. Experiment: start a node with `IGNEUM_POW_EPOCH_BLOCKS=60` and no file; its template's `epoch_blocks` must be the network's value. @@ -1676,30 +1678,49 @@ Evidence: `git ls-files | xargs grep -lF ` counts, `git log -S`. Experime ### X18. Two nodes with two override files connect, and only some mismatches fork "Your handshake compares the network name and nothing else. A PoW or difficulty mismatch forks and bans; a `finality` mismatch is a WARN; `rollout-v2.sh` throws the finality block away when it writes the file; the app rewrites the packaged file on every start." -Status: Open (4 October 2026). +Status: Fix built, pending rollout (4 October 2026, night; node branch `fud-consensus`, main repo branch `fud-consensus`). Was: Open (4 October 2026). -Answer: Correct. `protocol/flows/src/flow_context.rs:833` compares `network`; the version message has no params digest and no genesis hash. `pre_pow_validation.rs:37` and `pow_guard.rs:25-42` fork and ban on PoW and difficulty fields; `processes/finality.rs:669-688` only warns on a finality mismatch; `infra/cloud-devnet/rollout-v2.sh:20,35` rewrites the file as two fields; `app/igneum-app/src/engine.rs:742-760` rewrites `override-params.json` each start. Fix: a digest of the effective consensus params plus the genesis hash in the version message, refused on mismatch; the finality WARN becomes a refusal with the reason; `rollout-v2.sh` merges rather than replaces. Review ids R4.1.2, R4.1.12. +The fix: `Params::consensus_digest()` (BLAKE2b-256, domain `IgneumParamsDigest`, every consensus field in a fixed tagged order: genesis, difficulty, mass and lane limits, blockrate, crescendo, the ten finality fields, the PoW schedule, the three activation heights; not the dead `timestamp_deviation_tolerance`, not seeders or ports; spec 2.8 lists it). The version message carries it (`paramsDigest`, field 11); `initialize_connection` refuses a peer whose digest differs with `ProtocolError::ParamsDigestMismatch` and one WARN naming both digests before any flow is registered, so a finality-only mismatch, which used to connect and WARN "names N voters" after the fact, never connects. A peer with no digest (an older build) is refused on mainnet and testnet and let in with a WARN on devnet and simnet while the devnet rolls (an allowance to remove afterwards). The node prints its digest at start. Unit test `consensus_digest_covers_every_consensus_field_and_nothing_else`. Measured (`docs/bench-log.md`, "round-4 consensus items", digest run; `tools/finality-attacks/fud.mjs digest`, fast time, ports 29400+): a listener on the shared fast-time override and a dialler whose finality block differs by one DAA second of window: the dialler was refused at the handshake on both sides ("Refusing peer ...: consensus params digest mismatch, local 4bf7... remote 7a40..." on the listener, the reject message on the dialler), 0 peers after 25 s on both; a third node on the shared override connected in 1 s. Control on the finality-fixes build 6aa69a45: the mismatched dialler connected (1 peer, no line). `rollout-v2.sh` still rewrites the file as two fields and the app still rewrites the packaged file (R4.1.12): with the digest both now fail loudly at the handshake instead of forking; the merge fix for the script is not done tonight. + +Answer (as found): Correct. `protocol/flows/src/flow_context.rs:833` compares `network`; the version message has no params digest and no genesis hash. `pre_pow_validation.rs:37` and `pow_guard.rs:25-42` fork and ban on PoW and difficulty fields; `processes/finality.rs:669-688` only warns on a finality mismatch; `infra/cloud-devnet/rollout-v2.sh:20,35` rewrites the file as two fields; `app/igneum-app/src/engine.rs:742-760` rewrites `override-params.json` each start. Fix: a digest of the effective consensus params plus the genesis hash in the version message, refused on mismatch; the finality WARN becomes a refusal with the reason; `rollout-v2.sh` merges rather than replaces. Review ids R4.1.2, R4.1.12. Evidence: the files above. Experiment: two nodes on different files; the handshake must fail with the field named. ### F23. The equivocation ban is node-local, so honest nodes refuse each other's certificates "Evidence detected from an RPC vote stamps the sink's DAA; evidence carried in a block stamps the carrier's DAA. Two honest nodes hold different `until` for the same key, their voter lists differ by one at every checkpoint between the two expiries, and `voter_count` refuses the other's certificate for good." -Status: Open (4 October 2026). +Status: Fix built, pending rollout (4 October 2026, night; node branch `fud-consensus`, main repo branch `fud-consensus`). Was: Open (4 October 2026); reproduced by the red team the same evening on the stock s1 scenario (n0 kept the ban until DAA 726 against 239 on n1 and n2, 9 and 3 to 4 certificates refused "names N voters"). -Answer: Correct. `ingest_evidence` (`processes/finality.rs:600-612`); `ingest_certificate` (`:680-688`). Certificate validity is not a function of the DAG, the same defect R3.9 found in the execution veto. Fix: stamp every ban with the DAA of the block that carries the evidence; evidence seen by RPC is only acted on once carried. Review id R4.1.3. +The fix: the node-local `stripped` map is gone. Evidence is kept as `EvidenceRecord` (the two votes, the carriers with their DAA scores) and the ban at a checkpoint C is a function of C's past (`bans_at`): the key is stripped at C when some carrier lies in C's past and `daa(C) < daa(lowest carrier in C's past) + ban`. Evidence detected over RPC or gossip strips nothing until a block carries it; the node puts it in its next templates. The weight table cache stays ban-free and `voters_at` applies the checkpoint's own bans, so a certificate built before the carrier existed verifies on a node that saw the evidence later, and a node that saw it over RPC counts the same voters as one that saw it in the block. Evidence records are bounded (4,096; dropped once the ban ended two windows below the sink or never carried within one ban of being seen; 16 carriers per record), the same for vote and certificate carriers. The persisted state is layout 2; a layout-1 blob is read and converted on start, so no devnet node loses its locks on the upgrade. Unit test `ban_is_decided_by_the_carrying_block_so_nodes_agree_on_every_voter_list` (three `TestConsensus` nodes on one chain: the voter list agrees on all three at every checkpoint, the key is a voter before the carrier and after the ban and nowhere in between, the third node verifies the first two's certificates at every locked index). Measured (`docs/bench-log.md`, "round-4 consensus items", ban run; `fud.mjs ban`, 480 s, six voters, one equivocation at index 9 by a voter on n0 over RPC, n2 cut off 45 s around it and healed, so it saw the evidence late from the carrier block): on the `fud-consensus` build every node names 5 voters at the same indices (10 to 12 in the final pass, 10 to 13 in the first), 0 certificates refused "names N voters", 0 conflicting certificates, 0 locked indices disagreeing, voter counts agree at every index with lines on two or more nodes, locks continue to index 14 or 15 on all three. Control on the finality-fixes build: n0 (the RPC detector) refused 2 certificates "names N voters, this node counts M", the rest agreed because each node built its own; the red team's stock s1 scenario the same evening gave 9 / 3 / 4 refusals. Spec 3.6 and the 3.10 row state the rule. + +Answer (as found): Correct. `ingest_evidence` (`processes/finality.rs:600-612`); `ingest_certificate` (`:680-688`). Certificate validity is not a function of the DAG, the same defect R3.9 found in the execution veto. Fix: stamp every ban with the DAA of the block that carries the evidence; evidence seen by RPC is only acted on once carried. Review id R4.1.3. Evidence: the files above. Experiment: `tools/finality-attacks` with one equivocation detected on node A by RPC and on node B from the carrying block 30 DAA later; count certificates refused with "names N voters" between the two expiries; after the fix, zero. +Red-team run, 4 October 2026 (evening, the 0.3.4 finality-fixes build with rule v3 on, `docs/review/redteam-2026-10-04.md` row 15): reproduced by the stock scenario 1 (two keys equivocating at every index, 4 honest voters, 3 nodes, fast time). The node that received the equivocators' votes by RPC re-detected at every index (46 detections) and held the ban until DAA 726; the two nodes that saw the evidence only in blocks detected it at indices 1 to 3 (8 detections, ban until 239) and let it expire. The first detection alone stamped `until` 174 and 176 on the RPC node against 176 on the others. From index 9 the voter lists differed by two keys and the nodes refused each other's certificates: 9 refusals "names 6 voters, this node counts 4" on the RPC node, 3 and 4 refusals "names 4 voters, this node counts 6" on the others. Every node still locked 15 of 15 only because each could aggregate its own certificate from the votes it held; with 8 named aggregators on a real network that fallback is `aggregator_fallback` later and a node whose certificate the rest refuse is one more aggregation round behind at every index. Rule v3 does not touch this path. Severity stays serious; the fix above stands. + ### F24. A checkpoint determination is never revisited "After a reorg deeper than `checkpoint_depth`, the node's record for that index names a block off its chain. Every certificate the network forms for that index is refused as conflicting, with no equivocation anywhere, and the node voted for a block that is not on its chain." -Status: Open (4 October 2026); extends F7 and C4. +Status: Fix built, pending rollout (4 October 2026, night; node branch `fud-consensus`, main repo branch `fud-consensus`). Was: Open (4 October 2026); extends F7 and C4. -Answer: Correct. `on_virtual_changed` (`processes/finality.rs:404-440`) inserts once and advances `next_index`; `:661-666` refuses any certificate whose checkpoint is not the node's record. Fix: re-determine an unlocked index when the virtual's chain at that blue score changes; refuse only a certificate that conflicts with a lock. Review id R4.1.4. +The fix: after every virtual change, every unlocked checkpoint record whose block is no longer a chain ancestor of the sink is determined again on the new chain ("re-determined" log line); the certificate held over the old block is dropped, the fold clock restarts. A certificate over a block other than the node's determination at an unlocked index (or at an index not yet determined, up to 64 ahead) is no longer logged CONFLICTING and discarded: it is held pending (4 per index) and verified when a re-determination names its block; a block that cannot be the index's checkpoint on any chain (C1 as a function of the DAG: blue score under the target, or the selected parent's not) is refused outright. CONFLICTING now means what 3.11.4 says: a certificate against a LOCK. The certificate verification itself is unchanged; a locked index is never re-determined (fork choice keeps the chain through it). The node's own keys do not re-vote at a re-determined index (that would be equivocation); the lock there comes from the network's certificate, which is the point. Unit tests `reorg_past_an_unlocked_checkpoint_re_determines_it_and_verifies_the_pending_certificate` and `a_locked_checkpoint_pins_the_chain_and_a_certificate_against_it_conflicts`. Measured (`docs/bench-log.md`, "round-4 consensus items", reorg run; `fud.mjs reorg`, n0 with 30% of the weight cut off for 180 s while the 70% side kept locking, then healed): on the `fud-consensus` build n0 re-determined its own 1 to 2 split indices on the majority chain, verified the pending certificates at determination (2 in the final pass), logged 0 CONFLICTING and 0 refusals, and ended with every index the majority locked locked on the same block (0 disagreeing). Control on the finality-fixes build: n0 refused 3 certificates "is for X, this node's checkpoint is Y" and never locked indices 8 and 9 that the majority locked (the permanent hole of this entry), 0 disagreeing because the hole is not a lock. The final pass also found that a chain which becomes the sink at a lower blue score than the old one (more blue work per block after the split's difficulty drift) re-determined index 9 at a sink below its depth, to a block below the target: fixed the same night (an index the new chain has not reached is un-determined and determined again when it has; unit test `a_shallower_sink_un_determines_the_indices_it_cannot_reach`) and re-run (`reorg-final2`, fork 977db931): the majority locked nothing during the split this time (Poisson again), n0 re-determined its one split index, verified 1 pending certificate at determination, 0 CONFLICTING, 0 refused, 0 disagreeing, no record below its target, every index the majority locked locked on n0 too. The red team's own reproductions on the new build (`tools/finality-attacks/redteam/rtfin.mjs`, fin-attacks miner at 6 blocks/s): `f24c` (3/3 split, 16 s cut) PASS with 0 refusals, 0 CONFLICTING, 0 stuck indices, 0 disagreeing; `f24b` (4/2 split, 24 s cut) FAIL on both builds for a reason outside F24: at 6 blocks/s the DAA score advances 6 a second, so the 24-s cut is 144 DAA, longer than the 120-DAA weight window and the 60-DAA merge depth; the two chains cannot merge at the heal and each side's table holds only its own keys (n1's lone locks read "100.0% of total, 100.0% of the table frozen at lock 39"), which is the partition longer than a window that spec 3.7 item 9 states and F21 conceded, not a reorg. The scenario's "under merge depth" assumes 1 DAA a second. Spec 3.2 C1 and C4, and the 3.10 rows, state the rule. + +Answer (as found): Correct. `on_virtual_changed` (`processes/finality.rs:404-440`) inserts once and advances `next_index`; `:661-666` refuses any certificate whose checkpoint is not the node's record. Fix: re-determine an unlocked index when the virtual's chain at that blue score changes; refuse only a certificate that conflicts with a lock. Review id R4.1.4. Evidence: the files above. Experiment: a 30 s cut on a 3-node devnet at d = 20; the losing side must accept the network's certificate at that index with no CONFLICTING line. +Red-team run, 5 October 2026, 00:56 (the 0.3.4 finality-fixes build, rule v3 on, fast time, `docs/review/redteam-2026-10-04.md` row 23b): reproduced on a clean merge. Two nodes with three voting keys each, cut for 16 s (about 50 blue blocks a side, under the 60-DAA merge depth), healed: sinks equal, 64 locks each, no conflicting certificate. Checkpoint 33 was determined during the cut on each side's own chain (two different blocks); after the reorg neither node re-determined it, neither block ever got a certificate (votes split 3/3), and finality went on from 34. A permanent one-index hole on every node, with no equivocation anywhere. The round-4 shape, a false CONFLICTING line, needs the other side's certificate to arrive, which a 3/3 split cannot form; the two 4/2 attempts (rows 22 and 23) showed the stuck indices and the refusals "this node's checkpoint is ..." but overshot merge depth, so they are confounded with F21. Rule v3 does not touch this path. The fix above stands; the two-node test is `rtfin.mjs f24c` in the red-team scratchpad (cut 16 s, 3/3), which should end with index 33 locked on both nodes. + +### F25. The fast-time harnesses cannot start a node, and the timestamp probe tests the old rule +"Both attack harnesses rebuild each node's override with `JSON.parse` and `JSON.stringify` of `infra/fast-time/override-60x.json`. That file now carries two `u64::MAX` sentinels (`difficulty_v2_activation_daa`, `proving_v0_activation_daa`); a JavaScript number cannot hold them, the round-trip writes `18446744073709552000`, and `igneumd` refuses the file as a floating point where a u64 is expected. Every `--fast-time` run of `tools/finality-attacks` and `tools/harness` fails at the first node. Scenario 2 of `tools/harness` still probes the 132 s future bound and reports FAIL against the 10 s rule the node has carried since the timestamp fix." + +Status: Fixed in both harnesses (4 October 2026, night: `tools/finality-attacks/lib/net.mjs` kept the sentinels as BigInt through the merge since the v3 runner of the evening; `tools/harness/lib/net.mjs` got the same reviver on branch `fud-memory`, merged into `fud-consensus`); the stale timestamp criterion of scenario 2 is not touched. Was: Open (4 October 2026, red-team run). Tooling, low severity: no consensus effect, but every fast-time attack run is blind until it is fixed. + +Answer: Correct, measured. The red-team run's first scenario errored on it (`docs/review/redteam-2026-10-04.md`, "Tooling defect"); `tools/proving-v0/run.mjs` already edits the file as text for this reason. Scenario 2 live probe: past floor pmt+1, future flip between +130.00 and +130.01 s of the probe's own offsets, every stamp from +10 s rejected; the node is right, the criterion is stale. Smallest fix: in `tools/finality-attacks/lib/net.mjs` and `tools/harness/lib/net.mjs` `overrideParams`, drop the two sentinel fields before `stringify` (absent means never) or splice the extra fields into the file text; in `tools/harness/scenarios/s2-timestamp.mjs`, probe `max(pmt + 1, parent - 10 s)` and the +10 s bound. Also stale: `tools/exec-attacks/scenario3_pgas.mjs` waits for an over-budget transaction to be included and skipped with `BlockProvingBudget`; since F-exec-B the mempool refuses it with the metered pgas, so the check should accept `ProvingGasAboveBlockLimit` from the pool (`docs/review/redteam-2026-10-04.md` row 28). And `tools/finality-attacks` scenario 5 compares the burster's share of the weight window with its share of the whole run, which only agree when the run is shorter than the window (row 17). + +Evidence: the first run's `/tmp/igneum-redteam-fin/n0/node.log` parse line (kept in the session scratchpad `rt/logs/fa_s8/n0/node.log`), `rt/logs/ord_s2.log`. + ### X19. Operational knobs and silences in the shipped node "A slow-clock node disconnects every peer on every relayed block and never says why; the handshake's `time_offset` is computed and unused; `IGNEUM_ATTACK_TS_OFFSET_MS` and `IGNEUM_POW_STRIKES` are compiled into the live binary; `timestamp_deviation_tolerance` is dead and still accepted." @@ -1801,6 +1822,28 @@ Answer: Correct. `site/litepaper.html:424`; the app's sources have no earnings, Evidence: the files above. +### M30. A block or transaction flood grows the 0.3.4 node by hundreds of megabytes in a minute +"On the 3 October ordering-layer node (no execution layer) the resource-exhaustion scenario grew RSS by 4, 11 and 14 MB and the 50x block flood by 30 MB. On the 0.3.4 build, same harness, same scenarios, same 60 s: template flood +6 MB, submit flood +269 MB, mempool flood +270 MB, block flood 302 to 1,082 MB on both nodes (567 MB at 10 s, 824 MB at 20 s). The harness calls it a pass because its bound is baseline + 512 MB; a peer that keeps going is not bounded by the harness." + +Status: Fix built, pending rollout (4 October 2026, night; node branch `fud-memory`, merged into `fud-consensus`; main repo branch `fud-memory`, merged into `fud-consensus`). Was: Open (4 October 2026, red-team run). Serious: a single peer at 50 blocks/s or 500 transactions/s is the devnet's own fast-miner event, and a node that grows 13 MB/s under it runs out of memory in minutes on the 2 to 4 GB cloud nodes. + +The cause, measured (not the one guessed below): every RSS step in both floods is one `PoW cache built` line, 256 MiB each. The lottery engine (`consensus/pow/src/igneum.rs`, `IgneumEngine`) keyed its resident entries by `(epoch seed, day)` and built a full 256 MiB cache per entry, `KEEP = 4`, although the cache depends on the day seed alone (`igneum_pow::Epoch::from_seed_bytes`: cache from the day bytes, program from the epoch seed). The floods ran on the 60x profile, where an epoch rolls every 60 DAA, so the 50x block flood rolled it every 10 to 20 s and paid a cache each time; the 3 October run was on the devnet profile (3,600-DAA epochs, no roll in 60 s), which is what differed, not the execution layer. The s6 figures were cumulative from one starting RSS: the mempool flood's "+270 MB" was the submit flood's growth carried forward (its own cost is 1 MB), and 197 chain blocks cost under 1 MB in `ExecState.records`. On the live devnet the same engine costs 256 MiB per hourly epoch roll up to `KEEP`, which is the steady-state growth the coordinator saw on the 0.3.4 app node (1,081 MB at 27 min, 2,258 MB at 4 h 14 min, about 270 MB an hour). The fix: caches keyed by day, `KEEP_DAYS = 3` (3 x 256 MiB resident, plus at most 2 in-flight builds), programs keyed by `(epoch seed, day)` at a few KB each (`KEEP = 8`, LRU); an epoch roll on the same day builds no cache; node and miner compute the identical hash through `EpochRef` (unit test `epoch_rolls_share_the_day_cache`). No consensus rule changed. Measured (`docs/bench-log.md`, "ledger M30", two-node fast-time floods, before on the shipping finality-fixes build, after on `fud-memory` 796f758d): s6 submit flood +263 / +257 MB with 1 cache build each, after +3 / +2 MB and 0 builds; s7 50x block flood 302 to 1,085 MB with 3 builds, after 302 to 318 MB and 0 builds; template and mempool floods unchanged at +6 and +1 MB. Steady state, measured on two nodes at 1 block/s with no flood for 1,500 blocks on the 60x profile (`tools/harness/scenarios/s8-steady.mjs`, bench-log "ledger M30", steady-state paragraph): the shipping build reached 1,342 MB by block 514 after 9 cache builds (one per epoch roll; five 256 MiB chunks, the fifth an evicted one the allocator keeps) and 1,371 MB at 1,529 blocks; the fixed build held 319 MB at 510 blocks with 1 build and 603 MB at 1,526 blocks with 2 (the fast-time day rolled once), so on the devnet profile it is 256 MiB flat, 512 MiB around midnight UTC, 768 MiB worst case. Both builds then climb 30 MB per 1,000 blocks (30.7 before, 30.2 after), which is not the PoW cache (72 MB of non-cache footprint at 1,526 blocks against 23 MB at 0; the reading, not a measurement, is the consensus database's buffers and rusty-kaspa's entry-sized caches filling); the live node's 36 MB per 1,000 blocks between epoch rolls fits it. The app node's 2,258 MB at 4 h 14 min exceeds what this node build can reach (about 1.7 GB at 15,000 blocks) and includes its GPU worker, which needs its own `vmmap -summary`. Still growing by design and not bounded tonight: `ExecState.records` (1 to 2 KB per chain block, approximate, 100 to 170 MB a day at 1 block/s; a window must cover the proving sortition window and the by-number RPC history), `SNAPSHOT_RING = 64` state clones scaling with state size, the finality key registry (grows with distinct vote keys; votes, certificates, locks and evidence are trimmed). The harness now reports per-load RSS deltas and cache-build counts and takes `IGNEUM_HARNESS_BASE_PORT` and `IGNEUM_HARNESS_TMP`. + +Answer: Measured, cause not yet isolated. What changed between the two builds is the execution layer, which this node links: `igneum/exec/src/service.rs` keeps every `ChainBlockRecord` in `ExecState.records` (`:304`, `:419`, pushed and never truncated) plus `tx_index` and `inclusions` maps per transaction, and the mempool flood's 14,998 rejected transactions cost 270 MB, so rejected transactions are retained somewhere too. Smallest fix: bound `ExecState.records` to the record window plus the pruning depth and drop `tx_index`/`inclusions` entries with them; discard a rejected transaction's bytes at rejection; then re-run `tools/harness` s6 and s7 and require growth under 50 MB, the 3 October figure. + +Evidence: `/tmp/igneum-redteam-ord/results/s6-exhaustion.json` and `s7-flood.json` (samples carry `rss_a`, `rss_b` every 10 s), kept in the session scratchpad `rt/logs/ord_s6`, `rt/logs/ord_s7`; the 3 October numbers in `docs/bench-log.md`, "consensus attack harness" entry. + +### M31. The 0.3.4 node cannot produce a block template on mainnet, testnet or simnet parameters +"`getBlockTemplate` on a `--simnet` node from the finality-fixes build answers every call with `Coinbase payload is above max length (204). Try to shorten the extra data.` and the network never makes a block. The coinbase of this build carries the vote-key reveal, the proof-record section and the finality section; only `DEVNET_PARAMS` was raised to `MAX_COINBASE_PAYLOAD_LEN_WITH_FINALITY` (16,384). `MAINNET_PARAMS`, `TESTNET_PARAMS` and `SIMNET_PARAMS` still carry Kaspa's 204 (`consensus/core/src/config/params.rs:705, 766, 828` against `:900`)." + +Status: Fix built, pending rollout (4 October 2026, night; node branch `fud-consensus`). Was: Open (4 October 2026, red-team run). Serious for anything that is not the devnet: a mainnet or testnet genesis on these parameters cannot be mined by a voting miner at all; harmless on the live devnet, whose parameters carry the raise. + +The fix: `max_coinbase_payload_len` is `MAX_COINBASE_PAYLOAD_LEN_WITH_FINALITY` (16,384) on mainnet, testnet and simnet as on devnet, and the template builder fits the finality section into what the payload has left (`finality::encode_section_within`, certificates first, then evidence, then votes; the RPC computes the budget from the fixed part, the script, the node version, the miner's extra data and the record section), so a coinbase can never exceed the limit on any network whatever the voter count. The worst case did not fit even on devnet before: 8 certificates over 8,192 voters, 8 pieces of evidence and 48 votes come to about 30 KB, so the per-item bounds alone were not a bound. Unit test `largest_coinbase_fits_on_every_network` (consensus-core): the fixed part, the version, a key reveal, a full record section (8 records of 274 bytes) and the cut finality section fit under every network's limit, the cut keeps every certificate and every piece of evidence and at least 8 votes, the uncut section would not fit, and a budget under one certificate yields an empty section. The exec-attacks network (`tools/exec-attacks/net.sh`, `--simnet`) can drop its override once this build ships; not re-run tonight. + +Answer: Measured on the execution-layer attack network (`tools/exec-attacks/net.sh` runs `--simnet` with no override): three nodes up, 0 blocks, every template refused with that line (`docs/review/redteam-2026-10-04.md` row 27). Smallest fix: set `max_coinbase_payload_len: MAX_COINBASE_PAYLOAD_LEN_WITH_FINALITY` on the three other networks, and add a unit test that builds a coinbase with a key reveal, the maximum record section and a full certificate and checks it under every network's limit. The red-team run worked around it with `{"max_coinbase_payload_len": 16384}` in an override file. + +Evidence: session scratchpad `rt/logs/exec_b/miner_node1.log` (the template error, repeated once per second), `rt/logs/exec_b/n1/node.log` (28 lines, genesis executed, nothing after). + ### E17. Unlogged inputs behind the economics, minor "The cap's 110 MH/s and its draw are not in the bench-log; the Mac's draw is not logged; the economy sim's one measured input is a 229 MH/s card against today's 124; mining-versus-pool flips from 4.9x for mining on today's devnet to 930x for proving at 10,000 cards and no document says it depends on fleet size; the app-share text omits '100,000-gas calls' and the open base unit; emission ran at up to 2x schedule; shard-scale prover cost is unmeasured on any GPU; the iGPU default off is right." diff --git a/docs/spec/02-consensus.md b/docs/spec/02-consensus.md index 5661bb949..0abfa8e53 100644 --- a/docs/spec/02-consensus.md +++ b/docs/spec/02-consensus.md @@ -152,3 +152,12 @@ Block number, timestamp, blockhash, coinbase and prevrandao over the ordered seq ## 2.7 What the devnet showed and did not show Measured (`docs/bench-log.md`, devnet entry): three kaspad nodes at Kaspa's devnet parameters (10 BPS, k 124) held identical block counts, DAA scores and sink hashes at 18 of 19 ten-second samples under a 27 MH/s CPU miner; the DAA raised difficulty from genesis bits at block 6,018 and the block rate fell from 56 to 62 blocks/s toward the 10 BPS target. GHOSTDAG k was not exercised (a single serial miner never produced parallel blocks); a second miner is the next step. Nothing in that run used an Igneum parameter. + +## 2.8 One parameter set per network (ledger G12 and X18, 4 October 2026 night) + +Status: Implemented on the node's `fud-consensus` branch, pending rollout. + +1. A node's consensus parameters come from its network's constants (`Params`) and, on devnet and simnet only, from an override file. The override file is refused on mainnet (since devnet v4) and the environment never sets a consensus parameter outside devnet and simnet: `IGNEUM_POW_EPOCH_BLOCKS`, `IGNEUM_POW_EPOCH_LEAD` and `IGNEUM_POW_DAY_MS` are applied on devnet and simnet after the file, and on mainnet and testnet are ignored with one line at start ("Ignoring ... the environment never sets a consensus parameter outside devnet and simnet"). The PoW schedule the node runs is installed from `Params` on every network at start; nothing in the node reads the environment for it later. A miner takes all three schedule values from the block template (`pow_epoch.day_ms` joined `epoch_blocks` and `epoch_lead`), never from its environment. +2. The params digest. Every node computes `Params::consensus_digest()`: BLAKE2b-256 under the domain `IgneumParamsDigest` over, in a fixed tagged order, the network name, the genesis hash, bits, timestamp and DAA score, the difficulty windows and rule, the coinbase and mass limits, the lane limits, the storage mass parameter, the deflationary schedule, `skip_proof_of_work`, the block level and proof parameters, every blockrate field (target time, k, sample rates, parents, mergeset, merge depth, finality depth, pruning depth, maturity), the crescendo activation, every finality field (3.10), the PoW schedule and the three activation heights (`difficulty_v2_activation_daa`, `proving_v0_activation_daa`, `finality_v3_activation_daa`). Not covered: seeders, ports, the maximum difficulty target (a constant of the code) and `timestamp_deviation_tolerance` (dead, read by nothing). The node prints the digest at start. +3. The handshake. The p2p version message carries the digest (`paramsDigest`, field 11). A peer whose digest differs is refused at the handshake with one WARN naming both digests ("Refusing peer ...: consensus params digest mismatch, local X remote Y (the peer's override file, environment or build differs)") and the error `ParamsDigestMismatch`; no flow is registered and no block is exchanged. A peer that sends no digest (a build older than this rule) is refused on mainnet and testnet; on devnet and simnet it is let in with a WARN per connection while the devnet rolls, an allowance to remove once every devnet node carries the digest. +4. Why: round 4 found that two nodes with two override files connected and diverged per field, a finality mismatch only as a WARN after the fact (X18), and that the environment set the epoch length on every network including mainnet (G12). Unit tests `consensus_digest_covers_every_consensus_field_and_nothing_else` (every consensus field moves the digest, the dead field does not, the networks differ, the same file gives the same digest) and `env_pow_schedule_is_devnet_and_simnet_only`; the two-node run in `docs/bench-log.md`, "round-4 consensus items" (digest run). diff --git a/docs/spec/03-finality.md b/docs/spec/03-finality.md index aae93991c..b5c0a7a64 100644 --- a/docs/spec/03-finality.md +++ b/docs/spec/03-finality.md @@ -29,10 +29,10 @@ All in public, on the hashrate charts. 51% never reaches 2/3 while honest miners ## 3.2 Checkpoints -- **C1.** Checkpoint i is the selected-chain block at blue score 30 i. It is determined once the virtual's blue score reaches 30 i + d. d = 60 at 1 block per second is a placeholder (ledger F7): the gate 3 devnet records the reorg-depth distribution and sets d so that a vote split at one index is rare and self-heals at the next. d scales with block rate. A lock lands about 90 to 120 s after a transaction (Designed; simulated lock latency after the checkpoint block is median 2.5 s, p99 4.6 s at a 2-s inter-region delay, `sim/results_v2.md` A). +- **C1.** Checkpoint i is the selected-chain block at blue score 30 i. It is determined once the virtual's blue score reaches 30 i + d. d = 60 at 1 block per second is a placeholder (ledger F7): the gate 3 devnet records the reorg-depth distribution and sets d so that a vote split at one index is rare and self-heals at the next. d scales with block rate. Re-determination (rule of 4 October 2026, night, ledger F24): while index i is not locked, a node whose selected chain moves past C_i (a reorg deeper than d) determines index i again on its new chain; its own votes for the old block stand (a key never signs two blocks at one index) and a certificate the network formed over the new block, received meanwhile and held pending, is then verified. A locked index is never re-determined (3.11.4): fork choice keeps the chain through its block, and a certificate for another block there is a conflict (C4). A lock lands about 90 to 120 s after a transaction (Designed; simulated lock latency after the checkpoint block is median 2.5 s, p99 4.6 s at a 2-s inter-region delay, `sim/results_v2.md` A). - **C2.** A vote is a BLS signature over `(chain_id, i, hash(C_i))` under a fixed domain-separation tag. Votes gossip as their own message type. - **C3.** A lock certificate for index i is an aggregate BLS signature over one checkpoint block hash with a bitmap of signers, whose signed weight meets Q3. Every block carries the highest certificate its producer knows. A block whose selected chain does not pass through every certified checkpoint in its past is invalid (section 2.4). -- **C4.** A node holding a certificate for index i rejects any other certificate for index i and publishes the pair as evidence (section 3.6). +- **C4.** A node holding a LOCK at index i rejects any other certificate for index i and publishes the pair as evidence (section 3.6). A certificate over a block that is not the node's own determination at an index it has not locked is not a conflict: the chain may still move to that block (C1 re-determination, ledger F24), so the node keeps it pending, bounded, until it does or the index is left behind. A certificate naming a block that cannot be index i's checkpoint on any chain (its blue score is under 30 i, or its selected parent's is not) is refused outright. - **C5.** No certificate may form in the chain's first 3,600 DAA seconds (design document). See 3.8 for the proposed first-month rule. ## 3.3 Quorum @@ -118,7 +118,7 @@ What a node does when it holds two valid certificates at one index after a parti ## 3.6 Equivocation evidence -Two votes by one key for different checkpoint blocks at one index are equivocation. The evidence (the two votes) is a transaction that any block MAY include. On inclusion: the key's weight is zero for the rest of the current window and its blocks earn no weight for the next 2,592,000 DAA s. There is no coin penalty. In the simulation, evidence is detected only at the heal and the penalty is forward-looking only; the model does not revoke the conflicting certificates (`sim/results_v2.md`, "cannot tell us"), which is why 3.5's post-heal proposal strikes the equivocators' weight retroactively for the re-evaluation. +Two votes by one key for different checkpoint blocks at one index are equivocation. The evidence (the two votes) is a transaction that any block MAY include. On inclusion: the key's weight is zero for the rest of the current window and its blocks earn no weight for the next 2,592,000 DAA s. There is no coin penalty. The ban is a function of the chain (rule of 4 October 2026, night, ledger F23): at checkpoint C the key is stripped exactly when some block in the past of C carries the evidence and `daa(C) < daa(e) + 2,592,000`, where e is the carrier of lowest DAA score in C's past (the evidence block). Evidence a node has seen but no block in C's past carries strips nothing at C, so a node that detected the equivocation itself, from two votes over RPC or gossip, puts the evidence in its next block and waits for the chain. Every node with C's past computes the same voter list at C, whenever and however it first saw the evidence; before this rule a node stamped its own detection time and honest nodes refused each other's certificates for the length of the discrepancy. In the simulation, evidence is detected only at the heal and the penalty is forward-looking only; the model does not revoke the conflicting certificates (`sim/results_v2.md`, "cannot tell us"), which is why 3.5's post-heal proposal strikes the equivocators' weight retroactively for the re-evaluation. ## 3.7 Residual risks, stated @@ -167,10 +167,10 @@ Status of this section: Implemented in `vendor/igneum-node` (reading guide in `d | W1 | `vote_key_hash` = BLAKE2b (domain `IgneumVoteKeyHash`) of the 48-byte compressed G1 key. The key is revealed with a proof of possession in the miner's coinbase extra data (`IGNK` plus 288 hex characters) and a node registers it only when the hash matches the header. A vote carries the public key too, so a key that votes is revealed by its vote | The reveal is hex, not binary, because the template RPC carries extra data as a UTF-8 string. Mainnet should carry the reveal in a dedicated field or transaction | | W2 | Blue blocks per key in `(daa(C) - window, daa(C)]`, counted along C's selected chain through every chain block's mergeset blues, C included | O(window) per checkpoint: fine at the devnet window of 7,200 DAA seconds, not at 2,592,000. Mainnet needs an incremental window kept per chain block | | W3, W5 | Dust excludes a key from the voter list and from both denominators | Key succession (W5) is not implemented | -| C1 | Checkpoint i is the lowest selected-chain block with blue score at least 30 i (blue scores along the chain can skip values), determined when the sink's blue score reaches 30 i + d, d = 20 on devnet. A determination is never revisited | d = 20 is below the placeholder 60; the devnet reorg-depth distribution that sets d has not been recorded | +| C1 | Checkpoint i is the lowest selected-chain block with blue score at least 30 i (blue scores along the chain can skip values), determined when the sink's blue score reaches 30 i + d, d = 20 on devnet. Re-determination (branch `fud-consensus`, 4 October 2026 night, ledger F24): after every virtual change, every unlocked record whose block is no longer a chain ancestor of the sink is determined again on the new chain (`on_virtual_changed`, "re-determined" log line); the certificate held over the old block is dropped, the fold clock restarts, and the certificates kept pending over the new block (`pending_certificates`, at most 4 per index, indices up to 64 ahead of the next determination) are verified. A locked record is never revisited. Unit test `reorg_past_an_unlocked_checkpoint_re_determines_it_and_verifies_the_pending_certificate` (a 6-block side chain's certificate is pending with no conflict, the 15-block side chain overtakes, index 13 is re-determined and locks from it; a block with the wrong blue score is refused) and `a_locked_checkpoint_pins_the_chain_and_a_certificate_against_it_conflicts` (a side chain twice as long does not become the sink past a lock, the certificate against the lock is the one conflict) | d = 20 is below the placeholder 60; the devnet reorg-depth distribution that sets d has not been recorded. Measured in `docs/bench-log.md`, "round-4 consensus items" (reorg run) | | C2 | BLS signature over `"igneum-vote-v1/" \|\| chain_id \|\| 0 \|\| index \|\| hash(C_i)` under `IGNEUM_VOTE_V1_BLS12381G2_XMD:SHA-256_SSWU_RO_NUL_`; the chain id is the prefixed network name (`igneum-devnet`, `igneum-devnet-7`); votes are p2p message 70 and ride in the coinbase extra data of every block | | | C3 | Certificate = index, checkpoint, voter count, signer bitmap over the canonical voter list (keys above dust and not stripped, sorted by key hash), aggregate signature, aggregator key hash and sortition proof. Every template carries the certificates not yet in its past | The validity rule (a block whose selected chain misses a certified checkpoint is invalid) is NOT enforced; only fork choice (F1, F2) is | -| C4 | A second certificate at an index for another block is kept and logged (`conflicting_certificates`) | Not published as evidence. The rule is now fixed by 3.11 item 4 (the node keeps the certificate it verified first, never re-evaluates it, and reports the conflict); the node does not yet clear `finality_active` or expose `finality_conflict` when the pair appears | +| C4 | A certificate at an index for a block other than the one LOCKED there is kept and logged as CONFLICTING (`conflicting_certificates`); at an unlocked index it is held pending (F24 above), not logged as a conflict | Not published as evidence. The rule is now fixed by 3.11 item 4 (the node keeps the certificate it verified first, never re-evaluates it, and reports the conflict); the node does not yet clear `finality_active` or expose `finality_conflict` when the pair appears. Until 4 October 2026 night a reorg deeper than d made the node log every certificate at the moved index as CONFLICTING (ledger F24) | | C5, 3.8 | `min_daa` = `weight_window` (2,592,000 DAA s on mainnet, 7,200 on devnet; a unit test pins the equality). `evaluate` never locks, and `ingest_certificate` refuses a certificate from any source, while the checkpoint's DAA score is below `min_daa`; the node logs "finality not active, window filling, N of M" at every determination until the sink's DAA score reaches `min_daa` and reports the same through `getFinalityCheckpoints` (`finality_reason`, `window_filled_daa`, `window_full_daa`). Unit test `processes::finality::tests::no_certificate_while_the_window_is_filling`: one key holding 100% of the weight signs every checkpoint of a 150-block chain at a 60-DAA window; nothing certifies below DAA 60, a hand-built certificate at an early index is refused, every checkpoint from DAA 60 locks (fin-fixes, 4 October 2026) | Implemented on 3.8's recommendation ahead of the launch-month simulation (O-3.1), which is still not run; gate 3 can lower the gate but not remove it without reopening ledger F1. The sink's DAA score the report compares is the one the virtual processor last handed the manager, so a restarted node reports the window as filling until its first virtual resolution | | Q1, Q2 | Presence window 20 indices on devnet (240 mainnet). Block reading: participation counts the indices in `[i - P, i - 1]` at which a vote by the key is carried by any block, blue or red, in the past of C_i; a key whose first block in the window is younger than P x 30 DAA seconds counts the full window; every template carries up to 48 votes not already in its past, certificates and evidence first | The per-block vote bound (48) is the devnet value of O-3.3. Participation is credited for any vote by the key at the index, whatever block it names; 3.11.1 requires the vote to name the checkpoint on the crediting chain, else a key can stay in the active denominator by voting for blocks of its own and never add to a certificate (O-3.19) | | Q3 | Integer tests: `3 x signed x P >= 2 x active_num` (active_num = sum of weight x participation count) and `3 x signed >= 2 x total` (was `30 x signed >= 17 x total` until 4 October 2026; `FinalityParams::FLOOR_NUM / FLOOR_DEN` = 2/3 on branch `devnet-v4`, with `quorum_met`, `floor_met` and `locks` as pure functions), both inclusive, both at C_i; bans known at evaluation time are applied to the voter list. Unit test `floor_is_two_thirds_of_total_and_inclusive`: 4 of 6 locks, 3 of 6 does not, 67 of 100 locks, 66 does not, the total test implies the active test for every participation. Measured on the three-node, six-voter network of `docs/bench-log.md`, "finality floor 2/3" (4 October 2026): no lock on either side of a 3/3 split, the 4 side of a 4/2 split locks at exactly two thirds | | @@ -181,7 +181,7 @@ Status of this section: Implemented in `vendor/igneum-node` (reading guide in `d | F1, F2 | In `resolve_virtual` the highest locked checkpoint that is in the future of the depth-based finality point and in the past of some body tip replaces the finality point: tips outside its future are not sink candidates | A lock that no body tip passes through is logged and ignored for that resolution | | F3 | Not implemented: the pruning point and `virtual_finality_point` ignore locks | Must land before any pruning network | | F5 | Not implemented (trusted certificate at start) | | -| 3.6 | A second vote by one key at one index for another block is evidence: the key's weight is zero until `detection DAA + ban` (7,200 DAA seconds on devnet), the evidence is carried in blocks and re-detected from blocks | Node-local detection timestamps the ban with the sink's DAA score; a block-carried evidence uses the carrying block's DAA score | +| 3.6 | A second vote by one key at one index for another block is evidence, carried in blocks (`EvidenceRecord`: the two votes, the carriers with their DAA scores). Branch `fud-consensus` (4 October 2026 night, ledger F23): the ban at checkpoint C is computed from C's own past (`bans_at`): the key is stripped at C when a carrier lies in C's past and `daa(C) < daa(lowest carrier) + ban` (7,200 DAA seconds on devnet); detection over RPC or gossip only puts the evidence into this node's templates ("detected here: carried in this node's next block"). Evidence records are bounded (4,096, the oldest dropped; a record goes once its ban ended two windows below the sink or it was never carried within one ban of being seen; 16 carriers per record). Unit test `ban_is_decided_by_the_carrying_block_so_nodes_agree_on_every_voter_list`: three nodes on one chain, one takes the equivocating vote over RPC, two see it from the carrier block only; the voter list agrees on all three at every checkpoint, the key is a voter before the carrier and after the ban and nowhere in between, and the third node verifies the first two's certificates at every locked index | A carrier the reachability store no longer holds (pruned) counts as in the past of every checkpoint more than the merge depth younger than it. Until 4 October 2026 night the ban was stamped node-locally (ledger F23). Measured in `docs/bench-log.md`, "round-4 consensus items" (ban run) | | 3.9 | `getFinalityCheckpoints` reports `finality_active` (the window is full and a lock exists within the last P indices), the latest lock, and since 4 October 2026 `finality_reason` (`active`, `window filling, N of M` with N the sink's DAA score capped at `min_daa` and M `min_daa`, or `paused`) with `window_filled_daa` and `window_full_daa`; the miner prints a `FINALITY` line whenever the reason changes | `last_certified` as a DAA score is not reported; the conflict reason of 3.11 item 4 (two certificates at one index) is not reported (O-3.17), so a conflict still reads as `active` or `paused` | | 3.11 item 6 (seed source) | The devnet keys the hourly program on the header's own `daa_score` (`epoch_seed`, `docs/review/round-3-2026-10-03.md`, R3.26), not on a checkpoint block | The `seed_source` rule (section 4.3 with the uncertified fallback of 3.11 item 6) is not implemented; nothing on the devnet exercises a seed during a finality pause | | 3.11 test table | The four-miner test network of the bench-log entry is the only measurement on a real DAG: 93 checkpoints, 0 conflicting certificates, one equivocation strip, one 12-checkpoint pause under the floor, one heal | d = 20, presence 20 indices and a 7,200-s window are devnet values; the measured pause and heal are at those values, not the mainnet ones | diff --git a/docs/spec/08-client-security.md b/docs/spec/08-client-security.md index 472ad6755..ab7fd7112 100644 --- a/docs/spec/08-client-security.md +++ b/docs/spec/08-client-security.md @@ -53,3 +53,10 @@ The two findings it answers. An app that auto-updates on ten thousand machines i | Seed shown and confirmed before mining | required | Decided | | Hardware wallet option | required | Decided | | The permanent line | "Nobody from Igneum will ever ask for your seed." | Decided, verbatim | +| Consensus parameters from the environment | none outside devnet and simnet; the override file refused on mainnet | Implemented on `fud-consensus`, pending rollout (8.7) | +| Params digest in the handshake | BLAKE2b-256 of the effective params; a mismatch is refused | Implemented on `fud-consensus`, pending rollout (8.7, section 2.8) | + +## 8.7 The node's parameters are the network's (4 October 2026 night, ledger G12 and X18) + +1. The official client MUST NOT let an environment variable, a setting or an update change a consensus parameter on mainnet or testnet. The node it wraps ignores `IGNEUM_POW_*` outside devnet and simnet and refuses the override file on mainnet (section 2.8); the app passes an override file only to a devnet or simnet node, and the packaged file (`node_override_params`) exists for the devnet's height switches alone. +2. A node refuses any peer whose consensus params digest differs from its own (section 2.8). The client SHOULD show the digest its node printed at start, so an operator can compare two machines by eye (not built yet); a refused peer is reported by the node's log line, never silently. diff --git a/tools/finality-attacks/fud.mjs b/tools/finality-attacks/fud.mjs new file mode 100644 index 000000000..b9b3be9ec --- /dev/null +++ b/tools/finality-attacks/fud.mjs @@ -0,0 +1,256 @@ +// Round-4 consensus items runner (4 October 2026, night): ledger F23 (deterministic equivocation bans), F24 +// (re-determination after a deep reorg) and G12/X18 (the params digest in the handshake) on the fast-time 3-node +// network of v3.mjs. Ports 29400 and up, network igneum-devnet-940, data under /tmp/igneum-fin-fud; the live devnet +// is never touched. +// +// node tools/finality-attacks/fud.mjs digest ban reorg # the three scenarios on the fud-consensus build +// IGNEUMD=... IGNEUM_MINER=... node tools/finality-attacks/fud.mjs ban # another build (the control: finality-fixes) +// DELAY_MS=100 BPS=1 node tools/finality-attacks/fud.mjs ... # one-way delay per proxied link, total block rate +// +// Topology (v3.mjs): n1 listens; n0 dials n1 through proxy P0, n2 dials n1 through proxy P2; a proxy adds DELAY_MS +// one way and can be cut and healed. +// +// digest n1 runs the shared override; n0 dials it with a finality block that differs by one DAA second of window +// (another consensus params digest): the handshake must refuse it and n1 must stay without peers; then n2 +// dials with the shared override and connects. Under the old build the mismatched n0 connects. +// ban six voters, two per node, equal shares; voter a0 (on n0) equivocates once at index EQ_INDEX. P2 is cut +// just before that index is determined and healed 45 s later, so n2 sees the evidence late, from the block +// that carries it, while n0 saw it over RPC and n1 from the block at once. Through the ban's expiry every +// node must build or accept certificates over the same voter count at every index: no "names N voters" +// refusal, no conflicting certificate, no locked index disagreeing, the stripped index range identical. +// reorg 4/2 keys with the 4 side (n1, n2) at 70% of the weight; P0 is cut for SPLIT s so n0 determines at least +// one checkpoint on its own chain, then healed: n0 must re-determine those indices on the majority chain +// and lock them from the network's certificates, with no CONFLICTING line and no index locked on two +// different blocks across the nodes. Under the old build n0 logs CONFLICTING for each such index. + +const NODE_ROOT = process.env.IGNEUM_NODE_ROOT || '/Users/joshm/Projects/igneum/'; +process.env.IGNEUM_FIN_BASE_PORT ||= '29400'; +process.env.IGNEUM_FIN_SUFFIX ||= '940'; +process.env.IGNEUM_FIN_TMP ||= '/tmp/igneum-fin-fud'; +process.env.IGNEUM_FAST_TIME ||= '1'; +process.env.IGNEUMD ||= `${NODE_ROOT}vendor/igneum-node-fud/target/release/igneumd`; +process.env.IGNEUM_MINER ||= `${NODE_ROOT}vendor/igneum-node-fud/target/release/igneum-miner`; +const DELAY_MS = +(process.env.DELAY_MS || 100); +const BPS = +(process.env.BPS || 1); +const EQ_INDEX = +(process.env.EQ_INDEX || 9); +const WARM = +(process.env.WARM || 230), SPLIT = +(process.env.SPLIT || 180), HEAL = +(process.env.HEAL || 150); +const BAN_CUT = +(process.env.BAN_CUT || 45), BAN_RUN = +(process.env.BAN_RUN || 480); +const TAG = process.env.TAG || 'fud'; +// rule v3 from checkpoint DAA 0 on every node (the fast-time file says never) +process.env.IGNEUM_FIN_OVERRIDE_JSON ||= JSON.stringify({ finality_v3_activation_daa: 0 }); + +const { Node, Miner, Proxy, stopAll, sleep, log, assertBinaries, TMP, IGNEUMD } = await import('./lib/net.mjs'); +const { mkdirSync, writeFileSync, appendFileSync, copyFileSync, existsSync } = await import('node:fs'); +mkdirSync(TMP, { recursive: true }); +// every scenario keeps its node logs (the next scenario wipes the node directories) +function keepLogs(name, nodes) { + const dir = `${TMP}/logs-${name}`; + mkdirSync(dir, { recursive: true }); + for (const n of nodes) if (existsSync(n.logFile)) copyFileSync(n.logFile, `${dir}/${n.name}.log`); + return dir; +} +const results = []; +const out = (line) => { console.log(line); appendFileSync(`${TMP}/results-${TAG}.md`, line + '\n'); }; + +const lockedMap = (cp) => new Map((cp?.checkpoints || []).filter(c => c.state === 'locked').map(c => [c.index, c.hash])); +const maxLocked = (cp) => Math.max(0, ...lockedMap(cp).keys()); +async function checkpoints(node, last = 800) { return node.rpc.call('getFinalityCheckpoints', { last }).catch(() => null); } +async function peers(node) { const r = await node.rpc.call('getConnectedPeerInfo', {}).catch(() => null); return (r?.peerInfo || r?.infos || []).length; } + +// per index, the voter count every certificate line on a node names (built / received / replaced / folded) +function voterCounts(node) { + const m = new Map(); + for (const l of node.grepLog(/Finality: certificate/)) { + let x; + if ((x = l.match(/certificate built for checkpoint (\d+) .* by (\d+) of (\d+) voters/))) m.set(+x[1], +x[3]); + else if ((x = l.match(/certificate at index (\d+) received: (\d+) of (\d+) voters/))) m.set(+x[1], +x[3]); + else if ((x = l.match(/certificate at index (\d+) replaced by a heavier one: (\d+) of (\d+) voters/))) m.set(+x[1], +x[3]); + else if ((x = l.match(/certificate for checkpoint (\d+) folded: (\d+) of (\d+) voters/))) m.set(+x[1], +x[3]); + } + return m; +} +function disagreeing(cps) { + const maps = cps.map(lockedMap); + const all = new Set(maps.flatMap(m => [...m.keys()])); + let n = 0; + for (const k of all) { const hs = new Set(maps.filter(m => m.has(k)).map(m => m.get(k))); if (hs.size > 1) n++; } + return n; +} +const count = (node, re) => node.grepLog(re).length; + +async function network() { + const n1 = await new Node(1).start(); + const p0 = await new Proxy(0, n1.p2pPort, { delayMs: DELAY_MS }).start(); + const p2 = await new Proxy(1, n1.p2pPort, { delayMs: DELAY_MS }).start(); + const n0 = await new Node(0, { connect: [p0.addr] }).start(); + const n2 = await new Node(2, { connect: [p2.addr] }).start(); + return { n0, n1, n2, p0, p2 }; +} + +async function digest() { + const name = `digest-${TAG}`; + const n1 = await new Node(1).start(); + // n0: the same file but one DAA second more of weight window: another digest + const n0 = await new Node(0, { connect: [n1.p2p], override: { finality: { weight_window: 121 } } }).start(); + await sleep(25000); + const refusedOn0 = count(n0, /consensus params digest mismatch/), refusedOn1 = count(n1, /consensus params digest mismatch/); + const peers1 = await peers(n1), peers0 = await peers(n0); + const digestLine = (n) => (n.grepLog(/Consensus params digest:/)[0] || '').replace(/^.*digest: /, '').split(' ')[0]; + // n2: the shared file, connects + const n2 = await new Node(2, { connect: [n1.p2p] }).start(); + let connected = null; + for (let i = 0; i < 25; i++) { await sleep(1000); if ((await peers(n2)) > 0) { connected = i + 1; break; } } + const peers1After = await peers(n1); + const refusedOn2 = count(n2, /consensus params digest mismatch/); + keepLogs(name, [n0, n1, n2]); + await stopAll(); + const pass = refusedOn0 > 0 && refusedOn1 > 0 && peers1 === 0 && peers0 === 0 && connected != null && refusedOn2 === 0; + out(`\n### ${name}: n1 on the shared fast-time override, n0 with finality.weight_window 121 (one DAA second more), n2 on the shared override\n`); + out('| measure | n0 (mismatched) | n1 (listener) | n2 (matching) |'); + out('|---|---|---|---|'); + out(`| params digest printed at start | ${digestLine(n0) || 'none'} | ${digestLine(n1) || 'none'} | ${digestLine(n2) || 'none'} |`); + out(`| "consensus params digest mismatch" lines | ${refusedOn0} | ${refusedOn1} | ${refusedOn2} |`); + out(`| peers after 25 s (n0, n1) and after n2 dialled (n1) | ${peers0} | ${peers1} then ${peers1After} | ${connected == null ? 'not connected in 25 s' : 'connected after ' + connected + ' s'} |`); + const sample = n0.grepLog(/consensus params digest mismatch/)[0]; + if (sample) out(`\nn0's line: \`${sample.replace(/^.*?(Refusing|WARN)/, '$1').slice(0, 300)}\``); + results.push({ name, pass }); +} + +async function ban() { + const name = `ban-${TAG}`; + const { n0, n1, n2, p2 } = await network(); + const miners = []; + for (const [node, label, eq] of [[n0, 'a0', true], [n0, 'a1', false], [n1, 'b0', false], [n1, 'b1', false], [n2, 'c0', false], [n2, 'c1', false]]) + miners.push(new Miner(node, { label, share: 1 / 6, bps: BPS, secs: BAN_RUN, equivocateAt: eq ? EQ_INDEX : undefined }).start()); + const t0 = Date.now(); + // cut n2 off just before index EQ_INDEX is determined on n0 (when EQ_INDEX - 1 is), heal BAN_CUT s later + let cutAt = null, healAt = null, eqSeenAt = null; + while (Date.now() - t0 < BAN_RUN * 1000) { + const cp = await checkpoints(n0, 50); + const t = Math.round((Date.now() - t0) / 1000); + if (cutAt == null && cp && cp.nextIndex >= EQ_INDEX) { p2.cut(); cutAt = t; log(`${name}: P2 cut at ${t} s (n0 next index ${cp.nextIndex})`); } + if (eqSeenAt == null && count(n0, /EQUIVOCATION by key/) > 0) { eqSeenAt = t; log(`${name}: n0 detected the equivocation at ${t} s`); } + if (cutAt != null && healAt == null && t - cutAt >= BAN_CUT) { p2.heal(); healAt = t; log(`${name}: P2 healed at ${t} s`); } + await sleep(1000); + } + const cps = await Promise.all([n0, n1, n2].map(n => checkpoints(n))); + for (const m of miners) await m.stop(); + const nodes = [n0, n1, n2]; + const refusals = nodes.map(n => count(n, /names \d+ voters, this node counts/)); + const conflicts = nodes.map(n => count(n, /CONFLICTING certificate/)); + const equiv = nodes.map(n => count(n, /EQUIVOCATION by key/)); + const carried = nodes.map(n => count(n, /EQUIVOCATION by key .* carried by block/)); + const counts = nodes.map(voterCounts); + const indices = new Set(counts.flatMap(m => [...m.keys()])); + let agree = 0, differ = 0; + const stripped = nodes.map(() => []); + for (const i of [...indices].sort((a, b) => a - b)) { + const vs = counts.map(m => m.get(i)).filter(v => v != null); + if (vs.length >= 2) { if (new Set(vs).size === 1) agree++; else differ++; } + counts.forEach((m, k) => { if (m.get(i) != null && m.get(i) < 6) stripped[k].push(i); }); + } + const range = (xs) => xs.length ? `${xs[0]}..${xs[xs.length - 1]} (${xs.length})` : 'none'; + const locks = cps.map(maxLocked); + const disagree = disagreeing(cps); + keepLogs(name, nodes); + await stopAll(); + const sameRange = new Set(stripped.map(range)).size === 1 && stripped[0].length > 0; + const pass = refusals.every(r => r === 0) && conflicts.every(c => c === 0) && disagree === 0 && differ === 0 && sameRange && locks.every(l => l > EQ_INDEX + 3); + out(`\n### ${name}: ${BAN_RUN} s, ${BPS} blocks/s in all, 6 voters, delay ${DELAY_MS} ms, a0 equivocates once at index ${EQ_INDEX}; P2 cut at ${cutAt ?? 'never'} s, healed at ${healAt ?? 'never'} s; n0 detected the equivocation at ${eqSeenAt ?? 'never'} s\n`); + out('| measure | n0 (saw it over RPC) | n1 (from the block at once) | n2 (from the block, after the heal) |'); + out('|---|---|---|---|'); + out(`| EQUIVOCATION lines (of which "carried by block") | ${equiv[0]} (${carried[0]}) | ${equiv[1]} (${carried[1]}) | ${equiv[2]} (${carried[2]}) |`); + out(`| certificates refused "names N voters, this node counts M" | ${refusals.join(' | ')} |`); + out(`| CONFLICTING certificate lines | ${conflicts.join(' | ')} |`); + out(`| indices whose certificates name 5 voters (a0 stripped) | ${stripped.map(range).join(' | ')} |`); + out(`| max locked index at the end | ${locks.join(' | ')} |`); + out(`\nindices with certificate lines on at least two nodes: voter counts agree at ${agree}, differ at ${differ}; locked indices disagreeing across the three nodes: ${disagree}`); + results.push({ name, pass }); +} + +async function reorg() { + const name = `reorg-${TAG}`; + const { n0, n1, n2, p0 } = await network(); + const secs = WARM + SPLIT + HEAL + 30; + const miners = []; + for (const [node, label, share] of [[n0, 'q0', 0.15], [n0, 'q1', 0.15], [n1, 'p0', 0.175], [n1, 'p1', 0.175], [n2, 'p2', 0.175], [n2, 'p3', 0.175]]) + miners.push(new Miner(node, { label, share, bps: BPS, secs }).start()); + await sleep(WARM * 1000); + const before = await Promise.all([n0, n1, n2].map(n => checkpoints(n))); + const beforeMax = before.map(maxLocked); + const preNext = (await checkpoints(n0, 5))?.nextIndex; + log(`${name}: cut at ${WARM} s: max locked ${beforeMax.join('/')}, n0 next index ${preNext}`); + p0.cut(); + await sleep(SPLIT * 1000); + const during = await Promise.all([n0, n1, n2].map(n => checkpoints(n))); + const ownDetermined = (during[0]?.nextIndex ?? 0) - preNext; + const duringMax = during.map(maxLocked); + p0.heal(); + const tHeal = Date.now(); + let reconnected = null; + while (Date.now() - tHeal < HEAL * 1000) { + if (reconnected == null && (await peers(n0)) > 0) reconnected = Math.round((Date.now() - tHeal) / 1000); + await sleep(2000); + } + const after = await Promise.all([n0, n1, n2].map(n => checkpoints(n))); + for (const m of miners) await m.stop(); + const nodes = [n0, n1, n2]; + const redetermined = nodes.map(n => count(n, /re-determined/)); + const conflicts = nodes.map(n => count(n, /CONFLICTING certificate/)); + const pending = nodes.map(n => count(n, /kept pending until the chain decides/)); + const afterMax = after.map(maxLocked); + const disagree = disagreeing(after); + // n0's locks at the indices it determined on its own chain: the same block as n1 holds + const m0 = lockedMap(after[0]), m1 = lockedMap(after[1]); + const splitIdx = [...Array(Math.max(0, ownDetermined)).keys()].map(k => preNext + k); + const agreed = splitIdx.filter(i => m0.has(i) && m1.has(i) && m0.get(i) === m1.get(i)).length; + // the build before F24 refused a certificate over another block with "is for X, this node's checkpoint is Y" and + // kept it as conflicting; the F24 build logs the same words with "kept pending" + const refusedOld = nodes.map(n => n.grepLog(/this node's checkpoint is/).filter(l => !/kept pending/.test(l)).length); + const verified = nodes.map(n => count(n, /pending certificate at index \d+ over \S+ verified/)); + const unverified = nodes.map(n => count(n, /did not verify once the index was determined/)); + // every index n1 locked, n0 locked on the same block by the end (the split indices included) + const missing = [...m1.keys()].filter(i => !m0.has(i)); + keepLogs(name, nodes); + await stopAll(); + // the majority may not lock every split index (70% nominal is noise away from the floor), so the test is: every + // index the majority locked, n0 locked on the same block, and nothing n0 holds is off the chain or below its target + const belowTarget = (after[0]?.checkpoints || []).filter(c => c.blueScore < 30 * c.index).map(c => c.index); + const pass = ownDetermined >= 1 && conflicts.every(c => c === 0) && disagree === 0 && afterMax[0] > duringMax[0] && redetermined[0] >= 1 && missing.length === 0 && belowTarget.length === 0; + out(`\n### ${name}: warm ${WARM} s, split ${SPLIT} s (n0 alone with 30% of the weight), heal window ${HEAL} s, ${BPS} blocks/s in all, delay ${DELAY_MS} ms\n`); + out('| measure | n0 (cut off, 30%) | n1 (70% side) | n2 (70% side) |'); + out('|---|---|---|---|'); + out(`| max locked index at the cut | ${beforeMax.join(' | ')} |`); + out(`| max locked index at the heal | ${duringMax.join(' | ')} |`); + out(`| max locked index at the end | ${afterMax.join(' | ')} |`); + out(`| "re-determined" lines | ${redetermined.join(' | ')} |`); + out(`| certificates kept pending | ${pending.join(' | ')} |`); + out(`| CONFLICTING certificate lines | ${conflicts.join(' | ')} |`); + out(`| certificates refused over another block, pre-F24 wording | ${refusedOld.join(' | ')} |`); + out(`| pending certificates verified at determination (did not verify) | ${verified.map((v, i) => `${v} (${unverified[i]})`).join(' | ')} |`); + out(`| indices n1 locked that this node did not lock | ${nodes.map(n => (n === n0 ? missing.join(' ') || 'none' : '')).join(' | ')} |`); + out(`| records whose block is below the index's target blue score | ${nodes.map(n => (n === n0 ? belowTarget.join(' ') || 'none' : '')).join(' | ')} |`); + out(`\nn0 determined ${ownDetermined} checkpoint(s) on its own chain during the split (indices ${splitIdx.join(', ') || 'none'}); after the heal n0 holds the same locked block as n1 at ${agreed} of them; locked indices disagreeing across the three nodes: ${disagree}; n0 reconnected ${reconnected == null ? 'not within the heal window' : reconnected + ' s after the gate reopened'}`); + const lines = n0.grepLog(/re-determined|CONFLICTING/).slice(0, 4); + for (const l of lines) out(` ${l.replace(/^.*?(Finality:)/, '$1').slice(0, 260)}`); + results.push({ name, pass }); +} + +const ALL = { digest, ban, reorg }; +async function main() { + assertBinaries(); + log(`tag ${TAG}; node ${IGNEUMD}; delay ${DELAY_MS} ms; ${BPS} blocks/s; override ${process.env.IGNEUM_FIN_OVERRIDE_JSON}`); + const asked = process.argv.slice(2).filter(a => !a.startsWith('--')); + for (const key of asked.length ? asked : ['digest', 'ban', 'reorg']) { + const fn = ALL[key]; + if (!fn) { log(`unknown scenario ${key}`); continue; } + log(`=== ${key} (${TAG}) starting ===`); + try { await fn(); } catch (e) { log(`${key} threw: ${e.stack || e}`); results.push({ name: key, pass: false }); await stopAll(); } + log(`=== ${key} done ===`); + } + out('\n' + results.map(r => `[${r.pass ? 'PASS' : 'FAIL'}] ${r.name}`).join('\n')); + writeFileSync(`${TMP}/results-${TAG}.json`, JSON.stringify(results, null, 2)); + await stopAll(); + process.exit(results.some(r => !r.pass) ? 1 : 0); +} +main(); diff --git a/tools/finality-attacks/lib/net.mjs b/tools/finality-attacks/lib/net.mjs index 06977388a..ac1ddc1fd 100644 --- a/tools/finality-attacks/lib/net.mjs +++ b/tools/finality-attacks/lib/net.mjs @@ -37,24 +37,27 @@ const started = []; // everything to stop at exit export const log = (...a) => console.log(new Date().toISOString().slice(11, 23), ...a); export const sleep = (ms) => new Promise(r => setTimeout(r, ms)); -export function overrideParams() { +// extra: a per-node object merged last (the F23/F24/X18 runner gives one node another finality block); name: the +// file's suffix so two nodes never share a file +export function overrideParams(extra = {}, name = '') { mkdirSync(TMP, { recursive: true }); - const file = `${TMP}/override.json`; + const file = `${TMP}/override${name ? '-' + name : ''}.json`; // u64::MAX ("never" for the height switches) is not a JavaScript number: keep it as a BigInt through the merge and // write it back as the integer literal the node's parser wants const big = (k, v, ctx) => (typeof v === 'number' && !Number.isSafeInteger(v) && ctx?.source ? BigInt(ctx.source) : v); const base = FAST_TIME ? JSON.parse(readFileSync(FAST_TIME_FILE, 'utf8'), big) : {}; - const merged = { ...base, skip_proof_of_work: true, ...EXTRA_OVERRIDE }; - if (base.finality && EXTRA_OVERRIDE.finality) merged.finality = { ...base.finality, ...EXTRA_OVERRIDE.finality }; + const merged = { ...base, skip_proof_of_work: true, ...EXTRA_OVERRIDE, ...extra }; + if (base.finality && (EXTRA_OVERRIDE.finality || extra.finality)) merged.finality = { ...base.finality, ...EXTRA_OVERRIDE.finality, ...extra.finality }; const text = JSON.stringify(merged, (k, v) => (typeof v === 'bigint' ? `BIGINT:${v}` : v)).replace(/"BIGINT:(\d+)"/g, '$1'); writeFileSync(file, text); return file; } export class Node { - constructor(index, { connect = [], name } = {}) { + constructor(index, { connect = [], name, override } = {}) { this.index = index; this.name = name || `n${index}`; + this.override = override; // a per-node override object merged over the shared one (see overrideParams) this.grpcPort = BASE_PORT + index * 10; this.p2pPort = BASE_PORT + index * 10 + 1; this.jsonPort = BASE_PORT + index * 10 + 2; @@ -70,7 +73,7 @@ export class Node { const a = ['--devnet', `--devnet-suffix=${DEVNET_SUFFIX}`, '--nodnsseed', '--disable-upnp', '--nologfiles', '--enable-unsynced-mining', '--utxoindex', `--appdir=${this.dir}`, `--rpclisten=127.0.0.1:${this.grpcPort}`, `--rpclisten-json=127.0.0.1:${this.jsonPort}`, - `--listen=127.0.0.1:${this.p2pPort}`, `--override-params-file=${overrideParams()}`, '--loglevel=info', '--yes']; + `--listen=127.0.0.1:${this.p2pPort}`, `--override-params-file=${overrideParams(this.override || {}, this.override ? this.name : '')}`, '--loglevel=info', '--yes']; if (this.connect.length) a.push(`--connect=${this.connect.join(',')}`); else a.push('--outpeers=0'); return a; } @@ -112,6 +115,7 @@ export class Miner { if (o.label) a.push('--label', o.label); if (o.vote === false) a.push('--no-vote'); if (o.equivocate) a.push('--equivocate'); + if (o.equivocateAt != null) a.push('--equivocate-at', String(o.equivocateAt)); if (o.dropVotes) a.push('--drop-votes'); if (o.sybil) a.push('--sybil', o.sybil); if (o.pulse) a.push('--pulse', o.pulse); diff --git a/tools/finality-attacks/redteam/flood.mjs b/tools/finality-attacks/redteam/flood.mjs new file mode 100644 index 000000000..f8fe4e4f7 --- /dev/null +++ b/tools/finality-attacks/redteam/flood.mjs @@ -0,0 +1,106 @@ +// Red-team: flood of invalid proof records against the proof pool of the finality-fixes build (proving v0 active). +// One redteam node (eth RPC), 3 vmine voters to reach activation and assign shards, then a flood of well-formed-length +// garbage records through igneum_submitProofRecord. Measures reject throughput and node CPU per rejected record. +import { spawn, spawnSync } from 'node:child_process'; +import { mkdirSync, rmSync, openSync, readFileSync, writeFileSync, existsSync } from 'node:fs'; +import { createHash, randomBytes } from 'node:crypto'; +import { connectRpc } from '../lib/rpc.mjs'; + +const ROOT = '/Users/joshm/Projects/igneum/'; +const IGNEUMD = `${ROOT}vendor/igneum-node-redteam/target/release/igneumd`; +const MINER = `${ROOT}vendor/igneum-node-fin-attacks/target/release/igneum-miner`; +const OVERRIDE = '/tmp/igneum-redteam-override-prove-v3.json'; +const TMP = '/tmp/igneum-redteam-flood'; +const BASE = 29680, SUFFIX = 968; +const RECLEN = 2 + 32 + 8 + 4 + 48 + 20 + 32 + 32 + 96; // 274 +const log = (...a) => console.log(new Date().toISOString().slice(11, 23), ...a); +const sleep = (ms) => new Promise(r => setTimeout(r, ms)); +const started = []; +for (const b of [IGNEUMD, MINER]) if (!existsSync(b)) { console.error(`missing ${b}`); process.exit(2); } +rmSync(TMP, { recursive: true, force: true }); mkdirSync(TMP, { recursive: true }); + +class Node { + constructor(i, connect = []) { this.i = i; this.grpc = BASE + i * 10; this.p2p = BASE + i * 10 + 1; this.json = BASE + i * 10 + 2; this.evm = BASE + i * 10 + 3; this.connect = connect; this.dir = `${TMP}/n${i}`; this.logFile = `${this.dir}/node.log`; } + async start() { + mkdirSync(this.dir, { recursive: true }); + const a = ['--devnet', `--devnet-suffix=${SUFFIX}`, '--nodnsseed', '--disable-upnp', '--nologfiles', '--enable-unsynced-mining', '--utxoindex', '--unsaferpc', + `--appdir=${this.dir}`, `--rpclisten=127.0.0.1:${this.grpc}`, `--rpclisten-json=127.0.0.1:${this.json}`, `--evm-rpclisten=127.0.0.1:${this.evm}`, + `--listen=127.0.0.1:${this.p2p}`, `--override-params-file=${OVERRIDE}`, '--loglevel=info', '--yes']; + if (this.connect.length) a.push(...this.connect.map(c => `--connect=${c}`)); else a.push('--outpeers=0'); + const out = openSync(this.logFile, 'a'); + this.proc = spawn(IGNEUMD, a, { stdio: ['ignore', out, out] }); started.push(this.proc); + await sleep(900); this.rpc = await connectRpc(`ws://127.0.0.1:${this.json}`); + log(`n${this.i} up pid ${this.proc.pid} evm ${this.evm}`); return this; + } + async eth(method, params = []) { + const body = JSON.stringify({ jsonrpc: '2.0', id: 1, method, params }); + const r = await fetch(`http://127.0.0.1:${this.evm}`, { method: 'POST', headers: { 'content-type': 'application/json' }, body }); + const j = await r.json(); if (j.error) throw new Error(`${method}: ${JSON.stringify(j.error)}`); return j.result; + } +} +function miner(node, label) { + const a = ['vmine', `grpc://127.0.0.1:${node.grpc}`, '600', '--label', label, '--share', String(1 / 3), '--bps', '1']; + const out = openSync(`${TMP}/miner-${label}.log`, 'a'); const p = spawn(MINER, a, { stdio: ['ignore', out, out] }); started.push(p); return p; +} +function cpuOf(pid) { try { return parseFloat(spawnSync('ps', ['-o', '%cpu=,time=', '-p', String(pid)], { encoding: 'utf8' }).stdout.trim().split(/\s+/)[0]) || 0; } catch { return 0; } } +function cpuSecs(pid) { try { const t = spawnSync('ps', ['-o', 'time=', '-p', String(pid)], { encoding: 'utf8' }).stdout.trim(); const m = t.match(/(?:(\d+)-)?(\d+):(\d+):(\d+)|(\d+):(\d+)\.(\d+)/); if (!m) return 0; if (m[2] != null) return (+(m[1]||0))*86400 + (+m[2])*3600 + (+m[3])*60 + (+m[4]); return (+m[5])*60 + (+m[6]) + (+('0.'+m[7])); } catch { return 0; } } + +// Build a well-formed-length record with controllable version and a matching/mismatching proof_hash. +function craftRecord({ version = 1, proofMatches = false } = {}) { + const proof = randomBytes(256); + const rec = Buffer.alloc(RECLEN); + let o = 0; + rec.writeUInt16LE(version & 0xffff, o); o += 2; // version + randomBytes(32).copy(rec, o); o += 32; // block + rec.writeBigUInt64LE(BigInt(1 + Math.floor(Math.random() * 1000)), o); o += 8; // number + rec.writeUInt32LE(0, o); o += 4; // shard + randomBytes(48).copy(rec, o); o += 48; // pubkey + randomBytes(20).copy(rec, o); o += 20; // payout + randomBytes(32).copy(rec, o); o += 32; // statement + const ph = proofMatches ? createHash('sha256').update(proof).digest() : randomBytes(32); + ph.copy(rec, o); o += 32; // proof_hash + randomBytes(96).copy(rec, o); o += 96; // signature + return { record: '0x' + rec.toString('hex'), proof: '0x' + proof.toString('hex') }; +} + +const out = { cases: [] }; +try { + const n0 = await new Node(0).start(); + ['v0', 'v1', 'v2'].forEach(l => miner(n0, l)); + const status0 = await n0.eth('igneum_getProvingStatus'); + log(`proving status: activationDaa=${parseInt(status0.activationDaa,16)} verifier=${status0.verifier}`); + // wait for activation + a few assigned shards + let daa = 0, waited = 0; + while (daa < 55 && waited < 180) { await sleep(2000); waited += 2; const s = await n0.eth('igneum_getProvingStatus').catch(() => null); if (s) daa = parseInt(s.tipDaa, 16); if (waited % 10 === 0) log(`daa ${daa}`); } + log(`reached daa ${daa}`); + + async function floodCase(name, opts, n) { + const c0 = cpuSecs(n0.proc.pid); const t0 = Date.now(); + let accepted = 0, rejected = 0; const reasons = {}; + for (let k = 0; k < n; k++) { + const { record, proof } = craftRecord(opts); + try { const r = await n0.eth('igneum_submitProofRecord', [{ record, proof }]); if (r.accepted) accepted++; else { rejected++; reasons[r.reason] = (reasons[r.reason] || 0) + 1; } } + catch (e) { rejected++; const m = String(e.message).slice(0, 60); reasons[m] = (reasons[m] || 0) + 1; } + } + const wall = (Date.now() - t0) / 1000; const c1 = cpuSecs(n0.proc.pid); + const row = { case: name, submitted: n, accepted, rejected, wallSecs: +wall.toFixed(2), rate: +(n / wall).toFixed(1), nodeCpuSecs: +(c1 - c0).toFixed(2), cpuMsPerRecord: +(((c1 - c0) * 1000) / n).toFixed(3), reasons }; + out.cases.push(row); log(`CASE ${name}: ${JSON.stringify(row)}`); + } + await floodCase('proof_hash-mismatch (cheapest)', { proofMatches: false, version: 1 }, 2000); + await floodCase('bad-version (passes proof_hash)', { proofMatches: true, version: 0xbbbb }, 2000); + await floodCase('v1-garbage (reaches record lookup)', { proofMatches: true, version: 1 }, 2000); + + const up = await n0.eth('eth_blockNumber').catch(() => null); + const status1 = await n0.eth('igneum_getProvingStatus').catch(() => null); + out.nodeUpAfter = up != null; + out.poolAfter = status1 && status1.pool; + log(`node up after flood: ${out.nodeUpAfter}; pool ${JSON.stringify(out.poolAfter)}`); + out.ok = out.nodeUpAfter && out.cases.every(c => c.accepted === 0); +} catch (e) { out.error = e.message; log(`FAILED: ${e.message}`); } +finally { + writeFileSync(`${TMP}/flood.json`, JSON.stringify(out, null, 2)); + console.log(JSON.stringify(out, null, 2)); + for (const p of started.reverse()) { try { p.kill('SIGINT'); } catch {} } + await sleep(1500); for (const p of started) { try { p.kill('SIGKILL'); } catch {} } + process.exit(out.ok ? 0 : 1); +} diff --git a/tools/finality-attacks/redteam/override-60x-v3.json b/tools/finality-attacks/redteam/override-60x-v3.json new file mode 100644 index 000000000..d56775c68 --- /dev/null +++ b/tools/finality-attacks/redteam/override-60x-v3.json @@ -0,0 +1 @@ +{"timestamp_deviation_tolerance": 132, "past_median_time_window_size": 27, "difficulty_window_size": 661, "min_difficulty_window_size": 150, "difficulty_rule": "igneum-dual", "coinbase_payload_script_public_key_max_len": 150, "max_coinbase_payload_len": 16384, "max_tx_inputs": 1000, "max_tx_outputs": 1000, "max_signature_script_len": 250000, "max_script_public_key_len": 10000, "mass_per_tx_byte": 1, "mass_per_script_pub_key_byte": 10, "mass_per_sig_op": 1000, "block_mass_limits": {"compute": 500000, "storage": 500000, "transient": 1000000}, "block_lane_limits": {"lanes_per_block": 50, "gas_per_lane": 1000000000}, "storage_mass_parameter": 1000000000000, "deflationary_phase_daa_score": 0, "pre_deflationary_phase_base_subsidy": 50000000000, "skip_proof_of_work": false, "max_block_level": 250, "pruning_proof_m": 1000, "blockrate": {"target_time_per_block": 1000, "ghostdag_k": 18, "past_median_time_sample_rate": 10, "difficulty_sample_rate": 4, "max_block_parents": 10, "mergeset_size_limit": 180, "merge_depth": 60, "finality_depth": 720, "pruning_depth": 13838, "coinbase_maturity": 2}, "pre_crescendo_target_time_per_block": 1000, "crescendo_activation": 0, "genesis_bits": 487587840, "finality": {"checkpoint_interval": 30, "checkpoint_depth": 20, "weight_window": 120, "dust": 5, "presence_window": 1, "aggregators": 8, "equivocation_ban": 120, "min_daa": 120, "aggregator_fallback": 1}, "pow_epoch_blocks": 60, "pow_epoch_lead": 10, "pow_day_ms": 1440000, "finality_v3_activation_daa": 0} \ No newline at end of file diff --git a/tools/finality-attacks/redteam/rtfin.mjs b/tools/finality-attacks/redteam/rtfin.mjs new file mode 100644 index 000000000..dcf5bdf4e --- /dev/null +++ b/tools/finality-attacks/redteam/rtfin.mjs @@ -0,0 +1,251 @@ +// Red-team custom finality scenarios against the finality-fixes build (v3 active), fast-time 60x. +// node rtfin.mjs withhold34 | part5050 | f23 | f24 (run with IGNEUM_FAST_TIME=1) +// Reuses the patched lib/net.mjs (node = vendor/igneum-node-redteam build, miner = fin-attacks, override = v3). +import { Node, Miner, Proxy, stopAll, sleep, log, TMP, IGNEUMD, MINER } from '../lib/net.mjs'; +import { mkdirSync, writeFileSync } from 'node:fs'; +mkdirSync(TMP, { recursive: true }); + +const maxLocked = (cp) => (cp?.checkpoints || []).filter(c => c.state === 'locked').reduce((m, c) => Math.max(m, c.index), 0); +const numLocked = (cp) => (cp?.checkpoints || []).filter(c => c.state === 'locked').length; +const hashAt = (cp, idx) => (cp.checkpoints.find(c => c.index === idx && c.state === 'locked') || {}).hash; +const out = []; +function record(o) { out.push(o); log(`RESULT ${o.scenario}: ${o.pass ? 'PASS' : 'FAIL'} :: ${o.observed}`); } + +// 34% of weight silent (never signs); remaining 66% < 2/3 of total, so finality must PAUSE, not fork. +async function withhold34() { + const secs = 300; + const n0 = await new Node(0).start(); + const n1 = await new Node(1, { connect: [n0.p2p] }).start(); + const miners = []; + // one silent producer at 34% share, five voters sharing 66% + miners.push(new Miner(n0, { label: 'silent', share: 0.34, bps: 6, secs, vote: false }).start()); + for (const [nd, l, sh] of [[n0, 'v0', 0.132], [n0, 'v1', 0.132], [n1, 'v2', 0.132], [n1, 'v3', 0.132], [n1, 'v4', 0.132]]) + miners.push(new Miner(nd, { label: l, share: sh, bps: 6, secs }).start()); + await sleep(secs * 1000 + 3000); + const cps = await Promise.all([n0, n1].map(n => n.rpc.call('getFinalityCheckpoints', { last: 500 }).catch(() => null))); + const w = await n0.rpc.call('getFinalityWeights', {}).catch(() => ({})); + const locked = cps.map(numLocked); + const maxIdx = cps.map(maxLocked); + const conflicts = [n0, n1].map(n => n.grepLog(/CONFLICTING certificate/).length); + // agreement on every commonly-locked index (no fork) + const common = Math.min(...maxIdx); + let agree = true; + for (let i = 1; i <= common; i++) { const h = cps.map(c => hashAt(c, i)).filter(Boolean); if (new Set(h).size > 1) agree = false; } + // finality should report paused / window not fully signed; locks should be few or none while 34% is silent + const paused = cps.some(c => c && c.finalityActive === false) || locked.every(l => l === 0); + const pass = conflicts.every(c => c === 0) && agree; + for (const m of miners) await m.stop(); + record({ scenario: 'withhold34 (34% silent, pause not fork)', + expected: 'finality pauses (signing weight 66% < 2/3 of total); 0 conflicting certs; no fork', + observed: `locked per node ${locked.join('/')}, maxIdx ${maxIdx.join('/')}, conflicts ${conflicts.join('/')}, cross-node agree=${agree}, finalityActive ${cps.map(c=>c&&c.finalityActive).join('/')}, totalWeight ${w.totalWeight} activeWeight ${w.activeWeight}, pausedObserved=${paused}`, + pass }); + await stopAll(); +} + +// 50/50 (3/3) partition kept longer than the old W/(3R) bound but within one weight window; F21 must hold 0 conflicting locks. +async function part5050() { + // fast-time: window 120 DAA, ~1 blk/s/side after split -> old bound W/(3R)=~80s. Keep the split ~105s (> old bound, < one window), then heal. + const warm = 200, split = 105, healWin = 160; + const secs = warm + split + healWin + 90; + const n0 = await new Node(0).start(); + const proxy = await new Proxy(0, n0.p2pPort).start(); + const n1 = await new Node(1, { connect: [proxy.addr] }).start(); + const miners = []; + for (const l of ['a0', 'a1', 'a2']) miners.push(new Miner(n0, { label: l, share: 1 / 6, bps: 6, secs }).start()); + for (const l of ['b0', 'b1', 'b2']) miners.push(new Miner(n1, { label: l, share: 1 / 6, bps: 6, secs }).start()); + await sleep(warm * 1000); + const w0 = await n0.rpc.call('getFinalityWeights', {}).catch(() => ({})); + const before0 = maxLocked(await n0.rpc.call('getFinalityCheckpoints', { last: 400 })); + const before1 = maxLocked(await n1.rpc.call('getFinalityCheckpoints', { last: 400 })); + log(`part5050 cut at warm ${warm}s: window daa ~${w0.daaScore}, voters ${w0.voters}, maxLocked ${before0}/${before1}`); + proxy.cut(); + const t0 = Date.now(); let maxNew0 = before0, maxNew1 = before1, breach0 = null, breach1 = null; + while (Date.now() - t0 < split * 1000) { + const c0 = maxLocked(await n0.rpc.call('getFinalityCheckpoints', { last: 600 }).catch(() => null)); + const c1 = maxLocked(await n1.rpc.call('getFinalityCheckpoints', { last: 600 }).catch(() => null)); + if (c0 > maxNew0) maxNew0 = c0; if (c1 > maxNew1) maxNew1 = c1; + if (breach0 == null && c0 > before0) breach0 = Math.round((Date.now() - t0) / 1000); + if (breach1 == null && c1 > before1) breach1 = Math.round((Date.now() - t0) / 1000); + await sleep(3000); + } + const newLocks = (maxNew0 - before0) + (maxNew1 - before1); + proxy.heal(); + await sleep(healWin * 1000); + const cpsA = await n0.rpc.call('getFinalityCheckpoints', { last: 900 }); + const cpsB = await n1.rpc.call('getFinalityCheckpoints', { last: 900 }); + const after0 = maxLocked(cpsA), after1 = maxLocked(cpsB); + // disagreeing locked indices across nodes after heal (a finality fork) + const common = Math.min(after0, after1); + let disagree = 0; for (let i = 1; i <= common; i++) { const a = hashAt(cpsA, i), b = hashAt(cpsB, i); if (a && b && a !== b) disagree++; } + const conflicts = [n0, n1].map(n => n.grepLog(/CONFLICTING certificate/).length); + for (const m of miners) await m.stop(); + const pass = newLocks === 0 && disagree === 0 && conflicts.every(c => c === 0) && after0 > maxNew0 && after1 > maxNew1; + record({ scenario: 'part5050 (50/50 split > old bound, within one window)', + expected: 'F21 frozen table: 0 new locks either side during the split, 0 disagreeing locked indices, 0 conflicting certs, locks resume after heal', + observed: `split ${split}s (> old W/3R ~80s, window 120 DAA); new locks ${newLocks} (first new side0=${breach0??'none'}s side1=${breach1??'none'}s); disagreeing locked indices after heal ${disagree}; conflicting certs ${conflicts.join('/')}; resumed ${after0>maxNew0&&after1>maxNew1}`, + pass }); + await stopAll(); +} + +// F23: a short equivocation burst, then the ban expires. Two honest nodes stamp the ban at different DAA, so their +// voter lists differ by one key around the expiry and each refuses the other's certificate (voter_count mismatch). +async function f23() { + const secs = 360; // > ~3 windows so the ban (120 DAA) expires well inside the run + const eqSecs = 40; // the equivocator stops early, so the ban has a definite expiry + const n0 = await new Node(0).start(); + const n1 = await new Node(1, { connect: [n0.p2p] }).start(); + const n2 = await new Node(2, { connect: [n0.p2p] }).start(); + const miners = []; + // one equivocator on n0 for a short burst, five honest voters for the whole run + miners.push(new Miner(n0, { label: 'eq', share: 1 / 6, bps: 6, secs: eqSecs, equivocate: true }).start()); + for (const [nd, l] of [[n0, 'h0'], [n1, 'h1'], [n1, 'h2'], [n2, 'h3'], [n2, 'h4']]) + miners.push(new Miner(nd, { label: l, share: 1 / 6, bps: 6, secs }).start()); + await sleep(secs * 1000 + 3000); + const nodes = [n0, n1, n2]; + const ws = await Promise.all(nodes.map(n => n.rpc.call('getFinalityWeights', {}).catch(() => null))); + const strippedUntil = ws.map(w => (w?.keys || []).filter(k => k.strippedUntilDaa > 0).map(k => k.strippedUntilDaa)); + // the F23 signature: per-node divergence in the ban expiry, and voter-count-mismatch refusals / CONFLICTING after the expiry + const voterCountRefusals = nodes.map(n => n.grepLog(/names \d+ voters, this node counts \d+/).length); + const conflicts = nodes.map(n => n.grepLog(/CONFLICTING certificate/).length); + const equivDetections = nodes.map(n => n.grepLog(/EQUIVOCATION by key/).length); + const cps = await Promise.all(nodes.map(n => n.rpc.call('getFinalityCheckpoints', { last: 600 }).catch(() => null))); + const maxIdx = cps.map(maxLocked); + const common = Math.min(...maxIdx.filter(x => x > 0)); + let disagree = 0; for (let i = 1; i <= common; i++) { const h = cps.map(c => hashAt(c, i)).filter(Boolean); if (new Set(h).size > 1) disagree++; } + // distinct ban-expiry values across nodes = node-local stamping divergence + const flatUntil = strippedUntil.flat(); + const distinctUntil = new Set(flatUntil).size; + for (const m of miners) await m.stop(); + const broke = voterCountRefusals.some(c => c > 0) || conflicts.some(c => c > 0) || disagree > 0; + record({ scenario: 'F23 (equivocation ban node-local; honest nodes refuse each other\'s certs)', + expected: 'ban expiry identical across honest nodes; 0 voter-count refusals; 0 CONFLICTING; 0 disagreeing locked indices', + observed: `equiv detections ${equivDetections.join('/')}; stripped-until per node ${strippedUntil.map(a=>a.join(',')||'-').join(' | ')} (distinct values ${distinctUntil}); voter-count-mismatch refusals ${voterCountRefusals.join('/')}; CONFLICTING ${conflicts.join('/')}; disagreeing locked indices ${disagree}; maxLocked ${maxIdx.join('/')}`, + pass: !broke }); + await stopAll(); +} + +// F24: a deep reorg (> checkpoint_depth) moves a determined checkpoint's block off a node's chain. The node never +// re-determines the index, so certificates for the new chain's determination hit cp.hash != cert.checkpoint and are +// pushed to conflicting_certificates (false CONFLICTING) with no equivocation anywhere. +async function f24() { + // minority node determines checkpoints on its own chain during a split, then the majority chain reorgs it deep on heal. + const warm = 160, split = 150, healWin = 200; + const secs = warm + split + healWin + 90; + const n0 = await new Node(0).start(); // majority (4 keys) + const proxy = await new Proxy(0, n0.p2pPort).start(); + const n1 = await new Node(1, { connect: [proxy.addr] }).start(); // minority (2 keys) + const miners = []; + for (const l of ['p0', 'p1', 'p2', 'p3']) miners.push(new Miner(n0, { label: l, share: 1 / 6, bps: 6, secs }).start()); + for (const l of ['q0', 'q1']) miners.push(new Miner(n1, { label: l, share: 1 / 6, bps: 6, secs }).start()); + await sleep(warm * 1000); + const before1Det = (await n1.rpc.call('getFinalityCheckpoints', { last: 400 }).catch(() => ({}))).nextIndex; + proxy.cut(); + await sleep(split * 1000); // both sides advance and determine checkpoints independently + const splitDet1 = (await n1.rpc.call('getFinalityCheckpoints', { last: 400 }).catch(() => ({}))).nextIndex; + proxy.heal(); // the heavier majority chain wins; n1 reorgs deep past its own determinations + await sleep(healWin * 1000); + const nodes = [n0, n1]; + const conflicts = nodes.map(n => n.grepLog(/CONFLICTING certificate/).length); + const certMismatch = nodes.map(n => n.grepLog(/certificate at index \d+ is for .*, this node's checkpoint is/).length); + const equivDetections = nodes.map(n => n.grepLog(/EQUIVOCATION by key/).length); + const cps = await Promise.all(nodes.map(n => n.rpc.call('getFinalityCheckpoints', { last: 900 }).catch(() => null))); + const maxIdx = cps.map(maxLocked); + const common = Math.min(...maxIdx.filter(x => x > 0)); + let disagree = 0; for (let i = 1; i <= common; i++) { const a = hashAt(cps[0], i), b = hashAt(cps[1], i); if (a && b && a !== b) disagree++; } + for (const m of miners) await m.stop(); + const broke = conflicts.some(c => c > 0) || certMismatch.some(c => c > 0) || disagree > 0; + record({ scenario: 'F24 (checkpoint determination never revisited after deep reorg; false CONFLICTING)', + expected: 'after the deep reorg the node re-determines the moved index; 0 false CONFLICTING without equivocation; 0 disagreeing locked indices', + observed: `n1 nextIndex warm=${before1Det} split=${splitDet1}; CONFLICTING ${conflicts.join('/')}; cert-checkpoint-mismatch ${certMismatch.join('/')}; equivocation detections ${equivDetections.join('/')}; disagreeing locked indices ${disagree}; maxLocked ${maxIdx.join('/')}`, + pass: !broke }); + await stopAll(); +} + + +// F24b: the same cut held UNDER merge depth (60 DAA at 60x), long enough for the minority to determine one or two +// checkpoints on its own chain (checkpoint_depth 20 blue), so the heal is a real reorg, not a permanent split. +async function f24b() { + const warm = 160, split = 24, healWin = 150; + const secs = warm + split + healWin + 60; + const n0 = await new Node(0).start(); + const proxy = await new Proxy(0, n0.p2pPort).start(); + const n1 = await new Node(1, { connect: [proxy.addr] }).start(); + const miners = []; + for (const l of ['p0', 'p1', 'p2', 'p3']) miners.push(new Miner(n0, { label: l, share: 1 / 6, bps: 6, secs }).start()); + for (const l of ['q0', 'q1']) miners.push(new Miner(n1, { label: l, share: 1 / 6, bps: 6, secs }).start()); + await sleep(warm * 1000); + const c0 = await n1.rpc.call('getFinalityCheckpoints', { last: 400 }).catch(() => ({})); + proxy.cut(); log(`f24b cut: n1 nextIndex ${c0.nextIndex}`); + await sleep(split * 1000); + const c1 = await n1.rpc.call('getFinalityCheckpoints', { last: 400 }).catch(() => ({})); + const d1 = await n1.rpc.call('getBlockDagInfo').catch(() => ({})); + proxy.heal(); log(`f24b heal: n1 nextIndex ${c1.nextIndex} daa ${d1.virtualDaaScore}`); + await sleep(healWin * 1000); + const nodes = [n0, n1]; + const conflicts = nodes.map(n => n.grepLog(/CONFLICTING certificate/).length); + const certMismatch = nodes.map(n => n.grepLog(/this node's checkpoint is/).length); + const equivDetections = nodes.map(n => n.grepLog(/EQUIVOCATION by key/).length); + const powRejected = nodes.map(n => n.grepLog(/PoW rejected/).length); + const cps = await Promise.all(nodes.map(n => n.rpc.call('getFinalityCheckpoints', { last: 900 }).catch(() => null))); + const sinks = await Promise.all(nodes.map(n => n.rpc.call('getBlockDagInfo').then(d => d.sink || (d.tipHashes||[])[0]).catch(() => null))); + const maxIdx = cps.map(maxLocked); + const common = Math.min(...maxIdx.filter(x => x > 0)); + let disagree = 0; for (let i = 1; i <= common; i++) { const a = hashAt(cps[0], i), b = hashAt(cps[1], i); if (a && b && a !== b) disagree++; } + // indices n1 determined during the split that it never locked while n0 did + const stuck = (cps[1]?.checkpoints || []).filter(c => c.index >= (c0.nextIndex||0) && c.index < (c1.nextIndex||0) && c.state !== 'locked' && hashAt(cps[0], c.index)).map(c => c.index); + for (const m of miners) await m.stop(); + const broke = certMismatch[1] > 0 || stuck.length > 0 || disagree > 0; + record({ scenario: 'F24b (deep reorg under merge depth; determination never revisited)', + expected: 'after a reorg deeper than checkpoint_depth the losing node re-determines the moved indices and accepts the network certificates; 0 false CONFLICTING, 0 stuck indices, 0 disagreeing locks', + observed: `n1 determined ${c0.nextIndex}..${(c1.nextIndex||1)-1} during the ${split}s cut; after heal: cert-for-other-block refusals ${certMismatch.join('/')}, CONFLICTING ${conflicts.join('/')}, equivocation ${equivDetections.join('/')}, PoW-rejected ${powRejected.join('/')}, sinks equal ${sinks[0] && sinks[0] === sinks[1]}, disagreeing locked indices ${disagree}, n1 indices stuck unlocked that n0 locked [${stuck.join(',')}], maxLocked ${maxIdx.join('/')}`, + pass: !broke }); + await stopAll(); +} + +async function f24c() { + const warm = 160, split = 16, healWin = 150; + const secs = warm + split + healWin + 60; + const n0 = await new Node(0).start(); + const proxy = await new Proxy(0, n0.p2pPort).start(); + const n1 = await new Node(1, { connect: [proxy.addr] }).start(); + const miners = []; + for (const l of ['p0', 'p1', 'p2']) miners.push(new Miner(n0, { label: l, share: 1 / 6, bps: 6, secs }).start()); + for (const l of ['q0', 'q1', 'q2']) miners.push(new Miner(n1, { label: l, share: 1 / 6, bps: 6, secs }).start()); + await sleep(warm * 1000); + const c0 = await n1.rpc.call('getFinalityCheckpoints', { last: 400 }).catch(() => ({})); + proxy.cut(); log(`f24c cut: n1 nextIndex ${c0.nextIndex}`); + await sleep(split * 1000); + const c1 = await n1.rpc.call('getFinalityCheckpoints', { last: 400 }).catch(() => ({})); + const d1 = await n1.rpc.call('getBlockDagInfo').catch(() => ({})); + proxy.heal(); log(`f24c heal: n1 nextIndex ${c1.nextIndex} daa ${d1.virtualDaaScore}`); + await sleep(healWin * 1000); + const nodes = [n0, n1]; + const conflicts = nodes.map(n => n.grepLog(/CONFLICTING certificate/).length); + const certMismatch = nodes.map(n => n.grepLog(/this node's checkpoint is/).length); + const equivDetections = nodes.map(n => n.grepLog(/EQUIVOCATION by key/).length); + const powRejected = nodes.map(n => n.grepLog(/PoW rejected/).length); + const cps = await Promise.all(nodes.map(n => n.rpc.call('getFinalityCheckpoints', { last: 900 }).catch(() => null))); + const sinks = await Promise.all(nodes.map(n => n.rpc.call('getBlockDagInfo').then(d => d.sink || (d.tipHashes||[])[0]).catch(() => null))); + const maxIdx = cps.map(maxLocked); + const common = Math.min(...maxIdx.filter(x => x > 0)); + let disagree = 0; for (let i = 1; i <= common; i++) { const a = hashAt(cps[0], i), b = hashAt(cps[1], i); if (a && b && a !== b) disagree++; } + // indices n1 determined during the split that it never locked while n0 did + const loser = (cps[0] && cps[1] && maxIdx[0] >= maxIdx[1]) ? 1 : 0; const stuck = (cps[loser]?.checkpoints || []).filter(c => c.index >= (c0.nextIndex||0) && c.index < (c1.nextIndex||0) && c.state !== 'locked' && hashAt(cps[1 - loser], c.index)).map(c => c.index); + for (const m of miners) await m.stop(); + const broke = certMismatch.some(x => x > 0) || stuck.length > 0 || disagree > 0; + record({ scenario: 'F24c (3/3 split, 16 s cut under merge depth; determination never revisited)', + expected: 'after a reorg deeper than checkpoint_depth the losing node re-determines the moved indices and accepts the network certificates; 0 false CONFLICTING, 0 stuck indices, 0 disagreeing locks', + observed: `n1 determined ${c0.nextIndex}..${(c1.nextIndex||1)-1} during the ${split}s cut; after heal: cert-for-other-block refusals ${certMismatch.join('/')}, CONFLICTING ${conflicts.join('/')}, equivocation ${equivDetections.join('/')}, PoW-rejected ${powRejected.join('/')}, sinks equal ${sinks[0] && sinks[0] === sinks[1]}, disagreeing locked indices ${disagree}, n1 indices stuck unlocked that n0 locked [${stuck.join(',')}], maxLocked ${maxIdx.join('/')}`, + pass: !broke }); + await stopAll(); +} + +const which = process.argv[2]; +const map = { withhold34, part5050, f23, f24, f24b, f24c }; +if (!map[which]) { console.error('usage: node rtfin.mjs withhold34|part5050|f23|f24'); process.exit(2); } +log(`=== ${which} starting (node ${IGNEUMD}, miner ${MINER}) ===`); +map[which]().then(() => { + writeFileSync(`${TMP}/rt-${which}.json`, JSON.stringify(out, null, 2)); + console.log(JSON.stringify(out, null, 2)); + process.exit(out.every(r => r.pass) ? 0 : 1); +}).catch(async (e) => { log(`${which} threw: ${e.stack || e}`); await stopAll(); process.exit(3); }); diff --git a/tools/finality-attacks/redteam/simnet-coinbase-override.json b/tools/finality-attacks/redteam/simnet-coinbase-override.json new file mode 100644 index 000000000..d2e9a420c --- /dev/null +++ b/tools/finality-attacks/redteam/simnet-coinbase-override.json @@ -0,0 +1 @@ +{"max_coinbase_payload_len": 16384, "skip_proof_of_work": true} \ No newline at end of file diff --git a/tools/harness/README.md b/tools/harness/README.md index b492a6d5d..d3b5095b4 100644 --- a/tools/harness/README.md +++ b/tools/harness/README.md @@ -22,7 +22,9 @@ the harness exercises the ordering layer, not a weakened copy of it. The test network uses `127.0.0.1` ports 27200 and up and data under `/tmp/igneum-harness`. It never touches the live devnet (gRPC 26610, P2P 26611, observer 26640/26641/28640), the PC node at 192.168.68.67, or any port other -agents use (up to 27199). Loopback peers are never gossiped (`components/addressmanager/src/lib.rs`), so no link +agents use (up to 27199). `IGNEUM_HARNESS_BASE_PORT=29500 IGNEUM_HARNESS_TMP=/tmp/my-harness` moves the ports (node +`i` takes base + 10i, proxies base + 900 + i) and the data and results directories, so two agents can run the +harness at once (4 October 2026). Loopback peers are never gossiped (`components/addressmanager/src/lib.rs`), so no link forms that a scenario did not ask for. Everything the harness starts is stopped at the end, including on SIGINT. ## Build @@ -48,6 +50,7 @@ node tools/harness/run.mjs # the full catalogue, priority order 5 node tools/harness/run.mjs s5 s2 --quick # named scenarios, short durations node tools/harness/run.mjs --no-bench-log # do not append to docs/bench-log.md node tools/harness/scenarios/s1-withhold.mjs --quick # one scenario on its own +node tools/harness/run.mjs s7 --quick --live-only # s7 Part B only (the vmine flood against real nodes; no simulator binary needed) node tools/harness/run.mjs s3 s4 --fast-time # the 60x fast-time profile (infra/fast-time): merge depth 60 s, so the # partition and eclipse cuts are 10, 30 and 62 s instead of 600, 1,800 and # 3,700; nodes and the simulator come from vendor/igneum-node/target-integration @@ -68,6 +71,7 @@ leaves the test network stopped. Exit code is non-zero if any scenario failed. | 5 | Malformed and boundary inputs on every p2p message and RPC method the fork touches | live + p2p | fork-divergence header and RPC rows; ledger M15: rejected without a crash or a cache build | | 6 | Resource exhaustion (50x template, submit and mempool floods from one peer) | live | honest template p95 under 200 ms, node under its memory bound; numbers recorded | | 7 | Fast-miner flood (50x joins at once, the devnet event) | live + sim | node stays responsive; controller trajectory recorded for the difficulty branch | +| 8 | Steady state: one honest miner at 1 block/s, no flood, 1,500 blocks (ledger M30 follow-up) | live | recorded only: RSS of the mining node and the follower at 0, 500, 1,000 and 1,500 blocks, cache builds, `vmmap -summary` at each milestone (`IGNEUM_STEADY_BLOCKS`, `IGNEUM_STEADY_MAX_MIN`) | Scenario 5 overlaps the `r3-fixes` branch (ledger M15): that branch runs the cheap checks before the lottery engine, caps cache builds and bans the peer. On this node branch (HEAD, `d62708a8`, before r3-fixes) the engine diff --git a/tools/harness/lib/net.mjs b/tools/harness/lib/net.mjs index f0317bc39..e905793a9 100644 --- a/tools/harness/lib/net.mjs +++ b/tools/harness/lib/net.mjs @@ -1,5 +1,7 @@ // Private test network of igneumd processes on 127.0.0.1, ports 27200 and up, data under /tmp/igneum-harness. // Nothing here touches the live devnet (26610/26611, 26640/26641, 28640) or any port below 27200. +// IGNEUM_HARNESS_BASE_PORT and IGNEUM_HARNESS_TMP move the ports and the data directory, so two agents can run the +// harness at the same time (4 October 2026: 29500+ and a private directory for the memory-flood re-run). // // Topology is explicit: a node with `connect: [...]` dials only those addresses and accepts no inbound // connections (kaspad/src/daemon.rs: connect_peers sets outbound target and inbound limit to 0); a node without @@ -24,8 +26,16 @@ export const TARGET = process.env.IGNEUM_HARNESS_TARGET || (FAST_TIME ? `${ROOT} export const IGNEUMD = process.env.IGNEUMD || `${TARGET}/igneumd`; export const PROBE = process.env.IGNEUM_P2P_PROBE || `${TARGET}/igneum-p2p-probe`; export const SIM = process.env.IGNEUM_HARNESS_SIM || `${TARGET}/igneum-harness-sim`; -export const TMP = '/tmp/igneum-harness'; -export const BASE_PORT = 27200; +export const TMP = process.env.IGNEUM_HARNESS_TMP || '/tmp/igneum-harness'; +export const BASE_PORT = parseInt(process.env.IGNEUM_HARNESS_BASE_PORT || '27200', 10); +if (!Number.isInteger(BASE_PORT) || BASE_PORT < 27200 || BASE_PORT > 64000) throw new Error(`IGNEUM_HARNESS_BASE_PORT ${process.env.IGNEUM_HARNESS_BASE_PORT} is not a port in 27200..64000`); + +// u64::MAX ("never" for the height switches) is not a JavaScript number: keep it as a BigInt through the merge and +// write it back as the integer literal the node's parser wants (ledger F25; the same reviver as +// tools/finality-attacks/lib/net.mjs). +const bigReviver = (k, v, ctx) => (typeof v === 'number' && !Number.isSafeInteger(v) && ctx?.source ? BigInt(ctx.source) : v); +const bigStringify = (o) => JSON.stringify(o, (k, v) => (typeof v === 'bigint' ? `BIGINT:${v}` : v)).replace(/"BIGINT:(\d+)"/g, '$1'); +export function readParamsFile(file) { return JSON.parse(readFileSync(file, 'utf8'), bigReviver); } const started = []; // everything to stop at exit @@ -38,8 +48,8 @@ export function overrideParams(extra = {}) { // skip_proof_of_work: the harness miner never hashes; every other rule (timestamps, DAA, GHOSTDAG, merge // depth, mass, coinbase) runs unchanged. Devnet parameters otherwise (1 BPS, k 18, merge depth 3,600), or the // 60x fast-time profile under --fast-time (merge depth 60, finality window 120 DAA, 60-block epochs). - const base = FAST_TIME ? JSON.parse(readFileSync(FAST_TIME_FILE, 'utf8')) : {}; - writeFileSync(file, JSON.stringify({ ...base, skip_proof_of_work: true, ...extra })); + const base = FAST_TIME ? readParamsFile(FAST_TIME_FILE) : {}; + writeFileSync(file, bigStringify({ ...base, skip_proof_of_work: true, ...extra })); return file; } @@ -47,8 +57,8 @@ export function overrideParams(extra = {}) { // devnet, 60 under --fast-time). A scenario cut "beyond merge depth" is mergeDepth + 100 s on the devnet and // mergeDepth + 2 s at 60x. export function clockParams() { - const o = FAST_TIME ? JSON.parse(readFileSync(FAST_TIME_FILE, 'utf8')) : {}; - const mergeDepth = o.blockrate?.merge_depth ?? 3600; + const o = FAST_TIME ? readParamsFile(FAST_TIME_FILE) : {}; + const mergeDepth = Number(o.blockrate?.merge_depth ?? 3600); return { fastTime: FAST_TIME, mergeDepth, scale: 3600 / mergeDepth }; } diff --git a/tools/harness/run.mjs b/tools/harness/run.mjs index 014a898ec..94cb24c74 100644 --- a/tools/harness/run.mjs +++ b/tools/harness/run.mjs @@ -3,7 +3,9 @@ // 127.0.0.1 ports 27200+ and /tmp/igneum-harness, writes a results table per run to docs/bench-log.md, and leaves // the test network stopped. The live devnet (26610/26611, 26640/26641, 28640) and the PC node are never touched. // -// node tools/harness/run.mjs [scenario ...] [--quick] [--no-bench-log] [--fast-time] +// node tools/harness/run.mjs [scenario ...] [--quick] [--no-bench-log] [--fast-time] [--live-only] +// --live-only skips the simulator part of s7 (the vmine flood against real nodes needs only igneumd). +// IGNEUM_HARNESS_BASE_PORT and IGNEUM_HARNESS_TMP move the ports and the data directory (lib/net.mjs). // scenarios: s5 s2 s1 s3 s6 s4 s7 (default: priority order 5,2,1,3,6,4,7) // --quick runs shorter block counts and durations for a smoke run. // --fast-time runs the network and the simulator on infra/fast-time/override-60x.json (every clock-like consensus @@ -12,7 +14,7 @@ // // See tools/harness/README.md. -import { stopAll, assertBinaries, FAST_TIME, TARGET } from './lib/net.mjs'; +import { stopAll, assertBinaries, FAST_TIME, TARGET, BASE_PORT, TMP } from './lib/net.mjs'; import { benchLogEntry, appendBenchLog } from './lib/report.mjs'; import { stubRows } from './scenarios/stubs.mjs'; import { execSync } from 'node:child_process'; @@ -25,6 +27,7 @@ const SCENARIOS = { s5: () => import('./scenarios/s5-malformed.mjs'), s6: () => import('./scenarios/s6-exhaustion.mjs'), s7: () => import('./scenarios/s7-flood.mjs'), + s8: () => import('./scenarios/s8-steady.mjs'), // steady state, not in the priority order (25 min, no criterion) }; const PRIORITY = ['s5', 's2', 's1', 's3', 's6', 's4', 's7']; @@ -34,7 +37,7 @@ function machineLine() { try { mem = (parseInt(execSync('sysctl -n hw.memsize').toString().trim(), 10) / 2 ** 30).toFixed(0) + ' GB'; } catch { } let load = ''; try { load = execSync('uptime').toString().match(/load averages?: ([\d. ]+)/)?.[1] || ''; } catch { } const profile = FAST_TIME ? 'skip_proof_of_work devnet on the 60x fast-time profile, infra/fast-time/override-60x.json' : 'skip_proof_of_work devnet'; - return `Machine: ${cpu}, ${mem}, load ${load.trim()}. Private test network of igneumd (release, ${profile}) on 127.0.0.1 ports 27200+, data /tmp/igneum-harness; the live devnet and the PC node were not touched. Harness: tools/harness/, binaries ${TARGET}.`; + return `Machine: ${cpu}, ${mem}, load ${load.trim()}. Private test network of igneumd (release, ${profile}) on 127.0.0.1 ports ${BASE_PORT}+, data ${TMP}; the live devnet and the PC node were not touched. Harness: tools/harness/, binaries ${TARGET}.`; } // Assemble one bench-log entry from result JSONs already written under /tmp/igneum-harness/results, without @@ -59,6 +62,7 @@ async function main() { const quick = args.includes('--quick'); const noBench = args.includes('--no-bench-log'); const assemble = args.includes('--assemble'); + const liveOnly = args.includes('--live-only'); if (assemble) { const allRows = await assembleFromResults(); @@ -70,7 +74,7 @@ async function main() { machine: machineLine(), rows: allRows, notes: [ - 'Full JSON per scenario under /tmp/igneum-harness/results and /tmp/igneum-harness/sim. The simulator (igneum/harness-sim in the fork worktree) runs real consensus code in virtual time with PoW skipped, as rusty-kaspa simpa does; the live scenarios (5, 6, 7 Part B) drive real igneumd processes over wRPC and the fork\'s own p2p (igneum/p2p-probe).', + `Full JSON per scenario under ${TMP}/results and ${TMP}/sim. The simulator (igneum/harness-sim in the fork worktree) runs real consensus code in virtual time with PoW skipped, as rusty-kaspa simpa does; the live scenarios (5, 6, 7 Part B) drive real igneumd processes over wRPC and the fork's own p2p (igneum/p2p-probe).`, 'Finality and difficulty-controller scenarios are stubs here: their criteria are written and they run against those branches once merged into the harness worktree (see tools/harness/scenarios/stubs.mjs).', ], }); @@ -89,7 +93,7 @@ async function main() { console.log(`\n==== scenario ${key}${quick ? ' (quick)' : ''}${FAST_TIME ? ' (fast-time 60x)' : ''} ====`); try { const mod = await SCENARIOS[key](); - const { rows } = await mod.run({ quick }); + const { rows } = await mod.run({ quick, liveOnly }); allRows.push(...rows); } catch (e) { console.error(`scenario ${key} threw: ${e.stack || e}`); @@ -110,7 +114,7 @@ async function main() { machine: machineLine(), rows: allRows, notes: [ - 'Full JSON per scenario under /tmp/igneum-harness/results and /tmp/igneum-harness/sim. The simulator (igneum/harness-sim in the fork worktree) runs real consensus code in virtual time with PoW skipped, as rusty-kaspa simpa does; the live scenarios (5, 6, 7 Part B) drive real igneumd processes over wRPC and the fork\'s own p2p (igneum/p2p-probe).', + `Full JSON per scenario under ${TMP}/results and ${TMP}/sim. The simulator (igneum/harness-sim in the fork worktree) runs real consensus code in virtual time with PoW skipped, as rusty-kaspa simpa does; the live scenarios (5, 6, 7 Part B) drive real igneumd processes over wRPC and the fork's own p2p (igneum/p2p-probe).`, 'Finality and difficulty-controller scenarios are stubs here: their criteria are written and they run against those branches once merged into the harness worktree (see tools/harness/scenarios/stubs.mjs).', ], }); diff --git a/tools/harness/scenarios/s6-exhaustion.mjs b/tools/harness/scenarios/s6-exhaustion.mjs index f7f081dfb..2d67311f4 100644 --- a/tools/harness/scenarios/s6-exhaustion.mjs +++ b/tools/harness/scenarios/s6-exhaustion.mjs @@ -5,7 +5,9 @@ // mempool: submitTransaction at 500/s of transactions spending unknown outputs (rejected one by one; funded // transactions need a wallet key, not done here). // Criterion: the honest peer's template latency p95 stays under 200 ms and both nodes stay under their memory bound -// (baseline RSS + 512 MB), alive, on one sink. Numbers are recorded per load. +// (baseline RSS + 512 MB), alive, on one sink. Numbers are recorded per load: `rss_start` and `rss_delta` are the +// load's own growth (RSS at its start to its peak); `rss_peak` minus `rss_baseline` is cumulative since the warm-up. +// `cache_builds` counts the node's "PoW cache built" log lines during the load (ledger M30: each is 256 MiB). import { Node, stopAll, dagInfo, log, sleep, assertBinaries } from '../lib/net.mjs'; import { Rpc } from '../lib/rpc.mjs'; @@ -34,6 +36,8 @@ export async function run({ quick = false } = {}) { async function load(name, perSec, fire, maxInflight = 200) { probeB.samples = []; probeA.samples = []; const t0 = Date.now(); let sent = 0, ok = 0, err = 0; const lat = []; let inflight = 0; + const rssStart = { a: a.rssMb(), b: b.rssMb() }; + const builds0 = { a: a.grepLog(/PoW cache built/).length, b: b.grepLog(/PoW cache built/).length }; const rssSeries = []; const tick = setInterval(() => rssSeries.push({ t_s: (Date.now() - t0) / 1000, a: a.rssMb(), b: b.rssMb() }), 2000); while (Date.now() - t0 < loadSecs * 1000) { @@ -51,11 +55,13 @@ export async function run({ quick = false } = {}) { const r = { requests_sent: sent, accepted: ok, rejected_or_error: err, rate_per_s: +(sent / loadSecs).toFixed(0), request_latency_ms: summarize(lat), honest_template_ms: bStats, attacked_node_template_ms: aStats, - rss_series: rssSeries, rss_peak: { a: Math.max(rss0.a, ...rssSeries.map(x => x.a)), b: Math.max(rss0.b, ...rssSeries.map(x => x.b)) }, + rss_series: rssSeries, rss_start: rssStart, rss_peak: { a: Math.max(rss0.a, ...rssSeries.map(x => x.a)), b: Math.max(rss0.b, ...rssSeries.map(x => x.b)) }, both_alive: a.alive() && b.alive(), }; + r.rss_delta = { a: r.rss_peak.a - rssStart.a, b: r.rss_peak.b - rssStart.b }; + r.cache_builds = { a: a.grepLog(/PoW cache built/).length - builds0.a, b: b.grepLog(/PoW cache built/).length - builds0.b }; results[name] = r; - log(`s6 ${name}: ${r.rate_per_s}/s, honest p95 ${bStats.p95} ms (base ${baseB.p95}), rss a ${r.rss_peak.a} b ${r.rss_peak.b}, alive ${r.both_alive}`); + log(`s6 ${name}: ${r.rate_per_s}/s, honest p95 ${bStats.p95} ms (base ${baseB.p95}), rss a ${rssStart.a}->${r.rss_peak.a} b ${rssStart.b}->${r.rss_peak.b} MB (+${r.rss_delta.a}/+${r.rss_delta.b}), cache builds ${r.cache_builds.a}/${r.cache_builds.b}, alive ${r.both_alive}`); return r; } @@ -83,11 +89,11 @@ export async function run({ quick = false } = {}) { const data = { baseline_template_ms: { a: baseA, b: baseB }, rss_baseline: rss0, loads: results, recovery_template_ms: recovery, final: { blocks_a: ia.blockCount, blocks_b: ib.blockCount, same_sink: sameSink }, alive, mem_bound_mb: MEM_BOUND_MB }; const worst = Math.max(...Object.values(results).map(r => r.honest_template_ms.p95)); - const peakRss = Object.entries(results).map(([k, r]) => `${k.split('_')[0]} +${Math.max(r.rss_peak.a - rss0.a, r.rss_peak.b - rss0.b)}MB`).join(', '); + const peakRss = Object.entries(results).map(([k, r]) => `${k.split('_')[0]} +${Math.max(r.rss_delta.a, r.rss_delta.b)}MB (${r.cache_builds.a}/${r.cache_builds.b} cache builds)`).join(', '); const row = { scenario: '6 resource exhaustion (50x template, submit and mempool floods from one peer)', criterion: 'honest template p95 < 200 ms and both nodes under baseline RSS + 512 MB, alive, one sink', - result: `honest template p95 worst ${worst} ms across loads (baseline ${baseB.p95} ms); ${Object.entries(results).map(([k, r]) => k.replace('_flood', '').replace('_', ' ') + ' ' + r.rate_per_s + '/s').join(', ')}; RSS growth ${peakRss}; alive ${alive}; same sink ${sameSink}`, + result: `honest template p95 worst ${worst} ms across loads (baseline ${baseB.p95} ms); ${Object.entries(results).map(([k, r]) => k.replace('_flood', '').replace('_', ' ') + ' ' + r.rate_per_s + '/s').join(', ')}; RSS growth per load ${peakRss}, cumulative +${Math.max(...Object.values(results).map(r => Math.max(r.rss_peak.a - rss0.a, r.rss_peak.b - rss0.b)))}MB over baseline ${rss0.a}/${rss0.b}; alive ${alive}; same sink ${sameSink}`, pass: alive && latencyOk && underBound && sameSink, }; saveResult('s6-exhaustion', { rows: [row], data }); diff --git a/tools/harness/scenarios/s7-flood.mjs b/tools/harness/scenarios/s7-flood.mjs index 871398d60..47d450852 100644 --- a/tools/harness/scenarios/s7-flood.mjs +++ b/tools/harness/scenarios/s7-flood.mjs @@ -11,11 +11,14 @@ import { Miner, LatencyProbe, difficultyRatio } from '../lib/miner.mjs'; import { saveResult } from '../lib/report.mjs'; import { runSim } from '../lib/sim.mjs'; -export async function run({ quick = false } = {}) { +export async function run({ quick = false, liveOnly = false } = {}) { assertBinaries(); const rows = []; const data = {}; // ---------- Part A: simulator ---------- const leave = quick ? 1200 : 1800, secs = quick ? 2400 : 4800; + if (liveOnly) { + rows.push({ scenario: '7 fast-miner flood, controller trajectory (sim)', criterion: 'trajectory recorded', result: 'skipped (--live-only)', pass: null }); + } else { const r = runSim('s7-flood', ['--scenario', 'flood', '--join-at', '600', '--leave-at', String(leave), '--flood-mult', '50', '--secs', String(secs), '--sample-secs', '30', '--seed', '77']); const g = r.genesis_time_ms; const traj = r.samples.map((s, i, a) => ({ t_s: (s.t - g) / 1000, bits: s.bits[0], ratio: +s.difficulty_ratio[0].toFixed(3), blocks: s.block_counts[0], rate: i ? +((s.block_counts[0] - a[i - 1].block_counts[0]) / ((s.t - a[i - 1].t) / 1000)).toFixed(2) : 0, daa: s.daa_scores[0] })); @@ -26,6 +29,7 @@ export async function run({ quick = false } = {}) { const settledAfterLeave = afterLeave.find(t => Math.abs(t.rate - 1) < 0.25 && afterLeave.slice(afterLeave.indexOf(t), afterLeave.indexOf(t) + 4).every(x => Math.abs(x.rate - 1) < 0.25)); data.sim = { trajectory: traj, peak_rate_bps: peakRate, peak_difficulty_ratio: peakRatio, trough_rate_after_leave: troughRate, settled_after_join_s: settled ? settled.t_s - 600 : null, settled_after_leave_s: settledAfterLeave ? settledAfterLeave.t_s - leave : null, counts: r.counts, wall_s: r.wall_s }; rows.push({ scenario: '7 fast-miner flood, controller trajectory (sim, Kaspa sampled DAA on HEAD)', criterion: 'trajectory recorded for the difficulty branch (bits, blocks per second, settle times)', result: `50x joins at 600 s: peak ${peakRate} blocks/s, difficulty x${peakRatio.toFixed(1)}, within 25% of 1 BPS after ${data.sim.settled_after_join_s ?? 'never'} s; leaves at ${leave} s: trough ${troughRate} blocks/s, back within 25% after ${data.sim.settled_after_leave_s ?? 'never'} s`, pass: true }); + } // ---------- Part B: live responsiveness ---------- const a = await new Node(0, { name: 's7a' }).start(); @@ -39,7 +43,10 @@ export async function run({ quick = false } = {}) { const flood = new Miner({ node: a, share: 1, label: 'flood-s7', rateMult: 50 }); await flood.start(); const floodSecs = quick ? 60 : 180; const samples = []; - for (let i = 0; i < floodSecs / 10; i++) { await sleep(10000); const ia = await dagInfo(a); const ib = await dagInfo(b); samples.push({ t_s: (i + 1) * 10, blocks_a: ia.blockCount, blocks_b: ib.blockCount, difficulty_a: ia.difficulty, sink_same: ia.sink === ib.sink, rss_a: a.rssMb(), rss_b: b.rssMb(), flood_accepted: flood.accepted, flood_rejected: flood.rejected, honest_accepted: honest.accepted }); log(`s7 live ${(i + 1) * 10}s: a ${ia.blockCount} b ${ib.blockCount} same=${ia.sink === ib.sink} flood ${flood.accepted}/${flood.rejected} honest ${honest.accepted}`); } + // Ledger M30: the node's "PoW cache built" lines (256 MiB each) are counted beside every RSS sample + const builds = (n) => n.grepLog(/PoW cache built/).length; + const builds0 = { a: builds(a), b: builds(b) }; + for (let i = 0; i < floodSecs / 10; i++) { await sleep(10000); const ia = await dagInfo(a); const ib = await dagInfo(b); samples.push({ t_s: (i + 1) * 10, blocks_a: ia.blockCount, blocks_b: ib.blockCount, difficulty_a: ia.difficulty, sink_same: ia.sink === ib.sink, rss_a: a.rssMb(), rss_b: b.rssMb(), cache_builds_a: builds(a) - builds0.a, cache_builds_b: builds(b) - builds0.b, flood_accepted: flood.accepted, flood_rejected: flood.rejected, honest_accepted: honest.accepted }); log(`s7 live ${(i + 1) * 10}s: a ${ia.blockCount} b ${ib.blockCount} same=${ia.sink === ib.sink} flood ${flood.accepted}/${flood.rejected} honest ${honest.accepted} rss ${a.rssMb()}/${b.rssMb()} MB cache builds ${builds(a) - builds0.a}/${builds(b) - builds0.b}`); } flood.stop(); const under = probe.stats(); probe.samples = []; await sleep(5000); @@ -47,14 +54,14 @@ export async function run({ quick = false } = {}) { const alive = a.alive() && b.alive(); const ia = await dagInfo(a), ib = await dagInfo(b); await stopAll(); - data.live = { baseline_template_ms: base, under_flood_template_ms: under, after_flood_template_ms: after, samples, rss_before: rss0, rss_peak: { a: Math.max(...samples.map(s => s.rss_a)), b: Math.max(...samples.map(s => s.rss_b)) }, flood: { accepted: flood.accepted, rejected: flood.rejected, errors: flood.errors }, honest: { accepted: honest.accepted, rejected: honest.rejected }, final: { blocks_a: ia.blockCount, blocks_b: ib.blockCount, same_sink: ia.sink === ib.sink, difficulty_a: ia.difficulty, ratio: difficultyRatio(ia.difficulty ? 0 : 0) }, alive }; - rows.push({ scenario: '7 fast-miner flood, live (50 blocks/s from one peer)', criterion: 'node stays responsive: honest template p95 < 200 ms, both nodes alive, same sink', result: `flood accepted ${flood.accepted} blocks in ${floodSecs} s (${(flood.accepted / floodSecs).toFixed(1)}/s); honest template p50/p95/max ${under.p50}/${under.p95}/${under.max} ms under flood (baseline ${base.p50}/${base.p95}/${base.max}); rss a ${rss0.a}->${data.live.rss_peak.a} MB, b ${rss0.b}->${data.live.rss_peak.b} MB; alive ${alive}; same sink ${ia.sink === ib.sink}`, pass: alive && under.p95 < 200 && ia.sink === ib.sink }); + data.live = { baseline_template_ms: base, under_flood_template_ms: under, after_flood_template_ms: after, samples, rss_before: rss0, rss_peak: { a: Math.max(...samples.map(s => s.rss_a)), b: Math.max(...samples.map(s => s.rss_b)) }, cache_builds: { a: builds(a) - builds0.a, b: builds(b) - builds0.b, before_flood: builds0 }, flood: { accepted: flood.accepted, rejected: flood.rejected, errors: flood.errors }, honest: { accepted: honest.accepted, rejected: honest.rejected }, final: { blocks_a: ia.blockCount, blocks_b: ib.blockCount, same_sink: ia.sink === ib.sink, difficulty_a: ia.difficulty, ratio: difficultyRatio(ia.difficulty ? 0 : 0) }, alive }; + rows.push({ scenario: '7 fast-miner flood, live (50 blocks/s from one peer)', criterion: 'node stays responsive: honest template p95 < 200 ms, both nodes alive, same sink', result: `flood accepted ${flood.accepted} blocks in ${floodSecs} s (${(flood.accepted / floodSecs).toFixed(1)}/s); honest template p50/p95/max ${under.p50}/${under.p95}/${under.max} ms under flood (baseline ${base.p50}/${base.p95}/${base.max}); rss a ${rss0.a}->${data.live.rss_peak.a} MB, b ${rss0.b}->${data.live.rss_peak.b} MB (cache builds ${data.live.cache_builds.a}/${data.live.cache_builds.b}); alive ${alive}; same sink ${ia.sink === ib.sink}`, pass: alive && under.p95 < 200 && ia.sink === ib.sink }); saveResult('s7-flood', { rows, data }); return { rows, data }; } if (import.meta.url === `file://${process.argv[1]}`) { - const r = await run({ quick: process.argv.includes('--quick') }); + const r = await run({ quick: process.argv.includes('--quick'), liveOnly: process.argv.includes('--live-only') }); console.log(JSON.stringify(r.rows, null, 2)); process.exit(0); } diff --git a/tools/harness/scenarios/s8-steady.mjs b/tools/harness/scenarios/s8-steady.mjs new file mode 100644 index 000000000..fbd9e3810 --- /dev/null +++ b/tools/harness/scenarios/s8-steady.mjs @@ -0,0 +1,77 @@ +// Scenario 8: steady state, no flood (ledger M30 follow-up, 5 October 2026). Node A takes one honest virtual miner +// at 1 block/s; node B follows. Every 60 s: RSS of both nodes, the block count, the count of "PoW cache built" lines +// in each node log, and (at the block milestones) `vmmap -summary` of both nodes, so malloc-held memory can be told +// from mapped. Runs until `blocks` chain blocks (IGNEUM_STEADY_BLOCKS, default 1,500) or `maxMinutes` +// (IGNEUM_STEADY_MAX_MIN, default 40), whichever first. No pass criterion: the row records RSS at 0, 500, 1,000 and +// 1,500 blocks, so the ledger can state MB per 1,000 blocks before and after a change. + +import { Node, stopAll, dagInfo, log, sleep, assertBinaries, TMP } from '../lib/net.mjs'; +import { Miner } from '../lib/miner.mjs'; +import { saveResult } from '../lib/report.mjs'; +import { execSync } from 'node:child_process'; +import { mkdirSync, writeFileSync } from 'node:fs'; + +const MILESTONES = [0, 500, 1000, 1500]; + +function vmmapSummary(node, tag) { + if (!node.alive()) return null; + try { + const out = execSync(`vmmap -summary ${node.proc.pid}`, { timeout: 60000, stdio: ['ignore', 'pipe', 'ignore'] }).toString(); + mkdirSync(`${TMP}/vmmap`, { recursive: true }); + writeFileSync(`${TMP}/vmmap/${node.name}-${tag}.txt`, out); + const pick = (re) => { const m = out.match(re); return m ? m[1].trim() : null; }; + return { + physical_footprint: pick(/Physical footprint:\s+([^\n]+)/), + malloc_total: pick(/\nMALLOC\s+[^\n]*?\s(\S+)\s+\S+\s+\S+\s+\S+\s+\S+\s+\S+\s*\n/) || pick(/TOTAL\s+([^\n]+)/), + file: `${TMP}/vmmap/${node.name}-${tag}.txt`, + }; + } catch (e) { return { error: String(e.message).slice(0, 120) }; } +} + +export async function run({ blocks = parseInt(process.env.IGNEUM_STEADY_BLOCKS || '1500', 10), maxMinutes = parseInt(process.env.IGNEUM_STEADY_MAX_MIN || '40', 10) } = {}) { + assertBinaries(); + const a = await new Node(0, { name: 's8a' }).start(); + const b = await new Node(1, { name: 's8b', connect: [a.p2p] }).start(); + await sleep(5000); + const builds = (n) => n.grepLog(/PoW cache built/).length; + const samples = []; const milestones = {}; + const t0 = Date.now(); + const sample = async (tag) => { + const ia = await dagInfo(a); const ib = await dagInfo(b); + const s = { t_s: Math.round((Date.now() - t0) / 1000), blocks_a: ia.blockCount, blocks_b: ib.blockCount, daa_a: ia.virtualDaaScore ?? null, rss_a: a.rssMb(), rss_b: b.rssMb(), cache_builds_a: builds(a), cache_builds_b: builds(b), same_sink: ia.sink === ib.sink, load: (() => { try { return execSync('uptime').toString().match(/load averages?: ([\d. ]+)/)?.[1].trim(); } catch { return ''; } })() }; + samples.push(s); + log(`s8 ${s.t_s}s: blocks ${s.blocks_a}/${s.blocks_b} rss ${s.rss_a}/${s.rss_b} MB cache builds ${s.cache_builds_a}/${s.cache_builds_b} same=${s.same_sink} load ${s.load}`); + if (tag) { s.vmmap_a = vmmapSummary(a, tag); s.vmmap_b = vmmapSummary(b, tag); milestones[tag] = s; } + return s; + }; + await sample('0'); + const honest = new Miner({ node: a, share: 1, label: 'honest-s8' }); await honest.start(); + let nextMilestone = 1; + while (Date.now() - t0 < maxMinutes * 60000) { + await sleep(60000); + const s = await sample(null); + while (nextMilestone < MILESTONES.length && s.blocks_a >= MILESTONES[nextMilestone]) { + const tag = String(MILESTONES[nextMilestone]); s.vmmap_a = vmmapSummary(a, tag); s.vmmap_b = vmmapSummary(b, tag); milestones[tag] = s; nextMilestone++; + } + if (s.blocks_a >= blocks) break; + } + honest.stop(); + const end = await sample('end'); + const alive = a.alive() && b.alive(); + await stopAll(); + const at = (k) => milestones[k] ? `${milestones[k].rss_a}/${milestones[k].rss_b} MB at ${milestones[k].blocks_a} blocks (${milestones[k].cache_builds_a}/${milestones[k].cache_builds_b} builds)` : 'not reached'; + const row = { + scenario: '8 steady state, one honest miner at 1 block/s, no flood (RSS per 1,000 blocks)', + criterion: 'recorded: RSS of the mining node / the follower at 0, 500, 1,000 and 1,500 blocks, cache builds', + result: `0: ${at('0')}; 500: ${at('500')}; 1000: ${at('1000')}; 1500: ${at('1500')}; end ${end.blocks_a} blocks in ${end.t_s} s: ${end.rss_a}/${end.rss_b} MB; honest accepted ${honest.accepted} rejected ${honest.rejected} errors ${honest.errors}; alive ${alive}`, + pass: null, + }; + saveResult('s8-steady', { rows: [row], data: { samples, milestones, honest: { accepted: honest.accepted, rejected: honest.rejected, errors: honest.errors }, alive } }); + return { rows: [row], data: { samples, milestones } }; +} + +if (import.meta.url === `file://${process.argv[1]}`) { + const r = await run({}); + console.log(JSON.stringify(r.rows, null, 2)); + process.exit(0); +}