diff --git a/docs/spec/02-consensus.md b/docs/spec/02-consensus.md index 5661bb94..0abfa8e5 100644 --- a/docs/spec/02-consensus.md +++ b/docs/spec/02-consensus.md @@ -152,3 +152,12 @@ Block number, timestamp, blockhash, coinbase and prevrandao over the ordered seq ## 2.7 What the devnet showed and did not show Measured (`docs/bench-log.md`, devnet entry): three kaspad nodes at Kaspa's devnet parameters (10 BPS, k 124) held identical block counts, DAA scores and sink hashes at 18 of 19 ten-second samples under a 27 MH/s CPU miner; the DAA raised difficulty from genesis bits at block 6,018 and the block rate fell from 56 to 62 blocks/s toward the 10 BPS target. GHOSTDAG k was not exercised (a single serial miner never produced parallel blocks); a second miner is the next step. Nothing in that run used an Igneum parameter. + +## 2.8 One parameter set per network (ledger G12 and X18, 4 October 2026 night) + +Status: Implemented on the node's `fud-consensus` branch, pending rollout. + +1. A node's consensus parameters come from its network's constants (`Params`) and, on devnet and simnet only, from an override file. The override file is refused on mainnet (since devnet v4) and the environment never sets a consensus parameter outside devnet and simnet: `IGNEUM_POW_EPOCH_BLOCKS`, `IGNEUM_POW_EPOCH_LEAD` and `IGNEUM_POW_DAY_MS` are applied on devnet and simnet after the file, and on mainnet and testnet are ignored with one line at start ("Ignoring ... the environment never sets a consensus parameter outside devnet and simnet"). The PoW schedule the node runs is installed from `Params` on every network at start; nothing in the node reads the environment for it later. A miner takes all three schedule values from the block template (`pow_epoch.day_ms` joined `epoch_blocks` and `epoch_lead`), never from its environment. +2. The params digest. Every node computes `Params::consensus_digest()`: BLAKE2b-256 under the domain `IgneumParamsDigest` over, in a fixed tagged order, the network name, the genesis hash, bits, timestamp and DAA score, the difficulty windows and rule, the coinbase and mass limits, the lane limits, the storage mass parameter, the deflationary schedule, `skip_proof_of_work`, the block level and proof parameters, every blockrate field (target time, k, sample rates, parents, mergeset, merge depth, finality depth, pruning depth, maturity), the crescendo activation, every finality field (3.10), the PoW schedule and the three activation heights (`difficulty_v2_activation_daa`, `proving_v0_activation_daa`, `finality_v3_activation_daa`). Not covered: seeders, ports, the maximum difficulty target (a constant of the code) and `timestamp_deviation_tolerance` (dead, read by nothing). The node prints the digest at start. +3. The handshake. The p2p version message carries the digest (`paramsDigest`, field 11). A peer whose digest differs is refused at the handshake with one WARN naming both digests ("Refusing peer ...: consensus params digest mismatch, local X remote Y (the peer's override file, environment or build differs)") and the error `ParamsDigestMismatch`; no flow is registered and no block is exchanged. A peer that sends no digest (a build older than this rule) is refused on mainnet and testnet; on devnet and simnet it is let in with a WARN per connection while the devnet rolls, an allowance to remove once every devnet node carries the digest. +4. Why: round 4 found that two nodes with two override files connected and diverged per field, a finality mismatch only as a WARN after the fact (X18), and that the environment set the epoch length on every network including mainnet (G12). Unit tests `consensus_digest_covers_every_consensus_field_and_nothing_else` (every consensus field moves the digest, the dead field does not, the networks differ, the same file gives the same digest) and `env_pow_schedule_is_devnet_and_simnet_only`; the two-node run in `docs/bench-log.md`, "round-4 consensus items" (digest run). diff --git a/docs/spec/03-finality.md b/docs/spec/03-finality.md index aae93991..b5c0a7a6 100644 --- a/docs/spec/03-finality.md +++ b/docs/spec/03-finality.md @@ -29,10 +29,10 @@ All in public, on the hashrate charts. 51% never reaches 2/3 while honest miners ## 3.2 Checkpoints -- **C1.** Checkpoint i is the selected-chain block at blue score 30 i. It is determined once the virtual's blue score reaches 30 i + d. d = 60 at 1 block per second is a placeholder (ledger F7): the gate 3 devnet records the reorg-depth distribution and sets d so that a vote split at one index is rare and self-heals at the next. d scales with block rate. A lock lands about 90 to 120 s after a transaction (Designed; simulated lock latency after the checkpoint block is median 2.5 s, p99 4.6 s at a 2-s inter-region delay, `sim/results_v2.md` A). +- **C1.** Checkpoint i is the selected-chain block at blue score 30 i. It is determined once the virtual's blue score reaches 30 i + d. d = 60 at 1 block per second is a placeholder (ledger F7): the gate 3 devnet records the reorg-depth distribution and sets d so that a vote split at one index is rare and self-heals at the next. d scales with block rate. Re-determination (rule of 4 October 2026, night, ledger F24): while index i is not locked, a node whose selected chain moves past C_i (a reorg deeper than d) determines index i again on its new chain; its own votes for the old block stand (a key never signs two blocks at one index) and a certificate the network formed over the new block, received meanwhile and held pending, is then verified. A locked index is never re-determined (3.11.4): fork choice keeps the chain through its block, and a certificate for another block there is a conflict (C4). A lock lands about 90 to 120 s after a transaction (Designed; simulated lock latency after the checkpoint block is median 2.5 s, p99 4.6 s at a 2-s inter-region delay, `sim/results_v2.md` A). - **C2.** A vote is a BLS signature over `(chain_id, i, hash(C_i))` under a fixed domain-separation tag. Votes gossip as their own message type. - **C3.** A lock certificate for index i is an aggregate BLS signature over one checkpoint block hash with a bitmap of signers, whose signed weight meets Q3. Every block carries the highest certificate its producer knows. A block whose selected chain does not pass through every certified checkpoint in its past is invalid (section 2.4). -- **C4.** A node holding a certificate for index i rejects any other certificate for index i and publishes the pair as evidence (section 3.6). +- **C4.** A node holding a LOCK at index i rejects any other certificate for index i and publishes the pair as evidence (section 3.6). A certificate over a block that is not the node's own determination at an index it has not locked is not a conflict: the chain may still move to that block (C1 re-determination, ledger F24), so the node keeps it pending, bounded, until it does or the index is left behind. A certificate naming a block that cannot be index i's checkpoint on any chain (its blue score is under 30 i, or its selected parent's is not) is refused outright. - **C5.** No certificate may form in the chain's first 3,600 DAA seconds (design document). See 3.8 for the proposed first-month rule. ## 3.3 Quorum @@ -118,7 +118,7 @@ What a node does when it holds two valid certificates at one index after a parti ## 3.6 Equivocation evidence -Two votes by one key for different checkpoint blocks at one index are equivocation. The evidence (the two votes) is a transaction that any block MAY include. On inclusion: the key's weight is zero for the rest of the current window and its blocks earn no weight for the next 2,592,000 DAA s. There is no coin penalty. In the simulation, evidence is detected only at the heal and the penalty is forward-looking only; the model does not revoke the conflicting certificates (`sim/results_v2.md`, "cannot tell us"), which is why 3.5's post-heal proposal strikes the equivocators' weight retroactively for the re-evaluation. +Two votes by one key for different checkpoint blocks at one index are equivocation. The evidence (the two votes) is a transaction that any block MAY include. On inclusion: the key's weight is zero for the rest of the current window and its blocks earn no weight for the next 2,592,000 DAA s. There is no coin penalty. The ban is a function of the chain (rule of 4 October 2026, night, ledger F23): at checkpoint C the key is stripped exactly when some block in the past of C carries the evidence and `daa(C) < daa(e) + 2,592,000`, where e is the carrier of lowest DAA score in C's past (the evidence block). Evidence a node has seen but no block in C's past carries strips nothing at C, so a node that detected the equivocation itself, from two votes over RPC or gossip, puts the evidence in its next block and waits for the chain. Every node with C's past computes the same voter list at C, whenever and however it first saw the evidence; before this rule a node stamped its own detection time and honest nodes refused each other's certificates for the length of the discrepancy. In the simulation, evidence is detected only at the heal and the penalty is forward-looking only; the model does not revoke the conflicting certificates (`sim/results_v2.md`, "cannot tell us"), which is why 3.5's post-heal proposal strikes the equivocators' weight retroactively for the re-evaluation. ## 3.7 Residual risks, stated @@ -167,10 +167,10 @@ Status of this section: Implemented in `vendor/igneum-node` (reading guide in `d | W1 | `vote_key_hash` = BLAKE2b (domain `IgneumVoteKeyHash`) of the 48-byte compressed G1 key. The key is revealed with a proof of possession in the miner's coinbase extra data (`IGNK` plus 288 hex characters) and a node registers it only when the hash matches the header. A vote carries the public key too, so a key that votes is revealed by its vote | The reveal is hex, not binary, because the template RPC carries extra data as a UTF-8 string. Mainnet should carry the reveal in a dedicated field or transaction | | W2 | Blue blocks per key in `(daa(C) - window, daa(C)]`, counted along C's selected chain through every chain block's mergeset blues, C included | O(window) per checkpoint: fine at the devnet window of 7,200 DAA seconds, not at 2,592,000. Mainnet needs an incremental window kept per chain block | | W3, W5 | Dust excludes a key from the voter list and from both denominators | Key succession (W5) is not implemented | -| C1 | Checkpoint i is the lowest selected-chain block with blue score at least 30 i (blue scores along the chain can skip values), determined when the sink's blue score reaches 30 i + d, d = 20 on devnet. A determination is never revisited | d = 20 is below the placeholder 60; the devnet reorg-depth distribution that sets d has not been recorded | +| C1 | Checkpoint i is the lowest selected-chain block with blue score at least 30 i (blue scores along the chain can skip values), determined when the sink's blue score reaches 30 i + d, d = 20 on devnet. Re-determination (branch `fud-consensus`, 4 October 2026 night, ledger F24): after every virtual change, every unlocked record whose block is no longer a chain ancestor of the sink is determined again on the new chain (`on_virtual_changed`, "re-determined" log line); the certificate held over the old block is dropped, the fold clock restarts, and the certificates kept pending over the new block (`pending_certificates`, at most 4 per index, indices up to 64 ahead of the next determination) are verified. A locked record is never revisited. Unit test `reorg_past_an_unlocked_checkpoint_re_determines_it_and_verifies_the_pending_certificate` (a 6-block side chain's certificate is pending with no conflict, the 15-block side chain overtakes, index 13 is re-determined and locks from it; a block with the wrong blue score is refused) and `a_locked_checkpoint_pins_the_chain_and_a_certificate_against_it_conflicts` (a side chain twice as long does not become the sink past a lock, the certificate against the lock is the one conflict) | d = 20 is below the placeholder 60; the devnet reorg-depth distribution that sets d has not been recorded. Measured in `docs/bench-log.md`, "round-4 consensus items" (reorg run) | | C2 | BLS signature over `"igneum-vote-v1/" \|\| chain_id \|\| 0 \|\| index \|\| hash(C_i)` under `IGNEUM_VOTE_V1_BLS12381G2_XMD:SHA-256_SSWU_RO_NUL_`; the chain id is the prefixed network name (`igneum-devnet`, `igneum-devnet-7`); votes are p2p message 70 and ride in the coinbase extra data of every block | | | C3 | Certificate = index, checkpoint, voter count, signer bitmap over the canonical voter list (keys above dust and not stripped, sorted by key hash), aggregate signature, aggregator key hash and sortition proof. Every template carries the certificates not yet in its past | The validity rule (a block whose selected chain misses a certified checkpoint is invalid) is NOT enforced; only fork choice (F1, F2) is | -| C4 | A second certificate at an index for another block is kept and logged (`conflicting_certificates`) | Not published as evidence. The rule is now fixed by 3.11 item 4 (the node keeps the certificate it verified first, never re-evaluates it, and reports the conflict); the node does not yet clear `finality_active` or expose `finality_conflict` when the pair appears | +| C4 | A certificate at an index for a block other than the one LOCKED there is kept and logged as CONFLICTING (`conflicting_certificates`); at an unlocked index it is held pending (F24 above), not logged as a conflict | Not published as evidence. The rule is now fixed by 3.11 item 4 (the node keeps the certificate it verified first, never re-evaluates it, and reports the conflict); the node does not yet clear `finality_active` or expose `finality_conflict` when the pair appears. Until 4 October 2026 night a reorg deeper than d made the node log every certificate at the moved index as CONFLICTING (ledger F24) | | C5, 3.8 | `min_daa` = `weight_window` (2,592,000 DAA s on mainnet, 7,200 on devnet; a unit test pins the equality). `evaluate` never locks, and `ingest_certificate` refuses a certificate from any source, while the checkpoint's DAA score is below `min_daa`; the node logs "finality not active, window filling, N of M" at every determination until the sink's DAA score reaches `min_daa` and reports the same through `getFinalityCheckpoints` (`finality_reason`, `window_filled_daa`, `window_full_daa`). Unit test `processes::finality::tests::no_certificate_while_the_window_is_filling`: one key holding 100% of the weight signs every checkpoint of a 150-block chain at a 60-DAA window; nothing certifies below DAA 60, a hand-built certificate at an early index is refused, every checkpoint from DAA 60 locks (fin-fixes, 4 October 2026) | Implemented on 3.8's recommendation ahead of the launch-month simulation (O-3.1), which is still not run; gate 3 can lower the gate but not remove it without reopening ledger F1. The sink's DAA score the report compares is the one the virtual processor last handed the manager, so a restarted node reports the window as filling until its first virtual resolution | | Q1, Q2 | Presence window 20 indices on devnet (240 mainnet). Block reading: participation counts the indices in `[i - P, i - 1]` at which a vote by the key is carried by any block, blue or red, in the past of C_i; a key whose first block in the window is younger than P x 30 DAA seconds counts the full window; every template carries up to 48 votes not already in its past, certificates and evidence first | The per-block vote bound (48) is the devnet value of O-3.3. Participation is credited for any vote by the key at the index, whatever block it names; 3.11.1 requires the vote to name the checkpoint on the crediting chain, else a key can stay in the active denominator by voting for blocks of its own and never add to a certificate (O-3.19) | | Q3 | Integer tests: `3 x signed x P >= 2 x active_num` (active_num = sum of weight x participation count) and `3 x signed >= 2 x total` (was `30 x signed >= 17 x total` until 4 October 2026; `FinalityParams::FLOOR_NUM / FLOOR_DEN` = 2/3 on branch `devnet-v4`, with `quorum_met`, `floor_met` and `locks` as pure functions), both inclusive, both at C_i; bans known at evaluation time are applied to the voter list. Unit test `floor_is_two_thirds_of_total_and_inclusive`: 4 of 6 locks, 3 of 6 does not, 67 of 100 locks, 66 does not, the total test implies the active test for every participation. Measured on the three-node, six-voter network of `docs/bench-log.md`, "finality floor 2/3" (4 October 2026): no lock on either side of a 3/3 split, the 4 side of a 4/2 split locks at exactly two thirds | | @@ -181,7 +181,7 @@ Status of this section: Implemented in `vendor/igneum-node` (reading guide in `d | F1, F2 | In `resolve_virtual` the highest locked checkpoint that is in the future of the depth-based finality point and in the past of some body tip replaces the finality point: tips outside its future are not sink candidates | A lock that no body tip passes through is logged and ignored for that resolution | | F3 | Not implemented: the pruning point and `virtual_finality_point` ignore locks | Must land before any pruning network | | F5 | Not implemented (trusted certificate at start) | | -| 3.6 | A second vote by one key at one index for another block is evidence: the key's weight is zero until `detection DAA + ban` (7,200 DAA seconds on devnet), the evidence is carried in blocks and re-detected from blocks | Node-local detection timestamps the ban with the sink's DAA score; a block-carried evidence uses the carrying block's DAA score | +| 3.6 | A second vote by one key at one index for another block is evidence, carried in blocks (`EvidenceRecord`: the two votes, the carriers with their DAA scores). Branch `fud-consensus` (4 October 2026 night, ledger F23): the ban at checkpoint C is computed from C's own past (`bans_at`): the key is stripped at C when a carrier lies in C's past and `daa(C) < daa(lowest carrier) + ban` (7,200 DAA seconds on devnet); detection over RPC or gossip only puts the evidence into this node's templates ("detected here: carried in this node's next block"). Evidence records are bounded (4,096, the oldest dropped; a record goes once its ban ended two windows below the sink or it was never carried within one ban of being seen; 16 carriers per record). Unit test `ban_is_decided_by_the_carrying_block_so_nodes_agree_on_every_voter_list`: three nodes on one chain, one takes the equivocating vote over RPC, two see it from the carrier block only; the voter list agrees on all three at every checkpoint, the key is a voter before the carrier and after the ban and nowhere in between, and the third node verifies the first two's certificates at every locked index | A carrier the reachability store no longer holds (pruned) counts as in the past of every checkpoint more than the merge depth younger than it. Until 4 October 2026 night the ban was stamped node-locally (ledger F23). Measured in `docs/bench-log.md`, "round-4 consensus items" (ban run) | | 3.9 | `getFinalityCheckpoints` reports `finality_active` (the window is full and a lock exists within the last P indices), the latest lock, and since 4 October 2026 `finality_reason` (`active`, `window filling, N of M` with N the sink's DAA score capped at `min_daa` and M `min_daa`, or `paused`) with `window_filled_daa` and `window_full_daa`; the miner prints a `FINALITY` line whenever the reason changes | `last_certified` as a DAA score is not reported; the conflict reason of 3.11 item 4 (two certificates at one index) is not reported (O-3.17), so a conflict still reads as `active` or `paused` | | 3.11 item 6 (seed source) | The devnet keys the hourly program on the header's own `daa_score` (`epoch_seed`, `docs/review/round-3-2026-10-03.md`, R3.26), not on a checkpoint block | The `seed_source` rule (section 4.3 with the uncertified fallback of 3.11 item 6) is not implemented; nothing on the devnet exercises a seed during a finality pause | | 3.11 test table | The four-miner test network of the bench-log entry is the only measurement on a real DAG: 93 checkpoints, 0 conflicting certificates, one equivocation strip, one 12-checkpoint pause under the floor, one heal | d = 20, presence 20 indices and a 7,200-s window are devnet values; the measured pause and heal are at those values, not the mainnet ones | diff --git a/docs/spec/08-client-security.md b/docs/spec/08-client-security.md index 472ad675..ab7fd711 100644 --- a/docs/spec/08-client-security.md +++ b/docs/spec/08-client-security.md @@ -53,3 +53,10 @@ The two findings it answers. An app that auto-updates on ten thousand machines i | Seed shown and confirmed before mining | required | Decided | | Hardware wallet option | required | Decided | | The permanent line | "Nobody from Igneum will ever ask for your seed." | Decided, verbatim | +| Consensus parameters from the environment | none outside devnet and simnet; the override file refused on mainnet | Implemented on `fud-consensus`, pending rollout (8.7) | +| Params digest in the handshake | BLAKE2b-256 of the effective params; a mismatch is refused | Implemented on `fud-consensus`, pending rollout (8.7, section 2.8) | + +## 8.7 The node's parameters are the network's (4 October 2026 night, ledger G12 and X18) + +1. The official client MUST NOT let an environment variable, a setting or an update change a consensus parameter on mainnet or testnet. The node it wraps ignores `IGNEUM_POW_*` outside devnet and simnet and refuses the override file on mainnet (section 2.8); the app passes an override file only to a devnet or simnet node, and the packaged file (`node_override_params`) exists for the devnet's height switches alone. +2. A node refuses any peer whose consensus params digest differs from its own (section 2.8). The client SHOULD show the digest its node printed at start, so an operator can compare two machines by eye (not built yet); a refused peer is reported by the node's log line, never silently. diff --git a/tools/finality-attacks/fud.mjs b/tools/finality-attacks/fud.mjs new file mode 100644 index 00000000..4ba063c7 --- /dev/null +++ b/tools/finality-attacks/fud.mjs @@ -0,0 +1,232 @@ +// Round-4 consensus items runner (4 October 2026, night): ledger F23 (deterministic equivocation bans), F24 +// (re-determination after a deep reorg) and G12/X18 (the params digest in the handshake) on the fast-time 3-node +// network of v3.mjs. Ports 29400 and up, network igneum-devnet-940, data under /tmp/igneum-fin-fud; the live devnet +// is never touched. +// +// node tools/finality-attacks/fud.mjs digest ban reorg # the three scenarios on the fud-consensus build +// IGNEUMD=... IGNEUM_MINER=... node tools/finality-attacks/fud.mjs ban # another build (the control: finality-fixes) +// DELAY_MS=100 BPS=1 node tools/finality-attacks/fud.mjs ... # one-way delay per proxied link, total block rate +// +// Topology (v3.mjs): n1 listens; n0 dials n1 through proxy P0, n2 dials n1 through proxy P2; a proxy adds DELAY_MS +// one way and can be cut and healed. +// +// digest n1 runs the shared override; n0 dials it with a finality block that differs by one DAA second of window +// (another consensus params digest): the handshake must refuse it and n1 must stay without peers; then n2 +// dials with the shared override and connects. Under the old build the mismatched n0 connects. +// ban six voters, two per node, equal shares; voter a0 (on n0) equivocates once at index EQ_INDEX. P2 is cut +// just before that index is determined and healed 45 s later, so n2 sees the evidence late, from the block +// that carries it, while n0 saw it over RPC and n1 from the block at once. Through the ban's expiry every +// node must build or accept certificates over the same voter count at every index: no "names N voters" +// refusal, no conflicting certificate, no locked index disagreeing, the stripped index range identical. +// reorg 4/2 keys with the 4 side (n1, n2) at 70% of the weight; P0 is cut for SPLIT s so n0 determines at least +// one checkpoint on its own chain, then healed: n0 must re-determine those indices on the majority chain +// and lock them from the network's certificates, with no CONFLICTING line and no index locked on two +// different blocks across the nodes. Under the old build n0 logs CONFLICTING for each such index. + +const NODE_ROOT = process.env.IGNEUM_NODE_ROOT || '/Users/joshm/Projects/igneum/'; +process.env.IGNEUM_FIN_BASE_PORT ||= '29400'; +process.env.IGNEUM_FIN_SUFFIX ||= '940'; +process.env.IGNEUM_FIN_TMP ||= '/tmp/igneum-fin-fud'; +process.env.IGNEUM_FAST_TIME ||= '1'; +process.env.IGNEUMD ||= `${NODE_ROOT}vendor/igneum-node-fud/target/release/igneumd`; +process.env.IGNEUM_MINER ||= `${NODE_ROOT}vendor/igneum-node-fud/target/release/igneum-miner`; +const DELAY_MS = +(process.env.DELAY_MS || 100); +const BPS = +(process.env.BPS || 1); +const EQ_INDEX = +(process.env.EQ_INDEX || 9); +const WARM = +(process.env.WARM || 230), SPLIT = +(process.env.SPLIT || 180), HEAL = +(process.env.HEAL || 150); +const BAN_CUT = +(process.env.BAN_CUT || 45), BAN_RUN = +(process.env.BAN_RUN || 480); +const TAG = process.env.TAG || 'fud'; +// rule v3 from checkpoint DAA 0 on every node (the fast-time file says never) +process.env.IGNEUM_FIN_OVERRIDE_JSON ||= JSON.stringify({ finality_v3_activation_daa: 0 }); + +const { Node, Miner, Proxy, stopAll, sleep, log, assertBinaries, TMP, IGNEUMD } = await import('./lib/net.mjs'); +const { mkdirSync, writeFileSync, appendFileSync } = await import('node:fs'); +mkdirSync(TMP, { recursive: true }); +const results = []; +const out = (line) => { console.log(line); appendFileSync(`${TMP}/results-${TAG}.md`, line + '\n'); }; + +const lockedMap = (cp) => new Map((cp?.checkpoints || []).filter(c => c.state === 'locked').map(c => [c.index, c.hash])); +const maxLocked = (cp) => Math.max(0, ...lockedMap(cp).keys()); +async function checkpoints(node, last = 800) { return node.rpc.call('getFinalityCheckpoints', { last }).catch(() => null); } +async function peers(node) { const r = await node.rpc.call('getConnectedPeerInfo', {}).catch(() => null); return (r?.peerInfo || r?.infos || []).length; } + +// per index, the voter count every certificate line on a node names (built / received / replaced / folded) +function voterCounts(node) { + const m = new Map(); + for (const l of node.grepLog(/Finality: certificate/)) { + let x; + if ((x = l.match(/certificate built for checkpoint (\d+) .* by (\d+) of (\d+) voters/))) m.set(+x[1], +x[3]); + else if ((x = l.match(/certificate at index (\d+) received: (\d+) of (\d+) voters/))) m.set(+x[1], +x[3]); + else if ((x = l.match(/certificate at index (\d+) replaced by a heavier one: (\d+) of (\d+) voters/))) m.set(+x[1], +x[3]); + else if ((x = l.match(/certificate for checkpoint (\d+) folded: (\d+) of (\d+) voters/))) m.set(+x[1], +x[3]); + } + return m; +} +function disagreeing(cps) { + const maps = cps.map(lockedMap); + const all = new Set(maps.flatMap(m => [...m.keys()])); + let n = 0; + for (const k of all) { const hs = new Set(maps.filter(m => m.has(k)).map(m => m.get(k))); if (hs.size > 1) n++; } + return n; +} +const count = (node, re) => node.grepLog(re).length; + +async function network() { + const n1 = await new Node(1).start(); + const p0 = await new Proxy(0, n1.p2pPort, { delayMs: DELAY_MS }).start(); + const p2 = await new Proxy(1, n1.p2pPort, { delayMs: DELAY_MS }).start(); + const n0 = await new Node(0, { connect: [p0.addr] }).start(); + const n2 = await new Node(2, { connect: [p2.addr] }).start(); + return { n0, n1, n2, p0, p2 }; +} + +async function digest() { + const name = `digest-${TAG}`; + const n1 = await new Node(1).start(); + // n0: the same file but one DAA second more of weight window: another digest + const n0 = await new Node(0, { connect: [n1.p2p], override: { finality: { weight_window: 121 } } }).start(); + await sleep(25000); + const refusedOn0 = count(n0, /consensus params digest mismatch/), refusedOn1 = count(n1, /consensus params digest mismatch/); + const peers1 = await peers(n1), peers0 = await peers(n0); + const digestLine = (n) => (n.grepLog(/Consensus params digest:/)[0] || '').replace(/^.*digest: /, '').split(' ')[0]; + // n2: the shared file, connects + const n2 = await new Node(2, { connect: [n1.p2p] }).start(); + let connected = null; + for (let i = 0; i < 25; i++) { await sleep(1000); if ((await peers(n2)) > 0) { connected = i + 1; break; } } + const peers1After = await peers(n1); + const refusedOn2 = count(n2, /consensus params digest mismatch/); + await stopAll(); + const pass = refusedOn0 > 0 && refusedOn1 > 0 && peers1 === 0 && peers0 === 0 && connected != null && refusedOn2 === 0; + out(`\n### ${name}: n1 on the shared fast-time override, n0 with finality.weight_window 121 (one DAA second more), n2 on the shared override\n`); + out('| measure | n0 (mismatched) | n1 (listener) | n2 (matching) |'); + out('|---|---|---|---|'); + out(`| params digest printed at start | ${digestLine(n0) || 'none'} | ${digestLine(n1) || 'none'} | ${digestLine(n2) || 'none'} |`); + out(`| "consensus params digest mismatch" lines | ${refusedOn0} | ${refusedOn1} | ${refusedOn2} |`); + out(`| peers after 25 s (n0, n1) and after n2 dialled (n1) | ${peers0} | ${peers1} then ${peers1After} | ${connected == null ? 'not connected in 25 s' : 'connected after ' + connected + ' s'} |`); + const sample = n0.grepLog(/consensus params digest mismatch/)[0]; + if (sample) out(`\nn0's line: \`${sample.replace(/^.*?(Refusing|WARN)/, '$1').slice(0, 300)}\``); + results.push({ name, pass }); +} + +async function ban() { + const name = `ban-${TAG}`; + const { n0, n1, n2, p2 } = await network(); + const miners = []; + for (const [node, label, eq] of [[n0, 'a0', true], [n0, 'a1', false], [n1, 'b0', false], [n1, 'b1', false], [n2, 'c0', false], [n2, 'c1', false]]) + miners.push(new Miner(node, { label, share: 1 / 6, bps: BPS, secs: BAN_RUN, equivocateAt: eq ? EQ_INDEX : undefined }).start()); + const t0 = Date.now(); + // cut n2 off just before index EQ_INDEX is determined on n0 (when EQ_INDEX - 1 is), heal BAN_CUT s later + let cutAt = null, healAt = null, eqSeenAt = null; + while (Date.now() - t0 < BAN_RUN * 1000) { + const cp = await checkpoints(n0, 50); + const t = Math.round((Date.now() - t0) / 1000); + if (cutAt == null && cp && cp.nextIndex >= EQ_INDEX) { p2.cut(); cutAt = t; log(`${name}: P2 cut at ${t} s (n0 next index ${cp.nextIndex})`); } + if (eqSeenAt == null && count(n0, /EQUIVOCATION by key/) > 0) { eqSeenAt = t; log(`${name}: n0 detected the equivocation at ${t} s`); } + if (cutAt != null && healAt == null && t - cutAt >= BAN_CUT) { p2.heal(); healAt = t; log(`${name}: P2 healed at ${t} s`); } + await sleep(1000); + } + const cps = await Promise.all([n0, n1, n2].map(n => checkpoints(n))); + for (const m of miners) await m.stop(); + const nodes = [n0, n1, n2]; + const refusals = nodes.map(n => count(n, /names \d+ voters, this node counts/)); + const conflicts = nodes.map(n => count(n, /CONFLICTING certificate/)); + const equiv = nodes.map(n => count(n, /EQUIVOCATION by key/)); + const carried = nodes.map(n => count(n, /EQUIVOCATION by key .* carried by block/)); + const counts = nodes.map(voterCounts); + const indices = new Set(counts.flatMap(m => [...m.keys()])); + let agree = 0, differ = 0; + const stripped = nodes.map(() => []); + for (const i of [...indices].sort((a, b) => a - b)) { + const vs = counts.map(m => m.get(i)).filter(v => v != null); + if (vs.length >= 2) { if (new Set(vs).size === 1) agree++; else differ++; } + counts.forEach((m, k) => { if (m.get(i) != null && m.get(i) < 6) stripped[k].push(i); }); + } + const range = (xs) => xs.length ? `${xs[0]}..${xs[xs.length - 1]} (${xs.length})` : 'none'; + const locks = cps.map(maxLocked); + const disagree = disagreeing(cps); + await stopAll(); + const sameRange = new Set(stripped.map(range)).size === 1 && stripped[0].length > 0; + const pass = refusals.every(r => r === 0) && conflicts.every(c => c === 0) && disagree === 0 && differ === 0 && sameRange && locks.every(l => l > EQ_INDEX + 3); + out(`\n### ${name}: ${BAN_RUN} s, ${BPS} blocks/s in all, 6 voters, delay ${DELAY_MS} ms, a0 equivocates once at index ${EQ_INDEX}; P2 cut at ${cutAt ?? 'never'} s, healed at ${healAt ?? 'never'} s; n0 detected the equivocation at ${eqSeenAt ?? 'never'} s\n`); + out('| measure | n0 (saw it over RPC) | n1 (from the block at once) | n2 (from the block, after the heal) |'); + out('|---|---|---|---|'); + out(`| EQUIVOCATION lines (of which "carried by block") | ${equiv[0]} (${carried[0]}) | ${equiv[1]} (${carried[1]}) | ${equiv[2]} (${carried[2]}) |`); + out(`| certificates refused "names N voters, this node counts M" | ${refusals.join(' | ')} |`); + out(`| CONFLICTING certificate lines | ${conflicts.join(' | ')} |`); + out(`| indices whose certificates name 5 voters (a0 stripped) | ${stripped.map(range).join(' | ')} |`); + out(`| max locked index at the end | ${locks.join(' | ')} |`); + out(`\nindices with certificate lines on at least two nodes: voter counts agree at ${agree}, differ at ${differ}; locked indices disagreeing across the three nodes: ${disagree}`); + results.push({ name, pass }); +} + +async function reorg() { + const name = `reorg-${TAG}`; + const { n0, n1, n2, p0 } = await network(); + const secs = WARM + SPLIT + HEAL + 30; + const miners = []; + for (const [node, label, share] of [[n0, 'q0', 0.15], [n0, 'q1', 0.15], [n1, 'p0', 0.175], [n1, 'p1', 0.175], [n2, 'p2', 0.175], [n2, 'p3', 0.175]]) + miners.push(new Miner(node, { label, share, bps: BPS, secs }).start()); + await sleep(WARM * 1000); + const before = await Promise.all([n0, n1, n2].map(n => checkpoints(n))); + const beforeMax = before.map(maxLocked); + const preNext = (await checkpoints(n0, 5))?.nextIndex; + log(`${name}: cut at ${WARM} s: max locked ${beforeMax.join('/')}, n0 next index ${preNext}`); + p0.cut(); + await sleep(SPLIT * 1000); + const during = await Promise.all([n0, n1, n2].map(n => checkpoints(n))); + const ownDetermined = (during[0]?.nextIndex ?? 0) - preNext; + const duringMax = during.map(maxLocked); + p0.heal(); + const tHeal = Date.now(); + let reconnected = null; + while (Date.now() - tHeal < HEAL * 1000) { + if (reconnected == null && (await peers(n0)) > 0) reconnected = Math.round((Date.now() - tHeal) / 1000); + await sleep(2000); + } + const after = await Promise.all([n0, n1, n2].map(n => checkpoints(n))); + for (const m of miners) await m.stop(); + const nodes = [n0, n1, n2]; + const redetermined = nodes.map(n => count(n, /re-determined/)); + const conflicts = nodes.map(n => count(n, /CONFLICTING certificate/)); + const pending = nodes.map(n => count(n, /kept pending until the chain decides/)); + const afterMax = after.map(maxLocked); + const disagree = disagreeing(after); + // n0's locks at the indices it determined on its own chain: the same block as n1 holds + const m0 = lockedMap(after[0]), m1 = lockedMap(after[1]); + const splitIdx = [...Array(Math.max(0, ownDetermined)).keys()].map(k => preNext + k); + const agreed = splitIdx.filter(i => m0.has(i) && m1.has(i) && m0.get(i) === m1.get(i)).length; + await stopAll(); + const pass = ownDetermined >= 1 && conflicts.every(c => c === 0) && disagree === 0 && afterMax[0] > duringMax[0] && redetermined[0] >= 1; + out(`\n### ${name}: warm ${WARM} s, split ${SPLIT} s (n0 alone with 30% of the weight), heal window ${HEAL} s, ${BPS} blocks/s in all, delay ${DELAY_MS} ms\n`); + out('| measure | n0 (cut off, 30%) | n1 (70% side) | n2 (70% side) |'); + out('|---|---|---|---|'); + out(`| max locked index at the cut | ${beforeMax.join(' | ')} |`); + out(`| max locked index at the heal | ${duringMax.join(' | ')} |`); + out(`| max locked index at the end | ${afterMax.join(' | ')} |`); + out(`| "re-determined" lines | ${redetermined.join(' | ')} |`); + out(`| certificates kept pending | ${pending.join(' | ')} |`); + out(`| CONFLICTING certificate lines | ${conflicts.join(' | ')} |`); + out(`\nn0 determined ${ownDetermined} checkpoint(s) on its own chain during the split (indices ${splitIdx.join(', ') || 'none'}); after the heal n0 holds the same locked block as n1 at ${agreed} of them; locked indices disagreeing across the three nodes: ${disagree}; n0 reconnected ${reconnected == null ? 'not within the heal window' : reconnected + ' s after the gate reopened'}`); + const lines = n0.grepLog(/re-determined|CONFLICTING/).slice(0, 4); + for (const l of lines) out(` ${l.replace(/^.*?(Finality:)/, '$1').slice(0, 260)}`); + results.push({ name, pass }); +} + +const ALL = { digest, ban, reorg }; +async function main() { + assertBinaries(); + log(`tag ${TAG}; node ${IGNEUMD}; delay ${DELAY_MS} ms; ${BPS} blocks/s; override ${process.env.IGNEUM_FIN_OVERRIDE_JSON}`); + const asked = process.argv.slice(2).filter(a => !a.startsWith('--')); + for (const key of asked.length ? asked : ['digest', 'ban', 'reorg']) { + const fn = ALL[key]; + if (!fn) { log(`unknown scenario ${key}`); continue; } + log(`=== ${key} (${TAG}) starting ===`); + try { await fn(); } catch (e) { log(`${key} threw: ${e.stack || e}`); results.push({ name: key, pass: false }); await stopAll(); } + log(`=== ${key} done ===`); + } + out('\n' + results.map(r => `[${r.pass ? 'PASS' : 'FAIL'}] ${r.name}`).join('\n')); + writeFileSync(`${TMP}/results-${TAG}.json`, JSON.stringify(results, null, 2)); + await stopAll(); + process.exit(results.some(r => !r.pass) ? 1 : 0); +} +main(); diff --git a/tools/finality-attacks/lib/net.mjs b/tools/finality-attacks/lib/net.mjs index 06977388..ac1ddc1f 100644 --- a/tools/finality-attacks/lib/net.mjs +++ b/tools/finality-attacks/lib/net.mjs @@ -37,24 +37,27 @@ const started = []; // everything to stop at exit export const log = (...a) => console.log(new Date().toISOString().slice(11, 23), ...a); export const sleep = (ms) => new Promise(r => setTimeout(r, ms)); -export function overrideParams() { +// extra: a per-node object merged last (the F23/F24/X18 runner gives one node another finality block); name: the +// file's suffix so two nodes never share a file +export function overrideParams(extra = {}, name = '') { mkdirSync(TMP, { recursive: true }); - const file = `${TMP}/override.json`; + const file = `${TMP}/override${name ? '-' + name : ''}.json`; // u64::MAX ("never" for the height switches) is not a JavaScript number: keep it as a BigInt through the merge and // write it back as the integer literal the node's parser wants const big = (k, v, ctx) => (typeof v === 'number' && !Number.isSafeInteger(v) && ctx?.source ? BigInt(ctx.source) : v); const base = FAST_TIME ? JSON.parse(readFileSync(FAST_TIME_FILE, 'utf8'), big) : {}; - const merged = { ...base, skip_proof_of_work: true, ...EXTRA_OVERRIDE }; - if (base.finality && EXTRA_OVERRIDE.finality) merged.finality = { ...base.finality, ...EXTRA_OVERRIDE.finality }; + const merged = { ...base, skip_proof_of_work: true, ...EXTRA_OVERRIDE, ...extra }; + if (base.finality && (EXTRA_OVERRIDE.finality || extra.finality)) merged.finality = { ...base.finality, ...EXTRA_OVERRIDE.finality, ...extra.finality }; const text = JSON.stringify(merged, (k, v) => (typeof v === 'bigint' ? `BIGINT:${v}` : v)).replace(/"BIGINT:(\d+)"/g, '$1'); writeFileSync(file, text); return file; } export class Node { - constructor(index, { connect = [], name } = {}) { + constructor(index, { connect = [], name, override } = {}) { this.index = index; this.name = name || `n${index}`; + this.override = override; // a per-node override object merged over the shared one (see overrideParams) this.grpcPort = BASE_PORT + index * 10; this.p2pPort = BASE_PORT + index * 10 + 1; this.jsonPort = BASE_PORT + index * 10 + 2; @@ -70,7 +73,7 @@ export class Node { const a = ['--devnet', `--devnet-suffix=${DEVNET_SUFFIX}`, '--nodnsseed', '--disable-upnp', '--nologfiles', '--enable-unsynced-mining', '--utxoindex', `--appdir=${this.dir}`, `--rpclisten=127.0.0.1:${this.grpcPort}`, `--rpclisten-json=127.0.0.1:${this.jsonPort}`, - `--listen=127.0.0.1:${this.p2pPort}`, `--override-params-file=${overrideParams()}`, '--loglevel=info', '--yes']; + `--listen=127.0.0.1:${this.p2pPort}`, `--override-params-file=${overrideParams(this.override || {}, this.override ? this.name : '')}`, '--loglevel=info', '--yes']; if (this.connect.length) a.push(`--connect=${this.connect.join(',')}`); else a.push('--outpeers=0'); return a; } @@ -112,6 +115,7 @@ export class Miner { if (o.label) a.push('--label', o.label); if (o.vote === false) a.push('--no-vote'); if (o.equivocate) a.push('--equivocate'); + if (o.equivocateAt != null) a.push('--equivocate-at', String(o.equivocateAt)); if (o.dropVotes) a.push('--drop-votes'); if (o.sybil) a.push('--sybil', o.sybil); if (o.pulse) a.push('--pulse', o.pulse);