igneum-network/igneum
1
0
Fork 0
Code Issues Pull requests Releases Activity

Ledger close round 2: X14 signing concentration from block payloads; X5 observer columns on a branch (O-X.1)

Browse source 
X14: the signing half. No RPC exposes a certificate's signer set, so
tools/finality-attacks/x14-concentration.mjs now walks the selected
chain over the window, decodes the coinbase finality section (IGNF
trailer: votes, certificates, evidence, the IGNK reveal), rebuilds the
canonical voter list at every checkpoint from headers the way the node's
compute_weights does, and maps every bitmap through it. Read-only on the
Mac observer node under the run lock, node version and DAA recorded,
two readings kept (the window straddles the 0.3.10 restart). Result at
23:00:44 UTC, DAA 138,542: signed weight over the heaviest certificate
per index, 27 keys, top-1 10.0%, top-3 29.1%, top-10 77.3%; hashing
6.4/19.2/60.1, aggregation 44.9/84.1/100, proving 100/100/100. Checks:
240 of 240 rebuilt voter lists equal the node's count, 194 of 194
certificates mapped, 27 reveals against BLAKE2b with 0 mismatches.
Status moved to Answered with evidence for all four; the old status
kept after "Was:". Bench-log entry appended.

X5 (paragraph only; Status stays Decision owner: the project lead): the observer
columns of O-X.1 on this branch, not deployed, the running observer
untouched: live_peer_asn (offline prefix table, no third-party lookup),
live_key_machines (machine fingerprint per vote key from the log
intake), live_pool_statements (signed JSON {pool, keys[], signed_at},
Ed25519, parser and verifier), live_concentration (nightly top-1/3/10
for the four concentrations plus N_ind labelled "proposed definition").
Pure functions in tools/observer/lib/concentration.mjs and
lib/nightly.mjs; keyed BLAKE2b in tools/finality-attacks/lib/blake2b.mjs;
9 node:test tests, all passing.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
igneum-labs 2026-10-05 23:07:00 +00:00
parent a4b3a5d980
commit 223d073fa9
12 changed files with 1022 additions and 8 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

46
docs/bench-log.md
View file

@ -1717,3 +1717,49 @@ Owed: the tampered pack against the real worker on PC 2's RTX 5090 (the job tool
**Not run.** The X20 fast-time simnet timing (a simnet chain is a few hundred blocks; the unit test's step count is the evidence, the 10^6-block chain is still the owed experiment) and the X19 two-node skew run (needs an `attack-switches` build of the node).
## 5 October 2026 (night), ledger close round 2: X14 signing concentration from block payloads
Machine: the Mac observer node (`vendor/igneum-node-0310/target-integration/release/igneumd`, fork commit 21d4c73c, `getInfo.serverVersion` 2.1.0, appdir `observer-v4`, wRPC `ws://127.0.0.1:28640`, exec RPC `http://127.0.0.1:26790`), read-only, nothing submitted. The node was restarted onto the 0.3.10 build at 21:49:38 UTC (`ps -o lstart`), about DAA 134,276 (approximate: the DAA at the reading minus the process age at one DAA per second), so the window below straddles that restart and every reading carries a second row from DAA 134,300. Command: `tools/lock/with-lock.sh run node tools/finality-attacks/x14-concentration.mjs --node-build "vendor/igneum-node-0310 21d4c73c target-integration, observer-v4 restarted 22:49:38 local" --since-daa 134300` (the round-1 script extended; the chain walk, the voter-table rebuild and the payload decoder are in `tools/observer/lib/concentration.mjs`, the key hash in `tools/finality-attacks/lib/blake2b.mjs`), run slot run-1, 10 s. Fleet at the reading (`node tools/console.mjs machines`, read-only, 23:03 UTC): Mac app 0.3.10 on node 2.1.0-a24ab01a (its own node, not the observer's), PC 1 the same and stopped 33 min before, PC 2 app 0.3.10 on node 2.1.0 (the 0.3.10 build, its app restarted inside the window: accepted count reset), PC 37ba0461 the same, Sam's Mac app 0.3.9 stopped 2 h before. The rollout was mid-flip; the bytes a block carries are the same whichever node reads them.
**Method.** No RPC exposes a certificate's signer set (round 1). Every block carries its certificates and votes in the coinbase extra data as `items || len_le32 || "IGNF"` (spec 03 C2, C3, Q2; `consensus/core/src/finality.rs` `encode_section`, `Certificate::write`: `index_le64 || checkpoint || voter_count_le32 || bitmap_len_le32 || bitmap || signature || aggregator || proof`; `Vote::write`: `index || checkpoint || pubkey || signature || sortition`, 280 bytes; identical on 0.3.6 and 0.3.10). The bitmap indexes the canonical voter list at the certificate's checkpoint (keys above dust, not stripped, sorted by key hash), which `getFinalityWeights` reports for the latest checkpoint only, so the script rebuilds it at every checkpoint in the window from headers the way `compute_weights` does (the checkpoint plus every chain block's mergeset blues back along the selected chain, counted when `window_start < daa <= daa(C)`, dust 5, sorted): 13,171 chain blocks walked (`getBlock` with transactions, from the sink to DAA 120,542, 1 s), 4,831 non-chain blue headers read, 18,002 blue blocks. Votes carry the public key; the key hash is BLAKE2b-256 keyed `IgneumVoteKeyHash` (`crypto/hashes/src/hashers.rs`), implemented in `lib/blake2b.mjs` and checked against the RFC 7693 vectors and the BLAKE2 keyed KAT. Checks on the live data: 240 of 240 rebuilt voter lists equal the node's `voters` count per checkpoint; 194 of 194 certificates mapped (every `voter_count` equals the rebuilt list's length and every certificate names the node's checkpoint block at its index); 27 `IGNK` reveals (pubkey to header `vote_key_hash`) against BLAKE2b, 0 mismatches; 0 evidence items in the window; 0 items naming a checkpoint outside the window. Only chain blocks are parsed (the task's scope); the node's block reading also credits votes carried by red blocks.
**Reading 2 (the one cited), 23:00:44 UTC, DAA 138,542 at the start and 138,555 at the end, window 7,200 DAA, weights at checkpoint 4,522 (DAA 138,514), 5,731 chain blocks in the window (DAA 131,343 to 138,542).**
| quantity | source | keys | total | top-1 | top-3 | top-10 |
|---|---|---|---|---|---|---|
| hashing (blue blocks per vote key; 27 of 27 above dust 5) | `getFinalityWeights` | 27 | 7,200 blocks | 6.4% | 19.2% | 60.1% |
| signing, heaviest certificate per index (the certificate the lock test reads; 126 indices) | block payloads | 27 | 732,650 signed weight | 10.0% | 29.1% | 77.3% |
| signing, every distinct certificate (194 certificates at 126 indices, carried 250 times) | block payloads | 27 | 1,080,056 | 10.5% | 30.5% | 77.6% |
| signing, every carriage (the same certificates once per carrying block) | block payloads | 27 | 1,443,739 | 10.1% | 29.5% | 76.3% |
| signing, votes carried (4,469 distinct (index, key, block) votes, weighted by the key's weight at C_i; 138 by keys under dust at their checkpoint) | block payloads | 27 | 1,234,377 | 8.4% | 24.4% | 70.0% |
| signing, votes unweighted (votes per key) | block payloads | 27 | 4,469 votes | 4.7% | 13.5% | 42.5% |
| aggregation (certificates built per named aggregator over 132 certified or locked checkpoints; 63 anonymous) | `getFinalityCheckpoints.certificateAggregator` | 10 | 69 certificates | 44.9% | 84.1% | 100% |
| aggregator sortition (keys named eligible, 240 checkpoints) | `getFinalityCheckpoints.aggregators` | 27 | 1,365 eligibilities | 8.7% | 24.9% | 69.7% |
| proving (paid proof records per prover key hash, 5,756 chain blocks from DAA 131,314; 84 carried, 32 rejected) | `igneum_getSegment.proofRecords` | 1 | 52 paid shards | 100% | 100% | 100% |
| signing since DAA 134,300, heaviest per index (88 indices, 149 certificates carried 202 times by 3,323 chain blocks) | block payloads | 26 | 543,555 | 9.9% | 28.7% | 75.9% |
| signing since DAA 134,300, every distinct certificate | block payloads | 26 | 856,271 | 10.5% | 30.8% | 77.4% |
| signing since DAA 134,300, votes weighted (3,018 votes) | block payloads | 27 | 773,597 | 6.9% | 20.3% | 62.3% |
The node's own aggregate over the same 132 locked checkpoints: `signedWeight` over `totalWeight` p50 74.6%, min 66.7%, max 98.3%; `votesSeen` p50 18 of 27 voters. Signed weight is the sum, over the certificates counted, of each signer's blue blocks at that certificate's checkpoint.
Signing per key, heaviest certificate per index (top 12 of 27):
| key (first 8) | signed weight | share | certificates signed of 126 | weight at checkpoint 4,522 |
|---|---|---|---|---|
| a72f7b7c | 73,473 | 10.0% | 126 | 462 |
| ab8a9b21 | 71,886 | 9.8% | 126 | 460 |
| 6dfe55b3 | 67,941 | 9.3% | 126 | 442 |
| ea53bd41 | 67,854 | 9.3% | 126 | 447 |
| 7b8ef6fd | 67,688 | 9.2% | 126 | 435 |
| fe40e510 | 61,005 | 8.3% | 116 | 448 |
| 987175f0 | 57,620 | 7.9% | 104 | 457 |
| c41e4cc2 | 51,610 | 7.0% | 98 | 414 |
| a405c3e6 | 24,800 | 3.4% | 92 | 204 |
| e15fe94b | 22,601 | 3.1% | 87 | 171 |
| b7e3c337 | 20,735 | 2.8% | 87 | 381 |
| 00cec3ae | 20,142 | 2.7% | 85 | 381 |
**Reading 1, 22:58:20 UTC, DAA 138,377 at the start and 138,385 at the end, weights at checkpoint 4,516 (both rows kept by the rollout-night rule).** Hashing 27 keys, 7,199 blocks: 6.5%, 19.5%, 60.8%. Aggregation 12 keys, 76 certificates over 139 certified or locked checkpoints (63 anonymous): 42.1%, 77.6%, 97.4%. Sortition 27 keys, 1,389 eligibilities: 8.7%, 24.7%, 69.8%. Proving 1 key, 52 paid shards: 100%. Signing: 200 distinct certificates at 131 indices carried 256 times by 5,738 chain blocks, 200 mapped; 239 of 239 tables agreed; 4,506 votes; the five share rows printed as NaN (the script passed the per-key map where the shares function expected its summary; fixed, the second reading taken 2 minutes later), the per-key table was right (a72f7b7c 75,531 signed weight, 10.0%, 131 of 131).
**What the numbers mean, and what follows.** Signing is more concentrated than hashing (top-10 77.3% against 60.1%, top-3 29.1% against 19.2%) because a certificate carries a median 18 of 27 voters: five keys signed every one of the 126 heaviest certificates and eight signed 98 or more, and those eight hold 70.8% of the lock weight; the other 19 keys appear in 85 to 92 of 126 and split the rest. Every certificate still locked at or above two thirds of total (min 66.7%), so the missing votes cost nothing tonight; on a chain where the eight were one party, that party would be 71% of every lock. The phase 5 gate ("top-10 share of window weight under 50%", `site/journey.json`) fails tonight on all four measures (hashing 60.1%, signing 77.3%, aggregation 100%, proving 100%), as a devnet of 27 keys on 5 machines (3 live at the reading; 5.4 to 9 keys per machine, approximate from the console) must; the gate is a public-testnet test, and the columns to read it from stored data are on branch `ledger-observer` (ledger X5, round 2; not deployed). For the operator page a `getFinalityCertificate(index)` RPC that returns the bitmap with the voter list at C_i would make this a few calls instead of a 13,171-block walk; the walk takes 1 s on the devnet window and would take about 6 min of RPC at the mainnet window (2,592,000 blocks at the same rate, approximate), so the RPC is the mainnet form. Raw output: `/tmp/igneum-x14-results.md` (overwritten by each run).

6
docs/fud-ledger.md
View file

@ -837,6 +837,8 @@ Evidence: `site/journey.json` phase 5 gate.
Cross-reference (external review, 3 October 2026, night): the definition and the four concentration metrics are X14, O-X.1.
Round 2 (5 October 2026, night), the observer columns (O-X.1), built on branch `ledger-observer` and NOT deployed; the running observer is untouched and the definition stays this decision (item 3). `tools/observer/observer.mjs` on that branch adds four tables and three timers (`tools/observer/README.md`, "Independence columns and the nightly table"): (a) `live_peer_asn`, the autonomous system per announcing peer address from `getConnectedPeerInfo` against an offline prefix table (`tools/observer/asn-table.txt`, empty tonight, to be filled from a dated BGP dump: RIPEstat, Team Cymru bulk whois or a pyasn dump of RouteViews; no third party is asked at run time; private ranges read `local`, a miss stays `source = 'none'`); (b) `live_key_machines`, the machine fingerprint per vote key from the log intake (`miner_logs.machine`, the id8 every STATUS line's run id carries, joined to the miner's `identity N 'label' vote_key_hash=...` line in the same upload); (c) `live_pool_statements`, the pool statement format: a signed JSON `{format: "igneum-pool-statement-1", pool, keys[], signed_at, pubkey, sig}`, Ed25519 over canonical bytes, verified against `pool-statements/registry.json` (pool label to key; empty tonight), refused when signed by another key, older than 35 days, with repeated keys or a malformed shape, and stored with the reason when it fails; (d) `live_concentration`, one row a day at 00:05 UTC with top-1, top-3 and top-10 shares for hashing (`getFinalityWeights`), signing (the stored `live_certificates` bitmaps over their voter tables), proving (`live_proofs` paid shards per prover) and aggregation (`certificateAggregator` over the window's checkpoints), plus `n_ind` with `n_ind_definition = 'proposed'`: distinct (ASN, fingerprint, pool) classes among keys above dust, the silent-key rule of the decision request applied (a key with no fingerprint is its own class only when its ASN is used by no other key), and `n_ind_unattributed` counting keys with no attribute at all. Pure functions in `tools/observer/lib/concentration.mjs` and `lib/nightly.mjs`; 9 unit tests under node's test runner (`node --test 'tools/observer/test/*.test.mjs'`: the payload decoder against hand-built fixtures, the voter-table rebuild rule, shares, N_ind on the 21-keys-5-machines reading of this entry (5) and the silent-fleet rule, the pool statement's signing and every refusal, the ASN table's longest prefix, the nightly row), all passing on the Mac. Open: a block names its producer by vote key and a peer announces an address, and nothing ties the two, so the ASN per key is null until the app reports its public address with its uploads (an app-owner item); the ASN table and the pool registry are empty until filled by hand. Tonight's reading from X14's round 2 (23:00 UTC, DAA 138,542): 27 keys above dust over 5 machines in the window, 3 of them live at the reading, so N_ind by fingerprint is at most 5 (approximate, from the console; the fingerprint column will give the exact figure once deployed) and the top-10 share of window weight is 60.1%, against the gate's "under 50%".
### X6. The one-click app is a honeypot vector
"An installer that creates a wallet, holds keys and mines, promoted to people who have never run a miner. Fake copies will be the first Google result. Defender flags every miner as malware."
@ -1559,7 +1561,7 @@ Evidence: `site/journey.json` phases 4 and 5; `docs/commercial/prover-customer-b
### X14. Concentration is unmeasured in four places
"Define independent for the 1,000-miner gate, then report concentration in hashing, checkpoint signing, proving and aggregation, because a thousand miners behind two pools is two."
Status: Answered with evidence for hashing, aggregation and proving, Open for signing (5 October 2026, night, ledger close round 1: no RPC exposes a certificate's signer set, so the signing concentration needs the observer extract of O-X.1; bench-log "5 October 2026 (night), ledger close round 1: X14 concentration in hashing, signing, proving and aggregation from the Mac node's RPCs, and E16 one live block"); the independence definition stays the project lead's (X5). Was: Open, measurement scheduled (O-X.1); extends X5. Sweep (5 October 2026): hashing concentration computed from the devnet record (`sim/difficulty/records/live-2026-10-04.csv`, 8,090 blocks, 18 vote keys): top-1 12.8%, top-3 34.5%, top-9 98.0%, the nine being devnet v4's nine identities run by three machines (approximate), so by machine the devnet is three parties. Signing, proving and aggregation concentration need certificate and proof-record extracts the observer does not keep yet (O-X.1).
Status: Answered with evidence for all four (5 October 2026, night, ledger close round 2: signing concentration read from the certificates and votes the chain blocks carry in their coinbase finality section, top-1 10.0%, top-3 29.1%, top-10 77.3% of signed weight over the heaviest certificate at each of 126 indices in the window; bench-log "5 October 2026 (night), ledger close round 2: X14 signing concentration from block payloads"); the independence definition stays the project lead's (X5). Was: Answered with evidence for hashing, aggregation and proving, Open for signing (5 October 2026, night, ledger close round 1: no RPC exposes a certificate's signer set, so the signing concentration needs the observer extract of O-X.1; bench-log "5 October 2026 (night), ledger close round 1: X14 concentration in hashing, signing, proving and aggregation from the Mac node's RPCs, and E16 one live block"); the independence definition stays the project lead's (X5). Was: Open, measurement scheduled (O-X.1); extends X5. Sweep (5 October 2026): hashing concentration computed from the devnet record (`sim/difficulty/records/live-2026-10-04.csv`, 8,090 blocks, 18 vote keys): top-1 12.8%, top-3 34.5%, top-9 98.0%, the nine being devnet v4's nine identities run by three machines (approximate), so by machine the devnet is three parties. Signing, proving and aggregation concentration need certificate and proof-record extracts the observer does not keep yet (O-X.1).
Answer: Correct. X5 conceded that "independent" needs a measurable definition (distinct ASNs, benchmark hardware fingerprints, pool attestations) and left it to phase 4. The four concentrations can each be computed from chain data: blue blocks per vote key and per pool (hashing), signed weight per key in certificates (checkpoint signing), proof records per prover key (proving, once P12's fix puts the key in the statement), and certificates and proof records per aggregator key (aggregation). The gate reports all four as top-1, top-3 and top-10 shares over 30 days, beside the independence count, on the live page. Transaction choice inside pools is a separate measurement and is already scheduled: whether members of the reference pool use declared templates (spec 9.4.2, mode C) is recorded under O-9.5.
@ -1567,6 +1569,8 @@ Evidence: X5, F10, P12, spec 9.4.2. Experiment: O-X.1. Review: external, point 6
Run (5 October 2026, night): `tools/lock/with-lock.sh run node tools/finality-attacks/x14-concentration.mjs` (new; the Mac observer node's wRPC and exec RPC, read-only, 7 s) at DAA 125,005, window 7,200 DAA, plus `node tools/console.mjs machines`. Hashing (`getFinalityWeights`, 29 keys, 25 above dust, 6,734 blocks): top-1 25.9%, top-3 37.3%, top-10 67.0%. Aggregation (`certificateAggregator` over 210 certified or locked checkpoints, 187 named, 23 anonymous, 20 keys): top-1 17.1%, top-3 43.3%, top-10 85.0%; sortition eligibility (1,492 namings over 225 checkpoints, 27 keys): top-1 7.1%, top-3 20.4%, top-10 61.9%. Proving (`igneum_getSegment.proofRecords` over 4,474 chain blocks, 275 records carried, 131 paid, 144 rejected): one key and one payout address hold 100% of the paid shards (PC 2's prover; 519 shards paid on the chain in all). Signing: NOT exposed; `RpcCheckpoint` carries `signedWeight`, `votesSeen`, `voters`, `aggregators` and `certificateAggregator` and no signer set or bitmap, so per-key signing concentration cannot be computed from any RPC on this line; what is exposed over the 210 locked checkpoints is `signedWeight` over `totalWeight` p50 71.1%, min 38.8%, max 100% (a locked checkpoint at 38.8% shows the two fields are not the pair the lock test used) and `votesSeen` p50 16 of 25. Key-to-machine ratio: 29 keys against 4 machines on the console at 19:14 UTC (5.8 to 7.3 keys per machine, depending on whether the evening's fifth machine is counted). Bench-log heading: "5 October 2026 (night), ledger close round 1: X14 concentration in hashing, signing, proving and aggregation from the Mac node's RPCs, and E16 one live block".
Round 2 (5 October 2026, night): the signing half, from block payloads. No RPC exposes a certificate's signer set, but every block carries its certificates and votes in the coinbase extra data (spec 03 C2, C3, Q2: `items || len_le32 || "IGNF"` at the end of the payload; `consensus/core/src/finality.rs` `encode_section`, `Certificate::write`, `Vote::write`, the same bytes on 0.3.6 and 0.3.10), and a certificate's bitmap indexes the canonical voter list at its checkpoint. `getFinalityWeights` reports that list for the latest checkpoint only, so `tools/finality-attacks/x14-concentration.mjs` (extended, not a second script) rebuilds the list at every checkpoint in the window from headers the way the node does (`compute_weights`: the checkpoint plus every chain block's mergeset blues back along the selected chain, `window_start < daa <= daa(C)`, dust, sorted by key hash) and maps every bitmap through it; votes carry the public key and the key hash is BLAKE2b-256 keyed `IgneumVoteKeyHash` (`lib/blake2b.mjs`, RFC vectors plus every `IGNK` reveal in the window as the check). Run: `tools/lock/with-lock.sh run node tools/finality-attacks/x14-concentration.mjs --node-build ... --since-daa 134300`, read-only, 10 s, at 23:00:44 UTC on the Mac observer node (`vendor/igneum-node-0310` 21d4c73c, `target-integration/release/igneumd`, `serverVersion` 2.1.0, restarted for 0.3.10 at 21:49:38 UTC, about DAA 134,276, so the window straddles that restart and the reading carries a second row from DAA 134,300), DAA 138,542 at the start and 138,555 at the end, window 7,200 DAA, weights at checkpoint 4,522. Fleet at the reading (console, read-only): Mac and PC 1 apps on node 2.1.0-a24ab01a (PC 1 stopped 33 min before), PC 2 and PC 37ba0461 on node 2.1.0 (the 0.3.10 build), Sam's Mac stopped 2 h before; the chain bytes are the same whichever node reads them. Checks: 240 of 240 rebuilt voter lists agree with the node's `voters` count, 194 of 194 certificates mapped (every `voter_count` equals the rebuilt list), 27 reveals against BLAKE2b with 0 mismatches, 0 evidence items, 0 items naming a checkpoint outside the window. Result, signed weight per key over the heaviest certificate at each of 126 indices (the certificate the lock test reads; 194 distinct certificates carried 250 times by 5,731 chain blocks): 27 keys, top-1 10.0%, top-3 29.1%, top-10 77.3%; over every distinct certificate 10.5%, 30.5%, 77.6%; over every carriage 10.1%, 29.5%, 76.3%; votes carried (4,469 distinct, weighted by the key's weight at C_i) 8.4%, 24.4%, 70.0%, unweighted 4.7%, 13.5%, 42.5%. Beside the other three at the same reading: hashing (27 keys, 7,200 blocks) 6.4%, 19.2%, 60.1%; aggregation (10 named keys, 69 certificates over 132 certified or locked checkpoints, 63 anonymous) 44.9%, 84.1%, 100%; proving (52 paid shards) 100%, 100%, 100%. Since DAA 134,300 (after the observer node's restart, 88 indices): 9.9%, 28.7%, 75.9%. Signing is more concentrated than hashing (top-10 77.3% against 60.1%) because the node sees a median 18 of 27 voters per checkpoint: five keys signed all 126 certificates and eight signed 98 or more, and those eight hold 70.8% of the lock weight (per-key table in the bench-log); the other 19 keys sign 85 to 92 of 126 and carry the rest. What it means: the phase 5 gate ("top-10 share of window weight under 50%") fails tonight on all four measures, as a four-live-machine devnet must (27 keys against 3 machines live at the reading, 5 over the window: 5.4 to 9 keys per machine, approximate from the console); the gate is a public-testnet test and the observer now has the columns to read it (X5, round 2). A `getFinalityCertificate(index)` RPC returning the bitmap and the voter list at C_i would make the reading a few calls instead of a 13,171-block walk; not needed for the ledger, noted for the operator page. An earlier reading at 22:58:20 UTC (DAA 138,377) gave hashing 6.5%, 19.5%, 60.8% and aggregation 42.1%, 77.6%, 97.4% (12 keys, 76 certificates); its signing rows were unreadable (a formatting fault in the script, fixed before the second reading) and both readings are kept in the bench-log.
### X15. Remove the founders from a test network and show what continues
"'The chain runs without its founders' is a sentence. Take the team's miners, provers, aggregators, seed nodes, observer and site off a running testnet and show what keeps producing blocks, proofs and locks."

86
tools/finality-attacks/lib/blake2b.mjs Normal file
View file

@ -0,0 +1,86 @@
// BLAKE2b (RFC 7693), keyed and with a chosen output length, in plain Node 22 (BigInt words, no dependencies).
// Needed because the fork's `vote_key_hash` is BLAKE2b-256 keyed with the domain string `IgneumVoteKeyHash`
// (vendor/igneum-node-036/crypto/hashes/src/hashers.rs, the `blake2b_hasher!` macro: hash_length 32, key = the
// domain) over the 48-byte compressed G1 public key, and Node's crypto exposes neither the key nor a 32-byte length.
// Checked against the RFC's unkeyed test vector in `selfTest()` and, in x14-concentration.mjs, against every key
// reveal the chain carries (reveal pubkey -> header vote_key_hash).
const IV = [
0x6a09e667f3bcc908n, 0xbb67ae8584caa73bn, 0x3c6ef372fe94f82bn, 0xa54ff53a5f1d36f1n,
0x510e527fade682d1n, 0x9b05688c2b3e6c1fn, 0x1f83d9abfb41bd6bn, 0x5be0cd19137e2179n,
];
const SIGMA = [
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15],
[14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3],
[11, 8, 12, 0, 5, 2, 15, 13, 10, 14, 3, 6, 7, 1, 9, 4],
[7, 9, 3, 1, 13, 12, 11, 14, 2, 6, 5, 10, 4, 0, 15, 8],
[9, 0, 5, 7, 2, 4, 10, 15, 14, 1, 11, 12, 6, 8, 3, 13],
[2, 12, 6, 10, 0, 11, 8, 3, 4, 13, 7, 5, 15, 14, 1, 9],
[12, 5, 1, 15, 14, 13, 4, 10, 0, 7, 6, 3, 9, 2, 8, 11],
[13, 11, 7, 14, 12, 1, 3, 9, 5, 0, 15, 4, 8, 6, 2, 10],
[6, 15, 14, 9, 11, 3, 0, 8, 12, 2, 13, 7, 1, 4, 10, 5],
[10, 2, 8, 4, 7, 6, 1, 5, 15, 11, 9, 14, 3, 12, 13, 0],
];
const M64 = (1n << 64n) - 1n;
const rotr = (x, n) => ((x >> BigInt(n)) | (x << BigInt(64 - n))) & M64;
function compress(h, block, t, last) {
const m = new Array(16);
for (let i = 0; i < 16; i++) m[i] = block.readBigUInt64LE(i * 8);
const v = h.concat(IV);
v[12] ^= BigInt(t) & M64;
v[13] ^= (BigInt(t) >> 64n) & M64;
if (last) v[14] ^= M64;
const G = (a, b, c, d, x, y) => {
v[a] = (v[a] + v[b] + x) & M64; v[d] = rotr(v[d] ^ v[a], 32);
v[c] = (v[c] + v[d]) & M64; v[b] = rotr(v[b] ^ v[c], 24);
v[a] = (v[a] + v[b] + y) & M64; v[d] = rotr(v[d] ^ v[a], 16);
v[c] = (v[c] + v[d]) & M64; v[b] = rotr(v[b] ^ v[c], 63);
};
for (let r = 0; r < 12; r++) {
const s = SIGMA[r % 10];
G(0, 4, 8, 12, m[s[0]], m[s[1]]); G(1, 5, 9, 13, m[s[2]], m[s[3]]);
G(2, 6, 10, 14, m[s[4]], m[s[5]]); G(3, 7, 11, 15, m[s[6]], m[s[7]]);
G(0, 5, 10, 15, m[s[8]], m[s[9]]); G(1, 6, 11, 12, m[s[10]], m[s[11]]);
G(2, 7, 8, 13, m[s[12]], m[s[13]]); G(3, 4, 9, 14, m[s[14]], m[s[15]]);
}
for (let i = 0; i < 8; i++) h[i] ^= v[i] ^ v[i + 8];
}
/** BLAKE2b of `data` (Buffer or Uint8Array) with an optional key (at most 64 bytes) and output length 1 to 64. */
export function blake2b(data, { key = Buffer.alloc(0), outlen = 64 } = {}) {
if (outlen < 1 || outlen > 64 || key.length > 64) throw new Error('blake2b: bad parameters');
const h = IV.slice();
h[0] ^= BigInt(0x01010000 ^ (key.length << 8) ^ outlen);
const msg = Buffer.from(data);
const blocks = [];
if (key.length) { const kb = Buffer.alloc(128); Buffer.from(key).copy(kb); blocks.push([kb, 128]); }
if (msg.length === 0) { if (!key.length) blocks.push([Buffer.alloc(128), 0]); }
else for (let i = 0; i < msg.length; i += 128) { const b = Buffer.alloc(128); const n = Math.min(128, msg.length - i); msg.copy(b, 0, i, i + n); blocks.push([b, n]); }
let t = 0;
for (let i = 0; i < blocks.length; i++) { t += blocks[i][1]; compress(h, blocks[i][0], t, i === blocks.length - 1); }
const out = Buffer.alloc(64);
for (let i = 0; i < 8; i++) out.writeBigUInt64LE(h[i], i * 8);
return out.subarray(0, outlen);
}
/** The fork's vote key hash: BLAKE2b-256 keyed `IgneumVoteKeyHash` over the 48-byte compressed G1 key (hex in, hex out). */
export function voteKeyHash(pubkeyHex) {
return blake2b(Buffer.from(pubkeyHex, 'hex'), { key: Buffer.from('IgneumVoteKeyHash'), outlen: 32 }).toString('hex');
}
/** RFC 7693 appendix A: BLAKE2b-512("abc"); plus the keyed vector of the BLAKE2 reference tests for key = bytes 0..63. */
export function selfTest() {
const abc = blake2b(Buffer.from('abc')).toString('hex');
const want = 'ba80a53f981c4d0d6a2797b69f12f6e94c212f14685ac4b74b12bb6fdbffa2d17d87c5392aab792dc252d5de4533cc9518d38aa8dbf1925ab92386edd4009923';
if (abc !== want) return { ok: false, detail: `BLAKE2b-512("abc") = ${abc}` };
const empty = blake2b(Buffer.alloc(0)).toString('hex');
const wantEmpty = '786a02f742015903c6c6fd852552d272912f4740e15847618a86e217f71f5419d25e1031afee585313896444934eb04b903a685b1448b755d56f701afe9be2ce';
if (empty !== wantEmpty) return { ok: false, detail: `BLAKE2b-512("") = ${empty}` };
// blake2b-kat.txt (BLAKE2 reference): in = "", key = 00..3f, out = 10ebb67700b1868efb4417987acf4690ae9d972fb7a590c2f02871799aaa4786b5e996e8f0f4eb981fc214b005f42d2ff4233499391653df7aefcbc13fc51568
const key = Buffer.from(Array.from({ length: 64 }, (_, i) => i));
const kat = blake2b(Buffer.alloc(0), { key }).toString('hex');
const wantKat = '10ebb67700b1868efb4417987acf4690ae9d972fb7a590c2f02871799aaa4786b5e996e8f0f4eb981fc214b005f42d2ff4233499391653df7aefcbc13fc51568';
if (kat !== wantKat) return { ok: false, detail: `keyed BLAKE2b-512("") = ${kat}` };
return { ok: true };
}

156
tools/finality-attacks/x14-concentration.mjs
View file

@ -14,12 +14,32 @@
// concentration by paid shards per key hash over the window's chain blocks
// getBlock(hash, true) the coinbase outputs: the 20% pool output with its OP_RETURN tag (E16)
// Never submits anything.
//
// Round 2 (5 October 2026, night, ledger close round 2): the SIGNING half from block payloads. No RPC exposes a
// certificate's signer set, but every block carries its certificates and votes in the coinbase extra data (spec 03
// C2, C3, Q2; `items || len_le32 || "IGNF"` at the end of the payload, consensus/core/src/finality.rs
// encode_section, Certificate::write, Vote::write). A certificate's bitmap indexes the canonical voter list at its
// checkpoint (keys above dust, not stripped, sorted by key hash), and `getFinalityWeights` reports that list for the
// latest checkpoint only, so the list at every checkpoint in the window is rebuilt here from headers exactly as the
// node does it (consensus/src/processes/finality.rs compute_weights: C itself plus every chain block's mergeset
// blues back along the selected chain, counted when `window_start < daa <= daa(C)`, dust applied, sorted). The
// rebuilt list is checked against the node's `voters` count per checkpoint and against every certificate's
// `voter_count`. Votes carry the public key; the key hash is BLAKE2b-256 keyed `IgneumVoteKeyHash` (lib/blake2b.mjs),
// checked against every `IGNK` reveal the window's coinbases carry.
// --node-build <label> the binary behind the RPC (recorded in the heading; the rollout night rule)
// --since-daa <n> a second, narrower reading over chain blocks at or above this DAA score (both rows kept)
// --chain-only skip the round-1 proving and aggregation parts (signing only)
import { connectRpc } from './lib/rpc.mjs';
import { voteKeyHash, selfTest } from './lib/blake2b.mjs';
import { decodeFinalitySection, voterTableAt, signingShares } from '../observer/lib/concentration.mjs';
const args = process.argv.slice(2);
const opt = (n, d) => { const i = args.indexOf(n); return i >= 0 ? args[i + 1] : d; };
const WRPC = opt('--wrpc', 'ws://127.0.0.1:28640');
const EXEC = opt('--exec', 'http://127.0.0.1:26790');
const NODE_BUILD = opt('--node-build', '');
const SINCE_DAA = Number(opt('--since-daa', 0)) || 0;
const CHAIN_ONLY = args.includes('--chain-only');
let id = 0;
async function exec(method, params = []) {
const r = await fetch(EXEC, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ jsonrpc: '2.0', id: ++id, method, params }), signal: AbortSignal.timeout(30000) });
@ -57,13 +77,14 @@ for (const c of inWin) for (const a of c.aggregators || []) eligible.set(String(
const signedFrac = locked.map(c => Number(c.signedWeight) / Number(c.totalWeight));
const votesSeen = locked.map(c => Number(c.votesSeen));
// proving: paid shards per prover key hash over the window's chain blocks
const budgets = await exec('igneum_getBudgets');
let budgets = { chainBlocks: 1 }, tipSeg = { mergeset: [] }, status = { paidShards: 0, pool: null, verifier: null, poolBalanceWei: '0', paidWei: '0' };
if (!CHAIN_ONLY) budgets = await exec('igneum_getBudgets');
const tip = Number(budgets.chainBlocks) - 1;
const tipSeg = await exec('igneum_getSegment', ['latest']);
if (!CHAIN_ONLY) tipSeg = await exec('igneum_getSegment', ['latest']);
const tipDaa = Number(tipSeg.mergeset?.[0]?.daaScore ?? 0);
const proving = new Map(), payoutAddr = new Map();
let paidRecords = 0, rejectedRecords = 0, carried = 0, chainBlocksRead = 0, firstDaa = null, example = null;
for (let n = tip; n >= 1; n--) {
for (let n = CHAIN_ONLY ? 0 : tip; n >= 1; n--) {
let seg; try { seg = await exec('igneum_getSegment', ['0x' + n.toString(16)]); } catch (e) { break; }
const daa = Number(seg.mergeset?.find(m => m.hash === seg.hash)?.daaScore ?? seg.mergeset?.[0]?.daaScore ?? 0);
if (daa && daa < Number(w.daaScore) - window) break;
@ -78,7 +99,7 @@ for (let n = tip; n >= 1; n--) {
} else if (!r.valid) rejectedRecords++;
}
}
const status = await exec('igneum_getProvingStatus');
if (!CHAIN_ONLY) status = await exec('igneum_getProvingStatus');
// E16: the carrying block's coinbase on the UTXO side
let coinbase = null;
if (example) {
@ -86,12 +107,118 @@ if (example) {
const cb = b.block?.transactions?.[0];
coinbase = { hash: example.carrierHash, outputs: (cb?.outputs || []).map(o => ({ amount: o.value ?? o.amount, script: String(o.scriptPublicKey?.scriptPublicKey ?? o.scriptPublicKey?.script ?? JSON.stringify(o.scriptPublicKey)).slice(0, 80) })), txCount: b.block?.transactions?.length, payloadBytes: cb ? String(cb.payload).length / 2 : null };
}
// ---------------------------------------------------------------------------------------------
// Round 2: signing concentration from block payloads
const b2 = selfTest(); if (!b2.ok) throw new Error('blake2b self-test failed: ' + b2.detail);
const info = await rpc.call('getInfo', {});
const sinkDaa = Number(dag.virtualDaaScore);
const winStart = sinkDaa - window; // the window: chain blocks with daa in (winStart, sinkDaa]
const dust = Number(w.params?.dust ?? 5);
const MERGE_DEPTH = 3600; // devnet merge depth bound at 1 bps (params.rs merge_depth_bound)
// checkpoints by index (the node's list reaches back past the window)
const cpByIndex = new Map((cps.checkpoints || []).map(c => [Number(c.index), c]));
const cpInWin = [...cpByIndex.values()].filter(c => Number(c.daaScore) > winStart);
const minCpDaa = cpInWin.length ? Math.min(...cpInWin.map(c => Number(c.daaScore))) : sinkDaa;
const walkStop = minCpDaa - window - MERGE_DEPTH; // the earliest table needs blues back to minCpDaa - window
// 1. the selected chain from the sink back to walkStop, with every chain block's finality items inside the window
const chain = []; // newest first: { hash, daa, key, blues: [hash], items }
const inflight = 8;
{
let hash = dag.sink;
const tWalk = Date.now();
while (hash && hash !== '0'.repeat(64)) {
const needTx = true; // the payload is needed in the window; cheap enough everywhere
let b; try { b = (await rpc.call('getBlock', { hash, includeTransactions: needTx })).block; } catch (e) { break; }
const h = b.header, v = b.verboseData || {};
const daa = Number(h.daaScore);
const items = daa > winStart ? decodeFinalitySection(b.transactions?.[0]?.payload || '') : null;
chain.push({ hash, daa, key: String(h.voteKeyHash), blues: v.mergeSetBluesHashes || [], items, isChain: v.isChainBlock });
if (daa < walkStop) break;
hash = v.selectedParentHash;
}
console.error(`chain walk: ${chain.length} chain blocks from DAA ${chain[0]?.daa} down to ${chain[chain.length - 1]?.daa} in ${Math.round((Date.now() - tWalk) / 1000)} s`);
}
const chainSet = new Map(chain.map(c => [c.hash, c]));
// 2. every blue block with its merger: chain blocks are merged by their child chain block; non-chain blues need a header read
const blues = [];
const need = [];
for (let i = chain.length - 1; i >= 0; i--) {
const c = chain[i];
for (const bh of c.blues) {
const cb = chainSet.get(bh);
if (cb) blues.push({ hash: bh, daaScore: cb.daa, voteKeyHash: cb.key, mergerDaa: c.daa });
else need.push([bh, c.daa]);
}
}
// the sink itself (merged by nobody yet) is counted when it is a checkpoint, like any chain block: give it its own daa
blues.push({ hash: chain[0].hash, daaScore: chain[0].daa, voteKeyHash: chain[0].key, mergerDaa: Number.MAX_SAFE_INTEGER });
{
const tHdr = Date.now();
let i = 0, failed = 0;
await Promise.all(Array.from({ length: inflight }, async () => {
while (i < need.length) {
const [bh, mergerDaa] = need[i++];
try { const b = (await rpc.call('getBlock', { hash: bh, includeTransactions: false })).block; blues.push({ hash: bh, daaScore: Number(b.header.daaScore), voteKeyHash: String(b.header.voteKeyHash), mergerDaa }); }
catch { failed++; }
}
}));
console.error(`non-chain blues: ${need.length} headers read (${failed} failed) in ${Math.round((Date.now() - tHdr) / 1000)} s; ${blues.length} blue blocks in all`);
}
// 3. the voter table at every checkpoint in the window, checked against the node's voters count
const tables = new Map(); let tableAgree = 0, tableDisagree = [];
for (const c of cpInWin) {
const t = voterTableAt({ hash: c.hash, daaScore: Number(c.daaScore) }, blues, { weightWindow: window, dust });
t.hash = c.hash; tables.set(Number(c.index), t);
if (t.voters.length === Number(c.voters)) tableAgree++; else tableDisagree.push(`${c.index}: rebuilt ${t.voters.length}, node ${c.voters}`);
}
// 4. the items carried by chain blocks in the window
const windowChain = chain.filter(c => c.daa > winStart);
const sinceChain = SINCE_DAA ? windowChain.filter(c => c.daa >= SINCE_DAA) : null;
const reveals = new Map(); let revealChecked = 0, revealMismatch = 0;
for (const c of windowChain) { const r = c.items?.reveal; if (r) { reveals.set(r.pubkey, c.key); } }
for (const [pk, kh] of reveals) { revealChecked++; if (voteKeyHash(pk) !== kh) revealMismatch++; }
function tally(blocksList) {
const certInstances = [], certDistinct = new Map(), votesDistinct = new Map(), votesPerKeyCount = new Map(), votesWeight = new Map();
let evidence = 0, outsideCerts = 0, outsideVotes = 0, unknownVoteKey = 0;
for (const c of blocksList) {
const it = c.items; if (!it) continue;
evidence += it.evidence.length;
for (const cert of it.certificates) {
if (!tables.has(cert.index)) { outsideCerts++; continue; }
certInstances.push(cert);
const k = `${cert.index}|${cert.checkpoint}|${cert.bitmap}`;
if (!certDistinct.has(k)) certDistinct.set(k, cert);
}
for (const v of it.votes) {
const t = tables.get(v.index);
if (!t) { outsideVotes++; continue; }
const kh = voteKeyHash(v.pubkey);
const k = `${v.index}|${kh}|${v.checkpoint}`;
if (votesDistinct.has(k)) continue;
votesDistinct.set(k, true);
votesPerKeyCount.set(kh, (votesPerKeyCount.get(kh) || 0) + 1);
const wgt = t.perKey.get(kh); if (wgt === undefined) { unknownVoteKey++; continue; }
votesWeight.set(kh, (votesWeight.get(kh) || 0) + wgt);
}
}
// the heaviest distinct certificate per index (the one the lock test reads)
const heaviest = new Map();
for (const cert of certDistinct.values()) { const cur = heaviest.get(cert.index); if (!cur || cert.signerPositions.length > cur.signerPositions.length) heaviest.set(cert.index, cert); }
const all = signingShares([...certDistinct.values()], tables);
const inst = signingShares(certInstances, tables);
const heavy = signingShares([...heaviest.values()], tables);
return { certInstances: certInstances.length, certDistinct: certDistinct.size, heaviest: heaviest.size, evidence, outsideCerts, outsideVotes, unknownVoteKey, votes: votesDistinct.size, all, inst, heavy, votesWeight, votesPerKeyCount, blocks: blocksList.length, indices: new Set([...certDistinct.values()].map(c => c.index)).size };
}
const S = tally(windowChain);
const S2 = sinceChain ? tally(sinceChain) : null;
const dagEnd = await rpc.call('getBlockDagInfo', {});
rpc.close();
const H = topShares(hashing), A = topShares(aggregation), E = topShares(eligible), P = topShares(proving), PA = topShares(payoutAddr);
const sorted = (xs) => [...xs].sort((a, b) => a - b);
const q = (xs, p) => xs.length ? sorted(xs)[Math.min(xs.length - 1, Math.floor(p * xs.length))] : NaN;
const out = [];
out.push(`### x14-concentration: Mac observer node ${WRPC} (exec ${EXEC}), read-only, ${new Date().toISOString()}, DAA ${dag.virtualDaaScore}, window ${window} DAA (weights at checkpoint ${w.checkpointIndex}, DAA ${w.daaScore}), ${Math.round((Date.now() - t0) / 1000)} s\n`);
out.push(`### x14-concentration: Mac observer node ${WRPC} (exec ${EXEC}), read-only, ${new Date().toISOString()}, node version ${info.serverVersion}${NODE_BUILD ? ' (' + NODE_BUILD + ')' : ''}, DAA ${dag.virtualDaaScore} at start and ${dagEnd.virtualDaaScore} at the end, window ${window} DAA (weights at checkpoint ${w.checkpointIndex}, DAA ${w.daaScore}), ${Math.round((Date.now() - t0) / 1000)} s\n`);
out.push('| quantity | keys | total | top-1 | top-3 | top-10 |');
out.push('|---|---|---|---|---|---|');
out.push(row(`hashing (blue blocks per vote key, getFinalityWeights; ${voters} of ${H.keys} keys are voters above dust ${w.params?.dust})`, H, 'blocks'));
@ -99,7 +226,24 @@ out.push(row(`aggregation (certificates built per named aggregator key over ${ce
out.push(row(`aggregator sortition (keys named eligible per checkpoint, ${inWin.length} checkpoints)`, E, 'eligibilities'));
out.push(row(`proving (paid proof records per prover key hash over ${chainBlocksRead} chain blocks from DAA ${firstDaa} to the tip; ${carried} records carried, ${rejectedRecords} rejected)`, P, 'paid shards'));
out.push(row('proving by payout address (the same records by the EVM address paid)', PA, 'paid shards'));
out.push(`| signing | not exposed: no RPC returns a certificate's signer set or bitmap (RpcCheckpoint carries signedWeight, votesSeen, voters, aggregators, certificateAggregator); over ${locked.length} locked checkpoints signed weight over total is p50 ${pct(q(signedFrac, 0.5))}, min ${pct(Math.min(...signedFrac))}, max ${pct(Math.max(...signedFrac))}; votes seen p50 ${q(votesSeen, 0.5)} of ${w.voters} voters | | | | |`);
const sRows = (S, label) => {
out.push(row(`signing, certificates${label} (signed weight per key over ${S.certDistinct} distinct certificates at ${S.indices} indices, carried ${S.certInstances} times by ${S.blocks} chain blocks; ${S.all.mapped} mapped, ${S.all.unmapped.length} unmapped)`, topShares(S.all.weight), 'weight'));
out.push(row(`signing, heaviest certificate per index${label} (${S.heaviest} certificates, the one the lock test reads)`, topShares(S.heavy.weight), 'weight'));
out.push(row(`signing, every carriage${label} (the same certificates counted once per block that carries them)`, topShares(S.inst.weight), 'weight'));
out.push(row(`signing, votes${label} (distinct (index, key, block) votes carried, weighted by the key's weight at C_i; ${S.votes} votes, ${S.unknownVoteKey} by keys under dust at their checkpoint)`, topShares(S.votesWeight), 'weight'));
out.push(row(`signing, votes unweighted${label} (votes per key)`, topShares(S.votesPerKeyCount), 'votes'));
};
sRows(S, '');
if (S2) sRows(S2, ` since DAA ${SINCE_DAA}`);
out.push(`| signing, the node's own aggregate | over ${locked.length} locked checkpoints signed weight over total is p50 ${pct(q(signedFrac, 0.5))}, min ${pct(Math.min(...signedFrac))}, max ${pct(Math.max(...signedFrac))}; votes seen p50 ${q(votesSeen, 0.5)} of ${w.voters} voters | | | | |`);
out.push('');
out.push(`Signing method: ${windowChain.length} chain blocks in the window (DAA ${winStart + 1} to ${sinkDaa}), ${chain.length} chain blocks walked back to DAA ${chain[chain.length - 1]?.daa} for the tables, ${blues.length} blue blocks; voter tables rebuilt at ${cpInWin.length} checkpoints, ${tableAgree} agree with the node's voters count${tableDisagree.length ? ', disagreeing: ' + tableDisagree.slice(0, 8).join('; ') : ''}; key reveals ${revealChecked} checked against BLAKE2b (${revealMismatch} mismatches); evidence items ${S.evidence}; certificates for checkpoints outside the window ${S.outsideCerts}, votes outside ${S.outsideVotes}${S.all.unmapped.length ? '; unmapped: ' + S.all.unmapped.slice(0, 6).map(u => u.index + ' ' + u.reason).join('; ') : ''}.`);
out.push('');
out.push('Signing per key, heaviest certificate per index (top 12):');
out.push('| key (first 8) | signed weight | share | certificates signed | weight now (getFinalityWeights) |');
out.push('|---|---|---|---|---|');
{ const tot = [...S.heavy.weight.values()].reduce((a, b) => a + b, 0) || 1;
[...S.heavy.weight.entries()].sort((a, b) => b[1] - a[1]).slice(0, 12).forEach(([k, v]) => out.push(`| ${k.slice(0, 8)} | ${v} | ${pct(v / tot)} | ${S.heavy.count.get(k)} | ${hashing.get(k) ?? '-'} |`)); }
out.push('');
out.push(`Proving status (igneum_getProvingStatus): ${status.paidShards} shards paid in all, ${JSON.stringify(status.pool)} in the pool, verifier ${status.verifier}, pool balance ${(Number(BigInt(status.poolBalanceWei)) / 1e18).toFixed(3)} IGN, paid ${(Number(BigInt(status.paidWei)) / 1e18).toFixed(3)} IGN.`);
out.push('');

13
tools/observer/README.md
View file

@ -57,3 +57,16 @@ Created on start if missing.
## Reading it
`site/api/live.mjs` serves `/api/live` from these tables in five indexed queries (`proving` from `live_state.proving`; every block carries `shards: [{i, n, state, prover, lag, payout, pgas}]` and `proven`). `LIVE_TABLE_PREFIX` on the API reads a test observer's tables. `site/live.html` polls it every 2 s. The site shows OFFLINE when `live_state.updated_at` is older than 30 s.
## Independence columns and the nightly table (O-X.1, ledger X5 and X14; branch `ledger-observer`, 5 October 2026 night, NOT deployed)
The running observer is untouched. On this branch `observer.mjs` adds four tables and three timers so the phase 5 gate ("N_ind >= 1,000 over 30 days with top-10 share of window weight under 50%", `site/journey.json`) can be read from stored data. The independence definition is the project lead's decision (`docs/plans/ledger-decisions.md` item 3); the table labels its column `proposed`.
| Table | Rows | Filled by | Columns |
|---|---|---|---|
| `live_peer_asn` | one per announcing peer address | `peerAsnTick` every 10 min: `getConnectedPeerInfo` against the offline prefix table `tools/observer/asn-table.txt` (`IGNEUM_ASN_TABLE` overrides). No third-party lookup is made at run time; the table is filled by hand from a dated BGP dump (RIPEstat, Team Cymru bulk whois, or a pyasn dump of RouteViews). Private ranges read `local`; a miss stays `source = 'none'` | `address`, `asn`, `asn_name`, `source`, `first_seen_at`, `updated_at` |
| `live_key_machines` | one per (vote key, machine) | `keyMachineTick` every 10 min: the log intake (`miner_logs`, same database as `tools/logs.mjs`), every upload of the last day with `vote_key_hash=<64 hex>` (the miner's identity line, `igneum/miner/src/main.rs`), keyed by the upload's `machine` (the STATUS line's run id carries the same id8) | `key_hash`, `machine`, `label`, `first_seen_at`, `last_seen_at` |
| `live_pool_statements` | one per pool | `poolStatementTick` every 10 min: `tools/observer/pool-statements/<pool>.json` (`IGNEUM_POOL_STATEMENTS` overrides) parsed and verified against `registry.json`; a failing statement is stored with `verified = false` and the reason | `pool`, `pubkey`, `keys`, `signed_at`, `verified`, `error`, `statement`, `received_at` |
| `live_concentration` | one per day | `nightlyTick` at 00:05 UTC from what the observer holds: `getFinalityWeights` (hashing), the window's checkpoints (aggregation by `certificateAggregator`), `live_certificates` with their stored voter tables (signing: signed weight per key over the bitmap), `live_proofs` in state `paid` (proving), and the three tables above (N_ind) | `day`, `daa`, `checkpoint_index`, `node_version`, `hashing`, `signing`, `proving`, `aggregation` (each `{keys, total, top1, top3, top10, unit, ...}`), `n_ind`, `n_ind_definition` (`proposed`), `n_ind_eligible`, `n_ind_unattributed`, `computed_at` |
What is open: a block names its producer by vote key and a peer announces an address, and nothing ties the two, so the ASN per key is null until the app reports its public address with its uploads (an app-owner item); `n_ind_unattributed` counts the keys with no attribute at all. The pure functions live in `tools/observer/lib/concentration.mjs` (payload decoder, voter table rebuild, shares, N_ind, pool statement, ASN table) and `lib/nightly.mjs` (the row); `tools/finality-attacks/x14-concentration.mjs` uses the same decoder and table rebuild against a live node. Tests: `node --test 'tools/observer/test/*.test.mjs'` (9 tests).

6
tools/observer/asn-table.txt Normal file
View file

@ -0,0 +1,6 @@
# Offline prefix table for the observer's peer-address column (ledger X5 and X14, O-X.1 (a)).
# One row per line: <IPv4 prefix in CIDR> <ASN> [name]. Longest prefix wins. Private and loopback ranges answer
# "local" without a row. IPv6 is not read yet. The observer never asks a third party at run time; this file is the
# lookup. Fill it from a BGP dump taken by hand (RIPEstat, Team Cymru's whois bulk service, or a pyasn dump of the
# RouteViews RIB), with the dump's date on the line, and commit the file. Empty tonight (5 October 2026): no row is
# written from memory, so every public peer reads source 'none' until a dump is loaded.

278
tools/observer/lib/concentration.mjs Normal file
View file

@ -0,0 +1,278 @@
// Concentration and independence (ledger X5 and X14, O-X.1): pure functions shared by the observer's nightly table
// and by tools/finality-attacks/x14-concentration.mjs. No RPC, no database, no network in this file.
//
// 1. decodeFinalitySection(payload): the finality items a block's coinbase carries (spec 03 C2, C3, 3.6; the fork's
// consensus/core/src/finality.rs: `items || len_le32 || "IGNF"` at the end of the coinbase extra data; tag 1 vote
// 280 B, tag 2 certificate, tag 3 evidence 560 B) and the key reveal (`IGNK` plus 288 hex) before it.
// 2. voterTableAt(checkpoint, blues, params): the canonical voter list at a checkpoint, rebuilt the way the node
// builds it (consensus/src/processes/finality.rs compute_weights and refresh_voters), so a certificate's bitmap
// can be read as key hashes.
// 3. topShares(counts): top-1, top-3, top-10 shares of a Map key -> weight.
// 4. independenceClasses(keys, attributes): N_ind, the PROPOSED definition of ledger X5 (decision item 3 for the project lead):
// distinct (ASN, machine fingerprint, pool) classes among keys above dust.
// 5. parsePoolStatement / verifyPoolStatement: the pool attestation format, a signed JSON {pool, keys[], signed_at}.
// 6. asnOf(address, table): the autonomous system of a peer address from an offline prefix table.
import { createPublicKey, verify as edVerify } from 'node:crypto';
const VOTE_LEN = 8 + 32 + 48 + 96 + 96; // 280
const EVIDENCE_LEN = 2 * VOTE_LEN; // 560
const hex = (u8) => Buffer.from(u8).toString('hex');
export function payloadBytes(p) {
if (p instanceof Uint8Array) return p;
if (Array.isArray(p)) return Uint8Array.from(p);
if (typeof p === 'string') return Uint8Array.from(Buffer.from(p, 'hex'));
return new Uint8Array();
}
/** The extra data of a coinbase payload: `blue_score u64 || subsidy u64 || script version u16 || script len u8 || script || extra`. */
export function coinbaseExtra(payload) {
const p = payloadBytes(payload);
if (p.length < 19) return new Uint8Array();
return p.subarray(19 + p[18]);
}
function readVote(b, o) {
if (b.length < o + VOTE_LEN) return null;
const dv = new DataView(b.buffer, b.byteOffset, b.byteLength);
return {
index: Number(dv.getBigUint64(o, true)),
checkpoint: hex(b.subarray(o + 8, o + 40)),
pubkey: hex(b.subarray(o + 40, o + 88)),
signature: hex(b.subarray(o + 88, o + 184)),
sortition: hex(b.subarray(o + 184, o + 280)),
};
}
function readCertificate(b, s) {
if (b.length < s + 48) return null;
const dv = new DataView(b.buffer, b.byteOffset, b.byteLength);
const voterCount = dv.getUint32(s + 40, true);
const bl = dv.getUint32(s + 44, true);
const end = s + 48 + bl + 96 + 32 + 96;
if (bl > 1 << 20 || b.length < end) return null;
const bitmap = b.subarray(s + 48, s + 48 + bl);
const signerPositions = [];
for (let i = 0; i < bitmap.length; i++) for (let bit = 0; bit < 8; bit++) if (bitmap[i] & (1 << bit)) { const pos = i * 8 + bit; if (pos < voterCount) signerPositions.push(pos); }
return {
index: Number(dv.getBigUint64(s, true)), checkpoint: hex(b.subarray(s + 8, s + 40)), voterCount,
bitmap: hex(bitmap), signerPositions, signature: hex(b.subarray(s + 48 + bl, s + 144 + bl)),
aggregator: hex(b.subarray(s + 144 + bl, s + 176 + bl)), aggregatorProof: hex(b.subarray(s + 176 + bl, end)),
bytes: hex(b.subarray(s, end)), length: end - s,
};
}
/**
* Every finality item of a coinbase payload (or of its extra data: the section is a trailer, so either works) plus
* the key reveal before it. A malformed section yields the items decoded before the fault, as the node does.
*/
export function decodeFinalitySection(payload, { raw = false } = {}) {
const p = payloadBytes(payload);
const extra = raw ? p : coinbaseExtra(p);
const out = { votes: [], certificates: [], evidence: [], reveal: null, sectionBytes: 0, malformed: false };
const n = extra.length;
let before = extra;
if (n >= 8 && String.fromCharCode(...extra.subarray(n - 4)) === 'IGNF') {
const len = new DataView(extra.buffer, extra.byteOffset, n).getUint32(n - 8, true);
if (len + 8 <= n) {
const body = extra.subarray(n - 8 - len, n - 8);
before = extra.subarray(0, n - 8 - len);
out.sectionBytes = len + 8;
let o = 0;
while (o < body.length) {
const tag = body[o];
if (tag === 1) { const v = readVote(body, o + 1); if (!v) { out.malformed = true; break; } out.votes.push(v); o += 1 + VOTE_LEN; }
else if (tag === 2) { const c = readCertificate(body, o + 1); if (!c) { out.malformed = true; break; } out.certificates.push(c); o += 1 + c.length; }
else if (tag === 3) { const a = readVote(body, o + 1), b2 = readVote(body, o + 1 + VOTE_LEN); if (!a || !b2) { out.malformed = true; break; } out.evidence.push({ first: a, second: b2 }); o += 1 + EVIDENCE_LEN; }
else { out.malformed = true; break; }
}
}
}
// the reveal: "IGNK" then 288 lowercase hex characters (48-byte key, 96-byte proof of possession)
const text = Buffer.from(before).toString('latin1');
const at = text.indexOf('IGNK');
if (at >= 0 && text.length >= at + 4 + 288) {
const h = text.slice(at + 4, at + 4 + 288);
if (/^[0-9a-f]{288}$/.test(h)) out.reveal = { pubkey: h.slice(0, 96), pop: h.slice(96) };
}
return out;
}
/**
* The canonical voter list at a checkpoint, as the node computes it.
* `checkpoint`: { hash, daaScore }. `blues`: every blue block that may count, as { hash, daaScore, voteKeyHash,
* mergerDaa } where mergerDaa is the DAA score of the chain block whose mergeset holds it (a chain block is held by
* its child chain block; the checkpoint itself is counted whatever its merger). `params`: { weightWindow, dust }.
* `bans`: optional Map keyHash -> DAA score the ban ends at (a stripped key leaves the list while daa < until).
* Returns { voters: [keyHash] sorted as the node sorts (byte order of the hash), perKey: Map keyHash -> blocks,
* total: the voters' blocks, keys: every key seen in the window }.
*/
export function voterTableAt(checkpoint, blues, params, bans = new Map()) {
const daaC = Number(checkpoint.daaScore);
const start = daaC - Number(params.weightWindow);
const perKey = new Map();
for (const b of blues) {
const d = Number(b.daaScore);
if (!(d > start && d <= daaC)) continue;
if (b.hash !== checkpoint.hash && Number(b.mergerDaa) > daaC) continue;
perKey.set(b.voteKeyHash, (perKey.get(b.voteKeyHash) || 0) + 1);
}
const dust = Number(params.dust);
const voters = [...perKey.entries()].filter(([k, n]) => n >= dust && !(bans.has(k) && daaC < bans.get(k))).map(([k]) => k).sort();
return { voters, perKey, total: voters.reduce((s, k) => s + perKey.get(k), 0), keys: perKey.size };
}
/** Top-1, top-3 and top-10 shares of a Map key -> weight (numbers), with the count of keys and the total. */
export function topShares(counts) {
const vals = [...counts.values()].map(Number).sort((a, b) => b - a);
const total = vals.reduce((s, v) => s + v, 0);
const top = (n) => vals.slice(0, n).reduce((s, v) => s + v, 0) / (total || 1);
return { keys: vals.length, total, top1: top(1), top3: top(3), top10: top(10) };
}
/**
* Signed weight per key over a set of certificates. `certs`: [{ index, checkpoint, voterCount, signerPositions }];
* `tables`: Map index -> the voter table at that index's checkpoint ({ voters, perKey, hash }). A certificate whose
* checkpoint hash is not the table's, or whose voter_count is not the table's length, is counted in `unmapped` with
* its reason and adds nothing. Returns { weight: Map key -> signed weight, count: Map key -> certificates signed,
* mapped, unmapped: [{index, reason}] }.
*/
export function signingShares(certs, tables) {
const weight = new Map(), count = new Map(), unmapped = [];
let mapped = 0;
for (const c of certs) {
const t = tables.get(c.index);
if (!t) { unmapped.push({ index: c.index, reason: 'no table for the index' }); continue; }
if (t.hash && t.hash !== c.checkpoint) { unmapped.push({ index: c.index, reason: 'certificate names another block' }); continue; }
if (t.voters.length !== c.voterCount) { unmapped.push({ index: c.index, reason: `voter_count ${c.voterCount} against a rebuilt list of ${t.voters.length}` }); continue; }
mapped++;
for (const p of c.signerPositions) {
const k = t.voters[p];
weight.set(k, (weight.get(k) || 0) + t.perKey.get(k));
count.set(k, (count.get(k) || 0) + 1);
}
}
return { weight, count, mapped, unmapped };
}
// ---------------------------------------------------------------------------------------------
// Independence (ledger X5, decision item 3): PROPOSED definition, not adopted
/**
* N_ind under the proposed definition: the number of distinct (ASN, machine fingerprint, pool) classes among the
* keys above dust. `keys`: [{ keyHash, blocks }]; `attributes`: Map keyHash -> { asn, fingerprint, pool } with any
* field null when unknown. The decision request's addition: a key with no fingerprint (a miner that sends no logs)
* is its own class only when its ASN is used by no other key; otherwise it joins the class of that ASN with no
* fingerprint and no pool. Returns { nInd, classes: Map classKey -> [keyHash], eligible, unattributed }.
*/
export function independenceClasses(keys, attributes, dust) {
const eligible = keys.filter(k => Number(k.blocks) >= Number(dust));
const asnUsers = new Map();
for (const k of eligible) { const a = attributes.get(k.keyHash); if (a && a.asn) asnUsers.set(a.asn, (asnUsers.get(a.asn) || 0) + 1); }
const classes = new Map();
let unattributed = 0;
for (const k of eligible) {
const a = attributes.get(k.keyHash) || {};
let cls;
if (!a.asn && !a.fingerprint && !a.pool) { unattributed++; cls = `unattributed:${k.keyHash}`; }
else if (!a.fingerprint) cls = asnUsers.get(a.asn) > 1 ? `asn:${a.asn}|fp:-|pool:${a.pool || '-'}` : `asn:${a.asn}|solo:${k.keyHash}`;
else cls = `asn:${a.asn || '-'}|fp:${a.fingerprint}|pool:${a.pool || '-'}`;
if (!classes.has(cls)) classes.set(cls, []);
classes.get(cls).push(k.keyHash);
}
return { nInd: classes.size, classes, eligible: eligible.length, unattributed };
}
// ---------------------------------------------------------------------------------------------
// Pool statement: {pool, keys[], signed_at} signed by the pool's Ed25519 key
export const POOL_STATEMENT_FORMAT = 'igneum-pool-statement-1';
/**
* The bytes a pool signs: the canonical JSON of {format, pool, keys (sorted, lowercase hex), signed_at} with the
* keys in that order and no whitespace, so two encoders agree byte for byte.
*/
export function poolStatementBytes({ pool, keys, signed_at }) {
const body = { format: POOL_STATEMENT_FORMAT, pool, keys: [...keys].map(k => String(k).toLowerCase()).sort(), signed_at };
return Buffer.from(JSON.stringify(body), 'utf8');
}
/**
* Parses a pool statement envelope: JSON text of {format, pool, keys[], signed_at, pubkey, sig}. Returns
* { ok, statement, error }. Shape only; verifyPoolStatement checks the signature.
*/
export function parsePoolStatement(text) {
let j;
try { j = typeof text === 'string' ? JSON.parse(text) : text; } catch (e) { return { ok: false, error: `not JSON: ${e.message}` }; }
if (!j || typeof j !== 'object') return { ok: false, error: 'not an object' };
if (j.format !== POOL_STATEMENT_FORMAT) return { ok: false, error: `format is ${JSON.stringify(j.format)}, want ${POOL_STATEMENT_FORMAT}` };
if (typeof j.pool !== 'string' || !/^[a-z0-9][a-z0-9.-]{0,62}$/.test(j.pool)) return { ok: false, error: 'pool must be a lowercase label (letters, digits, dot, dash, at most 63 characters)' };
if (!Array.isArray(j.keys) || j.keys.length === 0 || j.keys.length > 100000) return { ok: false, error: 'keys must be a non-empty list (at most 100,000)' };
for (const k of j.keys) if (typeof k !== 'string' || !/^[0-9a-fA-F]{64}$/.test(k)) return { ok: false, error: `key ${JSON.stringify(k)} is not a 64-hex vote key hash` };
if (new Set(j.keys.map(k => k.toLowerCase())).size !== j.keys.length) return { ok: false, error: 'keys repeat' };
if (typeof j.signed_at !== 'string' || Number.isNaN(Date.parse(j.signed_at)) || !/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d+)?Z$/.test(j.signed_at)) return { ok: false, error: 'signed_at must be an ISO 8601 UTC instant' };
if (typeof j.pubkey !== 'string' || !/^[0-9a-f]{64}$/.test(j.pubkey)) return { ok: false, error: 'pubkey must be a 32-byte Ed25519 key in hex' };
if (typeof j.sig !== 'string' || !/^[0-9a-f]{128}$/.test(j.sig)) return { ok: false, error: 'sig must be a 64-byte Ed25519 signature in hex' };
return { ok: true, statement: { pool: j.pool, keys: j.keys.map(k => k.toLowerCase()).sort(), signed_at: j.signed_at, pubkey: j.pubkey, sig: j.sig } };
}
/**
* Verifies a parsed statement's Ed25519 signature against the pool's registered key (hex). `registry` maps the pool
* label to its key; a statement signed by a key that is not the pool's is refused even when the signature verifies.
* `maxAgeMs` refuses a statement older than that (default 35 days: one weight window plus slack).
*/
export function verifyPoolStatement(statement, registry, { now = Date.now(), maxAgeMs = 35 * 86400e3 } = {}) {
const want = registry instanceof Map ? registry.get(statement.pool) : registry?.[statement.pool];
if (!want) return { ok: false, error: `pool ${statement.pool} has no registered key` };
if (want.toLowerCase() !== statement.pubkey) return { ok: false, error: `signed by ${statement.pubkey.slice(0, 8)}, the pool's key is ${want.slice(0, 8)}` };
const age = now - Date.parse(statement.signed_at);
if (age < -300e3) return { ok: false, error: 'signed_at is in the future' };
if (age > maxAgeMs) return { ok: false, error: `signed ${Math.round(age / 86400e3)} days ago, over the ${Math.round(maxAgeMs / 86400e3)}-day limit` };
let key;
try { key = createPublicKey({ key: Buffer.concat([Buffer.from('302a300506032b6570032100', 'hex'), Buffer.from(statement.pubkey, 'hex')]), format: 'der', type: 'spki' }); }
catch (e) { return { ok: false, error: `bad public key: ${e.message}` }; }
const ok = edVerify(null, poolStatementBytes(statement), key, Buffer.from(statement.sig, 'hex'));
return ok ? { ok: true } : { ok: false, error: 'signature does not verify' };
}
// ---------------------------------------------------------------------------------------------
// Autonomous system of a peer address: an offline prefix table, no third-party lookups
/** Parses a table of `prefix asn [name]` lines (IPv4 CIDR), # comments allowed. Returns [{ net, bits, asn, name }]. */
export function parseAsnTable(text) {
const rows = [];
for (const raw of String(text).split('\n')) {
const line = raw.replace(/#.*/, '').trim();
if (!line) continue;
const m = /^(\d+\.\d+\.\d+\.\d+)\/(\d+)\s+(\S+)(?:\s+(.*))?$/.exec(line);
if (!m) continue;
const bits = Number(m[2]);
if (bits < 0 || bits > 32) continue;
rows.push({ net: ipv4(m[1]), bits, asn: m[3], name: (m[4] || '').trim() });
}
return rows.sort((a, b) => b.bits - a.bits);
}
function ipv4(s) {
const p = s.split('.').map(Number);
if (p.length !== 4 || p.some(x => !(x >= 0 && x <= 255))) return null;
return ((p[0] << 24) | (p[1] << 16) | (p[2] << 8) | p[3]) >>> 0;
}
/**
* The autonomous system of an address (`ip`, `ip:port` or `[v6]:port`) from the parsed table, longest prefix first.
* Private and loopback ranges answer `local`; an IPv6 address or a miss answers null (the stub: the table is the
* offline lookup; a live lookup against a BGP source is named in the README and not called from here).
*/
export function asnOf(address, table) {
const s = String(address || '').trim();
if (s.startsWith('[')) return null;
const host = s.includes(':') ? s.split(':')[0] : s;
const n = ipv4(host);
if (n === null) return null;
const a = n >>> 24, b = (n >>> 16) & 255;
if (a === 10 || a === 127 || (a === 192 && b === 168) || (a === 172 && b >= 16 && b <= 31) || (a === 169 && b === 254)) return { asn: 'local', name: 'private or loopback' };
for (const r of table) if (r.net !== null && (r.bits === 0 || ((n ^ r.net) >>> (32 - r.bits)) === 0)) return { asn: r.asn, name: r.name };
return null;
}

75
tools/observer/lib/nightly.mjs Normal file
View file

@ -0,0 +1,75 @@
// The nightly concentration row (ledger X5 and X14, O-X.1): top-1, top-3 and top-10 shares for the four
// concentrations over the weight window, plus N_ind under the PROPOSED independence definition (decision item 3
// for the project lead, docs/plans/ledger-decisions.md). Pure: takes what the observer has read and returns the row; the
// observer stores it in live_concentration once a day. No RPC and no database here.
//
// Inputs:
// weights the getFinalityWeights answer (keys[] with keyHash, blocks, voter; params.dust; checkpointIndex, daaScore)
// checkpoints the window's checkpoints as the node reports them (certificateAggregator per certified or locked one)
// certificates stored certificates [{ index, bitmap_hex, voter_count, voters: [{pubkey, weight}] }] (live_certificates:
// the voter table as the node reported it when the lock landed; signed weight = sum of voters[p].weight
// over the set bits); only those whose checkpoint is in the window
// proofs paid shards per prover id over the window: Map prover -> count (live_proofs rows in state paid)
// attributes Map keyHash -> { asn, fingerprint, pool } from live_peer_asn, live_key_machines, live_pool_statements
// meta { nodeVersion, daa, day }
import { topShares, independenceClasses } from './concentration.mjs';
import { voteKeyHash } from '../../finality-attacks/lib/blake2b.mjs';
const ZERO = '0'.repeat(64);
// live_certificates.voters rows carry the public key (what a verifier needs); the key hash is derived here
const keyOf = (v) => v.keyHash || (v.pubkey ? voteKeyHash(v.pubkey) : null);
export function signedWeightFromStored(certificates) {
const weight = new Map();
let used = 0, skipped = 0;
for (const c of certificates) {
const voters = Array.isArray(c.voters) ? c.voters : [];
if (voters.length !== Number(c.voter_count)) { skipped++; continue; }
used++;
const bm = Buffer.from(String(c.bitmap_hex || ''), 'hex');
for (let i = 0; i < bm.length; i++) for (let bit = 0; bit < 8; bit++) {
const p = i * 8 + bit;
if (p < voters.length && (bm[i] & (1 << bit))) {
const k = keyOf(voters[p]);
if (k) weight.set(k, (weight.get(k) || 0) + Number(voters[p].weight));
}
}
}
return { weight, used, skipped };
}
const shares = (s) => ({ keys: s.keys, total: s.total, top1: s.top1, top3: s.top3, top10: s.top10 });
export function nightlyRow({ weights, checkpoints = [], certificates = [], proofs = new Map(), attributes = new Map(), meta = {} }) {
const dust = Number(weights?.params?.dust ?? 100);
const keys = (weights?.keys || []).map(k => ({ keyHash: String(k.keyHash), blocks: Number(k.blocks) }));
const hashing = topShares(new Map(keys.map(k => [k.keyHash, k.blocks])));
const agg = new Map();
let anonymous = 0;
for (const c of checkpoints) {
if (c.state !== 'locked' && c.state !== 'certified') continue;
const a = String(c.certificateAggregator || ZERO);
if (a === ZERO) anonymous++; else agg.set(a, (agg.get(a) || 0) + 1);
}
const aggregation = topShares(agg);
const signed = signedWeightFromStored(certificates);
const signing = topShares(signed.weight);
const proving = topShares(proofs);
const ind = independenceClasses(keys, attributes, dust);
return {
day: meta.day || new Date().toISOString().slice(0, 10),
daa: meta.daa ?? Number(weights?.daaScore ?? 0),
checkpoint_index: Number(weights?.checkpointIndex ?? 0),
node_version: meta.nodeVersion || null,
hashing: { ...shares(hashing), unit: 'blue blocks', above_dust: keys.filter(k => k.blocks >= dust).length },
signing: { ...shares(signing), unit: 'signed weight', certificates: signed.used, skipped: signed.skipped },
proving: { ...shares(proving), unit: 'paid shards' },
aggregation: { ...shares(aggregation), unit: 'certificates', anonymous },
n_ind: ind.nInd,
n_ind_definition: 'proposed', // ledger X5, decision item 3: not adopted until the project lead decides
n_ind_eligible: ind.eligible,
n_ind_unattributed: ind.unattributed,
computed_at: new Date().toISOString(),
};
}

164
tools/observer/observer.mjs
View file

@ -34,13 +34,24 @@
// fills past minutes instead of painting a spike. If no blockAdded arrives for 60 s while the node's block_count
// advances, the observer resubscribes; after two failed attempts it exits 2 so tools/observer/run.sh restarts it.
import { readFileSync } from 'node:fs';
import { readFileSync, readdirSync, existsSync } from 'node:fs';
import { homedir } from 'node:os';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { parseAsnTable, asnOf, parsePoolStatement, verifyPoolStatement } from './lib/concentration.mjs';
import { nightlyRow } from './lib/nightly.mjs';
const RPC = process.env.IGNEUM_RPC || 'ws://127.0.0.1:28610';
const RETAIN_HOURS = Number(process.env.LIVE_RETAIN_HOURS || 24);
const T = (process.env.LIVE_TABLE_PREFIX || '').replace(/[^a-z0-9_]/gi, '');
const TB = `${T}live_blocks`, TS = `${T}live_state`, TE = `${T}live_events`, TC = `${T}live_checkpoints`, TX = `${T}live_certificates`, TP = `${T}live_proofs`;
// O-X.1 (ledger X5, X14; branch ledger-observer, 5 October 2026 night): the independence columns and the nightly table
const TA = `${T}live_peer_asn`, TM = `${T}live_key_machines`, TQ = `${T}live_pool_statements`, TN = `${T}live_concentration`;
const HERE = dirname(fileURLToPath(import.meta.url));
const ASN_TABLE = process.env.IGNEUM_ASN_TABLE || join(HERE, 'asn-table.txt');
const POOL_DIR = process.env.IGNEUM_POOL_STATEMENTS || join(HERE, 'pool-statements');
const NIGHTLY_AT_UTC_MIN = 5; // 00:05 UTC, once per day
const ATTRIB_EVERY_MS = 10 * 60_000; // peers' ASNs, the key-to-machine map and the pool statements
const EVM = process.env.IGNEUM_EVM_RPC || 'http://127.0.0.1:26800';
const STATE_EVERY_MS = 2000;
const FLUSH_EVERY_MS = 500;
@ -174,6 +185,48 @@ async function setupSchema() {
headers jsonb NOT NULL,
headers_complete boolean NOT NULL DEFAULT false,
created_at timestamptz NOT NULL DEFAULT now())`,
// O-X.1 (a): the autonomous system per announcing peer address, from the offline table (no third-party lookups)
`CREATE TABLE IF NOT EXISTS ${TA} (
address text PRIMARY KEY,
asn text,
asn_name text,
source text NOT NULL DEFAULT 'table',
first_seen_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now())`,
// O-X.1 (b): the machine fingerprint per vote key, from the app's log uploads (miner_logs.machine and the miner's
// "identity N 'label' vote_key_hash=..." line); one row per (key, machine)
`CREATE TABLE IF NOT EXISTS ${TM} (
key_hash text NOT NULL,
machine text NOT NULL,
label text,
first_seen_at timestamptz NOT NULL DEFAULT now(),
last_seen_at timestamptz NOT NULL DEFAULT now(),
PRIMARY KEY (key_hash, machine))`,
// O-X.1 (c): pool statements, a signed JSON {pool, keys[], signed_at}, verified against the pool's registered key
`CREATE TABLE IF NOT EXISTS ${TQ} (
pool text PRIMARY KEY,
pubkey text NOT NULL,
keys jsonb NOT NULL,
signed_at timestamptz NOT NULL,
verified boolean NOT NULL,
error text,
statement jsonb NOT NULL,
received_at timestamptz NOT NULL DEFAULT now())`,
// O-X.1 (d): the nightly table: top-1/3/10 shares of the four concentrations plus N_ind (PROPOSED definition)
`CREATE TABLE IF NOT EXISTS ${TN} (
day date PRIMARY KEY,
daa bigint,
checkpoint_index bigint,
node_version text,
hashing jsonb NOT NULL,
signing jsonb NOT NULL,
proving jsonb NOT NULL,
aggregation jsonb NOT NULL,
n_ind int,
n_ind_definition text NOT NULL DEFAULT 'proposed',
n_ind_eligible int,
n_ind_unattributed int,
computed_at timestamptz NOT NULL DEFAULT now())`,
];
for (const s of stmts) await sql(s);
}
@ -880,6 +933,111 @@ async function provingState() {
};
}
// ---------- O-X.1: independence columns and the nightly concentration table (ledger X5 and X14) ----------
// On the branch ledger-observer (5 October 2026, night); not deployed. The running observer is untouched.
let asnTable = null;
function loadAsnTable() {
if (asnTable) return asnTable;
try { asnTable = parseAsnTable(readFileSync(ASN_TABLE, 'utf8')); } catch { asnTable = []; }
return asnTable;
}
// (a) every connected peer's announcing address against the offline prefix table. A miss stays a row with asn null
// (source 'none'), so a later table fills it in; nothing here asks a third party.
async function peerAsnTick(rpc) {
try {
const peers = (await rpc.call('getConnectedPeerInfo', {})).infos || [];
const table = loadAsnTable();
for (const p of peers) {
const address = String(p.address || '');
if (!address) continue;
const a = asnOf(address, table);
await sql(`INSERT INTO ${TA} (address, asn, asn_name, source) VALUES ($1, $2, $3, $4)
ON CONFLICT (address) DO UPDATE SET asn = EXCLUDED.asn, asn_name = EXCLUDED.asn_name, source = EXCLUDED.source, updated_at = now()`,
[address, a ? a.asn : null, a ? a.name : null, a ? 'table' : 'none']);
}
} catch (e) { log('peer asn tick failed', e.message); }
}
// (b) the key-to-machine map from the log intake: every upload of the last day whose lines carry
// "vote_key_hash=<64 hex>" (the miner's identity line, igneum/miner/src/main.rs), keyed by the upload's machine id
// (the STATUS line's run id carries the same id8). The intake lives in the same database (tools/logs.mjs).
async function keyMachineTick() {
try {
const rows = await sql(`SELECT machine, label, lines, received_at FROM miner_logs
WHERE received_at > now() - interval '1 day' AND lines LIKE '%vote_key_hash=%' ORDER BY received_at ASC LIMIT 2000`);
const seen = new Map();
for (const r of rows) {
const machine = String(r.machine || '').slice(0, 64);
if (!machine) continue;
for (const m of String(r.lines).matchAll(/vote_key_hash=([0-9a-f]{64})/g)) seen.set(`${m[1]}|${machine}`, { key: m[1], machine, label: r.label, at: r.received_at });
}
for (const v of seen.values()) {
await sql(`INSERT INTO ${TM} (key_hash, machine, label, first_seen_at, last_seen_at) VALUES ($1, $2, $3, $4, $4)
ON CONFLICT (key_hash, machine) DO UPDATE SET label = EXCLUDED.label, last_seen_at = GREATEST(${TM}.last_seen_at, EXCLUDED.last_seen_at)`,
[v.key, v.machine, v.label || null, v.at]);
}
} catch (e) { log('key machine tick failed', e.message); }
}
// (c) pool statements from POOL_DIR/<pool>.json, verified against POOL_DIR/registry.json ({pool: ed25519 pubkey hex});
// a statement that fails is stored with verified false and the reason, never silently dropped
async function poolStatementTick() {
try {
if (!existsSync(POOL_DIR)) return;
let registry = {};
try { registry = JSON.parse(readFileSync(join(POOL_DIR, 'registry.json'), 'utf8')); } catch { }
for (const f of readdirSync(POOL_DIR)) {
if (!f.endsWith('.json') || f === 'registry.json') continue;
const text = readFileSync(join(POOL_DIR, f), 'utf8');
const parsed = parsePoolStatement(text);
if (!parsed.ok) { log(`pool statement ${f}: ${parsed.error}`); continue; }
const st = parsed.statement;
const v = verifyPoolStatement(st, registry);
await sql(`INSERT INTO ${TQ} (pool, pubkey, keys, signed_at, verified, error, statement)
VALUES ($1, $2, $3::jsonb, $4, $5, $6, $7::jsonb)
ON CONFLICT (pool) DO UPDATE SET pubkey = EXCLUDED.pubkey, keys = EXCLUDED.keys, signed_at = EXCLUDED.signed_at, verified = EXCLUDED.verified, error = EXCLUDED.error, statement = EXCLUDED.statement, received_at = now()`,
[st.pool, st.pubkey, JSON.stringify(st.keys), st.signed_at, v.ok, v.ok ? null : v.error, JSON.stringify(st)]);
}
} catch (e) { log('pool statement tick failed', e.message); }
}
// the attributes of every key: (ASN, machine fingerprint, pool). The key-to-address join is open: a block names its
// producer by vote key, a peer announces an address, and nothing yet ties the two (the app must report its public
// address with its uploads, an app-owner item), so asn is null here until that lands. The nightly row says how many
// keys are unattributed.
async function keyAttributes() {
const attrs = new Map();
const machines = await sql(`SELECT key_hash, machine FROM ${TM} WHERE last_seen_at > now() - interval '30 days'`);
for (const r of machines) { const a = attrs.get(r.key_hash) || { asn: null, fingerprint: null, pool: null }; a.fingerprint = a.fingerprint || r.machine; attrs.set(r.key_hash, a); }
const pools = await sql(`SELECT pool, keys FROM ${TQ} WHERE verified`);
for (const r of pools) for (const k of (r.keys || [])) { const a = attrs.get(k) || { asn: null, fingerprint: null, pool: null }; a.pool = r.pool; attrs.set(k, a); }
return attrs;
}
// (d) the nightly table, once a day at 00:05 UTC: the four concentrations and N_ind (PROPOSED definition, decision
// item 3), from what the observer already holds: the weights (hashing), the window's checkpoints (aggregation), the
// stored certificates with their voter tables (signing) and the paid shards (proving)
let nightlyDoneFor = null;
async function nightlyTick(rpc) {
const now = new Date();
const day = now.toISOString().slice(0, 10);
if (nightlyDoneFor === day || now.getUTCHours() !== 0 || now.getUTCMinutes() < NIGHTLY_AT_UTC_MIN) return;
nightlyDoneFor = day;
try {
const have = await sql(`SELECT day FROM ${TN} WHERE day = $1`, [day]);
if (have.length) return;
const weights = await rpc.call('getFinalityWeights', {});
const window = Number(weights.params?.weightWindow || 0);
const report = await rpc.call('getFinalityCheckpoints', { last: Math.ceil(window / 30) + 40 });
const lowDaa = Number(weights.daaScore) - window;
const checkpoints = (report.checkpoints || []).filter(c => Number(c.daaScore) > lowDaa);
const certs = await sql(`SELECT index, bitmap_hex, voter_count, voters FROM ${TX} WHERE daa_score > $1`, [lowDaa]);
const paid = await sql(`SELECT prover, count(*)::int AS n FROM ${TP} WHERE state = 'paid' AND carrier_daa > $1 GROUP BY prover`, [lowDaa]);
const proofs = new Map(paid.map(r => [r.prover, Number(r.n)]));
const row = nightlyRow({ weights, checkpoints, certificates: certs, proofs, attributes: await keyAttributes(), meta: { nodeVersion, daa: Number(weights.daaScore), day } });
await sql(`INSERT INTO ${TN} (day, daa, checkpoint_index, node_version, hashing, signing, proving, aggregation, n_ind, n_ind_definition, n_ind_eligible, n_ind_unattributed)
VALUES ($1, $2, $3, $4, $5::jsonb, $6::jsonb, $7::jsonb, $8::jsonb, $9, $10, $11, $12) ON CONFLICT (day) DO NOTHING`,
[row.day, row.daa, row.checkpoint_index, row.node_version, JSON.stringify(row.hashing), JSON.stringify(row.signing), JSON.stringify(row.proving), JSON.stringify(row.aggregation), row.n_ind, row.n_ind_definition, row.n_ind_eligible, row.n_ind_unattributed]);
log(`nightly concentration ${day}: hashing top-1 ${pct(row.hashing.top1)}%, signing top-1 ${pct(row.signing.top1)}% over ${row.signing.certificates} certificates, N_ind ${row.n_ind} (proposed definition, ${row.n_ind_unattributed} of ${row.n_ind_eligible} keys unattributed)`);
} catch (e) { nightlyDoneFor = null; log('nightly concentration failed', e.message); }
}
async function prune() {
try {
await sql(`DELETE FROM ${TB} WHERE received_at < now() - ($1 || ' hours')::interval`, [String(RETAIN_HOURS)]);
@ -988,7 +1146,11 @@ async function main() {
setInterval(pollRecords, STATE_EVERY_MS);
setInterval(() => tick(rpc), STATE_EVERY_MS);
setInterval(prune, PRUNE_EVERY_MS);
// O-X.1 columns (branch ledger-observer): the attribute ticks every 10 minutes, the nightly row at 00:05 UTC
setInterval(() => { peerAsnTick(rpc); keyMachineTick(); poolStatementTick(); }, ATTRIB_EVERY_MS);
setInterval(() => nightlyTick(rpc), 60_000);
tick(rpc); prune();
peerAsnTick(rpc); keyMachineTick(); poolStatementTick();
}
process.on('unhandledRejection', e => log('unhandled', e && e.message));

10
tools/observer/pool-statements/README.md Normal file
View file

@ -0,0 +1,10 @@
# Pool statements
One file per pool, `<pool>.json`, the signed JSON of `tools/observer/lib/concentration.mjs` (`parsePoolStatement`):
```
{"format":"igneum-pool-statement-1","pool":"example-pool","keys":["<64 hex vote key hash>", ...],
"signed_at":"2026-10-05T23:00:00Z","pubkey":"<32-byte Ed25519 key, hex>","sig":"<64-byte Ed25519 signature, hex>"}
```
The signature is over the canonical bytes `poolStatementBytes`: `{"format":...,"pool":...,"keys":[sorted lowercase],"signed_at":...}` with no whitespace. `registry.json` maps the pool label to its key; the observer refuses a statement signed by another key, one older than 35 days, or one whose keys repeat. Nothing is registered yet (5 October 2026).

4
tools/observer/pool-statements/registry.json Normal file
View file

@ -0,0 +1,4 @@
{
"_comment": "pool label -> the pool's Ed25519 public key (32 bytes, hex). A statement signed by any other key is stored with verified false. Empty until the first pool registers (ledger X5, O-X.1 (c)).",
"_format": "see tools/observer/README.md, Pool statements"
}

186
tools/observer/test/concentration.test.mjs Normal file
View file

@ -0,0 +1,186 @@
// node --test tools/observer/test/ (Node 22, no dependencies)
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { generateKeyPairSync, sign as edSign } from 'node:crypto';
import { decodeFinalitySection, coinbaseExtra, voterTableAt, topShares, signingShares, independenceClasses,
parsePoolStatement, verifyPoolStatement, poolStatementBytes, parseAsnTable, asnOf } from '../lib/concentration.mjs';
import { nightlyRow, signedWeightFromStored } from '../lib/nightly.mjs';
import { blake2b, voteKeyHash, selfTest } from '../../finality-attacks/lib/blake2b.mjs';
// ---------- payload fixtures (the fork's Vote::write, Certificate::write, encode_section) ----------
const u64 = (n) => { const b = Buffer.alloc(8); b.writeBigUInt64LE(BigInt(n)); return b; };
const u32 = (n) => { const b = Buffer.alloc(4); b.writeUInt32LE(n); return b; };
const fill = (n, v) => Buffer.alloc(n, v);
const vote = (index, cp, pub) => Buffer.concat([Buffer.from([1]), u64(index), cp, pub, fill(96, 0xaa), fill(96, 0xbb)]);
const cert = (index, cp, voterCount, bitmap, aggregator) => Buffer.concat([Buffer.from([2]), u64(index), cp, u32(voterCount), u32(bitmap.length), bitmap, fill(96, 0xcc), aggregator, fill(96, 0xdd)]);
const section = (items) => { const body = Buffer.concat(items); return Buffer.concat([body, u32(body.length), Buffer.from('IGNF')]); };
const coinbase = (extra) => Buffer.concat([u64(1), u64(2), Buffer.from([0, 0]), Buffer.from([3]), Buffer.from([1, 2, 3]), extra]);
const PUB = Buffer.from('b95b6d4f2f7e9887727aab42e68d25dd76e2d65c0135366c6de1a813f20eefd996cf7fc2e58bf90919cd2ea399cbd6c4', 'hex');
const CP = fill(32, 0x11);
test('blake2b: RFC vectors and the live vote key hash', () => {
assert.deepEqual(selfTest(), { ok: true });
// read from the Mac observer node on 5 October 2026 (getFinalityWeights.keys[0]): pubkey -> keyHash
assert.equal(voteKeyHash(PUB.toString('hex')), 'a72f7b7c4725dfc18df87752598932a0ed40bbef839691b06a4c88c5527315c5');
assert.equal(blake2b(Buffer.from('abc'), { outlen: 32 }).length, 32);
});
test('decodeFinalitySection reads votes, certificates, evidence and the reveal from a coinbase payload', () => {
const reveal = Buffer.from('2.1.0/IGNK' + PUB.toString('hex') + 'ef'.repeat(96));
const bm = Buffer.from([0b00001011]); // positions 0, 1, 3
const agg = fill(32, 0x22);
const ev = Buffer.concat([Buffer.from([3]), vote(7, CP, PUB).subarray(1), vote(7, fill(32, 0x33), PUB).subarray(1)]);
const extra = Buffer.concat([reveal, section([vote(5, CP, PUB), cert(5, CP, 4, bm, agg), ev])]);
const d = decodeFinalitySection(coinbase(extra));
assert.equal(d.votes.length, 1); assert.equal(d.votes[0].index, 5); assert.equal(d.votes[0].pubkey, PUB.toString('hex')); assert.equal(d.votes[0].checkpoint, CP.toString('hex'));
assert.equal(d.certificates.length, 1);
const c = d.certificates[0];
assert.equal(c.index, 5); assert.equal(c.voterCount, 4); assert.deepEqual(c.signerPositions, [0, 1, 3]); assert.equal(c.aggregator, agg.toString('hex'));
assert.equal(d.evidence.length, 1); assert.equal(d.evidence[0].first.index, 7); assert.notEqual(d.evidence[0].first.checkpoint, d.evidence[0].second.checkpoint);
assert.equal(d.reveal.pubkey, PUB.toString('hex'));
assert.equal(d.malformed, false);
// raw extra data decodes the same; a hex string too
assert.equal(decodeFinalitySection(extra, { raw: true }).certificates.length, 1);
assert.equal(decodeFinalitySection(coinbase(extra).toString('hex')).votes.length, 1);
});
test('decodeFinalitySection: a bitmap bit beyond voter_count is ignored, a payload without a section is empty, a truncated section yields what decoded', () => {
const bm = Buffer.from([0b11111111]);
const c = decodeFinalitySection(coinbase(section([cert(1, CP, 3, bm, fill(32, 0))]))).certificates[0];
assert.deepEqual(c.signerPositions, [0, 1, 2]);
assert.deepEqual(decodeFinalitySection(coinbase(Buffer.from('2.1.0/'))), { votes: [], certificates: [], evidence: [], reveal: null, sectionBytes: 0, malformed: false });
const good = vote(1, CP, PUB), bad = Buffer.concat([Buffer.from([2]), u64(1), CP, u32(9), u32(5000)]);
const body = Buffer.concat([good, bad]);
const d = decodeFinalitySection(Buffer.concat([body, u32(body.length), Buffer.from('IGNF')]), { raw: true });
assert.equal(d.votes.length, 1); assert.equal(d.malformed, true);
assert.equal(coinbaseExtra(Buffer.alloc(5)).length, 0);
});
test('voterTableAt rebuilds the node rule: window, merger, dust, sort order, bans', () => {
const K = (c) => c.repeat(64);
const blues = [
// daa, key, mergerDaa
{ hash: 'c', daaScore: 100, voteKeyHash: K('b'), mergerDaa: 101 }, // the checkpoint itself (merged by its child)
{ hash: 'x1', daaScore: 99, voteKeyHash: K('b'), mergerDaa: 100 },
{ hash: 'x2', daaScore: 98, voteKeyHash: K('a'), mergerDaa: 100 },
{ hash: 'x3', daaScore: 97, voteKeyHash: K('a'), mergerDaa: 100 },
{ hash: 'x4', daaScore: 95, voteKeyHash: K('a'), mergerDaa: 103 }, // merged after C: not in C's past
{ hash: 'x5', daaScore: 90, voteKeyHash: K('a'), mergerDaa: 100 }, // window (90, 100]: 90 is out
{ hash: 'x6', daaScore: 96, voteKeyHash: K('d'), mergerDaa: 100 }, // one block: dust
];
const t = voterTableAt({ hash: 'c', daaScore: 100 }, blues, { weightWindow: 10, dust: 2 });
assert.equal(t.perKey.get(K('a')), 2); assert.equal(t.perKey.get(K('b')), 2); assert.equal(t.perKey.get(K('d')), 1);
assert.deepEqual(t.voters, [K('a'), K('b')]); assert.equal(t.total, 4); assert.equal(t.keys, 3);
const banned = voterTableAt({ hash: 'c', daaScore: 100 }, blues, { weightWindow: 10, dust: 2 }, new Map([[K('a'), 200]]));
assert.deepEqual(banned.voters, [K('b')]);
const expired = voterTableAt({ hash: 'c', daaScore: 100 }, blues, { weightWindow: 10, dust: 2 }, new Map([[K('a'), 100]]));
assert.deepEqual(expired.voters, [K('a'), K('b')]);
});
test('topShares and signingShares', () => {
const s = topShares(new Map([['a', 50], ['b', 30], ['c', 20]]));
assert.equal(s.keys, 3); assert.equal(s.total, 100); assert.equal(s.top1, 0.5); assert.equal(s.top3, 1); assert.equal(s.top10, 1);
assert.deepEqual(topShares(new Map()), { keys: 0, total: 0, top1: 0, top3: 0, top10: 0 });
const tables = new Map([[5, { hash: 'h5', voters: ['k1', 'k2', 'k3'], perKey: new Map([['k1', 10], ['k2', 20], ['k3', 30]]) }]]);
const r = signingShares([
{ index: 5, checkpoint: 'h5', voterCount: 3, signerPositions: [0, 2] },
{ index: 5, checkpoint: 'h5', voterCount: 3, signerPositions: [2] },
{ index: 5, checkpoint: 'other', voterCount: 3, signerPositions: [0] },
{ index: 5, checkpoint: 'h5', voterCount: 4, signerPositions: [0] },
{ index: 6, checkpoint: 'h6', voterCount: 3, signerPositions: [0] },
], tables);
assert.equal(r.mapped, 2); assert.equal(r.unmapped.length, 3);
assert.equal(r.weight.get('k1'), 10); assert.equal(r.weight.get('k3'), 60); assert.equal(r.count.get('k3'), 2);
assert.match(r.unmapped[0].reason, /another block/); assert.match(r.unmapped[1].reason, /voter_count 4/); assert.match(r.unmapped[2].reason, /no table/);
});
test('independenceClasses: the X5 reading (21 keys on 5 machines) and the silent-fleet rule', () => {
const keys = []; const attrs = new Map();
for (let i = 0; i < 21; i++) { const k = `k${i}`; keys.push({ keyHash: k, blocks: 100 }); attrs.set(k, { asn: 'AS1', fingerprint: `m${i % 5}`, pool: null }); }
keys.push({ keyHash: 'dust', blocks: 1 });
const r = independenceClasses(keys, attrs, 5);
assert.equal(r.nInd, 5); assert.equal(r.eligible, 21); assert.equal(r.unattributed, 0);
// the same machines behind a pool statement are still 5 classes; a second pool on the same machines is 10
for (const [k, a] of attrs) a.pool = 'p1';
assert.equal(independenceClasses(keys, attrs, 5).nInd, 5);
attrs.get('k0').pool = 'p2';
assert.equal(independenceClasses(keys, attrs, 5).nInd, 6);
// silent keys (no fingerprint): own class only when the ASN is theirs alone
const silent = [{ keyHash: 's1', blocks: 10 }, { keyHash: 's2', blocks: 10 }, { keyHash: 's3', blocks: 10 }];
const sa = new Map([['s1', { asn: 'AS9', fingerprint: null, pool: null }], ['s2', { asn: 'AS9', fingerprint: null, pool: null }], ['s3', { asn: 'AS10', fingerprint: null, pool: null }]]);
const rs = independenceClasses(silent, sa, 5);
assert.equal(rs.nInd, 2);
// a key with nothing known counts as its own class and is reported as unattributed
const none = independenceClasses([{ keyHash: 'n1', blocks: 10 }, { keyHash: 'n2', blocks: 10 }], new Map(), 5);
assert.equal(none.nInd, 2); assert.equal(none.unattributed, 2);
});
test('pool statement: sign, parse, verify, and every refusal', () => {
const { publicKey, privateKey } = generateKeyPairSync('ed25519');
const pubHex = publicKey.export({ format: 'der', type: 'spki' }).subarray(-32).toString('hex');
const keys = ['B'.repeat(64), 'a'.repeat(64)];
const signed_at = new Date(Date.now() - 3600e3).toISOString();
const body = { pool: 'ember-pool', keys, signed_at };
const sig = edSign(null, poolStatementBytes(body), privateKey).toString('hex');
const text = JSON.stringify({ format: 'igneum-pool-statement-1', ...body, pubkey: pubHex, sig });
const p = parsePoolStatement(text);
assert.equal(p.ok, true, p.error);
assert.deepEqual(p.statement.keys, ['a'.repeat(64), 'b'.repeat(64)]);
const registry = { 'ember-pool': pubHex };
assert.deepEqual(verifyPoolStatement(p.statement, registry), { ok: true });
assert.deepEqual(verifyPoolStatement(p.statement, new Map([['ember-pool', pubHex]])), { ok: true });
// tampered keys
const t = { ...p.statement, keys: [...p.statement.keys, 'c'.repeat(64)] };
assert.match(verifyPoolStatement(t, registry).error, /does not verify/);
// unknown pool, wrong registered key, stale, future
assert.match(verifyPoolStatement({ ...p.statement, pool: 'other' }, registry).error, /no registered key/);
assert.match(verifyPoolStatement(p.statement, { 'ember-pool': 'ff'.repeat(32) }).error, /the pool's key/);
assert.match(verifyPoolStatement(p.statement, registry, { now: Date.now() + 40 * 86400e3 }).error, /over the 35-day limit/);
assert.match(verifyPoolStatement({ ...p.statement, signed_at: new Date(Date.now() + 3600e3).toISOString() }, registry).error, /future/);
// shape refusals
assert.match(parsePoolStatement('nope').error, /not JSON/);
assert.match(parsePoolStatement({ format: 'x' }).error, /format/);
assert.match(parsePoolStatement({ format: 'igneum-pool-statement-1', pool: 'Bad Pool' }).error, /pool must be/);
assert.match(parsePoolStatement({ format: 'igneum-pool-statement-1', pool: 'p', keys: [] }).error, /non-empty/);
assert.match(parsePoolStatement({ format: 'igneum-pool-statement-1', pool: 'p', keys: ['zz'] }).error, /not a 64-hex/);
assert.match(parsePoolStatement({ format: 'igneum-pool-statement-1', pool: 'p', keys: ['a'.repeat(64), 'A'.repeat(64)] }).error, /repeat/);
assert.match(parsePoolStatement({ format: 'igneum-pool-statement-1', pool: 'p', keys: ['a'.repeat(64)], signed_at: 'yesterday' }).error, /ISO 8601/);
assert.match(parsePoolStatement({ format: 'igneum-pool-statement-1', pool: 'p', keys: ['a'.repeat(64)], signed_at, pubkey: 'x' }).error, /pubkey/);
assert.match(parsePoolStatement({ format: 'igneum-pool-statement-1', pool: 'p', keys: ['a'.repeat(64)], signed_at, pubkey: pubHex, sig: 'x' }).error, /sig/);
});
test('asn table: longest prefix, private ranges, misses, IPv6', () => {
const table = parseAsnTable('# comment\n10.0.0.0/8 AS-IGNORED private anyway\n203.0.113.0/24 AS64500 Example Net\n203.0.0.0/16 AS64501 Wider\nbad line\n2001:db8::/32 AS1 v6-not-read\n');
assert.equal(table.length, 3);
assert.deepEqual(asnOf('203.0.113.9:26611', table), { asn: 'AS64500', name: 'Example Net' });
assert.deepEqual(asnOf('203.0.7.1', table), { asn: 'AS64501', name: 'Wider' });
assert.equal(asnOf('198.51.100.1:1', table), null);
assert.deepEqual(asnOf('192.168.68.67:26611', table), { asn: 'local', name: 'private or loopback' });
assert.deepEqual(asnOf('127.0.0.1:26611', table), { asn: 'local', name: 'private or loopback' });
assert.equal(asnOf('[2001:db8::1]:26611', table), null);
assert.equal(asnOf('', table), null);
assert.equal(parseAsnTable('').length, 0);
});
test('nightly row: the four shares and N_ind labelled proposed', () => {
const pk = PUB.toString('hex');
const weights = { params: { dust: 5 }, checkpointIndex: 4500, daaScore: 138000, keys: [
{ keyHash: 'a72f7b7c4725dfc18df87752598932a0ed40bbef839691b06a4c88c5527315c5', blocks: 60, voter: true },
{ keyHash: 'k2', blocks: 40, voter: true }, { keyHash: 'k3', blocks: 2, voter: false }] };
const checkpoints = [{ state: 'locked', certificateAggregator: 'agg1' }, { state: 'locked', certificateAggregator: '0'.repeat(64) }, { state: 'proposed', certificateAggregator: 'agg2' }];
// stored as live_certificates stores them: voters with pubkey and weight, in canonical order; bitmap bits 0 and 1
const certificates = [
{ index: 1, bitmap_hex: '03', voter_count: 2, voters: [{ pubkey: pk, weight: 60 }, { keyHash: 'k2', weight: 40 }] },
{ index: 2, bitmap_hex: '01', voter_count: 2, voters: [{ pubkey: pk, weight: 60 }, { keyHash: 'k2', weight: 40 }] },
{ index: 3, bitmap_hex: '01', voter_count: 3, voters: [{ pubkey: pk, weight: 60 }] }, // table length disagrees: skipped
];
const sw = signedWeightFromStored(certificates);
assert.equal(sw.used, 2); assert.equal(sw.skipped, 1);
assert.equal(sw.weight.get('a72f7b7c4725dfc18df87752598932a0ed40bbef839691b06a4c88c5527315c5'), 120); assert.equal(sw.weight.get('k2'), 40);
const row = nightlyRow({ weights, checkpoints, certificates, proofs: new Map([['p1', 3], ['p2', 1]]), attributes: new Map([['k2', { asn: null, fingerprint: 'm1', pool: null }]]), meta: { nodeVersion: '2.1.0', day: '2026-10-06' } });
assert.equal(row.day, '2026-10-06'); assert.equal(row.daa, 138000); assert.equal(row.node_version, '2.1.0');
assert.equal(row.hashing.top1, 60 / 102); assert.equal(row.hashing.above_dust, 2);
assert.equal(row.signing.top1, 120 / 160); assert.equal(row.signing.certificates, 2); assert.equal(row.signing.skipped, 1);
assert.equal(row.proving.top1, 0.75); assert.equal(row.aggregation.keys, 1); assert.equal(row.aggregation.anonymous, 1);
assert.equal(row.n_ind, 2); assert.equal(row.n_ind_definition, 'proposed'); assert.equal(row.n_ind_eligible, 2); assert.equal(row.n_ind_unattributed, 1);
});