igneum/docs/plans/mission-item-12-peer-directory.md
2026-10-07 13:56:48 +00:00

12 KiB

Mission item 12: the weight-backed peer directory, prototype (status log)

Lane: horizon (mission items 4 and 12), 7 October 2026, after item 4's gate line. Branches: peer-directory (this repo, on master) and peer-directory-node (the fork, on release-0.3.20-node dc141409; targets 0.3.21). Times UK (BST) unless marked Z.

The item (mission.md 2.12, invent.md 7.1): a coinbase section where a key above dust may publish its node's address; a fresh node dials from that list weighted by 30-day weight, beside the DNS seeds. Gate: a fresh node with genesis peers only reaches 8 outbound from the directory; a 34 percent attacker eclipses under 0.1 percent of 1,000 fresh starts (simulation is fine); the NAT fraction measured on the fleet (owed to the fleet lane, asked 7 October 13:0x UK; nothing rented). Dropped if the NAT fraction makes the list useless: this lane reports, it does not decide.

1. What exists to build on (read before writing)

Where What Used how
fork consensus/core/src/finality.rs FinalityItem (tags 1 vote, 2 certificate, 3 evidence, 4 leave), encode_section / decode_section (`items len
fork consensus/src/processes/finality.rs ingest_leave (block-carried, dated by the lowest carrier), template_candidates / section_from (what a template carries), leave_active (switch by DAA), FinalityState.leaves (persisted) the ingest, the carriage and the switch pattern the directory is a map key hash to (address, carrier DAA), persisted in the state the same way
fork components/connectionmanager/src/lib.rs handle_outbound_connections dials from the address manager's prioritised random iterator, then the DNS seeders when connections are still missing the directory draw goes between the two: addresses drawn by weight from the directory are fed to the address manager and dialled before the seeders are asked
fork components/addressmanager/src/lib.rs add_address, iterate_prioritized_random_addresses (weighted by failure count and prefix bucket) the store a dialled address lives in directory addresses enter it like seeder answers; the weight decides which enter, the store's own rule decides the order
fork consensus/core/src/config/params.rs consensus_digest (a switch at never is outside the digest; set, it is in) the activation pattern every 0.3.16 switch follows peer_directory_activation_daa, never on every network

2. Design, as built (prototype; fork branch peer-directory-node on release-0.3.20-node dc141409, commits 0cad5106 and db28d331 (tests); repo branch peer-directory on master 819d536b, commits 9a64d18c and 3e0dfa17 (gate), pushed to origin)

Item Rule
Wire AddressAnnounce { daa, ip: [u8; 16], port, pubkey, signature } (core finality.rs), 171 B, signed by the vote key the header names under IGNEUM_ADDR_V1 over `"igneum-addr-v1/"
Carriage in the MINER's extra data as `IGNP
Reading FinalityManager::on_block_body at or above peer_directory_activation_daa: the announcement must be by the block's own key, verify, and name a routable-in-form address; one node-local entry per key, the newest carrier wins (not persisted: a restarted node refills it from the blocks it sees; a prototype's gap)
Report ConsensusApi::peer_directory() rows with each key's weight at the latest determined checkpoint (0 under dust); entries older than a weight window dropped on the way
Draw ConsensusApi::peer_directory_draw(n, exclude): without replacement, proportional to weight, keys under dust never drawn
Dialling ConnectionManager::handle_outbound_connections: after the address store's own iterator and before the DNS seeders, when connections are still missing, 2 x missing addresses from the draw enter the address store and are dialled at once (DirectoryDraw closure wired in protocol/flows/src/service.rs through the consensus's blocking session)
Switch peer_directory_activation_daa, never on every network (the four network constants), in the digest once set, in OverrideParams and the 60x file
Not done the RPC that lists the directory (the harness reads getConnectedPeerInfo and the node log instead); persistence across restarts; Ember and the phone's use of the list (invent.md 7.1's third hour)

3. Gate plan

Gate How
a fresh node with genesis peers only reaches 8 outbound from the directory fast-time network of 10 listing nodes (each announcing its loopback address) plus one fresh node started with --outpeers=8, --nodnsseed and one --addpeer to a genesis peer that serves blocks only; the fresh node's getConnectedPeerInfo shows 8 outbound within 5 min, all from the directory
a 34 percent attacker eclipses under 0.1 percent of 1,000 fresh starts sim/peer-directory/eclipse.py: the draw as implemented (weighted, without replacement, 8 peers) over a table where the attacker's keys hold 34 percent of the weight and list 34 percent of the addresses; 1,000 starts; expected 1.8e-4 at 8 peers (invent.md model E); also 16 peers and the honest-majority-of-peers reading
the NAT fraction RECEIVED from the fleet lane (ac055d60427caab99) at 13:1x UK, read at 12:06Z from each standing live-devnet node's own log, nothing rented: reachable from outside 2 of 14 (hub-1, RunPod, 26611 mapped: 2,844 inbound peer connections against 86 outbound; pool-1, RunPod, mapped 4463: 1,692 against 99); not reachable 12 of 14 (every Vast box: 0 inbound peer connections each, 96 to 542 outbound, no --externalip, no mapped port). So 86 percent of the fleet's nodes sit behind NAT and hold their 4 to 5 peers by dialling out; the two reachable nodes carry every inbound connection. Caveat (theirs): a rented fleet overstates the NAT share for datacentre operators and understates it for home miners; a fair read for "a node started with the defaults". Left out: the four Devnet 2 pods and the Hetzner seed. The marker was "Connected to incoming peer" against "Connected to outgoing peer" (the RPC server's local accepts are not peers).

4. Runs

The eclipse simulation (sim/peer-directory/eclipse.py, box 2, nice 10, 13:5x to 14:0x UK)

The draw as implemented (without replacement, proportional to weight at the latest checkpoint), 200 honest keys with Zipf weights, the attacker at 34 percent of the weight split over m keys of equal weight, seed 7.

starts peers attacker keys eclipsed rate model E a^n attacker majority of the peers
1,000 8 8 0 0 1.79e-4 4.0 percent
1,000 8 64 1 1.0e-3 1.79e-4 9.8 percent
1,000 8 1,000 0 0 1.79e-4 10.7 percent
1,000 16 8, 64, 1,000 0, 0, 0 0 3.19e-8 0, 9.2, 9.2 percent
100,000 8 8 0 0 1.79e-4 4.8 percent
100,000 8 64 14 1.4e-4 1.79e-4 10.7 percent
100,000 8 1,000 22 2.2e-4 1.79e-4 11.6 percent

The gate's line (under 0.1 percent of 1,000 fresh starts) is met at 100,000 starts: 1.4e-4 and 2.2e-4, where 1,000 starts cannot resolve it (1 in 1,000 is exactly the line). Two readings beyond model E: with fewer attacker keys than peers an eclipse is impossible under the draw without replacement (the dust threshold, 100 blocks a window on mainnet, prices each attacker key at 100 blocks), and the attacker holds a MAJORITY of a fresh node's peers in about 11 percent of starts at 8 peers, which is the number that matters for a relay-level attack rather than a full eclipse; 16 peers takes that to 9 percent and the eclipse to zero in 1,000.

Unit tests (box 2)

Test Result
core address_announce_round_trips_and_verifies_under_its_key_only 1 of 1
node the_peer_directory_lists_a_blocks_own_key_at_its_newest_address_only_when_switched_on (known-failed first: the switch at never lists nothing) 1 of 1
cargo check -p kaspad -p igneum-miner --features kaspad/igneum-pow on the branch ok

The fast-time gate (box 2, tools/fast-time-remote.sh --box 2, binary of peer-directory-node 0cad5106, 13:27 to 13:34 UK)

infra/fast-time/peer-directory.mjs: ten listing nodes at one CPU thread each, every miner announcing its node's loopback p2p address, every node advertising an unroutable external ip (10.255.0.0/16) so ordinary address gossip hands a fresh node dead addresses only; after 200 s a fresh node starts with --outpeers=8 and one --addpeer to node 0 and is watched 180 s.

Case Must Got Numbers
switch off, expect one (the known-failed case) PASS PASS node 0 logged 0 listings; the fresh node held 1 outbound (node 0) for the whole watch, 0 draw lines, synced 448 blocks
switch off, expect eight (the harness's own failed shape) FAIL FAIL 1 outbound, 0 from the directory
switch on, expect eight (THE GATE LINE) PASS PASS node 0 logged 10 listings; the fresh node reached 8 outbound at 231.7 s (about 30 s after it started), 9 connections from the directory in one draw, synced 414 blocks

Three harness faults on the way, each fixed: the log directory missing on the box (every case died at the redirect); the peer count read isOutbound where the fork's RpcPeerInfo is is_outbound; and the wrapper's first run checking the worktree root out on the box (fixed on horizon, 10140f9f, carried here).

5. Verdict line for main

The prototype does what invent.md 7.1 asked, on the numbers: a fresh node with one genesis peer and dead gossip reached 8 outbound from the directory in about 30 s; the eclipse by a 34 percent attacker is 1.4e-4 to 2.2e-4 at 8 peers over 100,000 starts (under the 0.1 percent line; 0 with 8 or fewer attacker keys, since the draw is without replacement), and the attacker holds a majority of a fresh node's peers in about 11 percent of starts, which is the number to watch. The NAT reading (2 of 14 standing nodes reachable, 86 percent behind NAT with no inbound path) makes the list a SEED SUPPLEMENT, not a replacement: today it would list the two reachable fleet nodes beside the seeds, and a home miner's node (the 12-of-14 class unless the app maps a port) is a reader of the list, never a listing. Per tier: a home miner's node gains weight-backed peers beside the seeds at no cost and lists nothing by default; a rig or a pool node with a mapped port opts in with --announce and 171 bytes a block; the phone and Ember verify mode are not wired (not done). Listed as useful or dropped is main's and the project lead's call; this lane's reading is "keep as a seed supplement behind its switch", which costs 1.7 MB a day per node at 10,000 listing keys (approximate, from the item size) and nothing while the switch is never.

Open: an RPC that lists the directory; persistence across restarts; Ember and the phone; the testnet object and every network file carry no peer_directory field (the switch stays never until a cut sets it).

6. Rebase for 0.3.21 (14:0x UK, the shipper's order)

peer-directory-node rebased onto release-0.3.20-node c4459193 in one round, no conflict: tip 2e32d5f6, on both box mirrors under its name. Suite on build-2 at 2e32d5f6: igneum-miner 19 of 19, connectionmanager 0 tests, consensus lib 116 of 117 (the ban flake only; cargo stopped before consensus-core's target). Digest on tools/fleet/override.json: 7bd98cc4..., the same as the pin's binary. Handed to the node lane a283f5f0d364ceef0.

Digest on the file hub-1 runs (/root/fleet/override.json, sixteen fields, the copy at /tmp/igneum-devnet/override-v3-live.json on build-1): eada4bda8aa8368c2b2c3d17744bc7a70a0ff0e996dad681884d3ac5de1207eb, the shipper's string, read 14:1x UK from the branch binary on build-2; 7bd98cc4 was master's tools/fleet/override.json, another file. The live digest is unchanged by this branch.