igneum-network/igneum
1
0
Fork 0
Code Issues Pull requests Releases Activity

exec sync: the plan (what happened, the fork commits, the copy measurements, the per-tier consequence) and the bench-log entry; the exec-sync harness

Browse source 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
igneum-labs 2026-10-06 13:46:31 +00:00
parent b06f7ae09c
commit 72a0a05cf4
3 changed files with 199 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

15
docs/bench-log.md
View file

@ -1884,3 +1884,18 @@ Measurement (`/Users/joshm/Projects/igneum/tools/lock/with-lock.sh measure bash
| v1 shard | cpu (no feature, `target/cpu`) | 20 | 10,049,917; 11; 0.31 | 585.8 (587.71) | 11; 11,534,336; 1,093,455; 390,964 | 10,583,965,696 (10.58) | 9,005.0 | 223,882 (222,668) | 0.010 | 0.21 / 0.009 to 0.012 / 32.9 MB, three times | yes |
Reading. (1) The same statement costs RISC Zero 10,049,917 user cycles against SP1's 4,717,439 (bench-log 5 October, the S_p curve), 2.13x, with 10.7% more in paging and reserved cycles at po2 20. (2) On this CPU the v1 shard takes 574 s at po2 20; SP1's CPU prover on the same Mac under the same lock this morning took 71.7 s for the same shard (the proof-system watch, `bench/proof-systems/rows/20261006-1146-MacBook-Pro-sp1.json`, another agent's row): 8.0x. (3) The succinct receipt is 223,882 bytes against SP1's 1,272,897-byte compressed proof, 5.7x smaller, and the same size at every po2 and for the empty shard. (4) The verify is 9 to 10 ms in process; the whole verify mode is 0.21 s, of which 0.18 to 0.19 s is `compute_image_id` over the 3.67 MB program binary at start (the SP1 host's verify mode ran 0.26 s on the watch's row). (5) Memory against the segment limit: 8.3 to 8.4 GB of RSS at po2 19 and for any one-segment shard, 10.7 GB at po2 20, 20.4 GB at po2 21; the limit is the dial, and po2 19 costs 22% more time (24 segments instead of 11). (6) The empty shard is one 2^19 segment and still 47.9 s: the per-receipt floor of segment prove plus lift to a succinct receipt on this CPU. (7) `/usr/bin/time` reports 15.7 of the 18 cores busy over the po2 20 run (9,035 s user in 575 s wall). (8) The ratio `metal` build to CPU build on the v1 shard at po2 20 is 574.4 s to 585.8 s, 0.98: the same code, run-to-run noise, as the sources say. (9) Nothing here is a GPU number: PC 2 (CUDA) measures the GPU memory and time; the Mac's Metal number does not exist in 3.0.6.
### 6 October 2026, 13:17Z to 13:45Z, the exec restart on a copy of node 1's data dir (Mac, `tools/lock/with-lock.sh run`)
`cp -c -R /tmp/igneum-devnet/node1 /tmp/igneum-devnet/node1-copy`, the exec-sync node (branch exec-sync-0313) on ports 29990+ with node 1's override file plus `exec_restart_number` 27276, `exec_restart_hash` bb45cf0d..., `exec_restart_trust_daa` 200000 (the script `node1-copy-test.sh`, every figure a RESULT line):
| Figure | Value |
|---|---|
| Chain facts | sink chain block 130,073 (later 130,272); pruning point = retention root = chain block 27,276, DAA 45,537, body held; bodies below gone |
| Replay from 27,276 to the sink | 92 s (run 4, no trust rule), 93 s (run 5, trust rule) wall from the node's start |
| Paid shards without the trust rule | 0 (every record vetoed by the native-statement check) |
| Paid shards with the trust rule | 1,482, 1,825.699 IGN |
| PC 2's payout balance at the tip | 265,359.6 IGN (run 4), 267,648.4 IGN (run 5) |
| Persisted file | 114,375,262 B then 114,830,936 B; 31 accounts; 1,200 full records |
| Resume after a restart | 8 s and 9 s to answer, startedFrom "snapshot", no replay |

47
docs/plans/exec-sync.md Normal file
View file

@ -0,0 +1,47 @@
# Exec sync: the executor's state after the pruning point left genesis (6 October 2026)
## What happened
The devnet's pruning point left genesis on 6 October 2026 at about 11:40Z (chain block 27,276, DAA 45,537). Every
node pruned the block bodies and acceptance rows below it and kept the headers and ghostdag rows. The execution
layer's state lived only in memory and was rebuilt from genesis at every start; every node had restarted for
0.3.11 and 0.3.12. From 11:40Z every node read `eth_blockNumber` 0x0 and `igneum_getProvingStatus` active false:
no balances, no proving work list, no payouts, mining untouched. No hand ran `--archival`, so no data dir on the
network holds the bodies the state was built from, and the state cannot be rebuilt from genesis anywhere.
## The fix, fork branch `exec-sync-0313` (off the 0.3.12 node 83089544), node only
| Commit | What |
|---|---|
| 4f05e56a | the exec state persisted to the data dir every 5 minutes and at stop (two generations; resumed at start when its tip is still a chain block); `--igneum-exec-snapshot=<path>[,<sha256>]` and `igneum_exportExecSnapshot`; the loud status when the follower cannot execute (warn every minute, `igneum_getExecStatus.blocked`, the proving status's `execSync`) |
| 3bd31a20 | the snapshot served and fetched over p2p at protocol 16 (messages 76 and 77, 1 MiB chunks, the sha256 on every chunk); a blocked executor asks one peer every 20 s; `--igneum-exec-snapshot=peer` |
| efa6924d, 7cb712f5, 80674943 | the archival walk: the selected-parent chain read from the ghostdag store when the virtual-chain query refuses a tip below the retention root; the mergeset order rebuilt from ghostdag rows when the acceptance row is pruned |
| 3dd9b2c9 | the exec restart: `exec_restart_number` and `exec_restart_hash` in the override object (in the digest once set): header-only records below R, the EVM state fresh at R, executed from R's body on; the status reports the pruning point and the retention root with their chain block numbers |
| 05e93f0e | `exec_restart_trust_daa`: a shard record carried below it pays as carried, without the native-statement veto and the assignee check |
## Measured on a copy of node 1's data dir (the Mac, `tools/lock/with-lock.sh run`)
| Figure | Value |
|---|---|
| The pruning point and retention root, node 1 and the observer | chain block 27,276, hash bb45cf0dd2d7cc97ebfa5a2701527c09a8ede5d32de74efead9caa293b15688a, DAA 45,537; its body held on both |
| Bodies below it | gone ("cannot find full block" at chain block 1) |
| Replay from 27,276 to the sink (130,073, then 130,272) | 92 s and 93 s wall from the node's start, over 2,000 chain blocks a second |
| Without the trust rule | paidShards 0: every historical record vetoed (the records attest the original state roots) |
| With `exec_restart_trust_daa` 200,000 | paidShards 1,482, paidWei 1,825.70 IGN (node 1 read 1,261 and 1,573 at 08:30Z; the chain moved on) |
| PC 2's payout address at the tip | 267,648 IGN |
| The persisted file | 114,830,936 bytes, 130,273 records (1,200 full), 31 accounts |
| Resume after a restart | 9 s, no replay |
## What it means
| Tier | Before the fix | After the cut with the three fields |
|---|---|---|
| Every node, home miner, rig, pool | an empty EVM since 11:40Z: no balances, no proving, no payouts; mining fine | the chain's state from chain block 27,276 on, rebuilt in about 90 s at the switch; every restart after that resumes in seconds |
| Miners | rewards invisible | rewards since R back (PC 2: 267,648 IGN); the rewards of chain blocks 0 to 27,275 (the devnet's first 12 hours) lost, the project lead's call on seeding them |
| Provers | payouts invisible | every carried record since proving v0 paid as carried (1,482 shards); new proofs verify against the new state from the trust DAA on |
| Joiners today | an empty EVM for ever | IBD gives the pruning point 27,276 and the bodies from it, so a fresh node replays the same 90 s with no snapshot; once the pruning point moves past R the p2p snapshot (protocol 16) starts it |
## Open
The fresh-join time on a rented box (in progress); the trust DAA's value (at or above the switch); the first 12
hours' rewards (the project lead); the p2p snapshot's fast-time harness case; `--archival` on at least one hand from now on.

137
tools/exec-sync/net.mjs Normal file
View file

@ -0,0 +1,137 @@
#!/usr/bin/env node
// Exec sync harness (6 October 2026): the executor's persisted state and the snapshot file, on a private fast-time
// simnet (ports 29970+, suffix 957; never the live devnet). Cases, each asserted:
// 1. known-finished: node A mines past N chain blocks; `igneum_exportExecSnapshot` writes a file; its tip, state
// root and sha256 are reported; A is stopped (the state is persisted at stop) and started again: it resumes from
// the data dir's snapshot (startedFrom "snapshot", snapshotTip = the persisted tip) and keeps executing
// 2. a fresh node B with --igneum-exec-snapshot=<file>,<sha256> and no bodies of its own beyond what it syncs:
// startedFrom "snapshot", snapshotTip = the file's tip, eth_blockNumber reaches A's tip, every miner balance
// and the state root at the file's tip equal A's
// 3. known-failed: a fresh node C with the file and a WRONG pin ignores the file (startedFrom "genesis" here, since
// a simnet peer still serves every body) and says why in its log
// 4. a fresh node D with the file and NO consensus data of its own but the file's tip unknown: refused with
// "unknown to consensus" (the file's tip is from another chain: a second simnet)
// Usage: node tools/exec-sync/net.mjs [--blocks 60]
// IGNEUM_EXEC_BIN=<dir with igneumd and igneum-miner> (default vendor/igneum-node/target-exec-sync/release)
import { spawn, spawnSync } from 'node:child_process';
import { existsSync, mkdirSync, readFileSync, rmSync, writeFileSync, openSync } from 'node:fs';
import { createHash } from 'node:crypto';
const ROOT = new URL('../../', import.meta.url).pathname;
const REL = process.env.IGNEUM_EXEC_BIN || `${ROOT}vendor/igneum-node/target-exec-sync/release`;
const IGNEUMD = `${REL}/igneumd`;
const MINER = `${REL}/igneum-miner`;
const TMP = '/tmp/igneum-exec-sync';
const BASE = 29970, SUFFIX = 957;
const flag = (name, d) => { const i = process.argv.indexOf(name); return i >= 0 ? Number(process.argv[i + 1]) : d; };
const BLOCKS = flag('--blocks', 60);
const FILE = `${ROOT}infra/fast-time/override-60x.json`;
for (const b of [IGNEUMD, MINER]) if (!existsSync(b)) { console.error(`missing ${b}`); process.exit(2); }
rmSync(TMP, { recursive: true, force: true }); mkdirSync(TMP, { recursive: true });
const override = `${TMP}/override.json`;
// the exec-sync node is cut from 0.3.12 and knows no proof-system-2 field: drop it as text (JSON.parse would turn
// the u64::MAX "never" values into floats the node refuses)
writeFileSync(override, readFileSync(FILE, 'utf8').replace(/\s*"proving_v2_activation_daa":\s*\d+,?/, ''));
const t0 = Date.now();
const log = (m) => console.log(`${new Date().toISOString().slice(11, 23)} t=${((Date.now() - t0) / 1000).toFixed(1)}s ${m}`);
const sleep = (ms) => new Promise(r => setTimeout(r, ms));
const hexn = (h) => Number(BigInt(h));
const started = [];
class Node {
constructor(i, connect = [], extra = [], suffix = SUFFIX) {
this.i = i; this.grpcPort = BASE + i * 10; this.p2pPort = BASE + i * 10 + 1; this.jsonPort = BASE + i * 10 + 2; this.evmPort = BASE + i * 10 + 3;
this.connect = connect; this.extra = extra; this.suffix = suffix; this.dir = `${TMP}/n${i}`; this.logFile = `${this.dir}/node.log`;
}
get evm() { return `http://127.0.0.1:${this.evmPort}`; }
async start() {
mkdirSync(this.dir, { recursive: true });
const a = ['--devnet', `--devnet-suffix=${this.suffix}`, '--nodnsseed', '--disable-upnp', '--nologfiles', '--enable-unsynced-mining', '--utxoindex', '--unsaferpc',
`--appdir=${this.dir}`, `--rpclisten=127.0.0.1:${this.grpcPort}`, `--rpclisten-json=127.0.0.1:${this.jsonPort}`, `--evm-rpclisten=127.0.0.1:${this.evmPort}`,
`--listen=127.0.0.1:${this.p2pPort}`, `--override-params-file=${override}`, '--loglevel=info', '--yes', ...this.connect.map(c => `--addpeer=${c}`), ...this.extra];
const out = openSync(this.logFile, 'a');
this.proc = spawn(IGNEUMD, a, { stdio: ['ignore', out, out], env: { ...process.env, IGNEUM_PROOF_VERIFY: 'trust' } });
started.push(this.proc);
for (let k = 0; k < 120; k++) { try { await this.eth('eth_chainId'); return this; } catch { await sleep(500); } }
throw new Error(`node ${this.i} did not answer on ${this.evm}`);
}
async stop() { try { this.proc.kill('SIGINT'); } catch { } for (let k = 0; k < 60; k++) { if (this.proc.exitCode !== null) return; await sleep(500); } try { this.proc.kill('SIGKILL'); } catch { } }
async eth(method, params = []) {
const r = await fetch(this.evm, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ jsonrpc: '2.0', id: 1, method, params }) });
const j = await r.json(); if (j.error) throw new Error(`${method}: ${JSON.stringify(j.error)}`); return j.result;
}
logText() { try { return readFileSync(this.logFile, 'utf8'); } catch { return ''; } }
}
function mine(node, label) {
const out = openSync(`${TMP}/miner-${label}.log`, 'a');
const p = spawn(MINER, ['mine', `127.0.0.1:${node.grpcPort}`, '--label', label, '--payout', '0x4343434343434343434343434343434343434343', '--threads', '1'], { stdio: ['ignore', out, out] });
started.push(p); return p;
}
const checks = [];
const check = (name, ok, detail) => { checks.push({ name, ok }); log(`${ok ? 'PASS' : 'FAIL'} ${name}${detail ? ': ' + JSON.stringify(detail).slice(0, 300) : ''}`); if (!ok) throw new Error(`check failed: ${name}`); };
async function waitTip(n, want, secs = 600) { for (let k = 0; k < secs * 2; k++) { const tip = hexn(await n.eth('eth_blockNumber')); if (tip >= want) return tip; await sleep(500); } throw new Error(`node ${n.i} did not reach ${want}`); }
function sha256(path) { return '0x' + createHash('sha256').update(readFileSync(path)).digest('hex'); }
async function main() {
const a = await new Node(0).start();
log(`node A up on ${a.evm}`);
const miner = mine(a, 'exec-sync-a');
await waitTip(a, BLOCKS, 900);
const tipA = hexn(await a.eth('eth_blockNumber'));
const st0 = await a.eth('igneum_getExecStatus');
check('A executes from genesis', st0.startedFrom === 'genesis' && st0.blocked === null && hexn(st0.executedTip) >= BLOCKS, st0);
// case 1: the export, the stop, the resume
const file = `${TMP}/snapshot.bin`;
const ex = await a.eth('igneum_exportExecSnapshot', [file]);
const sha = sha256(file);
check('the export writes the file with its tip, root and sha256', existsSync(file) && ex.sha256 === sha && hexn(ex.tip) >= BLOCKS && ex.records === hexn(ex.tip) + 1, { tip: ex.tip, bytes: ex.bytes, sha256: ex.sha256, accounts: ex.accounts });
const blockAtTip = await a.eth('eth_getBlockByNumber', [ex.tip, false]);
check('the export\'s state root is the tip block\'s', blockAtTip.stateRoot === ex.stateRoot, { root: ex.stateRoot });
try { miner.kill('SIGINT'); } catch { }
await a.stop();
const persistedLine = a.logText().split('\n').find(l => l.includes('exec state persisted at stop'));
check('A persisted its state at stop', !!persistedLine, persistedLine && persistedLine.slice(-120));
await a.start();
const resumed = a.logText().split('\n').filter(l => l.includes('exec sync: resumed from')).pop();
const st1 = await a.eth('igneum_getExecStatus');
check('A resumed from the data dir\'s snapshot', st1.startedFrom === 'snapshot' && hexn(st1.snapshotTip) >= BLOCKS && !!resumed, { startedFrom: st1.startedFrom, snapshotTip: st1.snapshotTip, line: resumed && resumed.slice(-160) });
const miner2 = mine(a, 'exec-sync-a2');
await waitTip(a, hexn(st1.snapshotTip) + 5, 300);
check('A keeps executing after the resume', hexn(await a.eth('eth_blockNumber')) > hexn(st1.snapshotTip));
// case 2: a fresh node from the file with the right pin
const b = await new Node(1, [`127.0.0.1:${a.p2pPort}`], [`--igneum-exec-snapshot=${file},${sha}`]).start();
const tipNow = hexn(await a.eth('eth_blockNumber'));
await waitTip(b, tipNow, 600);
const stB = await b.eth('igneum_getExecStatus');
check('B started from the file', stB.startedFrom === 'snapshot' && hexn(stB.snapshotTip) === hexn(ex.tip) && stB.snapshotSha256 === sha, { startedFrom: stB.startedFrom, snapshotTip: stB.snapshotTip });
const blkB = await b.eth('eth_getBlockByNumber', [ex.tip, false]);
check('B\'s state root at the file\'s tip equals A\'s', blkB.stateRoot === ex.stateRoot, { b: blkB.stateRoot });
const payout = '0x4343434343434343434343434343434343434343';
const h = '0x' + tipNow.toString(16);
const [balA, balB] = await Promise.all([a.eth('eth_getBalance', [payout, h]), b.eth('eth_getBalance', [payout, h])]);
check('the miner\'s balance at the same height equals on A and B', balA === balB && BigInt(balA) > 0n, { balA, balB, height: tipNow });
const [rootA, rootB] = await Promise.all([a.eth('eth_getBlockByNumber', [h, false]), b.eth('eth_getBlockByNumber', [h, false])]);
check('the state root at the same height equals on A and B', rootA.stateRoot === rootB.stateRoot, { height: tipNow });
// case 3: known-failed, the wrong pin
const wrong = '0x' + 'ab'.repeat(32);
const c = await new Node(2, [`127.0.0.1:${a.p2pPort}`], [`--igneum-exec-snapshot=${file},${wrong}`]).start();
await sleep(3000);
const stC = await c.eth('igneum_getExecStatus');
const cline = c.logText().split('\n').find(l => l.includes('is not the pinned'));
check('known-failed: C refuses the file under a wrong pin and says so', stC.startedFrom !== 'snapshot' && !!cline, { startedFrom: stC.startedFrom, line: cline && cline.slice(-140) });
// case 4: known-failed, a file from another chain
const z = await new Node(3, [], [], 958).start();
const zm = mine(z, 'exec-sync-z');
await waitTip(z, 12, 300);
const fileZ = `${TMP}/snapshot-z.bin`;
const exZ = await z.eth('igneum_exportExecSnapshot', [fileZ]);
try { zm.kill('SIGINT'); } catch { }
const d = await new Node(4, [`127.0.0.1:${a.p2pPort}`], [`--igneum-exec-snapshot=${fileZ},${exZ.sha256}`]).start();
await sleep(3000);
const stD = await d.eth('igneum_getExecStatus');
const dline = d.logText().split('\n').find(l => l.includes('unknown to consensus') || l.includes('not on this node') || l.includes('is not this network'));
check('known-failed: D refuses a snapshot from another chain and says why', stD.startedFrom !== 'snapshot' && !!dline, { startedFrom: stD.startedFrom, line: dline && dline.slice(-160) });
try { miner2.kill('SIGINT'); } catch { }
log(`RESULT exec sync harness: PASSED (${checks.length} checks) in ${((Date.now() - t0) / 1000).toFixed(1)} s; snapshot ${ex.bytes} bytes at tip ${hexn(ex.tip)}, ${ex.accounts} accounts`);
}
main().then(() => cleanup(0)).catch(e => { log(`FAILED: ${e.message}`); cleanup(1); });
function cleanup(code) { for (const p of started.reverse()) { try { p.kill('SIGINT'); } catch { } } setTimeout(() => { for (const p of started) { try { p.kill('SIGKILL'); } catch { } } process.exit(code); }, 3000); }