igneum/proving/igneum-prove/host/src/main.rs
igneum-labs 3671918fb4 prover: segment-aligned work (proving v1): a free prover claims a whole segment, proves every shard from one export in one chain run, submits the shard records and the segment record; the host's chain mode takes --prev and writes per-shard records with --save-shards
One prover at 2.2% coverage never had 8 consecutive proven blocks (C47, 6 October 2026); claiming whole segments makes it complete about 1 segment in 75 instead of none. The choice is deterministic per key (FNV of first block and key hash) over the untouched whole segments inside their deadline by a margin (240 DAA or 1.5x the last segment's time); the per-block path stays as the fallback. Unit tests for the grid, the grouping, the margin, the attempted set and the per-key order; 120 app tests, 8 core and 9 host tests pass.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 07:12:23 +00:00

1034 lines
64 KiB
Rust

//! igneum-prove-host: proves one Igneum chain block fixture with SP1, shard by shard (design 5.1 and 5.3).
//!
//! Usage: igneum-prove-host <fixture.json> [--mode native|execute|shard|block|all] [--shard N]
//! [--prover 0x<payout address>] [--out <results.json>]
//!
//! Modes build on each other. `native` cuts the block into shards at `S_p`, builds every shard's witness, runs
//! every shard statement natively, checks that the shards chain (roots, links) and sum (gas, pgas) to the whole
//! block, aggregates them natively, and shows that a tampered witness fails. `execute` runs every shard guest
//! and the aggregator guest in SP1's executor for the cycle counts (no proof). `shard` proves one shard (the
//! `--shard` index, default 0) in all three SP1 stages: execute, core, compressed, each verified. `block` makes
//! the compressed proof of every shard and the aggregated block proof, verified. `all` is shard then block.
//! The prover comes from `SP1_PROVER` (cpu, the default; cuda, Linux x86_64 with the `cuda` feature; mock).
//! Every stage prints a `STAGE ... start` line and one `RESULT` line with a UTC timestamp, so a silent gap
//! between stages is visible (ledger P20). The host holds a Tokio runtime for the whole run and drops the
//! proof system inside it, so the CUDA client's destructor finds a runtime (ledger P20).
//!
//! The guests are pinned (5 October 2026, `pinned.rs`): the host embeds the ELFs and verifying keys committed
//! under `elf/`, checks their hashes against `elf/manifest.json` at every start, and in the prove modes refuses
//! to continue when SP1's key setup does not derive the manifest's program id. `--mode id` prints the pinned
//! ids with no setup. `--mode verify` uses SP1's light verifier and the pinned verifying key: no prover client,
//! no key generation (the 114 s to 138 s the Mac's node spent per proof on 5 October).
mod pinned;
mod proof_system;
use alloy_primitives::{Address, B256};
use anyhow::{anyhow, bail, Context, Result};
use igneum_prove_core::agg::{self, AggInput, BlockOutput};
use igneum_prove_core::shard::{build_shards, shard_statement, BuiltShard, ShardInput, ShardOutput};
use igneum_prove_core::Fixture;
use proof_system::{ProofSystem, SegmentClaim, ShardWitness, Sp1ProofSystem, Sp1ShardProof, StubProofSystem};
use sha2::Digest;
use std::time::{Duration, Instant};
fn now() -> String {
let secs = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.as_secs()).unwrap_or(0);
let (h, m, s) = ((secs / 3600) % 24, (secs / 60) % 60, secs % 60);
let days = secs / 86400;
// Civil date from days since the epoch (Howard Hinnant's algorithm).
let z = days as i64 + 719468;
let era = z.div_euclid(146097);
let doe = z - era * 146097;
let yoe = (doe - doe / 1460 + doe / 36524 - doe / 146096) / 365;
let y = yoe + era * 400;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let d = doy - (153 * mp + 2) / 5 + 1;
let mo = if mp < 10 { mp + 3 } else { mp - 9 };
let y = if mo <= 2 { y + 1 } else { y };
format!("{y:04}-{mo:02}-{d:02}T{h:02}:{m:02}:{s:02}Z")
}
fn stage(name: &str) {
println!("STAGE {name} start {}", now());
}
fn main() -> Result<()> {
// Ledger P20: the SP1 CUDA client spawns its shutdown on the current Tokio runtime from `Drop`; hold one for
// the whole run so the drop at the end finds it, and give the spawned tasks time to finish.
let runtime = tokio::runtime::Runtime::new()?;
let guard = runtime.enter();
let result = run();
drop(guard);
runtime.shutdown_timeout(Duration::from_secs(10));
result
}
fn run() -> Result<()> {
let args: Vec<String> = std::env::args().collect();
let arg = |name: &str| args.iter().position(|a| a == name).and_then(|i| args.get(i + 1)).cloned();
let mode = arg("--mode").unwrap_or_else(|| "all".into());
let pinned = pinned::Pinned::load()?;
if mode == "id" {
println!("RESULT id: {}", pinned.describe());
return Ok(());
}
if mode == "verify" {
// proving v0 (spec 7.7): the node's proof pool verifies a submitted shard proof off the consensus path
return run_verify(&pinned, &arg("--proof").context("--proof <file>")?, &arg("--statement").context("--statement 0x<keccak of the public values>")?);
}
if mode == "verify-segment" {
// proving v1 (spec 7.8): the node's pool verifies an aggregated segment proof against the pinned aggregator key
return run_verify_segment(&pinned, &arg("--proof").context("--proof <file>")?, &arg("--statement").context("--statement 0x<keccak of the block public values>")?);
}
let prover: Address = arg("--prover").map(|s| s.parse()).transpose()?.unwrap_or_else(|| Address::from_slice(&[0x19; 20]));
let out_path = arg("--out");
if mode == "aggregate" {
// proving v1: the live aggregator, from shard proof files (the node's pool) and the previous segment proof
return run_aggregate(&pinned, &arg("--proofs").context("--proofs <a.bin,b.bin,...> (the segment's shard proofs in shard order)")?, &arg("--parent").context("--parent 0x<parent chain block hash>")?, arg("--prev").as_deref(), out_path.as_deref());
}
if mode == "chain" {
// proving v1: N consecutive fixtures proven shard by shard, each block aggregated with the previous block's
// proof (the chain rule of design 5.3), the measurement of docs/plans/proving-v1.md step 2
let list = arg("--chain").or_else(|| args.get(1).filter(|a| !a.starts_with("--")).cloned()).context("--chain <f1.json,f2.json,...> (consecutive fixtures)")?;
let fixtures: Vec<String> = list.split(',').map(|s| s.trim().to_string()).filter(|s| !s.is_empty()).collect();
// --save-shards writes every shard's compressed proof next to the results (block-N-shard-i-compressed.bin),
// so `--mode aggregate` can re-run the aggregation of the same proofs under other settings
let save_shards = args.iter().any(|a| a == "--save-shards");
// --prev <file>: the previous segment's aggregated proof; the chain continues from it (chain_len grows past
// the segment length, the chain rule of spec 7.8) instead of starting fresh. The app's segment path (6 October
// 2026) passes it when the node reports the previous segment paid and its proof in the pool.
let prev = arg("--prev");
return run_chain(&pinned, &fixtures, prover, out_path.as_deref(), save_shards, prev.as_deref());
}
let path = args.get(1).filter(|a| !a.starts_with("--")).context("usage: igneum-prove-host <fixture.json> [--mode native|execute|shard|compressed|block|all] [--shard N] [--budget <test pgas>] [--prover 0x..] [--out results.json]; --mode chain --chain <f1,f2,...> [--prover 0x..] [--out results.json] [--save-shards] [--prev prev.bin]; --mode aggregate --proofs <a.bin,...> --parent 0x.. [--prev prev.bin] [--out results.json]; --mode verify --proof <file> --statement 0x..; --mode verify-segment --proof <file> --statement 0x..; --mode id")?;
let shard_index: usize = arg("--shard").map(|s| s.parse()).transpose()?.unwrap_or(0);
// `--budget <pgas>`: re-plan the fixture's block at a TEST budget (the S_p curve of 5 October 2026); the fixture's
// own per-shard plan is then not compared (the chain and the sums still are), and `--out` records the cut
let test_budget: Option<u64> = arg("--budget").map(|s| s.parse()).transpose()?;
let fixture: Fixture = serde_json::from_str(&std::fs::read_to_string(path).with_context(|| format!("read {path}"))?)?;
if fixture.format != igneum_prove_core::fixture::FORMAT {
bail!("fixture format {} is not {} (regenerate with igneum-prove-export)", fixture.format, igneum_prove_core::fixture::FORMAT);
}
let block = &fixture.block;
let txs: usize = block.blocks.iter().map(|b| b.txs.len()).sum();
let prover_kind = std::env::var("SP1_PROVER").unwrap_or_else(|_| "cpu".into());
// The fee set that meters this block: the fixture's schedule read at the block's DAA score (5 October 2026,
// `fees_v1_activation_daa`); the plan's budget must be that set's S_p unless the fixture says test cut.
let fee_set = block.fees.at(block.env.daa_score);
if let Some(b) = test_budget {
println!("TEST CUT: the block is re-planned at a budget of {b} pgas (the fixture's plan at {} is not compared)", fixture.plan.shard_budget);
}
if test_budget.is_none() && fixture.plan.consensus && fixture.plan.shard_budget != fee_set.shard_proving_gas_budget {
bail!("the fixture's plan says consensus budget {} but the fee set at DAA score {} ({}) has S_p {}; regenerate with igneum-prove-export", fixture.plan.shard_budget, block.env.daa_score, fee_set.name(), fee_set.shard_proving_gas_budget);
}
println!(
"igneum-prove-host sources {}: fixture {path}: chain {} block {} ({}) at DAA score {}, {txs} transactions in {} including blocks, {} accounts in the pre-state, plan {} shard(s) at S_p = {} pgas{}; fee schedule {}: this block meters with {} (intrinsic {} pgas, B_p {}); SP1_PROVER={prover_kind}; prover payout {prover}; {}",
env!("IGNEUM_PROVE_SOURCES"),
block.chain_id,
block.env.number,
block.env.hash,
block.env.daa_score,
block.blocks.len(),
block.pre_state.len(),
fixture.plan.shards.len(),
fixture.plan.shard_budget,
if fixture.plan.consensus { "" } else { " (TEST CUT below the consensus budget)" },
block.fees.describe(),
fee_set.name(),
fee_set.pgas.intrinsic_pgas_per_tx,
fee_set.block_proving_gas_limit,
now()
);
let mut results = serde_json::Map::new();
results.insert("fixture".into(), path.clone().into());
results.insert("block".into(), block.env.number.into());
results.insert("prover".into(), prover_kind.clone().into());
results.insert("shard_budget".into(), fixture.plan.shard_budget.into());
results.insert("consensus_budget".into(), fixture.plan.consensus.into());
results.insert("daa_score".into(), block.env.daa_score.into());
results.insert("fee_set".into(), fee_set.name().into());
results.insert("fees_v1_activation_daa".into(), if block.fees.v1_activation_daa == u64::MAX { serde_json::Value::Null } else { block.fees.v1_activation_daa.into() });
results.insert("sp1_crate_version".into(), "6.8.1".into());
results.insert("sp1_circuit_version".into(), sp1_sdk::SP1_CIRCUIT_VERSION.into());
// 1. Native: the cut, the witnesses, every shard statement, the chain and the sums, against the fixture.
stage("native");
let t = Instant::now();
let budget = test_budget.unwrap_or(fixture.plan.shard_budget);
let (outcome, pre_root, shards) = build_shards(block, budget, prover);
let native_s = t.elapsed().as_secs_f64();
results.insert("budget".into(), budget.into());
results.insert("test_cut".into(), test_budget.is_some().into());
let e = &fixture.expected;
let same = pre_root == e.pre_state_root && outcome.state_root == e.post_state_root && outcome.receipts_root == e.receipts_root && outcome.tx_commitment == e.tx_commitment && outcome.gas_used == e.gas_used && outcome.pgas_used == e.pgas_used;
println!(
"RESULT native: {:.4} s, pre {} post {} receipts {} gas {} pgas {} executed {} skipped {}: {} at {}",
native_s,
pre_root,
outcome.state_root,
outcome.receipts_root,
outcome.gas_used,
outcome.pgas_used,
outcome.executed.len(),
outcome.skipped.len(),
if same { "MATCHES the fixture's expected values" } else { "DIFFERS from the fixture's expected values" },
now()
);
if !same {
bail!("native execution differs from the fixture; regenerate the fixture with igneum-prove-export");
}
if e.node_state_root != e.post_state_root {
bail!("the fixture's node state root differs from its expected post-state root; the exporter must not have produced this file");
}
if test_budget.is_none() && shards.len() != fixture.plan.shards.len() {
bail!("the plan has {} shards here and {} in the fixture", shards.len(), fixture.plan.shards.len());
}
let (mut sum_gas, mut sum_pgas) = (0u64, 0u64);
let mut prev_root = pre_root;
let mut prev_link = igneum_prove_core::Carry::default().link();
for (i, s) in shards.iter().enumerate() {
let o = &s.output;
let (accounts, slots, leaves, hashes) = s.input.witness.stats();
let bytes = bincode::serialize(&s.input)?.len();
if let (None, Some(x)) = (test_budget, fixture.plan.shards.get(i)) {
if o.pre_root != x.pre_root || o.post_root != x.post_root || o.receipts_root != x.receipts_root || o.link_in != x.link_in || o.link_out != x.link_out || o.gas_used != x.gas_used || o.pgas_used != x.pgas_used {
bail!("shard {}: the native statement differs from the fixture's plan", o.shard_index);
}
}
if o.pre_root != prev_root || o.link_in != prev_link {
bail!("shard {}: does not continue the previous shard", o.shard_index);
}
prev_root = o.post_root;
prev_link = o.link_out;
sum_gas += o.gas_used;
sum_pgas += o.pgas_used;
results.entry("shard_witness_bytes").or_insert_with(|| serde_json::Value::Array(Vec::new())).as_array_mut().unwrap().push(serde_json::Value::from(bytes as u64));
println!(
"RESULT shard {} native: txs {}..{} ({} executed, {} skipped), gas {}, pgas {}{}, witness {accounts} accounts {slots} slots {leaves} leaves {hashes} hashes, input {bytes} bytes, pre {} post {}",
o.shard_index,
s.spec.tx_start,
s.spec.tx_end,
o.executed,
o.skipped,
o.gas_used,
o.pgas_used,
if s.spec.over_budget { " (ONE TRANSACTION ABOVE S_p)" } else { "" },
o.pre_root,
o.post_root
);
}
if prev_root != outcome.state_root || sum_gas != outcome.gas_used || sum_pgas != outcome.pgas_used {
bail!("the shards do not chain to the block's post-root or do not sum to its gas and pgas");
}
let native_block = agg::aggregate(&AggInput { shard_vk: [0; 8], shards: shards.iter().map(|s| s.output.to_bytes()).collect(), parent_hash: block.env.parent_hash, prev: None }, &mut |_, _| {});
if native_block.post_root != outcome.state_root || native_block.tx_commitment != outcome.tx_commitment || native_block.gas_used != outcome.gas_used {
bail!("the native aggregation does not reproduce the block");
}
println!("RESULT plan: {} shard(s) chain from {} to {} and sum to gas {} pgas {}: the cut equals the block; native aggregation receipts {} provers {}", shards.len(), pre_root, outcome.state_root, sum_gas, sum_pgas, native_block.receipts, native_block.provers);
results.insert("native_seconds".into(), native_s.into());
results.insert("shards".into(), (shards.len() as u64).into());
results.insert("shard_pgas".into(), shards.iter().map(|s| serde_json::Value::from(s.output.pgas_used)).collect::<Vec<_>>().into());
results.insert("witness_bytes".into(), shards.iter().map(|s| serde_json::Value::from(bincode::serialize(&s.input).map(|b| b.len() as u64).unwrap_or(0))).collect::<Vec<_>>().into());
// 2. Tampered witnesses must fail: a changed balance moves the pre-root away from the node's; a changed
// storage value breaks the storage root check; a dropped account makes the execution unprovable.
tamper_checks(&shards[0])?;
// 3. The devnet stub behind the same trait (design 5.7), so the pipeline shape is exercised too.
let stub = StubProofSystem::new([7u8; 32]);
let stub_shards: Vec<_> = shards.iter().map(|s| stub.prove_shard(&ShardWitness { input: s.input.clone() })).collect::<Result<_>>()?;
let stub_seg = stub.aggregate(None, &stub_shards)?;
let claim = SegmentClaim { segment: block.env.hash, pre_root, post_root: outcome.state_root, receipts: native_block.receipts };
println!("RESULT stub: ProofSystem v{} program {} verify_segment {}", StubProofSystem::VERSION, stub.program_id(), stub.verify_segment(&stub_seg, &claim));
if mode == "native" {
return finish(results, out_path);
}
// 4. SP1, version 1 behind the trait: two programs.
stage("setup");
let t = Instant::now();
let sp1 = Sp1ProofSystem::from_env(pinned.shard_elf(), pinned.agg_elf())?;
let setup_s = t.elapsed().as_secs_f64();
let secs = |k: &str| sp1.last_timing(k).map(|d| d.as_secs_f64()).unwrap_or(0.0);
println!("RESULT setup: {:.2} s (prover client {:.2} s, shard keys {:.2} s, aggregator keys {:.2} s), ProofSystem v{} shard program id {} aggregator id {} at {}", setup_s, secs("client"), secs("setup-shard"), secs("setup-aggregator"), Sp1ProofSystem::VERSION, sp1.program_id(), sp1.aggregator_id(), now());
if sp1.program_id() != pinned.shard_id || sp1.aggregator_id() != pinned.agg_id {
bail!("SP1's key setup derived shard program id {} and aggregator id {} from the embedded guests, the pinned manifest says {} and {}: this host would make proofs no other node accepts (re-pin with proving/igneum-prove/pin-guests.sh)", sp1.program_id(), sp1.aggregator_id(), pinned.shard_id, pinned.agg_id);
}
println!("RESULT pinned: setup matches the manifest ({})", pinned.describe());
results.insert("client_seconds".into(), secs("client").into());
results.insert("setup_seconds".into(), setup_s.into());
results.insert("shard_program_id".into(), sp1.program_id().to_string().into());
results.insert("aggregator_id".into(), sp1.aggregator_id().to_string().into());
let r = match mode.as_str() {
"execute" => run_execute(&sp1, &shards, block.env.parent_hash, &mut results),
"shard" => run_shard(&sp1, &shards, shard_index, out_path.as_deref(), &mut results),
"compressed" => run_compressed(&sp1, &shards, shard_index, out_path.as_deref(), &mut results),
"block" => run_block(&sp1, &shards, &claim, out_path.as_deref(), &mut results),
"all" => run_shard(&sp1, &shards, shard_index, out_path.as_deref(), &mut results).and_then(|_| run_block(&sp1, &shards, &claim, out_path.as_deref(), &mut results)),
other => Err(anyhow!("unknown mode {other}")),
};
// The proof system (and with it the CUDA client) is dropped here, inside the runtime guard of `main`.
drop(sp1);
r?;
finish(results, out_path)
}
fn tamper_checks(shard: &BuiltShard) -> Result<()> {
let quiet = std::panic::take_hook();
std::panic::set_hook(Box::new(|_| {}));
let outcome = |name: &str, input: ShardInput, expect_root: B256| -> Result<()> {
let r = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| shard_statement(&input)));
let verdict = match &r {
Err(_) => "REJECTED (the statement cannot be proven)".to_string(),
Ok(o) if o.pre_root != expect_root => format!("REJECTED (pre-root {} is not the node's {})", o.pre_root, expect_root),
Ok(_) => "ACCEPTED".to_string(),
};
println!("RESULT tamper {name}: {verdict}");
if verdict.starts_with("ACCEPTED") {
bail!("a tampered witness ({name}) was accepted");
}
Ok(())
};
let expect = shard.output.pre_root;
// (a) A balance in an account leaf, re-encoded so the leaf still decodes: the witness is consistent with
// itself and the pre-root it yields is simply not the node's.
let mut a = shard.input.clone();
let addresses: Vec<B256> = a.witness.accounts.iter().map(|acc| alloy_primitives::keccak256(acc.address)).collect();
if let Some((_, v)) = a.witness.trie.leaves.iter_mut().find(|(k, _)| addresses.contains(k)) {
let mut acc = <alloy_trie::TrieAccount as alloy_rlp::Decodable>::decode(&mut v.as_ref()).expect("leaf decodes");
acc.balance += alloy_primitives::U256::from(1);
*v = alloy_rlp::encode(acc).into();
}
outcome("account balance", a, expect)?;
// (b) A storage value, where a shard touches storage; else a code byte.
let mut b = shard.input.clone();
let mut touched = false;
for acc in b.witness.accounts.iter_mut() {
if let Some((_, v)) = acc.storage.leaves.first_mut() {
let mut bytes = v.to_vec();
let last = bytes.len() - 1;
bytes[last] ^= 0x01;
*v = bytes.into();
touched = true;
break;
}
}
if !touched {
for acc in b.witness.accounts.iter_mut() {
if !acc.code.is_empty() {
let mut bytes = acc.code.to_vec();
bytes[0] ^= 0x01;
acc.code = bytes.into();
touched = true;
break;
}
}
}
if touched {
outcome("storage or code", b, expect)?;
}
// (c) An account dropped from the witness: its leaf stays in the trie (the pre-root still matches) but the
// execution reads it and the strict database refuses.
let mut c = shard.input.clone();
let victim = c.txs.first().and_then(|t| igneum_evm_types::decode_and_check(&t.raw, c.chain_id).ok()).map(|tx| tx.sender);
if let Some(sender) = victim {
c.witness.accounts.retain(|acc| acc.address != sender);
outcome("dropped account", c, expect)?;
}
std::panic::set_hook(quiet);
Ok(())
}
fn run_execute(sp1: &Sp1ProofSystem, shards: &[BuiltShard], parent_hash: B256, results: &mut serde_json::Map<String, serde_json::Value>) -> Result<()> {
let mut cycles_all = Vec::new();
let mut outputs = Vec::new();
for s in shards {
stage(&format!("execute shard {}", s.output.shard_index));
let (out, report, dt) = sp1.execute_shard(&s.input)?;
check_shard_output(&out, &s.output)?;
let cycles = report.total_instruction_count();
println!("RESULT execute shard {}: {} cycles, prover gas {}, {:.2} s, {:.0} cycles per EVM gas, {:.0} cycles per pgas, syscalls {} at {}", out.shard_index, cycles, report.gas().unwrap_or(0), dt.as_secs_f64(), cycles as f64 / out.gas_used.max(1) as f64, cycles as f64 / out.pgas_used.max(1) as f64, report.syscall_counts.values().sum::<u64>(), now());
cycles_all.push(serde_json::Value::from(cycles));
outputs.push(out);
}
stage("execute aggregator");
let (out, report, dt) = sp1.execute_aggregator(&outputs, parent_hash)?;
let cycles = report.total_instruction_count();
println!("RESULT execute aggregator: {} cycles over {} shards (deferred proof verification off), {:.2} s, post-root {} at {}", cycles, out.shard_count, dt.as_secs_f64(), out.post_root, now());
results.insert("shard_cycles".into(), cycles_all.into());
results.insert("aggregator_cycles".into(), cycles.into());
Ok(())
}
/// Writes a proof with a 4 MB buffer and prints how long it took. `SP1ProofWithPublicValues::save` serialises
/// straight into an unbuffered `File`, which on WSL2 means millions of tiny writes across the /mnt/c bridge:
/// the 18 MB core proof of a full shard took the RTX 5090 run over 15 minutes to save (ledger P20, second gap).
fn save_proof<T: serde::Serialize>(proof: &T, path: std::path::PathBuf) {
let t = Instant::now();
let res = std::fs::File::create(&path)
.map_err(|e| anyhow!("{e}"))
.and_then(|f| {
let mut w = std::io::BufWriter::with_capacity(4 << 20, f);
bincode::serialize_into(&mut w, proof).map_err(|e| anyhow!("{e}"))?;
std::io::Write::flush(&mut w).map_err(|e| anyhow!("{e}"))
});
match res {
Ok(()) => println!("saved {} in {:.1} s", path.display(), t.elapsed().as_secs_f64()),
Err(e) => println!("could not save {}: {e}", path.display()),
}
}
fn run_shard(sp1: &Sp1ProofSystem, shards: &[BuiltShard], index: usize, out_dir: Option<&str>, results: &mut serde_json::Map<String, serde_json::Value>) -> Result<()> {
let s = shards.get(index).ok_or_else(|| anyhow!("shard {index} is not in the plan ({} shards)", shards.len()))?;
let i = s.output.shard_index;
results.insert("shard_index".into(), i.into());
stage(&format!("execute shard {i}"));
let (out, report, dt) = sp1.execute_shard(&s.input)?;
check_shard_output(&out, &s.output)?;
let cycles = report.total_instruction_count();
println!("RESULT execute shard {i}: {} cycles, prover gas {}, {:.2} s, {:.0} cycles per EVM gas, {:.0} cycles per pgas, syscalls {} at {}", cycles, report.gas().unwrap_or(0), dt.as_secs_f64(), cycles as f64 / out.gas_used.max(1) as f64, cycles as f64 / out.pgas_used.max(1) as f64, report.syscall_counts.values().sum::<u64>(), now());
results.insert("cycles".into(), cycles.into());
results.insert("prover_gas".into(), report.gas().unwrap_or(0).into());
results.insert("execute_seconds".into(), dt.as_secs_f64().into());
stage(&format!("core shard {i}"));
let (core, dt) = sp1.prove_shard_core(&s.input)?;
let (ok, vdt) = sp1.verify_shard(&core, &s.output);
let bytes = bincode::serialize(&core)?.len();
println!("RESULT core shard {i}: prove {:.1} s, proof {} bytes, verify {:.3} s, {}; post-root {} at {}", dt.as_secs_f64(), bytes, vdt.as_secs_f64(), if ok { "VERIFIED" } else { "VERIFY FAILED" }, s.output.post_root, now());
if !ok {
bail!("core proof of shard {i} did not verify");
}
results.insert("core_prove_seconds".into(), dt.as_secs_f64().into());
results.insert("core_verify_seconds".into(), vdt.as_secs_f64().into());
results.insert("core_proof_bytes".into(), bytes.into());
if let Some(dir) = out_dir.and_then(|p| std::path::Path::new(p).parent()) {
save_proof(&core, dir.join(format!("block-{}-shard-{i}-core.bin", s.input.env.number)));
}
stage(&format!("compressed shard {i}"));
let proof = sp1.prove_shard(&ShardWitness { input: s.input.clone() })?;
let dt = sp1.last_timing("compressed").unwrap_or_default();
let (ok, vdt) = sp1.verify_shard(&proof.proof, &s.output);
let bytes = bincode::serialize(&proof.proof)?.len();
println!("RESULT compressed shard {i}: prove {:.1} s, proof {} bytes, verify {:.3} s, {}; post-root {} prover {} at {}", dt.as_secs_f64(), bytes, vdt.as_secs_f64(), if ok { "VERIFIED" } else { "VERIFY FAILED" }, proof.output.post_root, proof.output.prover, now());
if !ok {
bail!("compressed proof of shard {i} did not verify");
}
results.insert("compressed_prove_seconds".into(), dt.as_secs_f64().into());
results.insert("compressed_verify_seconds".into(), vdt.as_secs_f64().into());
results.insert("compressed_proof_bytes".into(), bytes.into());
if let Some(dir) = out_dir.and_then(|p| std::path::Path::new(p).parent()) {
save_proof(&proof.proof, dir.join(format!("block-{}-shard-{i}-compressed.bin", s.input.env.number)));
}
Ok(())
}
/// Proving v0 (spec 7.7): the prover's mode. Execute (the cycle count), then the compressed proof of one shard,
/// verified; writes `<out dir>/block-N-shard-i-compressed.bin` and records the statement (keccak of the public
/// values, what the proof record carries) and the proof's SHA-256 (the record's `proof_hash`).
fn run_compressed(sp1: &Sp1ProofSystem, shards: &[BuiltShard], index: usize, out_dir: Option<&str>, results: &mut serde_json::Map<String, serde_json::Value>) -> Result<()> {
let s = shards.get(index).ok_or_else(|| anyhow!("shard {index} is not in the plan ({} shards)", shards.len()))?;
let i = s.output.shard_index;
results.insert("shard_index".into(), i.into());
stage(&format!("execute shard {i}"));
let (out, report, dt) = sp1.execute_shard(&s.input)?;
check_shard_output(&out, &s.output)?;
let cycles = report.total_instruction_count();
println!("RESULT execute shard {i}: {} cycles, {:.2} s at {}", cycles, dt.as_secs_f64(), now());
results.insert("cycles".into(), cycles.into());
results.insert("execute_seconds".into(), dt.as_secs_f64().into());
stage(&format!("compressed shard {i}"));
let proof = sp1.prove_shard(&ShardWitness { input: s.input.clone() })?;
let dt = sp1.last_timing("compressed").unwrap_or_default();
let (ok, vdt) = sp1.verify_shard(&proof.proof, &s.output);
let bytes = bincode::serialize(&proof.proof)?;
let statement = alloy_primitives::keccak256(s.output.to_bytes());
let proof_hash: [u8; 32] = sha2::Sha256::digest(&bytes).into();
println!("RESULT compressed shard {i}: prove {:.1} s, proof {} bytes, verify {:.3} s, {}; statement {} proof sha256 0x{} prover {} at {}", dt.as_secs_f64(), bytes.len(), vdt.as_secs_f64(), if ok { "VERIFIED" } else { "VERIFY FAILED" }, statement, hex::encode(proof_hash), proof.output.prover, now());
if !ok {
bail!("compressed proof of shard {i} did not verify");
}
results.insert("compressed_prove_seconds".into(), dt.as_secs_f64().into());
results.insert("compressed_verify_seconds".into(), vdt.as_secs_f64().into());
results.insert("compressed_proof_bytes".into(), bytes.len().into());
results.insert("statement".into(), statement.to_string().into());
results.insert("proof_sha256".into(), format!("0x{}", hex::encode(proof_hash)).into());
results.insert("block_hash".into(), s.input.env.hash.to_string().into());
results.insert("number".into(), s.input.env.number.into());
results.insert("prover".into(), proof.output.prover.to_string().into());
let dir = out_dir.and_then(|p| std::path::Path::new(p).parent().map(|d| d.to_path_buf())).unwrap_or_else(|| std::path::PathBuf::from("."));
let file = dir.join(format!("block-{}-shard-{i}-compressed.bin", s.input.env.number));
std::fs::write(&file, &bytes)?;
results.insert("proof_file".into(), file.display().to_string().into());
println!("proof written to {}", file.display());
Ok(())
}
/// `--mode verify --proof <file> --statement 0x..`: verifies the compressed proof against the PINNED shard
/// verifying key with SP1's light verifier (no prover client, no key generation) and checks that the keccak of
/// its public values is the statement. Prints the program id the proof was made with next to ours, so a proof
/// from a host with another guest build is rejected with the reason in the node log. Exit 0 = verified,
/// 3 = not verified (what the node's proof pool reads).
fn run_verify(pinned: &pinned::Pinned, proof_path: &str, statement: &str) -> Result<()> {
use sp1_sdk::blocking::{LightProver, Prover};
let bytes = std::fs::read(proof_path).with_context(|| format!("read {proof_path}"))?;
let want: B256 = statement.parse().context("statement is not 32 bytes of hex")?;
stage("setup");
let t = Instant::now();
let verifier = LightProver::new();
println!("RESULT setup: {:.3} s (light verifier, pinned key), shard program id {} at {}", t.elapsed().as_secs_f64(), pinned.shard_id, now());
stage("verify");
let t = Instant::now();
let proof: sp1_sdk::SP1ProofWithPublicValues = bincode::deserialize(&bytes).context("the file is not a bincode SP1 proof")?;
let got = alloy_primitives::keccak256(proof.public_values.as_slice());
let output = ShardOutput::from_bytes(proof.public_values.as_slice());
let claimed = pinned::claimed_program_id(&proof);
let same_program = claimed == Some(pinned.shard_id);
let crypto_ok = same_program && verifier.verify(&proof, &pinned.shard_vk, None).is_ok();
let ok = crypto_ok && got == want && output.is_some();
let dt = t.elapsed().as_secs_f64();
let program = match claimed {
Some(c) if same_program => format!("program id {c} (ours)"),
Some(c) => format!("program id {c} IS NOT OURS {} (the prover runs another guest build)", pinned.shard_id),
None => "not a compressed proof".to_string(),
};
match &output {
Some(o) => println!("RESULT verify: {} in {dt:.3} s; block {} shard {} prover {} statement {got} (want {want}) {program} proof {} bytes at {}", if ok { "VERIFIED" } else { "NOT VERIFIED" }, o.number, o.shard_index, o.prover, bytes.len(), now()),
None => println!("RESULT verify: NOT VERIFIED in {dt:.3} s; public values are not a shard statement; {program} at {}", now()),
}
if ok {
Ok(())
} else {
std::process::exit(3)
}
}
fn run_block(sp1: &Sp1ProofSystem, shards: &[BuiltShard], claim: &SegmentClaim, out_dir: Option<&str>, results: &mut serde_json::Map<String, serde_json::Value>) -> Result<()> {
let block_t = Instant::now();
let mut proofs: Vec<Sp1ShardProof> = Vec::with_capacity(shards.len());
let mut shard_seconds = Vec::new();
for s in shards {
let i = s.output.shard_index;
stage(&format!("compressed shard {i} (block mode)"));
let p = sp1.prove_shard(&ShardWitness { input: s.input.clone() })?;
let dt = sp1.last_timing("compressed").unwrap_or_default();
let (ok, vdt) = sp1.verify_shard(&p.proof, &s.output);
println!("RESULT block shard {i}: compressed prove {:.1} s, proof {} bytes, verify {:.3} s, {}, pgas {} at {}", dt.as_secs_f64(), bincode::serialize(&p.proof)?.len(), vdt.as_secs_f64(), if ok { "VERIFIED" } else { "VERIFY FAILED" }, p.output.pgas_used, now());
if !ok {
bail!("compressed proof of shard {i} did not verify");
}
shard_seconds.push(serde_json::Value::from(dt.as_secs_f64()));
proofs.push(p);
}
stage(&format!("aggregate {} shards", proofs.len()));
let seg = sp1.aggregate(None, &proofs)?;
let dt = sp1.last_timing("aggregate").unwrap_or_default();
let ok = sp1.verify_segment(&seg, claim);
let vdt = sp1.last_timing("verify-block").unwrap_or_default();
let bytes = bincode::serialize(&seg.proof)?.len();
let total = block_t.elapsed().as_secs_f64();
println!(
"RESULT block: {} shards, aggregate prove {:.1} s, proof {} bytes, verify {:.3} s, {}; block {} pre {} post {} receipts {} tx commitment {} gas {} pgas {} provers {}; shard proofs plus aggregation {:.1} s at {}",
seg.output.shard_count,
dt.as_secs_f64(),
bytes,
vdt.as_secs_f64(),
if ok { "VERIFIED (shard program id and claim checked)" } else { "VERIFY FAILED" },
seg.output.number,
seg.output.pre_root,
seg.output.post_root,
seg.output.receipts,
seg.output.tx_commitment,
seg.output.gas_used,
seg.output.pgas_used,
seg.output.provers,
total,
now()
);
if !ok {
bail!("the block proof did not verify");
}
results.insert("block_shard_prove_seconds".into(), shard_seconds.into());
results.insert("aggregate_prove_seconds".into(), dt.as_secs_f64().into());
results.insert("aggregate_verify_seconds".into(), vdt.as_secs_f64().into());
results.insert("block_proof_bytes".into(), bytes.into());
results.insert("block_total_seconds".into(), total.into());
if let Some(dir) = out_dir.and_then(|p| std::path::Path::new(p).parent()) {
save_proof(&seg.proof, dir.join(format!("block-{}-aggregated.bin", seg.output.number)));
}
let _ = BlockOutput::LEN;
Ok(())
}
/// Loads a fixture with the checks `run` makes (format, the plan's budget against the fee set at its DAA score).
fn load_fixture(path: &str) -> Result<Fixture> {
let fixture: Fixture = serde_json::from_str(&std::fs::read_to_string(path).with_context(|| format!("read {path}"))?)?;
if fixture.format != igneum_prove_core::fixture::FORMAT {
bail!("fixture format {} is not {} (regenerate with igneum-prove-export)", fixture.format, igneum_prove_core::fixture::FORMAT);
}
let fee_set = fixture.block.fees.at(fixture.block.env.daa_score);
if fixture.plan.consensus && fixture.plan.shard_budget != fee_set.shard_proving_gas_budget {
bail!("{path}: the fixture's plan says consensus budget {} but the fee set at DAA score {} ({}) has S_p {}; regenerate with igneum-prove-export", fixture.plan.shard_budget, fixture.block.env.daa_score, fee_set.name(), fee_set.shard_proving_gas_budget);
}
Ok(fixture)
}
fn setup_sp1(pinned: &pinned::Pinned, results: &mut serde_json::Map<String, serde_json::Value>) -> Result<Sp1ProofSystem> {
stage("setup");
let t = Instant::now();
let sp1 = Sp1ProofSystem::from_env(pinned.shard_elf(), pinned.agg_elf())?;
let setup_s = t.elapsed().as_secs_f64();
println!("RESULT setup: {:.2} s, ProofSystem v{} shard program id {} aggregator id {} at {}", setup_s, Sp1ProofSystem::VERSION, sp1.program_id(), sp1.aggregator_id(), now());
println!("RESULT sp1 knobs: {}", Sp1ProofSystem::env_knobs());
results.insert("sp1_env_knobs".into(), Sp1ProofSystem::env_knobs().into());
if sp1.program_id() != pinned.shard_id || sp1.aggregator_id() != pinned.agg_id {
bail!("SP1's key setup derived shard program id {} and aggregator id {} from the embedded guests, the pinned manifest says {} and {}: this host would make proofs no other node accepts (re-pin with proving/igneum-prove/pin-guests.sh)", sp1.program_id(), sp1.aggregator_id(), pinned.shard_id, pinned.agg_id);
}
results.insert("setup_seconds".into(), setup_s.into());
results.insert("shard_program_id".into(), sp1.program_id().to_string().into());
results.insert("aggregator_id".into(), sp1.aggregator_id().to_string().into());
results.insert("prover".into(), std::env::var("SP1_PROVER").unwrap_or_else(|_| "cpu".into()).into());
Ok(sp1)
}
/// What a segment record carries about an aggregated proof (spec 7.8): the public values, their keccak (the
/// statement), the proof's SHA-256 and where the proof bytes went.
fn segment_results(seg: &proof_system::Sp1SegmentProof, out_dir: &std::path::Path, results: &mut serde_json::Map<String, serde_json::Value>) -> Result<(B256, usize)> {
let bytes = bincode::serialize(&seg.proof)?;
let pv = seg.proof.public_values.as_slice().to_vec();
let statement = alloy_primitives::keccak256(&pv);
let proof_hash: [u8; 32] = sha2::Sha256::digest(&bytes).into();
let file = out_dir.join(format!("segment-{}-aggregated.bin", seg.output.number));
std::fs::write(&file, &bytes)?;
results.insert("segment_number".into(), seg.output.number.into());
results.insert("segment_block_hash".into(), seg.output.block_hash.to_string().into());
results.insert("segment_chain_len".into(), seg.output.chain_len.into());
results.insert("segment_public_values".into(), format!("0x{}", hex::encode(&pv)).into());
results.insert("segment_statement".into(), statement.to_string().into());
results.insert("segment_proof_sha256".into(), format!("0x{}", hex::encode(proof_hash)).into());
results.insert("segment_proof_bytes".into(), bytes.len().into());
results.insert("segment_proof_file".into(), file.display().to_string().into());
results.insert("segment_provers".into(), seg.output.provers.to_string().into());
println!("segment proof written to {} ({} bytes); statement {statement} proof sha256 0x{}", file.display(), bytes.len(), hex::encode(proof_hash));
Ok((statement, bytes.len()))
}
fn out_dir_of(out_path: Option<&str>) -> std::path::PathBuf {
out_path.and_then(|p| std::path::Path::new(p).parent().map(|d| d.to_path_buf())).filter(|d| !d.as_os_str().is_empty()).unwrap_or_else(|| std::path::PathBuf::from("."))
}
/// `--mode chain`: every fixture in order, consecutive on the chain (number and parent hash), each block's shards
/// proven compressed and aggregated with the previous block's aggregated proof (`AggInput.prev`, the chain rule),
/// every proof verified. One RESULT line per shard, per block (with the running totals) and for the chain.
fn run_chain(pinned: &pinned::Pinned, fixtures: &[String], prover: Address, out_path: Option<&str>, save_shards: bool, prev_path: Option<&str>) -> Result<()> {
if fixtures.is_empty() {
bail!("--chain needs at least one fixture");
}
let mut results = serde_json::Map::new();
results.insert("mode".into(), "chain".into());
results.insert("fixtures".into(), fixtures.iter().map(|f| serde_json::Value::from(f.as_str())).collect::<Vec<_>>().into());
let mut loaded = Vec::with_capacity(fixtures.len());
for path in fixtures {
let f = load_fixture(path)?;
if let Some(prev) = loaded.last().map(|(_, f): &(String, Fixture)| &f.block.env) {
if f.block.env.number != prev.number + 1 || f.block.env.parent_hash != prev.hash {
bail!("{path}: block {} (parent {}) does not follow block {} ({}); the chain needs consecutive fixtures", f.block.env.number, f.block.env.parent_hash, prev.number, prev.hash);
}
}
loaded.push((path.clone(), f));
}
let first = loaded[0].1.block.env.number;
let last = loaded[loaded.len() - 1].1.block.env.number;
println!("igneum-prove-host sources {}: chain of {} consecutive blocks {first}..={last}, prover payout {prover}, SP1_PROVER={}; {}", env!("IGNEUM_PROVE_SOURCES"), loaded.len(), std::env::var("SP1_PROVER").unwrap_or_else(|_| "cpu".into()), now());
// native first: every block's cut and every shard statement must reproduce the fixture before a proof is made
stage("native");
let mut built: Vec<(u64, B256, Vec<BuiltShard>, B256)> = Vec::with_capacity(loaded.len());
for (path, f) in &loaded {
let (outcome, pre_root, shards) = build_shards(&f.block, f.plan.shard_budget, prover);
let e = &f.expected;
if pre_root != e.pre_state_root || outcome.state_root != e.post_state_root || outcome.receipts_root != e.receipts_root || outcome.gas_used != e.gas_used || outcome.pgas_used != e.pgas_used || shards.len() != f.plan.shards.len() {
bail!("{path}: native execution differs from the fixture; regenerate it with igneum-prove-export");
}
if let Some((_, _, prev_shards, _)) = built.last() {
let prev_post = prev_shards.last().map(|s| s.output.post_root).unwrap_or_default();
if pre_root != prev_post {
bail!("{path}: block {} starts from state root {pre_root}, the previous block ended at {prev_post}: the state does not chain", f.block.env.number);
}
}
println!("RESULT chain native block {}: {} shard(s), pgas {}, gas {}, pre {} post {}", f.block.env.number, shards.len(), outcome.pgas_used, outcome.gas_used, pre_root, outcome.state_root);
built.push((f.block.env.number, f.block.env.hash, shards, pre_root));
}
// the previous segment's proof: the chain continues from it (its block must be the parent of the first fixture)
let mut prev: Option<proof_system::Sp1SegmentProof> = match prev_path {
None => None,
Some(p) => {
let bytes = std::fs::read(p).with_context(|| format!("read {p}"))?;
let proof: sp1_sdk::SP1ProofWithPublicValues = bincode::deserialize(&bytes).with_context(|| format!("{p} is not a bincode SP1 proof"))?;
let output = BlockOutput::from_bytes(proof.public_values.as_slice()).with_context(|| format!("{p}: public values are not a block statement"))?;
if output.number + 1 != first || output.block_hash != loaded[0].1.block.env.parent_hash {
bail!("--prev attests block {} ({}), the chain starts at block {first} with parent {}: the previous proof must be the parent block's", output.number, output.block_hash, loaded[0].1.block.env.parent_hash);
}
println!("RESULT chain prev: block {} chain_len {} (the chain continues from it)", output.number, output.chain_len);
Some(proof_system::Sp1SegmentProof { proof, output })
}
};
let base_len = prev.as_ref().map(|p| p.output.chain_len).unwrap_or(0);
let sp1 = setup_sp1(pinned, &mut results)?;
let chain_t = Instant::now();
let mut blocks_json = Vec::with_capacity(built.len());
let (mut shard_total, mut agg_total, mut shards_total) = (0.0f64, 0.0f64, 0usize);
let out_dir = out_dir_of(out_path);
let mut shard_files: Vec<String> = Vec::new();
for (number, _hash, shards, _) in &built {
let block_t = Instant::now();
let mut proofs: Vec<Sp1ShardProof> = Vec::with_capacity(shards.len());
let mut shard_secs = Vec::new();
// with --save-shards: what a shard's proof record carries (the statement, the proof's sha256, the file), so
// the app's segment path signs and submits every shard of the chain from one run
let mut shard_records: Vec<serde_json::Value> = Vec::new();
for s in shards {
let i = s.output.shard_index;
stage(&format!("chain block {number} compressed shard {i}"));
let p = sp1.prove_shard(&ShardWitness { input: s.input.clone() })?;
let dt = sp1.last_timing("compressed").unwrap_or_default().as_secs_f64();
let (ok, vdt) = sp1.verify_shard(&p.proof, &s.output);
println!("RESULT chain block {number} shard {i}: compressed prove {dt:.1} s, proof {} bytes, verify {:.3} s, {}, pgas {} at {}", bincode::serialize(&p.proof)?.len(), vdt.as_secs_f64(), if ok { "VERIFIED" } else { "VERIFY FAILED" }, p.output.pgas_used, now());
if !ok {
bail!("compressed proof of block {number} shard {i} did not verify");
}
shard_secs.push(dt);
shard_total += dt;
if save_shards {
let file = out_dir.join(format!("block-{number}-shard-{i}-compressed.bin"));
let bytes = bincode::serialize(&p.proof)?;
std::fs::write(&file, &bytes).with_context(|| format!("write {}", file.display()))?;
let proof_hash: [u8; 32] = sha2::Sha256::digest(&bytes).into();
shard_records.push(serde_json::json!({
"number": number, "block_hash": s.input.env.hash.to_string(), "shard": i, "statement": alloy_primitives::keccak256(s.output.to_bytes()).to_string(),
"proof_sha256": format!("0x{}", hex::encode(proof_hash)), "proof_bytes": bytes.len(), "proof_file": file.display().to_string(), "prove_seconds": dt,
}));
shard_files.push(file.display().to_string());
}
proofs.push(p);
}
shards_total += proofs.len();
stage(&format!("chain block {number} aggregate {} shards{}", proofs.len(), if prev.is_some() { " with the previous block proof" } else { "" }));
let seg = sp1.aggregate(prev.as_ref(), &proofs)?;
let adt = sp1.last_timing("aggregate").unwrap_or_default().as_secs_f64();
let sdt = sp1.last_timing("aggregate-stdin").unwrap_or_default().as_secs_f64();
agg_total += adt;
let claim = SegmentClaim::from_block(&seg.output);
let ok = sp1.verify_segment(&seg, &claim);
let vdt = sp1.last_timing("verify-block").unwrap_or_default().as_secs_f64();
let bytes = bincode::serialize(&seg.proof)?.len();
let block_s = block_t.elapsed().as_secs_f64();
let cumulative = chain_t.elapsed().as_secs_f64();
println!(
"RESULT chain block {number}: {} shards ({:.1} s of shard proofs), aggregate prove {adt:.1} s (stdin {sdt:.3} s, {} deferred proofs), proof {bytes} bytes, verify {vdt:.3} s, {}; chain_len {}, agg_vk {}; this block {block_s:.1} s, cumulative {cumulative:.1} s over {} block(s) at {}",
seg.output.shard_count,
shard_secs.iter().sum::<f64>(),
proofs.len() + usize::from(prev.is_some()),
if ok { "VERIFIED (shard program id, aggregator id and claim checked)" } else { "VERIFY FAILED" },
seg.output.chain_len,
seg.output.agg_vk,
blocks_json.len() + 1,
now()
);
if !ok {
bail!("the aggregated proof of block {number} did not verify");
}
let expected_len = base_len + blocks_json.len() as u64 + 1;
if seg.output.chain_len != expected_len {
bail!("block {number}: chain_len {} is not {expected_len}", seg.output.chain_len);
}
blocks_json.push(serde_json::json!({
"number": number, "shards": seg.output.shard_count, "shard_prove_seconds": shard_secs, "aggregate_prove_seconds": adt, "aggregate_stdin_seconds": sdt,
"aggregate_verify_seconds": vdt, "proof_bytes": bytes, "chain_len": seg.output.chain_len, "block_seconds": block_s, "cumulative_seconds": cumulative,
"post_root": seg.output.post_root.to_string(), "statement": alloy_primitives::keccak256(seg.output.to_bytes()).to_string(),
"shard_records": shard_records,
}));
prev = Some(seg);
}
let seg = prev.unwrap();
let total = chain_t.elapsed().as_secs_f64();
results.insert("base_chain_len".into(), base_len.into());
let (statement, bytes) = segment_results(&seg, &out_dir, &mut results)?;
println!(
"RESULT chain: {} blocks {first}..={last}, {shards_total} shards, shard proofs {shard_total:.1} s, aggregation {agg_total:.1} s, end to end {total:.1} s; final proof {bytes} bytes attests chain_len {} (statement {statement}), pre {} post {} provers {} at {}",
built.len(),
seg.output.chain_len,
built[0].3,
seg.output.post_root,
seg.output.provers,
now()
);
results.insert("blocks".into(), blocks_json.into());
results.insert("first".into(), first.into());
results.insert("last".into(), last.into());
results.insert("shards".into(), shards_total.into());
results.insert("shard_prove_seconds_total".into(), shard_total.into());
results.insert("aggregate_prove_seconds_total".into(), agg_total.into());
results.insert("chain_seconds".into(), total.into());
results.insert("shard_proof_files".into(), shard_files.into_iter().map(serde_json::Value::from).collect::<Vec<_>>().into());
drop(sp1);
finish(results, out_path.map(|s| s.to_string()))
}
/// `--mode aggregate`: the live aggregator (the app's segment step, spec 7.8). `--proofs` names the shard proof
/// files of one or more consecutive chain blocks: blocks separated by `;`, a block's shards by `,` (what the node's
/// pool holds, `igneum_getProofBytes`); `--parent` is the first block's parent chain block hash; `--prev` the
/// previous segment's aggregated proof when the chain continues. Each block is aggregated with the previous
/// block's proof in one process (one key setup); the output is the last block's aggregated proof, its public
/// values, the statement and the proof hash the segment record carries.
fn run_aggregate(pinned: &pinned::Pinned, proofs: &str, parent: &str, prev_path: Option<&str>, out_path: Option<&str>) -> Result<()> {
let first_parent: B256 = parent.parse().context("--parent is not 32 bytes of hex")?;
let mut results = serde_json::Map::new();
results.insert("mode".into(), "aggregate".into());
let mut blocks: Vec<Vec<Sp1ShardProof>> = Vec::new();
let mut parent_hash = first_parent;
for group in proofs.split(';').map(str::trim).filter(|s| !s.is_empty()) {
let mut shards: Vec<Sp1ShardProof> = Vec::new();
for path in group.split(',').map(str::trim).filter(|s| !s.is_empty()) {
let bytes = std::fs::read(path).with_context(|| format!("read {path}"))?;
let proof: sp1_sdk::SP1ProofWithPublicValues = bincode::deserialize(&bytes).with_context(|| format!("{path} is not a bincode SP1 proof"))?;
let output = ShardOutput::from_bytes(proof.public_values.as_slice()).with_context(|| format!("{path}: public values are not a shard statement"))?;
if let Some(c) = pinned::claimed_program_id(&proof) {
if c != pinned.shard_id {
bail!("{path}: shard proof made with program id {c}, ours is {}", pinned.shard_id);
}
}
shards.push(Sp1ShardProof { proof, output, parent_hash });
}
if shards.is_empty() {
bail!("a block in --proofs names no file");
}
shards.sort_by_key(|s| s.output.shard_index);
if let Some(prev) = blocks.last().and_then(|b: &Vec<Sp1ShardProof>| b.first()) {
if shards[0].output.number != prev.output.number + 1 {
bail!("block {} does not follow block {}: the blocks of --proofs must be consecutive", shards[0].output.number, prev.output.number);
}
}
parent_hash = shards[0].output.block_hash;
blocks.push(shards);
}
if blocks.is_empty() {
bail!("--proofs names no file");
}
let mut prev = match prev_path {
None => None,
Some(p) => {
let bytes = std::fs::read(p).with_context(|| format!("read {p}"))?;
let proof: sp1_sdk::SP1ProofWithPublicValues = bincode::deserialize(&bytes).with_context(|| format!("{p} is not a bincode SP1 proof"))?;
let output = BlockOutput::from_bytes(proof.public_values.as_slice()).with_context(|| format!("{p}: public values are not a block statement"))?;
Some(proof_system::Sp1SegmentProof { proof, output })
}
};
let first = blocks[0][0].output.number;
let last = blocks[blocks.len() - 1][0].output.number;
println!("igneum-prove-host sources {}: aggregate blocks {first}..={last} ({} shard proofs){}; {}", env!("IGNEUM_PROVE_SOURCES"), blocks.iter().map(|b| b.len()).sum::<usize>(), prev.as_ref().map(|p| format!(", chaining to block {} (chain_len {})", p.output.number, p.output.chain_len)).unwrap_or_default(), now());
let sp1 = setup_sp1(pinned, &mut results)?;
let t_all = Instant::now();
let mut per_block = Vec::new();
for shards in &blocks {
let number = shards[0].output.number;
stage(&format!("aggregate block {number}, {} shards", shards.len()));
let deferred = shards.len() + usize::from(prev.is_some());
let seg = sp1.aggregate(prev.as_ref(), shards)?;
let adt = sp1.last_timing("aggregate").unwrap_or_default().as_secs_f64();
let sdt = sp1.last_timing("aggregate-stdin").unwrap_or_default().as_secs_f64();
let claim = SegmentClaim::from_block(&seg.output);
let ok = sp1.verify_segment(&seg, &claim);
let vdt = sp1.last_timing("verify-block").unwrap_or_default().as_secs_f64();
println!("RESULT aggregate block {number}: {} shards, prove {adt:.1} s (stdin {sdt:.3} s, {deferred} deferred proofs), proof {} bytes, verify {vdt:.3} s, {}; chain_len {}, post {} at {}", seg.output.shard_count, bincode::serialize(&seg.proof)?.len(), if ok { "VERIFIED" } else { "VERIFY FAILED" }, seg.output.chain_len, seg.output.post_root, now());
if !ok {
bail!("the aggregated proof of block {number} did not verify");
}
per_block.push(serde_json::json!({ "number": number, "shards": seg.output.shard_count, "aggregate_prove_seconds": adt, "aggregate_stdin_seconds": sdt, "deferred_proofs": deferred, "aggregate_verify_seconds": vdt, "chain_len": seg.output.chain_len }));
prev = Some(seg);
}
let seg = prev.unwrap();
let (statement, bytes) = segment_results(&seg, &out_dir_of(out_path), &mut results)?;
println!("RESULT aggregate: blocks {first}..={last} in {:.1} s, final proof {bytes} bytes, chain_len {}, statement {statement} at {}", t_all.elapsed().as_secs_f64(), seg.output.chain_len, now());
results.insert("blocks".into(), per_block.into());
results.insert("first".into(), first.into());
results.insert("last".into(), last.into());
results.insert("aggregate_seconds".into(), t_all.elapsed().as_secs_f64().into());
drop(sp1);
finish(results, out_path.map(|s| s.to_string()))
}
/// `--mode verify-segment --proof <file> --statement 0x..`: the node's verifier for an aggregated segment record
/// (spec 7.8): SP1's light verifier against the PINNED aggregator key, the public values' keccak against the
/// statement, the shard program id inside the statement against ours, the aggregator id inside it against ours
/// (or zero when the proof chains to nothing). Exit 0 = verified, 3 = not verified.
fn run_verify_segment(pinned: &pinned::Pinned, proof_path: &str, statement: &str) -> Result<()> {
use sp1_sdk::blocking::{LightProver, Prover};
let bytes = std::fs::read(proof_path).with_context(|| format!("read {proof_path}"))?;
let want: B256 = statement.parse().context("statement is not 32 bytes of hex")?;
stage("setup");
let t = Instant::now();
let verifier = LightProver::new();
println!("RESULT setup: {:.3} s (light verifier, pinned aggregator key), aggregator id {} shard program id {} at {}", t.elapsed().as_secs_f64(), pinned.agg_id, pinned.shard_id, now());
stage("verify-segment");
let t = Instant::now();
let proof: sp1_sdk::SP1ProofWithPublicValues = bincode::deserialize(&bytes).context("the file is not a bincode SP1 proof")?;
let got = alloy_primitives::keccak256(proof.public_values.as_slice());
let output = BlockOutput::from_bytes(proof.public_values.as_slice());
let claimed = pinned::claimed_program_id(&proof);
let same_program = claimed == Some(pinned.agg_id);
let crypto_ok = same_program && verifier.verify(&proof, &pinned.agg_vk, None).is_ok();
let ids_ok = output.as_ref().map(|o| o.shard_vk == pinned.shard_id && (o.agg_vk == pinned.agg_id || (o.chain_len == 1 && o.agg_vk == B256::ZERO))).unwrap_or(false);
let ok = crypto_ok && ids_ok && got == want;
let dt = t.elapsed().as_secs_f64();
let program = match claimed {
Some(c) if same_program => format!("aggregator id {c} (ours)"),
Some(c) => format!("aggregator id {c} IS NOT OURS {} (the aggregator runs another guest build)", pinned.agg_id),
None => "not a compressed proof".to_string(),
};
match &output {
Some(o) => println!(
"RESULT verify-segment: {} in {dt:.3} s; block {} ({}) chain_len {} shards {} statement {got} (want {want}) {program}; inner ids {}; proof {} bytes at {}",
if ok { "VERIFIED" } else { "NOT VERIFIED" },
o.number,
o.block_hash,
o.chain_len,
o.shard_count,
if ids_ok { "ours".to_string() } else { format!("NOT OURS (shard {} agg {} chain_len {})", o.shard_vk, o.agg_vk, o.chain_len) },
bytes.len(),
now()
),
None => println!("RESULT verify-segment: NOT VERIFIED in {dt:.3} s; public values are not a block statement; {program} at {}", now()),
}
if ok {
Ok(())
} else {
std::process::exit(3)
}
}
fn check_shard_output(out: &ShardOutput, native: &ShardOutput) -> Result<()> {
if out != native {
bail!("the guest's public values differ from the native run:\n guest {out:?}\n native {native:?}");
}
Ok(())
}
fn finish(results: serde_json::Map<String, serde_json::Value>, out_path: Option<String>) -> Result<()> {
if let Some(p) = out_path {
std::fs::write(&p, serde_json::to_string_pretty(&serde_json::Value::Object(results))?)?;
println!("results written to {p}");
}
Ok(())
}
/// Fixtures on both sides of the fee switch (5 October 2026, `fees_v1_activation_daa`): one pinned guest, two
/// tables. The prototype fixtures carry no `fees` field (the devnet schedule, prototype, never); the v1 fixtures
/// were cut from a simnet run whose override file set the switch at DAA score 800, a prototype block below it and
/// two v1 blocks above it from ONE export, replayed from genesis across the switch (the exporter's state-root check
/// on every segment is what shows the port's schedule is the node's on both sides).
#[cfg(test)]
mod fee_switch_tests {
use igneum_prove_core::config::{FeeParams, FeeSchedule};
use igneum_prove_core::shard::{build_shards, shard_statement};
use igneum_prove_core::Fixture;
fn load(name: &str) -> Fixture {
let path = format!("{}/../../fixtures/{name}", env!("CARGO_MANIFEST_DIR"));
serde_json::from_str(&std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("{path}: {e}"))).unwrap()
}
/// The native run of a fixture reproduces its plan and its expected values under its own schedule.
fn native_matches(f: &Fixture) {
let prover = alloy_primitives::Address::from_slice(&[0x19; 20]);
let (outcome, pre_root, shards) = build_shards(&f.block, f.plan.shard_budget, prover);
assert_eq!(pre_root, f.expected.pre_state_root);
assert_eq!(outcome.state_root, f.expected.post_state_root);
assert_eq!(outcome.state_root, f.expected.node_state_root, "the node's root");
assert_eq!(outcome.pgas_used, f.expected.pgas_used);
assert_eq!(shards.len(), f.plan.shards.len());
for (s, x) in shards.iter().zip(&f.plan.shards) {
assert_eq!((s.output.pre_root, s.output.post_root, s.output.pgas_used, s.output.gas_used), (x.pre_root, x.post_root, x.pgas_used, x.gas_used), "shard {}", x.index);
assert_eq!(s.output.to_bytes().len(), 328, "the statement layout is unchanged");
}
}
#[test]
fn a_fixture_without_fees_is_the_prototype_side() {
let f = load("block-338-shard1.json");
assert_eq!(f.block.fees, FeeSchedule::DEVNET);
assert_eq!(f.block.env.daa_score, 0);
let set = f.block.fees.at(f.block.env.daa_score);
assert_eq!(set, FeeParams::PROTOTYPE);
assert_eq!(f.plan.shard_budget, 7_500_000);
assert!(f.plan.consensus);
native_matches(&f);
}
#[test]
fn the_prototype_block_below_the_switch() {
let f = load("fees-switch-prototype.json");
assert_eq!(f.block.fees.v1_activation_daa, 800);
assert!(f.block.env.daa_score < 800, "daa {}", f.block.env.daa_score);
assert_eq!(f.block.fees.at(f.block.env.daa_score), FeeParams::PROTOTYPE);
assert_eq!(f.plan.shard_budget, 7_500_000);
assert_eq!(f.block.env.base_fee_exec, 1_000_000_000, "the prototype floor");
native_matches(&f);
}
#[test]
fn the_v1_blocks_at_and_above_the_switch() {
for (name, shards) in [("fees-v1-shards2.json", 2usize), ("fees-v1-shards3.json", 3)] {
let f = load(name);
assert_eq!(f.block.fees.v1_activation_daa, 800);
assert!(f.block.env.daa_score >= 800, "{name}: daa {}", f.block.env.daa_score);
assert_eq!(f.block.fees.at(f.block.env.daa_score), FeeParams::CALIBRATED_V1);
assert_eq!(f.plan.shard_budget, 30_000, "{name}");
assert!(f.plan.consensus);
assert_eq!(f.plan.shards.len(), shards, "{name}");
assert_eq!(f.block.env.base_fee_exec, 100_000_000_000, "{name}: the v1 execution floor");
assert_eq!(f.block.env.base_fee_proving, 10_000_000_000_000, "{name}: the v1 proving floor");
// every executed transaction carries the v1 intrinsic (300) and no plain transfer costs more
let txs: usize = f.block.blocks.iter().map(|b| b.txs.len()).sum();
assert!(f.expected.pgas_used >= 300 * txs as u64, "{name}: pgas {} for {txs} transactions", f.expected.pgas_used);
native_matches(&f);
}
}
/// The same shard input under the other table gives another statement: the native veto is what pins the
/// schedule a prover claims (the statement layout does not carry it).
#[test]
fn moving_the_switch_changes_the_statement() {
let f = load("fees-v1-shards2.json");
let prover = alloy_primitives::Address::from_slice(&[0x19; 20]);
let (_, _, shards) = build_shards(&f.block, f.plan.shard_budget, prover);
let mut input = shards[0].input.clone();
let v1 = shard_statement(&input);
assert_eq!(v1.to_bytes(), shards[0].output.to_bytes());
// the switch moved past this block: the prototype table meters the same transactions with the intrinsic
// 200 but a modexp entry 100x v1's (1,000 + 10 per byte against 10 + 1 per 10 bytes), so this modexp shard
// costs MORE under the prototype (28,228 against 22,172 pgas on fees-v1-shards2): another pgas total,
// another statement either way
input.fees = FeeSchedule { base: FeeParams::PROTOTYPE, v1_activation_daa: u64::MAX };
let proto = shard_statement(&input);
assert_ne!(proto.pgas_used, v1.pgas_used, "prototype {} vs v1 {}", proto.pgas_used, v1.pgas_used);
assert!(proto.pgas_used > v1.pgas_used, "the prototype modexp entry is the larger one: prototype {} vs v1 {}", proto.pgas_used, v1.pgas_used);
assert_ne!(proto.to_bytes(), v1.to_bytes());
// and the other way on the prototype fixture: the switch at 0 makes it v1
let p = load("fees-switch-prototype.json");
let (_, _, pshards) = build_shards(&p.block, p.plan.shard_budget, prover);
let mut pin = pshards[0].input.clone();
let before = shard_statement(&pin);
pin.fees = FeeSchedule { base: FeeParams::PROTOTYPE, v1_activation_daa: 0 };
let after = shard_statement(&pin);
// under v1 the floors rise to 100 gwei, above these transactions' fee cap, so they skip: another pgas
// total (the v1 intrinsic per skipped copy), another post-root, another statement
assert_ne!(after.pgas_used, before.pgas_used);
assert_ne!(after.post_root, before.post_root, "the v1 floors change what runs and what is burned, so the post-root moves");
assert_ne!(after.to_bytes(), before.to_bytes());
}
}