diff --git a/docs/benchmarks/repro.md b/docs/benchmarks/repro.md new file mode 100644 index 000000000..4c4cc72e0 --- /dev/null +++ b/docs/benchmarks/repro.md @@ -0,0 +1,219 @@ +# The reproducible benchmark package + +6 October 2026 (built on the evening of 5 October). One command per platform reproduces the numbers on the bench table +on an outsider's own machine on day one: `bench/repro.sh` (Linux, macOS) and `bench/repro.ps1` (Windows), shipped as +`igneum-repro-.tar.gz` and `.zip` next to the public downloads (`packaging/ota/publish-public.sh --repro`, aliases +`/public/igneum-repro.tar.gz` and `/public/igneum-repro.zip`; not deployed tonight). Package v0.1.0-repro was built from +commit `39141f5` plus the package sources of branch `repro-bench`, commit `66ccd25` (the shipped binaries were built from that +tree before the commit; the script fixes after the runs change no binary), +and run end to end on the three project machines. The deltas against the bench log are below. + +Why it exists: `docs/evidence.md` has 30 claims and none is reproduced externally. The ladder from "tested by the team" +to "reproduced externally" needs "the command published and a third party's run with the same result". This is the +command. The reward for running it is section 8.1 of `docs/benchmarks/proving-e2e.md`, quoted unchanged below. + +## 1. What one run does + +| Step | What runs | Binary | What it reports | +|---|---|---|---| +| 1 | The machine | the workers' `--list` | OS, CPU, memory; every GPU each worker sees (name, driver, memory) | +| 2 | The lottery hash vectors of the published genesis pack (`proto-cuda/packs/igneum-genesis-mh`: seed `igneum-genesis`, day 2026-10-03, generator 2, program id `bcc1248b10cc90f2`, memory-hard 1 GiB dataset) | `igneum-pow check-pack` on the CPU; `igneum-worker-cuda --bench`, `igneum-worker-opencl --bench --pack`, `igneum-bench --pack` on every GPU | bit-exact or not: 3 warps, 96 lanes, the 256 MiB cache FNV, on the CPU through the Rust interpreter and on every GPU through its own compiler (NVRTC, the vendor's OpenCL compiler, Metal) | +| 3 | The hash benchmark, 120 s per card at 1 GiB, `--batch-log2 24` | the same workers, `--seconds 120` | MH/s over the summed dispatch time, the hashes done, the fingerprint of the first 2^24 outputs at base nonce 0 (FNV-1a 64 over 16.7 million hashes: equal on two machines means every one of them agreed) | +| 4 | The random-read probe at 4, 64, 256 and 1024 MiB | `--memprobe` | dependent random 4-byte reads per second (the hash's access pattern) and their latency at 256 lanes, eight independent chains, 16 and 64-byte lines, the coalesced stream, an integer chain | +| 5 | The chip-resistance sweep: the same program at 4, 64, 256 and 1024 MiB, 5 batches each | `--bench --dataset-mib N` (`--sweep` on Metal) | MH/s per size; in-cache rate over the 1 GiB rate; the hash's share of the card's random-read ceiling at 1 GiB (MH/s x 128 loads against the chase) | +| 6 | One fixture shard proven with the pinned guest and verified (optional) | `igneum-prove-host --mode shard --shard 0`, then `--mode verify` | execute, core and compressed proof times, proof bytes, VERIFIED or not; skipped with the reason when there is no 12 GB NVIDIA card or no prover host | +| 7 | The result | the script | `results/igneum-repro--