From eca4111c34aab4c1f8a7a2462277755bbbce22d0 Mon Sep 17 00:00:00 2001 From: igneum-labs <337424239+igneum-labs@users.noreply.github.com> Date: Tue, 6 Oct 2026 01:16:24 +0000 Subject: [PATCH] Reproducible benchmark package: one command per platform (bench/repro.sh, repro.ps1), run on the Mac and PC 2; PC 1 owed bench/: repro.sh (Linux, macOS) and repro.ps1 (Windows) print the machine, check the genesis pack's lottery hash vectors on the CPU and every GPU, hash for 120 s per card at 1 GiB, probe random reads at 4, 64, 256 and 1024 MiB, sweep the same sizes, prove and verify one fixture shard when a 12 GB NVIDIA card and a prover host exist, and write one JSON result (the bench table's row shape, every command, every binary's sha256, the tolerances) plus a table. make-package.sh builds igneum-repro-.tar.gz and .zip from the existing cross-build scripts; ingest.mjs checks a result against bench/reference.json and adds "reproduced externally" rows to site/miner-bench.json only when every number agrees; publish-public.sh --repro publishes next to the downloads (not deployed). The workers gained --list, --bench --pack --seconds --dataset-mib and --memprobe (CUDA from the readwidth branch, OpenCL memprobe from opencl-rdna4, Metal new) with RESULT and DEVICE lines; igneum-pow gained check-pack; the pack reader accepts a string-seed pack. Runs (docs/benchmarks/repro.md, the bench-log entry, evidence rows 4, 5, 15): the Mac under the measure lock (Metal 27.674 MH/s, Apple OpenCL 26.875, 96/96 on CPU, Metal and OpenCL, 2^24 fingerprint 25f96e7dce90bd4e, 3.40 G random reads/s, CPU verify 0.611 ms per warp); PC 2 (96/96 through NVRTC, NVIDIA OpenCL and AMD OpenCL with the same fingerprint; gfx1036 3.312 MH/s; the 5090 rows beside the app's live miner, 62.4 MH/s, not the card's; the live host proved and verified the shard through step 6, compressed 33.9 s). PC 1 went down before its slot; owed. Classes closed: the PC job parses the package script before touching a card and bench/ is in the Windows 5.1 parse job; the card-off is confirmed by nvidia-smi's compute-apps list and the prover's state read from settings.json, never api/state; PowerShell case-only variable pairs fail CI (bench/jobs/ps-case-check.sh); the package builder keeps the Mac workers outside the stage and checks every binary with file -b. Co-Authored-By: Claude Fable 5.1 --- .github/workflows/windows.yml | 3 +- .gitignore | 3 + bench/README.md | 80 +++ bench/ingest.mjs | 104 ++++ bench/jobs/collect-results.mjs | 33 ++ bench/jobs/pc-repro.ps1 | 168 +++++++ bench/jobs/ps-case-check.sh | 13 + bench/make-package.sh | 93 ++++ bench/reference.json | 64 +++ bench/repro.ps1 | 242 +++++++++ bench/repro.sh | 290 +++++++++++ docs/bench-log.md | 110 +++++ .../igneum-repro-macos-20261005T215136Z.json | 13 + .../igneum-repro-macos-20261005T215136Z.log | 467 ++++++++++++++++++ .../igneum-repro-macos-20261005T215136Z.md | 24 + .../igneum-repro-windows-20261006T004515Z.log | 288 +++++++++++ .../igneum-repro-windows-20261006T005218Z.log | 316 ++++++++++++ .../igneum-repro-windows-20261006T005622Z.log | 316 ++++++++++++ docs/benchmarks/repro.md | 218 ++++++++ docs/evidence.md | 8 +- igneum-pow/src/main.rs | 127 ++++- packaging/ota/publish-public.sh | 34 +- proto-cuda/nvrtc/packfile.h | 10 +- proto-cuda/nvrtc/worker.cpp | 308 +++++++++++- proto-metal/main.swift | 341 ++++++++++++- proto-opencl/host.c | 132 ++++- site/bench.html | 39 +- site/build.mjs | 2 +- site/miners.html | 2 +- tools/ci/windows/check-ps51.ps1 | 2 +- 30 files changed, 3805 insertions(+), 45 deletions(-) create mode 100644 bench/README.md create mode 100644 bench/ingest.mjs create mode 100644 bench/jobs/collect-results.mjs create mode 100644 bench/jobs/pc-repro.ps1 create mode 100755 bench/jobs/ps-case-check.sh create mode 100755 bench/make-package.sh create mode 100644 bench/reference.json create mode 100644 bench/repro.ps1 create mode 100755 bench/repro.sh create mode 100644 docs/benchmarks/repro-2026-10-06/mac/igneum-repro-macos-20261005T215136Z.json create mode 100644 docs/benchmarks/repro-2026-10-06/mac/igneum-repro-macos-20261005T215136Z.log create mode 100644 docs/benchmarks/repro-2026-10-06/mac/igneum-repro-macos-20261005T215136Z.md create mode 100644 docs/benchmarks/repro-2026-10-06/pc2/igneum-repro-windows-20261006T004515Z.log create mode 100644 docs/benchmarks/repro-2026-10-06/pc2/igneum-repro-windows-20261006T005218Z.log create mode 100644 docs/benchmarks/repro-2026-10-06/pc2/igneum-repro-windows-20261006T005622Z.log create mode 100644 docs/benchmarks/repro.md diff --git a/.github/workflows/windows.yml b/.github/workflows/windows.yml index 377b452e9..2dfd30789 100644 --- a/.github/workflows/windows.yml +++ b/.github/workflows/windows.yml @@ -47,6 +47,7 @@ on: - 'proving/windows-wsl2/**' - 'relay/clients/**' - 'relay/playbooks/**' + - 'bench/**' - 'brand/icons/**' - 'tools/ci/windows/**' - '.github/workflows/windows.yml' @@ -85,7 +86,7 @@ jobs: Install-Module -Name PSScriptAnalyzer -Force -Scope CurrentUser -AllowClobber } Import-Module PSScriptAnalyzer - $folders = @('proto-cuda/windows-app', 'proto-cuda/windows-miner', 'proto-cuda/windows-node', 'proving/windows-wsl2', 'relay/clients', 'relay/playbooks', 'packaging/windows', 'tools/ci/windows') + $folders = @('proto-cuda/windows-app', 'proto-cuda/windows-miner', 'proto-cuda/windows-node', 'proving/windows-wsl2', 'relay/clients', 'relay/playbooks', 'packaging/windows', 'tools/ci/windows', 'bench') $total = 0 foreach ($f in $folders) { $results = Invoke-ScriptAnalyzer -Path $f -Recurse -Severity Warning, Error -ExcludeRule PSAvoidUsingWriteHost, PSUseShouldProcessForStateChangingFunctions, PSUseSingularNouns, PSAvoidUsingPositionalParameters diff --git a/.gitignore b/.gitignore index e60bf5fb6..4f0427e83 100644 --- a/.gitignore +++ b/.gitignore @@ -31,3 +31,6 @@ vendor/igneum-node-ship/ # Trademark instruction packs name the director and the applicant company; never in the repository brand/trademark/pbip-pack/ brand/trademark/*.zip + +# the reproducible benchmark package (bench/make-package.sh output) +bench/build/ diff --git a/bench/README.md b/bench/README.md new file mode 100644 index 000000000..33c4d68e0 --- /dev/null +++ b/bench/README.md @@ -0,0 +1,80 @@ +# Igneum reproducible benchmark + +One command reproduces the numbers on the Igneum bench table on your own machine, on day one, with no secrets, no node, no +wallet and no network connection. The result is one JSON file you can send back, signed by nothing, that names every +command run and the sha256 of every binary used. + +## Run it + +| Platform | Command | +|---|---| +| Linux (x86_64, NVIDIA driver or any OpenCL driver) | `tar xzf igneum-repro-.tar.gz && cd igneum-repro- && ./repro.sh` | +| macOS (Apple silicon) | the same tar.gz; `./repro.sh` | +| Windows 10 or 11 (x64) | unzip `igneum-repro-.zip`, then in PowerShell: `powershell -ExecutionPolicy Bypass -File repro.ps1` | + +Options: `--seconds 120` (per card, the default), `--only cuda:0,opencl:1` (one card; the indexes are the ones `--list` +prints at the start), `--no-probe`, `--no-sweep`, `--no-prove`, `--out `. On Windows the same as `-Seconds`, +`-Only`, `-NoProbe`, `-NoSweep`, `-NoProve`, `-Out`. + +Stop mining on the card first. A number taken while a miner, a game or another benchmark holds the card is not a number; +the result file does not know, so you do. + +Time: about 4 minutes per card (120 s hash benchmark, three short sweep sizes, the probe table), plus the CPU check (one +second). The optional proving step takes about a minute on an RTX 5090 class card and needs a prover host you built +(below). + +## What runs, in order + +| Step | What | What it needs | What it reports | +|---|---|---|---| +| 1 | The machine | nothing | OS, CPU, memory, every GPU each worker sees (name, driver, memory) | +| 2 | The lottery hash vectors of the published genesis pack (`packs/igneum-genesis-mh`: seed `igneum-genesis`, day 2026-10-03, generator 2, memory-hard 1 GiB dataset, program id `bcc1248b10cc90f2`) | a CPU; every GPU present | bit-exact or not: 3 warps, 96 lanes, the cache FNV, on the CPU through the Rust interpreter and on every GPU through its own compiler (NVRTC, the OpenCL driver, Metal) | +| 3 | The hash benchmark, 120 s per card at 1 GiB | a GPU with 1.3 GB free | MH/s, the hashes done, and the fingerprint of the first 2^24 outputs (equal fingerprints on two machines mean every one of those 16.7 million hashes agreed) | +| 4 | The random-read probe at 4, 64, 256 and 1024 MiB | a GPU | dependent random 4-byte reads per second (the hash's access pattern) and their latency, eight independent chains, 64-byte lines, the coalesced stream bandwidth, an integer chain | +| 5 | The chip-resistance sweep: the same program at 4, 64, 256 and 1024 MiB | a GPU | MH/s per size; the ratio in-cache to 1 GiB; the hash's share of the card's random-read ceiling (MH/s x 128 loads against step 4's chase at 1 GiB) | +| 6 | One fixture shard proven with the pinned guest and verified (optional) | an NVIDIA card with 12 GB or more, Linux or WSL2, a built `igneum-prove-host` (`IGNEUM_PROVE_HOST`) | execute, core and compressed proof times, the verify time, VERIFIED or not; otherwise the line says why it was skipped | +| 7 | The result | nothing | `results/igneum-repro--