a08e5f77c1
tools/ci/test-map.json gains bench:pc1-packs (GPU-02, GPU-03, GPU-05, POW-08 on PC 1's 5090), bench:pc1-amd (GPU-01, GPU-05 on the RX 7600), bench:fleet-pods (GPU-01, GPU-03, POW-08 on the rented cards), census:class-v6 (POW-02), POW-07 under suite:pow (the FP32 branch unreachable on master) and UX-08's published-tuning half; GPU-04 and GPU-07 read NOT RUN with the Devnet 3 off order as the reason; the harness map page is regenerated. tools/ci/p01-vectors.py is P01 part A as the hash lane shaped it: the real worker driven with pool.rs's job lines at an all-pass target, every found hash diffed against the CPU reference list, the evidence JSON with the counts, the first ten disagreements, the pack and program ids, the device line and the manifest sha; exit 1 on any disagreement or unanswered nonce; self-tested with a fake worker (clean PASS, one wrong hash FAIL naming the nonce, one unanswered nonce FAIL naming it). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
376 lines
18 KiB
JSON
376 lines
18 KiB
JSON
{
|
|
"title": "Igneum 2.0 test harness map",
|
|
"registry": "docs/plans/igneum-2.0-test-registry.json",
|
|
"rule": "a case maps to a cell only where the cell's tests visibly answer it; coverage names what the cell proves and what remains; a mapped cell's green writes RUNNING, PASS only when coverage is full and the evidence file exists; an automated case with no cell reads NOT RUN with its reason, never PASS by inference",
|
|
"cells": {
|
|
"suite:pow": {
|
|
"command": "tools/build-remote.sh --priority gate -- test --release (from igneum-pow/)",
|
|
"box_class": "suite (gate priority)",
|
|
"fixtures": [
|
|
"F0",
|
|
"F1"
|
|
],
|
|
"cases": [
|
|
"POW-02",
|
|
"POW-07",
|
|
"POW-08"
|
|
],
|
|
"coverage": {
|
|
"POW-02": "partial: program derivation, ds55 geometry, the v6 fold, the packs and the spec read-back tests; the independent re-implementation and the index-fold census are the hash lane's harnesses",
|
|
"POW-08": "partial: the freeze list and the generator version pin; the public-claim scrub is the site gate's",
|
|
"POW-07": "the mixed FP32 branch is excluded and unreachable: no class flag reaches an FP op in igneum-pow on master (the mixedfp lane's branch is not on master); evidence the grep of the emitters on master, recorded by the hash lane"
|
|
}
|
|
},
|
|
"suite:app": {
|
|
"command": "tools/build-remote.sh --priority gate -- test --release (from app/igneum-app/)",
|
|
"box_class": "suite (gate priority)",
|
|
"fixtures": [
|
|
"F0",
|
|
"F1"
|
|
],
|
|
"cases": [
|
|
"UX-02",
|
|
"UX-03",
|
|
"UX-07",
|
|
"VER-08"
|
|
],
|
|
"coverage": {
|
|
"UX-02": "partial: the engine state machine's pause, stop and knob tests; the operator study is UX-01's",
|
|
"UX-03": "partial: the card and earnings rendering tests; the net-earnings model against real bills is COM/ECO evidence",
|
|
"UX-07": "partial: the refused-update and manifest tests; the update-return lane's install classes are its own rows",
|
|
"VER-08": "partial: the app's own state-word tests; the wallet and light client rows are VER-01 to VER-03"
|
|
}
|
|
},
|
|
"suite:core": {
|
|
"command": "tools/build-remote.sh --priority gate -- test --release -p kaspa-consensus-core (from the node fork)",
|
|
"box_class": "suite (gate priority)",
|
|
"fixtures": [
|
|
"F0",
|
|
"F1"
|
|
],
|
|
"cases": [
|
|
"FIN-01",
|
|
"INC-01",
|
|
"ROT-06"
|
|
],
|
|
"coverage": {
|
|
"FIN-01": "partial: the finality field and checkpoint tests on the object; ordering under load is the consensus suite and the sim",
|
|
"INC-01": "partial: emission, fee and payout arithmetic tests; the reconciliation over a live chain is the economics lane's",
|
|
"ROT-06": "partial: the dataset activation and digest tests; a live activation is the fast-time harness"
|
|
}
|
|
},
|
|
"suite:consensus": {
|
|
"command": "tools/build-remote.sh --priority gate -- test --release -p kaspa-consensus (from the node fork)",
|
|
"box_class": "suite (gate priority)",
|
|
"fixtures": [
|
|
"F0",
|
|
"F1"
|
|
],
|
|
"cases": [
|
|
"FIN-01",
|
|
"FIN-03",
|
|
"FIN-04",
|
|
"FIN-05",
|
|
"FIN-07",
|
|
"ROT-05",
|
|
"ZKP-01"
|
|
],
|
|
"coverage": {
|
|
"FIN-01": "partial: the finality processes' agreement tests",
|
|
"FIN-03": "partial: rule v4's anchored table and authority expiry tests (bee41b5e)",
|
|
"FIN-04": "partial: the frozen table without its time expiry (rule v4)",
|
|
"FIN-05": "partial: key succession and another scheme's vote refused",
|
|
"FIN-07": "partial: majority-continuity recovery (rule v4)",
|
|
"ROT-05": "partial: the finality-stopped pause tests",
|
|
"ZKP-01": "partial: a proof-less or wrong-statement block refused by consensus"
|
|
}
|
|
},
|
|
"suite:exec": {
|
|
"command": "tools/build-remote.sh --priority gate -- test --release -p igneum-exec (from the node fork)",
|
|
"box_class": "suite (gate priority)",
|
|
"fixtures": [
|
|
"F0",
|
|
"F1"
|
|
],
|
|
"cases": [
|
|
"ZKP-01",
|
|
"ZKP-03",
|
|
"OPS-04",
|
|
"EVM-02"
|
|
],
|
|
"coverage": {
|
|
"ZKP-01": "partial: a real proof verifies, a wrong statement and garbage are refused (nativeverify)",
|
|
"ZKP-03": "partial: a snapshot under another digest refused, the epoch streams bound to the digest",
|
|
"OPS-04": "partial: snapshot install, replay and genesis recapture after a crash",
|
|
"EVM-02": "partial: the replay tests on the day stream; the signature and chain-id binding vectors are an EVM harness still to map"
|
|
}
|
|
},
|
|
"suite:miner": {
|
|
"command": "tools/build-remote.sh --priority gate -- test --release -p igneum-miner (from the node fork)",
|
|
"box_class": "suite (gate priority)",
|
|
"fixtures": [
|
|
"F0",
|
|
"F1"
|
|
],
|
|
"cases": [
|
|
"UX-06",
|
|
"UX-04",
|
|
"POW-01"
|
|
],
|
|
"coverage": {
|
|
"UX-06": "partial: the miner's own template selection tests",
|
|
"UX-04": "partial: payout label and share accounting tests; custody is the pool lane's row",
|
|
"POW-01": "partial: the pack re-check seam against the pack's 96 vectors (recheck_pack); the million-vector campaign is harness:p01-vectors"
|
|
}
|
|
},
|
|
"suite:p2p-flows": {
|
|
"command": "tools/build-remote.sh --priority gate -- test --release -p kaspa-p2p-flows (from the node fork)",
|
|
"box_class": "suite (gate priority)",
|
|
"fixtures": [
|
|
"F0",
|
|
"F1"
|
|
],
|
|
"cases": [
|
|
"OPS-06",
|
|
"ZKP-06"
|
|
],
|
|
"coverage": {
|
|
"OPS-06": "partial: malformed relay messages refused; traffic containment under load is an OPS harness",
|
|
"ZKP-06": "partial: proof relay coverage tests; aggregation completeness is the proving lane's test set"
|
|
}
|
|
},
|
|
"gate:pre-push": {
|
|
"command": "tools/ci/pre-push.sh (every landing; the heavy checks ship to a box)",
|
|
"box_class": "gate",
|
|
"fixtures": [
|
|
"F1"
|
|
],
|
|
"cases": [
|
|
"GOV-05",
|
|
"GOV-06",
|
|
"UX-08"
|
|
],
|
|
"coverage": {
|
|
"GOV-05": "partial: every gate check carries a self-test that fires on a known failure and passes a known success (77 checks); the suites' own oracles are their mutation rows",
|
|
"GOV-06": "partial: the launch-gates, forbidden-strings and scope checks on the served site",
|
|
"UX-08": "partial: the site gate on every served page, plus docs/build/tuning.md (the published tuning and build-settings half, the hash lane's); the open builds and the fee conditions are the shipper's and the site lane's"
|
|
}
|
|
},
|
|
"check:freeze": {
|
|
"command": "consensus/pow/build.rs rule 19 at every kaspa-pow build; packaging/pow-freeze.txt and packaging/d1-freeze.txt",
|
|
"box_class": "build",
|
|
"fixtures": [
|
|
"F0"
|
|
],
|
|
"cases": [
|
|
"GOV-01"
|
|
],
|
|
"coverage": {
|
|
"GOV-01": "partial: the linked igneum-pow tree's fingerprint must match a listed freeze or the build fails, the binary prints which; the manifest (igneum_getManifest) names the object digest; the signed F0 manifest is the node lane's row tonight"
|
|
}
|
|
},
|
|
"harness:fast-time-60x": {
|
|
"command": "the v5 fast-time harness on the devnet object at 60x (infra/fast-time/override-60x.json; the node lane's run shape)",
|
|
"box_class": "harness (lease pool, never build-1)",
|
|
"fixtures": [
|
|
"F0",
|
|
"F3"
|
|
],
|
|
"cases": [
|
|
"ROT-01",
|
|
"ROT-02"
|
|
],
|
|
"coverage": {
|
|
"ROT-01": "partial: hourly boundaries crossed at 60x with every node agreeing",
|
|
"ROT-02": "partial: the weekly and family boundary crossings at 60x"
|
|
}
|
|
},
|
|
"harness:finality-sim": {
|
|
"command": "igneum/harness-sim (the finality lane's scenarios: partition, split populations, long partition)",
|
|
"box_class": "harness",
|
|
"fixtures": [
|
|
"F0",
|
|
"F4"
|
|
],
|
|
"cases": [
|
|
"FIN-02",
|
|
"FIN-08"
|
|
],
|
|
"coverage": {
|
|
"FIN-02": "partial: split honest populations in the simulator; the real-node run is the fault network F4",
|
|
"FIN-08": "partial: the combined boundary and fault scenarios in the simulator; model checking is the formal review"
|
|
}
|
|
},
|
|
"harness:p01-vectors": {
|
|
"command": "tools/ci/p01-vectors.py --worker <the kit's worker> --pack <pack dir> --reference /srv/artefacts/packs/p01-vectors/<pack>.txt --count 1000000 --out <evidence.json>: the worker driven with pool.rs's job lines at an all-pass target, every found hash diffed against the CPU reference list (igneum-pow hash-bound --count 1000000, prehash 00..01, the genesis seeds), exit 1 on any disagreement or unanswered nonce",
|
|
"box_class": "pods (one backend each)",
|
|
"fixtures": [
|
|
"F0",
|
|
"F2"
|
|
],
|
|
"cases": [
|
|
"POW-01"
|
|
],
|
|
"coverage": {
|
|
"POW-01": "the million-vector campaign per backend per pack, bit-for-bit; PASS only when every supported backend reads a million vectors with zero disagreement; the malformed-input half is harness:parser-malformed"
|
|
}
|
|
},
|
|
"harness:parser-malformed": {
|
|
"command": "cargo test --release -p <crate> --test parser_malformed (building; igneum-exec, rpc-core, igneum-miner)",
|
|
"box_class": "suite",
|
|
"fixtures": [
|
|
"F5"
|
|
],
|
|
"cases": [
|
|
"POW-06"
|
|
],
|
|
"coverage": {
|
|
"POW-06": "partial: ten thousand seeded malformed cases per parser refused without panic in bounded time; the verifier work bound is the proving lane's measurement"
|
|
}
|
|
},
|
|
"bench:pc1-packs": {
|
|
"command": "tools/ca3-v4-amend/pc1-ca4-packs.ps1 on PC 1 (RTX 5090, the signed-jobs channel, --cards-off, the Power Helper lock at 1,300 MHz, nvidia-smi at 1 Hz, 250 x 2^24 per row, the kit's CUDA worker --bench)",
|
|
"box_class": "PC 1 bench",
|
|
"fixtures": [
|
|
"F0",
|
|
"F1"
|
|
],
|
|
"cases": [
|
|
"GPU-02",
|
|
"GPU-03",
|
|
"GPU-05",
|
|
"POW-08"
|
|
],
|
|
"coverage": {
|
|
"GPU-02": "partial: the paired stock and locked rows on the same board, host and workload (the rate held, 2.37 against 3.26 microjoules per hash on the class v5 pack); the historical 34 to 41 percent claim's full configuration set is owed",
|
|
"GPU-03": "partial: hl-v6-all against hl-v6-foldrw on the 5090, the window's cost per unit of work, registers and blocks per SM, no spill; the mandatory-SKU set is owed",
|
|
"GPU-05": "partial: v3-1g, v3-ds29, v3-ds30, ds55, ds4g-ms complete with their fingerprints, energy per step at stock and at the knee on the 5090 and the 7600; the 8.5 and 11.5 GiB candidates are owed",
|
|
"POW-08": "partial: rw-w4, rw-w32, rw-w64 and the 1p5x long-program packs as regression controls for the excluded levers"
|
|
}
|
|
},
|
|
"bench:pc1-amd": {
|
|
"command": "tools/ca3-v4-amend/pc1-amd-cardin-bench.ps1 on PC 1 (RX 7600, OpenCL, gfx1102, 30 dispatches of 2^24 per size, rate only)",
|
|
"box_class": "PC 1 bench",
|
|
"fixtures": [
|
|
"F0",
|
|
"F1"
|
|
],
|
|
"cases": [
|
|
"GPU-01",
|
|
"GPU-05"
|
|
],
|
|
"coverage": {
|
|
"GPU-01": "partial: the 8 GB AMD cell, fingerprints and rates at 1, 2, 4 and 5.5 GiB; one cell of P02's twelve",
|
|
"GPU-05": "partial: the 8 GB tier's fit rows"
|
|
}
|
|
},
|
|
"bench:fleet-pods": {
|
|
"command": "the fleet lane's one-shot RunPod cards (5090, 4090, H100, 3090) with the same worker --bench and nvidia-smi sampling, the lock through nvidia-smi -lgc",
|
|
"box_class": "rented pods",
|
|
"fixtures": [
|
|
"F0",
|
|
"F1"
|
|
],
|
|
"cases": [
|
|
"GPU-01",
|
|
"GPU-03",
|
|
"POW-08"
|
|
],
|
|
"coverage": {
|
|
"GPU-01": "partial: the NVIDIA cells of P02 that RunPod carries",
|
|
"GPU-03": "partial: the 4090 cell",
|
|
"POW-08": "partial: W = 16 on four cards"
|
|
}
|
|
},
|
|
"census:class-v6": {
|
|
"command": "the census lane's harness on class-v6-census-fold: the acceptance rule with (c''') on every candidate, the attempts census, the per-site index-bit read, attack-f8 at 2^22 on the live state",
|
|
"box_class": "harness (the lease pool)",
|
|
"fixtures": [
|
|
"F0",
|
|
"F3"
|
|
],
|
|
"cases": [
|
|
"POW-02"
|
|
],
|
|
"coverage": {
|
|
"POW-02": "partial: the fold's index statistics and the grammar bounds on 256 seeds plus the F8 set; the malformed programs are owed to harness:parser-malformed"
|
|
}
|
|
}
|
|
},
|
|
"not_run": {
|
|
"GOV-02": "the approval is recorded in the registry's approval field; the automated half (thresholds frozen before any run_status) is the gate rule landing by 21:00",
|
|
"GOV-04": "the evidence vault F9 (raw and negative evidence preserved) is the gate rule landing by 21:00: a PASS must carry its evidence file",
|
|
"GOV-08": "the stale-evidence rule (evidence older than the manifest sha reads NOT RUN) is the gate rule landing by 21:00",
|
|
"GPU-04": "the memory-clock ladder has no harness tonight: PC 1 mines nothing under the Devnet 3 off order",
|
|
"GPU-06": "accepted work under ordinary connectivity needs the fault network F4",
|
|
"GPU-07": "the sustained thermal and power soak has no harness tonight: PC 1 mines nothing under the Devnet 3 off order",
|
|
"POW-05": "amortised cheap winning attempts are the attack lanes' grind and era harnesses (tools/attack/f7-era, f9-grind), not in the release matrix; their rows come from those lanes",
|
|
"ADV-04": "selective participation needs the economic model F7 and a live chain window",
|
|
"ADV-06": "process-advantage separation is the adversary lanes' chip study",
|
|
"ADV-07": "lifetime rows are the adversary lanes' models",
|
|
"ROT-03": "miner-voted bring-forward needs a vote harness on the fault network F4",
|
|
"ROT-04": "seed-selection resistance is the census harness (the class v6 invention lane), not yet in the matrix",
|
|
"ROT-07": "layer ablation is a research harness, not a release suite",
|
|
"ROT-08": "the no-new-rules counterfactual is a research harness",
|
|
"ECO-01": "the economic model F7 is the economics lane's; no automated harness in the matrix",
|
|
"ECO-02": "F7",
|
|
"ECO-03": "F7",
|
|
"ECO-04": "F7",
|
|
"ECO-05": "F7",
|
|
"ECO-06": "F7",
|
|
"ECO-07": "F7",
|
|
"EVM-01": "no EVM conformance-vector harness is mapped tonight; the exec suite does not run the reference test vectors",
|
|
"EVM-03": "the two-dimensional fee tests need the proving-limit harness",
|
|
"EVM-04": "block context and randomness vectors are an EVM harness still to map",
|
|
"EVM-05": "contract journeys need the DEX lane's integration set on Devnet 4",
|
|
"EVM-06": "wallet, RPC and indexer validation is the explorer and wallet lanes' rows",
|
|
"EVM-07": "execution DoS workloads need the workload catalogue F3",
|
|
"EVM-08": "verifier upgrade tests need the enforced-proving lane's upgrade scenario",
|
|
"ZKP-02": "program and verifier binding vectors are the proving lane's test set, not yet in the matrix",
|
|
"ZKP-04": "reward substitution tests are the proving lane's",
|
|
"ZKP-05": "proof payment idempotence under races needs the capacity harness",
|
|
"ZKP-08": "acceptance-path audit needs the authority test set",
|
|
"CAP-01": "the consumer-shard reproduction is the fleet lane's pods",
|
|
"CAP-02": "proving on the mining configuration is the fleet lane's",
|
|
"CAP-03": "the request-to-payment path is the proving fleet's measurement",
|
|
"CAP-04": "sustained load is the proving fleet's",
|
|
"CAP-05": "overload and recovery is the proving fleet's",
|
|
"CAP-06": "assignment windows are the proving fleet's",
|
|
"CAP-07": "reassignment is the proving fleet's",
|
|
"CAP-08": "customer-verifiable output is the reference apps plus the fleet",
|
|
"INC-02": "revenue separation is the economics lane's model",
|
|
"INC-03": "the modified-client task choice needs an adversarial client harness",
|
|
"INC-04": "demand spikes are the economic model",
|
|
"INC-05": "reservation abuse needs the capacity harness",
|
|
"INC-06": "difficulty and timestamp manipulation needs the fault network F4",
|
|
"INC-07": "self-dealing fees need the economic model and a live window",
|
|
"INC-08": "failure concentration is a live-window measurement",
|
|
"FIN-06": "old-key compromise analysis is the finality lane's long-range scenario, not yet in the sim",
|
|
"VER-01": "light-client bootstrap is the reference apps lane (/light); no automated harness mapped tonight",
|
|
"VER-02": "the evolving-authority statements are the reference apps lane's",
|
|
"VER-03": "payment proof (/receipt) is the reference apps lane's",
|
|
"VER-05": "state reconstruction without founder storage is the OPS no-founder exercise",
|
|
"VER-06": "withholding and corruption detection needs the fault network F4",
|
|
"VER-07": "wallet key protection is the wallet lane's security row",
|
|
"OPS-01": "the no-founder exercise is an operations run, not a suite",
|
|
"OPS-02": "bootstrap diversity needs the fault network F4",
|
|
"OPS-03": "update distribution separation is the update-return lane's row",
|
|
"OPS-05": "proving workload isolation is the fleet's pod row",
|
|
"OPS-07": "runbook detection is an operations run",
|
|
"OPS-08": "repeat independent operation is a cross-release observation",
|
|
"UX-05": "voting keys through pooling is the pool lane's row",
|
|
"COM-01": "commercial evidence, no automated harness",
|
|
"COM-02": "commercial evidence",
|
|
"COM-03": "commercial evidence",
|
|
"COM-04": "commercial evidence",
|
|
"COM-05": "commercial evidence",
|
|
"COM-06": "commercial evidence",
|
|
"COM-07": "commercial evidence",
|
|
"LEAD-02": "leadership comparison, an observation window",
|
|
"LEAD-03": "observation window",
|
|
"LEAD-04": "observation window",
|
|
"LEAD-05": "observation window",
|
|
"LEAD-06": "observation window",
|
|
"LEAD-08": "observation window"
|
|
}
|
|
}
|