Commit graph

1640 commits

Author SHA1 Message Date
igneum-labs
ffb644fe2b reliability injector: the catch-up step's flag no longer shadows the process list
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit bfab69e9db)
2026-10-07 13:12:08 +00:00
igneum-labs
71fa3e5b82 execrpc: one gate for every execution-layer RPC call the app makes (ledger N7, main's rule for 0.3.19)
A node before the exec RPC bounds fix (every 0.3.17 node) dies when a method that resolves a block number or indexes the record vector is asked while its exec follower holds no record; PC 1 crash-looped on two callers in one night (eth_getBlockByNumber from the clock sample, then igneum_getAssignedShards from the prover loop: 'panicked at igneum/exec/src/rpc.rs:808:35: range start index 1 out of range for slice of length 0'). Every caller (prover.rs's evm_rpc, update.rs's clock sample, extnode's rpc for chainfacts and the external-node probe) now goes through execrpc::call: SAFE_ON_EMPTY methods go out, GATED ones wait for igneum_getExecStatus's executedTipHash, an unclassified method is refused. The test every_caller_goes_through_the_gate scans src/ for JSON-RPC requests built elsewhere and for unclassified exec method names.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 13:12:08 +00:00
igneum-labs
f3238c9b3e Miner app: plug, tune, play (the project lead, 7 October 2026): node readiness gate, the retry ladder, no permanent fault, fault lines to the intake, the signed cards job, the fault-class register
docs/plans/miner-faults.md: MF-1 to MF-7, each with its rule, test and gate line.
- MF-1/MF-2: a worker starts and is judged only when the node is READY (synced and igneum_getExecStatus reports an
  executed tip; execrpc::probe every 5 s off the engine thread); the node watchdog never counts the catch-up (settled
  once read synced; 30 min cap before that; any RPC answer is a sign of life); the watchdog restarts on a ladder 10 s,
  30 s, 2 min, 5 min, then every 5 min for ever (watchdog::RETRY_LADDER_S); the faulted state and the one-restart
  budget are gone (tools/ci/permanent-fault-check.sh in the gate); a node-caused restart resets the ladder at sync.
- MF-3: the hot-plug pass starts a recovered or revived card's worker (unchanged rule, now in the register).
- MF-4: the status clock starts at ready (program loaded), loading bounded by 300 s; a self-test failure holds the
  card 30 min with the reason on its row, released on a driver change; a crash loop climbs the ladder; the pack is
  exported once a minute for every card (a refused pack forces one).
- MF-5: the app reads template_wait=, template_ms=, identities_active= from the 0.3.20 miner's STATUS; waiting on
  the node is never the card's fault; the row says node slow; every node-wait label clears on the first rate.
- MF-6: a miners hold belongs to the job that took it and releases when that job is gone or at its own cap.
- MF-7: the engine owns every igneum-miner it started: an untracked one on this engine's node RPC is killed at start,
  after every stop and every minute, one line and one fault report per kill; a restart kills the old process first.
- Every fault line posts one FAULT line to the log intake (label fault-<id8>, app and node version, 60/h cap).
- The signed cards job kind (per card enabled, identities, power_pct; refused for a card the machine lacks; applied
  through the app's own card path, persisted, read back): packaging/ota/publish-jobs.sh add --kind cards.
- LG-4 as a job: relay/playbooks/first-share.ps1 and tools/fleet/first-share-gate.mjs (no Windows box yet).
- tools/reliability: the fault injector with one step per class (catch-up, card-appears, own-restart, zero-ladder,
  no-status, node-silent, one-card-fails, orphan-miner); fake-worker.mjs lists devices and fails self-tests on command.
- master's build tooling (d4bc5a94) and release-0.3.20's igneum-pow taken into the worktree for the box routes.
Box: app 198 + 27 + 8 tests green on igneum-build-2; the tree gate green (33 checks).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 13:11:48 +00:00
igneum-labs
a4e6ab7b8a Driver table: a referer per row, sent as curl -e (drivers.amd.com answers 403 without an amd.com one; the AMD row carries it); a plain user agent on the download
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit f43c9d4d7c)
2026-10-07 13:09:47 +00:00
igneum-labs
f5d682e98e Driver table: the NVIDIA 617.42 and AMD 26.9.2 rows measured on igneum-build-2 (sha256, size, Authenticode subject from the vendors' own files); the stray #[test] above the Intel test's doc comment removed
NVIDIA GeForce Game Ready 617.42 WHQL (6 October 2026): us.download.nvidia.com/Windows/617.42/..., 990,853,168
bytes, sha256 f115c927..., subject CN=NVIDIA Corporation (DigiCert G4). AMD Software Adrenalin 26.9.2 WHQL
(29 September 2026, the win11-b build): drivers.amd.com/drivers/whql-amd-software-adrenalin-edition-26.9.2-win11-b.exe
(an amd.com Referer required), 1,000,800,840 bytes, sha256 593c1d73..., subject CN=Advanced Micro Devices (Sectigo).
Read on the box: curl, sha256sum, the PKCS7 out of the PE security directory through openssl pkcs7 -print_certs.
The table's placeholders are gone: every row installs. The drivertable test sample, the mock and the view test name
the real NVIDIA release. detect.rs:990 carried a #[test] above the doc comment of the Intel test from the cherry-pick,
the test build's one warning ("duplicated attribute"): removed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 4f8b37b6be)
2026-10-07 13:09:47 +00:00
igneum-labs
108624cb19 Driver check: the app detects a missing or old driver per card and installs it on one click from the vendor's own server (branch driver-check, 7 October 2026)
the project lead: can the drivers be packaged with the miner, for all cards, with the system knowing which to install if not
present. Not bundled: detected and installed on one click. A per-vendor table rides the signed manifest (drivers.json:
min_version, the version on offer, the vendor's URL, size, sha256 and its source page, the silent arguments, the
restart exit codes, the Authenticode signer, the Linux and HiveOS package), validated by the signer and the app alike
(src/drivertable.rs, shared), written to <app data>/drivers.json by ota.rs. Each card's driver version is read at
every detection (nvidia-smi's driver_version; Windows' DriverVersion for AMD and Intel) and compared; a missing or
old driver puts the offer on the card's row, on the dashboard and on the first-run list. The click downloads with
curl (resume), checks size, sha256 and the Authenticode subject, runs the installer through one elevated prompt
(platform::elevated_command, the PC 1 driver job's shape), reports restart required with a Restart now button, and
never restarts by itself; the miners keep mining. macOS: no step; Linux and HiveOS: the package line. Dry run through
IGNEUM_DRIVER_DRY_RUN or the table. Tests: the table, the versions, the offers per tier, the exit codes, the
Authenticode verdicts, the download against a mocked vendor server on 127.0.0.1, the UI's strip per state; the mock's
drivers scenarios; captures light and dark in docs/plans/driver-check-shots. publish-manifest.sh --drivers carries the
table. Also the doubled #[test] in detect.rs from the cherry-pick.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit a0b3982998)
2026-10-07 13:09:46 +00:00
igneum-labs
3d42a22e71 release-0.3.20 plan: section 35, the project lead's orders (the floor file publishes, the sweep in waves, the Arc-tested worker exe, 0.3.21 tonight)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 13:09:09 +00:00
igneum-labs
6d24730df5 release rules: 4a every gate starts on every candidate as it builds, 4b warm pods per gate class, 5 the sweep in waves (the project lead, 7 October 2026)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 13:07:49 +00:00
igneum-labs
5a0b84b932 release-0.3.20 plan: section 34, every artefact on c4459193 with shas, the workers with the Arc fix, the hive package and its smoke, the Windows inputs and run, the staging plan
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 13:00:53 +00:00
igneum-labs
a4b2293b1a release-0.3.20 plan: the cases rerun's slip to about 16:10 BST, the choice put to main
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:53:10 +00:00
igneum-labs
7c591e3619 release-0.3.20 plan: the c4459193 seed pair; the pair set complete
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:48:51 +00:00
igneum-labs
c1b60cc9d3 release-0.3.20 plan: c20-1's kept start on the pin, the Mac build started
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:47:20 +00:00
igneum-labs
34ee819a70 release-0.3.20 plan: c4459193's two node gates pass; the Mac binaries start
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:46:47 +00:00
igneum-labs
04eb4b4eb7 release-0.3.20 plan: the c4459193 hands pair
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:41:51 +00:00
igneum-labs
d87a511b2b release-0.3.20 plan: the cases rerun started on c20-1
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:41:44 +00:00
igneum-labs
d329bd7743 release-0.3.20 plan: main accepts the three lines; the cut set stands
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:41:24 +00:00
igneum-labs
a213628fcd release-0.3.20 plan: the cases rerun on c4459193 (the class byte touch), the 610 read (the driver proves; the server per architecture is the rule)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:40:45 +00:00
igneum-labs
89ebce25d2 release-0.3.20 plan: the wipe canary's start stamp on c19-1
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:40:00 +00:00
igneum-labs
3aa518138a release-0.3.20 plan: CASES END pass on dc141409, the carry question to c4459193
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:39:47 +00:00
igneum-labs
1d3cab98ed release-0.3.20 plan: the wipe canary pod c19-1 on c4459193, the 10 GB tier closed
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:38:30 +00:00
igneum-labs
b39864c2f6 release-0.3.20 plan: main accepts the wipe canary's clock
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:36:56 +00:00
igneum-labs
4d7068e0e1 release-0.3.20 plan: the wipe canary moves to a second pod so its line lands inside the checkpoint
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:36:32 +00:00
igneum-labs
af3e72df00 release-0.3.20 plan: the 3080 hour's number (a 10 GB card proves alone, not beside its miner), the build-server sequence on c4459193
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:34:34 +00:00
igneum-labs
75b1646931 release-0.3.20 plan: section 33, b7cc37e7 struck (the 10 s stop), the pin c4459193, the carry ruling, main's F8 and slot rulings, the 12 GB rule in and green, the roll complete, the server-architecture fact, the 0.3.21 driver-check tip
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:33:26 +00:00
igneum-labs
196ee0101d Proving refuses a card under 12 GB and says why (main's rule, 7 October 2026): MIN_VRAM_MB_PROVE_ANY 11,800 and the sentence "proving needs a 12 GB card; mining continues" in provedefault.rs with its test, the prover loop refusing before the sync wait when every present NVIDIA card is under it, the tile sentence in the UI with its view test; the fleet's p1-3080 died at the compressed step 29 of 29 with 0 paid; the measured SP1 threshold moves the line
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:13:33 +00:00
igneum-labs
677a08a7cf release-0.3.20 plan: the driver check to 0.3.21, the clock rule, the 10 GB prover tier and the rule put to main
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:11:42 +00:00
igneum-labs
c78c407c5e release-0.3.20 plan: 6b94c823's two gate lines (clean on the checks that bear on the binary), the harness sequencing fixes
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:10:20 +00:00
igneum-labs
e3b2f8a667 release rules file (the standing rules as main set them, the kept-datadir gate as rule 4); the 0.3.20 plan's row for main's three additions
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:07:42 +00:00
igneum-labs
6bd5741c5e release-0.3.20 plan: section 32, N12 and N13 on the line, the pin b7cc37e7 with no fallback commit, the kept-datadir gate, the stale packs
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 12:06:22 +00:00
igneum-labs
efbad7b158 proto-cuda/packs-ca3-v4 at the hash lane's 8c728ca3: the amended class v4 packs re-exported under the sub-version-1 stamp (program id 1a4230699a6b9c60 on every v4 pack; the pre-amendment c120d7963abdcd96 is the must-differ vector); igneum-pow's recheck tests read these
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:55:16 +00:00
igneum-labs
3af54b17e3 release-0.3.20 plan: the build-server lane's four pairs in flight, the node line's missing rust-toolchain.toml owed for 0.3.21
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:54:38 +00:00
igneum-labs
dc63c587b7 release-0.3.20 plan: clock correction (the UK stamps since 12:40 BST ran ahead; true times from the commit times)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:54:22 +00:00
igneum-labs
657ca5ad14 release-0.3.20 plan: the sixteen-field interop fact (0.3.17 relays byte-5 headers), the void run's failed checks explained
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:54:01 +00:00
igneum-labs
21c2d0ade6 release-0.3.20 plan: the amended v4 packs' ids and fingerprints (the hash lane's table), G1, the pairing line in flight
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:53:51 +00:00
igneum-labs
908c06f1de release-0.3.20 plan: section 31, the candidate pin, igneum-pow pairing, the speculative builds
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:52:21 +00:00
igneum-labs
7221da2328 igneum-pow at the hash lane's 8c728ca3 (the amended class v4 for AP-F8-1: the source rule keyed on the class with the shadow's pass count, the v4 unit test following the amendment; the pair for release-0.3.20-node 6a3432a3 and its fallback 6b94c823)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:51:28 +00:00
igneum-labs
b0afb70463 release-0.3.20 plan: the fallback pin 6b94c823 on the mirror
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:47:57 +00:00
igneum-labs
0f38c0e8c4 release-0.3.20 plan: the stale finality test on 8097d600, the fallback pin is a test-only commit on top
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:47:10 +00:00
igneum-labs
976820b9a2 release-0.3.20 plan: epoch 231 on the old file confirmed with the reference, the staging recipe for the floor-moved file
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:43:30 +00:00
igneum-labs
a38f7bf2ed release-0.3.20 plan: main's rulings (watchdog test meets the condition, the floor-moved file staged not published, the 13 October sweep date in the lock lines, the epoch-231 question for the report)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:41:45 +00:00
igneum-labs
c75a9fc8c0 release-0.3.20 plan: the node lane's clock estimate, how the two conditions are met, the kit's wait rule, the 13 October floor deadline
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:41:15 +00:00
igneum-labs
c0af072cbf release-0.3.20 plan: main's two conditions on the second node commit, the publish order
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:40:07 +00:00
igneum-labs
8c2032265d release-0.3.20 plan: the node line at 8097d600 (what it carries), the second commit, the 15:30 UK cut rule
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:39:21 +00:00
igneum-labs
3b5525773f release-0.3.20 plan: the fleet's prover-roll read (12 GB provers paired and unpaid, the claim race, the fix on the node line)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:36:26 +00:00
igneum-labs
dfa404b304 release-0.3.20 plan: cards_leave_off in (918cd15a), the third app gate, the reliability lane's hashes named
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:32:39 +00:00
igneum-labs
918cd15a1d Remote jobs: cards_leave_off, a run job's --cards-off cards restored as OFF on any exit (persisted by the app's own card path)
The way to hold a card out of mining past a job without a script on api/cards (the 6 October rule): the runner's hold
is built with enabled=false (identities and cap kept), the report line says LEFT OFF, publish-jobs.sh carries
--cards-leave-off. For the Arc B580 on PC 1 while its worker fix rides to the shipped app. Test:
cards_leave_off_restores_the_card_as_off_with_its_settings_kept (igneum-app 157 of 157 on the box).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 9e794503d2e9689ae3a0996e703cb97ea6d95cef)
2026-10-07 11:31:55 +00:00
igneum-labs
6054cf7a5c release-0.3.20 plan: main's hold for the isSynced fix (16:00 UK checkpoint)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:29:39 +00:00
igneum-labs
85729b4f2c release-0.3.20 plan: the app gate green twice on the box, the push
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:28:48 +00:00
igneum-labs
509b838631 release-0.3.20 plan: the Arc rotr fix and the Intel app files are in (bench d numbers, the two hashes, the box tests)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:25:31 +00:00
igneum-labs
99a1268b83 Intel Arc in the app (0.3.20): vendor intel from the OpenCL vendor string and the Windows name, the INTEL badge and its token, the no-cap row naming IGCL, the measure-only tune words, the view test and the ui-mock scenario (the six app files of intel-arc 18453bb1, applied as its own diff; the test's prove sentence in this tree's wording)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:25:05 +00:00