Source: the scratch engine's own log in collect ember-c35-collect-1 (06:59Z): 22:31:02Z '0.3.10 is available: downloading',
22:31:05Z 'update: starting the installer first ... ota-apply.ps1', and the installed app's 'quit:' at 22:31:06Z.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
PC 1, 22:31 UTC: the installed engine's quit hung 24 minutes in the jobs runner's abort, waiting for EOF on the script's
stdout pipe whose write end the second engine and its miners had inherited (Process.Start with redirection inherits
every inheritable handle), while the orphaned miners mined on against the relaunched app.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
LoadClass::hot (Option<HotClass { mb, k, added }>) beside mix, slots, scratch, mixer_mult, growth and era; V3_CLASS = { era: None, hot: None, ..LoadClass::MX4 } (hot stays None: the hot code is behind the flag, measured and not adopted). Op::Hot, the hot slots drawn after the scratch slots, no width roll (v2_loads allows the added form's extra slots), the id suffix hot/<S><k>[added], the hot parsing inside parse_loads, the era branch first in name(). HotTable under seed_words("igneum-hot/" || epoch seed) with the cache chain and tag umHT, read at H[mulhi(src, HOT_WORDS)]; DatasetSource::hot attached by new_class_day and from_seed_bytes_class; the acceptance stand-in dataset_elem(idx, S[2], S[3]); the three emitters (hot argument after the init words, ht_segment and igneum_hot_fill beside the layout-aware cores); packfile.h hot fields beside class, era, attempt and mixerMult; OpenCL host, Metal packbench and NVRTC worker fill H on the device and self-test it. Eight packs under proto-cuda/packs-ca2-hot (replaced hot32k4 hot64k4 hot96k4 hot64k2 hot64k8, added hot32k4a hot64k4a hot96k4a) re-exported on the merged crate: vectors unchanged, program.h and program.json carry the mixer fields. docs/plans/hot-table.md (design, spec text, per-tier budget, chip model, Mac and PC measurements, the decision: layer 5 out of v3, the 3.0 note); bench-log entry and addenda with job ids and worker sha256s; the two PC playbooks.
Checks on this commit: cargo test --release 53 + 19 pass (the pinned v2, mx4, era, readwidth and hot packs); the pinned packs under proto-cuda/packs, packs-ca2-mixer, packs-ca2-era and packs-readwidth untouched; Metal packbench and Apple OpenCL --bench-pack on all eight hot packs (run lock, 2^20 at base 0): 96/96 lanes, hot table head, last line and FNV PASS, one fingerprint per pack on both harnesses, equal to the fingerprints before the rebase (hot32k4 679e5e83378d3790, hot64k4 d4c9e456b039fdef, hot96k4 7c98eceffee9fd73, hot64k2 f43b10a95879b8e5, hot64k8 c11309d743be9392, hot32k4a afb700b2d997c847, hot64k4a ba214baa9c1a9e85, hot96k4a 29e1916aed6deff5). No era or mixer behaviour changed: every resolution kept the ca2-v3 side and appended the hot branch.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
the project lead, 5 October 2026, 22:45 BST: "make sure we have ember tuning every single card for efficiency out of the box, the
more data = the better the tune, make an awesome system." Built on lever 3 (docs/plans/miner-eff.md), lever 2's signed
tuning section (docs/design/miner-tuning.md), the AMD telemetry helper (423936b, its --tune/--set-gmax/--set-plimit/
--reset contract) and the Power control switch (057f0ec). Design, data flow, tiers and the privacy line:
docs/plans/ember-tune.md.
- src/ember.rs (new): two knobs per card (power limit %, core clock cap MHz; memory clock never touched), the full plan
(power ladder 100..50%, then the clock ladder 90..60% at the chosen power), the confirm plan (the fleet prior and one
neighbour), the baseline plan (measure only), the marks (faulted, hot, memory_clock_dropped, unapplied, no_readings),
the choice (best MH/W within 1% of the top rate, then rate, then draw), the fleet record (a hash of the install id,
no address), the prior lookup and the kill switch (tuning.ember), the state machine on a fake clock. 9 unit tests.
- engine.rs: tick_sweep schedules every NVIDIA, AMD and Apple card (120 s steady, 600 s to the boundary, no job hold,
no pause, weekly, again after a driver major or program-class change, never under the manifest kill switch); the
probe (nvidia-smi clocks.max.gr + driver_version and the direct/helper mode; igneum-gpu-telemetry --tune for AMD);
tune_apply (nvidia-smi -pl / -lgc 0,<MHz> / -rgc directly or through the helper; the AMD helper per request);
Cmd::TuneProbe, Cmd::TuneSet; faults from rejected and mismatched hashes mark the step; the TUNE lines and the TUNE
{json} record, uploaded with the log; the Tuned line on the card state. The NVIDIA helper starts only with Power
control on: the --sweep job never counts as permission (no prompt on a PC with nobody there).
- sweep.rs: the helper protocol gains lgc/rgc (clock cap and reset) and resets the clocks after 20 idle minutes.
- state.rs, config.rs: the tune fields (clock cap, driver, class, source, the Tuned line); the nvidia-smi telemetry
query carries clocks.gr and clocks.mem; the AMD sample line's plimit_pct and gmax_mhz are parsed.
- ui: "Tuned: X MH/s at Y W (Z MH/W)" with the point, the source and when; measure-only cards say why; the Ember Tune
switch; tune-line.test.mjs.
- relay/lib/ember.mjs + relay/test/ember.test.mjs: the aggregation per (card model | driver major | program class):
median point, MH/W, spread, samples, machines; five samples converge, an outlier does not move the median, baselines
make no prior, de-duplication, the manifest merge keeps lever 2's cards. api/console.mjs fn=tuning and
tools/console.mjs tuning; tools/tuning.mjs --priors [--write tuning.json] [--site] [--tuning-off].
- site: the fleet priors table on /miners (site/miner-priors.json), the lever text.
- relay/playbooks/ember-tune-pc1.ps1: the PC 1 run (second engine with --sweep from a scratch copy of the install).
Measured tonight: see the bench log entry that follows the PC 1 run. The 9070 XT left PC 1's bus at 20:40 UTC and the
5090 needs the administrator prompt the project lead cannot answer asleep, so tonight's PC 1 run is the baseline plan on the 5090
through the whole pipeline; the two-knob tune on both cards is owed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Round 1 (run-readwidth-{5090,9070}-20261005) delivered the probes and refused every pack: pf_load demanded the chain's 32-byte epoch seed and the experiment packs carry igneum-pow --seed strings.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
From 18:23Z both Windows workers (CUDA on PC 1 and PC 2, OpenCL on PC 1 after the 18:34Z node restart)
refused every pack for epoch 34 with "the epoch seed bytes do not give the pack's IGNEUM_SEEDW_INIT", and
the miner and the app restarted them every 5 to 60 s until 18:44Z and beyond. The packs were correct.
The generator retries a rejected candidate with seed || k_le32 (attempt_words); epoch 34's attempt 0 was
rejected (246 of 16384 final register values saturated, limit 163) and attempt 1 accepted, so the pack
carried attempt 1's words while pf_load (proto-cuda/nvrtc/packfile.h) derived the expected words from
the bare seed. Both workers also matched jobs to pairs by those bare-seed words, so even a loaded pack
of a retried program would have answered "epoch seed mismatch" on every job.
The rule, in one place per language:
- packfile.h: pf_program_words(bytes, attempt); pf_load reads IGNEUM_PROGRAM_ATTEMPT and checks the
attempt's words; the refusal says "program pack and its seeds disagree: IGNEUM_SEEDW_INIT is not
attempt N of the epoch seed ..." in plain words.
- worker.cpp and proto-opencl/host.c: a job belongs to a pair when the seed hex the node sent is the
pair's (pairIs); the compiled-in placeholder pack keeps the word comparison.
- igneum-pow/src/packcheck.rs: verify_pack_texts / verify_pack_dir, the same rule in Rust; the miner
checks every pack it writes with it before a worker sees it (vendor/igneum-node pack-loop branch).
Tests pin the attempt vectors of epoch 34 on both sides (one vector, two implementations), that
epoch 34 is attempt 1 and epoch 33 attempt 0, a known-good pack of a later attempt, a known-mismatched
(out of date) pack, and self-contradicting packs.
- proto-cuda/nvrtc/emu/packfile-test.c (+ .sh, in CI): pf_load on a known-good attempt-1 pack, the
checked-in attempt-0 pack, and the known-mismatched bare-words pack.
The app (app/igneum-app):
- watchdog.rs: PACK_OUT_OF_DATE_CODE 44, PackRebuilds (at most 3 pack exports per epoch, then the card
shows the reason), pack_refusal (the worker's "error 0 pack" line and the miner's "PACK OUT OF DATE"
line), pack_epoch_of; tests on the incident lines, known-good and known-mismatched.
- engine.rs: exit 44 exports the pack again before the restart instead of a blind restart, the strip
says "program pack out of date, rebuilding", the card and the log name the condition; at the cap the
card is marked failed with the reason and tries again in 10 minutes.
The relay (relay/lib/parse.mjs): PACK_MISMATCH; the card reads "pack mismatch, rebuilding (N refusals
in the tail, M restarts)" in `node tools/console.mjs machines` instead of a bare restart count; tests
on the PC 2 tail of 18:27Z and a healthy tail.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
5 October 2026: an RX 9070 XT went into PC 1 through a Sonnet USB4 box while the app ran and nothing noticed; the
app detected cards once at start. Now src/hotplug.rs compares every enumeration with the list (key, else vendor +
name when unique; a card whose tool did not answer is never called removed): a new usable card starts a worker,
enabled by default like a card at start, with "New card: <name>, mining" on the strip and in the log; a card
Windows lists with a problem code (Win32_VideoController Status / ConfigManagerErrorCode) is shown as "<name>: not
usable (Code 43)" with the reboot-or-reinstall hint and no worker; a card that disappears has its worker stopped
(quit, 8 s) and its row says removed for five minutes, then hides; an unchanged list touches nothing. The Windows
host sends "detect" on WM_DEVICECHANGE; the engine polls every 60 s (300 s on macOS, no GPU hot-plug there).
detect.rs: the Ryzen iGPU is "gfx1036" to the OpenCL worker, so the APU gfx codes count as integrated, plus the
adapter row's Intel processor string and a dedicated memory under 1 GB; integrated defaults to off with "integrated
GPU, off by default (2 to 3 MH/s for 30 W)" on the row, and the user's choice is kept across re-detections and
restarts (settings, found by key or by vendor + name when the index moved).
Console: the engine logs "cards: <name> [<kind>, <state>] | ..." at start, on every change and every 10 minutes;
relay/lib/parse.mjs reads it and the hot-plug events, the machines API and tools/console.mjs machines show them.
Tests: hotplug.rs (added, removed, moved, errored, recovered, revived, unchanged, twins, user override kept,
the console line), detect.rs (PC 1's adapter lines, the Mac, kind classification, the unusable row),
notices.test.mjs (card notices), relay parse.test.mjs (cards line). cargo test -p igneum-app: 91 passed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
GET /wake?since=<stamp> is public (the apps hold no token) and rate limited (30 a minute per IP). It holds up to
45 s, re-reading the stamp every 2 s, and answers {stamp, at, added, changed, held_ms} the moment the stored stamp
differs from since, else the unchanged stamp at the deadline. POST /r/<token>/wake {stamp, added} (the relay's
auth, also x-relay-token or x-igneum-key on /wake) records a stamp; one row per stamp in relay_wake, created by the
first POST. maxDuration 60 s for api/wake.mjs in vercel.json. api/relay.mjs is untouched.
The handler lives in lib/wake.mjs with its dependencies injected; relay/test/wake.test.mjs drives it with a fake
database, a fake clock and a fake sleep (the hold, the change, the deadline, the rate limit, the hold cap, auth, a
database error). CI's site job runs it with the other relay tests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Conflicts resolved: state.rs keeps both the sweep fields (miner-eff) and the race fields (miner-perf); bench-log.md
keeps both appended entries; publish-manifest.sh keeps master's --override implementation (8082576, the "every
height switch" rule, --verify-only, --tries, the retrying live check) and adds miner-perf's --tuning / --no-tuning
with the carry-over of consensus.override and tuning from the current manifest. One --override case, one parser.
src/sweep.rs (new): the cap steps 100% to 50% in 10% steps clamped to the card's limits, per-step rows (mean draw
from nvidia-smi power.draw, mean worker interval rate), the choice (best MH/W, ties to the higher rate then the lower
cap), the nvidia-smi power parser, a state machine on an explicit clock (15 s settle, 60 s hold, 30 s cap readback
limit), the elevated helper scripts (one administrator prompt per sweep: a command file polled by one elevated
process, self-restoring after 20 idle minutes), the unsupported reasons (Apple silicon, AMD). 9 unit tests with
PC 1's recorded RTX 5090 numbers (575 W default, 460 W cap, 290 W draw, memory temperature [N/A]).
Engine: scheduler (once after install, then weekly; one card at a time; only while the card mines, after 120 s
steady, never under a remote job hold, a pause, or inside 600 s of the hour boundary), the cap-mode probe (direct
when the engine runs elevated, else the helper), abort on any fault (card leaves mining, worker error, GPU 90 C,
job, pause, quit) with the cap restored, the chosen cap held and recorded, SWEEP table lines in the app log,
--sweep mode (sweep every supported card, print the table on stdout, leave the caps, quit). Cap floor 50% (was 60).
A readback that matches the asked cap now counts as applied (PC 1 showed "cap NOT applied" for hours at 460 W).
Dashboard: live eff MH/W on each tile, the sweep line (phase, last result, or why unsupported), Sweep now / Stop /
Unpin, "pinned" and "chosen by the sweep" on the cap line, the Settings toggle, the cards-page note, slider min 50.
A cap moved by hand pins the card: the sweep records but does not change it.
PC 1 measurement: relay/playbooks/sweep-5090.ps1 (a run job, elevated, miners stopped; a second engine with --sweep
in a scratch data folder, RESULT SWEEP lines) and docs/plans/miner-eff.md with the publish command. Not published.
Untested on a card.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The engine turns a worker's race line into one TUNING {json} line in the app log (card model as the worker names it,
driver, arch, program class loads and wide loads, every variant's MH/s, winner, gain, the card's power cap and
draw, MH per watt), which the existing intake receives; the card state carries the variant for the dashboard and
one event per race. tools/tuning.mjs aggregates the records from miner_logs per card model (median MH/s or MH per
watt, at least 3 samples, de-duplicated per race) and writes tuning.json; publish-manifest.sh --tuning puts it in
the signed manifest (and now takes --override for consensus.override; both are carried over from the current
manifest when not given, --no-tuning drops it); manifest.rs parses it; ota.rs writes <app data>/tuning.json and
removes it when the manifest drops it; procs::spawn takes an environment and every miner starts with
IGNEUM_TUNING_FILE, which its worker reads at every prepare. Dry run of the publisher against a scratch folder:
tuning and override written, carried over, dropped, signature verified.
docs/plans/miner-perf.md: the signed jobs for PC 1 (fetch the race build of the NVRTC worker, then
relay/playbooks/race-5090.ps1 with the miners stopped: 17 variants, 3 rounds, twice) with the exact publish
commands for the main session; not published by the agent.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The intake key sits in every miner package, so the relay now lets it report only (drop text and files, ack, done,
register, upload). Posting a run or task, or renaming and re-roling a machine, needs the console token.
The prove host wrote proofs through SP1's unbuffered save: on WSL2 under /mnt/c the 18 MB core proof of a shard
took longer to save than to prove. Proofs now go through a 4 MB buffer with a timed 'saved' line, and
prove-shard.sh keeps results on the Linux side and copies them per stage. Ledger P20 updated.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The console marked any job with a RESULT line as done, so a running shard job read as finished. Done now means
the SUMMARY line carries finished_at or the job's closing 'job <id>: <status> (exit N)' line is present.
prove-shard.sh dropped the third fixture argument (block-344-shards4) because it read only $2.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
