Source: the scratch engine's own log in collect ember-c35-collect-1 (06:59Z): 22:31:02Z '0.3.10 is available: downloading',
22:31:05Z 'update: starting the installer first ... ota-apply.ps1', and the installed app's 'quit:' at 22:31:06Z.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
PC 1, 22:31 UTC: the installed engine's quit hung 24 minutes in the jobs runner's abort, waiting for EOF on the script's
stdout pipe whose write end the second engine and its miners had inherited (Process.Start with redirection inherits
every inheritable handle), while the orphaned miners mined on against the relaunched app.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
the project lead, 5 October 2026, 22:45 BST: "make sure we have ember tuning every single card for efficiency out of the box, the
more data = the better the tune, make an awesome system." Built on lever 3 (docs/plans/miner-eff.md), lever 2's signed
tuning section (docs/design/miner-tuning.md), the AMD telemetry helper (423936b, its --tune/--set-gmax/--set-plimit/
--reset contract) and the Power control switch (057f0ec). Design, data flow, tiers and the privacy line:
docs/plans/ember-tune.md.
- src/ember.rs (new): two knobs per card (power limit %, core clock cap MHz; memory clock never touched), the full plan
(power ladder 100..50%, then the clock ladder 90..60% at the chosen power), the confirm plan (the fleet prior and one
neighbour), the baseline plan (measure only), the marks (faulted, hot, memory_clock_dropped, unapplied, no_readings),
the choice (best MH/W within 1% of the top rate, then rate, then draw), the fleet record (a hash of the install id,
no address), the prior lookup and the kill switch (tuning.ember), the state machine on a fake clock. 9 unit tests.
- engine.rs: tick_sweep schedules every NVIDIA, AMD and Apple card (120 s steady, 600 s to the boundary, no job hold,
no pause, weekly, again after a driver major or program-class change, never under the manifest kill switch); the
probe (nvidia-smi clocks.max.gr + driver_version and the direct/helper mode; igneum-gpu-telemetry --tune for AMD);
tune_apply (nvidia-smi -pl / -lgc 0,<MHz> / -rgc directly or through the helper; the AMD helper per request);
Cmd::TuneProbe, Cmd::TuneSet; faults from rejected and mismatched hashes mark the step; the TUNE lines and the TUNE
{json} record, uploaded with the log; the Tuned line on the card state. The NVIDIA helper starts only with Power
control on: the --sweep job never counts as permission (no prompt on a PC with nobody there).
- sweep.rs: the helper protocol gains lgc/rgc (clock cap and reset) and resets the clocks after 20 idle minutes.
- state.rs, config.rs: the tune fields (clock cap, driver, class, source, the Tuned line); the nvidia-smi telemetry
query carries clocks.gr and clocks.mem; the AMD sample line's plimit_pct and gmax_mhz are parsed.
- ui: "Tuned: X MH/s at Y W (Z MH/W)" with the point, the source and when; measure-only cards say why; the Ember Tune
switch; tune-line.test.mjs.
- relay/lib/ember.mjs + relay/test/ember.test.mjs: the aggregation per (card model | driver major | program class):
median point, MH/W, spread, samples, machines; five samples converge, an outlier does not move the median, baselines
make no prior, de-duplication, the manifest merge keeps lever 2's cards. api/console.mjs fn=tuning and
tools/console.mjs tuning; tools/tuning.mjs --priors [--write tuning.json] [--site] [--tuning-off].
- site: the fleet priors table on /miners (site/miner-priors.json), the lever text.
- relay/playbooks/ember-tune-pc1.ps1: the PC 1 run (second engine with --sweep from a scratch copy of the install).
Measured tonight: see the bench log entry that follows the PC 1 run. The 9070 XT left PC 1's bus at 20:40 UTC and the
5090 needs the administrator prompt the project lead cannot answer asleep, so tonight's PC 1 run is the baseline plan on the 5090
through the whole pipeline; the two-knob tune on both cards is owed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
igneum-prove-core carries the node's fees.rs (PgasTable, FeeParams PROTOTYPE and CALIBRATED_V1,
FeeSchedule::at); the shard input and every fixture carry the schedule and the block's DAA score; the executor
reads the set at that score, raises the base fees to its floors and meters with its intrinsic, B_p and modexp
entry, as the node's execute_segment does. The 328-byte statement is unchanged: the node's native veto pins the
schedule (a new layout would be a consensus change for every node). Exporter: schedule and daaScore from the
dump (gen.mjs writes them), per-segment switch on replay, S_p from the set. Fixtures from one simnet chain across
the switch at DAA 800: fees-switch-prototype (block 51), fees-v1-shards2 (351), fees-v1-shards3 (355); 358
segments replayed, every state root the node's. Host tests on both sides. Guest re-pinned: shard
0x2b1a81cb..., aggregator 0x474678f3...; pinned-guests-check passes.
Node fork 2b6d23ef unchanged (igneum-exec tests 11 passed). Digest for the override with
fees_v1_activation_daa 210000: ab8847da538dead1dc10e046dfaadab3c1c35928e3748810c4e050d4a886087a.
Runbook docs/plans/fee-switch-devnet.md; infra/devnet/restart-hand-nodes.sh and restart-seed.sh take the
override object (the hand nodes and the seed run 20139145 today and must move to 2b6d23ef first). One line on
the live page, spec 5.11, the testnet README, the floor analysis, the bench log and the journey.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The 13:19:41Z refusal on PC 2: fetch_jobs took igneum-jobs.json and .sig in two requests while the edge was
still serving the previous deployment for one of them. The signer wraps the verified pair into one object and
reads it back; the app fetches that object (the pair only when none is published); publish-jobs.sh writes and
mirrors all three files and verifies every folder after the deploy; tools/jobs.mjs reads the envelope.
Tests: jobs.rs signed_envelope_binds_file_and_signature, packaging/ota/test-publish-jobs.sh (24 checks).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
logs.mjs --rotation reads the old fingerprints from the dated .old-* files once the .next files are gone; fresh-repo.sh scrubs the dated secret files too; ship-app.mjs mirrors payload-inputs.json.sig.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
On 5 October 2026 the Mac's host (shard program id 0x0559759b...) rejected every
proof from PC 2's host (0x05db1aca...). Both were built from the same guest
sources: host/build.rs compiled the guests on each machine and the ELF depends
on where it is built (cargo's -C metadata for a path crate includes the checkout
path; a worktree on the same Mac gave a third id, 0x0dfade07...). The node's
verifier also spent 114 s to 138 s per proof in the prover client and both key
setups before a 0.1 s to 0.4 s verify.
- elf/: both guest ELFs, their verifying keys and manifest.json (sha256, ids);
host/src/pinned.rs embeds and checks them at every start; the prove modes
refuse when SP1's setup does not derive the manifest's id
- --mode verify: LightProver with the pinned key, no prover client, no key
setup; prints the proof's own program id next to ours ("IS NOT OURS")
- --mode id; igneum-prove-pin and pin-guests.sh to re-pin; build.rs builds a
guest only under IGNEUM_BUILD_GUESTS=1
- tools/ci/pinned-guests-check.sh: elf/ must match its manifest, no script
builds a guest outside pin-guests.sh; make-package.sh and build-dmg.sh print
the pinned ids
- unit tests on the pinned set; bench-log entry with the three ids, the cause
and the timing: 127.0 s wall per verify before, 1.8 s to 2.4 s after
- rollout order in proving/README.md: every prover and verifier moves together
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
publish-jobs.sh --deploy POSTs the new stamp (published_at plus 8 hex of the file's sha256) and the added id to the
relay's /wake once the live file verifies. The relay token goes in a 600-mode header file, never on the command line
or the screen. Prints "woke the apps (stamp ...)" or a one-line warning; the apps' 2-minute poll still catches it.
tools/jobs.mjs status reads relay_wake (one row per publish with the ids it added) and prints "woken +N s after the
publish" for a machine's latest job that a publish added; nothing when the table does not exist yet.
docs/plans/release-0.3.6.md: "Instant jobs" section with the design, the expected latency and a TODO row per machine
for the measured number once 0.3.6 is live. packaging/ota/README.md: the 10-minute poll is history.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
