igneum/packaging/ota

Over-the-air updates (packaging/ota)

the project lead's rule (4 October 2026): every app updates itself and downloads the update without being asked. The Igneum Miner app on Windows and macOS does, and the node, the miner and the GPU workers ship inside it, so a consensus upgrade (a height-activated rule such as difficulty v2) reaches every node before its activation height.

The launcher packages (proto-cuda/windows-app, the igneum-windows-v4.zip console launchers, the Terminal DMGs 0.1.0 and 0.2.0) are NOT auto-updated, by design: they are the engineering path and are replaced by hand.

The pieces

Piece	Where	What it does
manifest	dl.igneum.network/dl/<token>/igneum-app-latest.json + .sig	version, per-platform file (url, sha256, size, kind), min_supported_version, notes, consensus activation height
signer	app/igneum-app/src/bin/ota-sign.rs (igneum-ota-sign, built with the app, never shipped)	keygen, sign, verify, sha256; includes src/manifest.rs so it signs what the app verifies
publisher	packaging/ota/publish-manifest.sh	copies the DMG or installer into the downloads folder, writes the canonical manifest, signs it, prints or runs the deploy
verifier	app/igneum-app/src/manifest.rs	Ed25519 check of the manifest bytes with the compiled-in public key, parse, version compare, safe-moment rule, unit tests
updater	app/igneum-app/src/ota.rs	check, download with resume, verify, stage, apply at a safe moment, rollback; update in /api/state
dashboard	app/igneum-app/ui	one banner (available, downloading, ready, applying, red bar for a close fork), Settings: Check now, Install now, automatic switch
Windows installer	packaging/windows/Igneum-Miner.iss	CloseApplications=yes, RestartApplications=no, a [Run] relaunch on /IGNOTA=1
CI loop	packaging/windows/fetch-ci-artifacts.sh	after copying the installer it calls publish-manifest.sh --win (the Mac entry is carried over); --deploy ships both

Keys

Generated once on the Mac (4 October 2026), never in the repo or in CI:

app/igneum-app/target/release/igneum-ota-sign keygen ~/.config/igneum/ota-signing-key ~/.config/igneum/ota-signing-key.pub

ota-signing-key is the 32-byte seed as hex, mode 0600. The public key is the constant OTA_PUBLIC_KEY_HEX in app/igneum-app/src/manifest.rs; igneum-ota-sign embedded prints it with its fingerprint (SHA-256 of the 32 key bytes). publish-manifest.sh refuses to sign when the embedded key is not the one in ~/.config/igneum.

Key rotation: a new key means a new app build (the constant), published and signed with the OLD key, then the next manifest signed with the new one. Apps that skipped the bridge build stop updating and show "manifest signature does not verify"; they are updated by hand from the download page.

Publishing a version

1. Bump version in app/igneum-app/Cargo.toml (and app/windows/version.h, resources/igneum-app.rc, as the CI smoke run demands). Commit, push: the Windows installer builds on GitHub.

2. Mac: packaging/mac/build-dmg.sh, then

    packaging/ota/publish-manifest.sh --version 0.3.1 --mac packaging/mac/dist/Igneum-Miner-0.3.1.dmg \
        --notes "difficulty v2 and over-the-air updates" [--activation-height 120000 --deadline-note "difficulty v2"]
   

   writes dl/<token>/igneum-app-latest.json with the Mac entry only and prints the deploy command. Deploying now is fine: a Windows app finds no windows entry and does nothing.

3. Windows: packaging/windows/fetch-ci-artifacts.sh --deploy copies the installer, adds the Windows entry to the same manifest (same version, Mac entry carried over), deploys the downloads folder.

4. Every app checks within the hour (Settings > Check now at once): it downloads, verifies and installs at the next safe moment. The event feed shows each step; app-<run>.log has the detail.

--min-supported 0.3.0 marks older versions unsupported: they install at once, without waiting for a safe moment, and show the red bar. --activation-height N does the same once a node's DAA score is within 1,800 blocks of N.

What the app does

Check on start (20 to 50 s in) and every 60 minutes plus up to 10 minutes of per-machine jitter; after an error, again in 10 minutes. Both files come through curl (the engine carries no TLS stack); the signature is checked over the manifest bytes before parsing; a version that is not newer, or a manifest without this platform, ends the round.

Download into <app data>/app/updates/ (~/Library/Application Support/Igneum/app/updates, %LOCALAPPDATA%\igneum\app\updates) with curl -C - (resume) and --retry 3; then the size and the sha256 from the manifest; a file already there with the right hash is not fetched again. The banner shows the percentage.

Stage. macOS: mount the DMG (or unpack the zip), copy Igneum Miner.app to .Igneum Miner.app.new next to the running bundle (same volume: the swap is two renames), run its engine with --version and demand the manifest's version. When the folder is not writable the state is manual: the banner says so and offers "Open the download". Windows: the installer is the staged file.

Safe moment (manifest::safe_to_apply): node synced, no hourly program boundary within 180 s (program.eta_s), no worker starting. Urgent (fork within 1,800 blocks, or unsupported version, or Install now) skips the wait. A ready update that found no safe moment for 6 hours applies anyway (an unsynced node mines nothing).

Apply. The engine writes update-pending.json (from, to, starts), starts the helper detached and leaves through its normal quit path: miners first (8 s grace), then the node (30 s), the last log upload, EXIT for the window.

• macOS helper ota-apply.sh: waits for the engine, asks the window (network.igneum.miner) to quit, moves the bundle to Igneum Miner.app.previous, the staged one in, strips quarantine, open -n. If the new engine is not running after 30 s it opens once more; if that fails too it puts .previous back and reports.
• Windows helper ota-apply.ps1: waits for the engine, runs Igneum-Miner-Setup-<v>.exe /VERYSILENT /SUPPRESSMSGBOXES /NORESTART /CLOSEAPPLICATIONS /IGNOTA=1 /LOG=... as administrator (ONE UAC prompt: the installer is PrivilegesRequired=admin because of Program Files and the firewall rule). The installer's PrepareToInstall runs stop-igneum.ps1 (ends the window and anything left), replaces the files, and the [Run] entry on /IGNOTA=1 relaunches igneum-app.exe --launch as the signed-in user. A declined prompt or a non-zero exit relaunches the old app and reports the error in the banner (Install now retries).

Rollback. The helper writes update-result.json; the new engine reads it on start and reports "updated to X from Y" or the error. The new engine counts its starts in update-pending.json and deletes the file after 90 healthy seconds; a third start without reaching that point restores the previous version (macOS: the .previous bundle; Windows: the previous version's installer kept in updates/, so the FIRST update from 0.3.0 has no rollback target on Windows, said so in the state) and shows "rolled back" in Settings.

Settings: auto_update (default on). Off: downloads still happen, the banner waits for Install now. The forced screenshots: ?update=available|downloading|ready|waiting|applying|urgent|manual|error|updated on the dashboard URL.

Testing

Unit tests: cargo test in app/igneum-app (manifest parse, a bad signature and a tampered manifest refused, sha256 of a file, version ordering incl. pre-releases, safe-moment rules, fork closeness, unsupported versions).

Dry run on the Mac, 4 October 2026 (packaging/mac/README.md has the private-devnet recipe; ports 29700+): the 0.3.0 bundle from the tree ran under its window host against a private devnet with IGNEUM_APP_UPDATE_MANIFEST=http://127.0.0.1:29790/dl/<token>/igneum-app-latest.json (a python3 -m http.server over a folder written by publish-manifest.sh --base-url ... --dest ...; loopback http is the one non-https URL the parser accepts), found the 0.3.1 manifest, downloaded and verified the DMG, staged the bundle, waited for the worker to start, applied, and came back as 0.3.1 with "updated to Igneum Miner 0.3.1 from 0.3.0" in the event feed. The screenshots are docs/design/app-screens/update-*.png. Windows: reviewed only, see TEST.md.

Remote jobs (4 October 2026)

the project lead's rule: one app on both PCs that the Mac can send commands and files to over the line, so everything is tested and built without a person at the PC. The channel is igneum-jobs.json plus igneum-jobs.json.sig, next to the update manifest, signed with the same OTA key and verified by the same code; the apps poll it every 10 minutes. The relay (relay/) stays for the Mac and for humans; its PC agent is replaced by this.

Piece	Where
model: parse, verify, targeting, expiry, the once-per-id ledger (jobs-state.json in the app data folder), unit tests	app/igneum-app/src/jobs.rs
runner: poll, requirement probes, the kinds, reports, the dashboard state	app/igneum-app/src/jobrun.rs
signer: igneum-ota-sign sign-jobs and verify-jobs (a bad job is refused at signing)	app/igneum-app/src/bin/ota-sign.rs
publisher: `packaging/ota/publish-jobs.sh add	list
reader: the published file (signature checked), status per machine, a job's result, watch	tools/jobs.mjs

A job: {id, kind, title, created_at, expires_at, target: {machine_ids: [...] | "all", platform, requires}, params, report: "log-intake"}. Machine ids are the per-install id (16 hex) or its first 8 (PC 1 ae432dc7, PC 2 1ccfe586). Requirements the engine probes: wsl, wsl-prover (cargo, ~/.sp1 and ~/igneum-prove inside Ubuntu-24.04), nvidia; an unknown one is never met and the job waits until it expires.

Kind	What the app does	Params
run	writes the script body to <app data>/app/jobs/<id>/ and runs it (powershell or bash), output captured, exit code reported	script, shell, elevated, stop_miners_first, timeout_minutes (60, at most 600)
fetch	downloads by https, checks the sha256 (and size), into the job folder or a named app folder; can extract	url, sha256, size, dir jobs/prove/packs/updates, to, extract, extract_dir, fresh
collect	uploads files matching globs under the app data root, and/or a command's output, to the intake	globs, command
restart	miners, node (the miners follow) or the app (a detached helper opens it again)	what
update-now	the updater checks and installs a newer version at once
shard-benchmark	miners stopped, GPU under 5%, the prove zip fetched fresh, prove-shard.sh inside WSL under the cap, RESULT and STAGE lines and results/*.json uploaded, miners back	zip_url, sha256, size, fixtures, cap_minutes (90), distro, wsl_user

Reports: every job uploads to the log intake as run_id job-<id>-<machine id8> with the label job-<kind>; the first line is SUMMARY {json} (status, exit, times, the RESULT lines, per-kind extras), then the captured output. Long jobs upload a running report every 5 minutes; collected files and result files are separate labels (file-<name>, result-<name>, prove-log) under the same run_id.

Safety: the file is rejected when the signature or any job's expires_at or params fail; a job id runs once per machine (a crash mid-job counts); nothing writes outside the app data folder except an explicit run script, which is the operator's responsibility; the dashboard shows the running job and a history (id, kind, started, exit, report uploaded); Settings has "Allow remote jobs from Igneum (signed)", ON by default on this devnet build, with the key fingerprint, and OFF aborts the running job. elevated needs someone at the UAC prompt (or UAC set to elevate without prompting); unattended it fails after the prompt times out. A restart app or update-now job relaunches through the usual quit path.

Publishing and reading:

packaging/ota/publish-jobs.sh add --kind shard-benchmark --target 1ccfe586 --title "Shard proof run on the 5090" --deploy
packaging/ota/publish-jobs.sh add --kind run --target ae432dc7,1ccfe586 --script fix.ps1 --title "..." [--elevated] [--stop-miners]
packaging/ota/publish-jobs.sh list | remove <id> | sign
node tools/jobs.mjs | status | <id> [--all] | watch <id>

Tested on the Mac, 4 October 2026: the unit tests (parse, bad jobs refused, signature and tampering, times, targeting, the once-only ledger, globs), the signer with the real key, the publisher against a scratch folder, the reader against the live intake; the crate also compiles for x86_64-pc-windows-gnu. Not yet run on a PC: the first job goes to PC 2 (1ccfe586) once 0.3.2 is installed there (docs/plans/shard-test-pc2.md).