igneum/docs/security/keys.md

Keys: inventory, backup, the signing-key plan (5 October 2026)

Internal. Every key the project depends on sat unencrypted in ~/.config/igneum on one Mac with no backup. This file is the inventory written from the real files and the scripts that read them, the backup and restore commands, the plan for a second OTA signing key, and the emergency path if the one key leaks. No value is written here. The only fingerprint quoted is the public key's. Owner of every rotation below: Josh, unless a row says otherwise.

Modes read on 5 October 2026 18:50 UTC: every secret file 0600; the folder itself was 0755 and is 0700 since this branch (vercel/ and txgen/ too). ota-signing-key.pub, build-slots and vercel/config.json (team ids, no token) are 0644, which is fine.

1. The inventory

Column "lost" = the Mac dies and there is no backup. Column "leaked" = someone else holds the value.

Name	Where it lives	What it unlocks (readers)	If lost	If leaked	Rotate: who, how	Rotation status
ota-signing-key (Ed25519 seed, 32 bytes as hex, 65 B, 0600, made 4 Oct 12:32)	the Mac only. Never in CI, never in the repo. Its public half is compiled into every app (app/igneum-app/src/manifest.rs:28, OTA_PUBLIC_KEY_HEX, fingerprint sha256 8f186e37b48cfddf52d2b37478c562d78a74236fc87f0a675393ed6a35baac4e) and on the Mac as ota-signing-key.pub	signs the update manifest igneum-app-latest.json (packaging/ota/publish-manifest.sh, publish-public.sh, tools/ship-app.mjs), the remote jobs file igneum-jobs.json (publish-jobs.sh, tools/jobs.mjs; kind: run jobs execute on every app machine, jobrun.rs:800) and the Windows build inputs payload-inputs.json (packaging/windows/push-inputs.sh, verified in CI with the embedded key, windows.yml:172). Verified by ota.rs:1030, jobrun.rs:800-811, igneum-ota-sign verify-inputs embedded	no new manifest, job or Windows build can be signed. Every installed 0.3.x keeps checking hourly and never updates again. The only way forward is a hand reinstall by every miner of a build that embeds a new key: the fleet is stranded	whoever holds it signs a manifest or a jobs file the apps accept. With the manifest URL (the folder token is in every app) and a way to serve it (the Vercel token, or DNS) they run code on every machine	Josh only. No fleet rotation path exists today (section 4): igneum-ota-sign keygen, change OTA_PUBLIC_KEY_HEX, ship, and every machine must apply that build first	never rotated; one key; backup = this branch
ota-signing-key.pub (65 B, 0644)	the Mac; the same bytes in the app	the local check of every publish (publish-manifest.sh, ship-app.mjs:563, test-publish-jobs.sh)	nothing: igneum-ota-sign embedded prints it from any app build	nothing, it is public	with the private key	with the private key
relay-token (28 B, 0600, 4 Oct 19:23)	the Mac; RELAY_TOKEN on Vercel igneum-relay; baked into the relay clients on PC 1 and PC 2 (relay/clients/make-clients.sh:8-12, igneum-agent.ps1); the phone bookmark	the relay URL /r/<token>/: read and post the feed, the drop box, and POST task kind: run on the PCs with only this token (docs/fud-ledger.md X23/X24, still open). Readers: tools/relay.mjs, console.mjs, build-job.mjs, ship-app.mjs, packaging/ota/publish-jobs.sh	generate a new one, set the env, rebuild the clients, redistribute to the PCs and the phone; nothing is unrecoverable	elevated command execution on PC 1 and PC 2, and every file on the relay	Josh: new value into the file, vercel env rm/add RELAY_TOKEN production --scope igneum, deploy, make-clients.sh, install on both PCs, delete the old	rotated 4 Oct 2026 (the .old-2026-10-04 copy is dead and can go)
relay-key (48 B, 0600, 4 Oct 19:23)	the Mac; RELAY_KEY on igneum-relay; in the relay clients on the PCs	the same relay, as the x-igneum-key header for scripts (relay/lib/relay.mjs:34-44; its own key since round 4 X23, no longer the intake key)	as the token	as the token	as the token	new 4 Oct 2026
dl-token (11 B, 0600, 4 Oct 19:25; new value since the 5 Oct rotation)	the Mac; DL_TOKEN GitHub secret (the Windows runner writes it to ~/.config/igneum/dl-token, windows.yml:147); DL_TOKEN on igneum-relay (the console); in every installed app's manifest URL (0.3.6 and later; 0.3.5 and older carry the old one)	the private downloads folder dl/<token>/ on dl.igneum.network: installers, the manifest, the jobs file, the CI inputs. Readers: packaging/mac/build-dmg.sh, packaged-config.sh, packaging/ota/*.sh, packaging/windows/*.sh, tools/ship-app.mjs, logs.mjs, jobs.mjs, console.mjs, build-job.mjs, relay.mjs, app/igneum-app/src/config.rs	the folder name is in every installed app's igneum-app.json and in the GitHub secret's consumer; recoverable from any install	the installers and the signed files are readable (the signature still guards what the app accepts); low	Josh, by docs/plans/rotation-phase-2.md (a second folder, a build that carries the new token, delete the old folder when tools/logs.mjs --rotation reads 0 behind)	rotated 5 Oct 2026; the old folder dies on 7 Oct (8f)
dl-token.old-2026-10-05 (12 B)	the Mac; still the folder name 0.3.5 and older apps read; in 2 commits of the history (masked on master; tools/repo/fresh-repo.sh rewrites it)	the OLD folder until 8f	nothing	as above, low	delete on 12 Oct per 8f step 6, after the fresh repository is pushed (the rewrite reads it)	dying
log-intake-key (32 B, 0600, 4 Oct 19:25; the NEW value since 5 Oct)	the Mac; Vercel igneum as LOG_INTAKE_KEY_NEXT (new) and LOG_INTAKE_KEY (old) until 8f; the same pair on igneum-relay; GitHub secrets LOG_INTAKE_KEY_NEXT (new) and LOG_INTAKE_KEY (old); in every installed app 0.3.6 and later (igneum-app.json); PC build scripts via IGNEUM_INTAKE_KEY	POST /api/log on the site and fn=upload on the relay (site/api/log.mjs:45, relay/lib/relay.mjs:44): write-only telemetry. Readers: packaging/mac/packaged-config.sh, infra/gpu-bench/upload.sh, tools/build-job.mjs, logs.mjs, ship-app.mjs, repo/fresh-repo.sh, app/igneum-app/src/config.rs	Vercel and GitHub keep it write-only (Hidden), so a new key must be generated and repackaged (phase 2 again)	junk rows in Neon and junk uploads to Blob; no read; low	Josh, by phase 2	rotated 4 to 5 Oct 2026; the old key dies on 7 Oct (8f)
log-intake-key.old-2026-10-05 (24 B)	the Mac; the 0.3.0 to 0.3.5 installs and the 0.2.0 launcher machines; in 9 commits of the history (0 tracked files on master; fresh-repo.sh rewrites it)	the intake, until 8f	nothing	low (write-only)	delete on 12 Oct per 8f step 6	dying
hetzner-token (64 B, 0600, 3 Oct 22:43)	the Mac only	the Hetzner Cloud API: create and delete servers (infra/seed-nodes/config.sh:29, infra/cloud-devnet/lib/common.sh:25-27): the seed nodes and the cloud devnet, on Josh's bill	make a new one in the Hetzner console; the servers stay	servers created on the bill, the seed nodes and the devnet deleted, every server listed	Josh: Hetzner console, Security, API tokens: new token, write the file, delete the old	never rotated
desec-token (28 B, 0600, 3 Oct 19:34)	the Mac only	the deSEC DNS API for the igneum.network zone (infra/seed-nodes/dns.sh:4-11; the relay CNAME lives there, relay/README.md:73). CLAUDE.md says the domains sit on Vercel nameservers (3 Oct); dns.sh is the later file. Approximate until Josh confirms which nameservers answer today	make a new one at deSEC	the zone: point dl, relay or the site anywhere, get a certificate for it, serve a fake manifest (the signature still guards the apps) and a fake site; high	Josh: deSEC, token management	never rotated
dev-fee-devnet.json (249 B, 0600; purpose, address, private_key)	the Mac only	the devnet (chain 4463) funder for tools/txgen/run.mjs (--funder, line 57). Devnet coins only	devnet funds; regenerate	devnet coins; nothing real	any time: a new wallet, fund it on the devnet	none needed
dev-fee-release.json (234 B, 0600; an address only)	the Mac	DEV_FEE_ADDRESS, Josh's payout address from the Igneum Wallet (docs/design/miner-dev-fee.md:50). No private key here: the key is in the Igneum Wallet on Josh's Mac, outside this folder and outside this backup	the address is in the fork's release-0.3.6 source	nothing, an address is public	not a secret. The wallet's own key needs its own backup (open)	n/a
txgen/wallets.json (3,090 B, 0600; 16 devnet wallets with keys)	the Mac only	the devnet load generator (tools/txgen/run.mjs:56)	regenerate	devnet coins; nothing real	any time	none needed
vercel/auth.json (397 B, 0600; token, refreshToken, expiresAt) and vercel/config.json (team ids, 0644)	the Mac only (--global-config ~/.config/igneum/vercel)	the igneum team: projects igneum (site), igneum-dl (downloads), igneum-relay; deploys, env vars (add, remove, pull the non-sensitive ones), domains. Readers: packaging/ota/*.sh, packaging/windows/*.sh, tools/ship-app.mjs	vercel login again as the igneum.network Google login; nothing unrecoverable	deploy anything to the three hosts, including a fake manifest at the real URL (still unsigned without the OTA key), read BLOB_READ_WRITE_TOKEN, delete projects; high	Josh: Vercel, Settings, Tokens: revoke; vercel logout/login. The access token expires (it is OAuth with a refresh token; 5 Oct: expiry the same evening)	expires on its own; the refresh token does not
env (406 B, 0600; DATABASE_URL, DATABASE_URL_UNPOOLED)	the Mac; DATABASE_URL on all of igneum and igneum-relay	Neon igneum (soft-voice-31914738, London): telemetry rows, faucet grants, the relay feed, tasks and wake stamps, the jobs ledger. Readers: tools/build-job.mjs, jobs.mjs, logs.mjs, tuning.mjs, observer/observer.mjs, infra/cloud-devnet/experiments/observer.sh, site/api/*.mjs, relay/lib/relay.mjs	Neon console, reset the role password; nothing unrecoverable	read every upload and relay message; write rows, including relay tasks the PC agents poll and run; high	Josh: Neon, reset password, then the file and both projects' DATABASE_URL, redeploy	never rotated
build-slots, dlsite-dir, pytools/	the Mac	a build cap, a local folder path, a pip copy of git-filter-repo	nothing	nothing	not secrets; excluded from the backup	n/a
GitHub tokens: igneum-josh (admin:org, repo, workflow) and joshmalone117 (gist, read:org, repo, workflow)	the macOS keychain through gh (gh auth status), not in this folder	the organisation igneum-network and the repository: push, Actions, the repository secrets, the Windows runner	gh auth login again	push to master (the site deploys on push), rewrite secrets, run workflows that receive DL_TOKEN and the intake key; the runner never holds the signing key, so no release can be signed from it; high	Josh: GitHub, Settings, Applications, revoke GitHub CLI; gh auth login	never rotated
GitHub repository secrets DL_TOKEN, LOG_INTAKE_KEY, LOG_INTAKE_KEY_NEXT	GitHub, write-only copies of the files above	the Windows build (windows.yml:132-152)	re-set from the files (gh secret set)	as the files	with the files; 8f step 3 drops _NEXT	in rotation
Vercel env igneum: FAUCET_KEY (two environments), LOG_INTAKE_KEY, LOG_INTAKE_KEY_NEXT, DATABASE_URL	Vercel, Hidden (not readable back)	FAUCET_KEY is the faucet wallet's private key (site/api/faucet.mjs:2): it exists ONLY on Vercel, not on the Mac, so it is not in this backup	the faucet wallet's funds are unreachable. Devnet today (chain 4463), so nothing real; the public testnet (4462) "gets its own key": generate THAT one on the Mac into ~/.config/igneum/faucet-testnet.json first, then vercel env add from the file, so the backup covers it	the faucet's balance; a testnet drain	Josh: new wallet, vercel env rm/add, move the balance	file-first rule for the testnet key (open)
Vercel env igneum-relay: DL_TOKEN, LOG_INTAKE_KEY, LOG_INTAKE_KEY_NEXT, RELAY_KEY, RELAY_TOKEN, DATABASE_URL, BLOB_READ_WRITE_TOKEN	Vercel. All Hidden except BLOB_READ_WRITE_TOKEN, which is a plain variable (vercel env ls prints its prefix and vercel env pull fetches it)	the Blob store of the relay (files, build outputs from the PCs)	regenerate in the Vercel dashboard (Storage, the Blob store, tokens)	read, write and delete every relay file; medium	Josh: dashboard; re-add as Sensitive (vercel env add BLOB_READ_WRITE_TOKEN production --sensitive) so it stops being readable	recommend: make it Sensitive
Vercel env igneum-dl	none	the downloads host deploys from the folder; nothing secret in env				n/a

Copies that live outside the Mac and are not in the backup: the relay token and key inside the client scripts on PC 1, PC 2 and the phone; the intake key and the folder token inside every installed app's igneum-app.json; the dated old values in ~/igneum-dl-old-20261005 (folder files, not keys). None of them is needed to rebuild the Mac.

2. The backup and the restore

tools/keys/backup.sh --dry-run          # what would go in: names, modes, sizes; creates nothing
tools/keys/backup.sh                    # ~/Desktop/igneum-keys-<date>.dmg, AES-256, hdiutil's own passphrase prompt
                                        # (twice: create, then the verify attach); verified by sha256, listed, detached
tools/keys/restore.sh <dmg> --check     # every file in the image against ~/.config/igneum: match / DIFFERS / missing
tools/keys/restore.sh <dmg> --to <dir>  # copy back (0700 dirs, 0600 files, .pub 0644), refuses to overwrite without --force
tools/keys/test-backup.sh               # the end-to-end test on a scratch folder with a throwaway passphrase

The image holds every file of ~/.config/igneum except build-slots, dlsite-dir and pytools/, plus README.txt (the listing and a copy of this file). The passphrase never passes through argv, the shell history or a file: hdiutil prompts on the terminal (--agent for the macOS dialog). --stdinpass exists for the test harness only.

What Josh does with the image: two copies on two media that are not this Mac (a USB stick at home and a second stick or an encrypted cloud folder), the Desktop copy deleted, the passphrase on paper away from both media. Run it again after every rotation and replace both copies; keep one older image. restore.sh --check after each run.

Test record, 5 October 2026: tools/keys/test-backup.sh passed all 8 steps (dry run lists 16 files and creates nothing; create and verify report 17 files byte-identical; --list; --check matches; a changed live file makes --check fail and name the file; --to restores 16 files with 0600/0644 and 0700, refuses a second run without --force; a wrong passphrase is refused; the raw image bytes do not contain the test values). The real folder was run in --dry-run only.

3. What is exposed today, and what was done

Finding	Fix
One copy of every key, on one disk, unencrypted	this branch: the encrypted image and the two-media rule (section 2)
~/.config/igneum was 0755 (listable by any local user; the files themselves were 0600)	chmod 700 on the folder, vercel/ and txgen/ (done 5 Oct)
The old intake key sits in 9 commits of the history and the old folder token in 2 (0 tracked files on master)	known: tools/repo/fresh-repo.sh rewrites both after 8f; the fresh repository plan (docs/plans/history-rewrite.md). Not changed here
BLOB_READ_WRITE_TOKEN on igneum-relay is a plain env var, readable by anyone with project access	recommend: re-add as Sensitive (section 1)
FAUCET_KEY exists only on Vercel, write-only, no copy anywhere	recommend: the testnet faucet key is generated on the Mac into the folder first, then set from the file
relay-token.old-2026-10-04 is a dead value still on disk	recommend: rm -P it (nothing reads it; fresh-repo.sh reads only the intake and dl files)
The dev-fee payout key lives in the Igneum Wallet, outside this folder and this backup	open: the wallet's own backup
Published Hardhat and Anvil developer keys in tools/evm-smoke/smoke.mjs and tools/exec-attacks/lib/common.mjs	public test vectors, allowlisted in the CI check; never fund those addresses on testnet 4462 or mainnet
Nothing in CI, no token in any script: every script reads a file under ~/.config/igneum or an env var (checked: git grep of every file name above and of the current values, 0 hits in tracked files)	tools/ci/no-secrets-check.sh keeps it so (section 5)

4. The OTA signing key: today, the second key, the emergency path

Today. One Ed25519 key signs three things: the update manifest, the jobs file and the Windows build inputs. Its public half is one constant, OTA_PUBLIC_KEY_HEX, read at ota.rs:1030, jobrun.rs:168,800,811 and by igneum-ota-sign embedded|verify-inputs. Every app in the field trusts that one key and nothing else: on 5 October 2026 18:40 UTC the intake shows 0.3.8 (one Mac), 0.3.7 (one Windows PC), 0.3.5 (Sam's Mac) and three machines without a version line (node tools/logs.mjs --rotation). The signature file is 64 raw bytes as 128 hex, no key id.

The second key, as the next step (one release, about two hours of work).

Step	What
1	igneum-ota-sign keygen a second key (K2) on the Mac with the output directed INTO a mounted igneum-keys image, so its private half exists only inside the encrypted copies; keep ota-signing-key-2.pub on disk. The current key is K1
2	manifest.rs: OTA_PUBLIC_KEYS: &[&str] = &[K1, K2]; verify_signature tries each key in turn (two Ed25519 verifies, microseconds); fingerprint() of each; embedded prints both. ota.rs, jobrun.rs, jobs.rs, inputs.rs and ota-sign.rs take the slice instead of the constant. Tests: a manifest signed by K2 verifies, by a third key fails, the existing K1 vectors still pass. The .sig format does not change, so 0.3.x apps keep verifying K1 signatures of the same files
3	The same release carries revocation: the manifest gains an optional revoked_keys: [<fingerprint>], signed like the rest. An app that verifies a manifest with the OTHER key and sees its current key listed writes updates/revoked.json and refuses that key from then on. Without this, a leaked K1 stays trusted forever (step 2 alone covers loss, not leak)
4	Ship it as 0.3.9, signed with K1, so every 0.3.5 to 0.3.8 machine takes it on its hourly check. From then on the publisher keeps signing with K1; K2 stays offline. tools/logs.mjs --rotation gains a column for the embedded fingerprints the app logs (it already logs fingerprint at jobrun.rs:168)
5	When every machine reports 0.3.9 or later, K2 is live as the fallback: K1 lost = mount the image, sign with K2, ship; K1 leaked = sign the next manifest with K2 with K1 in revoked_keys

If K1 leaks TODAY, before 0.3.9: a manifest signed with any new key is useless. Every 0.3.x app verifies with K1 only (ota.rs:1030), rejects the new signature ("manifest signature does not verify"), stays on its version and keeps fetching the same URL every hour, where it will accept anything K1 signed. Plainly: the fleet cannot be moved to a new key by the update path, and it stays open to whoever holds K1 for as long as that URL serves. The mitigation, in order:

Order	Action	Effect
1	Take the manifest and the jobs file off the folder (dl/<token>/igneum-app-latest.json, .sig, igneum-jobs.json, .sig): a deploy of the downloads folder without them	the apps read 404 as "no manifest at the update URL yet" and stay put; the attacker's signed manifest has nowhere to be served unless they also hold the Vercel token or the DNS
2	Rotate the folder token (phase 2 again) and the Vercel token, the deSEC token, the Neon password, the relay token and key, the GitHub tokens: one leaked file from this folder means the folder was read	the old URL dies; the attacker cannot place a file at the new one
3	Build 0.3.9 (section 4 steps 1 to 3, with K1 NOT in the trusted list) and sign its manifest with K1, once, into the NEW folder; publish it before anything else goes there	every machine that applies it stops trusting K1. The race is real: a machine that fetched an attacker's K1 manifest first is owned; logs.mjs shows which machines moved
4	Machines that did not move (asleep, offline, or owned) are reinstalled by hand from the public links (dl.igneum.network/public/, packaging/README-ship.md)	the only path for an app that never saw step 3

Until 0.3.9 ships, the K1 file is the single point of failure in both directions, and this branch's backup covers the loss direction only. The leak direction is covered by step 3 of the plan, and by keeping K1 off this disk: the publish scripts read ~/.config/igneum/ota-signing-key by path, so the file can become a symlink into a mounted image that is attached only for the minutes of a publish (a follow-up, not done here; publish-manifest.sh:36, ship-app.mjs:305).

5. The CI check

tools/ci/no-secrets-check.sh runs in ci.yml (site job) after a --self-test that must fire on a known-bad tree (a tracked ota-signing-key, a dl-token.old-<date>, an igneum-app.json, FAUCET_KEY=0x<64 hex>, signingKey = "<64 hex>", x-igneum-key: <64 hex>) and stay quiet on a known-good one (a sha256 next to another word, a 32-hex id, the public key in manifest.rs, a .test.mjs vector). Over the tree it refuses any tracked file named like a key of ~/.config/igneum (with .next and .old-<date> variants, *.env, .env*, a bare env, auth.json, wallets.json, igneum-relay-clients.zip, igneum-log-key.txt) and any 64-hex value (optionally 0x) assigned to a name ending in token, key, secret, password or passphrase, outside test files, proving/fixtures/, infra/cloud-devnet/results/ and *.log. Allowlisted by path with the reason in the script: the OTA public key, and the published Hardhat and Anvil accounts in the devnet tools. 5 October 2026: self-test passed, 776 files checked, 0 hits. Hits are printed with the hex masked.