diff --git a/docs/plans/counter-asic-3-status.md b/docs/plans/counter-asic-3-status.md index 0c551d60..f25a9b57 100644 --- a/docs/plans/counter-asic-3-status.md +++ b/docs/plans/counter-asic-3-status.md @@ -411,7 +411,17 @@ Self-test PASS on every pack (cache FNV 448274a57f508cbc); both rows in the ledg | p34 | 1.25x | | site 1, 1.35 percent | | | p4 | 1.22x | | site 1, 1.45 percent | unattributed, 1.57x on sub-version 1 | -Every failing seed is ONE low-entropy load site; the mechanism is lineage-blind (AP-F8-1's residual) and the acceptance could not see it because it never ran the shadow block (AP-F8-3). SUB-VERSION 3's FIRST COMMIT: ca3-v4-amend ddacfbd3, 14:20:37Z (origin and build, gate GREEN): the acceptance executes the shadow block as the hash does, the test pins it to verify.rs on the seven v4 programs; PROGRAM_SUBVERSION_V4 = 3; byte 7; (A) and (B) held in a stash. Epoch-0 id a785001687d8688a (must-differ c120d7963abdcd96, 1a4230699a6b9c60, a788661687db4bb3; the devnet seed still accepts at attempt 1, so its program is sub-version 2's under the new id); the seven fingerprints unchanged from sub-version 2 for the same reason (e370fb2080b7dbb1, b7237555d31fc3cf, b6b167fa15dfe2c9, 28bdf65eff33f2c4, e26d38c46f3f1b16, dd8fdf6ff4f59eed, 8bf40f5cb858d835; control 90f794dd556f7a3b; Metal = Apple OpenCL 14:18:41 to 14:19:13Z), so the fleet and PC 2 G1s already read them and the G1 on sub-version 3's packs is a re-run of a known result; packs zip packs-ca3-v4-sub3 sha256 4f2445c50c58d76a5544023492d8b858d0b07c5e372d31f9c90c4ce51f829154; the ledger entry carries AP-F8-2's exhaustion half FIXED-AND-PASSED at 8bdcbdd8 and AP-F8-3 fixed here; the suite and the 4,096-seed census at ddacfbd3 on box 2. THE LOCALISATION (the hash lane): p23's band is dataset- and nonce-independent and reproduces in the acceptance's own execution: site 7 (instruction 38) reads r6 after 25 mulhi, 31 or r6 |= r4, 35 xor r6 ^= r4, which is r6 and not r4, an AND mask the lineage rule counts as fresh because the xor's operand is the or's; over 2^20 evaluations on the closed-form words site 7 reads 874,953 distinct word indices against about 1,046,500 at every other site (0.84 of uniform, 2.2 s on one box-2 core); over 2^24, 8,979,203 against about 16,260,000 (0.55, 35 s). THE COST LINES FOR THE SECOND COMMIT (0.3.22): structural (an abstract value class tracking shared operands) 0 s per attempt, this idiom only; the distinct-index ratio at 2^20 2.2 s per chosen candidate; at 2^24 35 s; no dataset-aware form needed (no cache fill). The hash lane's recommendation: the ratio at 2^20 with the threshold set from the clean seeds' per-site spread (p23's site 0.84; every clean site 0.995 to 1.000), plus the structural rule for the idiom so the ratio is a never-firing check on it. The attack-pass lane's verdict: sub-version 3 is the stream to gate and cannot read green by 19:00Z; byte 5 for 0.3.21 stands on its evidence. THE THRESHOLD NUMBERS FOR SUB-VERSION 3's SECOND COMMIT (the hash lane, box 2, the per-site distinct-index ratio over 2^20 evaluations against the window expectation N minus N^2 / 2W, on the 64 F8 programs as ddacfbd3 draws them, 2.8 s per seed on one core, all sixteen sites): the 55 clean seeds' per-seed minimum 0.9962 to 1.0000 (median 0.9999); over all 880 clean site rows min 0.9960, p1 0.9990, p5 1.0000, median 1.0000. The nine failing seeds' minimum site: p23 0.8361 (site 7), p18 0.9274 (site 6), p19 0.9335 (site 15), p15 0.9432 (site 2), p56 0.9654 (site 2); then p34 0.9927, p4 0.9961, p8 0.9963, p10 0.9963. A threshold of 0.98 sits 0.016 under the clean minimum and 0.015 over the strong five's maximum and rejects exactly those five; the weak four (1.22x to 1.50x in F8's gate) sit inside the clean spread at 2^20 and no threshold reaches them without rejecting clean seeds; a 2^24 run (35 s per candidate) on the weak four, p23 and five clean seeds measures whether they separate there (p23 went 0.84 to 0.55). The structural rule is in and bites where the band was: with the shared-operand relation tracked in the draw and in (a'), p23's attempt 1 draws site 7 from r5 instead of r6 (the or-then-xor on r4 marked r6 lossy); the devnet seed still accepts at attempt 1 (id unchanged); the (a') fixpoint carries the relation. THE COORDINATOR'S LINE: decide by the 2^24 lines: if they separate the weak four from the clean seeds with a gap at least the 2^20 gap, commit (iii), the 2^20 ratio at 0.98 always plus the 2^24 ratio only when the 2^20 per-seed minimum sits under a band edge set from the clean p1 with margin (0.999 fires it on about 1 percent of clean candidates, 35 s once an hour on a node); otherwise commit (i), the 2^20 ratio at 0.98, and name the weak four (p4, p8, p10, p34) as the open tail in the ledger and the commit, unattributed-and-chased, not absorbed. 0.3.21's STAGING (the node lane): the order dry-merges onto 55768f88 with nothing moving to 0.3.22; the late-join fix is 52e96c94 (70e4601e rebased onto 55768f88, exec suite 33 green with both new tests); f067f7c1, b0444f51 and 437f0438 merge clean in order; 2e32d5f6's one conflict (DST_ADDRESS beside pool-finish's DST_BINDING in consensus/core/src/finality.rs) kept both; the live-file digest eada4bda after each (every switch at never); the staging waits on the shipper's sweep-end word; the re-pin held. PC 2 DOWN AGAIN (main, 16:5x UK): the project lead takes PC 2 down for cable work (PC 1 back but his desk); both PCs out of the sweep's waves, each updates on its poller on return; no PC job to PC 1; the Windows G1 completed before the outage, nothing reruns. 0.3.21's SECOND GATE LINE on 55768f88 (sha256 279b1b690e854fc9): the ten-minute mixed-version gate beside the 5899f603 pair, 13:37:40Z to 13:47:52Z, SUMMARY PASS (one digest b0afb2ee on five nodes; 223 new and 381 old blocks accepted by the old hub, 0 rejected; counts equal at 319, 486 and 604 through both clean joins and the restart step at 13:45:22Z; no panic); the node lane's two lines on 0.3.21's first candidate complete, in plan 6.9 on ca3-v4-node; the fleet's set on it (the bare-child 12 GB line, the wipe, the kept read, the cases) is the fleet's. 0.3.21's FIRST GATE LINE on 55768f88 (sha256 279b1b690e854fc9, the string read back; pairing igneum-pow 8c728ca3 at byte 5): the digest gate 13:35:41Z to 13:37:19Z SUMMARY PASS (a89be8a7 on both binaries with the peers; db9a85f9 refused, no peer; the live file's eada4bda unmoved); the ten-minute mixed-version gate from 13:37:40Z, line about 13:50Z. The 0.3.21 order as the shipper sent it: 55768f88; f067f7c1 and 70e4601e; b0444f51; 6eb21fc9; db28d331; then the re-pin from 8bdcbdd8 on the coordinator's word; suites between, the digest read after every one; the mirror's release-0.3.20-node back at the pin c4459193, release-0.3.21-node open at 55768f88. THE LATE-JOIN COMMIT (N9's second half, the node lane): 70e4601e on the box mirror as branch proof-hold-fix, from c4459193, two files (igneum/exec/src/proving.rs, protocol/flows/src/v10/proving.rs); the gap was the fetch side on the joiner (the served record ran the native check against the joiner's trailing exec state before anything was stored, the check refused it, the proof was never held, the body rule read "not held" for 20 s and failed the IBD); the fix holds the proof by hash before the checks (the pool entry still needs them) and the serve side says when it holds fewer than asked; the exec suite 32 passed at 13:26Z with the known-failed shape first, the flows check green 13:28Z, igneumd on build-1 at the 0321 worktree path built 13:32Z, sha256 17649eeb2f7d1290, string read back; with the testnet lane (the resume form, B alone); it joins the 0.3.21 staging as its own commit. THE WIPE CANARY ON c19-1, c4459193 (sha 45be9b02d1b002f5, string read back): FORM END rc 0 at 13:50:53Z. Wipe synced 13:35:50Z (57 minutes, inside the 98-minute class); mining 13:36:00Z to 13:47:07Z, 66 mined, 66 accepted, 0 rejected, isSynced true at the tip throughout; the hub holds 41 of its blocks in its last 700 with 0 rejects (13:47:09Z); the restart on its kept datadir at 13:47:15Z: the old process stopped at once (the new process's first lock line seven seconds after the marker; the watchdog held nothing, the b7cc37e7 fault closed), synced again at 13:48:39Z after 84 s, 109 templates read with max 3,432 ms and 0 timeouts; the kept read on pool-1's 0.3.17 copy on the same pod passed at 13:38Z (the rewrite line once, a clean second start). The pin's set on c4459193: the digest gate PASS, the mixed-version gate PASS, the wipe canary PASS, the kept read PASS, the restart PASS, the 12 GB line proves and verifies (paid is a race, not a gate); CASES END from c20-1 (about 14:50Z) is the last pin line. THE INTEROP FACT stands from the void run: the 5899f603 hub accepted 235 object-byte-5 blocks from the 8097d600 node with 0 rejected, one digest on all five nodes on the live sixteen-field file. The gates: the digest test and the kaspa-pow vector test (the amended devnet epoch-0 id 1a4230699a6b9c60 must equal, c120d7963abdcd96 must differ, the v3 control unchanged) on the box; the mixed-version Devnet 2 gate (the amended 0.3.20 node beside a 5899f603 node for ten minutes on the live file without the v4 fields) after the Mac build; the fresh-join canary the 0.3.20 cut's | +Every failing seed is ONE low-entropy load site; the mechanism is lineage-blind (AP-F8-1's residual) and the acceptance could not see it because it never ran the shadow block (AP-F8-3). SUB-VERSION 3's FIRST COMMIT: ca3-v4-amend ddacfbd3, 14:20:37Z (origin and build, gate GREEN): the acceptance executes the shadow block as the hash does, the test pins it to verify.rs on the seven v4 programs; PROGRAM_SUBVERSION_V4 = 3; byte 7; (A) and (B) held in a stash. Epoch-0 id a785001687d8688a (must-differ c120d7963abdcd96, 1a4230699a6b9c60, a788661687db4bb3; the devnet seed still accepts at attempt 1, so its program is sub-version 2's under the new id); the seven fingerprints unchanged from sub-version 2 for the same reason (e370fb2080b7dbb1, b7237555d31fc3cf, b6b167fa15dfe2c9, 28bdf65eff33f2c4, e26d38c46f3f1b16, dd8fdf6ff4f59eed, 8bf40f5cb858d835; control 90f794dd556f7a3b; Metal = Apple OpenCL 14:18:41 to 14:19:13Z), so the fleet and PC 2 G1s already read them and the G1 on sub-version 3's packs is a re-run of a known result; packs zip packs-ca3-v4-sub3 sha256 4f2445c50c58d76a5544023492d8b858d0b07c5e372d31f9c90c4ce51f829154; the ledger entry carries AP-F8-2's exhaustion half FIXED-AND-PASSED at 8bdcbdd8 and AP-F8-3 fixed here; the suite and the 4,096-seed census at ddacfbd3 on box 2. THE LOCALISATION (the hash lane): p23's band is dataset- and nonce-independent and reproduces in the acceptance's own execution: site 7 (instruction 38) reads r6 after 25 mulhi, 31 or r6 |= r4, 35 xor r6 ^= r4, which is r6 and not r4, an AND mask the lineage rule counts as fresh because the xor's operand is the or's; over 2^20 evaluations on the closed-form words site 7 reads 874,953 distinct word indices against about 1,046,500 at every other site (0.84 of uniform, 2.2 s on one box-2 core); over 2^24, 8,979,203 against about 16,260,000 (0.55, 35 s). THE COST LINES FOR THE SECOND COMMIT (0.3.22): structural (an abstract value class tracking shared operands) 0 s per attempt, this idiom only; the distinct-index ratio at 2^20 2.2 s per chosen candidate; at 2^24 35 s; no dataset-aware form needed (no cache fill). The hash lane's recommendation: the ratio at 2^20 with the threshold set from the clean seeds' per-site spread (p23's site 0.84; every clean site 0.995 to 1.000), plus the structural rule for the idiom so the ratio is a never-firing check on it. The attack-pass lane's verdict: sub-version 3 is the stream to gate and cannot read green by 19:00Z; byte 5 for 0.3.21 stands on its evidence. THE THRESHOLD NUMBERS FOR SUB-VERSION 3's SECOND COMMIT (the hash lane, box 2, the per-site distinct-index ratio over 2^20 evaluations against the window expectation N minus N^2 / 2W, on the 64 F8 programs as ddacfbd3 draws them, 2.8 s per seed on one core, all sixteen sites): the 55 clean seeds' per-seed minimum 0.9962 to 1.0000 (median 0.9999); over all 880 clean site rows min 0.9960, p1 0.9990, p5 1.0000, median 1.0000. The nine failing seeds' minimum site: p23 0.8361 (site 7), p18 0.9274 (site 6), p19 0.9335 (site 15), p15 0.9432 (site 2), p56 0.9654 (site 2); then p34 0.9927, p4 0.9961, p8 0.9963, p10 0.9963. A threshold of 0.98 sits 0.016 under the clean minimum and 0.015 over the strong five's maximum and rejects exactly those five; the weak four (1.22x to 1.50x in F8's gate) sit inside the clean spread at 2^20 and no threshold reaches them without rejecting clean seeds; a 2^24 run (35 s per candidate) on the weak four, p23 and five clean seeds measures whether they separate there (p23 went 0.84 to 0.55). The structural rule is in and bites where the band was: with the shared-operand relation tracked in the draw and in (a'), p23's attempt 1 draws site 7 from r5 instead of r6 (the or-then-xor on r4 marked r6 lossy); the devnet seed still accepts at attempt 1 (id unchanged); the (a') fixpoint carries the relation. THE COORDINATOR'S LINE: decide by the 2^24 lines: if they separate the weak four from the clean seeds with a gap at least the 2^20 gap, commit (iii), the 2^20 ratio at 0.98 always plus the 2^24 ratio only when the 2^20 per-seed minimum sits under a band edge set from the clean p1 with margin (0.999 fires it on about 1 percent of clean candidates, 35 s once an hour on a node); otherwise commit (i), the 2^20 ratio at 0.98, and name the weak four (p4, p8, p10, p34) as the open tail in the ledger and the commit, unattributed-and-chased, not absorbed. SUB-VERSION 3's SECOND COMMIT: ca3-v4-amend 017e7037 (017e70376489251e18564c0abce7e466e606c8b3; origin and build; gate GREEN; CI run 37639406567 queued), choice (i) by the 2^24 lines: 2^24 does not separate the weak four from the clean seeds (weak p34 0.9181, p4 0.9614, p8 0.9630, p10 0.9612; clean p44 0.9612, p52 0.9613, p3 0.9971, p2 1.0004, p5 1.0004; p23's chain attempt 1 1.0004), two clean seeds sitting on the weak four's value, so any 2^24 floor that reaches them rejects clean seeds. Committed: the per-site distinct-index ratio at 0.98 over 2^20 alone (ACCEPT_UNITS_DISTINCT_V4 = 4096, MIN_DISTINCT_RATIO_V4 = 0.98; expectation per site N minus N^2/2W, window 2^28 >> min(win, 2), the shadow executed), no 2^24 stage, (B) unwired; the structural shared-operand rule in the draw and in (a'); the open tail named in the ledger and the commit: p4, p8, p10, p34 (0.9927 to 0.9963 at 2^20), unattributed and chased. + +| Threshold line | Value | +|---|---| +| Floor at 2^20 | 0.98 | +| Clean minimum over the 55 clean seeds' 880 site rows | 0.9960 (p1 0.9990, median 1.0000) | +| Strong five | p23 0.8361, p18 0.9274, p19 0.9335, p15 0.9432, p56 0.9654 | +| Margin | 0.015 to each side | +| Cost per chosen candidate | one 2^20 pass, 2.1 to 2.2 s on one box-2 core, once an hour on a node | + +Known-failed test class_v4_distinct_ratio_rejects_the_low_entropy_band green on box 2 (12.95 s): p15 attempt 3 (52638ea2e8b0fd68) site 2 at 0.943; p18 attempt 2 (9a37e9489d8ba698) site 6 at 0.927; p19 attempt 0 (79d7441de0689223) site 15 at 0.933; p56 attempt 2 (486a8ad2701ec3b5) site 2 at 0.965; p23 attempt 1 (d65122675f16a1c7) draws site 7 from r5 and passes, and with r6 put back is refused by (a') UnfreshLoadSource and, run anyway, by the ratio at 0.836. Stream unchanged from ddacfbd3 (re-export diff 0 on all eight packs): id a785001687d8688a, the seven fingerprints and the packs zip sha256 4f2445c5... as recorded. In flight on box 2: the crate suite and the 4,096 + F8 census at 017e7037; then the attack-pass lane's 64-seed gate at 2^24 and 10^6 exhaustion count. Owed as before: G2, G3, the ladder re-measure, AMD (PC 1), the 2019-class core, the fleet and PC 2 G1 on the sub-version 3 packs (a re-run of a known result). 0.3.21's STAGING (the node lane): the order dry-merges onto 55768f88 with nothing moving to 0.3.22; the late-join fix is 52e96c94 (70e4601e rebased onto 55768f88, exec suite 33 green with both new tests); f067f7c1, b0444f51 and 437f0438 merge clean in order; 2e32d5f6's one conflict (DST_ADDRESS beside pool-finish's DST_BINDING in consensus/core/src/finality.rs) kept both; the live-file digest eada4bda after each (every switch at never); the staging waits on the shipper's sweep-end word; the re-pin held. PC 2 DOWN AGAIN (main, 16:5x UK): the project lead takes PC 2 down for cable work (PC 1 back but his desk); both PCs out of the sweep's waves, each updates on its poller on return; no PC job to PC 1; the Windows G1 completed before the outage, nothing reruns. 0.3.21's SECOND GATE LINE on 55768f88 (sha256 279b1b690e854fc9): the ten-minute mixed-version gate beside the 5899f603 pair, 13:37:40Z to 13:47:52Z, SUMMARY PASS (one digest b0afb2ee on five nodes; 223 new and 381 old blocks accepted by the old hub, 0 rejected; counts equal at 319, 486 and 604 through both clean joins and the restart step at 13:45:22Z; no panic); the node lane's two lines on 0.3.21's first candidate complete, in plan 6.9 on ca3-v4-node; the fleet's set on it (the bare-child 12 GB line, the wipe, the kept read, the cases) is the fleet's. 0.3.21's FIRST GATE LINE on 55768f88 (sha256 279b1b690e854fc9, the string read back; pairing igneum-pow 8c728ca3 at byte 5): the digest gate 13:35:41Z to 13:37:19Z SUMMARY PASS (a89be8a7 on both binaries with the peers; db9a85f9 refused, no peer; the live file's eada4bda unmoved); the ten-minute mixed-version gate from 13:37:40Z, line about 13:50Z. The 0.3.21 order as the shipper sent it: 55768f88; f067f7c1 and 70e4601e; b0444f51; 6eb21fc9; db28d331; then the re-pin from 8bdcbdd8 on the coordinator's word; suites between, the digest read after every one; the mirror's release-0.3.20-node back at the pin c4459193, release-0.3.21-node open at 55768f88. THE LATE-JOIN COMMIT (N9's second half, the node lane): 70e4601e on the box mirror as branch proof-hold-fix, from c4459193, two files (igneum/exec/src/proving.rs, protocol/flows/src/v10/proving.rs); the gap was the fetch side on the joiner (the served record ran the native check against the joiner's trailing exec state before anything was stored, the check refused it, the proof was never held, the body rule read "not held" for 20 s and failed the IBD); the fix holds the proof by hash before the checks (the pool entry still needs them) and the serve side says when it holds fewer than asked; the exec suite 32 passed at 13:26Z with the known-failed shape first, the flows check green 13:28Z, igneumd on build-1 at the 0321 worktree path built 13:32Z, sha256 17649eeb2f7d1290, string read back; with the testnet lane (the resume form, B alone); it joins the 0.3.21 staging as its own commit. THE WIPE CANARY ON c19-1, c4459193 (sha 45be9b02d1b002f5, string read back): FORM END rc 0 at 13:50:53Z. Wipe synced 13:35:50Z (57 minutes, inside the 98-minute class); mining 13:36:00Z to 13:47:07Z, 66 mined, 66 accepted, 0 rejected, isSynced true at the tip throughout; the hub holds 41 of its blocks in its last 700 with 0 rejects (13:47:09Z); the restart on its kept datadir at 13:47:15Z: the old process stopped at once (the new process's first lock line seven seconds after the marker; the watchdog held nothing, the b7cc37e7 fault closed), synced again at 13:48:39Z after 84 s, 109 templates read with max 3,432 ms and 0 timeouts; the kept read on pool-1's 0.3.17 copy on the same pod passed at 13:38Z (the rewrite line once, a clean second start). The pin's set on c4459193: the digest gate PASS, the mixed-version gate PASS, the wipe canary PASS, the kept read PASS, the restart PASS, the 12 GB line proves and verifies (paid is a race, not a gate); CASES END from c20-1 (about 14:50Z) is the last pin line. THE INTEROP FACT stands from the void run: the 5899f603 hub accepted 235 object-byte-5 blocks from the 8097d600 node with 0 rejected, one digest on all five nodes on the live sixteen-field file. The gates: the digest test and the kaspa-pow vector test (the amended devnet epoch-0 id 1a4230699a6b9c60 must equal, c120d7963abdcd96 must differ, the v3 control unchanged) on the box; the mixed-version Devnet 2 gate (the amended 0.3.20 node beside a 5899f603 node for ten minutes on the live file without the v4 fields) after the Mac build; the fresh-join canary the 0.3.20 cut's | | Main's rulings (7 October, morning) | no generator change to v4 on the live devnet; the record's null is the window model with numbers, sent by the hash lane to the attack-pass lane so AP-F8-1 re-gates against it; a fault beyond the model (a low-entropy source at site 15) stops at the coordinator with the two options priced (a 0.3.19 class amendment before the flip, or the flip held at the floor), nothing shipping without the project lead's word; the tighter tail, an acceptance bound on the hot-set share, is a CLASS V5 item (sent to the v5 lane a6410f3b8abefb762 with the 64-seed census as its gate; the bound's number follows from the model) | ### AP-F4-1, the weak-day MUL draw (the attack-pass lane, 7 October, morning): PASS against v4, a class v5 rule diff --git a/docs/plans/relay-deploy-2026-10-07.md b/docs/plans/relay-deploy-2026-10-07.md new file mode 100644 index 00000000..fdebbac5 --- /dev/null +++ b/docs/plans/relay-deploy-2026-10-07.md @@ -0,0 +1,16 @@ +# Relay deploy, 7 October 2026 (the X23 tree, MF-11) + +| What | Value | +|---|---| +| Deployed | 2026-10-07 15:38 BST, from branch update-return bd9b4f4e, relay/ | +| Production deployment | https://igneum-relay-iabqarnby-igneum.vercel.app (inspect KHyscUSVKueXueqxvZBRKkaUHwJR), aliased https://relay.igneum.network | +| Previous production | https://igneum-relay-bgy767z40-igneum.vercel.app | +| Env added | RELAY_RUN_PUB (the public half of ~/.config/igneum/relay-run-key, made by `node tools/relay.mjs keygen` the same hour); RELAY_INTAKE_COMPAT unset | +| Read-backs | /wake answers; fn=machines carries the bound field; the console page answers 200 and c/machines carries poll fields; a v1-shaped register by hostname (key tier) answers named:false bound:false; a signed start-app from the Mac answers 200 (item 394); an unsigned run from master's old tool is refused | + +## Rollback + +`cd relay && npx --yes vercel@latest --global-config ~/.config/igneum/vercel --scope igneum rollback https://igneum-relay-bgy767z40-igneum.vercel.app` +(or `vercel promote https://igneum-relay-bgy767z40-igneum.vercel.app`). Seconds; nothing to undo in the database: the X23 code adds relay_machines.secret_hash and the +table relay_wake_seen, both ignored by the old code. What a rollback loses: signed run verification (the old relay never +had it), the start-app kind, the ping; PC 2's new agent registers by hostname on either. diff --git a/relay/README.md b/relay/README.md index 4fdad445..4b319e7e 100644 --- a/relay/README.md +++ b/relay/README.md @@ -22,9 +22,29 @@ Mac: `node tools/console.mjs post --kind log --title "..." --body "..."` writes The app log (label `win-` or `mac-`) reaches the intake since b8b349a (4 Oct 2026, 0.3.3); apps before that show `app ?`. The parsers (labels, miner tail, app tail, the stale mark) live in `relay/lib/parse.mjs` with no dependencies, and `relay/lib/auth.mjs` holds the constant-time secret compare; `node --test relay/test/parse.test.mjs relay/test/auth.test.mjs` runs their tests, and CI runs them in the site job. -## The secret is the path +## The secrets, since 5 October 2026 (night): three tiers, headers, no token in a URL -The web page lives at `/r//` and every API call sits under `/r//api/`. The token is 20 base32 characters generated once and stored at `~/.config/igneum/relay-token` on the Mac (and as `RELAY_TOKEN` in the project). Anyone with the link can read and post, so the link stays with the project lead. Scripts may present the log intake key in `x-igneum-key` instead (`RELAY_KEY`, the same value as `~/.config/igneum/log-intake-key`). There is no other login. Blob file URLs carry a random segment and a random suffix; they are not listed anywhere. +| Secret | Where it lives | Sent as | May | +|---|---|---|---| +| the console token (20 base32 characters) | `~/.config/igneum/relay-token`, `RELAY_TOKEN` on the project | the `x-relay-token` header from every tool and client; the URL path `/r//` only for the phone's page (`ui.html`) and the calls that page makes (`/r//api/`, `/r//c/`, `/r//wake`) | everything, except that a `run` task also needs the run signature below | +| the relay key (48 characters, the relay's own since 4 October 2026) | `~/.config/igneum/relay-key`, `RELAY_KEY` | `x-igneum-key` | read the feed, items, files, inbox, machines; post notes, files, results; register; ack; done. Never `task`, `run`, `name`, `role`, `secret`, `delete` | +| the log-intake key (inside every shipped package) | `LOG_INTAKE_KEY`, `LOG_INTAKE_KEY_NEXT` | `x-igneum-key` | only `upload` and a `drop` of kind `file` or `text`: the PC apps' build-job outputs (`jobrun.rs`). No reads, no results, nothing else. `RELAY_INTAKE_COMPAT=0` on the project closes this tier once the apps carry a relay key of their own | +| the run key (Ed25519, `node tools/relay.mjs keygen`) | `~/.config/igneum/relay-run-key` (seed, 0600) and `.pub`; the public half as `RELAY_RUN_PUB` on the project | `flags.sig` on every `run` task, over the canonical text in `relay/lib/guard.mjs` (`runCanon`: machine, nonce, body sha256, the elevated, reboot_continue and reboot flags) | without a verifying signature the relay answers 401 to a `run`; without `RELAY_RUN_PUB` every `run` is refused | +| a machine secret per PC (64 hex, `node tools/relay.mjs secret PC1`) | `~/.config/igneum/relay-machines/` on the Mac; `machine-secret.txt` beside the agent on that PC (from `make-clients.sh --machine`); its sha256 on the relay (`relay_machines.secret_hash`) | `x-machine-secret` on `register`, `result`, `done`; `flags.mac` on every `run` (an HMAC-SHA256 tag the agent verifies before it executes, because Windows PowerShell 5.1 has no Ed25519) | a result's `from` is the machine the secret proves (a mismatch is 403); a bound hostname cannot register without it; the agent runs nothing whose tag does not verify, and never the same nonce twice | + +Review round 4 (X23) found one tier with every power: the intake key inside every package queued PowerShell on PC 1 as administrator. Now the token alone cannot queue a `run` either: the Mac signs it with the run key (checked by the relay) and tags it with the target's secret (checked by the agent). Compare is constant time (`lib/auth.mjs`). Blob file URLs carry a random segment and a random suffix; they are not listed anywhere. 120 calls a minute per IP, 10 failed authentications a minute per IP, then 429. + +### Rotation (what each secret's change breaks, and the order) + +| Rotate | How | What stops, until | Order | +|---|---|---|---| +| the console token | new value in `~/.config/igneum/relay-token` and `RELAY_TOKEN`; a new zip per PC (`make-clients.sh --machine`) carried by hand; the phone gets the new link | every PC client and the phone page, until the new zip and link are in place; the Mac tools at once (they read the file) | any time; the old zip on a PC is dead the moment the project env changes | +| the relay key | new value in `relay-key` and `RELAY_KEY`; new zips | the PC clients' reads and reports until the new zip; `tools/build-job.mjs` falls back to the token | with the token, same zip | +| the intake key | `LOG_INTAKE_KEY_NEXT` on the relay AND the site intake first, repackage every app with the next key (`packaged-config.sh`), publish; then move `_NEXT` to `LOG_INTAKE_KEY`; then `RELAY_INTAKE_COMPAT=0` once no shipped app uploads build outputs with it | build-job uploads from every app that still carries the old key; log uploads from every shipped package until it updates | the long one: apps update over days, so both values are accepted during the window (`docs/plans/rotation-phase-2.md`) | +| the run key | `node tools/relay.mjs keygen` after moving the old `relay-run-key` aside; `RELAY_RUN_PUB` on the project | every `run` queued with the old key is refused at the relay; nothing on a PC changes (the PCs hold no run key) | any time, in one step | +| a machine secret | `node tools/relay.mjs secret PC1 --rotate` (rebinds the sha256), `make-clients.sh --machine PC1`, carry the zip | that PC's results and registration until the new zip is there; queued `run` tasks tagged with the old secret are refused by the agent | per machine, any time | + +What the 5 October rotation already did: the relay token (4 October), the intake key and the dl token (`.old-2026-10-05` beside the live files). What is still the project lead's: the hosted `igneum-relay-clients.zip` off the downloads host (it holds the old token and key; dead values now, but the file is the shape X23 names), `RELAY_RUN_PUB` on the project after `keygen`, one `secret` per PC and the zips carried by hand. ## What is stored where @@ -33,49 +53,62 @@ The web page lives at `/r//` and every API call sits under `/r//ap | Items (text, title, who, kind, flags, read and done marks) | Neon table `relay_items` (database `igneum`) | body 1 MB | | Machines (name, hostname, role, GPU and WSL facts, last seen) | Neon table `relay_machines` | | | Files | Vercel Blob store `igneum-relay` (public URLs with random path and suffix, London) | 50 MB per file through a client token; 4 MB when pushed through the function | -| The token and key | `~/.config/igneum/relay-token`, `~/.config/igneum/relay-key` (the relay's own key since 4 October 2026, round 4 X23; the log-intake key no longer opens the relay); project env | never in the repo | +| The token and keys | `~/.config/igneum/relay-token`, `relay-key`, `relay-run-key`, `relay-machines/`; project env (`RELAY_TOKEN`, `RELAY_KEY`, `RELAY_RUN_PUB`, `LOG_INTAKE_KEY`, `LOG_INTAKE_KEY_NEXT`) | never in the repo | +| Retention | rows older than 30 days are deleted with their blobs, checked on a feed read at most every 10 minutes per instance; `delete` removes the blob with the row (X26) | 30 days | -Kinds: `text` (a note), `file`, `task` (for a person or a Claude session on a PC), `run` (a script the agent executes), `result` (what a task produced, linked by `task_id`). Roles: `miner`, `prover`, `bench`, `mac`, `phone`. +Kinds: `text` (a note), `file`, `task` (for a person or a Claude session on a PC), `run` (a script the agent executes; signed and tagged, see above), `result` (what a task produced, linked by `task_id`; `from` is the machine the secret proves, `flags.unbound` marks one from a machine with no secret yet). Roles: `miner`, `prover`, `bench`, `mac`, `phone`. -## API (all under `/r//api/`) +## API (`/api/relay?fn=` with the headers above; `/r//api/` from the phone's page only) | Call | Does | |---|---| -| `GET feed?since=&before=&machine=&limit=` | items newest first (200 by default) plus every machine with its unread count | +| `GET feed?since=&before=&machine=&limit=` | items newest first (50 by default, 100 at most) plus every machine with its unread count and whether it is bound | | `GET item?id=` | one item with its full body | | `GET file?id=[&download=1]` | 302 to the file | -| `GET inbox?machine=PC1&kind=run|task&ack=1` | unread, not done tasks for that machine; `ack=1` marks them read | -| `GET machines` | names, roles, hostnames, last seen | -| `POST drop` | JSON `{from,to,kind,title,body,file_name,file_url,size,task_id,flags}`; or raw bytes with `Content-Type: application/octet-stream` and `x-file-name` (4 MB cap) | -| `POST task` | same fields; `kind` `task` or `run`; `run` needs one named machine and flags `{elevated, reboot_continue}` | +| `GET inbox?machine=PC1&kind=run|task` | unread, not done tasks for that machine; a GET never marks anything | +| `POST inbox {machine, kind, ack:true}` | the same list, marked read (the agents use this) | +| `GET machines` | names, roles, hostnames, last seen, bound | +| `POST drop` | JSON `{from,to,kind,title,body,file_name,file_url,size,task_id,flags}`; or raw bytes with `Content-Type: application/octet-stream` and `x-file-name` (4 MB cap). A `result` needs `x-machine-secret` when its machine is bound | +| `POST task` | same fields; `kind` `task` or `run`; `run` needs one named machine and `flags {elevated, reboot_continue, reboot, nonce, sig, mac}`; `tools/relay.mjs run` fills the last three in. 401 without a verifying `sig`, 409 on a reused nonce | | `POST upload` | `{name,size}` returns a one-hour Blob client token and `put_url`; PUT the bytes there, then `drop` with the returned `url` | -| `POST ack {ids}` `POST done {id,exit_code}` `POST delete {id}` | marks | -| `POST register {hostname,info}` | a machine checks in; returns its name, role and whether it is named | -| `POST name {hostname,name}` `POST role {name,role}` | naming and roles, from the Mac | +| `POST ack {ids}` `POST done {id,exit_code}` `POST delete {id}` | marks; `delete` takes the blob with the row (token only) | +| `POST register {hostname,info}` | a machine checks in; with `x-machine-secret` the secret names it whatever the hostname says (both PCs report DESKTOP-KMCV30N); `info.user` and `info.dir` are dropped | +| `POST secret {name, secret_hash}` | binds a machine to the sha256 of its secret (token only; `tools/relay.mjs secret` does it) | +| `POST name {hostname,name}` `POST role {name,role}` | naming and roles, from the Mac (token only) | + +Tests: `node --test relay/test/guard.test.mjs relay/test/handler.test.mjs relay/test/clients.test.mjs` (the rules, the handler against a fake database and fake blobs, the clients' shape), beside the parse, auth and wake suites; CI runs all six. Wake (`api/wake.mjs`, 0.3.6, 5 October 2026): `GET /wake?since=` is public (the apps hold no token) and rate limited, 30 a minute per IP. It holds up to 45 s and answers `{stamp, at, added, changed, held_ms}` the moment the stored stamp differs from `since`, else the unchanged stamp at the deadline; without `since` it answers at once. `POST /r//wake {stamp, added}` (or `POST /wake` with `x-relay-token` or `x-igneum-key`) records the stamp; `packaging/ota/publish-jobs.sh` sends it after every verified deploy, with the ids it added. One row per stamp in Neon table `relay_wake` (created by the first POST); `tools/jobs.mjs status` reads the rows for the woken latency. The function's `maxDuration` is 60 s (`vercel.json`). Tests: `relay/test/wake.test.mjs` drives the handler with a fake database and clock. ## Mac -`node tools/relay.mjs` (feed), `read `, `drop ""|`, `task PC2 "title" [file]`, `run PC2 "title" script.ps1 [--elevated] [--reboot-continue]`, `watch`, `inbox PC1`, `machines`, `role PC2 prover`, `name DESKTOP-XYZ PC2`, `ack|done|rm `, `url`. Playbooks live in `relay/playbooks/`; `run` fills `__DL_BASE__` in from `~/.config/igneum/dl-token`. +`node tools/relay.mjs` (feed), `read `, `drop ""|`, `task PC2 "title" [file]`, `run PC2 "title" script.ps1 [--elevated] [--reboot-continue] [--reboot]`, `keygen`, `secret PC2`, `watch`, `inbox PC1 [--ack]`, `machines`, `role PC2 prover`, `name DESKTOP-XYZ PC2`, `ack|done|rm `, `url`. Playbooks live in `relay/playbooks/`; a playbook reads the downloads base from `$env:RELAY_DL_BASE` (bash: `$RELAY_DL_BASE`), which the agent holds; `run` refuses a body that says `__DL_BASE__` or carries the dl token (X26). A script asks for a restart by printing `RELAY-REBOOT` on a line of its own, and only a task queued with `--reboot` or `--reboot-continue` restarts the PC. ## PCs -`relay/clients/make-clients.sh` bakes the URL, key and token into copies of the clients and writes `~/Desktop/igneum-relay-clients.zip`. Unzip anywhere on the PC. `send.bat` for people and Claude sessions (see `CLAUDE-PC.md`), `igneum-agent.bat` for the automatic runner: double-click once, leave it open. It registers the PC (hostname, GPUs, WSL, nvcc), polls every 20 s, runs each `run` task in order, posts a `result` (exit code, last 64 KB inline, full log as a file when longer) and marks it done. A script that prints `RELAY-REBOOT` triggers `shutdown /r /t 10`; with `reboot_continue` the agent re-arms (scheduled task at logon with highest privileges, RunOnce as a fallback) and re-runs the task after the restart with `RELAY_PASS` incremented. The PC must sign in by itself for that to be unattended. +`relay/clients/make-clients.sh --machine PC1` bakes the URL, key, token, downloads base and that PC's secret into copies of the clients and writes `~/Desktop/igneum-relay-clients-PC1.zip`. Carry it by hand; never through the downloads host. Unzip anywhere on the PC. `send.bat` for people and Claude sessions (see `CLAUDE-PC.md`), `igneum-agent.bat` for the automatic runner: double-click once, leave it open. It registers the PC (GPUs, WSL, nvcc; no username, no folder), polls every 20 s, checks each `run` task's tag and nonce against its secret (a task that fails is answered with exit 77 and nothing of it runs), runs the rest in order, posts a `result` (exit code, last 64 KB inline, full log as a file when longer) and marks it done. A script that prints `RELAY-REBOOT` on its own line, in a task queued with `--reboot` or `--reboot-continue`, triggers `shutdown /r /t 10`; with `reboot_continue` the agent arms a logon task (highest privileges, RunOnce as a fallback) for that one restart and re-runs the task after it with `RELAY_PASS` incremented. The agent removes the logon task and the RunOnce key every time it starts and when it exits (Ctrl+C included; a closed window is caught by the next start). Nothing is armed on an ordinary start (X25). The PC must sign in by itself for a restart to be unattended. An unknown hostname that registers appears in the feed with a "name this machine" box, or `node tools/relay.mjs name PC2`. PC1 is DESKTOP-KMCV30N. -## The job-channel ping (MF-11, 7 October 2026) +## The agent as a logon task, start-app, and the job-channel ping (MF-11, 7 October 2026) -PC 2 lost power at 10:46Z on 7 October 2026 and nothing said so until a person read the intake. From 0.3.21 every app wake request carries `machine=&v=&job=`; `/wake` records one row per machine in `relay_wake_seen` before it holds (`relay/lib/wake.mjs`: `recordSeen`, `seenList`, `pingState`, `PING_SILENT_S` = 900 s); the console's Machines tab reads it as "job channel polled N ago, last job X" and, after 15 minutes without a poll, "job channel silent since