diff --git a/docs/plans/counter-asic-3-status.md b/docs/plans/counter-asic-3-status.md index e90f8404e..3309a0aca 100644 --- a/docs/plans/counter-asic-3-status.md +++ b/docs/plans/counter-asic-3-status.md @@ -654,6 +654,29 @@ THE FINALITY RUNS' NOTE ON MASTER (01:22 UK, the steward, read back): box master KIT 2 NAMED (01:27 UK, the node lane): kit2b-line 5d53a591 = kit 1's tree + fix (2) (the foreign-seed capture with the own-chain rule and the step-level watchdog) + the wedge fix, no record store (2.0.3, its one-held-answer fix gating with a devnet-4 read near the tip): six suites green on build-8, the pair on build-1 and build-2 ("igneumd 2.0.2"), the canary set 01:07, a fresh node from an empty datadir at hub-1's tip in 15 minutes (01:22), a copy of build-1's dead 6,162 island healed onto hub-1's chain twice (00:36, 00:50) under the polling that wedged every earlier fast node, 0 watchdog lines; the shipper, the fleet (kit2-sha.txt) and the build-server lane with the line; the fleet's fix-2 roll arming on it with kept datadirs everywhere (every island healing by itself, what the 06:00 gate measures). For 08:00: the two kits (490ebcfe, 5d53a591), the D1 record on master 345a295bb, the native finality runs complete (4566eb1e4, 8d41e0854), the site's "forming node only" served (be0aa051), and by 06:00 the three 2.0.3 shas or their clocks (the recovery stamp and the conflict rule, the record store's held answer, P22 activation-gated) with the reset recommendation's final form after the roll's first convergence read. +KIT 2's BATCH ON MASTER (01:34 UK, the steward, read back): box master 4f8e2c1e (ci-202-kit2 e8f6a695, 92 checks, under the lock): the batch 202-5d53a591-1176efb9 (node 5d53a591 = fix (2) + the wedge fix on kit 1's tree; the tip 1176efb9 verified: the manifest and the pin at 5d53a591, the check green, no crate move since 77b5acc5): the kaspad check with igneum-pow and the five node suites GREEN on build-4 (core 185, exec 95, miner 33, p2p-flows 40, consensus 143), pow 113 and app 389 on 77b5acc5; the logs at build-1:/srv/artefacts/tas/202-5d53a591-1176efb9/; F0's pair row naming kit 2 (5d53a591, 1176efb9) the roll, kit 1 re-cut (490ebcfe, 5db25637) the clean fallback, cf32be79 the record-store fold on 2.0.3; the rule-33 row waiting on kit 2's four-block record; the steward's fifteen landings tonight in order (12b5cf42, 45b87ee8, ee3e0245, d2e1c192, c4d5df15, 14592dbd, 26203e35, b792c8cc the F0, 76041234, 9b040ab0, ff1da144, 55279204, 74898e2f, 1089028b, 4f8e2c1e); the board 190: NOT RUN 176, FAIL 7, PASS 7, 71 in progress. KIT 2's MAC CANARY (01:35, the shipper): the artefact Igneum-Miner-2.0.2-5d53a591-1176efb9.dmg (46,002,244 B, sha256 66137cd8303356da) carrying igneumd 2.0.2 commit 5d53a591, fingerprint cbc5bd0aa10585c8, igneum-app 2.0.2, 4 peers, edition public, 0 jobs or wake strings, live in both dl token folders at 01:35:42 (byte-equal by sha over HTTPS), the manifest untouched (2.0.1 urgent); the canary installed from the artefact at 01:35:43 on the mini (processes after quit 0, the datadir empty at start, started 01:35:57), the mini's kit-1 miner quit at 01:35:50 for it (one slip once: the line to main owed before the quit, not after; the push and the start chained in one block); on PASS the Mac entry publishing with --release-sha 1176efb9 and the mac block; a red by 02:10 leaving kit 1's entry (its mac block PASS) as the fallback; the mini mining again by about 02:00. THE COORDINATOR'S SEVENTY-FIRST LANDING RED ONCE (01:3x): the merge tool's gate on ff6a05510 red on "the REV suite is generated from Review B's findings and dispatch and matches them" (the registry's REV suite differing from the generator's output): the branch cut from the master before the steward's REV-affecting landings and gated without them (the race the lock does not cover for a tree cut earlier); the check green on the tree rebased onto 4f8e2c1e; re-landed. +THE SEVENTY-FIRST LANDING on master at 408f08df (01:48 UK, re-landed on the rebase). + +KIT 2's FLEET KIT, ONE SLIP (02:12 UK, the shipper): 5d53a591-node-lane.tgz due about 01:45 not on build-1 at 02:12: the build-server lane's run-202.sh running the suite gate before the cut, inside cargo test --release -p kaspa-mining under a 2,400 s timeout at load 11 to 20, the suite logs under /srv/artefacts/tas/202-5d53a591-1176efb9/ and no tarball, hive package or Windows payload; the fleet canary (lp-4090-11), the hive canary (lp-4090-43), the convergence roll and PC 2's Windows canary waiting on the file; the fleet's watcher firing the minute it lands, both canaries in parallel, the roll on the fleet PASS, the 06:00 gate on epoch seconds; the new clock: the kit's path and time from the build-server lane by 02:30, the kit not before about 02:45, the fleet and hive blocks about 45 minutes after; the Mac lane not held: the mini's canary on kit 2 synced from empty in 2,016 s on hub-1's chain (DAA 35,041 against hub-1's 35,040 at 02:10), in its 5-min mining read, the Mac entry publishing alone on its PASS; kit 1 the fallback. + +KIT 2's MAC ENTRY LIVE (02:20:37 UK, the shipper, read back): igneum-app-latest.json 2.0.2 on igneum-2.0-devnet in both dl token folders, the mac Igneum-Miner-2.0.2-5d53a591-1176efb9.dmg (66137cd8, 46,002,244 B), ui 1.0.2, min_supported_version 2.0.1, the public alias /public/igneum-miner-mac.dmg; Mac only (a Windows engine on the manifest logging "2.0.2 is published but has no windows build yet"; the hive alias unchanged at a16add6a); deployed from the Mac by r202/deploy.sh under pid 2455 (02:17:25 to 02:20:57) through the rule-33 guard: the mini's fresh-install canary PASS at 02:14:58 (synced from empty in 2,016 s on hub-1's chain, 18 blocks in 5 min at 12.65 MH/s, 0 refusals, tip 16731 against hub-1's 16722, quit in 4 s; build-1:/srv/canary/1176efb9/mini/); the record tools/ci/canary/1176efb9.json (the mac block, kit 1's 5db25637 beside it) landing on master (canary-202 5904c4b4); the publish from a tool branch publish-202 (1176efb9 plus master's canary guard; no binary from it). ONE STALL ON THE MINI, found and fixed: the canary's relaunch after its bounded quit not bringing the engine back (api/quit ending the engine at 02:14:55, the window host outliving it, the known Mac class, `open -a` fronting the dead host), the mini mining nothing for 8 min 20 s to 02:23:14; fixed by hand at 02:23 (the host ended by its recorded pid, open -a, the engine answering in 2 s); the mini synced and mining on 2.0.2 kit 2 at DAA 35,856 at 02:24:54 (hub-1 17,103 blocks); the canary script now ending the host by pid before the open and waiting for api/state; the app-side fix (the window host ending with its engine after api/quit) a 2.0.3 item for the window lane; the shipper's slip once: the second stop of the mini's miner tonight without a line to main first (its sequencing), the step line before the hand from here. Pending on the build-server lane's kit: the fleet and hive canaries and entries, PC 2's Windows canary and entry, the convergence roll, the cut line with the four tips, the Discord card; the 07:45 mini line armed. +THE KIT CUT RE-ORDERED (02:3x UK, the coordinator's line): kit 2's fleet cross kit not being cut at 02:30 (build-1 holding no tarball and running no cross build; the build-server lane spending its turn on a merge-to-master of its ship-v3-sysroot tip on the Mac since 02:28 and not answering the shipper's 02:12 ask by its 02:30 default); the shipper's ask with its default (the Mac entry alone through the night, the fleet on 2.0.1's kit, the hive and Windows entries the morning's, the 06:00 operator reset the convergence fallback); the coordinator's order: the cut outranks every landing of the build-server lane's tonight (the Mac merge stopped by pid if it blocks the cut; run-202.sh's suite gate skipped or parallel on the steward's green batch), the start time by 02:50, the path and sha when read back; only the build-server lane cutting a fleet binary under rule 2. The rule-33 records on master (fa8139bd, the host line scrubbed as 8dc522ba). +INT-07 ON KIT 2's RULE-33 RECORD (02:39 UK, the steward, read back): box master bf4334dc (ci-int07-kit2 f25e7f28, 92 checks, under the lock): INT-07 NOT RUN in progress on tools/ci/canary/1176efb9.json (landed by the shipper as fa8139bd: the mac block PASS under the check, one of four; the row PASS when the fleet, hive and windows blocks join); F0's rule-33 row recording the Mac entry published on kit 2 at 02:20 through publish-manifest's guard and kit 1's 94e4c6ba FAIL as that sha's record; two recorder classes found and fixed with self-tests: a cell's "in_progress": true dropped (only the legacy word RUNNING setting the flag; every batch in the new form tonight reading "not in progress" on master, the 202 and same-work rows taking the flag at their next replay), and a cell's or batch's note not stored in the record (fixed at 1089028b); the identity grep covering the canary records and refusing the owner's first name in a host line (the two records' host strings scrubbed, both PASS); THE BOARD: 190 cases, NOT RUN 177, FAIL 6, PASS 7, 72 in progress; the steward's sixteenth landing. + +THE PLACED 32-LANE 18-FAMILY CORE (02:40 UK, the adversary lane, read back from adv-g; ADV-01, D3 "physical implementation (placed)"): placed, the clock tree built and globally routed with parasitics estimated from the global route (the estimate calibrated against the SPEF on the 32-lane genesis core at 02:39 within 1 percent, so a placed row; the detailed router at 9,000 violations and falling, its SPEF row the morning's): 416,633 cells with six macros; 9.54 pJ per lane-op on the class v4 draw at ASAP7 (8.9 to 11.0), 6.68 at N5, 4.81 at N3; k 0.65 same node and 0.47 a node ahead at the 5090's lock; every reserve family live at 4 points 8.45 (-11 percent); on the complete GDDR7 machine 1.5x same node (1.3x to 1.7x) and 1.8x a node ahead (1.5x to 2.0x), the same as the placed 8-lane full core (9.36): the lane count buying the 18-family core nothing once its wires are in, while the genesis-only 32-lane core placed with SPEF reads 4.80 (k 0.33, the machine 2.1x) and the bank's placed cost grows with the width (+20 percent of the energy at 8 lanes, +99 percent at 32); THE ONE-LINE READING FOR THE REGISTER: a chip that carries the bank sits at 1.5x same node on the DRAM board (1.3x to 1.7x) and 1.9x to 2.1x on the stored-half hybrid, placed at both lane counts; P04 FAIL at the hybrid, at the line on the board; the delta "complete 7a9fa6148" to the landing hand with the batch adversary-20261009-placed-32lane (RUNNING, method model); adv-g up overnight at USD 0.24 an hour for the SPEF row; the adversary lane released. +THE 32-LANE DELTA ON MASTER (02:48 UK, the landing hand): 46890417 (7a9fa614; the batch adversary-20261009-placed-32lane); thirty-four landings by the hand tonight. + +THE THREE VERIFIED HUBS DEAD AND RESTARTED (02:47 to 02:49 UK, the fleet, once): lp-4090-04, -05 and -06 on hub-1's chain dead at 02:47 (-04 and -06: "thread 'virtual-processor' panicked at consensus/src/pipeline/virtual_processor/processor.rs:542:30"; -05 since 01:28 after "Finality: CONFLICTING certificate at index 686"; no OOM), found when the node lane's fresh syncs lost their peers; restarted by pid files on their kept datadirs at 02:48:43, 02:49:09 and 02:49:35, ports open, the seed guard extended to them; THE READ BEHIND IT: 213.173.107.74:16516 is hub-1's standing LIVE-devnet node (the pre-2.0 chain), not its devnet-4 node (listening on 36611 with no external map), so hub-1 was never a reachable devnet-4 peer and the dial list on its chain is the three lp hubs; the roll's and the reset's SEED lists carrying those three, the gate re-armed at 02:50 (the earlier of kit 2's roll end plus two hours and 06:00, the decision file first, the sixty-second window); the kit for 5d53a591 not landed on build-1 (the build-server lane past the 02:45 and 02:50 defaults), neither fleet canary run, nothing rolled; the fleet on 2.0.1 and seventeen islands into the gate unless the kit lands first. Two node classes for the 2.0.3 list: the virtual-processor panic at processor.rs:542 on two hubs; the node dying after a CONFLICTING certificate at index 686. +THE SHIPPER'S 02:50 DEFAULT FIRED AND A FRESH BUILD-SERVER LANE SPAWNED (02:52 to 03:0x UK): at 02:52 build-1 holding no kit for 5d53a591 and running no cross build; the standing build-server lane's transcript not advanced since 22:49 (inside one tool call; its Mac merge ended on its own; its 22:49 background landing loop pushing ship-v3-sysroot to master directly on a clean merge, bypassing the gate: a class for the record); the coordinator took the shipper's option under the night rule's new-lane clause: a fresh build-server lane on Opus with one brief (NODE_SHA=5d53a591 APP_SHA=1176efb9 bash scratchpad/app202/run-202.sh from the build-server worktree at ship-v3-sysroot 5c494c08; the served 0317 prover pair; the ISA gate and the packing guard; STAGE_DL off; the four rules, the kill guard, the canary rule, no Mac builds), the kit's path and sha to the shipper, the fleet and the coordinator when read back on build-1, a line by 04:00 if not; the stuck lane not stopped (no running build; its armed items a record source if it wakes). The mini stopped mining at 02:42 when the three lp hubs died (its sync source on hub-1's chain), 828 DAA behind with 9 peers and catching up at 02:50; the live 2.0.2 Mac entry carrying 213.173.107.74:16516 (hub-1's pre-2.0 node, never a devnet-4 peer; the handshake refusing it; the three lp hubs carrying every sync) as one of four targets; the shipper's recommendation taken: hub-1's devnet-4 node not restarted onto 26611 tonight (a consensus-level hand under the night rule), 2.0.3's dial list the morning's call. +A COORDINATOR FAULT, ONCE (02:53 to 02:58 UK): on the shipper's option the coordinator spawned a fresh build-server lane for the kit 2 cut at 02:53, a minute before main spawned its own (a6647fc718543fa73) and stopped the stuck lane; two runs of run-202.sh would collide on the worktrees and the boxes; TaskStop refusing from the coordinator, the duplicate told to stop before any build and end; main's lane the cutter, the shipper and the fleet told. THE NODE LANE's 02:55 LINE: the two hub classes named with clocks: (1) the virtual-processor panic at processor.rs:542:30 on lp-4090-04 and -06 (2.0.0 nodes on kept datadirs of hub-1's chain, no OOM), the fleet asked for the 30 log lines by 03:30, the fix on 2.0.3 with a known-failed test, 08:00 for the read-back's line and the fix by noon unless the lines show a 2.0.2 kit path; (2) lp-4090-05 dead at 01:28 after "Finality: CONFLICTING certificate at index 686" and "[igneum-exec] exec state not persisted: the file's tip 13907 is above this state's 13906": the conflict the 3.11.4 reporting class already fixed on 2.0.3 (c8d22a85: the node reporting "conflict" and locking no further), the exec-persist refusal a second class (a snapshot file ahead of the state after a reorg under the conflict), 2.0.3 with the same clocks; neither blocking kit 2's roll (5d53a591 carrying neither fix but passing every read; the seed guard restarting a dead hub by pid within thirty seconds); the fresh-sync canary's dial list the three lp hubs; the 2.0.3 line's reads (02:06 to 02:46) confounded by the dying hubs, the HEAL lane's read of the 938c3726 log the one real signal (the node's native statements disagreeing with every carried record from 03:07:14 box time, 36 veto lines, its state root and epoch-3 stream diverging, hub-1's headers failing the lottery check: a statement divergence in what 2.0.3 adds, the P22 activation the first suspect, the record store second, not fix (2)); the decisive four-node read on the restarted hubs (kit 2 the control; kit 2 + P22; + the finality fix; + the record store), each counted for vetoes and the cut crossing, verdicts about 03:15. +The duplicate lane stood down at 02:5x (reads only, no process on any box; stopped by main), leaving two faults for the cutter: the Mac's /bin/bash 3.2.57 lacking declare -A (run-202.sh step 4's ZIP/KIT[public|lab] arrays colliding on index 0) and run-202.sh writing only to /srv/artefacts/202-5d53a591-1176efb9, not /srv/workers/fleet (the fleet kit tarball needing the seed-pair step from bs0325/run-7cfa422a.sh); both sent to the cutter a6647fc718543fa73. +THE HUBS' PANIC EXPLAINED AND FIXED (02:57 UK, the fleet, once): lp-4090-04 and -06 died on "Too many open files (os error 24)" (the exec snapshot write, then RocksDB's consensus log append unwrapped at processor.rs:542:30) at the node's 8,192 open-file soft limit with 300 to 400 descriptors at rest under the fleet's inbound peers; the loop on every fleet box now raising the node to the container's hard limit before each start (read back in the start line); the three hubs restarted once more by pid files on their kept datadirs at 02:56:49, 02:57:20 and 02:57:52 with limits 524,288, 524,288 and 1,048,576, ports open, syncing to hub-1's tip; the rest of the fleet taking the raised limit at the roll's restart; the class for 2.0.3 (the unwrap at processor.rs:542 a clean refusal, the descriptor budget a start-time check). +KIT 2's CUT STARTED (02:57:45 UK, main's fresh build-server lane a6647fc718543fa73): run-202.sh pid 6630 (pid file scratchpad/app202/run-202.pid; node 5d53a591, app 1176efb9), both faults in (the six per-product arrays as plain ZIP_public/ZIP_lab variables read by indirection, bash -n green under /bin/bash 3.2; the fleet-kit step from run-7cfa422a.sh after the pairs, guarded: no lab string, 0 AVX-512 lines, the commit string present, else nothing staged, writing /srv/workers/fleet/5d53a591-node-lane.tgz plus .sha256 and reading the sha, --version, glibc and kit-isa back on build-1); the fleet kit's ETA about 03:20, the full run about 03:45; one cargo process of another lane's on build-1 before the start (pid 432074, a mixer fuzz suite: cargo test --release --test mixer --test scratch -- fuzz --nocapture), left alone. +THE MINI's STALL (03:06 UK, the shipper, once): the mini (2.0.2 kit 2) at DAA 36,567 since 02:50 with 8 peers, synced false, the card waiting (hub-1's feed at 38,314 at 03:06); its node log: a peer rejecting its IBD at 02:32:50 with "this node's executor did not reach the class v5 epoch's seed block within 300 s", "Sync rate 0.49 is below threshold 0.5" at 02:37, and from 02:42 (its last accepted block) processing blocks only at its own tip region while connected to the public seeds 64.119.209.250:21703 and 69.145.85.93:17873 among its 8 (the address book bringing them in past the dial list); the cause at the join: the three lp hubs dying 02:28 to 02:47 and restarting on kept datadirs, the mini's resync from them tripping the executor window on the M6 (the proof-laden epochs pacing at about 250 DAA a minute there); the node lane's 2.0.3 class (the IBD executor window against a slow executor; the log on the mini at ~/Library/Logs/Igneum/node-20261009-012315.log); the coordinator's word: one restart of the app at 03:20 if still not synced (the step line before the hand), a home machine's app restart not a consensus hand, a second stall the recorded one for the morning. +The mini's stall cleared on its own at 03:08:01 UK (DAA 38,532, synced, mining on 2.0.2 kit 2 with 8 peers once the three lp hubs were back at the tip); 18 minutes without mining (02:42 to 03:08) recorded; the restart withdrawn, no hand on the mini. + +THE FLEET KIT CUT (03:18 UK, the build-server lane a6647fc718543fa73 under rule 2; read back from build-1 by the coordinator at 03:19 and 03:20 UK and by the shipper at 03:19): /srv/workers/fleet/5d53a591-node-lane.tgz, sha256 8fb8c8f52c27670fef988ff15ffd11c47b9024d62e0dab0bb4e0412974ed1012 equal to its sidecar, 26,973,076 B; inside, igneumd fbf1d6e53021febd printing "igneumd 2.0.2" with the commit string 5d53a591 (glibc 2.34, kit-isa 0 AVX-512 lines on the box gate) and igneum-miner 66fded0f5ce15bde (kit-isa 0, glibc 2.34), the freeze fingerprint cbc5bd0aa10585c8 read back in both; the served 0317 prover pair inside. One guard fix, said once by the cutter: attempt 1 (pid 6630) went red at 03:12 on the script's check that the miner carries the commit string, which no miner ever has (the commit string is igneumd's); the check is igneumd-only now and attempt 2 (pid 23908) rebuilt from the warm cache. The hive package, the Windows payload and the installer kit follow from the cutter under /srv/artefacts/202-5d53a591-1176efb9 by about 03:45 UK; on them the shipper's push to the dl folders, the hive canary on lp-4090-43, PC 2's Setup and canary through the relay lane (04:15 and 05:00). The fleet lane's watcher fires on the file: the lp-4090-11 fleet canary, then the one convergence roll on kept datadirs, the gate as armed (the earlier of two hours after the roll ends and 06:00 UK). hub-1's tip at 03:20 UK: 19,520 at DAA 39,154 (the reference feed on build-1); the mini mining on kit 2 at hub-1's tip. + A SHARED-DEVNET FACT FROM THE FLEET (not this lane's, with the shipper and the infra lane): the Hetzner live seed 188.245.5.161:26611 is still on the old override object (digest eada4bda) 1 h 40 min after the 0.3.20 sweep (the fleet never touches Hetzner nodes, so it was outside the sweep); the 0.3.21 wipe canary c22-1 took five digest-mismatch rejects from it; an app with the packaged peers is refused at the seed and syncs through node1 and the hub only, a fresh joiner with only the seed cannot join, the 14 voters and the hub are unaffected; the owner puts the floor file ov16-floor-900000.json (sha 294f1f80) and the c4459193 pin on it. 0.3.21's STAGING (the node lane): the order dry-merges onto 55768f88 with nothing moving to 0.3.22; the late-join fix is 52e96c94 (70e4601e rebased onto 55768f88, exec suite 33 green with both new tests); f067f7c1, b0444f51 and 437f0438 merge clean in order; 2e32d5f6's one conflict (DST_ADDRESS beside pool-finish's DST_BINDING in consensus/core/src/finality.rs) kept both; the live-file digest eada4bda after each (every switch at never); the staging waits on the shipper's sweep-end word; the re-pin held. PC 2 DOWN AGAIN (main, 16:5x UK): the founder takes PC 2 down for cable work (PC 1 back but his desk); both PCs out of the sweep's waves, each updates on its poller on return; no PC job to PC 1; the Windows G1 completed before the outage, nothing reruns. 0.3.21's SECOND GATE LINE on 55768f88 (sha256 279b1b690e854fc9): the ten-minute mixed-version gate beside the 5899f603 pair, 13:37:40Z to 13:47:52Z, SUMMARY PASS (one digest b0afb2ee on five nodes; 223 new and 381 old blocks accepted by the old hub, 0 rejected; counts equal at 319, 486 and 604 through both clean joins and the restart step at 13:45:22Z; no panic); the node lane's two lines on 0.3.21's first candidate complete, in plan 6.9 on ca3-v4-node; the fleet's set on it (the bare-child 12 GB line, the wipe, the kept read, the cases) is the fleet's. 0.3.21's FIRST GATE LINE on 55768f88 (sha256 279b1b690e854fc9, the string read back; pairing igneum-pow 8c728ca3 at byte 5): the digest gate 13:35:41Z to 13:37:19Z SUMMARY PASS (a89be8a7 on both binaries with the peers; db9a85f9 refused, no peer; the live file's eada4bda unmoved); the ten-minute mixed-version gate from 13:37:40Z, line about 13:50Z. The 0.3.21 order as the shipper sent it: 55768f88; f067f7c1 and 70e4601e; b0444f51; 6eb21fc9; db28d331; then the re-pin from 8bdcbdd8 on the coordinator's word; suites between, the digest read after every one; the mirror's release-0.3.20-node back at the pin c4459193, release-0.3.21-node open at 55768f88. THE LATE-JOIN COMMIT (N9's second half, the node lane): 70e4601e on the box mirror as branch proof-hold-fix, from c4459193, two files (igneum/exec/src/proving.rs, protocol/flows/src/v10/proving.rs); the gap was the fetch side on the joiner (the served record ran the native check against the joiner's trailing exec state before anything was stored, the check refused it, the proof was never held, the body rule read "not held" for 20 s and failed the IBD); the fix holds the proof by hash before the checks (the pool entry still needs them) and the serve side says when it holds fewer than asked; the exec suite 32 passed at 13:26Z with the known-failed shape first, the flows check green 13:28Z, igneumd on build-1 at the 0321 worktree path built 13:32Z, sha256 17649eeb2f7d1290, string read back; with the testnet lane (the resume form, B alone); it joins the 0.3.21 staging as its own commit. THE WIPE CANARY ON c19-1, c4459193 (sha 45be9b02d1b002f5, string read back): FORM END rc 0 at 13:50:53Z. Wipe synced 13:35:50Z (57 minutes, inside the 98-minute class); mining 13:36:00Z to 13:47:07Z, 66 mined, 66 accepted, 0 rejected, isSynced true at the tip throughout; the hub holds 41 of its blocks in its last 700 with 0 rejects (13:47:09Z); the restart on its kept datadir at 13:47:15Z: the old process stopped at once (the new process's first lock line seven seconds after the marker; the watchdog held nothing, the b7cc37e7 fault closed), synced again at 13:48:39Z after 84 s, 109 templates read with max 3,432 ms and 0 timeouts; the kept read on pool-1's 0.3.17 copy on the same pod passed at 13:38Z (the rewrite line once, a clean second start). The pin's set on c4459193: the digest gate PASS, the mixed-version gate PASS, the wipe canary PASS, the kept read PASS, the restart PASS, the 12 GB line proves and verifies (paid is a race, not a gate); CASES END from c20-1 (about 14:50Z) is the last pin line. THE INTEROP FACT stands from the void run: the 5899f603 hub accepted 235 object-byte-5 blocks from the 8097d600 node with 0 rejected, one digest on all five nodes on the live sixteen-field file. The gates: the digest test and the kaspa-pow vector test (the amended devnet epoch-0 id 1a4230699a6b9c60 must equal, c120d7963abdcd96 must differ, the v3 control unchanged) on the box; the mixed-version Devnet 2 gate (the amended 0.3.20 node beside a 5899f603 node for ten minutes on the live file without the v4 fields) after the Mac build; the fresh-join canary the 0.3.20 cut's | | Main's rulings (7 October, morning) | no generator change to v4 on the live devnet; the record's null is the window model with numbers, sent by the hash lane to the attack-pass lane so AP-F8-1 re-gates against it; a fault beyond the model (a low-entropy source at site 15) stops at the coordinator with the two options priced (a 0.3.19 class amendment before the flip, or the flip held at the floor), nothing shipping without the founder's word; the tighter tail, an acceptance bound on the hot-set share, is a CLASS V5 item (sent to the v5 lane a6410f3b8abefb762 with the 64-seed census as its gate; the bound's number follows from the model) |