igneum-network/igneum
1
0
Fork 0
Code Issues Pull requests Releases Activity

fast-time crossing: the first cross run's summary (19:22:01Z: rung 1 at epoch 6, v5 at epoch 8 at rung 1, every id equal to the CPU verifier's on 13 of 13 rows, byte 6 at 9,985 bps; the restart step's IBD refusal and fork) and five harness-scope corrections it showed: a fork row (a seed that is not n0's template seed) is listed and never counted; the step line is logged twice per step on the 0.3.23 line, so a second step is a new (epoch, rung) pair; a submit RPC timeout is counted apart from a PoW refusal; the stale node's accepted block on an old template after the refresh is the previous epoch's; the miners expected on a row allow for the restarted node

Browse source 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
igneum-labs 2026-10-07 19:24:24 +00:00
parent 17e2a2e1ed
commit 5e82aebafc
3 changed files with 1775 additions and 12 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1630
docs/design/class-v5-harness/fasttime/cross.json Normal file
View file

File diff suppressed because it is too large Load diff

118
docs/design/class-v5-harness/fasttime/cross.log Normal file
View file

@ -0,0 +1,118 @@
19:04:22.115 override: 4 key(s) the binary does not know dropped (sig_scheme, sig_scheme_activation_daa, finality_succession_activation_daa, latency_ladder_cache_rung)
19:04:22.116 case cross: 4 nodes, bytes 6/6/6/6, ladder up/up/up/up; class v4 sub-version 3 from genesis at rung 0; class window 60 DAA x 7 (the first epoch that can cross to v5 is 8, DAA 480), ladder window 45 DAA x 7 (the first epoch that can step is 6, DAA 360); v5 floor 900 (epoch 15); stale node 3; restart node 2 (stop at DAA 455, start at 500); 60 DAA per epoch, lead 10; run 960 s or 11 epochs; node /srv/builds/igneum-wt-v5-fasttime/vendor/igneum-node-v5-fasttime/target/release/igneumd
19:04:27.082 CPU verifier on the Devnet 3 genesis pack: cli e5a4ac5978462156 pack e5a4ac5978462156 expected e5a4ac5978462156: OK
19:04:28.641 n0 up pid 1119100 json 29862 p2p 29861 exec 29863, byte 6, ladder up
19:04:30.155 n1 up pid 1119363 json 29872 p2p 29871 exec 29873, byte 6, ladder up
19:04:31.664 n2 up pid 1119613 json 29882 p2p 29881 exec 29883, byte 6, ladder up
19:04:33.169 n3 up pid 1119913 json 29892 p2p 29891 exec 29893, byte 6, ladder up
19:04:33.170 n0: Program class v5 from the override file: enabled, the floor at epoch 15 (DAA score 900 rounded up to the epoch boundary at 900); the class v | this node signals object version 6 | this node signals up
19:04:33.170 n1: Program class v5 from the override file: enabled, the floor at epoch 15 (DAA score 900 rounded up to the epoch boundary at 900); the class v | this node signals object version 6 | this node signals up
19:04:33.170 n2: Program class v5 from the override file: enabled, the floor at epoch 15 (DAA score 900 rounded up to the epoch boundary at 900); the class v | this node signals object version 6 | this node signals up
19:04:33.170 n3: Program class v5 from the override file: enabled, the floor at epoch 15 (DAA score 900 rounded up to the epoch boundary at 900); the class v | this node signals object version 6 | this node signals up
19:04:33.170 n0 digest: 6fa8e71d7fb9aa0d
19:04:34.187 epoch -1 -> 0 at daa 0, 7.1 s: template class 3 (next 3) rung 0 (27 passes, next rung 0), v5 share at the sink 0 bps (weakest of 7: 0), ladder up weakest 0 bps, this node signals byte 6, v5 by signal at epoch none, step at epoch 0
19:04:34.220 t=7.1 s daa 0 epoch 0 class 3 rung 0 v5 0 bps (weakest 0) up weakest 0 blocks/sink per node 0/234e082d 0/234e082d 0/234e082d 0/234e082d
19:04:49.304 t=22.2 s daa 0 epoch 0 class 3 rung 0 v5 0 bps (weakest 0) up weakest 0 blocks/sink per node 0/234e082d 0/234e082d 0/234e082d 0/234e082d
19:05:04.453 t=37.4 s daa 2 epoch 0 class 3 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 2/72937082 2/72937082 2/72937082 2/72937082
19:05:19.546 t=52.5 s daa 11 epoch 0 class 3 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 11/c177d283 11/c177d283 11/c177d283 11/c177d283
19:05:34.593 t=67.5 s daa 15 epoch 0 class 3 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 15/90704f1f 15/90704f1f 15/90704f1f 15/90704f1f
19:05:49.627 t=82.5 s daa 30 epoch 0 class 3 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 30/7b78ee46 30/7b78ee46 30/7b78ee46 30/7b78ee46
19:06:04.694 t=97.6 s daa 48 epoch 0 class 3 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 48/5a9eb898 48/5a9eb898 48/5a9eb898 48/5a9eb898
19:06:19.740 t=112.7 s daa 58 epoch 0 class 3 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 58/20e0b73b 58/20e0b73b 58/20e0b73b 58/20e0b73b
19:06:20.744 epoch 0 -> 1 at daa 60, 113.7 s: template class 3 (next 3) rung 0 (27 passes, next rung 0), v5 share at the sink 10000 bps (weakest of 7: 0), ladder up weakest 0 bps, this node signals byte 6, v5 by signal at epoch none, step at epoch 0
19:06:22.752 stream for epoch 1 (seed 3b9a15224d8a9b0c): root 0x2e1a306737b355d2 17 records, from n0
19:06:35.245 t=128.2 s daa 64 epoch 1 class 3 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 64/c37c9383 64/c37c9383 64/c37c9383 64/c37c9383
19:06:50.659 t=143.6 s daa 65 epoch 1 class 3 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 65/a6316dba 65/a6316dba 65/a6316dba 65/a6316dba
19:07:06.009 t=158.9 s daa 68 epoch 1 class 3 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 68/36d6c468 68/36d6c468 68/36d6c468 68/36d6c468
19:07:22.025 t=174.9 s daa 68 epoch 1 class 3 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 68/36d6c468 68/36d6c468 68/36d6c468 68/36d6c468
19:07:37.512 t=190.4 s daa 69 epoch 1 class 3 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 69/72c5d44c 69/72c5d44c 69/72c5d44c 69/72c5d44c
19:07:53.000 t=205.9 s daa 80 epoch 1 class 3 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 80/ccd6c335 80/ccd6c335 80/ccd6c335 80/ccd6c335
19:08:08.048 t=221.0 s daa 108 epoch 1 class 3 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 108/a743a62a 108/a743a62a 108/a743a62a 108/a743a62a
19:08:15.067 epoch 1 -> 2 at daa 121, 228.0 s: template class 4 (next 4) rung 0 (27 passes, next rung 0), v5 share at the sink 10000 bps (weakest of 7: 0), ladder up weakest 0 bps, this node signals byte 6, v5 by signal at epoch none, step at epoch 0
19:08:18.074 stream for epoch 2 (seed 0eccc013f9b60e0d): root 0xc0a3a18d4d925353 17 records, from n0
19:08:23.090 t=236.0 s daa 121 epoch 2 class 4 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 121/e3094893 121/e3094893 121/e3094893 121/e3094893
19:08:38.152 t=251.1 s daa 143 epoch 2 class 4 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 143/cc1ed2db 143/cc1ed2db 143/cc1ed2db 143/cc1ed2db
19:08:53.244 t=266.2 s daa 175 epoch 2 class 4 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 175/3d87f664 175/3d87f664 175/3d87f664 175/3d87f664
19:08:57.255 epoch 2 -> 3 at daa 180, 270.2 s: template class 4 (next 4) rung 0 (27 passes, next rung 0), v5 share at the sink 10000 bps (weakest of 7: 0), ladder up weakest 0 bps, this node signals byte 6, v5 by signal at epoch none, step at epoch 0
19:08:58.261 stream for epoch 3 (seed 2b01fdd54ea150f5): root 0x362eac479bfe7ebb 17 records, from n0
19:09:08.309 t=281.2 s daa 183 epoch 3 class 4 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 183/f24d5245 183/f24d5245 183/f24d5245 183/f24d5245
19:09:23.353 t=296.3 s daa 212 epoch 3 class 4 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 212/4bb4ea6d 212/4bb4ea6d 212/4bb4ea6d 212/4bb4ea6d
19:09:38.399 t=311.3 s daa 228 epoch 3 class 4 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 228/c9e5ca83 228/c9e5ca83 228/c9e5ca83 228/c9e5ca83
19:09:49.423 epoch 3 -> 4 at daa 241, 322.3 s: template class 4 (next 4) rung 0 (27 passes, next rung 0), v5 share at the sink 10000 bps (weakest of 7: 0), ladder up weakest 0 bps, this node signals byte 6, v5 by signal at epoch none, step at epoch 0
19:09:50.429 stream for epoch 4 (seed 2b65b0e1ac694375): root 0xcf8c29a8a71c192e 17 records, from n0
19:09:53.437 t=326.4 s daa 241 epoch 4 class 4 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 241/af0370c7 241/af0370c7 241/af0370c7 241/af0370c7
19:10:08.476 t=341.4 s daa 255 epoch 4 class 4 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 255/f68e301c 255/f68e301c 255/f68e301c 255/f68e301c
19:10:23.530 t=356.4 s daa 269 epoch 4 class 4 rung 0 v5 10000 bps (weakest 0) up weakest 0 blocks/sink per node 269/0a0af1a2 269/0a0af1a2 269/0a0af1a2 269/0a0af1a2
19:10:38.578 t=371.5 s daa 284 epoch 4 class 4 rung 0 v5 10000 bps (weakest 0) up weakest 10000 blocks/sink per node 284/32d217a5 284/32d217a5 284/32d217a5 284/32d217a5
19:10:53.637 t=386.6 s daa 298 epoch 4 class 4 rung 0 v5 10000 bps (weakest 0) up weakest 10000 blocks/sink per node 298/3da5a782 297/7968d7f4 297/7968d7f4 298/3da5a782
19:10:57.646 epoch 4 -> 5 at daa 300, 390.6 s: template class 4 (next 4) rung 0 (27 passes, next rung 0), v5 share at the sink 10000 bps (weakest of 7: 0), ladder up weakest 10000 bps, this node signals byte 6, v5 by signal at epoch none, step at epoch 0
19:10:58.655 stream for epoch 5 (seed 56e5ee99d68c7f63): root 0x89fe68cf51ddd005 17 records, from n0
19:11:08.684 t=401.6 s daa 302 epoch 5 class 4 rung 0 v5 10000 bps (weakest 0) up weakest 10000 blocks/sink per node 302/e2f27369 302/e2f27369 302/e2f27369 308/298bbcae
19:11:23.730 t=416.6 s daa 324 epoch 5 class 4 rung 0 v5 10000 bps (weakest 0) up weakest 10000 blocks/sink per node 324/833cfdb5 324/833cfdb5 324/833cfdb5 324/833cfdb5
19:11:38.786 t=431.7 s daa 337 epoch 5 class 4 rung 0 v5 10000 bps (weakest 0) up weakest 10000 blocks/sink per node 337/ebd1a71d 337/ebd1a71d 337/ebd1a71d 337/ebd1a71d
19:11:53.825 epoch 5 -> 6 at daa 360, 446.7 s: template class 4 (next 4) rung 1 (35 passes, next rung 1), v5 share at the sink 10000 bps (weakest of 7: 0), ladder up weakest 10000 bps, this node signals byte 6, v5 by signal at epoch none, step at epoch 6
19:11:53.825 LADDER STEP: the template is rung 1 (35 shadow passes) from epoch 6 (daa 360) at 446.7 s wall
19:11:53.826 t=446.7 s daa 360 epoch 6 class 4 rung 1 v5 10000 bps (weakest 0) up weakest 10000 blocks/sink per node 360/8aa8dd99 360/8aa8dd99 360/8aa8dd99 360/8aa8dd99
19:11:54.833 stream for epoch 6 (seed 2351bd291bb1ae36): root 0xc78294404f032d42 17 records, from n0
19:12:08.878 t=461.8 s daa 361 epoch 6 class 4 rung 1 v5 10000 bps (weakest 0) up weakest 10000 blocks/sink per node 361/8aa8dd99 361/8aa8dd99 361/8aa8dd99 361/8aa8dd99
19:12:24.016 t=476.9 s daa 371 epoch 6 class 4 rung 1 v5 10000 bps (weakest 0) up weakest 10000 blocks/sink per node 371/aa32fdd9 371/aa32fdd9 371/aa32fdd9 366/d431f65a
19:12:39.078 t=492.0 s daa 394 epoch 6 class 4 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 394/3f9f5fac 394/3f9f5fac 394/3f9f5fac 394/3f9f5fac
19:12:54.126 t=507.0 s daa 415 epoch 6 class 4 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 415/0e86785b 415/0e86785b 415/0e86785b 415/0e86785b
19:13:01.143 epoch 6 -> 7 at daa 420, 514.1 s: template class 4 (next 4) rung 1 (35 passes, next rung 1), v5 share at the sink 10000 bps (weakest of 7: 10000), ladder up weakest 10000 bps, this node signals byte 6, v5 by signal at epoch none, step at epoch 6
19:13:03.154 stream for epoch 7 (seed 3c52e672c1378f2a): root 0x22531991b734ea59 17 records, from n0
19:13:09.166 t=522.1 s daa 423 epoch 7 class 4 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 423/d36c2e38 423/d36c2e38 423/d36c2e38 423/d36c2e38
19:13:24.210 t=537.1 s daa 449 epoch 7 class 4 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 449/8ba9cb86 449/8ba9cb86 449/8ba9cb86 449/8ba9cb86
19:13:30.232 RESTART STEP: n2 and its miner stopped at daa 456 (543.1 s), 24 DAA before the expected v5 boundary 480
19:13:39.264 t=552.2 s daa 470 epoch 7 class 4 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 470/69d3ab7f 470/69d3ab7f down 470/69d3ab7f
19:13:49.309 epoch 7 -> 8 at daa 481, 562.2 s: template class 5 (next 5) rung 1 (35 passes, next rung 1), v5 share at the sink 10000 bps (weakest of 7: 10000), ladder up weakest 10000 bps, this node signals byte 6, v5 by signal at epoch 8, step at epoch 6
19:13:49.309 CLASS SWITCH: the template is class v5 from epoch 8 (daa 481) at rung 1 (35 passes), 562.2 s wall
19:13:51.320 stream for epoch 8 (seed 69d3ab7f1e01a481): root 0x22bc70bdb33afe63 17 records, from n0
19:13:54.324 t=567.2 s daa 481 epoch 8 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 481/37947df8 481/37947df8 down 481/37947df8
19:14:09.378 t=582.3 s daa 482 epoch 8 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 482/3d87fc70 481/37947df8 down 481/37947df8
19:14:25.976 n2 RESTARTED pid 1156550 json 29882 p2p 29881 exec 29883, byte 6, ladder up (own datadir kept)
19:14:25.981 RESTART STEP: n2 started on its own datadir at daa 500 (598.9 s), 20 DAA after the boundary; its miner restarted with its key
19:14:25.983 t=598.9 s daa 500 epoch 8 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 503/fb374a55 503/fb374a55 456/32ee058c 503/fb374a55
19:14:41.057 t=614.0 s daa 516 epoch 8 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 516/5f40e6e0 516/5f40e6e0 456/32ee058c 516/5f40e6e0
19:14:56.142 t=629.1 s daa 533 epoch 8 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 533/6fdd7334 533/6fdd7334 462/03506bf8 533/6fdd7334
19:15:00.149 epoch 8 -> 9 at daa 540, 633.1 s: template class 5 (next 5) rung 1 (35 passes, next rung 1), v5 share at the sink 10000 bps (weakest of 7: 10000), ladder up weakest 10000 bps, this node signals byte 6, v5 by signal at epoch none, step at epoch 6
19:15:00.154 stream for epoch 9 (seed 8982354512f2f9e7): root 0x7726cb57697c6d7b 17 records, from n0
19:15:11.214 t=644.1 s daa 540 epoch 9 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 540/56291e9f 540/56291e9f 470/8d4c5095 540/56291e9f
19:15:26.336 t=659.3 s daa 543 epoch 9 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 543/8f9ad85c 543/8f9ad85c 480/c9f7bf3b 543/8f9ad85c
19:15:41.431 t=674.3 s daa 551 epoch 9 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 551/076ad538 551/076ad538 485/a0f4a2e6 551/076ad538
19:15:56.486 t=689.4 s daa 569 epoch 9 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 569/3d1706a1 569/3d1706a1 501/c4299c77 569/3d1706a1
19:16:11.572 t=704.5 s daa 587 epoch 9 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 587/e2c372d7 587/e2c372d7 518/50ba5302 587/e2c372d7
19:16:25.655 epoch 9 -> 10 at daa 600, 718.6 s: template class 5 (next 5) rung 1 (35 passes, next rung 1), v5 share at the sink 10000 bps (weakest of 7: 10000), ladder up weakest 10000 bps, this node signals byte 6, v5 by signal at epoch none, step at epoch 6
19:16:26.658 t=719.6 s daa 600 epoch 10 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 600/04e5328b 600/04e5328b 535/66897093 600/04e5328b
19:16:28.673 stream for epoch 10 (seed 76ba0aa9f559b0d5): root 0x0f0f2b27619ae20c 17 records, from n0
19:16:41.745 t=734.7 s daa 602 epoch 10 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 602/cbed939a 601/3d48d6a4 540/790011b0 602/cbed939a
19:16:57.039 t=750.0 s daa 616 epoch 10 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 616/d1a8f4b5 616/d1a8f4b5 546/0d490be1 616/d1a8f4b5
19:17:13.099 t=766.0 s daa 616 epoch 10 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 616/d1a8f4b5 616/d1a8f4b5 548/3dcabef7 616/d1a8f4b5
19:17:28.626 t=781.5 s daa 621 epoch 10 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 621/576085f1 621/576085f1 556/18734417 621/576085f1
19:17:44.266 t=797.2 s daa 626 epoch 10 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 626/67fe2511 626/67fe2511 559/39c4eeba 626/67fe2511
19:17:59.945 t=812.9 s daa 626 epoch 10 class 5 rung 1 v5 10000 bps (weakest 10000) up weakest 10000 blocks/sink per node 626/67fe2511 626/67fe2511 560/9f7a47cd 626/67fe2511
19:18:12.697 epoch 10 -> 11 at daa 660, 825.6 s: template class 5 (next 5) rung 1 (35 passes, next rung 1), v5 share at the sink 10000 bps (weakest of 7: 10000), ladder up weakest 10000 bps, this node signals byte 6, v5 by signal at epoch none, step at epoch 6
19:18:15.713 stream for epoch 11 (seed 500aef809809c6e5): root 0xcddb9bd28d420dfc 17 records, from n0
19:22:01.404 SUMMARY FAIL (cross) 2026-10-07T19:22:01.403Z: rung 1 (35 passes) from epoch 6 at DAA 360; v5 from epoch 8 at DAA 481 at rung 1; epochs e0:v3:r0:0bps e1:v3:r0:0bps e2:v4:r0:0bps e3:v4:r0:0bps e4:v4:r0:0bps e5:v4:r0:0bps e6:v4:r1:0bps e7:v4:r1:10000bps e8:v5:r1:10000bps e9:v5:r1:10000bps e10:v5:r1:10000bps e11:v5:r1:10000bps; chain bytes {"0":1,"6":660} (9985 bps at byte 6), ladder bits {"none":1,"up":660}; blocks 482 / 179 across DAA 480; rejected miners 2/1/2/91 nodes 0/0/0/89; accepted after the flip 141/119/242/26; sinks b9c9bba58d8c6936 b9c9bba58d8c6936 2ba9d42f8fab0328 b9c9bba58d8c6936 (DIFFER) at 660/660/599/660; step lines 4/4 (epochs 6/6/6/6), v5 signal lines 4/4 (epochs 8/8/8/8, weakest 10000/10000/10000/10000); stale n3 accepted after the first refresh 1 rejected 89; restart n2 stopped at DAA 456, started at DAA 500, lock faults 0, accepted after 242; dn3 id OK
19:22:01.404 PROGRAM ID epoch 0 v3 rung 0 (27 passes) seed 234e082d653dc69d: miners 736e1544a578165e (4 of 4) cli 83dab9f31f2e2dba other-class 343287b6df5996ed MISMATCH
19:22:01.404 PROGRAM ID epoch 1 v3 rung 0 (27 passes) seed 3b9a15224d8a9b0c: miners 30dbef8825021877 (4 of 4) cli aba29f8ade88d44f other-class c37261923a60173e state 0x2e1a306737b355d2 (17 records) MISMATCH
19:22:01.404 PROGRAM ID epoch 2 v4 rung 0 (27 passes) seed 0eccc013f9b60e0d: miners ea3daa803376e580 (4 of 4) cli ea3daa803376e580 other-class 309132d8ccafd62b state 0xc0a3a18d4d925353 (17 records) OK
19:22:01.404 PROGRAM ID epoch 3 v4 rung 0 (27 passes) seed 2b01fdd54ea150f5: miners d348741f15db4166 (4 of 4) cli d348741f15db4166 other-class 35539da9b3f03635 state 0x362eac479bfe7ebb (17 records) OK
19:22:01.404 PROGRAM ID epoch 4 v4 rung 0 (27 passes) seed 2b65b0e1ac694375: miners 9771fc152d30f39b (4 of 4) cli 9771fc152d30f39b other-class ff4e2d35d8848012 state 0xcf8c29a8a71c192e (17 records) OK
19:22:01.404 PROGRAM ID epoch 5 v4 rung 0 (27 passes) seed 56e5ee99d68c7f63: miners 9b149ff74ac77a2c (4 of 4) cli 9b149ff74ac77a2c other-class 7c0db63ee8a1815f state 0x89fe68cf51ddd005 (17 records) OK
19:22:01.404 PROGRAM ID epoch 6 v4 rung 1 (35 passes) seed 2351bd291bb1ae36: miners f78d859523e585dd (4 of 4) cli f78d859523e585dd other-class 62f42228b38e6016 state 0xc78294404f032d42 (17 records) OK
19:22:01.404 PROGRAM ID epoch 7 v4 rung 1 (35 passes) seed 3c52e672c1378f2a: miners 77a05541bd6d2408 (4 of 4) cli 77a05541bd6d2408 other-class 163a039c8517b6e5 state 0x22531991b734ea59 (17 records) OK
19:22:01.404 PROGRAM ID epoch 8 v5 rung 1 (35 passes) seed 69d3ab7f1e01a481: miners bda8cd253ab68bcc (3 of 4) cli bda8cd253ab68bcc other-class bdafffa1c8373f09 rung0 97e0320e0d2c4d43 state 0x22bc70bdb33afe63 (17 records) OK
19:22:01.404 PROGRAM ID epoch 8 v5 rung 1 (35 passes) seed 8d4c5095f1e287d3: miners 449c4509e2b5b161 (1 of 4) cli 449c4509e2b5b161 other-class ab38c4b0d956b900 rung0 69f43cfa5de1d13c state 0x22bc70bdb33afe63 (17 records) OK
19:22:01.404 PROGRAM ID epoch 9 v5 rung 1 (35 passes) seed 8982354512f2f9e7: miners 655700d99fc2964a (3 of 4) cli 655700d99fc2964a other-class 0727788cba489ea7 rung0 6cd9b04c788be5d1 state 0x7726cb57697c6d7b (17 records) OK
19:22:01.404 PROGRAM ID epoch 9 v5 rung 1 (35 passes) seed 45c2631258e49c3d: miners 60de85f2f257efdf (1 of 4) cli 60de85f2f257efdf other-class 2ea8a834a1b97e36 rung0 ecc28a774de5c0aa state 0x7726cb57697c6d7b (17 records) OK
19:22:01.404 PROGRAM ID epoch 10 v5 rung 1 (35 passes) seed 76ba0aa9f559b0d5: miners 0d6222e5fcf147e4 (3 of 4) cli 0d6222e5fcf147e4 other-class a772e50dadf83b61 rung0 fce005e5294345eb state 0x0f0f2b27619ae20c (17 records) OK
19:22:01.404 FAILED CHECK started_on_class_v4_at_rung_0
19:22:01.404 FAILED CHECK no_second_step
19:22:01.404 FAILED CHECK v5_ids_equal_the_cli_v5_id
19:22:01.404 FAILED CHECK a_v5_epoch_per_window_refresh
19:22:01.404 FAILED CHECK zero_rejected_by_honest_miners
19:22:01.404 FAILED CHECK sinks_agree
19:22:01.404 FAILED CHECK block_counts_agree
19:22:01.404 FAILED CHECK stale_node_refused_from_the_first_refresh
19:22:01.404 FAILED CHECK restarted_node_resynced
19:22:01.404 summary: /tmp/igneum-fast-time-v5x-cross/summary.json and /srv/builds/igneum-wt-v5-fasttime/docs/design/class-v5-harness/fasttime/cross.json

39
infra/fast-time/class-v5-crossing.mjs
View file

@ -181,7 +181,7 @@ process.on('unhandledRejection', async (e) => { log(`FAILED: ${e?.stack || e}`);
const minerLog = (i) => { try { return readFileSync(`${TMP}/cpu${i}.log`, 'utf8').split('\n'); } catch { return []; } };
// the miner stamps every line with epoch seconds (`1791374997.353 ACCEPTED block ...`)
const lineWall = (l) => { const m = /^(\d{10}\.\d+) /.exec(l); return m ? +m[1] * 1000 : null; };
const acceptedAfter = (i, wallMs) => wallMs == null ? null : minerLog(i).filter(l => /ACCEPTED block/.test(l) && (lineWall(l) ?? 0) >= wallMs).length;
const acceptedAfter = (i, wallMs, freshOnly = false) => wallMs == null ? null : minerLog(i).filter(l => /ACCEPTED block/.test(l) && (lineWall(l) ?? 0) >= wallMs && !(freshOnly && /gens old/.test(l))).length;
const STEP_LINE = /Latency ladder step by miner signal: epoch (\d+) moves to rung (\d+) \((\d+) shadow passes, from rung (\d+)\).*?weakest up (\d+) bps/;
const SIGNAL_LINE = /Program class v5 by miner signal: epoch (\d+) \(share (\d+) bps.*?weakest (\d+) bps/;
const FLOOR_LINE = /Program class v5 from the override file: enabled, the floor at epoch (\d+)/;
@ -329,23 +329,38 @@ for (const e of [...programs.values()].sort((a, b) => a.epoch - b.epoch)) {
if (!ep || !ep.eraSeed) { idRows.push({ epoch: e.epoch, class: e.class, seed: e.seed, miners_id: e.id, miners: e.miners.size, reps: null, cli: null, cli_other: null, state_root: null, note: 'no template row for this epoch' }); continue; }
const st = streams.get(e.epoch);
const reps = ep.reps ?? 0;
const row = { epoch: e.epoch, class: e.class, rung: ep.rung, reps, seed: e.seed, miners_id: e.id, miners: e.miners.size, disagree: !!e.disagree, state_root: st?.root ?? null, state_records: st?.records ?? null, roots_per_node: st?.roots ?? {} };
if (e.class === 'v5') { row.cli = st?.path ? cliId(e.seedHex, ep.eraSeed, 'v5', reps, st.path) : null; row.cli_other = cliId(e.seedHex, ep.eraSeed, 'v4', reps, null); row.cli_rung0 = st?.path ? cliId(e.seedHex, ep.eraSeed, 'v5', 0, st.path) : null; }
// a seed that is not n0's template seed for the epoch is a FORK row (the first cross run: the restarted node's own chain
// printed its own epoch 8 and 9 seeds, "1 of 4"); listed, never counted as the main chain's
const fork = !!ep.seed && e.seedHex !== ep.seed;
const row = { epoch: e.epoch, class: e.class, rung: ep.rung, reps, seed: e.seed, fork, miners_id: e.id, miners: e.miners.size, disagree: !!e.disagree, state_root: fork ? null : (st?.root ?? null), state_records: fork ? null : (st?.records ?? null), roots_per_node: fork ? {} : (st?.roots ?? {}) };
if (e.class === 'v5') { row.cli = st?.path && !fork ? cliId(e.seedHex, ep.eraSeed, 'v5', reps, st.path) : null; row.cli_other = cliId(e.seedHex, ep.eraSeed, 'v4', reps, null); row.cli_rung0 = st?.path ? cliId(e.seedHex, ep.eraSeed, 'v5', 0, st.path) : null; }
else { row.cli = cliId(e.seedHex, ep.eraSeed, e.class, reps, null); row.cli_other = cliId(e.seedHex, ep.eraSeed, e.class, reps === RUNG0 ? RUNG1 : 0, null); }
idRows.push(row);
}
const v4Rows = idRows.filter(r => r.class === 'v4'), v5Rows = idRows.filter(r => r.class === 'v5');
const mainRows = idRows.filter(r => !r.fork), forkRows = idRows.filter(r => r.fork);
const v4Rows = mainRows.filter(r => r.class === 'v4'), v5Rows = mainRows.filter(r => r.class === 'v5');
// the miners expected on a main-chain row: every honest node's, less the restarted node while it is off the chain
const MINERS_EXPECTED = Math.max(2, honest.length - (restartNode ? 1 : 0));
const rung0Rows = v4Rows.filter(r => r.reps === RUNG0 || r.reps === 0), rung1Rows = v4Rows.filter(r => r.reps === RUNG1);
// rejections, acceptance, the lines
const rejectedMiner = nodes.map(n => minerLog(n.i).filter(l => /rejected nonce=|submit error/.test(l)).length);
// a chain refusal is `rejected nonce=`; a `submit error ... RPC request timeout` is the node's RPC under load (five on the
// first cross run at load 338 from another lane's build), counted apart and reported, never a PoW refusal
const rejectedMiner = nodes.map(n => minerLog(n.i).filter(l => /rejected nonce=/.test(l)).length);
const submitTimeouts = nodes.map(n => minerLog(n.i).filter(l => /submit error/.test(l)).length);
const rejectedNode = nodes.map(n => n.grepLog(REJECT_LINE).length);
const accepted = nodes.map(n => minerLog(n.i).filter(l => /ACCEPTED block/.test(l)).length);
const acceptedAfterFlip = nodes.map(n => acceptedAfter(n.i, flip?.wall));
const refreshEpoch = flip ? epochs.get(flip.epoch + 1) : null;
const staleAcceptedAfterRefresh = stale && refreshEpoch ? acceptedAfter(STALE, refreshEpoch.wall) : null;
// a block the stale miner built on the previous epoch's template ("template N gens old") and submitted seconds after the
// boundary is that epoch's block, valid under that epoch's stream (one such on the first cross run); a block on a fresh
// template of the new epoch with the frozen stream is what the rule refuses
const staleAcceptedAfterRefresh = stale && refreshEpoch ? acceptedAfter(STALE, refreshEpoch.wall, true) : null;
const staleOldTemplateAfterRefresh = stale && refreshEpoch ? acceptedAfter(STALE, refreshEpoch.wall) - staleAcceptedAfterRefresh : null;
const stepLines = nodes.map(n => n.grepLog(STEP_LINE).map(l => l.replace(/^.*?(Latency ladder step by miner signal)/, '$1')));
const stepParsed = stepLines.map(ls => { const m = ls[0] && STEP_LINE.exec(ls[0]); return m ? { epoch: +m[1], rung: +m[2], reps: +m[3], weakest_up: +m[5] } : null; });
// the 0.3.23 line logs one step twice (the template's lead path and the boundary); a second STEP is a new (epoch, rung) pair
const distinctSteps = stepLines.map(ls => new Set(ls.map(l => { const m = STEP_LINE.exec(l); return m ? `${m[1]}:${m[2]}` : l; })).size);
const signalLines = nodes.map(n => n.grepLog(SIGNAL_LINE).map(l => l.replace(/^.*?(Program class v5 by miner signal)/, '$1')));
const signalParsed = signalLines.map(ls => { const m = ls[0] && SIGNAL_LINE.exec(ls[0]); return m ? { epoch: +m[1], share: +m[2], weakest: +m[3] } : null; });
const floorLines = nodes.map(n => n.grepLog(FLOOR_LINE)[0]?.replace(/^.*?(Program class v5 from)/, '$1') ?? null);
@ -380,7 +395,7 @@ const checks = {
stepped_at_the_first_full_window_epoch: step != null && step.epoch === FIRST_STEP_EPOCH,
step_line_on_every_node_same_epoch: stepParsed.every(s => s != null && s.rung === 1) && new Set(stepParsed.map(s => s?.epoch)).size === 1 && stepParsed[0]?.epoch === step?.epoch,
step_weakest_up_at_or_above_threshold: stepParsed.every(s => s != null && s.weakest_up >= LADDER_THRESHOLD),
no_second_step: step != null && afterStep.length >= 2 && afterStep.every(r => r === 1) && stepLines.every(ls => ls.length === 1),
no_second_step: step != null && afterStep.length >= 2 && afterStep.every(r => r === 1) && distinctSteps.every(n => n === 1),
rung0_ids_equal_the_cli_rung0_id: rung0Rows.length > 0 && rung0Rows.every(r => r.cli != null && r.cli === r.miners_id && !r.disagree),
rung1_ids_equal_the_cli_rung1_id: rung1Rows.length > 0 && rung1Rows.every(r => r.cli != null && r.cli === r.miners_id && !r.disagree && r.cli_other !== r.miners_id),
// the crossing: v5 by signal at byte 6 exactly, at the first full-window epoch, before the floor, on every honest node
@ -395,7 +410,7 @@ const checks = {
v5_rides_at_rung_1: flip != null && v5Epochs.length > 0 && v5Epochs.every(([, v]) => v.rung === 1 && v.reps === RUNG1),
// one program id per epoch on every node, equal to the CPU verifier's class v5 id for that seed, state and rung
a_stream_for_every_v5_epoch: v5Epochs.length > 0 && v5Epochs.every(([e]) => streams.get(e)?.path),
v5_ids_equal_the_cli_v5_id: v5Rows.length >= 2 && v5Rows.every(r => r.cli != null && r.cli === r.miners_id && r.miners >= honest.length && !r.disagree),
v5_ids_equal_the_cli_v5_id: v5Rows.length >= 2 && v5Rows.every(r => r.cli != null && r.cli === r.miners_id && r.miners >= MINERS_EXPECTED && !r.disagree),
v5_ids_differ_from_the_same_seed_v4_id: v5Rows.length > 0 && v5Rows.every(r => r.cli_other != null && r.cli_other !== r.miners_id),
v5_ids_differ_from_the_rung0_v5_id: v5Rows.length > 0 && v5Rows.every(r => r.cli_rung0 != null && r.cli_rung0 !== r.miners_id),
state_roots_agree_across_nodes: rootRows.length > 0 && rootRows.every(r => new Set(r.roots).size === 1),
@ -423,15 +438,15 @@ const summary = {
node: IGNEUMD, miner: CPU_MINER, pow: IGNEUM_POW, override_keys_dropped: DROPPED, dn3_genesis_check: dn3, ladder_step: step, class_switch: flip, run_ended_at_s: endAt, final_daa: lastDaa, max_epoch_seen: maxEpochSeen,
epochs: Object.fromEntries(epochRows), streams: Object.fromEntries([...streams.entries()].map(([e, s]) => [e, { root: s.root, records: s.records, block: s.block, roots_per_node: s.roots }])),
blocks: { total: blocks.length, before_boundary: before.length, after_boundary: after.length, object_bytes: objectBytes, byte_6_share_bps: byte6Share, ladder_bits: ladderBits, up_share_bps: upShare },
program_id_rows: idRows, accepted_per_miner: accepted, accepted_after_flip_per_miner: acceptedAfterFlip, rejected_by_miners: rejectedMiner, rejected_by_nodes: rejectedNode,
stale: stale ? { node: STALE, accepted_total: accepted[STALE], accepted_after_first_refresh: staleAcceptedAfterRefresh, rejected_by_its_node: rejectedNode[STALE], rejected_lines: rejectedMiner[STALE], first_refresh_epoch: flip ? flip.epoch + 1 : null } : null,
program_id_rows: idRows, fork_rows: forkRows.length, miners_expected_on_a_row: MINERS_EXPECTED, accepted_per_miner: accepted, accepted_after_flip_per_miner: acceptedAfterFlip, rejected_by_miners: rejectedMiner, submit_timeouts_per_miner: submitTimeouts, rejected_by_nodes: rejectedNode,
stale: stale ? { node: STALE, accepted_total: accepted[STALE], accepted_after_first_refresh: staleAcceptedAfterRefresh, old_template_blocks_after_refresh: staleOldTemplateAfterRefresh, rejected_by_its_node: rejectedNode[STALE], rejected_lines: rejectedMiner[STALE], first_refresh_epoch: flip ? flip.epoch + 1 : null } : null,
restart: restartNode ? { node: RESTART, ...restart, lock_fault_lines: restartFaults.slice(0, 5), ibd_refusal_lines: restartIbdRefusals.length, ibd_refusal_example: restartIbdRefusals[0]?.replace(/^.*?\] /, '').slice(0, 300) ?? null, accepted_after_restart: restartAcceptedAfter, rejected_by_its_node: rejectedNode[RESTART] } : null,
sinks, block_counts: counts, step_lines: stepLines.map(ls => ls.map(l => l.slice(0, 300))), signal_lines: signalLines.map(ls => ls.map(l => l.slice(0, 300))), floor_lines: floorLines, samples,
};
writeFileSync(`${TMP}/summary.json`, JSON.stringify(summary, null, 2));
try { mkdirSync(OUT, { recursive: true }); writeFileSync(`${OUT}/${CASE}.json`, JSON.stringify(summary, null, 2)); } catch (e) { log(`could not write ${OUT}/${CASE}.json: ${e.message}`); }
log(`SUMMARY ${pass ? 'PASS' : 'FAIL'} (${CASE}) ${summary.utc}: ${step ? `rung ${step.rung} (${step.reps} passes) from epoch ${step.epoch} at DAA ${step.daa}` : 'no rung step'}; ${flip ? `v5 from epoch ${flip.epoch} at DAA ${flip.daa} at rung ${flip.rung}` : 'no v5 epoch'}; epochs ${epochRows.map(([e, v]) => `e${e}:v${v.class}:r${v.rung}:${v.v5_weakest_bps}bps`).join(' ')}; chain bytes ${JSON.stringify(objectBytes)} (${byte6Share} bps at byte 6), ladder bits ${JSON.stringify(ladderBits)}; blocks ${before.length} / ${after.length} across DAA ${BOUNDARY}; rejected miners ${rejectedMiner.join('/')} nodes ${rejectedNode.join('/')}; accepted after the flip ${acceptedAfterFlip.join('/')}; sinks ${sinks.join(' ')} (${checks.sinks_agree ? 'agree' : 'DIFFER'}) at ${counts.join('/')}; step lines ${stepParsed.filter(Boolean).length}/${N} (epochs ${stepParsed.map(s => s?.epoch ?? '-').join('/')}), v5 signal lines ${signalParsed.filter(Boolean).length}/${N} (epochs ${signalParsed.map(s => s?.epoch ?? '-').join('/')}, weakest ${signalParsed.map(s => s?.weakest ?? '-').join('/')}); stale n${STALE ?? '-'} accepted after the first refresh ${staleAcceptedAfterRefresh ?? 'n/a'} rejected ${STALE != null ? rejectedNode[STALE] : 'n/a'}; restart n${RESTART ?? '-'} ${restart.stopped ? `stopped at DAA ${restart.stopped.daa}` : 'not stopped'}${restart.started ? `, started at DAA ${restart.started.daa}` : ''}${restart.resynced ? `, resynced ${restart.resynced.after_s} s later` : ''}, lock faults ${restartFaults.length}, IBD refusals ${restartIbdRefusals.length}, accepted after ${restartAcceptedAfter ?? 'n/a'}; dn3 id ${dn3 ? (dn3.ok ? 'OK' : 'MISMATCH') : 'skipped'}`);
for (const r of idRows) log(`PROGRAM ID epoch ${r.epoch} ${r.class} rung ${r.rung ?? '?'} (${r.reps ?? '?'} passes) seed ${r.seed}: miners ${r.miners_id} (${r.miners} of ${N}${r.disagree ? ', DISAGREE' : ''}) cli ${r.cli} other-class ${r.cli_other}${r.cli_rung0 ? ` rung0 ${r.cli_rung0}` : ''}${r.state_root ? ` state ${String(r.state_root).slice(0, 18)} (${r.state_records} records)` : ''} ${r.cli === r.miners_id ? 'OK' : 'MISMATCH'}`);
log(`SUMMARY ${pass ? 'PASS' : 'FAIL'} (${CASE}) ${summary.utc}: ${step ? `rung ${step.rung} (${step.reps} passes) from epoch ${step.epoch} at DAA ${step.daa}` : 'no rung step'}; ${flip ? `v5 from epoch ${flip.epoch} at DAA ${flip.daa} at rung ${flip.rung}` : 'no v5 epoch'}; epochs ${epochRows.map(([e, v]) => `e${e}:v${v.class}:r${v.rung}:${v.v5_weakest_bps}bps`).join(' ')}; chain bytes ${JSON.stringify(objectBytes)} (${byte6Share} bps at byte 6), ladder bits ${JSON.stringify(ladderBits)}; blocks ${before.length} / ${after.length} across DAA ${BOUNDARY}; rejected miners ${rejectedMiner.join('/')} nodes ${rejectedNode.join('/')} (submit timeouts ${submitTimeouts.join('/')}); accepted after the flip ${acceptedAfterFlip.join('/')}; sinks ${sinks.join(' ')} (${checks.sinks_agree ? 'agree' : 'DIFFER'}) at ${counts.join('/')}; step lines ${stepParsed.filter(Boolean).length}/${N} (epochs ${stepParsed.map(s => s?.epoch ?? '-').join('/')}), v5 signal lines ${signalParsed.filter(Boolean).length}/${N} (epochs ${signalParsed.map(s => s?.epoch ?? '-').join('/')}, weakest ${signalParsed.map(s => s?.weakest ?? '-').join('/')}); stale n${STALE ?? '-'} accepted after the first refresh ${staleAcceptedAfterRefresh ?? 'n/a'} rejected ${STALE != null ? rejectedNode[STALE] : 'n/a'}; restart n${RESTART ?? '-'} ${restart.stopped ? `stopped at DAA ${restart.stopped.daa}` : 'not stopped'}${restart.started ? `, started at DAA ${restart.started.daa}` : ''}${restart.resynced ? `, resynced ${restart.resynced.after_s} s later` : ''}, lock faults ${restartFaults.length}, IBD refusals ${restartIbdRefusals.length}, accepted after ${restartAcceptedAfter ?? 'n/a'}; dn3 id ${dn3 ? (dn3.ok ? 'OK' : 'MISMATCH') : 'skipped'}`);
for (const r of idRows) log(`PROGRAM ID epoch ${r.epoch}${r.fork ? ' FORK' : ''} ${r.class} rung ${r.rung ?? '?'} (${r.reps ?? '?'} passes) seed ${r.seed}: miners ${r.miners_id} (${r.miners} of ${N}${r.disagree ? ', DISAGREE' : ''}) cli ${r.cli} other-class ${r.cli_other}${r.cli_rung0 ? ` rung0 ${r.cli_rung0}` : ''}${r.state_root ? ` state ${String(r.state_root).slice(0, 18)} (${r.state_records} records)` : ''} ${r.cli === r.miners_id ? 'OK' : 'MISMATCH'}`);
for (const [k, v] of Object.entries(checks)) if (v === false) log(`FAILED CHECK ${k}`);
log(`summary: ${TMP}/summary.json and ${OUT}/${CASE}.json`);
await finish(pass ? 0 : 1);