From 341a2ad134a2df9faa99e7f5f45434843303e3d2 Mon Sep 17 00:00:00 2001 From: igneum-labs <337424239+igneum-labs@users.noreply.github.com> Date: Mon, 5 Oct 2026 09:47:12 +0000 Subject: [PATCH] release-0.3.6 plan: section 9, the 0.3.6 Windows runtime incident and the 0.3.7 hotfix --- docs/plans/release-0.3.6.md | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/docs/plans/release-0.3.6.md b/docs/plans/release-0.3.6.md index 4f5efa56b..192cf80f3 100644 --- a/docs/plans/release-0.3.6.md +++ b/docs/plans/release-0.3.6.md @@ -502,3 +502,34 @@ on app 0.3.5, node 2.1.0-20139145; rotation 0 of 6 on the new key and folder. | Machine | Seen on 0.3.6 | What its log shows | |---|---|---| | PC 1 ae432dc7 (Windows, the first over-the-air update through the fixed helper) | 09:37:31Z watch tick (the first tick after the restart); the 0.3.6 engine's first line is at 09:35:17Z, 71 s after the manifest went live | run `win-ae432dc7-20261005-093517`: `IGNEUM-APP version=0.3.6 machine=ae432dc7 platform=windows`; `[ok] updated to Igneum Miner 0.3.6 from 0.3.5`; `config: intake ... key 477bb0ef (packaged); manifest .../dl//igneum-app-latest.json folder ed9c4d2e (packaged)` (the NEXT key and folder: rotation 1 of 6); `node proof verifier: command (...\igneum-prove-verify.exe)`; `igneumd started (pid 26572)`; `update check: 0.3.6 is current`; `wake: listening at https://relay.igneum.network/wake`, and at 09:36:47Z `update to 0.3.6 complete (from 0.3.5); keeping the previous version for a rollback`; 09:37:15Z a wake stamp change fetched the jobs file within seconds (the instant-jobs path live on Windows). The 0.3.5 engine's own download, stage and helper lines were not uploaded before it quit (its last app-log upload was 09:30:14Z; the node log kept uploading to 09:35:14Z): fetched afterwards with a collect job (below) | + +## 9. Incident 09:4x BST and the 0.3.7 hotfix + +Both PCs took 0.3.6 over the air (the Windows OTA path works: PC 1's engine restarted 71 s after the manifest went live, +PC 2 by 09:39Z), and on both the new igneumd.exe refused to start: "Entry Point Not Found: +_ZNKSt25__codecvt_utf8_utf16_baseIwE10do_unshiftERiPcS2_RS2_ could not be located in igneumd.exe". Mining was down on +both PCs; the Mac (d937c69d) ran 0.3.6 with node 2b6d23ef from 09:39Z. The coordinator republished 0.3.5 in the OLD +folder (0.3.5 machines stay put; the NEXT folder kept 0.3.6) and the project lead approved a DLL swap job (fix-runtime-036) and +0.3.7 through the pipeline. + +Cause, read with `x86_64-w64-mingw32-objdump -p`: every igneumd.exe so far imports libstdc++-6.dll (0.3.5's Mac +cross-build, 197 symbols; the PC build, 198), `-C link-arg=-static` notwithstanding. 0.3.5 shipped Mac-built exes with +the Mac toolchain's DLL (GCC 16.2.0): a match. 0.3.6 shipped PC-built exes (Ubuntu 24.04's mingw, GCC 13) with the same +Mac DLL, and GCC 16's libstdc++ no longer exports seven symbols the GCC 13 exe imports (five +`std::__codecvt_utf8_utf16_base` methods, `basic_stringbuf::seekpos`, and one more). The `-static` flag and +the toolchain rule were both assumed, neither checked: the lesson is the gate below. + +0.3.7 = release-0.3.6 tip + (commit 7c... "Igneum Miner 0.3.7"): + +| Change | Where | Check | +|---|---|---| +| The gate: every symbol an exe imports from a shipped lib*.dll must be exported by that DLL | `packaging/windows/check-runtime-dlls.sh`, run by `push-inputs.sh` before signing and by `make-payload.sh` before the installer is packed (skips with a note where objdump is missing, the runner) | known-bad (the 0.3.6 PC exes with the Mac DLL): FAILED, 7 missing, the morning's symbol first; known-good (0.3.5's Mac exes with the same DLL): 197 of 197 exported, pass | +| DLLs from the toolchain that linked the exes | `push-inputs.sh` takes lib*.dll next to the exes first, then the Mac toolchain (make-payload.sh already did); `jobbuild.rs` windows stage copies the PC toolchain's `libstdc++-6.dll`, `libgcc_s_seh-1.dll` (gcc `-print-file-name`) and `libwinpthread-1.dll` into the pack with RESULT lines; `build-job.mjs` accepts the small DLL PEs and places them next to the exes | unit test on the stage script; the PC path itself runs only once the 0.3.7 app is on PC 1 (the stage scripts come from the app on the PC) | +| `-static-libstdc++` tried | `proto-cuda/windows-node/cross-build.sh` | measured on the 0.3.7 cross-build below: does the import disappear? | +| version 0.3.7 | the six files | `ship-app.mjs --check` | + +The 0.3.7 Windows exes come from the Mac cross-build (the 0.3.5 way, Homebrew mingw-w64 GCC 16.2.0, target dir +cloned from `target-release-win`), paired with the same toolchain's DLLs; the PC-built pairing would need the GCC 13 +DLLs, which the 0.3.6 app's build job cannot upload. The DMG is rebuilt for the 0.3.7 engine with the same node +binaries (2b6d23ef) and the 0.3.5 prover. The ship publishes to BOTH folders (OLD folder too: the 0.3.5 machines go +straight to 0.3.7).