- provision.sh step_runner: actions/runner 2.338.0 (sha256 checked) at /opt/actions-runner under a dedicated user `runner`
(no sudo, not in build's group), rustup 1.99.0 pinned with both targets, sccache against /srv/sccache in READ_ONLY mode
on its own server port, Node 22 and mingw from the system, GitHub's svc.sh unit with a Nice 10 drop-in; registered on
igneum-network/igneum as igneum-build-1 (labels self-hosted, linux, x64, igneum-build-1) through
infra/build-server/runner/register.sh (gh as igneum-labs, the token on ssh stdin, never logged). Idempotent after
the env files moved behind svc.sh install (its env.sh rewrites them). libicu74 and python3-numpy added to APT.
- main's slots ruling: SLOTS default 2; remote-run.sh sets CARGO_BUILD_JOBS 90 when it holds the only taken slot and 45
when both are held, BR_MEASURE=1 takes the `measure` file exclusively and excludes builds (builds hold it shared),
lock files open in append mode (the old `exec {fd}>` truncated a busy slot's holder line on every probe), env
IGNEUM_BUILD_SLOTS_DIR and IGNEUM_BUILD_LOG_DIR win over the profile, `--self-test-slots` with five cases (the old
script fails it with JOBS=none); build-remote.sh and cross-remote.sh pass -j only when --jobs is given.
- infra/build-server/prover/cpu-trial.sh: the SP1 CPU prover on one fixture shard under the measure hold with a VmHWM
poller; 6 Oct 2026 run: core 34.2 s, compressed 85.9 s, peak RSS 28.2 GB on 96 threads, so no standing CPU prover.
- docs/plans/ci-self-hosted.md: the proposed runs-on change for ci.yml behind the repository variable IGNEUM_CI_RUNNER
(GitHub-hosted is the fallback), and why windows.yml cannot move to a Linux box. Workflows untouched.
- docs/plans/build-server.md section 7.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
4.8 KiB
CI on the box: the self-hosted runner and the workflow change (proposal, 6 October 2026)
The runner igneum-build-1 (labels self-hosted, linux, x64, igneum-build-1) is installed by infra/build-server/provision.sh
step_runner and registered by infra/build-server/runner/register.sh (docs/plans/build-server.md section 7). The workflows are
NOT changed here: the shipper owns .github/workflows tonight. This is the proposed diff for main.
1. The shape: one repository variable decides, GitHub-hosted is the fallback
GitHub has no "try this runner, else that one" in runs-on: a list of labels means ALL of them must match one runner, so
[self-hosted, igneum-build-1, ubuntu-latest] would never schedule. The fallback is therefore a repository variable read in
the expression. IGNEUM_CI_RUNNER = box sends the job to the box; unset or anything else keeps ubuntu-latest. Flipping it
back is one click in Settings > Secrets and variables > Actions > Variables (or gh variable set IGNEUM_CI_RUNNER --body box
and gh variable delete IGNEUM_CI_RUNNER as igneum-labs), with no commit and no queue lost: a job already queued for the box
stays queued; the next push goes to GitHub's machines.
2. ci.yml (the two jobs that compile or compute; the site job stays on GitHub's machines)
jobs:
pow:
name: igneum-pow tests, igneum-census build
- runs-on: ubuntu-latest
+ runs-on: ${{ vars.IGNEUM_CI_RUNNER == 'box' && fromJSON('["self-hosted", "linux", "x64", "igneum-build-1"]') || 'ubuntu-latest' }}
steps:
- uses: actions/checkout@v4
- name: toolchain
run: rustc --version && cargo --version
@@
sims:
name: simulators, quick modes
- runs-on: ubuntu-latest
+ runs-on: ${{ vars.IGNEUM_CI_RUNNER == 'box' && fromJSON('["self-hosted", "linux", "x64", "igneum-build-1"]') || 'ubuntu-latest' }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
+ if: vars.IGNEUM_CI_RUNNER != 'box' # the box has python3 and numpy from provision.sh; setup-python would download a second Python
with:
python-version: '3.12'
- - run: python3 -m pip install --quiet numpy
+ - run: python3 -m pip install --quiet numpy
+ if: vars.IGNEUM_CI_RUNNER != 'box'
What the box gives these two jobs: rustc 1.99.0 pinned (GitHub's ubuntu-latest carries whatever stable it ships; the box
is the Mac's version, so CI compiles what the agents compile), sccache hits from the agents' cache (read-only), 48 cargo
jobs. The site job is Node and shell checks and takes under a minute on GitHub's runners; moving it buys nothing and would
put tools/ci/public-api-check.mjs (a live HTTPS check) behind the box's egress for no reason.
Why the toolchain line still runs: on the box rustc --version must print 1.99.0; a mismatch means provision.sh and the
runner's rustup disagree (R5 in build-server.md) and the job should say so in its first step.
3. windows.yml: no change possible on this box
Every job of windows.yml runs on windows-latest for a reason the box cannot answer: the engine builds on the MSVC
target, the window host needs the Windows SDK and WebView2, the installer needs Inno Setup, the smoke run executes the exes
and the launcher under Windows PowerShell 5.1. A Linux runner has none of that. The only self-hosted option for this
workflow is a Windows runner on PC 1 or PC 2 (actions/runner for Windows under a service account), which conflicts with
the rule that the PCs keep only GPU and Windows-runtime JOBS through the signed job system, and is not proposed tonight.
What the box already does for Windows is upstream of this workflow: tools/cross-remote.sh builds igneumd.exe and
igneum-miner.exe (the payload inputs) in 1 min 44 s, and the night battery rebuilds them for the reproducibility record.
4. What to check after the flip (main, the first run on the box)
| Check | Where | Pass |
|---|---|---|
| the job landed on the box | the run's "Set up job" log says Runner name: 'igneum-build-1' |
yes |
| the toolchain | the toolchain step prints rustc 1.99.0 |
yes |
| sccache hits | add sccache --show-stats as a step once, or read /srv/sccache size before and after: the runner's config is READ_ONLY, so the size must NOT change |
size unchanged |
| the agents were not starved | /srv/builds/_log/builds.jsonl secs of the builds during the run against the same crate's earlier lines |
within the usual spread |
| the fallback | gh variable delete IGNEUM_CI_RUNNER, push a no-op commit: the job runs on ubuntu-latest again |
yes |
Open: a CI job on the box does not take a build slot (/srv/builds/_locks/build-<k>), it runs at Nice 10 with 48 jobs; if a
CI job ever delays a release build visibly, the fix is a step at the top of the job that takes a slot through
infra/build-server/remote-run.sh's flock, the same file the agents use.