igneum-network/igneum
1
0
Fork 0
Code Issues Pull requests Releases Activity

Build server: zig and cargo-zigbuild on the box; build-remote.sh --ship builds glibc 2.36 Linux artefacts for seeds and HiveOS; the glibc ceiling check

Browse source 
Main's order of 7 October 2026 after a seed took 14 restarts and three minutes down on a glibc 2.39 binary. provision.sh:
step_zig (zig 0.17.0 from ziglang.org, sha256 from the official download index) and cargo-zigbuild 0.23.4 in the cargo tools.
tools/build-remote.sh --ship [--glibc 2.36]: cargo zigbuild --target x86_64-unknown-linux-gnu.2.36 with zig as the C/C++
toolchain (the Mac's infra/cross/build-linux.sh recipe), artefacts from the target-triple dir, each checked by
tools/ci/glibc-ceiling-check.sh (need at most the ceiling; self-test fires on 2.38 against 2.36, passes 2.34 and 2.36;
--symbols reads a saved objdump -T text so CI needs no ELF tools). tools/workers-remote.sh builds the two GPU workers with
zig at 2.36 by default (GLIBC=native for clang). Proof on the box: fork 3bfe346f igneumd needs GLIBC_2.34 (47,023,120 B,
sha256 345dfb95...), igneum-miner GLIBC_2.34 (9,248,808 B, d09dc27b...), 3 min 17 s cold through zig; the workers at 2.36.
Rule: anything that ships to a seed or a HiveOS rig is built with --ship; a plain build (glibc 2.39) is for the box and
Ubuntu 24.04 hosts only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
igneum-labs 2026-10-06 23:37:31 +00:00
parent 2750361894
commit 7089aa161f
5 changed files with 96 additions and 12 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
docs/plans/build-server.md
View file

@ -124,7 +124,7 @@ Self-test: `tools/build-remote.sh --self-test-repro [--full]` from a fork worktr
| Gap | Why it matters | Next step | | Gap | Why it matters | Next step |
|---|---|---| |---|---|---|
| No zig / cargo-zigbuild | the devnet seed (Debian 12, glibc 2.36) takes the Mac's zig build; a native box build links glibc 2.39, which Debian 13 seeds accept and HiveOS (Ubuntu 18/20 base) does not | install zig 0.17 + cargo-zigbuild in provision.sh, add `--target x86_64-unknown-linux-gnu.2.36` mode to build-remote.sh | | zig / cargo-zigbuild: DONE 7 Oct 2026 (main's order after a seed took 14 restarts and three minutes down on a glibc 2.39 binary) | provision.sh `step_zig` (zig 0.17.0 from ziglang.org, sha256 of the official index) and cargo-zigbuild 0.23.4 in `step_cargo_tools`; `tools/build-remote.sh --ship [--glibc 2.36]` runs `cargo zigbuild --target x86_64-unknown-linux-gnu.2.36`, fetches from the target-triple dir and runs `tools/ci/glibc-ceiling-check.sh` (need at most 2.36) on every artefact; `tools/workers-remote.sh` builds with `zig cc/c++ -target x86_64-linux-gnu.2.36` by default (`GLIBC=native` for clang). Rule: anything that ships to a seed or a HiveOS rig is built with `--ship`; a plain build is glibc 2.39 for the box and Ubuntu 24.04 hosts only. Proof (fork 3bfe346f, cold through zig, 3 min 17 s): igneumd 47,023,120 B sha256 345dfb95... needs GLIBC_2.34; igneum-miner 9,248,808 B sha256 d09dc27b... needs GLIBC_2.34; the check's self-test fires on 2.38 against 2.36 and passes 2.34 and 2.36 | the Mac's infra/cross/build-linux.sh is the same recipe and can retire once two seed releases shipped from the box |
| No macOS target | agents who run nodes on the Mac still build there | out of scope (needs the macOS SDK on Linux); the fleet or the box's own Devnet 2 seed takes the test-network runs instead | | No macOS target | agents who run nodes on the Mac still build there | out of scope (needs the macOS SDK on Linux); the fleet or the box's own Devnet 2 seed takes the test-network runs instead |
| No CI runner | DONE 6 October 2026, 19:19Z (section 7): the runner `igneum-build-1` is online under user `runner`, never build; the workflow change is proposed in docs/plans/ci-self-hosted.md | main flips `IGNEUM_CI_RUNNER=box` after the shipper's cut | | No CI runner | DONE 6 October 2026, 19:19Z (section 7): the runner `igneum-build-1` is online under user `runner`, never build; the workflow change is proposed in docs/plans/ci-self-hosted.md | main flips `IGNEUM_CI_RUNNER=box` after the shipper's cut |
| Byte identity with the Mac's exes | different C/C++ toolchain (Homebrew mingw vs Ubuntu GCC 13) and embedded source paths | not a goal; the box is identical with itself build to build, cross-remote.sh reports sha256 and the DLL list per exe | | Byte identity with the Mac's exes | different C/C++ toolchain (Homebrew mingw vs Ubuntu GCC 13) and embedded source paths | not a goal; the box is identical with itself build to build, cross-remote.sh reports sha256 and the DLL list per exe |

22
infra/build-server/provision.sh
View file

@ -332,6 +332,22 @@ step_node() {
changed node "$(/usr/local/bin/node --version) from nodejs.org (sha256 checked)" changed node "$(/usr/local/bin/node --version) from nodejs.org (sha256 checked)"
} }
# zig (main, 7 October 2026): the glibc 2.36 Linux artefacts for Debian 12 seeds and HiveOS rigs come from cargo-zigbuild with zig as
# the C/C++ toolchain, as infra/cross/build-linux.sh does on the Mac (tonight's seed took 14 restarts and three minutes down on a
# glibc 2.39 native build). The release the Mac uses (0.17.0), from ziglang.org with the sha256 of the official download index.
ZIG_VERSION="${ZIG_VERSION:-0.17.0}"
ZIG_SHA256="${ZIG_SHA256:-1cbe9df9f27e6b78d14ccbca43b6703a404ef79ef1c463de901d7f088d4e2026}" # zig-x86_64-linux-0.17.0.tar.xz, index.json 7 Oct 2026
step_zig() {
local have dir tar
have=$(/usr/local/bin/zig version 2>/dev/null || true)
if [ "$have" = "$ZIG_VERSION" ]; then ok zig "$have"; return; fi
dir=/usr/local/lib/zig; tar="zig-x86_64-linux-$ZIG_VERSION.tar.xz"
install -d "$dir"
( cd "$dir" && curl -fsSLO "https://ziglang.org/download/$ZIG_VERSION/$tar" && echo "$ZIG_SHA256 $tar" | sha256sum -c --quiet - && tar -xJf "$tar" && rm -f "$tar" )
ln -sfn "$dir/zig-x86_64-linux-$ZIG_VERSION/zig" /usr/local/bin/zig
changed zig "$(/usr/local/bin/zig version) from ziglang.org (sha256 checked) at /usr/local/bin/zig"
}
step_sshd() { step_sshd() {
local f=/etc/ssh/sshd_config.d/10-igneum-build.conf tmp local f=/etc/ssh/sshd_config.d/10-igneum-build.conf tmp
tmp=$(mktemp) tmp=$(mktemp)
@ -543,7 +559,9 @@ step_runner() {
step_cargo_tools() { step_cargo_tools() {
local cargo="$BUILD_HOME/.cargo/bin/cargo" any=0 local cargo="$BUILD_HOME/.cargo/bin/cargo" any=0
if [ ! -x "$BUILD_HOME/.cargo/bin/cargo-audit" ]; then as_build "$cargo install cargo-audit --locked" >/dev/null 2>&1 || as_build "$cargo install cargo-audit" >/dev/null; any=1; fi if [ ! -x "$BUILD_HOME/.cargo/bin/cargo-audit" ]; then as_build "$cargo install cargo-audit --locked" >/dev/null 2>&1 || as_build "$cargo install cargo-audit" >/dev/null; any=1; fi
[ "$any" = 1 ] && changed cargo-tools "$(as_build "$BUILD_HOME/.cargo/bin/cargo-audit --version")" || ok cargo-tools "$(as_build "$BUILD_HOME/.cargo/bin/cargo-audit --version")" # cargo-zigbuild (main, 7 October 2026): the glibc 2.36 Linux artefacts for Debian 12 seeds and HiveOS rigs (tools/build-remote.sh --ship)
if [ ! -x "$BUILD_HOME/.cargo/bin/cargo-zigbuild" ]; then as_build "$cargo install cargo-zigbuild --locked" >/dev/null 2>&1 || as_build "$cargo install cargo-zigbuild" >/dev/null; any=1; fi
[ "$any" = 1 ] && changed cargo-tools "$(as_build "$BUILD_HOME/.cargo/bin/cargo-audit --version"), $(as_build "$BUILD_HOME/.cargo/bin/cargo-zigbuild --version")" || ok cargo-tools "$(as_build "$BUILD_HOME/.cargo/bin/cargo-audit --version"), $(as_build "$BUILD_HOME/.cargo/bin/cargo-zigbuild --version")"
} }
# the night battery (infra/build-server/night): the script and remote-run.sh into /srv/builds/_bin, the two units, the timer # the night battery (infra/build-server/night): the script and remote-run.sh into /srv/builds/_bin, the two units, the timer
@ -579,6 +597,7 @@ step_summary() {
printf 'raid: %s\n' "$(grep -E '^md' /proc/mdstat 2>/dev/null | tr '\n' ';' || echo none)" printf 'raid: %s\n' "$(grep -E '^md' /proc/mdstat 2>/dev/null | tr '\n' ';' || echo none)"
printf 'rust: %s | %s | targets %s\n' "$(as_build "$BUILD_HOME/.cargo/bin/rustc --version")" "$(as_build "$BUILD_HOME/.cargo/bin/cargo --version")" "$(as_build "$BUILD_HOME/.cargo/bin/rustup target list --installed" | tr '\n' ' ')" printf 'rust: %s | %s | targets %s\n' "$(as_build "$BUILD_HOME/.cargo/bin/rustc --version")" "$(as_build "$BUILD_HOME/.cargo/bin/cargo --version")" "$(as_build "$BUILD_HOME/.cargo/bin/rustup target list --installed" | tr '\n' ' ')"
printf 'sccache: %s, %s\n' "$(as_build "$BUILD_HOME/.cargo/bin/sccache --version")" "$(cat "$BUILD_HOME/.config/sccache/config" | tr '\n' ' ')" printf 'sccache: %s, %s\n' "$(as_build "$BUILD_HOME/.cargo/bin/sccache --version")" "$(cat "$BUILD_HOME/.config/sccache/config" | tr '\n' ' ')"
printf 'zig: %s, cargo-zigbuild %s (glibc 2.36 Linux artefacts: tools/build-remote.sh --ship, tools/workers-remote.sh)\n' "$(/usr/local/bin/zig version 2>/dev/null || echo missing)" "$(as_build "$BUILD_HOME/.cargo/bin/cargo-zigbuild --version 2>/dev/null | awk '{ print \$NF }'" || echo missing)"
printf 'mingw: %s\n' "$(x86_64-w64-mingw32-gcc-posix --version | head -1)" printf 'mingw: %s\n' "$(x86_64-w64-mingw32-gcc-posix --version | head -1)"
printf 'clang: %s | lld: %s\n' "$(clang --version | head -1)" "$(ld.lld --version | head -1)" printf 'clang: %s | lld: %s\n' "$(clang --version | head -1)" "$(ld.lld --version | head -1)"
printf 'node: %s | git: %s | tmux: %s\n' "$(/usr/local/bin/node --version)" "$(git --version)" "$(tmux -V)" printf 'node: %s | git: %s | tmux: %s\n' "$(/usr/local/bin/node --version)" "$(git --version)" "$(tmux -V)"
@ -616,6 +635,7 @@ do_provision() {
step_ufw step_ufw
step_runner step_runner
step_cargo_tools step_cargo_tools
step_zig
step_night step_night
step_summary step_summary
log "done" log "done"

33
tools/build-remote.sh
View file

@ -12,6 +12,15 @@
# tools/build-remote.sh --jobs 48 -- check # tools/build-remote.sh --jobs 48 -- check
# tools/build-remote.sh --target-dir target-exp -- build --release another persistent target dir on the box # tools/build-remote.sh --target-dir target-exp -- build --release another persistent target dir on the box
# tools/build-remote.sh --no-fetch -- clippy --all-targets nothing comes back (tests, check, clippy) # tools/build-remote.sh --no-fetch -- clippy --all-targets nothing comes back (tests, check, clippy)
# tools/build-remote.sh --ship [--glibc 2.36] anything that SHIPS to a seed (Debian 12) or a HiveOS
# rig: cargo zigbuild for x86_64-unknown-linux-gnu.<glibc>
# (zig as the C/C++ toolchain, as the Mac's
# infra/cross/build-linux.sh), artefacts from
# target/x86_64-unknown-linux-gnu/release, each checked by
# tools/ci/glibc-ceiling-check.sh (need at most <glibc>).
# A plain build is native glibc 2.39: the box, the fleet's
# Ubuntu 24.04 hosts, never a seed or a rig (7 Oct 2026:
# a seed took 14 restarts on a 2.39 binary).
# tools/build-remote.sh --self-test-repro [--full] from a fork worktree: igneum-miner built twice a minute # tools/build-remote.sh --self-test-repro [--full] from a fork worktree: igneum-miner built twice a minute
# apart without sccache into one target dir must give one # apart without sccache into one target dir must give one
# sha256, and a per-run target path must not (prost's # sha256, and a per-run target path must not (prost's
@ -48,7 +57,7 @@ BS_TOOL=build-remote
. "$HERE/../infra/build-server/lib.sh" . "$HERE/../infra/build-server/lib.sh"
# JOBS empty = the box decides: 90 alone, 45 beside another slot holder (remote-run.sh, main's ruling 6 Oct 2026) # JOBS empty = the box decides: 90 alone, 45 beside another slot holder (remote-run.sh, main's ruling 6 Oct 2026)
JOBS="${JOBS:-}"; OUT=""; ARTEFACTS=""; TARGET_DIR="target"; FETCH=1; CARGO_ARGS=(); SELFTEST=0; FULL=0 JOBS="${JOBS:-}"; OUT=""; ARTEFACTS=""; TARGET_DIR="target"; FETCH=1; CARGO_ARGS=(); SELFTEST=0; FULL=0; SHIP=0; GLIBC="${GLIBC:-2.36}"
while [ $# -gt 0 ]; do while [ $# -gt 0 ]; do
case "$1" in case "$1" in
--jobs) JOBS="$2"; shift 2 ;; --jobs) JOBS="$2"; shift 2 ;;
@ -57,6 +66,8 @@ while [ $# -gt 0 ]; do
--target-dir) TARGET_DIR="$2"; shift 2 ;; --target-dir) TARGET_DIR="$2"; shift 2 ;;
--no-fetch) FETCH=0; shift ;; --no-fetch) FETCH=0; shift ;;
--self-test-repro) SELFTEST=1; shift ;; --self-test-repro) SELFTEST=1; shift ;;
--ship) SHIP=1; shift ;;
--glibc) GLIBC="$2"; shift 2 ;;
--full) FULL=1; shift ;; --full) FULL=1; shift ;;
--) shift; CARGO_ARGS=("$@"); break ;; --) shift; CARGO_ARGS=("$@"); break ;;
-h|--help) sed -n '2,32p' "$0"; exit 0 ;; -h|--help) sed -n '2,32p' "$0"; exit 0 ;;
@ -103,21 +114,29 @@ fi
exit 0 exit 0
fi fi
# --ship: the zig path and the target triple dir for the artefacts
SHIP_TARGET=x86_64-unknown-linux-gnu
if [ "$SHIP" = 1 ]; then TARGET_SUB="$SHIP_TARGET/release"; else TARGET_SUB="release"; fi
# defaults per crate # defaults per crate
case "$BS_KIND:$BS_CRATE_REL" in case "$BS_KIND:$BS_CRATE_REL" in
node:*) node:*)
[ -n "${CARGO_ARGS[*]:-}" ] || CARGO_ARGS=(build --release -p kaspad -p igneum-miner --features kaspad/igneum-pow) [ -n "${CARGO_ARGS[*]:-}" ] || CARGO_ARGS=(build --release -p kaspad -p igneum-miner --features kaspad/igneum-pow)
[ -n "$ARTEFACTS" ] || ARTEFACTS="$TARGET_DIR/release/igneumd $TARGET_DIR/release/igneum-miner" ;; [ -n "$ARTEFACTS" ] || ARTEFACTS="$TARGET_DIR/$TARGET_SUB/igneumd $TARGET_DIR/$TARGET_SUB/igneum-miner" ;;
repo:app/igneum-app) repo:app/igneum-app)
[ -n "${CARGO_ARGS[*]:-}" ] || CARGO_ARGS=(build --release) [ -n "${CARGO_ARGS[*]:-}" ] || CARGO_ARGS=(build --release)
[ -n "$ARTEFACTS" ] || ARTEFACTS="$TARGET_DIR/release/igneum-app $TARGET_DIR/release/igneum-ota-sign $TARGET_DIR/release/igneum-prove-verify" ;; [ -n "$ARTEFACTS" ] || ARTEFACTS="$TARGET_DIR/$TARGET_SUB/igneum-app $TARGET_DIR/$TARGET_SUB/igneum-ota-sign $TARGET_DIR/$TARGET_SUB/igneum-prove-verify" ;;
repo:proving/igneum-prove) repo:proving/igneum-prove)
[ -n "${CARGO_ARGS[*]:-}" ] || CARGO_ARGS=(build --release) [ -n "${CARGO_ARGS[*]:-}" ] || CARGO_ARGS=(build --release)
[ -n "$ARTEFACTS" ] || ARTEFACTS="$TARGET_DIR/release/igneum-prove-host $TARGET_DIR/release/igneum-prove-export" ;; [ -n "$ARTEFACTS" ] || ARTEFACTS="$TARGET_DIR/$TARGET_SUB/igneum-prove-host $TARGET_DIR/$TARGET_SUB/igneum-prove-export" ;;
*) *)
[ -n "${CARGO_ARGS[*]:-}" ] || CARGO_ARGS=(build --release) ;; [ -n "${CARGO_ARGS[*]:-}" ] || CARGO_ARGS=(build --release) ;;
esac esac
case "${CARGO_ARGS[0]}" in build) ;; *) [ -n "${ARTEFACTS_SET:-}" ] || { FETCH=0; ARTEFACTS=""; } ;; esac # test, check, clippy: nothing to fetch case "${CARGO_ARGS[0]}" in build) ;; *) [ -n "${ARTEFACTS_SET:-}" ] || { FETCH=0; ARTEFACTS=""; } ;; esac # test, check, clippy: nothing to fetch
if [ "$SHIP" = 1 ]; then
[ "${CARGO_ARGS[0]}" = build ] || bs_die "--ship is for cargo build"
CARGO_ARGS=(zigbuild "${CARGO_ARGS[@]:1}" --target "$SHIP_TARGET.$GLIBC")
BR_TARGET_SHIP="$SHIP_TARGET.$GLIBC"
fi
[ -n "$OUT" ] || OUT="$BS_CRATE/target-remote" [ -n "$OUT" ] || OUT="$BS_CRATE/target-remote"
bs_log "$BS_KIND crate $BS_WT/$BS_CRATE_REL at $BS_SHA ($BS_BRANCH) -> $BS_HOST:$BS_REMOTE_CRATE; cargo ${CARGO_ARGS[*]} -j ${JOBS:-auto}; target dir $TARGET_DIR" bs_log "$BS_KIND crate $BS_WT/$BS_CRATE_REL at $BS_SHA ($BS_BRANCH) -> $BS_HOST:$BS_REMOTE_CRATE; cargo ${CARGO_ARGS[*]} -j ${JOBS:-auto}; target dir $TARGET_DIR"
@ -130,12 +149,12 @@ bs_log "sources in place after $(( $(date +%s) - t_sync0 )) s (every changed fil
# rerun-if-changed and is never run again by cargo (release-0.3.11 plan: `cargo clean -p kaspa-build-info` first); so when # rerun-if-changed and is never run again by cargo (release-0.3.11 plan: `cargo clean -p kaspa-build-info` first); so when
# the commit the box builds differs from the last one built in this target dir, that one crate is cleaned (a relink, seconds) # the commit the box builds differs from the last one built in this target dir, that one crate is cleaned (a relink, seconds)
pre="" pre=""
if [ "$BS_KIND" = node ] && [ "${CARGO_ARGS[0]}" = build ]; then if [ "$BS_KIND" = node ] && { [ "${CARGO_ARGS[0]}" = build ] || [ "${CARGO_ARGS[0]}" = zigbuild ]; }; then
pre="[ \"\$(cat '.build-remote-sha-$TARGET_DIR' 2>/dev/null)\" = '$BS_SHA' ] || CARGO_TARGET_DIR='$TARGET_DIR' cargo clean -q --release -p kaspa-build-info 2>/dev/null; " pre="[ \"\$(cat '.build-remote-sha-$TARGET_DIR' 2>/dev/null)\" = '$BS_SHA' ] || CARGO_TARGET_DIR='$TARGET_DIR' cargo clean -q --release -p kaspa-build-info 2>/dev/null; "
fi fi
cmd="$(bs_repro_env)${pre}CARGO_TARGET_DIR='$TARGET_DIR' cargo $(printf '%q ' "${CARGO_ARGS[@]}")${JOBS:+-j $JOBS} 2>&1 | tee -a '$BS_REMOTE_WT/.build-remote.log'; rc=\${PIPESTATUS[0]}; [ \$rc = 0 ] && echo '$BS_SHA' > '.build-remote-sha-$TARGET_DIR'; ( exit \$rc )" # a subshell exit: the runner reads \$? and still prints its RESULT line cmd="$(bs_repro_env)${pre}CARGO_TARGET_DIR='$TARGET_DIR' cargo $(printf '%q ' "${CARGO_ARGS[@]}")${JOBS:+-j $JOBS} 2>&1 | tee -a '$BS_REMOTE_WT/.build-remote.log'; rc=\${PIPESTATUS[0]}; [ \$rc = 0 ] && echo '$BS_SHA' > '.build-remote-sha-$TARGET_DIR'; ( exit \$rc )" # a subshell exit: the runner reads \$? and still prints its RESULT line
label="$BS_WT/$BS_CRATE_REL cargo ${CARGO_ARGS[*]}" label="$BS_WT/$BS_CRATE_REL cargo ${CARGO_ARGS[*]}"
BR_KIND=$(bs_kind build-remote "${CARGO_ARGS[0]}"); BR_COMMAND="cargo ${CARGO_ARGS[*]}"; BR_TARGET=x86_64-unknown-linux-gnu BR_KIND=$(bs_kind build-remote "$( [ "${CARGO_ARGS[0]}" = zigbuild ] && echo build || echo "${CARGO_ARGS[0]}")"); BR_COMMAND="cargo ${CARGO_ARGS[*]}"; BR_TARGET="${BR_TARGET_SHIP:-x86_64-unknown-linux-gnu}"
for ((i = 0; i < ${#CARGO_ARGS[@]}; i++)); do [ "${CARGO_ARGS[$i]}" = --target ] && BR_TARGET="${CARGO_ARGS[$((i + 1))]:-}"; done for ((i = 0; i < ${#CARGO_ARGS[@]}; i++)); do [ "${CARGO_ARGS[$i]}" = --target ] && BR_TARGET="${CARGO_ARGS[$((i + 1))]:-}"; done
BR_ARTEFACTS=""; [ "$FETCH" = 1 ] && BR_ARTEFACTS="$ARTEFACTS" BR_ARTEFACTS=""; [ "$FETCH" = 1 ] && BR_ARTEFACTS="$ARTEFACTS"
export BR_KIND BR_COMMAND BR_TARGET BR_ARTEFACTS export BR_KIND BR_COMMAND BR_TARGET BR_ARTEFACTS
@ -162,6 +181,8 @@ if [ "$FETCH" = 1 ] && [ -n "$ARTEFACTS" ]; then
bs_log "artefact $dest: $(bs_size "$dest") bytes, sha256 $(bs_sha256 "$dest"), $(file -b "$dest" | cut -c1-60)" bs_log "artefact $dest: $(bs_size "$dest") bytes, sha256 $(bs_sha256 "$dest"), $(file -b "$dest" | cut -c1-60)"
# the commit-string gate (rule of 6 October 2026): a node binary without its commit in its strings fails the run # the commit-string gate (rule of 6 October 2026): a node binary without its commit in its strings fails the run
case "$BS_KIND:$(basename "$dest")" in node:igneumd) "$HERE/ci/commit-string-check.sh" "$dest" "$BS_SHA" || bs_die "commit-string gate failed for $a" ;; esac # only kaspad depends on kaspa-build-info case "$BS_KIND:$(basename "$dest")" in node:igneumd) "$HERE/ci/commit-string-check.sh" "$dest" "$BS_SHA" || bs_die "commit-string gate failed for $a" ;; esac # only kaspad depends on kaspa-build-info
# the glibc ceiling of anything that ships (main, 7 Oct 2026): a seed or a rig refuses a binary needing more than 2.36
if [ "$SHIP" = 1 ]; then "$HERE/ci/glibc-ceiling-check.sh" "$dest" "$GLIBC" || bs_die "glibc ceiling gate failed for $a"; fi
done done
fi fi
bs_wt_unlock bs_wt_unlock

40
tools/ci/glibc-ceiling-check.sh Executable file
View file

@ -0,0 +1,40 @@
#!/usr/bin/env bash
# The glibc ceiling of a shipped Linux binary (main, 7 October 2026: a seed took 14 restarts and three minutes down on a binary
# built natively on Ubuntu 24.04, glibc 2.39, while Debian 12 seeds have 2.36 and HiveOS rigs less). Rule: anything that ships to
# a seed or a rig needs at most GLIBC_2.36; tools/build-remote.sh --ship and tools/workers-remote.sh build with zig for that and run
# this on every artefact they fetch. Reads the versioned symbol needs (`objdump -T`, or `nm -D` where objdump is absent) and compares
# the highest GLIBC_x.y with the ceiling.
#
# tools/ci/glibc-ceiling-check.sh <elf> [ceiling, default 2.36] exit 0 when the need is at or under the ceiling
# tools/ci/glibc-ceiling-check.sh --symbols <file> [ceiling] the same from a saved `objdump -T` text (the self-test, CI without ELF tools)
# tools/ci/glibc-ceiling-check.sh --self-test fires on a 2.38 need against 2.36, passes 2.34 against 2.36 and 2.36 against 2.36
set -euo pipefail
ceiling_of() { grep -oE 'GLIBC_[0-9]+\.[0-9]+' | sed 's/GLIBC_//' | sort -t. -k1,1n -k2,2n | tail -1; }
le() { [ "$(printf '%s\n%s\n' "$1" "$2" | sort -t. -k1,1n -k2,2n | tail -1)" = "$2" ]; } # $1 <= $2 as glibc versions
judge() { # <need> <ceiling> <label>
local need="$1" max="$2" label="$3"
[ -n "$need" ] || { echo "glibc-ceiling: $label needs no versioned glibc symbol (static, or not an ELF): ok"; return 0; }
if le "$need" "$max"; then echo "glibc-ceiling: $label needs GLIBC_$need, ceiling $max: ok"; else echo "glibc-ceiling: $label needs GLIBC_$need, OVER the ceiling $max (a Debian 12 seed or a HiveOS rig refuses it)" >&2; return 1; fi
}
case "${1:-}" in
--self-test)
t=$(mktemp -d); trap 'rm -rf "$t"' EXIT
printf '0000 DF *UND* 0000 GLIBC_2.34 pthread_create\n0000 DF *UND* 0000 GLIBC_2.38 strlcpy\n0000 DF *UND* 0000 GLIBC_2.2.5 malloc\n' > "$t/high.txt"
printf '0000 DF *UND* 0000 GLIBC_2.34 pthread_create\n0000 DF *UND* 0000 GLIBC_2.2.5 malloc\n' > "$t/low.txt"
printf '0000 DF *UND* 0000 GLIBC_2.36 arc4random\n' > "$t/edge.txt"
"$0" --symbols "$t/high.txt" 2.36 >/dev/null 2>&1 && { echo "glibc-ceiling self-test: a 2.38 need PASSED against 2.36 (blind)"; exit 1; }
"$0" --symbols "$t/low.txt" 2.36 >/dev/null || { echo "glibc-ceiling self-test: a 2.34 need FAILED against 2.36"; exit 1; }
"$0" --symbols "$t/edge.txt" 2.36 >/dev/null || { echo "glibc-ceiling self-test: a 2.36 need FAILED against 2.36 (the ceiling is inclusive)"; exit 1; }
"$0" --symbols "$t/high.txt" 2.39 >/dev/null || { echo "glibc-ceiling self-test: a 2.38 need FAILED against 2.39"; exit 1; }
echo "glibc-ceiling self-test: fires on 2.38 against 2.36; passes 2.34 and 2.36 against 2.36, 2.38 against 2.39"; exit 0 ;;
--symbols) judge "$(ceiling_of < "$2")" "${3:-2.36}" "$2" ;;
"") echo "usage: glibc-ceiling-check.sh <elf> [ceiling] | --symbols <file> [ceiling] | --self-test" >&2; exit 2 ;;
*)
f="$1"; max="${2:-2.36}"
[ -f "$f" ] || { echo "glibc-ceiling: no file $f" >&2; exit 2; }
if command -v objdump >/dev/null 2>&1 && objdump -T "$f" >/dev/null 2>&1; then need=$(objdump -T "$f" | ceiling_of)
elif command -v nm >/dev/null 2>&1; then need=$(nm -D "$f" 2>/dev/null | ceiling_of)
elif command -v strings >/dev/null 2>&1; then need=$(strings "$f" | ceiling_of) # the Mac: no ELF objdump; the version strings are in .dynstr
else echo "glibc-ceiling: no objdump, nm or strings to read $f" >&2; exit 2; fi
judge "$need" "$max" "$(basename "$f")" ;;
esac

11
tools/workers-remote.sh
View file

@ -28,13 +28,15 @@ BS_LOCAL_DIRS="proto-cuda proto-opencl"; BS_VENDOR_REPOS=""
bs_log "workers from $BS_WT at $BS_SHA ($BS_BRANCH) -> $BS_HOST:$BS_REMOTE_WT (proto-cuda, proto-opencl)" bs_log "workers from $BS_WT at $BS_SHA ($BS_BRANCH) -> $BS_HOST:$BS_REMOTE_WT (proto-cuda, proto-opencl)"
bs_sync_sources bs_sync_sources
PLACEHOLDER=proto-cuda/packs/igneum-devnet-v4-epoch0 # the OpenCL worker's compile-time pack, as build-workers-linux.sh PLACEHOLDER=proto-cuda/packs/igneum-devnet-v4-epoch0 # the OpenCL worker's compile-time pack, as build-workers-linux.sh
GLIBC="${GLIBC:-2.36}" # the workers ship to HiveOS rigs and Debian 12 seeds: zig targets glibc 2.36 as the Mac's build-workers-linux.sh does; GLIBC=native uses clang (2.39: the box, Ubuntu 24.04 hosts)
if [ "$GLIBC" = native ]; then CXX='clang++ -std=c++17'; CC='clang -std=gnu99'; else CXX="zig c++ -target x86_64-linux-gnu.$GLIBC -std=c++17"; CC="zig cc -target x86_64-linux-gnu.$GLIBC -std=gnu99"; fi
cmd="$(bs_repro_env)"'. /etc/profile.d/igneum-build.sh cmd="$(bs_repro_env)"'. /etc/profile.d/igneum-build.sh
CUDA=$(ls -d /usr/local/cuda-12.* 2>/dev/null | sort -V | tail -1); [ -n "$CUDA" ] || { echo "no CUDA headers under /usr/local/cuda-12.*: provision.sh step_cuda"; exit 2; } CUDA=$(ls -d /usr/local/cuda-12.* 2>/dev/null | sort -V | tail -1); [ -n "$CUDA" ] || { echo "no CUDA headers under /usr/local/cuda-12.*: provision.sh step_cuda"; exit 2; }
[ -f /usr/include/CL/cl.h ] || { echo "no /usr/include/CL/cl.h: apt opencl-c-headers"; exit 2; } [ -f /usr/include/CL/cl.h ] || { echo "no /usr/include/CL/cl.h: apt opencl-c-headers"; exit 2; }
mkdir -p out-workers-box mkdir -p out-workers-box
echo "workers-remote: $(clang++ --version | head -1); CUDA headers $CUDA/include; $(git rev-parse --short HEAD)" echo "workers-remote: '"$CXX"' ($(zig version 2>/dev/null || clang++ --version | head -1)); CUDA headers $CUDA/include; $(git rev-parse --short HEAD)"
clang++ -std=c++17 -O2 -Wall -Wextra -I proto-cuda/nvrtc -I "$CUDA/include" -static-libstdc++ -static-libgcc -o out-workers-box/igneum-worker-cuda proto-cuda/nvrtc/worker.cpp -ldl -lpthread || exit 1 '"$CXX"' -O2 -Wall -Wextra -I proto-cuda/nvrtc -I "$CUDA/include" -static-libstdc++ -static-libgcc -o out-workers-box/igneum-worker-cuda proto-cuda/nvrtc/worker.cpp -ldl -lpthread || exit 1
clang -std=gnu99 -D_GNU_SOURCE -O2 -Wall -Wextra -Wno-format-truncation -DIGNEUM_CL_DYNAMIC -DCL_TARGET_OPENCL_VERSION=120 -I /usr/include -I '"$PLACEHOLDER"' -DIGNEUM_KERNEL_PATH='"'"'"kernel_bound.cl"'"'"' -static-libgcc -o out-workers-box/igneum-worker-opencl proto-opencl/host.c -ldl -lm -lpthread || exit 1 '"$CC"' -D_GNU_SOURCE -O2 -Wall -Wextra -Wno-format-truncation -DIGNEUM_CL_DYNAMIC -DCL_TARGET_OPENCL_VERSION=120 -I /usr/include -I '"$PLACEHOLDER"' -DIGNEUM_KERNEL_PATH='"'"'"kernel_bound.cl"'"'"' -static-libgcc -o out-workers-box/igneum-worker-opencl proto-opencl/host.c -ldl -lm -lpthread || exit 1
for b in igneum-worker-cuda igneum-worker-opencl; do printf "workers-remote: %s glibc ceiling %s, needs %s\n" "$b" "$(objdump -T out-workers-box/$b | grep -oE "GLIBC_[0-9.]+" | sort -V | tail -1)" "$(readelf -d out-workers-box/$b | grep -oE "\[lib[^]]+\]" | tr -d "[]" | tr "\n" " ")"; done' for b in igneum-worker-cuda igneum-worker-opencl; do printf "workers-remote: %s glibc ceiling %s, needs %s\n" "$b" "$(objdump -T out-workers-box/$b | grep -oE "GLIBC_[0-9.]+" | sort -V | tail -1)" "$(readelf -d out-workers-box/$b | grep -oE "\[lib[^]]+\]" | tr -d "[]" | tr "\n" " ")"; done'
BR_KIND=other BR_COMMAND="clang++ worker.cpp; clang host.c" BR_TARGET=x86_64-unknown-linux-gnu BR_ARTEFACTS="out-workers-box/igneum-worker-cuda out-workers-box/igneum-worker-opencl"; export BR_KIND BR_COMMAND BR_TARGET BR_ARTEFACTS BR_KIND=other BR_COMMAND="clang++ worker.cpp; clang host.c" BR_TARGET=x86_64-unknown-linux-gnu BR_ARTEFACTS="out-workers-box/igneum-worker-cuda out-workers-box/igneum-worker-opencl"; export BR_KIND BR_COMMAND BR_TARGET BR_ARTEFACTS
t0=$(date +%s); set +e; bs_remote_run "$BS_REMOTE_WT" "$BS_WT/proto-cuda workers" "$cmd" 2>&1 | tee "/tmp/workers-remote-$$.log"; rc=${PIPESTATUS[0]}; set -e t0=$(date +%s); set +e; bs_remote_run "$BS_REMOTE_WT" "$BS_WT/proto-cuda workers" "$cmd" 2>&1 | tee "/tmp/workers-remote-$$.log"; rc=${PIPESTATUS[0]}; set -e
@ -45,6 +47,7 @@ mkdir -p "$OUT"
for b in igneum-worker-cuda igneum-worker-opencl; do for b in igneum-worker-cuda igneum-worker-opencl; do
bs_rsync -p "$BS_HOST:$BS_REMOTE_WT/out-workers-box/$b" "$OUT/$b" || bs_die "no $b on the box" bs_rsync -p "$BS_HOST:$BS_REMOTE_WT/out-workers-box/$b" "$OUT/$b" || bs_die "no $b on the box"
bs_log "artefact $OUT/$b: $(bs_size "$OUT/$b") bytes, sha256 $(bs_sha256 "$OUT/$b"), $(file -b "$OUT/$b" | cut -c1-50)" bs_log "artefact $OUT/$b: $(bs_size "$OUT/$b") bytes, sha256 $(bs_sha256 "$OUT/$b"), $(file -b "$OUT/$b" | cut -c1-50)"
[ "$GLIBC" = native ] || "$HERE/ci/glibc-ceiling-check.sh" "$OUT/$b" "$GLIBC" || bs_die "glibc ceiling gate failed for $b"
done done
{ printf 'igneum workers, linux x86_64, built on igneum-build-1 with clang++ (static libstdc++) on %s from %s (%s); run-time libraries dlopen-ed\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$BS_SHA" "$BS_BRANCH"; } > "$OUT/version.txt" { printf 'igneum workers, linux x86_64, built on igneum-build-1 (glibc %s, static libstdc++) on %s from %s (%s); run-time libraries dlopen-ed\n' "$GLIBC" "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$BS_SHA" "$BS_BRANCH"; } > "$OUT/version.txt"
bs_wt_unlock bs_wt_unlock