6f5787ed71
Main, 7 October 2026, 15:07 UK: one global exclusive measure flock across unrelated measurements stalled build-1 at load 120 with free slots (a stopped probe held it 5.5 h; an exclusive waiter queued every new shared taker) and build-2 behind a one-core VDF bench. lease.sh (installed at /srv/builds/_bin/lease by provision.sh and by hand on both boxes) takes one flock per core for a pinned measurement and the quiet file for a whole-box one; remote-run.sh takes quiet shared only for unbounded runs, excludes leased cores from its set, and its keeper refreshes the holder file and calls lease reap (a STOPPED holder of a lease, quiet or a slot for 5 minutes is killed with a line in _log/reaped.log). Keepers close the lock descriptors they inherit (an orphaned sleep held a slot and the worktree lock 20 s past the release; the slot self-test had rotted on that since the worktree lock landed). tools/ci/box-locks-check.sh runs lease.sh --self-test and remote-run.sh --self-test-slots on build-1. provision.sh also carries the 16 libraries headless Chromium needs (installed by hand on both boxes at 14:5x UK) and a headless self-test step. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
21 lines
1.4 KiB
Bash
Executable file
21 lines
1.4 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# The box-side lock self-tests, run on igneum-build-1 over ssh (flock, taskset and GNU timeout live there, not on the Mac; the Mac
|
|
# gate carried no slot self-test since the worktree lock landed on 6 October 2026 and the slot test rotted unseen). Both scripts
|
|
# are copied as they are in this tree (never the box's installed copies) into a scratch path and run against scratch lock
|
|
# directories: infra/build-server/lease.sh --self-test (per-core leases, the quiet class, the reaper) and
|
|
# infra/build-server/remote-run.sh --self-test-slots (slots, jobs, quiet beside builds and suites, leased cores excluded).
|
|
#
|
|
# tools/ci/box-locks-check.sh # exit 1 when either self-test fails or the box cannot be reached
|
|
set -euo pipefail
|
|
cd "$(dirname "$0")/../.."
|
|
. infra/build-server/lib.sh
|
|
bs_host 1
|
|
stamp="ci-$$-$(date +%s)"
|
|
scp -q "${BS_SSH_OPTS[@]}" infra/build-server/lease.sh "$BS_HOST:/tmp/lease-$stamp.sh"
|
|
scp -q "${BS_SSH_OPTS[@]}" infra/build-server/remote-run.sh "$BS_HOST:/tmp/rr-$stamp.sh"
|
|
rc=0
|
|
bs_ssh "bash /tmp/lease-$stamp.sh --self-test 2>&1 | grep -E '^lease self-test'; exit \${PIPESTATUS[0]}" || rc=1
|
|
bs_ssh "IGNEUM_REMOTE_RUN_UNDER_TEST=/tmp/rr-$stamp.sh bash /tmp/rr-$stamp.sh --self-test-slots 2>&1 | grep -E '^self-test-slots'; exit \${PIPESTATUS[0]}" || rc=1
|
|
bs_ssh "rm -f /tmp/lease-$stamp.sh /tmp/rr-$stamp.sh" || true
|
|
[ "$rc" = 0 ] && echo "box-locks: the lease tool and the slot runner pass their self-tests on $BS_HOST"
|
|
exit $rc
|