igneum/docs/ops/build-queue.md

4.6 KiB

Build queue: the nine build boxes and their standing work

Set up 9 October 2026, 09:1x UK, on the order relayed by the coordinator: every box has one owner lane and a standing queue, a reader on build-1 reads this file every 30 minutes against the merged workers.json and reports an idle box as a fault. The live copy is build-1:/srv/queue/build-queue.md (the reader reads that one; this file is its source on master, landed through the gate).

Rules that bind every entry

  • Kill by recorded pid or pid file only, never by a name or a pattern (pkill and killall refuse on every box, exit 97; tools/ci/kill-by-name-check.sh reads the tree). Every job writes its pid file before it starts; a stop reads that file.
  • A box never builds for the fleet: fleet binaries come only from the cross-build kit (x86-64-v3, the ISA gate at 0 AVX-512 lines).
  • Nothing on the chain by hand; the devnet, hub-1 and every node are the node and fleet lanes' through their own harnesses.
  • An entry carries its owner lane, its box, its pid-file path and its clock (UK). A box with no live pid file from its queue and a load under 1.0 for a 30-minute read is a fault, reported by the reader to the coordinator (/srv/queue/faults.log).
  • A lane that finishes an entry replaces it with the next or hands the box back here with a line; an empty box is the fault.

Owners (one lane per box)

box threads owner lane standing use
build-1 96 build-server lane cuts and kits, the hands (observer-node, node1), the capacity fuzz slices, the workers page, the queue reader
build-2 96 site lane the site gate (Playwright), the scene-parity suites; spare slots for the node lane's suites
build-3 32 node lane the long consensus fuzz and property suites (kaspa-consensus, kaspa-consensus-core)
build-4 96 adversary lane the chip model (OpenROAD, kepler-formal): the 20 to 25 percent floorplan for the converged SPEF row
build-5 32 vCPU research lane TV-02's two independent supply-replay implementations, CPU only
build-6 32 vCPU HEAL lane the long p2p and exec property suites (kaspa-p2p-flows, igneum-exec)
build-7 96 node lane the 2.0.3 flows items: their suites and known-failed tests; the heal-on harness
build-8 96 fleet lane 2.0.3 kit canaries the moment a chain moves; the heal-off harness; the dn4 roll node
build-9 96 steward the board's rows that need only a box (ZKP, EVM, VER fixtures), the pow fuzz on the side

Queue (first entries, 9 October 2026)

# entry owner box pid file clock (UK)
1 2.0.3 flows items: the suites and the known-failed tests on release-2.0.3-node-k6 (bf60948a and after) node lane build-7 /srv/queue/pids/build-7-node-flows.pid from 09:30, continuous
2 long fuzz and property suites, consensus (kaspa-consensus, kaspa-consensus-core) node lane build-3 /srv/queue/pids/build-3-consensus-fuzz.pid from 09:45, continuous
3 long fuzz and property suites, exec and p2p (igneum-exec, kaspa-p2p-flows, kaspa-p2p-lib) HEAL lane build-6 /srv/queue/pids/build-6-exec-p2p-fuzz.pid from 09:45, continuous
4 long fuzz and property suites, pow (kaspa-pow, igneum-pow mixer and scratch) node lane build-9 /srv/queue/pids/build-9-pow-fuzz.pid from 09:45, continuous
5 TV-02: two independent supply-replay implementations, CPU only research lane build-5 /srv/queue/pids/build-5-tv02-replay.pid from 10:00, until both agree
6 chip model: the 20 to 25 percent floorplan for the converged SPEF row (about four host-hours) adversary lane build-4 /srv/queue/pids/build-4-floorplan.pid running (OpenROAD 844081, kepler-formal 1797623), about 13:30
7 the board's rows that need only a box: ZKP, EVM and VER fixtures (the steward names them) steward build-9 /srv/queue/pids/build-9-board-rows.pid from 10:00, by row
8 2.0.3 kit canaries the moment a chain moves (the evidence kits under /srv/workers/fleet) fleet lane build-8 /srv/queue/pids/build-8-kit-canary.pid on the chain's move
9 the site gate and the scene-parity suites site lane build-2 /srv/queue/pids/build-2-site-gate.pid on each landing
10 cuts, kits, the hands, the capacity fuzz slices, the workers page, the queue reader build-server lane build-1 /srv/queue/queue-reader.pid and the cut pid files under the cutter's scratch continuous

A lane starts its entry by writing its pid file (<pid> <start UTC> <label>, the format of the slot files) under /srv/queue/pids on build-1 (or on its own box, mirrored there by the lane) and ends it by removing the file and replacing the entry.