a0eeb725f9
Thirteen failure emails between 15:26 and 16:53 UK. The classes and what closes them: - the box-locks check on a hosted runner (10 runs): closed bybd6fcb88and165e8b35earlier - windows-ci's stale payload-inputs pin (3 runs): closed on master by 4b4e1bc1; update-return's dispatches still carry e69e8a39 - three hosted site jobs on master hung in the tree gate for over two hours (no timeout-minutes): ci.yml now carries site 15, changes 10, pow 60, sims 45, the overlap sweep runs under a 10-minute wall clock where GNU timeout exists, and tools/ci/workflow-timeouts-check.sh fails a job without a budget (self-test: a job without the key, a wrong budget) - a branch merged with no ci run of its own (era-vdf61421005, 16:31 UK): master's igneum-pow suite went red and five docs-only merges landed green over it because their runs skip the compile job. tools/ci/ci-state.mjs reads the runs API through gh (a commit's newest run, master's last COMPILED run, a branch's last red); merge-to-master.sh pushes an unrun branch for a run, waits for a queued one printing the clock, refuses a red one and refuses any merge onto a red master except the declared fix (--fixes-master); the pre-push hook refuses a push to master whose commit, or whose merge's branch parent, has no green run on that exact sha; a feature-branch push prints the branch's previous red first. Self-tests with a fake gh in all three. - ci-red.yml fires on failure, cancelled and timed_out and hands the conclusion to red-watch.mjs, whose line names the kind (CI red, CI cancelled, CI timed out); the self-test reads the workflow file for the three conclusions - tools/ci/retry-once.sh: one retry before red for the box-locks check, the scene parity check and the live public API check (each keeps its own skip line on a runner without the resource) GitHub's branch protection cannot be applied: the organisation is on the free plan and the repository is private (the API answers 403, "Upgrade to GitHub Pro or make this repository public"), so the two scripts are the enforcement; the rule is one line in CLAUDE.md under the CI block. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
3.2 KiB
CI checks
| Check | What it fails | Since |
|---|---|---|
no text overlaps (overlap-check.mjs) |
A served page, or a miner or wallet screen (behind IGNEUM_OVERLAP_APPS=1), where a visible run of text is covered by another element (a pill over a caption, a label over a value, a card over its neighbour, text under the header at rest), clipped by an overflow-hidden ancestor, or past the viewport; a page that scrolls sideways. Five widths, light and dark, the home hero at rest and at each step. A fixture with one deliberate overlap of each kind must be flagged first (--self-test). Needs a headless Chromium: CI installs Playwright; the Mac ships the pages to build-2 (infra/build-server/overlap-browser.sh). |
7 October 2026: the hero's step pill sat on the caption's second line ("Two thirds of the weight sign. The checkpoint locks.") at every desktop width, found by the project lead on the live site |
| master takes only CI-passed commits (ci-state.mjs, merge-to-master.sh, the hook's master_ci_ok) | A push to master whose commit, or whose merge's branch parent, has no green ci run on that exact sha (the runs API through gh: red, queued, none or gh unreachable all refuse); a merge onto a master whose last compiled run is red, unless declared the fix (--fixes-master). The merge tool pushes an unrun branch for a run and waits for a queued one with the clock. A feature-branch push prints the branch's previous red first (--branch-red). | 7 October 2026: era-vdf's tip 0e2d6b1c merged with no ci run; master's igneum-pow suite red from 16:31 UK under five docs-only green merges |
| every workflow job carries timeout-minutes (workflow-timeouts-check.sh) | A job in .github/workflows without timeout-minutes, or a budget off its measured line (site 15, changes 10, pow 60, sims 45). | 7 October 2026: three hosted site jobs on master hung over two hours each in the tree gate; the six-hour default was the only stop |
| a box or network check gets one retry (retry-once.sh) | Nothing by itself: wraps the box-locks check, the scene parity check and the live public API check so a first failure is printed and retried once; the second is the verdict. The checks keep their own skip line on a runner without the resource. | 7 October 2026 |
| the red watcher fires on cancelled and timed-out runs too (ci-red.yml, red-watch.mjs) | The watcher's if missing any of failure, cancelled, timed_out, or the conclusion not handed to the record step (the self-test reads the workflow file); the line names the kind: CI red, CI cancelled, CI timed out. | 7 October 2026 |
| kill by exact command or pid file (owed as a check) | 6 October 2026, 21:09Z: a Mac-side pkill -f <log file name> matched nothing (the log name was a redirect, not part of the command line), the roll-everything script lived on and wiped a box it had been told to hold. Rule: a job is stopped by its pid file (tools/fleet/fleet-bg.sh start|stop <name>) or by a pattern anchored on its exact command line (^python3 -u /root/fleet/in/box-prover.py), never by a word that may or may not appear in it. The check that flags a pkill -f/pgrep -f whose literal is a path or a name that never starts a command line is owed to the CI lane |