CI on every push: igneum-pow tests, census build, simulator quick modes, site build + link check, identity grep
GitHub Actions workflow (.github/workflows/ci.yml) on push and pull_request with three jobs on the free runners: igneum-pow `cargo test --release` and the igneum-census build; the two Python simulators' --quick modes under a 120-second timeout; the site build, an internal link check of site/*.html (tools/ci/link-check.mjs) and a gh-free identity grep of the public export list (tools/ci/identity-check.sh over tools/ci/forbidden-strings.txt: machine names, LAN and overlay addresses, home paths, local time zones, the log-intake key pattern; never a key or a name). The node fork is too big for CI today and the workflow says so. sim/finality_v2.py --quick is now a genuine smoke run (one day or hour per scenario, one partition and one eclipse setting): 149 s at nice 19 on a loaded Mac, was 745 s. sim/difficulty/sim.py gains --quick (up50 and warmup-hard, kaspa and igneum controllers, 36 s). One bench-log time-zone label reworded so the identity grep passes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
igneum-josh
parent
6461540057
commit
33dba682f0
7 changed files with 210 additions and 11 deletions
63
.github/workflows/ci.yml
vendored
Normal file
63
.github/workflows/ci.yml
vendored
Normal file
|
|
@ -0,0 +1,63 @@
|
|||
# CI on every push and pull request (private repository, free runner minutes).
|
||||
#
|
||||
# What runs: the lottery-hash crate's tests (igneum-pow, release profile), the census tool's build, the two Python
|
||||
# simulators' --quick modes (each under two minutes), the site build with an internal link check, and the gh-free
|
||||
# identity grep of the public export list (tools/ci/forbidden-strings.txt).
|
||||
#
|
||||
# What does not run, on purpose: the node fork (vendor/igneum-node*, a rusty-kaspa fork of about 500 crates with
|
||||
# rocksdb, blst and the execution layer) is gitignored here and too big for the free runners today (a cold build is
|
||||
# 20 to 55 minutes on 2 to 8 vCPU, docs/bench-log.md). The workflow builds igneum-pow only; the fork's own tests run
|
||||
# on the Mac and the seed node (infra/seed-nodes, infra/fast-time).
|
||||
name: ci
|
||||
on:
|
||||
push:
|
||||
pull_request:
|
||||
jobs:
|
||||
pow:
|
||||
name: igneum-pow tests, igneum-census build
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: toolchain
|
||||
run: rustc --version && cargo --version
|
||||
- name: igneum-pow tests (release)
|
||||
working-directory: igneum-pow
|
||||
run: cargo test --release
|
||||
- name: igneum-census build (release)
|
||||
working-directory: igneum-census
|
||||
run: cargo build --release
|
||||
sims:
|
||||
name: simulators, quick modes
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.12'
|
||||
- run: python3 -m pip install --quiet numpy
|
||||
- name: finality_v2.py --quick (under two minutes)
|
||||
working-directory: sim
|
||||
run: time timeout 120 python3 finality_v2.py --quick > finality_quick.md
|
||||
- name: difficulty/sim.py --quick (under two minutes)
|
||||
working-directory: sim/difficulty
|
||||
run: time timeout 120 python3 sim.py --quick > difficulty_quick.md
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: sim-quick-output
|
||||
path: |
|
||||
sim/finality_quick.md
|
||||
sim/difficulty/difficulty_quick.md
|
||||
site:
|
||||
name: site build, link check, identity grep
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '22'
|
||||
- name: site build
|
||||
run: node site/build.mjs
|
||||
- name: internal link check of site/*.html
|
||||
run: node tools/ci/link-check.mjs
|
||||
- name: identity grep of the public export list
|
||||
run: bash tools/ci/identity-check.sh
|
||||
|
|
@ -39,6 +39,7 @@ variation (Poisson alone gives 0.129 at 60 blocks per minute).
|
|||
|
||||
python3 sim.py synthetic set, every controller
|
||||
python3 sim.py --ts-noise 500 the same with +-500 ms timestamp jitter
|
||||
python3 sim.py --quick CI smoke run: up50 and warmup-hard, kaspa and igneum (about a minute)
|
||||
python3 sim.py --tune parameter sweep for the Igneum rule
|
||||
python3 sim.py --record devnet-2026-10-03.csv --events 2026-10-03T19:28:05+00:00,2026-10-03T19:42:40+00:00
|
||||
exact replay check of Kaspa's rule on the record,
|
||||
|
|
|
|||
|
|
@ -715,7 +715,14 @@ def main():
|
|||
ap.add_argument("--events", default="", help="real record: comma list of ISO times of miner events")
|
||||
ap.add_argument("--dump", default="", help="profile:controller, write per-block trajectory to stdout as CSV and exit")
|
||||
ap.add_argument("--custom", default="", help="extra profile: name=genesis_bits_hex,base_MHs,duration_s,t_s:factor,t_s:factor,...")
|
||||
ap.add_argument("--quick", action="store_true",
|
||||
help="smoke run for CI: profiles up50 and warmup-hard, controllers kaspa and igneum (about a minute); --profiles and --controllers still override")
|
||||
args = ap.parse_args()
|
||||
if args.quick:
|
||||
if args.profiles == "all":
|
||||
args.profiles = "up50,warmup-hard"
|
||||
if args.controllers == "kaspa,monero,lwma60,lwma120,igneum,igneum-literal":
|
||||
args.controllers = "kaspa,igneum"
|
||||
names = args.controllers.split(",")
|
||||
profiles = synthetic_profiles(args.seed)
|
||||
if args.profiles != "all":
|
||||
|
|
|
|||
|
|
@ -809,16 +809,17 @@ def scenario_e(args):
|
|||
out = ["### E. Partition: honest weight split across sides for a set time, then healed", ""]
|
||||
configs = [P(denom="active", pmode="cert", delay=args.delay), P(denom="active", pmode="seen", delay=args.delay),
|
||||
P(denom="total", delay=args.delay)]
|
||||
splits = [("50/50", [0.5, 0.5]), ("33/33/34", [0.33, 0.33, 0.34])]
|
||||
q = getattr(args, "quick", False)
|
||||
splits = [("50/50", [0.5, 0.5])] if q else [("50/50", [0.5, 0.5]), ("33/33/34", [0.33, 0.33, 0.34])]
|
||||
rows_conf = []
|
||||
rows_first = []
|
||||
for name, fr in splits:
|
||||
for att in (0.0, 0.34):
|
||||
for dur in (30, 90, 150):
|
||||
for att in ((0.0,) if q else (0.0, 0.34)):
|
||||
for dur in ((30,) if q else (30, 90, 150)):
|
||||
rc = [name, pct(att, 0), dur]
|
||||
rf = [name, pct(att, 0), dur]
|
||||
for p in configs:
|
||||
r = run_partition(args.seed, fr, att, dur, p)
|
||||
r = run_partition(args.seed, fr, att, dur, p, **({"pre_min": 15, "post_min": 30} if q else {}))
|
||||
rc.append("%d%s" % (r["conflicts"], "" if r["conflicts"] == 0 else " (first at %s min)" % fm(r["first_conflict_min"])))
|
||||
rf.append(" / ".join(fm(x) for x in r["side_first_lock"]) + " ; post-heal stalls %d" % r["post_stalls"])
|
||||
rows_conf.append(rc)
|
||||
|
|
@ -930,8 +931,9 @@ def scenario_f(args):
|
|||
out = ["### F. Eclipse of one pool holding 20% of weight", ""]
|
||||
configs = [P(denom="active", pmode="cert", delay=args.delay), P(denom="active", pmode="seen", delay=args.delay),
|
||||
P(denom="total", delay=args.delay)]
|
||||
q = getattr(args, "quick", False)
|
||||
rows = []
|
||||
for dur in (1, 2, 4):
|
||||
for dur in ((1,) if q else (1, 2, 4)):
|
||||
for p in configs:
|
||||
r = run_eclipse(args.seed, dur, False, p)
|
||||
rows.append([dur, p.label(), "%.3f" % r["min_part"], fm(r["drop_min"]), fm(r["recover_min"]), r["conflicts"],
|
||||
|
|
@ -945,7 +947,7 @@ def scenario_f(args):
|
|||
configs2 = configs + [P(denom="active", pmode="cert", floor=0.8, delay=args.delay),
|
||||
P(denom="active", pmode="cert", floor=0.85, delay=args.delay),
|
||||
P(denom="active", pmode="cert", floor=1.0, delay=args.delay)]
|
||||
for dur in (1, 2, 4):
|
||||
for dur in ((1,) if q else (1, 2, 4)):
|
||||
for p in configs2:
|
||||
r = run_eclipse(args.seed, dur, True, p)
|
||||
rows.append([dur, p.label(), "%.3f" % r["min_part"], fm(r["recover_min"]), r["conflicts"], fm(r["first_conflict_min"]),
|
||||
|
|
@ -1427,14 +1429,17 @@ def main(argv=None):
|
|||
args = ap.parse_args(argv)
|
||||
set_floor(args.floor)
|
||||
q = args.quick
|
||||
args.days_a = 3 if q else 60
|
||||
# --quick is the smoke run: every scenario end to end at a length that finishes in under two minutes on a
|
||||
# 2-vCPU CI runner (the full set takes about an hour). The numbers it prints are not the results of
|
||||
# results_v2.md; only the full run is.
|
||||
args.days_a = 1 if q else 60
|
||||
args.days_delay = 1 if q else 3
|
||||
args.days_b = 4 if q else 35
|
||||
args.hours_c = 3 if q else 6
|
||||
args.hours_c_total = 3 if q else 72
|
||||
args.days_b = 1 if q else 35
|
||||
args.hours_c = 1 if q else 6
|
||||
args.hours_c_total = 1 if q else 72
|
||||
args.days_d35 = 1 if q else 3
|
||||
args.days_d50 = 1 if q else 12
|
||||
args.days_g = 3 if q else 25
|
||||
args.days_g = 1 if q else 25
|
||||
print("# finality_v2 output")
|
||||
print()
|
||||
p = P()
|
||||
|
|
|
|||
20
tools/ci/forbidden-strings.txt
Normal file
20
tools/ci/forbidden-strings.txt
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
# Patterns that must never appear in the public export (grep -E, one per line, # comments ignored).
|
||||
# This is the committed, non-secret subset of the public mirror's identity list (igneum-public/tools/identity.local,
|
||||
# which stays private because its remaining entries would themselves name what must stay out). Machine names, LAN
|
||||
# and overlay addresses, home paths, local time zones and the log-intake key pattern. Never a key, never a name.
|
||||
# Checked by tools/ci/identity-check.sh over the export list of igneum-public/tools/sync.sh after its generic scrub.
|
||||
DESKTOP-[A-Z0-9]{7}
|
||||
MacBook
|
||||
192\.168\.
|
||||
100\.[0-9]+\.[0-9]+\.[0-9]+
|
||||
\+0100
|
||||
\bBST\b
|
||||
/Users/
|
||||
C:\\Users
|
||||
~/Desktop
|
||||
log-intake
|
||||
LOG_INTAKE
|
||||
intake[_-]?key
|
||||
Tailscale
|
||||
tailscale
|
||||
ts\.net
|
||||
63
tools/ci/identity-check.sh
Executable file
63
tools/ci/identity-check.sh
Executable file
|
|
@ -0,0 +1,63 @@
|
|||
#!/usr/bin/env bash
|
||||
# Identity grep of the public export list, gh-free, for CI (tools/ci/forbidden-strings.txt).
|
||||
#
|
||||
# Copies the paths that igneum-public/tools/sync.sh publishes into a temporary directory, applies the same generic
|
||||
# scrub that sync.sh applies (model names for machines, <lan-ip> for LAN addresses, ~ for home paths, UTC stamps),
|
||||
# then greps the result with the committed pattern list. A hit means a change would reach the public mirror with
|
||||
# a machine name, a LAN address, a home path or the log-intake key pattern that the generic scrub does not catch.
|
||||
# The private rules of the mirror (sync.local.sed, identity.local) are not here; they run at export time.
|
||||
#
|
||||
# tools/ci/identity-check.sh # exit 1 on any hit, with file:line
|
||||
set -euo pipefail
|
||||
HERE="$(cd "$(dirname "$0")" && pwd)"
|
||||
REPO="$(cd "$HERE/../.." && pwd)"
|
||||
PATTERNS="$HERE/forbidden-strings.txt"
|
||||
|
||||
# The export list of igneum-public/tools/sync.sh (keep in step with it).
|
||||
DIRS=(docs/spec docs/analysis sim igneum-pow igneum-census tools/harness proto-cuda/packs docs/benchmarks tools/finality-attacks tools/exec-attacks)
|
||||
FILES=(docs/provenance.md docs/bench-log.md docs/evidence.md proto-cuda/README.md proto-cuda/CHECKLIST.md proto-cuda/host.cu proto-cuda/build.sh
|
||||
proto-cuda/build.bat proto-cuda/.gitignore proto-cuda/emu/emu.sh proto-cuda/emu/shim.cpp proto-cuda/emu/cuda_runtime.h proto-metal/README.md
|
||||
proto-metal/MEMHARD.md proto-metal/TESTS.md proto-metal/main.swift proto-opencl/README.md proto-opencl/WAVEFRONT.md proto-opencl/host.c
|
||||
proto-opencl/build.sh proto-opencl/build.bat proto-opencl/.gitignore proto-opencl/emu/emu.sh proto-opencl/emu/emu_main.cpp proto-opencl/emu/emu_opencl.h)
|
||||
|
||||
TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT
|
||||
for d in "${DIRS[@]}"; do [ -d "$REPO/$d" ] && { mkdir -p "$TMP/$(dirname "$d")"; cp -R "$REPO/$d" "$TMP/$d"; }; done
|
||||
for f in "${FILES[@]}"; do [ -f "$REPO/$f" ] && { mkdir -p "$TMP/$(dirname "$f")"; cp "$REPO/$f" "$TMP/$f"; }; done
|
||||
# prune what sync.sh prunes
|
||||
find "$TMP" \( -name target -o -name __pycache__ -o -name out -o -name 'build-*' -o -name node_modules -o -name results -o -name runs \) -prune -exec rm -rf {} + 2>/dev/null || true
|
||||
find "$TMP" \( -name .DS_Store -o -name '*.pyc' \) -type f -delete
|
||||
|
||||
# the generic scrub of sync.sh step 3 (its public half; the rules are public text, not secrets)
|
||||
TEXT_FILES="$(find "$TMP" -type f \( -name '*.md' -o -name '*.rs' -o -name '*.py' -o -name '*.mjs' -o -name '*.sh' -o -name '*.bat' \
|
||||
-o -name '*.c' -o -name '*.cu' -o -name '*.cl' -o -name '*.h' -o -name '*.cpp' -o -name '*.swift' -o -name '*.metal' -o -name '*.json' \
|
||||
-o -name '*.csv' -o -name '*.toml' -o -name '*.txt' \) -print)"
|
||||
while IFS= read -r f; do
|
||||
[ -n "$f" ] || continue
|
||||
perl -pi -e '
|
||||
s/the PC node at 192\.168\.[0-9.]+/the RTX 5090 node on the LAN/g;
|
||||
s/\bthe PC node\b/the RTX 5090 node/g;
|
||||
s/\bWindows PC\b/an RTX 5090 on Windows/g;
|
||||
s/\bthe PC\x27s\b/the RTX 5090 machine\x27s/g;
|
||||
s/\bthe PC\b/the RTX 5090 machine/g;
|
||||
s/\bPC (joins|start|period)\b/RTX 5090 $1/g;
|
||||
s/192\.168\.[0-9]+\.[0-9]+/<lan-ip>/g;
|
||||
s/DESKTOP-[A-Z0-9]{7}/<pc-hostname>/g;
|
||||
s/~\/Desktop\//`/g; s/``/`/g;
|
||||
s/~\/\.cargo\/bin\/cargo/cargo/g;
|
||||
s/\/Users\/[A-Za-z0-9_.-]+/~/g;
|
||||
s/C:\\Users\\[A-Za-z0-9_.-]+/%USERPROFILE%/g;
|
||||
s/(\d{1,2}:\d{2}(?::\d{2})? UTC) = \d{1,2}:\d{2} B[S]T/$1/g;
|
||||
s/(\d{1,2}):(\d{2})(:\d{2})? to (\d{1,2}):(\d{2})(:\d{2})? B[S]T/sprintf("%02d:%s%s to %02d:%s%s UTC",($1+23)%24,$2,$3\/\/"",($4+23)%24,$5,$6\/\/"")/ge;
|
||||
s/(\d{1,2}):(\d{2})(:\d{2})? B[S]T/sprintf("%02d:%s%s UTC",($1+23)%24,$2,$3\/\/"")/ge;
|
||||
' "$f"
|
||||
done <<< "$TEXT_FILES"
|
||||
perl -pi -e 's/\(Mac side only;/(Apple M5 Max side only;/g; s/\bthe Mac\x27s\b/the Apple M5 Max\x27s/g; s/\bthe Mac\b/the Apple M5 Max/g;' "$TMP/docs/bench-log.md" 2>/dev/null || true
|
||||
|
||||
PAT="$(grep -vE '^\s*(#|$)' "$PATTERNS")"
|
||||
HITS="$(grep -rEn -f <(printf '%s\n' "$PAT") "$TMP" || true)"
|
||||
if [ -n "$HITS" ]; then
|
||||
echo "identity grep: HITS in the public export list (after the generic scrub):"
|
||||
printf '%s\n' "$HITS" | sed "s#^$TMP/##" | cut -c1-200
|
||||
exit 1
|
||||
fi
|
||||
echo "identity grep: 0 hits over $(printf '%s\n' "$TEXT_FILES" | grep -c .) files"
|
||||
40
tools/ci/link-check.mjs
Normal file
40
tools/ci/link-check.mjs
Normal file
|
|
@ -0,0 +1,40 @@
|
|||
// Internal link check of the site: every href="..." and src="..." in site/*.html that is not an external URL,
|
||||
// an anchor, a mailto or a data URI must resolve to a file under site/ (clean URLs: /bench -> bench.html).
|
||||
// Fragment links (#id) inside a page must name an element id in that page.
|
||||
// node tools/ci/link-check.mjs exit 1 on the first broken link, listing every one
|
||||
import { readFileSync, readdirSync, existsSync, statSync } from 'node:fs';
|
||||
import { join, dirname } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const site = join(dirname(fileURLToPath(import.meta.url)), '..', '..', 'site');
|
||||
const pages = readdirSync(site).filter(f => f.endsWith('.html'));
|
||||
const broken = [];
|
||||
let checked = 0;
|
||||
|
||||
function resolves(target) {
|
||||
const clean = target.replace(/[?#].*$/, '');
|
||||
if (clean === '' || clean === '/') return true;
|
||||
const rel = clean.replace(/^\//, '');
|
||||
const candidates = [join(site, rel), join(site, `${rel}.html`), join(site, rel, 'index.html')];
|
||||
return candidates.some(c => existsSync(c) && statSync(c).isFile());
|
||||
}
|
||||
|
||||
for (const page of pages) {
|
||||
const html = readFileSync(join(site, page), 'utf8');
|
||||
const ids = new Set([...html.matchAll(/\sid="([^"]+)"/g)].map(m => m[1]));
|
||||
for (const m of html.matchAll(/\s(?:href|src)="([^"]*)"/g)) {
|
||||
const t = m[1];
|
||||
if (/^(https?:|mailto:|data:|tel:|javascript:)/i.test(t) || t.startsWith('//')) continue;
|
||||
checked++;
|
||||
if (t.startsWith('#')) { if (t.length > 1 && !ids.has(t.slice(1))) broken.push(`${page}: fragment ${t}`); continue; }
|
||||
const [path, frag] = t.split('#');
|
||||
if (!resolves(path)) { broken.push(`${page}: ${t}`); continue; }
|
||||
if (frag) {
|
||||
const rel = path.replace(/[?].*$/, '').replace(/^\//, '');
|
||||
const file = [join(site, rel), join(site, `${rel}.html`), join(site, rel, 'index.html')].find(c => existsSync(c) && statSync(c).isFile());
|
||||
if (file && file.endsWith('.html') && !new Set([...readFileSync(file, 'utf8').matchAll(/\sid="([^"]+)"/g)].map(x => x[1])).has(frag)) broken.push(`${page}: ${t} (no id ${frag})`);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (broken.length) { console.error(`link check: ${broken.length} broken internal link(s) of ${checked}:`); for (const b of broken) console.error(` ${b}`); process.exit(1); }
|
||||
console.log(`link check: ${checked} internal links across ${pages.length} pages, 0 broken`);
|
||||
Loading…
Reference in a new issue