diff --git a/relay/README.md b/relay/README.md index 12be0b3e..38a82d68 100644 --- a/relay/README.md +++ b/relay/README.md @@ -8,7 +8,7 @@ Text, files and tasks between the project lead's devices without Gmail: the Mac, | Tab | Shows | Source | |---|---|---| -| Machines | one card per machine: app and node version, height (daa), synced, hash rate, accepted blocks, peers, faults, power and temperatures, last seen; red after 3 min without an upload | Neon `miner_logs` (the log intake in `site/api/log.mjs`): newest upload per label, the last 20 KB parsed server side (miner `STATUS` lines, node log, the app's `stability:` lines) | +| Machines | one card per machine: app and node version, height (daa), synced, hash rate, accepted blocks, peers, faults, power and temperatures, last seen; red after 3 min without an upload, grey "stopped (quit\|update)" when the app's last upload ends on its own quit lines; a worker card whose STATUS line is over 180 s old is marked stale and left out of the machine total; the OTA state is the newest update line the app logged (downloading, downloaded, staged, installing, updated, failed, current); worker labels nvidia, amd, mac, metal, opencl, other and intel | Neon `miner_logs` (the log intake in `site/api/log.mjs`): newest upload per label, the last 20 KB parsed server side (miner `STATUS` lines, node log, the app's `stability:` lines) | | Jobs | the signed jobs file with per-machine status (queued, running, done + exit code) and the result line; tap a run for the full upload | `igneum-jobs.json` on the downloads host (fetched server side with `DL_TOKEN`), results from `miner_logs` rows whose `run_id` is `job--` | | Builds | the OTA manifest (version, notes, platforms, sizes), the last CI fetch, build events, the downloads folder listing | `igneum-app-latest.json` and `igneum-windows-ci.json` on the downloads host; `console_items` kind `build` (posted by `packaging/windows/fetch-ci-artifacts.sh` and `packaging/ota/publish-manifest.sh`) and key `dl` (`tools/console.mjs sync-dl`) | | Chain | blocks, identities, hash estimate, difficulty, last lock, finality state, peers, blocks per minute sparkline, events, Hetzner results | `https://igneum.network/api/live` fetched server side; `console_items` key `hetzner` (`sync-hetzner`) | @@ -20,7 +20,7 @@ The console function is `api/console.mjs`, reached through the rewrite `/r/` or `mac-`) never reaches the intake, because `app/igneum-app/src/main.rs` names the file with its own `stamp_now()` while `engine.rs` uploads `app-.log`. Until that is fixed the Machines tab has no app version, no `stability:` power and temperature lines and no `status:` lines; everything else comes from the node and miner logs. +The app log (label `win-` or `mac-`) reaches the intake since b8b349a (4 Oct 2026, 0.3.3); apps before that show `app ?`. The parsers (labels, miner tail, app tail, the stale mark) live in `relay/lib/parse.mjs` with no dependencies, and `relay/lib/auth.mjs` holds the constant-time secret compare; `node --test relay/test/parse.test.mjs relay/test/auth.test.mjs` runs their tests, and CI runs them in the site job. ## The secret is the path diff --git a/tools/observer/README.md b/tools/observer/README.md index 2e1c60a8..66d346c1 100644 --- a/tools/observer/README.md +++ b/tools/observer/README.md @@ -16,7 +16,7 @@ Environment, every value optional: | `DATABASE_URL` | read from `~/.config/igneum/env` | Neon connection string. Never commit it. | | `LIVE_RETAIN_HOURS` | `24` | Hours of blocks kept in `live_blocks`. Older rows are deleted once a minute. | | `LIVE_TABLE_PREFIX` | empty | Prefix for every table name, so a test observer against a test network can write `fintest_live_*` without touching the site. | -| `IGNEUM_EVM_RPC` | `http://127.0.0.1:26800` | The execution layer's JSON-RPC (http) of a node on the proving build, for `igneum_getShardPlan`, `igneum_getProofRecords` and `igneum_getProvingStatus`. The default is the Mac app's node; the observer node itself has no EVM listener yet (start it with `--evm-rpclisten=127.0.0.1:` once it runs the proving build and point this at it). A node without the RPCs (method not found) gives `proving = {supported: false}`, rechecked every 5 minutes; an unreachable endpoint is retried every 20 s. | +| `IGNEUM_EVM_RPC` | `http://127.0.0.1:26800` | The execution layer's JSON-RPC (http) of a node on the proving build, for `igneum_getShardPlan`, `igneum_getProofRecords` and `igneum_getProvingStatus`. On the Mac that port is the Igneum Miner app's node, so an app restart (an OTA at its slot minute, a quit) blips the proving feed with `fetch failed` lines until it is back; the observer's own node (`observer-v4`) has no `--evm-rpclisten`. The default is the Mac app's node; the observer node itself has no EVM listener yet (start it with `--evm-rpclisten=127.0.0.1:` once it runs the proving build and point this at it). A node without the RPCs (method not found) gives `proving = {supported: false}`, rechecked every 5 minutes; an unreachable endpoint is retried every 20 s. | A node started by another tool may listen on gRPC only. Then run your own non-mining peer with a JSON listener, on ports that do not clash with the devnet's (gRPC 26610, P2P 26611): @@ -50,6 +50,10 @@ Created on start if missing. | `live_state.proving` | jsonb, updated every 2 s | `supported` (false with `reason` when the node has no proving RPCs or the endpoint is unreachable), `active`, `activation_daa`, `tip_daa`, `verifier`, `pool` (entries, pending, verified, failed), `paid_shards_total`, `shard_budget_pgas`, `blocks_10m`, `blocks_fully_proven_10m`, `shards_proven_10m`, `shards_paid_10m`, `median_proof_lag_s` (median `lag_daa` of the last 10 minutes; the devnet targets one DAA step per second), `provers_10m`, `open_blocks`, `pending_plans`, `evm_rpc`. | | `live_state.finality` | jsonb, updated every 2 s | `params`, `chain_id`, `next_index`, `finality_active`, `latest_locked_index`, `latest_locked_hash`, `latest_locked_blue_score`, `weights` (`total_weight`, `active_weight`, `voters`, `keys[]` with `id`, `blocks`, `voter`, `participation`, `stripped_until_daa`, `revealed`). | +## Keeping it current (autosync) + +`tools/observer/autosync.sh` (loop: `nohup bash tools/observer/autosync.sh &` from the shared checkout; log `/tmp/igneum-devnet/autosync.out`) fetches origin/master every 5 min and fast-forwards the shared checkout when it is clean under `tools/observer` and `site/api`; a failed fast-forward logs git's reason and the dirty files the incoming commits also touch. Whoever moved HEAD (this loop or a pull by hand), the observer is restarted (a `kill`; `run.sh` starts it again in 3 s) whenever the checked-out `observer.mjs` or `run.sh` differs from what the running one started from: the key is the two blob ids hashed, kept in `/tmp/igneum-devnet/observer.tree` and rewritten at each restart. `tools/observer/autosync.sh check` prints the key, the marker and whether a restart is due (exit 3 when it is). A change to `autosync.sh` itself does not restart the observer, but the running loop keeps its old code: replace it by hand (kill it, start it again) after such a change; write the marker first (`git rev-parse HEAD:tools/observer/observer.mjs HEAD:tools/observer/run.sh | tr -d '\n' | shasum | cut -c1-40 > /tmp/igneum-devnet/observer.tree`) when the running observer is already current, or the new loop restarts it once for nothing. Each restart logs `seeded N checkpoint states` and, if a lock is recorded over a seeded state, names that state. + ## Reading it `site/api/live.mjs` serves `/api/live` from these tables in five indexed queries (`proving` from `live_state.proving`; every block carries `shards: [{i, n, state, prover, lag, payout, pgas}]` and `proven`). `LIVE_TABLE_PREFIX` on the API reads a test observer's tables. `site/live.html` polls it every 2 s. The site shows OFFLINE when `live_state.updated_at` is older than 30 s.