# Igneum devnet observer

Watches one Igneum node over wRPC JSON and writes what it sees to Neon, so `/api/live` and `/live` on the site can show the devnet as it runs. Node 22 or newer, no dependencies.

## Run

```
node tools/observer/observer.mjs
```

Environment, every value optional:

| Variable | Default | Meaning |
|---|---|---|
| `IGNEUM_RPC` | `ws://127.0.0.1:28610` | The node's wRPC JSON url. 28610 is the igneum-devnet wRPC JSON port (`consensus/core/src/network.rs`). Start the node with `--rpclisten-json=127.0.0.1:28610` or the port of your choice. |
| `DATABASE_URL` | read from `~/.config/igneum/env` | Neon connection string. Never commit it. |
| `LIVE_RETAIN_HOURS` | `24` | Hours of blocks kept in `live_blocks`. Older rows are deleted once a minute. |
| `LIVE_TABLE_PREFIX` | empty | Prefix for every table name, so a test observer against a test network can write `fintest_live_*` without touching the site. |
| `IGNEUM_EVM_RPC` | `http://127.0.0.1:26800` | The execution layer's JSON-RPC (http) of a node on the proving build, for `igneum_getShardPlan`, `igneum_getProofRecords` and `igneum_getProvingStatus`. On the Mac that port is the Igneum Miner app's node, so an app restart (an OTA at its slot minute, a quit) blips the proving feed with `fetch failed` lines until it is back; the observer's own node (`observer-v4`) has no `--evm-rpclisten`. The default is the Mac app's node; the observer node itself has no EVM listener yet (start it with `--evm-rpclisten=127.0.0.1:<port>` once it runs the proving build and point this at it). A node without the RPCs (method not found) gives `proving = {supported: false}`, rechecked every 5 minutes; an unreachable endpoint is retried every 20 s. |

A node started by another tool may listen on gRPC only. Then run your own non-mining peer with a JSON listener, on ports that do not clash with the devnet's (gRPC 26610, P2P 26611):

```
vendor/igneum-node/target/release/igneumd --devnet --nodnsseed --disable-upnp \
  --appdir=/tmp/igneum-obsnode --rpclisten=127.0.0.1:26640 --rpclisten-json=127.0.0.1:28640 \
  --listen=127.0.0.1:26641 --connect=127.0.0.1:26611 --nologfiles
IGNEUM_RPC=ws://127.0.0.1:28640 node tools/observer/observer.mjs
```

## What it does

- Subscribes to `blockAdded` and `virtualChainChanged`; polls `getBlockDagInfo`, `getInfo`, `getConnectedPeerInfo`, `getSinkBlueScore` and `estimateNetworkHashesPerSecond` every 2 s.
- Decodes the miner address from the coinbase payload script (same bech32 variant as `crypto/addresses`). The fork's `vote_key_hash` header field is stored per block; the first 8 hex characters are the miner's short id on the site.
- `engine` is the miner's tag in the coinbase extra data after the node's version prefix. The node exposes no engine name over RPC, so this is null on devnet v0.
- When the node refuses the hash-rate estimate (it needs a 1,000-block window) the observer reports blue work added per second over the last 10 minutes instead.
- Proving v0 (4 Oct 2026, spec 7.7): every chain block (the `isChainBlock` flag of a new block, or a `virtualChainChanged` addition) has its shard plan read over `IGNEUM_EVM_RPC` as `{blockHash}` (the chain block hash is the same hash on both layers), one `live_proofs` row per shard in state `planned`; a plan the EVM node has not executed yet is retried with a growing delay (up to 30 tries). The proof records of the chain blocks of the last 10 minutes are polled in rotation (80 blocks per 2 s tick while active, 10 before activation, four calls in flight); a shard moves to `proving` (a record in the node's pool), `verified` (the SP1 proof verified by the node's verifier, or the record carried by a block and checked by consensus, which is what happens before activation) or `paid` (a carrying segment paid it). `lag_daa` is the carrier's DAA score minus the block's; `prover` is the first 8 hex characters of the record's vote key hash. A block whose every shard is paid, or older than 10 minutes, leaves the rotation. Events: `proving` (activation reached, first paid shard seen) and `prover_seen` (one per prover per run). A reorg drops the removed chain blocks from the rotation.
- Explorer (5 Oct 2026, `docs/plans/explorer.md`): every block row also carries what `/explorer`, `/block/<hash>` and `/address/<addr>` show, all read from the `blockAdded` notification itself (no extra RPC per block, measured: 282 against 283 wRPC calls per minute, 785 against 776 EVM calls in the same minute, before and after): `tx_count` (EVM transactions in the block), `evm_miner` (the coinbase's `IGNA` payout address, else the vote key hash's low 20 bytes as `consensus/core/src/evm.rs` falls back to), `proof_records` (records in the `IGNP` section, 274 bytes each), `subsidy_sompi` (the `E(daa)` the payload declares), `paid_sompi` (the coinbase outputs' sum), `selected_parent`, `number` (the chain block number, filled in when the shard plan arrives) and `detail` (header fields, mergeset, coinbase outputs, EVM transaction hashes, certificate indices). `live_state.rpc_load` counts this process's RPC calls per minute; `live_state.supply_check` is the hourly comparison of the newest 500 blocks with the emission rule (`site/lib/emission.mjs`, spec 2.5): the declared subsidy against `blockSubsidy(daa, 1)`, and each block's outputs against the declared subsidies of the blocks it merges. The observer now imports `site/lib/emission.mjs` and `site/lib/eth.mjs` (keccak for transaction hashes); autosync restarts only on `observer.mjs` and `run.sh` changes, so a change to those two libraries needs a restart by hand.
- Finality v2 (3 Oct 2026): subscribes to `FinalityLock` (the node's lock event) and polls `getFinalityCheckpoints` every 2 s and `getFinalityWeights` every 10 s. Every checkpoint the node reports is upserted into `live_checkpoints`; a checkpoint turning `locked` writes the event `checkpoint N locked (xx% of weight, yy% of active, v votes of n voters) at block h`. The weights snapshot (total, active, per key) goes into `live_state.finality`. A node from before the finality layer answers the RPC with an error; the observer then logs once and skips finality.

## Tables

Created on start if missing.

| Table | Rows | Columns |
|---|---|---|
| `live_blocks` | one per block, kept `LIVE_RETAIN_HOURS` | `hash`, `blue_score`, `daa_score`, `timestamp_ms`, `parents` (count), `parent_hashes`, `is_chain_block`, `vote_key_hash`, `miner_address`, `engine`, `received_at`, `color`; explorer: `tx_count`, `evm_miner`, `proof_records`, `subsidy_sompi`, `paid_sompi`, `selected_parent`, `number`, `detail` (jsonb). Indexes on `received_at`, `timestamp_ms`, `(vote_key_hash, received_at)`, `(evm_miner, received_at)`, `(miner_address, received_at)`, `number`. |
| `live_state` | one row, updated every 2 s | `block_count`, `header_count`, `blue_score`, `difficulty`, `hashes_per_second_estimate`, `peers`, `mempool`, `node_version`, `network`, `blocks_60s`, `blocks_per_minute` (60 pairs of minute epoch ms and count), `observer_started_at`, `updated_at` |
| `live_events` | one per event, kept 7 days | `ts`, `kind`, `text`. Kinds: `observer`, `miner_seen`, `miner_quiet`, `miner_back`, `peer_joined`, `peer_left`, `difficulty` (step over 5%), `checkpoint_locked`. |
| `live_checkpoints` | one per checkpoint index, kept 7 days | `index`, `hash`, `blue_score`, `daa_score`, `state` (proposed, certified, locked), `signed_weight`, `active_weight`, `total_weight`, `fraction_active`, `fraction_total`, `votes_seen`, `voters`, `aggregators` (key hashes whose sortition proof made them aggregators), `locked_at`, `first_seen_at`, `updated_at`. |
| `live_proofs` | one per planned shard of a chain block, kept `LIVE_RETAIN_HOURS` | `block_hash`, `shard`, `shards` (in the plan), `block_number`, `block_daa`, `block_ts`, `pgas`, `state` (planned, proving, verified, paid), `prover` (id8), `verified`, `carried_by`, `carrier_number`, `carrier_daa`, `lag_daa`, `payout_wei`, `received_at`, `updated_at`. Primary key `(block_hash, shard)`, index on `received_at`. |
| `live_state.proving` | jsonb, updated every 2 s | `supported` (false with `reason` when the node has no proving RPCs or the endpoint is unreachable), `active`, `activation_daa`, `tip_daa`, `verifier`, `pool` (entries, pending, verified, failed), `paid_shards_total`, `shard_budget_pgas`, `blocks_10m`, `blocks_fully_proven_10m`, `shards_proven_10m`, `shards_paid_10m`, `median_proof_lag_s` (median `lag_daa` of the last 10 minutes; the devnet targets one DAA step per second), `provers_10m`, `open_blocks`, `pending_plans`, `evm_rpc`. |
| `live_state.finality` | jsonb, updated every 2 s | `params`, `chain_id`, `next_index`, `finality_active`, `latest_locked_index`, `latest_locked_hash`, `latest_locked_blue_score`, `weights` (`total_weight`, `active_weight`, `voters`, `keys[]` with `id`, `blocks`, `voter`, `participation`, `stripped_until_daa`, `revealed`). |

## Keeping it current (autosync)

`tools/observer/autosync.sh` (loop: `nohup bash tools/observer/autosync.sh &` from the shared checkout; log `/tmp/igneum-devnet/autosync.out`) fetches origin/master every 5 min and fast-forwards the shared checkout when it is clean under `tools/observer` and `site/api`; a failed fast-forward logs git's reason and the dirty files the incoming commits also touch. Whoever moved HEAD (this loop or a pull by hand), the observer is restarted (a `kill`; `run.sh` starts it again in 3 s) whenever the checked-out `observer.mjs` or `run.sh` differs from what the running one started from: the key is the two blob ids hashed, kept in `/tmp/igneum-devnet/observer.tree` and rewritten at each restart. `tools/observer/autosync.sh check` prints the key, the marker and whether a restart is due (exit 3 when it is). A change to `autosync.sh` itself does not restart the observer, but the running loop keeps its old code: replace it by hand (kill it, start it again) after such a change; write the marker first (`git rev-parse HEAD:tools/observer/observer.mjs HEAD:tools/observer/run.sh | tr -d '\n' | shasum | cut -c1-40 > /tmp/igneum-devnet/observer.tree`) when the running observer is already current, or the new loop restarts it once for nothing. Each restart logs `seeded N checkpoint states` and, if a lock is recorded over a seeded state, names that state.

## Reading it

`site/api/stats.mjs`, `site/api/supply.mjs` and `site/api/explorer.mjs` serve `/api/stats`, `/api/supply` and `/api/explorer` (docs/api/public-stats.md). `site/api/live.mjs` serves `/api/live` from these tables in five indexed queries (`proving` from `live_state.proving`; every block carries `shards: [{i, n, state, prover, lag, payout, pgas}]` and `proven`). `LIVE_TABLE_PREFIX` on the API reads a test observer's tables. `site/live.html` polls it every 2 s. The site shows OFFLINE when `live_state.updated_at` is older than 30 s.