miner-ui-4 (0.3.18): a miner whose template fetches time out is not faulted on silence

PC 1's read-back: the node reported synced while its finality replay blocked the template RPC for three minutes; the miners printed only timeouts and the watchdog faulted every card. The timeout line is the miner's heartbeat: silence clocks start over from it, the card says it waits for templates, one Activity line per episode. Recorded sequence as the test; 173 box tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

app/igneum-app/src/engine.rs

@@ -4026,6 +4026,22 @@ impl Engine {
         let Some(i) = self.miners.iter().position(|m| m.card == card) else { return };
         let label = self.miners[i].label.clone();
         let card_name = self.st().mining.cards.get(card).map(|c| c.name.clone()).unwrap_or_else(|| label.clone());
+        if text.contains("template fetch timed out") {
+            // PC 1, 7 October 2026 04:51Z: the node reported synced while its finality replay blocked the template
+            // RPC for three minutes; the miner printed only these lines, and the watchdog faulted every card for
+            // silence. The line is the miner's heartbeat: the card waits on the node, with one Activity line per episode
+            let now = Instant::now();
+            let t = self.secs(now);
+            let first = !self.miners[i].watch.templates_blocked();
+            self.miners[i].watch.event(t, crate::watchdog::Event::TemplateTimeout);
+            if first {
+                self.shared.event("info", &format!("{card_name}: the node is not answering block templates yet; the miner keeps asking (the node catches up after a restart)"));
+            }
+            if let Some(c) = self.st().mining.cards.get_mut(card) {
+                c.message = "waiting for the node to answer block templates".into();
+            }
+            return;
+        }
         if stderr {
             if text.contains(" rejected nonce=") {
                 if let Some(run) = self.sweep.as_mut() {

app/igneum-app/src/watchdog.rs

@@ -94,6 +94,11 @@ pub enum Event<'a> {
     /// The node became synced: a card faulted for silence while the node was not ready tries again (PC 1, 7 October
     /// 2026 04:51Z: every card faulted "no status line" during the post-relaunch sync and never came back).
     NodeSynced,
+    /// The miner printed "template fetch timed out": it is alive and the node is not answering templates (PC 1,
+    /// 7 October 2026 04:51Z: the node reported synced from the first second while its finality replay blocked the
+    /// template RPC for three minutes; with no template the miner prints no status line). Silence is not the card's
+    /// fault while this goes on; the miner is judged again once templates flow.
+    TemplateTimeout,
 }
 
 #[derive(Debug, Clone, PartialEq)]
@@ -117,6 +122,15 @@ pub struct CardWatch {
     restarts: u32,
     faulted: Option<String>,
     last_fault: String,
+    /// the miner's last line was a template timeout (the node not answering), cleared by the next status line
+    templates_blocked: bool,
+}
+
+impl CardWatch {
+    /// True while the miner's last word was a template timeout: the node, not the card, holds the hashing.
+    pub fn templates_blocked(&self) -> bool {
+        self.templates_blocked
+    }
 }
 
 impl CardWatch {
@@ -170,6 +184,7 @@ impl CardWatch {
                 Action::None
             }
             Event::Status(s) => {
+                self.templates_blocked = false;
                 self.last_status_s = Some(now_s);
                 if s.hash_now > 0.0 {
                     self.zero_since = None;
@@ -215,6 +230,16 @@ impl CardWatch {
                 *self = Self::default();
                 Action::None
             }
+            Event::TemplateTimeout => {
+                // the miner's own heartbeat: every silence clock starts over from this line
+                self.started_s = Some(now_s);
+                if self.last_status_s.is_some() {
+                    self.last_status_s = Some(now_s);
+                }
+                self.zero_since = None;
+                self.templates_blocked = true;
+                Action::None
+            }
             Event::NodeSynced => {
                 if self.faulted.as_deref().map(|f| NODE_FAULTS.iter().any(|p| f.starts_with(p))).unwrap_or(false) {
                     *self = Self::default();
@@ -522,6 +547,28 @@ mod tests {
     /// while the node is not synced and never releases the fault.
     #[test]
     fn a_worker_started_while_the_node_syncs_is_not_faulted_and_a_faulted_one_returns_at_sync() {
+        // the recorded sequence (relaunch-engine-and-workers.txt): node synced=true from the first second, the miner
+        // started at +14 s, subscribed, then every template fetch timed out at 5 s for 180 s; no status line was printed
+        let mut r = CardWatch::new();
+        r.event(14.0, Event::Started);
+        let mut t = 19.0;
+        while t <= 200.0 {
+            r.event(t, Event::TemplateTimeout);
+            assert_eq!(r.tick(t + 1.0, true), Action::None, "at {t}: a miner whose template fetches time out is not faulted on silence");
+            assert!(r.templates_blocked());
+            t += 5.0;
+        }
+        // the node answers templates again: the miner prints status and is healthy
+        healthy(&mut r, 210.0, 400.0);
+        assert!(!r.templates_blocked());
+        assert_eq!(r.faulted(), None);
+        // a miner that stops printing even timeouts is judged again from its last line
+        let mut q = CardWatch::new();
+        q.event(0.0, Event::Started);
+        q.event(50.0, Event::TemplateTimeout);
+        assert_eq!(q.tick(109.0, true), Action::None);
+        assert!(matches!(q.tick(111.0, true), Action::Restart(_)), "60 s of nothing at all after the last timeout line");
+
         let mut w = CardWatch::new();
         w.event(0.0, Event::Started);
         // the node reads unsynced for 200 s (the relaunch's sync): no verdict at all

docs/plans/miner-ui-4.md

@@ -113,3 +113,4 @@ Tests: UI 39; the app crate on the box 152 passed (the N4 pair, the shot path, t
 - Known-failed case: PC 1 at the 0.3.17 relaunch (04:51:43Z, docs/plans/release-0.3.17.md PC 1 row): the app started its workers while the node was still syncing; the card watchdog clocked the silence anyway (a miner with no template prints no status), restarted each once, then faulted every card "no status line from the miner for 90 s (restarted once already)", pid 0, 0.0 MH/s on; faulted cards were never restarted, including at the synced transition.
 - Fix (src/watchdog.rs, src/engine.rs): a miner is judged only while the node is synced (the start, no-status and zero-rate clocks hold at now while it is not); a worker silent from its start with the node synced is one restart then a fault at 60 s (`START_S`), 90 s stays for a running one; a fault the node's readiness explains (`faulted_by_node`) is released by `Event::NodeSynced` when the node syncs, the card starts again with an Activity line; one Activity line per card at its first worker start. Test `a_worker_started_while_the_node_syncs_is_not_faulted_and_a_faulted_one_returns_at_sync` (recorded state), 173 box tests.
 - The pre-apply card set already persists (cards' enabled flags live in settings and the relaunch planned every enabled card); the fault, not the plan, was the failure.
+- Read-back changed the reading (relaunch-engine-and-workers.txt, 05:40 UK): PC 1's node reported synced=true from the first second; the miners started at +14, +25 and +38 s, subscribed, then every template fetch timed out at 5 s for 180 s while the node ran its finality catch-up (checkpoints 8053 to 8055 locking), so no status line was printed; the watchdog restarted each at 90 s and faulted each at 180 s. Rule added: "template fetch timed out" is the miner's heartbeat (`Event::TemplateTimeout`): the silence clocks start over from the line, the card reads "waiting for the node to answer block templates", one Activity line per episode, and the miner is judged again from its last line (60 s of nothing at all after it is still a restart). The synced-gate half stays. Test extended with the recorded sequence; 173 box tests. The node-side half (the template RPC blocking under the finality replay after a restart) is with the node lane.