The Dead Component Pattern

The Hermes calibrator was alive, well, and producing zero signal. Every call succeeded. Every log line was healthy. Every unit test passed. And every signal it contributed to cleared the threshold on its own merits — or, in 96% of cases, failed to clear because the 25 points the calibrator theoretically could have added were zero instead.

This is the dead component pattern. It is the single most common silent failure mode in production ML and scoring systems, and it exists because error-based monitoring was never designed to catch it.

The Anatomy

A dead component has four properties that collectively make it invisible to standard health checks:

1. No exceptions. The component gracefully handles all failure cases internally. Network errors, missing data, malformed inputs — all caught, all swallowed, all replaced with a default return value.
2. No warnings. Nothing in the code path logs warning or error. The default path looks identical to the successful path in log output.
3. Stable latency. The component returns quickly because there is nothing to compute — returning zero is faster than running the full pipeline. Latency dashboards show improvement, not degradation.
4. Correct output. The return value is a well-formed zero (or empty list, or False boolean) that satisfies the caller's type expectations. Downstream code continues execution with no idea anything went wrong.

The component is simultaneously doing nothing and passing every health check. The bug is the gap between "responding" and "contributing" — and the gap is invisible unless you measure it.

Inline Diagram — The Invisibility Matrix

Why It Hides for So Long

The dead component is usually hidden by a second failure: the downstream pipeline does not distinguish between "component contributed zero" and "component was not yet computed." Both look like a zero in the composite score. The bot processes the signal, writes the score to the database, and moves on. The next signal comes in. The same thing happens. The pipeline never stops running; it just stops producing output that clears the threshold.

For Hermes, the second failure was that signals_cleared_threshold_24h was not an alerted metric. Zero signals per day looked identical to "it was a slow news cycle." The calibrator's role in the blockage was invisible until someone — Knox — thought to ask: "What does the component distribution look like?"

The Detection Pattern

The only reliable detection for dead components is direct fire-rate monitoring (Lesson 258):

# In the scoring component itself
value = compute_contribution(signal)
fired = value > 0.001  # or whatever epsilon makes sense
metrics.increment(f"component.{name}.total")
if fired:
    metrics.increment(f"component.{name}.fired")

# In alerting config
fire_rate_24h = metrics.rate(f"component.{name}.fired", "24h") / \
                metrics.rate(f"component.{name}.total", "24h")
alert_if: fire_rate_24h < 0.20  # warning
alert_if: fire_rate_24h < 0.05  # critical

Three lines of instrumentation. Two alert thresholds. That is the entire fix — and it is the single cheapest investment any scoring system can make against silent failure.

The Rule

Silence is not health. Every scoring component emits fire rate. Every fire rate has alert thresholds. The dead component pattern has been observed in Hermes, in Foresight's news feed, and in Shiva's market-weather signal — it is not a rare edge case. It is a default failure mode that appears wherever scoring systems depend on external data sources. Monitor for it, or keep finding it in post-mortems.