Signal Engines: From Market Data to Conviction
A signal engine is a pipeline that turns raw market data into a directional conviction score — and the silent killer is a near-dead component carrying weight it never earns.
What the engine actually does
A trading agent does not see the market — it sees a number. That number is the output of a signal engine: a pipeline that transforms raw market data into a scalar conviction score, which then crosses a threshold to become a directional decision.
Understanding the pipeline end-to-end is not an academic exercise. Every production problem an autonomous trading agent hits — silent periods, inconsistent fills, profitable backtests that bleed live — has a root somewhere in this pipeline. Knowing where to look is what separates an operator from someone who just watches the logs.
The pipeline from top to bottom
The pipeline has five stages, and each stage can fail in a distinct way.
Market data arrives as candles, orderbook snapshots, and volume readings. The failure mode here is stale or missing data: if the feed is delayed and the agent doesn't detect it, it is trading on yesterday's prices.
Feature extractors convert raw data into meaningful signals: RSI, momentum, spread-to-volatility ratio, bid-ask imbalance. These are computed fresh on every tick and fed downstream to the filters. The failure mode is drift: a feature that made sense during the backtest period may not carry the same meaning in current conditions.
Weighted filters are the core of the engine. Each filter receives the feature set, evaluates its specific hypothesis, and returns a score between 0 and 1. The score gets multiplied by the filter's weight before joining the aggregate. This is where the architecture provides a key design benefit: each filter is independently named, weighted, and — critically — toggleable at runtime. You can disable a filter while the agent runs without a restart.
The aggregate score is the weighted sum across all active filters. This is what gets compared to the threshold.
The threshold gate converts a continuous conviction score into a binary decision. Cross the threshold and you get a directional signal; fall short and the agent does nothing. The threshold can be conditioned on regime.
The silent killer: dead components
This is the most important lesson in the signal engine chapter, and it is almost never taught explicitly.
A filter can be technically present and doing actual math on every tick while contributing almost nothing to the aggregate — because it almost never fires. When a filter's recent fire rate approaches zero, its output is near zero on nearly every evaluation. Its weight still exists. Its contribution is near zero. In the weighted sum, it dilutes every other component's contribution without adding any signal.
The insidious part: this is invisible in standard monitoring. The filter is running. The code is executing. The weight is applied. The aggregate looks lower than expected, and the threshold becomes harder to cross, and the agent quietly stops trading — possibly for weeks.
This is not hypothetical. A signal engine can be operating at near-zero effectiveness for an extended period before anyone notices, because the aggregate score never drops to zero — it just never gets high enough to cross the threshold. The agent looks like it's working (it's running, processing data, evaluating filters) while producing no trades.
The audit process: for each filter, measure how often its output is non-zero over a meaningful recent window (say, 500 ticks). A filter below 5% fire rate with non-trivial weight deserves immediate investigation. Either the filter's hypothesis is not holding in current conditions (and the weight should be lowered or the filter disabled), or the feature inputs it depends on have drifted.
Runtime toggle invariant
The independently-toggleable nature of filters is a genuine operational advantage. You can disable a misbehaving filter in production without a deploy. But this capability has a strict invariant that production experience reveals every time it is violated:
The toggle map key must match the filter's id field exactly.
If you disable "momentum_filter" in the toggle map and the filter's id is "momentumFilter", nothing happens. The disable call succeeds — there is no error, no log line, no indication of the mismatch. The filter keeps running. The toggle simply resolves to nothing.
This is the class of bug that costs hours in production because all the evidence says the toggle worked: the API call returned 200, the config update was recorded, the agent restarted clean. But the filter is still active, and you cannot see that without checking the effective filter set against the canonical id list.
The rule: when adding a filter, choose its id field first, then use that exact string everywhere — in the filter definition, in the toggle map, in the documentation, in the audit log. Never derive the id from the filter name through any transformation (camelCase to snake_case, spaces to hyphens) — use the literal string.
Regime awareness: where backtests lie
A signal calibrated in a ranging market will bleed in a trend. This is not a failure of the signal logic — it is a failure to scope the conditions under which the signal is valid.
The regime detector provides that scope. It classifies the current market context — uptrending, downtrending, ranging, high-volatility — and gates the threshold accordingly. A short-biased signal that performs well in sideways conditions might have a different threshold in a trending environment, or might be fully disabled in a strong uptrend.
Without regime awareness, the engine applies the same conviction requirement across all market conditions. A signal calibrated for 60%-win-rate conditions in a ranging market might have a 40% win rate in a trend — and the agent has no way to distinguish between the two situations.
The regime input feeds the threshold gate directly: the same aggregate score might cross the threshold in a ranging market and miss it in a trending one, or vice versa. This is the mechanism that keeps the agent calibrated to the current environment rather than to the environment it was trained in.
How is the regime actually classified? The common, robust approaches are simple by design: a moving-average slope (price above a long SMA with positive slope = uptrend, below with negative slope = downtrend, flat = ranging) and a volatility rank (recent realized volatility against its own historical percentile = calm vs. high-volatility). Keep the classifier coarse and slow to flip — a regime that changes every tick is noise, not context — and remember the trap: a regime classifier is itself a model, so it can overfit. If you tune it only on favorable history it will mislabel live conditions, and the signal inherits that error.
Build it: score with fire-rate guard and regime gate
You are going to implement the core scoring function — the weighted aggregate with a near-dead component warning, a regime gate, and a threshold returning a directional signal.
What comes next
You have seen how a signal engine converts data into conviction. The next lesson takes the output of that engine and answers the second hard question: given a positive-expectancy signal, how much of your capital do you commit to each trade? That is the math of Kelly, fractional sizing, and sizing off available balance — not total equity.