// Track

Quantitative Scoring System Design

Build scoring systems that actually fire

Component weighting, fire-rate monitoring, ceiling analysis, arithmetic backtesting, and the math that prevents dead components from silently killing your signal. Drawn from the Hermes score rebalancing that took a bot from 2,646 signals and zero trades to live in one session.

6 lessons~51 min total

Lessons are shown in recommended order. Complete them in sequence for the best experience — or jump to any lesson.

Lesson 257·9 min read

Load-Bearing vs Additive Components

Hermes had a 25-point calibrator that contributed zero on 96.2% of signals. That single dead component made the 70-point threshold mathematically unreachable. Load-bearing vs additive decomposition is the pre-flight check every scoring system needs.

Lesson 258·9 min read

Fire-Rate Monitoring

Every scoring component needs a fire_rate_24h metric. Below 20%, it warns. Below 5%, it pages. The Hermes calibrator contributed zero on 96% of signals for weeks and nothing alerted — because the system was counting errors, not contributions.

Lesson 259·8 min read

Ceiling Analysis Before Shipping

The pre-flight check every scoring system needs. Build the spreadsheet before you write the scoring code. What is the maximum achievable score without each component? Can the threshold be cleared on realistic fire rates? The Hermes failure would have been caught in five minutes by anyone willing to do arithmetic first.

Lesson 260·8 min read

Arithmetic Backtesting from Stored Components

The Hermes PR #28 backtest took five minutes via SQL instead of two hours via re-running 64 Perplexity and Grok API calls. If your scoring is deterministic and your components are persisted, backtesting a rebalance is arithmetic — not another agent run.

Lesson 261·8 min read

Piecewise Continuity

CodeRabbit caught what the code-only agent missed: a calibrator branch boundary where the score jumped from 18.4 to 17.6 as the input crossed 0.15. Piecewise rubrics need boundary audits — and reviewers are non-optional.

Lesson 262·9 min read

Rebalance Without Moving the Threshold

PR #28 walkthrough. Grok 30 to 35, Perplexity 30 to 50, Calibrator 25 to 20. Threshold held at 70. Zero signals clearing became eight in 24 hours. Why restructuring weights is always safer than lowering the bar.