Piecewise Continuity
CodeRabbit caught what the code-only agent missed: a calibrator branch boundary where the score jumped from 18.4 to 17.6 as the input crossed 0.15. Piecewise rubrics need boundary audits — and reviewers are non-optional.
PR #28 had a bug that no test caught and no human had time to spot. The calibrator component had a piecewise formula with four branches. Its input p is the calibration-match strength: how closely the signal's implied probability agrees with the matched prediction-market probability, normalized to [0, 1] — higher means better-calibrated. This was the original (buggy) version that shipped in the first PR draft:
# ORIGINAL — buggy version (pre-CodeRabbit fix)
def calibrator_score(p: float) -> float:
if p < 0.10:
return 0.0
elif p < 0.15:
return 18.4 * (p - 0.10) / 0.05 # ramps 0 → 18.4 linearly
elif p < 0.25:
return 17.6 - 3.6 * (p - 0.15) / 0.10 # starts at 17.6, drops to 14.0
elif p < 0.40:
return 18.0 + 2.0 * (p - 0.25) / 0.15 # starts at 18, ends at 20
else:
return 20.0
Check the boundary at p = 0.15. Approaching from below (the limit as p → 0.15⁻, which is the upper end of the elif p < 0.15 branch — the branch is not taken at exactly 0.15 itself): the lower branch gives 18.4 * 0.05 / 0.05 = 18.4. The upper branch at p = 0.15 gives 17.6 - 3.6 * 0.0 / 0.10 = 17.6. A slightly better calibration input (just above 0.15) scores 0.8 points lower than a slightly worse one (just below 0.15). That is a cliff.
CodeRabbit flagged it. And running the full boundary audit before fixing surfaced a second problem the review comment had not mentioned: an unowned 4-point jump at p = 0.25, where the third branch ends at 14.0 and the fourth branch starts at 18.0. The PR was blocked until both boundaries were corrected. The fixed version:
# FIXED — after CodeRabbit review and a full boundary audit
def calibrator_score(p: float) -> float:
if p < 0.10:
return 0.0
elif p < 0.15:
return 18.0 * (p - 0.10) / 0.05 # ramps 0 → 18 linearly
elif p < 0.25:
return 18.0 # plateau at 18
elif p < 0.40:
return 18.0 + 2.0 * (p - 0.25) / 0.15 # ramps 18 → 20
else:
return 20.0
Now run the lesson's audit on both versions — every boundary, both branch expressions evaluated at the exact transition value:
| Boundary | Original: below → at | Fixed: below → at | Verdict |
|---|---|---|---|
| 0.10 | 0.0 → 0.0 | 0.0 → 0.0 | Continuous in both |
| 0.15 | 18.4 → 17.6 | 18.0 → 18.0 | Original: 0.8-point cliff — fixed is continuous |
| 0.25 | 14.0 → 18.0 | 18.0 → 18.0 | Original: 4.0-point jump — fixed is continuous |
| 0.40 | 20.0 → 20.0 | 20.0 → 20.0 | Continuous in both |
The fixed version is continuous at all four boundaries and monotonically non-decreasing: better calibration agreement never scores lower. Both original bugs were invisible to any test that did not probe exactly at the boundary values — which is precisely why the audit walks every boundary, not just the one a reviewer happened to flag.
Inline Diagram — Continuity Audit Steps
The Audit Pattern
For every piecewise scoring function, run this audit before shipping:
- List every branch boundary. Every
if/elifthreshold. Everycasecutoff. Every comparator likep > 0.15. - For each boundary B: compute the lower-branch expression at exactly B, and the upper-branch expression at exactly B. They should be equal or differ by a design-justified amount.
- If they differ unexpectedly: either the formula is wrong or the shape choice is wrong. Both need an owner to explain.
A ten-line function with four boundaries gets four boundary checks. Five minutes of arithmetic.
Why Reviewers Catch This
Code-writing agents have a consistent blind spot for numerical continuity. They pattern-match on "piecewise function with thresholds" and produce branches that look symmetrical but are not. The test suite usually picks round numbers that never land on a boundary. The author trusts the tests. The bug ships.
CodeRabbit, and code-review LLMs in general, are better at this specific task because they operate on the code itself as a mathematical object. Given a piecewise expression, asking "are the branches continuous at the boundaries?" is a clean analysis task that does not require running the code.
The Rule
Piecewise functions get boundary audits. Every branch point gets tested at the exact transition value. Every discontinuity needs an owner. And CodeRabbit comments on scoring math are treated as blockers until resolved — not nits to be skimmed.