Quality Gates in the Pipeline

Done Is Not a Feeling

"I think it works" is not done. "CI is green" is not done. "The PR is merged" is not done. In spec-driven development, done has a precise definition that can be checked against five gates before the PR is ever opened.

The definition of done exists because agents — like humans — will declare work complete at the earliest possible moment. Without explicit gates, "complete" means "the happy path works and I ran one test." With explicit gates, "complete" means the five conditions below are met.

The Five Gates

The fifth gate — E2E verified on the target machine — is the most commonly skipped. It is the gate Knox's MEMORY.md specifically calls out in the "Past incident (2026-04-07)" story: eight PRs applied to your laptop only, every localhost check green, the Vercel frontend immediately broke when it hit the production server backend. Gate 5 exists to prevent this exact class of failure.

The Coverage Floor

Every repo in the organization must maintain ≥90% test coverage. This is not a target — it is a floor. CI blocks merge when coverage drops below it. The vitest configuration enforces it:

// vitest.config.ts coverage settings
{
  "coverage": {
    "thresholds": {
      "statements": 90,
      "branches": 90,
      "functions": 90,
      "lines": 90
    }
  }
}

But the 90% floor is only meaningful if the tests under it are real:

Coverage theater — tests that execute code but assert nothing — is the most dangerous form of false confidence. A codebase with 95% coverage and all tests checking toBeDefined() has the same defect exposure as a codebase with 0% coverage, plus the false belief that it is well-tested.

The test for a real assertion: if you changed the return value of the function under test, would any test fail? If the answer is no, the test is theater.

Test-First for Bug Fixes

The definition of done includes a specific rule for bug fixes: the regression test must be written before the fix.

The protocol:

1. Read the bug report
2. Write a test that fails with the current code — proving the bug exists
3. Fix the bug until the test passes
4. Verify no other tests broke

This order is non-negotiable. A test written after the fix can be written to match the implementation rather than to verify the bug is gone. The test-first rule ensures the test is a real guard, not a formality.

In Knox's production environment, this rule saved the trading bot multiple times. The calibrator had a coverage gap on its edge-case paths. When the score component audit revealed 96.2% dead components, the fix included regression tests for every fire-rate edge case before touching the calibrator logic. The tests caught two additional failure modes during development.

The Security Gate

Gate 4 (no security regressions) has four specific checks that are easy to miss:

dangerouslySetInnerHTML without sanitization: Any new use of this React prop must pass through DOMPurify or equivalent. Unsanitized user content in the DOM is an XSS surface.

Tokens in URL params: ?token=... is visible in logs, browser history, and referrer headers. Bearer tokens belong in Authorization headers, not query strings.

os.getenv() in API routers: Environment variables in routers bypass dependency injection and make testing harder. The pattern is process.env.OPENAI_API_KEY accessed through a config module, not directly in route handlers.

No new secrets in source code: The sec-scan at ~/.local/bin/sec-scan runs before any repo adoption. It must also run before any PR that touches configuration files. Exit code 2 (FAIL) blocks merge.

When Does Localhost Verification Count?

Gate 5 specifies E2E verification on the target machine. But not every feature requires SSH to the production server. The rule is: does your change affect a service that runs on the production server?

• Frontend-only changes (no new API calls, no new server components): Localhost verification + Vercel preview URL is sufficient.
• Backend changes (new API routes, modified database queries, new cron behavior): Production server verification required. ssh 198.51.100.5 and verify the change is deployed and functional.
• Full-stack changes: Both. Frontend verified against Vercel preview; backend verified on the production server.

The common mistake is verifying the backend change on your laptop's shadow stack (localhost:8001) and calling it done. The shadow stack does not have the same Docker volumes, environment variables, or git state as the production server. It proves local correctness, not production correctness.

What's Next

The next lesson covers semantic review — reviewing the diff against the spec rather than against intuition, generating a PR description that maps to acceptance criteria, and the CodeRabbit triage workflow that distinguishes merge-blocking Major+ issues from advisory notes.