From Eyeball-QA to Guardrails

The Smell

The reel took three rounds of human feedback before it cleared the bar.

Round one: desync. The audio and visuals drifted — captions lagged the spoken words, motion felt disconnected from speech. Back to me.

Round two: dense captions plus a title that duplicated a caption. The screen was full of text when it should have been full of image. The title headline and a caption phrase said the same thing. Back again.

Round three: AI b-roll where diagrams belonged, and an image the owner didn't want in the piece at all.

Every single one of those issues was caught by the owner's eyes at review. That's the smell. When a human is the only quality gate, quality scales with their attention — not with the system. The owner had to watch the full reel three times to ship one reel. That's a three-to-one failure rate at the most expensive stage of the pipeline.

Mapping Each Failure to Its Guardrail

Let me walk through each one and what it actually catches.

1. Desync → VO-coherence gate at compose time

The root cause of desync wasn't sloppy editing — it was that the pipeline was using proportionally-scaled timings instead of Whisper-anchored timings. Proportional scaling calculates caption positions as a fraction of total duration. It looks right when you generate it. It drifts when the actual VO pacing doesn't match the assumed distribution.

The fix is mandatory, not optional: timing_source must equal "whisper_anchored" in the composition manifest. If it doesn't, the compose step refuses to run. Then assert every timestamp is within [0, vo_duration] — catch any value that slipped outside the valid window before a single frame renders.

This is also the highest-leverage guardrail in the set because it kills three problems at once: desync, dense captions caused by proportional full-transcript layout, and float-overlap lint noise from fractional millisecond boundaries. One root fix, three symptom classes eliminated.

2. Dense captions → caption-density contract assertion

The density contract is three numbers: no more than 11 phrases per minute, no more than 5 words per caption, no more than 45% runtime coverage. These aren't arbitrary — they come from watching what happens when you exceed each threshold. Above 11 phrases/min the viewer starts reading instead of watching. Above 5 words/caption the eye can't parse it fast enough to keep watching. Above 45% coverage there's no breathing room and the visual gets crowded out.

This runs at compose time. The pipeline counts phrases, measures words, calculates coverage. If any threshold is exceeded, it fails with the specific violation before a single frame renders. No eyeball required.

3. Title == caption → title≠caption shingle check

The title duplicating a caption sounds minor. It isn't. The title is a thematic hook — it sets expectation. A caption is a spoken line — it transcribes action. When they share text, one of them isn't doing its job. Usually the caption is carrying the title's work, which means the title is wasted screen real estate.

The assertion: no shared 3-word shingle between any title headline and any caption phrase. Three words because two-word coincidences are common and usually harmless — "the pipeline", "this works". Three words in sequence is semantic equivalence. If that fires, the title needs to be rewritten to do its actual job.

4. Wrong assets → asset preflight before any spend

This one has a different failure mode from the others. The first three fail at compose time — expensive in wall-clock terms but cheap in API terms. This one can fail after the pipeline has burned actual money: render credits, API credits, time. An invalid avatar-id or empty credit pool only surfaces when the render call returns an error, which is after the clock has been running.

The preflight runs before any spend. Three checks: avatar-id is live and accessible, the correct API credit pool is funded, voice is resolved. If any of those fails, it exits immediately with a message that says exactly what's wrong and what to fix. "Avatar ID abc123 returned 404 — check your HeyGen account" is an actionable message. A failed render log is not.

5. Float overlaps → frame-quantize at emit, not patch per-build

This one is architectural. Float-overlap lint noise is what you get when caption timestamps are stored as floating-point milliseconds and two adjacent captions end/start at values that round to the same frame number. It produces warnings, occasionally visual overlap, always lint noise in the build log.

The per-build patch is: look for overlaps, adjust by 1ms. This works. It also means you're running a patch on every build because the emit layer is still producing fractional timestamps. Frame-quantize at emit instead: snap every timestamp to the nearest frame boundary during generation. The fractional values are never produced. No overlap can occur. No patch is needed. This is the structural fix — make it impossible at the source rather than correctable at the output.

The Meta-Lesson

What these five guardrails have in common is the move they make: from "someone notices this at review" to "the pipeline cannot proceed with this defect."

That move is the whole lesson.

A pipeline with only human review is a pipeline whose quality ceiling is bounded by how much attention a human can sustain across how many review cycles. A pipeline with build-time assertions is a pipeline whose quality floor is bounded by what the assertions cover. The ceiling is still human judgment — tone, pacing feel, whether the story lands. But the floor is machine-enforced, and the floor keeps rising every time you convert a feedback round into an assertion.

This is the same compounding-loop discipline that applies to code pipelines — retro → lessons → guardrails — applied to a media pipeline. The retro from three rounds of feedback produced five guardrails. Those five guardrails mean the next reel ships in one review cycle because the mechanical failures can't reach review. The owner's attention is spent on the things that require their judgment, not on catching defects that a 50ms assertion could have caught first.

What Goes in the Lessons File

After this retro, the lessons.md entry looks like this:

## 2026-05-31 — Video Generation / Quality Gates

**Mistake**: Reel required 3 rounds of human feedback to clear bar.
**Root cause**: No automated gates — pipeline had human review as the only quality check.
**Rules**:
1. timing_source must equal "whisper_anchored" — assert at compose time or refuse run.
2. Caption density contract: ≤ 11 phrases/min, ≤ 5 words/caption, ≤ 45% coverage.
3. No 3-word shingle shared between title and any caption.
4. Asset preflight before any spend: avatar-id live, credit pool funded, voice resolved.
5. Frame-quantize all timestamps at emit — fractional ms is never produced.

Three fields. What went wrong. Root cause. The specific, actionable rules that prevent recurrence. Not "be more careful about captions." The exact contract that the pipeline must pass before any human ever sees the output.

That's how a retro compounds into a better pipeline.