One HTML Timeline, One Capture Pass
HyperFrames replaces four compounding FFmpeg re-encodes with a single declarative HTML composition captured once. Here's the 4-track model, the quality argument, and the four gotchas that caused real bugs.
The Problem With Four Re-Encodes
Before HyperFrames, my video pipeline used FFmpeg for final assembly. The sequence looked roughly like this: concatenate clips and crossfade → burn ASS caption file → overlay the circular PiP avatar → mux the audio track in and trim it to final length. Four discrete operations. Four full-video re-encodes.
The compounding quality loss is real. Every lossy encode pass degrades the output. By the fourth pass you're not starting from the original capture anymore — you're encoding an encode of an encode. And the code surface was three Python modules with their own error handling, retry logic, and state.
The 4-Track Model
The runtime uses data-track-index to composite content in layers. There are four tracks by convention, each with a specific role:
Every timed element in any track uses the same three attributes: data-start (seconds from video start), data-duration (how long it's visible), and data-track-index (which layer it sits on). The runtime handles the rest — it knows when to show each element, in what order, and with what z-index stacking.
Why One Pass Matters
The FFmpeg assembly model accumulated four failure surfaces. If the PiP overlay step failed, you had already spent time on the concat and caption steps. If the audio mux failed, you had burned three passes. Each step had to read in the full video output of the previous step.
The HyperFrames model inverts this. The composition is declared first, entirely. The runtime renders it once. If something is wrong structurally — a missing class="clip", a duplicate id, an invalid font reference — the linter catches it before any render begins. The render is the final step, not an intermediate one.
This is the mental model I keep coming back to: trust the linter before you render. A 60-second render to discover a structural error is expensive. A lint check to catch it is free.
The Four Gotchas
These are not theoretical. Each one caused a real bug during the HyperFrames migration.
Gotcha 1: Missing class="clip" on timed elements. Every element with data-start and data-duration must also carry class="clip". Without it, the runtime does not register the element as a timed clip — it treats it as always-visible content. A caption that should appear at second 4 and disappear at second 6 instead covers the entire video. The linter errors on this, so in practice you discover it at lint time. But if you're hand-editing HTML and skip the lint step, this is the first thing to check.
Gotcha 2: Media elements without a unique id. Every <video> and <audio> element must have a unique id. The runtime uses the id to track element identity across seek operations. Without it, media elements render frozen — they initialize once but cannot be restarted or re-initialized when the timeline seeks back. The symptom is silence on the audio track or a static first frame on a video element. Fix is simple: id="main-vo", id="bg-music", one per element.
Gotcha 3: Animating opacity on a data-* element. The runtime forces opacity: 1 on every active clip — any element carrying data-* attributes. This is intentional: the runtime controls visibility, not the element's own opacity. If you write a GSAP tween targeting opacity on a data-* element, it is a silent no-op. The avatar appears and disappears instantly instead of fading. The fix is to wrap the element in a plain <div> with no data-* attributes and animate the wrapper. The inner element still controls timing; the outer wrapper controls the visual transition. This is how the circular PiP fade-in is implemented.
Gotcha 4: CSS variables in font-family. The HyperFrames compiler does static font analysis at build time. It reads literal font-family strings, maps them against supported Google Fonts, and auto-fetches them. A CSS variable like var(--font-display) is opaque to the static analyzer. The compiler cannot resolve what font name the variable holds, so nothing is fetched, and text falls back to the system default. Write the literal font name directly: font-family: "Space Grotesk". This is a compile-time constraint, not a runtime one.
The Code Surface Reduction
Before the HyperFrames migration, final assembly lived in three modules:
concat_clips.py— handled clip ordering, crossfades, and duration normalizationburn_captions.py— read ASS subtitle files and re-encoded with caption overlaypip_overlay.py— composited the avatar PiP and handled the audio mux
Each module had its own retry logic, error handling, and output paths. A change to the caption timing format required touching burn_captions.py, which also meant re-testing the audio mux because the output file name convention was shared.
After the migration, there is one HTML composition file. Caption timing, avatar positioning, audio start/end, and base clip sequencing all live in the same declarative document. A change to caption timing is a change to data-start and data-duration attributes on caption elements — no Python, no subprocess call, no output file to track. The linter validates the structure. The renderer captures it.
The tradeoff is that the composition format has its own constraints — the four gotchas above are real constraints, not edge cases. But they are caught at lint time, not discovered after a multi-minute render cycle.
What's Next
The next lesson covers a real sync bug that shipped: when proportional scaling a storyboard to a voiceover placed the wrong caption over the wrong visual, and why Whisper word-level timestamps are the only structural fix.