The Four-Stage Architecture — Source → Draft → Image → Publish

Why Architecture Before Implementation

Every production content pipeline follows the same pattern: ingest something, transform it into content, generate visual assets, and deliver to a destination. The specific tools change. The four-stage structure does not.

Before you write any implementation code, you need to define the architecture — specifically, what each stage receives, what it produces, and what the failure response is if either of those conditions fails. Without this definition, your pipeline is a sequence of optimistic function calls, not a contract-enforced system.

The Four Stages in Detail

Knox's blog-autopilot implements the canonical four-stage architecture with gather.py, claude.py, generate_image.py, and deliver.py — four scripts, each responsible for one stage, each producing a defined output that the next stage validates before starting.

The stage architecture has a critical structural property: each stage is independently testable. You can run python gather.py alone with a test topic and inspect intake.json. You can run python claude.py with a pre-populated intake.json and inspect draft.md. This testability is not an accident — it is a consequence of the contract boundary between stages.

Stage Contracts Are Not Documentation

The handoff contract is the interface between stages. It defines which fields are required, which are optional, and what happens if a required field is missing or invalid.

The failure modes section of each contract is the most important part. The common mistake is to define only the happy-path contract — "DRAFT produces draft_text and frontmatter_json" — without defining what happens when it doesn't. The failure mode is where the invisible bugs live.

The SOURCE Stage

The SOURCE stage's job is to convert raw inputs (RSS feeds, YouTube transcripts, manual topics) into a structured, deduplicated intake.json entry. It must provide:

• topic_id — a UUID that is the dedup key for the entire pipeline. If this ID already exists in intake.json with status: done, the pipeline skips and claims the next pending entry.
• raw_content — the transcript, article text, or scraped content that DRAFT will use as source material
• source_url — required for attribution and as the provenance link in the published article
• source_type — determines which voice profile DRAFT applies and how deeply it synthesizes vs summarizes

The DRAFT Stage

The DRAFT stage takes the intake entry and produces a publishable article. It must validate its input contract before starting — if word_count in the output is below 800, the stage fails fast rather than passing a stub to IMAGE.

The DRAFT stage is also where the voice profile is applied. This is not a free-text instruction in the prompt — it is a structured configuration file (voice_profile.txt) that specifies tone, vocabulary, structure, and prohibited patterns. Without a voice profile, the pipeline produces generic AI text that reads as AI text.

Model routing happens at the DRAFT stage: long-form analysis uses claude-sonnet, short explainers use claude-haiku. This decision is not made at runtime by the model — it is configured in the stage's routing logic based on target word count.

The IMAGE Stage

The IMAGE stage takes the title and excerpt from DRAFT's frontmatter and produces a hero image. The stage's primary provider is Leonardo Phoenix 1.0 — use the Phoenix model ID from your Leonardo dashboard or API docs. If Leonardo returns a failure or the quality gate fails (file size < 50KB, MIME type not image/*), the stage falls back to Gemini Imagen, then OpenAI gpt-image-1.

This fallback chain is not complexity for its own sake. Leonardo fails on approximately 3% of runs — typically on high-demand mornings or when the free tier limit is hit. Without a fallback chain, 3% of Monday articles publish without a hero image. With the chain, 0% do.

The PUBLISH Stage

The PUBLISH stage takes the draft, frontmatter, and hero image path and delivers to the configured destinations. For jeremyknox.ai, this means committing to a new branch and opening a GitHub PR. For Discord, it fires the webhook. For X, it generates the carousel text but does not post — X has no viable publish API in this stack (the paid v2 write tier exists but isn't economical for an individual account here), so this step produces content Knox pastes manually.

The PUBLISH stage's critical gate is the delivery receipt: the PR URL is written to the run manifest. If the PR URL is not present, the stage failed. This receipt is what distinguishes "the script ran" from "the content arrived."

Prove Each Hop

The architecture rule that governs this entire track comes from Knox's production delivery discipline: prove each hop — source → transform → destination → consumer.

This means you do not declare a pipeline working because the script exited 0. You declare it working because you can show that the source had the right data, the transformation produced a valid artifact, the artifact arrived at the destination, and the consumer (the blog reader, the Discord channel member) can access it.

A pipeline with an unverified handoff is a pipeline with an invisible failure point. The contracts make the failure points visible.