ASK KNOX
beta
Learn/Production Content Pipelines/Lesson 451
LESSON 451

The Production Content Pipeline Playbook

Assemble everything from this track into a self-running, self-healing content machine — and run the audit lab to prove it.

11 min read·Production Content Pipelines

This Track in One Map

Ten lessons. One goal: a content pipeline that runs every Monday, Wednesday, and Friday without you, recovers when it fails, and produces articles that sound like they were written by Knox. The Pipeline Observability lesson that follows this playbook covers measuring pipeline health over time — detecting silent degradation before it becomes a hard failure.

Before the audit lab, let's see the full system assembled end-to-end.

This is what Knox's blog-autopilot actually does — 2.5 minutes from cron fire to PR open. Four scripts. Intake state. A fallback chain. A delivery receipt. The only human step is PR merge.

The Playbook in Seven Phases

Building this from scratch follows a defined sequence. You cannot shortcut Phase E (Image fallback chain) because you didn't experience a Leonardo failure yet. You cannot defer Phase G (Health checks) to "when something breaks." Production architecture must be complete before the pipeline runs unsupervised.

Phase A: Architecture (Before Writing Any Code)

Before the first line of code, write the four-stage architecture with handoff contracts. This is not documentation — it is the design that determines whether your pipeline fails visibly or silently.

Write the contracts in order:

  1. What does SOURCE produce for DRAFT? (topic_id, raw_content, source_url, source_type)
  2. What does DRAFT produce for IMAGE? (draft_text, title, excerpt, word_count)
  3. What does IMAGE produce for PUBLISH? (image_path, image_provider, aspect_ratio, quality_pass)
  4. What does PUBLISH produce as the final receipt? (pr_url, discord_message_id)

For each field, define: type, required/optional, gate action if missing (hard block vs soft remediate).

Phase B: Scheduling (The launchd Foundation)

Write the launchd plist with all required fields before deploying to the production server:

  • StartCalendarInterval with Mon/Wed/Fri at 08:00
  • EnvironmentVariables block with every variable from your .env file
  • WorkingDirectory as an absolute path
  • StandardOutPath and StandardErrorPath configured and writable
  • RunAtLoad=false and KeepAlive=false (scheduled pipelines, not persistent services)

Test the plist in isolation: launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/<plist>, then launchctl list | grep <plist-name>. Verify the schedule fires on the next Mon/Wed/Fri slot.

Phase C-F: The Four Stages

Implement each stage as an independent, testable script. The test for each stage is simple: run it with a synthetic input and inspect the output. If the output matches the contract, the stage is correct. If not, debug the stage — not the pipeline.

The most common implementation gaps:

  • SOURCE: missing dedup logic (URL hash check)
  • DRAFT: voice profile not loaded as system context (most common quality bug)
  • IMAGE: quality gate not implemented (accepting empty responses)
  • PUBLISH: no idempotency check (duplicate PRs on restart)

Phase G: Health Checks (The Last and Most Important)

Health checks are the most frequently deferred phase. "I'll add alerting once the pipeline is working" is how zombie pipelines go undiscovered for two weeks.

The minimum viable health check for a Mon/Wed/Fri pipeline:

from datetime import datetime

# Run as a separate launchd job, every 2 hours on publish days
def health_check():
    now = datetime.now()
    if now.weekday() not in [0, 2, 4]:  # 0=Mon, 2=Wed, 4=Fri
        return  # not a publish day

    if now.hour < 11:
        return  # publish runs at 08:00 — give it until 11:00 before judging

    last_pr = get_last_pr_timestamp()  # query GitHub API
    if last_pr.date() < now.date():
        send_discord_alert(
            "Pipeline health check FAILED",
            f"No PR created today (publish day). Last PR: {last_pr}"
        )

This check fires every 2 hours on publish days, but only alerts after 11:00 — and only if the most recent PR is not from today. Both guards matter: without the hour guard, the midnight and 06:00 checks fire before the 08:00 run has even happened; without the date comparison, a successful 08:00 publish would still trip a naive "older than 4 hours" age check by mid-afternoon. The check detects the zombie pipeline scenario — process alive, no output — which the process heartbeat misses.

The Audit Lab

Before declaring a pipeline production-grade, run the 28-check audit lab. This is not optional. The audit lab is the certification process.

For each of the 28 checks, document:

  • What you tested
  • What the evidence is (command output, file contents, log lines)
  • Pass or fail

For any hard gate failure, fix before declaring production. For any soft gate failure, document the known issue and the planned remediation date.

A pipeline that passes the audit lab is a production pipeline. A pipeline that has not run the audit lab is a development pipeline, regardless of how well it has been running.

Compounding the Pipeline Over Time

Production pipelines improve. The first version of Knox's blog-autopilot had no fallback chain, no voice profile versioning, and no retry count limit. These gaps were discovered in production — a Leonardo failure with no fallback, an article that read as generic AI text, a topic that failed permanently and re-queued endlessly.

Each discovery produced a fix. Each fix was documented in lessons.md. Each lessons.md entry was stored to Semantic Memory Layer via memory_remember. The next pipeline Knox builds starts from a higher baseline because the current pipeline's failure modes are documented, stored, and searchable.

This is the spec-driven development track applied to production systems: the retro loop makes every pipeline better, and the Semantic Memory Layer storage makes those lessons available across sessions and across projects.

The Final Test

A production content pipeline passes this test: Knox goes on a two-week vacation. When he returns, the pipeline has published six articles (Mon/Wed/Fri × 2 weeks), all in his voice, all with hero images, all with ecosystem backlinks, all with Discord notifications delivered. Three failed runs were auto-recovered. One Leonardo failure was handled by the Gemini fallback. No manual intervention was required.

If a pipeline requires Knox's attention to function, it is not a production pipeline. It is a workflow Knox has to maintain. The distinction is the difference between leverage and overhead. This playbook produces the former.