ASK KNOX
beta
Learn/Your First AI Content Pipeline/Lesson 365
LESSON 365

Publishing via Git PR: Human in the Loop

Auto-generating is not the same as auto-publishing — a pull request is the review gate that keeps quality high and brand damage low while the pipeline calibrates.

8 min read·Your First AI Content Pipeline

Introduction

The PUBLISH stage has one job: take the draft and the image and make them reviewable. Not live — reviewable. The distinction matters.

The instinct when building an automated content pipeline is to go all the way: generate the article, generate the image, commit directly to main, and let Vercel deploy it. That instinct is wrong at the start, and only becomes defensible after the pipeline has proven itself across many cycles.

The is not bureaucracy. It is the mechanism that lets you run an automated pipeline without betting your site's reputation on every single AI output.

This lesson covers why the PR model is the right default, how deliver.py opens one, what the Discord notification carries, and when — after how much evidence — it is reasonable to move toward fuller automation.

Core Concept

Why Not Commit Directly to Main?

A pipeline that commits directly to main and triggers a Vercel auto-deploy has removed all human recourse between "AI wrote this" and "readers see this." That is a high-risk default, especially in the early cycles of a new pipeline.

Consider what can go wrong in one pipeline run without a review gate:

  • The voice profile drifted — the draft sounds like generic AI, not you
  • The AI included a factual claim it confabulated — a statistic, a product name, a date that is wrong
  • The topic record had an edge case — the article was technically about the right topic but came out in a different direction than intended
  • The image prompt produced something visually off-brand

Any of these would be caught immediately in a 10-minute PR review. Without the gate, they go live. One bad article is a recoverable event. A pattern of them erodes trust.

The PR gate is cheap: 10 to 20 minutes of review time per article. The downside of removing it prematurely is expensive.

How deliver.py Opens a PR

The PUBLISH stage in the jeremyknox.ai pipeline follows this sequence:

1. Load draft.mdx and image_path.txt
2. Assemble final MDX file with complete frontmatter:
   - title, date, category, excerpt (from topic.json)
   - coverImage (from image_path.txt)
   - tags, readTime, status: "published"
3. Create a feature branch: content/auto-{slug}-{date}
4. git add the MDX file and image
5. git commit with message: "auto: {article title}"
6. git push -u origin {branch}
7. gh pr create \
     --title "{article title}" \
     --body "Auto-generated article. Review before merging." \
     --label "auto-content"
8. Capture PR URL from gh pr create output
9. Send Discord notification: "New article ready for review: {title}\n{PR URL}"
10. Write pr_url.txt for pipeline state log
11. Exit 0

Steps 6 through 9 are what make the PR useful as a workflow tool. The feature branch means the article is staging in git but not in production. The Discord notification means the human knows immediately when something is ready without polling GitHub. The PR URL in the notification means a single click gets you to the diff.

The Discord Notification

The notification fired in step 9 is worth designing carefully. Here is what the jeremyknox.ai notification includes:

  • Article title
  • Category and estimated read time
  • First two sentences of the draft (excerpt preview)
  • Direct link to the GitHub PR
  • Direct link to the staged file diff

This lets the reviewer make a first-pass quality call from Discord without opening GitHub. If the title looks wrong or the excerpt reads as generic AI slop, they know to look carefully before merging. If both look good, the full review in the diff is confirming rather than discovering.

When to Remove the Gate

The PR review gate is a calibration tool. You remove it — or move it to an exception-based model — when you have evidence that the pipeline produces consistently high quality.

"Evidence" means:

  • 30 to 50 published articles with no major quality failures
  • A scoring mechanism that grades each draft before delivery (voice consistency, factual claim density, structure adherence)
  • Automatic alerts when the draft score drops below threshold — so the gate triggers on low-confidence outputs only
  • A revert procedure that is fast: if something goes live and needs to come down, you can revert and redeploy in under five minutes

The jeremyknox.ai pipeline is currently at the "PR gate on every article" stage. Moving to automatic merge is on the roadmap, but requires a draft scoring layer that does not exist yet.

Practical Application

Here is a concrete example of the PR that deliver.py opens:

Branch:  content/auto-ai-memory-systems-2026-05-30
Title:   Why AI Agents Need Persistent Memory
Labels:  auto-content
Body:
  Auto-generated by blog-autopilot pipeline.
  Run ID: 2026-05-30T09:04:22Z
  Topic source: topic_queue.json item #47
  Image provider: Leonardo AI Phoenix 1.0
  Draft length: 1,187 words
  
  REVIEW CHECKLIST:
  [ ] Voice sounds like Knox, not generic AI
  [ ] No factual claims that need verification
  [ ] Headline matches the article body
  [ ] Hero image is on-brand and not bizarre
  [ ] Frontmatter fields are all correct

The checklist in the PR body is not there for the pipeline — it is there for the reviewer. It converts a vague "looks good?" review into a structured 10-minute pass. Each item has a clear binary: pass or fail.

If any item fails, the reviewer edits the draft directly in the branch (GitHub's web editor or a local checkout), pushes the fix, and merges. The edit history in git shows exactly what changed between AI generation and human review — useful data for improving the voice profile over time.

Common Mistakes

Publishing directly to main from the start. The first sign the pipeline has a quality problem is when a bad article goes live and you have to emergency revert. Do not use production readers as your quality gate.

No PR labels. Without a label like auto-content, PRs from the pipeline blend into your other development PRs. Label them so you can filter, track, and measure the pipeline's output separately.

Not capturing the PR URL. If deliver.py opens a PR but does not send the URL to Discord, you have to go find it manually every time. gh pr create prints the new PR's URL to stdout — capture it with PR_URL=$(gh pr create ...) (or fetch it after creation with gh pr view --json url --jq .url) and treat it as a required output of the PUBLISH stage.

Reviewing only the prose, not the frontmatter. Frontmatter errors (wrong date, missing coverImage path, incorrect category) will cause rendering failures or missing images on the site. Add frontmatter to your review checklist explicitly.

Forgetting to pull after merge. The pipeline runs on a branch. After you merge, the main branch moves forward. The next pipeline run needs to branch from the current main. Make sure deliver.py always starts from a fresh git pull on main before creating the feature branch — not from whatever state the previous run left behind.

Summary

  • The PR gate is the human review step that catches AI quality failures before they reach readers.
  • deliver.py assembles final MDX, commits to a feature branch, opens a PR with gh pr create, and sends a Discord notification with the PR URL.
  • The Discord notification should include the title, excerpt preview, and a direct PR link so review requires minimal navigation.
  • Remove the gate only after 30 to 50 cycles of proven quality, a draft scoring mechanism, and a fast revert procedure.
  • Review checklist in the PR body converts a vague approval into a structured quality pass.

What's Next

The next lesson covers the cron — moving from manually running the pipeline to a scheduled job that fires automatically. Category round-robin, state tracking, and how to handle failures when nobody is watching.