Keeping Docs Alive: Freshness, Ownership & CI Gates
Docs rot because nothing enforces freshness at merge time — CI gates, ownership signals, and deletion discipline are the only defenses that work.
The Economics of Rot
Documentation rot is not a failure of effort — it is a failure of incentive structure. Writing a doc has a clear, immediate reward: the problem you were solving gets solved. Updating a doc has a diffuse, delayed cost: the next person who reads it will be misled, but you will likely never see that outcome.
This is not a discipline problem. It is an economics problem. Under time pressure, every team chooses the next feature over updating the runbook, even when they know they should. Convention-based doc maintenance always loses to the deadline.
The solution is not to ask for more discipline. The solution is to change the incentive structure by moving the enforcement point to merge time.
The decay curve is predictable. A doc that is accurate on day zero loses trustworthiness as code evolves around it without a corresponding doc update. By week twelve, it is missing critical steps. By week twenty-four, it may actively guide someone into a broken state. By week fifty-two, it is a liability: readers approach it with confidence and leave with wrong information.
The problem compounds with agents. A human reader may sense that a doc is stale and verify before acting. An agent reads a doc at the start of every session and acts on it with confidence. Stale docs that humans dismiss with skepticism can cause agents to produce incorrect work for weeks before anyone notices.
Freshness Signals
Before building CI gates, it helps to understand what freshness signals exist and what each one is capable of catching.
The simplest signal is a last-updated date at the top of the doc. It requires no tooling and no automation — just a convention. The cost is also its value: it works only if the human who edits the doc remembers to update the date. Under pressure, they will not. The date will eventually lie.
A named owner in the doc header or in a CODEOWNERS file improves on this. PRs that touch files owned by a particular person route that person for review. When a code change could make a doc stale, the review system nudges the owner to check it. The failure mode: the owner leaves the team and the CODEOWNERS entry becomes orphaned.
CI link checkers are the first automated signal. Tools like markdown-link-check or remark-validate-links verify that every URL in your markdown files returns a non-4xx response, and that every internal anchor resolves. This catches dead links automatically — a renamed file or a deleted section breaks the link test, not a human review. The limitation: link checkers verify that files exist and URLs respond; they cannot verify that the semantic content is accurate.
Drift tests (introduced in the previous lesson) go further: they assert that specific claims in the documentation are still true. A README key-files test asserts that each path in the table exists on disk. A schema drift test asserts that required fields still match the TypeScript type. These are targeted, brittle-by-design — they fail when the documented state diverges from reality, which is precisely what you want.
The academy uses this pattern directly. The __tests__/content-integrity.test.ts file asserts the exact number of lessons in the platform. This is not a soft warning — it is a hard CI failure. When someone adds a lesson MDX file without registering it in lib/tracks.ts, the count assertion fails and the PR cannot merge. The test converts a missing registration from "easy to overlook" to "impossible to ship."
The Four Gates
A comprehensive docs CI layer has four gates, each catching a different failure class.
Gate 1: Link check. Every URL and internal anchor in markdown files resolves. Any file referenced in a table or link exists on disk. This is the baseline. It runs in seconds and catches a large class of stale references automatically.
Gate 2: Count integrity. If your project has a known structure — N lessons in a track, M endpoints in an API, K environment variables in a config — assert the count in a test. Count mismatches mean the structure changed without the corresponding registration. This is the gate that makes "add lesson, update tracks.ts" a forced pair rather than a convention.
Gate 3: Schema drift. For API references and interface contracts, a drift test pins the schema-to-type relationship. Any field rename, removal, or type change fails the test unless the schema is updated in the same commit. This gate protects consumers of the interface.
Gate 4: Required docs in PR. A GitHub Actions check that detects when a PR changes a code directory that has a corresponding documentation file, but the documentation file is not touched. For example: a PR that modifies src/api/ should be required to touch docs/api-reference.md if that file exists, or provide an explicit exemption. This is the most powerful gate because it catches semantic staleness, not just structural staleness — but it is also the most complex to implement and tune.
Reviewing Docs Like Code
A subtle shift in process that compounds over time: when you review a PR, diff the documentation too.
git diff origin/main...HEAD -- '*.md' '*.mdx'
Most PR reviewers look at the TypeScript and skip the markdown. But a rename in src/lib/tracks.ts that is not reflected in the README key-files table is a bug — it just does not manifest until the next agent session or on-call rotation.
The habit: when code changes, ask whether any doc claims something about that code. If yes, the doc is part of the diff. A doc that claims "the tracks are registered in lib/tracks.ts" is a contract. The PR that renames the file without updating that claim is an incomplete PR.
Deleting Dead Docs
The hardest lesson in doc maintenance is the deletion decision.
A doc that was once accurate but is now wrong falls into one of three categories:
- Superseded by a new doc: link to the new doc from the old location, then delete the old one after a migration period.
- The thing it documented no longer exists: delete immediately. There is no migration to manage.
- Partially stale: the parts that are wrong must be updated or removed. A doc where 40% is accurate and 60% is wrong is not a partially useful doc — it is a confidence trap.
The rule: if you cannot commit to maintaining a doc, delete it. A missing doc forces a question. A wrong doc provides a confident wrong answer. The wrong answer is always worse.
This is especially true for runbooks referenced during incidents. A runbook with one wrong step is not a 90%-accurate runbook — it is a document that will cause an incident responder to take a damaging action under pressure. Delete and rebuild, or leave it out of the on-call wiki entirely.
The Ownership Triangle
Every doc that will be maintained long-term needs three things: a structural home (where it lives in the repo), a named owner (who is accountable for keeping it current), and an enforcement mechanism (what ensures it actually stays current under pressure).
The enforcement mechanism is the one most teams skip. They add the owner but not the gate. The owner becomes responsible for a doc that has no automated check, so the doc stays current only as long as the owner has bandwidth. When the owner changes projects, the doc enters decay.
The CI gates above are the enforcement mechanism. They do not replace ownership — they make ownership meaningful. The named owner is the person who gets paged when a gate fails. The gate is what ensures the failure is caught.
Now build the enforcement mechanism from scratch: a vitest test that reads the README, extracts the key-files table, and fails when any referenced path no longer exists on disk.