The Learning Loop: Lessons Files & Post-Mortems

The Correction That Resets to Zero

A coding agent makes a mistake. You catch it, correct it, explain the right approach. The session ends. The next session, the same agent is dispatched on a related task. The same mistake appears.

This is not a model quality issue. This is a documentation issue.

Agents do not retain session memory across sessions. Each new session starts fresh, with no knowledge of what the previous session discovered, corrected, or learned. The agent does not remember your explanation. It does not carry the correction forward. Without an external mechanism to persist that learning, every correction resets to zero when the session ends.

The external mechanism is a lessons file.

The Lessons File Pattern

The format is deliberately minimal. Three fields per entry:

Mistake. What went wrong, described concretely. Not "there was an issue with authentication" — "the sign-up form accepted a valid-looking response from the CAPTCHA provider but the token was already consumed, causing all subsequent retries to silently fail."

Root cause. Why it went wrong. The violated assumption, the missing test, the wrong inference. The root cause is not the same as the symptom. The symptom was "sign-ups failing silently." The root cause was "assumption that single-use tokens reset automatically on provider-side failure — an assumption that was not tested."

Rule. The actionable instruction that prevents recurrence. The rule must be specific enough that its absence can be verified — either by a code reviewer or by an automated test. "Single-use-token flows must reset the token on every failed call. The test matrix must include a fail→retry sequence." Both conditions are checkable.

## [2026-06-04] — Authentication / Single-Use Tokens

**Mistake**
A CAPTCHA widget consumed its single-use verification token on a failed sign-up
attempt. Subsequent retries silently failed — the token was already spent. The UI
displayed a success indicator throughout, masking the failure completely.

**Root cause**
The implementation assumed the CAPTCHA provider resets the token automatically
on a failed attempt. It does not. The test matrix covered only the happy path —
the fail→retry sequence was never tested.

**Rule**
Single-use-token flows must reset the token on every failed call before allowing
the caller to retry. The test matrix for any authentication flow using a single-use
token must include the fail→retry sequence: trigger a failure, confirm the token
resets, confirm the retry succeeds with a fresh token.

This entry is 12 lines. It takes 10 minutes to write after an incident. An agent reading it at the next session start has an active constraint: when implementing any single-use-token flow, check that the token resets on failure, and when writing tests for such a flow, include the fail→retry test case. The correction has persisted across the session boundary.

The Compounding Effect

The lessons file works because it accumulates. The first correction prevents the first mistake from recurring. The tenth correction prevents ten mistakes from recurring simultaneously. By the fiftieth entry, the agent is operating with fifty constraints that were purchased through fifty real incidents — each one cheaper to prevent than to re-encounter.

This is the compounding loop. Each incident generates one new rule. Each rule reduces the probability of that class of mistake in all future sessions. The initial cost of writing the rule is fixed; the prevention benefit repeats every session indefinitely.

The compounding effect is also why the rule must be written as a rule, not as a note. "We had a problem with single-use tokens once" is a note. An agent reading it learns: there was a problem once. "Single-use-token flows must reset the token on every failed call" is a rule. An agent reading it applies the constraint to the current implementation. The rule creates the prevention. The note does not.

When the Incident Is Bigger: Post-Mortems

For incidents with broader impact — extended outages, data integrity issues, security events, failures that affected real users — a lessons file entry is not sufficient. These incidents warrant a full post-mortem, sometimes called a Root Cause Analysis (RCA).

The post-mortem has more structure because larger incidents have more dimensions: multiple contributing factors, detection delays, alerting gaps, response process failures. A lessons file entry distills the outcome into a single rule. A post-mortem preserves the full causal chain so that the rule is understood in context.

The structure matters for a specific reason: the mitigation-resolution distinction. Most incident responses stop at mitigation — the service is back up, the immediate pressure is off. The root cause is still present. An incident that ends at mitigation will recur. An incident that ends at resolution — root cause closed, rule extracted, test matrix updated — will not.

The timeline section of a post-mortem exists to measure detection latency: how long between "something broke" and "someone knew." For the CAPTCHA incident above, the symptom started when the first retry silently failed. Detection happened when a user reported it on social media. The detection lag was hours. The alerting gap: there was no alert on sign-up failure rate.

The rule extracted from the detection lag is separate from the rule extracted from the root cause:

• Root cause rule: reset single-use tokens on failed calls, test fail→retry.
• Alerting rule: sign-up failure rate above a threshold must produce an alert.

Both rules go into lessons.md. Both prevent different aspects of the same incident class.

Blameless Post-Mortems

The post-mortem format described here is deliberately blameless. "The engineer failed to reset the token" is not a root cause — it is an assignment of blame that prevents real analysis. The root cause is a system-level failure: the implementation assumed behavior that the provider documentation contradicted, and there was no test to catch the assumption.

Blameless analysis finds the systemic failure so that fixing the system prevents recurrence for everyone, not just the specific engineer who made the specific mistake. The CAPTCHA token problem does not recur because the rule is now active in every coding session — not because the original engineer is more careful.

This matters in agent-driven development for a straightforward reason: agents do not learn from blame. An agent does not become more careful because you told it to be careful. It operates under constraints. The constraint extracted from the blameless root cause — "reset tokens, test fail→retry" — is something an agent can act on. Blame is not.

What a Good Rule Looks Like

The test for a good rule: can it be verified?

A verifiable rule:

• "Single-use-token flows must reset the token on every failed call." → Check: does the implementation reset the token in the error handler?
• "The test matrix for single-use-token flows must include fail→retry." → Check: does the test file contain a test that triggers a failure then a successful retry?

An unverifiable rule:

• "Be more careful with single-use tokens." → No test can verify this. No code reviewer can check it. No agent can apply it.
• "Document authentication flows better." → What does "better" mean? Better than what?

If the rule cannot be verified by a test or a code review, it is not a rule — it is a sentiment. Rewrite it as a verifiable constraint before adding it to lessons.md.

The InDecision platform once shipped a sign-up flow where the fail→retry path silently failed for exactly this reason. The rule extracted: all single-use-token flows must be accompanied by two mandatory test cases — the happy path and the fail→retry path. That rule is active in every coding session that touches authentication. The bug class is closed.

Now extract the rule yourself.

What's Next

The next lesson covers docs as contracts — API references, schemas, and type definitions where the doc IS the interface truth. ADRs record why decisions were made. Runbooks capture how operations are performed. Lessons files extract what went wrong into rules. Contract docs define what a system exposes to its callers — and the mechanism for ensuring the contract and the implementation never drift.