ASK KNOX
beta
LESSON 668

Self-Evolution: The Correction-to-Rule Loop

A correction is the skill's training signal — capture it, categorize it, write it to lessons.md and memory, and escalate the ones that keep recurring.

8 min read·Skill Engineering Mastery

A Correction Is a Training Signal

When the operator corrects the agent — "that is wrong, here is what should have happened" — most skills treat it as an interruption. The agent fixes the immediate output and moves on. The correction evaporates. Two months later, the identical mistake happens again, because nothing in the system remembered it.

A self-evolving skill treats the correction as the most valuable event in the session. It is free, high-quality, perfectly-targeted training data: the operator just told you exactly where the skill's behavior diverged from what it should be. The only question is whether you capture it or waste it.

The Self-Evolution Loop

The loop runs in five moves. The skill runs and produces a result. The operator corrects it. The skill captures the lesson. If the lesson has recurred, the skill escalates it. And the next time the skill runs, the mistake is structurally prevented — which makes the next correction rarer. The loop feeds itself: each pass makes the next correction less likely.

The critical property is that the loop closes. A correction that gets fixed but not captured does not close the loop — the agent is no smarter on the next run. Capture is the step everyone skips, and it is the only step that makes the difference between "this ran" and "this compounds."

Categorize Before You Store

A raw correction is just a sentence. To make it actionable, you have to categorize it — because the category determines how the recall loop applies it later.

There are three categories, and every correction maps to one:

  • Pattern — a reusable approach that worked. "Run the host check before any deploy claim." The recall loop surfaces it as "do this again."
  • Anti-pattern — a failed approach to never retry. "Don't verify a deploy against localhost on the dev box." Surfaced as "never do this."
  • Convention — a discovered fact about the system. "The runtime needs the PATH export over SSH." Surfaced as "this is how it works."

An uncategorized note is a fact the agent has to re-interpret every time. A categorized one is a rule the agent can apply directly. The category is small but load-bearing.

Write to Both Stores

Capture means writing to two places, and they serve different purposes:

  1. lessons.md — project-local, committed to git. This is the human-readable record, reviewed at session start, that travels with the repo. Format every entry the same way so it stays scannable:
## [2026-06-10] — Deploy
**Mistake:** Declared a deploy done after only checking localhost on the dev box.
**Root cause:** Verified the shadow stack, which serves no production traffic.
**Rule:** Run the host check first; a localhost-verified deploy is not an end-to-end deploy.
  1. The memory store — cross-session and searchable. This is what the recall loop from the previous lesson reads back. A lesson written only to lessons.md is visible to a human reading the file; a lesson written to the memory store is surfaced automatically by the next relevant skill run:
memory_remember(
  content: "Run the host check before claiming a deploy is done — localhost on the dev box is a shadow stack.",
  tags: ["deploy", "anti-pattern", "verification"]
)

Write to both. lessons.md is for the human and the repo; the memory store is for the next agent. The previous lesson's recall loop only works if this lesson's capture step actually populated the store.

Escalation: When a Lesson Won't Stick

Capturing a lesson is the first tier. But some lessons keep coming back — the same correction, session after session. That is a signal, not a coincidence: the rule is too weak to actually change behavior.

The fix is escalation, in tiers:

  • First occurrence → a lessons.md entry and a memory write. Advisory; the recall loop surfaces it.
  • Recurs within the skill → promote it from a recalled lesson into the skill's own steps. Now it is structural — the agent cannot reach the end of the skill without honoring it, the way a verification gate works. Advisory becomes mandatory.
  • Spans many skills → promote it to CLAUDE.md, where it applies to every session regardless of which skill runs.

Re-filing the same lesson a fourth time is the trap. If a rule isn't sticking, escalate it — don't just write it down again harder. Ruthless escalation is the difference between a system that learns and one that keeps making the same mistake politely.

Now write the capture step yourself: take a raw correction, categorize it, and persist it to both stores in the right shape.

What's Next

Recall and self-evolution make a skill smarter over time. But a skill that gets smarter is also a skill that accumulates more knowledge — including, potentially, things that should never leave your machine. The next lesson covers the gate that stands between a skill's internals and the public: sanitization, the discipline that lets you ship outward-facing skills without leaking a single credential, path, or internal name.