RAG Is a Production System, Not a Demo
RAG has a failure taxonomy, SLOs, and an owner — treat it like any other production service or it will lie to users at scale.
What RAG Actually Is
Retrieval-augmented generation is the practice of answering a question using a model that has access to a dynamically retrieved slice of a knowledge base, rather than relying purely on what was baked into its weights at training time. The pipeline has three discrete stages: retrieve matching content from an index, augment the prompt with that content, and generate an answer grounded in it.
That three-word description understates the operational complexity. Each stage can fail independently. Each failure mode produces a different class of wrong answer. And none of those failures look like an error — they look like a confident, well-formed response that happens to be misleading.
This track uses a concrete running example throughout: this academy's own AI tutor. When you ask a question in the /learn interface, it embeds your query, retrieves the most relevant lesson chunks from a pgvector index, assembles them into a prompt alongside your question, and generates a cited answer. That pipeline — corpus of MDX lessons → embedding → pgvector retrieval → cited generation — is not theoretical. It has SLOs, it has had incidents, and it has taught us exactly which failure modes are expensive to debug.
Why Naive RAG Lies
A naive RAG implementation does the minimum: embed some documents, retrieve the top few by cosine similarity, paste them into a prompt. In a demo, this works. In production, it lies in four ways.
Stale corpus. The index is a cache of the corpus. Every cache needs invalidation. If a lesson is rewritten and the index is not updated, every question about that topic gets an answer sourced from the old version. The model does not know the source is stale — it answers with the same confidence it would show for fresh content. Users can go weeks receiving outdated information while the source documents are correct and the engineers believe the system is healthy.
Bad chunks. The chunk size decision at ingest time propagates forward into every query. Chunks that are too small give the model precise retrieval but starved context — the retrieved passage is accurate but too short to contain the full explanation. Chunks that are too large dilute the similarity score, making the retrieval system less precise. There is no chunk size that works for every document type, which is why structure-aware splitting is a production discipline rather than a configuration knob.
No thresholds. A system with no similarity threshold will always return something — even when the query has nothing to do with the corpus. The model then synthesizes an answer from loosely related content. From the user's perspective, this looks like confident hallucination. From the system's perspective, retrieval "succeeded."
No receipts. An answer without citations cannot be verified. A user who wants to confirm a claim has no path to the source. An engineer debugging a wrong answer has no trail to follow. Citations are not a UI nicety — they are the auditing mechanism that makes RAG trustworthy.
RAG vs Long Context vs Fine-Tuning
These three approaches are often presented as competing. They are not — they answer different questions.
Fine-tuning bakes knowledge into the model's weights. It is expensive, irreversible for a given checkpoint, and produces no attribution. It is right when you need the model to have a different style or to handle a specialized format, not when you need it to know fresh facts.
Long-context prompting passes a large document directly in the prompt. It works when the relevant content is a known, bounded document — a single API specification, a known codebase. It does not work when the corpus is large, changing, or user-permission-gated: you cannot fit the entire academy into a single prompt, you cannot update it without changing the prompt, and you cannot show different users different subsets.
RAG wins when knowledge needs to be fresh, attributable, or access-controlled. The index can be updated incrementally without changing the model or the prompt. Every retrieved chunk carries a source ID that becomes a citation. Retrieval can be filtered with a WHERE clause that gates which chunks each user can see.
The academy's tutor uses RAG for exactly these reasons: lessons are added and updated constantly, every answer should cite a lesson, and future work may tier answers by user subscription level.
The Production Frame
The difference between a RAG demo and a RAG production system is the presence of three things: SLOs — service-level objectives, the measurable targets a system promises to hold (a max staleness, a min faithfulness) — an owner, and a defined failure taxonomy.
A production RAG system has a freshness SLO: the maximum allowed lag between a source document update and the corresponding chunk being re-embedded. This academy's system triggers an ingest job on every lesson merge to CI — that is the freshness SLO enforcement mechanism.
It has a faithfulness SLO: every answer is expected to ground its claims in the retrieved chunks. An answer that makes a claim not supported by any cited chunk is a faithfulness violation, measurable with an LLM-judge.
It has an owner: someone who receives the alert when the freshness SLO is violated, who investigates when recall drops on the golden set, and who is responsible for the corpus quality gates.
And it has a named failure taxonomy — the six modes in the diagram above — so when something goes wrong, the debugging conversation starts at the right stage of the pipeline rather than treating the system as an opaque black box.
The Academy as a Running Example
Every lesson in this track is grounded in the same pipeline. The corpus is the lesson MDX files in the academy repository. The ingest gate checks for PII and credentials before any chunk is embedded. The chunker uses structure-aware splitting on Markdown headings so code blocks are never split. The index is pgvector — the Postgres extension that stores embedding vectors and runs similarity search — with an HNSW index (a graph structure that makes nearest-neighbor lookups fast) on the embedding column. Both are defined fully in the embeddings lesson.
When you work through the BuildChallenge exercises in this track, you are implementing pieces of a real system — not a toy. The typed pipeline skeleton in this lesson's challenge enforces the stage contracts as TypeScript types, so a misconnected stage produces a compile error rather than a runtime bug discovered in production.
The real system has had incidents. A content update was not re-embedded for several days because a cron job was not configured after a deployment. Users received stale answers about a lesson that had been substantially rewritten. The fix — a CI hook that triggers the ingest job on every content merge — is the exact pattern the re-embed lifecycle lesson covers in detail.
Understanding the full pipeline before diving into any individual stage is the right starting point. Every decision you make about chunking, every gate you add at ingest, every threshold you set at retrieval — all of it affects the quality of every answer downstream. RAG is a system. Build it like one.