Trust Scores and Contradiction Detection

Introduction

You have a memory store. Queries are running before tasks start. Information is being retrieved and applied. But here is a problem that emerges quickly in practice: not everything in your memory store deserves equal trust.

Some facts were entered carefully after deliberate decisions. Others were guesses made in the heat of a debugging session. Some are six months stale. Some are contradicted by something stored last week.

Trust scores and contradiction detection are the mechanisms that keep a memory store from becoming a liability. They are what separates a knowledge OS from a pile of potentially wrong notes.

Core Concept

Why Uniform Trust Fails

Imagine you are debugging a production outage. Under pressure, you store a quick hypothesis: "the timeout might be too low — try 30 seconds." Later that day, you find the real cause: a database index was missing. You store the correct finding: "add index on user_id column in events table."

You never go back and clean up the hypothesis. Three months later, an agent queries memory about timeouts. It finds your hypothesis — stored with the same weight as every other fact — and treats it as established fact.

This is the failure mode of uniform trust. Without confidence signals, the memory store cannot distinguish between:

• "I verified this in production, it is definitely true" (0.95)
• "I read this in a blog post and it seemed right" (0.5)
• "This was a guess under pressure, probably wrong" (0.2)

Trust scores make this distinction explicit.

Trust Score Basics

A trust score is a number between 0.0 and 1.0 assigned to each stored fact:

• 0.9-1.0: Verified, high-confidence fact. Directly observed or confirmed by multiple sources.
• 0.7-0.8: Likely correct. Verified in one context but not extensively tested.
• 0.5-0.6: Moderate confidence. Based on a reasonable inference or a single observation.
• 0.2-0.4: Low confidence. A hypothesis, a guess, or secondhand information.
• 0.0-0.1: Speculative. Should probably be labeled as such or not stored at all.

When an agent retrieves a fact with a trust score below a threshold (say, 0.6), it should treat it differently — not as established fact but as something to verify before acting on.

Trust scores should also decay over time. A fact stored with 0.9 confidence becomes 0.7 after six months without reconfirmation, because the world may have changed. Freshness matters.

Contradiction Detection

Contradiction detection is the process of identifying when a new fact being stored conflicts with an existing stored fact.

The naive approach: just store the new fact. The problem: now the store has two conflicting facts, and future retrieval is unpredictable — the agent might surface either one.

The correct approach:

Step 1 — Check for conflicts before storing. When a new fact arrives, query the store for related facts. If the new fact contradicts an existing one, do not silently overwrite.

Step 2 — Surface the contradiction. Tell the human: "I have 'the max retry count is 3' stored from 2026-01-15 at confidence 0.85. You are now telling me 'the max retry count is 5.' Which is correct?"

Step 3 — Resolve and update. Once the human confirms, update the store with the correct fact and mark the old one as superseded (not deleted — more on that in the “Pruning” lesson).

This pattern — detect, surface, resolve — keeps the human in the loop for any fact that matters. The agent should not make judgment calls about which conflicting fact to believe. That is a human decision.

Storage-Time Contradictions vs Retrieval-Time Supersession

The escalation rule above applies at storage time — when a new fact arrives that conflicts with what is already stored, the agent must not silently pick a winner. But there is a second, different situation: at retrieval time, the agent finds two already-stored entries about the same fact, and every signal points the same way — one entry is both more recent and higher confidence than the other.

That is not a genuine contradiction between peers. It is supersession: the store simply never cleaned up the stale entry. When recency and confidence agree, the agent should act on the newer, higher-confidence entry — and note in its output that an older entry appears superseded, so the human can prune it later. Stopping work to ask "which deployment region?" when one entry is 8 months old at 0.6 confidence and the other is 2 weeks old at 0.9 adds friction without adding safety.

The escalation rule kicks back in the moment the signals disagree: the newer entry has lower confidence, the confidences are close, or the fact is high-stakes. Then it is a real judgment call between conflicting facts — and judgment calls between conflicting facts belong to the human.

When Contradictions Are Not Contradictions

A sophisticated memory system distinguishes between true contradictions and context-dependent variation:

• "The timeout is 30 seconds" in the staging environment context
• "The timeout is 120 seconds" in the production environment context

These are not contradictions. They are facts with different scopes. Good memory entries always include scope: what environment, what project, what date range, what context.

A contradiction only exists when two facts claim to be true in the same scope at the same time. Building scope metadata into your memory entries (project: X, environment: Y, date: Z) dramatically reduces false-positive contradictions.

Practical Application

Here is how trust scores and contradiction detection play out in a production semantic memory layer used by an agent framework and coding agent sessions daily.

A fact is stored: "The order execution service uses limit orders only. NEVER market orders in this context. Market orders are explicitly disabled." This is stored with confidence 0.95 — it was established after a real incident where the wrong order type caused unexpected fills.

Weeks later, an agent is helping configure a new trading pair. It discovers a configuration pointing to market order mode. The contradiction detector fires: "Conflict detected — new configuration uses market orders, but stored high-confidence fact says limit orders only."

Rather than silently using the new configuration or automatically overwriting the stored fact, the agent surfaces both: "I see a conflict. The configuration I found uses market orders, but stored memory says limit orders only at 0.95 confidence. Which should I follow?"

This is the right behavior. The stored fact was correct. The configuration was a mistake. Because the agent caught the contradiction and escalated rather than resolving it silently, a potentially expensive error was avoided.

Common Mistakes

Assigning high confidence to everything. If everything is 0.9, the score is meaningless. Be honest about certainty levels. Hypotheses get 0.3. Gut feelings get 0.4. Verified facts get 0.8-0.95. Reserve 1.0 for invariants that are definitionally true.

Letting the agent resolve contradictions silently. An agent silently choosing between conflicting facts is making a decision the human should make. Always escalate contradiction resolution. The agent's job is to detect and surface; the human's job is to decide.

Not decaying scores over time. A 0.9 fact from two years ago that has not been reconfirmed is not a 0.9 fact today. Build time-based decay into your trust scoring. Even a simple rule — "drop 0.1 per 90 days without reconfirmation" — dramatically improves reliability.

Confusing low confidence with uselessness. A low-confidence memory is still useful — as a lead to investigate, not as a fact to act on. Label it clearly ("possible: the issue may be related to X") and use it to guide inquiry rather than as a basis for action.

Summary

• Trust scores (0.0-1.0) encode how reliable a stored fact is — without them, guesses and verified facts look identical
• Trust scores should decay over time — a stale high-confidence fact is no longer high-confidence
• Contradiction detection catches when new facts conflict with stored ones before they corrupt the store
• The correct contradiction response: detect, surface both facts, escalate to human for resolution
• Retrieval-time supersession is different: when recency and confidence both favor the same stored entry, act on it and note that the older entry appears superseded
• Not all apparent contradictions are real — scope metadata (project, environment, date) resolves most false positives
• The agent detects and surfaces; the human resolves

What's Next

Trust scores help you evaluate what you have stored. But what makes a memory worth storing in the first place? The next lesson covers the one-fact rule and what distinguishes a durable memory from noise — because the discipline of writing good memories is just as important as the discipline of querying them.