Building a Knowledge OS — The Semantic Memory Layer Pattern

Introduction

Files and CLAUDE.md handle session context. Memory files handle project state. But neither can answer questions that cross projects, span months, or require semantic understanding of "what's similar to this problem I solved before?"

That is the problem the knowledge OS solves. Knox's implementation — Semantic Memory Layer — is a production system running on the production server at port 8002, serving 10 MCP tools across 2,968 indexed chunks from 43 repos.

This lesson teaches the architecture, the query-first habit, and the specific tools that make semantic memory a practical engineering advantage.

Semantic Memory Layer Architecture

The system has four layers:

Clients (producers + consumers): Every Claude Code session and the Agent Gateway agent produce memories (via memory_remember) and consume memories (via memory_query, memory_recall). The 10 MCP tools in the memory-service server are the interface.

Ingestion layer: The ingest-corpus GitHub Action runs on every push to main in the academy repo, embedding new MDX content into ChromaDB. memory_remember handles agent-authored memories. memory_reindex force-reindexes when the index drifts from disk state.

Storage layer: ChromaDB holds vector embeddings across 6 namespaces: claude-code-memory, trading-kb, project-docs, lessons, knowledge-base, daily-logs. Each memory has metadata: namespace, date, project, confidence score.

Service layer: FastAPI backend at port 8002 on the production server. NOT on your laptop — running both simultaneously creates a split-brain condition where two instances share the port (SO_REUSEPORT) serving divergent ChromaDB states. The canonical deployment is Docker on the production server only.

The 10 MCP Tools

The 10 tools in the memory-service MCP server cover the full knowledge OS workflow:

Tool	Purpose
memory_query	Full-content semantic search — returns results with memory_id
memory_search	Compact discovery — broad, cheap, returns summaries
memory_recall	Single memory fetch by ID — token-efficient deep dive
memory_remember	Store a new memory with metadata
memory_forget	Soft-delete a memory (marks as deleted, not removed)
memory_context	Session context assembly — pass a topic, get a curated bundle
memory_gaps	Identify missing coverage for a topic
memory_events	Query for time-bound events
memory_status	Index stats — chunk count, namespace breakdown
memory_reindex	Force-reindex when index drifts

The Query-First Habit in Practice

The habit is simple: before the first file read of any session, run memory_query("project-name").

What this surfaces for a trading-bot session:

• The Phemex unit semantics lesson (qty=1 as 1 BTC — critical before any live order work)
• The USDT-margined futures only constraint (the trading bot ONLY uses the USDT futures wallet)
• The launchd supervision setup (the trading bot service + its watchdog service)
• The runtime config override path (the bot's admin API PATCH /config overrides .env)
• Prior session decisions about pair expansion candidates

Without the query-first habit, a session that modifies the order submission code risks:

• Missing the Phemex unit semantics constraint and submitting a dangerously wrong qty
• Not knowing the USDT-margined only constraint and accidentally touching spot wallet logic
• Missing the launchd supervision setup and using the wrong restart mechanism

The Semantic Memory Layer query surfaces all of this in one call — before any code is touched.

Query vs Recall: The Decision Tree

The two-tool pattern is the most efficient query workflow:

1. memory_search or memory_query for discovery — broad semantic search returns summaries and memory_ids
2. memory_recall(memory_id) for content — single-hop fetch of the specific memory

The memory_query results explicitly include memory_id. Chain them: memory_query("trading bot exchange limits") → results include memory_id → memory_recall(id) → full text.

This two-hop pattern uses fewer total tokens than running memory_query twice or using speculative keyword searches.

Namespaces for Context Isolation

The 6 namespace design prevents context pollution across domains:

• claude-code-memory — behavioral feedback, session protocols, agent discipline rules
• trading-kb — Phemex semantics, Polymarket API, USDC.e vs USDC, Foresight calibration
• project-docs — architecture notes, project state, deployment configs
• lessons — project-level lessons from lessons.md entries
• knowledge-base — academy content, deep-dives, research
• daily-logs — Agent Gateway operational logs by date

When querying for trading context, filtering to trading-kb namespace prevents irrelevant academy lessons from contaminating the results. When auditing a session for context engineering patterns, claude-code-memory is the relevant namespace.

memory_query accepts namespace filters. Use them.

Building Your Own Knowledge OS

You do not need a full Semantic Memory Layer deployment to practice the knowledge OS pattern. The minimum viable knowledge OS:

1. memory_remember after every session — even a one-sentence summary of what was decided
2. memory_query before every new session — start with the query, then read files
3. lessons.md as structured supplement — the formatted lessons that Semantic Memory Layer picks up via ingest

At 50 memories, semantic search starts returning noticeably relevant results. At 200 memories across 3+ projects, cross-project pattern detection becomes possible — "this problem in the trading bot resembles the problem I saw in foresight three months ago."

At 2,968 chunks, the knowledge OS becomes the most valuable asset in the system. It is irreplaceable in a way that the model is not — you cannot recreate 2 years of accumulated session context from a new Claude subscription.

Summary

• Semantic Memory Layer: FastAPI + ChromaDB, port 8002, production server only, 10 MCP tools
• Query-first habit: memory_query("project") before any file read in every session
• The two-hop pattern: memory_search/query for discovery → memory_recall for content
• 6 namespaces prevent cross-domain context pollution
• Build your own: memory_remember after sessions, memory_query at session start, lessons.md as structured supplement

What's Next

The next lesson confronts the failure mode that degrades every knowledge OS over time: context rot. Staleness, contradiction, and the freshness signal — why stale context actively misleads and how to detect it before it causes damage.