Context Camp: Knowledge Management for Personal Agents
Why RAG isn’t dead, how I run personal AI agents on a portable Markdown vault, and three ways to retrieve context.
I had a fantastic time joining Hugo Bowne-Anderson and Doug Turnbull for Context Camp to talk about Knowledge Management for Personal Agents. An agent’s memory is only useful if it can retrieve the right thing at the right time. In the session, I shared what I’ve learnt from running my own personal agent setup on top of an Obsidian and Markdown knowledge base, arguing why RAG is far from dead and why robust indexing is the cornerstone of effective agentic search.
👉 Watch the full Context Camp recording on Maven
There is a common misconception that ever-expanding model context windows eliminate the need for retrieval. In practice, dumping an entire knowledge base into a prompt produces bloated token bills, slower responses, and subtle hallucinations. What works is disciplined context management: keeping your knowledge in a durable, app-agnostic store and giving your agents a versatile toolbox of retrieval methods.
One Vault, Several Agent Runtimes
The bedrock of my setup is a single, local Markdown vault. Rather than locking notes inside proprietary agent databases or siloed vector stores, plain text ensures that my knowledge remains completely independent of any single tool, model, or platform.
The same durable files serve multiple runtimes simultaneously:
Human interface: Obsidian for visual authoring, linking, and structured database-style Base views.
Always-on background agent: Hermes running on a remote host to handle autonomous tasks, research, and asynchronous triage.
Interactive coding harnesses: Tools like Codex, OpenCode, Warp, and Cursor for direct interactive work.
Shared instructions and skills: Every agent shares the same versioned
AGENTS.mdinstructions, note templates, and Agent Skills.
Because the vault is standard Markdown synchronised across machines, switching to a new agent or model requires zero data migration. You simply point the new tool at the vault.
How the Vault Is Structured
To keep a knowledge base of thousands of notes navigable for both humans and AI without bureaucratic folder hierarchies, the vault relies on a few simple conventions:
Flat, typed, and linked: Notes live at the vault root with explicit YAML frontmatter / Properties (
Type,People,Projects,Date). Frontmatter relationships use Obsidian wikilinks so connections form a navigable graph.The active working set (
Now: true): Instead of moving active files into temporary folders, active notes are marked withNow: true. When work concludes, toggling the property tofalsekeeps the record in place for historical discovery.Durable capture streams: Transcripts from Google Meet and Zoom (via Monologue), agent chat sessions, journals, and web clippings are imported into typed, structured notes rather than vanishing in ephemeral histories.
Versioned instructions: A repository-level
AGENTS.mdteaches agents vault conventions, routing rules, template standards, and safety boundaries.
Three Ways to Retrieve Knowledge
No single search method solves every problem. An agent relying solely on cosine similarity across chunked vector embeddings misses exact keyword filters, while basic keyword search fails on conceptual questions. A capable agent requires three distinct retrieval modes:
1. Structured queries (Obsidian CLI & Properties): When searching for known entities, types, or relationships (e.g.
obsidian search query='[Type:meeting] [People:Niko]'), structured property filtering is fast, deterministic, and highly token-efficient.2. Indexed search (QMD & Hybrid Lexical/Semantic Search): When the query is conceptual or the exact note title is unknown, hybrid search combining BM25 keyword matching, dense vector embeddings, and reranking identifies the most relevant passages across large corpora.
3. Direct file operations & code (
rg,grep, direct reads): For literal code inspection, regex searches, and grounded verification of candidate files.
The golden rule: Never answer purely from snippet fragments. Agents should use structured queries or semantic search to discover candidate paths, then perform direct file reads on complete source notes to verify evidence before answering.
A Complete, Runnable Demo
To accompany the talk, I built a complete, runnable demo repository based on a fictional research station—the Brackenlight Field Station. It packages Obsidian, the Obsidian CLI, Hermes, OpenCode, and a pre-warmed QMD service inside a single Docker container operating over a mounted Markdown vault.
The demo walks through a realistic evidence chain: following an inconsistent citizen-science moth census from debrief observations, through moonlight-bias synthesis, to an agreed protocol decision and a volunteer field kit.
Explore the Materials
From Personal Agents to Enterprise Scale
If you want to take these retrieval principles beyond personal knowledge bases and apply them to company-wide systems, product catalogues, and production evaluation pipelines, Doug Turnbull and Hugo Bowne-Anderson are running Build Production-Ready AI Agents for the Enterprise on Maven. It covers multi-step retrieval, filtering, reranking, and rigorous evaluation—and you can use code CONTEXT10 for 10% off.