The memory hierarchy: CLAUDE.md, memory vault, and persistent context

TLDR; Claude Code can retain selected project knowledge through CLAUDE.md and auto-memory, but it does not preserve the full conversation between sessions. Use CLAUDE.md for project rules, auto-memory for corrections and recurring project context, a separate vault for knowledge that transfers across projects, and files for task decisions that must survive compaction.

I spent forty minutes on a client’s Rails API one morning, establishing naming conventions, walking Claude through the domain model, agreeing on how polymorphic associations should be structured. Good session. Productive. Then I closed the terminal and went to lunch.

That afternoon, I opened a new session on the same project. Claude had no idea what we’d discussed. It suggested a naming convention we’d explicitly rejected. It asked about the domain model as if encountering it for the first time. Forty minutes of context, gone.

This is the fundamental constraint of working with Claude Code: a new session does not restore the full conversation that came before it. Claude Code can preload project instructions and selected auto-memory, but an architecture decision that lived only in yesterday’s chat may still be absent. The model has persistence mechanisms; they are selective, file-based, and only as useful as the information that reaches them.

Four layers address this problem, each at a different scope. CLAUDE.md handles project-level instructions that should persist across sessions. Auto-memory captures selected corrections and project observations. A memory vault captures knowledge that transfers across projects. Conversation context is working memory you manage actively. Knowing which layer to use for which kind of knowledge is the difference between repeating yourself and giving Claude a reliable path back to the decision.

CLAUDE.md

CLAUDE.md is the author-controlled layer of Claude Code’s long-term project context. It loads automatically at the start of ordinary sessions and resumed conversations. Custom subagents normally inherit applicable project instructions, while built-in Explore and Plan agents use narrower context.

The file system has four tiers:

1. Managed policy (an IT-deployed file outside the repo, e.g. /etc/claude-code/CLAUDE.md on Linux): organisation-wide instructions, present only where an employer has configured one.
2. ~/.claude/CLAUDE.md: global instructions, applied to every project. Your preferred language, coding style, absolute rules.
3. CLAUDE.md (project root): project-specific instructions, committed to git, shared with your team.
4. CLAUDE.local.md (project root): private project overrides, gitignored. Personal shortcuts that don’t belong in the shared config.

Claude reads all four in order, broadest scope to narrowest, and merges them into a single instruction set.

What belongs

The litmus test: would removing this line cause Claude to make a mistake? If not, cut it.

What survives that test:

• Build and test commands Claude can’t guess (npm run dev, bundle exec rspec, the flags that matter)
• Non-default conventions that aren’t in the code itself (naming patterns, architectural boundaries, forbidden patterns)
• Common gotchas specific to this codebase (the Tailwind v4 max-w-* bug in my own project, for instance)
• Hard boundaries: things Claude should never do in this project

What doesn’t belong: anything Claude can discover by reading the code. File structure explanations, module purposes, existing patterns: Claude can see all of this. Documenting it in CLAUDE.md pays tokens to tell Claude what it already knows.

The token cost

Every line in CLAUDE.md is recurring context. It loads at session start and in the subagent contexts that inherit project instructions. I audited a client project where CLAUDE.md had grown to 280 lines, roughly 8,500 tokens, consuming 4.3% of a 200K window before I’d done anything.

Across 15 sessions in a day, that’s 127,500 tokens just in CLAUDE.md loading. Not huge in absolute cost, but context window space is the scarcer resource. More instructions doesn’t mean more compliance. Lakshmi Narasimhan argues that frontier models reliably follow roughly 150–200 instructions before compliance decays, and Claude Code’s system prompt already consumes about 50 of those slots. Your CLAUDE.md gets maybe 100–150 effective instructions. After that, you’re paying tokens for lines Claude deprioritises.

I trimmed that 280-line file to 85 lines. Same output quality, and slightly better on one task, possibly because less irrelevant context left more attention for the instructions that mattered. Boris Cherny, who created Claude Code, is reported to run a roughly 100-line CLAUDE.md and sustains high-throughput PR shipping with multiple parallel Claude instances. His approach: add a rule only when Claude makes a specific mistake. If Claude hasn’t made the mistake, the rule is premature.

The target I use: under 200 lines for a project CLAUDE.md, Anthropic’s own figure, and as short as you can manage for global (mine is about 105). Move everything else to skills whose descriptions load at startup but whose full instructions and references load on demand, or to path-scoped .claude/rules/ files that load only for the files they cover. The skills guide covers this migration in detail.

The memory vault

CLAUDE.md handles what Claude should know about this project. But what about knowledge that transfers? The debugging insight you had on a Rails project that applies to every Rails project. The API quirk you discovered that’ll save time next quarter when a different client hits the same integration.

This is the gap that Claude Code’s built-in memory system partially fills, and the gap that led me to build something more.

The built-in: auto-memory

Claude Code ships with an automatic memory system. When you interact with Claude, it can save observations to markdown files in a per-project memory directory, indexed by a one-line pointer per file in a MEMORY.md entrypoint. The official docs describe the files as freeform topics Claude names itself, a debugging file, an API-conventions file; in practice I’ve also seen deployments type them explicitly, user (who you are), feedback (how you want Claude to work), project (what’s happening), reference (where to find things). Either way, the index line is the load-bearing part: it’s what decides whether a memory ever gets read again.

Auto-memory is useful. It captures preferences and corrections without manual effort. When I told Claude to stop adding trailing summaries to every response, it saved that as a feedback memory and stopped doing it in future sessions. When I corrected a collection naming issue in 11ty, it remembered the fix next time.

But the system has structural limits that matter as projects grow. The official documentation and a Zilliz/Milvus source analysis describe constraints worth planning around, as of v2.1.88:

• Startup preload limit: Claude Code loads the first 200 lines or 25KB of MEMORY.md at startup; deeper memory files remain available to read when referenced or retrieved
• No semantic retrieval: memory search is string matching, not intent-based
• No cross-project sharing: memory is scoped to the git repository (shared across its worktrees and subdirectories) and stays on one machine
• Session-scoped context: each conversation still starts fresh, loading memories anew
• No automatic cross-tool standard: other coding tools do not automatically adopt Claude Code’s memory conventions, although file-based memories can be shared deliberately; Claude Code itself reads AGENTS.md natively since v2.1.277

Community writeups also describe Auto Dream, a background consolidation agent said to merge duplicates, resolve contradictions, and prune stale entries after idle periods. Treat it as unconfirmed: as of September 2026 it appears in reverse-engineered analyses, not in Anthropic’s official docs or the Claude Code changelog, and at least one outlet describes it as unreleased. Housekeeping like that would be welcome when it lands; don’t build your workflow around it yet.

Cross-project knowledge

This is where the built-in system falls short and the community has gotten creative.

I wrote about this earlier when the memory system debate caught fire: Karpathy’s LLM Wiki, Graphify’s knowledge graphs, MemPalace’s spatial memory. Each tool is impressive, but they’re all solving the same underlying problem: knowledge that’s useful beyond the project where you learned it has nowhere native to live.

What I built is a private Claude Code plugin called memory-vault. The storage is deliberately simple: a folder at ~/.claude/knowledge-vault/ symlinked to iCloud Drive, so it syncs across every machine without a server, daemon, or vector database. Inside, entries are single markdown files sorted into categories, technical, solutions, decisions, domains, tooling, with frontmatter for tags and confidence levels.

Each entry follows a consistent structure: Summary, Key Findings, Gotchas, Related (with wikilink backlinks). The categories are coarse by design. Good tags on entries beat a deep taxonomy.

The filtering rule: would this help in a different project? If yes, it goes in the vault. If no, it stays in the project’s auto-memory. “GRDB’s DatabasePool uses WAL mode for concurrent reads” is vault material; it’ll save me time on any future Swift project. “The Client X API requires auth header format Y” is project memory, useless anywhere else.

The vault is now past 490 entries, and a structured markdown index navigated directly is still enough for my use: Claude reads the index and follows links without needing semantic search. I do not treat a particular entry count as a universal crossover. The point where vector retrieval earns its overhead depends on document size, query ambiguity, update frequency, and how well a manual index is maintained.

I know where my own line is because I’ve crossed it once, on a different kind of project. Knowledge Navigator, a local-first memory system I’m building where retrieval quality is the product itself, runs embeddings with a Recall@k eval harness, and swapping its embedding model means re-embedding every stored memory, gated on that eval. That machinery is justified there because recall is the whole point. For a personal knowledge vault, it’s overhead: markdown plus a disciplined index keeps working far longer than the vector-database pitch suggests.

Conversation context

The context window itself is memory, the most immediate kind and the most perishable. Every file Claude reads, every exchange, every tool output lives in the conversation until the session ends or compaction clears it.

This is covered thoroughly in the session craft guide, but the memory angle adds one thing worth stating: conversations compress, files don’t.

When auto-compact fires or you run /compact, conversation history is summarised to free context. The exact before-and-after size varies by model, context window, configuration, and release. The summarisation is lossy: an architecture decision from forty minutes ago might compress to a single sentence or disappear. A plan written to a markdown file on disk remains intact and can be read again after compaction or in the next session.

This is why the better habit is writing decisions to files, not to the chat. The pattern that holds up across the practitioners I’ve watched: annotate plan files inline across multiple rounds rather than discussing changes in conversation. The plan survives compaction because it lives on the filesystem. The conversation about the plan evaporates.

The practical pattern: early in any complex task, ask Claude to write the implementation plan to a markdown file. As decisions get made, have Claude update the file. When compaction fires, explicitly point Claude back to that plan if it is still needed. When you start fresh the next day, the plan is waiting, uncompressed, for Claude to read again.

Treat conversation context like working memory: use it actively, don’t trust it to persist, and externalise anything that needs to survive the session.

Which layer to use

If it’s a rule Claude should always follow in this project, CLAUDE.md. If it’s a correction or preference, auto-memory. If it’s transferable knowledge, the vault. If it’s a decision for the current task, write it to a file. If it’s temporary working context, guide the compaction.

The most common mistake is putting too much in CLAUDE.md. Deployment procedures, client-specific domain knowledge, API integration patterns: these feel like they belong in project instructions, but they’re situational. Move them to skills. CLAUDE.md should be the lean, essential instruction set that applies to every task in this project. Everything else loads on demand.

Building a vault that gets recalled

A vault nobody searches is a folder of files. Findability depends on three things.

Naming and descriptions

The memory file’s description field in frontmatter is the make-or-break detail. When Claude decides whether a memory is relevant to the current task, it reads the MEMORY.md index; each entry is a one-line description. Make it specific:

• Bad: “SwiftUI gotcha”, too vague to match any particular search
• Good: “SwiftUI alert bindings can’t use .constant(), must use @State binding”, searchable, specific, immediately useful

The description should answer: if I were searching for this knowledge six months from now, what words would I use?

The capture ritual

The hardest part of any memory system isn’t storage; it’s the moment of reflection at the end of a session where you ask whether anything here is worth keeping.

My vault plugin ships with a Stop hook that runs at the end of every session. It scans the transcript for keywords that suggest research or debugging happened, words like “turns out”, “figured out”, “root cause”, “workaround”, and if it finds enough of them, it nudges me:

This session involved research or debugging. Consider using /remember to save key learnings.

It never blocks. It never forces anything. Most sessions produce nothing. The ones that do produce a single entry, often a paragraph, that saves me real time the next time the same class of problem appears.

Capture is only half the ritual. The counterpart command, /recall, searches the vault at the start of a task, so the entry saved six months ago actually gets read instead of just archived. A vault you save to but never query is a diary, not a memory.

The keywords are tuned to how I work. I debug out loud and tend to say “turns out” when I’ve found a root cause. If you build something similar, pick your own triggers. The nudge is the mechanism; the keywords are just how you tune it.

Memory records what I learned; it doesn’t stop the model repeating a mistake. When a lesson keeps being ignored, I promote it to a hook rule with its own tests, the loop described in my hook-rules guide.

Structure for recall

Every vault entry follows the same four-section format: Summary, Key Findings, Gotchas, Related. This consistency is functional, not aesthetic. When Claude retrieves an entry, it knows exactly where to look for the actionable information. The Summary gives the quick answer. Key Findings provide the detail. Gotchas flag the traps. Related points to connected entries.

The [[wikilink]] backlinks in the Related section create a navigable web. When Claude finds one entry, the backlinks lead it to related knowledge it might not have searched for directly. The vault becomes a graph rather than a list, sparse and hand-curated, but enough to surface connections.

Past a few hundred entries, upkeep becomes its own job. I run a vault-curator agent periodically for hygiene: dangling wikilinks, orphaned entries nothing points to, singleton tags that should merge into an existing one, high-traffic entries that have gone stale. It reports rather than auto-edits, curation stays a human decision, but without the pass, a growing vault quietly rots into exactly the folder-of-files it was built to avoid.

A team that wants an assistant to answer from its own documents faces the same rot at a larger scale: stale sources give confident wrong answers, and building that retrieval with an owner for its sources is Applied AI work.

---

Related reading

• Session craft, when to start fresh, resume, or compact, the full session lifecycle and context management model
• Token economics, what Claude Code actually costs, understanding the cost of each memory layer in tokens
• Skills, hooks, agents, and commands, the definitive guide, moving knowledge from CLAUDE.md to on-demand skills
• Claude memory systems: the cross-project problem is the real one, the blog post that started the vault approach
• Claude Code hook examples: turning mistakes into tested rules, for the lessons memory alone doesn’t enforce

Sources

• Claude Code Docs, CLAUDE.md configuration, Official documentation for CLAUDE.md files and memory. Documentation
• CLAUDE.md Best Practices, Lakshmi Narasimhan, Argues for an instruction-compliance ceiling and gives sizing guidelines. Blog post
• How Boris Uses Claude Code, Boris Cherny’s ~100-line CLAUDE.md approach and parallel session workflow. Secondary source (fan-made collection)
• Claude Code Has 4 Memory Layers, Zilliz/Milvus, Source code analysis documenting memory architecture and structural limits as of v2.1.88. Technical analysis
• Auto Dream Mechanics, Claude Fast, Background consolidation agent phases and trigger conditions. Secondary analysis; feature unconfirmed in official docs as of September 2026
• Andrej Karpathy, LLM Wiki architecture, The pattern of incrementally building a structured markdown wiki rather than retrieving from raw documents. Primary source
• How Anthropic teams use Claude Code, Persistent plan artifacts and file-based context management. White paper

Common questions

How does Claude remember things between sessions?

It does not on its own, so three layers solve it: CLAUDE.md for project instructions, a memory vault for cross-project knowledge, and active context management within a session. Each layer handles a different scope of memory.

What is the difference between CLAUDE.md and a memory vault?

CLAUDE.md holds instructions for one project and loads every session. The vault holds knowledge worth reusing across projects and is recalled on demand. One is per-project rules; the other is portable, searchable findings.

How do you build a memory vault that actually gets recalled?

Write entries so they surface on the queries you will actually ask, and make saving and recalling part of the workflow. A vault only helps if its entries are findable and the habit of checking it is built in.