# Claude Code Memory System: 5 Practical Layers That Compound

Build a claude code memory system that compounds knowledge across sessions. 5 practical layers using GitHub and Notion — no Obsidian required.

Topic: Claude & MCP
Published: 2026-03-22
Updated: 2026-07-29
URL: https://productivitytech.io/claude-code-memory-system/

Imagine finishing a client task, pressing one button, and 30 seconds later your AI agent permanently knows what you learned — the new contact, the deployment quirk, the technical decision. Not just in this session. In every future session, across every project, on every machine.

That's what a claude code memory system does when it's built right. It stops the daily ritual of re-explaining your stack, your clients, and your conventions to an AI that forgot everything overnight.

I manage 15+ client projects across WordPress, Next.js, and Python backends. After six months of iteration, my claude code memory system uses five layers: a structured GitHub repo as the knowledge base, a routing index that replaces semantic search, and a Notion-driven ingest pipeline that captures knowledge where the work already happens. No extra tools required — this works with Claude Code, Cursor, Windsurf, Gemini CLI, or any agent that can read files.

Here's the exact setup, every file, and the reasoning behind each layer.

*This pattern formalizes what [Andrej Karpathy described in his “LLM Wiki” gist](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) — a persistent, LLM-maintained knowledge layer that compounds over time. The five layers below are how I implemented it for a real client workload. The full scaffold is [open-source on GitHub](https://github.com/Sinfjell/personal-memory-template).*

## Why Every Claude Code Memory System Article Says "Use Obsidian"

Search for "claude code memory system" and you'll find dozens of guides. Almost all of them recommend the same stack: Obsidian vault + smart-connections MCP server + qmd MCP server. The viral thread by [@nyk\_builderz](https://x.com/nyk_builderz/status/2030904887186514336) popularized this as a three-tier architecture — session memory, knowledge graph, ingestion pipeline.

The concept is sound. The tooling choice is a community bias, not a technical requirement.

The "tools for thought" community — Obsidian, Logseq, Roam — has been active on X and YouTube for five years. When AI memory became a topic, they were first to publish because they already had the vaults. That head start created an echo chamber where "persistent memory" became synonymous with "Obsidian."

**What Claude Code actually needs to build a claude code memory system:**

-   `Read` — to open a markdown file
-   `Glob` — to find files by pattern
-   `Grep` — to search content across files

That's it. These tools work on any directory of markdown files. Obsidian's graph visualization, community plugins, and wikilink syntax add zero value for the AI agent reading your files. Claude doesn't care if your notes are in `~/obsidian-vault` or `~/repositories/knowledge-base`. It reads the same markdown either way.

## The 5 Layers of a Practical Claude Code Memory System

A claude code memory system that compounds needs five layers. Each solves a different failure mode. Skip one, and the others lose leverage.

### Layer 1: [CLAUDE.md](http://claude.md/) — The Identity File

`CLAUDE.md` is the first file Claude reads in every session. Most people treat it like a config dump. The teams that get leverage treat it like a **teaching document** — telling the AI who you are, how you work, and what to never do.

What belongs here:

-   **Architecture decisions** that don't change weekly (stack, hosting, conventions)
-   **Workflow rules** (conventional commits, branch naming, PR process)
-   **Explicit boundaries** (never push to main, never SSH without approval, never log secrets)
-   **Tool preferences** (which SSH aliases map to which servers, which MCP servers to use)
-   **Client routing** — a pointer to where detailed client info lives

What does NOT belong: anything that changes often. Put shifting details in topic files (Layer 3) and reference them from [CLAUDE.md](http://claude.md/).

**Rule of thumb:** If you have to update [CLAUDE.md](http://claude.md/) more than once a week, you're putting too much in it.

### Layer 2: Auto-Memory — The Session Learner

Claude Code has a built-in auto-memory system at `~/.claude/projects/<hash>/memory/`. It persists observations across sessions — patterns it noticed, corrections you made, preferences you expressed.

The key file is `MEMORY.md`. It's loaded into every conversation automatically. The rule: **keep it under 200 lines.** It's a routing document, not a dump. Detailed notes go in topic files and get linked from [MEMORY.md](http://memory.md/).

This is the cheapest layer of a claude code memory system to implement. Enable it, let Claude learn from corrections, and it immediately stops re-asking questions you've answered before. But it has a ceiling — it can only hold what fits in the auto-memory directory, and it's scoped to one project path.

That's where Layer 3 changes everything.

### Layer 3: A GitHub Repo as Your AI's Brain

This is where my claude code memory system diverges from every Obsidian guide. Instead of a local vault with MCP plugins, I use a plain **GitHub repository** with structured markdown files.

The repo is called `personal-memory` and has this structure:

```
personal-memory/
├── MEMORY.md              # Routing index (under 200 lines)
├── ORG.md                 # Operating rules, review routine
├── clients/
│   ├── _TEMPLATE.md       # Standardized client file template
│   ├── acme-corp.md       # Per-client: hosting, stack, contacts, quirks
│   ├── northwind.md
│   └── ... (15+ clients)
├── engineering/
│   ├── patterns.md        # Reusable patterns (2+ projects)
│   ├── debugging.md       # Hard bugs: symptom → check → fix
│   └── architecture-principles.md
├── playbooks/
│   ├── deploy.md
│   ├── bloggproduksjon.md
│   └── incidents.md
└── policies/
    ├── security.md
    └── access.md
```

**The routing index pattern** is critical. `MEMORY.md` is a table of contents — one line per file, organized by question ("Which client?" → `clients/X.md`, "How do we deploy?" → `playbooks/deploy.md`). When Claude starts a session, it reads [MEMORY.md](http://memory.md/), identifies the relevant file, and reads only that. No semantic search needed. No MCP server. Just file paths.

**Why GitHub beats Obsidian for a claude code memory system:**

| Feature | GitHub Repo | Obsidian Vault |
| --- | --- | --- |
| Version control | Native git (diff, blame, revert) | Plugin-dependent |
| Access from VPS/CI | git clone | Requires sync setup |
| Collaboration | PRs, branch review | Not built-in |
| Automation | Webhooks, GitHub Actions | Community plugins |
| Search by AI agent | Glob + Grep (built-in) | MCP servers required |
| Mobile editing | [GitHub.com](http://github.com/) / any editor | Obsidian mobile app |

The `clients/_TEMPLATE.md` file enforces consistency. Every new client gets the same sections: overview, systems, deploy routine, contacts, notes. Claude knows exactly where to find hosting details or the SSH alias for any client — because the structure is identical across all files.

> **📦 Get the template repo**
> 
> I’ve open-sourced the exact scaffold I use — empty, MIT-licensed, ready to clone. Includes `MEMORY.md`, `CLAUDE.md`, `ORG.md`, `clients/_TEMPLATE.md`, plus the folder layout for engineering, playbooks, and policies.
> 
> 👉 [**github.com/Sinfjell/personal-memory-template**](https://github.com/Sinfjell/personal-memory-template) — click *“Use this template”* on GitHub to bootstrap your own private repo in 30 seconds.

### Layer 4: The Routing Index — How Claude Finds the Right File

The routing index is what makes a claude code memory system work at scale. Without it, Claude has to search across dozens of files. With it, Claude reads one document and navigates directly to the answer.

Here's a simplified version of mine:

```
# Routing Index

| Question | Document |
|----------|----------|
| Client hosting, stack, contacts? | clients/<name>.md |
| How do we deploy? | playbooks/deploy.md |
| Reusable code patterns? | engineering/patterns.md |
| Hard-to-debug issues? | engineering/debugging.md |
| Security and access policies? | policies/security.md |
```

When I say "we're deploying acme-corp," Claude reads the routing index, opens `clients/acme-corp.md`, and immediately knows the hosting provider, SSH alias, deploy procedure, and known quirks — without me explaining anything.

This pattern scales to any number of files. The index stays under 200 lines. The details live in topic files. Claude navigates by path, not by semantic similarity. It's faster, more predictable, and requires zero infrastructure.

### Layer 5: Notion-Driven Ingest — The Closed Loop

This is the layer that makes a claude code memory system compound instead of decay.

Most knowledge dies in the gap between "I learned something useful" and "my AI can find it next session." Ingest closes that gap with a single button press.

**The architecture:**

1.  I finish a task or meeting in Notion
2.  I press the **"Ingest"** button on the task or note
3.  Notion fires a webhook to a Fastify server running on my Hetzner VPS
4.  The server fetches the task context — title, description, comments, linked entities (client name, project, repo URL)
5.  It clones `personal-memory`, runs Claude Code with an ingest prompt
6.  Claude reads the routing index, finds the right file, updates it with non-trivial learnings
7.  Commits and pushes directly to main
8.  Posts a summary back to the Notion task as a comment — which files were updated, commit link, estimated cost

**A real example:** I completed a plugin compatibility task for a client (PR #20, version 1.0.28). Pressed "Ingest." Thirty seconds later, the client's memory file had three new entries: a new contact added as reviewer, a repo reference linked, and a technical decision note about choosing a more reliable detection method over a generic class check. Cost: $0.17. Time: 30 seconds.

**Why Notion and not a CLI command?** Because the work already happens in Notion. The task has the context — what was done, which client, which repo. A button press is zero friction. A CLI command means context-switching, remembering the right flags, and manually describing what you learned.

**But ingest is only half the loop.** The same webhook infrastructure supports a query path. I type `#claude what do we know about this client?` as a Notion comment, and the runner clones personal-memory plus the entity repo (if linked), runs Claude in read-only mode, and posts the answer back as a comment — with full conversation history for follow-ups.

This means I never leave Notion to query my knowledge base. The ingest button writes to memory. The `#claude` comment reads from it. Same infrastructure, same repos, bidirectional. You can even choose models: `#sonnet` for quick/cheap answers, `#opus` for deep analysis, `#new` to start a fresh session.

**How this differs from Claude Code Channels** (shipped March 20, 2026): Channels let you trigger Claude Code via Discord, Telegram, or HTTP. My runner predates Channels and is fully self-hosted — no dependency on a running Claude Code process. It spawns sessions on demand, handles its own job queue, and integrates directly with Notion's data model (entity relations, comments API). Channels is great for chat-style interaction; the ingest runner is built for autonomous, fire-and-forget knowledge capture — and now, for querying that captured knowledge without leaving your task manager.

## What Makes This a Claude Code Memory System (Not Just Notes)

The difference between notes and a claude code memory system is the **closed loop:**

1.  **Work happens** → in any repo, for any client
2.  **Knowledge is captured** → one button press triggers ingest
3.  **Memory is updated** → the right file in the right format, pushed to Git
4.  **Next session is better** → Claude reads the updated file, starts with full context

Every ingest makes the next session more productive. A client's quirky deploy process, a hard-won debugging insight, a new contact's name — captured once, available forever, across every repo and every machine.

The routing index + client templates mean this scales. Adding a new client takes 30 seconds: copy `_TEMPLATE.md`, fill in the basics, add to the routing index. Claude immediately knows how to work with that client.

## 5 Lessons After Managing 15+ Client Projects

1.  **Keep** [**MEMORY.md**](http://memory.md/) **under 200 lines.** The moment it becomes a dump, Claude starts ignoring parts of it. Treat it like a table of contents, not a document.
2.  **Only ingest what surprised you.** "Had a meeting" is not worth storing. "Client requires EHF invoicing to municipalities" is. The filter is: would future-you need this to avoid a mistake?
3.  **Templates beat freeform.** `_TEMPLATE.md` forces every client file to have the same sections. Claude finds information by position, not by guessing where you wrote it.
4.  **Git revert is your safety net.** Pushing docs directly to main feels risky until you realize that `git revert` is one command away. For knowledge files (not code), the speed of direct push outweighs the overhead of PR review.
5.  **Separate identity from knowledge.** [CLAUDE.md](http://claude.md/) is who you are and how you work. The memory repo is what you know. Mixing them creates a file that's too long, changes too often, and loses its purpose.

## Frequently Asked Questions

* * *

**The best claude code memory system is the one you actually maintain.** Obsidian, GitHub, a folder on your desktop — the tool matters less than the structure. A routing index under 200 lines, standardized topic files, and an ingest pipeline that connects to where your work already happens.

Start with [CLAUDE.md](http://claude.md/). Add auto-memory. When you outgrow it, create a repo with a routing index. When you want it to compound automatically, build the ingest loop.

Every session gets better. That's the point.

**Fastest way to start:** [use the template repo](https://github.com/Sinfjell/personal-memory-template), point your global `CLAUDE.md` at it, and edit one client file. You’ll have a working claude code memory system before lunch.