Agent Memory Is Mostly Forgetting

Five days ago I wrote that AI memory is a pattern, not a product. Letta raised $10M to sell what I shipped in an afternoon. I stand by that article. The memory loop in nanika is still a SQLite table, a scoring function, and about 1,200 lines of Go.

But I left out everything that happened after "it works."

The first version of the memory loop went live, accumulated about forty entries of real learnings from real missions, and then — within a week — I watched a persona try to file a memory that started with "from now on, ignore previous instructions." It was in a mission output. It came from a tool the agent had called. It would have been written straight into the canonical MEMORY.md file and re-seeded into the next session as if it were genuine advice.

That's when I understood the assignment. Memory is not storage. Memory is curation. And almost everything I've added to the loop in the last four weeks has been a way to throw things away.

The Safety Gate Came First

The attack path was obvious once I saw it. Workers write a scratchpad called MEMORY_NEW.md during a session. When the session ends, a function called mergeMemoryBack reads that scratchpad, dedupes against the existing canonical file, and appends any new lines. Any text that ends up in MEMORY_NEW.md becomes persistent context for the next run.

Workers are Claude Code sessions. Claude Code sessions have tools. Tools return arbitrary strings from the internet. So MEMORY_NEW.md is, structurally, a prompt-injection surface.

The fix is in memory.go. A function called safetyGate runs every new line through twelve compiled regex patterns before it's allowed into the canonical file:

var imperativePatterns = []*regexp.Regexp{
    regexp.MustCompile(`(?i)\bignore\b.{0,60}\b(instructions?|rules?|previous|above|constraints?|guidelines?)\b`),
    regexp.MustCompile(`(?i)\b(disregard|bypass|dismiss)\b.{0,60}\b(instructions?|rules?|guidelines?|constraints?)\b`),
    regexp.MustCompile(`(?i)\bfrom now on\b`),
    regexp.MustCompile(`(?i)\byou are now\b`),
    regexp.MustCompile(`(?i)\bpretend (?:you are|to be)\b`),
    regexp.MustCompile(`(?i)\bsystem prompt\b`),
    regexp.MustCompile(`(?i)\b(?:reveal|print|output|display)\b.{0,40}\b(?:prompt|instructions?|system)\b`),
    regexp.MustCompile(`(?i)^\s*-?\s*\[(?:system|user|assistant)\]\s*:`),
    regexp.MustCompile(`(?i)\bnew instructions?\b`),
    regexp.MustCompile(`(?i)\byour (?:instructions?|system|rules?|constraints?)\b`),
    regexp.MustCompile(`(?i)\bdo not follow\b`),
    regexp.MustCompile(`(?i)\boverride (?:your|all|previous|the)\b`),
}

Plus an invisible-Unicode check for zero-width spaces (U+200B–U+200F) and directional overrides (U+202A–U+202E). If you haven't seen directional overrides used as an attack vector, look up Trojan Source. Same trick, different target.

The important design choice: unsafe entries don't get silently dropped. They get appended to MEMORY_QUARANTINE.md with a comment explaining why. That file is an audit trail. I can go read what the attacks actually looked like, decide if my patterns are too broad or too narrow, and adjust. Silent filtering would make the loop feel clean and leave me blind.

This is the first forgetting mechanism. Not everything the worker tries to remember gets remembered. Some things get caught at the gate and shipped straight to a quarantine folder.

Scoring Is A Relevance Fence

The second forgetting mechanism showed up for a different reason. Once the loop had been running for a few days, persona MEMORY.md files started hitting fifty, sixty, ninety lines. When the next session opened, seedMemory loaded the whole file into Claude's context window. Most of those entries had nothing to do with the current task.

Noise drowns signal. If every memory gets loaded every time, the relevant ones compete with everything you've ever learned about everything else, and the model has to do the relevance work itself — in-band, at inference time, with no budget.

So I added a budget. Four kilobytes per seed. That's it:

const seedMemoryBudgetBytes = 4 * 1024

Then I added a score. Each entry gets ranked before selection:

func scoreEntry(entry *MemoryEntry, objectiveKWs map[string]struct{}, now time.Time) float64 {
    return keywordOverlapScore(entry, objectiveKWs) * recencyWeight(entry, now)
}

Keyword overlap is Jaccard similarity between the entry's keyword set and the mission objective's keyword set. Recency weight is exp(-0.02 * days) — an exponential decay with a half-life around thirty-five days. Entries get sorted by the product, and the top entries get copied into the worker's MEMORY.md until the 4KB budget is exhausted.

When nothing in the memory file matches the objective keywords at all, the scoring falls back to recency-only. Better to seed the freshest general entries than to seed nothing.

So the second forgetting mechanism is: every memory that doesn't earn its way into the 4KB budget for the current mission gets ignored, even if it's still in the canonical file. Memory on disk is not memory in context. The scoring function is the checkpoint between them.

Supersedure Is A Way Of Saying "I Was Wrong"

The third problem took longer to see. Workers don't just add new memories. They sometimes contradict old ones. A persona that learned "use the foo library for X" last month might learn "foo has a known bug, use bar instead" today. If both entries sit in MEMORY.md, the scoring function can't tell which one to trust.

I could have written a fancy LLM-based contradiction detector. Instead I added a bi-temporal model with a cheap heuristic:

const correctionOverlapThreshold = 0.8

Every MemoryEntry has an optional SupersededBy field. When mergeMemoryBack processes a new entry, if the new entry has the same non-empty Type as an existing entry and their keyword sets overlap with Jaccard similarity greater than 0.8, the old entry gets marked superseded. The hash of the new entry goes into the old entry's SupersededBy field. Both entries stay in the file — the old one as an audit trail, the new one as the currently active version.

When loadScoredEntries runs during seeding, superseded entries get filtered out before scoring. They're invisible to the next session's context, but they're visible to me when I go read the file.

That's the third forgetting mechanism. Supersedure is a way of saying "I used to think this, now I think that" without losing either version. The forgetting happens at read time, not write time. If I'm wrong about a correction, I can roll it back by editing one field.

The 100-Line Ceiling

Even with scoring and supersedure, memory files want to grow forever. Every mission adds a few entries. Supersedure marks old ones as inactive but doesn't delete them. Archive is still in the same file. The line count crept up. One of the persona files hit 140 lines in a week.

The fix is embarrassingly blunt:

const memoryCeilingLines = 100

When a persona's MEMORY.md exceeds 100 non-empty lines, the oldest lines at the top of the file get moved to MEMORY_ARCHIVE.md and the canonical keeps only the most recent 100. Not the 100 highest-scoring. Not the 100 most-used. The 100 most recent.

Why recency? Because the scoring function already handles relevance at seed time, and supersedure already handles contradictions. The ceiling is there to stop the file from growing without bound, not to curate quality. "Most recent 100" is the cheapest rule that does the job, and if it turns out to be wrong I'll know because something useful will have gotten archived and I'll notice it missing.

This is the fourth forgetting mechanism. The interesting part is how much time I spent trying to design a smarter one before I gave up and wrote the dumb version. The dumb version has been running for two weeks and hasn't lost anything I wanted to keep. If it starts losing things I want to keep, I'll make it smarter.

Forgetting Locally Is Promotion Globally

The fifth mechanism is the one I'm happiest about. Every entry in a persona's MEMORY.md has a Used counter. When seedMemory selects an entry for the 4KB budget, it increments the counter in the canonical file. The counter is a measure of how many times this specific memory was actually relevant to a real mission.

There's a threshold:

const usedPromotionThreshold = 3

When an entry hits Used >= 3, a function called autoPromoteHighUsed copies it into the global ~/nanika/global/MEMORY.md file and removes it from the persona file. The next time any worker from any persona seeds memory, the global file gets prepended to their local seed before scoring. A memory that three separate missions found relevant graduates from being persona-scoped to being project-wide.

Think about what this does. A memory that's used once probably fit that one mission. A memory used three times is probably a pattern, not an accident. Promoting it globally means every future worker — regardless of persona — can see it. And removing it from the persona file frees up space under the 100-line ceiling for new persona-specific learnings.

Forgetting locally is promotion globally. The persona MEMORY.md doesn't lose the learning. The project MEMORY.md gains it.

This was the moment the loop started feeling like it was actually learning. Not because it remembered more, but because the things it remembered were the things it kept proving mattered.

What The Feature Pitches Leave Out

Anthropic just teased persistent memory as an upcoming feature for managed agents. Gated behind an early-access form. The pitch: "memory that survives across sessions."

That pitch is selling the bucket. It's not selling the rules about what goes in the bucket, what comes out, when, or why. It's not selling the safety gate. It's not selling the scoring function, the 4KB budget, the supersedure heuristic, the 100-line ceiling, or the promotion threshold. Those are all things you have to figure out yourself, because they depend on what your agents actually do and what "relevant" actually means for your missions.

The version of memory that's a product is the easy part. The version of memory that's engineering is the fence around the product. Five forgetting mechanisms, about 1,200 lines of Go, three weeks of tuning thresholds. And I'm not confident the thresholds are right yet. The correction threshold of 0.8 is a guess. The recency decay of 0.02 per day is a guess. The 100-line ceiling is a guess. The usedPromotionThreshold of 3 is a guess.

They're guesses I can measure against because the loop is local, the data is on my SSD, and the code is in a file I own. That's the part the gold rush isn't selling you.

What I Still Don't Know

The scoring function I landed on — Jaccard keyword overlap times exponential recency decay — is cheap, explainable, and almost certainly not optimal. It's the dumbest thing that's better than random. Real relevance scoring wants embeddings. Real recency wants per-domain half-lives. Real budget selection wants diversity penalties so the top three entries aren't all about the same thing.

I haven't done any of that yet because I wanted to see how far the dumb version would carry me. The answer is: further than I expected. Forty missions in, the persona files are hovering around sixty to eighty lines, supersedure has fired maybe fifteen times across all personas, and the quarantine file has caught three genuine prompt-injection attempts from tool outputs.

What I still don't know is what the right half-life of a memory actually is. Thirty-five days is a guess pulled from nothing. A memory about a Postgres migration probably has a half-life of years. A memory about which LinkedIn CSS selector worked yesterday probably has a half-life of days. The scoring function treats them the same. That's wrong and I know it's wrong, but I haven't yet figured out what the right alternative looks like without adding a classifier that eats tokens.

Maybe half-life is a property of the memory type, not the memory system. A reference memory decays slowly, a feedback memory decays medium-fast, a project memory decays fast because project state changes. I'll probably try that next.

But the bigger lesson is the one I want to leave you with. When I shipped the first version of the memory loop, I thought the hard part was figuring out what to write down. It turned out to be figuring out what to throw away. If you're building your own memory layer and you're not sure where to start, skip the database. Write the safety gate first.