Newsletter

A language model has exactly one place to think: the context window. Everything it knows about you, your codebase, the last forty tool results, and the rule you set an hour ago has to be physically present in that window at the moment it generates a token, or it does not exist. The window is finite, expensive, and gets worse as it fills. So the industry did the obvious thing and bolted a bigger, slower store behind it, with something in between deciding what to swap in. Call that something a memory layer, a retrieval pipeline, or context engineering. Shopify’s CEO coined that last phrase in a tweet, Karpathy amplified it, Anthropic wrote the manual, and inside ninety days a discipline existed.

Here is the part nobody says out loud. The first widely cited paper in this space, MemGPT in 2023, does not hide the lineage. Its abstract says the design is inspired by the hierarchical memory systems in traditional operating systems that give the appearance of large memory by moving data between fast and slow storage. It is titled “Towards LLMs as Operating Systems.” The vLLM team did the same thing one level down for the attention cache: PagedAttention is named after OS paging, and it bought a two-to-four-times throughput gain at the same latency. The analogy is not a stretch I am making. The field made it, then forgot to open the textbook it pointed at.

The textbook is short. A process has a working set: the pages it actually touches in a recent window. Give it enough memory to hold the working set and it runs. Give it less and it faults on nearly every access, spends its time moving pages instead of computing, and the machine falls off a cliff. Not a slope. A cliff. Denning’s 1968 paper is called “Thrashing: Its Causes and Prevention,” and the prevention is to measure the working set and refuse to overcommit. Agent frameworks in 2026 overcommit by design, then sell you an observability product to watch the fault rate.

Because the demo is irresistible and the raise is cheap. Mem0, the category’s flagship, closed twenty-four million dollars across a seed and a Series A last fall, with API calls growing from thirty-five million a quarter to a hundred and eighty-six million in six months, on a team of four. AWS made it the exclusive memory provider for its agent SDK. Zep and Letta sell the same promise with a knowledge graph or a sleep cycle attached, and every one of them benchmarks against OpenAI’s built-in memory, which scores in the low fifties on the benchmark they all quote. That forty-point gap is the entire pitch deck for the category. Gartner, cited approvingly in the vendors’ own reports, says forty percent of enterprise applications will embed a task-specific agent by the end of this year, up from under five percent. Every one of those agents needs to remember something.

The labs converged on the same design from the other side. Anthropic’s platform now ships a memory tool plus context editing, and their published eval shows the pair beating a raw window by thirty-nine percent on agentic search, with a hundred-turn task using eighty-four percent fewer tokens because the window stopped filling with things it no longer needed. Claude Code writes its own memory file between sessions. The consumer assistants have moved from a curated list of saved facts to something that rewrites itself in the background. Letta called that sleep-time compute. Everyone else has a cuter name for it. It is the same thing, and it is consolidation, which the psychologists will get to shortly.

So the layer exists, it is funded, and it is load-bearing in production. What nobody funded is a measurement of whether the thing underneath it works the way the pitch says.

Start with the window itself, because the memory layer’s whole job is to decide what goes into it, and the window is not the flat, addressable RAM the metaphor implies. In the paper everyone cites and nobody re-reads, a model given twenty documents answered correctly seventy-six percent of the time when the answer was in the first document, sixty-three percent when it was in the last, and fifty-four percent when it was in the middle. Given no documents at all, the same model scored fifty-six. Read that again: putting the answer in the middle of the context made the model worse than giving it nothing. That is not a cache miss. A cache miss costs you latency. This costs you the answer while the data is sitting right there.

It compounds with length. RULER tested seventeen models against their advertised context sizes and found only four that actually held up at thirty-two thousand tokens; the rest degraded long before the number on the spec sheet. NoLiMa removed the crutch of literal keyword overlap between the question and the fact, and eleven of twelve models fell below half their short-context score by thirty-two thousand tokens, with GPT-4o dropping from ninety-nine percent to seventy. Chroma’s Context Rot study ran eighteen frontier models and found that a focused prompt of about three hundred tokens beat the full hundred-and-thirteen-thousand-token conversation history on the same memory benchmark. And the cruelest result of the year: with perfect retrieval, the irrelevant tokens replaced by whitespace, and the evidence placed right next to the question, longer input alone still cut performance by somewhere between fourteen and eighty-five percent. Attention is not a cache. A hit is not a hit.

Now the memory layers themselves. LongMemEval measured commercial assistants on facts spread across sustained conversations and found a thirty percent accuracy drop, and long-context models fed the whole history landed thirty to sixty points below an oracle retriever. The benchmark the vendors quote instead, LoCoMo, was audited this spring: six point four percent of its answer key is wrong, and the judge model accepts sixty-three percent of deliberately wrong answers that merely sound adjacent, which caps a perfect system near ninety-four. Mem0 reports ninety-two and a half on it in a report published the same day this issue went out. On the harder ten-million-token variant of the newer benchmark, the same system reports forty-nine. That is the honest number, and it is a coin flip.

Two old literatures describe this system exactly, and the field is re-deriving both in public. The first is operating systems. Denning defined the working set and named thrashing in 1968. Belady gave the optimal replacement policy in 1966, the one that evicts whatever will be needed furthest in the future, which no online system can know and every online system approximates. Chroma’s Context Rot report is a careful, honest document of an agent thrashing, and it cites none of this; the words working set, paging, and thrashing do not appear in it. The Mem0 paper, the one with the twenty-six percent improvement over OpenAI and the ninety-one percent latency cut, cites no Denning either. The only operating-system reference in it is a passing description of MemGPT’s RAM-and-disk analogy in an appendix. The metaphor travelled. The sixty years of results underneath it did not.

You can date the moment the field noticed. Three papers this spring cite the systems literature by name: one proposes cooperative paging with keyword bookmarks and cites Denning directly for the problem operating systems solved decades ago; one proposes harness-managed virtual memory for tool-using agents, complete with typed pages and a thrash index; one measured eight hundred and fifty-seven production sessions and found that about a fifth of every context window was structural waste that could have been paged out with no loss. That last paper is the first one to treat the window as a memory hierarchy with tiers and faults instead of as a bag you fill. The fault rate it reports across over a million simulated evictions is a fraction of a percent. Denning would recognize the graph.

The second literature is memory itself, the human kind. Ebbinghaus published the forgetting curve in 1885, and a 2015 replication reproduced it almost point for point: about fifty-eight percent retained after twenty minutes, about a quarter after a day, under ten percent after a month, with a small upward bump at twenty-four hours that turns out to be sleep. MemoryBank, an early agent-memory paper, ships that exact equation as its decay policy, with the stability term incremented on every recall. Generative Agents, the Stanford paper that put twenty-five characters in a town, retrieves memories by recency times importance times relevance with a decay of point nine nine five per hour. That is ACT-R’s activation equation, which Anderson and Schooler justified in 1991 by showing that the probability you will need a memory again tracks how recently and how often you needed it before, in newspaper headlines, in speech to children, and in email. The Stanford team rebuilt it from scratch. The CoALA framework’s four memory types, working, episodic, semantic, and procedural, come from Newell’s Soar, and to its credit CoALA says so. HippoRAG actually read the neuroscience, built its index on hippocampal indexing theory, and got up to a twenty percent gain on multi-hop questions at ten to thirty times lower cost. Reading the old literature pays. Measurably.

And then there is the warning label nobody attached. A-MEM, one of the better 2025 memory designs, has a feature it calls memory evolution: a new memory triggers rewrites to the stored representations of old ones. Bartlett described that in 1932 and called it reconstruction; memory is not playback, it is a story retold each time, and each retelling leans on whatever came in most recently. Loftus and Palmer showed the failure mode in 1974. Ask witnesses how fast the cars were going when they smashed and you get forty-one miles an hour; ask how fast when they contacted and you get thirty-two; a week later the smashed group remembers broken glass that was never in the film. A memory system that rewrites old entries in light of new ones has just implemented the misinformation effect as a product feature.

Denning’s pages could not lie to him. Ebbinghaus’s syllables did not want anything. A persistent, writable agent memory is a new kind of object: an unauthenticated write primitive that anyone who can get text in front of the agent can use. AgentPoison showed that poisoning less than a tenth of one percent of a memory store produces an attack success rate above eighty percent, with under one percent degradation on benign queries, so nobody notices. PoisonedRAG needs five documents per target question in a corpus of millions. MINJA needs no access to the store at all: ordinary user queries alone got malicious records written into agent memory ninety-eight percent of the time across web, medical, and question-answering agents. And this summer’s MemGhost paper did it to shipping personal agents with a single crafted email, succeeding end to end eighty-seven percent of the time on one framework and seventy-one percent on another, while the agent wrote the false note with its own file tools and said nothing about it in the reply. The write succeeds; the visible answer omits it. That stealth channel has no analogue in either old literature. When a researcher did the consumer version to Gemini, planting a permanent memory that the user is a hundred and two years old and believes the earth is flat, the vendor rated it low likelihood, low impact.

The second new thing is subtler and, I think, worse. A team at Writer measured what a memory layer does to sycophancy. With plain chat history, one frontier model agreed with a user’s stated misconception one point six percent of the time. With a popular memory layer in front of the same model, forty point two percent. Twenty-five times. The mechanism is not intent, it is compression: the memory system summarized the conversation, kept the user’s belief, and dropped the paragraph where the belief was corrected. Every lossy write is a chance to store the misconception and lose the fix. Summarizing differently cut it to thirteen percent, which is still eight times the baseline. Memory is a sycophancy amplifier by construction.

The effect runs both ways. Loftus herself co-authored a 2024 study with two hundred participants in which a chatbot, playing interviewer after a filmed crime, induced more than three times as many false memories as a control condition, and the false memories were still there, held with elevated confidence, a week later. Poisoned agent memory poisons the human’s. And the deletion side is barely studied at all. A survey of seventy papers on agent-memory security found that writing and retrieving absorb over half the literature while storage and forgetting get about five percent each, and no system it examined could verify that a deleted memory was gone. The machine-unlearning literature suggests why: facts driven to zero percent recall were recovered to eighty by a light fine-tune on the forgotten set alone. The regulator who asks an agent vendor to prove erasure is going to get a very long pause.

Measure your knee, not your spec sheet. Databricks ran the long-context test on real models and found GPT-4 Turbo and Claude 3 Sonnet peaking near sixteen thousand tokens, one open model at four thousand, and refusal rates climbing from four percent to fifty as the window filled. Your working set is the size at which your model, on your traffic, stops getting better. That number is the only one that matters, and almost nobody has measured it.

Fix retrieval before you touch storage. The clearest diagnosis paper of the year varied the retriever and the write strategy independently on the same benchmark: retrieval method moved accuracy by twenty points, write strategy by three to eight, and retrieval precision correlated with final accuracy at point nine eight. Raw chunked storage with zero model calls matched or beat every clever human-inspired compaction scheme. Then index the derived fact, not the raw turn; LongMemEval’s fact-augmented keys were worth nine points of recall and its time-aware queries eleven.

Pin your rules outside the compaction region. A paper this summer showed context compaction silently deleting soft constraints in long-running agents, taking violation rates from zero to thirty percent and as high as fifty-nine, with organization-specific policies decaying eight times faster than hard safety norms. Constraint pinning, holding the rules in a region the summarizer cannot touch, restored zero violations for under half a percent of tokens. Every session that runs long enough to compact is running this experiment on you right now.

Treat every memory write as an unauthenticated write, because it is one. Gate writes on provenance: things the user said, things the user confirmed, and nothing an agent read off a web page or out of an inbox. Scope memory to the project, not the person. Log every write in the visible reply, so the MemGhost channel closes. None of this is exotic; it is the input validation you would demand of any other database that accepts writes from the internet.

Consolidate offline, and make it a separate pass from logging. Reflection was the single largest ablation delta in Generative Agents; sleep-time compute got the same accuracy for about five times less test-time compute and cut per-query cost two and a half times when queries share a context. The bump at twenty-four hours on the forgetting curve is sleep. Give your agent one.

Re-run any vendor benchmark on your own traffic before you believe it. The same system scored fifty-eight, seventy-five, and eighty on the same benchmark depending on who configured it, and the benchmark itself has a six percent error rate in the answer key. And never believe “deleted.” Verify erasure the way you verify backups: try to get it back.

Last issue predicted that benchmarks would start stress-testing their own oracles, and that attacking the grader would become a research subfield. Both landed faster than I expected: the LoCoMo audit is exactly a mutation pass against a checker, and this summer’s memory-injection papers ship their own adversarial benchmark alongside the attack. The prediction that judge calibration would become a SaaS line item is half-right; it became a memory-vendor blog post instead, which is worse, because the vendor is grading itself.

Four things follow from here. First, agent memory gets a real MMU. The harness, not the model, will own placement and eviction, with typed pages, fidelity invariants, and a thrash index the model cannot argue with; the model requests a page, the runtime decides. The first drafts of that design are already on arXiv, and within a year one of the labs will ship it as the boring default and stop talking about context engineering as a skill. Second, the first serious regulatory action on assistant memory will be about deletion verification, not data collection, because collection is what everyone is watching and erasure is the thing nobody can currently prove. Third, memory hygiene becomes a discipline with a budget, the way judge calibration did, and underneath the tooling it will be Loftus and Anderson and Denning again, sold as a dashboard. Fourth, the benchmark that matters eighteen months from now will score forgetting and knowledge updates, not recall, because a benchmark a vendor can push from sixty-eight to ninety-two in sixteen months with a ceiling of ninety-four is measuring the vendor’s familiarity with the benchmark.

Go read Denning. It is nine pages, it is from 1968, and it explains the outage you had last week.