A paper posted to arXiv on 25 October 2026, “Memory Tax: Dissecting the Net Value of Agent Memory”, finds that “many dedicated memory systems underperform naive RAG in many cases”. Its authors are from the University of Illinois Urbana Champaign, Tsinghua University, Renmin University of China, Verdent.ai and the University of Waterloo.
On the LongMemEval-s benchmark with the GPT-5.6 Luna backbone, two-round retrieval over the raw history scored 90.0% in the paper’s table. The 11 memory systems tested scored between 63.0% and 84.0% on the same test.
Short explainer · Agent memory vs plain RAG
A vertical news Short narrated from the published text of this piece and related FourWeekMBA pieces.
An evergreen Business Engineer explainer on the difference between an AI model and the harness around it. It explains the general idea and does not discuss the ‘Memory Tax’ paper.
The key insight: As we read it, the paper measures memory against a baseline its authors say many memory papers overlook: they write that many designs compare mainly with other memories, while naive RAG “is often overlooked”.
The paper compares dedicated memory systems for AI agents with naive RAG, which keeps the raw interaction history and retrieves relevant chunks directly. The authors say they use RAG “as the comparison reference for assessing the net value of memory”.
They tested 11 open-source systems in four groups: flat stores (A-Mem, SimpleMem, MemOS), hierarchical or lifecycle systems (MemoryOS, LightMem, OpenViking), graph-based systems (HippoRAG2, PlugMem, MAGMA) and trained policies (Mem-T, Memory-R2).
The benchmarks were LongMemEval-s, PersonaMem-v2 and MINT-Eval, run on three backbones: Qwen3.5-4B, Qwen3.6-27B and GPT-5.6 Luna. The paper says that due to cost constraints it evaluated 20% of PersonaMem-v2 and 10% of MINT-Eval.
LongMemEval-s accuracy with the Qwen3.6-27B backbone for three memory systems, alone and with raw-history retrieval added and reranked, from Table 4 of He et al., ‘Memory Tax’, arXiv 2610.34073 (25 October 2026). Two-round RAG scored 81.4 on the same test (Table 1).
What Was Posted on X
RAG is so back!
Another paper showing that plain RAG keeps winning the agent memory comparison.
They show that many dedicated memory systems score below naive retrieval over the raw history on LongMemEval, PersonaMem, and MINT-Eval, while costing much more.
They split a memory system’s value into two parts. Memory benefit counts questions it answers that RAG misses. Memory tax counts questions RAG answers that the memory system gets wrong.
Why does the tax appear
Building memory rewrites the history. Some facts get dropped or distorted during extraction, and some survive but become harder to retrieve. Memory systems also vary more than RAG when you change the backbone model.
A simple fix helped. Retrieving from the raw history alongside memory and reranking both together reduced the tax.
The practical takeaway here is that before adding a memory layer to your agent, compare it with plain retrieval over the transcript on your own data.
Elvis Saravia wrote on X on 27 October that the paper is another one showing plain RAG keeps winning the agent memory comparison. He summarises that many dedicated memory systems score below naive retrieval over the raw history on LongMemEval, PersonaMem and MINT-Eval while costing much more, and that retrieving from the raw history alongside memory and reranking both together reduced the tax. His takeaway: before adding a memory layer to an agent, compare it with plain retrieval over the transcript on your own data.
Where Plain Retrieval Won
LongMemEval-s accuracy with the GPT-5.6 Luna backbone from Table 1 of He et al., ‘Memory Tax’, arXiv 2610.34073 (25 October 2026): two-round RAG 90.0 against six of the 11 memory systems tested; all 11 scored between 63.0 and 84.0.Animated: offline memory-building time on LongMemEval-s with GPT-5.6 Luna, from Table 19 of the ‘Memory Tax’ paper (100-corpus subset): RAG 0.11 hours, HippoRAG2 11.6, A-Mem 117.0 and MemoryOS 126.5.
The paper’s summary of its main table: “many memory systems fail to beat RAG in many cases”. On LongMemEval-s with Qwen3.6-27B, two-round RAG scored 81.4%; the best memory system on that test, A-Mem, scored 79.4%.
The result is not uniform. The same table shows some memory systems ahead of two-round RAG in some settings: on MINT-Eval with GPT-5.6 Luna, Mem-T scored 38.4% and PlugMem 37.8%, against 34.5% for RAG.
The Memory Tax, Defined
The authors compare each memory system with RAG question by question. Memory solves some queries RAG misses, and loses others RAG already solves. “We refer to the former as memory benefit and the latter as memory tax. When the tax outweighs the benefit, memory falls below RAG overall.”
They trace the tax to three causes. On faithfulness, sampled facts survived memory construction with no loss 39.4% of the time in OpenViking and 98.4% in MemoryOS. On accessibility, the paper says memory can reduce access “by fragmenting, diluting, or rephrasing it”.
On robustness, the paper says that with the memory construction fixed, “changing the QA backbone causes larger performance variation than naive RAG for many systems”.
What It Costs to Build Memory
The paper says many memory systems underperform naive RAG “despite substantially higher costs”. Its full LongMemEval-s results with GPT-5.6 Luna list offline building time of 0.11 hours for RAG, against 117.0 hours for A-Mem and 126.5 hours for MemoryOS.
The same table lists 51.7 million tokens of memory construction for Mem-T, a construction step RAG does not have. The Luna runs used a subset of 100 of the benchmark’s 500 corpora, the paper says.
The Fix the Paper Tested
As a proof of concept, the authors “retrieve from source history alongside normal memory retrieval and jointly rerank them before QA”. They report “consistent improvements and reduction of memory tax across several memory systems”.
With Qwen3.6-27B on LongMemEval-s, SimpleMem rose from 73.6% to 82.8% and its memory tax fell from 12.0 to 5.2. Mem-T rose from 75.0% to 80.0%, and MemoryOS from 73.4% to 76.4%.
As we read it, the result is about the baseline as much as the systems: on LongMemEval-s with GPT-5.6 Luna, two-round RAG scored 90.0 in the paper’s table, and no memory system scored above 84.0.
As we read it, the cost side runs the other way: the paper lists 0.11 hours of offline building for RAG on that test, against 117.0 hours for A-Mem and 126.5 hours for MemoryOS.
As we read it, the paper’s fix keeps memory and adds the raw history back: retrieving both and reranking lifted SimpleMem from 73.6 to 82.8 on LongMemEval-s with Qwen3.6-27B.
He et al., ‘Memory Tax’, arXiv, 25 October 2026
“When the tax outweighs the benefit, memory falls below RAG overall.”
Three Implications
FOR TEAMS BUILDING AGENTS The paper says naive RAG remains a strong baseline, and that many memory systems underperform it across diverse settings.
FOR MEMORY-SYSTEM MAKERS The authors say they hope their notion of memory tax can shift more attention from standalone performance to the net value of memory.
FOR BUYERS COMPARING TOOLS The paper reports token cost, wall-clock time, LLM calls and storage size alongside accuracy for every system it tested.
The framework puts memory among the new sources of advantage: “Orchestration, memory, tool use, and workflow execution have become the new sources of competitive advantage.”
As we read it, the ‘Memory Tax’ paper adds a test for that layer: whether a memory system answers more than plain retrieval over the same history. The framework was written in July 2026 and does not discuss this paper.
What Is Not Established
This is a preprint posted to arXiv. Scores are judged by an LLM backed by Qwen3.6-27B. PersonaMem-v2 and MINT-Eval were run on samples, and the GPT-5.6 Luna runs on LongMemEval-s, including the headline result, used 100 of its 500 corpora. The authors say naive RAG already scores high on these personal-agent benchmarks, “suggesting a retrieval-friendly task setting”.
The paper tests open-source representatives; it does not price or name commercial memory products, and its results may differ on other tasks or data.
This Week In The AI Supercycle
Execution Fault Lines, Concentrated Demand, and Geopolitical Firewalls
This week’s issue: the AI build is not short of demand for its parts. It is short of cheap money, and its end revenue rests on a narrow base of large, often venture-funded buyers.
A 25 October 2026 arXiv paper from UIUC-led researchers found that many of 11 agent memory systems scored below plain retrieval over raw history, and that adding that retrieval back alongside memory raised the scores of the systems it tested.
A note on sourcing. All results are from the arXiv preprint “Memory Tax: Dissecting the Net Value of Agent Memory” (He et al., 25 October 2026); the figures cited are from its sections 1 to 4 and 6 and its Tables 1, 2, 4 and 19. The X post by Elvis Saravia pointed us to it. This is not investment advice.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
