AI memory’s harder test: What should an agent keep, update or ignore?
New research shifts attention from whether an AI agent can retrieve a memory to whether that memory is still valid and worth using. The results are promising, but they are research findings—not a new memory feature in your assistant.
2026-11-15 · The Konshus Team · Sources checked before publishing
Want more news like this?
- One short email a week with the biggest changes in AI memory — ChatGPT, Claude, Gemini, new companies and research.
- Every story is checked against its sources before it goes out.
- No spam: just the weekly news, no sales emails, and we never sell or share your address.

An assistant that remembers an old preference can seem attentive. If that preference has changed, the same behavior becomes a mistake. And if the assistant spends time storing and searching details that never help, memory can make it less efficient without making it more useful.
That is the thread connecting several AI-memory papers released or updated in the November 12–15 period. They examine different agents and tasks, so their results cannot be rolled into one score. Together, though, they sharpen a practical question: What makes a memory worth keeping and acting on?
What changed
The clearest new test is Memora, a benchmark for personalized agents working with conversations that unfold over weeks or months. Rather than testing recall alone, it evaluates remembering, reasoning and recommending. Its authors also introduce Forgetting-Aware Memory Accuracy, or FAMA, a measure that penalizes an agent for relying on a memory that has become obsolete or has been invalidated. They report frequent reuse of invalid memories and difficulty reconciling changing information across the four language models and six memory agents they tested.
Other papers approach the decision from the agent’s side. MAPE searches for agent-memory designs through program evolution instead of relying solely on designs chosen by people. Its authors report gains of up to 12 percentage points in absolute accuracy over manually designed baselines on LoCoMo and LongMemEval, at 35% of the serving cost. CoReMem asks coding agents to account for the costs of writing and reading memories across successive repository tasks. On its ChainSWE evaluation, the authors report that a 27B actor’s task-resolution rate increased from 21.0% to 25.7%, while using fewer steps and fewer total cache-aware tokens than the comparison model.
These are research reports, not announcements that a consumer assistant has acquired a new memory feature. Their numbers also answer different questions: Memora tests how agents handle changing personal information; MAPE evaluates memory designs on established benchmarks; and CoReMem measures performance on coding tasks. None of those results, by itself, tells us how a familiar chatbot will treat a correction you make tomorrow.
The problem with remembering too well
A basic memory test asks whether an agent can recover something from earlier in a conversation. Memora’s more demanding question is whether it knows when not to rely on what it recovered. If a user changes a preference, a system may have both the old and the new statement available. Finding the old one is then not a success; using it could be the error the test is meant to catch.
That distinction matters because keeping a record and maintaining an accurate account are different jobs. An agent needs some way to recognize a correction, decide what it supersedes and avoid treating a once-valid detail as permanently true. Memora’s reported failures make this a measurable problem rather than merely a hypothetical one. They do not, however, establish that every deployed assistant fails in the same way: the paper evaluates the systems in its benchmark.
For readers, this suggests a better question to ask of any proposed memory feature. Do not stop at “Can it remember my preferences?” Ask what happens when you reverse one, when two past statements conflict or when a detail no longer applies. A system that can quote an old conversation may still need a reliable way to judge which parts deserve authority now.
A useful memory must earn its cost
CoReMem examines a different failure mode: saving information because it might help later, without adequately accounting for the effort of storing and retrieving it. For a coding agent moving through repository tasks, a note about one task could be valuable on the next. It could also consume resources to write and read while contributing nothing. The authors’ reported improvement is notable because it comes alongside lower step and token counts in their comparison, not simply from giving the agent more room to remember.
MAPE offers another route to improvement. Instead of assuming that one hand-built memory workflow is best, it evolves candidate designs and tests them. The authors report that effective patterns include retaining raw conversations as evidence and composing retrieved evidence for the question at hand. That points to a distinction between preserving a source and deciding what the source proves. A stored conversation can help an agent check its work; an isolated summary may be easier to use but may leave out the context needed to spot a correction.
Neither paper settles how much information a personal assistant should retain. A result on coding tasks does not establish the right policy for personal details, and an accuracy-and-cost result on benchmarks is not a privacy assessment. The useful lesson is narrower: “more memory” is not automatically the same as “better memory.” The information must improve a later decision enough to justify the work of keeping and using it.
Can an agent pick up where it left off?
Two more studies examine memory across interruptions. AstroMemBench tests memory systems for embodied spacecraft agents and compares six configurations. Its authors report that, after a session restart, Mem0 outperformed retrieval-augmented generation by 12.25 points in their evaluation. The setting is specialized, but the question is easy to grasp: can an agent resume a task with the right state after its immediate context is gone?
A revised paper on coding-agent memory post-training takes another approach. Its CAMG-II suite gives agents long-horizon environments in which they can use files as persistent memory across context replacements. The authors report gains over their strongest on-policy-distillation baseline and improvements on Terminal-Bench 2.1 and SWE-Bench Pro V2. Here, memory is not just a fact retrieved from a store. It can be a workspace the agent creates, revises, searches and reuses.
Those examples show why a single definition of “AI memory” can be misleading. Remembering a person’s changed preference, carrying forward notes about a codebase and restoring the state of an interrupted task require different decisions. A technique that helps with one may not transfer neatly to another.
What it means
For developers, these papers offer more discriminating tests. An agent should be evaluated not only on whether it recalls relevant information, but also on whether it rejects superseded information, avoids paying for unhelpful memories and resumes work accurately after an interruption. The studies offer candidate methods and reported results, not a universal recipe.
For everyone using an assistant, the central distinction is between retention and trustworthiness. Keeping a detail is only the first step. The harder work is knowing whether it remains current, whether it applies to the task and whether acting on it is better than asking again. This week’s research makes that harder work more visible. It does not yet show that everyday assistants have solved it.
Our take (opinion)
The most important shift here is from treating forgetting as a defect to treating selective forgetting—or, at least, declining to use an outdated detail—as a sign of good judgment. Memora is especially valuable for making an invalid memory count against an agent rather than rewarding it for retrieving something from the past.
We would be cautious about turning any of these paper results into a claim that personal-assistant memory is fixed. The studies use different tasks and comparisons, and reported gains need not carry over to a person correcting a sensitive detail in an ordinary conversation. Cost matters, too, but efficiency should not become an excuse to retain information indefinitely because storage is convenient.
A trustworthy assistant should be able to do more than repeat what it knows. It should have a defensible reason to use a memory now—and a way to stop using it when that reason disappears.
Questions
Was a new consumer AI memory feature launched?
No such launch is documented in the research covered here. These developments are papers and benchmarks, not a new feature available to users.
What is an invalid memory?
In the Memora research, it is a previously retained detail that has become obsolete or has been invalidated. The benchmark penalizes agents for relying on it.
Does more stored information make an agent better?
Not necessarily. The CoReMem work explicitly weighs the costs of writing and reading memories, while Memora tests whether agents avoid using information that is no longer valid.
Do these results prove my assistant will remember corrections?
No. The reported results come from specific research evaluations. They do not establish how a particular consumer assistant will handle a changed preference.
Sources
- Benchmarking Long-Term Memory for Personalized Agents
- Discovering Effective Agent Memory Designs through Program Evolution
- CoReMem: Cost-Internalizing Memory for Successive Coding Tasks with Prospective Valuation
- AstroMemBench: A Benchmark for Memory Systems in Embodied Spacecraft Agents
- Coding Agent Memory Post-Training II
The AI memory news, once a week.
If you're interested in AI memory, want your own AI to remember you better, or just want to stay up to date, this is the newsletter for you. Every week we comb through the news for the big shifts in AI memory — new companies, new features from ChatGPT, Claude and Gemini, new research and new ideas — and sum up what matters in one short email.
For people who use AI every day, builders and founders, and anyone curious where AI memory is going.
About Konshus: a private memory Vault that works across ChatGPT, Claude, Gemini and more. Learn more.