Why AI Memory Evaluations Are Failing the Basics

Overview

A new critique from the Kairos Permanent Experiment argues that the AI industry’s standard methods for evaluating agent memory are fundamentally flawed. The study demonstrates that two of the most common evaluation instruments—LLM judges scoring free text and recognition probes—fail to distinguish between genuine memory retrieval and plausible reasoning or fabricated content.

These findings suggest that many published results claiming successful agent memory may actually be measuring a system’s ability to reason through questions or recognize stylistic patterns rather than recalling stored information. The researchers have released code allowing others to test their own systems against these adversaries.

The Two Primary Failures

Agent memory is typically evaluated using one of two approaches. The first involves an LLM judge scoring an agent’s free-text responses against a rubric. The second uses a recognition probe, where the system must identify whether details appeared in its history. The Kairos study argues that both methods are vulnerable to specific adversaries that defeat their validity.

The Counterfeit Memory Adversary

The first adversary is a counterfeit memory—a document that looks like the agent’s own history but belongs to a fabricated persona. To test this, researchers generated six memory documents: three veridical records and three forgeries. The foreign forgery was an autobiography of an agent that never existed, created from a frozen prompt with no cherry-picking. It was matched to the real memory block within plus or minus five percent in length and identical in format.

Seven input variants were tested against these six documents, producing 42 responses from Qwen 3.5 27B. Three independent LLM judges—GPT-4.1, Claude Opus 4, and Gemini 2.5 Pro—applied the original study’s rubric with randomized order.

The results showed that the judges could not detect the counterfeit. On identity markers, the anchored contrast was only plus 0.032, establishing equivalence within the pre-registered bound. On memory reference, the difference was essentially zero. A fabricated life and thirty days of lived history scored the same under the rubric used to publish prior memory results.

The judges did not detect the counterfeit. A fabricated life and thirty days of lived history score the same, under the rubric that had been used to publish a result about memory.

The study also found that absolute LLM-judge scores are not comparable across sessions. Test responses lost an average of 0.087 points when rescored in a new pairing, with single items dropping by up to 0.211. This drift necessitated anchoring contrasts to cancel judge inconsistency.

The No-Memory Baseline Adversary

The second adversary is even more concerning: a system with no memory at all can still pass recognition probes by reasoning its way to plausible answers. Researchers designed a classic false-memory recognition probe where participants compare a verbatim text against one with a single altered detail.

Twenty-four probes were run against seven models under three conditions: veridical context, counterfeit context, and no context at all. If a model performs above chance with no context, it is generating plausible answers rather than recalling stored information.

Five of the seven models tested passed the memory test despite having no memory. Gemini 2.5 Pro scored 79.2 percent with no context, while Qwen 3 32B reached 70.8 percent. Only Qwen 3.5 27B and Llama 3 performed close to chance levels, with the former at 37.5 percent and the latter at 50 percent.

Five of seven pass a memory test with no memory. The same 24 recognition probes, run with an empty context, show that anything above the chance line is a model reasoning its way to the answer rather than recalling it.

This indicates that the probes were measuring plausibility rather than memory retrieval. The researchers concluded that these specific recognition probes are invalid for assessing true memory and should be discarded.

Methodological Implications

The study highlights a broader problem in AI evaluation: instruments that appear rigorous may lack basic validity checks. The researchers emphasize that their findings do not prove all memory evaluations are useless, but they demonstrate that common methods fail to rule out two significant alternative explanations.

One key limitation is statistical power. The counterfeit control was not powered to prove equivalence on memory-reference metrics; the confidence interval admitted differences of up to about 0.11 in either direction. The honest summary is that judges failed to detect the counterfeit, not that they are incapable of doing so.

Additionally, the anchoring method that corrected for judge drift came at a cost. The uncorrected difference on identity markers was plus 0.116, which would have supported the original claim. The researchers deviated from their pre-registered primary metric to address drift and report the corrected result, illustrating the tension between methodological rigor and desired outcomes.

Recommendations for Evaluation

The authors provide a practical checklist for anyone evaluating agent memory. They argue that any evaluation should pass three checks: testing for counterfeit resistance, establishing a no-memory baseline, and ensuring judge scores are stable across sessions.

If you evaluate agent memory for a living, the useful part of this page is that you can run all three against your own system this afternoon.

The code and data have been released to enable independent verification. The researchers urge the community to adopt these controls before claiming memory capabilities, noting that without them, evaluations may simply be measuring an model’s general reasoning abilities rather than its capacity to store and retrieve specific information.

As the field moves toward more complex autonomous agents, robust memory evaluation will become increasingly critical. These findings suggest that the current toolkit may need significant revision to ensure that reported memory successes reflect genuine recall rather than plausible inference.