Benchmarks

BEAM leaderboard

BEAM (Beyond a Million Tokens, Tavakoli et al., ICLR 2026) measures long-term memory in AI agents. It asks 2,000 questions over 100 conversations at four history sizes: 100K, 500K, 1M and 10M tokens. Harny scores 85.1 at the 128K tier, the best published score there.

BEAM accuracy by history size

Harny’s BEAM results by history size.

Harny85.1

BEAM score by history size: Harny scores 85.1 at the 128K history tier.
System	100K	500K	1M	10M	Source
Harny	85.1	—	—	—	harny.ai

The BEAM paper, published at ICLR 2026 →

How to read the table

Harny’s figure is a full BEAM split: all 400 questions at the 128K tier, across all ten abilities, scored on BEAM’s own rubric with the judge gemini-3.5-flash-lite.

Results at each size

Harny’s results at the history size measured. For every other published result at each size, use the public leaderboard.

100K tokens

System	Accuracy	Source
Harny	85.1	harny.ai

Results published in research papers

These results come from the BEAM paper itself, at the same smallest tier.

BEAM results from the benchmark paper: its own method and its RAG baseline.
System	Size	Accuracy	Paper
LIGHT, the BEAM paper’s own method	100K	35.8	Tavakoli et al., 2026
RAG baseline	100K	32.3	Tavakoli et al., 2026

What BEAM measures

BEAM holds long conversations at four sizes and asks questions whose answers sit somewhere in that history. Each question tests one of ten abilities: abstention, contradiction resolution, event ordering, information extraction, instruction following, knowledge update, multi-session reasoning, preference following, summarization, temporal reasoning.

BEAM history sizes, with the number of conversations and questions at each size.
History size	Conversations	Questions
100K tokens	20	400
500K tokens	35	700
1M tokens	35	700
10M tokens	10	200

The paper →The dataset and code →Harny results by ability →

Another benchmark named BEAM

Eywa (Joshi, 2026) reports results on a 700-question technical-memory benchmark that is also called BEAM. It is a different benchmark from the one on this page, and its scores are not comparable to the results here. Eywa on arXiv →

Keep reading

Benchmark results and the per-ability breakdown How to benchmark agent memory LoCoMo results

Run it on your data

Ask Harny about changed facts, earlier sessions and questions your history cannot answer.

Get early access