Get API key
Benchmarks

Benchmark results and methodology

Harny holds the highest published score on BEAM at 128K tokens: 85.1, on everyday models. The memory your agents run on is the most accurate you can buy at this size, and it runs on models you can afford at scale.

BEAM accuracy by history size

BEAM asks what a memory system still knows after a very long conversation. Harny’s figure is from the 128K tier.

  • Harny85.1
BEAM score by history size: Harny scores 85.1 at the 128K history tier.
System100K500K1M10MSource
Harny85.1———harny.ai
The BEAM paper, published at ICLR 2026 →

85.1 on BEAM at 128K tokens.

Test Harny on your own data

Send your own data and ask it questions.

Get early access

BEAM results by ability

Accuracy for the ten BEAM abilities, from the same run as the chart above.

85.1Overall accuracy at the 128K tier, over 400 questions across 20 conversations.
  • Abstentionn=4092.5
  • Contradiction resolutionn=4086.3
  • Event orderingn=4056.5
  • Information extractionn=4089.7
  • Instruction followingn=4092.5
  • Knowledge updaten=4098.8
  • Multi-session reasoningn=4075.1
  • Preference followingn=4095.0
  • Summarizationn=4083.2
  • Temporal reasoningn=4081.3

508 of 600 on DolphinBench, in our internal runs.

LoCoMo

99.3Accuracy on the four standard categories, 150 questions across weeks-long conversations, scored on the corrected answer key. With trap questions included, 88.3.
LoCoMo accuracy on two question sets of 150 questions each: the four standard categories, and all five including trap questions.
Question setAccuracyn
Standard questions: single-hop, multi-hop, temporal, open-domain99.3150
With trap questions: all five categories, equal weight88.3150
The LoCoMo paper →The dataset and code →

LongMemEval

96.9Accuracy on 150 questions that depend on facts told in earlier sessions and never repeated.
The LongMemEval paper →The dataset and code →

Tokens per task and cost per answer

98%Fewer tokens than handing your agent the raw tool output: 50× less for your agent to read.
4×Cheaper per answer.
How we measured →

Evaluation scope

Current public results cover BEAM, LoCoMo and LongMemEval accuracy. These are the datasets, graders and measures behind every figure above.

Datasets

BEAM
The full 128K tier: 400 questions over 20 conversations, graded across ten abilities on BEAM’s own rubric.
LongMemEval
150 questions drawn from the benchmark.
LoCoMo
150 questions per set, scored against the corrected answer key. 99 of the 1,540 published answers were corrected.

How a result is reported

Judges
gemini-3.5-flash-lite grades BEAM. deepseek-v4-pro grades LongMemEval and LoCoMo.
Metrics
Accuracy per question. BEAM’s headline is the mean of its ten ability scores.
The BEAM page, with every size and ability →How to benchmark agent memory →

Run it on your data

Ask Harny about changed facts, earlier sessions and questions your history cannot answer.

Get early access See the integration →