Benefits Navigator

Updated 5/26/2026

Industry Partner

Tests accurate, accessible guidance through public-benefit eligibility and application workflows.

View:Task:


1Claude Fable 5 Run settings: temperature 0, max tokens 16384, extended reasoning, 4 repeated runs.	71.30% ± 1.33	$5.13	602s
2GPT-5.6 Sol Run settings: temperature 0, max tokens 16384, extended reasoning, 5 repeated runs.	67.22% ± 1.50	$6.20	651s
3Claude Opus 4.8 Run settings: temperature 0, max tokens 16384, extended reasoning, 6 repeated runs.	65.41% ± 1.67	$3.72	1158s
4Muse Spark 1.1OPEN Run settings: temperature 0.2, max tokens 16384, extended reasoning, 4 repeated runs.	64.38% ± 0.65	$0.31	338s
5Claude Sonnet 5 Run settings: temperature 0, max tokens 16384, extended reasoning, 3 repeated runs.	64.12% ± 0.48	$3.27	543s
6GPT-5.5 Run settings: temperature 0, max tokens 16384, standard reasoning, 5 repeated runs.	62.31% ± 0.82	$4.66	593s
7Gemini 3.5 Flash Run settings: temperature 0, max tokens 16384, standard reasoning, 4 repeated runs.	60.54% ± 1.33	$0.71	210s
8GLM 5.2OPEN Run settings: temperature 0.2, max tokens 16384, standard reasoning, 3 repeated runs.	60.30% ± 1.16	$0.66	446s
9Grok 4.5 Run settings: temperature 0, max tokens 16384, standard reasoning, 6 repeated runs.	60.12% ± 0.99	$1.92	338s
10Qwen 3.7 PlusOPEN Run settings: temperature 0.2, max tokens 16384, standard reasoning, 5 repeated runs.	60.10% ± 1.50	$0.57	364s

Motivation

Generic leaderboards rarely show whether a model can complete the work people actually depend on. This evaluation turns a real workflow into a controlled, repeatable test with visible uncertainty, cost, and latency.

Results

Industry average accuracy comparison

Mean score with standard-error whiskers across repeated runs.

Claude Fable 571.30% ± 1.33

GPT-5.6 Sol67.22% ± 1.50

Claude Opus 4.865.41% ± 1.67

Key takeaways

Eligibility reasoning is strong when household facts are complete.
Models need clearer escalation behavior for jurisdiction-specific exceptions.
Plain-language prompting improves accessibility without reducing accuracy.

Methodology

Every system is run against the same frozen task set and scorer. Results are the mean across repeated trials; uncertainty is reported as the standard error of the mean. Costs use public API pricing at evaluation time and latency is measured end to end.

Evaluation items: 460
Repeated runs: 3–6 / system

Updates