Strategic Games Arena

Updated 5/30/2026

Pramaana Original

Evaluates planning, adaptation, and opponent modeling in text-based imperfect-information games.

View:Task:


1Claude Fable 5 Run settings: temperature 0, max tokens 16384, extended reasoning, 3 repeated runs.	68.00% ± 1.16	$4.39	471s
2Muse Spark 1.1OPEN Run settings: temperature 0.2, max tokens 16384, extended reasoning, 3 repeated runs.	65.88% ± 0.48	$0.26	264s
3GPT-5.6 Sol Run settings: temperature 0, max tokens 16384, extended reasoning, 4 repeated runs.	63.92% ± 1.33	$5.30	509s
4Claude Opus 4.8 Run settings: temperature 0, max tokens 16384, extended reasoning, 5 repeated runs.	62.11% ± 1.50	$3.18	905s
5Claude Sonnet 5 Run settings: temperature 0, max tokens 16384, extended reasoning, 6 repeated runs.	60.82% ± 1.67	$2.80	425s
6GPT-5.5 Run settings: temperature 0, max tokens 16384, standard reasoning, 4 repeated runs.	59.01% ± 0.65	$3.99	464s
7Gemini 3.5 Flash Run settings: temperature 0, max tokens 16384, standard reasoning, 3 repeated runs.	57.24% ± 1.16	$0.61	164s
8GLM 5.2OPEN Run settings: temperature 0.2, max tokens 16384, standard reasoning, 6 repeated runs.	57.00% ± 0.99	$0.56	348s
9Grok 4.5 Run settings: temperature 0, max tokens 16384, standard reasoning, 5 repeated runs.	56.82% ± 0.82	$1.64	264s
10Qwen 3.7 PlusOPEN Run settings: temperature 0.2, max tokens 16384, standard reasoning, 4 repeated runs.	56.80% ± 1.33	$0.48	285s

Motivation

Generic leaderboards rarely show whether a model can complete the work people actually depend on. This evaluation turns a real workflow into a controlled, repeatable test with visible uncertainty, cost, and latency.

Results

Industry average accuracy comparison

Mean score with standard-error whiskers across repeated runs.

Claude Fable 568.00% ± 1.16

Muse Spark 1.165.88% ± 0.48

GPT-5.6 Sol63.92% ± 1.33

Key takeaways

Models plan well but adapt slowly to adversarial opponents.
Compact state summaries improve long-horizon consistency.
Negotiation performance varies sharply by prompting profile.

Methodology

Every system is run against the same frozen task set and scorer. Results are the mean across repeated trials; uncertainty is reported as the standard error of the mean. Costs use public API pricing at evaluation time and latency is measured end to end.

Evaluation items: 600
Repeated runs: 3–6 / system

Updates