PLPramaana

Adaptive Tutoring Dialogue

Updated 6/12/2026
Pramaana Original

Evaluates pedagogical guidance, misconception detection, and age-appropriate response adaptation.

Model leaderboard

1Claude Fable 5
Run settings: temperature 0, max tokens 16384, extended reasoning, 3 repeated runs.		69.32% ± 0.48$5.13471s
2GPT-5.6 Sol
Run settings: temperature 0, max tokens 16384, extended reasoning, 4 repeated runs.		67.31% ± 0.65$6.20509s
3Muse Spark 1.1OPEN
Run settings: temperature 0.2, max tokens 16384, extended reasoning, 3 repeated runs.		66.50% ± 1.16$0.31264s
4Claude Opus 4.8
Run settings: temperature 0, max tokens 16384, extended reasoning, 5 repeated runs.		65.50% ± 0.82$3.72905s
5Claude Sonnet 5
Run settings: temperature 0, max tokens 16384, extended reasoning, 6 repeated runs.		64.21% ± 0.99$3.27425s
6Gemini 3.5 Flash
Run settings: temperature 0, max tokens 16384, standard reasoning, 3 repeated runs.		63.36% ± 0.48$0.71164s
7GPT-5.5
Run settings: temperature 0, max tokens 16384, standard reasoning, 4 repeated runs.		62.40% ± 1.33$4.66464s
8GLM 5.2OPEN
Run settings: temperature 0.2, max tokens 16384, standard reasoning, 6 repeated runs.		60.39% ± 1.67$0.66348s
9Grok 4.5
Run settings: temperature 0, max tokens 16384, standard reasoning, 5 repeated runs.		60.21% ± 1.50$1.92264s
10Qwen 3.7 PlusOPEN
Run settings: temperature 0.2, max tokens 16384, standard reasoning, 4 repeated runs.		58.12% ± 0.65$0.57285s

Results reflect a fixed evaluation snapshot. API prices and provider behavior may change after the recorded run date.

License: Research preview

Motivation

Generic leaderboards rarely show whether a model can complete the work people actually depend on. This evaluation turns a real workflow into a controlled, repeatable test with visible uncertainty, cost, and latency.

Results

Industry average accuracy comparison

Mean score with standard-error whiskers across repeated runs.

Claude Fable 569.32% ± 0.48
GPT-5.6 Sol67.31% ± 0.65
Muse Spark 1.166.50% ± 1.16

Key takeaways

  • The best tutors ask targeted questions before supplying an answer.
  • Concise feedback scores higher than exhaustive correction.
  • Age adaptation remains inconsistent on ambiguous prompts.

Methodology

Every system is run against the same frozen task set and scorer. Results are the mean across repeated trials; uncertainty is reported as the standard error of the mean. Costs use public API pricing at evaluation time and latency is measured end to end.

Evaluation items
380
Repeated runs
3–6 / system

Updates

  1. 6/12/2026

    Latest frontier model results added.

  2. 5/12/2026

    Scorer calibration and refusal handling updated.

  3. 2/18/2026

    Evaluation set published.