Jev leads new benchmark for decision models

Artificial Analysis has launched its Decision Index v1.0 to compare models that select structured answers and return probabilities for each option in a single call. The benchmark combines four evaluations covering classification, tool calling, knowledge, hallucination, and reasoning, while measuring cost, speed, and confidence calibration. Jev 1.13 leads decision models with 58, ahead of GLiDE and Microsoft-Decision-1, both on 54.

The results highlight substantial efficiency advantages over language models. Jev 1.13 costs $42 per one million tasks and takes 0.13 seconds per task. Claude Haiku 5.5 (Low) scores higher at 61, but costs $184 per one million tasks and takes 1.4 seconds per task. Leading decision models perform comparably to the tested language models on classification and tool calling, while language models retain an advantage on knowledge and reasoning.

Confidence remains a weakness: all evaluated decision models are generally overconfident, particularly on knowledge and reasoning tasks. Calibration is stronger on classification and tool calling. The benchmark also tests whether models acknowledge uncertainty by choosing “I don’t know.” Microsoft-Decision-1 leads decision models on this non-hallucination measure at 40%, followed by Jev 1.13 at 35% and Mercury Decide at 32%. The measure tracks abstentions among questions a model did not answer correctly.