System One Bench

Head to head

Kev 0.8B vs Solar Decide Flash

Kev 0.8B scores 63.6 and Solar Decide Flash 63.6 on the System One Index. Kev 0.8B is more accurate on 7 of 16 text datasets, Solar Decide Flash on 9. Same cases, same request, same scorer.

Overall

Kev 0.8BSolar Decide Flash
System One Index63.663.6
English71.061.3
Arabic54.966.5
Vision Index––
Calibration error4.3%14.4%
Median latency–586 ms
Cost per 1,000 calls–$0.042

Text datasets, accuracy

Kev 0.8BSolar Decide Flash
AG News English91.086.0
SST-2 English90.090.5
SST-5 English47.044.5
DAIR Emotion English56.054.5
Banking77 English80.51.0
BoolQ English76.086.0
XNLI English English81.365.3
Typed decisions English46.362.5
XNLI French French66.061.3
XNLI Arabic Arabic58.752.0
MASSIVE scenario Arabic38.546.0
MASSIVE intent Arabic51.563.0
OSACT4 offensive Arabic49.084.5
Tweet sentiment Arabic57.065.5
MIRACL rerank Arabic62.571.5
MIRACL pairwise Arabic67.083.0

Model pages: Kev 0.8B · Solar Decide Flash. Latency for API models is the median per call through OpenRouter, timed from one machine.