System One Bench

Head to head

Decider V1.1 27B vs Solar Decide Flash

Decider V1.1 27B scores 83.4 and Solar Decide Flash 63.6 on the System One Index. Decider V1.1 27B is more accurate on 16 of 16 text datasets, Solar Decide Flash on 0. Same cases, same request, same scorer.

Overall

Decider V1.1 27BSolar Decide Flash
System One Index83.463.6
English81.161.3
Arabic85.066.5
Vision Index83.0–
Calibration error5.7%14.4%
Median latency263 ms586 ms
Cost per 1,000 calls$0.009$0.042

Text datasets, accuracy

Decider V1.1 27BSolar Decide Flash
AG News English92.086.0
SST-2 English96.090.5
SST-5 English60.544.5
DAIR Emotion English60.054.5
Banking77 English81.01.0
BoolQ English92.586.0
XNLI English English93.365.3
Typed decisions English73.462.5
XNLI French French90.761.3
XNLI Arabic Arabic89.352.0
MASSIVE scenario Arabic72.546.0
MASSIVE intent Arabic90.063.0
OSACT4 offensive Arabic95.584.5
Tweet sentiment Arabic72.065.5
MIRACL rerank Arabic87.071.5
MIRACL pairwise Arabic88.583.0

Image datasets, accuracy

Decider V1.1 27BSolar Decide Flash
SST-2, as an image96.0–
Arabic sentiment, as an image51.0–
Oxford-IIIT Pet84.0–
VQAv2 yes/no89.5–
CountBench94.5–

Model pages: Decider V1.1 27B · Solar Decide Flash. Latency for API models is the median per call through OpenRouter, timed from one machine.