System One Bench

Head to head

d1-3B vs Solar Decide

d1-3B scores 78.4 and Solar Decide 68.8 on the System One Index. d1-3B is more accurate on 11 of 16 text datasets, Solar Decide on 5. Same cases, same request, same scorer.

Overall

d1-3BSolar Decide
System One Index78.468.8
English78.464.7
Arabic78.373.7
Vision Index––
Calibration error2.4%20.3%
Median latency–715 ms
Cost per 1,000 calls–$0.042

Text datasets, accuracy

d1-3BSolar Decide
AG News English88.586.0
SST-2 English95.591.5
SST-5 English56.053.5
DAIR Emotion English63.554.0
Banking77 English88.01.0
BoolQ English85.087.0
XNLI English English86.074.7
Typed decisions English64.869.9
XNLI French French78.767.3
XNLI Arabic Arabic73.358.0
MASSIVE scenario Arabic71.555.5
MASSIVE intent Arabic83.575.0
OSACT4 offensive Arabic84.586.0
Tweet sentiment Arabic63.571.5
MIRACL rerank Arabic82.584.5
MIRACL pairwise Arabic89.085.5

Model pages: d1-3B · Solar Decide. Latency for API models is the median per call through OpenRouter, timed from one machine.