System One Bench

Head to head

d1-3B vs Drex Max

d1-3B scores 78.4 and Drex Max 84.1 on the System One Index. d1-3B is more accurate on 0 of 16 text datasets, Drex Max on 16. Same cases, same request, same scorer.

Overall

d1-3BDrex Max
System One Index78.484.1
English78.482.8
Arabic78.384.5
Vision Index––
Calibration error2.4%6.2%
Median latency–277 ms
Cost per 1,000 calls–$0.021

Text datasets, accuracy

d1-3BDrex Max
AG News English88.590.5
SST-2 English95.596.0
SST-5 English56.059.0
DAIR Emotion English63.565.0
Banking77 English88.090.5
BoolQ English85.093.0
XNLI English English86.095.3
Typed decisions English64.873.0
XNLI French French78.791.3
XNLI Arabic Arabic73.389.3
MASSIVE scenario Arabic71.573.0
MASSIVE intent Arabic83.588.0
OSACT4 offensive Arabic84.593.5
Tweet sentiment Arabic63.569.0
MIRACL rerank Arabic82.589.5
MIRACL pairwise Arabic89.089.5

Model pages: d1-3B · Drex Max. Latency for API models is the median per call through OpenRouter, timed from a cloud server with 8 calls in flight.