System One Bench

Head to head

Drex v2.0 vs d3 edge

Drex v2.0 scores 78.6 and d3 edge 65.9 on the System One Index. Drex v2.0 is more accurate on 14 of 16 text datasets, d3 edge on 2. Same cases, same request, same scorer.

Overall

Drex v2.0d3 edge
System One Index78.665.9
English77.869.1
Arabic79.662.2
Vision Index––
Calibration error1.5%18.7%
Median latency294 ms–
Cost per 1,000 calls$0.015–

Text datasets, accuracy

Drex v2.0d3 edge
AG News English88.080.0
SST-2 English94.080.5
SST-5 English56.039.5
DAIR Emotion English55.557.0
Banking77 English80.590.0
BoolQ English92.579.5
XNLI English English87.384.0
Typed decisions English68.742.3
XNLI French French77.366.0
XNLI Arabic Arabic71.359.3
MASSIVE scenario Arabic63.541.0
MASSIVE intent Arabic82.555.0
OSACT4 offensive Arabic94.572.5
Tweet sentiment Arabic69.561.5
MIRACL rerank Arabic87.070.0
MIRACL pairwise Arabic89.076.0

Model pages: Drex v2.0 · d3 edge. Latency for API models is the median per call through OpenRouter, timed from a cloud server with 8 calls in flight.