System One Bench

Head to head

Clef Omni vs Solar Decide Flash

Clef Omni scores 79.7 and Solar Decide Flash 63.6 on the System One Index. Clef Omni is more accurate on 15 of 16 text datasets, Solar Decide Flash on 1. Same cases, same request, same scorer.

Overall

Clef OmniSolar Decide Flash
System One Index79.763.6
English78.661.3
Arabic80.566.5
Vision Index72.2–
Calibration error3.4%14.4%
Median latency208 ms586 ms
Cost per 1,000 calls$0.074$0.042

Text datasets, accuracy

Clef OmniSolar Decide Flash
AG News English89.086.0
SST-2 English97.090.5
SST-5 English51.044.5
DAIR Emotion English56.054.5
Banking77 English96.01.0
BoolQ English90.086.0
XNLI English English82.765.3
Typed decisions English67.062.5
XNLI French French82.761.3
XNLI Arabic Arabic73.352.0
MASSIVE scenario Arabic89.046.0
MASSIVE intent Arabic91.563.0
OSACT4 offensive Arabic90.584.5
Tweet sentiment Arabic50.065.5
MIRACL rerank Arabic83.071.5
MIRACL pairwise Arabic86.583.0

Image datasets, accuracy

Clef OmniSolar Decide Flash
SST-2, as an image89.5–
Arabic sentiment, as an image28.5–
Oxford-IIIT Pet88.0–
VQAv2 yes/no87.0–
CountBench68.0–

Model pages: Clef Omni · Solar Decide Flash. Latency for API models is the median per call through OpenRouter, timed from one machine.