System One Bench

Head to head

Clef Omni vs GPT-6 Luna Decisions

Clef Omni scores 79.7 and GPT-6 Luna Decisions 76.0 on the System One Index. Clef Omni is more accurate on 9 of 16 text datasets, GPT-6 Luna Decisions on 7. Same cases, same request, same scorer.

Overall

Clef OmniGPT-6 Luna Decisions
System One Index79.776.0
English78.675.2
Arabic80.578.8
Vision Index72.281.6
Calibration error3.4%9.8%
Median latency208 ms215 ms
Cost per 1,000 calls$0.074$0.040

Text datasets, accuracy

Clef OmniGPT-6 Luna Decisions
AG News English89.087.5
SST-2 English97.093.5
SST-5 English51.061.5
DAIR Emotion English56.058.5
Banking77 English96.077.5
BoolQ English90.085.5
XNLI English English82.766.7
Typed decisions English67.071.0
XNLI French French82.762.0
XNLI Arabic Arabic73.354.0
MASSIVE scenario Arabic89.066.0
MASSIVE intent Arabic91.584.5
OSACT4 offensive Arabic90.593.5
Tweet sentiment Arabic50.080.0
MIRACL rerank Arabic83.086.0
MIRACL pairwise Arabic86.587.5

Image datasets, accuracy

Clef OmniGPT-6 Luna Decisions
SST-2, as an image89.588.5
Arabic sentiment, as an image28.567.0
Oxford-IIIT Pet88.087.0
VQAv2 yes/no87.080.5
CountBench68.085.0

Model pages: Clef Omni ยท GPT-6 Luna Decisions. Latency for API models is the median per call through OpenRouter, timed from one machine.