System One Bench

Head to head

Clef Omni vs Laya Router

Clef Omni scores 79.7 and Laya Router 60.4 on the System One Index. Clef Omni is more accurate on 14 of 16 text datasets, Laya Router on 2. Same cases, same request, same scorer.

Overall

Clef OmniLaya Router
System One Index79.760.4
English78.664.7
Arabic80.553.2
Vision Index72.2–
Calibration error3.4%20.0%
Median latency208 ms–
Cost per 1,000 calls$0.074–

Text datasets, accuracy

Clef OmniLaya Router
AG News English89.092.5
SST-2 English97.088.5
SST-5 English51.034.0
DAIR Emotion English56.053.5
Banking77 English96.044.0
BoolQ English90.081.5
XNLI English English82.787.3
Typed decisions English67.036.4
XNLI French French82.776.0
XNLI Arabic Arabic73.368.0
MASSIVE scenario Arabic89.039.0
MASSIVE intent Arabic91.532.5
OSACT4 offensive Arabic90.582.5
Tweet sentiment Arabic50.037.0
MIRACL rerank Arabic83.031.5
MIRACL pairwise Arabic86.582.0

Image datasets, accuracy

Clef OmniLaya Router
SST-2, as an image89.5–
Arabic sentiment, as an image28.5–
Oxford-IIIT Pet88.0–
VQAv2 yes/no87.0–
CountBench68.0–

Model pages: Clef Omni · Laya Router. Latency for API models is the median per call through OpenRouter, timed from one machine.