system-one-bench

~/system-one-bench / models/celeris-1-decision

Celeris · Celeris API

Celeris-1 Decision

Celeris-1 Decision by Celeris scores 83.6 on the System One Index (5th of 39). Calibration error 2.6%.

System One Index
83.6
5th of 39
English
83.2
Arabic
83.1
Vision Index
–
No image input
Calibration error
2.6%
Lower is better
Median latency
–
Cost per 1,000 calls
$0.014
At list price

Text datasets

DatasetAccuracyRankBest on the boardCalibration error
AG News English90.510th of 3992.53.5%
SST-2 English95.58th of 3998.04.1%
SST-5 English61.06th of 3962.56.3%
DAIR Emotion English72.54th of 3991.57.0%
Banking77 English89.014th of 3998.53.3%
BoolQ English89.517th of 3993.03.1%
XNLI English English91.38th of 3995.39.2%
Typed decisions English76.43rd of 3980.04.8%
XNLI French French90.02nd of 3991.39.1%
XNLI Arabic Arabic80.07th of 4190.05.5%
MASSIVE scenario Arabic73.59th of 4189.56.3%
MASSIVE intent Arabic88.011th of 4191.56.7%
OSACT4 offensive Arabic94.56th of 4197.05.1%
Tweet sentiment Arabic69.015th of 4180.06.3%
MIRACL rerank Arabic86.519th of 4190.53.0%
MIRACL pairwise Arabic90.012th of 4191.54.0%

This run

Model id
celeris-1-decision
Runs on
Celeris API
Run date
2026-11-08
Latency
API; not timed (round 11 ran on Celeris's own API from a machine whose round trips don't compare with the board's)
Questions
4,850, 0 failed (counted wrong)

Closed weights, on Celeris's own System One endpoint. Cost is input tokens at the $0.04 per 1M list price, since the API reports no cost. It accepts images but scored at chance on every image dataset (50.0% on SST-2 from an image, against 95.5% on the same reviews as text), so it has no vision score. Doesn't answer Box questions.

Compare

Ranks count the models charted on the leaderboard. Every model gets the same cases and the same request, under the benchmark rules.