~/system-one-bench / models/celeris-1-decision
Celeris · Celeris API
Celeris-1 DecisionCeleris-1 Decision by Celeris scores 83.6 on the System One Index (5th of 39). Calibration error 2.6%.
| Dataset | Accuracy | Rank | Best on the board | Calibration error |
|---|---|---|---|---|
| AG News English | 90.5 | 10th of 39 | 92.5 | 3.5% |
| SST-2 English | 95.5 | 8th of 39 | 98.0 | 4.1% |
| SST-5 English | 61.0 | 6th of 39 | 62.5 | 6.3% |
| DAIR Emotion English | 72.5 | 4th of 39 | 91.5 | 7.0% |
| Banking77 English | 89.0 | 14th of 39 | 98.5 | 3.3% |
| BoolQ English | 89.5 | 17th of 39 | 93.0 | 3.1% |
| XNLI English English | 91.3 | 8th of 39 | 95.3 | 9.2% |
| Typed decisions English | 76.4 | 3rd of 39 | 80.0 | 4.8% |
| XNLI French French | 90.0 | 2nd of 39 | 91.3 | 9.1% |
| XNLI Arabic Arabic | 80.0 | 7th of 41 | 90.0 | 5.5% |
| MASSIVE scenario Arabic | 73.5 | 9th of 41 | 89.5 | 6.3% |
| MASSIVE intent Arabic | 88.0 | 11th of 41 | 91.5 | 6.7% |
| OSACT4 offensive Arabic | 94.5 | 6th of 41 | 97.0 | 5.1% |
| Tweet sentiment Arabic | 69.0 | 15th of 41 | 80.0 | 6.3% |
| MIRACL rerank Arabic | 86.5 | 19th of 41 | 90.5 | 3.0% |
| MIRACL pairwise Arabic | 90.0 | 12th of 41 | 91.5 | 4.0% |
celeris-1-decisionClosed weights, on Celeris's own System One endpoint. Cost is input tokens at the $0.04 per 1M list price, since the API reports no cost. It accepts images but scored at chance on every image dataset (50.0% on SST-2 from an image, against 95.5% on the same reviews as text), so it has no vision score. Doesn't answer Box questions.
Ranks count the models charted on the leaderboard. Every model gets the same cases and the same request, under the benchmark rules.