Together AI · OpenRouter API
Tev1 4B Experimental by Together AI scores 71.5 on the System One Index (16th of 24). Calibration error 5.3%, median latency 232 ms.
| Dataset | Accuracy | Rank | Best on the board | Calibration error |
|---|---|---|---|---|
| AG News English | 91.5 | 4th of 24 | 92.5 | 4.0% |
| SST-2 English | 94.0 | 12th of 24 | 97.0 | 3.7% |
| SST-5 English | 43.0 | 20th of 24 | 62.5 | 26.7% |
| DAIR Emotion English | 56.5 | 16th of 24 | 91.5 | 20.6% |
| Banking77 English | 1.0 | 22nd of 24 | 98.5 | 0.3% |
| BoolQ English | 91.0 | 3rd of 24 | 92.5 | 5.8% |
| XNLI English English | 87.3 | 9th of 24 | 93.3 | 4.7% |
| Typed decisions English | 67.0 | 11th of 24 | 79.0 | 4.7% |
| XNLI French French | 82.7 | 4th of 24 | 90.7 | 5.9% |
| XNLI Arabic Arabic | 76.0 | 5th of 26 | 89.3 | 8.5% |
| MASSIVE scenario Arabic | 56.5 | 20th of 26 | 89.5 | 13.9% |
| MASSIVE intent Arabic | 70.0 | 18th of 26 | 91.5 | 9.6% |
| OSACT4 offensive Arabic | 84.5 | 15th of 26 | 95.5 | 4.6% |
| Tweet sentiment Arabic | 70.5 | 7th of 26 | 80.0 | 6.8% |
| MIRACL rerank Arabic | 85.5 | 11th of 26 | 90.5 | 8.7% |
| MIRACL pairwise Arabic | 87.5 | 10th of 26 | 91.0 | 3.6% |
togethercomputer/tev1-4b-experimentalAccepts at most 20 options per question, so all 200 Banking77 cases (77 options) failed and count as wrong.
Ranks count the models charted on the leaderboard. Every model gets the same cases and the same request, under the benchmark rules.