Respan ยท OpenRouter API
Span-02 by Respan scores 79.5 on the System One Index (11th of 30) and 86.1 on the Vision Index (1st of 11). Calibration error 10.5%, median latency 204 ms.
| Dataset | Accuracy | Rank | Best on the board | Calibration error |
|---|---|---|---|---|
| AG News English | 86.5 | 23rd of 30 | 92.5 | 9.9% |
| SST-2 English | 95.5 | 4th of 30 | 97.0 | 2.5% |
| SST-5 English | 57.5 | 8th of 30 | 62.5 | 21.8% |
| DAIR Emotion English | 56.0 | 21st of 30 | 91.5 | 31.3% |
| Banking77 English | 82.0 | 18th of 30 | 98.5 | 5.3% |
| BoolQ English | 90.0 | 9th of 30 | 93.0 | 6.1% |
| XNLI English English | 83.3 | 21st of 30 | 95.3 | 4.4% |
| Typed decisions English | 72.1 | 10th of 30 | 79.0 | 13.0% |
| XNLI French French | 73.3 | 21st of 30 | 91.3 | 11.8% |
| XNLI Arabic Arabic | 66.0 | 23rd of 32 | 90.0 | 17.9% |
| MASSIVE scenario Arabic | 71.0 | 11th of 32 | 89.5 | 13.3% |
| MASSIVE intent Arabic | 86.0 | 9th of 32 | 91.5 | 2.7% |
| OSACT4 offensive Arabic | 97.0 | 1st of 32 | 97.0 | 0.9% |
| Tweet sentiment Arabic | 76.0 | 2nd of 32 | 80.0 | 10.1% |
| MIRACL rerank Arabic | 88.5 | 7th of 32 | 90.5 | 7.9% |
| MIRACL pairwise Arabic | 91.5 | 1st of 32 | 91.5 | 4.7% |
| Dataset | Accuracy | Rank | Best on the board | Calibration error |
|---|---|---|---|---|
| SST-2, as an image | 92.5 | 9th of 11 | 97.0 | 1.8% |
| Arabic sentiment, as an image | 70.0 | 1st of 11 | 70.0 | 11.5% |
| Oxford-IIIT Pet | 92.5 | 1st of 11 | 92.5 | 8.3% |
| VQAv2 yes/no | 87.0 | 6th of 11 | 89.5 | 6.7% |
| CountBench | 88.5 | 7th of 11 | 95.5 | 8.7% |
respan/span-02Open weights. Unlike Span-01, which only takes a conversation, it reads the same state as every other model. Doesn't answer Box questions.
Ranks count the models charted on the leaderboard. Every model gets the same cases and the same request, under the benchmark rules.