Perplexity ยท OpenRouter API
Decider V1.1 27B by Perplexity scores 83.4 on the System One Index and 83.0 on the Vision Index. Calibration error 5.7%. An earlier run, kept for reference.
| Dataset | Accuracy | Rank | Best on the board | Calibration error |
|---|---|---|---|---|
| AG News English | 92.0 | 92.5 | 7.4% | |
| SST-2 English | 96.0 | 97.0 | 1.3% | |
| SST-5 English | 60.5 | 62.5 | 5.5% | |
| DAIR Emotion English | 60.0 | 91.5 | 13.0% | |
| Banking77 English | 81.0 | 98.5 | 7.4% | |
| BoolQ English | 92.5 | 93.0 | 2.3% | |
| XNLI English English | 93.3 | 95.3 | 5.3% | |
| Typed decisions English | 73.4 | 79.0 | 9.4% | |
| XNLI French French | 90.7 | 91.3 | 5.4% | |
| XNLI Arabic Arabic | 89.3 | 90.0 | 4.6% | |
| MASSIVE scenario Arabic | 72.5 | 89.5 | 13.0% | |
| MASSIVE intent Arabic | 90.0 | 91.5 | 4.5% | |
| OSACT4 offensive Arabic | 95.5 | 97.0 | 2.9% | |
| Tweet sentiment Arabic | 72.0 | 80.0 | 6.1% | |
| MIRACL rerank Arabic | 87.0 | 90.5 | 7.0% | |
| MIRACL pairwise Arabic | 88.5 | 91.5 | 5.9% |
| Dataset | Accuracy | Rank | Best on the board | Calibration error |
|---|---|---|---|---|
| SST-2, as an image | 96.0 | 97.0 | 4.1% | |
| Arabic sentiment, as an image | 51.0 | 70.0 | 17.6% | |
| Oxford-IIIT Pet | 84.0 | 92.5 | 8.8% | |
| VQAv2 yes/no | 89.5 | 89.5 | 3.8% | |
| CountBench | 94.5 | 95.5 | 4.8% |
perplexity/pplx-decider-v1.1-27bDropped out of OpenRouter's catalog in October 2026 (its slug now returns 404) and replaced by Decider V1.2. Left out of the charts. Its latency was timed on 15 October 2026 with the other API models.
Ranks count the models charted on the leaderboard. Every model gets the same cases and the same request, under the benchmark rules.