Perplexity ยท OpenRouter API
Decider Flash V1.2 by Perplexity scores 81.9 on the System One Index (7th of 30) and 77.5 on the Vision Index (10th of 11). Calibration error 1.8%, median latency 104 ms.
| Dataset | Accuracy | Rank | Best on the board | Calibration error |
|---|---|---|---|---|
| AG News English | 89.0 | 13th of 30 | 92.5 | 3.1% |
| SST-2 English | 95.0 | 11th of 30 | 97.0 | 3.7% |
| SST-5 English | 54.0 | 14th of 30 | 62.5 | 6.8% |
| DAIR Emotion English | 57.0 | 15th of 30 | 91.5 | 14.1% |
| Banking77 English | 90.5 | 6th of 30 | 98.5 | 4.7% |
| BoolQ English | 89.0 | 14th of 30 | 93.0 | 3.2% |
| XNLI English English | 92.7 | 3rd of 30 | 95.3 | 3.2% |
| Typed decisions English | 73.0 | 6th of 30 | 79.0 | 4.3% |
| XNLI French French | 88.0 | 3rd of 30 | 91.3 | 4.9% |
| XNLI Arabic Arabic | 85.3 | 3rd of 32 | 90.0 | 5.6% |
| MASSIVE scenario Arabic | 70.5 | 12th of 32 | 89.5 | 8.3% |
| MASSIVE intent Arabic | 86.0 | 9th of 32 | 91.5 | 4.1% |
| OSACT4 offensive Arabic | 90.5 | 9th of 32 | 97.0 | 9.3% |
| Tweet sentiment Arabic | 71.5 | 7th of 32 | 80.0 | 4.4% |
| MIRACL rerank Arabic | 88.5 | 7th of 32 | 90.5 | 4.4% |
| MIRACL pairwise Arabic | 90.5 | 6th of 32 | 91.5 | 3.1% |
| Dataset | Accuracy | Rank | Best on the board | Calibration error |
|---|---|---|---|---|
| SST-2, as an image | 93.5 | 6th of 11 | 97.0 | 4.1% |
| Arabic sentiment, as an image | 35.0 | 8th of 11 | 70.0 | 28.3% |
| Oxford-IIIT Pet | 84.0 | 9th of 11 | 92.5 | 13.0% |
| VQAv2 yes/no | 87.5 | 4th of 11 | 89.5 | 4.5% |
| CountBench | 87.5 | 9th of 11 | 95.5 | 5.2% |
perplexity/pplx-decider-flash-v1.24B sibling of Decider V1.2. Doesn't answer Box questions.
Ranks count the models charted on the leaderboard. Every model gets the same cases and the same request, under the benchmark rules.