System One Bench

Head to head

GPT-6 Luna Decisions vs Solar Decide

GPT-6 Luna Decisions scores 76.0 and Solar Decide 68.8 on the System One Index. GPT-6 Luna Decisions is more accurate on 12 of 16 text datasets, Solar Decide on 4. Same cases, same request, same scorer.

Overall

GPT-6 Luna DecisionsSolar Decide
System One Index76.068.8
English75.264.7
Arabic78.873.7
Vision Index81.6–
Calibration error9.8%20.3%
Median latency215 ms715 ms
Cost per 1,000 calls$0.040$0.042

Text datasets, accuracy

GPT-6 Luna DecisionsSolar Decide
AG News English87.586.0
SST-2 English93.591.5
SST-5 English61.553.5
DAIR Emotion English58.554.0
Banking77 English77.51.0
BoolQ English85.587.0
XNLI English English66.774.7
Typed decisions English71.069.9
XNLI French French62.067.3
XNLI Arabic Arabic54.058.0
MASSIVE scenario Arabic66.055.5
MASSIVE intent Arabic84.575.0
OSACT4 offensive Arabic93.586.0
Tweet sentiment Arabic80.071.5
MIRACL rerank Arabic86.084.5
MIRACL pairwise Arabic87.585.5

Image datasets, accuracy

GPT-6 Luna DecisionsSolar Decide
SST-2, as an image88.5–
Arabic sentiment, as an image67.0–
Oxford-IIIT Pet87.0–
VQAv2 yes/no80.5–
CountBench85.0–

Model pages: GPT-6 Luna Decisions · Solar Decide. Latency for API models is the median per call through OpenRouter, timed from one machine.