Published model quality scores
157 scores that the makers of open AI models, and public leaderboards, published for the models in our finder. We copy the number as printed and link the source and the date we read it.
Published by others. These are not Findra scores and they are not comparable across rows: tests, settings and thinking modes differ, and some rows are for a related model (the note says which). Use them to compare a model with itself or with the same test on another model. Download the data: model-scores.json (free to reuse, credit Findra). To see which laptops can run a model, use the finder.
- Llama 3.2 3B 3.2B
- Qwen3 4B 4B
- Gemma 3 4B 4.3B
- Mistral 7B v0.3 7.2B
- Llama 3.1 8B 8B
- Qwen3 8B 8.2B
- Gemma 3 12B 12.2B
- DeepSeek-R1 Distill Qwen 14B 14.8B
- Phi-4 14B 14.7B
- Qwen3 14B 14.8B
- gpt-oss 20B 20.9B
- Gemma 3 27B 27.4B
- Qwen3 30B (A3B) 30.5B
- Qwen3 32B 32.8B
- Llama 3.1 70B 70.6B
- gpt-oss 120B 117B
Llama 3.2 3B
Ollama tag llama3.2:3b
| Test | Score | Setting | Source |
|---|---|---|---|
| MMLU | 63.4% | 5-shot, macro_avg/acc | source read 2026-10-22 Maker (Meta) model card, Llama 3.2 3B Instruct bf16 column |
| IFEval | 77.4% | 0-shot, avg of prompt/instruction acc loose/strict | source read 2026-10-22 Maker (Meta) model card, Llama 3.2 3B Instruct bf16 column; average of four IFEval accuracies, not strict-only |
| MATH (CoT) | 48% | 0-shot, final_em | source read 2026-10-22 Maker (Meta) model card, Llama 3.2 3B Instruct bf16 column |
| GPQA | 32.8% | 0-shot, acc | source read 2026-10-22 Maker (Meta) model card, Llama 3.2 3B Instruct bf16 column; plain GPQA as printed, not named Diamond |
Qwen3 4B
Ollama tag qwen3:4b
| Test | Score | Setting | Source |
|---|---|---|---|
| MMLU-Redux | 83.7% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 17; post-trained hybrid model, thinking mode; MMLU-Redux is a cleaned MMLU re-label, not plain MMLU |
| GPQA-Diamond | 55.9% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 17; post-trained hybrid model, thinking mode |
| IFEval strict prompt | 81.9% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 17; post-trained hybrid model, thinking mode |
| LiveCodeBench v5 | 54.2% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 17; post-trained hybrid model, thinking mode |
| MATH-500 | 97% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 17; post-trained hybrid model, thinking mode |
| AIME'24 | 73.8% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 17; post-trained hybrid model, thinking mode |
| MMLU-Redux | 77.3% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 18; post-trained hybrid model, non-thinking mode; MMLU-Redux is a cleaned MMLU re-label, not plain MMLU |
| GPQA-Diamond | 41.7% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 18; post-trained hybrid model, non-thinking mode |
| IFEval strict prompt | 81.2% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 18; post-trained hybrid model, non-thinking mode |
| LiveCodeBench v5 | 21.3% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 18; post-trained hybrid model, non-thinking mode |
| MATH-500 | 84.8% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 18; post-trained hybrid model, non-thinking mode |
| AIME'24 | 25% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 18; post-trained hybrid model, non-thinking mode |
| MMLU | 72.99% | not stated | source read 2026-10-22 Maker (Qwen team) tech report Table 7; score is for the BASE model Qwen3-4B-Base, not the chat model |
| MMLU-Pro | 50.58% | not stated | source read 2026-10-22 Maker (Qwen team) tech report Table 7; score is for the BASE model Qwen3-4B-Base, not the chat model |
Gemma 3 4B
Ollama tag gemma3:4b
| Test | Score | Setting | Source |
|---|---|---|---|
| MMLU-Pro | 43.6% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report Table 6, instruction-tuned (IT) model; Table 6 caption says zero-shot |
| GPQA Diamond | 30.8% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report Table 6, instruction-tuned (IT) model; Table 6 caption says zero-shot |
| LiveCodeBench | 12.6% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report Table 6, instruction-tuned (IT) model; Table 6 caption says zero-shot |
| MATH | 75.6% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report Table 6, instruction-tuned (IT) model; Table 6 caption says zero-shot |
| MMLU | 58.1% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report appendix Table 18, instruction-tuned (IT) model; 0-shot per the report's IT details table |
| HumanEval | 71.3% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report appendix Table 18, instruction-tuned (IT) model; 0-shot per the report's IT details table |
| IFEval | 90.2% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report appendix Table 18, instruction-tuned (IT) model; 0-shot per the report's IT details table |
Mistral 7B v0.3
Ollama tag mistral:7b
| Test | Score | Setting | Source |
|---|---|---|---|
| IFEval | 54.65% | 0-shot, strict prompt+instruction accuracy | source read 2026-10-22 Hugging Face Open LLM Leaderboard v2 (not the maker), row mistralai/Mistral-7B-Instruct-v0.3 bfloat16 chat model, submitted 2024-06-12 |
| MMLU-PRO (raw accuracy) | 30.75% | 5-shot | source read 2026-10-22 Hugging Face Open LLM Leaderboard v2 (not the maker), row mistralai/Mistral-7B-Instruct-v0.3 bfloat16 chat model, submitted 2024-06-12; the leaderboard's normalized MMLU-PRO column prints 23.06 |
| GPQA (raw accuracy) | 27.94% | 0-shot | source read 2026-10-22 Hugging Face Open LLM Leaderboard v2 (not the maker), row mistralai/Mistral-7B-Instruct-v0.3 bfloat16 chat model, submitted 2024-06-12; plain GPQA, 4-way multiple choice so chance is 25; normalized GPQA column prints 3.91 |
| MATH Lvl 5 | 3.85% | 4-shot, exact match | source read 2026-10-22 Hugging Face Open LLM Leaderboard v2 (not the maker), row mistralai/Mistral-7B-Instruct-v0.3 bfloat16 chat model, submitted 2024-06-12; level 5 problems only |
Llama 3.1 8B
Ollama tag llama3.1:8b
| Test | Score | Setting | Source |
|---|---|---|---|
| MMLU | 69.4% | 5-shot, macro_avg/acc | source read 2026-10-22 Maker (Meta) model card, instruction-tuned model (Llama 3.1 Instruct); also on the unsloth HF mirror |
| MMLU (CoT) | 73% | 0-shot, macro_avg/acc | source read 2026-10-22 Maker (Meta) model card, instruction-tuned model (Llama 3.1 Instruct) |
| MMLU-Pro (CoT) | 48.3% | 5-shot, macro_avg/acc | source read 2026-10-22 Maker (Meta) model card, instruction-tuned model (Llama 3.1 Instruct) |
| IFEval | 80.4% | not stated | source read 2026-10-22 Maker (Meta) model card, instruction-tuned model (Llama 3.1 Instruct); card prints no shot count or metric for IFEval |
| GPQA | 30.4% | 0-shot, em | source read 2026-10-22 Maker (Meta) model card, instruction-tuned model (Llama 3.1 Instruct); plain GPQA as printed, not named Diamond |
| HumanEval | 72.6% | 0-shot, pass@1 | source read 2026-10-22 Maker (Meta) model card, instruction-tuned model (Llama 3.1 Instruct) |
| MATH (CoT) | 51.9% | 0-shot, final_em | source read 2026-10-22 Maker (Meta) model card, instruction-tuned model (Llama 3.1 Instruct) |
Qwen3 8B
Ollama tag qwen3:8b
| Test | Score | Setting | Source |
|---|---|---|---|
| MMLU-Redux | 87.5% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 17; post-trained hybrid model, thinking mode; MMLU-Redux is a cleaned MMLU re-label, not plain MMLU |
| GPQA-Diamond | 62% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 17; post-trained hybrid model, thinking mode |
| IFEval strict prompt | 85% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 17; post-trained hybrid model, thinking mode |
| LiveCodeBench v5 | 57.5% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 17; post-trained hybrid model, thinking mode |
| MATH-500 | 97.4% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 17; post-trained hybrid model, thinking mode |
| AIME'24 | 76% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 17; post-trained hybrid model, thinking mode |
| MMLU-Redux | 79.5% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 18; post-trained hybrid model, non-thinking mode; MMLU-Redux is a cleaned MMLU re-label, not plain MMLU |
| GPQA-Diamond | 39.3% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 18; post-trained hybrid model, non-thinking mode |
| IFEval strict prompt | 83% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 18; post-trained hybrid model, non-thinking mode |
| LiveCodeBench v5 | 22.8% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 18; post-trained hybrid model, non-thinking mode |
| MATH-500 | 87.4% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 18; post-trained hybrid model, non-thinking mode |
| AIME'24 | 29.1% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 18; post-trained hybrid model, non-thinking mode |
| MMLU | 76.89% | not stated | source read 2026-10-22 Maker (Qwen team) tech report Table 6; score is for the BASE model Qwen3-8B-Base, not the chat model |
| MMLU-Pro | 56.73% | not stated | source read 2026-10-22 Maker (Qwen team) tech report Table 6; score is for the BASE model Qwen3-8B-Base, not the chat model |
Gemma 3 12B
Ollama tag gemma3:12b
| Test | Score | Setting | Source |
|---|---|---|---|
| MMLU-Pro | 60.6% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report Table 6, instruction-tuned (IT) model; Table 6 caption says zero-shot |
| GPQA Diamond | 40.9% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report Table 6, instruction-tuned (IT) model; Table 6 caption says zero-shot |
| LiveCodeBench | 24.6% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report Table 6, instruction-tuned (IT) model; Table 6 caption says zero-shot |
| MATH | 83.8% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report Table 6, instruction-tuned (IT) model; Table 6 caption says zero-shot |
| MMLU | 71.9% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report appendix Table 18, instruction-tuned (IT) model; 0-shot per the report's IT details table |
| HumanEval | 85.4% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report appendix Table 18, instruction-tuned (IT) model; 0-shot per the report's IT details table |
| IFEval | 88.9% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report appendix Table 18, instruction-tuned (IT) model; 0-shot per the report's IT details table |
DeepSeek-R1 Distill Qwen 14B
Ollama tag deepseek-r1:14b
| Test | Score | Setting | Source |
|---|---|---|---|
| AIME 2024 | 69.7% | pass@1 | source read 2026-10-22 Maker (DeepSeek) model card distilled-model table; pass@1 from 64 samples, temperature 0.6, top-p 0.95, max 32768 tokens |
| MATH-500 | 93.9% | pass@1 | source read 2026-10-22 Maker (DeepSeek) model card distilled-model table; pass@1 from 64 samples, temperature 0.6, top-p 0.95, max 32768 tokens |
| GPQA Diamond | 59.1% | pass@1 | source read 2026-10-22 Maker (DeepSeek) model card distilled-model table; pass@1 from 64 samples, temperature 0.6, top-p 0.95, max 32768 tokens |
| LiveCodeBench | 53.1% | pass@1 | source read 2026-10-22 Maker (DeepSeek) model card distilled-model table; pass@1 from 64 samples, temperature 0.6, top-p 0.95, max 32768 tokens |
| CodeForces rating | 1481 Elo | not stated | source read 2026-10-22 Maker (DeepSeek) model card distilled-model table; pass@1 from 64 samples, temperature 0.6, top-p 0.95, max 32768 tokens |
| MMLU-Redux | 84.1% | thinking mode | source read 2026-10-22 NOT the maker: measured by the Qwen team and printed in the Qwen3 tech report Table 17 as a comparison column; DeepSeek publishes no MMLU for this model |
| IFEval strict prompt | 72.6% | thinking mode | source read 2026-10-22 NOT the maker: measured by the Qwen team and printed in the Qwen3 tech report Table 17 as a comparison column; DeepSeek publishes no IFEval for this model |
Phi-4 14B
Ollama tag phi4:14b
| Test | Score | Setting | Source |
|---|---|---|---|
| MMLU | 84.8% | simple-evals | source read 2026-10-22 Maker (Microsoft) model card Model Quality table, OpenAI simple-evals |
| GPQA | 56.1% | simple-evals | source read 2026-10-22 Maker (Microsoft) model card Model Quality table, OpenAI simple-evals; plain GPQA as printed, not named Diamond |
| MATH | 80.4% | simple-evals | source read 2026-10-22 Maker (Microsoft) model card Model Quality table, OpenAI simple-evals |
| HumanEval | 82.6% | simple-evals | source read 2026-10-22 Maker (Microsoft) model card Model Quality table, OpenAI simple-evals |
| MMLU-Pro | 70.4% | not stated | source read 2026-10-22 Maker (Microsoft) phi-4 tech report Table 1; in the second benchmark group (not simple-evals); setting not stated in the table |
| IFEval | 63% | not stated | source read 2026-10-22 Maker (Microsoft) phi-4 tech report Table 1; in the second benchmark group (not simple-evals); setting not stated in the table |
Qwen3 14B
Ollama tag qwen3:14b
| Test | Score | Setting | Source |
|---|---|---|---|
| MMLU-Redux | 88.6% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 15; post-trained hybrid model, thinking mode; MMLU-Redux is a cleaned MMLU re-label, not plain MMLU |
| GPQA-Diamond | 64% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 15; post-trained hybrid model, thinking mode |
| IFEval strict prompt | 85.4% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 15; post-trained hybrid model, thinking mode |
| LiveCodeBench v5 | 63.5% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 15; post-trained hybrid model, thinking mode |
| MATH-500 | 96.8% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 15; post-trained hybrid model, thinking mode |
| AIME'24 | 79.3% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 15; post-trained hybrid model, thinking mode |
| MMLU-Redux | 82% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 16; post-trained hybrid model, non-thinking mode; MMLU-Redux is a cleaned MMLU re-label, not plain MMLU |
| GPQA-Diamond | 54.8% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 16; post-trained hybrid model, non-thinking mode |
| IFEval strict prompt | 84.8% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 16; post-trained hybrid model, non-thinking mode |
| LiveCodeBench v5 | 29% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 16; post-trained hybrid model, non-thinking mode |
| MATH-500 | 90% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 16; post-trained hybrid model, non-thinking mode |
| AIME'24 | 31.7% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 16; post-trained hybrid model, non-thinking mode |
| MMLU | 81.05% | not stated | source read 2026-10-22 Maker (Qwen team) tech report Table 5; score is for the BASE model Qwen3-14B-Base, not the chat model |
| MMLU-Pro | 61.03% | not stated | source read 2026-10-22 Maker (Qwen team) tech report Table 5; score is for the BASE model Qwen3-14B-Base, not the chat model |
gpt-oss 20B
Ollama tag gpt-oss:20b
| Test | Score | Setting | Source |
|---|---|---|---|
| MMLU | 80.4% | low reasoning | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| MMLU | 84% | medium reasoning | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| MMLU | 85.3% | high reasoning | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| GPQA Diamond | 56.8% | low reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| GPQA Diamond | 66% | medium reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| GPQA Diamond | 71.5% | high reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| AIME 2024 | 42.1% | low reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| AIME 2024 | 80% | medium reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| AIME 2024 | 92.1% | high reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| AIME 2025 | 37.1% | low reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| AIME 2025 | 72.1% | medium reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| AIME 2025 | 91.7% | high reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| Codeforces Elo | 1366 Elo | low reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| Codeforces Elo | 1998 Elo | medium reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| Codeforces Elo | 2230 Elo | high reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
Gemma 3 27B
Ollama tag gemma3:27b
| Test | Score | Setting | Source |
|---|---|---|---|
| MMLU-Pro | 67.5% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report Table 6, instruction-tuned (IT) model; Table 6 caption says zero-shot |
| GPQA Diamond | 42.4% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report Table 6, instruction-tuned (IT) model; Table 6 caption says zero-shot |
| LiveCodeBench | 29.7% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report Table 6, instruction-tuned (IT) model; Table 6 caption says zero-shot |
| MATH | 89% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report Table 6, instruction-tuned (IT) model; Table 6 caption says zero-shot |
| MMLU | 76.9% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report appendix Table 18, instruction-tuned (IT) model; 0-shot per the report's IT details table |
| HumanEval | 87.8% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report appendix Table 18, instruction-tuned (IT) model; 0-shot per the report's IT details table |
| IFEval | 90.4% | 0-shot | source read 2026-10-22 Maker (Google) Gemma 3 tech report appendix Table 18, instruction-tuned (IT) model; 0-shot per the report's IT details table |
| Chatbot Arena Elo | 1338 Elo | not stated | source read 2026-10-22 Maker (Google) Gemma 3 tech report Table 5; preliminary result received 8 Mar 2025 from LMArena, 95% CI +8/-9 |
Qwen3 30B (A3B)
Ollama tag qwen3:30b
| Test | Score | Setting | Source |
|---|---|---|---|
| MMLU-Redux | 89.5% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 15; post-trained hybrid model, thinking mode; MMLU-Redux is a cleaned MMLU re-label, not plain MMLU |
| GPQA-Diamond | 65.8% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 15; post-trained hybrid model, thinking mode |
| IFEval strict prompt | 86.5% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 15; post-trained hybrid model, thinking mode |
| LiveCodeBench v5 | 62.6% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 15; post-trained hybrid model, thinking mode |
| MATH-500 | 98% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 15; post-trained hybrid model, thinking mode |
| AIME'24 | 80.4% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 15; post-trained hybrid model, thinking mode |
| MMLU-Redux | 84.1% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 16; post-trained hybrid model, non-thinking mode; MMLU-Redux is a cleaned MMLU re-label, not plain MMLU |
| GPQA-Diamond | 54.8% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 16; post-trained hybrid model, non-thinking mode |
| IFEval strict prompt | 83.7% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 16; post-trained hybrid model, non-thinking mode |
| LiveCodeBench v5 | 29.8% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 16; post-trained hybrid model, non-thinking mode |
| MATH-500 | 89.8% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 16; post-trained hybrid model, non-thinking mode |
| AIME'24 | 32.8% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 16; post-trained hybrid model, non-thinking mode |
| MMLU | 81.38% | not stated | source read 2026-10-22 Maker (Qwen team) tech report Table 5; score is for the BASE model Qwen3-30B-A3B-Base, not the chat model |
| MMLU-Pro | 61.49% | not stated | source read 2026-10-22 Maker (Qwen team) tech report Table 5; score is for the BASE model Qwen3-30B-A3B-Base, not the chat model |
Qwen3 32B
Ollama tag qwen3:32b
| Test | Score | Setting | Source |
|---|---|---|---|
| MMLU-Redux | 90.9% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 13; post-trained hybrid model, thinking mode; MMLU-Redux is a cleaned MMLU re-label, not plain MMLU |
| GPQA-Diamond | 68.4% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 13; post-trained hybrid model, thinking mode |
| IFEval strict prompt | 85% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 13; post-trained hybrid model, thinking mode |
| LiveCodeBench v5 | 65.7% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 13; post-trained hybrid model, thinking mode |
| MATH-500 | 97.2% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 13; post-trained hybrid model, thinking mode |
| AIME'24 | 81.4% | thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 13; post-trained hybrid model, thinking mode |
| MMLU-Redux | 85.7% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 14; post-trained hybrid model, non-thinking mode; MMLU-Redux is a cleaned MMLU re-label, not plain MMLU |
| GPQA-Diamond | 54.6% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 14; post-trained hybrid model, non-thinking mode |
| IFEval strict prompt | 83.2% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 14; post-trained hybrid model, non-thinking mode |
| LiveCodeBench v5 | 31.3% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 14; post-trained hybrid model, non-thinking mode |
| MATH-500 | 88.6% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 14; post-trained hybrid model, non-thinking mode |
| AIME'24 | 31% | non-thinking mode | source read 2026-10-22 Maker (Qwen team) tech report Table 14; post-trained hybrid model, non-thinking mode |
| MMLU | 83.61% | not stated | source read 2026-10-22 Maker (Qwen team) tech report Table 4; score is for the BASE model Qwen3-32B-Base, not the chat model |
| MMLU-Pro | 65.54% | not stated | source read 2026-10-22 Maker (Qwen team) tech report Table 4; score is for the BASE model Qwen3-32B-Base, not the chat model |
Llama 3.1 70B
Ollama tag llama3.1:70b
| Test | Score | Setting | Source |
|---|---|---|---|
| MMLU | 83.6% | 5-shot, macro_avg/acc | source read 2026-10-22 Maker (Meta) model card, instruction-tuned model (Llama 3.1 Instruct); also on the unsloth HF mirror |
| MMLU (CoT) | 86% | 0-shot, macro_avg/acc | source read 2026-10-22 Maker (Meta) model card, instruction-tuned model (Llama 3.1 Instruct) |
| MMLU-Pro (CoT) | 66.4% | 5-shot, macro_avg/acc | source read 2026-10-22 Maker (Meta) model card, instruction-tuned model (Llama 3.1 Instruct) |
| IFEval | 87.5% | not stated | source read 2026-10-22 Maker (Meta) model card, instruction-tuned model (Llama 3.1 Instruct); card prints no shot count or metric for IFEval |
| GPQA | 46.7% | 0-shot, em | source read 2026-10-22 Maker (Meta) model card, instruction-tuned model (Llama 3.1 Instruct); plain GPQA as printed, not named Diamond |
| HumanEval | 80.5% | 0-shot, pass@1 | source read 2026-10-22 Maker (Meta) model card, instruction-tuned model (Llama 3.1 Instruct) |
| MATH (CoT) | 68% | 0-shot, final_em | source read 2026-10-22 Maker (Meta) model card, instruction-tuned model (Llama 3.1 Instruct) |
gpt-oss 120B
Ollama tag gpt-oss:120b
| Test | Score | Setting | Source |
|---|---|---|---|
| MMLU | 85.9% | low reasoning | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| MMLU | 88% | medium reasoning | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| MMLU | 90% | high reasoning | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| GPQA Diamond | 67.1% | low reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| GPQA Diamond | 73.1% | medium reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| GPQA Diamond | 80.1% | high reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| AIME 2024 | 56.3% | low reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| AIME 2024 | 80.4% | medium reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| AIME 2024 | 95.8% | high reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| AIME 2025 | 50.4% | low reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| AIME 2025 | 80% | medium reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| AIME 2025 | 92.5% | high reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| Codeforces Elo | 1595 Elo | low reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| Codeforces Elo | 2205 Elo | medium reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |
| Codeforces Elo | 2463 Elo | high reasoning, no tools | source read 2026-10-22 Maker (OpenAI) gpt-oss model card Table 3; MXFP4 weights as released |