Updated 26 Oct 2026
PUBLISHED BENCHMARK RESULTS2023 TO PRESENT
AI model
benchmark scores
Published results for major AI models, with the source and method for each score.
223models
486benchmarks
2,087reported results
37labs
Scored models released or announced since 2023. Each model–benchmark result is counted once.
Data review in progress. Some scores and benchmark definitions still await source verification.
01
Benchmark scores
Compare published scores and the methods behind them.
One selected result per model and benchmark.
Green marks the best result with matching detailed method records; this is not independent verification. Amber is a reference when methods differ or are incomplete. Open a score for its source and method.486 benchmarks · 223 models · newest → oldest
| BenchmarkTest set / measure | NanbeigeNanbeige4.3-33B25 Oct 2026 | Moonshot AIKimi K3.1 Pro23 Oct 2026 | NVIDIANemotron 3.5 Bolt E4B 4GB (Q4_K_M)23 Oct 2026 | FastinoGLiDE v1.222 Oct 2026 | InclusionAILing-3.0-flash-Sante22 Oct 2026 | SubquadraticSubQ Snap 1.122 Oct 2026 | Basetenelephant-1 (9B)21 Oct 2026 | Basetenmammoth-1 (27B)21 Oct 2026 | RecursiveRecursive Flint21 Oct 2026 |
|---|---|---|---|---|---|---|---|---|---|
| Reasoning & mathsAIME2026 test · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Reasoning & mathsGPQA DiamondPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | ||
| CodingNL2RepoIssues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| CodingSciCodePercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| CodingSWE-Bench ProIssues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| CodingTerminal-BenchVersion 2.1 · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | ||
| Agents & toolsAutomationBenchVersion 1.0.6 · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsSkillsBenchTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalJobBenchTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| CodingTerminal-BenchVersion 4.0 · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | ||
| Agents & toolsToolathlonVerified · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| CodingSWE-Atlas-QnAGrading variant unspecified · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| CodingDeepSWEVersion unspecified · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalGDPvalRevised rubrics · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsClaw-EvalGeneral subset · pass^3 · Reported pass^3 | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsDeepPlanningReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsEnterpriseOps-GymReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsYC-benchReported dollars · Reported dollars | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsτ³-BenchSubset unspecified · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Reasoning & mathsAA-LCRVersion unspecified · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | ||
| Reasoning & mathsHumanity’s Last ExamNo tools · Percentage correct | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | |
| CodingDeepSWEv1.1 · Issues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| CodingKimi Code BenchVersion 2.0 · Issues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| CodingSWE-MarathonVersion 1.1 · First-attempt pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsAPEX-AgentsTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsBrowseCompPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | ||
| ProfessionalOfficeQA ProPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| MultimodalBabyVisionNo tools · Percentage correct | —No tools —With Python | —No tools | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python |
| MultimodalCharXiv ReasoningNo tools · Percentage correct | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | |
| MultimodalMMMU ProNo tools · Percentage correct | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | |
| MultimodalPerceptionBenchPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| MultimodalZeroBenchNo tools · Five-attempt pass rate | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | |
| CodingTerminal-Bench-ScienceVersion 0.1 · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| MultimodalChartographyPercentage correct | —No tools —With tools | —No tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools |
| ProfessionalGDP.pdfCriteria passed | —No tools —With tools | —No tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools |
| ProfessionalLegal Agent BenchmarkPublic set · All-pass rate | —All criteria passed —Criteria passed | —Criteria passed | —All criteria passed —Criteria passed | —All criteria passed —Criteria passed | —All criteria passed —Criteria passed | —All criteria passed —Criteria passed | —All criteria passed —Criteria passed | —All criteria passed —Criteria passed | —All criteria passed —Criteria passed |
| CodingProgramBenchAlmost@1 · Tasks meeting the almost-solved threshold | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| CodingPostTrainBenchv1.2 · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalGDPval-AA2.1 · Rating | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalAA-Briefcase1.1 · Rating | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsAutomationBenchArtificial Analysis · Strict pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsAgents’ Last Examv1.1 · all 122 tasks · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsAgents’ Last Examv1.1 · CLI subset · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| MultimodalBenchCAD Vision2Code1,000-task subset · With tools · Voxel IoU | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| MultimodalBenchCAD Vision2Code1,000-task subset · No tools · Voxel IoU | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Reasoning & mathsBioMysteryBenchv11 · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| CodingKimi Code Benchv3.0 · Long-horizon · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalOfficeQA Prov2 · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalFrontier FinanceFinance-agent v2 harness · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsHANDBOOK.mdv1 · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsOSWorldv2.1 · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalBusiness OperationKimi internal · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalDECK-ArenaKimi internal · Rating | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| MultimodalDesign BenchKimi internal · Rating | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| SafetyKimi Hallu Bench HardWith tools · Claim-level hallucination rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| SafetyKimi Hallu BenchNo tools · Claim-level hallucination rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalGDP.pdfNo tools · strict pass · Strict pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Reasoning & mathsHumanity’s Last Exam DiamondNo tools · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| MultimodalZeroBenchMain set · no tools · avg@5 · Mean accuracy over five runs | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| SafetyExploitBenchProgressive flags · pass@5 · Flags reached · pass@5 | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| SafetyExploitGym6 hours · percentage · Exploitation success rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Safetyxdev-bench-hardKimi internal · Three-attempt pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalHLAB-Valsv1.1 · All criteria passed · Task pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| General knowledgeMultiChallengePercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsBFCLVersion 3 · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| General knowledgeIFBenchPrompt loose · Prompts passed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsτ²-benchDomain average · Average accuracy | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| General knowledgeRoleBenchCustom reference comparison · Wins against reference | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| General knowledgeRoleMRCExcludes multi-turn · Judge average | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| General knowledgeEQ-BenchVersion 3 · Rubric · Rubric score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| General knowledgeBrevity BenchmarkNVIDIA internal · Quality and conciseness | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsWhen2CallMulti-turn · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsToolSandboxAverage accuracy | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| General knowledgeIHEvalConflict subset · Conflict accuracy | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsAgentIFInstructions satisfied | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| General knowledgeStructEval-TAverage accuracy | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| General knowledgeInverseIFEvalOverall accuracy | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| General knowledgeRULER64K tokens · Overall accuracy | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| General knowledgeLongMemEvalOverall accuracy | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| General knowledgeLongBenchVersion 2 · 48K tokens · Overall accuracy | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| General knowledgeSysBenchInstructions satisfied | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| General knowledgeGaRAGeRelevance-aware factuality · Relevance-aware factuality | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| General knowledgeRAGTruthF1 · F1 score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsRelyToolBenchNative/real · Native/real pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsHugging Face Decision Indexv0.4.1 · full · Full skill index | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsMCP-AtlasPublic set · Public test set · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsτ³-BankingTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalGDPval-AAv2 · Rating | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsDeepSearchQAReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalHealthBench ProfessionalAdjustment unspecified · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalAFUMED-ComplexReasoningReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalAFUMED-DrugContraindication accuracy | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalMedQA-USMLEPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalDiagnosisArena-MCQPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalReDisQAReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalMedMCQAPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| SafetyAFUSAFE-MedSCEReported safety score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| ProfessionalMedXpertQAText subset · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| Agents & toolsArtificial Analysis Decision Indexv1.0 · Index score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |||
| CodingSWE-bench VerifiedIssues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| CodingSWE-Bench ProVersion 2 · Issues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | |
| MultimodalAA Word Error Rate Index2 · Non-streaming · Word error rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeIFBenchPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingDeepSWEv1.0 · Issues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsAA-LCRv1.1 · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalGDPval EloVersion unspecified · Reported rating | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeAA-Omniscience PublicPublic set · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeMRCR v2Context unspecified · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingSWE-Bench AtlasIssues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsBrowseComp+Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsAgents’ Last ExamTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyCyberGymTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsHumanity’s Last ExamText-only · No tools · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalBabyVisionClaude Code tools · First-attempt pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalCharXiv ReasoningTools not specified · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingPostTrainBench1.1 · Weighted aggregate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalBankerToolBenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsLOCA-Bench256K context · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingNL2Repo-BenchTwo corrected tasks · October 2026 · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingSWE-Refactor-BenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalGDP.pdf-AAAA · reported score · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsOSWorld2.0 · scoring unspecified · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingNL2Repo-BenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingApexBenchv1.1 · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalBabyVisionWith tools · harness unspecified · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalZeroBenchMain set · tools · aggregation unspecified · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeLongBenchVersion 2 · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsAA-LCRPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsCritPtScore | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingLiveCodeBenchFirst-attempt pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingSWE-bench MultilingualIssues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsDeepSearchQAF1 score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingSWE-Atlas-QnAStrict · Strict accuracy | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsAutomationBenchPublic split · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsMCP-AtlasSplit unspecified · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeAA-OmnisciencePublic index · Public penalised index | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingSWE-Bench Pro2 · Hard · Issues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyOpenAI Voice-Native SafetyOctober 2026 · Production · Sexual · Safety score (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyOpenAI Voice-Native SafetyOctober 2026 · Production · Illicit Behaviour · Safety score (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyOpenAI Voice-Native SafetyOctober 2026 · Production · Mental Health · Safety score (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyOpenAI Voice-Native SafetyOctober 2026 · Production · Personal Data · Safety score (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyOpenAI Voice-Native SafetyOctober 2026 · Production · Emotional Reliance · Safety score (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyOpenAI Voice-Native SafetyOctober 2026 · Production · Self Harm · Safety score (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyOpenAI Voice-Native SafetyOctober 2026 · Synthetic · Sexual · Safety score (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyOpenAI Voice-Native SafetyOctober 2026 · Synthetic · Illicit Behaviour · Safety score (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyOpenAI Voice-Native SafetyOctober 2026 · Synthetic · Mental Health · Safety score (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyOpenAI Voice-Native SafetyOctober 2026 · Synthetic · Personal Data · Safety score (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyOpenAI Voice-Native SafetyOctober 2026 · Synthetic · Emotional Reliance · Safety score (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyOpenAI Voice-Native SafetyOctober 2026 · Synthetic · Self Harm · Safety score (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyOpenAI Voice-Native SafetyOctober 2026 · Synthetic · Hate · Safety score (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyOpenAI Voice-Native SafetyOctober 2026 · Synthetic · Gore · Safety score (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeIFM Arabic evaluation suite2026-10-15 · Jais 3 release suite · Arabic benchmark average | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalOpen Universal Arabic ASR2026-10 · Six public sets · WER · Word error rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalOpen Universal Arabic ASR2026-10 · Six public sets · CER · Character error rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsOSWorld-VerifiedTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingSciCodeTest subproblems · Background provided · 288 subproblems · No tools · Subproblems passed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeAA-OmniscienceIndex · Penalised knowledge index | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeArtificial Analysis Intelligence Index4.3 · Index score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalWorldVQATools · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalSimpleVQATools · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalIntPhys 2Physical-consistency accuracy | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalVideo-MME-v22 · Subtitles, no audio · Grouped benchmark score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeAA-OmnisciencePercentage correct | —Accuracy —Non-hallucination | —Accuracy —Non-hallucination | —Accuracy —Non-hallucination | —Accuracy —Non-hallucination | —Accuracy —Non-hallucination | —Accuracy —Non-hallucination | —Accuracy —Non-hallucination | —Accuracy —Non-hallucination | —Accuracy —Non-hallucination |
| Agents & toolsPinchBenchTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeIFBenchInstruction loose · Constraints satisfied | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsWebArenaVision · 812 tasks · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingLiveCodeBenchVersion 6 · First-attempt pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsτ²-benchTelecom subset · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalScreenSpot ProPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeMMLU-ProX10-language average · Mean accuracy | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsMathArena Apex ShortlistShortlist · No tools · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeGraphWalks950 prompts · Up to 1M tokens · Node-set F1 | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeIFBenchInstruction strict · Constraints satisfied | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeNVIDIA Chat Benchmark3,713-prompt internal set · Pairwise wins | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeWMT24++English → nine languages · XCOMET-XXL · XCOMET-XXL score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsτ²-benchRetail · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsτ²-benchAirline · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingSciCode65 whole problems · Background provided · Whole problems passed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalV*Static · No visual-search tool · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalOCR-ReasoningPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalOCRBench2 · English · English score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalOCRBench2 · Chinese · Chinese score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalVideo-MMEVersion 2 · No subtitles · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMMLongBench-DocPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsOSWorld-Verified361 tasks · Partial credit · Mean task score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeDecision Index0.3 · Public fast subset · 1,068 rows · Public fast-subset index | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalAA Speech to Speech Index2.0 · Index score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalBig Bench Audio1.2 · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalFull Duplex BenchAA subset of v1 and v1.5 · Conversational dynamics score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsτ-VoiceTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalSpeech Agent ArenaPreference · Preference rating | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalSpeech Agent ArenaTask success · Task success rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsEVA-BenchAccuracy · Accuracy | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeGraphWalksUp to 128K · F1 score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsLAB-BenchVersion 2 · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsAutomationBenchPrivate set · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalFinance AgentVersion 2 · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalHarvey LABVals v1 · All-pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalLVBenchPercentage correct | —Static —Agentic | —Static —Agentic | —Static —Agentic | —Static —Agentic | —Static —Agentic | —Static —Agentic | —Static —Agentic | —Static —Agentic | —Static —Agentic |
| CodingFrontierSWEVersion 2 · Score (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsOSWorld2.0 · August 2026 tasks · Partial credit · Partial score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalVals IndexSeptember 2026 snapshot · Weighted score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingVibe Code BenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsRiemannBenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeGraphWalks256K–1M · BFS · F1 score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyCWE-Bench1 · Repair success rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingFrontierSWEScore | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsWideSearchTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsDRACONormalised score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsHumanity’s Last ExamSet and tools unspecified · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsSuperMatBenchQA · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsSuperMatBenchAgent · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalFinSearchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalLegal Research BenchPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalPLawBenchPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalLegalBenchVals evaluation · Accuracy | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalGDP.pdfTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalHealthBench ProfessionalLength-adjusted rubric score | —Length-adjusted —Raw score | —Length-adjusted —Raw score | —Length-adjusted —Raw score | —Length-adjusted —Raw score | —Length-adjusted —Raw score | —Length-adjusted —Raw score | —Length-adjusted —Raw score | —Length-adjusted —Raw score | —Length-adjusted —Raw score |
| ProfessionalHarvey Legal Agent Benchmark1.1 · AA all-pass · All-pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgePersonal Superintelligence IndexComposite score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsOSWorld2.1 · Online and offline · Partial score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsOSWorld2.1 · Binary reward · Strict pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsMemoryBenchMeta evaluation · Weighted rubric score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsFocusBenchMeta evaluation · Persistent-agent score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingSWE-MarathonVersion unspecified · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyExploitBenchTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeMicrosoft Decision RobustnessEight perturbations · Decision-flip rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingHumanEval+First-attempt pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingMBPP+First-attempt pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsBFCLv4 · JetBrains five-part macro-average · Five-part macro-average | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingFrontierCodev1.1 Main · Composite task score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsOSWorld2.1 · Offline · Reported offline score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsHugging Face Decision Indexv0.2.1 · public · Skill index | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalHealthBenchRubric score | —Raw score —Length-adjusted | —Raw score —Length-adjusted | —Raw score —Length-adjusted | —Raw score —Length-adjusted | —Raw score —Length-adjusted | —Raw score —Length-adjusted | —Raw score —Length-adjusted | —Raw score —Length-adjusted | —Raw score —Length-adjusted |
| ProfessionalHealthBench HardLength-adjusted · Length-adjusted rubric score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalHealthBench HardRaw · Rubric score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalHealthBench ConsensusLength-adjusted · Length-adjusted rubric score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalHealthBench ConsensusRaw · Rubric score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalMentalHealthBenchOverall · Task-clipped rubric score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalAA-BriefcaseRating | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingCyberGymEnd-to-end · End-to-end success | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingCybenchCTF tasks solved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingCoding Agent IndexVersion unspecified · Index score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalDense200Grounding accuracy | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyB3 AI SecurityAttack resistance · Attack resistance | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalKORAScore (0–2) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeMMLU-ProPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsAIME2025 test · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalArtificial Analysis Streaming ASRFinal transcript · Final transcript word error rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyExploitGymNo time limit · Exploitation success rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyExploitBenchJune–August 2026 subset · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetySEC-Bench ProTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyOpenAI User-Flagged FactualityResponses containing factual errorsLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingCursorBench4.0 · Issues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsAndroidWorldTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsOSWorldHolo4 release · version unspecified · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingAI Coding Daily70-point total · Points / 70 | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsFast Decisions17-dataset suite · Micro-task accuracy | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalCT-RATEDirect yes/no · 18 labels · Macro-F1 (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalCT-RATEDirect yes/no · 18 labels · Macro-AUROC (0–1) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingFrontierCode1.1 Main · AA · Pass rate · All blocking checks passed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsAgents’ Last ExamScore · Score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingMiMo Code MiniReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyMiMo Cyber MiniReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsMiMo General MiniReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMiMo Visual MiniReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingProgramBenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingMiMo Code BenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyMiMo Cyber BenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMiMo Visual CodingReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyCyberGymMiMo-corrected environments · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyExploitGymMiMo · budget unspecified · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyExploitBenchMiMo · reported score · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeArtificial Analysis Intelligence Indexv4.3.2 · Index score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsAutomationBench-AAv1.0.6 held-out · Partial objective credit · Mean objective score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalLegal Agent BenchmarkHeld-out · All criteria passed · All-pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalEEBenchV1 core corpus · Functional reward | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalCADGenBenchGeneration · Generation reward | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalLatchBio Capabilities1.0 · Mean capability score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalHealthBench ProfessionalAdjustment unspecified · Reported rubric score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsAgents’ Last ExamCLI · version unspecified · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalFrontier FinanceHarness version unspecified · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingStep Code BenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalGrok short-phrase transcriptionInternal test set · Word error rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsClawEval-MMAverage score · Score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsClawEval-MMThree-attempt pass rate · Three-attempt pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalERQAPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalRealWorldQAPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsWildClawBench-MMReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsUniClawBenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsAgenticVBenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsOmniGAIAReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalDailyOmniReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalWorldSenseReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalAVUTReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalJoinAVBenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalOmniVideoBenchReported score | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code |
| MultimodalVideo-MME-v2Reported score | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code |
| MultimodalLVOmniBenchReported score | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code | —Static —Qwen Code |
| MultimodalOmniClozeReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalOmniCap-IFCSR · CSR score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalOmniCap-IFISR · ISR score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalQIVDReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalStreamingBenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalAliMeeting TestDER · Diarisation error rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalAliMeeting TestCPWER · Concatenated minimum-permutation word error rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalAISHELL-4DER · Diarisation error rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalAISHELL-4CPWER · Concatenated minimum-permutation word error rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMagicData-RAMCDER · Diarisation error rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMagicData-RAMCCPWER · Concatenated minimum-permutation word error rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMLC-SLM (en)DER · Diarisation error rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMLC-SLM (en)CPWER · Concatenated minimum-permutation word error rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalWenetSpeech (Net)Word error rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalWenetSpeech (Meeting)Word error rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalFLEURS-ASRQwen 60-language set · Word error rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalFLEURS-S2TTQwen 60-language set · BLEU score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalSpotSoundBenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMMAUReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMMARReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMMSUReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalLongAudioSpanAccuracy · Accuracy score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalLongAudioSpanRubric · Rubric score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalLongAudioSpanChain · Chain score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMuchoMusic-RULReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalHumMusQAReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMusTBenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalAudio MultiChallengeReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalWildSpeechReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalVoiceBenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingSWE-Bench ProQwen-refined set · September 2026 · Issues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsCoWorkBenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMathVisionPercentage correct | —Corrected set · No code interpreter —Corrected set · With code interpreter | —Corrected set · No code interpreter —Corrected set · With code interpreter | —Corrected set · No code interpreter —Corrected set · With code interpreter | —Corrected set · No code interpreter —Corrected set · With code interpreter | —Corrected set · No code interpreter —Corrected set · With code interpreter | —Corrected set · No code interpreter —Corrected set · With code interpreter | —Corrected set · No code interpreter —Corrected set · With code interpreter | —Corrected set · No code interpreter —Corrected set · With code interpreter | —Corrected set · No code interpreter —Corrected set · With code interpreter |
| MultimodalVision2WebMean judge score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalLegalResearchBenchPrivate validation · 200 questions · Overall correctness | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsτ³-Bench BankingVoice · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingTerminal-BenchVersion 3.0 · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsAutomationBenchTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingCodeforcesRating | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsMathArena ApexFirst-attempt pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyExploitGymTime budget unspecified · Exploitation pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalZeroBenchMain set · Claude Code tools · Five-attempt pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeMRCR v2128K · 8 needle · Percentage correct | —128K · 8 needle —256K–512K · 8 needle —512K–1M · 8 needle | —128K · 8 needle —256K–512K · 8 needle —512K–1M · 8 needle | —128K · 8 needle —256K–512K · 8 needle —512K–1M · 8 needle | —128K · 8 needle —256K–512K · 8 needle —512K–1M · 8 needle | —128K · 8 needle —256K–512K · 8 needle —512K–1M · 8 needle | —128K · 8 needle —256K–512K · 8 needle —512K–1M · 8 needle | —128K · 8 needle —256K–512K · 8 needle —512K–1M · 8 needle | —128K · 8 needle —256K–512K · 8 needle —512K–1M · 8 needle | —128K · 8 needle —256K–512K · 8 needle —512K–1M · 8 needle |
| Reasoning & mathsARC-AGIVersion 1 · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsARC-AGIVersion 2 · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsARC-AGIVersion 3 · Relative Human Action Efficiency | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsFrontierMathTier 4 v2 · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingFrontierCodeVersion 1.1 Extended · Composite task score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalBenchCAD Vision2CodeVoxel IoU | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools |
| ProfessionalOpenScore String Quartets1 − OMR-NED | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalOpenAI Internal Design TasksTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeArtificial Analysis Intelligence IndexVersion 4.1.1 · Index score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalOpenAI Internal Data Science TasksTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingOpenAI Internal Database Migration TasksTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingArtificial Analysis Coding Agent IndexVersion 1.4 · Index score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalGeneBench ProPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalMedChemBenchOpenAI internal set · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalLifeSciBenchPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingSRE-BenchFirst-attempt pass rate | —First attempt —Within four attempts | —First attempt —Within four attempts | —First attempt —Within four attempts | —First attempt —Within four attempts | —First attempt —Within four attempts | —First attempt —Within four attempts | —First attempt —Within four attempts | —First attempt —Within four attempts | —First attempt —Within four attempts |
| SafetyOpenAI Internal Computer-Use SafetyUnsafe-action rateLower is better | —Standard review —With AutoReview | —Standard review —With AutoReview | —Standard review —With AutoReview | —Standard review —With AutoReview | —Standard review —With AutoReview | —Standard review —With AutoReview | —Standard review —With AutoReview | —Standard review —With AutoReview | —Standard review —With AutoReview |
| SafetyOpenAI Internal Circumvention EvaluationCircumvention rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyExploitGym HoneypotHoneypot-trigger rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyImpossible ExploitGymSafe handling rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyOpenAI Internal Hallucination EvaluationHallucination rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsHMMTFebruary 2026 · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsBFCLVersion 4 · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsWildClawBenchTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsMCPMarkTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsBioMysteryBenchHard set · Tasks solved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsBioMysteryBenchHuman-solved comparison · Tasks solved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsHumanity’s Last ExamVerified set · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsOSWorld2.0 binary · Score | —Binary reward —Partial reward | —Binary reward —Partial reward | —Binary reward —Partial reward | —Binary reward —Partial reward | —Binary reward —Partial reward | —Binary reward —Partial reward | —Binary reward —Partial reward | —Binary reward —Partial reward | —Binary reward —Partial reward |
| SafetyGray Swan IPI BenchmarkASR@15 · Attack success at 15 · Attack success rateLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyGoogle Internal Vulnerability Discovery20-language set · Vulnerabilities found | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyCWE-BenchTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingCursorBenchVersion 3.2 · Issues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalOfficeQAPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingSWE-bench MultimodalIssues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsCritPT-CorrectedFirst-attempt pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeGlobal MMLU42 languages · 42-language average · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeMILU11 languages · 11-language average · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalHarvey LAB-AA120 held-out tasks · All-pass rate | —All criteria passed —Criteria passed | —All criteria passed —Criteria passed | —All criteria passed —Criteria passed | —All criteria passed —Criteria passed | —All criteria passed —Criteria passed | —All criteria passed —Criteria passed | —All criteria passed —Criteria passed | —All criteria passed —Criteria passed | —All criteria passed —Criteria passed |
| Agents & toolsOSWorld2.0 · August 2026 · Anthropic modified tasks · Partial score | —Partial credit —Strict pass | —Partial credit —Strict pass | —Partial credit —Strict pass | —Partial credit —Strict pass | —Partial credit —Strict pass | —Partial credit —Strict pass | —Partial credit —Strict pass | —Partial credit —Strict pass | —Partial credit —Strict pass |
| Reasoning & mathsArXivMathJune 2026 · Percentage correct | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools | —No tools —With tools |
| ProfessionalSpatialBench VerifiedScore | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalSingleCellBenchScore | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalProteinGym HardRank correlation | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalProtein DesignScore | —Sequence generation —Library ranking | —Sequence generation —Library ranking | —Sequence generation —Library ranking | —Sequence generation —Library ranking | —Sequence generation —Library ranking | —Sequence generation —Library ranking | —Sequence generation —Library ranking | —Sequence generation —Library ranking | —Sequence generation —Library ranking |
| ProfessionalOrganic ChemistryVersion 2 · Score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalMolecular Biology ProtocolsScore | —Troubleshooting —Understanding | —Troubleshooting —Understanding | —Troubleshooting —Understanding | —Troubleshooting —Understanding | —Troubleshooting —Understanding | —Troubleshooting —Understanding | —Troubleshooting —Understanding | —Troubleshooting —Understanding | —Troubleshooting —Understanding |
| Agents & toolsSkillsBenchTencent · 79 text tasks · three-run mean · Mean reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsCoWorkBenchReported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsRecreationBenchTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalVision2WebMean judged score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingSWE-Bench ProQwen-refined set · August 2026 · Issues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingApexBenchFirst-attempt pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingDSBenchHard set · Issues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingPostTrainBenchPost-training aggregate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingProgramBenchAlmost-solved threshold · Programs meeting the almost-solved threshold | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyExploitGym2 hours · Tasks solved | —2-hour budget —6-hour budget | —2-hour budget —6-hour budget | —2-hour budget —6-hour budget | —2-hour budget —6-hour budget | —2-hour budget —6-hour budget | —2-hour budget —6-hour budget | —2-hour budget —6-hour budget | —2-hour budget —6-hour budget | —2-hour budget —6-hour budget |
| Reasoning & mathsHumanity’s Last ExamDiamond · text-only · no tools · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyExploitBenchGLM · union across 3 revisions · Mean capability coverage | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyExploitGymGLM · 6-hour TPS-normalised budget · Exploitation pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingQwenSWEBenchIssues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingSWE-MMPublic dev · modified tasks · Issues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMathVisionNo tools · Percentage correct | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python | —No tools —With Python |
| MultimodalOmniDocBenchVersion 1.5 · Document parsing score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingDSBenchFullStack set · Issues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingCode ArenaRating | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsOSWorldVersion 2.0 · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingAPEX-SWEIssues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeRULERVersion 1 · 1M tokens · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeBEAM128K context · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsGAIAVersion 2 · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMMMU ProStandard 10 protocol · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyCI MemoriesCoverage · Coverage | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyCI MemoriesViolation rate · ScoreLower is better | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetySiren AgentDojoAttack success · ScoreLower is better | —Attack success rate —Utility | —Attack success rate —Utility | —Attack success rate —Utility | —Attack success rate —Utility | —Attack success rate —Utility | —Attack success rate —Utility | —Attack success rate —Utility | —Attack success rate —Utility | —Attack success rate —Utility |
| MultimodalOmniDocBenchVersion 1.5 · Meta scoring · Document parsing score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsSkillsBench86 tasks · With skills · 86 tasks · Partial credit · Mean task reward | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingMeta Internal Coding BenchIssues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeOneMillion-BenchExpert set · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingAndroid Bench95-task public subset · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingMLS-Bench-LiteIssues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingPaperBenchScore | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingQwenQoderBenchIssues resolved | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingQwenReactBenchRating | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingQwenSVGBenchRating | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsWorkSpaceBenchTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalPRBenchFinance · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalPRBenchLegal · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeMRCR v2256K · 8 needle · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsSkillsBenchVersion 1.1 · 87 tasks · Mean benchmark score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsMCPMark-VerifiedTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsResearchRubricsScore | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsSaaS-BenchTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalCorpFinVersion 2 · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalSpreadsheetBenchVersion 2 · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMMVUPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalVideo-MMEWith subtitles · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalWorldVQAForceAnswer · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalOmniDocBenchVersion and measure unspecified · Reported score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsE-Commerce Bench2026 initial release · Year-end assets · Mean final assets · CNY thousands | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingFrontierBenchv0.1 · Mean task reward | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingSWE MarathonFirst-attempt pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeSimpleQA VerifiedPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsFrontierMathTier 1–3 v2 · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsToolathlonTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsSkillsBench1.1 · 87 tasks · With Skills · Resolution rate · Resolution rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyFirefox 147 exploit developmentTasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsHumanity’s Last ExamText-only · With tools · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsHumanity’s Last ExamWith tools · subset unspecified · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeAdvanced IFRubric score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeAIR-BenchPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeLongFactClaim precision | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeTruthfulQAPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingTerminal-BenchVersion 2.0 · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalMedXpertQAPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyCyberSecEval 4Autocomplete · Code safety score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| SafetyCyberSecEval 4Instruct · Code safety score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalFinanceAgentv1.1 · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| ProfessionalGDPvalWin or tie rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsFrontierMathTier 1–3 · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsFrontierMathTier 4 · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingTerminal-BenchHard set · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingTerminal-BenchEarlier release · version unspecified · Tasks completed | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Agents & toolsSimpleQAWith search tools · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingLiveCodeBenchJanuary–May 2025 · Reported score (%) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeMMLUPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsAIME2024 test · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsMATH-500Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeSimpleQAPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingFullStackBenchPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeMMMLU14 languages · 14-language average · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| MultimodalMMMUValidation set · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsMGSMPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingLiveCodeBenchOctober2024–February2025 · 1Oct2024–1Feb2025 · First-attempt pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsMATHPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingHumanEvalFirst-attempt pass rate | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsBIG-Bench HardPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsGSM8KPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsGPQA MainMain set · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| CodingLiveCodeBenchAugust–November 2024 · Reported score (%) | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeDROPF1 score | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| Reasoning & mathsGPQASubset unspecified · Percentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeARC-ChallengePercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |
| General knowledgeHellaSwagPercentage correct | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded | —No result recorded |