PUBLISHED BENCHMARK RESULTS2023 TO PRESENT

AI model
benchmark scores

Published results for major AI models, with the source and method for each score.

223models
486benchmarks
2,087reported results
37labs

Scored models released or announced since 2023. Each model–benchmark result is counted once.

Data review in progress. Some scores and benchmark definitions still await source verification.

01

Benchmark scores

Compare published scores and the methods behind them.

One selected result per model and benchmark.
Green marks the best result with matching detailed method records; this is not independent verification. Amber is a reference when methods differ or are incomplete. Open a score for its source and method.486 benchmarks · 223 models · newest → oldest
BenchmarkTest set / measureNanbeigeNanbeige4.3-33B25 Oct 2026Moonshot AIKimi K3.1 Pro23 Oct 2026NVIDIANemotron 3.5 Bolt E4B 4GB (Q4_K_M)23 Oct 2026FastinoGLiDE v1.222 Oct 2026InclusionAILing-3.0-flash-Sante22 Oct 2026SubquadraticSubQ Snap 1.122 Oct 2026Basetenelephant-1 (9B)21 Oct 2026Basetenmammoth-1 (27B)21 Oct 2026RecursiveRecursive Flint21 Oct 2026
Reasoning & mathsAIME2026 test · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsGPQA DiamondPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingNL2RepoIssues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSciCodePercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSWE-Bench ProIssues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingTerminal-BenchVersion 2.1 · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsAutomationBenchVersion 1.0.6 · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsSkillsBenchTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalJobBenchTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingTerminal-BenchVersion 4.0 · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsToolathlonVerified · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSWE-Atlas-QnAGrading variant unspecified · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingDeepSWEVersion unspecified · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalGDPvalRevised rubrics · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsClaw-EvalGeneral subset · pass^3 · Reported pass^3—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsDeepPlanningReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsEnterpriseOps-GymReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsYC-benchReported dollars · Reported dollars—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsτ³-BenchSubset unspecified · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsAA-LCRVersion unspecified · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsHumanity’s Last ExamNo tools · Percentage correct
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
CodingDeepSWEv1.1 · Issues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingKimi Code BenchVersion 2.0 · Issues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSWE-MarathonVersion 1.1 · First-attempt pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsAPEX-AgentsTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsBrowseCompPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalOfficeQA ProPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalBabyVisionNo tools · Percentage correct
—No tools
—With Python
—No tools
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
MultimodalCharXiv ReasoningNo tools · Percentage correct
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
MultimodalMMMU ProNo tools · Percentage correct
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
MultimodalPerceptionBenchPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalZeroBenchNo tools · Five-attempt pass rate
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
CodingTerminal-Bench-ScienceVersion 0.1 · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalChartographyPercentage correct
—No tools
—With tools
—No tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
ProfessionalGDP.pdfCriteria passed
—No tools
—With tools
—No tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
ProfessionalLegal Agent BenchmarkPublic set · All-pass rate
—All criteria passed
—Criteria passed
—Criteria passed
—All criteria passed
—Criteria passed
—All criteria passed
—Criteria passed
—All criteria passed
—Criteria passed
—All criteria passed
—Criteria passed
—All criteria passed
—Criteria passed
—All criteria passed
—Criteria passed
—All criteria passed
—Criteria passed
CodingProgramBenchAlmost@1 · Tasks meeting the almost-solved threshold—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingPostTrainBenchv1.2 · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalGDPval-AA2.1 · Rating—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalAA-Briefcase1.1 · Rating—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsAutomationBenchArtificial Analysis · Strict pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsAgents’ Last Examv1.1 · all 122 tasks · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsAgents’ Last Examv1.1 · CLI subset · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalBenchCAD Vision2Code1,000-task subset · With tools · Voxel IoU—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalBenchCAD Vision2Code1,000-task subset · No tools · Voxel IoU—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsBioMysteryBenchv11 · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingKimi Code Benchv3.0 · Long-horizon · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalOfficeQA Prov2 · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalFrontier FinanceFinance-agent v2 harness · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsHANDBOOK.mdv1 · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsOSWorldv2.1 · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalBusiness OperationKimi internal · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalDECK-ArenaKimi internal · Rating—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalDesign BenchKimi internal · Rating—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyKimi Hallu Bench HardWith tools · Claim-level hallucination rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyKimi Hallu BenchNo tools · Claim-level hallucination rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalGDP.pdfNo tools · strict pass · Strict pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsHumanity’s Last Exam DiamondNo tools · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalZeroBenchMain set · no tools · avg@5 · Mean accuracy over five runs—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyExploitBenchProgressive flags · pass@5 · Flags reached · pass@5—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyExploitGym6 hours · percentage · Exploitation success rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Safetyxdev-bench-hardKimi internal · Three-attempt pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalHLAB-Valsv1.1 · All criteria passed · Task pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeMultiChallengePercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsBFCLVersion 3 · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeIFBenchPrompt loose · Prompts passed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsτ²-benchDomain average · Average accuracy—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeRoleBenchCustom reference comparison · Wins against reference—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeRoleMRCExcludes multi-turn · Judge average—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeEQ-BenchVersion 3 · Rubric · Rubric score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeBrevity BenchmarkNVIDIA internal · Quality and conciseness—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsWhen2CallMulti-turn · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsToolSandboxAverage accuracy—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeIHEvalConflict subset · Conflict accuracy—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsAgentIFInstructions satisfied—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeStructEval-TAverage accuracy—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeInverseIFEvalOverall accuracy—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeRULER64K tokens · Overall accuracy—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeLongMemEvalOverall accuracy—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeLongBenchVersion 2 · 48K tokens · Overall accuracy—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeSysBenchInstructions satisfied—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeGaRAGeRelevance-aware factuality · Relevance-aware factuality—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeRAGTruthF1 · F1 score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsRelyToolBenchNative/real · Native/real pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsHugging Face Decision Indexv0.4.1 · full · Full skill index—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsMCP-AtlasPublic set · Public test set · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsτ³-BankingTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalGDPval-AAv2 · Rating—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsDeepSearchQAReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalHealthBench ProfessionalAdjustment unspecified · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalAFUMED-ComplexReasoningReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalAFUMED-DrugContraindication accuracy—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalMedQA-USMLEPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalDiagnosisArena-MCQPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalReDisQAReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalMedMCQAPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyAFUSAFE-MedSCEReported safety score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalMedXpertQAText subset · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsArtificial Analysis Decision Indexv1.0 · Index score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSWE-bench VerifiedIssues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSWE-Bench ProVersion 2 · Issues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalAA Word Error Rate Index2 · Non-streaming · Word error rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeIFBenchPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingDeepSWEv1.0 · Issues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsAA-LCRv1.1 · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalGDPval EloVersion unspecified · Reported rating—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeAA-Omniscience PublicPublic set · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeMRCR v2Context unspecified · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSWE-Bench AtlasIssues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsBrowseComp+Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsAgents’ Last ExamTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyCyberGymTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsHumanity’s Last ExamText-only · No tools · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalBabyVisionClaude Code tools · First-attempt pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalCharXiv ReasoningTools not specified · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingPostTrainBench1.1 · Weighted aggregate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalBankerToolBenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsLOCA-Bench256K context · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingNL2Repo-BenchTwo corrected tasks · October 2026 · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSWE-Refactor-BenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalGDP.pdf-AAAA · reported score · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsOSWorld2.0 · scoring unspecified · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingNL2Repo-BenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingApexBenchv1.1 · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalBabyVisionWith tools · harness unspecified · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalZeroBenchMain set · tools · aggregation unspecified · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeLongBenchVersion 2 · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsAA-LCRPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsCritPtScore—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingLiveCodeBenchFirst-attempt pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSWE-bench MultilingualIssues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsDeepSearchQAF1 score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSWE-Atlas-QnAStrict · Strict accuracy—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsAutomationBenchPublic split · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsMCP-AtlasSplit unspecified · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeAA-OmnisciencePublic index · Public penalised index—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSWE-Bench Pro2 · Hard · Issues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyOpenAI Voice-Native SafetyOctober 2026 · Production · Sexual · Safety score (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyOpenAI Voice-Native SafetyOctober 2026 · Production · Illicit Behaviour · Safety score (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyOpenAI Voice-Native SafetyOctober 2026 · Production · Mental Health · Safety score (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyOpenAI Voice-Native SafetyOctober 2026 · Production · Personal Data · Safety score (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyOpenAI Voice-Native SafetyOctober 2026 · Production · Emotional Reliance · Safety score (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyOpenAI Voice-Native SafetyOctober 2026 · Production · Self Harm · Safety score (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyOpenAI Voice-Native SafetyOctober 2026 · Synthetic · Sexual · Safety score (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyOpenAI Voice-Native SafetyOctober 2026 · Synthetic · Illicit Behaviour · Safety score (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyOpenAI Voice-Native SafetyOctober 2026 · Synthetic · Mental Health · Safety score (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyOpenAI Voice-Native SafetyOctober 2026 · Synthetic · Personal Data · Safety score (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyOpenAI Voice-Native SafetyOctober 2026 · Synthetic · Emotional Reliance · Safety score (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyOpenAI Voice-Native SafetyOctober 2026 · Synthetic · Self Harm · Safety score (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyOpenAI Voice-Native SafetyOctober 2026 · Synthetic · Hate · Safety score (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyOpenAI Voice-Native SafetyOctober 2026 · Synthetic · Gore · Safety score (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeIFM Arabic evaluation suite2026-10-15 · Jais 3 release suite · Arabic benchmark average—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalOpen Universal Arabic ASR2026-10 · Six public sets · WER · Word error rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalOpen Universal Arabic ASR2026-10 · Six public sets · CER · Character error rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsOSWorld-VerifiedTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSciCodeTest subproblems · Background provided · 288 subproblems · No tools · Subproblems passed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeAA-OmniscienceIndex · Penalised knowledge index—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeArtificial Analysis Intelligence Index4.3 · Index score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalWorldVQATools · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalSimpleVQATools · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalIntPhys 2Physical-consistency accuracy—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalVideo-MME-v22 · Subtitles, no audio · Grouped benchmark score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeAA-OmnisciencePercentage correct
—Accuracy
—Non-hallucination
—Accuracy
—Non-hallucination
—Accuracy
—Non-hallucination
—Accuracy
—Non-hallucination
—Accuracy
—Non-hallucination
—Accuracy
—Non-hallucination
—Accuracy
—Non-hallucination
—Accuracy
—Non-hallucination
—Accuracy
—Non-hallucination
Agents & toolsPinchBenchTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeIFBenchInstruction loose · Constraints satisfied—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsWebArenaVision · 812 tasks · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingLiveCodeBenchVersion 6 · First-attempt pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsτ²-benchTelecom subset · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalScreenSpot ProPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeMMLU-ProX10-language average · Mean accuracy—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsMathArena Apex ShortlistShortlist · No tools · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeGraphWalks950 prompts · Up to 1M tokens · Node-set F1—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeIFBenchInstruction strict · Constraints satisfied—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeNVIDIA Chat Benchmark3,713-prompt internal set · Pairwise wins—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeWMT24++English → nine languages · XCOMET-XXL · XCOMET-XXL score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsτ²-benchRetail · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsτ²-benchAirline · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSciCode65 whole problems · Background provided · Whole problems passed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalV*Static · No visual-search tool · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalOCR-ReasoningPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalOCRBench2 · English · English score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalOCRBench2 · Chinese · Chinese score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalVideo-MMEVersion 2 · No subtitles · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMMLongBench-DocPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsOSWorld-Verified361 tasks · Partial credit · Mean task score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeDecision Index0.3 · Public fast subset · 1,068 rows · Public fast-subset index—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalAA Speech to Speech Index2.0 · Index score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalBig Bench Audio1.2 · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalFull Duplex BenchAA subset of v1 and v1.5 · Conversational dynamics score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsτ-VoiceTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalSpeech Agent ArenaPreference · Preference rating—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalSpeech Agent ArenaTask success · Task success rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsEVA-BenchAccuracy · Accuracy—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeGraphWalksUp to 128K · F1 score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsLAB-BenchVersion 2 · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsAutomationBenchPrivate set · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalFinance AgentVersion 2 · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalHarvey LABVals v1 · All-pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalLVBenchPercentage correct
—Static
—Agentic
—Static
—Agentic
—Static
—Agentic
—Static
—Agentic
—Static
—Agentic
—Static
—Agentic
—Static
—Agentic
—Static
—Agentic
—Static
—Agentic
CodingFrontierSWEVersion 2 · Score (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsOSWorld2.0 · August 2026 tasks · Partial credit · Partial score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalVals IndexSeptember 2026 snapshot · Weighted score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingVibe Code BenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsRiemannBenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeGraphWalks256K–1M · BFS · F1 score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyCWE-Bench1 · Repair success rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingFrontierSWEScore—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsWideSearchTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsDRACONormalised score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsHumanity’s Last ExamSet and tools unspecified · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsSuperMatBenchQA · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsSuperMatBenchAgent · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalFinSearchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalLegal Research BenchPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalPLawBenchPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalLegalBenchVals evaluation · Accuracy—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalGDP.pdfTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalHealthBench ProfessionalLength-adjusted rubric score
—Length-adjusted
—Raw score
—Length-adjusted
—Raw score
—Length-adjusted
—Raw score
—Length-adjusted
—Raw score
—Length-adjusted
—Raw score
—Length-adjusted
—Raw score
—Length-adjusted
—Raw score
—Length-adjusted
—Raw score
—Length-adjusted
—Raw score
ProfessionalHarvey Legal Agent Benchmark1.1 · AA all-pass · All-pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgePersonal Superintelligence IndexComposite score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsOSWorld2.1 · Online and offline · Partial score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsOSWorld2.1 · Binary reward · Strict pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsMemoryBenchMeta evaluation · Weighted rubric score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsFocusBenchMeta evaluation · Persistent-agent score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSWE-MarathonVersion unspecified · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyExploitBenchTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeMicrosoft Decision RobustnessEight perturbations · Decision-flip rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingHumanEval+First-attempt pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingMBPP+First-attempt pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsBFCLv4 · JetBrains five-part macro-average · Five-part macro-average—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingFrontierCodev1.1 Main · Composite task score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsOSWorld2.1 · Offline · Reported offline score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsHugging Face Decision Indexv0.2.1 · public · Skill index—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalHealthBenchRubric score
—Raw score
—Length-adjusted
—Raw score
—Length-adjusted
—Raw score
—Length-adjusted
—Raw score
—Length-adjusted
—Raw score
—Length-adjusted
—Raw score
—Length-adjusted
—Raw score
—Length-adjusted
—Raw score
—Length-adjusted
—Raw score
—Length-adjusted
ProfessionalHealthBench HardLength-adjusted · Length-adjusted rubric score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalHealthBench HardRaw · Rubric score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalHealthBench ConsensusLength-adjusted · Length-adjusted rubric score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalHealthBench ConsensusRaw · Rubric score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalMentalHealthBenchOverall · Task-clipped rubric score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalAA-BriefcaseRating—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingCyberGymEnd-to-end · End-to-end success—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingCybenchCTF tasks solved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingCoding Agent IndexVersion unspecified · Index score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalDense200Grounding accuracy—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyB3 AI SecurityAttack resistance · Attack resistance—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalKORAScore (0–2)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeMMLU-ProPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsAIME2025 test · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalArtificial Analysis Streaming ASRFinal transcript · Final transcript word error rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyExploitGymNo time limit · Exploitation success rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyExploitBenchJune–August 2026 subset · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetySEC-Bench ProTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyOpenAI User-Flagged FactualityResponses containing factual errorsLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingCursorBench4.0 · Issues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsAndroidWorldTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsOSWorldHolo4 release · version unspecified · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingAI Coding Daily70-point total · Points / 70—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsFast Decisions17-dataset suite · Micro-task accuracy—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalCT-RATEDirect yes/no · 18 labels · Macro-F1 (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalCT-RATEDirect yes/no · 18 labels · Macro-AUROC (0–1)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingFrontierCode1.1 Main · AA · Pass rate · All blocking checks passed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsAgents’ Last ExamScore · Score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingMiMo Code MiniReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyMiMo Cyber MiniReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsMiMo General MiniReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMiMo Visual MiniReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingProgramBenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingMiMo Code BenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyMiMo Cyber BenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMiMo Visual CodingReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyCyberGymMiMo-corrected environments · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyExploitGymMiMo · budget unspecified · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyExploitBenchMiMo · reported score · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeArtificial Analysis Intelligence Indexv4.3.2 · Index score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsAutomationBench-AAv1.0.6 held-out · Partial objective credit · Mean objective score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalLegal Agent BenchmarkHeld-out · All criteria passed · All-pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalEEBenchV1 core corpus · Functional reward—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalCADGenBenchGeneration · Generation reward—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalLatchBio Capabilities1.0 · Mean capability score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalHealthBench ProfessionalAdjustment unspecified · Reported rubric score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsAgents’ Last ExamCLI · version unspecified · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalFrontier FinanceHarness version unspecified · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingStep Code BenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalGrok short-phrase transcriptionInternal test set · Word error rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsClawEval-MMAverage score · Score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsClawEval-MMThree-attempt pass rate · Three-attempt pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalERQAPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalRealWorldQAPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsWildClawBench-MMReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsUniClawBenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsAgenticVBenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsOmniGAIAReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalDailyOmniReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalWorldSenseReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalAVUTReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalJoinAVBenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalOmniVideoBenchReported score
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
MultimodalVideo-MME-v2Reported score
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
MultimodalLVOmniBenchReported score
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
—Static
—Qwen Code
MultimodalOmniClozeReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalOmniCap-IFCSR · CSR score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalOmniCap-IFISR · ISR score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalQIVDReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalStreamingBenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalAliMeeting TestDER · Diarisation error rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalAliMeeting TestCPWER · Concatenated minimum-permutation word error rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalAISHELL-4DER · Diarisation error rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalAISHELL-4CPWER · Concatenated minimum-permutation word error rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMagicData-RAMCDER · Diarisation error rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMagicData-RAMCCPWER · Concatenated minimum-permutation word error rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMLC-SLM (en)DER · Diarisation error rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMLC-SLM (en)CPWER · Concatenated minimum-permutation word error rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalWenetSpeech (Net)Word error rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalWenetSpeech (Meeting)Word error rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalFLEURS-ASRQwen 60-language set · Word error rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalFLEURS-S2TTQwen 60-language set · BLEU score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalSpotSoundBenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMMAUReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMMARReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMMSUReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalLongAudioSpanAccuracy · Accuracy score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalLongAudioSpanRubric · Rubric score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalLongAudioSpanChain · Chain score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMuchoMusic-RULReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalHumMusQAReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMusTBenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalAudio MultiChallengeReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalWildSpeechReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalVoiceBenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSWE-Bench ProQwen-refined set · September 2026 · Issues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsCoWorkBenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMathVisionPercentage correct
—Corrected set · No code interpreter
—Corrected set · With code interpreter
—Corrected set · No code interpreter
—Corrected set · With code interpreter
—Corrected set · No code interpreter
—Corrected set · With code interpreter
—Corrected set · No code interpreter
—Corrected set · With code interpreter
—Corrected set · No code interpreter
—Corrected set · With code interpreter
—Corrected set · No code interpreter
—Corrected set · With code interpreter
—Corrected set · No code interpreter
—Corrected set · With code interpreter
—Corrected set · No code interpreter
—Corrected set · With code interpreter
—Corrected set · No code interpreter
—Corrected set · With code interpreter
MultimodalVision2WebMean judge score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalLegalResearchBenchPrivate validation · 200 questions · Overall correctness—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsτ³-Bench BankingVoice · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingTerminal-BenchVersion 3.0 · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsAutomationBenchTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingCodeforcesRating—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsMathArena ApexFirst-attempt pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyExploitGymTime budget unspecified · Exploitation pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalZeroBenchMain set · Claude Code tools · Five-attempt pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeMRCR v2128K · 8 needle · Percentage correct
—128K · 8 needle
—256K–512K · 8 needle
—512K–1M · 8 needle
—128K · 8 needle
—256K–512K · 8 needle
—512K–1M · 8 needle
—128K · 8 needle
—256K–512K · 8 needle
—512K–1M · 8 needle
—128K · 8 needle
—256K–512K · 8 needle
—512K–1M · 8 needle
—128K · 8 needle
—256K–512K · 8 needle
—512K–1M · 8 needle
—128K · 8 needle
—256K–512K · 8 needle
—512K–1M · 8 needle
—128K · 8 needle
—256K–512K · 8 needle
—512K–1M · 8 needle
—128K · 8 needle
—256K–512K · 8 needle
—512K–1M · 8 needle
—128K · 8 needle
—256K–512K · 8 needle
—512K–1M · 8 needle
Reasoning & mathsARC-AGIVersion 1 · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsARC-AGIVersion 2 · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsARC-AGIVersion 3 · Relative Human Action Efficiency—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsFrontierMathTier 4 v2 · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingFrontierCodeVersion 1.1 Extended · Composite task score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalBenchCAD Vision2CodeVoxel IoU
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
ProfessionalOpenScore String Quartets1 − OMR-NED—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalOpenAI Internal Design TasksTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeArtificial Analysis Intelligence IndexVersion 4.1.1 · Index score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalOpenAI Internal Data Science TasksTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingOpenAI Internal Database Migration TasksTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingArtificial Analysis Coding Agent IndexVersion 1.4 · Index score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalGeneBench ProPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalMedChemBenchOpenAI internal set · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalLifeSciBenchPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSRE-BenchFirst-attempt pass rate
—First attempt
—Within four attempts
—First attempt
—Within four attempts
—First attempt
—Within four attempts
—First attempt
—Within four attempts
—First attempt
—Within four attempts
—First attempt
—Within four attempts
—First attempt
—Within four attempts
—First attempt
—Within four attempts
—First attempt
—Within four attempts
SafetyOpenAI Internal Computer-Use SafetyUnsafe-action rateLower is better
—Standard review
—With AutoReview
—Standard review
—With AutoReview
—Standard review
—With AutoReview
—Standard review
—With AutoReview
—Standard review
—With AutoReview
—Standard review
—With AutoReview
—Standard review
—With AutoReview
—Standard review
—With AutoReview
—Standard review
—With AutoReview
SafetyOpenAI Internal Circumvention EvaluationCircumvention rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyExploitGym HoneypotHoneypot-trigger rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyImpossible ExploitGymSafe handling rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyOpenAI Internal Hallucination EvaluationHallucination rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsHMMTFebruary 2026 · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsBFCLVersion 4 · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsWildClawBenchTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsMCPMarkTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsBioMysteryBenchHard set · Tasks solved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsBioMysteryBenchHuman-solved comparison · Tasks solved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsHumanity’s Last ExamVerified set · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsOSWorld2.0 binary · Score
—Binary reward
—Partial reward
—Binary reward
—Partial reward
—Binary reward
—Partial reward
—Binary reward
—Partial reward
—Binary reward
—Partial reward
—Binary reward
—Partial reward
—Binary reward
—Partial reward
—Binary reward
—Partial reward
—Binary reward
—Partial reward
SafetyGray Swan IPI BenchmarkASR@15 · Attack success at 15 · Attack success rateLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyGoogle Internal Vulnerability Discovery20-language set · Vulnerabilities found—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyCWE-BenchTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingCursorBenchVersion 3.2 · Issues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalOfficeQAPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSWE-bench MultimodalIssues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsCritPT-CorrectedFirst-attempt pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeGlobal MMLU42 languages · 42-language average · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeMILU11 languages · 11-language average · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalHarvey LAB-AA120 held-out tasks · All-pass rate
—All criteria passed
—Criteria passed
—All criteria passed
—Criteria passed
—All criteria passed
—Criteria passed
—All criteria passed
—Criteria passed
—All criteria passed
—Criteria passed
—All criteria passed
—Criteria passed
—All criteria passed
—Criteria passed
—All criteria passed
—Criteria passed
—All criteria passed
—Criteria passed
Agents & toolsOSWorld2.0 · August 2026 · Anthropic modified tasks · Partial score
—Partial credit
—Strict pass
—Partial credit
—Strict pass
—Partial credit
—Strict pass
—Partial credit
—Strict pass
—Partial credit
—Strict pass
—Partial credit
—Strict pass
—Partial credit
—Strict pass
—Partial credit
—Strict pass
—Partial credit
—Strict pass
Reasoning & mathsArXivMathJune 2026 · Percentage correct
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
—No tools
—With tools
ProfessionalSpatialBench VerifiedScore—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalSingleCellBenchScore—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalProteinGym HardRank correlation—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalProtein DesignScore
—Sequence generation
—Library ranking
—Sequence generation
—Library ranking
—Sequence generation
—Library ranking
—Sequence generation
—Library ranking
—Sequence generation
—Library ranking
—Sequence generation
—Library ranking
—Sequence generation
—Library ranking
—Sequence generation
—Library ranking
—Sequence generation
—Library ranking
ProfessionalOrganic ChemistryVersion 2 · Score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalMolecular Biology ProtocolsScore
—Troubleshooting
—Understanding
—Troubleshooting
—Understanding
—Troubleshooting
—Understanding
—Troubleshooting
—Understanding
—Troubleshooting
—Understanding
—Troubleshooting
—Understanding
—Troubleshooting
—Understanding
—Troubleshooting
—Understanding
—Troubleshooting
—Understanding
Agents & toolsSkillsBenchTencent · 79 text tasks · three-run mean · Mean reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsCoWorkBenchReported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsRecreationBenchTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalVision2WebMean judged score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSWE-Bench ProQwen-refined set · August 2026 · Issues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingApexBenchFirst-attempt pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingDSBenchHard set · Issues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingPostTrainBenchPost-training aggregate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingProgramBenchAlmost-solved threshold · Programs meeting the almost-solved threshold—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyExploitGym2 hours · Tasks solved
—2-hour budget
—6-hour budget
—2-hour budget
—6-hour budget
—2-hour budget
—6-hour budget
—2-hour budget
—6-hour budget
—2-hour budget
—6-hour budget
—2-hour budget
—6-hour budget
—2-hour budget
—6-hour budget
—2-hour budget
—6-hour budget
—2-hour budget
—6-hour budget
Reasoning & mathsHumanity’s Last ExamDiamond · text-only · no tools · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyExploitBenchGLM · union across 3 revisions · Mean capability coverage—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyExploitGymGLM · 6-hour TPS-normalised budget · Exploitation pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingQwenSWEBenchIssues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSWE-MMPublic dev · modified tasks · Issues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMathVisionNo tools · Percentage correct
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
—No tools
—With Python
MultimodalOmniDocBenchVersion 1.5 · Document parsing score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingDSBenchFullStack set · Issues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingCode ArenaRating—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsOSWorldVersion 2.0 · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingAPEX-SWEIssues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeRULERVersion 1 · 1M tokens · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeBEAM128K context · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsGAIAVersion 2 · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMMMU ProStandard 10 protocol · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyCI MemoriesCoverage · Coverage—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyCI MemoriesViolation rate · ScoreLower is better—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetySiren AgentDojoAttack success · ScoreLower is better
—Attack success rate
—Utility
—Attack success rate
—Utility
—Attack success rate
—Utility
—Attack success rate
—Utility
—Attack success rate
—Utility
—Attack success rate
—Utility
—Attack success rate
—Utility
—Attack success rate
—Utility
—Attack success rate
—Utility
MultimodalOmniDocBenchVersion 1.5 · Meta scoring · Document parsing score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsSkillsBench86 tasks · With skills · 86 tasks · Partial credit · Mean task reward—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingMeta Internal Coding BenchIssues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeOneMillion-BenchExpert set · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingAndroid Bench95-task public subset · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingMLS-Bench-LiteIssues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingPaperBenchScore—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingQwenQoderBenchIssues resolved—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingQwenReactBenchRating—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingQwenSVGBenchRating—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsWorkSpaceBenchTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalPRBenchFinance · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalPRBenchLegal · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeMRCR v2256K · 8 needle · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsSkillsBenchVersion 1.1 · 87 tasks · Mean benchmark score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsMCPMark-VerifiedTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsResearchRubricsScore—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsSaaS-BenchTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalCorpFinVersion 2 · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalSpreadsheetBenchVersion 2 · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMMVUPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalVideo-MMEWith subtitles · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalWorldVQAForceAnswer · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalOmniDocBenchVersion and measure unspecified · Reported score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsE-Commerce Bench2026 initial release · Year-end assets · Mean final assets · CNY thousands—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingFrontierBenchv0.1 · Mean task reward—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingSWE MarathonFirst-attempt pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeSimpleQA VerifiedPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsFrontierMathTier 1–3 v2 · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsToolathlonTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsSkillsBench1.1 · 87 tasks · With Skills · Resolution rate · Resolution rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyFirefox 147 exploit developmentTasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsHumanity’s Last ExamText-only · With tools · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsHumanity’s Last ExamWith tools · subset unspecified · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeAdvanced IFRubric score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeAIR-BenchPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeLongFactClaim precision—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeTruthfulQAPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingTerminal-BenchVersion 2.0 · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalMedXpertQAPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyCyberSecEval 4Autocomplete · Code safety score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
SafetyCyberSecEval 4Instruct · Code safety score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalFinanceAgentv1.1 · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
ProfessionalGDPvalWin or tie rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsFrontierMathTier 1–3 · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsFrontierMathTier 4 · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingTerminal-BenchHard set · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingTerminal-BenchEarlier release · version unspecified · Tasks completed—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Agents & toolsSimpleQAWith search tools · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingLiveCodeBenchJanuary–May 2025 · Reported score (%)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeMMLUPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsAIME2024 test · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsMATH-500Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeSimpleQAPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingFullStackBenchPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeMMMLU14 languages · 14-language average · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
MultimodalMMMUValidation set · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsMGSMPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingLiveCodeBenchOctober2024–February2025 · 1Oct2024–1Feb2025 · First-attempt pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsMATHPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingHumanEvalFirst-attempt pass rate—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsBIG-Bench HardPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsGSM8KPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsGPQA MainMain set · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
CodingLiveCodeBenchAugust–November 2024 · Reported score (%)—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeDROPF1 score—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
Reasoning & mathsGPQASubset unspecified · Percentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeARC-ChallengePercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded
General knowledgeHellaSwagPercentage correct—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded—No result recorded