Skip to content
AIBRAINWORX / THE INDEPENDENT AI LABRESEARCH · HARDWARE · OPEN INTELLIGENCE
AI News

Epoch AI's Capabilities Index combines 50+ benchmarks to counter score saturation

Using Item Response Theory and Bayesian modeling, the ECI places frontier and open-weight models on a single continuous scale as older tests hit ceilings.

Official Epoch AI site graphic from epoch.ai
Epoch AI, official site preview graphic from epoch.ai; identification only, no endorsement.
Event / context: Epoch AI's Epoch Capabilities Index (ECI) was highlighted in November 2026 evaluations.. Source check: October 28, 2026.

As standard single benchmarks such as MMLU, GSM8K, and HumanEval saturate near 100 percent, Epoch AI is using its Epoch Capabilities Index (ECI) to track general model progress across more than 50 underlying evaluations on one continuous scale.

How the composite index works

Rather than averaging raw percentages across incompatible tests, the ECI applies Item Response Theory (IRT) and Bayesian modeling to estimate both a model's underlying capability and each benchmark's difficulty and discrimination slope. Historical anchor points—such as calibrating earlier reference models like Claude 3.5 Sonnet around 130 and GPT-5 around 150—allow newer, harder coding, math, and agentic benchmarks to be stitched onto the same scale as older tests retire.

Strengths and limits of a single number

A composite scale helps researchers and enterprise buyers compare long-term capability trends across proprietary and open-weight releases without chasing individual leaderboard spikes. At the same time, any single aggregate index can mask sharp task-specific tradeoffs—such as latency, cost, tool-use reliability, or domain-specific safety—so teams still need workload-specific evaluations before production deployment.

Original reporting; Epoch AI methodology reviewed November 16, 2026.

Follow your curiosity.

Pass the signal.