Entity
METR AI Time Horizons
METR's time-horizon metric quantifies AI capability by measuring the human task-completion time an AI can match with 50% success probability. A new statistical analysis challenges core assumptions in how the metric converts human time into AI difficulty.
This page was written and is updated by the paper from the source articles listed below. Importance is how central the topic is to current coverage; confidence is how well-sourced the page is; mentions counts the articles that fed it. How the wiki works →
METR's "time horizon" metric measures the human completion time of software tasks that an AI system can solve with 50% probability, offering a way to express AI capability progress in interpretable units of human task-time (arXiv:2610.12466, October 12). The metric has become a widely cited benchmark for tracking frontier AI capability trends over time.
A statistical reanalysis of the METR plot, covering 228 tasks and 26 AI systems, questioned a key assumption underlying the original methodology: that AI difficulty scales linearly with the log of human completion time (arXiv:2610.12466, October 12). Using splines and item-response theory instead, the authors found the relationship between human time and AI difficulty is "nearly flat" in the 2-30 minute range but "close to linear" elsewhere, suggesting the conversion between human time and AI difficulty is more complex than previously modeled (arXiv:2610.12466, October 12).
## Why It Matters
As AI capability claims increasingly hinge on metrics like time horizons — used in policy discussions, safety forecasting, and investment decisions — scrutiny of the statistical validity of these measurements has direct implications for how capability progress is reported and interpreted by regulators, labs, and investors.