
Apple Machine Learning Research published Toward a Theory of Evaluability on November 10, 2026, examining how to formalize foundation model evaluation when classical held-out test-set assumptions no longer hold.
Why held-out assumptions break down
Modern foundation models are evaluated across accuracy, safety, calibration, and fairness, yet internet-scale training makes it increasingly difficult to guarantee that evaluation sets were strictly excluded from training. Apple's paper proposes a theoretical framework for reasoning about evaluability without relying on that classical separation.
Original reporting; Apple Machine Learning Research summary reviewed November 16, 2026.



