Spectral bias and task-model alignment explain generalization in kernel regression and infinitely wide neural networks

Abstract

Canatar et al. propose a predictive theory of generalization in kernel regression applicable to real data. This theory explains various generalization phenomena observed in wide neural networks, which admit a kernel limit and generalize well despite being overparameterized.