Abstract : These lectures present some recent results on two phenomena that have been observed in deep neural networks. The first is benign overfitting: even without any explicit effort to control model complexity, deep learning methods find functions that give a near-perfect fit to noisy training data and yet exhibit good prediction performance in practice. We describe results that characterize this phenomenon in linear regression and in ridge regression. The second phenomenon that we consider is that of adversarial examples: functions computed by deep networkscan be extremely sensitive to small changes in their inputs. We show that this occurs in ReLU networks of constant depth with independent gaussian parameters because the functions that these networks compute are close to linear. The lectures include joint work with Seb Bubeck, Yeshwanth Cherapanamjeri, Phil Long, Gabor, Lugosi, and Alex Tsigler.

Keywords : mathematical statistics; statistical learning theory; linear regression; bias-variance trade-off; ridge regression