The Mathematical Theory of Neural Network-based Machine Learning

• 主讲人：Weinan E (Peking University)
• 举办方： Beijing-Moscow Mathematics Colloquium
• 时间： 2023-04-07 16:00 - 2023-04-07 17:00
• 地点： online

Recording: https://disk.pku.edu.cn:443/link/6333F9FBDF221B6B354F73F952FB7EE8
Valid Until: 2027-05-31 23:59

 

Abstract: The task of supervised learning is to approximate a function using a given set of data. In low dimensions, its mathematical theory has been established in classical numerical analysis and approximation theory in which the function spaces of interest (the Sobolev or Besov spaces), the order of the error and the convergence rate of the gradient-based algorithms are all well-understood. Direct extension of such a theory to high dimensions leads to estimates that suffer from the curse of dimensionality as well as degeneracy in the over-parametrized regime.

In this talk, we attempt to put forward a unified mathematical framework for analyzing neural network-based machine learning in high dimension (and the over-parametrized regime). We illustrate this framework using kernel methods, shallow network models and deep network models. For each of these methods, we identify the right function spaces (for which the optimal complexity estimates and direct and inverse approximation theorems hold), prove optimal generalization error estimates and study the behavior of gradient decent dynamics.

 

Bio:  Weinan E is a professor in the School of Mathematical Sciences and the Center for Machine Learning Research (CMLR) at Peking University. He is also a professor at the Department of Mathematics and Program in Applied and Computational Mathematics at Princeton University. His main research interest is numerical algorithms, machine learning and multi-scale modeling, with applications to chemistry, material sciences and fluid mechanics.

Weinan E was awarded the ICIAM Collatz Prize in 2003, the SIAM Kleinman Prize in 2009 and the SIAM von Karman Prize in 2014, the SIAM-ETH Peter Henrici Prize in 2019, and the ACM Gordon-Bell Prize in 2020. He is a member of the Chinese Academy of Sciences, and a fellow of SIAM, AMS and IOP. Weinan E is an invited plenary speaker at ICM 2022, an invited speaker at ICM 2002 in Beijing, ICIAM 2007 as well as the AMS National Meeting in 2003. He has also been an invited speaker at APS, ACS, AIChe annual meetings and the American Conference of Theoretical Chemistry.