I will give an elementary introduction of basic deep learning models and training algorithms from a mathematical viewpoint. In particular, I will relate some basic deep learning models with finite element and multigrid methods. I will also touch on some advanced topics to demonstrate the potential of new mathematical insight and analysis for improving the efficiency of deep learning technologies and, in particular, for their application to numerical solution of partial differential equations.[-]

In this series of lectures, I will report some recent development of the design and analysis of neural network (NN) based method, such as physics-informed neural networks (PINN) and the finite neuron method (FNM), for numerical solution of partial differential equations (PDEs). I will give an overview on convergence analysis of FNM, for error estimates (without or with numerical quadrature) and also for training algorithms for solving the relevant optimization problems. I will present theoretical results that explains the success as well as the challenges of PINN and FNM that are trained by gradient based methods such as SGD and Adam. I will then present some new classes of training algorithms that can theoretically achieve and numerically observe the asymptotic rate of the underlying discretization algorithms (while the gradient based methods cannot). Motivated by our theoretical analysis, I will finally report some competitive numerical results of CNN and MgNet using an activation function with compact support for image classifications.[-]

In this series of lectures, I will report some recent development of the design and analysis of neural network (NN) based method, such as physics-informed neural networks (PINN) and the finite neuron method (FNM), for numerical solution of partial differential equations (PDEs). I will give an overview on convergence analysis of FNM, for error estimates (without or with numerical quadrature) and also for training algorithms for solving the relevant optimization problems. I will present theoretical results that explains the success as well as the challenges of PINN and FNM that are trained by gradient based methods such as SGD and Adam. I will then present some new classes of training algorithms that can theoretically achieve and numerically observe the asymptotic rate of the underlying discretization algorithms (while the gradient based methods cannot). Motivated by our theoretical analysis, I will finally report some competitive numerical results of CNN and MgNet using an activation function with compact support for image classifications.[-]

In this series of lectures, I will report some recent development of the design and analysis of neural network (NN) based method, such as physics-informed neural networks (PINN) and the finite neuron method (FNM), for numerical solution of partial differential equations (PDEs). I will give an overview on convergence analysis of FNM, for error estimates (without or with numerical quadrature) and also for training algorithms for solving the relevant optimization problems. I will present theoretical results that explains the success as well as the challenges of PINN and FNM that are trained by gradient based methods such as SGD and Adam. I will then present some new classes of training algorithms that can theoretically achieve and numerically observe the asymptotic rate of the underlying discretization algorithms (while the gradient based methods cannot). Motivated by our theoretical analysis, I will finally report some competitive numerical results of CNN and MgNet using an activation function with compact support for image classifications.[-]