802679S Mathematical Principles of Deep Learning (5 cr)
Description
Universal approximation theorem, benefit of depth in neural networks (deep learning), infinite width networks, Barron spaces, Neural Tangent Kernel, Optimization and stochastic gradient descent, backpropagation, non differentiability, Nesterov acceleration, Generalisation properties, margin maximization, implicit Bias, Rademacher complexity.
Learning outcomes
Students will get familiar with mathematical concepts of neural networks and deep learning. We will introduce neural networks in a mathematical framework of function approximation and show that neural networks are universal approximator and how deep neural networks achieve efficient representations. We will discuss the concept of infinite width neural networks and Barron spaces. Furthermore, optimization methods, backpropagation and generalization properties of neural networks will be discussed. After the course, the student will understand neural networks from a fundamental mathematical perspective. The course is NOT aimed to discuss applications of neural networks beyond classification.
Additional information
Timing 3rd/last year during B.Sc., M.Sc. studies or PhD studies. Target group Students having mathematics, applied mathematics, or statistics as the major or a minor subject. Theoretical Computer Science. Background in mathematics is necessary.
Description of prerequisites
Core courses in the B.Sc curriculum of mathematical sciences, especially Analysis 1 NM00BD54, Analysis 2 NM00BD55,802651S Measure and Integral. Additionally as recommended: optimization (recommended), Mathematics of Imaging and Vision (beneficial, but not necessary).