Mathematical Foundations of Deep Neural Networks

Course Information

Mathematical Foundations of Deep Neural Networks, Spring 2024 (Ernest K. Ryu)

Table of Contents

Ch 1. Optimization and Stochastic Gradient Descent

• 1. Optimization Problem
• 2. Gradient Descent
• Chapter 1 Code

Ch 2. Shallow Neural Networks to Multilayer Perceptrons

• 3. Shallow Neural Networks
• 4. Deep Neural Networks
• Chapter 2 Code

Ch 3. Convolutional Neural Networks

• 5. Convolutional Neural Networks
• 6. Foundations of Design and Training of Deep Neural Networks
• 7. ImageNet Challenge
• Chapter 3 Code

Ch 4. CNNs for Other Supervised Learning Tasks

• 8. CNNs for Other Supervised Learning Tasks
• Chapter 4 Code

Ch 5. Unsupervised Learning

• 9. Autoencoder
• 10. Flow Models
• 11. Variational Autoencoders
• 12. Generative Adversarial Networks
• Chapter 5 Code

Ch A. Appendix - Basics of Monte Carlo

• 13. Basics of Monte Carlo
• Chapter A Code

Python Basics

• Python Lecture 1
• Python Lecture 2
• Python Lecture 3

Materials

• Latex PDF