Deep Learning
Neural networks from the single perceptron to modern transformers: how depth, the right activations, and gradient-based training let a model learn its own features instead of hand-crafted ones.
Prerequisites: the Machine Learning course (especially the perceptron in Linear classification), basic Python, calculus, and linear algebra.
Syllabus
- Introduction
- Multilayer perceptron
- Activation functions
- Loss functions and output layers
- Backpropagation
- Optimization
- Initialization and vanishing gradients
- Normalization
- Regularization and dropout
- Convolutional networks
- Embeddings and representation learning
- Recurrent networks
- LSTM and GRU
- Attention
- Transformers