Deep Learning
Neural networks from the single perceptron to modern transformers: how depth, the right activations, and gradient-based training let a model learn its own features instead of hand-crafted ones.
Prerequisites: the Machine Learning course, especially Multilayer neural networks, which builds the model and covers the losses and backpropagation used throughout this course. Basic Python, calculus, and linear algebra.
Syllabus
- Introduction
- Activation functions
- Optimization
- Training deep networks
- Convolutional networks
- Embeddings and representation learning
- Recurrent networks
- LSTM and GRU
- Attention
- Transformers