Deep Learning Optimizers: From Gradient Descent to Adam and Beyond

Gradient Computation in Deep Nets

Gradient Update Rules: From Vanilla Gradient Descent to Modern Optimizers

Backpropagation Algorithm