A journey into Optimization algorithms for Deep Neural Networks

An overview of the most popular optimization algorithms for training deep neural networks. From stohastic gradient descent to Adam, AdaBelief and second-order optimization