This article was written by Sebastian Ruder. Sebastian is a PhD student in Natural Language Processing and a research scientist at AYLIEN. He blogs about Machine Learning, Deep Learning, NLP, and startups. Gradient descent is one of the most popular algorithms to perform optimization and by far the most common way to optimize neural networks. At… Read More »An overview of gradient descent optimization algorithms