Awesome Deep Learning Resources

This is a rough list of my favorite deep learning resources. It has been useful to me for learning how to do deep learning, I use it for revisiting topics or for reference. I (Guillaume Chevalier) have built this list and got through all of the content listed here, carefully.

Contents

• Trends
• Online classes
• Books
• Posts and Articles
• Practical resources
• Librairies and Implementations
• Some Datasets
• Other Math Theory
• Gradient Descent Algorithms and optimization
• Complex Numbers & Digital Signal Processing
• Papers
• Recurrent Neural Networks
• Convolutional Neural Networks
• Attention Mechanisms
• Other
• YouTube and Videos
• Misc. Hubs and Links
• License

Trends

Here are the all-time Google Trends, from 2004 up to now, September 2017:

You might also want to look at Andrej Karpathy's new post about trends in Machine Learning research.

I believe that Deep learning is the key to make computers think more like humans, and has a lot of potential. Some hard automation tasks can be solved easily with that while this was impossible to achieve earlier with classical algorithms.

Moore's Law about exponential progress rates in computer science hardware is now more affecting GPUs than CPUs because of physical limits on how tiny an atomic transistor can be. We are shifting toward parallel architectures [read more]. Deep learning exploits parallel architectures as such under the hood by using GPUs. On top of that, deep learning algorithms may use Quantum Computing and apply to machine-brain interfaces in the future.

I find that the key of intelligence and cognition is a very interesting subject to explore and is not yet well understood. Those technologies are promising.

Online Classes

• DL&RNN Course - I created this richely dense course on Deep Learning and Recurrent Neural Networks.
• Machine Learning by Andrew Ng on Coursera - Renown entry-level online class with certificate. Taught by: Andrew Ng, Associate Professor, Stanford University; Chief Scientist, Baidu; Chairman and Co-founder, Coursera.
• Deep Learning Specialization by Andrew Ng on Coursera - New series of 5 Deep Learning courses by Andrew Ng, now with Python rather than Matlab/Octave, and which leads to a specialization certificate.
• Deep Learning by Google - Good intermediate to advanced-level course covering high-level deep learning concepts, I found it helps to get creative once the basics are acquired.
• Machine Learning for Trading by Georgia Tech - Interesting class for acquiring basic knowledge of machine learning applied to trading and some AI and finance concepts. I especially liked the section on Q-Learning.
• Neural networks class by Hugo Larochelle, Université de Sherbrooke - Interesting class about neural networks available online for free by Hugo Larochelle, yet I have watched a few of those videos.
• GLO-4030/7030 Apprentissage par réseaux de neurones profonds - This is a class given by Philippe Giguère, Professor at University Laval. I especially found awesome its rare visualization of the multi-head attention mechanism, which can be contemplated at the slide 28 of week 13's class.
• Deep Learning & Recurrent Neural Networks (DL&RNN) - The most richly dense, accelerated course on the topic of Deep Learning & Recurrent Neural Networks (scroll at the end).