Deep Learning Reading List
¡¡
Deep Learning for Speech/Videos
Deep (Reinforcement) Learning for Recommendations
Books
1. A text book on Deep Learning written by Ian Goodfellow, Yoshua Bengio, and Aaron Courville
2. A introductory book on Deep Learning Methods and Applications from Microsoft Research for signal and information processing tasks
Tutorials
1. A deep learning tutorial from LISA lab, University of Montreal
2. Deep Learning slides from Andrew Ng
3. A deep learning and applications tutorial from NIPS¡¯10
4. Neural Network Tutorials from Hinton
Papers
1. Deep learning from Nature by Yann LeCun, Yoshua Bengio, and Geoffrey Hinton
2. Deep Learning in Neural Networks: An Overview
3. Reducing the Dimensionality of Data with Neural Networks from Science
Open source code/tools
1. TensorFlow from Google
2. Caffe from Yangqing Jia of UC Berkeley
3. CNTK - Computational Network Toolkit - from Microsoft
4. DeepLearnToolbox - Matlab/Octave toolbox for deep learning
Datasets
2. Imagenet: image database organized according to the WordNethierarchy
3. The Street View House Numbers (SVHN) Dataset
4. Labeled Faces in the Wild
5. 20 Newsgroups: a popular data set for text classification and text clustering
6. MovieLens: for movie rating prediction/recommendation
7. Netflix Prize dataset
8. Book-Crossing dataset for book recommendation
Online courses and other resources
1. Online course for deep learning from Google
2. Online course for machine learning from Andrew Ng
3. Deep learning courses from Machine Learning Mastery
4. More deep learning references
Tutorials
1. Deep learning for computer vision from Stanford U
2. Deep learning for computer vision from Microsoft
Papers
1. ImageNet Classification with Deep Convolutional Neural Networks
2. DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition from UC Berkeley
3. CNN Features off-the-shelf: an Astounding Baseline for Recognition
4. DeepFace: Closing the Gap to Human-Level Performance in Face Verification from FAIR
5. Show and Tell: A Neural Image Caption Generator from Google
6. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention from U of Montreal
7. Going Deeper with Convolutions from Google
8. Deep Residual Learning for Image Recognition from Microsoft Research
9. Deep Networks with Stochastic Depth
10. Swapout: Learning an ensemble of deep architectures
Online courses and other resources
1. Deep learning in computer vision from U of Toronto
2. Convolutional Neural Networks for Visual Recognition from Stanford U
3. Application of deep learning to computer vision from Coursera
4. DeepVision conference
Book
1. Automatic Speech Recognition from Deng Li
Tutorials
1. Deep Neural Network and Its Application in Speech Recognition from Microsoft Research
2. Deep Learning for Speech/Language Processing from Deng Li
Papers
1. Deep Neural Networks for Acoustic Modeling in Speech Recognition from Hinton
2. Conversational Speech Transcription Using Context-Dependent Deep Neural Networks from Microsoft Research
3. Recent Advances in Deep Learning for Speech Research at Microsoft
6. Sequence to Sequence Learning with Neural Networks from Google
7. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation from Bengio
8. Large-scale Video Classification with Convolutional Neural Networks from Feifei Li
9. Long-term Recurrent Convolutional Networks for Visual Recognition and Description
10. Two-Stream Convolutional Networks for Action Recognition in Videos
11. Beyond Short Snippets: Deep Networks for Video Classification
12. Sequence to Sequence ¨C Video to Text
13. Learning Spatiotemporal Features with 3D Convolutional Networks
Online courses and other resources
1. Deep Learning for Perception from GTech
2. Tutorial on Learning Deep Architectures from Yann LeCun
3. Tutorial 4: Deep Learning for Speech Generation and Synthesis
Tutorials
1. Deep Learning for NLP (without Magic) from ACL 2012
2. Natural Language Processing Applications of Deep Learning from Bengio
3. Deep Learning for Question Answering
Papers
1. Efficient Estimation of Word Representations in Vector Space, i.e., word2vec
2. Natural Language Processing (Almost) from Scratch for POS and NER
3. Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach from Bengio
4. Show and Tell: A Neural Image Caption Generator from Google
5. Teaching Machines to Read and Comprehend from DeepMind
6. A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning
7. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation from Bengio
8. A Neural Conversational Model from Google
9. Empirical Study on Deep Learning Models for QA from IBM Watson
10. Open Question Answering with Weakly Supervised Embedding Models
11. Sequence to Sequence Learning with Neural Networks from Google
12. Neural Machine Translation by Jointly Learning to Align and Translate
13. Text Understanding from Scratch from Yann LeCun
Online courses and other resources
1. Deep Learning for Natural Language Processing from Stanford U
2. LSTM Networks for Sentiment Analysis
3. Deep Learning for Natural Language Processing from Oxford U
Tutorials
1. A New Look at the System, Algorithm and Theory Foundations of Distributed Machine Learning from Eric Xing
Papers
1. A fast learning algorithm for deep belief nets
2. Greedy Layer-Wise Training of Deep Networks
3. Representational Power of Restricted Boltzmann Machines and Deep Belief Networks
4. Deep Belief Networks are Compact Universal Approximators
5. On the Number of Linear Regions of Deep Neural Networks
6. Random Search for Hyper-Parameter Optimization
7. No More Pesky Learning Rates
8. Understanding the difficulty of training deep feedforward neural networks
9. Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
10. Deep Boltzmann Machines and the Centering Trick
11. Multi-Prediction Deep Boltzmann Machines
Online courses and other resources
1. Foundations and Challenges of Deep Learning from Bengio
My research mainly focuses on the sequential and spatio-temporal aspects of deep and reinforcement learning.
The following is a list of resources for references in this respect.
Books
1. A text book on Reinforcement Learning: An introduction
2. Synthesis lectures on Algorithms for Reinforcement Learning
Tutorials
1. Deep Reinforcement Learning tutorial from David Silver Google DeepMind ICML¡¯16
2. Reinforcement Learning tutorial from NIPS¡¯05 and another tutorial with slides from ICML¡¯06
3. Tutorial on Q-Learning
4. A brief introduction to Long Short-Term Memory (LSTM)
Papers
1. Reinforcement learning improves behaviour from evaluative feedback
2. Playing Atari with Deep Reinforcement Learning
3. Human-level control through deep reinforcement learning from Google DeepMind
4. Recurrent Models of Visual Attention
5. A Contextual-Bandit Approach to Personalized News Article Recommendation
6. Unbiased Offline Evaluation of Contextual-bandit-based News Article Recommendation Algorithms
7. Contextual Bandits in a Collaborative Environment
8. Collaborative Filtering Bandits
9. Building Machines That Learn and Think Like People
10. Sequence to Sequence Learning with Neural Networks
11. Reinforcement Learning In Continuous Time and Space
12. Technical note - Q-Learning
13. Unsupervised Learning of Video Representations using LSTMs
15. Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
Open source code/tools
1. Human Level Control through Deep Reinforcement Learning
Last updated On Sep. 22 2016