|
| Lecture | Date | Topic | Suggested Reading | Further Readings | Independent activity | Public domain software | |
|---|---|---|---|---|---|---|---|
| 1 | Fri Sept 11th | Historical overview of neural networks; Introduction to Deep Learning | Chapter 1, Learning Deep Architectures for AI | Hacker's guide to neural networks
Three Classes of Deep Learning Architectures | Set up Theano on your machine, and run the logistic regression example using the MNIST dataset | Deep learning tutorial using Theano Caffe: Deep learning package in C++ Mocha: Deep learning package in Julia | |
| 2 | Fri Sept 18th | Deep architectures: feedforward neural nets, convolutional networks, deep generative networks etc. | Chapter 4, Learning Deep Architectures for AI | Learning representations by back-propagating errors | Implement/download a feedforward neural net learner, and experiment with it on some datasets | Implementation of feedforward networks in Theano | |
| 3 | Fri Sept 25th | Energy-based models; Boltzmann machines; deep belief networks | Chapter 5, Learning Deep Architectures for AI | Fast Learning Algorithm for Deep Belief Networks | Train an RBM or a deep belief network on the MNIST dataset (this requires a relativelu fast PC) | Deep belief networks implemented in Theano | |
| 4 | Fri Oct 2nd | Stacked denoising autoencoders; Linear and nonlinear training regimes; Applications | Reducing the dimensionality of data with neural networks | Marginalized denoising autoencoders for domain adaptation | Implement the marginalized denoising autoencoder and compare with the regular stacked denoising autoencoder in Theano on the Amazon sentiment analysis data set | Implementation of denoising autoencoders in Theano | |
| 5 | Fri Oct 9th | Convolutional neural networks; Applications | Convolutional Neural Networks for Images, Speech, and Time-series | Learning Methods for Generic Object Recognition with Ivariance to Pose Stanford course on CNNs for computer vision | Experiment with CNNs in Theano | Implementation of CNN in Theano | |
| 6 | Fri Oct. 16th | Deep reinforcement learning | Human level control through deep reinforcement learning | Deep Learning for Real-Time Atari Game Play using Offline Monte-Carlo Tree-Search | Install Nathan's version of DQN on your PC | Nathan Sprague's implementation of deep reinforcement learning | |
| 7 | Fri. Oct. 23rd | Project Proposals | |||||
| 8 | Fri Oct 30th | Deep learning with memory: recursive neural nets; long short-term memory models | Long short-term memory networks | Deep sentence embedding using the long short-term memory networks Karpathy's blog on LSTMs | Experiment with LSTMs in Theano | LSTM Networks in Theano RNNs in Torch/Lua | |
| 9 | Nov. 6th | Deep learning and ensemble methods; regularization and dropout | Dropout training as adaptive regularization | Learning with pseudo ensembles | Experiment with various forms of dropout and compare them | ||
| 10 | Fri. Nov 13th | Algorithmic analysis of deep learning; non-convex computation and saddle points | The Loss Surfaces of Multilayer Networks | Provable Bounds for Learning Some Deep Represenations | |||
| 11 | Fri Nov 20th | Frontiers of deep learning | Reinforcement Learning Neural Turing Machines | Neural Turing Machines MIT Technology review article on Neural Turing Machines | |||
| 12 | Fri Dec 4th | Beyond deep learning: overview of Spring 2016 seminar on building a deep mind | |||||
| 13 | Fri Dec. 11th | Final project presentations | Final Project Reports due |