
Quick links: overview  textbooks  grading  announcements  schedule  homework  related courses  faq
Machine learning is the study of techniques that can identify patterns from data to make future predictions. Examples include classification of email as spam or not, predicting ratings of movies based on previous ratings, predicting future oil prices, identifying the species of a bird from its photograph, etc. Machine learning is a core tool in a number of areas such as natural language processing, speech recognition and computer vision.
The course will introduce a number of key concepts, techniques and algorithms. The focus will be on the mathematical foundations rather than the use of software packages as black box. The course requires appropriate mathematical background in probability and statistics, calculus, linear algebra and an ability to program in MATLAB. Some familiarity with MATLAB will be helpful, but students should be able to learn MATLAB during the course. Graduate students from outside computer science with sufficient background are also welcome to take the course (you will need to request an override by filing an override request).
Lec  Date  Topic  Slides  Reading  Homework 
1  Jan 20  Introduction to machine learning  1UP, 4UP  MLaPP 1.11.3, 2  p0 
Basic supervised learning  
2  Jan 22  Decision trees: learning algorithm, inductive bias  1UP, 4UP  Quinlan 1986  hw00 
No class, snow day  
3  Jan 29  Nearest neighbor classifier: geometry, decision boundaries, Bayes error/optimality, biasvariance tradeoff  1UP, 4UP  MLaPP 1.4, 6.16.4  hw01 
4  Feb 03  
5  Feb 05  Perceptron: geometry and convergence  1UP, 4UP  Freund and Schapire 99  hw02 
6  Feb 10  Feature and model selection: feature selection, normalization, crossvalidation, bootstrapping, statistical significance  1UP, 4UP  Guyon and Elisseeff 03  
7  Feb 12  hw03, p1  
No class, following Monday class schedule  
8  Feb 19  Beyond binary classification: multiclass, ranking, collective classification  1UP, 4UP  MLaPP 9.7  hw04 
Advanced supervised learning  
9  Feb 24  Linear models: surrogate loss, regularization, gradients, subgradients, margins, support vector machines  1UP, 4UP  MLaPP 6.5, 7.17.4, 8.18.3, 13  
10  Feb 26  hw05  
11  Mar 03  Probabilistic modeling: density estimation, MLE, priors, MAP, naive Bayes, conditional models, kernel density estimation  1UP, 4UP  MLaPP 2, 3, 5.15.2, 7.17.4  
12  Mar 05  hw06  
13  Mar 10  Neural networks: hidden layers, link functions, representation power, backpropagation, shallow or deep, convolutional neural networks  1UP, 4UP  MLaPP 16.5 NNaSI 6, 7 LeCun et al. 98 

14  Mar 12  hw07  
15  Mar 24  Kernel methods: feature mapping and kernels, kernel methods, representer theorem, computational tradeoffs, efficient kernel algorithms  1UP, 4UP  MLaPP 14  p2, fp 
16  Mar 26  hw08  
17  Mar 31  Ensemble methods: voting, bagging, boosting, random forests  1UP, 4UP  MLaPP 16  
Unsupervised learning  
18  Apr 02  Clustering: flat clustering alogrithms – kmeans, mean shift, spectral clustering; hierarchical clustering algorithms – agglomerative, divisive  1UP, 4UP  MLaPP 25 Quick shift, kmeans++ 
hw09 
19  Apr 07  
20  Apr 09  Dimensionality reduction: PCA, kernel PCA, spectral embedding  1UP, 4UP  MLaPP 12.2, 14.4 Tutorial 
hw10 
21  Apr 14  Expectation maximization: EM algorithm, GMMs, Naive Bayes  1UP, 4UP  MLaPP 11 Notes 

Other topics  
22  Apr 16  Hidden Markov Models: inference and learning  1UP, 4UP  MLaPP 17  hw11 
23  Apr 21  Reinforcement learning: Guest lecture by Kevin Spiteri  1UP, 4UP  
Apr 23  Project presentations  hw12  
Apr 28  Project presentations 
All weekly homeworks are due on the date the homework is posted on the schdule before the class starts via moodle. You're free to use the LaTex source in any way you want, but you'll need mydefs.sty and notes.sty to build them.
Weekly homework