CMPSCI 689: Machine Learning, Spring 2015

Instructor Subhransu Maji
Schedule Tue/Thurs 1:00pm-2:15pm, CS 142
Office hours Tue 2:15pm-3:15pm, or by appointment
TA Xiaojian Wu
TA office hours Thu 2:30pm-3:30pm Wed 3:30pm to 4:30pm, 220 LGRT
Homework moodle for hw submissions
Computing edlab (has MATLAB)

Quick links: overview | textbooks | grading | announcements | schedule | homework | related courses | faq

Overview

Machine learning is the study of techniques that can identify patterns from data to make future predictions. Examples include classification of email as spam or not, predicting ratings of movies based on previous ratings, predicting future oil prices, identifying the species of a bird from its photograph, etc. Machine learning is a core tool in a number of areas such as natural language processing, speech recognition and computer vision.

The course will introduce a number of key concepts, techniques and algorithms. The focus will be on the mathematical foundations rather than the use of software packages as black box. The course requires appropriate mathematical background in probability and statistics, calculus, linear algebra and an ability to program in MATLAB. Some familiarity with MATLAB will be helpful, but students should be able to learn MATLAB during the course. Graduate students from outside computer science with sufficient background are also welcome to take the course (you will need to request an override by filing an override request).

Textbooks

Grading

Announcements

Schedule (may change over the course of the semester!)

LecDate Topic Slides Reading Homework
1 Jan 20 Introduction to machine learning 1UP, 4UP MLaPP 1.1-1.3, 2 p0
Basic supervised learning
2 Jan 22 Decision trees: learning algorithm, inductive bias 1UP, 4UP Quinlan 1986 hw00
Jan 27 No class, snow day
3 Jan 29 Nearest neighbor classifier: geometry, decision boundaries, Bayes error/optimality, bias-variance tradeoff 1UP, 4UP MLaPP 1.4, 6.1-6.4 hw01
4 Feb 03
5 Feb 05 Perceptron: geometry and convergence 1UP, 4UP Freund and Schapire 99 hw02
6 Feb 10 Feature and model selection: feature selection, normalization, cross-validation, bootstrapping, statistical significance 1UP, 4UP Guyon and Elisseeff 03
7 Feb 12 hw03, p1
Feb 17 No class, following Monday class schedule
8 Feb 19 Beyond binary classification: multi-class, ranking, collective classification 1UP, 4UP MLaPP 9.7 hw04
Advanced supervised learning
9 Feb 24 Linear models: surrogate loss, regularization, gradients, subgradients, margins, support vector machines 1UP, 4UP MLaPP 6.5, 7.1-7.4, 8.1-8.3, 13
10 Feb 26 hw05
11 Mar 03 Probabilistic modeling: density estimation, MLE, priors, MAP, naive Bayes, conditional models, kernel density estimation 1UP, 4UP MLaPP 2, 3, 5.1-5.2, 7.1-7.4
12 Mar 05 hw06
13 Mar 10 Neural networks: hidden layers, link functions, representation power, back-propagation, shallow or deep, convolutional neural networks 1UP, 4UP MLaPP 16.5
NNaSI 6, 7
LeCun et al. 98
14 Mar 12 hw07
15 Mar 24 Kernel methods: feature mapping and kernels, kernel methods, representer theorem, computational tradeoffs, efficient kernel algorithms 1UP, 4UP MLaPP 14 p2, fp
16 Mar 26 hw08
17 Mar 31 Ensemble methods: voting, bagging, boosting, random forests 1UP, 4UP MLaPP 16
Unsupervised learning
18 Apr 02 Clustering: flat clustering alogrithms – k-means, mean shift, spectral clustering; hierarchical clustering algorithms – agglomerative, divisive 1UP, 4UP MLaPP 25
Quick shift, k-means++		 hw09
19 Apr 07
20 Apr 09 Dimensionality reduction: PCA, kernel PCA, spectral embedding 1UP, 4UP MLaPP 12.2, 14.4
Tutorial 		hw10
21 Apr 14 Expectation maximization: EM algorithm, GMMs, Naive Bayes 1UP, 4UP MLaPP 11
Notes
Other topics
22 Apr 16 Hidden Markov Models: inference and learning 1UP, 4UP MLaPP 17 hw11
23 Apr 21 Reinforcement learning: Guest lecture by Kevin Spiteri 1UP, 4UP
Apr 23 Project presentations hw12
Apr 28 Project presentations

Homework

All weekly homeworks are due on the date the homework is posted on the schdule before the class starts via moodle. You're free to use the LaTex source in any way you want, but you'll need mydefs.sty and notes.sty to build them.

Weekly homework Mini-projects Final project

Related courses

Here is a list of similar courses offered at various places:

FAQs

This year a number of machine learning courses are being offered this Spring at UMass CS, each of which has a slightly different focus. For example the 688 focusses on probabilistic graphical models, and 589 focusses on the application side of machine learning. Here are some frequenty asked questions that might help you to choose what is the right course for you (courtesy Ben Marlin).