\documentclass[fleqn]{article}
\usepackage{mydefs}
\usepackage{notes}
\usepackage{url}
\begin{document}
\lecture{Machine Learning}{HW06: Linear models}{CS 689, Spring 2015}
% IF YOU ARE USING THIS .TEX FILE AS A TEMPLATE, PLEASE REPLACE
% "CS 689, Spring 2015" WITH YOUR NAME AND UID.
Hand in via moodle at: \url{https://moodle.umass.edu/course/view.php?id=20836}.
Remember that only PDF submissions are accepted. We encourage using
\LaTeX\ to produce your writeups. See \verb+hw00.tex+ for an example
of how to do so. You can make a \verb+.pdf+ out of the \verb+.tex+ by
running ``\verb+pdflatex hw00.tex+''. You'll need mydefs.sty and notes.sty which can be downloaded from the course page.
\bee
\i What values of $\lambda$ in the regularized loss objective (Slide \#4) will lead to overfitting? What values will lead to underfitting?
\i The solution to the least-squares regression problem involves inverting the matrix $(\mathbf{X}^T\mathbf{X}+\lambda \mathbf{I}_D)$ which might not be invertible. Is this actually a problem?
\i Explain why the squared loss is not suitable for binary classification problems.
\i One disadvantage of the squared loss is that it has a tendency to be dominated by outliers -- the overall loss $\sum_n (y_n - \hat{y}_n)^2$, is influenced too much by points that have high $|y_n-\hat{y}_n|$. Suggest a modification to the squared loss that remedies this.
\ene
\end{document}