\documentclass[fleqn]{article}

\usepackage{mydefs}
\usepackage{notes}
\usepackage{url}

\begin{document}
\lecture{Machine Learning}{HW06: Linear models}{CS 689, Spring 2015}

% IF YOU ARE USING THIS .TEX FILE AS A TEMPLATE, PLEASE REPLACE
% "CS 689, Spring 2015" WITH YOUR NAME AND UID.

Hand in via moodle at: \url{https://moodle.umass.edu/course/view.php?id=20836}.
Remember that only PDF submissions are accepted.  We encourage using
\LaTeX\ to produce your writeups.  See \verb+hw00.tex+ for an example
of how to do so.  You can make a \verb+.pdf+ out of the \verb+.tex+ by
running ``\verb+pdflatex hw00.tex+''. You'll need mydefs.sty and notes.sty which can be downloaded from the course page.

\bee
\i What values of $\lambda$ in the regularized loss objective (Slide \#4) will lead to overfitting? What values will lead to underfitting?

\i The solution to the least-squares regression problem involves inverting the matrix $(\mathbf{X}^T\mathbf{X}+\lambda \mathbf{I}_D)$ which might not be invertible. Is this actually a problem?


\i Explain why the squared loss is not suitable for binary classification problems.

\i One disadvantage of the squared loss is that it has a tendency to be dominated by outliers -- the overall loss $\sum_n (y_n - \hat{y}_n)^2$,  is influenced too much by points that have high $|y_n-\hat{y}_n|$. Suggest a modification to the squared loss that remedies this.


\ene
\end{document}
