\documentclass[]{article}
\usepackage{lmodern}
\usepackage{amssymb,amsmath}
\usepackage{ifxetex,ifluatex}
\usepackage{fixltx2e} % provides \textsubscript
\ifnum 0\ifxetex 1\fi\ifluatex 1\fi=0 % if pdftex
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\else % if luatex or xelatex
\ifxetex
\usepackage{mathspec}
\makeatletter % undo the wrong changes made by mathspec
\let\RequirePackage\original@RequirePackage
\let\usepackage\RequirePackage
\makeatother
\else
\usepackage{fontspec}
\fi
\defaultfontfeatures{Ligatures=TeX,Scale=MatchLowercase}
\fi
% use upquote if available, for straight quotes in verbatim environments
\IfFileExists{upquote.sty}{\usepackage{upquote}}{}
% use microtype if available
\IfFileExists{microtype.sty}{%
\usepackage{microtype}
\UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts
}{}
\usepackage{hyperref}
\hypersetup{unicode=true,
pdftitle={Homework 7},
pdfborder={0 0 0},
breaklinks=true}
\urlstyle{same} % don't use monospace font for urls
\IfFileExists{parskip.sty}{%
\usepackage{parskip}
}{% else
\setlength{\parindent}{0pt}
\setlength{\parskip}{6pt plus 2pt minus 1pt}
}
\setlength{\emergencystretch}{3em} % prevent overfull lines
\providecommand{\tightlist}{%
\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}
\setcounter{secnumdepth}{0}
% Redefines (sub)paragraphs to behave more like sections
\ifx\paragraph\undefined\else
\let\oldparagraph\paragraph
\renewcommand{\paragraph}[1]{\oldparagraph{#1}\mbox{}}
\fi
\ifx\subparagraph\undefined\else
\let\oldsubparagraph\subparagraph
\renewcommand{\subparagraph}[1]{\oldsubparagraph{#1}\mbox{}}
\fi
\title{Homework 7}
\input{preamble.tex}
\title{Homework}
\begin{document}
\maketitle
\section{Homework 7}\label{homework-7}
Please answer the following questions in complete sentences in a clearly
prepared manuscript and submit the solution by the due date on
Blackboard (around Sunday, October 28th, 2018.)
Remember that this is a graduate class. There may be elements of the
problem statements that require you to fill in appropriate assumptions.
You are also responsible for determining what evidence to include. An
answer alone is rarely sufficient, but neither is an overly verbose
description required. Use your judgement to focus your discussion on the
most interesting pieces. The answer to ``should I include `something' in
my solution?'' will almost always be: Yes, if you think it helps support
your answer.
\subsection{Problem 0: Homework
checklist}\label{problem-0-homework-checklist}
\begin{itemize}
\item
Please identify anyone, whether or not they are in the class, with
whom you discussed your homework. This problem is worth 1 point, but
on a multiplicative scale.
\item
Make sure you have included your source-code and prepared your
solution according to the most recent Piazza note on homework
submissions.
\end{itemize}
\subsection{Problem 1: Condition
numbers}\label{problem-1-condition-numbers}
Consider the following computations. Discuss if they are
well-conditioned or ill-conditioned. If the answer depends on the types
of input, please provide some rough guidance. (e.g.~for subtraction,
it's ill-conditioned if the numbers are close by)
\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
The mean of a dataset \(\bar{x} = \frac{1}{N} \sum_{i=1}^N x_i\).
\item
The sample variance
\(V = \frac{1}{N-1} \sum_{i=1}^N (x_i - \frac{1}{N} \sum_{i=1}^N x_i)^2\).
\item
A matrix vector product \(\vy = \mA \vx\).
\item
Evaluating a neural network layer \(\vy = f(\mW^T \vx)\) where the
elements are \(y_i = f(w_i^T x)\) and \(f\) the soft-plus function
\(\log(1+e^x)\).
\end{enumerate}
\subsection{Problem 2: Experience with the
SVD}\label{problem-2-experience-with-the-svd}
Produce the analytic SVDs of the following matrices. (That is, no
numerical approximations, but feel free to let Julia give you a good
guess!). It's important to think about these questions because I may
give similar questions on a midterm or final and you'll be expected to
remember these. It's also handy to think about how to construct these,
even though we haven't seen any algorithms yet. You should be able to
work them all out directly from the definition.
\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
\(\bmat{0 & 3 \\ 0 & 0}\)
\item
\(\bmat{5 & 0 \\ 2 & 0}\)
\item
\(\bmat{5 & -5 \\ 2 & -2 \\ 0 & 0}\)
\item
\(\bmat{2 & 0 \\ 0 & -1}\)
\end{enumerate}
\subsection{Problem 3: Eigenfaces}\label{problem-3-eigenfaces}
For this problem, we'll be using the same face data from Yale that we
used on Homework 6. This is on our website:
\begin{verbatim}
*
*
\end{verbatim}
And this can be read into Julia via
\begin{verbatim}
A = readdlm("Yale_64.csv",',')
labels = readdlm("Yale_64_ids.csv")
\end{verbatim}
More about the data is here:
\url{http://www.cad.zju.edu.cn/home/dengcai/Data/FaceData.html} This was
converted from the Matlab file into the CSV files.
\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\setcounter{enumi}{1}
\item
Describe a scheme to center the vector associated with each face
picture. That is, if \(\vf\) is the vector associated with a face,
then explain how to add a scalar to \(\vf\) such that
\(\vx = \vf + \gamma \ve\) and the sum of \(\vx\) is zero.
\item
Describe and implement a matrix operation to center a matrix of all
faces.
\item
Compute the singular value decomposition of the matrix of centered
images as a ``pixels \(\times\) centered faces'' matrix.\\
What is the largest singular value?
\item
Plot the leading singular vector \(\vu_1\) as a \(32\times 32\) pixel
image. What do you see?
\item
If you're curious about this, this website has more info
\url{http://en.wikipedia.org/wiki/Eigenface}. (Except we are doing
eigendigits!) You should also look at the subdominant eigenvectors as
well.
\end{enumerate}
\subsection{Problem 4: Matrix Analysis}\label{problem-4-matrix-analysis}
Suppose that \(\mA\) is a symmetric positive definite matrix with \(1\)
on the diagonal. Address -- through whatever means you want, but make
sure you document anything you look at or work through -- the following
question. Does the convergence rate of Gauss-Seidel change with the
condition number? If so, how?
\end{document}