\documentclass{article}

% Packages required to support encoding
\usepackage{ucs}
\usepackage[utf8x]{inputenc}

% Packages required by code


% Packages always used
\usepackage{hyperref}
\usepackage{xspace}
\usepackage[usenames,dvipsnames]{color}
\hypersetup{colorlinks=true,urlcolor=blue}

\input{preamble.tex}


\begin{document} 
\hypertarget{lecture_1__norms_linear_systems_and_eigenvalues_1}{}\section*{{Lecture 1 - Norms, Linear Systems, and Eigenvalues}}\label{lecture_1__norms_linear_systems_and_eigenvalues_1}

\hypertarget{outline_2}{}\subsection*{{Outline}}\label{outline_2}

In this lecture, we'{}ll treat a few prerequisites from dense linear algebra.

\begin{enumerate}%
\item Norms
\item Linear systems
\item Eigenvalue problems

\end{enumerate}
After the lecture, you should be able to use these concepts to solve problems.

All of these notes are based Golub and van Loan, sections 2.2 and 2.3.

\hypertarget{norms_3}{}\subsection*{{Norms}}\label{norms_3}

\hypertarget{vector_norms_4}{}\subsubsection*{{Vector norms}}\label{vector_norms_4}

Let $\vx, \vy \in \RR^{n}$, and $\alpha \in \RR$. A function $f$ that satisfies the following three properties is called a \emph{vector norm}:

\begin{enumerate}%
\item $f(\vx) \ge 0$ and $f(\vx) = 0$ only when $\vx = 0$.
\item $f(\vx + \vy) \le f(\vx) + f(\vy)$
\item $f(\alpha \vx) = |\alpha| f(\vx)$.

\end{enumerate}
In such cases, we write

\begin{displaymath}
f(\vx) = \normof{\vx}.
\end{displaymath}
The most common vector norm is, by far, the \emph{Euclidean norm}, also called the \emph{2 norm}:

\begin{displaymath}
\normof{\vx} = \normof{\vx}_2 = \sqrt{\sum_{i=1}^n |x_i|^2}.
\end{displaymath}
Usually, this is the norm that people refer to when they aren'{}t specific about another choice.

A more general norm is the $p$-norm ($p > 1$):

\begin{displaymath}
\normof{\vx}_p = \left( \sum_{i=1}^n |x_i|^p \right)^{1/p}.
\end{displaymath}
This becomes the Euclidean norm when $p=2$, hence, when it is also called the 2-norm. There are two other common choices:

\begin{displaymath}
p = 1 \quad : \quad \normof{\vx}_1 = \sum_{i=1}^n |x_i|
\end{displaymath}
\begin{displaymath}
p = \infty \quad : \quad \normof{\vx}_{\infty} = \max_{1 \le i \le n} |x_i|
\end{displaymath}
\textbf{THEOREM (Equivalence of Norms)} Let $\normof{\vx}_a$ and $\normof{\vx}_b$ be any two vector norms on $\RR^n$. Then there exist fixed constants $c_1$ and $c_2$ such that

\begin{displaymath}
c_1 \normof{\vx}_a \le \normof{\vx}_b \le c_2 \normof{\vx}_a,
\end{displaymath}
which hold for any $\vx \in \RR^{n}$.

\emph{Example}

\begin{displaymath}
\normof{\vx}_2 \le \normof{\vx}_1 \le \sqrt{n} \normof{\vx}_2
\end{displaymath}
\begin{displaymath}
\normof{\vx}_{\infty} \le \normof{\vx}_2 \le \sqrt{n} \normof{\vx}_{\infty}
\end{displaymath}
\begin{displaymath}
\normof{\vx}_{\infty} \le \normof{\vx}_1 \le n \normof{\vx}_{\infty}
\end{displaymath}
\textbf{Vector convergence} The importance of this theorem is given by the following definition. Let $\vx\itn{1}, \vx\itn{2}, \ldots$ be a sequence of vectors. We say that $\vx\itn{k}$ \emph{converges to} $\vx$ if

\begin{displaymath}
\lim_{k\to\infty} \normof{\vx\itn{k} - \vx} = 0.
\end{displaymath}
Because of the equivalcence of norms, we can show this result for \emph{any vector norm}. For the forthcoming problems with PageRank, showing such results with the 1-norm will be especially nice.

\begin{quote}%
\emph{A small digression} \newline Many people call these norms the $\ell_1, \ell_2,$ and $\ell_{\infty}$ norms, or even the $L_1, L_2$, and $L_{\infty}$ norms. In my view, these are misnomers, although, they aren'{}t incorrect. Usually the $\ell_1$ and $L_1$ norms apply to sequences and functions, respectively: \newline $\ell_1 : \sum_{i=1}^\infty |x_i|$ \newline $L_1 : \int_{D} |f| \, d\mu$ (over an appropriately measureable space.) So I prefer 1-norm, 2-norm, and $\infty$-norm to the ``{}L''{} versions.


\end{quote}
\hypertarget{matrix_norms_5}{}\subsubsection*{{Matrix norms}}\label{matrix_norms_5}

Let $\mA \in \RR^{m \times n}$ be a matrix. Then any function $f$ where

\begin{enumerate}%
\item $f(\mA) \ge 0$ and $f(\mA) = 0$ only when $\mA = 0$


\item $f(\mA + \mB) \le f(\mA) + f(\mB)$


\item $f(\alpha \mA) = |\alpha| f(\mA)$ is a matrix norm and we write:

\begin{displaymath}
\normof{\mA} \text{ for } f(\mA).
\end{displaymath}


\end{enumerate}
A frequently used matrix norm is the Frobenius norm

\begin{displaymath}
\normof{\mA}_F = \sqrt{\sum_{i=1}^m \sum_{j=1}^n |A_{i,j}|^2 }
\end{displaymath}
Just like

\hypertarget{linear_systems_6}{}\subsection*{{Linear systems}}\label{linear_systems_6}

\hypertarget{a_problem_from__7}{}\subsubsection*{{A problem from \url{http://www.raymondcheong.com/rankings/background.html}}}\label{a_problem_from__7}

\begin{quote}%
A team'{}s ASM depends on the strength of the opponents. Suppose we had a measure of each team'{}s strength, $\vr$ (i.e. the ranking), such that $r_i - r_j$ is the expected score margin when team $i$ plays team $j$. For $\vr$ to be consistent with the available data, then the average discrepancy between the actual and expected score margin should be zero. In other words, for each team $i$, $\vr$ ought to satisfy the equation:


\end{quote}
\begin{displaymath}
\sum_{j \in Games(i)} (r_i - r_j) - (S_i - S_j) = 0
\end{displaymath}
\hypertarget{exercises_8}{}\subsection*{{Exercises}}\label{exercises_8}

These are not required.

\begin{enumerate}%
\item Give an example showing that the $p$-norm isn'{}t a norm when $p < 1$.


\item Show that $\lim_{p \to \infty} \normof{\vx}_p = \normof{x}_{\infty}$ as defined above.


\item Show that the $F_{\infty}$ norm is not submultiplicative via an example.



\end{enumerate}

\end{document}
