\documentclass{article}

% Packages required to support encoding
\usepackage{ucs}
\usepackage[utf8x]{inputenc}

% Packages required by code


% Packages always used
\usepackage{hyperref}
\usepackage{xspace}
\usepackage[usenames,dvipsnames]{color}
\hypersetup{colorlinks=true,urlcolor=blue}

\input{preamble.tex}


\begin{document} 
Let us begin by introducing basic notation for matrices and vectors.

We'{}ll use $\RR$ to denote the set of real-numbers and $\CC$ to denote the set of complex numbers.

We write the space of all $m \times n$ real-valued matrices as $\RR^{m \times n}$. Each

\begin{displaymath}
\mA \in \RR^{m \times n} \text{ is } 
\bmat{ 
A_{1,1} & \cdots & A_{1,n} \\
\vdots &  & \vdots \\
A_{m,1} & \cdots & A{m,n} 
}
\text{ where }
A_{i,j} \in \RR.
\end{displaymath}
With only a few exceptions, matrices are written as \emph{bold, capital} letters. Matrix elements are written as sub-scripted, \emph{unbold} letters. When clear from context,

\begin{displaymath}
A_{i,j} \text{ is written } A_{ij}
\end{displaymath}
instead, e.g. $A_{11}$ instead of $A_{1,1}$.

An short-hand notation for $\mA \in \RR^{m \times n}$ is

\begin{displaymath}
\mA : n \times n.
\end{displaymath}
\textbf{In class} I'{}ll usually write matrices with just upper-case letters.

We write the set of length-$n$ real-valued vectors as $\RR^{n}$. Each

\begin{displaymath}
\vx \in \RR^{n} \text{ is } \bmat{ x_{1} \\ \vdots \\ x_{n} }
\text{ where } x_i \in \RR.
\end{displaymath}
Vectors are denoted by \emph{lowercase, bold} letters. As with matrices, elements are sub-scripted, \emph{unbold} letters. Sometimes, we'{}ll write vector elements as

\begin{displaymath}
x_i \text{ or } [x]_i \text{ or } x(i).
\end{displaymath}
Usually, this choice is motivated by a particular application. \emph{Throughout the class, vectors are \textbf{column} vectors.}

\textbf{In class} I'{}ll usually write vectors with just lower-case letters. I may try and follow the convention of underlining vectors. We'{}ll see.

\hypertarget{operations_1}{}\section*{{Operations}}\label{operations_1}

\textbf{Transpose} Let $\mA : m \times n$, then

\begin{displaymath}
\mB : n \times m = \mA^T \implies B_{i,j} = A_{j,i}.
\end{displaymath}
\emph{Example} $\mA = \sbmat{ 2 & 3 \\ 1 & 4 \\ 3 & -1 } \quad \mA^T = \sbmat{ 2 & 1 & 3 \\ 3 & 4 & -1 }$

\textbf{Hermitian} Let $\mA \in \CC^{m \times n}$, then

\begin{displaymath}
\mB \in \CC^{n \times m} = \mA^* = \mA^{H} \implies B_{i,j} = \conj{A}_{j,i}.
\end{displaymath}
\emph{Example} $\mA = \sbmat{ 2 & 3 \\ i & 4 \\ & 3 & -i } \quad \mA^* = \sbmat{ 2 & -i & 3 \\ 3 & 4 & i }$

\textbf{Addition} Let $\mA : m \times n$ and $\mB : m \times n$, then

\begin{displaymath}
\mC : m \times n = \mA + \mB \implies C_{i,j} = A_{i,j} + B_{i,j}.
\end{displaymath}
\emph{Example} $\mA = \sbmat{ 2 & 3 \\ 1 & 4 \\ 3 & -1 }, \mB = \sbmat{ 1 & -1 \\ 2 & 3 \\ -1 & 1 }$ $\mA + \mB = \sbmat{3 & -2 \\ 3 & 2 \\ 2 & 0 }$.

\textbf{Scalar Multiplication} Let $\mA : m \times n$ and $\alpha \in \RR$, then

\begin{displaymath}
\mC : m \times n = \alpha \mA + \mB \implies C_{i,j} = \alpha A_{i,j}.
\end{displaymath}
\emph{Example} $\mA = \sbmat{ 2 & 3 \\ 1 & 4 \\ 3 & -1 }, 5 \mA = \sbmat{ 10 & 15 \\ 5 & 20 \\ 15 & -5 }$

\textbf{Matrix Multiplication} Let $\mA : m \times n$ and $\mB : n \times k$, then

\begin{displaymath}
\mC : m \times k = \mA \mB \implies C_{i,j} = \sum_{r=1}^{n} A_{i,r} B_{r,j}.
\end{displaymath}
\textbf{Matrix-vector Multiplication} Let $\mA : m \times n$ and $\vx \in \RR^{n}$, then

\begin{displaymath}
\vc \in \RR^{m} = \mA \vx \implies c_i = \sum_{j=1}^n A_{i,j} x_j.
\end{displaymath}
This operation is just a special case of matrix multiplication that follows from treating $\vx$ and $\vc$ as $n \times 1$ and $m \times 1$ matrices, respectively.

\textbf{Vector addition}, \textbf{Scalar vector multiplication} These are just special cases of matrix addition and scalar matrix multiplication where vectors are viewed as $n \times 1$ matrices.

\hypertarget{partitioning_2}{}\section*{{Partitioning}}\label{partitioning_2}

It is often useful to represent a matrix as a collection of vectors. In this case, we write

\begin{displaymath}
\mA : m \times n = \bmat{ \va_1 & \va_2 & \cdots & \va_n }
\end{displaymath}
where each $\va_j \in \RR^{m}$. \emph{This form corresponds to a partition into columns.}

Alternatively, we may wish to partition a matrix into rows.

\begin{displaymath}
\mA : m \times n  = \bmat{ \vr_1^T \\ \vr_2^T \\ \vdots \\ \vr_m^T }.
\end{displaymath}
Here, each $\vr_i \in \RR^{n}$.

Using the column partitioning:

\begin{displaymath}
\mA \vx = \bmat{ \va_1 & \va_2 & \cdots & \va_n } \bmat{ x_1 \\ x_2 \\ \vdots \\ x_n }
= \sum_j x_j \va_j.
\end{displaymath}
And with the row partitioning:

\begin{displaymath}
\[ \mA \vx = \bmat{ \vr_1^T \\ \vr_2^T \\ \vdots \\ \vr_m^T } \vx
= \bmat{ \vr_1^T \vx \\ \vr_2^T \vx \\ \vdots \\ \vr_m^T \vx }.
\end{displaymath}
Another useful partitioned representation of a matrix is into blocks:

\begin{displaymath}
\mA = \bmat{\mA_{1,1} & \mA_{1,2} \\               \mA_{2,1} & \mA_{2,2} }
\end{displaymath}
or

\begin{displaymath}
\mA = \bmat{\mA_{1,1} & \mA_{1,2} & \mA_{1,3} \\               \mA_{2,1} & \mA_{2,2} & \mA_{2,3} \\
               \mA_{3,1} & \mA_{3,2} & \mA_{3,3} }.
\end{displaymath}
Here, the sizes ``{}just have to work out''{} in the vernacular. Formally, all $\mA_{i,\cdot}$ must have the same number of rows and all $\mA_{\cdot,j}$ must have the same number of columns. This means the diagonal blocks are always square, but the off-diagonal blocks may not be.


\end{document}
