A matrix is a rectangular array of numbers. The size of a matrix is described as the number of rows by the number of columns. In other words, a matrix with $m$ rows and $n$ columns is said to be an $m \times n$ matrix. The entry in the $i^{th}$ row and $j^{th}$ column of a matrix $A$ is referred to as $A$’s $(i, j)^{th}$ entry, and it is denoted by $A_{i,j}$

Matrices are versatile structures with a variety of problem-solving uses. For example,
1. A matrix can be thought of as a list of column vectors, so we can use a matrix to package many column
vectors into a single mathematical object.
2. An $m \times n$ matrix can be thought of as a linear transformation from $\mathbb{R}^n$ to $\mathbb{R}^m$.

In this topic, we are going to develop both of these perspectives and define some operations which facilitate common manipulations that arise while handling matrices.

Matrix Operations

Matrix Addition for two $m \times n$ matrices $A$ and $B$ is the sum $A+B$ that results in another $m \times n$ matrix whose each entry is the sum of the corresponding entries in $A$ and $B$.

If $A = \begin{bmatrix}1 & 2\\4 & 5\end{bmatrix}$ and $B = \begin{bmatrix}3 & 2\\8 & 6\end{bmatrix}$, then compute $A+B$.

				
%use s2
// define the given matrices
var A = DenseMatrix(arrayOf(
doubleArrayOf(1.0, 2.0),
doubleArrayOf(4.0, 5.0)))
var B = DenseMatrix(arrayOf(
doubleArrayOf(3.0, 2.0),
doubleArrayOf(8.0, 6.0)))

// C = A+B
println(C)


2x2
[,1] [,2]
[1,] 4.000000, 4.000000,
[2,] 12.000000, 11.000000,

Scalar Multiplication

The product of a number $c$ and an $m \times n$ matrix $A$ is defined to be the $m \times n$ matrix each of whose entries is $c$ times the corresponding entry of $A$.

Compute the value of $2\begin{bmatrix}1 & 2\\4 & 5\end{bmatrix}$.

				
%use s2
// define matrix A
var A = DenseMatrix(arrayOf(
doubleArrayOf(1.0, 2.0),
doubleArrayOf(4.0, 5.0)))
//define C
val C = 2.0
// M = AC
val M = A.scaled(C)
println(M)


2x2
[,1] [,2]
[1,] 2.000000, 4.000000,
[2,] 8.000000, 10.000000, 

Matrix-Vector Multiplication

Let’s say $A$ is an $m \times n$ matrix and $x$ is a column vector in $\mathbb{R}^n$, then $Ax$ is defined as the linear combination of the columns of $A$ with weights given by the entries of $x$.

For a clear understanding, refer to the example below:

If matrix $A = \begin{bmatrix}1 & 2 & 3\\4 & 5 & 6\end{bmatrix}$ and vector $v=\begin{bmatrix}1\\2\\3\end{bmatrix}$, then $Av = 1\begin{bmatrix}1\\4\end{bmatrix}+2\begin{bmatrix}2\\5\end{bmatrix}+3\begin{bmatrix}3\\6\end{bmatrix} = \begin{bmatrix}14\\32\end{bmatrix}$.

Let us illustrate the same using S2.

				
%use s2
// define matrix A
var A = DenseMatrix(arrayOf(
doubleArrayOf(1.0, 2.0, 3.0),
doubleArrayOf(4.0, 5.0, 6.0)))
// define a vector
var v = DenseVector(arrayOf(1.0, 2.0, 3.0))

// B = Av
val B = A.multiply(v)
println(B)


[14.000000, 32.000000]

Matrix Product

If $A$ is an $m \times n$ matrix and $B$ is an $n \times p$ matrix, then the matrix product of these two matrices $AB$ is an $m \times p$ matrix $C$ whose $k^{th}$ column is defined to be the product of $A$ and the $k^{th}$ column of $B$.

For example, if $A = \begin{bmatrix}1 & 2\\4 & 5\end{bmatrix}$ and $B = \begin{bmatrix}3 & 4\\6 & 7\end{bmatrix}$, then the matrix product of $A, B$ is:

$AB = C = \begin{bmatrix}\begin{bmatrix}1 & 2\\4 & 5\end{bmatrix}\begin{bmatrix}3\\6\end{bmatrix} & \begin{bmatrix}1 & 2\\4 & 5\end{bmatrix}\begin{bmatrix}4\\7\end{bmatrix}\end{bmatrix}=\begin{bmatrix}3\begin{bmatrix}1\\4\end{bmatrix}+6\begin{bmatrix}2\\5\end{bmatrix} & 4\begin{bmatrix}1\\4\end{bmatrix}+7\begin{bmatrix}2\\5\end{bmatrix}\end{bmatrix}$

Therefore, $C = \begin{bmatrix}15 & 18\\42 & 21\end{bmatrix}$

Implementing using S2 is much easier as you can see below.

				
%use s2
// define matrices A &amp; B
var A = DenseMatrix(arrayOf(
doubleArrayOf(1.0, 2.0),
doubleArrayOf(4.0, 5.0)))
var B = DenseMatrix(arrayOf(
doubleArrayOf(3.0, 4.0),
doubleArrayOf(6.0, 7.0)))

// C = AB
val C = A.multiply(B)
println(C)


2x2
[,1] [,2]
[1,] 15.000000, 18.000000,
[2,] 42.000000, 51.000000, 

The Inverse of a Matrix

The inverse of an invertible matrix is another matrix which on multiplication with the given matrix results in an identity matrix.

For a matrix $A$, $A^{-1}$ is its inverse and these two matrices $A$ and $A^{-1}$ satisfy the equation: $AA^{-1} = A^{-1}A = I$ where $I$ denotes the $n \times n$ identity matrix which has ones along the diagonal starting at the top left entry and zeros elsewhere.

Invertible Matrix Theorem

Statement: Suppose that $A$ is an $n \times n$ matrix, then the following are equivalent(that is, for a given matrix they are either all true or all false).

1. The transformation $x \rightarrow Ax$ from $\mathbb{R}^n$ to $\mathbb{R}^n$ is bijective.
2. The range of $A$ is $\mathbb{R}^n$.
3. The null space of $A$ is $\left\{0\right\}$

Proof:

• Let us begin by showing that (2) and (3) are equivalent.
• If the columns of $A$ are linearly dependent, then the range of $A$ is spanned by fewer than $n$ vectors.
• Therefore, if the rank of $A$ is equal to $n$, then the columns of $A$ are linearly independent.
• This implies that a linear combination of the columns is equal to the zero vector only if the weights are all zero. In other words, the only solution of the equation $Ax = 0$ is the zero vector.
• In other words, the null space of $A$ is $\left\{0\right\}$.
• Conversely, if the null space of $A$ is $\left\{0\right\}$, then the columns of $A$ are linearly independent, and the rank of $A$ is therefore equal to $n$.
• By definition of bijectivity, (2) and (3) together imply (1), and (1) implies (2) and (3). Therefore, the
three given statements are equivalent.

Computing inverse of an invertible matrix using S2:

				
%use s2

// Create a matrix
val A = DenseMatrix(arrayOf(
doubleArrayOf(5.0, 4.0, 4.0, 1.0, 5.0, 4.0, 2.0, 4.0, 1.0, 1.0),
doubleArrayOf(4.0, 5.0, 2.0, 2.0, 1.0, 2.0, 4.0, 5.0, 5.0, 2.0),
doubleArrayOf(5.0, 5.0, 3.0, 3.0, 5.0, 2.0, 3.0, 4.0, 1.0, 3.0),
doubleArrayOf(1.0, 2.0, 5.0, 5.0, 3.0, 1.0, 4.0, 3.0, 3.0, 3.0),
doubleArrayOf(2.0, 2.0, 4.0, 2.0, 3.0, 1.0, 3.0, 5.0, 4.0, 4.0),
doubleArrayOf(5.0, 4.0, 5.0, 1.0, 1.0, 3.0, 2.0, 3.0, 3.0, 4.0),
doubleArrayOf(3.0, 4.0, 4.0, 3.0, 4.0, 3.0, 2.0, 5.0, 5.0, 5.0),
doubleArrayOf(3.0, 4.0, 3.0, 3.0, 2.0, 1.0, 4.0, 2.0, 2.0, 1.0),
doubleArrayOf(4.0, 1.0, 1.0, 1.0, 1.0, 4.0, 4.0, 2.0, 2.0, 1.0),
doubleArrayOf(1.0, 5.0, 5.0, 5.0, 1.0, 1.0, 2.0, 4.0, 1.0, 4.0)))

// Compute the inverse
val Ainv = Inverse(A)
println("inverse: $Ainv\n") //verification val I = A.multiply(Ainv) // A*A_inverse = I, the identity matrix println("A*Ainv = I:$I\n")
val det: Double = MatrixMeasure.det(I)
println("determinant of I: \$det")


inverse: 10x10
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,] -0.053795, 0.540354, 0.510748, 0.754387, -0.463907, 0.379793, -0.429973, -0.927418, -0.199307, -0.302318,
[2,] 0.001053, -0.302488, -0.254928, -0.616193, 0.184844, -0.173248, 0.320145, 0.765180, 0.035960, 0.192260,
[3,] 0.147015, -0.039141, -0.159575, 0.114186, 0.055723, 0.143278, -0.125604, 0.033857, -0.110351, -0.006015,
[4,] -0.038542, 0.511186, 0.375881, 0.827975, -0.618041, 0.145380, -0.229813, -0.892096, -0.138295, -0.131325,
[5,] 0.043452, -0.227621, -0.015788, -0.167973, 0.124814, -0.146489, 0.205971, 0.313659, -0.016667, -0.069269,
[6,] 0.105785, -0.243783, -0.286237, -0.295539, 0.006206, -0.156237, 0.295182, 0.273379, 0.264409, 0.178383,
[7,] -0.086773, -0.457569, -0.250290, -0.616079, 0.523478, -0.228678, 0.128076, 0.843330, 0.287830, 0.162570,
[8,] 0.124988, 0.342720, 0.122915, 0.217325, 0.045236, -0.040330, -0.304115, -0.571875, -0.039731, 0.070352,
[9,] -0.012258, 0.205563, -0.049505, 0.243281, -0.199434, 0.080622, 0.113154, -0.106082, -0.144272, -0.205092,
[10,] -0.239306, -0.394510, 0.010704, -0.450285, 0.297947, -0.029015, 0.238082, 0.378149, 0.185634, 0.129286,

A*Ainv = I: 10x10
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,] 1.000000, -0.000000, 0.000000, -0.000000, -0.000000, 0.000000, 0.000000, -0.000000, 0.000000, 0.000000,
[2,] 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 0.000000, -0.000000, -0.000000, 0.000000, 0.000000,
[3,] 0.000000, 0.000000, 1.000000, 0.000000, -0.000000, 0.000000, 0.000000, -0.000000, 0.000000, 0.000000,
[4,] -0.000000, -0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, -0.000000,
[5,] 0.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, -0.000000, 0.000000,
[6,] 0.000000, -0.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, -0.000000, 0.000000, 0.000000,
[7,] 0.000000, -0.000000, 0.000000, 0.000000, -0.000000, -0.000000, 1.000000, -0.000000, 0.000000, 0.000000,
[8,] 0.000000, -0.000000, 0.000000, -0.000000, 0.000000, -0.000000, 0.000000, 1.000000, 0.000000, 0.000000,
[9,] 0.000000, -0.000000, 0.000000, -0.000000, 0.000000, -0.000000, -0.000000, 0.000000, 1.000000, 0.000000,
[10,] 0.000000, -0.000000, 0.000000, 0.000000, 0.000000, 0.000000, -0.000000, -0.000000, -0.000000, 1.000000,

determinant of I: 1.0000000000000016