The Dot Product

Let’s say, there is a shop inventory that lists unit prices and quantities for each of the products they carry. If the shop has small-sized storage boxes each worth , medium-sized storage boxes each worth , and small storage boxes each worth . Now, the price vector and the quantity vector can be written as:

The net-worth of the boxes is calculated as:

This operation of multiplying two entries in pairs of vectors and summing arises often in applications of linear algebra and is also foundational in basic linear algebra theory.

**Definition:** The dot product of two vectors in is defined as:

Let us solve an example using S2.

Q: Determine given and

` ````
```%use s2
// define vectors x & y
var x = DenseVector(arrayOf(1.0, 2.0, 3.0, 4.0))
var y = DenseVector(arrayOf(4.0, 5.0, 6.0, 7.0))
// P = x.y
val P = x.multiply(y)
println(P)

[4.000000, 10.000000, 18.000000, 28.000000]

**Geometrical Connection:**

The dot product can also be calculated with the help of the angle between these two vectors as follows:

Let us find the dot product of the vectors in the figure below:

From the figure, vector can be written as:

Similarly, vector can be written as:

Given,

* Note:* if and only if the vectors and are orthogonal (perpendicular to each other).

**Real World Application of Vector Dot Product**

In Natural Language Processing, one of the basic ways to compare a finite number of text documents is to use

vectorized word counts. Let’s suppose the documents have a combined total of distinct words, which are arranged in some order. Each document is then associated with a vector of length whose entry indicates the number of times the word occurs in the associated document.

One way to measure similarity between two documents is to take the dot product of the associated

unit vectors: If two documents and have associated vectors and respectively, their similarity is defined by:

Similarity,

As defined earlier,

We know that,

Similarly we have, for any two documents and .

- Documents with no words in common are associated with orthogonal vectors and thus have similarity.
- If two documents have similarity , their associated vectors are scalar multiples of each other, meaning that they have the same words and that the words appear in the same proportions.

Let us try to find the vectorized word count similarity between the following sentences:

A: “Julie likes John more than Kelly likes John”

B: “Joel likes Jonam more than Julie likes John”

Respective word count in A & B:

Word | Count in A | Count in B |

Julie | 1 | 1 |

John | 2 | 1 |

Kelly | 1 | 0 |

Joel | 0 | 1 |

Jonam | 0 | 1 |

likes | 2 | 2 |

more | 1 | 1 |

than | 1 | 1 |

Therefore, the two vectors are:

and

Approximately,

Hence, the similarity between the two given sentences and , .

The Transpose

The dot product gives us a compact way to express the formula for an entry of a matrix product: to obtain

the entry of a matrix product , we dot the row of and the column of .

However, the matrix product by itself is not quite flexible enough to handle a common use case: suppose

we have two matrices and which contain tabular data stored in the same format.

For example, suppose that the columns of store the vectorized word counts for a series of emails sent from Kelly, while stores vectorized word counts for a series of emails sent from Ana. If we want to calculate the similarity of each of Kelly’s email to each of Ana’s emails, then we want to dot the columns of but not its rows with the columns of .

So we define the transpose of a matrix to be the matrix resulting from switching ‘s rows and columns.

**Definition: **If is an matrix, then its transpose is defined to be the matrix with rows whose row is equal to the column of , for each from to .

If , then

Let us implement the same using S2.

` ````
```%use s2
// define a matrix
var A = DenseMatrix(arrayOf(
doubleArrayOf(1.0, 2.0, 3.0),
doubleArrayOf(4.0, 5.0, 6.0)))
println(A)
// B = Tranpose of A
val B = A.t()
println("\n\nTranspose of A:\n")
println(B)

2x3 [,1] [,2] [,3] [1,] 1.000000, 2.000000, 3.000000, [2,] 4.000000, 5.000000, 6.000000, Transpose of A:

3x2 [,1] [,2] [1,] 1.000000, 4.000000, [2,] 2.000000, 5.000000, [3,] 3.000000, 6.000000,

**Note:**Transpose is a linear operator, meaning that , where is a constant and & are matrices.

Matrix Properties

Now that we know what the transpose of a matrix is, let’s learn about various matrix properties and their illustrations in S2.

**Symmetric Matrix**

A matrix in which its entry will be necessarily equal to its entry is known as a Symmetric Matrix.

In other words, if is an matrix satisfying the equation , we say that is symmetric.

` ````
```%use s2
// define a matrix
var A = DenseMatrix(arrayOf(
doubleArrayOf(1.0, 2.0, 3.0),
doubleArrayOf(2.0, 4.0, 5.0),
doubleArrayOf(3.0, 5.0, 6.0)))
println(A)
print("\n")
//prints 'true' if A is symmetric, else prints 'false'
val precision = 1e-15
println(MatrixPropertyUtils.isSymmetric(A,precision))

3x3 [,1] [,2] [,3] [1,] 1.000000, 2.000000, 3.000000, [2,] 2.000000, 4.000000, 5.000000, [3,] 3.000000, 5.000000, 6.000000, true

**Skew-Symmetric Matrix**

If is an matrix satisfying the equation , we say is Skew-Symmetric.

` ````
```%use s2
// define a matrix
var A = DenseMatrix(arrayOf(
doubleArrayOf(0.0, -6.0, 4.0),
doubleArrayOf(6.0, 0.0, -5.0),
doubleArrayOf(-4.0, 5.0, 0.0)))
println(A)
print("\n")
//prints 'true' if A is skew-symmetric, else prints 'false'
val precision = 1e-15
println(MatrixPropertyUtils.isSkewSymmetric(A,precision))

3x3 [,1] [,2] [,3] [1,] 0.000000, -6.000000, 4.000000, [2,] 6.000000, 0.000000, -5.000000, [3,] -4.000000, 5.000000, 0.000000, true

**Idempotent Matrix**

If is an matrix satisfying the equation or , we say is Idempotent.

` ````
```%use s2
// define a matrix
var A = DenseMatrix(arrayOf(
doubleArrayOf(2.0, -2.0, -4.0),
doubleArrayOf(-1.0, 3.0, 4.0),
doubleArrayOf(1.0, -2.0, -3.0)))
println(A)
//prints 'true' if A is idempotent, else prints 'false'
val precision = 6.0
println("\n")
println(MatrixPropertyUtils.isIdempotent(A,precision))

3x3 [,1] [,2] [,3] [1,] 2.000000, -2.000000, -4.000000, [2,] -1.000000, 3.000000, 4.000000, [3,] 1.000000, -2.000000, -3.000000, true

**Orthogonal Matrix**

If is an matrix satisfying the equation where is an Identity Matrix of order , we say is Orthogonal.

` ````
```%use s2
// define a matrix
var A = DenseMatrix(arrayOf(
doubleArrayOf(1.0, 0.0, 0.0),
doubleArrayOf(0.0, 0.0, -1.0),
doubleArrayOf(0.0, -1.0, 0.0)))
println(A)
//prints 'true' if A is orthogonal, else prints 'false'
val precision = 1e-15
println("\n")
println(MatrixPropertyUtils.isOrthogonal(A,precision))

3x3 [,1] [,2] [,3] [1,] 1.000000, 0.000000, 0.000000, [2,] 0.000000, 0.000000, -1.000000, [3,] 0.000000, -1.000000, 0.000000, true