The Dot Product
Let’s say, there is a shop inventory that lists unit prices and quantities for each of the products they carry. If the shop has small-sized storage boxes each worth
,
medium-sized storage boxes each worth
, and
small storage boxes each worth
. Now, the price vector
and the quantity vector
can be written as:
The net-worth of the boxes is calculated as:
This operation of multiplying two entries in pairs of vectors and summing arises often in applications of linear algebra and is also foundational in basic linear algebra theory.
Definition: The dot product of two vectors in
is defined as:
Let us solve an example using S2.
Q: Determine given
and
%use s2
// define vectors x and y
var x = DenseVector(arrayOf(1.0, 2.0, 3.0, 4.0))
var y = DenseVector(arrayOf(4.0, 5.0, 6.0, 7.0))
// P = x.y
val P = x.multiply(y)
println(P)
[4.000000, 10.000000, 18.000000, 28.000000]
Geometrical Connection:
The dot product can also be calculated with the help of the angle
between these two vectors as follows:
Let us find the dot product of the vectors in the figure below:

From the figure, vector can be written as:
Similarly, vector can be written as:
Given,
          Â
            Â
    Â
Note: if and only if the vectors
and
are orthogonal (perpendicular to each other).
Real World Application of Vector Dot Product
In Natural Language Processing, one of the basic ways to compare a finite number of text documents is to use
vectorized word counts. Let’s suppose the documents have a combined total of distinct words, which are arranged in some order. Each document is then associated with a vector of length
whose
entry indicates the number of times the
word occurs in the associated document.
One way to measure similarity between two documents is to take the dot product of the associated
unit vectors: If two documents and
have associated vectors
and
respectively, their similarity is defined by:
Similarity, Â
As defined earlier,
We know that,
Similarly we have, for any two documents
and
.
- Documents with no words in common are associated with orthogonal vectors and thus have
similarity.
- If two documents have similarity
, their associated vectors are scalar multiples of each other, meaning that they have the same words and that the words appear in the same proportions.
Let us try to find the vectorized word count similarity between the following sentences:
A: “Julie likes John more than Kelly likes John”
B: “Joel likes Jonam more than Julie likes John”
Respective word count in A & B:
Word | Count in A | Count in B |
Julie | 1 | 1 |
John | 2 | 1 |
Kelly | 1 | 0 |
Joel | 0 | 1 |
Jonam | 0 | 1 |
likes | 2 | 2 |
more | 1 | 1 |
than | 1 | 1 |
Therefore, the two vectors are:
and
Approximately,
Hence, the similarity between the two given sentences and
,
.
The Transpose
The dot product gives us a compact way to express the formula for an entry of a matrix product: to obtain
the entry of a matrix product
, we dot the
row of
and the
column of
.
However, the matrix product by itself is not quite flexible enough to handle a common use case: suppose
We have two matrices and
which contain tabular data stored in the same format.
For example, suppose that the columns of store the vectorized word counts for a series of emails sent from Kelly, while
stores vectorized word counts for a series of emails sent from Ana. If we want to calculate the similarity of each of Kelly’s emails to each of Ana’s emails, then we want to dot the columns of
but not its rows with the columns of
.
So we define the transpose of a matrix
to be the matrix resulting from switching
‘s rows and columns.
Definition: If is an
matrix, then its transpose
is defined to be the matrix with
rows whose
row is equal to the
column of
, for each
from
to
.Â
If , then
Let us implement the same using S2.
%use s2
// define a matrix
var A = DenseMatrix(arrayOf(
doubleArrayOf(1.0, 2.0, 3.0),
doubleArrayOf(4.0, 5.0, 6.0)))
println(A)
// B = Tranpose of A
val B = A.t()
println("\n\nTranspose of A:\n")
println(B)
2x3 [,1] [,2] [,3] [1,] 1.000000, 2.000000, 3.000000, [2,] 4.000000, 5.000000, 6.000000, Transpose of A:
3x2 [,1] [,2] [1,] 1.000000, 4.000000, [2,] 2.000000, 5.000000, [3,] 3.000000, 6.000000,
Note:Transpose is a linear operator, meaning that , where
is a constant and
&
are matrices.
Matrix Properties
Now that we know what the transpose of a matrix is, let’s learn about various matrix properties and their illustrations in S2.
Symmetric Matrix
A matrix in which its entry will be necessarily equal to its
entry is known as a Symmetric Matrix.
In other words, if is an
matrix satisfying the equation
, we say that
is symmetric.
%use s2
// define a matrix
var A = DenseMatrix(arrayOf(
doubleArrayOf(1.0, 2.0, 3.0),
doubleArrayOf(2.0, 4.0, 5.0),
doubleArrayOf(3.0, 5.0, 6.0)))
println(A)
print("\n")
//prints 'true' if A is symmetric, else prints 'false'
val precision = 1e-15
println(MatrixPropertyUtils.isSymmetric(A,precision))
3x3 [,1] [,2] [,3] [1,] 1.000000, 2.000000, 3.000000, [2,] 2.000000, 4.000000, 5.000000, [3,] 3.000000, 5.000000, 6.000000, true
Skew-Symmetric Matrix
If is an
matrix satisfying the equation
, we say
is Skew-Symmetric.
%use s2
// define a matrix
var A = DenseMatrix(arrayOf(
doubleArrayOf(0.0, -6.0, 4.0),
doubleArrayOf(6.0, 0.0, -5.0),
doubleArrayOf(-4.0, 5.0, 0.0)))
println(A)
print("\n")
//prints 'true' if A is skew-symmetric, else prints 'false'
val precision = 1e-15
println(MatrixPropertyUtils.isSkewSymmetric(A,precision))
3x3 [,1] [,2] [,3] [1,] 0.000000, -6.000000, 4.000000, [2,] 6.000000, 0.000000, -5.000000, [3,] -4.000000, 5.000000, 0.000000, true
Idempotent Matrix
If is an
matrix satisfying the equation
or
, we say
is Idempotent.
%use s2
// define a matrix
var A = DenseMatrix(arrayOf(
doubleArrayOf(2.0, -2.0, -4.0),
doubleArrayOf(-1.0, 3.0, 4.0),
doubleArrayOf(1.0, -2.0, -3.0)))
println(A)
//prints 'true' if A is idempotent, else prints 'false'
val precision = 6.0
println("\n")
println(MatrixPropertyUtils.isIdempotent(A,precision))
3x3 [,1] [,2] [,3] [1,] 2.000000, -2.000000, -4.000000, [2,] -1.000000, 3.000000, 4.000000, [3,] 1.000000, -2.000000, -3.000000, true
Orthogonal Matrix
If is an
matrix satisfying the equation
where
is an Identity Matrix of order
, we say
is Orthogonal.
%use s2
// define a matrix
var A = DenseMatrix(arrayOf(
doubleArrayOf(1.0, 0.0, 0.0),
doubleArrayOf(0.0, 0.0, -1.0),
doubleArrayOf(0.0, -1.0, 0.0)))
println(A)
//prints 'true' if A is orthogonal, else prints 'false'
val precision = 1e-15
println("\n")
println(MatrixPropertyUtils.isOrthogonal(A,precision))
3x3 [,1] [,2] [,3] [1,] 1.000000, 0.000000, 0.000000, [2,] 0.000000, 0.000000, -1.000000, [3,] 0.000000, -1.000000, 0.000000, true