Chapter 5: Matrices

Section 5.7 Matrices

Matrices Operation

Terminology

Data about many kinds of problems can often be represented using a rectangular arrangement of values; such an arrangement is called a matrix . Thus:
\[ A=\begin{pmatrix} 1 & 0 & 4 \\ 3 & -6 & 8 \\ \end{pmatrix} \] is a matrix with two rows and three columns. The dimensions of the matrix are the number of rows and columns; here A is a $ 2 \times 3 $ matrix.
Elements of a matrix A are denoted by $ a_{ij} $, where $i$ is the row number of the element in the matrix and $j$ is the column number. In the example matrix A, $a_{23} = 8$ because 8 is the element in row 2, column 3, of A.

Example 1:
Average temperatures in three different cities for each month can be neatly summarized in a $ 3 \times 12$ matrix. Here we interpret the 3 rows as the 3 cities and the 12 columns as the 12 months Januray-December. The average temperature in the third city in April, $a_{34}$,is 67.
$$ A=\begin{pmatrix} 23 & 26 & 38 & 47 & 58 & 71 & 78 & 77 & 69 & 55 & 39 & 33 \\ 14 & 21 & 33 & 38 & 44 & 57 & 61 & 59 & 49 & 38 & 25 & 21 \\ 35 & 46 & 54 & 67 & 78 & 86 & 91 & 94 & 89 & 75 & 62 & 51 \\ \end{pmatrix}$$

Square Matrices

We will often be interested in square matrices, in which the number of rows equals the number of columns . If A is an $n \times n$ square matrix, then the elements $ a_{11}, a_{22}, \dots, a_{nn}$ form the main diagonal of the matrix.
$$ A=\begin{pmatrix} 1 & 3 & 6 \\ 2 & 0 & 4 \\ -4 & 5 & 1 \\ \end{pmatrix}$$

Symmetric Matrix

A symmetric matrix is a square matrix that is equal to its transpose. In a symmetric matrix, $ a_{ij} = a_{ji} $.
$$ A=\begin{pmatrix} 1 & 5 & 7 \\ 5 & 0 & 2 \\ 7 & 2 & 6 \\ \end{pmatrix}$$ The upper triangular part (the portion above the main diagonal) is a reflection of the lower triangular part.

Identity Matrix

The identity matrix of size $n$ is the $n \times n$ square matrix with ones on the main diagonal and zeros elsewhere. It is denoted by I. If we multiply I times any $n \times n$ matrix A, we get A as the result. The identity matrix behaves similarly to the number 1 in regular arithmetic. The equation: $$ I·A = A·I = A$$ $$ A=\begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{pmatrix}$$

Applications of Identity Matrix

An identity matrix is used to find the inverse of a matrix.
An identity matrix is used to find the eigenvalues and eigenvectors.

Matrix Inverse

An $n \times n$ matrix A is invertible if there exists an $n \times n$ matrix B such that
$$A·B = B·A = I$$ In this case, B is called the inverse of A, denoted by $A^{-1}$.

Applications of Inverse Matrix

An inverse matrix is used to solve a system of linear equations.
Helps in computing eigenvalues and eigenvectors.

Matrix Operations

We can define arithmetic operations on matrices whose entries are numerical. These operations make matrices interesting objects to study in their own right, but they also make matrices more useful for certain tasks such as solving systems of equations.

Scalar Multiplication

Scalar multiplication , calls for multiplying each entry of a matrix by a fixed single number called a scalar. The result is a matrix with the same dimensions as the orginial matrix.
$$ A=\begin{pmatrix} 1 & 4 & 5 \\ 6 & -3 & 2 \\ \end{pmatrix}$$ by the scalar $r = 3$:
$$ \begin{aligned} 3A & = 3\begin{pmatrix} 1 & 4 & 5 \\ 6 & -3 & 2 \\ \end{pmatrix} \\ & = \begin{pmatrix} 1*3 & 4*3 & 5*3 \\ 6*3 & -3*3 & 2*3 \\ \end{pmatrix} \\ & = \begin{pmatrix} 3 & 12 & 15 \\ 18 & -9 & 6 \\ \end{pmatrix} \end{aligned} $$

Matrix Addition

Addition of two matrices A and B is defined only when A and B have the same dimensions; then it is simply a mater of adding the corresponding elements. Formally, if A and B are both $ n \times m $ matrices, then C = A + B is an $ n \times m$ matrix with entries: $ c_{ij} = a_{ij} + b_{ij}$.

$$ A=\begin{pmatrix} 1 & 3 & 6 \\ 2 & 0 & 4 \\ -4 & 5 & 1 \\ \end{pmatrix} \quad \quad B=\begin{pmatrix} 0 & -2 & 8 \\ 1 & 5 & 2 \\ 2 & 3 & 3 \\ \end{pmatrix} $$ The matrix $A + B$ is: $$ \begin{aligned} A + B & = \begin{pmatrix} 1 & 3 & 6 \\ 2 & 0 & 4 \\ -4 & 5 & 1 \end{pmatrix} + \begin{pmatrix} 0 & -2 & 8 \\ 1 & 5 & 2 \\ 2 & 3 & 3 \end{pmatrix} \\ & = \begin{pmatrix} 1+0 & 3+(-2) & 6+8 \\ 2+1 & 0+5 & 4+2 \\ -4+2 & 5+3 & 1+3 \end{pmatrix} \\ & = \begin{pmatrix} 1 & 1 & 14 \\ 3 & 5 & 6 \\ -2 & 8 & 4 \end{pmatrix} \end{aligned} $$

Matrix Subtraction

Subtraction of matrices is defined by $A - B = A + (-1)B$.

Matrix Multiplication

To compute A times B, $A · B$, the number of columns in A must equal the number of rows in B. Thus we can compute $A·B$ if A is an $n \times m$ matrix and B is an $m \times p$ matrix. The result is an $ n \times p$ matrix. An entry in row $i$, column $j$ of $A · B$ is obtained by multiplying all the elements in row $i$ of A by the corresponding elements in column $j$ of B and adding the results. Formally, $A · B = C$, where $ c_{ij} = \sum\limits_{k=1}^ma_{ik}b_{kj}$
$$ A=\begin{pmatrix} 2 & 4 & 3 \\ 4 & -1 & 2 \\ \end{pmatrix} \quad \quad B=\begin{pmatrix} 5 & 3 \\ 2 & 2 \\ 6 & 5 \\ \end{pmatrix} $$ A is a $ 2 \times 3$ matrix and B is a $3 \times 2$ matrix, so the prodcut $A · B$ exists and is a $2 \times 2$ matrix C. To find element $c_{11}$, we multiply corresponding elements of row 1 of A and column 1 of B and add the results.

The complete product is:
$$ \begin{aligned} A·B & = \begin{pmatrix} 2 & 4 & 3 \\ 4 & -1 & 2 \\ \end{pmatrix} \begin{pmatrix} 5 & 3 \\ 2 & 2 \\ 6 & 5 \\ \end{pmatrix} \\ & = \begin{pmatrix} 2*5 + 4*2 + 3*6 & 2*3 + 4*2 + 3*5 \\ 4*5 + -1*2 + 2*6 & 4*3 + -1*2 + 2*5 \\ \end{pmatrix} \\ & = \begin{pmatrix} 10 + 8 + 18 & 6 + 8 + 15 \\ 20 + -2 + 12 & 12 + -2 + 10 \\ \end{pmatrix} \\ & = \begin{pmatrix} 36 & 29 \\ 30 & 20 \\ \end{pmatrix} \end{aligned} $$

Matrix multiplication pseudocode

It is easy to write an algorithm for matrix multiplication by simply following the definition. A pseudocode version of the algorithm follows, where bracket notation $A[i,j]$ replaces the subscript notation $a_{ij}$.

Algorithm Matrix multiplication
//computes $n \times p$ matrix $A · B$ for $n \times m$ matrix $A$, $m \times p$ matrix $B$
//stores result in C

for i = 1 to n do
    for j = 1 to p do
	C[i,j] = 0
        for k = 1 to m do 
            C[i,j] = C[i,j] + A[i,k]*B[k,j]
        end for
    end for
end for
write out product matrix C

matrix2DArray.cpp
matrix2DVector.cpp

Time Complexity Analysis

Since we have three nested loops, each iterating up to $n$, $p$, and $m$, the total operations are: $ O(n \times p \times m) $

Tautologies 1

If A and B are $n \times m$ matrices and $r$ and $s$ are scalars, the following matrix equations are true:

$ 1. \quad 0 + A = A \\ 2. \quad A + B = B + A \\ 3. \quad (A+B)+C = A+(B+C) \\ 4. \quad r(A+B) = rA + rB \\ 5. \quad (r+s)A = rA + sA \\ 6. \quad r(sA) = (rs)A \\ $

Tautologies 2

Where A,B, and C are matrices of appropriate dimensions and $r$ and $s$ are scalars, the following matrix equations are true:

$ A·(B·C) = (A·B)·C \\ A(B+C) = A·B + A·C \\ (A+B)C = A·C + B·C \\ rA·sB = (rs)(A·B) \\ $

Practice

(1) For $r=2$, find $rA+B$. $$ A=\begin{pmatrix} 1 & 7 \\ -3 & 4 \\ 5 & 6 \\ \end{pmatrix} \quad \quad B=\begin{pmatrix} 4 & 0 \\ 9 & 2 \\ -1 & 4 \\ \end{pmatrix} $$
(2) Compute $A·B$ and $B·A$ for $$ A=\begin{pmatrix} 1 & 4 \\ 6 & -2 \\ \end{pmatrix} \quad \quad B=\begin{pmatrix} 3 & 6 \\ 3 & 4 \\ \end{pmatrix} $$

Applications of Matrices

Computer Graphics: Matrices are used for 3D transformations and image processing.
Machine Learning & AI: Matrices are widely used in neural networks, data representation, and statistical modeling.
Physics & Engineering: Matrices help in vector transformations and simulating systems.

Reference

saylor academy
Inverse Matrix