Lecture 12: CauchySchwartz Inequality,
GramSchmidt Process, QR Factorization and
Orthogonal Transformations
Longxiu Huang
Goal:
•
CauchySchwartz inequality. Angle between two vectors.
Correlation coefficient.
•
GramSchmidt process and QR factorization.
•
Orthogonal transformations, orthogonal matrices, matrix
transpose and the matrix of an orthogonal projection.
1 / 25
Table of Contents
1
CauchySchwartz inequality
2
GramSchmidt Process
3
QR Factorization
4
Orthogonal Transformation
5
The Transpose of a Matrix
6
The Matrix of an Orthogonal Projection
Theorem (Pythagorean theorem)
Let
~x, ~
y
∈
R
n
. The equation
k
~x
+
~
y
k
2
=
k
~x
k
2
+
k
~
y
k
2
holds if (and only if)
~x
and
~
y
are orthogonal.
2 / 25
Theorem (An inequality for the magnitude of proj
V
(
~x
)
)
Consider a subspace
V
∈
R
n
and a vector
~x
∈
R
n
. Then
k
proj
V
~x
k ≤ k
~x
k
.
The statement is an equality iff
~x
∈
V
.
3 / 25
Proof.
Since for any vector
~x
∈
R
n
, we have
~x
=
~x
⊥
+
proj
V
(
~x
)
.
By Pythagorean theorem, we get
k
~x
k
2
=
k
~x
⊥
k
2
+
k
proj
V
(
~x
)
k
2
≥ k
proj
V
(
~x
)
k
2
and thereby
k
proj
V
~x
k ≤ k
~x
k
.
4 / 25
Theorem (CauchySchwartz inequality))
If
~x
and
~
y
are vectors in
R
n
, then

~x
·
~
y
 ≤ k
~x
kk
~
y
k
.
The statement is an equality iff
~x
and
~
y
are parallel.
5 / 25
Definition (Angles between two vectors)
Consider two nonzero vectors
~x
and
~
y
in
R
n
. The angle
θ
between
these vectors is defined as
θ
=
arccos
~x
·
~
y
k
~x
kk
~
y
k
.
Note that
θ
∈
[0
, π
]
.
6 / 25
Table of Contents
1
CauchySchwartz inequality
2
GramSchmidt Process
3
QR Factorization
4
Orthogonal Transformation
5
The Transpose of a Matrix
6
The Matrix of an Orthogonal Projection
GramSchmidt Process
Consider a basis
~v
1
, . . . ,~v
m
of a subspace
V
of
R
n
. Let
V
1
=
span
(
~v
1
)
V
2
=
span
(
~v
1
,~v
2
)
.
.
.
V
m
=
span
(
~v
1
, . . . ,~v
m
) =
V
To find an orthonormal basis of
V
1
, we normalize the basis vector
~v
1
and get
~u
1
=
~v
1
k
~v
1
k
which forms an orthonormal basis of
V
1
.
7 / 25
GramSchmidt Process continues...
To find an orthonormal basis of
V
2
, we have
~u
1
∈
V
2
and look for a
unit vector
~u
2
which is orthogonal to
~u
1
such that
span
(
~u
1
, ~u
2
) =
V
2