Introduction to Normed Vector Spaces

Audrey Terras
March 29, 2009
1 Why worry about innite dimensional normed vector spaces?
We want to understand the integral from Langs perspective rather than that of your calculus book. Secondly we want to
understand convergence of series of functions - something that proved problematic for Cauchy in the 1800s. These things are
important for many applications in physics, engineering, statistics. We will be able to study vibrating things such as violin
strings, drums, buildings, bridges, spheres, planets, stock values. Quantum physics, for example, involves Hilbert space,
which is a type of normed vector space with a scalar product where all Cauchy sequences of vectors converge.
The theory of such normed vector spaces was created at the same time as quantum mechanics - the 1920s and 1930s.
So with this chapter of Lang you are moving ahead hundreds of years from Newton and Leibnitz, perhaps 70 years from
Riemann.
Fourier series involve orthogonal sets of vectors in an innite dimensional normed vector space:
C[a, /] = ) : [a, /] R[) continuous .
The 1
2
norm of a continuous function ) in C[a, /] is
|)|
2
=
0
@
b
Z
a
[)(r)[
2
dr
1
A
1=2
.
This is an analog of the usual idea of length of a vector ) = ()(1), ..., ()(:)) R
n
:
|)|
2
=
0
@
n
X
j=1
[)(,)[
2
1
A
1=2
.
There are other natural norms for ) C[a, /] such as:
|)|
1
=
b
Z
a
[)(r)[ dr.
|)|
1
= max
axb
[)(r)[ .
On nite dimensional vector spaces such as R
n
it does not matter what norm you use when you are trying to gure out
whether a sequence of vectors has a limit. However, in innite dimensional normed vector spaces convergence can disappear
if a dierent norm is used. Not all norms are equivalent in innite dimensions. See Homework 9, problem 6.
Note that C[a, /] is innite dimensional since the set 1, r, r
2
, r
3
, ..., r
n
, ... is an innite set of linearly independent
vectors. Prove this as follows. Suppose that we have a linear dependence relation
n
X
j=0
c
j
r
j
= 0, for all r in [a, /]. This
implies all the constants c
j
= 0. Why? Extra Credit. Prove this.
Innite dimensional vector spaces are thus more interesting than nite dimensional ones. Each (inequivalent) norm leads
to a dierent notion of convergence of sequences of vectors.
1
2 What is a Normed Vector Space?
In what follows we dene normed vector space by 5 axioms. We will not put arrows on our vectors. We will try to keep
vectors and scalars apart by using Greek letters for scalars. Our scalars will be real. Maybe by the end of next quarter we
may allow complex scalars. It simplies Fourier series.
Denition 1 A vector space \ is a set of vectors \ which is closed under addition and closed under multiplication
by scalars c R. This means n + \ and c \ and the following 5 axioms must hold for all n, , n \ and all
c, , R:
VS1. n + ( + n) = (n + ) + n
VS2. 0 \ s.t. 0 + =
VS3. \ \
0
\ s.t. +
0
= 0.
VS4. + n = n +
VS5. 1 = , c(,) = (c,), (c + ,) = c + ,, c(n + ) = cn + c
You may say we cheated by putting 4 axioms into VS5.
Denition 2 A vector space \ is a normed vector space if there is a norm function mapping \ to the non-negative real
numbers, written || , for \, and satisfying the following 3 axioms:
1. || _ 0 \ \ a:d || = 0 if and only if = 0.
2. |c| = [c[ || , \ \ and\c R. Here [c[ = absolute value of c.
3. |n + | _ |n| +|| , \n, \. Triangle Inequality.
Denition 3 The distance between 2 vectors n, in a normed vector space \ is dened by
d(n, ) = |n | .
Example 1. 3-Space.
R
3
=
8
<
:
0
@
r
1
r
2
r
3
1
A

r
1
, r
2
, r
3
R
9
=
;
.
Dene addition and multiplication by scalars as usual:
0
@
r
1
r
2
r
3
1
A
+
0
@
j
1
j
2
j
3
1
A
=
0
@
r
1
+ j
1
r
2
+ j
2
r
3
+ j
3
1
A
.
c
0
@
r
1
r
2
r
3
1
A
=
0
@
cr
1
cr
2
cr
3
1
A
, \ c R.
The usual norm is
|r|
2
=
q
r
2
1
+ r
2
2
+ r
2
3
i)r =
0
@
r
1
r
2
r
3
1
A
.
Other norms are possible:
|r|
2
= [r
1
[ +[r
2
[ +[r
3
[ or |r|
1
= max [r
1
[ , [r
3
[ , [r
3
[ .
The proof that these denitions make R
3
a normed vector space is tedious. No doubt we will make it a homework
problem.
2
Example 2. The space of continuous functions on an interval.
C[a, /] = ) : [a, /] R[) continuous .
For ), q C[a, /], dene () + q)(r) = )(r) + q(r) for all r [a, /] and dene for c R (c))(r) = c)(r) for all
r [a, /]. We leave it as an Exercise (see Homework 9) to check the axioms for a vector space. The most non-trivial one is
the one that says ) + q and c) are both continuous functions on [a, /].
Again there are many possible norms. We will look at 3:
|)|
2
=
0
@
b
Z
a
[)(r)[
2
dr
1
A
1=2
.
|)|
1
=
b
Z
a
[)(r)[ dr.
|)|
1
= max
axb
[)(r)[ .
Most of the axioms for norms are easy to check. Lets do it for the |)|
1
norm.
N1. || _ 0 \ \ and || = 0 if and only if = 0.
N2. |c| = [c[ || , \ \ and \c R .
N3. Triangle Inequality. |n + | _ |n| +|| , \ n, \.
Proof of N1.
Since [)(r)[ _ 0 for all r we know that the integral is _ 0, because the integral preserves inequalities (Lang, Thm. 2.1,
p. 104 or earlier in these notes). Suppose that
b
Z
a
[)(r)[ dr = 0. Since ) is continuous by Theorem 2.4 of Lang, p. 104 or
these notes near Figure 1, this implies )(r) = 0 for all r [a, /].
Proof of N2.
Also for any c R and ) C[a, /], we have |c)|
1
=
b
Z
a
[c)(r)[ dr =
b
Z
a
[c[ [)(r)[ dr = [c[
b
Z
a
[)(r)[ dr = [c[ |)|
1
. This
proves N2 for norms. Here we used the multiplicative property of absolute value as well as the linearity of the integral (i.e.,
scalars come out of the integral lecture notes p. 80).
Proof of N3. Using the denition of the 1-norm, and the triangle inequality for real numbers as well
as the fact that the integral preserves _, we see that
|) + q|
1
=
b
Z
a
[)(r) + q(r)[ dr _
b
Z
a
([)(r)[ +[q(r)[) dr.
To nish the proof, use the linearity of the integral to see that
b
Z
a
([)(r)[ +[q(r)[) dr =
b
Z
a
[)(r)[ dr +
b
Z
a
[q(r)[ dr = |)|
1
+|q|
1
.
Putting it all together gives |) + q|
1
_ |)|
1
+|q|
1
.
3
3 Scalar Products.
You have seen the dot (or scalar or inner) product in R
3
. It is
0
@
r
1
r
2
r
3
1
A

0
@
j
1
j
2
j
3
1
A
= r
1
j
1
+ r
2
j
2
+ r
3
j
3
.
It turns out there is a similar thing for C[a, /]. First lets dene the scalar product on a vector space and see how to get
a norm if in addition the scalar product is positive denite.
Denition 4 A (positive denite) scalar product < , n for vectors , n in a vector space \ is a real number < , n
such that the following axioms hold:
SP1. < , n =< n, , \ , n \ (symmetry)
SP2. < n, + n =< n, + < n, n , \ n, , n \
SP3. < c, n = c < , n , \ , n \ and \ccR
SP4. < , _ 0, \ \ and < , = 0 == = 0 (positive denite)
Axioms SP1,2,3 say < , n is linear in each variable holding the other variable xed. Axiom SP4 says the scalar
product is positive denite. We will always want to assume SP4 because we want to be able to get a norm out of the
scalar product.
Denition 5 If \ is a vector space with a (positive denite) scalar product < , n for , n \, dene the associated
norm by || =
_
< , , for all \
Before proving that this really gives a norm, lets look at some examples.
Example 1. In R
3
the scalar product is
< r, j =
0
@
r
1
r
2
r
3
1
A

0
@
j
1
j
2
j
3
1
A
= r
1
j
1
+ r
2
j
2
+ r
3
j
3
.
It is easy to check the axioms. For example, the positive deniteness follows from the fact that squares of real numbers are
_ 0 and sums of non-negative numbers are non-negative:
< r, r = r
2
1
+ r
2
2
+ r
2
3
_ 0.
And 0 =< r, r _ r
2
i
implies r
i
= 0 for all i and thus r = 0.
Example 2.
C[a, /] =) : [a, /] R[ ) is continuous
For ), q in C[a, /], dene the scalar product by
< ), q =
b
Z
a
)q.
Once more, it is not hard to use the properties of the integral to check axioms SP1,SP2,SP3 (extra credit exercise). To
see SP4, note that )(r)
2
_ 0 for all r [a, /] implies by the fact that integrals preserve _ that
< ), ) =
b
Z
a
)(r)
2
dr _ 0.
4
Now suppose that < ), ) = 0. Then by the positivity property of the integral, we know that )(r)
2
= 0 for all r [a, /]
which says that ) is the 0 function (the identity for addition in our vector space C[a, /]).
Then the norm associated to this scalar product is |)|
2
=
0
@
b
Z
a
[)(r)[
2
dr
1
A
1=2
.
The following theorem is so useful people from lots of countries got their names attached.
Theorem 6 Cauchy-Schwarz (Bunyakovsky) Inequality
Suppose that \ is a vector space with scalar product < , n . Then using our denition of the norm || =
_
< , ,
we have for all , n \ :
[< , n [ _ || |n| .
Proof. (Compare Problem 8 in Homework 8).
Let t R and look at
)(t) =< + tn, + tn .
By properties of the scalar product, we have 0 _ )(t) =< , +2t < , n +t
2
< n, n .
As a function of ), we see that )(t) = t
2
+ 1t + C, where =< n, n , 1 = 2 < , n and C =< , . So
the graph of )(t) is that of a parabola above or touching the t-axis. For example, we have drawn a parabola touching the
t-axis at one point.
Recall the quadratic formula for the roots r

of )(t) = t
2
+ 1t + C = 0,
r

=
1
_
1
2
4C
2
.
Since we have at most one real root it follows that
1
2
4C _ 0.
Now plug in =< n, n , 1 = 2 < , n and C =< , . This gives the Cauchy-Schwarz inequality.
Corollary 7 Under the hypotheses of the preceding theorem, || =
_
< , denes a norm on \ .
Proof. We must prove:
N1. || _ 0 \ \ and || = 0 if and only if = 0.
N2. |c| = [c[ || , \ \ and \c R .
N3. Triangle Inequality. |n + | _ |n| +|| , \ n, \.
We get N2 from SP3.For then |c|
2
=< c, c = c
2
< , = [c[
2
||
2
, \ \ and \ccR .
5
We get N1 from SP4. This says ||
2
=< , _ 0, \ \ and ||
2
=< , = 0 == = 0.
To prove the triangle inequality N3, we need to use the Cauchy-Schwarz inequality. This proof goes as follows. By the
linearity and symmetry of the scalar product we see that
|n + |
2
= < n + , n + =< n, n +2 < n, + < ,
= |n|
2
+ 2 < n, +||
2
_ |n|
2
+ 2 [< n, [ +||
2
as r _ [r[
_ |n|
2
+ 2 |n| || +||
2
by Cauchy Schwarz
= (|n| +||)
2
.
Now use the fact that the square root
_
preserves inequalities to nish the proof of the triangle inequality.
Whats the good of all this? Now we can happily denite limits of sequences of vectors
n
in our normed vector space
\ . Can you guess the denition of |i:
n!1

n
= 1 \ ?
Answer: \- 0,
"
Z
+
s.t. : _
"
implies |
n
1| < -. That is, just replace absolute value in the old
denition of limit with the norm.
Similarly we can dene a Cauchy sequence
n
in the normed vector space \ .
Another use of the scalar product is to dene orthogonal vectors in a vector space \ with a scalar product.
Denition 8 Two vectors , n \, a vector space \ with scalar product <, , are dened to be orthogonal if the scalar
product < , n = 0.
In a vector space with scalar product, you can also dene the angle 0 between 2 vectors , n \, by
< , n = || |n| cos 0.
You can draw the same picture you would draw in the plane since 2 vectors determine a plane and then use the cosine law
from high school trig to see this.
What is the cosine law? Using the triangles in Figure 2, it says
| n|
2
= ||
2
2 || |n| cos 0 +|n|
2
.
You also need to see, using the axioms for scalar product, that
| n|
2
=< n, n = ||
2
2 < , n +|n|
2
.
6
Figure 1: Visualizing vectors in a normed vector space and the angle between them.
7
4 Comparison of Norms
Denition 9 Suppose
n
is a sequence in a normed vector space \ with norm || , We say
lim
n!1

n
= 1
and "
n
converges to 1 \ " ==
lim
n!1
|
n
1| = 0.
Note that the last limit is a sequence of real numbers.
Question. We know that there are lots of norms on \ . How can we guarantee that 2 dierent norms ||

and ||

produce the same convergent sequences in \ ?

The answer is that equivalent norms produce the same convergent sequences where we dene equivalent as follows.
Denition 10 2 norms ||

and ||

on the vector space \ are equivalent i there are constants , 1 0 such that for all
\ we have
||

_ ||

_ 1||

.
In the preceding denition we are assuming that the constants and 1 are independent of \.
Why do equivalent norms lead to the same convergent sequences?
Answer. Suppose
n
is a convergent sequence for the ||

-norm; i.e., for some 1 \ we have lim

n!1
|
n
1|

= 0.
And suppose ||

is an equivalent norm. Then

|
n
1|

_ |
n
1|

_ 1|
n
1|

.
Since the outside sequences go to 0 as : , it follows by the squeeze lemma that the guy in the middle has to go to 0
as well. For limits preserve inequalities and we would have
0 _ lim
n!1
|
n
1|

_ 0.
which implies lim
n!1
|
n
1|

= 0.
Similarly lim
n!1
|
n
1|

= 0 implies lim
n!1
|
n
1| = 0.
Moral. It does not matter which of 2 equivalent norms you use to test a sequence for convergence.
Theorem 11 All norms on R
n
are equivalent.
Proof. See Lang p. 145.
Thus, for our purposes, it does not matter which norm you use on nite dimensional vector spaces. You get the same
denition of convergence of sequences. However, things are very dierent for innite dimensional vector spaces. Look at
Exercise 6 in Homework 9 for example. There you see a sequence of functions )
n
in C[0, 1] such that
lim
n!1
|)
n
0|
1
= 0 using the norm |)|
1
=
1
Z
0
[)(r)[ dr.
but
|)
n
0|
1
= 1 for all n, using the norm |)|
1
= max
axb
[)(r)[ .
It follows that the norms ||
1
and ||
1
on C[0, 1] are not equivalent.
Extra Credit Exercise. Using the same example, show that lim
n!1
|)
n
0|
2
= 0 using the norm|)|
2
=
0
@
1
Z
0
[)(r)[
2
dr
1
A
1=2
.
It follows that the norms ||
2
and ||
1
on C[0, 1] are not equivalent.
8
Proposition 12 The norms |)|
1
=
b
Z
a
[)(r)[ dr and |)|
2
=
0
@
b
Z
a
)(r)
2
dr
1
A
1=2
are not equivalent. However, we do have
the inequality
|)|
1
_
_
/ a |)|
2
.
Proof. To prove the inequality, use the Cauchy-Schwarz inequality on the functions [)[ and q(r) = 1 for all r [a, /]. This
gives
[< ), q [ _ |)|
2
|q|
2
.
So we have [< [)[ , 1 [ _ |[)[|
2
|1|
2
. Now this really means
|)|
1
=
b
Z
a
[)(r)[ dr _
0
@
b
Z
a
)(r)
2
dr
1
A
1=2
0
@
b
Z
a
1dr
1
A
1=2
=
_
/ a
0
@
b
Z
a
)(r)
2
dr
1
A
1=2
=
_
/ a |)|
2
.
To see that ||
1
and ||
2
are not equivalent norms, we take a = 0 and / = 1. Then we look at the following example.
Dene as in Lang, p. 147:
q
n
(r) =
_
:, for 0 _ r _
1
n
;
1
p
x
, for
1
n
_ r _ 1.
Note that q
n
is continuous. Why? Extra Credit Exercise. Show
|q
n
|
1
= 2
1
_
:
and
|q
n
|
2
=
p
1 + log :.
It follows that there cannot be a constant C 0 such that ||
2
_ C ||
1
at least on the interval [0, 1]. Can you extend
this idea to arbitrary intervals [a, /]?
9