EE103 (Fall 2011-12)

3. Linear equations
linear equations
example: polynomial interpolation
applications
geometrical interpretation
3-1
Linear equations
m equations in n variables x
1
, x
2
, . . . , x
n
:
a
11
x
1
+ a
12
x
2
+ + a
1n
x
n
= b
1
a
21
x
1
+ a
22
x
2
+ + a
2n
x
n
= b
2
.
.
.
a
m1
x
1
+ a
m2
x
2
+ + a
mn
x
n
= b
m
in matrix form: Ax = b, where
A =

a
11
a
12
a
1n
a
21
a
22
a
2n
.
.
.
.
.
.
.
.
.
.
.
.
a
m1
a
m2
a
mn

, x =

x
1
x
2
.
.
.
x
n

, b =

b
1
b
2
.
.
.
b
m

Linear equations 3-2

Example: polynomial interpolation
t a polynomial
p(t) = x
1
+ x
2
t + x
3
t
2
+ + x
n
t
n1
through n points (t
1
, y
1
), . . . , (t
n
, y
n
)
t
1
t
2
t
3
t
4
t
5
problem data (parameters): t
1
, . . . , t
n
, y
1
, . . . , y
n
problem variables: x
1
, . . . , x
n
Linear equations 3-3
write out the conditions on x:
p(t
1
) = x
1
+ x
2
t
1
+ x
3
t
2
1
+ + x
n
t
n1
1
= y
1
p(t
2
) = x
1
+ x
2
t
2
+ x
3
t
2
2
+ + x
n
t
n1
2
= y
2
.
.
.
p(t
n
) = x
1
+ x
2
t
n
+ x
3
t
2
n
+ + x
n
t
n1
n
= y
n
in matrix form: Ax = b with
A =

1 t
1
t
2
1
t
n1
1
1 t
2
t
2
2
t
n1
2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1 t
n
t
2
n
t
n1
n

, b =

y
1
y
2
.
.
.
y
n

n linear equations in n variables

Linear equations 3-4
Exercise
express as a set of linear equations: nd two cubic polynomials
p(t) = c
0
+ c
1
t + c
2
t
2
+ c
3
t
3
, q(t) = d
0
+ d
1
t + d
2
t
2
+ d
3
t
3
that satisfy the following properties:
p(t
1
) = y
1
, p(t
2
) = y
2
, p(t
3
) = y
3
(t
i
, y
i
given for i = 1, 2, 3)
q(t
5
) = y
5
, q(t
6
) = y
6
, q(t
7
) = y
7
(t
i
, y
i
given for i = 5, 6, 7)
p(t
4
) = q(t
4
), p

(t
4
) = q

(t
4
) (t
4
is given)
t
1
t
2
t
3
t
4
t
5
t
6
t
7
q(t)
p(t)
Linear equations 3-5
Applications
a set of linear equations Ax = b (with A mn) is
square if m = n
underdetermined if m < n
overdetermined if m > n
all three types arise in practice
broad categories of applications:
analysis or simulation
control or design
estimation or inversion
Linear equations 3-6
Analysis or simulation
y = Ax is a linear (or linearized) model of a physical system
x and y are quantities that describe the operation of the system
(e.g., currents & voltages, forces & displacements, . . . )
we know y; we are interested in determining x
examples
structure (e.g., building, bridge, . . . ): y
i
s are forces or loads, x
i
s are
the resulting displacements at specied points
electrical circuit: y
i
s are values of voltage and current sources, x
i
s are
the resulting voltages and currents in the circuit
Linear equations 3-7
Control or design
x is vector of design parameters or inputs (which we can choose)
y is vector of results, or outcomes
y = Ax describes how input choices aect results
problems
nd x so that y = y
des
nd all xs that result in y = y
des
(i.e., all designs that meet the specs)
among xs that satisfy y = y
des
, nd a small one (i.e., nd a small or
ecient x that meets specications)
Linear equations 3-8
example: nal position/velocity of mass from forces (see p. 1-10)
relation between forces and nal velocity/position:

y
1
y
2

19/2 17/2 1/2

1 1 1

x
1
x
2
.
.
.
x
10

x: sequence of forces
y
1
: nal position; y
2
: nal velocity
problems: given desired nal position/velocity y
des
nd force sequence x so that y = y
des
(2 linear equations in 10 vars.)
nd all xs that result in y = y
des
among xs that satisfy y = y
des
, nd one with small x
Linear equations 3-9
Estimation or inversion
y
i
is a measurement or sensor reading (which we know)
x
j
is parameter to be estimated or determined
y = Ax describes how parameters aect measurements
(a
ij
is the sensitivity of the ith sensor to the jth parameter)
problems
nd x, given y
nd all xs that result in y (i.e., xs consistent with measurements)
among all xs that result in y, nd x closest to some prior guess x
prior
if there is no x such that y = Ax, nd x such that y Ax (if the
sensor readings are inconsistent, nd x which is almost consistent)
Linear equations 3-10
Image reconstruction from projections
problem: estimate density (x, y, z) of a 3-dimensional object
partition object in N small boxes (pixels or voxels) and assume density
is constant on each pixel
X-ray beam with intensity I
0
measured intensity I = I
0
e

N
i=1
l
i

i
l
i

i
: (unknown) density in pixel i;
l
i
: (known) length of path through pixel i (up to a physical constant)
Linear equations 3-11
after taking logs: one linear equation in the variables
i
:
N

i=1
l
i

i
= log(I
0
/I)
repeat measurement with dierent source locations and beam directions:
M measurements yield M linear equations
N

i=1
l
ki

i
= log(I
0
/I
k
), k = 1, . . . , M
usually a huge number of variables (millions)
underdetermined set of linear equations (M N): many solutions
i
a popular method: pick a solution , consistent with the measurements,
and close to a prior guess
for example, choose
i
that minimizes

N
i=1
(
i

i
)
2
where
i
is a
prior guess of the density (e.g., uniform density)
Linear equations 3-12
Geometrical interpretation of Ax = b
special case: one equation in n variables (m = 1)
a
T
x = b (a = 0)
general solution: x =
b
a
2
a +v where v is an arbitrary vector with a
T
v = 0
a
b
a
2
a
x
v
{x | a
T
x = b}
the solution set is called a (hyper-)plane, a is the normal vector
for n = 2: a line perpendicular to a; n = 3: a plane perpendicular to a.
Linear equations 3-13
example (n = 2, a = (2, 1))
a
x
1
x
2
a
T
x = 5
a
T
x = 10
a
T
x = 5
a
T
x = 0
Linear equations 3-14
general case: m equations in n variables
Ax = b
write A in terms of its rows:
A =

a
11
a
12
a
1n
a
21
a
22
a
2n
.
.
.
.
.
.
.
.
.
.
.
.
a
m1
a
m2
a
mn

a
T
1
a
T
2
.
.
.
a
T
m

where a
i
=

a
i1
a
i2
.
.
.
a
in

then Ax = b can be written as

a
T
1
x = b
1
, a
T
2
x = b
2
, . . . , a
T
m
x = b
m
the solution set is the intersection of m hyperplanes a
T
i
x = b
i
(assuming the rows a
i
are nonzero)
Linear equations 3-15
example (n = 2): A =

2 1
1 1

, b =

10
2

x
a
1
a
2
a
T
2
x = 2
a
T
1
x = 10
solution is x = (4, 2)
Linear equations 3-16