i=1
i
z
i
, (1.2)
then
u =
n
i=1
i
f(
i
)z
i
.
Let us take the vector p(A), with p a polynomial of degree m 1, as an
approximate value of u. The error norm is
u p(A) =
_
_
_
_
_
n
i=1
i
(f p)(
i
)z
i
_
_
_
_
_
=
_
n
i=1
2
i
(f p)
2
(
i
). (1.3)
Thus, the problem has been reduced to discrete polynomial approximation to f
with the nodes
i
and the weights
2
i
.
With polynomial methods, computer time is mostly consumed by multiplying A
by vectors. Therefore, the integer m needed to attain a given accuracy is the most
important characteristic of such a method. One can easily see that for any xed m
the minimum in (1.3) is attained if and only if p is the initial m terms of the Fourier
series of f in the (
i
,
2
i
)orthogonal polynomials. The Lanczos process, which was
initially intended for computation of the spectrum of A, allows one to build up the
discrete orthogonal polynomials whose weight approximates (
i
,
2
i
) in a certain
sense.
For problem (1.1), the Lanczos process was rst applied to linear systems [20,
24, 25], and then to onedimensional [15, 16] and multidimensional [3, 12] parabolic
7/1/2000 16:10 PAGE PROOFS parlkah3
Krylov Subspace Approximation 207
and hyperbolic equations. The general scheme of applying the Lanczos process to
solving (1.1) (see 2) was suggested in [2, 4] and in another form in [28]. See also
[8, 9, 14, 21].
Error bounds for computation of (1.1) in general ( 3) and for some concrete forms
of f (most common, mentioned above) ( 4  6) are given below. The application of
our results to the proper Lanczos method (for calculating eigenpairs) is also stated
( 7).
Computer arithmetic noticeably inuences the Lanczos method ( 8). Our results
are extended to nite computer arithmetic in 9  10.
2. The Spectral Lanczos Decomposition Method
Consider the result of m steps in the Lanczos process for approximate computation
of eigenpairs of A [23, ch. 13]. In the Krylov subspace
/
m
= span (, A, . . . , A
m1
),
obtain the basis q
1
, . . . , q
m
as the result of GramSchmidt orthogonalization of the
vectors , A, . . . , A
m1
. The orthogonalization can be carried out indirectly by
using the following three term recurrence:
Aq
i
=
i1
q
i1
+
i
q
i
+
i
q
i+1
, i = 1, . . . , m,
where
0
q
0
is assumed to be 0, q
1
= /,
i
0. Denote by H the tridiagonal
symmetric matrix
H =
_
_
_
_
1
1
0
1
2
2
0
m1
m
_
_
_
_
,
by
i
, s
i
= (s
1i
, . . . , s
mi
)
T
, i = 1, . . . , m, the eigenvalues and corresponding nor
malized eigenvectors of H. Dene e
i
as the ith unit mvector, Q = (q
1
. . . q
m
),
y
i
= Qs
i
. Then (
i
, y
i
) are the approximate eigenpairs of the matrix A obtained
with Ritz method, as applied to /
m
. In view of (1.2) and the orthonormality of the
matrix (s
1
. . . s
m
), we have
= q
1
= Qe
1
= Q
m
i=1
s
1i
s
i
= 
m
i=1
s
1i
y
i
,
which is why it is natural to take the vector
u
m
= 
m
i=1
s
1i
f(
i
)y
i
= Q
m
i=1
s
1i
f(
i
)s
i
= Qf(H)e
1
as an approximation to u. This denition is correct, because the spectrum of H is
contained in the spectral segment of A.
The method that has just been described is called the spectral Lanczos decom
position method (shortly, SLDM).
7/1/2000 16:10 PAGE PROOFS parlkah3
208 Vladimir Druskin, Leonid Knizhnerman
Usually, computation of Q and H is the most timeconsuming part of the SLDM.
The way of computing Qf(H)e
1
depends on a concrete application. Sometimes a
tridiagonal system of linear algebraic equations (e.g., f(H) = (H i)
1
,
R) or a system of ordinary dierential equations with a tridiagonal matrix (e.g.,
f(H) = exp(tH)) can be solved. In some cases the use of Chebyshev expansion
of f is preferable. The standard way is to solve the full eigenproblem for H and to
use one of the two expressions:
m
i=1
s
1i
f(
i
)y
i
or Q
m
i=1
s
1i
f(
i
)s
i
,
dependently on how many functions (say, how many time moments t in exp(tH))
you want to handle simultaneously and in how many nodes of R
n
you want to know
the solution.
To terminate the Lanczos process properly, we compute u
m
for dierent m and
exploit a suitable stabilization criterion.
3. The General SLDM Error Bound
Assume further that A is not scalar, that is
n
= max
i
i
> min
i
i
=
1
. Assign
B =
(
n
+
1
)I 2A
1
, g(x) = f
_
n
+
1
(
n
1
)x
2
_
,
then I B I and the function g is dened on the segment [1, 1]. Let us
consider the Chebyshev series of the rst kind
g(x) =
k=0
g
k
T
k
(x). (3.4)
Theorem 3.1. If series (1.1) converges absolutely on [1, 1], then the inequality
u u
m
 2
k=m
[g
k
[
holds [4].
Theorem 3.1, in general, cannot be substantially improved. An example has been
found where the error is greater than the righthand side of theorem 3.1s inequality
multiplied by
_
3/8. This example is: A = (a
ij
), a
ij
= 1/
2, if [i j[ =1 and
i + j = 3, a
ij
= 1/2, if [i j[ = 1 and i + j > 3, and a
ij
= 0 elsewhere; = e
(n)
1
;
f = T
2m
, n > 2m; and u u
m
 =
1.5.
4. Computation of exp(tA)
Let us assume that A 0, i.e.
1
0. Evidently,
u(t) = exp(tA) = exp
_
t
n
(I B)
2
_
= exp
_
t
n
2
_
exp
_
n
B
2
_
k=1
I
k
(a)T
k
(x)
(I
k
is the Bessel function [11]), one easily nds that
u(t) = exp
_
t
n
2
_
_
I
0
_
t
n
2
_
+ 2
k=1
I
k
_
t
n
2
_
T
k
(B)
_
,
so that
g
k
= 2I
k
_
t
n
2
_
for k 1. (4.5)
Theorem 4.1. Let a =
tn
2
and m a. Then the error bound
u u
m


_
2 + O
_
m
a
__
a
m
exp
_
m
2
2a
+ O
_
m
4
a
3
__
holds [4].
The proof uses (4.5) and the asymptotics for I
k
.
The estimate shows that for large t the SLDM convergence is faster than expo
nential, and the number of Lanczos steps m to reach given accuracy is proportional
to
t.
5. Computation of exp(tA) with Splitting
SLDM can be eectively used for acceleration of timestepping parabolic equation
solvers.
Imagine that we want to compute exp(tA) from 4, using a splitting scheme,
for instance, the onecycle scheme with the transitional operator
A
=
_
I +
2
A
1
_
1
_
I
2
A
1
__
I +
2
A
2
_
1
_
I
2
A
2
_
,
where A = A
1
+ A
2
, A
1
0, A
2
0, A
1
A
2
= A
2
A
1
and > 0 is the time step.
Then the approximate solution can be written as
u
()
(t) = A
s
(5.6)
for any integer quotient s = t/. It is known that A
is symmetric and I A
I.
For other variants of splitting we may have 0 A
instead
of A.
Theorem 5.1. For the approximate solution u
()
m
, generated by m steps of Lanczos
process with A
, 0 A
_
1 + O
_
m
s
_
+ O
_
1
s m
__
exp
_
m
2
s
+ O
_
m
3
s
2
__
,
where s = t/ N, is valid [6].
7/1/2000 16:10 PAGE PROOFS parlkah3
210 Vladimir Druskin, Leonid Knizhnerman
Table 1. The results of the numerical experiment; m is the number of steps, l is the
number of multiplications/step/node.
Exact sol. Exact sol. Standard SLDM + LIM
dierent. semidiscr. SLDM splitting
t problem problem m = 15, l = 10 m = 3, l = 16 m = 3192, l = 6
0.004 0.58517(1) 0.58517(1) 0.58516(1) 0.58518(1) 0.58523(1)
0.008 0.54711(1) 0.54711(1) 0.54689(1) 0.54713(1) 0.54722(1)
0.016 0.47507(1) 0.47508(1) 0.47392(1) 0.47343(1) 0.47534(1)
0.032 0.35155(1) 0.35160(1) 0.34955(1) 0.34842(1) 0.35218(1)
0.064 0.18794(1) 0.18801(1) 0.18657(1) 0.18572(1) 0.18871(1)
0.128 0.53158(2) 0.53201(2) 0.52767(2) 0.52499(2) 0.53598(2)
0.256 0.42489(3) 0.42557(3) 0.42189(3) 0.41940(3) 0.43196(3)
0.512 0.27145(5) 0.27231(5) 0.26969(5) 0.26766(5) 0.28059(5)
1.024 0.11078(9) 0.11150(9) 0.11020(9) 0.10902(9) 0.11839(9)
The proof uses the decomposition
x
s
= 2
2s+1
m
i=0
_
2s
i
_
T
si
(x),
where
_
_
is a binomial coecient, T
k
is the Chebyshev polynomial, shifted
onto the segment [0, 1], and the
sign at
means that the member with T
0
is to
be divided by 2.
The estimate shows that, to attain the same accuracy, the updated solver only
needs the square root of the number of steps of the original one.
Numerical example. Let us consider the model problem
u +
u
t
= 0, (x, y) =]0, 1[
2
(t > 0),
u(x, y, t)[
t=0
= x(1 x)y(1 y), u[
(x,y)
= 0.
Approximating the Laplacian as usual on the rectangular grid with the step h =
1/51, we get a secondorder semidiscrete scheme with spatial operator A. This
scheme has been solved by the two methods described above and by local itera
tion [13]. The results of the comparison for the point (
25
51
,
25
51
) are presented in the
table, where m denotes the number of matrixvector multiplications. The splitting
methods time step equals 410
3
. The experiment demonstrates the large speedup
of SLDM compared with conventional explicit and implicit methods.
6. Other Matrix Functions [4]
We shall assume here that  = 1.
a) u(t) = cos(t
A), A 0.
7/1/2000 16:10 PAGE PROOFS parlkah3
Krylov Subspace Approximation 211
This vector function satises the hyperbolic Cauchy problem
Au +
d
2
u
dt
2
= 0, u(0) = ,
du
dt
(0) = 0.
If =
2
n
, x =
2t
, =
2m
x
1 and for all that 0 < 1, then
u u
m

exp
_
x(2)
1.5
3
_
2
4m
2
x
2
(some Oterms have been omitted). So, for m > t/, SLDM converges very fast,
being the maximal time step of the explicit scheme according to the Courant
FriedrichsLevy condition.
b) u(z) = exp(z
A),
1
> 0.
This vector function satises the elliptical Dirichlet problem
Au
d
2
u
dz
2
= 0 (z > 0), u(0) = , u(+) = 0.
If c =
n+1
n1
, then
u u
m

2(c)
m
1 (c)
1
,
denotes the inverse Zhukovsky function, (w) = w +
w
2
1. So, when applied
to the Ndimensional elliptic problem with coecients independent of one variable,
SLDM has the same estimate of arithmetical work as solution of the (N 1)
dimensional problem by conjugate gradients.
c) u = A
1
,
1
> 0 (a symmetric system of linear equations):
u u
m
 u
2
1
with = T
m
_
n+1
n1
_
1
.
7. Some Evaluations for Eigenvalues [7]
In order to draw conclusions about convergence of
i
towards the exact eigenvalues
of A, one can take in theorem 3.1 a hatlike polynomial, picking out the required
eigenvalue
r
, as f. We assume here that  = 1.
Theorem 7.1. Let us assume that A 1,
r
= 0,
r
,= 0 and m 3. Then i,
1 i m, exists such that
[
i
[
_
4
2
r
2
_ 1
m1
1
2
.
7/1/2000 16:10 PAGE PROOFS parlkah3
212 Vladimir Druskin, Leonid Knizhnerman
Remark If m log
_
4
2
r
2
_
, then the righthand side of the theorems evaluation
approximately equals
1
2m
log
_
4
2
r
2
_
.
Theorem 7.2. Let us assume that A 1,
r
= 0,
r
> 0, [
l
[ > 0 for
l ,= r, k = [
m1
2
] and
= T
k
_
1 +
2
1
2
_
1
satises
<
r
1 +
_
1
2
r
.
Then, there exists such an index j, 1 j m, that the estimate
[
j
[
_
2(1
2
r
)
_
1 +
_
1
2
r
_
is valid.
Theorem 7.2 is similar to Kaniel and Saad estimates [23, 12.4], but, unlike them,
concerns the internal part of the spectrum and is independent of r. The estimate
cannot be essentially improved. As to eigenvectors, a nontrivial estimate cannot be
obtained, but z
r
can be approximated by a linear combination of at most two y
i
with the error of the order of about .
8. Computer Arithmetic General Notes
Since the beginning of the paper we have been dealing with the exact arithmetic.
The real picture, of course, is not as bright as it has been drawn. The Lanczos
method was suggested by Lanczos as early as in 1950, but, because of its instability,
was kept in the background for two decades. The instability manifests itself in the
loss of orthonormality of y
k
s and in the appearance of the parasitic counterparts
to
i
s.
To achieve a suitable level of orthogonality, one can use dierent variants of
reorthogonalization [22, 26, 27]. Unfortunately, such modications are costly, and
a half the number of signicant digits is immediately lost when solving (1.1).
In 1970s Paige [1719] obtained important results clarifying behavior of the simple
(without any reorthogonalization) Lanczos process in computer arithmetic. It was
also empirically pointed out that, in spite of instability, there appear approximations
to all the s among the s, while m increases (so called Lanczos phenomenon [1]).
Using Paiges theorems, we have managed to prove, for computer arithmetic,
analogs of a number of results given here. Their sense can be summarized as follows:
the Lanczos process is unstable by itself, but the error bounds remain stable with
respect to roundo.
Yet Paiges estimates are outstanding theoretically, Parlett [23] and Paige [19]
reckon that they are numerically rough: in todays examples Paiges parameters (see
9) can be quite large, but the real precision is much better. Any improvement of
Paiges results shall automatically make stronger ours.
7/1/2000 16:10 PAGE PROOFS parlkah3
Krylov Subspace Approximation 213
Paige assumed that the matrix would be presented explicitly and, therefore, the
matrixvector multiplication would be calculated in the standard inner product
manner. In another (implicit or mixed) case the user can independently estimate
the computer error in the computation of Av, v R
n
, and make correspondent
evident changes in Paiges results.
9. Computer Arithmetic Problem (1.1)
Let be the elementary computer roundo error, c
1
the maximal number of nonzero
elements in rows of A,
1
=
_
7 +
c
1
 [A[ 
A
_
,
2
=
2 max[12(n + 4),
1
],
3
=
4.5
24
m
1
+
2m(n + 4), = m
2.5
A
2
(Paiges parameters). Also, let f be dened on [
1
,
n
+ ],
B =
(
n
+
1
)I 2A
1
+ 2
, g(x) = f
_
n
+
1
(
n
1
+ 2)x
2
_
and g
k
be dened by (3.4).
Theorem 9.1. If
m[6(n + 4) +
1
] 1, (n + 4) <
1
24
(9.7)
and series (3.4) converges absolutely on [1, 1], then the error bound
u u
m


4
_
2
g
A
1
m
3
1
+
10
24
k=m
[g
k
[
is valid for the computation of (1.1) by the simple Lanczos process, where
g
_
1
_
1
g(x)
2
dx
1 x
2
[5].
Evidently, theorem 9.1 is an analog of theorem 3.1 for the computer arithmetic.
Theorem 9.1 shows that the simple Lanczos process produces an approximate solu
tion of (1.1) with stable error evaluation. Figure 1 illustrates the dierence between
theorem 3.1 (in the form of theorem 4.1) and theorem 9.1 in the case of the matrix
exponential.
7/1/2000 16:10 PAGE PROOFS parlkah3
214 Vladimir Druskin, Leonid Knizhnerman

100 200 300 400
m
6
5
10
15
log
10
uum
= 3 10
16
= 0
Figure 1. Comparison of the SLDM error estimates for exact arithmetic (theorem 3.1)
and computer arithmetic (theorem 9.1); f(A) = exp(tA), n = 100, n 1000,
1 = 17, t = 50.
7/1/2000 16:10 PAGE PROOFS parlkah3
Krylov Subspace Approximation 215
10. Computer Arithmetic Lanczos Phenomenon [5]
Initially, the Lanczos phenomenon was investigated in [1], where a conditional result
was proved.
Taking in theorem 9.1 a caplike polynomial, picking out the necessary eigen
value, as f, one can obtain rigorous results on the convergence of eigenvalues in the
simple Lanczos process.
Let us assume that  = 1, A 0.9, the necessary eigenvalue of A is
r
= 0,
r
> 0 in expansion (1.2).
Theorem 10.1. (Lanczos phenomenon without using separability). Assume that
conditions (9.7), m 3,
0.1, (10.8)
m
2
1
1
4
0.095
r
take place. Then a number i, 1 i m, exists such that
[
i
[
_
10
6 r
_ 2
m2
1
2
. (10.9)
Remark If
2
m2
log
10
6
r
1,
then the righthand side of (10.9) approximately equals
1
1
m
log
10
6
r
.
Theorem 10.2. (Lanczos phenomenon using separability). Let m 2, [
l
[ 2 >
0 for l ,= r, k =
_
m1
2
3
,
2
r
m
_
13m
3
6
_
97
48
+ 1.8
m
1
_
+ 0.9
r
m
1
+
_
2
_
2
0.19
m
2
1
+
_
1 +
_
25m
24
_
T
k
_
1 +
2
1
2
_
1
_
_
49
48
+ 1.8
m
1
_
are satised. Then there exists such a number i, 1 i m, that [
i
[ < .
1
Recently L.Knizhnerman announced that
m can be omitted in Theorem 10.1.
7/1/2000 16:10 PAGE PROOFS parlkah3
216 Vladimir Druskin, Leonid Knizhnerman
Theorem 10.2 actually gives exponential decrease of the error down to the order
of about
1/3
. Exponential behavior at the rst stage can be proved by removing the
computer arithmetic terms. Vice versa, if the ideal arithmetic terms are negligibly
small, we, roughly, have the inequality
>
_
/, from what one can derive the
nal estimate
1/3
.
Evidently, theorems 10.1 and 10.2 are computer arithmetic analogs of theo
rems 7.1 and 7.2, respectively. They are illustrated by [5, tables 1, 2 and gure 1].
One can derive from [10] that in a favourable situation an eigenvalue in the Lanc
zos process can be nally approximated with error of order of
1/4
. Theorems 10.1
and 10.2, roughly speaking, improve this estimate up to
1/3
.
Acknowledgements
The authors thank the referees and Dr. M.Oristaglio for numerous remarks lead to
essential improvement of the manuscript.
REFERENCES
1. J. Cullum, R. A. Willoughby. The Lanczos phenomenon an interpretation based
upon conjugate gradient optimization. Linear Algebra and Applic., 29:63+, 1980.
2. V. L. Druskin, L. A. Knizhnerman. Using operational series in orthogonal polynomi
als for computation of functions of selfadjoint operators and background of Lanczos
phenomenon. Dep. at VINITI 02.03.1987, No. 1535 B87. (Russian).
3. V. L. Druskin, L. A. Knizhnerman. A spectral semidiscrete method for the numerical
solution of 3D nonstationary problems in electrical prospecting. Izv. Ac. Sci. U.S.S.R.,
Physics of Solid Earth, (8):63+, 1988. (Russian; translated into English).
4. V. L. Druskin, L. A. Knizhnerman. Two polynomial methods to compute functions of
symmetric matrices. J. Comp. Math. and Math. Phys., 29(12):1763+, 1989. (Russian;
translated into English
2
).
5. V. Druskin, L. Knizhnerman. Error evaluations for the simple Lanczos process when
calculating functions of symmetric matrices and eigenvalues. J. Comp. Math. and
Math. Phys., 31(7):970+, 1991. (Russian; translated into English).
6. V. Druskin, L. Knizhnerman. The Lanczos optimization of a splittingup method to
solve homogeneous evolutionary equations. J. Comp. Appl. Math., 42:221+, 1992.
7. V. Druskin, L. Knizhnerman. Evaluation for Krylov subspace approximation to inter
nal eigenvalues of large symmetric matrices and bounded selfadjoint operators with
continuous spectrum. SchlumbergerDoll Research, Research note, September 2 (1992).
8. R. A. Frishner, L. S. Tukerman, B. C. Dornblacer, T. V. Russo. A method of expo
nential propagation of large systems of sti nonlinear dierential equations. J. Sci.
Comp., 4:327+, 1989.
9. E. Gallopulos, Y. Saad. Ecient solution of parabolic equations by Krylov approxi
mation methods. SIAM J. Sci. Stat. Comp., 13(5):1236+, 1992.
10. A. Greenbaum. Behavior of slightly perturbed Lanczos and conjugategradient recur
rences. Linear Algebra and Applic., 113:7+, 1989.
11. Ed.: M. Abramowitz, J. Stegan. Handbook of Mathematical Functions. National Bureau
of Standards, Applied Mathematics series, No. 55, 1964.
12. A. Hordt, V. Druskin, L. Knizhnerman, K.M. Strack. Interpretation of 3 D ef
fects in deep transient electromagnetic soundings in the M unsterland area (Germany).
Geophysics, 57(9):1128+, 1992.
2
The U.S.A.s translation was called U.S.S.R. Comput. Math. and Math. Phys.
7/1/2000 16:10 PAGE PROOFS parlkah3
Krylov Subspace Approximation 217
13. V. O. Lokutsievskiy, O. V. Lokutsievskiy. Using the Chebyshev parameters for the nu
merical solution of some evolutionary problems. Inst. Appl. Math. Acad. Sci. U.S.S.R.,
Moscow, preprint No. 99, 1984. (Russian).
14. A. Nauts, R. E. Wyatt. New approach to many state quantum dynamics. Phys. Rev.
Letter, 51:2238+, 1983.
15. B. NourOmid, R. W. Clough. Dynamic analysis of structure using Lanczos co
ordinates. Earthquake Eng. and Structur. Dynamics, 12(4):565+, 1984.
16. B. NourOmid. Lanczos method for heat conduction analysis. Intern. J. Numer. Meth.
Eng., 24(1):251+, 1987.
17. C. C. Paige. The computation of eigenvalues and eigenvectors of very large sparse
matrices. Ph.D. Thesis, Univ. of London, 1971.
18. C. C. Paige. Error analysis of the Lanczos algorithm for tridiagonalizing of a symmetric
matrix. J. Inst. Math. Applic., 18(3):341+, 1976.
19. C. C. Paige. Accuracy and eectiveness of the Lanczos algorithm for the symmetric
eigenproblem. Linear Algebra and Applic., 34:235+, 1980.
20. C. C. Paige, M. A. Saunders. Solution of sparse indenite systems of linear equations.
SIAM J. Numer. Analys., 12(4):617+, 1975.
21. T. J. Park, J. C. Light. Unitary quantum time evolution by iterative Lanczos reduction.
J. Chem. Phys., 85:5870+, 1986.
22. B. N. Parlett, D. S. Scott. The Lanczos algorithm with selective orthogonalization.
Math. Comp., 33:217+, 1979.
23. B. N. Parlett. The symmetric eigenvalue problem. PrenticeHall, Englewood Cliffs,
N.Y., 1980.
24. B. N. Parlett. A new look at the Lanczos algorithm for solving symmetric systems of
linear equations. Linear Algebra and Applic., 29:323+, 1980.
25. H. D. Simon. The Lanczos algorithm for solving linear systems. Ph.D. Thesis, Univ.
of California, 1982.
26. H. D. Simon. Analysis of the symmetric Lanczos algorithm with reorthogonalization
methods. Linear Algebra and Applic., 61:101+, 1984.
27. A. V. Sobyanin. The roundo error analysis and stability in Lanczoslike methods.
Dep. Comput. Math. Acad. Sci. U.S.S.R., Moscow, preprint No. 87, 1985. (Russian).
28. H. A. van der Vorst. An iterative solution method for solving f(A)x = b using Krylov
subspace information obtained for the symmetric positive denite matrix A. J. Com
put. Appl. Math., 18(2):249: 1987.
7/1/2000 16:10 PAGE PROOFS parlkah3
Muito mais do que documentos
Descubra tudo o que o Scribd tem a oferecer, incluindo livros e audiolivros de grandes editoras.
Cancele quando quiser.