Discover millions of ebooks, audiobooks, and so much more with a free trial

Only $11.99/month after trial. Cancel anytime.

Special Matrices and Their Applications in Numerical Mathematics: Second Edition
Special Matrices and Their Applications in Numerical Mathematics: Second Edition
Special Matrices and Their Applications in Numerical Mathematics: Second Edition
Ebook493 pages5 hours

Special Matrices and Their Applications in Numerical Mathematics: Second Edition

Rating: 5 out of 5 stars

5/5

()

Read preview

About this ebook

This revised and corrected second edition of a classic book on special matrices provides researchers in numerical linear algebra and students of general computational mathematics with an essential reference.
Author Miroslav Fiedler, a Professor at the Institute of Computer Science of the Academy of Sciences of the Czech Republic, Prague, begins with definitions of basic concepts of the theory of matrices and fundamental theorems. In subsequent chapters, he explores symmetric and Hermitian matrices, the mutual connections between graphs and matrices, and the theory of entrywise nonnegative matrices. After introducing M-matrices, or matrices of class K, Professor Fiedler discusses important properties of tensor products of matrices and compound matrices and describes the matricial representation of polynomials. He further defines band matrices and norms of vectors and matrices. The final five chapters treat selected numerical methods for solving problems from the field of linear algebra, using the concepts and results explained in the preceding chapters.
LanguageEnglish
Release dateDec 1, 2013
ISBN9780486783482
Special Matrices and Their Applications in Numerical Mathematics: Second Edition

Related to Special Matrices and Their Applications in Numerical Mathematics

Titles in the series (100)

View More

Related ebooks

Mathematics For You

View More

Related articles

Reviews for Special Matrices and Their Applications in Numerical Mathematics

Rating: 5 out of 5 stars
5/5

1 rating0 reviews

What did you think?

Tap to rate

Review must be at least 10 words

    Book preview

    Special Matrices and Their Applications in Numerical Mathematics - Miroslav Fiedler

    Index

    1

    Basic concepts of matrix theory

    This introductory chapter is essentially a survey of well known definitions and general principles from matrix theory which will be useful in the sequel.

    1.1 Matrices

    A matrix of type m-by-n or, equivalently, an m × n matrix, is a rectangular array of ran numbers (usually real or complex) arranged in m rows and n columns (m, n positive integers):

    We call the number alk the entry of the matrix in the ith row and the kth. column. The set of m × n matrices with real entries will be denoted by Rm×n. Analogously, the set of m × n matrices with complex entries will be denoted by Cm×n. In some cases, entries can be polynomials, variables, functions, etc.

    In this terminology, matrices with only one column (thus, n = 1) will be called column vectors and matrices with only one row (thus, m = 1) row vectors. In such case, we write Rm instead of Rm×1 and vectors will always be column vectors, unless we specify otherwise.

    It is advantageous to denote the matrix by a single symbol, say, A, or [alk], C, etc.

    Two matrices A = [aik], B = [bik] are equal, written A = B, if and only if they are both m × n matrices for some positive integers m and n, and aik = bik for i = 1, …, m, j = 1, …, n.

    The importance of matrices is that one can introduce operations generalizing operations with numbers.

    Matrices of the same type can be added: If A = [aik], B = [bik] then A + B is the matrix [aik + bik]. The operation of addition is thus entrywise. We admit also multiplication of a matrix by a number (real, complex, a parameter, etc.): If A = [aik] and if α is a number (also called scalar), then αA is the matrix [αaik] of the same type as A.

    An m × n matrix A = [aik] can be multiplied by an n × p matrix B = [bkl] as follows: it is the m × p matrix C = [cil], where

    It is important to notice that the matrices A and B can be multiplied (in this order) only if the number of columns of A is the same as the number of rows in B. Also, the entries of A and B should be multiplicable. In general, the product AB is not equal to BA, even if the multiplication of both products is possible. On the other hand, the multiplication fulfils the associative law

    as well as (in this case, two) distributive laws:

    and

    whenever multiplications are possible.

    An m × n matrix is called square if m = n. We then say that it is square of order m.

    Of basic importance are the zero matrices of which all entries are zeros, and the identity matrices; these are square matrices, and have ones in the main diagonal and zeros elsewhere. Thus

    are identity matrices of order one, two and three. We denote zero matrices simply by 0, the identity matrices by 1, sometimes with a subscript denoting the order.

    The identity matrices of appropriate orders have the property that

    hold for any matrix A.

    Suppose now A = [aik] is an m × n matrix. Choose a subset M1 = {i1, i2, …, ir} of the set of row indices M = {1, 2, …, m}, and a subset N1 {k1, k2, …, ks} of the set of column indices N = {1, 2, …, n}. In addition, we assume that the integers ij and kl are ordered in such a way that i1 < i1 < … < ir and k1 < k2 < … < ks. The matrix

    is called a submatrix of A. We denote this submatrix by A(M1, N1). If A is a square matrix with M = N, then every square submatrix A(M1, M1) (with M1 = N1) is called a principal submatrix. The principal submatrix will be sometimes denoted also by A(M1).

    The most important type of submatrices are those whose sets of indices consist of consecutive integers. Such submatrices are called blocks. For example, in the matrix

    the submatrices

    are blocks. A matrix that is in this way subdivided by vertical and horizontal lines into blocks is said to be partitioned as indicated. The above matrix can thus be written as

    the matrices Pik have the property that those with first indices equal have the same number of rows and (those with) second indices equal have the same number of columns.

    An important property of partitioned matrices is that they behave to some extent as if the individual blocks were numbers. It is only necessary to preserve order in the calculation of products and to assure that the blocks have proper dimensions. Clearly, the addition of two matrices conformally partitioned into blocks can be performed in this manner. As regards multiplication, the following theorem holds (Exercise 1.1).

    Theorem 1.1.1 Let

    be matrices partitioned in such a way that for each i, i = l, …, r, and each k, k = 1, …, t, the number of columns in Pij equals the number of rows in Qjk, whenever j = 1, …, s.

    Then the product PQ can be partitioned into blocks Rik, i = 1, …, r, k = 1, …, t, where

    Example 1.1 Let us write the product

    with two-by-two blocks. Then

    i.e.

    as can be verified by direct multiplication.

    Let A = [aik] be an m × n matrix. The n × m matrix C = [cpq] whose entries are cpq = aqp, q = 1, …, m, p = 1, …, n is called the transpose of A; we denote it AT.

    The operation of transposition has two important properties:

    whenever AB is defined, and further

    for any matrix A.

    This notation will be used rather often, especially for column vectors. A column vector u with entries (components) u1, …, un will usually be written as [u1, …, un]T.

    We turn now our attention to special cases of square matrices. A square matrix A = [aik] of order n is diagonal if aik = 0 for i k, i, k = 1, …, n; the diagonal matrix with the diagonal entries a11, …, ann will be often written as diag (a11, …, ann). The matrix A is a lower triangular matrix if aik = 0 for i < k, i, k = 1, …, n, and it is an upper triangular matrix if aik = 0 for i < i, k = 1, …, n. For example, the first two of the matrices

    are lower triangular, the third is upper triangular, and the fourth matrix is a diagonal matrix which may be also written as diag (2, –1, 3). Clearly, the following statement holds.

    Theorem 1.1.2 The product of two lower triangular matrices of the same order is again a lower triangular matrix, and the product of two upper triangular matrices of the same order is upper triangular. The product of two diagonal matrices of the same order is again a diagonal matrix.

    A further important concept is that of a nonsingular matrix. We say that a square matrix A is nonsingular if there exists a matrix B such that AB = BA = 1. The matrix B is then called the inverse of A and is denoted by A–1. The following theorems can easily be proved.

    Theorem 1.1.3 Given a nonsingular matrix A, its inverse is determined uniquely. Furthermore, we have (A–1)–1 = A.

    Theorem 1.1.4 If A and B are nonsingular matrices of the same order, then their product AB is nonsingular as well and

    Hence, the inverse of the product is the product of the inverses in the reverse order.

    Theorem 1.1.5 If A is nonsingular, then its transpose AT is nonsingular as well and

    Hence, the inverse of the transpose is the transpose of the inverse.

    1.2 Determinants

    Determinants play a crucial role in distinguishing whether a square matrix is singular or nonsingular.

    The determinant det A of an n × n matrix A = [aik] is the number

    The summation is over all permutations P of the integers 1, …, n and the factor σ(P) is the sign of the permutation P. It is equal to +1 if the permutation P is even, and to – 1 if P is odd. The permutation

    is even if it has an even number of inversions, i.e. such pairs i, j for which i < j whereas ki > kj. An odd permutation has an odd number of inversions.

    We state two consequences of (1.5). Clearly, if l is an identity matrix, then

    The properties of permutations imply that for any square matrix A

    Let A = [aik] be an m × n matrix. Let M = {1, 2, …, m}, N = {1, 2, …, n} and let A(M1, M2) be a square submatrix of A (M1 ⊂ M, M2 ⊂ N, |M1| = |M2|).¹ Then, we are able to consider the determinant det A(M1, M2) of the submatrix; such a determinant will be called a minor of A. If A is a square matrix, then together with the above minor there exists the minor det A(M \ M1, N \ M2).² It is the determinant of the submatrix whose rows consists precisely of the rows of A (in the same order as they occur in A) that are not included in the submatrix A(M1, M2), and whose columns consist precisely of the columns of A that are not included in A(M1, M2), again in the same order as they are in A. The number (–1)s(M1)+s(M2) det A(M \ M1, N \ M2), where s(M1) is the sum of the row indices belonging to M1 and s(M2) is the sum of the column indices belonging to M2, is then called the cofactor of the minor det A(M1, M2). It is advantageous to denote the cofactor of det A(M1, M2) by codet A(M1, M2). If A is a square matrix, then the determinants of principal submatrices play an important role. It is customary to call them principal minors.

    Example 1.2 Let

    Then for M1 = {1, 3}, M2 = {2, 3} we have

    det A(M \ M1, N \ M2) = det[a21] = a21, s(M1) = 4, s(M2) = 5. Hence, codet A(M1, M2) = –a21.

    The following important theorem gives an expansion of the determinant of A by some of its rows (or columns).³

    Theorem 1.2.1 (Laplace expansion) Let M N = {1, 2, …, n} be a fixed set of indices which has m elements. Let A = [aij], i, j = 1, …, n, be a square matrix of order n. Then

    where the summation is over all the subsets Mj of the set of column indices that have m elements.

    Remark. Since

    the statement of the theorem may be rewritten as

    where the summation is over all the sets Mj N that have m elements.

    Applying Theorem 1.2.1 to the transpose of A and turning back to the original matrix, we obtain a similar formula giving the expansion of the determinant by its columns:

    where M is an arbitrary subset of N and the summation is over all the sets Mj N that have the same number of elements as M.

    As a corollary of the Laplace theorem we have the formulae (for A = [aik] of order n)

    where Aik is the cofactor of the entry (the expansion of det A by its ith row), and

    (the expansion of det A by its kth column).

    An immediate consequence of (1.10) and (1.11) is the next theorem.

    Theorem 1.2.2 Let A and B be square matrices of equal orders which differ only in the entries of one row or column. Let C be the matrix whose entries in this row or column are sums of the corresponding entries of A and B, the other entries being the same as in A and B. Then

    Assuming that A and B differ only in the first column and denoting the columns of A by A1, A2, …, An, and the columns of B by B1, A2, …, An, we can write the above formula as

    Repeatedly using this result, we establish a formula for the determinant of the sum of two matrices, A and B:

    Here, the summation is over i1 = 1, 2, i2 = 1, 2, …, in = 1, 2, and =

    .

    If we now use (1.9) to expand each determinant on the right-hand side by all its columns that are from A we arrive at the formula

    where the summation is over all pairs of subsets Mi, Mj, Mi N, Mj N, which have the same number of elements (|Mi| = |Mj|).

    A further consequence of (1.10) and (1.11) is the following theorem.

    Theorem 1.2.3 If a row or a column of a square matrix A contains only zero entries, then det A = 0.

    If two rows of A are equal, then applying the Laplace expansion 1.2.1 to these rows one obtains the following theorem.

    Theorem 1.2.4 If a square matrix A has two identical rows, then det A = 0. The same is true if A has two identical columns.

    Let us apply the theorem to the matrix B obtained from a square matrix A = [aik] of order n by replacing (for j i) the jth row of A by its ith. row (without changing the ith row). Expanding det B by its jth row we have, according to (1.10)

    The same procedure applied to columns gives

    The following important theorem can be proved by induction on r using the expansion (1.8) and Theorem 1.2.3.

    Theorem 1.2.5 Let A = [Aik], i, k = 1, …, r, be a partitioned square matrix whose diagonal blocks are square matrices and the blocks below diagonal are zero blocks: Aik = 0 for i > k (such a matrix is said to be upper block triangular), i.e.,

    Then,

    Remark. A similar formula holds for lower block triangular and for block diagonal matrices.

    The next theorem presents the celebrated Binet-Cauchy formula.

    Theorem 1.2.6 Let P be an m × n matrix and let Q be an n × m matrix, m < n. Set M = {1, 2, …, m}, N = {1, 2, …, n}. Then

    where the summation is over all the subsets Ni of N that have m elements.

    We do not give the proof here⁴ but we state an important corollary for m = n.

    Theorem 1.2.7 If A and B are square matrices of equal orders, then

    1.3 Nonsingular matrices. Inverse matrices

    We now turn to nonsingular matrices. First, we prove the following fundamental theorem.

    Theorem 1.3.1 A square matrix A = [aik] is nonsingular if and only if det A ≠ 0. In this case, the inverse A–1 = [αik] has the entries

    where Aki is the cofactor of aki in the matrix A.

    Proof. Suppose A has the inverse A–1. Then

    and by Theorem 1.2.7 and (1.6), we have

    Therefore,

    Conversely, suppose that det A ≠ 0. Define the matrix B = [αik] by αik = Aki/ det A. Then it follows that

    as well as

    by (1.10).

    Further, if i j, then

    according to (1.13).

    Equations (1.19) and (1.20) may also be written in the form

    A similar argument yields the relation

    Thus, B is the (unique) inverse of A

    Before proceeding further, we restate (1.18) as a theorem.

    Theorem 1.3.2 If A is a nonsingular matrix, then

    Theorem 1.3.1 enables us, at least theoretically, to find the solution of a system of linear equations written as

    in matrix notation, provided A is a nonsingular square matrix. Indeed, supposing x = [x1, …, xn]T is the column solution vector and left-multiplying the system by

    we get

    Here, Mj is the matrix obtained from A by replacing its jth column by b. This method of solving systems of linear equations is called Cramer’s rule. A consequence of this formula is the next theorem.

    Theorem 1.3.3 A system of n linear equations in n unknowns with a nonsingular matrix of coefficients has a unique solution.

    Theorem 1.3.4 Let A be a square matrix. If a matrix B satisfies the relation

    where l is the identity matrix, then B = A–1.

    Remark. Thus, if we want to prove that B is the inverse of A it is sufficient to verify (1.21). It is not necessary to show also that BA = l.

    Proof. By Theorems 1.2.7, 1.3.1, and by (1.6), A is nonsingular. The first column of B in (1.21) can be taken for the solution of the system

    But the first column of A–1 satisfies the equation as well. By Theorems 1.3.3, the first columns of B and A–1 are identical. The same can be proved for the remaining columns, and thus B = A

    Theorem 1.3.5 Let A be a square matrix. If the system Ax = 0 has a solution in which not all the unknowns are zero, then det A = 0.

    Proof. The system in question always has a (trivial) solution x = 0. If A were nonsingular, then

    Square matrices whose determinant is zero are called singular. Therefore, Theorem 1.3.5 gives a sufficient condition for A to be a singular matrix. Later we shall see that this condition is necessary as well (Theorem 1.5.3).

    The following theorem is an immediate consequence of the definition of a singular matrix and Theorem 1.2.7.

    Theorem 1.3.6 If one of the factors of a product of square matrices is singular, then the product is singular as well.

    Note also the following corollary of Theorem 1.2.5.

    Theorem 1.3.7 A block triangular matrix (lower or upper) is nonsingular if and only if all its diagonal blocks are nonsingular. In particular, a (lower or upper) triangular matrix is nonsingular if and only if all its diagonal entries are different from zero.

    We are now in a position to study the properties of the inverse of a nonsingular lower (or upper) triangular matrix.

    Theorem 1.3.8 If A = [aik] is a nonsingular lower (upper) triangular matrix, then its inverse A–1 is lower (upper) triangular as well.

    Proof. In view of Theorem 1.3.4 it is sufficient to show that there is a lower triangular matrix B satisfying

    The entries of such a matrix B, if it exists, fulfil the condition

    as well as

    Hence,

    However, from these equations the entries can be calculated successively by using i j = 1, 2, …, n – 1. Indeed, the right-hand side of (1.23) contains only the bkj’s with k j less than i j

    If B is the lower triangular matrix whose entries are the calculated bij’s, then it does hold AB = l, i.e. B = A–1. The corresponding statement for an upper triangular matrix is obtained by taking transposes and employing

    Remark. Note also the following two important consequences of (1.22) and (1.23):

    (1) If A = [aik] is a lower triangular matrix satisfying aii > 0 for every i and aij ≤ 0 for all i, j, i j, then all the entries of A–1 are nonnegative. (Naturally, the same is true for upper triangular matrices as well.)

    (2) If A is a nonsingular lower block triangular matrix, the proof applies without changes, except for writing

    (li are identity matrices) instead of (1.22) and considering aik and bik as blocks. Therefore, the theorem also holds for lower (or upper) block triangular matrices. Another corollary is the next theorem.

    Theorem 1.3.9 Lower triangular matrices of order n constitute a subset of square matrices which is closedwith respect to addition, multiplication, scalar multiplication and inversion (of nonsingular matrices). The same holds for upper triangular matrices.

    Moreover, carrying out the operations with triangular matrices we find that the kth diagonal entry of the result (k fixed, 1 ≤ k n) is, according to the matrix operation being performed, the sum, product or reciprocal of the kth diagonal entries of the operands and is not affected by other entries.

    1.4 Schur complement. Factorization

    We now introduce a further concept. Let

    be a partitioned matrix and A11 its nonsingular block. The Schur complement [A/A11] of the block A11 in A is then defined [34] to be the matrix

    (Here, A need not be square.)

    Theorem 1.4.1 Let A = [Aij], i, j = 1, …, r, be a partitioned matrix. Let the submatrices

    be nonsingular. Then there exist a lower block triangular matrix B having identity matrices as diagonal blocks, an upper block triangular matrix C having identity diagonal blocks, and a block diagonal matrix D such that

    Moreover, the matrices B, D, and C are determined uniquely by the conditions of the theorem; the diagonal blocks D1, …, Dr of D satisfy

    and D1, …, Dr–1 are nonsingular.

    Proof. First we prove the uniqueness assuming that D1, …, Dr–1 are nonsingular. Consider two possible factorizations of A, (and C are nonsingular, we may write

    By –1BD is upper block triangular. Hence both matrices are block diagonal, the former being equal to D–1B C–1 are identity matrices). Thus, D –1B = [Yik], i.e. Yjj = lj, Yik = 0 for i < k, and further

    for i > k, we have

    Since Di, i = 1, …, r–1 are nonsingular, we get Yik = 0 whenever i > k, Zik = 0 whenever i < k–1B = l C–1, and thus B , C .

    The existence of the factorization (1.26), and the validity of (1.27) and (1.28) will be proved by induction on r. For r = 1, both (1.26) and (1.27) hold. Let A have r > 1 block rows and assume the existence of (1.26) for matrices having r obtained from A by deleting its last block row and the last block column, has r with the diagonal blocks D1, …, Dr.

    Now write A in the partitioned form

    and try to find B, C, and D,

    such that A = BDC. This requirement gives

    The right-hand side is

    and it is therefore sufficient to satisfy the equations

    are nonsingular as well and the first two equations may be solved for C1r and Br1. We thus obtain

    The last of the equations is then satisfied by

    i.e., by Dr = [A]. Therefore, the factorization (1.26) exists and (1.28) holds also for k = r.

    At the same time we showed that D1, …, Dr

    The theorem has several corollaries.

    Theorem 1.4.2 Let A be a square matrix which is partitioned in the form

    where A11 is nonsingular. Then

    Hence A is nonsingular if and only if the Schur complement [A/A11] of the submatrix A11 in A is nonsingular. In such case, the inverse of the Schur complement [A/A11] is the submatrix of A–1 in the position complementary to that of A11.

    In other words, if A–1 is partitioned as was A above,

    then

    Proof. Using the formulae (1.27) and (1.28) of Theorem 1.4.1 with r = 2, we get

    But det A = det B det D det C by (1.26) and (1.17), where det B = det C = 1, det D = det D1 det D2 because of (1.15). Altogether, we have det A = det A11 det [A/A11], which proves (1.29).

    Now suppose that A, and thus also [A/A11] = D2, is nonsingular. The factorization (1.26) implies A–1 = C–1D–1B–1. But B–1, C–1, and D–1 are of the form

    so that

    This implies B22 = D–12 which is (

    If a square matrix A = [aik] of order n has all principal minors det A(Nk), Nk = {1, 2, …, k}, k = 1, …, n, different from zero, we call it strongly nonsingular. An immediate corollary of Theorem 1.4.1 is the next statement on strongly nonsingular matrices.

    Theorem 1.4.3 If A = [aik] is a strongly nonsingular matrix of order n, then there is a unique factorization A = BDC, where B and C are lower and upper triangular matrices, respectively, with ones along the main diagonal, and D is a diagonal matrix with diagonal entries d1, …, dn. Moreover, we have d1 = a11, dk = det A(Nk)/det A(Nk–1), k = 2, …, n.

    We shall now show another property of the Schur complement.

    Theorem 1.4.4 [8] Let

    be a partitioned matrix whose submatrices A11 and

    are both nonsingular. Then

    Proof. By Theorem 1.4.1, there exists a unique factorization A = BDC, where B is a lower block triangular matrix having three block rows and identity diagonal blocks, C is upper block triangular with analogous structure, and D is a block diagonal matrix with diagonal blocks D1, D2, D3, where

    On the other hand, A may be partitioned as

    and factored into factors with two block rows. According to (, where

    2 may be factored according to (1.26),

    so that

    It is easy to verify that the matrix

    written as a partitioned matrix with three block rows, has identity diagonal blocks and this is true of the matrix

    as well. Hence (1.33) is only another way of writing the factorization A = BDC3 =

    Enjoying the preview?
    Page 1 of 1