Inversion of Modified Symmetric Matrices

GERttARD ZIELKE
Martin-Luther-Universiffit Halle-Wittenberg,* German Democratic Republic

ABSTRACT. The problem to be solved is the computation of the inverse of a modified symmetric matrix if the inverse of the original symmetric matrix is Mready known. By "modified" it is meant that the two symmetric matrices differ from each other in two symmetric elements or columns ~nd rows. For both cases new identities and ALGOLalgorithms are derived. WORDSANDPHRASES: ALGOL procedures, inverse matrix adjustment, inversion, inverting modified matrices, matrix identities, matrix inversion, matrix perturbation, modified matrix, reciprocal matrix, symmetric matrix, symmetric matrix change
KEY

CR CATEGORIES :

5.14

1.

Introduction

Let A, B, and C be square matrices of order n, and further, let A and B be nonsingular and C arbitrary at first. The relation A = B + C is assumed to hold. Assuming B -1, the inverse of B, is known, the matrix A -1 is to be computed by means of B -1, Such a problem arises, e.g. when in a matrix already inverted additional modifications of the original matrix m u s t be considered. Practical cases of this kind occur where the data in a set of equations to be solved are to be corrected or varied, or, when a computer is used, where the input data are falsified because of transmission error. Often the problem will involve changing single elements or a few columns or rows. Since in most cases the correction matrix C is singular, i.e. this matrix has a rank r < n, it is natural to represent it as a product of two rectangular matrices, or more generally as C = U S V ' , where U and V are n X r matrices (r < n), S is a nonsingular r r matrix, and V t is the transpose of V. The outlined problem can be solved b y the following identity:
( B + U S V ' ) -1 = B -~ - B - 1 U ( S -1 + V ' B - I U ) - I V ' B -~.

(1)

Up to now this identity has been attributed to Woodbury [12]; see also Householder [8, pp. 123,124]. Apparently it seems to have escaped notice t h a t the identity (1) was found as early as 1944 by Duncan [5, p. 666, eq. (4.8) or (4.10)], and also was published in 1946 by G u t t m a n [6, p. 342, eq. (13)]. Of course, in both papers this formula cannot be easily discovered because it is only a by-product. Moreover, in the first paper the formula cannot be recognized immediately because of abbreviations. The well-known formulas of Sherman and Morrison for changing a column or a row [10] and for changing a single element [11] (see also Sherman [9]) are special cases of the general formula (1). * Institut ffir Numerische Mathematik

Journal of the Association for Computing Machinery, Vol. 15, Nn. 3, July 1968, pp. 402-408

Inversion of Modified Symmetric Matrices By a simple transformation from (1) we get (B + U S V ' ) -~ = B -~ - B - ~ U ( A + S V ' B - ~ U ) - I S V ' B -~,

403

(2)

in which the assumption of nonsingularity of S may now be dropped. /r denotes the unit matrix of order n. Both identities (1) "rod (2) can be verified directly. It is well known that the inversion of symmetric matrices by one of the conventional methods, e.g. the Gauss method, requires only about half the number of operations required for inversion of nonsymmetric matrices. There is a great probability that by use of modified formulas based on identities (1) and (2) a similar decrease may be expected. This is investigated in detail in the following sections. For simplicity we derive only formulas for simultaneous change of a column and a row which are symmetric to each other, as well as for simultaneous changing of two gymmetric elements. More complicated modifications can always be reduced to multiple application of both the simple cases, which are considered in what follows.

2.

Simultaneous Change of a Column and a Row Which Are Symmetric to Each Other

As a starting formula, one of the two identities (1) or (2) is used. It is not convenient to simplify formulas (1) and (2) to the symmetric case at this point. The inclusion of the symmetry yields an even more complicated form, but the evaluation will be shorter, because it is no longer necessary to calculate all the elements of the inverse; only the (n 2 + n ) / 2 elements of a triangular matrix must be calculated. The assumption is, of course, that in addition to B the correction matrix C = U S V ' will be symmetric. The most convenient way is to put S = I and U = V first. Indeed, by this choice the symmetry is guaranteed, but U = V has the disadvantage that not only are the two rectangular matrices U and U' (or in our special case the column u and the row u') added to B, but also the whole square matrix UU'. In the special case of simultaneous change of a column and its symmetric row, which we now investigate, the following consideration will prove successflfl. Each correction matrix C, which consists of an arbitrary column and row and otherwise only zeros, has at most rank 2. Hence it must be possible to represent C as a product of two rectangular matrices of types n X 2 and 2 X n. Indeed, if the ith column and row of B are to be changed, say, by addition of column u and a row u', then in the following identity (3) which results from (1) or (2) b y S = I, (B -+- UV') -~ = B -~ - B - ~ U ( A -t- V'B-~U)-~V'B -~, one can put U = (u,e~) and V = (e~,u). (4) (3)

The correction matrix then has the form C = UV' = ue/ + e~u'.

Journal of the Association for Computing Machinery, Vol. 15, No. 3, July 1968

404

GERHARD ZIELKE

Because in the product C = UV', c~ = 2u~, i.e. the modification of b, by u~ is executed twice; therefore an additional adjustment must be performed. From the beginning this correction c~ - u~ can be incorporated into the modification formula, so that a true identity is obtained again. It is surprising that the additional amount of computation is only a single addition. In order to work in the correction it is sufficient to modify (4) for U, viz.
U = (u u~e~, e~).

(5)

Thus the correction matrix finally is obtained in the form

UV' = uei' -{- e~u' - uieiei'.

By putting this together with U and V into (3) one obtains

( B -4- ue,' -f- e,u' -- u,e,e,') -~ = B -~ -- B -~ (u - u~e, ' e~) Q-I [e~'~ \u'/B-~'

(6)

where
Q = 12"~ u' B - ' ( u - uie,, e~) = ~u,B_l u _ u,u'B-~ 1 + u'B~,']"

Corresponding to Bodewig's notation [1, 2] in the foregoing formula, B-~ ~ = B-~e~ means the ith column of B -~ and B ~ ~ = ei'B -~ the ith row of B -1. Moreover the element (B-~), is denoted by fl~. The inverse of the second-order matrix Q,
Q= ( q , q2], q3 q4/

can be written in the following simple manner:

Q-~ = ~ -q~
- - q2

q~

w h e r e d = qffl~- q2q~andq~ = 1 - t - B ~ ) u - u d . , q~. = ~ . , q~ = u ' B - ~ u uiu'B~ ~, q~ = 1 - l - u ' B ~ ~. From formula (6) follows by matrix multiplications and suitable collecting of terms the final formula for the simultaneous change of a column and a row which are symmetric to each other,
l -1 . (B -4- ue~' + e~u' - u~e~e~')-1 = B -1 - it [B-~(k,v' -4- k ~B,. ) -4- v ( k J -4- k,B~t)],

(7)

w h e r e v = B-lu, w = u ' B ~ 1 = B~lu, kl = 1 + w, k2 = - ( u ' v + u~), k3 = -~,, d = kl 2 + k 2 ~ . . The symmetry of the right-hand side of (7) becomes obvious because the matrices vv', B~IB~ 1 , and kl(B-~lv ' A- vB~ 1) are symmetric. In the case of a nonsingular matrix B + UV' the determinant d always is different from zero. In order to prove this theorem the general case of the identity (2) may be considered, where the analogous theorem is valid; it says that the determinant det ( I + S V ' B - ' U) is always different from zero if the matrix B + U S V ' is nonsingular. This statement can be verified easily by representing the matrix

Journat of the Assoclatlon for C o m p u t i n g

Machlnery, Vol. 15, No. 3, July 1968

Inversion of Modified Symmetric Mat~'ices

405

in two different ways as a product of two block triangular matrices, viz.

By the transition to determinants one obtains d e t ( B + U S V ' ) = det B. det(I + S V ' B - ~ U ) . By assumption, B and B + U S V ' are nonsingular; consequently d e t ( I + SV~B-~U) must be different from zero. It should be noted that the formula for a symmetric changing of a column and a row without consideration of the double diagonal element is nearly the same as formula (7). In the expression for/c: = - ( u J + u~) only the term u~ is lacking. In other words, the two formulas differ only in one addition.
3. Simultaneous Change of Two Symmetric Elements

If in the matrix B only the two symmetric elements blj and b~i = b~] (i ~ j ) are to be changed, say, by addition of a single number c, then in (4) and (5) one has to put u -- ce~ and u~ = 0. Then the correction matrix takes the form
\ce/]= c(e~e,' + e~e/).

By putting this into the identity (3) one can proceed in the same way as in Section 2 and derive a formula similar to (7). This formula, however, can also be directly obtained from (7) by specialization. There, one has to put v = cB-~ei = cB~ ~ , w = ce~B.i = cflj~ = c ~ , k~ = 1 + cfl~, k2 = - c ~ e j B j 1 - c ~ f l j j , ks = B , , d = kl2 + k2~,. Then from (7) follows
-

[B + c ( e ~ e / + e~.ed)]-1 = B -~

1 [B_I (k~cBi. + k~B~ ~) + cS-~'(k3cBT. ~ + tc~S~)]. -1 .~

By factoring out the number c, also, one obtains the final formula for the simultaneous change of two symmetric elements,
c [B + c(eie/ + eie/)1-1 = B -1 -- d [BT~(hlB-~.I + h2B71) + B. (h3B~. + hlBT.1)], -1 -1 (8)

wherehl = 1 + c[3ii , h2 = - c S ~ j , h3 = -c~i~ , d = hi 2 - h2h3. The symmetry of the right-hand side of (8) is obvious when it is seen that by transition from i to j and vice versa, h2 and ha have to be interchanged.
~. A L G O L Procedures

With regard to computation of inverses according to formulas (7) and (8), two ~-LGOL procedures were developed, viz. the procedure INVSYM for the simultaneous changing of a column and a row which are symmetric to each other, and the procedure INVSYM 2 for the simultaneous changing of two symmetric elements. Both the procedures were tested on the Zeiss computer ZRA 1 of the Institut ftir Numerische Mathematik of the Martin-Luther-Universiti~t Halle-Wittenberg, and are prepublished in [13] and [14], respectively.
procedure INVSYM v a l u e n, i; i n t e g e r

(n, i, u, a, b); n, i; a r r a y u, a, b;

Journal of the Association for Computing Machinery, Vol, 15, No. 3, July 1968

406

GERHARD ZIELKE

c o m m e n t [ N V S Y M c o m p u t e s t h e i n v e r s e A -~ = a of a n o n s i n g u l a r s y m m e t r i c n t h - o r d e r m a t r i x A = B + ue~' + eiu ~ - u~eses' w h i c h arises f r o m ~ s y m m e t r i c m ~ t r i x B b y a d d i n g a rowu'= (ut , u2 , . . . , u s , . . . , un) to the i t h row ~nd ~ c o l u m n u t o t h e i t h c o l u m n . In t h i s p r o c e s s t h e d i a g o n a l e l e m e n t us is a d d e d o n l y once. T h e i n v e r s e m a t r i x B-~ = b is ass u m e d to be k n o w n ; b e g i n i n t e g e r k, l; r e a l k l , k2, k3, w, d, t; a r r a y r, S, V[1 :n]; f o r k := 1 s t e p 1 u n t i l n d o b e g i n t : = 0; for l := 1 step 1 until n do t := t + b[k, l] X u[/];

v[k] :=

end; W := 0; f o r k := 1 s t e p 1 u n t i l n d o w := w + u[k] X b[k, i]; kl := 1 + w; k3 : = - b [ i , i ] ; t:=O; f o r / ~ := 1 s t e p 1 u n t i l n d o t := t + u[k] X v[k]; k2 := - t -- u[i]; d := k l ~ 2 + k2 X b[i,i]; kl := k l / d ; k2 := k2/d; k3 := k3/d; f o r k := 1 s t e p l u n t i l n d o b e g i n r[k] : = kl X v[k] + k2 X b[i, k]; s[k] := k3 X v[k] + kl X b[i, k] end; f o r / c := i s t e p 1 u n t i l n d o for l := 1 s t e p 1 u n t i l k d o a[]c, l] := all, k] : = b[k, l] - b[k, i] X r[l] - v[k] X sill e n d I N VS Y M p r o c e d u r e I N V S Y M 2 (n, i, j , c, a, b); v a l u e n, i, j , c; i n t e g e r n , i , j ; r e a l c; a r r a y a, b; e o n ~ n t e n t I N V S Y M 2 c o m p u t e s t h e i n v e r s e A -1 = a of a n o n s i n g u l a r s y m m e t r i c n t h - o r d e r m a t r i x A = B + c(ese/ + e~ei') w h i c h arises f r o m ~ s y m m e t r i c m a t r i x B b y a c h t m g e c in t w o s y m m e t r i c e l e m e n t s Bsi ~nd Bji = Bii (i # j ) . T h e i n v e r s e m ~ t r i x B -1 = b is a s s u m e d to be k n o w n ; b e g i n i n t e g e r / % l; r e a l h l , h2, h3, d, a r r a y r, s[1 :n]; hl : = 1 + c X b[i,j]; h2 := - c X b [ j , j ] ; h3 := - c X b[i,i]; d : = h 1 ~ 2 h2 X h3; d := c/d; h l := hl X d; h2 := h2 X d; h3 := h3 X d; f o r k := 1 s t e p 1 u n t i l n d o b e g i n r[k] := h l X b[j, k] + h2 X b[i, k]; s[k] := h3 X b[j, k] + hl X b[i, k] end; f o r /z := 1 s t e p 1 u n t i l n d o for 1 := 1 s t e p 1 u n t i l k d o a[k, l] := a[l, k] := b[k, l] -- b[k, i] X rill -- b[k, j] X s[l] end INVSYM

5. Estimation of Amount of Calculation

In order to estimate the amount of calculation in matrix problems, especially in comparison with other methods, it is sufficient to consider the number of multiplications (divisions), as shown in the following. (a) Simultaneous change of a column and a row which are symmetric to e~ch other:

Inversion of Modified Symmetric Matrices

Melhod
F o r m u l a (7) of t h i s p a p e r (Zielke) Sherman-Morrison

407
Number of multiplications 2n~ -~ O(n) ~n 2 4" O(n)

(b) Simultaneous change of two symmetric elements:

Method
F o r m u l a (8) of t h i s p a p e r (Zielke) Sherman-Morrison

Number of multiplications n 2 -b O(n) ~n 2 ~- O(n)

6.

Bibliographic Notes

T h e first remark (without formula) that symmetric modifications in symmetric matrices lead to simpler formulas seems to be due to Sherman in 1951 (published :irL [9]). A formula relation for the adjustment of the inverse if, in particular, two symmetric elements are changed was probably first described by Bodewig [1, p. 103], and [2, p. 34]. Bodewig obtained this relation by the double use of the Shermani%iorrison formula [11] for changing a single element. As Bodewig himself notes, h i s formula in general will be less useful in practice than the Sherman-Morrison formula, because it involves five dyadic products, i.e. 5n 2 multiplications, whereas t h e double application of the latter formula requires only two dyadic products. I n the second edition of his Matrix Calculus [2], Bodewig even omitted the discussion on the modification of symmetric matrices. The contradiction included in the l a s t sentences concerning the amount of calculation is quickly clarified, viz. by a suitable collecting of terms Bodewig's formula can be further simplified so as to become identical with formula (8) of this paper, thus also containing two dyadic products only. A detailed discussion on modifying symmetric matrices was given by DOck [3]; see also [4]. He obtained the identities for the symmetric changing of two elements a s well as a column and a row (without consideration of the double diagonal element, however) from the general modification formula (3). In this form, however, these identities are not well suited to the computational process of the inversion because, in each case, they involve four dyadic products, i.e. 4n ~ multiplications. II~ order to state algorithms for the calculation DOck derived the corresponding formulas by double application of the Sherman-Morrison formula [11, 7] for changi n g a single element respectively for a column or row [10]. The amount of ealeulat i o n for Dtick's algorithm for the changing of two symmetric elements is about the s a m e as that according to formula (8) of this paper, namely n 2 --F O(n) multiplications. The amount of calculation of DOck's algorithm for the changing of a column 5 a n d a row which are symmetric to each other is ~n 2 --~ O(n) multiplications compared with 2n2 -~- O(n) multiplications by formula (7) of this paper. The two I)(iek algorithms have the disadvantage that they can break down even in the nonsingular case because of division by zero. The identities derived in this paper concerning the simultaneous symmetric change of two elements as well as a column and a row seem to be new in this form, arid the amount of calculation appears to be optimal. A(~KNOWLEDGMENT. The author wishes to express his appreciation to Prof. Dr. E. Sehincke, Director of the Institut fiir Numerische Mathematik, for his interest and
Journal of the Association for Computing Maohinery, Vol. 15, No. 3, July 1968

408

GERHARD

ZIELK]~

generous c o m m e n t s to t h i s paper. T h a n k s are also due to Prof. C. C. Cotlieb, T o r o n t o , a n d R. Schreyer, Halle, for critical r e a d i n g of t h e m a n u s c r i p t .

REFERENCES 1. BODEWIG, E. Zum Matrizenkalk~il. Proc. Koninkl. Ned. Akad. Wetenschap. /A}, 58 (1955), 95-106 (Indagationes Math. 17 (1955)). Matrix Calculus. North-Holland Publishing Co., Amsterdam, 1956; 2nd ed., 1959. 2. - - . 3. D~)cx, W. Iterative Verfahren und Ab(~nderungsmethoden zur Inversion yon Matrizen. Habilitationsschrift, TH Karl-Marx-Stadt, Germany, 1964. 4. . Inversion symmetrischer Matrizen durch Ab~nderungsmethoden. Z. Angew. Math. Mechanik 46 (1966), Sonderheft GAMM-Tagung Darmstadt 1965, pp. T41-T43. 5. DUNCAN,W . J . Some devices for the solution of large sets of simultaneous linear equations. Phil. Mag. /7}, 35 (1944), 660-670. 6. GUTTMAN, L. Enlargement methods for computing the inverse matrix. Ann. Math. Statist. 17 (1946), 336-343. 7. HERNDON,J . R . Adjust inverse of a matrix when an element is perturbed. Comm. ACM 4, 3 (March 1961), 180, Algorithm 51. 8. HOUSEHOLDER,A.S. The Theory o] Matrices in Numerical Analysis. Blaisdell Publishing Co., New York, 1964. 9. SHERMAN, J. Computations relating to inverse matrices. In Simultaneous Linear Equations and the Determination of Eigenvalues, Nat. Bur. Standards Appl. Math. Series, No. 29, US Gov't. Printing Off., Washington, D. C., 1953, pp. 123-124. 10. ANDMORRISON,W . J . Adjustment of an inverse matrix corresponding to changes ia the elements of a given column or a given row of the original matrix. Ann. Math. Statist. 20 (1949), 621. 11. - - AND - - - - . Adjustment of an inverse matrix corresponding to a change in one element of a given matrix. Ann. Math. Statist. 21 (1950), 124-127. 12. WOODBURV,M.A. Inverting modified matrices. Statist. Res. Group, Mem. Rep. No. 42, Princeton U., Princeton, N. J., June 14, 1950. 13. ZIELEE, G. ~nderung der Inversen einer symmetrischea Matrix bei symmetrischer Zeilen- und Spalten~nderung. Computing 8, 76-77 (1968), "Algorithmus," p. 6. 14. Adjustment of the inverse of a symmetric matrix when two symmetric elements are changed. Comm. ACM 11, 2 (Feb. 1968), p. 118, Algorithm 325.
-

RECEIVED NOVEMBER, 1967; RSVISED JANUARY, 1968

Journalof the Association

for Computing Machinery,

Vol. 16, No. 8, July 1968