Introduction to Scientic Computing

using Matlab

Ian Gladwell Department of Mathematics Southern Methodist University Dallas, TX 75275 Warren E. Ferguson, Jr. James G. Nagy Department of Mathematics and Computer Science Emory University Atlanta, GA 30322

c 2004, Warren E. Ferguson Jr., Ian Gladwell & James G. Nagy

ii

Contents
1 Getting Started with Matlab 1.1 Starting, Quitting, and Getting Help . . . . . . . 1.2 Basics of Matlab . . . . . . . . . . . . . . . . . 1.3 Matlab as a Scientic Computing Environment 1.3.1 Initializing Vectors with Many Entries . . 1.3.2 Creating Plots . . . . . . . . . . . . . . . 1.3.3 Script and Function M-Files . . . . . . . . 1.3.4 Dening Mathematical Functions . . . . . 1.3.5 Creating Reports . . . . . . . . . . . . . . 2 Numerical Computing 2.1 Numbers . . . . . . . . . . . . . . . . . . 2.1.1 Integers . . . . . . . . . . . . . . 2.1.2 Floating-Point Numbers . . . . . 2.2 Computer Arithmetic . . . . . . . . . . 2.2.1 Integer Arithmetic . . . . . . . . 2.2.2 Floating-Point Arithmetic . . . . 2.2.3 Quality of Approximations . . . 2.2.4 Propagation of Errors . . . . . . 2.3 Examples . . . . . . . . . . . . . . . . . 2.3.1 Plotting a Polynomial . . . . . . 2.3.2 Repeated Square Roots . . . . . 2.3.3 Estimating the Derivative . . . . 2.3.4 A Recurrence Relation . . . . . . 2.3.5 Summing the Exponential Series 2.3.6 Euclidean Length of a Vector . . 2.3.7 Roots of a Quadratic Equation . 2.4 Matlab Notes . . . . . . . . . . . . . . 2.4.1 Single Precision Computations in 2.4.2 Special Constants . . . . . . . . 2.4.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 2 7 7 9 13 17 19 21 21 21 22 25 25 25 27 28 29 29 30 31 32 34 36 38 40 40 40 41 45 45 46 47 53 53 57 59 61 61

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Matlab . . . . . . . . . . . . . . . . . . . . .

3 Solution of Linear Systems 3.1 Linear Systems . . . . . . . . . . . . . . . . . . . 3.1.1 Singular and Nonsingular Systems . . . . 3.1.2 Simply Solved Linear Systems . . . . . . . 3.2 Gaussian Elimination . . . . . . . . . . . . . . . . 3.2.1 Outline . . . . . . . . . . . . . . . . . . . 3.2.2 Implementing Gaussian Elimination . . . 3.2.3 The Role of Interchanges . . . . . . . . . 3.3 Gaussian Elimination and Matrix Factorizations 3.3.1 LU Factorization . . . . . . . . . . . . . . iii

iv 3.3.2 P A = LU Factorization . . . . . . . . . . . The Accuracy of Computed Solutions . . . . . . . Matlab Notes . . . . . . . . . . . . . . . . . . . . 3.5.1 Diagonal Linear Systems . . . . . . . . . . . 3.5.2 Triangular Linear Systems . . . . . . . . . . 3.5.3 Gaussian Elimination . . . . . . . . . . . . 3.5.4 Built-in Matlab Tools for Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

CONTENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 67 70 70 72 75 78 81 81 82 84 88 91 94 95 96 101 105 110 110 116 121 124 129 129 131 133 136 137 141 143 143 145 146 147 149 149 152 153 156 159 160 164 166 169 172 174 174 175 177 179

3.4 3.5

4 Curve Fitting 4.1 Polynomial Interpolation . . . . . . . . . . . . . . . . . . . . . . 4.1.1 The Power Series Form of The Interpolating Polynomial 4.1.2 The Newton Form of The Interpolating Polynomial . . . 4.1.3 The Lagrange Form of The Interpolating Polynomial . . 4.1.4 The Error in Polynomial Interpolation . . . . . . . . . . 4.2 Polynomial Splines . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Linear Polynomial Splines . . . . . . . . . . . . . . . . 4.2.2 Cubic Polynomial Splines . . . . . . . . . . . . . . . . . 4.3 Chebyshev Polynomials and Series . . . . . . . . . . . . . . . . 4.4 Least Squares Fitting . . . . . . . . . . . . . . . . . . . . . . . 4.5 Matlab Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.1 Polynomial Interpolation . . . . . . . . . . . . . . . . . 4.5.2 Polynomial Splines . . . . . . . . . . . . . . . . . . . . . 4.5.3 Chebyshev Polynomials and Series . . . . . . . . . . . . 4.5.4 Least Squares . . . . . . . . . . . . . . . . . . . . . . . . 5 Numerical Integration 5.1 Integrals and the Midpoint Rule . . . . . . . . . . . . . . . 5.2 Quadrature Rules . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 Error in the Trapezoidal and Composite Trapezoidal 5.2.2 Interpolatory Quadrature . . . . . . . . . . . . . . . 5.2.3 Degree of Precision and Peanos theorem . . . . . . 5.3 Adaptive Integration . . . . . . . . . . . . . . . . . . . . . . 5.4 Gauss and Lobatto Rules . . . . . . . . . . . . . . . . . . . 5.4.1 Gauss Rules . . . . . . . . . . . . . . . . . . . . . . . 5.4.2 Lobatto Rules . . . . . . . . . . . . . . . . . . . . . . 5.4.3 Comparing Gauss and Lobatto rules . . . . . . . . . 5.5 Transformation from a Canonical Interval . . . . . . . . . . 5.6 Matlab Notes . . . . . . . . . . . . . . . . . . . . . . . . . 5.6.1 Explicitly Known, Simple Integrand . . . . . . . . . 5.6.2 Explicitly Known, Complicated Integrand . . . . . . 5.6.3 Integration of Tabular Data . . . . . . . . . . . . . . 5.6.4 Improper Integrals . . . . . . . . . . . . . . . . . . . 6 Root Finding 6.1 FixedPoint Iteration . . . . . . 6.1.1 Analysis of Convergence . 6.2 The Newton Iteration . . . . . . 6.2.1 Calculating Square Roots 6.3 Secant Iteration . . . . . . . . . . 6.4 Root Bracketing Methods . . . . 6.4.1 Bisection . . . . . . . . . 6.4.2 False Position Iteration . 6.5 Quadratic Inverse Interpolation . 6.6 Roots of Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

CONTENTS 6.6.1 Synthetic Division . . . . . . . . . 6.6.2 Conditioning of Polynomial Roots Matlab Notes . . . . . . . . . . . . . . . 6.7.1 Fixed Point Iteration . . . . . . . . 6.7.2 Newton and Secant Methods . . . 6.7.3 Bisection Method . . . . . . . . . . 6.7.4 The roots and fzero Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

v 181 184 188 188 190 191 193 199 201 201 205 207 208 209 210

6.7

7 Univariate Minimization 7.1 Search Methods not Involving Derivatives 7.1.1 Golden Section Search . . . . . . . 7.1.2 Quadratic Interpolation Search . . 7.2 Searches using the Derivative . . . . . . . 7.2.1 Cubic Interpolation Search . . . . 7.2.2 Secant Search . . . . . . . . . . . . 7.3 Matlab Notes . . . . . . . . . . . . . . .

A Computer Arithmetic 215 A.1 Project The Eect of Rounding Errors . . . . . . . . . . . . . . . . . . . . . . . . 215 B Solution of Linear Systems 219 B.1 Project - Solving Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 C Curve Fitting 221 C.1 Project The Error in Polynomial Interpolation Runges Example . . . . . . . . 221 C.2 Project Piecewise Cubic Interpolation . . . . . . . . . . . . . . . . . . . . . . . . 222 C.3 Project Polynomial Interpolation Newton Form . . . . . . . . . . . . . . . . . 222 D Numerical Integration 223 D.1 Project Performance of a Globally Adaptive Integration Code Matlab . . . . 223 E Root Finding E.1 Project E.2 Project E.3 Project E.4 Project E.5 Project Polynomial Root Finding . . . . . Root nding Keplers Problem Bracketing Roots . . . . . . . . . . False Position/Bisection . . . . . . False Position/Modied Secant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 225 225 226 226 227

F Univariate Minimization 231 F.1 Project Computing local minima and their locations . . . . . . . . . . . . . . . . 231

vi

CONTENTS

Preface
This text book is aimed at science and engineering majors whose wish to learn the ideas behind elementary scientic computation including understanding how some of the best numerical software works. We devote a chapter to each of six topics: A brief introduction to matlab. Computer arithmetic based on the IEEE 754 standard, and its eects on scientic computing The solution of systems of linear equations, and a discussion of residuals and conditioning Curve tting including interpolation, splines and least squares tting Numerical integration including an adaptive algorithm Solving a single nonlinear equation including computing square roots and roots of polynomials Minimization of a function of one variable These topics were chosen because they give a broad enough coverage to present the basic ideas of scientic computing and some of the theory behind it to students who have taken only the courses in univariate calculus in addition to high school mathematics. We do use partial derivatives at one point, but at the level which can be taught within the context of the application. In addition, because we use Matlab throughout the book, students must have some familiarity with basic matrix algebra. However, a linear algebra course is not prerequisite; instead, we introduce vectors, matrices and matrix algebra in Chapter 1, Getting Started with Matlab. If the book is used in a course for which linear algebra is not prerequisite, then we suggest going more slowly through this introduction. For each topic we provide sucient theory so that the student can follow the reasoning behind the choices made in quality numerical software and can interpret the results of using such software. We have provided a large number of problems, some theoretical and some computational. It is intended that students use Chapter 1 and the information on the Matlab HELP pages to start these computational problems. It is partly our aim to provide an opportunity for students to practice their programming skills in a real world situation. Some problems require students to code up and use the pseudocodes in the book. Many problems require graphical output; it is assumed that the simple plotting package available through the plot function in Matlab will be used. The base versions of the numerical software library assumed in this text is Matlab: Version 7.0 (Release 14). This textbook has been developed over a number of years. It has been used in the classroom a number of times by all three authors and also by other teachers. The course has been aimed at all students who have met the prerequisites. They have included engineering, computer science, management science, physics, chemistry, economics, business and mathematics majors. The computer projects have all been used in assessment as have a number of the problems. The chapters in this textbook can be taught in almost any order as many of the topics are almost independent. And, we have experimented with teaching the chapters in a variety of orders. It is best if the Curve Fitting chapter is taught before the Numerical Integration chapter, and if the Root Finding chapter is taught before the Univariate Minimization chapter. Sometimes, we have started the course by teaching from the Computer Arithmetic chapter, especially since its examples and vii

viii

CONTENTS

computer projects make a good starting point for student assessment, but equally often we have started the course by teaching the Solution of Linear Systems chapter and occasionally we have started by rst teaching the Root Finding chapter. Each chapter is divided into a number of sections, and many of the sections are divided into subsections where this is necessary to provide a structure that is easy for the instructor and the student. Figures and tables are numbered through the chapter but all other numbered entities (denitions, examples, problems, remarks and theorems) are numbered through the sections in which they appear. Acknowledgements The authors wish to thank their friends and colleagues who have read prepublication versions of this text and, in some cases, used it in the classroom. In particular we wish to thank Michele Benzi of Emory University, Wayne Enright of the University of Toronto, Graeme Fairweather of the Colorado School of Mines, Leonard Freeman of the University of Manchester, Peter Moore of Southern Methodist University, David Sayers of the Numerical Algorithms Group Ltd., and Wen Zhang of Oakland University. The Authors Dr. Warren Ferguson was awarded a BS in Physics from Clarkson University in 1971 and a Ph.D. in Applied Mathematics from the California Institute of Technology in 1975. He was a member of the faculty in the Department of Mathematics at the University of Arizona from 19751980 and at Southern Methodist University, Dallas, from 19802000. In 2000, he joined the Intel Corporation. Dr. Ferguson is the author of many published research papers in applied mathematics and scientic computing. His particular interests are in the numerical solution of partial dierential equations and in computer arithmetic. Dr. Ian Gladwell was awarded a BA in Mathematics from the University of Oxford in 1966, an M.Sc. in Numerical Analysis and Computing in 1967 and a Ph.D. in Mathematics in 1970 from the University of Manchester. From 19671987 he was a member of the faculty in the Department of Mathematics at the University of Manchester. In 1986, he was a Royal Society Industrial Fellow at The Numerical Algorithms Group, Oxford. Since 1987 he has been a member of the faculty in the Department of Mathematics at Southern Methodist University, Dallas. He is the author of many published research papers on scientic computing and on mathematical software, and of publicly available numerical software. His particular research interests are in the numerical solution of ordinary dierential equations. Dr. James Nagy was awarded a BS in Mathematics in 1986 and an MS in Computational Mathematics in 1998 from Northern Illinois University. In 1991 he was awarded a Ph.D. in Applied Mathematics from North Carolina State University. He was a Postdoctoral Research Associate at the Institute of Mathematics and Its Applications, University of Minnesota, from 19911992, and a member of the faculty in the Department of Mathematics at Southern Methodist University, Dallas, from 19921999. Since 1999 he has been a member of the faculty in the Mathematics and Computer Science Department at Emory University, Atlanta. In 2001 he was selected to hold the Emory Professorship for Distinguished Teaching in the Social and Natural Sciences. He has published many research papers on scientic computing, numerical linear algebra, inverse problems, and image processing. His particular research interests are in the numerical solution of large scale structured linear systems.