Pre-Engineering 220

Introduction to MatLab &

Scientific Programming
J Kiefer

Gottfried W. Leibnitz:

It is unworthy for excellent men to lose hours like slaves in the labour of calculation
which could be safely relegated to anyone else if machines were used.
2013

Table of Contents
Table of Contents.............................................................................................................................1
I.
Introduction............................................................................................................................3
A.
Numerical Methods or Numerical Analysis....................................................................3
1.
Numerical Analysis......................................................................................................3
2.
Newtons Method for Solving a Nonlinear Equationan example............................3
3.
Series............................................................................................................................5
4.
Error.............................................................................................................................5
B.
Programming...................................................................................................................6
1.
Program Design...........................................................................................................6
2.
Branching.....................................................................................................................6
3.
Loops...........................................................................................................................6
4.
I/O................................................................................................................................6
5.
Precision Issues............................................................................................................7
6.
Debugging....................................................................................................................7
II. MatLab.................................................................................................................................8
A.
Program Features.............................................................................................................8
1.
Commands...................................................................................................................8
2.
Arrays........................................................................................................................10
3.
Array Operations........................................................................................................11
B.
Files................................................................................................................................11
1.
m-files........................................................................................................................11
2.
Script files..................................................................................................................12
3.
Function files.............................................................................................................12
C.
Plots...............................................................................................................................13
1.
Two Dimensional Graphs (pp. 133-158.....................................................................13
2.
Three Dimensional Graphs........................................................................................13
D.
Programs........................................................................................................................14
1.
Branches....................................................................................................................14
2.
Loops (pp. 190-200)..................................................................................................16
3.
Input/output (pp 114-118)..........................................................................................17
III. Numerical Solution of Nonlinear Equations........................................................................18
A.
Non-Linear Equationsone at a time...........................................................................18
1.
The Problem...............................................................................................................18
2.
Bisection....................................................................................................................18
3.
Newtons Method or the Newton-Raphson Method..................................................19
4.
Secant Method...........................................................................................................20
5.
Hybrid Methods.........................................................................................................20
B.
Systems of Nonlinear Equations....................................................................................21
1.
Newton-Raphson.......................................................................................................21
2.
Implicit Iterative Methods.........................................................................................21
IV. Linear Algebra......................................................................................................................23
A.
Matrix Arithmetic..........................................................................................................23
1.
Matrices.....................................................................................................................23
2.
Addition & Subtraction..............................................................................................23
3.
Multiplication............................................................................................................23

4.

Inverse Matrix............................................................................................................24
B.
Simultaneous Linear Equations.....................................................................................25
1.
The Problem...............................................................................................................25
2.
Gaussian Elimination.................................................................................................25
3.
Matrix Operations......................................................................................................26
4.
Gauss-Jordan Elimination..........................................................................................28
C.
Iterative Methods...........................................................................................................30
1.
Jacobi Method............................................................................................................30
2.
Gauss-Seidel Method.................................................................................................31
D.
Applications...................................................................................................................32
1.
Electrical Circuit........................................................................................................32
2.
Truss System..............................................................................................................33
V. Interpolation and Curve Fitting............................................................................................34
A.
Polynomial Interpolation...............................................................................................34
1.
Uniqueness.................................................................................................................34
2.
Newtons Divided Difference Interpolating Polynomial...........................................35
B.
Least Squares Fitting.....................................................................................................38
1.
Goodness of Fit..........................................................................................................38
2.
Least Squares Fit to a Polynomial.............................................................................38
3.
Least Squares Fit to Non-polynomial Function.........................................................40
MatLab Sidelight Number One..............................................................................................41
1.
Polynomials...............................................................................................................41
2.
Curve Fitting & Interpolation....................................................................................42
VI. Integration............................................................................................................................43
A.
Newton-Cotes Formul.................................................................................................43
1.
Trapezoid Rule...........................................................................................................43
2.
Extension to Higher Order Formul.........................................................................44
B.
Numerical Integration by Random Sampling................................................................47
1.
Random Sampling.....................................................................................................47
2.
Samples of Random Sampling...................................................................................48
3.
Integration..................................................................................................................48
MatLab Sidelight Number Two.............................................................................................53
1.
Nonlinear Equations..................................................................................................53
2.
Integration..................................................................................................................53
VII.
Ordinary Differential Equations....................................................................................55
A.
Linear First Order Equations.........................................................................................55
1.
One Step Methods......................................................................................................55
2.
Error...........................................................................................................................56
MatLab Sidelight Number Three...........................................................................................58
1.
First Order Ordinary Differential Equations (ODE)..................................................58
B.
Second Order Ordinary Differential Equations.............................................................59
1.
Reduction to a System of First Order Equations.......................................................59
2.
Difference Equations.................................................................................................60

I.
A.
1.

Introduction
Numerical Methods or Numerical Analysis
Numerical Analysis

a.
Definition
Concerned with solving mathematical problems by the operations of arithmetic. That is, we
manipulate (

/ ,, , etc.) numerical values rather than derive or manipulate analytical

mathematic expressions (

d
, dx, e x , x b , ln x , etc.).
dx

We will be dealing always with approximate values rather than exact formul.
b.
History
Recall the definition of a derivative in Calculus:

df
f
lim
g ( x) ,
dx x 0 x

where f f ( x 2 ) f ( x1 ) and x x 2 x1 . We will work it backwards, using

df
f

.
dx x

In fact, before Newton and Leibnitz invented Calculus, the numerical methods were the methods.
Mathematical problems were solved numerically or geometrically, e.g., Kepler and Newton with
their orbits and gravity. Many of the numerical methods still used today were developed by
Newton and his predecessors and contemporaries.
They, or their computers, performed numerical calculations by hand. Thats one reason it
could take Kepler so many years to formulate his Laws of planetary orbits. In the 19th and
early 20th centuries adding machines were used, mechanical and electric. In business, also,
payroll and accounts were done by hand.
Today, we use automatic machines to do the arithmetic, and the word computer no longer refers
to a person, but to the machine. The machines are cheaper and faster than people; however, they
still have to be told what to do, and when to do itcomputer programming.
2.

Newtons Method for Solving a Nonlinear Equationan example

a.
Numerical solution
Lets say we want to evaluate the cube root of 467. That is, we want to find a value of x such
that x 3 467 . Put another way, we want to find a root of the following equation:
f ( x) x 3 467 0 .

If f(x) were a straight line, then

df x xo
f ( x1 ) f ( xo )
x1 xo 0 .
dx
In fact, f ( x1 ) 0 , but lets say that f ( x1 ) 0
and solve for x1.

x1 xo

f x1 f xo
f xo
xo
df ( xo )
f xo .
dx

df ( x xo )
.
dx
Having now obtained a new estimate for the
root, we repeat the process to obtain a sequence
of estimated roots which we hope converges on
the exact or correct root.

Note that we are using f ( xo )

f x1
f x1
f x2
x3 x 2
f x2
x 2 x1

etc.
In our example, f ( x) x 3 467 and f ( x ) 3 x 2 . If we take our initial guess to be xo 6 ,
then by iterating the formula above, we generate the following table:
i

xi

f ( xi )

f ( xi )

0
1
2
3

6
8.324
7.796
7.759

-251
109.7718
6.8172
0.108

108
207.8706
182.3316
0.0350

f xo
251
6
8.32407
f x1o
108
f x1
109.7768
x2 x1
8.32407
7.79597
f x1
207.8706
f x2
6.817273
x3 x 2
7.79597
7.75858
f x 2
182.33156
x1 xo

[Note: The pocket calculator has a (yx) button, but a computer may do

x x x to get x3.]

b.
Analytical solution
How might we solve for the cube root of 467 analytically or symbolically? Take logarithms.
x 3 467

3 ln x ln 467

ln x

xe

ln 467
3

1
ln 467
3

= 7.758402264. . .

We used the (ln) button on our pocket calculator, followed by the (ex) button. In earlier times,
wed have used log tables. But, whence cometh those tables and how does the calculator
evaluate ln 467 or e2.0488?
3.

Series
ln x

x 1 1 x 1

x
2 x

1 x 1

3 x

x3 x5 x7

3! 5! 7!
x 2 x3 x 4
ex 1 x

2! 3! 4!

sin x x

The infinite series are exact. However, in practice we always keep a finite number of terms. In
principle, we can achieve arbitrary precision, if we have the necessary patience. Pocket
calculators and computer programs add up enough terms in a series to achieve a specified
precision, say 8 or 16 significant digits.
4.

Error

In this context, the term error does not refer to a mistake. Rather, it refers to the ideas of
deviation or of uncertainty. Every measured value is uncertain, according to the precision of the
measuring instrument. Every computed value is uncertain, according to the number of
significant digits carried along or according to the number of terms retained in the summation of
a series. Consequently, all numerical solutions are approximate.
Oftentimes, in discussing an example problem, the correct exact solution is known, so it is
possible to determine how an approximate numerical solution deviates from that exact solution.
Indeed, algorithms are often tested by applying them to problems having known exact solutions.
However, in real life, we dont know the correct exact solution. We cant know how far our
approximate solutions deviate from the correct, exact, but unknown solution. In other words, we
have to approximate the solution to a problem, but also we can only estimate the error.
Fortunately, we have means of estimating error. A goodly portion of the discussion in a
Numerical Methods textbook is devoted to rigorous estimation of error. In this course, we wont
concern ourselves with a detailed discussion of error analysis. Nonetheless, we want to be
always aware of the error issue, keeping in mind at least qualitatively the limitations of a
numerical solution. From time to time in the paragraphs that follow some aspects of the error
involved with a particular algorithm will be briefly discussed.

B.

Programming

The computer carries out the tedious arithmetic, but it must be told what to do. That is the
function of a computer program. A program may be written in one of any number of
programming languages, however there are certain features or issues that all languages have in
common.
1.

Program Design

a.

Stages
Conceptiondefine the problem
Develop the algorithmmap out or outline the solution
Codewrite the program
Debug & verifytrace the program; perform trial runs with known results; correct logical
& syntax errors

b.

Building blocks
Sequential operationsinstructions done one after the other in a specified order
Branching operationsselecting alternative sequences of operations
Looping operationsrepeating subsets of operations
I/O operationsreading and writing data

2.

Branching

a.

Simple yes or noselect between just 2 alternative actions

b.

Nested branchesa sequence of decisions or branches; decision tree

c.

Select casemore than two alternative actions

3.

Loops

a.

Counted loopa section of code is executed a specified number of times

b.

Conditional loopa section of code is iterated until a specified condition is met

c.
Infinite loopthe condition for ending the loop never is encountered, so the program
never ends
4.

I/O

a.

Inputkeyboard or data file

b.

Outputmonitor, output file, printer; numbers, text, graphics

5.

Precision Issues

a.
Binary
The computer does its arithmetic with binary numbers, that is, base-2. E.g., 0, 1, 10, 11, 100,
101, 110, 111, etc. We are accustomed to working and thinking with base-10 numbers. In
producing the machine language code (the executable) and carrying out calculations, all
numerical values are translated from base-10 to base-2 then back again for output. Usually, we
dont need to care about this. However, it can be a source of loss of precision in our numerical
values because the machine stores values with only finite precision.
b.
Precision
A single binary digit (0 or 1) is called a bit. Eight bits make up a byte. Within the machine, the
unit of information that is transferred at one time to/from the CPU and main memory is called a
word. The size of a word, or the word length, varies from one machine to another. Typically,
itll be from 4 to 64 bits. A 4-byte word contains 32 bits, etc.
One memory cell or memory location holds one or more words. Lets say its one word, or 4
bytes. Whatever information (number) is stored in one such memory cell must be expressible as
a string of 32 bits and no more. For instance, a non-terminating binary fraction will be
truncated, e.g., (0.1)10 = (0.00011001100110011. . .)2. Only 32 digits will be stored in memory.
When translated back into decimal, the number will be (0.09999997)10, not (0.1)10. Similarly, the
finite precision places a limit on the largest and the smallest numerical value that can be stored in
a memory cell.
In the back of our minds, we always remain aware of the physical limitations of the machine.
6.

Debugging

When syntax errors are all eliminated, the program may very well run smoothly to completion.
Perhaps it produces results which are clearly absurd; perhaps the results appear quite plausible.
A programmer must always take steps to convince itself that the program is working correctly;
the temptation to assume must be resisted.
One of the most insidious assumptions is that the program is doing what the programmer
intended it to do. Perhaps, a typing error has produced a statement that has no syntax error, but
does a different operation from that intended. Perhaps the logical sequence of steps written by
the programmer doesnt accomplish the task intended by the programmer. This why program
tracing is so important, why it is essential to insert print statements all through the program to
display the intermediate values of variables, why it is essential to check and double check such
things as argument lists and dimensions and the values of indiceschecking not what the
programmer intended, but what the program actually does.
The other, almost easier, aspect of debugging involves applying the program to a problem whose
solution is already known. It also involves repeating a numerical solution with different values
of various parameters such as step size and convergence tolerance. It involves comparing a
numerical solution for consistency with previous experience.

II. MatLab
A. Program Features
Work in MatLab is done in a variety of windows. The windows used most often are the
Command, Figure, Editor, and Help windows. When the program is started, three windows are
displayedCommand, Current Directory, and Command History windows. The first thing to do
upon starting the program is to select the Desktop Menu, select Desktop Layout, select
Command Window Only.
1.

Commands

a.
Command lines (p. 9)
Commands are entered at the command prompt (>>). When the enter key is pressed, the
command is executed and the output (if any) is displayed at once. All commands are recorded in
the Command History. Results from those previous commands are remembered.
More than one command may be entered on one line, separated by commas. The commands are
executed in order when enter is pressed.
A command can be continued to the next line with an ellipsis followed by enter.
The command history can be accessed with the up and down arrow keys.
Suppress command output--If a command is ended with a semicolon, display of its output (if
any) is suppressed. The product of the command is still available, just not displayed in the
command window.
Comments
Comment lines are started with the % symbol. They are not executed when the enter key is
pressed. A comment may also be attached to the end of a command, before pressing the enter
key.
Clearing the Command Window
The clc command clears the Command Window, but does not erase the command history.
b.

Arithmetic operators (p. 10)

Operation
Addition
Subtraction
Multiplication
Right division
Left division
Exponentiation

Symbol
+
*
/
\
^

precedence
4
4
3
3
3
2

Notice the distinction between right & left division. Left division is right division raised to the
1 power: 3\5 = 5/3.

Expressions enclosed in parentheses are evaluated first. Nested parentheses are executed from
innermost outward.
c.
Built-in functions (pp. 13-16)
Commonly used math functions are built-in. There are the usual sqrt, exp, sin, cos, etc., as
shown in the tables in the text. In addition, there are so-called rounding functions. The
argument, x, may be an expression.
Function
Description
Round(x)
Round to nearest integer
Fix(x)
Round toward zero
Ceil(x)
Round toward infinity
Floor(x)
Round toward infinity
Rem(x,y)
Remainder of x/y
Sign(x)
Returns the algebraic sign of x: 1, -1, or 0
d.
Scalars
A scalar is a numerical constant, like 5 or 8746 or 45.998, etc. A scalar variable is a name,
really the label of a memory location. A numerical value is stored in a variable. That numerical
value may be changed at any time. A variable name must begin with a letter, but may otherwise
contain letters, digits and the underscore character. There is a limit to how many characters the
name may be, but that varies with the MatLab version.
Built-in scalar variables: ans, pi, eps = 2^(-52), inf (infinity), and my favorite, NaN (not a
number). The variable ans is used to store the value of an expression or command that has not
been assigned a variable name. Caution! The built-in scalar variables may be reassigned,
whether inadvertently, or advertently.
The values stored in variables are all retained until or unless they are removed from memory
with the clear command. A list of variables presently in memory is obtained with the who or
whos commands.
e.
Assignment operator
Numerical values are assigned to a variable name with the assignment operator. The
assignment operator is the = sign, but it does not mean equal to. It means store this value in the
memory location labeled by the specified variable name. Only a single variable name can be on
the left-hand side, while the right-hand side may be a single number or a computable expression
including other, previously defined, variables.
The initial assignment of a value to a variable serves to define that variable. There is no special
declaration of variable types as is seen in some programming languages.
f.
Numerical display formats (p. 12-13)
The format command sets the display format of numerical values. See Table 1-2 in the text.
Basically, the number of digits displayed can be either 4 or 14(15) in either fixed point or
exponential notation.

2.
Arrays
An array is a matrix, or rather a matrix is an array of numbers. An n by m matrix has n rows and
m columns. All variables in Matlab are arrays, even scalars, which are 1x1 arrays.
a.
Vectors
A vector is a one-dimensional array. A row vector has one row and n columns. A row vector is
defined by listing its elements enclosed by square brackets and separated by commas or spaces.
E.g., a three element row vector is defined by A = [a1 , a2 , a3]. Similarly, a column vector is
defined by listing its elements enclosed by square brackets and separated by semicolons. B = [b1
; b2 ; b3] The column vector has one column and m rows.
Alternatively, row vectors may be defined by first element (zi), last element (zf) and the spacing
between the elements (q). Z = [ zi : q : zf ]
The linspace command creates a row vector by specifying the first and last element and the
number of elements. Z = linspace(z1,zf,n)
A character string is stored in MatLab as a vector, one character to one element. For instance,
B = Now is the time for all creates a 23-element vector, as there are 23 characters (including
spaces) in the phrase enclosed in the single quote marks. Each element may be addressed and
altered/replaced/deleted individually.
b.
Two-dimensional arrays
A = [first row ; second row ; third row ; . . .]
The rows can be specified as individual row vectors. The elements can be expressions.
Special arrays are zeros (elements all zero), ones (elements all ones), and eye (the identity
matrix).
The matrix transpose operator is the single quote mark. B = A B is the transpose of A.
c.
Addressing matrix elements
Individual elements of an array are referred to by their indices.
A(k) is the kth element of the vector A. B(m,n) is the element in the mth row & nth column.
It may be desirable to address an entire row or column of a matrix, perhaps a subset of a row or
column. In that case a colon (:) is used to indicate a range.
The 3rd through 6th elements of a vector are addressed by A(3:6), etc. Likewise, all the elements
of the mth row of a matrix are addressed by B(m,:). The m through n columns of all the rows of
a matrix B are designated by B(:,m:n).
The most general case would be a block of elements within the matrixB(m:n,p:q).

10

d.
Adding or deleting matrix elements
It is possible alter the sizes of a previously defined array variables. This done simply by
addressing additional vector(matrix) elements and assigning them values.
Say that A is a 4-element vector. We add elements to the vector by assigning values to the extra
elements. A(5)=5 , A(6)=7 , A97)=-98 , etc. Alternatively, a preexisting vector may be appended
to another. C=[ A B] or C=[G ; H]. Likewise, rows, columns, or entire matrices may be
appended to a matrix. Of course, the dimensions of the added rows, columns, & matrices must
match the matrix being enlarged.
A vector or matrix can be reduced in size, as well, by assigning nothing to some the elements,
thusly: B(:,4,9)=[]. This particular example will eliminate all rows from columns 4 9.
e.
Built in array manipulations
Some common array handling functions are built-in. These are listed on pages 41 43 of the
text.
3.

Array Operations

a.
Matrix operations
Arrays are multiplied, divided, added, subtracted, etc. according to the usual rules of matrix
arithmetic.
Inverse A-1 = A^-1
Left & right division X = A-1B = A\B
X = DC-1 = D/C
Left and right division arise because matrix multiplication is not commutative.
b.
Element by element operations
There exist also what are called element-by-element operations. In that case, an operation is
carried out on every element of an array. A period is added in front of the math operator to
indicate element-by-element operation. E.g., .* or .^
Notice that a dot product between two vectors can be carried out by an element-by-element
multiplication: sum(A.*B) = a1b1 + a2b2 + a3b3 + . . .
c.
Analyzing arrays
The built-in array functions are listed in Table 3-1, pages 64-65. These include Inv and Det.
B.

Files

1.
m-files
MatLab commands can be stored in a plain text file, and then run in the Command window.
The general term for a series of commands is a script. Writing such a series of commands is
called scripting. In MatLab, script files are saved with the extension .m, hence the term m-files.

11

The m-file may be created & edited in any plain text editor, such as Notepad, or by any word
processing program that is capable of storing plain text. There is also an Edit Window in
MatLab itself.
a.
Editor
Script files are created, edited, saved, and run in the Edit window.
b.
I/O (pp 95-117)
Input
Assign variables in the Command window before running the script.
Use the Input command or function within the script to interactively enter data.
Variable = input(message string)
Output
Dispwrites to the workspace
Fprintfallows formatting of the printed line(s).
2.

Script files

a.
Running
Run by entering the file name at the prompt in the Command window.
Run by pressing the run button in the Editor window
In either case, commands previously issued and variables previously defined in the Command
window are known to the script file.
b.
Comments & documentation
There must be comments throughout a script file describing the purpose of the script, defining
the variables used, describing the required input, etc. The purpose of the documentation is to
make plain what is happening in the script to yourself or another programmer at some later date,
not to mention to the instructor. Get in the habit early of over-commenting your scripts.
c.
Inline & feval
These are commands to create one-liners.
Functioname = inline(math expression as character string)
x = functioname(arguments)
variable = feval(function name,argument value)
3.
Function files
function command
A function file differs from the general script file in that it is self-contained. Variables assigned
in the work space (Command window) are not available inside the function file in general.
Likewise, variables assigned within the function file are not available outside the function file.
Variables have to be assigned inside the function file, or passed via the argument list in the
function statement, or of course by input commands. The first line of a function file is
function[arguments-out] = functioname(arguments-in)

12

Typically, the function is saved in the file functioname.m; that is, the file name is the same as the
function. The function is invoked by entering the functioname(arguments-in)
Data can be passed to the function through global variables, the argument-in list, and through
input commands within the function, as well as xlsread commands.
The function produces output through disp, fprintf, and plot commands within the script, or
through the arguments-out list.
It is possible to define variables to be global variables by including the Global command in all
script files, and the Command window as well.
Global variable list
C.
1.

Plots
Two Dimensional Graphs (pp. 133-158

a.
Line plots
Executing the plot or the fplot command automatically opens a Figure Window.
Plot(X,Y) plots Y vs X, where X & Y are vectors of the same length. If no other parameters
are specified, the graph is plotted in a bare-bones fashion, with a line connecting the data points,
but no axis titles, or data point symbols, etc. The axes are scaled over the intervals spanned by
the vectors X & Y.
However, there are parameters within the plot command as well as additional commands whose
purpose is to change the format of the graph. A graph can be formatted interactively within the
Figure Window, as well.
For plotting a function, there is the command fplot(function,xmin,xmax,ymin,ymax). The
function, y = f(x), is entered as a character string, as in 45*cos(3*x^3). The drawback of fplot
is that the f(x) cannot include variable names, only the dummy variable.
b.
Other plots
There are available other plotting commands that produce log graphs, bar graphs, pie charts, etc.
c.
Multiple graphs
It is possible to graph several curves on the same plot, using the Hold On and Hold Off
commands. Alternatively, it is possible to create several separate graphs on a single page with
the Subplot command.
2.

Three Dimensional Graphs

a.
Line plots (p 323)
Plot3(X,Y,Z)

13

This one is intended to plot X(t), Y(t), & Z(t) all as functions of a fourth parameter, t.
b.
Surface plots (pp 324-330)
Mesh(X,Y,Z) or Surf(X,Y,Z)
These commands plot Z(X,Y). The mesh command creates a wire-grid surface, while the surf
command adds color shading to the surface. There are variations of mesh & surf that produce
surface graphs of differing appearancemeshz, meshc, surfc, etc.
c.
Contour plots (p 330)
Contour(X,Y,Z,n) and variations.
d.
Special graphics (p 331)
Bar3(Y)
Sphere or [X,Y,Z]=Sphere(n) produces a set of (X,Y,Z) to be used by mesh or surf to plot a
sphere.
[X,Y,Z]=Cylinder(r) produces a set of points to be used by mesh or surf to draw a cylinder. r is
a vector that specifies the profile of the cylinder. r = some f(t)
e.
view command
The View command alters the angle at which a 3-d plot is viewed, by specifying the azimuth and
elevation angles of the view point.
View(az,el), with az and el specified in degrees, relative to the xz-plane and the xy-plane,
respectively.
D.

Programs

MatLab has many built-in functions and computing tools. Nonetheless, it becomes necessary to
write a special-purpose solution for a specific problem. No one commercial computing package
can address every possible situation, and no one lab can have every commercial product on hand.
Previously, we have used assignment statements to carry out calculations, and plot commands to
produce graphical output. Computer programs require also statements to make decisions, to
make comparisons and to carry out repetitive operations, not to mention input and output.
1.

Branches

a.

Relational & logical operators (p.174)

Operator
Description
<
Less than
>
Greater than
<=
Less than or equal to
>=
Greater than or equal to
==
Equal to**
~=
Not equal to
*The equal to operator consists of two equal signs, with no space between them.

14

If two numbers are compared, the result is 1 (logical true) or 0 (logical false). Comparing two
scalars yields a scalar 1 or 0. Arrays are compared element-by element. The result is a logical
array of 1s and 0s. Evidently, the two arrays must be the same size if they are to be compared
with each other. Similarly, a scalar is compared with an array element-by element, and the result
is logical array of 1s and 0s. The elements of logical arrays can be used to address elements in
ordinary arrays. Since the relational comparisons produce numerical values, relational operators
can be used within mathematical expressions. In mathematical expressions, the relational
operators are evaluated after all mathematical operators.
Logical operators
Operator
&
A&B
|
A|B
~
~A

Description
A AND B
=true if both A and B are
true, false otherwise
A OR B
=true if A or B is true,
false if both are false
NOT A
=true if A is false, false
if A is true

See the order of precedence on page 178. Notice that NOT comes after exponentiation and
before multiplication, etc., but that the other logical operators (AND, OR) come last.
There are a number of built-in logical functions, described on pages 179 180.
b.
If (pp. 182-190)
The IF statement is used to select between two courses of action. Several IF statements may be
nested to create a binary decision tree.
The decision is based on the truth or falsity of a statement or conditional expression. A
conditional expression is an expression consisting of relational and/or logical operators. The
expression will have the value true or false.
i.
if-end a block of commands is executed if the conditional expression is true,
skipped if its false.
if conditional expression
Matlab commands
end
ii.
if-else-end in this case, there are two blocks of MatLab commandsone is
executed if the conditional expression is true, the other if it is false.
if conditional expression
MatLab commands
else
Matlab commands

15

end
iii. if-elseif-else-end using two conditional expressions, one of three sets of Matlab
commands is executed.
if conditional expression
MatLab commands
elseif conditional expression
MatLab commands
else
Matlab commands
end
c.
Case
If we desire to select from among more than 2 or 3 cases, then it may be more convenient to use
the switch-case statement.
switch switch expression
case value1
MatLab commands
case value2
MatLab commands
case value3
MatLab commands
etc.
otherwise
MatLab commands
end
The switch expression is a scalar or string variable or an expression that can take on the values
value1, value2, value3, etc. If none of the specified values occur, then the block following the
otherwise command is executed. The otherwise command is optional.
2.
Loops (pp. 190-200)
Another thing we want a computer program to do automatically is to repeat an operation.
a.
Counting
The for-end loop executes a block of MatLab commands a specified number of times.
for k = f:s:t
MatLab commands
end
The loop executes for k = f, f+s, f+2s, f+3s, . . ., t. The increment, s, may be omitted in which
case it is assumed to be 1.
b.
Conditional
Alternatively, a loop may be executed as long as a conditional expression remains true.
while conditional expression

16

MatLab commands
end
The variables in the conditional expression must have initial values assigned, and at least one of
the variables must be changed within the loop.
3.

Input/output (pp 114-118)

a.
File input
variable = xlsread(filename,sheetname,range)import data from an Excel spreadsheet
b.
Import Wizard.
The Import Wizard is invoked by selecting Import Data in the File Menu.
c.
File output
fprintf--writes to a plain text disk file fprint(fid,arguments)
fid=open(filename)
fclose(fid)
xlswrite(filename,sheetname,range,variablename)--export to an Excel spreadsheet

17

III. Numerical Solution of Nonlinear Equations

A.

Non-Linear Equationsone at a time

There are closed form solutions for quadratic and even 3rd degree polynomial equations. Higher
degree polynomials can sometimes be factored. However, in general there is no closed form
analytical solution to non-linear equations.
1.

The Problem

a.
Roots & zeroes
We seek to find x such that f ( x) 0 or
perhaps such that f ( x ) g ( x) . In the
latter case, we merely set
h( x) f ( x) g ( x ) 0 . We are looking
for a root of the equation f ( x) 0 or a
zero of the function f(x).
b.
Graphical solution
Plot f(x) vs. xobserve where the graph
crosses the x-axis or plot f(x) and g(x) vs.
x and observe where the two curves
intersect. A graph wont give a precise root, but we can use the graph to choose an initial
estimate of the root.
2.
a.

Bisection
Setup

For brevity, say fo = f(xo) and f1 = f(x1), etc.

Say further that x is the desired root.
The graph shows us that f o f1 0 because
f(x) crosses the x-axis between [xo,x1].
b.
Algorithm
Let us find the midpoint of [xo,x1], and call it
b.
x x
i) b o 1 and then f b f (b)
2
ii) Does f b 0 ? If so, quit cause b .
iii) If not, then
if f o f b 0 , then set xo b and f o f b
or
if f b f o 0 , then set instead x1 b and f1 f b .

18

xo x1
.
2
v) If not, then repeat beginning with step (i).

iv) Is

x1 xo

? If so, quit and set

It is well also to count the iterations and to place a limit on the number of iterations that will be
performed. Otherwise, the program could be trapped in an infinite loop. Also, it is well to test
for the cases f o f b 0 and f1 f b 0 . It may be that the function does not cross the x-axis
between fo and f1, or crosses more than once.
3.

Newtons Method or the Newton-Raphson Method

a.
Taylors series
Any well-behaved function can be expanded in a Taylors series:
f ( xo )
f ( xo )
f ( x) f ( xo ) ( x xo ) f ( xo ) ( x xo ) 2
( x xo ) 3
.
2!
3!
Lets say that x is close to xo and keep just the first two terms.
f ( x) f ( xo ) ( x xo ) f ( xo )
We want to solve for x such that f(x) = 0.
f ( xo ) ( x xo ) f ( xo ) 0

f ( xo )
f ( xo )
In effect we have approximated f(x) by a straight line; x is the intercept of that line with the xaxis. It may or may not be a good approximation for the root .
x xo

b.

Algorithm
i) choose an initial estimate, xi
ii) compute f(xi) and f ( xi )
iii) compute the new estimate:
f ( xi )
xi 1 xi
f ( xi )
iv) return to step (ii) with i = i + 1

c.
Comments
It turns out that if the initial estimate of the
root is a good one, then the method is
guaranteed to converge, and rapidly. Even if
the estimate is not so good, the method will
converge to a rootmaybe not the one we
anticipated.

% Script to implement Newton's Method

del=1
x=3
k=0
func=inline('x^3-587')
deriv=inline('3*x^2')
while del >= 0.00005
k=k+1
if k <= 10
xnew=x-func(x)/deriv(x);
del=abs((xnew-x)/x);
disp(x);disp(func(x));disp(del)
x=xnew;
end
end

Also, if there is a f 0 point nearby the method can have trouble. Its always a good thing to
graph f(x) first.

19

4.

Secant Method

a.
Finite differences
A finite difference is merely the difference between two numerical values.
x x 2 x1 or x xi 1 xi
Derivatives are approximated by divided differences.
f ( xi 1 ) f ( xi ) f
f ( x)

xi 1 xi
x
We may regard this divided difference as an estimate of f at xi or at xi+1 or at the midpoint
between xi and xi+1.
b.
The Secant method
We simply replace f by the divided difference in the Newton-Raphson formula:
xi xi 1
xi 1 xi f ( xi )
.
f ( xi ) f ( xi 1 )
Notice the indices: i + 1, i, i 1. With the Secant Method, we dont use a functional form for f
. We do have to carry along two values of f, however.
Care must be taken that f ( xi ) f ( xi 1 ) not be too small, which would cause an overflow error
by the computer. This may occur if f ( xi ) f ( xi 1 ) due to the finite precision of the machine.
This may also give a misleading result for the convergence test of f ( xi ) f ( xi 1 ) . To avoid
that, we might use the relative deviation to test for convergence.
f ( xi ) f ( xi 1 )
f ( xi )

c.
Compare and contrast
Both the Newton-Raphson and Secant Methods locate just one root at a time.
Newton: requires evaluation of f and of f at each step; converges rapidly.
Secant: requires evaluation only of f at each step; converges less rapidly.
5.

Hybrid Methods

A hybrid method combines the use in one program of two or more specific methods. For
instance, we might use bisection to locate a root roughly, then use the Secant Method to compute
the root more precisely. For instance, we might use bisection to locate multiple roots of an
equation, then use Newton-Raphson to refine each one.

20

B.

Systems of Nonlinear Equations

Consider a system of n nonlinear equations with n unknowns.

f1 ( x1 , x 2 , x3 , , x n ) 0
f 2 ( x1 , x 2 , x3 , , x n ) 0

f n ( x1 , x 2 , x3 , , x n ) 0

1.

Newton-Raphson

a.
Matrix notation
Lets write the system of equations as a matrix equation.
f1
f2
f 0


fn
x1

x2
x

The unknowns form a column matrix also.

. We might write the system of equations


xn

compactly as

f( x ) 0.

b.
The Method
The Newton-Raphson method for simultaneous equations involves evaluating the derivative

f i

matrix, F , whose elements are defined to be Fij x . If the inverse F 1 exists, then we can
j

generate a sequence of approximations for the roots

of functions
{fi}.

x k 1 x k F 1 ( x k ) f ( x k )

At each step, all the partial derivatives must be evaluated and the F matrix inverted. The
iteration continues until all the f i 0 . If the inverse matrix does not exist, then the method fails.
If the number of equations, n, is more than a handful, the method becomes very cumbersome and
time consuming.
2.

Implicit Iterative Methods

The Newton-Raphson method is an iterative method in the sense that it generates a sequence of
successive approximations by repeating, or iterating, the same formula. However, the term
iterative method as commonly used refers to a particular class of algorithms which might more
descriptively be called implicit iterative methods. Such algorithms occur in many numerical
contexts as well see in subsequent sections of this course. At this point, we apply the approach
to the system of simultaneous nonlinear equations.

21

a.

General form
1

2


Let
be the solution matrix to the equation f ( x ) 0 . I.e., f ( ) 0 . Now, solve


n

algebraically each f i ( x ) 0 for xi. This creates a new set of equations, xi Fi ( x ) , where x
refers to the set of unknowns {xj} excluding xi. Algebraically, this looks funny, because each
unknown is expressed in terms of all the other unknowns, hence the term implicit. Of course,
what we really mean is

x k 1 F ( x k ) .
Alternatively, in terms of matrix elements, the equations take the form
xi , k 1 Fi ( x1, k , x 2, k , x n ,k ) .
b.
Algorithm
In a program, the iterative method is implemented thusly:

i) choose an initial guess, x o

ii) compute x1 F ( xo )

iii) test f ( x1 ) 0

iv) if yes, set x1 and exit

v) if no, compute x 2 F ( x1 ) , etc.

c.
Convergence

x k . For what conditions will this be true? Consider a region R in the

We hope that klim

number such that

h for 1 j n and suppose that for x in R there is a positive

Fi ( x )

. Then, it can be shown that if x o lies in R, the

x j

x j

space of {xi} such that

n

j 1

iterative method will converge. What does this mean, practically? It means that if the initial

guess, x o , is close enough to , then the method will converge to after some number, k,
of iterations. Big deal.

22

IV. Linear Algebra

A.

Matrix Arithmetic

The use of matrix notation to represent a system of simultaneous equations was introduced in
section III-B-1 above, mainly for the sake of brevity. In solving simultaneous linear equations,
matrix operations are central. There follows, therefore, a brief review of the salient properties of
matrices. Fuller discussion of the properties of matrices may be found in various texts,
particularly Linear Algebra texts.
1.

Matrices

A matrix is an n x m array of numbers. In these notes a matrix is symbolized by a letter with a

line on top, B ; n is the number of rows and m is the number of columns. If n = m, the matrix is
said to be a square matrix. If the matrix has only one column(row) it is said to be a column(row)
matrix. The jth element in the ith row of a matrix is indicated by subscripts, bij. Mathematically,
an entity like a matrix is defined by a list of properties and operations, for instance the rules for
adding or multiplying two matrices. Also, matrices can be regarded as one way to represent
members of a group in Group Theory.
b12 b13 b14
b
x1
11

x x 2
B b21 b22 b23 b24
x3
b31 b32 b33 b34
2.

Addition & Subtraction

a.
Definition
The addition is carried out by adding the respective
matrix elements.

C A B

cij aij bij

b.
Rules
The sum of two matrices is also a matrix. Only matrices having the same number of rows and
the same number of columns may be added. Matrix
addition
is commutative
and associative.

A B B A

3.
a.

( A B)C A(B C )

Multiplication
Definition

C AB

cij aik bkj ai1b1 j ai 2 b2 j ai 3b3 j

k

b.
Rules

The product of two matrices is also a matrix.

The
number
of
elements
in
a
row
of
A must equal

the number of elements in a column of B .

Matrix
multiplication is not commutative.

AB BA

23

A matrix may be multiplied by a constant, thusly: cij q aij . The result is also a matrix.
4.

Inverse Matrix

a.
Unit matrix
The unit matrix is a square matrix with the diagonal elements equal to one and the off-diagonal
elements all equal to zero. Heres a 3x3 unit matrix:
1

U 0
0

0
1
0

0
1

b.
Inverse

The inverse of a matrix, B , (denoted B 1 ) is a matrix such that BB 1 B 1 B U . The

inverse of a particular matrix may not exist, in which case the matrix is said to be singular.
The solution of a system of simultaneous equations in effect is a problem of evaluating the
inverse of a square matrix.

24

B.
1.

Simultaneous Linear Equations

The Problem

a.
Simultaneous equations
We wish to solve a system of n linear equations in n unknowns.
b11 x1 b12 x 2 b1n x n c1
b21 x1 b22 x 2 b2 n x n c 2

bn1 x1 bn 2 x 2 bnn x n c n

where the {bij} and the {ci} are constants.

b.
Matrix notation
The system of equations can be written as a matrix multiplication.

Bx c ,

where x

x1
x
2


xn

, c

c1
c
2


cn

and B

b11
b
21

bn1

b1n
b2 n

b12
b22

bnn

bn 2

When n is small ( n 40 , say) a direct or one-step method is used. For larger systems, iterative
methods are preferred.
2.

Gaussian Elimination

In a one-step approach, we seek to evaluate the

inverse of the B matrix.

Bx c

B 1 Bx x B 1c

The solution is obtained by carrying out the matrix multiplication B 1c .

a.
Elimination
You may have seen this in high school algebra. For brevitys sake, lets let n = 3.
b11 x1 b12 x 2 b13 x3 c1
b21 x1 b22 x 2 b23 x3 c 2
b31 x1 b32 x 2 b33 x3 c3

In essence, we wish to eliminate unknowns from the equations by a sequence of algebraic steps.
normalization
reduction

b21
and add to eqn. 2; replace eqn. 2.
b11
b31
ii) multiply eqn 1 by
and add to eqn. 3; replace eqn. 3.
b11

i) multiply eqn. 1 by

b11 x1 b12 x 2 b13 x3 c1

x 2 b23
x3 c 2
b22

25

 x 2 b33
x3 c3
b32

iii) multiply eqn. 2 by

b32
and add to eqn. 3; replace eqn. 3.

b22

b11 x1 b12 x 2 b13 x3 c1

x 2 b23
x3 c 2
b22
x3 c3
b33

We have eliminated x1 and x2 from eqn.3 and x1 from eqn. 2.

back substitution iv) solve eqn. 3 for x3, substitute in eqn. 2 & 1.
solve eqn. 2 for x2, substitute in eqn. 1.
solve eqn. 1 for x1.
b.
Pivoting
Due to the finite number of digits carried along by the machine, we have to worry about the
relative magnitudes of the matrix elements, especially the diagonal elements. In other words, the

inverse matrix, B 1 may be effectively singular even if not actually so. To minimize this
possibility, we commonly rearrange the set of equations to place the largest coefficients on the
diagonal, to the extent possible. This process is called pivoting.
e.g.
37x2 3x3 = 4
19x1 2x2 + 48x3 = 99
7x1 + 0.6x2 +15x3 = -9
rearrange
19x1 2x2 + 48x3 = 99
37x2 3x3 = 4
7x1 + 0.6x2 +15x3 = -9
or
7x1 + 0.6x2 +15x3 = -9
37x2 3x3 = 4
19x1 2x2 + 48x3 = 99
3.

Matrix Operations

In preparation for writing a computer program, well cast the elimination and back substitution in
the form of matrix multiplications.
a.

Augmented matrix

b b b c
11 12 13 1
A B : c b21 b22 b23 c 2
b31 b32 b33 c3

b.
Elementary matrices
Each single step is represented by a single matrix multiplication.

26

The elimination steps:

S1

1
b
21
b11
0

0 0

1 0

0 1

S2

1
0
b
31
b11

0
1

0
0

1
0

S3 0
1

b
0 32

b22

b b b
11 12 13
b23

S 3 S 2 S1 A 0 b22

0 0 b33

0
0

c1
c 2

c3

The first back substitution step:

Q1 0

0
0
1

b33

1
0

b
b
b13 c1
11 12

b23
c 2
Q1 S 3 S 2 S1 A 0 b22
0 0 1 x3

This completes one cycle. Next we eliminate one unknown from the second row using
1

S 4 0
0

0

b23

b
b
c
b
11 12 13 1
0 c 2
S 4 Q1 S 3 S 2 S1 A 0 b22

0 0 1 x3
1 0

1
Q2 0

b22

0 0
b
11
Q2 S 4 Q1 S 3 S 2 S1 A 0
0
This completes the second cycle. The final cycle is

27

b12

c1

b13

1 0

x2

0 1 x3

S 5 0
0

b13

0
1

1
0

S 6 0
0

b12
1
0

0
0

1
b

11
Q3 0
0

1 0 0 x1

Q3 S 6 S 5 Q2 S 4 Q1 S 3 S 2 S1 0 1 0 x 2
0 0 1 x3

We identify the inverse matrix B 1 Q3 S 6 S 5 Q2 S 4 Q1 S 3 S 2 S1 . Notice that the order of the

matrix multiplications is significant. Naturally, we want to automate this process, and generalize
to n equations.
4.

Gauss-Jordan Elimination

a.
Inverse matrix
We might multiply all the elementary matrices together before multiplying
by the augmented
1
1
matrix. That is, carry out the evaluation of B , then perform B A .
b.

Algorithm

a kjk

a kjk 1

i k

a kkk 1
k
k 1
k 1
k
aij aij aik a kj i k
k 1, n
i 1, n
j k, n 1

n = number of equations
k = index of the step or cycle
aij = elements of the
original
augmented matrix, A .
For each value of k, do the i = k line
first.

% Script to implement Gauss-Jordan Elimination

b=[4 1 2;1 3 1;1 2 5];
c=[16;10;12];
a=[b c];
np=size(b);
n=np(1);
for k=1:n
for m=k+1:n+1
a(k,m)=a(k,m)/a(k,k);
end
a(k,k)=1;
for l=1:n
if l~=k
for m=k+1:n+1
a(l,m)=a(l,m)-a(l,k)*a(k,m);
end
a(l,k)=0;
end
end
end
a

28

c.
Example
n = 3 and n + 1 = 4
4 x1 x 2 2 x3 16
x1 3 x 2 x3 10
x1 2 x 2 5 x3 12

k=0

4 1 2 16

A 1 3 1 10
1 2 5 12

e.g., for k = 1, i = 1, j = 1 & j = 4

1
a11

0
a11
0
a11

1
a14

0
a14
0
a11

16
4
4

0
0 1
a121 a 21
a 21
a11 1 1 1 0

k=1

1
1 4

11
A 0
4

0 7

k=2

k=3

4
2
1
6
2
9
8

5
1
0

11

2
A 0 1
11

0 0 46

11

38
11
24

11
46
11

1 0 0 3

A 0 1 0 2
0 0 1 1
3
x 2
1

29

C.

Iterative Methods

For n > about 40, the one-step methods take too long and accumulate too much round-off error.
1.

Jacobi Method

a.
Recursion formula
Each equation is solved for one of the unknowns.
1
b11
1
x 2 c 2 b21 x1 b23 x3 b2 n x n
b22

x1 c1 b12 x 2 b12 x3 b1n x n

x n c n bn1 x1 bn 2 x 2 bnn 1 x n 1

In short

1
xi ci bij x j

j 1
bii
i j

1
bnn

, i = 1, 2, 3, . . .,n.

Of course, we cannot have bii = 0 for any i. So before starting the iterative program, we may
have to reorder the equations. Further, it can be shown that if bii bij for each i, then the
method will converge, though it may be slowly. Heres an outline of the showing.
The first iteration is:

x 1 Ax 0 V

k k
0
k 1 0 k
k 1
x

A
x

A
A
A
x

A
AV A x A V
After several iterations,
k 1
3 2 1
k 1
3 2

b
A k 1 x 0 0 , which will happen if ij 1 .
We want klim

bii

b.
Algorithm

We need four arrays: x k , x k 1 , B , and c .
x10

0 x 20
Firstly, select an initial guess (k = 0) x
.

0
x n

Secondly, compute a new x (k + 1 = 1).

xik 1

Thirdly, test for convergence.

xik 1

xik

xik

ci

j 1
i j

1
bii

bij x kj

. Notice that all the xi must pass the test.

If all the xi do not pass the test, then repeat until they do.

30

2.

Gauss-Seidel Method

The Gauss-Seidel Method hopes to speed up the convergence by using newly computed values
of xi at once, as soon as each is available. Thus, in computing xnew(12), for instance, the values
of xnew(1), xnew(2), . . ., xnew(11) are used on the right hand side of the formula. We still need
to keep separate sets of xnew and xold in order to perform the convergence tests.
% Script to implement Gauss-Seidel
b=[4 1 2;1 3 1;1 2 5];
c=[16;10;12];
xold=[1;1;1];
xnew=xold;
np=size(b);
n=np(1);
flag=1;
while flag > 0
for k=1:n
sum=0;
for l=1:n
if k~=l
sum=sum+b(k,l)*xnew(l);
end
end
xnew(k)=(c(k)-sum)/b(k,k);
end
for k=1:n
if abs((xnew(k)-xold(k))/xold(k)) > 0.0005
xold=xnew;
break
else
flag=0;
end
end
end
xnew

31

D.

Applications

A couple of cases in engineering that give rise to simultaneous linear equations.

1.

Electrical Circuit

(7+2+6)x1 2x2 6x3 = 300

-2x1 + (2+5+4+1)x2 4x3 x4 = 0
-6x1 4x2 + (4+9+6)x3 9x4 = 0
-x2 9x3 + (9+1+11)x4 = 0
15
2
A
6

2
12
4
1

6 0 300
4 1 0
;
19 9 0

9 21 0

32

26.5

9.35
solution: x
13.3

6.13

2.

Truss System

1
0

0
4

0 1 1

0
A
0

0
1

0 0 0
5
1 3
0 0
5
4 1 0
0
5
0
0 0 1

600
0 3600
0
0

400

0
;
0 600

0
0

0 600
1
0
0

33

600
1000

600

1250
x

solution:
1083.33

866.67
600

600

V. Interpolation and Curve Fitting

Suppose one has a set of data pairs:
x
x1
x2
x3

f
f1
f2
f3

xm
fm
where fi is the measured (or known) value of f(x) at xi. We would like to find a function that will
approximate f(x) for all x in a specified range. There are two basic approaches: interpolation and
curve fitting.
A.

Polynomial Interpolation

With interpolation, the approximating function passes through the data points. Commonly, the
unknown f(x) is approximated by a polynomial of degree n, pn(x), which is required to pass
through all the data points, or a subset thereof.
1.

Uniqueness

Theorem: Given {xi} and {fi}, i = 1, 2, 3, . . ., n + 1, there exists one and only one polynomial of
degree n or less which reproduces f(x) exactly at the {xi}.
Notes
i) There are many polynomials of degree > n which also reproduce the {fi}.
ii) There is no guarantee that the polynomial pn(x) will accurately reproduce f(x) for
x xi . It will do so if f(x) is a polynomial of degree n or less.
Proof: We require that pn(x) = fi for all i = 1, 2, 3, . . ., n+1. This leads to a set of simultaneous
linear equations
a o a1 x1 a 2 x12 a n x1n f1
a o a1 x 2 a 2 x 22 a n x 2n f 2

a o a1 x n 1

a n x nn1 f n 1

a 2 x n21

which wed solve for the {ai}. As long as no two of the {xi} are the same, the solution to such a
set of simultaneous linear equations is unique.
The significance of uniqueness is that no matter how an interpolating polynomial is derived, as
long as it passes through all the data points, it is the interpolating polynomial. There are many
methods of deriving an interpolating polynomial. Here, well consider just one.

34

2.

Newtons Divided Difference Interpolating Polynomial

a.
Divided differences
The first divided difference is defined to be (notice the use of square brackets)
f (a ) f (b)
f a, b
, ab
ab
If f(x) is differentiable in the interval [a,b], then there exists
df ( )
f a, b
at least one point between a and b at which
dx
. In practice, we would take a as close to b as we can
(limited by the finite precision of the machine) and say that
f f a, b .
Higher order differences are defined as well:
order
0

notation
f x1

definition
f ( x1 )

f x 2 , x1

f x3 , x2 , x1

f x4 , x3 , x2 , x1

f x4 , x3 , x 2 f x3 , x2 , x1
x4 x1

f x 2 f x1
x 2 x1

f x3 , x 2 f x 2 , x1
x3 x1

f x n1 , x n , , x3 , x 2 f x n , x n 1 , , x 2 , x1
x n 1 x1

f x n1 , x n , , x 2 , x1

b.
Newtons divided difference formula
Build the formula up step by step:
i)

two data points (x1,f1) & (x2,f2). We wish to approximate f(x) for x1 < x < x2.
As a first order approximation, we use a straight line (p1(x) so that
f x, x1 f x 2 , x
f ( x ) f1
f f ( x)
2
x x1
x2 x
Solve for f(x)
f ( x) f1 ( x x1 ) f x 2 , x1 p1 ( x)

ii)

Now, if f(x) is a straight line, then f(x) = p1(x). If not, there is a remainder, R1.
R1 ( x) f ( x) p1 ( x) f ( x) f1 ( x x1 ) f x 2 , x1 ( x x1 )( x x 2 ) f x, x 2 , x1
35

We dont know f(x), so we cannot evaluate f[x,x2,x1]. However, if we had a third data point
we could approximate f x, x 2 , x1 f x3 , x 2 , x1 . Then we have a quadratic
f ( x) f1 ( x x1 ) f x 2 , x1 ( x x1 )( x x 2 ) f x3 , x 2 , x1 p 2 ( x) .
iii)

If f(x) is not a quadratic polynomial, then there is still a remainder, R2.

R2 ( x ) f ( x ) p 2 ( x )
To estimate R2, we need a fourth data point and the next order divided difference. . .
f x, x3 , x 2 , x1 f x 4 , x3 , x 2 , x1

iv)

Jump to the generalization for n + 1 data points:

f ( x ) p n ( x ) Rn ( x ) , where

p n ( x) f x1 ( x x1 ) f x 2 , x1 ( x x1 )( x x 2 ) f x3 , x 2 , x1
( x x1 )( x x 2 )( x x3 ) ( x x n ) f x n 1 , x n , , x 2 , x1

Notice that i) p3 p 2 ( x x1 )( x x 2 )( x x3 ) f x 4 , x3 , x2 , x1 , etc. and ii) the (x xi) factors are

also those of the previous term times one more factor.
c.
Inverse interpolation
The NDDIP lends itself to inverse interpolation. That is, given f(x), approximate x. In effect, we
are solving f(x) = 0 when f(x) is in the form of a table of data. Simply reverse the roles of the {fi}
and the {xi}.
n 1

i 1

i 2

j 1

x p n ( f ) f f 1 , f 2 , , f i f ( x ) f j f f 1

Set f(x) = 0 and evaluate x = pn(0). In practice, with a Fortran program, one would just reverse
the data columns and use the same code.
d.
Example
The difference table is computed thusly:
for j=1:n+1
diff(j,1)=f(j)
end
for j=2:n+1
for i=1:n+1-j+1
diff(i,j)=( diff(i+1,j-1)-diff(i,j-1) )/(x(i+j-1)-x(i))
end
end

36

Divided Difference Table for n = 6

f[ , , ]
f[ , , , ]
f[ , , , , ]
1.667
-2.583
1.583
-3.5
2.167
-0.96
0.833
-0.233
0.1
0.367
0.017
0.4

j
x
f
f[ , ]
1
1
-1.5
0.5
2
2
-1
3
3
2.5
0.5
-0.5
4
3
0.25
0.75
5
4
1
1.3
6
4.5
1.65
1.7
7
5
2.5
The sixth degree polynomial constructed from this table is
7

i 1

i 2

j 1

f[ , , , , , ]
-0.727
0.353

f[ , , , , , , ]
0.27

p 6 ( x) f x1 f x1 , x 2 , , xi ( x x j ) .

Line by line, the script might look like this:

fac = ex x(1)
p0 = diff(1,1)
p1 = p0 + fac*diff(1,2)
fac = fac*(ex-x(2))
p2 = p1 + fac*diff(1,3)
fac = fac*(ex-x(3))
p3 = p2 + fac*diff(1,4)
fac = fac*(ex-x(4))
p4 = p3 + fac*diff(1,5))
fac = fac*(ex-x(5))
p5 = p4 + fac*diff(1,6)
fac = fac*(ex-x(6))
p6 = p5 + fac*diff(1,7)
Notice that we must use a different variable name for the argument x from the name used for the
data array x(i).
Of course, its more general and flexible to use a loop.
fac = 1.0
p = diff(1,1)
for j=1:n
fac = fac*(ex-x(j))
p = p + fac*diff(1,j+1)
end
e.
Issues with high degree polynomials
If we have a large number of data points, 20 or 100 or 1000s, it does not pay to use the entire
data table to create a 20 or 100 or 1000th degree polynomial. The greater the degree, the more
often the pn goes up and down between the data points. Our confidence that f ( x) p n ( x)
actually decreases. Its better to interpolate on subsets of the data using a p3 or a p4 using data
points that surround the specified x. This process can be incorporated into the program. These
low-degree segments are sometimes called splines.

37

B.

Least Squares Fitting

Often, there are errors or uncertainties in the data values, 10.07 0.005 sec , for instance.
Perhaps forcing the approximating function to pass through the data points is not the wisest
approach.
An alternative approach is to assume a functional form for the unknown f(x) and adjust it to best
fit the uncertain data. A way to judge what is best is needed.
1.

Goodness of Fit

The method of least squares uses a particular measure of goodness of fit.

a.
Total squared error, E
First of all, never forget that the word error in this context means uncertainty. Now, lets say
{xi,fi} are the n+1 data values and f(x) is the assumed function. Then E is defined to be
n 1
1
E 2 f i f xi 2
i 1 i
The { i } are weighting factors that depend on the nature of the uncertainties in the data {fi}.
For measured values, the i f i , the experimental uncertainties. Often, we just take all the
i 1 , perhaps implying that the experimental uncertainties are all the same.. In that case,
n 1

E f i f xi 2 .
i 1

b.
Least squares fit
We wish to derive an f(x) which minimizes E. That means taking the derivative of E with respect
to each adjustable parameter in f(x) and setting it equal to zero. We obtain a set of simultaneous
linear equations with the adjustable parameters as the unknowns. These are called the normal
equations.
2.

Least Squares Fit to a Polynomial

Assume that f ( x) a bx cx 2 dx 3 .
a.
Total squared error
n 1

2
1
f i a bxi cxi2 dxi3
2
i 1 i
We have four adjustable parameters: a, b, c, and d. Notice that, unlike the interpolating
polynomial, there may be any number of data pairs, regardless of the number of parameters.
Lets take all the i 1 .

The partial derivative with respect to the adjustable parameters are

E
2 f i a bxi cxi2 dxi3
a
i

38

E
2 xi f i a bxi cxi2 dx i3
b
i
E
2 xi2 f i a bxi cxi2 dxi3
c

E
2 xi3 f i a bxi cxi2 dxi3
d
i

b.
Normal equations
Collect the like powers of xi and set the derivatives equal to zero.

fi
i

xi f i
i

a b xi c xi2 d xi3
i

a xi b
i

xi2

xi3

d xi4
i

xi2 f i a xi2 b xi3 c xi4 d xi5

i

xi3 f i

a
i

xi3

xi4

xi5

d xi6
i

In terms of the matrix elements we used in solving simultaneous linear equations,

c1 f i

c 2 xi f i

c3 xi2 f i
c4

xi3

fi

b11 1

b12 xi

b21 xi

b22 xi2 , etc.

The system is solved by any standard method, Gauss-Jordan, Gauss-Seidel, even by Cramers
method.
c.
Accuracy of fit
Wed like to have some statistical measure of how good the fit between the {fi} and f(x) is. This
will depend on the relation between E and the { i2 }. Lets consider a quantity called (N = n + 1)
N

f ( xi ) f i 2

.
i2
If all i 1 , then 2 E . Now, on another hand, if i f ( xi ) f i , then 2 N g , where g
is the number of adjustable parameters and N g is the number of degrees of freedom in the
i 1

mathematical model for the data. Wed like to see

2
1
Ng

for a good fit, while

indicates that the quality of the fit is ambiguous (sometimes called over fitted), and

2
1
Ng

2
1
Ng

indicates a poor fit.

3.

Least Squares Fit to Non-polynomial Function

The process is similar when fitting to a function that is not a polynomial. For instance, say that
f ( x) a ln x b cos x ce x .

39

We wish to fit this function to the data shown at right. In this case,
N = 10 and g = 3. The adjustable parameters are a, b and c.
10

E 2 f i a ln xi b cos xi ce xi
i 1

The normal equations are:

a ln xi 2 b ln xi cos xi c ln xi e xi f i ln xi

a ln xi cos xi b (cos xi ) 2 c cos xi e xi f i cos xi

a ln xi e xi b cos xi e xi c (e xi ) 2 f i e xi

xi
.24
.65
.95
1.24
1.73
2.01
2.23
2.52
2.77
2.99

fi
0.23
-0.26
-1.10
-0.45
0.27
0.10
-0.29
0.24
0.56
1.00

6.794a 5.348b 63.259c 1.616

5.347 a 5.108b 49.009c 2.383
63.259a 49.009b 1002.506c 26.773

When solved by the Gauss-Jordan method, these yield

a = -1.041
b = -1.261
c = 0.031

Least Squares Fit

data

f ( x) 1.041 ln x 1.261 cos x 0.031e x

f(x)

2
0.926

1
Ng
7

The goodness of fit between these data and

this function is ambiguous. A glance at a
graph verifies that the fit is iffy. [Thats the
technical term for it.]

2
1
0
-1
-2

2
x

40

fit

MatLab Sidelight Number One

1.

Polynomials

a.
Representation
In MatLab, polynomials are represented by a vector composed of the coefficients. Thusly,
p3 ( x) a3 x 3 a 2 x 2 a1 x a o p [a3 a 2 a1 a o ]

b.
Evaluate
A polynomial is evaluated using the command polyval. If p is the vector of coefficients and x is
the argument,
polyval(p,x) or y=polyval(p,x)
The roots command gives the roots of a polynomial, as elements of a vector.
r=roots(p)
If the roots of a polynomial are known, then the coefficient vector can be obtained by the poly
command.
p=poly(r)
c.
Add, multiply, & divide
Polynomials are added by adding the vectors of their coefficients. The shorter vector has to be
padded with zeros to make the two vectors the same length.
Multiplication of two polynomials is done with the conv command. c(x) = a(x) * b(x)
c = conv(a,b)
Division is done with the deconv command. u(x)/v(x) = q(x) + r(x)
[q,r] = deconv(u,v)
d.

derivatives

d
p ( x) k = polyder(p)
dx
d
k ( x)
( a ( x )b( x )) k=polyder(a,b)
dx
n( x )
d u ( x)

[n m] = polyder(u,v) followed by [k,r] = deconv(n,m)

m( x ) dx v ( x )

k ( x)

2.

Curve Fitting & Interpolation

41

a.
Least squares
MatLab fits data to a polynomial using the least squares method. Fitting an nth degree
polynomial to a table of (x,y) points. If the number of data points is m, then n must be m-1 or
less, and greater than 0.
p=polyfit(x,y,n)
Fitting to functions other than polynomials is done by rewriting the function in terms of a straight
line, for instance by taking the log of both sides, etc.
b.
Interactive fitting
In the Tools menu of the Figure Window is a Basic Fitting tool. This can be used to fit a function
to data interactively. See section 8.4 in the text.
c.
Interpolation
MatLab has four interpolating techniques built-in. The command is interpl. It estimates
yi = f(xi), given a set of {x,y}.
yi = interpl(x,y,xi,method)
The methods available are
nearest returns the value of the nearest data point
linear
carries out linear interpolation
spline
carries out interpolation using a cubic polynomial based on the data points
surrounding the interpolated point
pchip
carries out interpolation using a cubic Hermite polynomial.

42

VI. Integration
b

We wish to evaluate the following definite integral:

f ( x ) dx .

We use numerical methods when

i) f(x) is known analytically but is too complicated to integrate analytically or
ii) f(x) is known only as a table of data.
A.
1.

Newton-Cotes Formul
Trapezoid Rule

a.
Graphs
Graphically, a definite integral is the area between the x-axis and the curve f(x). Areas below the
axis are negative; areas above the axis are positive.

b.
Trapezoids
The area under the curve might be approximated most simply by a series of trapezoids and
triangles.

f f
f1 f 2
x 2 x1 2 3 x3 x 2
2
2

43

Notice that x1 = a and that x8 = b.

c.
Interpolating polynomial
In effect, we are replacing the integrand, f(x), by a straight line between each pair of points:
f ( xi ) f ( xi 1 )
p1 ( x) f ( xi 1 ) ( x xi 1 )
.
xi xi 1
This can be checked by integrating p1(x) analytically.
xi

xi 1

f f i 1
f f i 1 x 2
f i 1 ( x xi 1 ) i
dx f i 1 xi xi 1 i
xi xi 1
xi xi 1 2

xi

xi 1
xi 1

f i f i 1 xi
x
xi xi 1 xi 1

xi f i xi f i 1 xi 1 f i xi 1 f i 1

xi 1 f i xi 1 f i 1
2
2
2
2
x f
x f
x f
i i i 1 i i 1 i 1
2
2
2

xi f i 1 xi 1 f i 1

d.

xi f i 1
2

1
xi xi 1 f i f i 1 check.
2

Implementation
N

For N data points spanning [a,b], there are N 1 trapezoids. T xi xi 1

i 2

f i f i 1
2

If the data are uniformly spaced, then xi xi 1 h for all i, and

N 1
f
f
h N
T f i f i 1 h 1 N f i .
2 i2
2 i 2
2
The lines in the MatLab script might look like this:
n = 10
T = 0.0
for i=2:n
T = T + (x(i)-x(i-1))*(f(i)+f(i-1))/2.0
end
2.

Extension to Higher Order Formul

a.
Forward difference interpolating polynomial
Well take this opportunity to examine an alternative interpolating polynomialthe Forward
Difference Polynomial.
Imagine we have a table of data pairs (xi,fi) which are uniformly spaced, with spacing h. The
forward differences are just the familiar deltas.
f ( x1 ) f ( x 2 ) f ( x1 ) f 2 f1
first order:
2 f ( x1 ) f ( x 2 ) f ( x1 ) ( f ( x3 ) f ( x 2 )) ( f ( x 2 ) f ( x1 ))
second order:
Notice that the differences f ( x1 ) and 2 f ( x1 ) are regarded as being evaluated at x = x1.
Hence the term forward difference.

44

Notice, too, that the forward differences are related to the divided differences simply by
multiplying by the denominators.
f ( x1 ) h f x 2 , x1
2 f ( x1 ) 2h 2 f x3 , x 2 , x1

n
f ( x1 ) n!h n f x n 1 , x n , , x 2 , x1

Now, lets expand the integrand f(x) in a Taylors Series about x = x1. Further, to increase the
x x1
element of confusion, let
so that x x1 h .
h
( 1) 2
( 1)( 2) 3
f ( x) f ( x1 ) f ( x1 )
f ( x1 )
f ( x1 )
2!
3!
Depending on how many terms are kept, this will give a polynomial in or in x.
b.
Simpsons rule
Any number of formul may be created by replacing the integrand, f(x), with an interpolating
polynomial of some specified degree. If f ( x) p1 ( x ) f ( x1 ) f ( x1 ) , the Trapezoid Rule is
recovered.
Perhaps f(x) has some curvature, so a second degree interpolating polynomial may serve better.
x3
2
2
1 2

f
(
x
)
dx

h
p
(
x

h
)
d

2 1
f ( x1 ) f ( x1 ) 2 f ( x1 ) d
x1
0
0
1

h 2 f ( x1 ) 2f ( x1 ) 2 f ( x1 )
3

Expand the differences. . .

x3

f ( x)dx h 2 f ( x1 ) 2 f ( x2 ) 2 f ( x1 ) 3 f ( x3 ) 3 f ( x2 ) 3 f ( x1 )

x1

h
f ( x1 ) 4 f ( x2 ) f ( x3 )
3
This is Simpsons Rule, which integrates over segments of three data points (or two intervals of
h) in one step.

c.

Implementation
x3

f ( x)dx 3 f ( x1 ) 4 f ( x2 ) f ( x3 )

x1
x5

f ( x)dx 3 f ( x3 ) 4 f ( x4 ) f ( x5 )

x3

xn 1

f ( x)dx 3 f ( xn1 ) 4 f ( xn ) f ( xn1 )

xn 1

45

Add em up. . .
h

n 1

i 2

i 2

f ( xi ) 4 f ( xi ) f (a ) f (b)
f ( x)dx 3 2i
1
i2

Caveats:

i) the data points must be uniformly spaced.

ii) n + 1 must be odd, starting with 1 so that n

46

ba
is even.
h

B.
1.

Numerical Integration by Random Sampling

Random Sampling

a.
Pseudorandom numbers
Random numbers are a sequence of numbers, z1 , z2 , z3 ,L , lying in the interval (0,1). There is
no pattern in the progression of the numbers, nor is any number in the sequence related to any
other number by a continuous function. There are statistical tests for randomness in a sequence
of numbers but we wont bother with them here.
The operation of a computer is deterministic, so truly random numbers cannot be generated by a
computer program. However, sequences can be generated that appear to be random in that the
sequence passes some of the statistical tests for randomness. Such a sequence of numbers is
called pseudorandom.
Here is an algorithm for generating a sequence of pseudorandom numbers:
xi mod a xi 1 c, m
x
zi i
m
where a, c and m are integers and mod( ) is the modulus function. The pseudorandom number
uniformly distributed in the interval (0,1) is zi.
In MatLab , this looks like the following: [Using the built-in remainder function (rem).]
x = xo
for i=1:100
x1 = rem(a*x+c,em);
z = x1/em
x = x1;
end
This process generates a sequence of numbers {zi} that have some properties of random
numbers, but in fact the sequence repeats itselfits periodic. The exact sequence depends on
the initial value, xo, called the seed. Usually, m is a large integer, commonly a power of 2. The
numbers c and m can have no common factor (c can be zero) while a is a multiple of a prime
factor of m + 1. The period of the sequence is m, which is why m needs to be large. For
instance, we might take m 2 31 , c = 0 and a = 16807.
On the other hand, MatLab has built-in random number generating functions, shown in Table
3-2.
b.
Intervals
Suppose we want our pseudorandom numbers to lie in the interval (a,b) rather than (0,1). This is
easily done by scaling, or mapping onto the desired interval. Say 0 z 1 , then
y b a z a will lie in the interval (a,b).

47

c.
Distributions
The example random number generator mentioned above produces numbers uniformly
distributed in (0,1). This means that is (0,1) were divided into equal subintervals, an equal
number of random numbers is expected in each of those subintervals. The probability of the next
random number in the sequence falling in a particular subinterval is the same for all the
subintervals spanning (0,1).
It is possible to form sequences of pseudorandom numbers which obey some other distribution
function, such as Poisson or Gaussian, etc. We wont get into that here.
2.

Samples of Random Sampling

a.
Coin toss
We have two outcomes for each toss, of equal probability. Well generate an integer, either 1 or
2, using a pseudorandom number generator.
zi = a uniformly distributed pseudorandom number in (0,1)
j = int(2*zi) + 1 = 1 or 2
Well say that if j = 1, its heads, if j = 2 its tails.
b.
Roll of a die
In this case we have six outcomes, of equal probability (we hope). So we need to produce an
integer from 1 to 6.
j = int(6*zi)+1 = 1, 2, 3, 4, 5 or 6
Now, if it is known that the die is loaded, we use a different scheme, creating subintervals in
(0,1) whose lengths reflect the relative probabilities of the faces of the die coming up. For
instance, we might say that
zi

j
1
2
3
4
5
6

0 z i 0.2
0.2 z i 0.34
0.34 z i 0.56
0.56 z i 0.72
0.72 z i 0.89
0.89 z i 1

3.

Integration

Thinking again of the definite integral as an area under a curve, we envision a rectangle whose
area is equal to the total area under the curve f(x). The area of that equivalent rectangle is just the
length of the integration interval (a,b) times the average value of the integrand over that interval.
How to take that average? One way is to sample the integrand at randomly selected points.

48

a.

One dimensional definite integrals

1 n
f ( xi ) , where the {xi} form a pseudorandom sequence uniformly distributed in
n i 1

f ( x) dx

(0,1). Over some other interval,

f ( xi ) , where xi a, b .
f ( x)dx (b a) n
i 1
a

Since we are just averaging over a list of numbers, the error is O[

1
n

], just like the deviation of

the mean.
1

example:

sin xdx
0

sin xdx 3 sin 0.00075 sin 0.01335 sin 0.3904 0.1313

0

sin xdx 4 sin 0.00075 sin 0.01335 sin 0.3904 sin 0.8776 0.2910
0

sin xdx 5 sin 0.00075 sin 0.01335 sin 0.3904 sin 0.8776 sin 0.0992 0.2524
0

The exact result is 0.460.

b.
Multi-dimension integrals
The random sampling approach is particularly useful with 2- and 3-dimensional integrals. The
other methods of numerical integration quickly become too messy to set up.
1 1 1

f ( x, y, z )dxdydz

0 0 0

1 n
f ( xi , y i , z i ) ,
n i 1

where (xi,yi,zi) is an ordered triple, each member uniformly distributed on (0,1).

We may use three separate sequences of pseudorandom numbers or simply take numbers from
one sequence three at a time.
c.

Alternate integration regions

bz b y bx

i)

f ( x, y, z ) dxdydz bx a x b y a y bz a z

az a y ax

1 n
f ( xi , y i , z i )
n i 1

ii) Suppose the integration region is not rectangular. Then an extra step is needed, to test for and
discard random points that fall outside the integration region.
e.g., a circlediscard points for which xi2 yi2 1 , as shown in the following diagram.

49

Why do it this way; to ensure that the points are uniformly distributed in all directions. If points
are taken uniformly distributed in the radius, the points will be more widely spread the further
out from the center they lie, not uniformly spread over the area of the circle.
example: compute the volume of a sphere of radius R. In this situation, the integrand is 1.
R 2

sin d d dr

0 0 0

4
R3
3

Numerically,
1
V R R R R R R
n

2R 3
m

m 8R 3

i 1

xi2 yi2 zi2 R 2

Notice this: the total number of random points generated is n. However, only m of those lie
within the spherical volume. The spherical volume we obtain is equal to

m
times the volume
n

of a cube whose side is 2R. Its interesting to see what this fraction is.
Vsphere
Vcube

The ratio

4
R3

3 3 0.52359 .
6
8R

m
should approach this constant as we generate more points and include them in the
n

summation.
Another way to look at this

m
issue is to say that f(x) = 1 when xi2 yi2 z i2 R 2 and 0 when
n

xi2 yi2 z i2 R 2 . Then there is no distinction between n and m, and the summation is a sum of n

m zeros and m ones.

50

d.

Example

Evaluate

sin

ln( x y 1dxdy

1
1
x y
2
2

, where is the region

r2 .

1 1

I sin ln( x y 1dxdy r r

2
2

1
n

f ( xi , y i )

4r 2

i 1
2
2
1
1
2
x
i yi r
2
2

1
n

f ( xi , y i )

i 1
2
2
1
1
2
x
i yi r
2
2

[If you want to try it, for r = 0.5, I = 0.57.]

This is equivalent to averaging the integrand over a circular area, thusly

sin

ln( x y 1dxdy r 2

1
m

f ( xi , y i )

i 1
1
xi
2

1
2
yi r
2

Of course, often the shape of the region of integration isnt a simple rectangle or circle.
e. Example script
% Script to carry out 2-dim integration via random sampling
f=inline('sin(sqrt(log(x+y+1)))')
n=100;
r=0.5;
r2=r*r;
sum=0;
ax=0.5-r;
ay=0.5-r;
bx=r+0.5;
by=r+0.5;
m=0
for i=1:n
ex=rand*(bx-ax)+ax;
why=rand*(by-ay)+ay;
are=(ex-0.5)^2+(why-0.5)^2;
if are<=r2

51

sum=sum+f(ex,why);
m=m+1;
end
end
sum=sum*(by-ay)*(bx-ax)/m;
fprintf('Integral of f(x,y) over the circle = %g',sum)

52

MatLab Sidelight Number Two

1.

Nonlinear Equations

a.
Fzero
The built-in MatLab command for solving individual equations is named fzero.
x = fzero(function,x0)
The initial guess is x0; function is either a mathematical expression typed as a string, or the
name of a user defined function. The function has to entered in standard form: f(x) = 0.
function is the f(x). If entered as a string, the function cannot include redefined variables.
The initial guess can be entered as a single value, or as a 2-element vector such that the function
crosses the x-axis between x0(1) and x0(2). (As in the bisection or secant methods.)
b.
Maximun/minimum
A function to find the minimum of a function is fminbnd.
[x fval] = fminbnd(function,x1,x2)
The command finds the minimum of the function, if any, lying in the interval (x1,x2).
2.

Integration

a.
Integrand as function
The quad command evaluates a definite integral using an elaborated version of Simpsons Rule.
The method adjusts the step size as it goes along.
b

f ( x)dx

q = quad(function,a,b,tol)

The parameter tol is an optional tolerance. If tol is not specified, MatLab assumes tol 10 6 .
The quadl command carries out the integration using another method, the adaptive Lobatto
method. Thats quad-L.
b.
Integrand as data table
When the integrand is available as a table of data pairs, MatLab uses the Trapezoid Rule.
q = trapz(x,y)

53

c.
Random numbers
For generating uniformly distributed pseudorandom numbers, MatLab has the command rand.
A single random number in (0,1): z = rand
A vector of n random numbers in (0,1): z =zrand(1,n)
An nxn matrix of random numbers in (0,1): z = rand(n)
An mxn matrix of random numbers in (0,1): z = rand(m,n)
A row vector with n elements consisting of a random permutation of integers 1 n:
m = randperm(n)

54

VII.Ordinary Differential Equations

A.

Linear First Order Equations

We seek to solve the following equation for x(t):

dx
f ( x, t ) . There are analytical methods of
dt

solution: integration, separation of variables, infinite series, etc. In practice these may not be
convenient or even possible. In such cases we resort to a numerical solution. The x(t) takes the
form of a table of data pairs {ti,xi}, rather than a function.
1.

One Step Methods

a.
Taylors Series
Many numerical solutions derive from the Taylors series expansion
dx(t o ) (t t o ) 2 d 2 x(t o )
(t t o ) p d p x (t o )
x(t ) x (t o ) (t t o )

.
dt
2!
p!
dt 2
dt p
dx
f ( x, t ) , so we could substitute this into the series thusly:
dt
(t t o ) 2 df ( x o , t o )
(t t o ) p d p 1 f ( x o , t o )
x(t ) x (t o ) (t t o ) f ( x o , t o )

.
2!
dt
p!
dt p 1
df d 2 f d 3 f
However, to obtain
,
,
, etc., we have to use the chain rule.
dt
dt 2
dt 3
df
f
f dx

dt
t x dt
2
2
d f
2 f
2 f
f f
f
2 f

2
f

2
2
2

x
x
dt
t
dx

We are given

Its easy to see that this gets very messy rather quickly.
b.
Eulers Method
Lets keep just the first two terms of the Taylors series: x(t ) x(t o ) (t t o ) f ( xo , t o ) To ,
where the To is the sum of all the terms were droppingcall it the truncation error. In what
follows, we will have to distinguish between the correct or exact solution, x(t), and our
approximate solution, xi. We hope xi x(t i ) .
With the Euler Method, our algorithm is [given to, x(to) = xo and f(x,t)]
x1 xo (t1 t o ) f ( x o , t o )

x 2 x1 (t 2 t1 ) f ( x1 , t1 )
xi 1

xi (t i 1 t i ) f ( xi , t i )

example:

dx
13t , with to = 0 and xo = 4 and (t i 1 t i ) h 0.5 .
dt

55

The algorithm is: xi 1 xi (t i 1 t i )13t i .

The first few steps in the numerical solution are shown in the following table.
i
t
x
0
0
4
1
.5
4
2
1
7.25
3
1.5
13.75
4
2
23.5

example: Problem 9-35

dv
0.0035 v 2 3
dt
v o 83.333

i = 0, 1, 2, 3, 4, 5, . . .
t 0.1
vi 1 vi 0.1 * (0.0035 * vi2 3)

i
0
1
2
3
4
5
6
7
2.
a.

t (sec)
0
.1
.2
.3
.4
.5
.6
.7

v (m/s)
83.3
80.6
78.0
75.6
73.3
71.1
69.0
67.1

Error
Truncation error

xi 1 xi hf ( xi , t i ) Ti 1

Not only do we not know what the exact solution is, we dont know how far the numerical
solution deviates from the exact solution. In the case of a truncated Taylors series, we can
estimate the truncation error by evaluating the first term that is dropped. For Eulers formula,
thats the third term of the series.
Ti 1

h 2 df ( xi ) h 2

f ( xi )
2 dt
2

Heres a graph of both the exact (but unknown) and the numerical solutions.

56

The deviation from the exact x(t) may tend to increase as the total truncation error accumulates
from step to step, the further we get from the initial values (to,xo). The lesson ismake h small.
b.
Round-off error
Since the values are stored in finite precision, round-off error accumulates from step to step also.
Therefore, in traversing an interval o t b , wed like to have as few steps as possible. In other
words, we want h to be large. Consequently, the two sources of error put competing pressure on
our choice of step size, h. If we have some knowledge of x(t), we may be able to achieve a
balance between large and small step size. Otherwise, its trial and error.
c.
Higher order methods
The many numerical algorithms that have been developed over the years for solving differential
equation seek to reduce the effect of truncation error by using more terms from the Taylors
series, or in some way correcting for the truncation error at each step. In that way, fewer, larger
steps can be used.

57

MatLab Sidelight Number Three

1.
First Order Ordinary Differential Equations (ODE)
In standard form,
dy
f ( x, y )
dx

We want to solve for y(x). In Physics & Engineering, of course, we often have

dy
f ( y, t ) , in
dt

dv x
f (v x , x, t ) , etc.
dt
MatLab has several built-in ODE solvers, which all have the form of
[t,y]=solver_name(ODEfunc,tspan,y0)
ODEfunc is the name of the function file which defines the differential equation, the f(t,y).
tspan is a vector that specifies the interval of the independent variable spanned by the
solution.
y0 is the initial value of y.
[t,y] is the output, in the form of two column vectors. Subsequently, we would plot(t,y).

which case we wish to solve for y(t); or perhaps we have

a.

Function file

The function file calculates

dy
for given values of y & t. That is, t & y are input arguments to
dt

the function, and the value of f(t,y) is returned.

b.
Solvers
Table 9-1 lists some of the MatLab initial-value ODE solvers. Some are more sophisticated
than others; some are adapted to problems in which the solution is not smooth, or is rapidly
varying, etc. In most physical and engineering applications, things are smooth and not toorapidly varying, so most times ode45 should suffice.
[t,y] = ode45(function,[0:0.1:10],100)
plot(t,y)
MatLab also has boundary value and partial differential equation solvers, but those are not
discussed in the introductory text, nor in this class.
example: Problem 9-35
dv
0.0035 v 2 3 m/s 2
dt
v o 300 km/hr

v0 = 300/60/60*1000
[t,v] = ode45(drag,[0:0.1:15],v0)
plot(t,v)
function dvdt = drag(t,v)
dvdt = -0.0035*v*v-3;

58

B.

1.

Second Order Ordinary Differential Equations

d 2x
dx
x f (t , x, ) f (t , x, x ) , with initial conditions x(0) x o and x (0) vo .
2
dt
dt
Reduction to a System of First Order Equations

a.
New Variables
We start by introducing new variable names: z1 t ; z 2 x ; z 3 x ; z 4 x . The first three
variables are the solutions to the following differential equations:
z1 1
z 2 x z 3
z 3 x z 4
These form a set of three simultaneous first order differential equations,
z1 1
z 2 z 3
z 3 z 4 f ( z1 , z 2 , z 3 )
with the initial conditions z1 (0) 0 , z 2 (0) x o and z 3 (0) vo respectively.
b.
Solution
Any method, such as Eulers, may now be applied to each first order equation in turn. Thusly:
z1,i 1 z1,i h 1

z 2,i 1 z 2,i h z 3,i

z 3,i 1 z 3,i h f i .

The MatLab code might look like this:

z(1) = 0.0
z(2) = xo
z(3) = vo
h = 0.01
for i=1:100
z(1) = z(1) + h
z(2) = z(2) +h*z(3)
z(3) = z(3) + h*f(z(1),z(2),z(3))
end
c.

Example

x x 9 cos( t )
x(0) x o , x (0) v o
In this case, f (t , x, x ) x 9 cos( t ) , so the algorithm looks like
z1,i 1 z1,i h 1

z 2,i 1 z 2,i h z 3,i

z 3,i 1 z 3,i h z 3,i 9 cos( z1,i ) .

59

2.

Difference Equations

An alternative approach to second order ordinary differential equations is to replace the

derivatives with finite differences. The differential equation is replaced by a difference equation.
a.
Difference equation
Using forward divided differences, we obtain
x 2 xi xi 1
d 2 x 1 xi 1 xi xi xi 1
dx xi 1 xi

i 1
x

and x 2
.
h
h
h
dt
h2
dt
h

Lets say that we have the second order differential equation

x ax bx ct d .
The corresponding difference equation is
xi 1 2 xi xi 1
x xi
a i 1
bxi ct i d .
2
h
h

The next step is to solve for the latest x.

xi 1 2 xi xi 1 ahxi 1 ahxi bh 2 xi ch 2 t i dh 2

1 ah xi 1 2 ah bh 2 xi xi 1 ch 2 t i dh 2

1
2 ah bh 2 xi xi1 ch 2t i dh 2
1 ah
The initial conditions are applied by setting to = 0, x0 = xo and x 1 xo v o h .
xi 1

b.

Examples
i) x g
Here, a b c 0 and d = -g.
xi 1 2 xi xi 1 gh 2

ii) x x g
This time, a = -1, b = 0, c = 0 and d = -g.
xi 1

1
2 h xi xi 1 gh 2
1 h

c.
Discretization error
Replacing continuous derivatives with finite differences introduces what is known as
discretization error. Implicitly, we are assuming a straight line between xi and xi+1 and between
xi and xi1 as well. There will always be some xi 1 x(t i 1 ) at each step which will then
accumulate over the sequence of steps in the numerical solution.

60