7201 Song Chap5 (Compatibility Mode)

Computational Methods in Engineering: Chapter 5
Computational Methods in Engineering

A/P Song Qing
EEE Division 4
e-mail: eqsong@ntu.edu.sg
S2-B2c-113
Ref Text: 1) Scientific Computing – An Introductory Survey,
Second Edition by Michael Heath, McGraw Hill
2) Nonlinear Programming by Dimitri Bertsekas, Athena Scientific
3) Numerical Methods for Unconstrained Optimization and
Nonlinear Equations by J.E.Dennis and Robert B. Schnabel,
Prentice Hall
4) Robust Training Algorithms of Neural Networks
http:/www.ntu.edu.sg/home/eqsong
3 Sep 07 1
3 Sep 07 2
3 Sep 07 3
MAIN CONCEPTS
1. Overview of Optimization and Nonlinear Equations

i. Single nonlinear equation
ii. n coupled nonlinear equations
iii. Existence and Uniqueness for Single nonlinear equation
2. Existence and Uniqueness
i. Single and Multiple Root
ii. Conditioning
iii. Convergence Rate
3. Solution Computation
i. Interval Bisection Method for single nonlinear equation
ii. Fixed Point Problems for single nonlinear equation
iii. Newton Method for single nonlinear equation
iv. Secant Method for single nonlinear equation
v. Inverse Quadratic Interpolation
vi. Linear Fractional Interpolation
vii. Newton’s Method for coupled nonlinear equations
viii. Broyden’s Method for coupled nonlinear equations
3 Sep 07 4
Types of Optimization Problem
3 Sep 07 5
Existence of Optimal Solutions
3 Sep 07 6
Unconstrained Optimization: 1st Order Necessary Condition
3 Sep 07 7
Unconstrained Optimization: 2nd Order Necessary Condition
3 Sep 07 8
Rates of Convergence
3 Sep 07 9
Example: Rates of Convergence
3 Sep 07 10
3 Sep 07 11
3 Sep 07 12
3 Sep 07 13
3 Sep 07 14
3 Sep 07 15
3 Sep 07 16
3 Sep 07 17
3 Sep 07 18
3 Sep 07 19
3 Sep 07 20
3 Sep 07 21
3 Sep 07 22
3 Sep 07 23
3 Sep 07 24
3 Sep 07 25
3 Sep 07 26
3 Sep 07 27
3 Sep 07 28
3 Sep 07 29
3 Sep 07 30
3 Sep 07 31
3 Sep 07 32
3 Sep 07 33
3 Sep 07 34
3 Sep 07 35
3 Sep 07 36
3 Sep 07 37
3 Sep 07 38
3 Sep 07 39
3 Sep 07 40
3 Sep 07 41
3 Sep 07 42
3 Sep 07 43
3 Sep 07 44
3 Sep 07 45
3 Sep 07 46
3 Sep 07 47
3 Sep 07 48
3 Sep 07 49
3 Sep 07 50
3 Sep 07 51
3 Sep 07 52
3 Sep 07 53
Local Quadratic Convergence of Newton’s Method

Let f : R n → R n be continuously differentiable in an
open convex set D ⊂ R n . Assume that there exists x* ∈ R n
and r , β > 0, such that N ( x* , r ) ⊂ D, f ( x* ) = 0, J ( x* ) −1
exists with J ( x* ) −1 ≤ β , and J ( x) − J ( y ) ≤ γ x − y for
all x, y ∈ N ( x* , r ) [i.e. Lipschitz continuity]. Then there exists
ε > 0 such that for all x0 ∈ N ( x* , ε ), the sequence x1 , x2 , K
generated by
xk +1 = xk − J ( x* ) −1 f ( xk ), k = 0, 1, K
is well defined, converges to x* , and obeys
xk +1 − x* ≤ βγ xk − x* , k = 0, 1, K
2
Proof : Dennis Jr and Schnabel, Pg 90.

3 Sep 07 54
3 Sep 07 55
Global Methods for Nonlinear Equations
Given f : R n → R n
Find x* ∈ R n such that f ( x* ) = 0 (1)
The Newton step is given by
xk +1 = xk − J (xk ) f ( xk )
−1
Newton' s method is locally quadratically convergent but

not necessarily globally convergent.
Instead, consider the following minimization problem
1
minn F ( x) = f ( x)T f ( x) (2)
x∈R 2
Note :
a) Every solution to (1) is a solution to (2)
b) There may be local minimizes of (2) that are not solutions to (1)
3 Sep 07 56
3 Sep 07 57
3 Sep 07 58
i.e. f ( xk +1 ) ≈ f ( xk ) + Bk +1 ( x k +1 − xk )
3 Sep 07 59
Motivation for Broyden’s Formula

f ( xk +1 ) ≈ f ( xk ) + Ak +1 ( x k +1 − xk )
i.e. Ak +1 ( x k +1 − xk ) = Ak +1sk = f ( xk +1 ) − f ( xk ) = yk : Secant Equation
Choose
Bk +1 = Bk +
( yk − Bk sk )skT
then
T
s s
k k
Bk +1 − Bk =
( yk − Bk sk )skT =
( Ak +1 − Bk )sk skT
T
s s
k k skT sk
sk skT
≤ Ak +1 − Bk T
= Ak +1 − Bk
sk sk
for all Ak +1 satisfying the secant equation
3 Sep 07 60
Factorization of Bk+1
Consider the following rank 1 update :
Bk +1 = Bk + uvT
where u , v ∈ R n and Bk is nonsingular.
Let
Bk = Qk Rk and w = QkT u
Then,
(
Bk +1 = Qk Rk + uvT = Qk Rk + wv T )
Form the QR decomposition
~
Rk + wv T = QRk +1 (1)
Hence,
~
Bk +1 = Qk QRk +1 = Qk +1 Rk +1
Givens Rotation is used to obtain QR decomposition of (1) in O(n 2 ) operations
3 Sep 07 61
Bounded Deterioration Property of Broyden’s Method

Lemma : Let D ⊆ R n be an open convex set containing
xk , xk +1 with xk ≠ x* . Let f : R n → R n , J (v) − J ( x) ≤ γ v − x
for all v, x ∈ D, Bk ∈ R n×n , Bk +1 defined by Broyden's Method.
This image cannot currently be display ed.
Then for either the Frobenius norm ( A F = tr A A [( T

)] 1/ 2
) and
l2 − matrix norms,
3γ
Bk +1 − J ( xk +1 ) ≤ Bk − J ( xk ) +
xk +1 − xk 2
2
Furthermore, if x* ∈ D and J ( x) obeys the weaker Lipschitz condition
J ( x) − J ( x* ) ≤ γ x − x* for all x ∈ D (i.e. v = x* only)
Then the following bounded deterioration property holds :
γ
Bk +1 − J ( x* ) ≤ Bk − J ( x* ) +
2
( x k +1 − x* 2 + xk − x* 2
) (1)
3 Sep 07 62
Local Convergence of Broyden’s Method
• We will show that if x0 is sufficiently close to a root x*, where J(x*) is
nonsingular, and if B0 is sufficiently close to J(x0), then the sequence of
iterates {xk} converges superlinearly to x*
This image cannot currently be display ed.
Theorem : Let f : R n → R n be continuously differentiable in an
open convex set D ⊂ R n . Assume that there exists x* ∈ R n
and r , β > 0, such that N ( x* , r ) ⊂ D, f ( x* ) = 0, J ( x* ) −1
exists with J ( x* ) −1 ≤ β , and J ( x) − J ( y ) ≤ γ x − y for
all x, y ∈ N ( x* , r ) [i.e. Lipschitz continuity]. Then there exists
ε , δ > 0 such that if x0 − x* 2 ≤ ε and B0 − J ( x* ) ≤ δ , then the sequence
{xk } generated by Broyden's method is well defined and converges super -
linearly to x* . If {Bk }is assumed to satisfy just the bounded deterioration
property (1), then {xk } converges at least linearly to x* .
3 Sep 07
Proof : Dennis Jr and Schnabel, Pg 177. 63
3 Sep 07 64
3 Sep 07 65
3 Sep 07 66
3 Sep 07 67
3 Sep 07 68

7201 Song Chap5 (Compatibility Mode)

Enviado por

Dados do documento

Descrição original:

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

7201 Song Chap5 (Compatibility Mode)

Enviado por

Direitos autorais:

Formatos disponíveis

Computational Methods in Engineering: Chapter 5

Computational Methods in Engineering

1. Overview of Optimization and Nonlinear Equations

Types of Optimization Problem

Existence of Optimal Solutions

Unconstrained Optimization: 1st Order Necessary Condition

Example: Rates of Convergence

Local Quadratic Convergence of Newton’s Method

Proof : Dennis Jr and Schnabel, Pg 90.

Newton' s method is locally quadratically convergent but

Motivation for Broyden’s Formula

Bounded Deterioration Property of Broyden’s Method

Then for either the Frobenius norm ( A F = tr A A [( T

Você também pode gostar