Recurrence Relations

CS 3110

Textbook readings:
Chapter 4

CS 3110 Recurrence Relations

1/12

Overview

What are recurrence relations? What is their relationship to algorithm design? Methods to solve them: Substitution (induction in disguise) Recursion trees Master Theorem

CS 3110 Recurrence Relations

2/12

What Are Recurrence Relations?

A recurrence relation expresses the value of a function f for an argument n in terms of the values of f for arguments less than n. Examples:

f (n) =

1 if n 1 2f (n/2) + n if n > 1

f (n) =

5 if n 10 f (n/10) + f (n 1) + f (n/2) if n > 10

CS 3110 Recurrence Relations

3/12

Recurrence Relations and Recursive Algorithms

The running time of a recursive algorithm is easily expressed using a recurrence relation. Example: (Merge-Sort) Merge-Sort(A, p, r) 1 if r > p 2 then q p+r 2 3 Merge-Sort(A, p, q) 4 Merge-Sort(A, q + 1, r) 5 Merge(A, p, q, r)

CS 3110 Recurrence Relations

4/12

Recurrence Relations and Recursive Algorithms

The running time of a recursive algorithm is easily expressed using a recurrence relation. Example: (Merge-Sort) Merge-Sort(A, p, r) 1 if r > p 2 then q p+r 2 3 Merge-Sort(A, p, q) 4 Merge-Sort(A, q + 1, r) 5 Merge(A, p, q, r) Running time: T (n) = O(1) if n 1 2T (n/2) + O(n) if n > 1

CS 3110 Recurrence Relations

4/12

Recurrence Relations and Optimization Problems

Given: Set T of n tasks t1 , t2 , . . . , tn Set K of n contractors k1 , k2 , . . . , kn Contractor ki charges cij dollars for performing task tj Assign one task to every contractor Goal: Minimize the total cost C. A recurrence relation: C = c(T, K) c(T, K) = the minimal cost of assigning the tasks in T to the contractors in K. c(T, K) = 0 if T = min1in (ci1 + c(T \ {t1 }, K \ {ki })) if T =

CS 3110 Recurrence Relations

5/12

Solving Recurrence Relations

Given two algorithms for the same problem. Running times: Algorithm 1: T1 (n) = 2T1 (n/2) + O(n lg n) Algorithm 2: T2 (n) = 3T2 (n/2) + O(n) Which one is faster?

CS 3110 Recurrence Relations

6/12

Solving Recurrence Relations

Given two algorithms for the same problem. Running times: Algorithm 1: T1 (n) = 2T1 (n/2) + O(n lg n) Algorithm 2: T2 (n) = 3T2 (n/2) + O(n) Which one is faster? We need closed forms of T1 (n) and T2 (n). (Expressions for T1 (n) and T2 (n) that are not recurrence relations.)

CS 3110 Recurrence Relations

6/12

Solving Recurrence Relations

Given two algorithms for the same problem. Running times: Algorithm 1: T1 (n) = 2T1 (n/2) + O(n lg n) Algorithm 2: T2 (n) = 3T2 (n/2) + O(n) Which one is faster? We need closed forms of T1 (n) and T2 (n). (Expressions for T1 (n) and T2 (n) that are not recurrence relations.) To solve a recurrence relation means to derive such a closed form from the recurrence relation.

CS 3110 Recurrence Relations

6/12

Methods for Solving Recurrence Relations

Substitution method: Make a guess Verify the guess using induction Recursion trees: Visualize how the recurrence unfolds May lead to a guess to be veried using substitution If done carefully, may lead to an exact solution Master theorem: Cook-book solution to a common class of recurrence relations

CS 3110 Recurrence Relations

7/12

Substitution Method

Three steps: 1. Make a guess 2. Prove that the guess is correct assuming that it can be veried for all n less than some n0 . (inductive step) 3. Verify the guess for all n n0 . (base case)

CS 3110 Recurrence Relations

8/12

Substitution Method

Three steps: 1. Make a guess 2. Prove that the guess is correct assuming that it can be veried for all n less than some n0 . (inductive step) 3. Verify the guess for all n n0 . (base case) Why do we switch the two parts of the inductive proof?

CS 3110 Recurrence Relations

8/12

Substitution Method

Three steps: 1. Make a guess 2. Prove that the guess is correct assuming that it can be veried for all n less than some n0 . (inductive step) 3. Verify the guess for all n n0 . (base case) Why do we switch the two parts of the inductive proof? Our guess is vague.

CS 3110 Recurrence Relations

8/12

Substitution Method

Three steps: 1. Make a guess 2. Prove that the guess is correct assuming that it can be veried for all n less than some n0 . (inductive step) 3. Verify the guess for all n n0 . (base case) Why do we switch the two parts of the inductive proof? Our guess is vague. Example: T (n) cn lg n (that is, T (n) = O(n lg n))

We do not know c, and we do not want to know; we do not care. The inductive step may work only for certain values of c. The base case usually works for any value of c.
CS 3110 Recurrence Relations 8/12

How To Make a Guess

Experience, inspiration, black magic, . . . Recursion trees help us to visualize how the recurrence unfolds. If everything fails, prove loose upper and lower bounds and tighten them.

CS 3110 Recurrence Relations

9/12

Recursion Trees

T (n) = 3T (n/4) + n2 n2

Cost n2

n 2 4

n 2 4

n 2 4

3 16

n2

n 2 16

n 2 16

n 2 16

n 2 16

n 2 16

n 2 16

n 2 16

n 2 16

n 2 16

3 2 16

n2

T (1) T (1)

T (1)

O(nlog4 3 )

Cost of the tree = sum of the costs of the levels T (n) = (n2 )
CS 3110 Recurrence Relations 10/12

Master Theorem

Theorem: (Master Theorem) Let a 1 and b > 1, let f (n) be a function over the positive integers, and let T (n) be given by the following recurrence: T (n) = aT (n/b) + f (n) (i) If f (n) = O(nlogb a ), for some > 0, then T (n) = (nlogb a ). (ii) If f (n) = (nlogb a ), then T (n) = (nlogb a lg n). (iii) If f (n) = (nlogb a+ ), for some > 0, and af (n/b) cf (n), for some c < 1 and all n n0 , then T (n) = (f (n)).

CS 3110 Recurrence Relations

11/12

Summary

A recurrence relation expresses the value of a function f for an argument n in terms of the values of f for arguments less than n. Recurrence relations are useful for expressing the running times of recursive algorithms and the costs of optimal solutions to optimization problems. There are three methods to solve recurrence relations: Substitution method Recursion trees Master theorem

CS 3110 Recurrence Relations

12/12