Last Time

• NP
• NP completeness

Approximation Algorithms
Philip Bille
(slides heavily inspired by slides of Kevin Wayne)

Suite of Known NP-complete Problems Agenda

• Key NP-complete problems: • Approximation Algorithms

• Independent Set • Dealing with “hard”-problems.
• Vertex Cover • Load Balancing
• Set Cover • Center Selection
• Set Packing
• Satisfiability, 3-Sat, Circuit Sat
• Subset Sum
• Hamilton Cycle
• Traveling Salesperson
• Finding more:
• “A compendium of NP optimization problems”
• http://www.nada.kth.se/~viggo/problemlist/compendium.html
 • Q. Suppose I need to solve an NP-hard problem. What should I do?
NP-hardness • A. Theory says you're unlikely to find a poly-time algorithm.

• How do we talk about hardness of optimization problems Y? • Must sacrifice one of three desired features.
• Solve problem to optimality.
• Y is NP-hard if for all X in NP, X !P Y. • Solve problem in poly-time.
• Y is at “least as hard” as any problem in NP. • Solve arbitrary instances of the problem.

• Ex: Finding the largest independent set is NP-hard. • !-approximation algorithm.

• Guaranteed to run in poly-time.
• Guaranteed to solve arbitrary instance of the problem
• Guaranteed to find solution within ratio ! of true optimum.

• Challenge. Need to prove a solution's value is close to optimum, without

even knowing what optimum value is!

Load Balancing Load Balancing

• How to schedule a set of n jobs on m machines so that the loads on all
machines is a balanced as possible?
A B C D
F G
• Input. m identical machines; n jobs, job j has processing time tj.
• Job j must run contiguously on one machine.
E • A machine can process at most one job at a time.
• Def. Let J(i) be the subset of jobs assigned to machine i. The load of
machine i is Li = !j " J(i) tj.

H I J • Def. The makespan is the maximum load on any machine L = maxi Li.
• Load balancing. Assign each job to a machine to minimize makespan.
• Jobs A-J scheduled optimally on 3 machines:
D E H Machine 1 C B
D E H Machine 1 C B
A Machine
G 2
A Machine
G 2
I F Machine 3 J
I F Machine 3 J
Greedy Algorithm List Scheduling Algorithm
• Consider n jobs in some fixed order.
• Load balancing is NP-hard.
• Assign job j to machine whose load is smallest so far.
List-Scheduling(m, n, t1,t2,…,tn) {
• Q. What is a natural way to solve the problem?
for i = 1 to m {
• A. Greedy algorithm: Li # 0 load on machine i
J(i) # $ jobs assigned to machine i
• Consider n jobs in some arbitrary order. }
• Assign job j to machine whose load is smallest so far.
for j = 1 to n {
i = machine with smallest load
J(i) # J(i) % {j} assign job j to machine i
• Called the list scheduling algorithm
L i # Li + t j update load of machine i
}
}

• Running time?

List Scheduling Algorithm List Scheduling Algorithm

• Consider n jobs in some fixed order.
• Fast and simple greedy algorithm.
• Assign job j to machine whose load is smallest so far.
• Poly-time.
List-Scheduling(m, n, t1,t2,…,tn) {
• Finds a valid solution.
for i = 1 to m {
Li # 0 load on machine i
J(i) # $ jobs assigned to machine i
}
• How good is the solution produced by the list scheduling algorithm compared
with the optimal solution?
for j = 1 to n {
i = machine with smallest load
J(i) # J(i) % {j} assign job j to machine i
L i # Li + t j update load of machine i
}
}

• Running time?
• A. O(n log m) using a priority queue.
 A B C D E B C D E

F G F G

H I J H I J

Machine 1 A Machine 1

Machine 2 Machine 2

Machine 3 Machine 3

0 Time 0 Time

C D E D E

F G F G

H I J H I J

A Machine 1 A Machine 1

B Machine 2 B Machine 2

Machine 3 C Machine 3

0 Time 0 Time
 E

F G F G

H I J H I J

A Machine 1 A E Machine 1

B D Machine 2 B D Machine 2

C Machine 3 C Machine 3

0 Time 0 Time

H I J H I J

A E Machine 1 A E Machine 1

B D Machine 2 B D Machine 2 G

C F Machine 3 C F Machine 3

0 Time 0 Time
 I J J

A E HMachine 1 A E HMachine 1 I

B D Machine 2 G B D Machine 2 G G

C F Machine 3 C F Machine 3

0 Time 0 Time

D E H Machine 1 C B

A Machine
G 2

I F Machine 3 J

0
Optimal Schedule

A E HMachine 1 I A E HMachine 1 I

B D Machine 2 G B D Machine 2 G

C F Machine 3 J C F Machine 3 J

0 Time 0
List schedule
 List Scheduling Analysis List Scheduling Analysis

• Approximation guarantee: • We compare the greedy solution L to the optimal makespan L*.
• For any instance of the Load Balancing problem: If
• L is the makespan produced by the list scheduling algorithm • Lemma 1. The optimal makespan L* & maxj tj.
• L* is the minimum makespan (optimal solution) • Pf. Some machine must process the most time-consuming job. "
• then L* ! L ! 2L*.
• Lemma 2. The optimal makespan L * " 1
m#j
tj.
• => List scheduling is a 2-approximation algorithm to the Load Balancing • Pf. The total processing time is !j tj .
problem.
• One of m machines must do at least a 1/m fraction of total work. "
!

List Scheduling Analysis List Scheduling Analysis

• Theorem. Greedy algorithm is a 2-approximation. • Theorem. Greedy algorithm is a 2-approximation.

• Pf. Consider load Li of bottleneck machine i. • Pf. Consider load Li of bottleneck machine i.
• Let j be last job scheduled on machine i. • Let j be last job scheduled on machine i.
• When job j was assigned to machine i, i had smallest load. Its load before • When job j was assigned to machine i, i had smallest load. Its load before
assignment is Li - tj ' Li - tj ( Lk for all 1 ( k ( m. assignment is Li - tj ' Li - tj ( Lk for all 1 ( k ( m.
• Sum inequalities over all k and divide by m:
blue jobs scheduled before j
Li " tj # m1 $ k Lk
= m1 $ k t k
Lemma 2 # L*
machine i j

• Now Li = (Li !
"tj) + tj # 2L *.
0 1424 3 {
L i - tj L = Li # L* # L*
Lemma 1

!
Exercise Session: Tightness List Scheduling Analysis

• Is the analysis of the list scheduling algorithm tight? • Q. Is our analysis tight?
• A. Almost.
• Find an instance with large approximation ratio: We have shown L ! 2L*, but • Ex: m machines, m(m-1) jobs length 1 jobs, one job of length m
are there instances where L is close to 2L*?
machine 2 idle
machine 3 idle
machine 4 idle

m = 10 machine 5 idle
machine 6 idle
machine 7 idle
machine 8 idle
machine 9 idle
machine 10 idle

list scheduling makespan = 19

List Scheduling Analysis Longest Processing Time Rule

• Q. Is our analysis tight? • Longest processing time (LPT). Sort n jobs in descending order of processing
• A. Almost. time, and then run list scheduling algorithm.
• Ex: m machines, m(m-1) jobs length 1 jobs, one job of length m LPT-List-Scheduling(m, n, t1,t2,…,tn) {
Sort jobs so that t1 ! t2 ! … ! tn

for i = 1 to m {
Li # 0 load on machine i
J(i) # $ jobs assigned to machine i

m = 10 }

for j = 1 to n {
i = machine with smallest load
J(i) # J(i) % {j} assign job j to machine i
L i # Li + t j update load of machine i
}
}

optimal makespan = 10
Analysis Analysis

• Observation. If at most m jobs, then list-scheduling is optimal.
• Pf. Each job put on its own machine.
• Observation. If at most m jobs, then list-scheduling is optimal.
• Lemma 3. If there are more than m jobs, 2tm+1 ! L*
• Pf. Consider first m+1 jobs t1, …, tm+1.
• Since the ti's are in descending order (t1 " t2 "... " tm+1), each takes at least
tm+1 time.
• There are m+1 jobs and m machines, so by pigeonhole principle, at least
one machine j gets two jobs.
• The load on machine j is at least 2tm+1.

Analysis Load Balancing: LPT Rule

• Observation. If at most m jobs, then list-scheduling is optimal. • Q. Is our 3/2 analysis tight?
• A. No.
• Lemma 3. If there are more than m jobs, 2tm+1 ! L*
• Theorem. LPT rule is a 3/2-approximation algorithm. • Theorem. [Graham, 1969] LPT rule is a 4/3-approximation.
• Pf. If at most m jobs the result follows from observation. • Pf. More sophisticated analysis of same algorithm.
• Otherwise: Same basic approach as for list scheduling.
• If the bottleneck machine i only has one job then Li ! L*. • Q. Is Graham's 4/3 analysis tight?
• Let j be the last job scheduled on bottleneck machine i (j " m+1 by above). • A. Essentially yes.
• Since j " m+1 => tj ! tm+1 ! L*/2. Now:
3 L *.
• Ex: m machines, n = 2m+1 jobs, 2 jobs of length m+1, m+2, …, 2m-1 and
L i = (Li " t j ) + t j #
1424 3 { 2 one job of length m.
# 1 L*
# L* 2

Lemma 3

!
Center Selection Center Selection Problem

• How can we place k shopping centers on a map so that none of n customers
will have longer distance to travel than necessary?
• Input. Set of n sites s1, …, sn.
• Center selection problem. Select k centers C (can be anywhere) so that
maximum distance from a site to nearest center is minimized.
k=4

r(C)

center
site

Center Selection Problem Brute force search

• Notation. • Each site is a point in the plane, a center can be any point in the plane.
• dist(x, y) = distance between x and y. • Searching may take infinite time.
• dist(si, C) = min c " C dist(si, c) = distance from si to closest center.
• r(C) = maxi dist(si, C) = smallest covering radius.
• Goal. Find set of centers C that minimizes r(C), subject to |C| ! k.

• We assume a metric distance function:

• dist(x, x) = 0# # # # (identity)
• dist(x, y) = dist(y, x)# # # (symmetry)
• dist(x, y) ( dist(x, z) + dist(z, y)$ (triangle inequality)
A First Greedy Algorithm Knowing the Optimal Radius Helps

• Greedy algorithm. Put the first center at the best possible location for a • Suppose that we know the optimal radius r, i.e., that there exists a set C* of k
single center, and then keep adding centers so as to reduce the covering centers with radius r(C*) = r.
radius each time by as much as possible. • We use this information to efficiently find a set of k centers with radius at
most 2r.

• Later we remove the assumption by “guessing” values of r.

greedy center 1

center
k = 2 centers site

• Remark: arbitrarily bad!

Greedy Algorithm Analysis

Greedy-Center-Selection-with-r(r, k, n, s1,s2,…,sn) {
• Suppose:
• C is the set of centers found by the greedy algorithm.
C = $
• C* is an optimal set of centers with r(C*) = r.
S’ = S
while S’ is not empty { • We show that:
Select any site s and add s to C
Delete all sites from S’ with distance at most 2r from s
• r(C) ! 2r.
} • |C| ! k.
return C
} • => C is a valid solution and r(C) ! 2r(C*) => Greedy algorithm is a 2-
approximation.

• Intuition:
• If c is a chosen center, c is covered by an optimal center c* within distance
at most r.
• c covers everything within distance 2r => c covers all that c* covers.
Analysis Analysis

• Claim: r(C) ! 2r • Claim: |C| ! k.

• Pf: By design of algorithm. • Pf:
• There is a set of centers C* with |C*| = k and covering radius r(C*) = r.
• Each center c in C selected by greedy is a site in S => c is covered by at
Greedy-Center-Selection-with-r(r, k, n, s1,s2,…,sn) { least one center in C* within distance at most r. Let c* be the closest such
center. We say c is close to c*.
C = $
S’ = S • No other center c’ in C is close to c*. By contradiction:
while S’ is not empty {
> 2r c’
Select any site s and add s to C c
Delete all sites from S’ with distance at most 2r from s
} !r
return C !r
} c*

• => |C| ! |C*| = k.

Guessing the Optimal Radius An Improved Greedy Algorithm

• Try values of r in the range [0, max dist] using binary search. • Greedy algorithm. Repeatedly choose the next center to be the site farthest
• Can get close to factor 2 approximation. from any existing center.
Greedy-Center-Selection(k, n, s1,s2,…,sn) {

• Improved and simpler greedy algorithm avoids guessing r. C = $

repeat k times {
Select a site si with maximum dist(si, C)
Add si to C
}
site farthest from any center
return C
}

• Achieves factor 2-approximation

Center Selection Summary

• Theorem. Greedy algorithm is a 2-approximation for center selection • Approximation algorithms

problem. • Load Balancing
• Center selection
• Remark. Greedy algorithm always places centers at sites, but is still within a
factor of 2 of best solution that is allowed to place centers anywhere.

• Question. Is there hope of a 3/2-approximation? 4/3?

• Theorem. Unless P = NP, there no !-approximation for center-selection
problem for any ! < 2 (for general metrics, not euclidean).

Next Time

• Randomized Algorithms
• Exercises: KT Chap. 11: 1