Tutorial Exercise (Graph Algorithms)

Heaps and Priority Queues

CS3381 Design and Analysis of Algorithms Helena Wong, 2001

1. For an array of input numbers: 1,13,9,5,12,8,7,4,0,6,2,15 a.) Illustrate the operation of BUILD-MAX-HEAP on these data. b.) For the resultant heap A obtained in (a), illustrate the operation of HEAP-EXTRACT-MAX. c.) For the resultant heap A obtained in (a), illustrate the operation of MAX-HEAP-INSERT(A,10). 2. Write pseudocode for the procedures HEAP-MINIMUM, HEAP-EXTRACT-MIN, HEAPDECREASE-KEY, and MIN-HEAP-INSERT that implement a min-priority queue with a min-heap. 3. Why do we want the loop index i in line 2 of BUILD-MAX-HEAP to decrease from n/2 down to 1 rather than increase from 1 to n/2 ?
Answer: When MAX-HEAPIFY is called in the loop, it is assumed that the binary trees rooted at LEFT(i) and RIGHT(i) are max-heaps. To make sure this assumption holds in each iteration, we must handle the nodes from bottom to top, ie. want the loop index i in line 2 of BUILD-MAX-HEAP to decrease from n/2 down to 1 rather than increase from 1 to n/2 .

4. Argue the correctness of HEAP-INCREASE-HEY using the following loop invariant: At the start of each iteration of the while loop, the array A[1 .. n] satisfies the max-heap property, except that there may be one violation: A[i] may be larger than A[PARENT(i)] HEAP-INCREASE-KEY(A,i,key) A[i] = key While i > 1 and A[PARENT(i)] < A[i] Do exchange A[i] and A[PARENT(i)] i = PARENT(i)
15 13 5 12 8 9 7

4 0 6 2 1

Answer: The given loop invariant is not sufficient. We modify it as: At the start of each iteration of the while loop, the array A[1 .. n] satisfies the max-heap property, except that there may be one violation: A[i] may be larger than A[PARENT(i)]. In addition, the parent of A[i] is greater than any child of A[i]. Initialization: Before this routine is executed, it is supposed to be a max-heap. So, any parent of A[i] >= A[i] >= any child of A[i]. After line 1, A[i] is increased to the value of key, that may be larger than the parent of A[i] (but still, any parent of A[i] >= any child of A[i]). Then the iteration begins. Hence, before the first iteration, the loop invariant holds. Maintenance: At the beginning of each iteration, let D be the value of A[i] and P be the value of the parent of A[i]. Suppose the loop invariant is true before each iteration, P is greater than any child of A[i]. In the iteration, if P < D, then they are exchanged, and i is changed to keep the index of the parent, ie. Node i means the node storing the value of D, but that has already been moved upwards. And, P is now moved lower and has new children that are smaller than P. Only that the relationship between node i and its parent becomes unknown. So, after the iteration, the loop invariant holds for the new value of i. Termination: The iteration terminates when case 1: i=1 (ie. A[i] is the root, that has no parent) or case 2: parent of A[i] >= A[i]. In both cases, the truth of the loop invariant imples that A[1..n] satisfies the max-heap property.

Tutorial Exercise (Graph Algorithms)

CS3381 Design and Analysis of Algorithms Helena Wong, 2001

Data structures for Graph Problems 5. Given an adjacency-matrix representation of a directed graph, how long does it take to compute the out-degree of every vertex? How long does it take to compute the in-degrees? Answer: For a directed graph G=(V,E), the adjacency-matrix M is a |V|x|V| matrix such that each cell m[i][j] is 1 if there is an edge (vi,vj) in E, and m[i][j] = 0 otherwise. To compute the out-degree of a vertex vk, we count for all o=1 to |V| such that m[k][o]=1 [O(V)] To compute the in-degree of a vertex vk, we count for all i=1 to |V| such that m[i][k]=1 [O(V)] Given an adjacency-list representation of a directed graph, how long does it take to compute the out-degree of every vertex? How long does it take to compute the in-degrees? Answer: For a directed graph G=(V,E), the adjacency-list representation involves |V| link lists such that list i points to linked nodes storing the vertices vj where (vi,vj) is an edge in E. To compute the out-degree of a vertex vk, we count the linked nodes of list k. [O(V)] To compute the in-degree of a vertex vk, we count for each linked nodes in all the |V| lists such that the linked node is vk . [O(E)] 6. Give an adjacency-list representation for a complete binary tree on 7 vertices, assume that the vertices are numbered from 1 to 7 as in a binary heap. Give an adjacency-matrix representation. Breadth First Search 7. Show the distance and pred values that result from running breadth-first search on the following graph, using vertex 3 as the source.
1 4 2 5 3 6

Answer:
Q 1 /null 4 /null 3 3 0 /null 6 /null 1 /null 4 /null Q 5 2 /null 5 1 /3 6 3 0/null 6 1 /3 2 /null 5 /null

Q 1 /null 4 2 /5

6 2 /null 5 1/3

4 3 0/null 6 1/3 1 /null 4 2/5

4 3 0/null 6 1/3

2 /null 5 1/3

Q 1 /null 4 2/5

2 2 3 /4 5 1/3 3 0/null 6 1/3 4 2/5 1 /null

Q 2 3/4

3 0/null

5 1/3

6 1/3

Tutorial Exercise (Graph Algorithms)

CS3381 Design and Analysis of Algorithms Helena Wong, 2001

8. What is the running time of BFS if its input graph is represented by an adjacency matrix and the algorithm is modified to handle this form of input? Answer: [O(V2)] 9. Argue that in a breadth-first search, the value u.distance is independent of the order in which the vertices in each adjacency list are given. Using the following graph as an example, show that the breadth-first tree computed by BFS can depend on the ordering within adjacency lists.

r v

s w

t x

u y

Answer: u.distance is independent of the order in which the vertices in each adjacency list are given In Breadth-First Search, we explore the edges of a graph to reach every vertex from a given source vertex, with shortest paths. u.distance is the distance of the shortest path of vertex u, hence it is independent of the order in which the vertices in each adjacency list are given. In the example, if we take t as the source vertex, then from t we will go to u and x. Suppose u is in front of x in ts adjacency list (this means u is queued before x) , then well go to y through u (ys parent node will be u). However, if x is in front of u in ts adjacency list (this means x is queued before u), then well go to y through x (ys parent node will be x). Hence the breadth-first tree will be different depending on the order of u and x. But note that y.distance will still be the same. Depth First Search and Topological Sorting 10. Show how depth-first search works on the following graph. Assume that the For loops of the DFS procedure considers the vertices in alphabetical order, and assume that each adjacency list is ordered alphabetically. Show the discovery and finishing times for each vertex. Show the ordering of vertices produced by TOPOLOGICAL-SORT when it is run on this graph.
s v w x z y q t r u

Answer:
2/7 3/6 1/16 17/20

s w
4/5

q
8/15 9/12

r t x z y
13/14 10/11

18/19

Ordering of vertices: r,u,q,t,y,x,z,s,v,w Note that this graph is not acyclic: ie. It contains cycles, hence the topological sorting is questionable.

Tutorial Exercise (Graph Algorithms)

CS3381 Design and Analysis of Algorithms Helena Wong, 2001

11. Explain how a vertex u of a directed graph can end up in a depth-first tree containing only u, even though u has both incoming and outgoing edges in G. Answer:

u a z
Suppose the above graph is to be traversed by depth-first search, and the vertices are examined according to alphabetical order. We firstly examine a. Since it has no outgoing edges, we mark a with start time and finishing time as 1/2, ie. a is a tree with only one node. Then we examine u. Since u has only one outgoing edge that directs to an already examined node (a), we cant traverse further from u. Hence we mark u with start time and finishing time as 3/4, ie. u is a tree with only one node. In this example, although u has both incoming and outgoing edges, but it ends up as a depth-first tree containing only u. This is because (1) all its outgoing edges point to vertices that have been examined before u is examined, and (2) all its incoming edges are pointed from vertices that are examined only after u is examined. Minimum Spanning Trees 12. Run the Kruskals algorithm on the following graph:

1 4 4 4

1 6 3

2 4 5 7 7

2 5 8 3

3 6 6

Answer: Step Initialization 1 2 3 4 5 6 7

Edge Considered -{1,2} {2,3} {4,5} {6,7} {1,4} {2,5} {4,7}

Connected components {1} {2} {3} {4} {5} {6} {7} {1,2} {3} {4} {5} {6} {7} {1,2,3} {4} {5} {6} {7} {1,2,3} {4,5} {6} {7} {1,2,3} {4,5} {6,7} {1,2,3,4,5} {6,7} Rejected {1,2,3,4,5,6,7}

Tutorial Exercise (Graph Algorithms)

CS3381 Design and Analysis of Algorithms Helena Wong, 2001

13. Suppose that the graph G=(V,E) is represented as an adjacency matrix. implementation of Prims algorithm for this case that runs in O(V2) time. Answer:

Give a simple

MST-PRIM(G,r) /* G = (V,E) */ 1 for i=1 to |V| 2 dist[i] = 3 pred[i] = 4 dist[r] = 0 5 Create a min-priority queue Q for indexes of vertices according to values of dist 6 While Q is not empty 7 i = EXTRACT-MIN(Q) 8 For j = 1 to |V| 9 if m[i][j]=1 10 if j exists in Q and weight of (i,j) < dist[j] 11 pred[j] = i 12 dist[j] = weight of (i,j)
Single-Source Shortest Paths 14. Run the Bellman-Ford algorithm on the following directed graph, using vertex z as the source.
t 6 s 0 7 8 2 y 9 5 -2 -3 -4 z 7 x

Answer:
t 6 s 7 8 2 y t 6 s 2 7 8 2 9 5 -2 -3 -4 x 7 s 6 t 8 2 7 y t 6 7 0 z s 2 7 8 2 9 y 9 5 9 5 -2 -3 -4 0 z x t 6 7 0 z s 2 7 8 2 9 y 9 5 -2 -3 -4 0 z x 6 7 x

7
7

0
z x 7 -3 -4

5 -2

5 -2 -3 -4

9
y

Tutorial Exercise (Graph Algorithms)

CS3381 Design and Analysis of Algorithms Helena Wong, 2001

15. The basic Bellman-Ford algorithm is to relax the whole set of edges |V|-1 times. This is carried out in the |V|-1 iterations of the For loop. A more efficient version of the Bellman-Ford algorithm is to keep track of whether there is any changes to the d values in each iteration, and to stop when there is no change in one iteration. Implement this version by changing some codes of the basic Bellman-Ford algorithm and the relaxation routine. Answer: RELAX(u,v) 1 if v.d > u.d + w(u,v) 2 v.d = u.d + w(u,v) 3 v.pred = u 4 changes = true Bellman-Ford Algorithm(G,s) /*G=(V,E)*/ 1 INITIALIZE-SINGLE-SOURCE(G,s) 2 changes = true 3 Iterations_done=0; 4 while changes = true and Iterations_done<=|V|-1 5 change = false 6 for each edge (u,v) in E 7 RELAX(u,v) 8 Iterations_done++; 9 for each edge (u,v) in E 10 if v.d > u.d + w(u,v) 11 return false 12 return true

6 s ? 3 5 0 2 ? 7 ? -1 4 ? -2 2

1 ?

16. Run DAG-SHORTEST-PATHS on the following graph, using vertex s as the source. Answer:
6 1 6 5 2 7 5 3 s 0 3 1 2 6 1

5 2 7 -1 -2 s 0 3 6 5 s 0 3 6 5 2 7 5 3 s 0 3 4 2

10

-1 4

-2 2

-1 4

-2 2

5 2 7 -1 -2 5 3 10 7 5 s 0 3 4 6 2 1

11

-1 4

-2 2

5 2 7 -1 -2 5 3 10 7 5 s 0 3 4 2

Tutorial Exercise (Graph Algorithms)

CS3381 Design and Analysis of Algorithms Helena Wong, 2001

17. Give a simple example of a directed graph with negative-weight edges for which Dijkstras algorithm produces incorrect answers. Answer:
10 s 5 -6 4 s 10 10 0 5 -6 5 4 10 10 0 s -6 5 5 4 9 10 10 0 s -6 5 5 4 9 10 10 0 s -6 5 4 9 5 10 10 0 s 5 4 9 -6 4

The algorithm finishes here 10 10 0 s 5 4 9 -6 4

N ash stp th ot orte a

Tutorial Exercise (Graph Algorithms)

All-Pairs Shortest Paths

CS3381 Design and Analysis of Algorithms Helena Wong, 2001

18. Run SLOW-ALL-PAIRS-SHORTEST-PATHS on the following graph, showing the matrices that result for each iteration of the loop. Do the same for FASTER-ALL-PAIRS-SHORTEST-PATHS.
1 -4 4 2 3 1 2 7 5 2 3 10 -8 6

-1

Answer: SLOW-ALL-PAIRS-SHORTEST-PATHS
w 1 2 3 4 5 6 1 0 1 -4 2 0 2 7 5 3 0 10 4 2 0 5 -1 3 0 6 -8 0

l(2) 1 2 3 4 5 6 l(3) 1 2 3 4 5 6 l(4) 1 2 3 4 5 6 l(5) 1 2 3 4 5 6

1 0 -2 3 -4 8 6 1 0 -2 -2 -4 5 3 1 0 -2 -5 -4 5 3 1 0 -2 -5 -4 5 3

2 6 0 -3 10 7 5 2 6 0 -3 2 7 5 2 6 0 -3 2 7 5 2 6 0 -3 2 7 5

3 0 10 3 0 10 3 0 10 3 0 10

4 2 4 0 9 7 4 8 2 -1 0 9 7 4 8 2 -1 0 9 7 4 8 2 -1 0 9 7

5 -1 0 -5 0 5 -1 -3 2 -5 0 5 5 -1 -3 -3 -5 0 2 5 -1 -3 -6 -5 0 2

6 -8 0 6 -8 0 6 -8 0 6 -8 0

Tutorial Exercise (Graph Algorithms)

Answer: FASTER-ALL-PAIRS-SHORTEST-PATHS L(2) 1 2 3 1 0 6 2 -2 0 3 3 -3 0 4 -4 10 5 8 7 6 6 5 10 l(4) 1 2 3 4 5 6 l(8) 1 2 3 4 5 6 1 0 -2 -5 -4 5 3 1 0 -2 -5 -4 5 3 4 2 4 0 9 7 5 -1 0 -5 0 5 -1 -3 -3 -5 0 2 5 -1 -3 -6 -5 0 2 6 -8 0 6 -8 0 6 -8 0

CS3381 Design and Analysis of Algorithms Helena Wong, 2001

2 3 4 6 8 0 2 -3 0 -1 2 0 7 9 5 10 7 2 3 4 6 8 0 2 -3 0 -1 2 0 7 9 5 10 7

19. Suppose we also wish to compute the vertices on shortest paths in the SLOW-ALL-PAIRSSHORTEST-PATHS algorithm. Show how to compute the predecessor matrix from the completed matrix L of shortest-path weights in O(n3) time. Answer: Use the graph in question 18 as example. To compute the predecessor matrix pred[6][6], consider the shortest path for each pair of vertices (i,j). This shortest path should be any one of the followings: 1. Use the shortest path from i to vertex 1, then use the edge from 1 to j. => pred[i][j] = 1 2. Use the shortest path from i to vertex 2, then use the edge from 2 to j. => pred[i][j] = 2 3. Use the shortest path from i to vertex 3, then use the edge from 3 to j. => pred[i][j] = 3 4. Use the shortest path from i to vertex 4, then use the edge from 4 to j. => pred[i][j] = 4 5. Use the shortest path from i to vertex 5, then use the edge from 5 to j. => pred[i][j] = 5 6. Use the shortest path from i to vertex 6, then use the edge from 6 to j. => pred[i][j] = 6 From the completed matrix L, we know the shortest path weight for (i,j), hence we can compute the weights of the paths in the above 6 cases to see which one matches the shortest path weight in L. eg. if case 4 matches the shortest path weigh in L, then pred[i][j] = 4. COMPUTE-PRED() 1 for i= 1 to 6 2 for j=1 to 6 3 for k=1 to 6 4 if L[i][j] = L[i][k]+w[k][j] 5 then pred[i][j]=k

Tutorial Exercise (Graph Algorithms)

CS3381 Design and Analysis of Algorithms Helena Wong, 2001

20. Run the Floyd-Warshall algorithm on the graph for question 18. Show the matrix D (k) that results for each iteration of the outer loop. Answer: D(1) 1 2 3 4 5 6 1 2 3 0 1 0 2 0 -4 7 5 10 4 2 0 4 2 4 0 9 7 4 2 4 0 9 7 4 2 4 0 9 7 4 8 2 4 0 9 7 4 8 2 -1 0 9 7 5 -1 0 -5 0 5 -1 0 2 -5 0 5 5 -1 0 2 -5 0 5 5 -1 -3 -1 -5 0 2 5 -1 -3 -1 -5 0 2 5 -1 -3 -6 -5 0 2 6 -8 0 6 -8 0 6 -8 0 6 -8 0 6 -8 0 6 -8 0 Pred(1) 1 2 3 4 5 6 Pred(2) 1 2 3 4 5 6 Pred(3) 1 2 3 4 5 6 Pred(4) 1 2 3 4 5 6 Pred(5) 1 2 3 4 5 6 Pred(6) 1 2 3 4 5 6 1 2 4 1 2 2 4 2 2 1 2 2 4 2 2 1 4 4 4 4 4 1 4 4 4 4 4 1 4 4 4 4 4 2 3 5 6 2 3 5 6 2 3 5 6 2 3 5 6 2 5 3 5 5 6 2 5 6 5 5 6 3 6 3 6 3 6 3 6 3 6 3 6 4 2 4 2 2 2 2 4 2 2 2 2 4 2 2 2 2 4 2 2 2 2 2 4 2 2 2 2 2 5 1 1 1 5 1 1 1 1 1 5 1 1 1 1 1 5 1 1 1 1 1 5 1 1 1 1 1 5 1 1 1 1 1 6 3 6 3 6 3 6 3 6 3 6 3

D(2) 1 2 3 1 0 2 1 0 3 3 2 0 4 -4 5 8 7 6 6 5 10 D(3) 1 2 3 1 0 2 1 0 3 3 2 0 4 -4 5 8 7 6 6 5 10 D(4) 1 2 3 1 0 2 -2 0 3 0 2 0 4 -4 5 5 7 6 3 5 10 D(5) 1 1 0 2 -2 3 0 4 -4 5 5 6 3 2 3 6 0 2 0 2 7 5 10

D(6) 1 2 3 1 0 6 2 -2 0 3 -5 -3 0 4 -4 2 5 5 7 6 3 5 10

Tutorial Exercise (Graph Algorithms)

CS3381 Design and Analysis of Algorithms Helena Wong, 2001

21. How can the output of the Floyd-Warshall algorithm be used to detect the presence of a negativeweight cycle? Answer: 2 ways: A. Check the main-diagonal entries of the result matrix for a negative value. B. Run the normal FLOYD-WARSHALL algorithm one extra iteration (k=|V|) to see if any of the d value change. If there are negative cycles, then some shortest-path cost will be cheaper. If there are no such cycles, then no d-values will change because the algorithm gives the correct shortest paths.

22. Use Johnsons algorithm to find the shortest paths between all pairs of vertices in the graph for question 18. Show the values of w and computed by the algorithm. Answer: Step 1: Add a new vertex {s} and the 0-weight edges: 0 0 0 1 s -4 0 0 0 Step 2: Run BELLMAN-FORD(G,s) 0 4 2 3 -1 7 5 5 10 -8 6 1 2 2 3

0 0

0 0 s0 0 1 2 1 3 2 5 1 -8 0 6 0 s0 0 1 0

2 1 0

3 2 0 5 10 -8 0 6

-4 2 -1 7 04 3 5 0

-4 2 -1 7 0 04 3 0 5 0

Tutorial Exercise (Graph Algorithms)

0 0 0 s0 0 1 -4 2 1 0 3 2 0 5 1 -8 0 -8 6 s0 0 0 1 -4 0

CS3381 Design and Analysis of Algorithms Helena Wong, 2001

0 2 1 -3 3 2 0 5 10 -8 -8 6

-4 2 -1 7 0 04 3 -1 5 0

-4 2 -1 7 0 04 3 -5 5 0

0 0 0 s0 0 1 -4 2 1 -3 3 2 0 5 1 -8 0 -8 6 s0 0 0

0 0 1 -5 2 1 -3 3 2 0 5 10 -8 -8 6

-4 2 -1 7 -1 04 3 -5 5 0

-4 2 -1 7 -1 04 3 -5 5 0

0 0 0 s0 0 1 -5 2 1 -3 3 2 0 5 1 -8 0 -8 6

-4 2 -1 7 -1 04 3 -6 5 0

Since no edge can relax any more vertex, we conclude that this graph doesnt have negative-weight cycle.

Tutorial Exercise (Graph Algorithms)

CS3381 Design and Analysis of Algorithms Helena Wong, 2001

Step 3: For each edge, compute new weights w(u,v) = w(u,v) + (s,u) - (s,v)
0 0

0 0 0 s0 0 1 -5 2 1 -3 3 2 0 5 10 -8 -8 6
0

1
-5

-4 2 -1 7 -1 04 3 -6 5 0

0 -1 ( -(-6 +-5) ) 1 +-8) ) -8 ( )-( ) 0 ( -(0 +0 -8 -4 ( -(-5 +-1) ) = 0 =2 = 7 ( )-(-3 +-6 ) 0 =0 5 ( )-( ) +-8 -3 =4 2 ( )-( ) +-3 -1 =0 = 0 -1 -6 -8 3 ( )-( ) +-1 -6 = 8 5 6 0 4 0

1 ( )-(-5 +-3 ) =3

2
-3

2 0 -3 + -( ) =5

Step 4: For each vertex u, Run DIJKSTRA(G,u) using w to compute (u,v) for all v in V Then for each vertex v in V, compute (u,v)= (u,v) + (s,v) - (s,u)
1
3 0 0

4 8

2 0

Step 4.1: Run DIJKSTRA(G,v1). Result:

0
0

3 0 0

2 4
4

3
2 0

4 4

0 5

Compute

(v1,v2)= (v1,v2) + (s,v2) - (s,v1) (v1,v4)= (v1,v4) + (s,v4) - (s,v1) (v1,v5)= (v1,v5) + (s,v5) - (s,v1)
0 0 0 1 -5 2 1 -3 3 2 0 5 10 -8 -8 6
0 0

1
3

2 4+ (-3)-(-5) = 6
4 0 5

3
2 0

0
0

s0 0

-4 2 -1 7 -1 04 3 -6 5 0

4+ (-1)-(-5) = 8 4

0+ (-6)-(-5) = -1 5

Step 4.2: Run DIJKSTRA(G,v2) Step 4.3: Run DIJKSTRA(G,v3) Step 4.4: Run DIJKSTRA(G,v4) Step 4.5: Run DIJKSTRA(G,v5) Step 4.6: Run DIJKSTRA(G,v6)