Data Structures & Algorithm Analysis

Week 11 & 12: Sorting

Objectives:

At the end of this lesson, the student will be able to:

sort an array into ascending order by using the following methods: selection sort, insertion sort, shell sort, merge sort, quick sort and heap sort, sort a chain of nodes into ascending order by using an insertion sort, and determine the efficiency of a sort and comparing the algorithms of sorting.

Introduction

Sorting rearranging data in an array or list so that it is in ascending or descending order. Sorting can be done to any collection of items that can be compared with one another. Eg: arrange a row of books on bookshelf by title. The efficiency of a sorting algorithm is significant, particularly when large amounts of data are involved.

INSERTION SORT

Insertion Sort

If first two books are out of order

Remove second book Slide first book to right Insert removed book into first slot Remove that book Slide 2nd book to right Insert removed book into 2nd slot Etc.

Then look at third book, if it is out of order

Recheck first two books again

Insertion Sort

The placement of the third book during an insertion sort.

Insertion Sort

An insertion sort of books

Insertion Sort

Sort an array partitions that is, divides the array into two parts.

One part is sorted and initially contains just first element in the array The second part contains the remaining elements.

The algorithm removes the first element from the unsorted part and inserts it into its proper sorted position within the sorted part. An insertion sort is more efficient when an array is sorted.

public class Sort { public static void main(String [] args) { int [] tsnom = {3, 1, 2, 7, 5}; System.out.println(Before Insertion Sort"); print(tsnom); insertionSort(tsnom); System.out.println(After Insertion Sort"); print(tsnom); } public static void insertionSort (int[] ts) { for (int x=1; x < ts.length; x++) { int temp=ts[x]; int y=x; for (; y>0 && temp < ts[y-1];y--) ts[y]=ts[y-1]; ts[y]=temp; } } public static void print(int[] ts) { for(int x=0; x< ts.length; x++) System.out.print(ts[x]); System.out.println(); } }

Java Code for Insertion Sort

Insertion Sort
3 1 2 7 5 [0] [1] [2] [3] [4] int temp=ts[x]; int y=x; for (; y>0 && temp < ts[y-1];y--) ts[y]=ts[y-1]; ts[y]=temp
x=1 y=1 y=0

3 1 2 7 5 [0] [1] [2] [3] [4]

X=1 temp=1 1<3

1 2 3 7 5 [0] [1] [2] [3] [4]

x=3 y=3

1 3 2 7 5 [0] [1] [2] [3] [4]

X=2 y=2 y=1

X=3

temp=7 7<3

1 2 3 7 5 [0] [1] [2] [3] [4]

x=4 y=4 y=3

1 3 2 7 5 [0] [1] [2] [3] [4]

1 2 3 7 5 [0] [1] [2] [3] [4]

X=2 temp=2 2<3 2<1

1 3 3 7 5 [0] [1] [2] [3] [4] 1 2 3 7 5 [0] [1] [2] [3] [4]

X=4

temp=5 5<7 5<3

1 2 3 5 7 [0] [1] [2] [3] [4]

1 2 3 5 7 [0] [1] [2] [3] [4]

Insertion Sort
int temp=ts[x]; int y=x; for (; y>0 && temp < ts[y-1];y--) ts[y]=ts[y-1]; ts[y]=temp 3 1 2 7 5 [0] [1] [2] [3] [4]
X=1 temp=1 1<3

1 2 3 7 5 [0] [1] [2] [3] [4]

temp=7 x=3 temp=7 7<3

1 3 2 7 5 [0] [1] [2] [3] [4]

1 3 2 7 5 [0] [1] [2] [3] [4]

temp=2 x=2 temp=2 2<3 2<1

1 2 3 7 5 [0] [1] [2] [3] [4]

temp=5 x=4 temp=5 5<7 5<3

1 2 3 7 5 [0] [1] [2] [3] [4]

1 2 3 5 7 [0] [1] [2] [3] [4]

Insertion Sort of Chain of Linked Nodes

A chain of integers sorted into ascending order

Insertion Sort of Chain of Linked Nodes

During the traversal of a chain to locate the insertion point, save a reference to the node before the current one.

Insertion Sort of Chain of Linked Nodes

Breaking a chain of nodes into two pieces as the first step in an insertion sort: (a) the original chain; (b) the two pieces

Efficiency of insertion sort of a chain is O(n2)

Efficiency of Insertion Sort

Best time efficiency is O(n) Worst time efficiency is O(n2) Average time efficiency is O(n2) If array is closer to sorted order

Less work the insertion sort does More efficient the sort is

Insertion sort is acceptable for small array sizes

SELECTION SORT

Selection Sort

Rearranges the object by interchanging value. The sort locates the smallest/largest value in the array. Task: rearrange books on shelf by height

Shortest book on the left Look at books, select shortest book Swap with first book Look at remaining books, select shortest Swap with second book Repeat

Approach:

Selection Sort

Before and after exchanging shortest book and the first book.

Selection Sort

A selection sort of an array of integers into ascending order.

(method 1:choose the smallest value)

Before Sort

L1

L2

L3

L4

Sorted List

12

12

12

12

12

12

19 33

19 33

19 22

19 22

19 22

19 22

26 22

26 22

26 33

26 33

26 33

26 33

Max=33

Max=26

Max=22

Max=19

A selection sort of an array of integers into ascending order.

(method 2: choose the biggest value)

Selection Sort
public void selectionSort(Object array[]) { int minIndex; Object temp; for(int x = 0; x < array.length; x++) { minIndex = 0; //initialization the index of smallest value for(int y = x + 1; y < array.length; y++){ //find the index of the smallest value in the list if(array[y] < array[minIndex]) minIndex = y; } //move the smallest value into correct position (swap) temp = array[x]; array[x] = array[minIndex]; array[minIndex] = temp; } }//end selectionSort

The Efficiency of Selection Sort

Iterative method for loop executes n 1 times

For each of n 1 calls, inner loop executes n 2 times (n 1) + (n 2) + + 1 = n(n 1)/2 = O(n2) Also O(n2)

Recursive selection sort performs same operations

Efficiency of Selection Sort

Best time efficiency is O(n2) Worst time efficiency is O(n2) Average time efficiency is O(n2) Sufficient when to sort a small array.

SHELL SORT

Shell Sort

A variation of the insertion sort

But faster than O(n2)

Done by sorting subarrays of equally spaced indices Instead of moving to an adjacent location an element moves several locations away

Results in an almost sorted array This array sorted efficiently with ordinary insertion sort

Shell Sort
List (Before Sort) 81 94 11 96 12 35 17 95 28 58 41 75 15

Step 1: Divide list distance of 5

81 35 94 17 11 95 96 28

horizontal
12 58

41

75

15

Sort using insertion sort

35 41 81 17 75 94 11 15 95 28 96

vertical
12 58

Combine the list

35 17 11 28 12 41 75

horizontal
15 96 58 81 94 95

Step 2: Divide list distance of 3

Shell Sort

35 28 75 58 95

17 12 15 81

11 41 96 94

Sort using insertion sort

28
35 58 75 95

12
15 17 81

11
41 94 96

Combine the list

28 12 11 35 15 41 58 17 94 75 81 96 95

Shell Sort

Step 3: Divide list distance of 1

28 12 11 35 15 41 58 17 94 75 81 96 95

Sort using insertion sort

11 12 15 17 28 35 41 58 75 81 94 95 96

Combine the list sorted list

11 12 15 17 28 35 41 58 75 81 94 95 96

Efficiency of Shell Sort

Best time efficiency is O(n) Worst time efficiency is O(n1.5) Average time efficiency is O(n1.5)

MERGE SORT

Merge Sort

Divide an array into halves

Sort the two halves Merge them into one sorted array This is often part of a recursive algorithm However recursion is not a requirement

Referred to as a divide and conquer algorithm

Merge Sort
Algorithm mergeSort(a, first, last) // Sorts the array elements a[first] through a[last] recursively.

if (first < last)

{ mid = (first + last)/2

mergeSort(a, first, mid)

mergeSort(a, mid+1, last) Merge the sorted halves a[first..mid] and a[mid+1..last] }

Merge Sort

The effect of the recursive calls and the merges during a merge sort.

Merge Sort

Efficiency of the merge sort

Merge sort is O(n log n) in all cases It's need for a temporary array is a disadvantage The class Arrays has sort routines that uses the merge sort for arrays of objects

Merge sort in the Java Class Library

public static void sort(Object[] a); public static void sort(Object[] a, int first, int last);

public class MergeSort { public static void main(String [] args) { int [] tsnom = {34,8,64,51,32,21,30}; System.out.println(Before Merge Sort"); print(tsnom); mergeSorting(tsnom); System.out.println(After Merge Sort"); print(tsnom); } private static void mergeSorting(int []a, int[]tsSem, int left, int right) { if (left < right) { int mid = (left + right)/2; mergeSorting(a,tsSem,left,mid); // left side mergeSorting(a,tsSem,mid + 1, right); // right side merge(a,tsSem,left,mid + 1, right); } } public static void mergeSorting(int []a) { Java Code for Merge int [] tsSem=new int[a.length]; Sort continued mergeSorting(a,tsSem,0,a.length-1); next slide }

private static void merge(int [] a, int [] tsSem, int posleft, int posright, int lastright) { int lastleft=posright - 1; int possem=posleft; int bilelemen=lastright - posleft + 1; while(posleft <= lastleft && posright <= lastright) { if (a[posleft]<=a[posright]) tsSem[possem++]=a[posleft++]; else tsSem[possem++]=a[posright++]; } while(posleft <=lastleft) tsSem[possem++]=a[posleft++]; while(posright<=lastright) tsSem[possem++]=a[posright++]; for(int i=0;i<bilelemen;i++,lastright--) a[lastright]=tsSem[lastright]; }//merge public static void print(int[]a) { for (int i=0;i<a.length;i++) System.out.print("\t"+ a[i]); }//print } // class MergeSort

Java Code for Merge Sort

Merge Sort

Merge Sort has two level:a) Divide the list into halves

L=0 R=6 mid=(0+6)/2 =3 L=0 R=3 mid=(0+3)/2 =1 L=0 R=1 mid=(0+1)/2 =0

2 5 1 9 3 8 7 [0] [1] [2] [3] [4] [5] [6]

L=4 R=6 mid=(4+6)/2 =5

2 5 1 9 [0] [1] [2] [3]

L=2 R=3 mid=(2+3)/2 =2

L=4 R=5 mid=(4+5)/2 =4

3 8 7 [4] [5] [6]

2 5 [0] [1]

1 9 [2] [3]

3 8 [4] [5]

7 [6]
L=6 R=6

2 [0]
L=0 R=0

5 [1]
L=1 R=1

1 [2]
L=2 R=2

9 [3]
L=3 R=3

3 [4]
L=4 R=4

8 [5]
L=5 R=5

First example

Merge Sort Sort the list from bottom 2)Merge Merge/combine

1 2 3 5 7 8 9 [0] [1] [2] [3] [4] [5] [6]

1 2 5 9 [0] [1] [2] [3]

3 8 7 [4] [5] [6]

2 5 [0] [1]

1 9 [2] [3]

3 8 [4] [5]

7 [6]

2 [0]

5 [1]

1 [2]

9 [3]

3 [4]

8 [5]

First example

Merge Sort

L=0 R=6 mid=(0+6)/2 =3 L=0 R=3 mid =(0+3)/2 =1

34 8 64 51 32 21 30 [0] [1] [2] [3] [4] [5] [6]

L=4 R=6 mid =(4+6)/2 =5

34 8 64 51 [0] [1] [2] [3]

L=0 R=1 mid =(0+1)/2 =0

34 8 [0] [1]

64 51 [2] [3]

Ki=2 Ka=3 Tgh=(2+3)/2 =2

Ki=4 Ka=5 Tgh=(4+5)/2 =4

32 21 30 [4] [5] [6]

32 21 [4] [5]

30 [6]
L=6 R=6

L=0 R=0

34 [0]

8 [1]
L=1 R=1

64 [2]
L=2 R=2

51 [3]
L=3 R=3

32 [4]
L=4 R=4

21 [5]
L=5 R=5

Second example

Merge Sort

L=0 R=6 mid=(0+6)/2 =3 L=0 R=3 mid=(0+3)/2 =1 L=0 R=1 mid=(0+1)/2 =0

8 21 30 32 34 51 64 [0] [1] [2] [3] [4] [5] [6]

L=4 R=6 mid=(4+6)/2 =5

8 34 51 64 [0] [1] [2] [3]

Ki=2 Ka=3 Tgh=(2+3)/2 =2

Ki=4 Ka=5 Tgh=(4+5)/2 =4

21 30 32 [4] [5] [6]

8 34 [0] [1]

51 64 [2] [3]

21 32 [4] [5]

30 [6]
L=6 R=6

L=0 R=0

34 [0]

8 [1]
L=1 R=1

64 [2]
L=2 R=2

51 [3]
L=3 R=3

32 [4]
L=4 R=4

21 [5]
L=5 R=5

Second example

Efficiency of Merge Sort

Best time efficiency is O(n log n) Worst time efficiency is O(n log n) Average time efficiency is O(n log n)

QUICK SORT

Quick Sort

Divides the array into two pieces

Not necessarily halves of the array An element of the array is selected as the pivot

Referred to as a divide and conquer algorithm Elements are rearranged so that:

Elements in positions before pivot are less than the pivot Elements after the pivot are greater than the pivot

Quick Sort

A partition of an array during a quick sort

Quick Sort
Algorithm quickSort(a, first, last) // Sorts the array elements a[first] through a[last] recursively. if (first < last) { Choose a pivot Partition the array about the pivot pivotIndex = index of pivot quickSort(a, first, pivotIndex-1) // sort Smaller quickSort(a, pivotIndex+1, last) // sort Larger

Quick Sort

Quick sort is O(n log n) in the average case and best case O(n2) in the worst case Worst case can be avoided by careful choice of the pivot

Quick Sort

Quick sort rearranges the elements in an array during partitioning process After each step in the process

One element (the pivot) is placed in its correct sorted position Remain in their respective subarrays

The elements in each of the two sub arrays

The class Arrays in the Java Class Library uses quick sort for arrays of primitive types

Quick Sort

Quick Sort has two level:a) Divide the list into halves (the first number is a pivot)

Quick Sort
b) Merge/combine the list from bottom

public class QuickSort { public static void main(String [] args) { int [] tsnom = {33,17,6,21,56,29}; System.out.println(Before Quick Sort"); print(tsnom); quickSorting(tsnom,0,tsnom.length-1); System.out.println(After Quick Sort"); print(tsnom); } private static void quickSorting(int []a, int first, int last) { int indeksPivot; if (first < last) { indeksPivot=choosePivot(a,first,last); quickSorting(a,first,indeksPivot-1); quickSorting(a,indeksPivot+1,last); }//if }//quickSorting

Java Code for Quick Sort.. Continued next slide

//choosePivot method for chosing the first number and sort private static int choosePivot(int []a,int first, int last) { int p=first; int pivot=a[first]; //for sort, left is smaller then pivot and right for greater then pivot for (int i=first + 1; i <=last; i++) { if (a[i] < pivot) { a[p]=a[i]; a[i]=a[p+1]; a[p+1]=pivot; p++; } } return p; }//choosePivot public static void print(int[]a) { for (int i=0;i<a.length;i++) System.out.print("\t"+ a[i]); System.out.println("\n"); }//print }

Java Code for Quick Sort

Quick Sort
List of Number Pivot = 5 and the value of pivot is smaller than other numbers. 5 6 7 8 9

pivot

unknown

5
pivot

unknown

5
pivot

unknown

5
pivot

9
unknown

5
pivot After first partition

4 comparisons and 0 changing number

Worst case for quick sort

Quick Sort
5 6 7 8 9 [0] [1] [2] [3] [4] <5 6 <6 7 <7 8 <8 >5 7 8 9 >6 8 9

>7 9 >8 9

Efficiency of Quick Sort

Best time efficiency is O(n log n) Worst time efficiency is O(n2) Average time efficiency is O(n log n)

HEAP SORT

Reprise: The ADT Heap

A complete binary tree Nodes contain Comparable objects In a maxheap

Object in each node objects in descendants

Note contrast of uses of the word "heap"

The ADT heap The heap of the operating system from which memory is allocated when new executes

Reprise: The ADT Heap

Interface used for implementation of maxheap

public interface MaxHeapInterface { public void add(Comparable newEntry); public Comparable removeMax(); public Comparable getMax(); public boolean isEmpty(); public int getSize(); public void clear(); } // end MaxHeapInterface

Using an Array to Represent a Heap

Figure-1: (a) A complete binary tree with its nodes numbered in level order; (b) its representation as an array.

Using an Array to Represent a Heap

When a binary tree is complete

Can use level-order traversal to store data in consecutive locations of an array

Enables easy location of the data in a node's parent or children

Parent of a node at i is found at i/2 (unless i is 1) Children of node at i found at indices 2i and 2i + 1

Adding an Entry

Figure-2: The steps in adding 85 to the maxheap of Figure-1(a)

Adding an Entry

Begin at next available position for a leaf Follow path from this leaf toward root until find correct position for new entry As this is done

Move entries from parent to child Makes room for new entry

Adding an Entry

Figure-3: A revision of steps shown in Figure-2 to avoid swaps.

Adding an Entry

Figure-4: An array representation of the steps in Figure-3 continued

Adding an Entry

An array representation of the steps in Figure-3.

Adding an Entry

Algorithm for adding new entry to a heap

Algorithm add(newEntry) if (the array heap is full) Double the size of the array newIndex = index of next available array location parentIndex = newIndex/2 // index of parent of available location while (newEntry > heap[parentIndex]) { heap[newIndex] = heap[parentIndex] // move parent to available location // update indices newIndex = parentIndex parentIndex = newIndex/2 } // end while

Removing the Root

Figure-5: The steps to remove the entry in the root of the maxheap of Figure-3(d)

Removing the Root

Figure-6: The steps that transform a semiheap into a heap without swaps.

Removing the Root

To remove a heap's root

Replace the root with heap's last child

This forms a semiheap Then use the method reheap

Transforms the semiheap to a heap

Creating a Heap

Figure-7: The steps in adding 20, 40, 30, 10, 90, and 70 to a heap.

Creating a Heap

More efficient to use reheap than to use add

Figure-8: The steps in creating a heap by using reheap.

Heapsort

Possible to use a heap to sort an array Place array items into a maxheap Then remove them

Items will be in descending order Place them back into the original array and they will be in order

Heapsort

Figure-9: A trace of heapsort (a c)

Heapsort

Figure-9: A trace of heapsort (d f)

Heapsort

Figure-9: A trace of heapsort (g i)

Heapsort

Figure-9: A trace of heapsort (j l)

Heapsort

Implementation of heapsort

public static void heapSort(Comparable[] array, int n) { // create first heap for (int index = n/2; index >= 0; index--) reheap(array, index, n-1); swap(array, 0, n-1); Efficiency is O(n log n). for (int last = n-2; last > 0; last--) { reheap(array, 0, last); However, quicksort is swap(array, 0, last); usually a better choice. } // end for } // end heapSor private static void reheap(Comparable[] heap, int first, int last) { < Statements from Segment 27.11, replacing rootIndex with first and lastIndex with last. > } // end reheap

Heap Sort

Creating a heap

1 2 4

5
3 6

7 9

2
5 3

1
7

8 reheap(3)

1 2 4

5
3 6

7 9

2
5 3

8
7

7 reheap(2)

Heap Sort
Creating a heap

1 2 4

5
3 6

7 2

9
5 3

8
7

reheap(1)

1 2 4

9
3 6

5
5 3

8
7

Heap Sort
Creating a heap

1 2 4

9
3 6

7 2

5
5 3

8
7

1 remove

1 2 4

7
3 6

9 2

5
5 3

Heap Sort
Creating a heap

1 2

7
3 6

9
4

5
5 3

1 reheap(1) 8
3 6

1 2 4

9 2

5
5 3

1 remove

Heap Sort

Creating a heap

1 2 4

1
3

9
2

5
5 3

reheap(1)

Heap Sort
Creating a heap

1 2

7
3

9
4

5
5 3

remove

1
2 4

3
3

9 2

reheap(1)

Heap Sort
Creating a heap

1 2 4

5
3

9 2

remove

1
2

2
3

reheap(1)

Heap Sort
Creating a heap

1 2

3
3

2 remove

reheap(1)

Heap Sort
Creating a heap

1 2

remove

remove

Efficiency of Heap Sort

Best time efficiency is O(n log n) Worst time efficiency is O(n log n) Average time efficiency is O(n log n)

Comparing the Algorithms

Best Case Average Case Worst Case

Insertion sort Selection sort Shell sort Merge sort Quick sort Heap sort

O(n) O(n2) O(n) O(n log n) O(n log n) O(n log n)

O(n2) O(n2) O(n1.5) O(n log n) O(n log n) O(n log n)

O(n2) O(n2) O(n1.5) O(n log n) O(n2) O(n log n)

The time efficiencies of three sorting algorithms, expressed in Big Oh notation.

Conclusion Q & A Session