Sorting Algorithms

Guest Lecturer G. Alan Wang, ABD

MIS 531A Fall, 2005

Outline
Computation Complexity Simple Sorting Algorithms
Bubble Sort Insertion Sort Selection Sort

Complex Sorting Algorithms

Quick Sort Heap Sort
2

Computation Complexity
Definition: Measure the efficiency of an algorithm in terms of time or space required. Time tends to be more important. The efficiency of an algorithm is always stated as a function of the input size
The Big-O notation: O(f(n))

Worst case scenario

The maximum number of computation steps taken on any input size n The strategy is to find the closest upper bound of the worst Upper bound case scenario
Actual function Lower bound 3

Time Complexity
We assume each operation in a program take one time unit.
Time Units to Compute ------------------------------ 1 for the assignment. 1 assignment, n+1 tests, and n increments. n loops of 3 units for an assignment, an addition, and one multiplication. 1 for the return statement. ---------------------------------------Total: 1+(1+n+1+n)+3n+1 = 5n+4 = O(n) int sum (int n) { int partial_sum = 0; for (int i = 1; i <= n; i++) partial_sum = partial_sum + (i * i ); return partial_sum; }

Basic Time Complexity Functions

In an increasing order of complexity:
Constant time: O(1) Logarithmic time: O(logn) Linear time: O(n) Polynomial time: O(n2) Exponential time: O(2n)

Suppose each step takes 1 microseconds (10-6):

n O(1) O(logn) O(n) O(n^2) 10 1 1 10 100 100 1 2 100 10000 1000 1 3 1000 1 sec 10000 1 4 10000 1.67 min O(2^n) 1024 40196936841331500 years

Basic Time Complexity Functions

20

O(2^n)

O(n^2) O(n)

Running Time

15

10

O(logn)
0 0 5 10 Input Size 15 20

O(1)

25

Bubble Sort
Sorting takes an unordered collection and makes it an ordered one. Bubble sort algorithm*:
Compare adjacent elements. If the first is greater than the second, swap them. Do this for each pair of adjacent elements, starting with the first two and ending with the last two. At this point the last element should be the greatest. Repeat the steps for all elements except the last one. Keep repeating for one fewer element each time, until you have no more pairs to compare

Time complexity: O(n2) Demo: http://www.cs.princeton.edu/~ah/alg_anim/gawain4.0/BubbleSort.html Example: Sort the list {12, 5, 7, 9, 2, 6}
7

*: Excerpted from WIKIPEDIA, http://en.wikipedia.org/wiki/Bubble_sort

Insertion Sort
Algorithm*:
Start with the result being the first element of the input; Loop over the input array until it is empty, "removing" the first remaining (leftmost) element; Compare the removed element against the current result, starting from the highest (rightmost) element, and working left towards the lowest element; If the removed input element is lower than the current result element, copy that value into the following element to make room for the new element below, and repeat with the next lowest result element; Otherwise, the new element is in the correct location; save it in the cell left by copying the last examined result up, and start again from step 2 with the next input element. Time complexity: O(n2) Demo: http://web.engr.oregonstate.edu/~minoura/cs162/javaProgs/sort/InsertSort.ht ml Example: Sort the list {12, 5, 7, 9, 2, 6}
*: Excerpted from WIKIPEDIA, http://en.wikipedia.org/wiki/Bubble_sort
8

Selection Sort
Algorithm:
Pass through elements sequentially; In the ith pass, we select the element with the lowest value in A[i] through A[n], then swap the lowest value with A[i].

Time complexity: O(n2) Demo: http://www.cosc.canterbury.ac.nz/people/mukun dan/dsal/SSort.html Example: Sort the list {25, 57, 48, 37, 12}
9

Quick Sort
Quick sort, also known as partition sort, sorts by employing a divide-and-conquer strategy. Algorithm:
Pick an pivot element from the input; Partition all other input elements such that elements less than the pivot come before the pivot and those greater than the pivot come after it (equal values can go either way); Recursively sort the list of elements before the pivot and the list of elements after the pivot. The recursion terminates when a list contains zero or one element.

Example: Sort the list {25, 57, 48, 37, 12}

10

Quick Sort
Quick sort, also known as partition sort, sorts by employing a divide-and-conquer strategy. Algorithm:
Pick an pivot element from the input; Partition all other input elements such that elements less than the pivot come before the pivot and those greater than the pivot come after it (equal values can go either way); Recursively sort the list of elements before the pivot and the list of elements after the pivot. The recursion terminates when a list contains zero or one element.

Time complexity: O(nlogn) or O(n2) Demo: http://pages.stern.nyu.edu/~panos/java/Quicksort/ Example: Sort the list {25, 57, 48, 37, 12}
11

Heap
Definition: Almost Complete Binary Tree (ACBT) Descending heap:
ACBT + every node value parent node value

Ascending heap:
ACBT + every node value parent node value

12

Heap Sort
Heapify phase:
Create a descending heap Add element to a binary tree from top to bottom and from left to right When adding a new element, if the element is out of order, perform sift-up operations (a sequence of swap with parent)

Example: {25, 57, 48, 37, 12}

13

Heap Sort (Contd)

Sorting phase
Work backwards from bottom to top and from right to left Swap current element with root For the new root, perform sift-down operations (swap with the larger son).

14

Heap Sort (Contd)

Time complexity:
Heapify: O(nlog2n) Sorting: O(nlog2n) Overall: O(nlog2n) + O(nlog2n) = O(nlog2n)

15

Questions

16