Sorting Algorithms-Sesion 6

Sorting and Searching
Algorithms
Mdulo de Entrenamiento del ACM ICPC
Revisado al 21 de Mayo de 2014 - FIGMM UNI


2
Sorting
3
Why Study Sorting Algorithms?
There are a variety of situations that we can
encounter
Do we have randomly ordered keys?
Are all keys distinct?
How large is the set of keys to be ordered?
Need guaranteed performance?
Various algorithms are better suited to some
of these situations

To arrange a set of items in sequence.
It is estimated that 25~50% of all computing
power is used for sorting activities.
Possible reasons:
Many applications require sorting;
Many applications perform sorting when they
don't have to;
Many applications use inefficient sorting
algorithms.

CENG 213 Data Structures
Sorting Algorithms
There are many sorting algorithms, such as:
Selection Sort
Insertion Sort
Bubble Sort
Merge Sort
Quick Sort

The first three are the foundations for faster
and more efficient algorithms.
6
Insertion Sort
Idea: like sorting a hand of playing cards
Start with an empty left hand and the cards facing
down on the table.
Remove one card at a time from the table, and
insert it into the correct position in the left hand
compare it with each of the cards already in the hand,
from right to left
The cards held in the left hand are sorted
these cards were originally the top cards of the pile on
the table
7
To insert 12, we need to make
room for it by moving first 36
and then 24.
Insertion Sort

8
Insertion Sort

9
Insertion Sort

10
Insertion Sort
5 2 4 6 1 3
input array
left sub-array
right sub-array
at each iteration, the array is divided in two sub-arrays:
sorted
unsorted

11
Insertion Sort
12
INSERTION-SORT
Alg.: INSERTION-SORT(A)
for j 2 to n
do key A[ j ]
Insert A[ j ] into the sorted sequence A[1 . . j -1]
i j - 1
while i > 0 and A[i] > key
do A[i + 1] A[i]
i i 1
A[i + 1] key
Insertion sort sorts the elements in place
a
8
a
7
a
6
a
5
a
4
a
3
a
2
a
1
1 2 3 4 5 6 7 8
key

13
Loop Invariant for Insertion Sort

Alg.: INSERTION-SORT(A)
for j 2 to n
do key A[ j ]
i j - 1
do A[i + 1] A[i]
i i 1
A[i + 1] key
Invariant: at the start of the for loop the elements in A[1 . . j-1] are in
sorted order
14
Proving Loop Invariants
Proving loop invariants works like induction
Initialization (base case):
It is true prior to the first iteration of the loop
Maintenance (inductive step):
If it is true before an iteration of the loop, it remains true before the
next iteration
Termination:
When the loop terminates, the invariant gives us a useful property
that helps show that the algorithm is correct
Stop the induction when the loop terminates

15
Initialization:
Just before the first iteration, j =
2:
the subarray A[1 . . j-1] = A[1],
(the element originally in A[1])
is sorted

16
Maintenance:
the while inner loop moves A[j -1], A[j -2], A[j
-3], and so on, by one position to the right until
the proper position for key (which has the value
that started out in A[j]) is found
At that point, the value of key is placed into this
position.
17
Termination:
The outer for loop ends when j = n + 1 j-1 = n
Replace n with j-1 in the loop invariant:
the subarray A[1 . . n] consists of the elements
originally in A[1 . . n], but in sorted order

The entire array is sorted!
j j - 1
Invariant: at the start of the for loop the elements in A[1 . . j-1] are in
sorted order
18
Analysis of Insertion Sort
cost times
c
1
n
c
2
n-1
0 n-1
c
4
n-1
c
5

c
6

c
7

c
8
n-1

=
n
j
j
t
2
=

n
j
j
t
2
) 1 (
=

n
j
j
t
2
) 1 (
( ) ( ) ) 1 ( 1 1 ) 1 ( ) 1 ( ) (
8
2
7
2
6
2
5 4 2 1
+ + + + + + =

= = =
n c t c t c t c n c n c n c n T
n
j
j
n
j
j
n
j
j
INSERTION-SORT(A)
for j 2 to n
do key A[ j ]
i j - 1
do A[i + 1] A[i]
i i 1
A[i + 1] key
t
j
: # of times the while statement is executed at iteration j
19
Best Case Analysis
The array is already sorted
A[i] key upon the first time the while loop test is run
(when i = j -1)
t
j
= 1
T(n) = c
1
n + c
2
(n -1) + c
4
(n -1) + c
5
(n -1) + c
8
(n-1)
= (c
1
+ c
2
+ c
4
+ c
5
+ c
8
)n + (c
2
+ c
4
+ c
5
+ c
8
)
= an + b = O(n)
( ) ( ) ) 1 ( 1 1 ) 1 ( ) 1 ( ) (
8
2
7
2
6
2
5 4 2 1
+ + + + + + =

= = =
n
j
j
n
j
j
n
j
j
20
Worst Case Analysis
The array is in reverse sorted order
Always A[i] > key in while loop test
Have to compare key with all elements to the left of the j-th position
compare with j-1 elements t
j
= j

a quadratic function of n

T(n) = O(n
2
) order of growth in n
2

1 2 2
( 1) ( 1) ( 1)
1 ( 1)
2 2 2
n n n
j j j
n n n n n n
j j j
= = =
+ +
= => = => =

) 1 (
2
) 1 (
2
) 1 (
1
2
) 1 (
) 1 ( ) 1 ( ) (
8 7 6 5 4 2 1
+
+
|
.
|
\
|

+
+ + + = n c
n n
c
n n
c
n n
c n c n c n c n T
c bn an + + =
2
( ) ( ) ) 1 ( 1 1 ) 1 ( ) 1 ( ) (
8
2
7
2
6
2
5 4 2 1
+ + + + + + =

= = =
n
j
j
n
j
j
n
j
j
using
we have:
21
Comparisons and Exchanges in Insertion
Sort
INSERTION-SORT(A)
for j 2 to n
do key A[ j ]
i j - 1
do A[i + 1] A[i]
i i 1
A[i + 1] key
cost times
c
1
n
c
2
n-1
0 n-1
c
4
n-1
c
5

c
6

c
7

c
8
n-1

=
n
j
j
t
2
=

n
j
j
t
2
) 1 (
=

n
j
j
t
2
) 1 (
~ n
2
/2 comparisons
~ n
2
/2 exchanges

22
Insertion Sort - Summary
Advantages
Good running time for almost sorted arrays O(n)
Disadvantages
O(n
2
) running time in worst and average case
~ n
2
/2 comparisons and exchanges

23
Bubble Sort
Idea:
Repeatedly pass through the array
Swaps adjacent elements that are out of order

Easier to implement, but slower than Insertion
sort
1 2 3 n
i
1 3 2 9 6 4 8
j


24
Searching

25
Searching
Given a list of data find the location of a
particular value or report that value is not
present
linear search
intuitive approach
start at first item
is it the one I am looking for?
if not go to next item
repeat until found or all items checked
If items not sorted or unsortable this
approach is necessary
26
Introduccin a los Algoritmos de
busqueda
A search algorithm is a method of locating a
specific item of information in a larger
collection of data. This section discusses two
algorithms for searching the contents of an
array.

27
The Linear Search
This is a very simple algorithm.
It uses a loop to sequentially step through an
array, starting with the first element.
It compares each element with the value
being searched for and stops when that value
is found or the end of the array is reached.

28
Implementacin

#include <iostream.h>

// prototipo
int searchList(int [], int, int);

const int arrSize = 5;

void main(void)
{
int tests[arrSize] = {87, 75, 98, 100, 82};
int results;

29
Program continues
results = searchList(tests, arrSize, 100);
if (results == -1)
cout << "You did not earn 100 points on any
test\n";
else
{
cout << "You earned 100 points on test ";
cout << (results + 1) << endl;
}
}
30
Program continues

int searchList(int list[], int numElems, int value)
{
int index = 0; // Used as a subscript to search
array
int position = -1; // To record position of search value
bool found = false; // Flag to indicate if the value was
found
while (index < numElelments && !found)
{
if (list[count] == value)
{
found = true;
position = index;
}
index++;
}
return position;
}


31
T(N)? Big O? Best case, worst case,
average case????

Attendance Question??
What is the average case Big O of linear search
in an array with N items, if an item is present?
A. O(N)
B. O(N
2
)
C. O(1)
D. O(logN)
E. O(NlogN)

32

33
Merge Sort Algorithm
1. If a list has 1 element or 0
elements it is sorted
2. If a list has more than 2 split into
into 2 separate lists
3. Perform this algorithm on each of
those smaller lists
4. Take the 2 sorted lists and merge
them together
Don Knuth cites John von Neumann as the creator
of this algorithm
Mergesort
Mergesort algorithm is one of two important divide-and-
conquer sorting algorithms (the other one is quicksort).
It is a recursive algorithm.
Divides the list into halves,
Sort each halve separately, and
Then merge the sorted halves into one sorted array.

Mergesort - Example

Merge
const int MAX_SIZE = maximum-number-of-items-in-array;
void merge(DataType theArray[], int first, int mid, int last)
{
DataType tempArray[MAX_SIZE]; // temporary array
int first1 = first; // beginning of first subarray
int last1 = mid; // end of first subarray
int first2 = mid + 1; // beginning of second subarray
int last2 = last; // end of second subarray
int index = first1; // next available location in tempArray
for ( ; (first1 <= last1) && (first2 <= last2); ++index) {
if (theArray[first1] < theArray[first2]) {
tempArray[index] = theArray[first1];
++first1;
}
else {
++first2;
} }

Merge (cont.)
// finish off the first subarray, if necessary
for (; first1 <= last1; ++first1, ++index)

// finish off the second subarray, if necessary
for (; first2 <= last2; ++first2, ++index)

// copy the result back into the original array
for (index = first; index <= last; ++index)
theArray[index] = tempArray[index];
} // end merge

Mergesort
void mergesort(DataType theArray[], int first, int last) {
if (first < last) {
int mid = (first + last)/2; // index of
midpoint
mergesort(theArray, first, mid);
mergesort(theArray, mid+1, last);

// merge the two halves
merge(theArray, first, mid, last);
}
} // end mergesort

Mergesort - Example
6 3 9 1 5

4 7 2
5

4 7 2 6 3 9 1
6 3 9 1
7 2
5

4
6 3 1 9 5 4 2 7
3 6 1 9
2 7
4

5
2

4 5 7 1 3 6 9
1 2 3 4 5

7 8 9
divide
divide divide divide
divide divide
divide
merge merge
merge
merge
merge merge
merge
Mergesort Example2

Mergesort Analysis of Merge
A worst-case instance of the merge step in mergesort

Mergesort Analysis of Merge (cont.)
Merging two sorted arrays of size k

Best-case:
All the elements in the first array are smaller (or larger) than all the
elements in the second array.
The number of moves: 2k + 2k
The number of key comparisons: k
Worst-case:
The number of moves: 2k + 2k
The number of key comparisons: 2k-1

...... ......
......
0 k-1 0 k-1
0 2k-1

Mergesort - Analysis
Levels of recursive calls to mergesort, given an array of eight items

.
.
.
.
.
.
. . . . . . . . . . . . . . . . .
2
m

2
m-1

2
m-1

2
m-2
2
m-2
2
m-2
2
m-2

2
0

2
0

level 0 : 1 merge (size 2
m-1
)
level 1 : 2 merges (size 2
m-2
)
level 2 : 4 merges (size 2
m-3
)
level m
level m-1 : 2
m-1
merges (size 2
0
)

Worst-case
The number of key comparisons:
= 2
0
*(2*2
m-1
-1) + 2
1
*(2*2
m-2
-1) + ... + 2
m-1
*(2*2
0
-1)
= (2
m
- 1) + (2
m
- 2) + ... + (2
m
2
m-1
) ( m terms )

= m*2
m

= m*2
m
2
m
1
Using m = log n
= n * log
2
n n 1

O (n * log
2
n )
=
1
0
2
m
i
i

Mergesort Analysis
Mergesort is extremely efficient algorithm with respect
to time.
Both worst case and average cases are O (n * log
2
n )

But, mergesort requires an extra array whose size
equals to the size of the original array.

If we use a linked list, we do not need an extra array
But, we need space for the links
And, it will be difficult to divide the list into half ( O(n) )


47
Sorting Comments
Language libraries often have sorting
algorithms in them
Java Arrays and Collections classes
C++ Standard Template Library
Python sort and sorted functions
Hybrid sorts
when size of unsorted list or portion of array is
small use insertion sort, otherwise use
O(N log N) sort like Quicksort of Mergesort
Many other sorting algorithms exist.

48
Searching in a Sorted List
If items are sorted then we can divide and conquer
dividing your work in half with each step
generally a good thing
The Binary Search on List in Ascending order
Start at middle of list
is that the item?
If not is it less than or greater than the item?
less than, move to second half of list
greater than, move to first half of list
repeat until found or sub list size = 0

49
Binary Search
The binary search is much more efficient than the
linear search.
It requires the list to be in order.
The algorithm starts searching with the middle
element.
If the item is less than the middle element, it starts over searching the
first half of the list.
If the item is greater than the middle element, the search starts over
starting with the middle element in the second half of the list.
It then continues halving the list until the item is found.


50
Binary Search
list
low item middle item high item
Is middle item what we are looking for? If not is it
more or less than the target item? (Assume lower)
list
low middle high
item item item
and so forth

51
Binary Search in Action
2 3 5 7 11 13 17 19 23 29 31 37 41 47 43 53
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
public static int bsearch(int[] list, int target)
{ int result = -1;
int low = 0;
int high = list.length - 1;
int mid;
while( result == -1 && low <= high )
{ mid = low + ((high - low) / 2);
if( list[mid] == target )
result = mid;
else if( list[mid] < target)
low = mid + 1;
else
high = mid - 1;
}
return result;
}
// mid = ( low + high ) / 2; // may overflow!!!
// or mid = (low + high) >>> 1; using bitwise op

Attendance Question??
CS 307 Fundamentals
of Computer Science

52
What is the worst case Big O of binary search in
an array with N items, if an item is present?
A. O(N)
B. O(N
2
)
C. O(1)
D. O(logN)
E. O(NlogN)

53
Other Searching Algorithms
Interpolation Search
more like what people really do
Indexed Searching
Binary Search Trees
Hash Table Searching
Grover's Algorithm (Waiting for
quantum computers to be built)
best-first

Comparison of Sorting Algorithms


Sorting Algorithms-Sesion 6

Enviado por

Dados do documento

Descrição original:

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Sorting Algorithms-Sesion 6

Enviado por

Direitos autorais:

Formatos disponíveis

Sorting and Searching

Você também pode gostar