Algorithms

Insertion sort analysis

Sorting
• The formal definition of sorting is:
• Input: a sequence of n numbers (𝑎1 , 𝑎2 , 𝑎3 , … , 𝑎𝑛 )
• Output: a permutation (reordering) 𝑎′1 , 𝑎′ 2 , 𝑎′ 3 , … , 𝑎′ 𝑛 such that
𝑎′1 ≤ 𝑎′ 2 ≤ 𝑎′ 3 … ≤ 𝑎′𝑛

• Such an input sequence is called an instance of the sorting problem

• Sorting is a fundamental operation in computer science, many
problems use it as an intermediary step

2
Types of Sorting
• In-place Sorting

• Internal Sorting

• External Sorting

• Stable and Unstable Sorting

3
In-place Sorting
• A sorting algorithm is said to be in-place if it requires very little
additional space besides the initial array holding the elements that
are to be sorted
• “Very little” is taken to mean that for sorting n elements, O(log n)
extra space is required
• This is reasonable because in a purely mathematical analysis of the
algorithms, any sorting algorithm that operates on a contiguous array
requires O(log n) extra space, since this is the number of bytes
required to represent an index into the array

4
Cont’d
• Examples:
• Bubble Sort, Comb Sort, Selection Sort, Insertion Sort, Heap Sort, Shell Sort
and Quick Sort

5
Internal Sorting
• An internal sort is any data sorting process that takes place entirely
within the main memory of a computer
• This is possible whenever the data to be sorted is small enough and
can be held in the main memory
• For sorting larger datasets, it may be necessary to hold only a chunk
of data in memory at a time, since it won’t all fit
• The rest of the data is normally held on some larger, but slower
medium, like a hard-disk

6
Cont’d
• Examples:
• Bubble Sort, Comb Sort, Selection Sort, Insertion Sort, Heap Sort, Shell Sort,
Redix Sort and Quick Sort

7
External Sorting
• External sorting is a class of sorting algorithms that can handle
massive amounts of data
• External sorting is required when the data being sorted do not fit into
the main memory and instead they must reside in the slower external
memory (usually a hard drive)
• External sorting typically uses a hybrid sort-merge strategy
• In the sorting phase, chunks of data small enough to fit in main
memory are read, sorted, and then written to a temporary file
• In the merge phase, the sorted sub-files are combined into a single
larger file

8
Cont’d
• Examples:
• K-way External Merge Sort, Distributions Sort

9
Stable vs. Unstable Sorting
• A sorting algorithm is said to be stable if two objects with equal keys
appear in the same order in sorted output as they appear in the input
unsorted array
• Some sorting algorithms are stable by nature like Insertion sort, Merge
Sort, Bubble Sort, etc. And some sorting algorithms are not, like Heap Sort,
Quick Sort, etc.
• However, any given sorting algo which is not stable can be modified to be
stable
• There can be sorting algo specific ways to make it stable, but in general,
any comparison based sorting algorithm which is not stable by nature can
be modified to be stable by changing the key comparison operation so that
the comparison of two keys considers position as a factor for objects with
equal keys

10
The Insertion Sort
• Insertion Sort is a simple sorting algorithm: a comparison sort in which the
sorted array (or list) is built one entry at a time
• It is much less efficient on large lists than more advanced algorithms such
as quicksort, heapsort, or merge sort
• However, insertion sort provides several advantages
• Simple implementation
• Efficient for (quite) small data sets
• More efficient in practice than most other simple quadratic, i.e. O(n2) algorithms
such as selection sort or bubble sort; the best case (nearly sorted input) is O(n)
• Stable, i.e. does not change the relative order of elements with equal keys
• In-place, i.e. only requires a constant amount O(1) of additional memory space

11
Working of Insertion Sort

12
13
Algorithm
1. for j ←2 to length(A)
2. key ← A[ j ]
3. > A[ j ] is added in the sorted sequence A[1, .. j-1]
4. i←j-1
5. while i > 0 and A [ i ] > key
6. A[ i +1 ] ← A[ i ]
7. i ← i -1
8. A [i +1] ← key

14
Cont’d

15
Analysis of Insertion Sort

16
Cont’d

17
Loop Invariant
• Invariant: a function, quantity, or property which remains unchanged
when a specified transformation is applied
• Loop Invariant: a property of loop that is true before and after each
iteration of the loop, even after last iteration i.e., unchanged
• Helps us understand why an algorithm is correct
• Has three important things:
• Initialization: It is true prior to the first iteration of the loop
• Maintenance: If it is true before first iteration, it remains true before next
iteration
• Termination: When the loop terminates, the invariant gives us a useful
property that helps us show that the algorithm is correct

18
Loop Invariant and Correctness of Insertion
Sort
for j  2 to length(A)

• Initialization: we show loop invariant holds before first iteration

• When j = 2, the subarray A[1.. J-1] consists of a single element A[1], which is
in fact sorted
• So invariant holds prior to first iteration

19
Loop Invariant and Correctness of Insertion
Sort – Cont’d
for j  2 to length(A)

• Maintenance: we show loop each iteration maintains invariant

• The body of the for loop works by moving A[j-1], A[j-2], A[j-3] and so on by
one position to the right until the proper position for A[j] is found (lines 4 – 7)
• Value of A[j] is inserted (line 8)

20
Loop Invariant and Correctness of Insertion
Sort – Cont’d
for j  2 to length(A)

• Termination: When loop terminates:

• The outer loop terminates when j exceeds n or when j = n + 1
• The subarray at this point is A[1 .. n], the elements of the original array but in
sorted order
• But the subarray A[1 .. n] is the entire array

• Therefore the algorithm is correct

21
Best Case Analysis
• The array is already sorted

• We can express this running time as an + b for some constants a and

b
• This is a linear function so the running time is 𝑇 𝑛 = 𝜃(𝑛)

22
Worst Case Analysis

23
Cont’d

• This can be represented as an2 + bn + c

• Which is a quadratic function so the running time is 𝑇 𝑛 = 𝜃(𝑛2)
24
Assignment
• Exercise 2.1-3 from Intro. to Algorithms 3rd Edition
• Due Date – April 2, 2018
• Format – Handwritten

25