CO1- Data Structure (19SC1202)

Time Complexity
✓ It is the time taken by a computer to completely execute a program.
✓ Program may be in any language. (C, C++, Java etc…)

Asymptotic Notations
✓ It is the meaningful way of representation of time complexity.
They are
i) Big Oh [ O (g(n)) ] { Upper bound }
ii) Big Omega [ Ω (g(n)) ] { Lower bound }
iii) Big Theta [ Ɵ (g(n)) ] { Average }

The Order increases in this format:

O (1), O (log n), O (n/2), O (n), O (n2), O (n3) . . .

Analysis: Sum of n natural numbers

Algorithm: Time taken
1) Start ------ 0
2) Input: Read ‘n’ ------ 1
3) initialize: sum=0, i=1 ------ 2
4) Process: sum = sum + i ------ n
5) i = i +1 ------ n
6) if ( i < = n ) go to step 4 ------ n +1
7) Output:
8) print ‘sum’ ------ 1
9) Stop ------ 0
Total time in function f(n) = 3n + 4

Big Oh: O() { Upper bound – Worst Case}

The function f(n) = O(g(n)), if and only if there exists constants “c and n0 “ such that f(n) ≤ c .
g(n) for all value of n, where as n>n0 .

Analysis : Sum of n natural numbers

f(n) = 3n + 4 ; f(n) ≤ C . g(n)
3n + 4 ≤ C . g(n)
3n + 4 ≤ 3n + 4n (Upper bound)
If 3n + 4 ≤ 7n then C = 7 , g(n) = n
If n=1 then 7 ≤ 7 ; If n=2 then 10 ≤ 14 ;If n=3 then 13 ≤ 21
 ✓ Therefore f(n) = O(g(n)) = O(n)
✓ thus f(n) = O(n) [Worst case ]

Big Theta [ Ɵ () ] { Average }

The function f(n) = Ɵ(g(n)), if and only if there exists constants “c1, c2 and n0“ such that c1 .
g(n) ≤ f(n) ≤ c . g(n) for all value of n, where as n>n0 .

Analysis : Sum of n natural numbers

f(n) = 3n + 4 ; c1 . g(n) ≤ f(n) ≤ c . g(n)
C1 . g(n) ≤ 3n + 4 ≤ C2 . g(n)
3n ≤ 3n + 4 ≤ 7n (Lower and Upper bound)
If 3n ≤ 3n + 4 ≤ 7n then C1 = 3 , C2 = 7 and g(n) = n
If n=1 then 3 ≤ 7 ≤ 7 ; If n=2 then 6 ≤ 10 ≤ 14 ;If n=3 then 9 ≤ 13 ≤ 21

✓ Therefore f(n) = Ɵ(g(n)) = Ɵ(n)

✓ thus f(n) = Ɵ(n) [Average Case]

Big Omega [ Ω () ] { Lower bound }

The function f(n) = Ω(g(n)), if and only if there exists constants “c and n0 “ such that f(n) ≥ c .
g(n) for all value of n, where as n>n0 .

Analysis : Sum of n natural numbers

f(n) = 3n + 4 ; f(n) ≥ C . g(n)
3n + 4 ≥ C . g(n)
3n + 4 ≥ 3n (Lower bound)
If 3n + 4 ≥ 3n then C = 3 , g(n) = n
If n=1 then 7 ≥ 3
If n=2 then 10 ≥ 6
If n=3 then 13 ≥ 9
 ✓ Therefore f(n) = Ω(g(n)) = Ω(n)
✓ thus f(n) = Ω(n) [Best Case]
Example II : Sum of n natural numbers
Algorithm: Time taken
1) Start ------ 0
2) Input: Read n ------ 1
3) Process: sum = n * (n+1) / 2 ------ 1
4) print ‘sum’ ------ 1
5) Stop ------ 0
Total time in function f(n) = 3

f(n) = 3 . 1 = C . g(n) ; C = 3 , g(n) = 1

therefore f(n) = O(g(n) ) = O (1) = Ω(1) = Ɵ(1)

Time complexity Analysis of Algorithms With examples

1) O(1): Time complexity of a function (or set of statements) is considered as O(1) if it
doesn’t contain loop, recursion and call to any other non-constant time function.
// set of non-recursive and non-loop statements
For example swap() function has O(1) time complexity.

A loop or recursion that runs a constant number of times is also considered as O(1). For
example the following loop is O(1).
// Here c is a constant
for (int i = 1; i <= c; i++) {
// some O(1) expressions
}
2) O(n): Time Complexity of a loop is considered as O(n) if the loop variables is incremented
/ decremented by a constant amount. For example following functions have O(n) time
complexity.
// Here c is a positive integer constant
for (int i = 1; i <= n; i += c) {
// some O(1) expressions
}
for (int i = n; i > 0; i -= c) {
// some O(1) expressions
}
3) O(nc): Time complexity of nested loops is equal to the number of times the innermost
statement is executed. For example the following sample loops have O(n2) time complexity
for (int i = 1; i <=n; i += c) {
for (int j = 1; j <=n; j += c) {
// some O(1) expressions
}
}
for (int i = n; i > 0; i -= c) {
for (int j = i+1; j <=n; j += c) {
// some O(1) expressions
}
For example Selection sort and Insertion Sort have O(n2) time complexity.
4) O(Logn) Time Complexity of a loop is considered as O(Logn) if the loop variables is
divided / multiplied by a constant amount.
for (int i = 1; i <=n; i *= c) {
// some O(1) expressions
}
for (int i = n; i > 0; i /= c) {
// some O(1) expressions
}

For example Binary Search(refer iterative implementation) has O(Logn) time complexity. Let
us see mathematically how it is O(Log n). The series that we get in first loop is 1, c, c2, c3, …
ck. If we put k equals to Logcn, we get cLogcn which is n.

5) O(LogLogn) Time Complexity of a loop is considered as O(LogLogn) if the loop

variables is reduced / increased exponentially by a constant amount.

// Here c is a constant greater than 1

for (int i = 2; i <=n; i = pow(i, c)) {
// some O(1) expressions
}
//Here fun is sqrt or cuberoot or any other constant root
for (int i = n; i > 1; i = fun(i)) {
// some O(1) expressions
}

How to combine time complexities of consecutive loops?

When there are consecutive loops, we calculate time complexity as sum of time complexities
of individual loops.

for (int i = 1; i <=m; i += c) {

// some O(1) expressions
}
for (int i = 1; i <=n; i += c) {
// some O(1) expressions
}
Time complexity of above code is O(m) + O(n) which is O(m+n)
If m == n, the time complexity becomes O(2n) which is O(n).

How to calculate time complexity when there are many if, else statements inside loops?
As discussed here, worst case time complexity is the most useful among best, average and
worst. Therefore we need to consider worst case. We evaluate the situation when values in if-
else conditions cause maximum number of statements to be executed.

Time Complexity of Loop with Powers

What is the time complexity of below function?
void fun(int n, int k)
{
for (int i=1; i<=n; i++)
{
 int p = pow(i, k);
for (int j=1; j<=p; j++)
{
// Some O(1) work
}
}
}
Time complexity of above function can be written as 1k + 2k + 3k + … n1k.
Let us try few examples:
k=1
Sum = 1 + 2 + 3 ... n
= n(n+1)/2
= n2 + n/2

k=2
Sum = 12 + 22 + 32 + ... n12.
= n(n+1)(2n+1)/6
= n3/3 + n2/2 + n/6

k=3
Sum = 13 + 23 + 33 + ... n13.
= n2(n+1)2/4
= n4/4 + n3/2 + n2/4

In general, asymptotic value can be written as (nk+1)/(k+1) + Θ(nk)

Note that, in asymptotic notations like Θ we can always ignore lower order terms. So the time
complexity is Θ(nk+1 / (k+1))

Simple Statement
This statement takes O(1) time.
int y= n + 25;
If Statement
The worst case O(n) if the if statement is in a loop that runs n times, best case O(1)
if( n> 100)
{
…
}else{
..
..
}
For / While Loops
The for loop takes n time to complete and and so it is O(n).
for(int i=0;i<n;i++)
{
..
..
}

If the for loop takes n time and i increases or decreases by a constant, the cost is O(n)
for(int i = 0; i < n; i+=5)
sum++;

for(int i = n; i > 0; i-=5)

sum++;

Nested loops
If the nested loops contain sizes n and m, the cost is O(nm)
for(int i=0;i<n;i++)
{
for(int i=0;i<m;i++){
..
..
}
}

If the first loop runs n2 times and the inner loop runs n times or (vice versa), the cost is O(n3)
for(int j=0;j<n*n;j++)
{
for(int i=0;i<n;i++){
..
..
}
}

If the first loop runs n times and the inner second loop runs n2 times and the third loop runs
n2, then O(n5)
for(int i = 0; i < n; i++)
for( int j = 0; j < n * n; j++)
for(int k = 0; k < j; k++)
sum++;

If the for loop takes n time and i increases or decreases by a multiple, the cost is O(log(n))
for(int i = 1; i < =n; i*=2)
sum++;

for(int i = n; i > 0; i/=2)

sum++;

If the first loop runs N times and the inner loop runs log(n) times or (vice versa), the cost is
O(n*log(n))
for(int i=0;i<n;i++)
{
for(int j=1;i<=n;j*=4){
..
..
}
}
////////////////////////////////////////////////INSERTION SORT////////////////O(N2)/////////////////////

Insertion sort resembles the arrangement of cards in card-player game. When a new card is
cards as each successive card is dealt you might have observed the following sequence of events: the
first card is dealt and, of course, since it is the only card, it is already in your hand in the correct
place. The second card is dealt and then you decide to place it either before or after the first card.
With the third the decision that must is made is whether to insert it before the first or second cards
or after the third card. This basic behavior goes on as each new card is received.

Consider an array A with N elements. The insertion sort algorithm scans the elements from
left to right i.e., from a[I] to a[n], inserting each element a[k] into its proper position in the
previously sorted sub array i.e., a[1] to a[k-1].

Algorithm:
Step 1: A[1] by itself is sorted.
Step 2: A[2] is inserted before or after a[1], so that a[1] and a[2] are sorted.
Step 3: A[3] is inserted into its proper place with respect to the elements a[1] and a[2]
Step 4: This process is continued till all the elements are sorted.

To make the process of moving easier we introduce element a[0]=least value (according to
the data type) Whose key value is smaller than any other keys among a[1],a[2],……….,a[n]

Examples: Input = 5, 2, 4, 6, 1, 2

Output: 1, 2, 3, 4, 5, 6
Program:
#include <stdio.h>
void insertionsort( int a[], int n)
{
int i,j,t;
for (i = 1 ; i < n ; i++)
{
j = i;
while((j > 0) && (a[j] < a[j-1]))
{
t = a[j];
a[j] = a[j-1];
a[j-1] = t;
j--;
}
}
}

void main()
{
int n, a[100], i;

printf("Enter number of elements\n");

scanf("%d", &n);
printf("Enter %d integers\n", n);
for (i = 0; i < n; i++)
scanf("%d", &a[i]);
printf("Sorted list in ascending order:\n");

insertionsort(a,n);

for (i = 0; i < n; i++)

printf("%d\n", a[i]);
}

Time Complexity:
The time for sorting is measured in terms of the number of comparisons. In the worst case if
the array is in the reverse order then the inner while loop takes j-1 comparisons. That means
according insertion sort in pass-1 it takes one comparison. In pass-II, 2 comparisons and etc.
Hence the time complexity function f(n) can be calculated as follows :

F(n) = 1+2+……………………+(n-1) = n(n-1)/2 = O(n2)

In the average case there will be approximately (j-1)/2 comparisons for the inner while loop.
The time complexity f(n) in the average case may be calculated as follows:

F(n)=1/2 + 2/2+………………+(n-1)/2 = n(n-1)/4 = O(n2)

/////////////////////////////////////////////////SHELL SORT//////////////////////// O(N2)////////////////////////

It is introduced by D.L Shell which uses insertion sort on periodic sub-sequence of the input
to produce a faster sorting algorithm. It is also known as diminishing increment sort. It is the
1st sorting algorithm to break 'n2' barrier. It is fast, easy to understand, easy to implement.
However, its complexity analysis is little more sophisticated. It begins by comparison of an
element i.e at a distance or gap 'd' which is initially half of the number of elements in the
array. Further in each part, the value of 'd' is reduced to half.

The Shell sort is significantly slower than merge, heap, quick sort but it is relatively simple
algorithm. It is also known as excellent choice for repetitive sorting of smaller list.

Algorithm:
Step 1 − Initialize the value of ‘gap = N/2’
Step 2 − Divide the list into smaller sub-list of equal interval ‘gap’
Step 3 − Sort these sub-lists using insertion sort
Step 3 − Repeat until complete list is sorted

Example of shell Sort

Use Shell sort for the following array : 18, 32, 12, 5, 38, 30, 16, 2

Compare the elements at a gap of 4. i.e 18 with 38 and so on and swap if first number is
greater than second.
Compare the elements at a gap of 2 i.e 18 with 12 and so on.

Now the gap is 1. So now use insertion sort to sort this array.

After insertion sort. The final array is sorted.

Program:
#include <stdio.h>
void shellsort( int a[ ], int n)
{
int gap,i,j,t;
gap=n/2;
while(gap>0)
{
for( i=gap ;i<n;i++)
{
for(j=i-gap ; j>=0; j=j-gap)
{
if (a[j] >a[j+gap])
{
t=a[j];
a[j]=a[j+m];
a[j+m]=t;
}
}
}
gap=gap/2;
}
void main()
{
int n, a[10], i, j, t,m;
printf("Enter number of elements\n");
scanf("%d", &n);
printf("Enter %d integers\n", n);
for (i = 0; i < n; i++)
scanf("%d", &a[i]);

shellsort(a,n);

printf("Sorted list in ascending order:\n");

for (i = 0; i < n; i++)
printf("%d\n", a[i]);
}

Time Complexity:
Since in this algorithm insertion sort is applied in the large interval of elements and then
interval reduces in a sequence, therefore the running time of Shell sort is heavily dependent
on the gap sequence it uses .

Worst Case Time complexity: O (N2)

Average Case Time complexity: depends on gap sequence.
Best Case Time complexity: O(N*log N)
////////////////////////////////////////////////////////////// HEAP SORT //////////////O(N logN)////////////

Heap sort is a comparison based sorting technique based on Binary Heap data structure. It is
similar to selection sort where we first find the maximum element and place the maximum
element at the end. We repeat the same process for remaining element.

A Binary Heap is a Complete Binary Tree where items are stored in a special order such that
value in a parent node is greater(or smaller) than the values in its two children nodes. The
former is called as max heap and the latter is called min heap. The heap can be represented by
binary tree or array. Since a Binary Heap is a Complete Binary Tree, it can be easily
represented as array and array based representation is space efficient. If the parent node is
stored at index I, the left child can be calculated by 2 * I + 1 and right child by 2 * I + 2
(assuming the indexing starts at 0). Whereas I starts from n/2 – 1. ‘n' is total number of
elements in the array.

Algorithm:
Step 1. Build a max heap from the input data.
Step 2. At this point, the largest item is stored at the root of the heap. Replace it with the last
item of the heap followed by reducing the size of heap by 1. Finally, heapify the root of tree.
Step 3. Repeat above steps while size of heap is greater than 1.

Example:

10 5 3 4 1
Swap : a[0] and a[n-1]
1 5 3 4 10
Heapify the remaining elements of 1,5, 3, 4 except 10.
Repeat until to the array elements having single element.
Program:
#include<stdio.h>
void heapify(int a[], int n, int i)
{
int root,l,r,t;
root = i;
l = 2*i + 1;
r = 2*i + 2;
if (l<n && a[l] > a[root])
root = l;
if (r<n && a[r] > a[root])
root = r;
if (root != i)
{
t=a[i];
a[i] = a[root];
a[root] = t;
heapify(a, n, root);
}
}
void heapsort(int a[], int n)
{
int i,t;
for(i=n/2-1; i>=0; i--)
heapify(a, n, i);
for(i=n-1; i>=0; i--)
{
t= a[0];
a[0]= a[i];
a[i] = t;
heapify(a, i, 0);
}
}
int main()
{
int a[50],i,n;
printf("Enter total number of elements:");
scanf("%d", &n);
printf("Enter the elements:\n");
for(i = 0; i < n; i++)
scanf("%d", &a[i]);
heapsort(a,n);
printf("\n\nAfter Heap sort:\n");
for(i = 0;i < n; i++)
printf("%d\t", a[i]);
return 0;
}

Time Complexity: Time complexity of heapify is O(nLogn). Time complexity of

createAndBuildHeap() is O(n) and overall time complexity of Heap Sort is O(N LogN).
////////////////////////////////////////////MERGE SORT ///////////////////////////////// O(N logN)////////////

Merge sort is a sorting technique based on divide and conquer technique. With worst-case
time complexity being Ο(n log n), it is one of the most respected algorithms. Merge sort first
divides the array into equal halves and then combines them in a sorted manner.

Algorithm:
Merge-Sort (a, first, last)
Step 1: if (first < last)
{
Step 2: mid = (first+last)/2 ;
Step 3: Merge-Sort(a, first, mid) ;
Step 4: Merge-Sort(a, mid+1, last) ;
Step 5: Merge(a, first, mid, last) ;
}
The above procedure sorts the elements in the sub array a[first...last].
To sort given any a[n] invoke the algorithm with parameters (a, 0, n -1).
We can assume each element in a given array as a sorted sub array. Take adjacent arrays and
merge to obtain a sorted array of two elements. Next step, take adjacent sorted arrays, of size
two, in pair and merge them to get a sorted array of four elements. Repeat the step until
whole array is sorted.

Example: Input elements are : 28,27,43,2,7,82,10

Sorted output: 3,9,10,27,38,43,82.

Program:
#include <stdio.h>
void msort(int a[],int low,int mid,int high)
{
int temp[50],i,j,k;
i=low;
j=mid+1;
k=0;
while(i<=mid && j<=high)
{
if(a[i]<a[j])
temp[k++]=a[i++];
else
temp[k++]=a[j++];
}
while(i<=mid)
temp[k++]=a[i++];
while(j<=high)
temp[k++]=a[j++];
for(i=low,j=0;i<=high;i++,j++)
a[i]=temp[j];
}

void mergesort(int a[],int low,int high)

{
int mid;

if(low<high)
{
mid=(low+high)/2;
mergesort(a,low,mid);
mergesort (a,mid+1,high);
msort(a,low,mid,high);
}
}
void main()
{
int a[50] , i, n;
printf("Enter total number of elements:");
scanf("%d", &n);
printf("Enter the elements:\n");
for(i = 0; i < n; i++)
scanf("%d", &a[i]);

mergesort (a,0,n - 1);

printf("After merge sort:\n");

for(i = 0;i < n; i++)
printf("%d\t", a[i]);
}
////////////////////////////////////////////////////////QUICK SORT///////////////////// O(N log N)////////////

Quick sort is a highly efficient sorting algorithm and is based on partitioning of array of data
into smaller arrays. A large array is partitioned into two arrays one of which holds values
smaller than the specified value, say pivot, based on which the partition is made and another
array holds values greater than the pivot value.

Quick sort partitions an array and then calls itself recursively twice to sort the two resulting
subarrays. This algorithm is quite efficient for large-sized data sets as its average and worst
case complexity are of Ο(n log n), where n is the number of items.

Algorithm:
Step 1: Input all the ‘n’ elements in the array as a[n].
Step 2: Choose the pivot element as first element, also assign ‘i’ as first element and
‘j’ as last element.
Step 3. Move the position ‘i’ from left to right ( i++) if a[i] < = a[pivot]
Step 4. Move the position ‘j’ from right to left ( j++) if a[j] > a[pivot]
Step 5. If ‘i’ and ‘j’ stops moving (i<j) then swap a[i] and a[j].
Step 6. If ‘i’ crossed ‘j’ (j>i) then swap a[ j ] & a[pivot] and fix pivot at the position of ‘j’.
Step 7. Now this pivot element partitioned the array elements in to two sub arrays.
Step 8. Apply recursion operation from steps 2 to step 7 for left side sub array from first
element to j-1 position. Then apply for right side sub array from j+1 position to last.
Step 9. Finally this recursion sort the element in sorted order.

Example:
Proceed the above operation recursively until the elements are sorted.

Program:
#include<stdio.h>
void quicksort(int a[ ],int first,int last)
{
int pivot,j,t,i;
if(first<last)
{
pivot=first;
i=first;
j=last;
while(i<j)
{
while(a[i] <= a[pivot])
i++;
while(a[j]>a[pivot])
j--;
if(i<j)
{
t=a[i];
a[i]=a[j];
a[j]=t;
}
}
t=a[pivot];
a[pivot]=a[j];
a[j]=t;
quicksort(a,first,j-1);
quicksort(a,j+1,last);
}
}

int main()
{
int a[100],n,i;
printf("Enter size of the array: ");
scanf("%d",&n);
printf("Enter %d elements: ",n);
for(i=0;i<n;i++)
scanf("%d",&a[i]);

quicksort(a,0,n-1);

printf("Sorted elements: ");

for(i=0;i<n;i++)
printf(" %d",a[i]);
return 0;
}
//////////////////////BUCKET SORT ///////////////////////////O(N)//////////////////////////

Bucket sort is a comparison sort algorithm that operates on elements by dividing them into
different buckets and then sorting these buckets individually. Each bucket is sorted
individually using a separate sorting algorithm or by applying the bucket sort algorithm
recursively. Bucket sort is mainly useful when the input is uniformly distributed over a range.

Assume one has the following problem in front of them:

One has been given a large array of floating point integers lying uniformly between the lower
and upper bound. This array now needs to be sorted. A simple way to solve this problem
would be to use another sorting algorithm such as Merge sort, Heap Sort or Quick Sort.
However, these algorithms guarantee a best case time complexity of O(N log N).. However,
using bucket sort, the above task can be completed in O(N) time. Let's have a closer look at
it.

Algorithm:

bucketSort(arr[], n)
1) Create n empty buckets (Or lists).
2) Do following for every array element arr[i].
-> Insert arr[i] into bucket[n*array[i]] (Apply hashing principle)
3) Sort individual buckets using insertion sort.
4) Concatenate all sorted buckets.
///////////////////////////////////////External sorting//////////////////////////////////////////////////

External sorting is a technique in which the data is stored on the secondary memory, in which
part by part data is loaded into the main memory and then sorting can be done over there.
Then this sorted data will be stored in the intermediate files. Finally, these files will be
merged to get a sorted data. Thus by using the external sorting technique, a huge amount of
data can be sorted easily. In case of external sorting, all the data cannot be accommodated on
the single memory, in this case, some amount of memory needs to be kept on a memory such
as hard disk, compact disk and so on.

The requirement of external sorting is there, where the data we have to store in the main
memory does not fit into it. Basically, it consists of two phases that are:

1. Sorting phase: This is a phase in which a large amount of data is sorted in an

intermediate file.
2. Merge phase: In this phase, the sorted files are combined into a single larger file.

One of the best examples of external sorting is external merge sort.

External merge sort:

The external merge sort is a technique in which the data is stored in intermediate files and
then each intermediate files are sorted independently and then combined or merged to get a
sorted data.

For example: Let us consider there are 10,000 records which have to be sorted. For this, we
need to apply the external merge sort method. Suppose the main memory has a capacity to
store 500 records in a block, with having each block size of 100 records.
In this example, we can see 5 blocks will be sorted in intermediate files. This process will be
repeated 20 times to get all the records. Then by this, we start merging a pair of intermediate
files in the main memory to get a sorted output.

Two-Way Merge Sort

Two-way merge sort is a technique which works in two stages which are as follows here:

Stage 1: Firstly break the records into the blocks and then sort the individual record with the
help of two input tapes.

Stage 2: In this merge the sorted blocks and then create a single sorted file with the help of
two output tapes.

By this, it can be said that two-way merge sort uses the two input tapes and two output tapes
for sorting the data.

Algorithm for Two-Way Merge Sort:

Step 1) Divide the elements into the blocks of size M. Sort each block and then write on disk.

Step 2) Merge two runs

1. Read first value on every two runs.

2. Then compare it and sort it.
3. Write the sorted record on the output tape.

Step 3) Repeat the step 2 and get longer and longer runs on alternates tapes. Finally, at last,
we will get a single sorted list.
Analysis

This algorithm requires log(N/M) passes with initial run pass. Therefore, at each pass the N
records are processed and at last we will get a time complexity as O(N log(N/M).