Escolar Documentos
Profissional Documentos
Cultura Documentos
National University
Page
Introduction
An algorithm analysis of a program is a step by step procedure for accomplishing that program In order to learn about an algorithm, we need to analyze it This means we need to study the specification of the algorithm and draw conclusion about the implementation of that algorithm (the program) will perform in general
National University
Page
The issues that should be considered in analyzing an algorithm are: The running time of a program as a function of its inputs The total or maximum memory space needed for program data
National University
Page
The main factors that effect the running time are the algorithm itself, input data, the computer system, etc.
The performance of a computer is determined by The hardware The programming language used and The operating system To calculate the running time of a general C++ program, we first need to define a few rules. In our rules , we are going to assume that the effect of hardware and software systems used in the machines are independent of the running time of our C++ program
National University
Page
Rule 1:
The time required to fetch an integer from memory is a constant t(fetch), The time required to store an integer in memory is also a constant t(store)
For example the running time of x=y is: t(fetch) + t(store) because we need to fetch y from memory store it into x
Similarly the running time of x=1 is also t(fetch) + t(store) because typically any constant is stored in the memory before it is fetched.
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University Page 5
Rule 2
The time required to perform elementary operations on integers, such as addition t(+), subtraction t(-), multiplication t(*), division t(/), and comparison t(cmp), are all constants.
For Example the running time of y= x+1 is: 2t(fetch) + t(store) + t (+) because you need to fetch x and 1: then add them together: and place the result into y:
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University
Rule 3:
The time required to call a function is a constant, t(call) And the time required to return a function is a constant, t(return)
Rule 4:
The time required to pass an integer argument to a function or procedure is the same as the time required to store an integer in memory, t(store)
National University
Page
National University
Page
Rule 5:
The time required for the address calculation implied by an array subscripting operation like a[i] is a constant, t([ ]). This time does not include the time to compute the subscript expression, nor does it include the time to access (fetch or store) the array element For example, the running time of y = a[i] is: 3t(fetch) + t([ ]) + t(store) Because you need To fetch the value of i: To fetch the value of a: To find the address of a[i]: To fetch the value of a[i]: To store the value of a[i] into y:
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University
Rule 6:
The time required to calculate a fixed amount of storage from the heap using operator new is a constant, t(new) This time does not include any time required for initialization of the storage (calling a constructor). Similarly, the time required to return a fixed amount of storage to the heap using operator delete is a constant, t(delete). This time does not include any time spent cleaning up the storage before it is returned to the heap (calling destructor)
For example the running time of For example the running time of int* ptr = new int; delete ptr; is: is: t(new) + t(store) t(fetch ) + t(delete) Because you need Because you need To allocate a memory: t(new) To fetch the address from ptr : t(fetch) And to store its address into ptr: t(store) And delete specifies location: t(delete)
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University Page 10
1. int Sum (int n) 2. { 3. int result =0; 4. for (int i=1; i<=n; i=i+1) 5. result = result + i; 6. return result 7. }
Statement Time 3 t(fetch)+ t(store) 4a 4b 4c 5 6 t(fetch) + t(store) (2t(fetch)+t(cmp)) * (n+1) (2t(fetch)+t(+) +t(store)) *n (2t(fetch)+t(+) +t(store)) *n t(fetch)+ t(return) Code result = 0 i=1 i<=n ++i Result +=i Return result
Total
A.R. Hadaegh Dr. Ahmad R. Hadaegh
1. int func (int a[ ], int n, int x) 2. { 3. int result = a[n]; 4. for (int i=n-1; i>=0; i=i-1) 5. result =result *x + a[i]; 6. return result 7. }
Statement Time
3 4a 4b 4c 5 6
3t(fetch)+ t([ ]) + t(store) 2t(fetch) + t(-) + t(store) (2t(fetch)+t(cmp)) * (n+1) (2t(fetch)+t(-) +t(store)) *n (5t(fetch)+t([ ])+t(+)+t(*)+t(store)) *n t(fetch)+ t(return)
Total
[(9t(fetch) +2t(store) + t(cmp) +t([]) + t(*) + t(-)]*n + [(8t(fetch) + 2t(store) + t([]) + t(-) +t(cmp) + t (return) )]
National University Page 12
Using constant times such as t(fetch), t(store), t(delete), t(new), t(+), , ect makes our running time accurate However, in order to make life simple, we can consider the approximate running time of any constant to be the same time as t(1). For example, the running time of y=x+1 is 3 because it includes two fetches and one store in which all are constants For a loop there are two cases: If we know the exact number of iterations, the running time becomes constant t(1) If we do not know the exact number of iterations, the running time becomes t(n) where n is the number of iterations
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University Page 13
1. int Geometric (int x, int n) 2. { 3. int sum = 0; 4. for (int i=0; i<=n; ++i) 5. { 6. int prod = 1; 7. for (int j=0; j<i; ++j) 8. prod = prod * x; 9. sum = sum + prod; 10. } 11. return result 12. }
Statement 3 4a 4b 4c 6 7a 7b
7c 8
9 11
Time 2 2 3(n+2) 4(n+1) 2(n+1) 2(n+1) n 3 Si=0 i+1 n 4 Si=0 i n 4 Si=0 i 4(n+1) 2
Total
(11/2)n2 + (47/2)n + 27
National University
Page 14
1. int Power (int x, int n) 2. { 3. if (n= =0) 4. return 1; 5. else if (n%2 = = 0) // n is even 6. return Power (x*x, n/2); 7. else // n is odd 8. return x* Power (x*x, n/2); 9. }
Statement 3 4 5 6 8 n=0 3 2 n>0 (n is even) 3 5 10 + T( n/2 ) n>0 (n is odd) 3 5 12 + T( n/2 )
Total
18 + T( n/2 )
20 + T( n/2 )
National University
Page 15
T(n) =
for n=0 for n>0 and n is even for n> 0 and n is odd
Suppose n = 2k for some k>0. Obviously 2k is an even number, we get T(2k) = 18 + T(2k-1) = 18 + (18 + T(2k-2)) = 18 + 18 + (18 + T(2k-3)) = .. = .. = 18k + T(2k-k) = 18k + T(20) = 18k + T(1)
National University
Page 16
Since T(1) is add number, the running time of T(1) is: T(1) = 20 + T(0) = 20 + 5 = 25 Therefore, T(2k) = 18k + 25 If n = 2k, then log n = log2k indicating that k = log n Therefore, T(n) = 18log n + 25
National University
Page 17
Asymptotic Notation
Suppose the running time of two algorithms A and B are TA(n) and TB(n), respectively where n is the size of the problem
Unfortunately, this is not an easy answer. We cannot expect that the size of n to be known ahead of time. But we may be able to say that under certain situations TA(n) is better than TB(n) for all n >= n1
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University Page 18
To understand the running times of the algorithms we need to make some definitions: Definition: Consider a function f(n) which is non-negative for all integers n>=0. We say that f(n) is big oh of g(n) which we write (f(n) is O(g(n)) if there exists an integer no and a constant c > 0 such that for all integers n >=no, f(n) <=c g(n) Example: Show that f(n) = 8n + 128 is O(n2)
8n + 128 <= c n2 (lets set c = 1) 0 <= cn2 -8n -128 0 <= (n-16) (n+8) Thus we can say that for constant c =1 and n >= 16, f(n) is O(n2)
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University Page 19
f(n)
g1(n)=4n2 g2(n)=2n2 g3(n)=n2
400
f(n)=8n+128 200
10
15
20
National University
Page 20
Theorem: If f1(n) is O(g1(n)) and f2(n) is O(g2(n)), then f1(n) + f2(n) = O (max(g1(n), g2(n))) Proof: If f1(n) is O(g1(n)) then f1(n) <= c1g1(n) for some c1 and n >= n1 If f2(n) is O(g2(n)) then f2(n) <= c2g2(n) for some c2 and n >= n2 Let no= max(n1, n2) and co = 2(max(c1, c2)), consider the sum f1(n) + f2(n) for some n >= no
f1(n) + f2(n)
Theorem: If f1(n) is O(g1(n)) and f2(n) is O(g2(n)), then f1(n) * f2(n) = O(g1(n)*g2(n) ) Proof: If f1(n) is O(g1(n)) then f1(n) <= c1g1(n) for some c1 and n>=n1 If f2(n) is O(g2(n)) then f2(n) <= c2g2(n) for some c2 and n>=n2 Let no= max(n1, n2) and co = c1*c2, consider the product of f1(n)*f2(n) for some n>=no
f1(n) * f2(n)
<= <=
Proof:
If f(n) is O(g(n)) then f(n) <= c1g(n) for some c1 and n>=n1 If g(n) is O(h(n)) then g(n) <= c2h(n) for some c2 and n>=n2 Let no= max(n1, n2) and co = c1*c2, then f(n) <= <= <= c1g1(n) c1c2h(n) co h(n)
National University
Page 23
National University
Page 24
Second, it is common practice to drop constant coefficients. Thus, instead of O(3n2), we write O(n2). As a special case of this rule, if the function is a constant, instead of, say O(1024), we simply write O(1)
National University
Page 25
Let no = 8, we can say that for c=1 and no>=8 f(n) is W(n2)
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University Page 26
1500
f(n)=5n2-64n+256 1000 f(n)=n2 500 f(n)=2n2
10
15
20
25
National University
Page 27
Other definitions
Definition: Consider a function f(n) which is non-negative for all integers n>=0. We say that f(n) is theta of g(n) which we write (f(n) is Q(g(n)) if and only if f(n) is O(g(n)) and f(n) is W(g(n)) Definition: Consider a function f(n) which is non-negative for all integers n>=0. We say that f(n) is little o of g(n) which we write (f(n) is o(g(n)) if and only if f(n) is O(g(n)) and f(n) is not W(g(n)) Now lets consider some of the previous examples in terms of the big O notations:
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University Page 28
1. int func (int a[ ], int n, int x) 2. { 3. int result = a[n]; 4. for (int i=n-1; i>=0; --i) 5. result =result *x + a[i]; 6. return result 7. }
Statement Simple Big O Time model 3 5 O(1) 4a 4b 4c 5 6 4 3n + 3 4n 9n 2
Total
A.R. Hadaegh Dr. Ahmad R. Hadaegh
16n + 14
National University
Page 29
1. int PrefixSums (int a[ ], int n) 2. { 3. for (int j=n-1; i>=0; --j) 4. { 5. int sum = 0; 6. for (int i=0; i<=j; ++i) 7. sum = sum + a[i]; 8. a[j] = sum; 9. } 10. return result 11. }
Statement Big O 3a 3a 3c 5
6a
6b 6c 7 9
Total
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University
O(1) O(1)*O(n) O(1)*O(n) O(1)*O(n) O(1)*O(n) O(1)*O(n2) O(1)*O(n2) O(1)*O(n2) O(1)*O(n) O(n2)
Page 30