Teaching Growth of Functions Using Equivalence Classes
An Alternative to Big O Notation
Constantine Roussos
Lynchburg College
1501 Lakeside Drive, Lynchburg, VA 24501
804-544-8395
roussos@lynchburg.edu
ABSTRACT
Understanding growth of functions using the standard big O
definition and notation is a challenge for many undergraduate
students. This paper
1. presents an approach to teaching growth of functions that
utilizes equivalence classes and partial ordering,
2. identifies those mathematical concepts students should
comprehend in order to understand the principles underlying
growth of functions,
3. demonstrates pedagogical inadequacies in existing order of
complexity notation and definitions and
4. gives a rationale for restricting functions under consideration
to positive-valued, monotonic increasing.
Categories and Subject Descriptors
G.2 [Discrete Mathematics]: and F.2 [Analysis of Algorithms
and Problem Complexity]: – order of complexity, equivalence
classes, partial ordering, big O notation.
General Terms
Algorithms, Theory
Keywords
big O, equivalence classes, little o, order of complexity, theta,
complexity analysis, growth of functions.
1 INTRODUCTION
Traditionally, growth of functions has been taught using big O
notation. Although the concept and definition of big O are sound,
for many students they are hard to grasp. This is due, in part, to
the complexity of the definition of big O and, in part, to the fact
that big O is often presented before students have been exposed to
concepts necessary to understand the principles underlying
complexity analysis. Big O notation was introduced by P.
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise,
or republish, to post on servers or to redistribute to lists, requires prior
specific permission and/or a fee.
SIGCSE’04, March 3–7, 2004, Norfolk, Virginia, USA.
Copyright 2004 ACM 1-58113-798-2/04/0003…$5.00
170
Bachman in the book Analytishche Zahlen-theorie in 1892 [5]. I
consider the notation to be archaic and present arguments to show
that more modern mathematical notation better serves its function.
This paper presents growth functions as members of equivalence
classes and uses a modified big O definition to impose a partial
ordering on these equivalence classes.
Additionally, the functions under consideration are restricted to
positive-valued, non-decreasing (monotonic increasing) functions,
those normally encountered in algorithm analysis. This approach
is consistent with the intuitive idea of comparing two functions as
being less than, greater than or equal to.
The paper addresses the topic at two levels. The first level being
that reached in a Discrete Mathematics course and the second
being the beginning of a course in Design and Analysis of
Algorithms. The importance of proper sequencing of course topics
is addressed and note is made of the background material required
for students to understand the various topics covered.
Limits and derivatives are shown to be useful tools in computing
the order of complexity of a function but probably not useful for
defining function relationships. Examples are presented to
support this.
2 FUNCTIONS UNDER
CONSIDERATION
Since the purpose in teaching growth of functions is ordinarily to
give students an important tool for comparing the efficiency of
algorithms we restrict ourselves to functions associated with the
execution of algorithms. These functions, normally,
a) map from Z+ to Z+ (the positive integers).
b) are monotonic increasing (non-decreasing)
Definition 2.1: We shall refer to the family of functions defined
above as Discrete Growth Functions.
3 THE BASICS AND BIG O
Selection Sort requires f(N) = C1* N2 operations to sort a list of N
numbers in the worst case. Similarly, Bubble Sort requires
g(N)=C2*N2 operations to sort N numbers in the worst case.
(Note: All Ci are constants and all logs are base 2, denoted lg.)
The difference in the number of operations executed between
Selection Sort and Bubble Sort in the worst case, expressed as a
ratio, is a constant (C1/C2). In practice, a constant ratio may easily
be made insignificant by any number of factors including
a) the speed of the computer on which the program executes
b) coding efficiencies/ inefficiencies
c) the language in which the program is written, etc.
Note that the ratio, f(N)/g(N) = C1/C2, remains constant as N
increases. In complexity analysis such a constant multiple
difference is not considered to be significant. Hence, we consider
Selection Sort and Bubble Sort to be equally efficient even though
Selection Sort typically requires fewer operations and runs faster
than Bubble Sort.
Merge Sort, however, requires only h(N) = C3*N*lg(N)
operations. The ratio between Selection Sort and Merge Sort is
C1*N2/(C3*N*lg(N)) = (C1/C3)*N/lg(N).
This ratio increases without bound as N increases without bound.
I.e., h grows more slowly than f and we consider h to be less than
f in some meaningful sense. This holds true regardless of the
relative values of C1 and C3. Hence we do consider the running
times (efficiency) of these algorithms to be significantly different.
Traditionally, the means for expressing these differences has been
the big O comparison.
4 BIG O AND ITS SHORTCOMINGS
Definition 4.1: (Big O)
If f and g are real-valued functions then f(N) is O(g(N)) (big O of
g(N)) iff there exist constants C and k such that f(N) <= C*g(N)
whenever N>k. [2]
I. e., function f is less than a single constant multiple of function g
for all values of N that are sufficiently large.
Most texts state that f = O(g) [2]. However, many students do not
understand that the = in this context does not represent strict
equality and are confused about its use and special restrictions.
Consequently I avoid the use of that notation in favor of f is O(g)
as in [7].
We then say f is big O of G. Some authors (for ex. [2]) consider
O(g) to be a set. In this case we should say that f ∈ O(g). This is
sound mathematically and pedagogically.
I see three pedagogical impediments inherent in reliance on the
definition of Big O given above.
4.1 Pedagogical Impediments with Big O
a. The big O definition is too complex for many students.
b. The definition only applies to less than or equal to
comparisons; not equal to or strictly less than.
c. The notation of the big O definition does not lend itself to
mathematical manipulation or rigor.
Example 4.1: Let f(N) = N + 10 and let g(N) = 2*N.
Show that f is O(g) using Definition 4.1.
Solution:
Whenever N >= 10, f(N) <= g(N). Therefore we may choose C =
1 and k = 9 and use Definition 4.1 to show that f is O(g).
Also, note that f(N) <= 6*g(N) whenever N >= 1. Therefore we
may also choose C =6 and k = 0 when applying Definition 4.1. In
fact, there are an infinite number of choices of C and k that will
work with Definition 4.1 to show that f is O(g).
171
Unfortunately, many students have difficulty applying the
definition even to this simple example. The reason for the
difficulty seems to lie in the fact that the two constants C and k
yield two degrees of freedom and this can be confusing. Some
students assume that only certain, hard-to-find combinations of C
and k will actually work. Explaining to them that the two degrees
of freedom actually make it easier to find the required constants
does not seem to help. Those readers who have worked through
the definition of the limit of a sequence [4] or epsilon-delta proofs
for the limit of a real-valued function [8] will detect striking
similarities.
Fortunately, there is an easier way.
Definition 4.1a: If f and g are positive-valued functions defined
on the positive integers (recall condition a. for Discrete Growth
Function) then f(N) is O(g(N)) iff there exists a constant C such
that f(N) <= C*g(N) for all N [7]).
It is often most useful to compare two functions, f and g, by
examining the ratio f/g. To be consistent with this approach
Definition 4.1a may be restated as follows:
Definition 4.2: (Big O modified)
If f and g are functions that map from Z+ to Z+ then f(N) is
O(g(N)) iff there exists a constant C such that
f(N)/g(N) <= C for all N.
This simpler definition appears to be easier for students to grasp
and to use. This addresses issue a. of "Pedagogical Impediments".
5 EXTENSIONS TO BIG O
As stated above, Big O only compares functions for being less
than or equal to. Often we would like to know if one function is
equal to or strictly less than another. The traditional approach to
addressing this problem is to introduce two additional definitions.
Definition 5.1: f is Θ(g) (Theta of g) iff f is O(g) and g is O(f).
In other words, if f<=g and g<=f then f ≡ g.
It follows directly that if f is Θ(g) then g is Θ(f).
A Caveat:
The use of Θ has led to some confusion in order of complexity
discourse. A common statement is "The order of complexity of
function f is g". It is not always clear if this is to mean that f is
O(g) or f is Θ(g). The latter meaning seems to be more useful.
Many texts define less than in terms of limits [6].
Definition 5.2: f is o(g) (little o of g) iff lim(f(N)/g(N)) = 0 as N
⇒ ∞.
I.e., if f grows at a substantially slower rate than g then the ratio
f(N)/g(N) will approach 0 as N grows .
An inherent incompatibility between the definition of big O and
this definition of little o will be discussed later.
6 A DIFFERENT PARADIGM USING
EQUIVALENCE CLASSES
A more intuitive approach may be used to compare discrete
growth functions. We may partition functions into equivalence
classes [3] based upon their rates of growth by utilizing a
modified Θ definition.
Definition 6.1: (Function Equivalence) Functions f and g are
said to be equivalent (members of the same equivalence class ) iff
there exists constants C1 and C2 such that f(N)/g(N) <= C1 and
g(N)/f(N) <= C2 for all N ∈ Z+. We may write f ≡ g.
Let [f] denote the equivalence class containing function f.
It is an easy and worthwhile exercise to show that every element
of [f] generates the same equivalence class.
In order to ensure that definition 6.1 does, in fact, define
equivalence classes we must show the relation is reflexive,
symmetric and transitive [3]. All are easy to show so the proofs
may be given as exercises for students.
It is also worthwhile to note here that these equivalence classes
are sets of functions [2]. This means that the results students have
studied in set theory are relevant to the study of growth function
equivalence classes.
From Definition 6.1 we may easily conclude the following results.
The proofs may be given as exercises for students.
6.1 f ≡ g iff f ∈ [g].
6.2 f ∈ [g] iff g ∈ [f].
6.3 [f] = [g] iff f ≡ g
Prior to introducing students to Definition 6.1 students should be
shown what it means for one function to be less than another. I.e.,
present examples of two functions f and g where f(N) < g(N). This
might also be a good time to introduce the hierarchical growth
function chart that appears in nearly every text that covers Big O.
The chart should, at least, include lg(N), Nk and CN where C and k
are constants.
Also, students should be familiar with equivalence classes. A
classic example of equivalence classes is the integers modulo p
where p is normally a prime [2] and [3]. Using the integers
modulo 5 as an example most will recall that
{…, -5, 0, 5, 10, …} forms the equivalence class [0] and {…, -4,
1, 6, 11, …} forms the equivalence class [1], etc.
The background material given above should come early in a
student’s college studies. ACM Curriculum 2001 recommends
this prerequisite material be covered in the first year [1].
7 ORDERING FUNCTIONS
Next, we address the concept of less than or equal to for growth
function equivalent classes. Following a standard approach we
define a partial ordering [3] on the equivalence classes that we
have just defined.
Definition 7.1: (≤≤ Function Ordering)
[f] ≤ [g] iff there exists a constant C, such that f(N)/g(N)<=C for
all N ∈ Z+ where f ∈ [f], g ∈ [g].
This is just the (modified) definition for Big O.
It is important to note that we have defined a relation on
equivalence classes by referring to a relation on function values
(integers in this case). The distinction between relations and
operations on equivalence classes and relations and operations on
function values should be emphasized to students.
In order to show that Definition 7.1 does, in fact, define a partial
ordering we must show that the relation so defined is reflexive,
172
antisymmetric and transitive [3]. We must also show that the
choice of functions from [f] and [g] is irrelevant.
The proofs that these properties hold are easy to complete and
may be given as exercises for the student.
One may ask if Definition 7.1 actually defines a total ordering.
I.e. are every two equivalence classes (or every two discrete
growth functions) comparable?
It is not difficult to produce two functions f and g such that there
is no constant C1 such that f(N) <= C1*g(N) nor is there a
constant, C2, such that g(N) <= C2*f(N). In such a case f and g are
not comparable and, thus, Definition 7.1 defines only a partial
ordering (not a total ordering).
Ideally, to be consistent with familiar mathematical concepts, we
should like to be able to say that if [f] ≤ [g] and [f] ≠ [g] then [f] <
[g]. Defining [f] < [g] in this manner requires functions f and g to
be monotonically increasing. Example 7.1 illustrates this.
Example 7.1: (The need for monotonic increasing)
Let f(N) = N.
Let g(N) = N1/2 when N is a power of 2 and N otherwise.
Clearly g(N) <= 2*f(N) and, thus, [g] <= [f].
However, there is an infinite and unbounded number of points for
which g(N) = N1/2. At these points f(N)/g(N) = N1/2 . Therefore it
is impossible to find a constant C such that f(N)/g(N) <= C.
Hence, [f]≠[g]. We probably do not want to consider [g] to be <
[f] when f and g are so very nearly equal. Thus we have the
uncomfortable situation where [f] ≤ [g] and [f] ≠ [g] but it is not
true that [f] < [g]. The difficulty in this example is due, in large
part, to the downward drops in the value of g(N) when N is a
power of two. Requiring our functions to be monotonically
increasing eliminates such sharp drops.
Question 7: (inadequacy of little o for <)
Should we use the traditional little o definition (Definition 5.2) to
define < on growth functions?
Example 7.2 (inconsistency of little o and Big O)
Let g(N) = N and h(N) = N1/2 .
We shall construct a function, f(N), that shares an infinite,
unbounded number of points with g(N) and (within epsilon)
shares an infinite, unbounded number of points with h(N).
Let  represent the floor function [3].
Let q(N) = lg(lg(N)).
Let p(N) = 2q(N) .
I.e. the exponent of 2 above is lg(lg(N)).
Let f(N) = 2p(N).
The reader may verify that f(N) has the properties stated above.
Furthermore h(N) <= f(N) <= g(N) = N, for all N ∈ Z+
Hence, [f] ≤ [g].
But since f(N) (nearly) shares an infinite, unbounded number of
points with h(N) and [h] < [g] it is not true that [g] ≤ [f]. So [f] ≠
[g].
It can easily be shown that lim(f(N)/g(N) (as N ⇒ ∞) does not
exist even if a smooth (everywhere differentiable) function that
approximates f(N) is substituted for f(N). So if we use little o
(Definition 5.2) as the definition for < we have a case where f ≤ g
and f ≠g but it is not true that f < g. Thus, the limit definition for
little o is incompatible with the big O definition and our partial Theorem 8.2:
ordering, ≤. a. [C * f] = [f] where C is a constant.
b. [f + g] = max([f], [g]) if [f] and [g] are comparable.
The following definition for < is compatible with the definitions Note: if f and g are not comparable then [f + g] is distinct
for ≤ and =. from [f] and [g].
Definition 7.2: (Less Than Function Ordering) The above are not difficult to prove and may be given as exercises
[f] < [g] iff [f] ≤[g] and [f] ≠[g]. to advanced students.

8 THEOREMS AND PROOF 8.2 "Bracketing" for Complexity Analysis

A common and effective method of comparing functions is
TECHNIQUES sometimes referred to as "Bracketing".
8.1 Operations on Growth Function
Equivalence Classes Theorem 8.3: (Bracketing)
If f(N) <= g(N) <= h(N) (N>k) and [f] = [h] Then g ∈ [f]
Definition 8.1:
a. [f] + [g] = [f + g] The proof is left as an exercise for advanced students.
b. C * [f] = [C * f]
c. [f] * [g] = [f * g] Example 8.1:
d. [f] / [g] = [f / g] Find a simpler expression equivalent to N2/(N + lg(N)).

Here we have defined operations on equivalence classes in terms Solution 1: (Using Theorem 8.3, bracketing)
of operations on functions. These definitions are very powerful for Since 0 <= lg(N) <= N,
manipulating order of complexity expressions. However, it is very N2/(N + N) <= N2/(N + lg(N)) <= N2/(N + 0)
important to note that the definitions are meaningful only if the ∴ N/2 <= N2/(N + lg(N)) <= N
particular choice of an element from an equivalent class is [N/2] = [N]
irrelevant. ∴ N2/(N + lg(N)) ∈ [N]

We may show, for example, that in Definition 8.1d, [f / g] yields As straightforward as the above is, the use of Theorem 8.2 and
the same equivalence class for any choice of elements from [f] Definition 8.1d yield an almost trivial solution.
and [g]. See Theorem 8.1d below.
Solution 2: (Using Theorem 8.2)
Theorem 8.1d: If f1, f2 ∈ [f], and g1, g2 ∈ [g] Then Since [lg(N)] <= [N], [N + lg(N)] = [N]
[f1 / g1] = [f2 / g2] ∴ [N2/(N + lg(N))] = [N2/N] = [N]
Proof:
Let f1, f2 ∈ [f], and g1, g2 ∈ [g]. Definition 8.1 allows us to substitute any member of an
Then, f1 <= C1*f2 and g2 <= C2*g1 ⇒ g2 / C2 <= g1 equivalence class for any other member. This is a very intuitive
Since f1 <= C1*f2 and powerful method.
f1 / g1 <= C1 * f2 / g1
Since g2 / C2 <= g1, 8.3 Using Limits for Complexity Analysis
f1 / g1 <= C1 * f2 / g1 <= C1 * f2 /( g2 / C2) Limits can play a very useful role in simplifying complexity
f1 / g1 <= C1 * C2* f2 /g2 expressions.
(f1 / g1)/ (f2 / g2) <= C1 * C2
∴ [f1 / g1] ≤ [f2 / g2] Theorem 8.4:
a. lim(f(N)/g(N)) = C >0 ⇒ [f] = [g]
Similarly, by reversing the roles of f1, f2 and g1, g2 we can show b. lim(f(N)/g(N)) = 0 ⇒ [f] ≤ [g].
that
[f2/ g2] <= [f1,/ g1] thus giving the desired result that The proofs may be assigned to advanced students with good
[f1/ g1] = [f2,/ g2] Calculus skills.
Students with an understanding of Calculus may question why
Equivalence class notation simplifies proving theorems and derivatives are not used for determining function growth. Note:
solving order of complexity problems. Perhaps even more f′(N) denotes the derivative of f(N).
significant than proving theorems using this paradigm is the lack
of necessity for some theorems. For example, many texts state the Theorem 8.5:
following theorem and follow with a proof. a. lim(f′(N)/g′(N)) = C >0 ⇒ [f] = [g]
If f is O(g) and g is O(h) then f is O(h). b. lim(f′(N)/g′(N)) = 0 ⇒ [f] ≤ [g].

Since ≤ forms a partial ordering on the equivalence classes we Theorem 8.5 follows immediately from Theorem 8.4 and
have the following more natural statement: l'Hôspital's rule [3], [8].
If [f] ≤ [g] and [g] ≤ [h] then [f] ≤ [h] by the transitive property of
the partial ordering, ≤.

173
Question 8.1:
Should we use limit-based definitions for growth function
comparisons?
Example 8.3
A crude version of Merge Sort (CMS) requires the number of
input values to be a power of two. When the number of input
values is not a power of 2 the algorithm creates enough dummy
input values to raise the number up to the next power of two. A
better version of Merge Sort (BMS) is able to properly handle any
number of input values.
The number of comparisons required for BMS is
f(N) = C1*N*lg(N) and for CMS it is g(N) = C2*2lg(N)*lg(2lg(N))
where   represents the ceiling function [3].
Using Definition 6.1 one can show that [f] = [g] and most would
agree that f and g should be equivalent since they differ by a
factor of between two and four. However, due to the nature of the
step function g(N), lim(f(N)/g(N)) does not exist and thus f and g
would be considered incomparable if limits were used to define
equality.
9 A STILL BETTER METHOD?
All of the measures considered thus far for comparing functions
measure function values at a particular point, f(i)/g(i). One might
argue that we would be better served by a measure that considers
the relative values at all points. The ratio of the average number of
operations for function f as compared to g is lim∑(f(i)/g(i))/N
(1<= i <= N, N ⇒ ∞).
We may define [f]=[g] iff lim∑(f(i)/g(i))/N = C > 0. And likewise
define [f]<[g] iff lim∑(f(i)/g(i))/N = 0. Although computing
∑(f(i)/g(i))/N is often difficult, the literature [2] gives many
methods to apply, including approximations involving ∫f(i)/g(i). A
thorough investigation of the above definitions might be suitable
as an advanced undergraduate research topic.
174
10 CONCLUSIONS
We have utilized equivalence classes and partial ordering to create
a means of comparing function growth in order of complexity
analysis. The method is similar to Big O but offers several
pedagogical advantages including
a. easier to teach and learn
b. better extensibility
c. modern mathematical notation.
11 REFERENCES
[1] ACM Computing Curricula 2001, Chapter 7, 7.4
http://www.computer.org/education/cc2001/final/chapter07.htm
and http://www.computer.org/education/cc2001/final/cs115.htm
[2] Cormen, Leiserson and Rivest Algorithms, The MIT Press,
McGraw-Hill Book Co., 1990, 23-34, 41-52, 804
[3] Gersting, J. L. Structures for Computer Science, W.H.
Freeman and Co., Fourth Edition, 1999, 239, 242, 245,
282, 309
[4] Goldberg, R. R. Methods of Real Analysis, Blaisdell
Publishing Co., 1964, 26.
[5] Knuth, Donald E. The Art of Computer Programming,
Volume 1/Fundamental Algorithms, second edition,
Addison-Wesley Publishing Co., 1973, 104
[6] Manber, Udi Introduction to Algorithms, Addisin-Wesley
Publishing Co., 1989, 41.
[7] Smith, Jeffrey D. Design and Analysis of Algorithms,
PWS-Kent Publishing Co., 1989, 12.
[8] Thomas, George B. Jr. Calculus and Analytic Geometry,
Addison-Wesley Series in Mathematics, Third Edition,
June 1962, 41, 815
Ftp Access: Obtain an on-line copy of this paper via anonymous
ftp to cs-netlab-01.lynchburg.edu. Move to the Algorithms
subdirectory and retrieve the file Growth.doc.