Data Structures And Algorithms

Lecture 1 Introduction Week of Feb 18-22, 2013

What is an algorithm?
An algorithm is a sequence of unambiguous instructions for solving a problem, that is, for obtaining a required output for any legitimate input in a finite amount of time. Each step of an algorithm should be non-ambiguous Range of inputs for which an algorithm works should be specified Same algorithm can be represented in several different ways Several algorithms for solving the same problem could exist Algorithms for the same problem can be based on very different ideas and can solve the problem with dramatically different speeds

A simple example: finding the greatest common divisor of two nonnegative integers m and n
Problem definition: find the number that divides both m and n. Denote this number as gcd(m,n). We will discuss three different algorithms to illustrate the discussion on algorithms on the previous page. Algorithm 1. Will be based on Euclids Algorithm. Repeatedly apply the following until m mod n gives 0: gcd(m,n) = gcd (n, m mod n) where m mod n is the remainder of the division of m by n. For instance, gcd(80, 30) can be computed as gcd(80,30)=gcd(30,20)=gcd(20,10)=gcd(10,0)=10

Euclids Algorithm continued

Here is the description of the algorithm: Step 1: If n=0, return the value of m as the answer and stop; otherwise proceed to step 2 Step 2. Divide m by n and assign the value of the remainder to r. Step 3. Assign the value of n to m and the value of r to n. Go to step 1. Or by using pseudocode (a language which is a a mixture of ordinary language and a programming language) while n != 0 do rm mod n mn nr return m Note that this algorithm eventually comes to a stop: second number of the pair gets smaller and smaller with each iteration and it eventually becomes zero terminating the algorithm.

Greatest common divisor algorithms continued

Algorithm 2. Start with the smaller of m and n, decrement it by 1 until the number obtained divides both m and n. Step 1. Assign the value of min {m, n} to t Step 2. Divide m by t. If the remainder of this division is 0, go to Step 3; otherwise, go to step 4. Step 3. Divide n by t. If the remainder of this division is 0, return the value of t as the answer and stop; otherwise, proceed to step 4. Step 4. Decrease the value of t by 1. Go to step 2. Note that this algorithm fails when one of the input values is 0. This example demonstrates why it is important to understand the range of input values on which a given algorithm could work.

Greatest common divisor algorithms continued

Algorithm 3. Compute the product of the common factors of m and n as the greatest common divisor. Step 1. Find the prime factors of m Step 2. Find the prime factors of n Step 3. Identify all the common factors in the two primes expansions found in steps 1 and 2. Step 4. Compute the product of all the common factors and return it as the greatest common divisor. Note that this algorithm, as in the form given above, is not really legitimate. The reason is the fact that steps to find the primes are defined ambiguously: they require a list of prime numbers and it is not explained how to get such a list. Also, note that this algorithm is much more complex and slower than Euclids Algorithm.

Fundamentals of Algorithmic Problem Solving

Algorithms are procedural solutions to problems. Understand the problem: Read the problem description carefully, do a few examples by hand, and think about special cases. Ascertain the capabilities of a computational device: For practical problems, be aware of the speed and memory of the computational device available to you. Choosing between exact and approximate problem solving: Decide whether you need an exact or an approximate solution. Why opt for an approximate solution? Exact solutions to some problems may not exist or may be too expensive. For the latter consider a system with a large number of interacting particles, say N. Exact interaction energy calculation requires NXN operations. However, one may calculate an approximate value by considering nearby particles in an exact sum and the distant particles in terms of a multi-pole expansion in less than NXN operations. (Will talk more about this example in the future.) Decide on appropriate data structures: Some algorithms may rely on a special way of representing data. We will have a quick tour of data structures later. Methods of specifying an algorithm: Natural language may be too ambiguous to specify the steps of an algorithm. Preferred way is pseudocode, a mixture of a natural language and a programming language. No single form of pseudocode is specified, but they all resemble one another.

Fundamentals of Algorithmic Problem Solving continued

Proving an algorithms correctness: Need to prove the correctness of an algorithm sketched for a problem. For instance, Euclids algorithm for greatest common divisor is known to be correct because of the equality gcd(m,n) = gcd(n, m mod n). Analyzing an algorithm: Time efficiency, that is, how fast the algorithm runs, and space efficiency, that is, how much memory the algorithm requires need to be analyzed. Also, simplicity of an algorithm is also important: simpler algorithms are easier to understand and implement, and less likely to have bugs in its implementation. Coding an algorithm: Validity of the implementation of an algorithm is established by rigorous testing. In tests consider a representative set of variations of input to an algorithm with boundary input values with special emphasis.

A Quick Tour of Important Problem Types

There are some problems that have attracted considerable interest due to their practical importance or their specific characteristics:
Sorting Searching String Processing Graph Problems Combinatorial Problems Geometric problems Numerical problems

Sorting Rearranging the items of a given list in ascending or descending order. For instance, sorting the list of students by their last name or by their school id number. Sorting makes many questions about lists of items easier; for instance, searching for a specific item in a given list. There are plenty of sorting algorithms. Some of them achieve efficiencies of n log(n) comparisons for sorting n elements.

A Quick Tour of Important Problem Types

Searching Finding a given value in a given set. There are plenty of search algorithms from the straightforward but inefficient sequential search to efficient binary search which has its limitations. Searching may need to be considered with two other operations on a data set: addition and deletion. Data structures and algorithms should be chosen to strike a balance in requirements of each operation. (Think of Google, adding and deleting millions of pages, and searching among them.) String Processing A string is a sequence of characters which terminates with a special character. Searching for a word in a given word in a text is an area of interest. Graph problems A graph is a collection of points called vertices, some of which are connected by line segments called edges. Graphs can be used to model a wide variety of real-life applications like transportation and communication networks, project scheduling, and games. Basic graph algorithms include graph traversal algorithms (how can one visit all the points in a network?) and shortest path algorithms (what is the best route between two cities?).

A Quick Tour of Important Problem Types

Combinatorial Problems Problems that ask to find a combinatorial object such as a permutation, a combination, or a subset, that satisfies certain constraints and has some desired property, for instance, maximizes a value or minimizes a cost. Geometric Problems These are the problems that deal with geometric objects such as points, lines, and polygons. For instance, the closest-pair problem: given n points in a plane, find the closest pair among them. The convex hull problem: finding the smallest convex polygon that would include all the points of a given set. Numerical Problems Problems that involve solving equations and systems of equations, computing definite integrals, evaluating functions, and so on. Developing efficient algorithms to study the dynamics of large numbers of interacting particles on a computer is an active research area.