Dabu Claudia-Maria IS anul I

Introduction to Complexity Theory

Space Complexity

Time and space are two of the most important considerations when we seek practical solutions to computational problems. Space complexity shares many of the same features of time complexity. Therefore, space complexity serves as a further way of classifying problems according to their computational difculty. We will continue to use the TM as our model, but now we look at the tape space it consumes during its computation. Space Complexity of an algorithm is total space taken by the algorithm with respect to the input size. Space complexity includes both Auxiliary space and space used by input. Auxiliary Space is the extra space or temporary space used by an algorithm. Complexity classes correspond to bounds on resources. One such resource is space: the number of tape cells a TM uses when solving a problem. Defining complexity classes The measure DSPACE is used to define complexity classes, sets of all of the decision problems which can be solved using a certain amount of memory space. For each functionf(n), there is a complexity class SPACE(f(n)), the set of decision problems which can be solved by a deterministic Turing machine using space O(f(n)). There is no restriction on the amount of computation time which can be used, though there may be restrictions on some other complexity measures (like alternation). Several important complexity classes are defined in terms of DSPACE. These include:

REG = DSPACE(O(1)), where REG is the class of regular languages. In fact, REG = DSPACE(o(log log n)) (that is, (log log n) space is required to recognize any nonregular language). L = DSPACE(O(log n))

Dabu Claudia-Maria IS anul I

PSPACE =

EXPSPACE =

DSPACE is traditionally measured on a deterministic Turing machine. Several important space complexity classes are sublinear, that is, smaller than the size of the input. Thus, "charging" the algorithm for the size of the input, or for the size of the output, would not truly capture the memory space used. This is solved by defining the multi-string Turing machine with input and output, which is a standard multi-tape Turing machine, except that the input tape may never be written-to, and the output tape may never be read from. This allows smaller space classes, such as L (logarithmic space), to be defined in terms of the amount of space used by all of the work tapes (excluding the special input and output tapes). Since many symbols might be packed into one by taking a suitable power of the alphabet, for all c 1 and f such that f(n) 1, the class of languages recognizable in c f(n) space is the same as the class of languages recognizable in f(n) space. This justifies usage of big O notation in the definition.

A complexity class is a set of problems of related complexity. Simpler complexity classes are defined by the following factors:

The type of computational problem: The most commonly used problems are decision problems. However, complexity classes can be defined based on function problems,counting problems, optimization problems, promise problems, etc. The model of computation: The most common model of computation is the deterministic Turing machine, but many complexity classes are based on nondeterministic Turing machines, Boolean circuits, quantum Turing machines, monotone circuits, etc. The resource (or resources) that are being bounded and the bounds: These two properties are usually stated together, such as "polynomial time", "logarithmic space", "constant depth", etc.

Of course, some complexity classes have complex definitions that do not fit into this framework. Thus, a typical complexity class has a definition like the following: The set of decision problems solvable by a deterministic Turing machine within time f(n). (This complexity class is known as DTIME(f(n)).)
2

Dabu Claudia-Maria IS anul I

But bounding the computation time above by some concrete function f(n) often yields complexity classes that depend on the chosen machine model. For instance, the language {xx| x is any binary string} can be solved in linear time on a multi-tape Turing machine, but necessarily requires quadratic time in the model of single-tape Turing machines. If we allow polynomial variations in running time, Cobham-Edmonds thesis states that "the time complexities in any two reasonable and general models of computation are polynomially related" (Goldreich 2008, Chapter 1.2). This forms the basis for the complexity class P, which is the set of decision problems solvable by a deterministic Turing machine within polynomial time. The corresponding set of function problems is FP.
Many important complexity classes can be defined by bounding the time or space used by the algorithm. Some important complexity classes of decision problems defined in this manner are the following: Complexity class Model of computation Resource constraint

DTIME(f(n))

Deterministic Turing machine

Time f(n)

Deterministic Turing machine

Time poly(n)

EXPTIME

Deterministic Turing machine

Time 2poly(n)

NTIME(f(n))

Non-deterministic Turing machine Time f(n)

NP

Non-deterministic Turing machine Time poly(n)

NEXPTIME

Non-deterministic Turing machine Time 2poly(n)

DSPACE(f(n))

Deterministic Turing machine

Space f(n)

Deterministic Turing machine

Space O(log n)

Dabu Claudia-Maria IS anul I

PSPACE

Deterministic Turing machine

Space poly(n)

EXPSPACE

Deterministic Turing machine

Space 2poly(n)

NSPACE(f(n))

Non-deterministic Turing machine Space f(n)

NL

Non-deterministic Turing machine Space O(log n)

NPSPACE

Non-deterministic Turing machine Space poly(n)

NEXPSPACE

Non-deterministic Turing machine Space 2poly(n)

It turns out that PSPACE = NPSPACE and EXPSPACE = NEXPSPACE by Savitch's theorem. Other important complexity classes include BPP, ZPP and RP, which are defined using probabilistic Turing machines; AC and NC, which are defined using Boolean circuits and BQP and QMA, which are defined using quantum Turing machines. #P is an important complexity class of counting problems (not decision problems). Classes like IP and AM are defined using Interactive proof systems. ALL is the class of all decision problems. DSPACE is traditionally measured on a deterministic Turing machine. Several important space complexity classes are sublinear, that is, smaller than the size of the input. Thus, "charging" the algorithm for the size of the input, or for the size of the output, would not truly capture the memory space used. This is solved by defining the multi-string Turing machine with input and output, which is a standard multi-tape Turing machine, except that the input tape may never be written-to, and the output tape may never be read from. This allows smaller space classes, such as L (logarithmic space), to be defined in terms of the amount of space used by all of the work tapes (excluding the special input and output tapes). Since many symbols might be packed into one by taking a suitable power of the alphabet, for all c 1 and f such that f(n) 1, the class of languages recognizable in c f(n) space is the same as the class of languages recognizable in f(n) space. This justifies usage of big O notation in the definition.
4

Dabu Claudia-Maria IS anul I

Denition: Let M be a deterministic TM that halts on all inputs. We dene the space complexity of M to be the function f : NN, where f(n) is the maximum number of tape cells that M scans on any input of length n. If the space complexity of M is f(n), then we also say that M runs in space f(n). If M is a nondeterministic TM where in all branches halt on all inputs we dene its space complexity f(n) to be the maximum number of tape cells that M scans on any branch of its computation on any input of length n. Denition: Let f : NR+ be a function. The space complexity classes,SP ACE(f(n)) and NSP ACE(f(n)), are dened as follows, SPACE(f(n)) ={L|Lis decided by anO(f(n))space TM} NSPACE(f(n)) ={L|L is decided by an O(f(n))space NTM}

Low Space Classes Definitions (logarithmic space classes): L = SPACE(logn) NL = NSPACE(logn) Problem! How can a TM use only logn space if the input itself takes n cells?!

3Tape Machines

Dabu Claudia-Maria IS anul I

The model of computation we will use is a 3-tape Turing Machine. We use this model because it is easier to deal with it. We remind that any multi-tape TM can be simulated by an ordinary TM with a loss of eficiency that is only polynomial. For the reminder of this lecture notes, Turing Machine" will refer to a 3-tape Turing Machine. The 3 tapes are: 1. input tape. Read-only 2. output tape. Write-only. Usually considered unidirectional: this assumption is not essen-tial but useful. For decision problems, as considered below, one can omit the output-tape altogether and have the decision in the machine's state. 3. work tape. Read and write. Space complexity is measured by the bounds on the machine's position on this tape. Writing is not allowed on the input tape: this way space is measured only on the worktape. If we allowed writing on the input tape then the length of the input itself should be taken into account when measuring space. Thus we could only measure space complexities which are at least linear. In order to consider also sublinear space bounds we restrict the input tape to be read-only. Define W M (x) to be the index of the rightmost cell on the worktape scanned by M on input x. Define S M (n) = max|x|=n W M(x). For any language L define XL (x) so that if x XL (x) = 1 otherwise XL (x) = 0.
6

L then

Dabu Claudia-Maria IS anul I

Definition 4.1 (Dspace):

We may multiply s( ) by log2 |TM | where TM is the alphabet used by M. Otherwise, we could always linearly compress the number of space cells using a bigger alphabet. We may also add log2(|x|) to s( ), where x is the input. (However, this convention disallow treatment of sublogarithmic space,and therefore will not be done when discussing such space bounds.) This is done in order to have a correspondence to the number of configurations. Definition: A configuration of M is an instantaneous representation of the computation carried on by M on a given input x. Therefore if |x|= n a configuration gives information about the following: state of M (O(1) bits) contents of the work tape (s(n) bits) head position in the input tape (log (n) bits) head position in the work tape (log (s(n)) bits) Sub-Logarithmic Space Complexity Working with sublogarithmic space is not so useful. One may b e tempted to think that whatever can be done in o(log (n)) space can also be done in constant space. Formally this would mean Dspace(o(log (n)))
.

and since obviously that in fact

we may also (incorrectly) argue

This intuition comes from the following imprecise observation: if space is not constant, machine M must determine how much space to use. Determining how much space to use seems to require the machine counting up to at least |x|= n which needs O(log (n)) space. Therefore any M that uses less than O(log (n)) cells, is forced to use constant space. It turns out that this intuition is wrong and the reason is that the language itself can help in deciding how much space to use.

Dabu Claudia-Maria IS anul I

Going further on, we can consider Dspace (o(log log (n)) and Dspace (O(1)). We will show that these two complexity classes are equivalent. The kind of argument used to prove their equivalence extends the one used to prove the following simpler fact. Theorem . For any s(n) : s(n) > log(n)

Proof: Fix an input x: | x |= n and a deterministic machine M that accepts x in space s(n). Let be C the number of possible configurations of M on input x. Then an upper bound for C is:

where Q M is the set of states of M, n is the number of possible locations of the head on the input tape, s(n) is the number of possible locations of the head on the worktape and 2o(s) is the number of possible contents in the worktape { the exponent is o(s) because the alphabet is not necessarily binary. We can write s(n) 2o(s(n)) = 2o(s(n)) and since s is at least logarithmic, n<2o(s(n)) . Therefore C < 2o(s(n)) M cannot run on input x for a time t(n) > 2 s(n) . Otherwise, M will go through the same configuration at least twice, entering an infinite loop and never stop. Then necessarily M has to run in time t(n)*2o(s) . Theorem Dspace (o(log2log2(n)) = Dspace (O(1)) Proof: Consider a s( ) -bounded machine M on the alphabet { 0;1 }. Claim: given input x : |x| = n such that M accepts x, then M can be on every cell on the input tape at most k = 2 s(n) *s(n)*|Q M |= O( 2s(n) ) times. The reason being that if M were to be on the cell more than k times then it would be in the same configuration twice, and thus never terminate. We dene a semi-configuration as a configuration with the position on the input tape replaced by the symbol at the current input tape position. For every location i on the input tape, we consider all possible semi-configurations of M when passing location i. If the sequence of such configurations is C i = C1 i,, Cr i then by the above claim its length is bounded: r<O (2 s(n) ). The number of possible diferent sequences of semi-configurations of M, associated with any position on the input tape, is bounded by

Dabu Claudia-Maria IS anul I

Since s(n) = o(log2log2n) then that . We then show

and therefore there exists that .

such Thus

Dspace (O(1)) proving the theorem. The number of sequences of semi-configurations at any position in the input tape is < n1/3. So labelling n 1 positions on the input tape by at most n1 /3 sequences means that there must be at least three positions with the same sequence of semi-configurations. Say Where each of the positions with symbol a has the same sequence of semi-configurations attached to it. Hierarchy theorems The space function not in space hierarchy theorem shows that, for every space-constructible but

, there exists some language L which is decidable in space .

For the complexity classes defined in this way, it is desirable to prove that relaxing the requirements on (say) computation time indeed defines a bigger set of problems. In particular, although DTIME(n) is contained in DTIME(n2), it would be interesting to know if the inclusion is strict. For time and space requirements, the answer to such questions is given by the time and space hierarchy theorems respectively. They are called hierarchy theorems because they induce a proper hierarchy on the classes defined by constraining the respective resources. Thus there are pairs of complexity classes such that one is properly included in the other. Having deduced such proper set inclusions, we can proceed to make quantitative statements about how much more additional time or space is needed in order to increase the number of problems that can be solved. More precisely, the time hierarchy theorem states that
.

The space hierarchy theorem states that

.

The time and space hierarchy theorems form the basis for most separation results of complexity classes. For instance, the time hierarchy theorem tells us that P is strictly contained in EXPTIME, and the space hierarchy theorem tells us that L is strictly contained in PSPACE.