Technion IIT Department of Computer Science Fall 20089

Propositional Proof Complexity

Eli Ben-Sasson

Course 236604:

Table of Contents
1.1 1.2 1.3 1.4 1.5 1.6

Lecture 1: Introduction to Propositional Proof Complexity

What is a proof ? [Cook-Reckhow 79] So how hard is it to check? . . . . . . . . . . . . . . . . . . . . . . 11 11 12 12 12 13 . . . . . . . . . . . . . . . . . . . . . . . . . . . . What is the "Propositional" in Propositional Proof Complexity . . . . . . . . SAT Solvers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Sequential Proof Systems

Summary - The Interesting Questions Up Ahead . . . . . . . . . . . . . . . .

Lecture 2: Sequential Proof Systems

2.1 2.2 2.3 2.4 2.5 Denitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 22 22 24 Proof System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Resolution Proof System [Blake-1937] . . . . . . . . . . . . . . . . . . . . . . Resolution Proof System with Weakening . . . . . . . . . . . . . . . . . . . . The DPLL Algorithm [Davis-Putnam-Logemann-Loveland, 1962] . . . . . . . . . . . . . . . . . . . . 24

Lecture 3: More on Sequential proof systems

3.1 3.2 Cutting Planes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Survey of lower bounds on resolution proof length . . . . . . . . . . . . . . . 31 32

Lecture 4: Width Method

4.1 4.2 The Size-Width Relation Theorem . . . . . . . . . . . . . . . . . . . . . . . . Proof of Size-Width Relation (Theorem 4.5) . . . . . . . . . . . . . . . . . . 42 43

Lecture 5: The Width Method, part II Lecture 6: Width of Explicit Encodings Lecture 7: A Combinatorial Characterization of Width

Lecture 8: The Feasible Interpolation Method

8.1 8.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 81 82 82 83 Lower bounds on proof length using the Feasible Interpolation method . . . . 8.2.1 8.2.2 8.3 Results about feasible interpolation . . . . . . . . . . . . . . . . . . . . . . Lower bounds on proof length via the feasible interpolation method

Proof of feasible interpolation property for Resolution

. . . . . . . . . . . . .

Lecture 9: Lower bounds for Res[k] refutation length

9.1 9.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 91 91 91 92 92

Res[k]
9.2.1 9.2.2 9.2.3

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

History and Motivation Formal Denition Semantic Denition

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9.3

The General Idea

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Lecture 10: Random 3-CN F and Res[k] refutations

10.0.1 10.0.2 10.0.3 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 101 102 The proof plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Switching lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Lecture 11: Random 3-CN F and Res[k] refutations  Part II

11.1 From 3-CNF to a system of linear equations [M. Alekhnovich] . . . . . . . . . 111

Lecture 12: Random 3-CN F and Res[k] refutations  Part III Lecture 13: TBD
13.1 13.2 13.3 13.4

Res[k]
Frege

- Conclusion

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

131 131 131 132

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Resolution

Open Questions

Homework assignments
Assignment 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HW1 Assignment 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HW3 Assignment 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HW4

References

Lecture 1

Introduction to Propositional Proof Complexity

November 11th, 2008
Lecturer:

Eli Ben-Sasson

Scribe:

Gadi Aleksandrowicz and Daniel Genkin

1.1 What is a proof? [Cook-Reckhow 79]

Given a language

over a nite alphabet

claims  general statements of the over

we can think of the language as a collection of form x L, and of a proof as a sequence of claims

A good proof is one that assures us that some claim is true; this means that by

reading the proof we can be convinced that a claim is true, and that false claims cannot be proved in a similar manner. This is treated formally by the notion of a proof checking algorithm. We turn now to formal denitions:

Denition 1.1.
inputs:

proof system for a language L is an algorithm P which receives two x (a claim) and (a proof string), and outputs Accept or Reject, such that
A

the following hold: 1. 2. 3.

completeness: x L soundness: x L / eciency:

: P(x, )= Accept

: P(x, )= Reject

The running time of P on

(x, )

is "ecient" (usually polynomial)

1.2 So how hard is it to check?

We shall now assume that accepts a truth value assignment

L NP, for for a

instance SAT. There exists a proof system that SAT sentence

and accepts

i C()

=1

and

|| = ||

|C|O(1) . The hardness of nding a proof is not known and is equivalent to the

P=NP problem. We will now ask how hard is it to

check

a given proof: let's look at the following SAT

{C| :
a string

C()

= 0}

and similar claims like

C()

= 1.

It's much less obvious how to

prove such claims. Of course, we have a trivial proof system for SAT: for an input C and

0...0, 0...1, ..., 1...1

of a length

2n n

of all the possible assignments. The problem

is that this kind of proof is long, cumbersome and not elegant, and its hard (exponentially hard) to check. Let us assume that we have a circuit C with a given structure C

(C )

where C is a sub-

circuit. It is easy to prove that this C is unsatisable regardless of C 's structure, therefore sometimes one

can

prove unsatisability in a much more "elegant" way then going over all

of the assignments, and therefore one can check this proof

much

more easily

11

1.3 What is the "Propositional" in Propositional Proof Complexity

In this section we will try to explain A proposition is something that either

true

or

false.

Example:

"Today is a rainy day or today is not a rainy day" is a claim that always holds

(a tautology). We can think of this claim as

x(x) (x).

This means that some things are always correct

in any context. In general, a tautology is a boolean circuit who's output is always matter what the input is.

true

no

Now we turn to a result which connects the study of proof systems to that of the relation between

NP

and

coNP: NP = coNP if and only if there exists a super proof system has the following property: For all x L

Theorem 1.2 (Cook and Reckhow [1979]).

proof system for SAT, where a there exists such that 1. || |x|O(1) 2. P (x, ) = Accept
super

NP = coNP then since SAT coNP we have SAT NP. Therefore, by the denition of NP we have a polynomial proof system for SAT, i.e. an algorithm P such that O(1) such that P(x, ) = Accept- hence a for all x SAT there exists a proof , || |x| super proof system for SAT. On the other hand, assume a super proof system for SAT exists, then SAT NP - but SAT is known to be coNP-complete, and hence coNP NP. It is easy to see that this implies coNP = NP, since if L NP we have by denition L coNP and hence L NP, so again by denition we have L = L coNP.
If

Proof.

1.4 SAT Solvers

Another motivation for studying propositional proof complexity comes from the eld of SAT solvers. This is a complete eld of research deals with algorithms for eciently proving (or disproving) satisability, even though in general the problem is

NP-complete.

The

algorithms are not ecient in time on any given SAT clause, but they work well in practice in most cases. This is interesting, or instance, in formal verication one can encounter the following problem: Given a function computes

f,

meaning that

f and a circuit C , we want to verify x1 x2 ...xn : f (x1 , ...xn ) C(x1 , ...xn ). Yet again

that a

indeed

claim.

1.5 Sequential Proof Systems

Let us begin by stating formally what a sequential proof system is. 12

Denition 1.3.
words

sequential proof system

and denote it by

Consists of a set of possible claim, which are

,...,x , and derivation rules of the form x1 ,x2y n . We say that the axioms

x1 , . . . , x n

prove the claim

z,

{x1 , x2 , . . . , xn }

if there is a nite sequence

y1 , y2 , . . . , yk
1.

such that the following holds:

z = yk
for some j ) or yi is derived from previous yj1 ,...,yjk claims by one of the derivation rules (i.e. such that jt < i). yi either is an axiom (i.e.

2. For all

yi ,

yi

yi = x j

Let us now examine some examples of sequential proof systems. Given C

satised C

= (x y) (y z) x z one can look at this C as a set of axioms = {(x y), (y z), x, z} and we shall prove sequentiality that C x y, x y 0
which is unsatisable both yield a new constraint:

that need to be is unsatisable:

1. the constraints: 2. 3.

z y

and and

yz y

yields

both yield the empty clause

this kind of proof method is called

Resolution. Cutting Planes.

In this proof system the

Now let us consider a dierent proof system axioms are given as inequalities: 1. 2. 3. 4.

x + (1 y) 1 y + (1 z) 1 (1 x) 1 z1

(This is equivalent to

x y)

By summing up all the equations we get

34

- a contradiction.

This proof system is better than Resolution since proofs are shorter, and it has a geometric taste.

1.6 Summary - The Interesting Questions Up Ahead

We will try to deal with questions like Is it possible to prove that

P = NP

(even assuming

it is correct)?. We will also try to understand the connection between computations and proofs, and whether something that is easy to prove is also easy to compute and vice versa. Sometimes the answer is positive. Some important questions in the eld are 1. Is there a

super

proof system? (whether

NP = coNP)

- we can't say much about

this question.

13

2. Is there an optimal proof system? i.e. as good as any other system (similar to the optimal hardness of problems which are answer this question as well. 3. Specic systems, like Resolution, Cutting Planes, Frege - are they super? We know that Resolution and Cutting Planes are not super, but the question is still open for Frege. 4. How to nd a proof - sometimes we can nd a proof eciently.

NP-complete)

- we don't know how to

14

Lecture 2

Sequential Proof Systems

November 18th, 2008
Lecturer:

Eli Ben-Sasson

Scribe:

Carmi Grushko and Hanna Fadida

The only good system, is the sound system . In this lecture we will introduce one of the simplest proof systems  Resolution  and a SAT-solving algorithm related to it, DPLL. Like other proof systems discussed in this course, resolution allows to prove that a formula is unsatisable. To dene resolution we need a few preliminary denitions.

2.1 Denitions
Denition 2.1. Denition 2.2. Denition 2.3.
[literal] A [clause] A

literal clause

over

is either

or

x.

is a disjunction of literals.

[CNF formula] A

CNF formula

is a set of clauses.

We consider a CNF formula as a set of constraints; we need to nd an assignment satisfying at least one literal in each constraint. In a

sequential proof system,

proof

is composed of a sequence (list) of statements, each

following from previous statements or an axiom. stating which new statements can be

Derivation rules

govern this process by

derived

from existing ones. To prove a proposition,

a specic statement has to be sequentially derived: in this course, since we try to prove unsatisability, we try to derive the false statement.

Denition 2.4. Denition 2.5. Denition 2.6. Denition 2.7. Denition 2.8.
We also denote

[Sequential Proof System] A

sequential proof system

is denoted by

is a proof system

dened by a set of sound derivation rules [Derivation rule] A [Axiom] An [Proof ] A

derivation rule

Ai ,...,Ak . B

axiom

is a derivation rule in which

B.

proof

in such a system is a sequence of lines, in which a line is

obtained from previous lines by using one of the derivation rules. A derivation rule must be veriable in polynomial time. [length] The

length

of a proof

number of lines of derivation, and is denoted

L ().

in a sequential proof system

is the

L (C

0)

as the

minimal

possible length of a proof that

0.

Some rastafarian
21

Note.

A CNF formula can be interpreted in two ways -

1. as a sequence of characters (syntax), and 2. as a boolean function (semantics)

2.2 Proof System

Denition 2.9.
assignment [Implication] We say that

A implies B , A derives B ,

and denote by

A |= B

if for each

if

is satised by

then

must also be satised by it. and denote by

Denition 2.10.
be obtained from

[Derivation] We say that

by a sequence of derivation rules from Proof System

A P.

if

can

In this course, we will only deal with refutation proof systems. A refutation proof system is equivalent to a tautology proof system; recall that a proving in a refutation proof system means obtaining a contradiction, while in a tautology proof system, it is by obtaining a tautology (not necessarily without axioms and/or assumptions). If we can prove a CNF formula is unsatisable, it means its negation is a tautology. This way, we can show satisability of formulas, by showing their negation is unsatisable.

Denition 2.11. Denition 2.12.

[Soundness] A proof system is

sound

if

A,

if

then

A |= 0

(this

can also be written as

A : (A

P)

(A |=)).

[Completeness] A proof system is

complete

if

A,

if

A |= 0

then

0.

Note. A proof system for tautologies is dened symmetrically: and complete i A : (|= A) ( P A)

such a proof system is

sound

2.3 Resolution Proof System [Blake-1937]

Resolution
is one of the simplest propositional proof systems. It was rst introduced in Blake's Ph.D-thesis in 1937 (Blake [1997]). Beside its mathematical simplicity and elegance, Resolution is a very interesting proof system because it generalizes the Davis-Putman algorithm (also known as DPLL, to be discussed shortly in Section 2.5) and several other known proof-search procedures. provers existing today. This proof system has essentially one derivation rule, and all lines in a poof are clauses. Moreover, Resolution is the base for most automatic theorem

Resolution Rule:
clauses and

Derive

CD

from

{C x, D x}

Cx D x CD R, where

C, D

are any

is any variable.

Lemma 2.13.
Proof.

The resolution proof system is both

sound and complete.

We show that the resolution rule is locally sound and complete, and therefore show

that the system is both sound and complete.

22

1.

Soundness : , C, D, x : [C x] () = 1 and [D x] () = 1 then [C D] () = 1. Assume by contradiction that [C D] () = 0. This means that both C () = 0 and D () = 0. Therefore, x must be satised for the rst part to hold, whereas x must
be satised for the second part to hold, contradiction.

2.

Completeness: C : C |= 0 C R 0. If C |= 0 does not hold, we're done. If C |= 0, there pure, by which we mean that both x and x appear
C

must be a variable which is not in the formula. Let

be such a

variable. We prove by induction on the number of variables

in a formula

C,

that if

is a contradiction, then there is a proof of this in the resolution proof system .

Basis: Step:

n = 1:

All formulas are trivially equivalent to the formula

x x,

therefore by

resolution we get

0. n1

Assume the resolution rule is complete for all formulas with up to

variables, and we show that it holds for formulas with which is not pure. We divide the formula

n variables.

Let

x1

be a variable

(which is a contradiction) into three sets:

Cx1 (clauses that contain x1 ), Cx1 (clauses that contain x1 ) and C (the rest) such C = Cx1 Cx1 C . We note that |Cx1 | , |Cx1 | > 0. Dene A x B to be all possible resolutions between a A and b B such that variable x is removed. Denote C = (Cx1 Cx1 ) C . Note that because x1 does not appear in C we conclude C contains at most n 1 variables. We need to show that C |= 0. Assume by contradiction that an (n 1)-variable assignment exists, s.t. C () = 1. WLOG we can write Cx1 as {[Cx1 ]i x1 }. Denote Cx1 = {[Cx1 ]i } (meaning, we took out the pure variable x1 ) and similarly Cx1 = {[Cx1 ]i }. If both Cx1 () = 0 and Cx1 () = 0, there exists A Cx1 and B Cx1 such that A |= 0 and B |= 0. In this case, the clause A B is derived and appears in (Cx1 Cx1 ), and is not satised, hence the entire C is not satised. Therefore either Cx1 () = 1 or Cx1 () = 1. Assume wlog Cx1 () = 1. We extend into by dening (x1 ) = 0. We get WLOG that:
that (a)

|= Cx1 because Cx1 () = 1,

i. ii.

and and nally

|= Cx1 |= C

because

(x1 ) = 0,

because

C () = 1 C |= 0.
Since by denition,

Therefore,

|= C ,

contradiction. We get

By the inductive hypothesis we have from

using resolution,

C R 0.

0.

can be derived

23

We end this section by noting that in certain cases there do indeed exist short resolution proofs.

Claim 14.
Proof.

For each 2-CNF formula with n variables, it holds that L (C

4 n2 .

0) 4 n2 .

Since every derivation yields a clause with at most two literals, the total number of

clauses in a proof is at most the number of combinations, which is

Note. 3-CNF formulas can increase L exponentially. 2n . Also, there exists a 3-CNF formula C whose L (C

Each 3-CNF formula has

L (C

0)

0) = 2n .

Considering a specic assignment, a clause which is not satised by it can easily be found. We construct a formula whose proof is exponentially long by combining clauses derived from each possible assignment. This way, to prove unsatisability, one must consider all clauses.

2n

2.4 Resolution Proof System with Weakening

In addition to the resolution rule, we will at times use the

weakening rule:

Weakening Rule:

Derive

C l

from

C Cl

W), for any clauses C, l.

Intuitively, this rule should not provide us with and additional power. This can be proven formally  for an unsatisable formula with width

if there is a proof using resolution and weakening

and length l, then there is also a proof using only resolution that used width

and length at most

l.

However, allowing use of the weakening rule will, at times, make

our arguments simpler.

2.5 The DPLL Algorithm [Davis-Putnam-Logemann-Loveland, 1962]

A SAT solver is an algorithm which, when given a CNF formula, nds a satisfying assignment if there is one and outputs unsatisable if there are none. The DPLL algorithm, due to Davis and Putnam [1960]; Davis et al. [1962], nds a satisfying assignment by repeatedly simplifying a formula using two heuristics (to be detailed shortly), choosing a literal, assigning a value to it and then recursively trying to nd a satisfying assignment for the reduced formula. The two heuristics are

Pure literal elimination  as dened before, a pure literal is a variable that occurs in the formula with only one polarity (i.e., either only positively, or only negatively). Pure literals can always be assigned in a way that makes all clauses containing them true. Thus, these clauses do not constrain the search anymore and can be deleted.

24

x
0 0 0

z
0 1

z
1

y z

xz

x,z

Figure 1: DPLL run

Unit propagation  If a clause is a unit clause, i.e. it contains only a single unassigned literal, this clause can only be satised by assigning the necessary value to make this literal true. Thus, no choice is necessary.

A run of DPLL can be described as a tree, where each node represents a choice of a variable, and each edge is a choice of a value for that variable.

Claim 15.

Given a recursive tree obtained from running DPLL on an unsatisable formula as input constitutes a resolution proof with the same treelike structure, with length at most the running time of the algorithm. Proof.
We provide a proof sketch. An example of a DPLL run is shown in Figure 1. Each leaf represents a clause which is unsatised by the assignment generated by the path starting at the root and ending in this leaf. Consider the leftmost leaf (L) in the example, and the leaf next to it (R). Both

and

represent clauses that are unsatised under the assignments

000 and 001 respectively.

z , unsatisability is due to another variable. We can weaken R into x z , and resolve L and R to get the clause x. We do the same with the leaf at the end of path 01 (y ) and 00 (x), and in the end we resolve with x. In each step, given a node and its left- and right-subtrees, we would want
This means that regardless of the value of to eliminate the variable at the node. Since each subtree leads to a contradiction, it doesn't matter what value we give to the variable at the node. Therefore, we can eliminate it. In the end, we end up with the root of the tree with two subtrees of a variable and its negation (in this example,

and

x).

We then resolve the two to obtain

and complete the proof.

The following corollary follows:

Corollary 2.16.

L (C

t.R

0) Time (DPLL (C)).

This is because we have shown that each DPLL run induces a treelike Resolution proof, whose length equals the number of nodes in the tree. Therefore, the minimal length of a treelike resolution proof is at most Time (DPLL (C)). An augmented version of DPLL exists which is called

DPLL with clause learning

(we denote

it by DPLL ). In this version, when a conicting clause is reached, it is learned and a new

25

clause derived from this experience is added. The proof of the following claim is beyond the scope of this course and can be found in Beame et al. [2006].

Claim 17.

[without proof ] L (C

0) Time DPLL+ (C)

We conclude this lecture with a few remarks on the relation between general and treelike resolution proof size. We start by stating an obvious relation between the two

Theorem 2.18.
Proof.

L (C

0) L (C

t.R

0) t.R implies more constraints R 0) should be at most L (C

on the

We provide a proof sketch.

In general, since

proof, and since any

t.R.

proof is also a

proof,

L (C

t.R

0).

Concretely, Resolution proofs allow the reuse of past lines in the proof, regardless of their position. Treelike-Resolution, on the other hand, limits us to using lines in the proof which are directly above the current line. In essence, it is the dierence between a DAG and a tree. The proof of the following claim is beyond the scope of this course and can be found in Ben-Sasson et al. [2004].

Theorem 2.19.

There exist formulas for which L (C L formula it holds that L (C t.R 0) 2 log L log log L .

t.R

0) 2 log L . Furthermore, for each

26

Lecture 3

More on Sequential proof systems

December 23rd, 2008
Lecturer:

Eli Ben-Sasson

Scribe:

Eli Ben-Sasson

In this lecture we continued surveying fundamental proof systems and then described the history of lower bounds on proof length for resolution. We shall prove such lower bounds in the next lectures.

3.1 Cutting Planes

The next system we discuss is based on using properties of polytopes over

Rn

to prove the

unsatisability of CNF formulas. The idea behind this system is as follows. Suppose an unsatisable CNF. Interpret a clause over variables function of the form

is

x1 , . . . , xn

as a

half-space,

i.e., as a

ai xi b,
i=1
For instance, the clause

where

ai , b R.

(1)

C := x1 x2 x3

is the constraint

C := x1 + x2 + (1 x3 ) 1

x1 + x2 x3 0.

F = {C1 , . . . , Cm }, is satisable if and only if its corresponding set of half-spaces F = {C1 , . . . , Cm } has a solution {0, 1}n . From a geometric point of view, the set of points in Rn that satisfy F form n n a polytope PF R and F is satisable i PF {0, 1} = . So one way of proving that F is unsatisable is to show that PF contains no points in {0, 1}n and the cutting planes
It is not hard to verify that a CNF formula, i.e., a set of clauses system, dened next, gives a formal way to prove this.

planes

Denition 3.1

(Cutting Planes)

Lines in the sequential proof system known as

cutting

(CP) are half-spaces as dened in Equation 1. The system has two derivation rules.

Addition Rounded division

n n i=1 ai xi b, i=1 ai xi n (ai + ai )xi (b + b ) i=1

(2)

n i=1 ai xi b n ai b i=1 c xi c
A Cutting Planes (CP)

where

c N+

divides

ai , i = 1, . . . , n.

(3)

derivation

from a CNF formula

is a sequence of half-spaces where

each half-space is either corresponds to a clause

CF

or is derived from previous lines by

31

one of the two derivation rules mentioned above. A CP

refutation

of

is a derivation from

that ends with the contradictory half-space

0 1.

if and only if F has a CP-refutation. Proof.

that if

Theorem 3.2

(Cutting planes is complete and sound)

A CNF formula F is unsatisable

To prove soundness, we argue by induction on the number of lines in a derivation

{0, 1}n

satises

then

satises all lines of the derivation. This implies that if

is satisable then it does not have a CP-refutation, because

01

cannot be satised by

any

. C F.
Assuming

The base case is that a line corresponds to a clause clause

satises the

we also see that

satises the half-space corresponding to it.

Next, suppose a

line is derived by using the addition rule. Then assuming

satises both upper inequalities

. Finally, b and c N+ divides ai , then n ai i i=1 c is an integer and this integer is greater or equal to b/c. Thus, we may round b/c up and still n ai have i=1 c i b/c . This shows that all steps of a derivation all locally sound and by
in Equation 2 we use linearity to argue the bottom inequality also holds for consider the rounded division rule. If

n i=1 ai i

induction we conclude the soundness of all CP-derivations. To argue completeness we showed in class that CP that given a resolution refutation pretty much mimics

simulates

resolution. By this we mean

of

we can construct a CP refutation

of

that

the clause that is the

in the sense that the tth line of tth line of (details omitted).

is the half-space corresponding to

3.2 Survey of lower bounds on resolution proof length

Assuming

NP = coNP

implies that for every proof system

there exists an innite

sequence of formulas that require super-polynomial proof length in

P.

However, showing the

existence of such hard-to-prove formulas, which is the most important problem considered in proof complexity, is not a simple task. The very rst super-polynomial lower bound on a natural proof system was given in Tseitin [1968] for a restricted form of the resolution system known as

regular resolution

(which we

have not dened in class). It took nearly twenty years until the rst superpolynomial bound on resolution proof length emerged, shown in Haken [1985] for the family of pigeonhole principle formulas. Shortly after, Urquhart [1987] showed exponential lower bounds on

proof length for the family of formulas rst considered by Tseitin, and this was followed by Chvtal and Szemerdi [1988] who showed that a random CNF formula with suciently many clauses requires exponential length proofs in resolution. All of these lower bounds can be obtained in a unied way by the width method of Ben-Sasson and Wigderson [2001], which is the next topic we shall study. We point out that there are lower bound results on resolution proof length that do not follow from the width method. One example is the lower bound of Pudlk [1997] that relies on the feasible interpolation method to be discussed later on in the course (in Lecture 8).

32

Another example is that of the lower bounds for the weak pigeonhole principle Raz [2004]; Razborov [2003] which are outside the scope of this course.

33

Lecture 4

Width Method
December 2nd, 2008
Lecturer:

Eli Ben-Sasson

Scribe:

Tomer Kotek and Jonathan Yaniv

In this lecture as well as the next we will prove lower bounds on the length of a resolution derivation

for a certain class of formulas, namely

(m, ) expanding

formulas. We

start by dening the width of a formula

Denition 4.1.

The

width

of a CNF-clause

is dened to be

w(c) = #literals in c.
The width of a set of clauses (or a formula)

is dened as

w(S) = max w(c)

cS
and the width of

0 w(C

is dened

0) =

min derivation of C ,

0 w(),

C SAT C SAT

. (m, )expanding

We start by stating the main theorem, followed by the denition of what a formula is.

Theorem 4.2.

If C is kCNF over n variables and is unsatisable, and furthermore C is (m, )expanding, then it holds that
LR (C 0) exp (m k)2 n
mk 2

and
LT R (C 0) 2

. U U,
denote by

Denition 4.3.
neighborhood of

Let

G = U, V, E

be a bipartite graph. For

N (U )

the

N (U ) = v V | e = (u, v) E, u U
A graph

. U U
with

G is called an (m, ) expander |N (U )| (1 + )|U |.

if it holds that for every

|U | m,

41

Intuitively, an expander graph is a graph with high connectivity, in that every non-trivial cut of the graph contains a large number of edges. As a simple example, we note the vertex clique,

n-

Kn ,

is trivially an expander. We will be more interested in graphs in which

the maximal degree is small (such as graphs, we now dene

3, 5, log n, . . .
formulas. let

etc.). Having dened

(m, )expander

(m, )expanding CN F
formula

Denition 4.4.

Given a

C,

G(C)

be the following bipartite graph,

G(C) = C, V ars, E
with two vertex sets:

C,

the set of clauses in the formula, the set of variables in the formula,

V ars,

and with edge set We say a formula

E for which (ci , xj ) E i xj appears in cj C is (m, )expanding if G(C) is.

(either positively or negatively).

The proof of Theorem 4.2 follows from the theorem below, as well as an additional theorem we will prove in the next lecture, which states that for a

(m, ) expanding

formula,

w(C

0)

m . 2

4.1 The Size-Width Relation Theorem

Theorem 4.5 (Size-Width Relation Theorem).
1. w(C 2. w(C Proof.
0) w(C) + log LT R (C 0) 0)) c
to

Given a CNF C , it holds that:

0) w(C) + 1 + 3 n ln (LR (C c |x=a

of a clause

Recall the restriction

x=a

for

a {0, 1}

is dened as

c |x=a =

xa , x1a c / xa c otherwise
is a derivation of

1 c\ x1a

x0 = x and x1 = x. Assume = {c1 , . . . , cs } |x=a = {c1 , . . . , cs } where

where

from

C.

We denote

ci =

ci |x=a

ci C derives via resolution on cj1 y, cj2 y ci = cj A by weakening.

42

c cj2 j1 c A | x=a j1

Next we remove from now note that

= |x=a

all clauses with occurrences of

(such as

1 cj ).

We can

is a derivation of

cs |x=a

from

C |x=a .

We precede the proof of theorem 4.5 by the proof of two lemmas.

Lemma 4.6.

Given a clause A with w(C |x=a A) d, it holds that

w(C A x1a ) d + 1.

Proof.

We look at a derivation

1a , and prove of the derivation, adding x

A of width d. We perform weakening to all the clauses A x1a .

Lemma 4.7.

Let C be a kCNF formula. If

w(C |x=a 0) d 1 w(C |x=1a 0) d

Then w(C Proof.

0) max {k, d} k + d. w(C x1a ) d.

Denote the subset of clauses of

By lemma 4.6,

which contain

a the literal x by

x1a by Cx1a a nor x1a by C . So C = C a C and the subset of clauses that do not contain x x x1a C . a c . Since we are able to derive x1a Look at a clause c Cxa , then it is of the form x C xa .
Similarly, denote the subset of clauses that contain

in width

d,

we can now proceed by resolution to derive all the

in width

note this means we can in fact derive the clauses of use that .

w(C |x=1a 0) d

to derive

in

C |x=1a max {k, d}

in width

max {k, d}. We max {k, d}. Now we

w(C 0) max {k, d}? No, e.g. let C = {x, y, z, w, xyzw}. Clearly, it holds that w(C |x=1 0) 3 and w(C |x=0 0) 1. If the lemma implied w(C 0) d, then we would have in this case w(C 0) 3, but clearly in order to derive 0 from C we have to introduce the clause xyzw of width k = 4.
Is the lemma true for

w(C

0) d

instead of

4.2 Proof of Size-Width Relation (Theorem 4.5)

1. Let

0.

We prove the claim by induction on the length For

be a T.R. (Tree-like Resolution) derivation of C 0 of length S = LT R (C 0) > S and on the number of variables n. S = 1, the formula must be of the form x, x. In this case we can prove C 0 be performing the resolution rule. We get w(C 0) = 1 w(C) = 1 w(C) + log LT R (C 0).
In this case, the derivation ends with

Base Step

x x 0
43

x and x were derived by T.R. derivations Tx and Tx respectively. Denote Sx = |Tx | and Sx = |Tx |, then S = Sx + Sx + 1. We may assume w.l.o.g that S Sx Sx , then Sx 2 . We note there is a tree-like derivation of 0 from C |x=0 S of length 2 . By the induction hypothesis, there is a tree-like derivation of 0 S from C |x=0 of width k + log 2 = k + log S 1. We denote d = k + log S , then w(C |x=0 0) d 1. On the other hand, the formula C |x=1 contains n 1 variables, so by induction on n we have a derivation C |x=1 of width k + log Sx k + log S = d. By lemma 4.7 , w(C 0) log S + k.
where 2. Again we prove by induction over the length

and the number of variables

n.

Base Step
d

The proof is the same as the previous case. Let there be literals).

tS

many

dfat

clauses (that is, clauses that consist of at least

This means that there are at least

dt

occurrences of literals in the

derivation. By the Pigeon-hole Principle, there is a literal that appears in (and there for satises) at least is

dt 2n of these clauses. We assume w.l.o.g that this literal

dfat clauses in |x=1 , where is a derivation of minimal d 1 d and length (that is, of length S ), is at most t 1 2n . Set a = ad = 1 2n b . We prove by induction on n and b let b be the smallest integer such that t < a that w(C 0) d + b + k . x.
So the number of

Base Step

If b = 0, then t must 0) d d + b + k . We already saw

be

0.

Then there are no

dfat

clauses and

w(C

C |x=a

has a derivation with

By the induction hypothesis on induction hypothesis on

we

b, w(C |x=1 have w(C |x=0

d ab1 t 1 2n fat clauses. 0) d + k + b 1. By the 0) d + k + b. By lemma 4.7,

w(C

0) d + k + b. w(C

So,
(tS)

0) k log 2nd t + d d + log 2nd S

2n 2n

=d+

4n ln S d + ln S 2nd d ln 2n = 2 2n ln S < 3 n ln S

44

Lecture 5

The Width Method, part II

December 9th, 2008
Lecturer:

Eli Ben-Sasson

Scribe:

Niv Salti and Slava Chernoy

In this lecture we will focus on proving the main theorem from the previous lecture that helped us to nd lower bounds on the length of a proof. The theorem from the previous lecture:

Theorem 5.1.

If F is (, m)-expanding formula, then

w(F 0) m 2

In order to prove this theorem, we will use some other theorems and denitions. (Hall's marriage theorem). Assume G = (V, U, E) is a bipartite graph. We can marry every v V with u U monogamously i for every subset V V , |N (V )| |V |. Where N (V ) is the neighbours of all the nodes v V . Hall's theorem has the following corollary that will be crucial to our proof.

Theorem 5.2

Theorem 5.3
Proof.

(Tarsi). If F is minimal not-satisable formula, then |F | > |V ars(F )| By minimal we mean that removing any clause will make F satisable. Assume to the contrary that

|F | |V ars(F )|.

If a "marriage" exists (i.e., each clause

is mated to a unique variable and the variable's value is determined by that clause), satisable  a contradiction. exists a subset

is

Therefore, applying the Hall's theorem, we get that there

F F for which |V ars(F )| < |F |. We choose the maximum subset of that kind. If F = F , we get a contradiction. So F F holds. Consider a bipartite sub-graph derived by F = F \ F = and V = V ars(F ) \ V ars(F ) V ars(F ). Using the maximality of F , we apply the Hall's theorem and get that there is a marriage from F to V . By the minimality of F , F is satisable; we satisfy it by assigning values to V ars(F ). By existence of a marriage from F to V , we may also satisfy F without changing values already assigned to V ars(F ). Therefore, F is satisable  a
contradiction.

Denition 5.4. Claim 5.

Proof.
Let

We will say that

F |=min C

i

F |= C

and for each

F , F |= C .

If F |=min C , then |F | > |V ars(F )| |C|.

be the minimal is exactly |C|.
restriction that causes

C| = 0.

The number of variables

assigned by

Claim 6.

F | |=min 0.
51

Proof.

F | is unsatisable; otherwise, we conclude that F |= C . Let's assume to the contrary that F | is not minimal; i.e., there exists a subset F F s.t. F | |= 0. Consider an assignment to F . If agrees with , we get F () = 0. If contradicts (in at least one variable of clause C ), we get C() = 1. Therefore, F |= C  a contradiction to the minimality of F .
It is clear that Applying the Tarsi's theorem to

F | ,

we get that

|F | | > |V ars(F | )|.

This implies

|V ars(F )| = |V ars(F | ) + |C| < |F | | + |C| |F | + |C|.

Now we can return to the proof of the main theorem of this lecture.

Proof of Theorem 5.1. Given an unsatisable (, m)-expanding We dene a function as follows:

: Clauses
over

formula

over

variables.

variables

N+

(C) = min{|F ||F F, F |= C}

Notice that: 1. if 2. if 3.

C F, C

then

(C) = 1; D, E
using resolution, then

derives from (by

(C) (D) + (E);

(0) > m

(, m)-expanding

condition and the Hall's theorem).

Summing up all 3 conclusions above, we can conclude that in each proof there exist a clause

C that m/2 (C) m. F |=min C . Applying Claim

Let

be the minimal subset in size

l, m/2 l m,

s.t.

5, we get

|C| > |V ars(F )| |F | > (1 + ) l l = l

m . 2

Denition 5.7
lecting

(Random CNF)

Let

k,n Rm

be the probability distribution obtained by se-

clauses uniformly at random from the set of all

2k

n k

clauses of size

over

variables.

k,n F Rm

means that

is selected at random from this distribution.

A random

k -CNF

formula is a formula

k,n F Rm .

Lemma 5.8 (Random CNF Expanding).

For every k 3 and for every 0 < < 1/2, there 1+ k,n exists a constant K = K(k, ) that, with high probability, F Rn is an , K n/ k2 expanding CNF.

52

Using the last lemma, Theorem 5.1, and relations between width and resolution, we get the following lower bounds for random CNF.

Theorem 5.9

For every k 3 and for every 0 < < k,n 1/2, there exists a constant K = K(k, ) that, with high probability, F Rn has:
(Lower Bounds for Random CNF)

w(F LR (F

0)

K n 2 k2
1+

; K 2 k2
1+

0) exp n

k n

probability, F

Corollary 5.10

(Lower Bounds for Random 3-CNF)

3,n Rn

an ,

n/1+2

For every 0 < < 1/2, with high -expanding. Therefore,

n 1+2 ; n .

w(F LR (F

0) =

0) = exp

2+4

For = O(1), we get w(F

0) = (n) and LR (F

0) = exp((n)).

53

Lecture 6

Width of Explicit Encodings

December 16th, 2008
Lecturer:

Eli Ben-Sasson

Scribe:

Slava Chernoy

Recall from the previous lectures the property of expanding, then

(, m)-expanding. m . 2

If CNF

is

(, m)-

w(F
For random CNF

0)

F,

we got lower bounds using the property of

(, m)-expansion,

but for

Tseiten formulae this property can not be applied. To show this let's consider a graph with a small degree, say

deg(G) = 3.

Recall that

T (G) = {e v xe = l(v)|v V }. xe1 xe2 xe3 T (G) and expand it as a CNF, i.e., a set of clauses: {x1 x2 x3 , x1 x2 x3 , x1 x2 x3 , x1 x2 x3 }. Consider the bipartite constraint graph (V, U, E) of this set of clauses. For each clause above, there is a vertex in V, and for each variable xi , there is a vertex in U . Let V be a set of vertices formed by those 4 clauses, then E(V ) contains 3 vertices corresponding to the variables x1 , x2 , x3 . So for V we have |V | = 4 and |E(V )| = 3; i.e., the graph (V, U, E) is not (, m)-expander, T (G) is not (, m)-expanding, and the property cannot be applied. To overcome the problem we will
Let's choose a constraint use a dierent approach. We work with sets of arbitrary boolean constraints (as opposed to considering only clauses) and introduce a stronger form of expansion, called

boundary

expansion,

or

unique-neighbor expansion.
Assume

Denition 6.1 (Boundary).

V V.

Then the boundary of

is

V = {u U |u

has exactly one neighbour in

V }.
and

Using boundary expansion we introduce

boundary expander graphs

G

boundary expanding
(r, c)-boundary ex-

formulas.

Denition 6.2
pander
if

(Boundary Expander)

A bipartite graph

is called

V V, |V | r, |V | c |V |.
In words, every small set has a boundary that is large relative to its size.

61

Denition 6.3
expanding

(Boundary Expanding)

A set of constraints

if the bipartite constraint graph derived from (Sensitivity)

is a

F is called (r, c)-boundary (r, c)-boundary expander.

Denition 6.4
any

A boolean function assignment

I [k]

such a way

|I| l and for any that f will be satised.

s.t.

f : {0, 1}k {0, 1} is l-sensitive if for s.t. f () = 0, may be changed on I in

Example 6.5. The Boolean functions OR and XOR are 1-sensitive at least r of its inputs are 1) and ANDr are r -sensitive.
We say that

while Thr (outputs 1 i

F |=min C

if

F |= C ,

but for every

f F , F \ {f } |= C .

If a set of constraints F is (|F |, c)-boundary expanding s.t. F |=min C and every f F is l-sensitive, then |C| |F | (c l + 1). Proof. Assume is a minimal assignment s.t. C| |= 0 and || = |C|. Let's show that F | |=min 0. Assume rst that F | |= 0. Then there is an assignment satisfying F | , i.e., F | () = 1. Since C| is unsatisable, we get that F |= C  a contradiction. Now assume that F | |= 0, but F | |=min 0; i.e., there is a subset F F s.t. F | |= 0. Let be an assignment s.t. C() = 0. Since is a minimal assignment falsifying C , ; hence F () = 0. This implies F |= C  a contradiction. We have F | |=min 0 holds. Then for every f F , there exists an assignment s.t. (F \ {f }) | () = 1 and f | () = 0. Let's consider F . Every variable in F belongs to an unique f F . Let f has t neighbours (variables) in F . We claim that after restricting to at most l 1 of them remain free. Assume by contradiction that f | has at least l free variables in F . By the l-sensitive of f , we may change on those l variables and satisfy f . Since the variables appear in F , the value of (F \ {f }) | will not change. This implies that F | is satisable  a contrudiction. We have that after restricting to , at most |F |(l1) variables of F remain free. Therefore,
|C| = || |F | |F | (l 1) |F | c |F | (l 1) = |F | (c l + 1).

Claim 6.

Before we proceed, we obtain a lower bound for Tseiten formula on a the last claim.

-edge

expander using

Denition 6.7.

d-regular

graph

G(V, E)

(i.e.,

deg G d)

is called

-edge

expander if

V V, |V |

|V | , |E(V , V )| |V |. 4

Let's recall the denition and the properties (without proof ) of the width measure on clauses.

Denition 6.8 (Proper Width Measure).

Given a set of XOR-functions

F.

F (C) = min{|F | | F F, F = , F |=min C}.

62

Lemma 6.9.

Properties of Proper Width Measure

F (Axiom) = 1; F is sub-additive, i.e., any proof of F D, E , F (C) F (D) + F (E); C

0 must contain a clause C with 1 F (0) F (C) < F (0). 2

bipartite), then

Corollary 6.10.

If T (G) is a Tseiten formula on a -edge expander graph G (not necessarily

|G| , 8 0) exp ( ( |G|)) .
Let

w(T (G) LR (T (G)

0)

Proof. In the proof we denote = T (G) . that (0) = |G| = |V |, we conclude that

be a minimal proof of

T (G)

0.

Noting

must contain a clause

s.t.

|V | |V | (C) . 8 4 |V | |V | . Recall that every function f T (G) 4 is 1-sensitive and for every vertex v V , T (G) contains an unique function fv := e v xe = l(v). Clearly, {fv |v V } = {xe |e E(V , V )}. Therefore, |V | equals to the number of edges from V to V , i.e., |V | = |E(V , V )| . Since G is -edge expander, V is (|V |, )Let

V T (G)

s.t.

V |=min C

and

|V | 8

boundary expanding. Therefore, applying Claim 6 we get

w(T (G)

0) = w() |C| |V | (c l + 1) = |V | ( 1 + 1) = |V |

|V | . 8

Now we prove the main lemma of the lecture.

Lemma 6.11

If formula (set of boolean functions) F is a (r, c)-boundary expanding s.t. every its boolean function is l-sensitive, then
(Width Lower Bounds for Explicit Encodings)

w(F

0)

r (c l + 1) , 2

where F is a CNF equivalent of F .

c + 1 l the claim is trivial, so assume c + 1 > l. Let F be a (r, c)-boundary expanding set of l-sensitive functions. Assume F (0) r . Then by the denition of F , there is F F s.t. 0 < |F | = F (0) r and F |= 0. Moreover F is (|F |, c)-boundary expanding set of l-sensitive functions. Therefore, applying Claim 6 we get that 0 = |0| |F | (c l + 1) implying |F | = 0  a contradiction.
For 63

Proof.

F (0) > r. Then by the properties of F , there is a clause C in a proof of F 0 1 1 s.t. 2 r < F (C) < r . This implies that there is a subset F F s.t. 2 r < |F | = F (C) < r and F |= C . Using Claim 6 again, we get
So we have

w(F

1 0) |C| |F | (c l + 1) > r (c l + 1). 2

64

Lecture 7

A Combinatorial Characterization of Width

December 23rd, 2008
Lecturer:

Eli Ben-Sasson

Scribe:

Shahar Papini

This lecture introduces a dierent approach to deal with Width.

We will associate the

Width measure, with combinatorial games, and through a series of lemmas, we will show an equivalent combinatorial way to nd the Width of a formula. We will start by introducing some pebble games, and later, relate them to Width. Moreover, we will show a dierent

measure, the Space measure, which, roughly speaking, will represent the minimal space (e.g. RAM) we must use in order to prove a formula. As it appears, Space is bounded from below by Width, and we will prove that in our lecture. Let us start with some denitions.

Denition 7.1 (Proof graph). Let = {C0 , . . . , Cn } be a proof in a sequential proof system.
A graph

G = (V, E) is called a proof graph of if G is a rooted directed acyclic graph (DAG) for which there is a bijective mapping : {C0 , . . . , Cn } V such that if {Ci1 , . . . , Cik } Cj in the proof, then n. ((Cj ), (Cin )) E .

Note.
G

A given proof

may have several distinct proof graphs associated with it. We say that the space of a rooted DAG

Denition 7.2 (Pebbling cost (space) of a graph).

is at most

k,

if we can use at most

pebbles and reach the root in the following manner:

We can place a pebble on a node if we have placed pebbles on all of its immediate predecessors. Notice that this means we can always place a pebble on a node with 0 in-degree, if we have enough pebbles.

We can remove a pebble from any node.

The space of the graph is the minimal such

k.
is of space

Denition 7.3
k
if

(Space of a proof )

Let

be a resolution proof. We say that

is the smallest integer such that there exists a graph of Let

that is of space

k.

Denition 7.4 (Space of a formula).

be an unsatisable CNF formula.

Space(F ) = min Space()

Where the minimum is taken over all refutations

of

F.

The following result was proved in Hopcroft et al. [1977].

71

Claim 5.

For every rooted DAG with fan-in at most 2 and n vertices

Space(G) O n log n

We immediately get

Corollary 7.6.

For every unsatisable CNF formula F we have

Space(F ) O Length(F ) log(Length(F ))

We shall use an equivalent denition of refutation space.

Denition 7.7
called a the following:

(Refutation Space  Equivalent denition)

CNF. A conguration

space proof

of

. Let F be an unsatisable kMi is a set of clauses. A sequence M0 , . . . Mn of congurations is F if and only if M0 = and Mi+1 is derived from Mi using one of
Mi+1 = Mi {C}
where for

Axiom download: Inference:

C F. Mi
using one of the

Mi+1 = Mi {C},

is derived from clauses of

resolution derivation rules.

Erasure:

Mi+1 = Mi \{C}

for

C Mi .

The Space of such that

F is the minimal k 1 for which there is a space proof {M0 , M1 , . . . , Ms = {0}} |Mi | k for all i s.

We have the following connection between and proof space and treelike proof size.

Lemma 7.8 (Treelike Resolution Space-Length relation).

It holds that

Let F be an unsatisable k -CNF.

Space(F ) log(LT R (F )) + 2

Proof.

First, notice that in a treelike resolution proof, the graph of the proof is the tree used

to generate the treelike proof. So, we will use the pebble-cover denition of space. We will prove by induction on the size of the tree,

n,

that its root can be covered, obeying

the rules mentioned in the denition of space, using no more than For

log n + 2

pebbles.

n = 1,

the tree is a single node, and thus, we need one pebble to cover the tree.

For the general case, given a tree

of size

n,

the root has two children which form subtrees

T1 , T2 . |T | = 1 + |T1 | + |T2 |, so, without loss of generality, |T1 | |T | . Using the induction 2 hypothesis, we can cover T2 with no more than log(|T2 |) + 2 log(|T |) + 2 pebbles. After we do, we remove all pebbles except the one on T2 's root. Now, we do the same for T1 , using no more than lg(|T1 |) + 2 max(log(|T |) 1 + 2, 0)(Note that T1 might be an empty tree, but not T2 ), not including the pebble on T2 . Now remove all pebbles from T1 except
72

T (now there are 3 pebbles on the board). Overall we have used max(log |T | + 2, log(|T |) 1 + 2 + 1, 3). Since n 2, log n 1, so we used no more than log n + 2 pebbles.
the one on root. Now, place another pebble on the root of

T1 's

The next denition is an important one, as it introduces a kind of a combinatorial game, which is tightly associated with Width. Afterwards, we will show that the minimum

for

which this game has a winning strategy, is exactly the Width of the respective formula. We will also relate this combinatorial game, to the Space of a formula, and thus, will get a relation between Width and Space.

Denition 7.9

(Existential

game)

Given a

k -CNF

formula,

F,

the existential

game

is dened as follows: There are two players, Duplicator(D) and Spoiler(S). Each has

board. Spoiler's game board is composed of all the variables of is composed of all assignments to variables in

(e.g.

k pebbles, and its own game F . Duplicator's game board x = 0 or x = 1).

Spoiler goes rst, and in his turn he may do one of the following:

Place a pebble on a variable.

In this case, Duplicator, in his turn, must place a

matching pebble on an assignment to that variable.

Remove a pebble from a variable. In this case, Duplicator, in his turn, must remove a matching pebble from an assignment to that variable.

Spoiler wins the game if Duplicator's pebbles form a partial assignment that falsies one of

F 's

clauses. If Spoiler has a winning strategy, Spoiler is said to win the existential

game.

Otherwise, Duplicator is said to win the existential

game.

Lemma 7.10 (Width-Existential Game Relation 1).

Proof.
C. H
Let

If there is no refutation of k -CNF, F , of width d, then Duplicator has a winning strategy for the existential d + 1 game.
C
be all clauses that are derived by

with width no more than

d.

Let

be

all partial assignment for no more than

d+1

variables, that do not falsify any clause in

is obviously not empty since the empty assignment belongs to it (because there is no

refutation of

with width

d,

so

0 C ). /

We shall prove that Duplicator can always use a

partial assignment from

to answer Spoiler's moves.

Assume by way of contradiction that Spoiler asked for variable valid move.

x,

and Duplicator has no

Thus, he doesn't have an extenstion of current assignment

in

H, d

to also

include variable existential

x,

such that it is still in

(doesn't falsify clauses in

C ).

Since this is the variables

d+1

game, current partial assignment

must be of no more than

(spoiler just placed the He cannot extend is of the form

d + 1th pebble, or lower). x = 1, so there must be a clause f.

Same for

x,

or in other words, is of the form

has already been assigned a value in

C that after the partial assignment A x such that every variable in A x = 0 with B x. However, notice that
in

73

using the rules of resolution, those two clauses derive more than

A B.

Moreover, since

assigns no

variables, and completely covers

and

(as stated before), then the width of

AB
Thus,

is no more than

d.

Thus

AB

is in

C.

But this means that

falsies a clause in

C,

contradiction.

H is

a winning strategy, and Duplicator wins the existential (Width-Existential Game Relation 2)

d+1

game.

Lemma 7.11
Proof.
Let

Given k -CNF, F , if Duplicator has a winning strategy for the existential d + 1 game, then there is no refutation of F of width d.
H
be the set of all partial assignments that can be produced by Duplicator in a

winning strategy, in any stage of the existential assignment belongs to it

d + 1 game.

It is not empty, since the empty

Assume by way of contradiction that there is a refutation of

d, C0 , . . . , Cn . We will show using complete induction that each partial assignment in H does not falsify any clause Ci in the proof, and thus, this cannot be a refutation (since 0 is falsied by any
of width partial assignment). Base case:

C0

is a clause of

F,

and every partial assignment in

does not falsify clauses of

by denition.

General case: Assume all

C0 , . . . , Ci1 hold the claim. If Ci is derived by weakening, then H still does not falsify it (in a trivial way). Otherwise, Ci = A B , where Cj = A x and Cj = B x are some clauses in the proof (i.e. derived using resolution rule). Using the induction hypothesis, each partial assignment in H does not falsify Cj nor Cj . Now, Let f be any partial assignment in H .
every partial assignment in

If If

does not cover every variable in

A B,

then it obviously doesn't falsify

Ci .

f covers every variable in A B , since f is achieved in the middle of the existential k +1 game by denition of H , Spoiler may remove pebbles from all other variables, and
we are left with another partial assignment. Notice, that since this is a refutation of

d dierent variables, so Spoiler has enough pebbles to x (and Duplicator answers using his winning strategy, so the resulting partial assignment is in H ). But now the partial assignment also covers A x and B x. Using the induction hypothesis, it does not falsify them, and since it covers them, it satises them. Because Resolution is sound, the clause Ci = A B
width has at most place another pebble on variable derived from the previous two clauses using Resolution rules, is satised as well. These operations did not aect the satisability of assignments to variables of

d, Ci = A B

A B.

So,

A B , because we did not f satises Ci , and we are done.

change the

Given k -CNF, F , Duplicator has a winning strategy for the existential d + 1 game if and only if there is no refutation of F of width d.

Corollary 7.12.

Lemma 7.13 (Space-Existential Game Relation).

74

If Duplicator can win the s + k 1 pebble game for k -CNF F , then there is no refutation of F of space s or less.

Proof.
space

Assume by way of contradiction that there is a refutation of

F , C0 , C1 , ..., Cn ,

of

or less (here

Ci

is not a clause, but, a set of clauses). Let

be the set of all partial

assignments that can be produced by Duplicator in a winning strategy, in any stage of the existential

s+k1

game.

We will show that each

Ci

is satisable, ans thus, we cannot reach the contradiction clause

0.

To do this, we will prove by induction that each

satises it, s.t. Base case: Case 1:

|fi |(number

of variables

Ci has a partial assignment fi fi assigns) |Ci |(number of clauses in Ci ). Ci = Ci1 {C},

where

that

C0

is the trivial clause(1), and thus is satised by the empty partial assignment.

C F . Using the induction hypothesis, we have fi1 H , a partial assignment that satises Ci1 (All partial assignments in H do not falsify a clause in F ), and |fi1 | |Ci1 |. Since partial assignment fi1 is in H , and |fi1 | |Ci1 | = |Ci | 1 s 1, and C is a k -clause (because F is k -CNF), then we can extend f by those k or less variables, for a total of no more than s + k 1 variables, and end up with partial assignment g H , that satises Ci . That is true because Duplicator wins s + k 1 pebble game, so he must be able to extend fi1 (in H , his winning strategy). Obviously, since fi satises it, we can pick for each clause in Ci , a literal that satises it in fi , and remain with a partial assignment fi , s.t. |fi | |Ci |. Case 2: Ci has been deduced by axiom inference, Ci = Ci1 {C}, where C is derived from clauses of Ci1 . Using the induction hypothesis, we have fi1 H , a partial assignment that satises Ci1 , and |fi1 | |Ci1 |. Because Resolution is sound, the partial assignment fi1 satises Ci as well. |fi1 | |Ci1 | |Ci |. Case 3: Ci has been deduced by memory erasure, Ci = Ci1 \{C}, where C is derived from clauses of Ci1 . Using the induction hypothesis, we have fi1 H , a partial assignment that satises Ci1 , and |fi1 | |Ci1 |. Obviously, fi1 still satises Ci . Using same sense as in Case 1, we can take one variable from every clause of Ci that satises it, and build a partial assignment fi that still satises Ci , and |fi | |Ci |. The induction is complete, and every Ci is satised. But C0 , ..., Cn is a refutation, so Cn = 0, Ci
has been deduced by axiom download, even though it is satisable, contradiction! Thus, the assumption cannot be true, the space of the minimal refutation of

must be

s + 1.

Corollary 7.14.

Let F be an unsatisable k -CNF formula. Then it holds Space(F |= 0) W idth(F |= 0) k + 1. Proof. By Width-Existential game relations, since there is no refutation of F with width W idth(F |= 0) 1, then Duplicator wins the W idth(F |= 0) pebble game. Using SpaceExistential game relation, since the Duplicator won the W idth(F |= 0) pebble game, the space of a refutation of F must be at least W idth(F |= 0) k + 1. So, Space(F |= 0) W idth(F |= 0) k + 1.

75

Lecture 8

The Feasible Interpolation Method

December 30th, 2008
Lecturer:

Eli Ben-Sasson

Scribe:

David Markus

8.1 Introduction
In this lecture we dene the

feasible interpolation method,

which is used to prove lower

bounds on certain proof systems (currently - on Resolution and CP). The method to prove lower bounds is: 1. Prove that an proof system has the feasible interpolation property. 2. Use an hard-to-compute problem (for example it's not in

P),

integer factorization,
|F | nO(1)
is an

assuming that

and prove that if an proof system has the interpolation property, the

problem can be computed in

LP (F )O(1)

where

CN F ,

and

is the

length of the input to the problem. 3. Assuming that the computational problem is hard to compute, the proof of proof system can't be short. In this lecture:

in this

We will dene the feasible interpolation method, and show some known results. We will show two computational problems: problem. deciphering and the clique/colorability

We will prove that the

Resolution proof system has the feasible interpolation property.

8.2 Lower bounds on proof length using the Feasible Interpolation method
The

feasible interpolation method is a reduction from proof complexity to computation comIt relies on the following intuition. If it is hard to compute a problem then it

plexity.

is reasonable to expect that it should be hard to prove things which are related to this computation. The research regarding interpolation started in the study of mathematical logic. Craig's

Interpolation Theorem is a well-known example which we informally describe next. Assume

is rst order theory, and

x, y, zA(x, z) B(y, z)

when

x, y, z

are distinct variable

groups.

81

This formula is equal to

x,y A(x) B(y),

i.e.,

A(x, z) B(y, z), and if the value of z is set to constant value, either A(x) or B(y) must be true in T .

Theorem 8.1 (Craig's interpolation theorem, informal statement).

If T x, y, zA(x, z) B(y, z) then there is a formula C(z), which is called the interpolant of A and B , such that T A(x, z) C(z) and T C(z) B(y, z). (this means that if C(z) = 0 then A(x, z), and if C(z) = 1 then B(y, z)).

Krajek's idea: consider the notion of an interpolant in a propositional system and study

computational complexity. More to the point, assume F is a CNF, F (x, y, z) = A(x, z) B(y, z) is unsatisable. An interpolant for A B is an boolean circuit C(z) such that if C() = 0 then A(x, ) is unsatisable, and if C() = 1 then B(y, ) is unsatisable. Notice that if A B is unsatisable, there must be some interpolant.
its

Denition 8.2.
proof in proof for

For proof system

if for any unsatisable

P , we say that P has the feasible interpolation property A B , |C(z)| (LP (A B 0))O(1) . In words, if A B has a short

P then the size of the interpolant is small. (In the examples below, given a short A B , we'll show how to eectively construct from it a small size interpolant C ).

8.2.1 Results about feasible interpolation

Resolution has the feasible interpolation property. Cutting Planes has the feasible interpolation property. If Frege has the feasible interpolation property then it is possible to factor numbers using polynomial-size circuits. feasible interpolation.) (This is regarded as evidence against Frege having

if

T C0 -Frege

has the feasible interpolation property, then it is possible to break RSA.

(Open question: is the existence of feasible interpolation for Frege equivalent to

P=

NP?)

8.2.2 Lower bounds on proof length via the feasible interpolation method
We take a hard problem and dene an interpolant which will compute it. For example, assume

and

B
is is

such that

rst bit of rst bit of CNFs.

M M

0 1.

and

A(x, z) encodes the statement  z is an encryption of M , and the B(x, z) encodes the statement  z is an encryption of M , and the Additional variables may be needed to allow A, B to be formulated as
must compute the rst bit of

An interpolant for

AB

E 1 (z).

If computing this bit is

hard, i.e., if there in no small circuit to compute the rst bit of short proof in resolution or in cutting planes for

E 1 ,

then there is no

A B.

(On the other hand, for some

cryptographic functions we do know of a short Frege proof of

A B,

thus showing that

feasible interpolation is unlikely for Frege.) We have now conditional lower bound, because

82

the hardness of hard.

AB

is proved only by assuming that computing the interpolant

is also

For CP, the only known exponential lower bound on proof length is shown via the feasible interpolation method. (Open question: are random-CNFs hard for CP?). If there are

such that

A(x, ) = 0

and

B(y, ) = 1,
and

there may be several dierent legal

interpolants for

AB

and to prove a lower bound for CP, we need to start with

A, B

such

that no small circuit can separate

{|A(x, ) = 0}

{|B(y, ) = 1}.

If A(x, z)B(y, z) is an unsatisable CNF over distinct variables, then there exists an interpolant C(z ) for A(x, z) B(y, z) such that |C| O(LR (A B 0)). Moreover, if z appears only in positive literals (or only negative literals) in A, then C is a monotone circuit (it has only , gates).

Theorem 8.3 (Feasible Interpolation for Resolution).

Theorem 8.4

(Feasible Interpolation for Cutting Planes)

|C| O(LCP (A B 0)). Moreover, if z appears only in positive literals in A, then C is a monotone circuit.

For these A, B, C it holds that

Let m, n N and m < n. Z = {zij : 1 i < j n} Z represents n. zij = 1 if (i, j) is vertex in G. Am = {G|G includes m clique}, Bm = {G|G is (m 1) colorable}. Am Bm = . We dene A(x, z) = G Am , B(y, z) = G Bm . graph with size Notice that

Denition 8.5.

separates

A, B

if

C(z) = 0 g A, /

and

C(z) = 1 g B . /
n log n
2 3

Theorem 8.6.
|C| 2
1 n log n 3

if C separates A, B and C is monotone circuit, and m = .

, then

The previous theorem means that there is no eective interpolant for the two sets dened in Denition 8.5. We dene a CNF formula as follows.

x = {xli |l = 1..m, i = 1..n}, then A(x, z) = = 1..m}, {li xli |l = 1..m, i = i }, {li xl i |l = l }, {li xl i zij |l = l , i, j} A x x x is satisable i z contains a m-clique. The variables of z appear only in positive literals, so the circuit is monotone. m1 If we dene Y = {yis |l = 1..n, i = 1..m 1}, then B(y, z) = {s=1 yis |i = 1..n}, {is y yis |, s = s }, {is yjs zij |i, j, s} (in B z 's variables appear only in negative literals). y
Let

{n xli |l i=1

According to Theorem 8.4 and Theorem 8.6 above we have Despite that the CNF itself is short (|A

LCP (A B

0) 2

1 n log n 3

B| = O(x3 ))

8.3 Proof of feasible interpolation property for Resolution

The theorem is proved by providing way to construct an interpolant-circuit from a Resolution refutation.

83

1. Given an assignment to

z , we prove that it isn't possible to partition the clauses of the refutation to two groups: clauses which doesn't include variables from y , and clauses which doesn't include variables from x. 0)
belongs to the rst group, then

2. If the last clause (which is

is unsatisable, and if

it belongs to the second group,

is unsatisable.

3. We dene a boolean circuit which for a given belongs.

z,

can distinguish to which group

Proof of Theorem 8.3. By induction. We say that a clause is a x-clause if it doesn't contain y -variables, and say it is a y -clause if it doesn't contain x-variables (in both cases the clause may contain z -variables). Given assignment to z , the axioms of A are x-clauses, and the axioms of B are y -clauses.
When a resolution inference is used, 1.

Dl E l , there are three possible cases: DE

is literal over

z. xand

If

l() = 1 D. We did l

we can replace not mix

D E with E , y -variables.
and

and if

l() = 0

we can replace

DE

with

2. if

is variable over

x, D

are

x-clauses,

so after the Resolution we also get

x-clause.
3. if

is variable over

- as above.

Dl and E l can be from dierent groups only if l Z . When the induction ends, 0 belongs to the x-clauses or to the y -clauses. By this it is possible to decide if A is unsatisable or B
is unsatisable. To create circuit we dene new gate: Given

sel(l, a, b) =

a l=0 b l=1

as DAG, we replace it by a circuit of size ||, which is constructed from the constants {0, 1} and from the gates , , sel. the axioms of A will be replaced by 0, the axioms of B - by 1. In junction we replace: l xab l y ab l z sel(z, a, b) (z x) y (if we start with x and then there is y -clause, it's better to continue with it, because it doesn't have x-variables, so it is shorter)

84

Lecture 9

Lower bounds for Res[k] refutation length

January 6th, 2009
Lecturer:

Eli Ben-Sasson

Scribe:

Gadi Aleksandrowicz and Daniel Genkin

9.1 Introduction
Today we shall present a family of proof systems, denoted

Res[k],

where

is a parameter,

which generalizes our well-known resolution proof system (it turns out Resolution is exactly

Res[1]).

Our nal goal is showing lower bounds for proofs for random

CN F

formulas. To

this end, we use two new ideas: a switching lemma, and a concept of local soundness, which would be explained later. Formally, we aim to show that with high probability for a random CNF with and

variables

10n

clauses:

k : (k) > 0 :LRes[k] (F

Let us begin with a formal denition of

0) > exp ( (n ))

Res[k].

9.2 Res[k]
Historically,

9.2.1 History and Motivation

Res[k]
was dened about ten years ago by Krajicek. He noticed that there prove PHP,

are weakened formulations of bounded arithmetic such that if they can prove PHP, then

LRes[k] (PHP) no(1) . Thus, in order to show that these formulations do not we show bounds on Res[k] (i.e. counting is impossible in the system). o(1) In particular, a theorem by Ajtai: I0 PHP LboundedF rege (PHP) n

9.2.2 Formal Denition

As always, our proof systems are composed of lines that can appear in a valid proofs, and a set of derivation rules. The lines in

ki j=1 lij . Note that a

is a clause that is

k DN F formulas (clauses of size k each): CN F is a set of 1 DN F formulas, and in Resolution each line 1 DN F , and so Res [1] =Resolution. Res[k]
are

The derivation rules are the rules of Resolution, and in addition creation and decomposition of DNFs (for literals lij and 1.

k DN F C ):

-introduction:

l1 C,...,lk C V ( k lj )C j=1
91

Vk
2.

-elimination:
( V

j=1 lj C

lj C lj )C,...,l1 lk D CD

3. Resolution:

The third rule is redundant, since it can be simulated by using the standard Resolution rule and rules 1-2.

9.2.3 Semantic Denition

We now present an alternative (semantic) denition; we'll denote the following proof system as

S Res [k]. kDN F

formulas and we have only a single,semantic,

In this new proof system the lines are derivation law: 1. For

C1 , . . . , Ck , D k DN F ,

if

C1 , . . . , Ck |= D

then

C1 ,...,Ck . D

It suces to show a lower bound for this proof system in order to obtain one on denition of

Res[k].

The

S Res [k] is simpler than that of Res[k], although the system may be stronger P we can similarly dene LP (F 0) LSP (F 0).
a semantic proof system

(we may be able to prove more things in it, and with shorter proofs) For every proof system general we have

S P,

and in

9.3 The General Idea

We will make use of a new notion: decision trees. A decision tree is a binary tree in which every internal node has exactly two children. The inner nodes are denoted by variables, the edges by 0,1, and the leaves by 0/1. Given an

assignment, we traverse the tree according to the assignment and the output is the value in the leaf we reach. Therefore, a decision tree describes a function, so it makes sense to represent formulas using decision trees. Suppose now we have a proof in such that

S Res [k]:

A collection of lines,

= C1 , . . . , C s = 0

Ci

is a k-DNF.

Assume by contradiction that 1. Hit

is small, e.g.

s = no(1) .

with a random restriction - choose a subset of variables and assign values to

them. We'll remain with

| = C1 | , . . . , Cs |
o(1)

2. Claim that with probability greater than 0, every tree of small depth, e.g. 3. We now have that derivation). Now translate every

Ci |

can be described as a decision

d (Ti ) = log

n.
with

| = T1 , . . . , Ts = 0 Tjt

d (Ti )

small, and

Tj1 ,...,T jk (semantic Ti

to a formula in Resolution with small width. But no proof of

small width exists in Resolution - contradiction.

92

Lecture 10

Random 3-CN F and Res[k] refutations

January 13th, 2009
Lecturer:

Eli Ben-Sasson

Scribe:

Tomer Kotek and Jonathan Yaniv

10.0.1 Introduction
In this lecture we start the proof that almost surely a random 3-CN F does not have a short proof in the proof system

Res[k].

We will do so by assuming that a random 3CN F has

a short proof, and reach a contradiction. The contradiction will be reached by proving a switching lemma and thereby showing that the width of the refutation of

must be small

on the one hand, and on the other hand showing that it must be large because

has an

expanding graph. Today we will show the switching lemma, which says that under a certain distribution on restrictions,

is of small depth.
1

There are depth bounded functions which are tough to compute. For example, for XOR on

bits,

We

sized (n ) exp nO( d ) . Another function that is tough to compute is majority. expand the model to depth d and {, , }. In the proof we will use a switching lemma

as follows:

We hit the bottom level of the depth

d formula with a random restriction, which leaves

only n variables unassigned.

This reduces the depth of the function to after being hit, can be represented as a

d 1. Every DN F short CN F .

on the bottom level,

We do this exclusively until we get a depth-2 function. For XOR (for example), we get an exponentially large expression of depth

2.

We note that under restrictions, a XOR remains a XOR.

10.0.2 The proof plan

Now lets look at a random 3-CN F refutation in the a random consistent

F.

We assume by contradiction that

has a short

Res[k] system, where proof lines are kDN F s. With very high probability, restriction will set to 0 some clause in F . We need an assignment which is with F .

The proof plan is: 1. Assume

= D1 , . . . , D s

is a

Res[k]

refutation of

F.

Assume

is small (say

s=

nO(1) ).

101

2. Apply a good restriction to all lines in prove. For resolution, 3. Argue that small

Good means that

F |

is still hard to

W (F | 0)

is large. can be converted into a Resolution refutation of is small).

| = D1 | , . . . , Ds | width (for each Di | , d(Di | )

4. This is a contradiction, since the graph of

will be shown to be expanding. Therefore,

must be large.

The third step will follow from the switching lemma we will prove today.

10.0.3 Switching lemma

Denition 10.1.
variable in Let A set

of variables is a cover of a

DN F D,

if every term of

S.

The cover number of

is dened as the minimal size of a cover of

D D.

has a

be a

k CN F .
c k

Claim 2.
Proof.

If c(D) = c then D has

mutually disjoint terms.

c.
it's a

If not, then we can nd a cover smaller than

The probability that a term the probability that the

ti = li1 lik of D be set 1 is 2k . But since whole DN F gets set to 1 goes exponentially down! P r [D | = 1]
is small.

DN F ,

If If

c(D) = c

is large,

c(D) = c DN F .

is small, then after querying

c(D)

vars, the rest goes down to a

(k 1)

Switching Lemma

Theorem 10.3.
1. k 1

Let the following hold:

2. s0 , s1 , . . . , sk1 0 3. p1 , . . . , pk 0 4. R be a distribution over restrictions satisfying: For every i k and i DN F D, Then for every kDNF it holds that
k1 k
if

c(D) si1

then

P rR [D | = 1] pi

P rR d(F | ) >
i=0

si
i=1

Pk1
j=i

sj

pi ()

102

Corollary 10.4.

Let k N such that k 1 and R be a distribution on restrictions such that for every kDNF F , we have
P rR [F | = 1] < 2c(F ) ()

then for every kDNF F ,

P rR [d(F | ) > 2s] < k 2

s 4k

s Proof of Corollary. We set si = 4i and pi = 24si = 2si1 .We notice that k1 sj = j=i k1 1 si (1 + 1 + 16 + . . .) 2si . We can now apply the theorem with s0 , . . . , sk1 and j=i 4

p1 , . . . , p k ,

because if

c(D) > si1

then

P r [D | = 1] 2si1 = pi .
k1

by ()

So by the theorem,

P rR d(F | ) >
i=1
and

si P r [d(F | ) > 2s]

k1

P rR d(F | ) >
i=1

si
i=1

Pk1
j=i

k si

pi
i=1

22si 24si k 2

s 4k

Proof of Theorem 10.3  The Switching Lemma.

By induction on So

k 1:
can be represented by a

Base

If

c(D) s0

then

decision tree of size

D is a s0 . So,

clause with for every

s0 variables. kDN F F ,
k1 k

P rR d(F | ) >
i=0

si
i=1

pi

Pk1
j=i

si

,
This

even without applying a random restriction. If completes the base case.

c(D) > s0 , then ( ) implies ().

Step

Assume theorem holds for If If

k1 ( )

and consider a

k DN F D.

c(D) > sk1 ,

then again

implies

().

c(D) sk1 , then let T be a cover for D. Then |T | sk1 . For every {0, 1}T , consider D | which is a k 1 DN F . By the induction hypothesis,
k1

P rR d ((D | ) | ) >
i=0
We will connect to each node for each

s i < 2(

Pk2
j=i

sj )

pi . (k 1)DN F
.

representing

a tree for a

103

So:

k2

k1

P r {0, 1} , d ((D | ) | ) >

i=0 k1 Pk1
j=i

si < 2

k1

i=1

2(

Pk2
j=i

sj )

pi =

2(
i=1
Assume

sj )

pi .
k2 i=0 si . What concerns us is if

is good, i.e.

, d ((D | ) | )

and

conicts on some variables.

We will prune the paths in which

and

aren't

consistent.

Claim 5.

The dtree obtained by pruning T according to gives a decision tree that represents D | (is given as homework). This claim nishes the proof.

104

Lecture 11

Random 3-CN F and Res[k] refutations  Part II

January 20th, 2009
Lecturer:

Eli Ben-Sasson

Scribe:

Hanna Fadida and Carmi Grushko

In this lecture we investigate the complexity of random Given a Res [k] refutation that

= l1 , l2 , . . . , ls = 0

of a random

3 CN F s for Res [k] systems. 3 CNF, our goal is to show

- the proof size, is big enough.

In the last lecture we hit every line by a random

restriction and from each line we derived a decision tree with small depth, which can be converted into a narrow Resolution refutaion . In this step there are no contradictions

because it's possible that under the random restriction some of the clauses became false. In order to get a contradiction, we show that the surviving system of equations is still expanding and therefore requires high-width to refute.

11.1 From 3-CNF to a system of linear equations [M. Alekhnovich]

xe1 xe2 xe3 of a 3-CNF, construct the following equation over GF (2): 1 2 3 x1 + x2 + x3 = e1 + e2 + e3 where ei = 1 denotes the literal xi , and ei = 0 denotes xi .
Given a clause Note that we only restricted ourselves further, thus any lower bound using these XORs also applies to our original 3-CNF. Instead of proving a lower bound on the size of the refutation, we will prove a stronger bound on the refutation of the system

Ax = b.

We are going to prove a lower bound on a random system of linear equations over GF (2) with

variables and

m = 10n

restrictions, when each restriction is composed of three variables.

Our system will be represented by a matrix

Ax = b.

We will show that we can assume there

are a constant number of 1's in each column of the matrix.

Ax = b be a system of equations so that A is an (r, )-boundary expander with > 0 and r = (n). A matrix Amn over GF (2) is a (r, )-boundary expander if I [m] , |I| r : |I| |I| where I is the set of all columns j [n] such that there exists exactly one row i I , that hold Ai,j = 1.
Let

Lemma 11.1.

For every I [m], |I| < r, it holds that AI x = bI is satisable, where AI is the matrix with A's rows indexed by I . Proof. Assume the contrary. Then there exists a nonzero vector v GF (2)|I| such that v T AI = 0 but v T bI = 1. Now consider I = supp (v). Then |I | = 0, but |I | < |I| < r and therefore by expansion, |I | > |I | and in particular is nonzero, contradiction.
We now want to select a set of random variables, such that every random restriction on these variables will not refute any small set of restrictions, and therefore conserves the expansion property.

111

Given

J [n]

a set of variables, we dene

(the closure of

J )

and

IJ

via the following

algorithm:

Algorithm 1 Denition of the closure of J

J = J , IJ = while I [m] \IJ s.t. 0 < |I | < IJ IJ I J J supp (I )
Initialization:

r 2 and

I \J <

|I |

do

end while return IJ , J

Motivation:

IJ

is the set of lines we are going to remove from our system. These are the

lines whose both size and boundary are small. We show the following things: 1. If

|J|

is small, so does

and

IJ

2. If we remove

IJ

and

from

A,

it will remain a good expander. Although it might be

a worse expander than

was, but still a fairly good one.

Assuming the aforementioned, we now have a way to choose a random restriction in a way that will leave us with a formula that is still hard to prove. Denote obtained by removing from whose indices are in

A\J

the matrix that is

all rows of

whose indices are in

IJ

and all columns of

J.
r 2, 2

Lemma 11.2.
Proof.
and

If A\J is nonempty then A\J is

2

-boundary expander.
I [m] \IJ
such that

Assume the contrary. Therefore there exists a set

|I | <

r 2

A\J I

<

|I |.

Consider a column

j I

There is exactly one row

i I

with

Ai,j = 1.

In the matrix

matrix, and therefore

A\J , if j J , then this column doesn't appear at all in the j A\J I . Otherwise, j A\J . Therefore I A\J I J , and /
2

I \ J A\J I \ J <
Contradiction.

|I |is

hold. Therefore

can be unied into

IJ

in Algorithm 1.

Lemma 11.3.
Proof.

If |J| <

r 4

then IJ

|J|

2 IJ > |J|. Considering the last iteration 2 r of Algorithm 1 for which IJ |J| , we have IJ < 2 . In the next step, we nd I r such that |I | < 2 and I \J 2 |I |. Now consider I IJ . Its size is smaller than r r because each set is smaller than 2 . Therefore by expansion I IJ > I IJ . Therefore I IJ \J I IJ |J| > I IJ 2 I IJ = 2 I IJ . But, consider IJ after the last iteration, IJ = IJ I , IJ \J 2 IJ for the following reason: let i1 , . . . , it be a sequence of subsets of [m] which are taken when the algorithm runs
Suppose for the sake of contradiction that

112

IJ = t Il . Note that |Il \ (J supp (l <l Il ))| < |Il | because of the way l=1 2 t the algorithm adds Il 's into IJ . Now, IJ l=1 Il \ (J supp (l <l Il )). Therefore, IJ \J t |Il \ (J supp (l <l Il ))| t |Il | = IJ . Meaning, after the t'th l=1 2 l=1 2 iteration, we have IJ > IJ > 2 |J| (because of expansion). Contradiction.
such that

113

Lecture 12

Random 3-CN F and Res[k] refutations  Part III

January 27th, 2009
Lecturer:

Eli Ben-Sasson

Scribe:

Shahar Papini and Niv Salti

EBS: What have we done in this lecture? Where are we heading? See Lecture 10
for an example of a good set of scribe notes.

Denition 12.1.
Init:

Given

J [n], let J , IJ
and

and

A\J
2

be the output of the following algorithm:

IJ , J J r While (I [m], 0 < |I | 2 IJ IJ I J J N (I ) return IJ , J and A\J , s.t. A\J columns J from A.

I \J <

|I |)

is the minor of

A,

which is a result of removing rows

IJ

and

Lemma 12.2 (2).

Proof.

r If A\J is non-empty then it is a ( 2 , )-boundary expander. 2

(Seen in the previous lecture)

Lemma 12.3 (3).

Proof.

If |J| <

r 4

then IJ <

|J| IJ ,
when

Consider the last time, when we construct

IJ

|J|.

Let

be the

subset of constraints added to iteration exists). Let

IJ I = IJ I

in the next iteration(assuming by contradiction that such . Notice the following observations:

1. By the way we build 2.

IJ , |I \J| < =

|I |. |J| <
r 4 ).

IJ

|J| <

r 4

r 2 (Using the assumption that

|I | < r . Since the graph is (r, )-boundary expanding, r we get that |I | |I |, so, |I \J| |I ||J|. Since |I | > (that's how we chose IJ ), 2 we get that: |J| < 2 |I |. Therefore, |I \J| > |I | 2 |I | = 2 |I |. Contradiction!

|I | <

r 2 (by the algorithm), therefore,

Lemma 12.4 (4).

Proof.

r For all J [n], I [m], |I | < 2 , if I J , then I IJ .

Otherwise, we could've added For a term

to

IJ ,

by the algorithm.

Denition 12.5.

(a conjunction of literals), let

denote

IV ars(t) ,

and

V (t) =

V ars(t). t is called locally consistent, if t (At x = bt ) consistent, if every term in it is locally consistent.

is satisable.

A DNF is

locally

Lemma 12.6

If t is locally consistent, then I [m], |I| < t (AI x = bI ) is satisable.

(5) 121

r 2,

then the constraint

Proof.

Otherwise,

I I ,

s.t.

I V (t).

But, by Lemma 4,

I t.

Since

is locally

consistent, we get a contradiction!

Lemma 12.7 (6). If A is (r, ) expanding, and is a Res[k] refutation of Ax = b, then there exists a refutation = 0 1 , s.t.:
1. 0 is a resolution derivation of width Proof.
6k

2. 1 is smaller than k ||(|1 | k ||), and every line is 1 is locally consistent.

For any term Using Lemma 3,

t, |t| k , that is not locally consistent, t (At x = bt ) is unsatisable. 2 t k . So, V ars(t) 6k , since each line in t has three 1 s at most. t (At x = bt ) = 0 (At x = bt ) |= t

t from At x = bt . 0 is the union of all such derivations. In particular, consider Axiom clause (Ax = b). This axiom is a 1-DNF, if any term of it, i.e., any literal l Axiom, is not locally consistent, resolve l 0 with Axiom, and add to 1 only those axioms that are locally consistent. 1 follows , but whenever an axiom is mentioned in , we put in 1 the locally consistent version of it, and whenever a locally inconsistent term is introduced in :
by the implicational completeness of

Res,

there is some derivation of the clause

l1 l2 lk 0 D
We achieved this expression using inconsistent. Thus,

Dl1 Dl2 Dlk D(l1 l2 lk )

resolution steps. Notice that l1

l2 lk

is locally

|1 | k || A,b

and all lines in

are locally consistent.

Denition 12.8.
1. Pick 2. Let

Let

be a restriction obtained from:

X V ars

at random, s.t.

|X| =

r 4 .

IX

be the closure of

X,

and

X = V ars(IX ). AIX x = bIx ,

and set

3. Pick a random satisfying assignment to

A,b

according to it.

Note.

by Lemma 3,

X < r,

so by Lemma 1(from the previous lecture),

AIX x = bIx

is

satisable.

Theorem 12.9
Note.
If

(Switching Lemma)

If R is a set of restrictions s.t. for every k -DNF D,

d 2k

s.t. P rR [D|p = 1]
1] (1 2k )
Assume
C(D) k

2(C(D)) ,
C(D)2k k

then P rR [d(D|p ) > d] 2

1,

k -DNF D. 2k .
So,

is completely random, each term is set to

with probability

P r[D|p =

<e

= Exp((C(D)))

D = t1 t2 tc over disjoint variables, s.t. i.ti is locally consistent. P rR [t1 (AIx x = bIx ) = 1] 2k , because t1 is locally consistent, so by Lemma 5, t1 (AIx x = bIx ) is satisable. So, t1 (AI x = bI ) is actually an ane subspace of x : AIx x = bIx of x x co-dimension k .
122

Lecture 13

TBD

February 3rd, 2009

Lecturer:

Eli Ben-Sasson

Scribe:

Gadi Aleksandrowicz

13.1 Res[k] - Conclusion

We have seen proof that for a random length in

3CN F

there has to be a proof with super-polynomial

Res[k]. AC0 .
AC0
as well.

There are two ways of proving lower bounds for constant-depth circuits: 1. Switching Lemma - shows lower bounds for

2. Approximation Method - Gives lower bounds to We used the switching lemma in order to hit a

DN F

and transform it into a

CN F

with

small width. This transforms short proofs to proofs with small width in Resolution. What about circuits of depth 2? Unknown.

13.2 Frege
After one use of the switching lemma, the system becomes

Ax = b that was (r, ) boundary expander

AB,AC ) cannot BC

(r/2, /2)

boundary expander.

It is known that for

AC0 F rege (the main rule of Frege, Modes Ponens: y

of holes, such that

prove (in short proof ) the pigeonhole principal (this was shown by Ajtai). We pick at random

pigeons and a set

|x| = |y|,

and we put the selected pigeons at the

selected holes. We remain with PHP on a smaller set. We now turn to Bounded Frege. We consider the parity function prove that a circuit functions

g : {0, 1}n {0, 1}.

We

of depth

10 cannot compute parity by considering the and size n

fC , g

after hitting them with a restriction

: fC |

becomes simple, whereas

g|

remains complex - a contradiction. We deal similarly with proofs of PHP by a sequence of circuits and all circuits of depth

d:

we impose restriction

to get a

C0 , . . . , Cs with Cs = P HP sequence fC0 | , . . . , fCs | . The

problem is that some of the functions become constant (tautologies or contradictions). To mend this we change the way the gates in the circuits behave, so the functions won't be constant.

13.3 Resolution
When dealing with the Resolution proof system, we saw the following: 131

Lower bounds via the width method. A connection between the width and the space of a proof. The interpolation method which connected proof length to the complexity of computations. We have shown that a short proofs implies a small circuit for computing interpolants in Resolution or Cutting Planes.

Res[k]

- a lower bound for random

CN F .

13.4 Open Questions

Some open questions concerns the Frege proof system (which is the most likely to turn out to be a super proof system, if one exists): 1. Are there super-polynomial lower bounds to proofs in Frege? 2. Does such a lower bound imply lower bounds on circuit sizes? 3. Can a lower bound to Frege be given assuming a natural and interesting hardness conjecture? Let us concentrate on a specic case of the third question. First we introduce the NisanWigderson pseudorandom generator. In general, a pseudorandom generator is a function

G:

{0, 1} {0, 1} with n < m whose output's distribution (given the uniform distribution n m on {0, 1} as input) is indistinguishable from the uniform distribution on {0, 1} (there is,
of course, a formal denition which cannot be given now).

f : {0, 1}n {0, 1} (i.e. there is no small circuit computing f ). Suppose we have m sets I1 , . . . , IM with |Ij | = n for each set and |Ij Ik | < log n, and dene G(x) = f (x|I1 ) . . . f (x|Im ). This way every bit is calculated in
The NW-Generator is based on a hard function a complicated way, and there are only small dependencies between the dierent bits. Note that if G is a generator, i.e. have

m > n,

then for at least half of the string

b {0, 1}m

we

b Im(G) /

A conjecture by Razborov states that if and this

f is hard, and G is an NW-Generator built from f , b Im(G), then the contradiction G(x) = b is hard to prove in Frege. It is known that / claim is true for Resolution and for Res[ log n].

132

Homework Assignment 1
Denition 1
(Cutting Planes)

Published on 4.12.2008 Due by 16.12.2008

Cutting Planes (CP) is the name of a sequential proof n system that relies on geometry of polytopes in R . A line of a CP proof is a half-space n , i.e., an inequality over variables x , . . . , x , with integer coecients. It has the form in R 1 n n ai xi b, where a1 , . . . , an , b Z. The CP system has the following two inference rules i=1 which are called addition and division, respectively.
n n i=1 ai xi i=1 ai xi b, n (ai + ai )xi b + b i=1

n i=1 ai xi

where

n i=1 ai xi

c, a1 , . . . , an , b Z . b/c n+1

Problem 1.
pigeons and

Let

PHPn+1 n

denote the pigeonhole principle, as dened in class, with

n+1 pigeonholes. Prove: LCP (PHPn

0) =

nO(1) .

Our next question deals with the

Buss-Pudlk (BP) proof system of Pudlk and Buss [1994].

Suppose

This system can be described as a two-player game and we shall call the players Pavel and Sam (in honor of Pavel Pudlk and Sam Buss).

is an unsatisable boolean

formula over NOT, AND, OR gates (each gate has fan-in at most 2). Suppose Sam were to claim The circuit

is satised by an assignment

.

How could Pavel prove that Sam is

lying? One way would be to take Sam to court, and the BP-game is a formal protocol for conducting the trial and convicting Sam of lying. The game proceeds in rounds. On the some Boolean formula

ith

round Pavel presents a question which is simply

Ci ,

Sam replies with a bit

bi ,

and the court records for later use both

Ci

Ci on ? and Sam's th round with Pavel asking C (the circuit response is his reply. The game starts on the 0
and We interpret as the question What is the value of that we want to prove is unsatisable) and Sam responding with

bi .

Ci

1,

and the court records

C() = 1. The game continues in rounds as described above and ends when an immediate contradiction is reached. An immediate contradiction for the NOT gate (denoted ) is a pair of indices i, j such that Cj is the circuit Ci and bj = bi . An immediate contradiction of a AND gate is a triple i, j, k such that Ck is Ci Cj and bk = bi bj . (An immediate contradiction of a
both question and answer. This rst round commits Sam to his false claim that OR gate is similarly dened). To sum, a BP-proof of the circuit questions, edges are labeled by

C is a binary {0, 1} and each leaf

tree whose nodes are labeled by Pavel's is labeled by an immediate contradiction

that emerges from the sequence of questions and answers labeling the path that leads from the root to . The length of a BP-proof is the size of its tree and the BP-length of proving

C,

denoted

LBP (C
Prove:

0),

is the minimal size of such a BP-proof tree.

Problem 2.
LPB (C

LPB (PHPn+1 n

0) = nO(1) .

We end by noting that it is not known if the BP-system is cannot rule out the possibility that for

super.

In other words, we

every

unsatisable formula

of size

we have

0) =

nO(1) .
HW1

Simplication and hint:

The simplied question allows Sam to present as a query any

boolean circuit, i.e., Sam's questions are not limited to formulas. The hint is to use circuits

C t with t inputs and t + 1 outputs, where on input {0, 1}t the ith output of C t is 1 i has i 1's, i.e., the circuit C t counts the number of 1s in . The circuit C t should t/2 . Use these circuits to count various subsets of the be constructed inductively from C n+1 variables of PHPn .
The original proof of this upper bound works even when Sam is limited to asking only formulas and appears in Buss [1987].

HW2

Homework Assignment 2
Problem 3.

Prove a super polynomial lower bound on

Published on 23.12.2008 Due by 6.1.2009

LR (PHPn+1 n

0).

Hints:

((n), d/2)-unique neighbor expander graph with n+1 left vertices, n right vertices and out-degree d on the left hand side, where d is a suciently large constant (say, d = 10).
Use the existence of a bipartite Consider the restriction of

PHPn+1 n

induced by

where pigeon

is allowed to go only

to holes indexed by the neighbors of

in

G.

Dene a suitable width measure as done in class and show every refutation of the pigeonhole principle, restricted to

G,

must include a wide clause.

Problem 4.
in Lecture 7.

Prove: If

is an unsatisable

k -CNF, then space(F

0) W(F

0) k + 1.

Hint: use the combinatorial characterization of width of Atserias and Dalmau [2008], proved

Problem 5. The following problem is an important open question.

F be an unsatisable k -CNF over n variables (think of k as a constant and n k ). Suppose LR (F 0) nO(1) . By the size-width relation Theorem 4.5 we have W(F 0) O( n log n). Is there a O(1) and width O( n log n) simultaneously? resolution refutation of F of length n
Let If

The analogous question for treelike resolution, which is also open, is the following.

LT R (F O(log n)

0)

nO(1) then by the size-width relation we have

of

W(F

there a (not necessarily treelike) resolution refutation simultaneously?

of length

0) O(log n). Is nO(1) and width

If one seeks a treelike proof of length

nO(1)

and width

O(log n)

simultaneously, the answer

is negative. More to the point, there exist such that has

W(F 0) = O(1) L() W() n/ log n.

and

LT R (F

3-CNF formulas with n variables, O(n) clauses 0) = O(n) but for every treelike refutation one

See Ben-Sasson [2002] for details.

HW3

Homework Assignment 3
Problem 6.

Published on 8.1.2009 Due by 24.1.2009

Prove that cutting planes has the feasible interpolation property. Let

be disjoint sets of variables and let

A(x, y) B(x, z)

is unsatisable.

such that for all assignments

to

A(x, y), B(x, z) be An interpolant for x we have

and

x, y, z a pair of CNFs such that F (x, y, z) = F is a boolean circuit over inputs x

C() = 0 A(, y) SAT

Let

C() = 1 B(, z) SAT.

(the size of

s(F ) be the minimal size of an interpolant for F s(F ) (LCP (F

is the number of Boolean

gates in it). Prove:

0))O(1) .

Hints:

Use the result of Buss and Clote [1996] which says that if of length

has a CP-refutation

S,

then

has a CP-refutation of length

S O(1)

such that every coecient

appearing in the refutation can be represented using at most

S O(1)

bits.

Let

be a CP-refutation of

two lines, one belonging respectively.

F . Take each line in the proof and construct from it to A and the other to B . Call these sets of lines A , B

Argue that for every assignment

to

x,

at least one

A | , B | S O(1)

is a CP-refutation of

A(, y)

or

B(, y).
that decides which of

Conclude by arguing that there exists a circuit of size

A , B

is a legal refutation (of

A, B ,

respectively). Let

Problem 7.
T

Prove the following claim which was used in Lecture 9.

T1 , . . . , Ts

and

be decision trees of depth at most

each.

Let

F1 , . . . , Fs , F

be their corresponding

representations as If In words, if

d-CNFs.

The claim to be proved is: then

{T1 , . . . , Ts } |= T ,

W ({C1 , . . . , Cs } T

C ) (s + 1)d. C
can be derived from

{T1 , . . . , Ts } T
from

semantically imply

then all clauses of

C1 . . . Cs
decision tree

via a resolution derivation of width at most

(s + 1)d.

Hints: Construct one big

{T1 , . . . , Ts }

and argue the claim for the case of

T |= T

HW4

References
1. Albert Atserias and Victor Dalmau. A combinatorical characterization of resolution width. of Computer and System Sciences, 74(3):323334, May 2008. 2. Beame, Impagliazzo, Pitassi, and Segerlind. Formula caching in DPLL. In Colloquium on Computational Complexity, technical reports, 2006. 3. Eli Ben-Sasson. Size space tradeos for resolution. In Proceedings of the 34th sium on Theory of Computing (STOC '02), pages 457464, May 2002.
Journal

ECCCTR: Electronic

Annual ACM Sympo-

4. Eli Ben-Sasson, Russell Impagliazzo, and Avi Wigderson. Near optimal separation of treelike and general resolution. Combinatorica, 24(4):585603, September 2004. 5. Eli Ben-Sasson and Avi Wigderson. Short proofs are narrowresolution made simple. the ACM, 48(2):149169, March 2001. 6. Blake. Canonical expressions in boolean algebra. plexity (ECCC), 4(20), 1997.
Journal of

Electronic Colloquium on Computational Com-

7. Samuel R. Buss. Polynomial size proofs of the propositional pigeonhole principle. (4):916927, 1987.

J. Symb. Log

, 52

8. Samuel R. Buss and Peter Clote. Cutting planes, connectivity, and threshold logic, March 06 1996. URL http://citeseer.ist.psu.edu/678454.html;http://euclid.ucsd.edu/~sbuss/ ResearchWeb/cuttingplanes/paper.ps. 9. Vaek Chvtal and Endre Szemerdi. Many hard examples for resolution. (4):759768, October 1988.
Journal of the ACM

, 35

10. Stephen A. Cook and Robert A. Reckhow. The relative eciency of propositional proof systems. Symb. Log, 44(1):3650, 1979.

J.

11. Martin Davis, George Logemann, and Donald Loveland. A machine program for theorem proving. Communications of the ACM, 5(7):394397, July 1962. 12. Martin Davis and Hilary Putnam. A computing procedure for quantication theory. ACM, 7(3):201215, 1960. 13. Armin Haken. The intractability of resolution. August 1985.
Theoretical Computer Science Journal of the

, 39(2-3):297308, , 24

14. John Hopcroft, Wolfgang Paul, and Leslie Valiant. On time versus space. (2):332337, April 1977.

Journal of the ACM

15. Pavel Pudlk. Lower bounds for resolution and cutting plane proofs and monotone computations. Journal of Symbolic Logic, 62(3):981998, September 1997. 16. Pavel Pudlk and Samuel R. Buss. How to lie without being (easily) convicted and the length of proofs in propositional calculus. In Leszek Pacholski and Jerzy Tiuryn, editors, CSL, volume 933 of Lecture Notes in Computer Science, pages 151162. Springer, 1994. ISBN 3-540-60017-5.

17. Ran Raz. Resolution lower bounds for the weak pigeonhole principle. 115138, 2004.

Journal of the ACM

, 51(2):

18. Alexander A. Razborov. Resolution lower bounds for the weak functional pigeonhole principle. Theoretical Computer Science, 1(303):233243, June 2003. 19. Grigori Tseitin. On the complexity of derivation in propositional calculus. In A. O. Silenko, editor, Structures in Constructive Mathematics and Mathematical Logic, Part II, pages 115125. Consultants Bureau, New York-London, 1968. 20. Alasdair Urquhart. Hard examples for resolution. 1987.
Journal of the ACM

, 34(1):209219, January