Constraint Satisfaction Problems

[Read chapter 5 of Russell & Norvig]

Homework #2: Formulate the Following Problem as a CSP

Given a set of n data points and a distance function d(xi, xj) between all pairs of points. Partition the data set into a two subsets so that the maximum diameter of the two subsets is minimized.

Constraint satisfaction problems (CSPs)

Simple example of a formal representation language Later we'll define much more complex languages in PL and FOL. Trade-off b/w complexity and solvability Standard search problem: state is a "black box any data
structure that supports successor function and goal test state is defined by variables Xi with values from domain Di goal test is a set of constraints specifying allowable combinations of values for subsets of variables

Focus on useful general-purpose algorithms with more power than standard search algorithms (BFS, DFS) and don't require user generated heuristics like A*.

Example: Map-Coloring

Variables WA, NT, Q, NSW, V, SA, T Domains Di = {red,green,blue} Constraints: adjacent regions must have different colors e.g., WA NT

Example: Map-Coloring

Solutions are complete and consistent assignments e.g., WA = red, NT = green, Q = red, NSW = green,V = red,SA = blue,T = green

Constraint graph
Binary CSP: each constraint relates two variables Constraint graph: nodes are variables, arcs are constraints

Varieties of CSPs
Discrete variables
finite domains:
n variables, domain size d O(dn) complete assignments e e.g., Boolean CSPs, incl. Boolean satisfiability (NP-complete)

infinite domains:
integers, strings, etc. e.g., job scheduling, variables are start/end days for each job need a constraint language, e.g., StartJob1 + 5 StartJob3

Continuous variables
e.g., start/end times for Hubble Space Telescope observations linear constraints solvable in polynomial time by LP

Varieties of constraints
Unary constraints involve a single variable,
e.g., SA green

Binary constraints involve pairs of variables,

e.g., SA WA

Higher-order constraints involve 3 or more variables,

Eventually we'll be able to specify constraints as a sentence in a logic.

Real-world CSPs
Assignment problems
e.g., who teaches what class

Timetabling problems
e.g., which class is offered when and where?

Transportation scheduling Factory scheduling Notice that many real-world problems involve real-valued variables Related techniques involving constraints? Why can't we use it for our homework problem?

Standard search formulation (incremental)

Let's start with the straightforward approach, then fix it States are defined by the values assigned so far Initial state: the empty assignment { } Successor function: assign a value to an unassigned variable that does not conflict with current assignment
fail if no legal assignments

Goal test: the current assignment is complete This is the same for all CSPs Every solution appears at depth n with n variables use depth-first search Path is irrelevant, so can also use complete-state formulation b = (n - l )d at depth l, hence n! dn leaves

Backtracking search
Variable assignments are commutative, i.e.,
[ WA = red then NT = green ] same as [ NT = green then WA = red ]

=> Only need to consider assignments to a single variable at each node Depth-first search for CSPs with single-variable assignments is called backtracking search Can solve n-queens for n 25

Backtracking search Baseline Comparative Algorithm. Can solve n-Queens for n=25

Behavior of this algorithm for n=4 n-queens problem

14

Backtracking example

Backtracking example

Backtracking example

Backtracking example

What are the possible improvements to basic BT search? What guiding principles/aims can we use to help answer this question?

Improving backtracking efficiency

General-purpose methods can give huge gains in speed:
Which variable should be assigned next? In what order should its values be tried? Can we detect inevitable failure early?

When Building The Tree How Should We Order Variables?

What should we assign next, SA, QLD, NSW, VIC, TAS? Why?

Most constrained variable

Most constrained variable:
choose the variable with the fewest legal values

a.k.a. minimum remaining values (MRV) heuristic

Will the MRV Heuristic Help Us Choose the Root of the Tree?
If not, whats a reasonable method to choose and why?

Most constraining variable

Tie-breaker among most constrained variables Most constraining variable:
choose the variable with the most constraints on remaining variables

Okay. We How About Choosing What Values to Explore First?

If we are expanding QLD which value should we instantiate first?

Least constraining value

Given a variable, choose the least constraining value:
the one that rules out the fewest values in the remaining variables

Combining these heuristics makes 1000 queens feasible. How is this philosophically different to our heuristic search with A*

What was nice about A*'s behavior?

So how can we do pruning in this context?

Forward checking
Idea:
Keep track of remaining legal values for unassigned variables Terminate search when any variable has no legal values

Forward checking
Idea:
Keep track of remaining legal values for unassigned variables Terminate search when any variable has no legal values

Forward checking
Idea:
Keep track of remaining legal values for unassigned variables Terminate search when any variable has no legal values

Forward checking
Idea:
Keep track of remaining legal values for unassigned variables Terminate search when any variable has no legal values

Using these heuristics in combination. What node ordering heuristic is a natural partner to forward checking?

Forward Checking is an Example of Constraint Propagation.

Constraint propagation
Forward checking propagates information from assigned to unassigned variables, but doesn't provide early detection for all failures:

Forward checking would still continue searching. But why is there no point and why should we prune this part of the tree?

Constraint propagation
Forward checking propagates information from assigned to unassigned variables, but doesn't provide early detection for all failures:

NT and SA cannot both be blue! Constraint propagation repeatedly enforces constraints locally

In your Class Groups of Three

Use the allocated technique: minimum remaining values, most constrained variable, least constraining value or forward checking. Which is the odd one out and why? Show the generated search tree.

Arc consistency
Simplest form of propagation makes each arc consistent X Y is consistent iff
for every value x of X there is some allowed y

Arc consistency
Simplest form of propagation makes each arc consistent X Y is consistent iff
for every value x of X there is some allowed y

Arc consistency
Simplest form of propagation makes each arc consistent X Y is consistent iff
for every value x of X there is some allowed y

If X loses a value, neighbors of X need to be rechecked

Arc consistency
Simplest form of propagation makes each arc consistent X Y is consistent iff
for every value x of X there is some allowed y

If X loses a value, neighbors of X need to be rechecked Arc consistency detects failure earlier than forward checking Can be run as a preprocessor or after each assignment

So what if we find a nodes xi and xj are not arc consistent what should we do?

Arc consistency algorithm AC-3

Time complexity: O(n2d3)

Checking consistency of an arc is O(d2)

A Simple Example???

In your class groups use a CSP formulation to solve this problem. Given a set of n data points and a distance function d(xi, xj) between all pairs of points. Partition the data set into a two subsets so that the maximum diameter of the two subsets is minimized. Hint: Construct a simple example to verify your algorithm. Is the problem solvable in polynomial time with respect to the number of instances?

Core Decision Problem: Boottleneck Diameter Problem (BDP):

Given: Point set P = \{p_1, p_2, ..., p_n\}, the distance d(p_i, p_j) for each pair of points, a distance \alpha. Question: Is it possible to partition P into two sets P_1 and P_2 such that dia(P_1) <= \alpha and dia(P_2) <= \alpha? We can use 2SAT to solve BDP. For each point p_i we have a Boolean variable x_i. (The interpretation is that all points corresponding to variables set to True form one cluster and the remaining points form the other cluster.) The clauses of the 2SAT instance are constructed as follows. Examine each pair of points p_i and p_j (i \neq j). If d(p_i,p_j) > \alpha, add the clauses

(x_i or x_j) and (\bar{x_i) or \bar{x_j})

Putting It All Together: Sudoku

9 2 3 9 6 5 9 7 3 8 5 2 1 7 8 5 4 1 2 9 3 3 6 5 1 8 4 3 5 9 2 8 1 4

Sudoku
http://www.websudoku.com/ Each Sudoku has a unique solution that can be reached logically without guessing. Enter digits from 1 to 9 into the blank spaces. Every row must contain one of each digit. So must every column, as must every 3x3 square.

Problem Formulation As Search

What is the state space? What is the initial state? What is the successor function? What is the goal test? What is the path cost? Admissible heuristic?

Why does using these search techniques not make sense for this problem?

Problem Formulation As Search

What is the state space? What is the initial state? What is the successor function? What is the goal test? What is the path cost? Admissible heuristic?

Points
Large search space commutativity what about using bfs or dfs?
fixed depth

CSP as a Search Problem

Initial state: empty assignment Successor function: a value is assigned to any unassigned variable, which does not conflict with the currently assigned variables Goal test: the assignment is complete Path cost: irrelevant Branching factor b at the top level is nd. b=(n-l)d at depth l, hence n!dn leaves (only dn complete assignments).
We can do better.

What else is needed?

Not just a successor function and goal test But also a means to propagate the constraints imposed by one move on the others and an early failure test Explicit representation of constraints and constraint manipulation algorithms

Example: Sudoku CSP

variables: domains: constraints: Goal: Assign a value to every variable such that all constraints are satisfied

Example: Sudoku CSP

variables: X11, , X99 domains: {1,,9} constraints:
row constraint: X11 X12, , X11 X19 col constraint: X11 X12, , X11 X19 block constraint: X11 X12, , X11 X33 How do we encode existing board values as constraints?

Goal: Assign a value to every variable such that all constraints are satisfied

Great. Now Do Another Formulation using Propositional Logic

s_{xyz} is the proposition that tile location x,y value is z Why is it so important to state our problem in propositional or first order calculus?

Great. Now Do Another Formulation using Propositional Logic

Which Heuristic Additions to BT Search Should We Use and Why?

Use the heuristic allocated to you.
Most Restricted Variable? Most Constrained Node? Lease Constraining value Forward checking? Arc Consistency?

Name a sufficient condition when it will work Does this condition occur with the Sudoku formulation?

Other techniques for CSPs

k-consistency
Tradeoff between propagation and branching

Symmetry breaking

Structured CSPs

Tree-structured CSPs Removes The Need For Back-tracking

Algorithm for tree-structured CSPs

Nearly tree-structured CSPs

(Finding the minimum cutset is NP-complete.)