Solving Sudoku using Constrain Satisfaction Techniques

Mike O Brien, John. B. Johnson, Vamseedhar Chennareddy M. Afzal Upal Electrical Engineering & Computer Science Department University of Toledo, Toledo, OH
Abstract: Sudoku is a simple number puzzle that has achieved international popularity in the recent past. This project examines solving Sudoku puzzles as Constraint Satisfaction Problems (CSP). The primary goal of this project is to determine if the symmetry of a puzzle has a correlation with the time taken to solve when using constraint satisfaction programming. Experimental data supports the hypothesis that symmetric puzzles are indeed solved faster than asymmetric puzzles irrespective of the difficulty level modeled as a CSP.

In 1989, Loadstar/Softdisk Publishing published the first prototype of a computerized Sudoku puzzle generator. Later in 1997, retired Hong Kong judge Wayne Gould, worked over 6 years and developed a computer program to produce puzzles quickly. Though there have been many algorithms developed since then to generate and solve Sudoku, it still offers an exciting artificial intelligence challenge as there is no particular efficient way to solve all the puzzles. [2] Sudoku being puzzle with only one unique solution, while solving Sudoku there is always a finite number of integer entries until puzzle completion. So this places Sudoku in the difficult to solve NP-complete class that includes problems of great practical importance, such as scheduling, network routing, and gene sequencing. [3] As a member of the NP-complete subset, Sudoku is an ideal tool for investigating the whole class of NP problems: an efficient algorithm for any NP-complete problemthe toughest of NP problemsautomatically provides an efficient algorithm for solving all. [3] Although most experts believe that no such algorithm exists, they continually search for improved algorithms that provide shorter, if not the very shortest, paths to solutions. [4] The three major kinds of symmetry used in generating Sudokus are rotation, reflection and transposition. Symmetry is defined as an operation that translates a valid Sudoku grid into another valid Sudoku grid with the same number of given [10]. For the purposes of our experiment we have used reflection ( turn rotational) symmetry to generate puzzles. We also followed a general rule of generating hard puzzles with 20-25 set of givens and easy puzzles with 35-40 set of givens [10]. The primary goal of this project is to determine if the symmetry of puzzles with varying difficulty levels have a correlation with the time taken to solve when using constraint satisfaction programming.

1. Introduction & Background Sudoku is a reasoning and logic puzzle, designed by American Howard Garns, a retired architect and freelance puzzle constructor. It was first published in New York in 1979 by Dell Magazines under the title Number Place, but it began to acquire fame only after it was introduced in Japanese monthly paper, Nikolist in April 1984 as Suuji wa dokushin ni kagiru (which means single or the numbers must occur only once), abbreviated as Sudoku [1]. The aim of the puzzle is to enter an integer from 1 through 9 in each of the 81 cells, of a 99 grid made up of 33 regions, starting with an initial playing field of anywhere from 25-40 cells filled in. The only rules are that each row, column, and region must contain only one instance of each numeral. [2]

Figure 1 - A Sample Sudoku

2. Proposed Solution Constraint Satisfaction Programming (CSP) has evolved into an ambitious technology being used in various walks of life, such as mortgage payments, combinational locks, for short-term staff planning etc. It has become indispensable in the field of Artificial Intelligence (AI) as it points a future AI, where the user states a problem and the computer solves it1. As Sudoku is inherently a constraint based problem, this project is using CBP to evaluate its efficiency in solving Sudoku. Constraint satisfaction problems are defined by three items: 1) a finite set of variables 2) a function that maps each variable to a finite domain 3) a finite set of constraints. For our puzzle the variables are the integers 1 9 and the constraints on these variables are similar to the constraints of the well known N-queens puzzle or the Latin squares puzzles. [5] This can be considered a 9*9 grid where no two same integers can be in the same row, same column or the same sub-grid. Generation Strategy: We developed a puzzle generator to produce the various puzzle starting grids to test our CSP algorithm The CSP algorithm inputs a value for an open location on the grid and checks it validity based on the given constraints. By repeatedly choosing a value for another variable, it incrementally instantiate variables and extend a partial solution that specifies consistent values for some of the variables toward a complete solution using a smart backtracking algorithm. [See appendix for pseudo code] Solving Strategy: A list of possible numbers for each cell of the 9x9 board is stored and the values for each square that conflict with the initial configuration of the game board are crossed out. Then, for each square, S, in the board, algorithm will examine each value in its list of possible candidates, V. If assignment of any one of those values, V[i], to that square, S forces all possible candidates for another square to be eliminated, and later cross V[i] out from the list of possible candidates for S. During propagation, if any square's list is reduced to only one value, that value is assigned to that square. The puzzle is solved when every square has only one remaining possibility. If during propagation no more values can be eliminated, meaning that there are no obvious moves, we employ a forward-looking backtracking search. [See appendix for pseudo code]. This algorithm is able to back up or undo previously made decisions. To ensure the backtracking algorithm is complete, the solver maintains a list of every combination of decisions that led to an error. The algorithm then looks ahead before future decisions to

make sure that applying the next decision will not create a state where the current decision set matches one previous eliminated from the search space. In this way, the commutative property of constraint satisfaction problems reduces the number of decision paths that must be stored. 3. Evaluation An empirical evaluation of the proposed hypothesis involves executing the algorithm on several test puzzles and recording the time taken to compute each solution. The selected problem set is an important factor affecting the validity of the results of this experiment. Because we are attempting to expose the effects of symmetry, it is important that the difficulties in each problem set are as uniform as possible. All of the hard problems and all of the easy problems were generated from our puzzle generated algorithm and then loaded into puzzle solver with a timer. From each set of Easy Symmetric (ES), Easy Asymmetric (EA), Hard Symmetric (HS) and Hard Asymmetric (HA), a group of thirty randomly generated puzzles were used in the investigation. The constraint-based backtracking approach is a simple approach to finding a solution to any Sudoku (provided a solution exists). Even though the search tree could have up to 981 leaves [6], the algorithm can still find the solution relatively quickly.

Times for Solving Easy Sudokus

Times for Solving 4 different kind of Sudoku Puzzles in 30 test runs

Time Taken in Seconds

0.6

0.5 0.4 0.3 0.2

ES
EA

ES 0.02 0.01 0.01 0.02 0.02 0.01 0.01 0.01 0.13 0.1 0.02 0.04 0.01 0.02 0.02 0.01 0.01 0.01 0.04 0.1 0.02 0.01 0.01 0.02 0.02 0.01 0.01 0.01 0.01 0.1

EA 0.25 0.291 0.391 0.49 0.07 0.07 0.01 0.18 0.01 0.01 0.25 0.291 0.391 0.49 0.07 0.07 0.01 0.18 0.01 0.01 0.25 0.05 0.05 0.49 0.07 0.07 0.01 0.18 0.01 0.01

HS 0.07 0.13 0.17 0.16 0.07 0.21 0.101 0.01 0.01 0.01 0.07 0.13 0.17 0.16 0.07 0.21 0.016 0.01 0.01 0.01 0.07 0.13 0.17 0.16 0.07 0.21 0.101 0.01 0.01 0.01

HA 0.61 0.461 0.49 0.131 0.011 0.16 0.02 0.01 0.201 0.321 0.541 0.461 0.62 0.131 0.011 0.16 0.02 0.01 0.201 0.321 0.541 0.461 0.49 0.131 0.34 0.16 0.02 0.01 0.36 0.321

0.1 0 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 Number of Runs

Figure 2 - Times for Solving Easy Sudokus Because so many clues are given, the number of incorrect decisions and consequently the time spent backtracking by the solving algorithm is greatly reduced. For the easiest class of Sudoku, those containing symmetry and more than 38 givens, solving is made easy because of the obvious assignments present at each step of the search. This trend is evidenced in Figure 2 as the easy symmetric puzzles we all solved quickly.
Times for Solving Hard Sudokus
0.7 0.6

Time Taken in Seconds

0.5 0.4 0.3 0.2 0.1 0 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 Number of Runs

HS HA

Figure 3 - Times for Solving Hard Sudokus When solving more difficult puzzles, symmetry plays less of a role because the algorithm is bound to perform quite a bit of decision-making and backtracking. Even so, Figure 3 shows the pattern that symmetric puzzles are more easily solved. 4. Analysis When starting grid cells are placed in a symmetric pattern, our data supports that the number of steps taken by the solver is reduced substantially to make the average time to compute a solution approximately 80% faster for easy puzzles and approximately 60% faster for hard puzzles.

Table 1 - Times for Solving Sudoku

The variance in solution times for the same type of puzzles is attributed to the frequency and position of starting numbers filled in on the initial grid. When solving easy Sudoku puzzles with constraint programming, much of the guesswork is eliminated.

Total Time Taken to Solve 30 Different Sudokus of Each Kind

Hard Assymetric 8 7 6

Our work is unique in a sense because we are concerned specifically with the effects of the symmetry of the grid when solving Sudoku which is modeled as a CSP.
6. Conclusion and Future Work The motive of this project has been to model Sudoku as a CSP and to evaluate the correlation between the symmetry and time required to solve the puzzle. As expected our experimental results indicate that symmetry indeed does have an effect on the solution time. In Figure 4, the graph indicates the total time taken to solve Sudokus of two degrees of difficulty for a sample of ten test runs. The reason asymmetric puzzles take substantially longer time to solve when compared to symmetric puzzles is that, when modeled as a CSP there is more information available in symmetric puzzles which makes the solving more efficient. Two distinct modifications to this experiment have been identified that would be a promising avenue of future research. Computer results are always met with some degree of suspicion. So, firstly to increase the credibility of our results number of test runs should increase to 1000015000, difficulty levels should be increased to a minimum of five and symmetry should include vertical, horizontal, diagonal, and x-wing symmetries. The results of such an experiment would definitely advance the field of AI because it can effectively teach us more about the correlation between symmetry and the complexity of solving a Sudoku puzzle. Secondly, to further understand the intricacies of modeling a problem as a CSP, other problem domains should be evaluated to determine its overall performance against other strategies in the same domain. In particular we were plan to used WalkSAT algorithms [11], where we would start randomly assigning values in the solution grid and if the assignment satisfies all clauses, the algorithm stops and returns the assignment. Otherwise, the value of a variable is changed, and the process is repeated. 7. Bibliography [1] Apt, K.: Principles of Constraint Programming. Cambridge University Press (2003) [2] Van Hoeve, W.: The all different constraint: A survey. sherry.ifi.unizh.ch/ article/vanhoeve01alldifferent.html (2001)

Easy Assymetric
5 Time Taken 4 3 2

Hard Symmetric

Easy Symmetric
1 0

Figure 4 - Times for solving various Sudokus Though from the above figure, it is not intuitive, these results make sense when considering the nature of the problem. Puzzles that are symmetric have a much more evenly distributed set of givens. Givens in the same row, column, or region provide us with more constraints than those that are not, and symmetric grids create these pairs of givens with a greater frequency. Therefore, symmetric puzzles can be solved in less time when modeled as a CSP. 5. Related Work The work undertaken in this project is not particularly related to research of any specific individuals, but it does draw upon the principles of constraint programming and NP complete problems. All the literature sources used in this project have been referenced in the bibliography. The work of this project is similar to the research work of many others because we are trying to solve a large combinatorial problem (Sudoku) using CSP, whereas others have tried to solve large combinatorial problems such as n-queens, the five house puzzle, the progressive party problem, minesweeper [7] and more recently the social golfer problem using CSP. Our work is also a bit similar to the work of Jarvis and Falgenhauer [12], except they identified how many distinct symmetric and asymmetric solution grids could be enumerated, whereas we were trying to find out the relation of symmetry with the puzzle solving time. There is a lot of research available on the mathematics of Sudoku. Although it may seem like a trivial game, there are advanced solving techniques that require the player to simultaneously analyze many constraints at once. In this sense, Sudoku trains players to observe more logical rules at once. This is highly desirable in todays fast-paced world. As the ability to multi-tasking is increasingly important, exercises to increase the parallel processing capabilities of the human mind become more widely appreciated.

[3]. Dincbas, M., Simonis, H., Van Hentenryck, P.: Solving large combinatorial problems in logic programming. J. Log. Program. 8 (1990) 7593 [4] Harvey, W.: The fully social golfer problem. In: SymCon03: Third International Workshop on Symmetry in Constraint Satisfaction Problems. (2003) 7585 [5] Lauriere, J.: A language and a program for stating and solving combinatorial problems. Artificial Intelligence 10 (1978) 29127 [6] Regin, J.C.: A filtering algorithm for constraints of difference in CSPs. In: AAAI. (1994) 362367 [7] Stergiou, K., Walsh, T.: The difference all-difference makes. In: IJCAI-99. (1999)

[8] S.F.Bammel, J.Rothstein, The number of 99 Latin squares, Discrete Mathematics 11 (1975) 9395 [9] Wilson, R.: How to solve Sudoku. The Infinite Ideas Company (2005) [10] Mathematics of Sudoku: http://wikipedia.org/Sudoku/ (2003) 11. Bart Selman, Henry Kauntz, Bram Choen, "Local Search Strategies for Satisfiability Testing." (1996) [12] Felgenhauer and Jarvis, Enumerating possible Sudoku grids, :http://www.afjarvis.staff.sherf.ac.uk.sudoku/pdf (2005)