INTELLIGENT TUTORING SYSTEMS IN THE MATHEMATICS CLASSROOM

Intelligent Tutoring Systems: Social Constructivism in the Mathematics Classroom

Matthew W. Long

Boise State University

INTELLIGENT TUTORING SYSTEMS IN THE MATHEMATICS CLASSROOM Abstract

This paper focuses on the application of the theory of social-constructivism and Vygotskys concept of the Zone of Proximal Development in mathematics classrooms. Intelligent tutoring systems are designed around these systems of thought and have been implemented in many mathematics classrooms across the nation. The effectiveness of several of these systems on student learning is investigated and further research questions are proposed.

Keywords: intelligent tutoring systems, tutoring, mathematics, ACT Theory

INTELLIGENT TUTORING SYSTEMS IN THE MATHEMATICS CLASSROOM Intelligent Tutoring Systems: Social Constructivism in the Mathematics Classroom The math classroom is not thought of fondly by many learners. The words drum up the thought of dull, dry classes following the same monotonous routine day after day. Students deal with what is perceived as useless but mandatory (sometimes confusing) material in a variety of

ways. Some students may fear the endless definitions and example problems that make no sense, which build on the feeling of being lost that they have had for the past several classes. Others, eager to just get by, scribble down work for homework problems in an attempt to fool the teacher into believing the problem was attempted. Isnt there something that can be done to make math a little more engaging for the students? Could there be a way for students to ask questions and actually get help when they are completely lost instead of being berated by the teacher for interrupting class? Educational technology offers several options that try to deliver on these attitudes that inhibit learning. Computers, in particular, have been utilized to achieve a wide range of educational goals since their invention. One of the most ambitious of these goals is having the computer serve as a tutor to students. Origin of Intelligent Tutoring Systems These computer-based tutoring systems, known as Intelligent Tutoring Systems (ITSs), have their roots in the Artificial Intelligence (AI) Movement of the 1950s and 1960s. In the 1960s researchers created a number of Computer Assisted Instructional (CAI) systems that could generate sets of problems to enhance student performance in arithmetic and vocabulary recall (Uhr, 1969). These systems were based on an implied behaviorist transmission model that operated on the basis that if learners were presented information, they would be able to absorb it without difficulty. By the late 1960s and early 1970s, research had progressed to consider the learner as a factor in the overall instructional system (Suppes, 1967). Systems were developed

INTELLIGENT TUTORING SYSTEMS IN THE MATHEMATICS CLASSROOM that altered the presentation of new materials based on the history of a students responses. These systems were focused on the development of skills and recall and were effective at that task. Around this time educational psychologists began questioning the assumptions behind behaviorism. Constructivism and Piagets theories of learning began to emerge as a leading learning theory. By the early-1980s, researchers had turned the focus of AI development to

other things such as creating expert systems for troubleshooting and diagnostic systems. A study in 1984 by Benjamin Bloom however revived interest in ITS. Bloom found that human tutoring could produce an average learning gain of two standard deviations compared to conventional classroom instruction. The one-to-one aspect of tutoring was a key element in the gain, but was considered impossible to implement on a large scale due to the costs associated. Designers of CAI realized that computers could provide an economical alternative to human tutoring. Learning Theory Behind ITS Intelligent learning systems are based on the theoretical framework originally developed by Vygotsky (1978). The frameworks foundation is that instruction should be based on continual assessment of the learners Zone of Proximal Development (ZPD: Murray & Arroyo, 2002). Vygotsky defines the ZPD as the distance between the actual development level as determined by independent problem solving and the level of potential development as determined through problem solving under adult guidance or in collaboration with more capable peers (1978, p.86). This theory holds that students will learn the most if given problems that fall in their ZPD, and help is provided in the form of hints or decomposition of the problem into its components. This goes against the radical constructivist view that an attempt to break a competence into its components would result in failed education (Shepard, 1991). Wood &

INTELLIGENT TUTORING SYSTEMS IN THE MATHEMATICS CLASSROOM Wood (1996) later expanded on the nature of the guidance and collaboration that promotes

learning development. Their work identified the aspects of tutoring that allow the progression of responsibility for successful task completion and competence from the tutor to the learner. This framework closely models that of human tutors who, as mentioned earlier, can provide significant learning gains (Bloom, 1984). When students encounter problems that are outside their ZPD, tutors can guide students to the solution by offering hints, posing questions, providing examples, and helping the students discover the solution rather than just providing the answer (Graesser, McNamara & VanLehn, 2005). This process of helping the student achieve their level of potential development is often referred to as scaffolding. This style of instruction allows students to learn new skills while also building their confidence in their ability to conquer demanding problems. The scaling back and eventual removal of scaffolding as students become more successful and confident in their problem solving ability is known as fading (Murray & Arroyo, 2002). Theories of skill acquisition-a part of the learning by doing methodology-are also a major concept on which ITSs are built. One of the most precise and well-developed of these theories is the ACT Theory (Adaptive Control of Thought Theory: Anderson & Gluck, 2001). This theory assumes that human cognition takes place from an interaction between declarative knowledge and procedural knowledge. Declarative knowledge is made up of the acquisition of facts, images, and sounds. Procedural knowledge is an understanding of how to do things. The elements that make up procedural knowledge are known as rules or productions because they specify the conditions under which they are applicable and the actions that result from applying them. Solving a problem correctly is the result of applying the appropriate production rules that retrieve the declarative knowledge needed to solve the problem. Following the assumption that

INTELLIGENT TUTORING SYSTEMS IN THE MATHEMATICS CLASSROOM learning involves acquiring these production rules, computer based cognitive tutors designed by Anderson and others seek to diagnose whether students have acquired the necessary production rules to solve problems. If difficulties are encountered with certain production rules then the cognitive tutors can provide remediation. What are Intelligent Tutoring Systems? How do Intelligent Tutoring Systems Function/Operate?

Intelligent tutoring systems differ from other CAI due to a variation in their components. Freedman (2000) describes the traditional ITS model as having four components: the learning environment, the domain knowledge model, the student model, and the teaching model. The learning environment is where learners interact with the problems they are being asked to solve. Their actions in solving the problems are evaluated by the domain knowledge model to develop an assessment of the students knowledge, which is contained in the student model. The teaching model then helps deliver instruction based on an evaluation of the students actions and the student model. This interaction between the components-which results in individualized instruction for each student-sets it apart from other CAI. ITS software operates in much the same manner as a human tutor does. The availability of scaffolding resources within the software is the essential component of intelligent tutoring systems. The system selects problems that it expects will provide a moderate challenge to the student due to its evaluation of the students production rule weaknesses. By using scaffolding resources, the student should be able to solve increasingly difficult problems in a specific area, eventually demonstrating mastery through independent problem solving. The ITS also uses fading as students proficiency increases.

INTELLIGENT TUTORING SYSTEMS IN THE MATHEMATICS CLASSROOM Examples of ITSs Implemented in Mathematics Classrooms Performance outcomes on standardized tests are being increasingly emphasized in K-12 education, therefore, teachers are unlikely to incorporate technology unless they are confident that it will help their students master the tested material. There have been several ITSs field tested in mathematics classrooms around the United States which include PAT, ASSISTment, and AnimalWatch. PAT PAT (Practical Algebra Tutor) is an ITS based on Algebra 1 material that was first implemented in the mid-1990s in three urban high schools in Pittsburgh, PA. The system was designed to help students develop algebraic skills that could be used in the context of real-world problems (Koedinger, Anderson, Hadley, & Mark, 1997). The tutor used multiple representations of a problem including tables, graphs, and equations. The tutor provided timely, private feedback on errors and provided help on request. The students who used the PAT ITS outperformed their peers by 15% on standardized tests and by 100% on objectives that the PAT curriculum was focused on. Subsequent versions of this ITS, now known as Cognitive Tutor, have been designed as

complete courses. The software now uses the ACT-R (Adaptive Control of Thought Rational) Theory which is a refinement of the ACT Theory (Anderson and Gluck, 2001). ACT-R breaks down small units of knowledge into very brief tasks that only take tenths of a second. New chunks of information are constantly modifying knowledge structures. As knowledge is used, it becomes stronger and can be recalled rapidly; weak knowledge that is not used often can take a longer time to retrieve. The relative strength of knowledge components will determine which is used to accomplish a task, with stronger knowledge components being used more frequently. In

INTELLIGENT TUTORING SYSTEMS IN THE MATHEMATICS CLASSROOM addition, as knowledge is strengthened it becomes more proceduralized, which takes up less working memory. Learning involves the development and strengthening of the appropriate knowledge components. Reasoning is limited by the brains working memory capacity. Education is most efficient when it focuses student learning on the individual knowledge

components that are weakest in their brains (Ritter, Anderson, Koedinger, & Corbett, 2007). The curriculum includes the Cognitive Tutor software, student textbooks, and training materials for the teachers. If all resources are used, then instruction is structured so that the software is used two days a week and the other three days are classroom activities focused on the text. Cognitive Tutor continues to demonstrate its effectiveness with the Algebra course being used regularly by about 500,000 students per year (Koedinger, McLaughlin, & Heffernan, 2010). ASSISTment The ASSISTment system is a statewide ITS used by schools in Massachusetts to help prepare middle school students for the states year-end standardized exams. The system incorporates released test items from previous years exams (Razzaq, Feng, Nuzzo-Jones, Heffernan, Koedinger, Junker, Ritter, Knight, Aniszczyk, Choksey, Livak, Mercado, Turner, Upalekar, Walonoski, Macasek & Rasmussen, 2005). Students use the program 20 minutes a week. The program uses a fine-grained learning model similar to Cognitive Tutor which allows the system to identify specific skill weaknesses. Students receive scaffolding help if they answer the initial test item incorrectly. The scaffolding is made up of hints, which the student requests, and buggy messages, which are tutoring responses to a particular student error. The goal of the program is to provide teachers with an overview of the difficulties that individuals and classes as a whole are having. This feedback guides the teachers to tailor their instruction to focus on these difficulties.

INTELLIGENT TUTORING SYSTEMS IN THE MATHEMATICS CLASSROOM In the pilot year of implementation, evidence suggested that the program results in student learning and is a better predictor of student performance on the state exam than paper

based practice tests (Razzaq et al., 2005). Further studies show significant improvements in yearend exam scores in users of ASSISTment compared to non-users (Koedinger et al., 2010). Results also suggest that a students score and his/her amount of usage are directly correlated. Gains were shown by students whose teachers frequently used the results from ASSISTment to tailor their instruction. The sub-group of special education students received the most benefit from use of the system as they showed the most significant learning gains. The study also shows that these students are immersed in regular education classrooms with their peers at a much higher rate when compared to schools not using ASSISTment. This is probably due to the tutoring systems making immersion easier to implement. Math homework is a nightly occurrence for many students. While teachers think of homework as a chance for students to practice the concepts learned during class independently, students view it in a less positive manner. Learners who mastered the concepts during class view it as a waste of time, while those who werent as successful during class feel frustration as theyre now asked to complete the assignment without help. Mendicino, Razzaq, and Heffernan (2009) conducted a study of fifth grade math students to see if homework delivered via the ASSISTment system would improve learning compared to traditional paper-and-pencil homework. The study showed a large improvement in learning which lends credence to the idea that ITSs can help students learn from homework and not just learn more effectively during instructional time. However, the study was brief in nature, so there is a need for further research on the effect of ITS-for both learning from homework and for its effect on relatively younger learners.

INTELLIGENT TUTORING SYSTEMS IN THE MATHEMATICS CLASSROOM AnimalWatch

10

The last ITS is AnimalWatch. The system was designed to help students master the basic computation and fractions skills found in a pre algebra curriculum (Beal, Arroyo, Cohen, & Woolf, 2010). The system also focuses on word problem solving by using real information about various endangered species. These types of problems are increasingly being used in classrooms as they involve applying mathematical processes to real world situations. This helps counter the thought of many learners that what they are learning in class has no application outside of the classroom. Multiple studies were done using the system, and results indicate that learners who used a combination of human tutoring and the ITS improved their scores from pre to post test just as much as students who worked with a human tutor the whole time. Also, the ITS provided the largest gains in learning for those with weaker initial skills. The results also match that of Koedingers study of the ASSISTment system (2010) in that users with a higher usage of the system had larger gains. These students also used the scaffolding resources available to them more often than their peers who had stronger initial skills. Conclusion Research has shown intelligent tutoring systems to be effective in increasing learning in several different environments. There are a multitude of factors that lead to these learning gains, most of which center on increasing a learners Zone of Proximal Development. These include the scaffolding resources included in each system, instantaneous feedback, breakdown of tasks into knowledge components, and the tracking of students progress which allows tailoring of instruction. The systems are not limited to just increasing the effectiveness of learning during instructional time, but also increasing learning on homework and serving as more accurate

INTELLIGENT TUTORING SYSTEMS IN THE MATHEMATICS CLASSROOM

11

formative benchmark tests for students performance on high stakes tests than paper-and-pencil versions. However, future research should be conducted in several areas related to findings in the completed studies. If these systems are so effective, then why arent they being more widely adopted? (Is it an issue of cost, ignorance of their existence, or some other factor?) As Beal et. al (2010 ) wondered, would a large-scale study show that ITS improves learning from homework? Are the benefits on learning as evident on younger students as those in junior and senior high? Are ITS benefits limited to the content areas of mathematics and science? An overarching question to the entire industry comes from whether or not tutoring is really the most effective strategy. Are there other educational technology products that outperform ITS in student learning gains? The answers to these questions will help to bring about the continual refinement and improvement of mathematics education.

INTELLIGENT TUTORING SYSTEMS IN THE MATHEMATICS CLASSROOM References Anderson, J.R. & Gluck, K. (2001). What role do cognitive architectures play in intelligent

12

tutoring systems? In D. Klahr & S.M. Carver (eds.) Cognition & Instruction: Twenty-five years of progress, 227-262. Erlbaum. Beal, C.R., Arroyo, I.M., Cohen, P.R., & Woolf, B.P. (2010). Evaluation of AnimalWatch: An intelligent tutoring system for arithmetic and fractions. Journal of Interactive Online Learning, 9(1), 64-77. Retrieved from www.ncolr.org/jiol Bloom, B.S. (1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13(6), 4-16. Freedman, R. (2000). What is an intelligent tutoring system? Intelligence, 11(3), 15-16. Graesser, A.C., McNamara, D.S., & VanLehn, K. (2005). Scaffolding deep comprehension strategies through Point&Query, AutoTutor, and iSTART. Educational Psychologist, 40(4), 225-234. Koedinger, K.R., Anderson, J.R., Hadley, W.H., & Mark, M.A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8(1), 30-43. Retrieved from www.iaied.org Koedinger, K., McLaughlin, E., & Heffernan, N. (2010) A quasi-experimental evaluation of an on-line formative assessment and tutoring system. Journal of Educational Computing Research. 43(1), 489-510. Mendicino, M., Razzaq, L., & Heffernan, N.T. (2009). A comparison of traditional homework to computer-supported homework. Journal of Research on Technology in Education, 41(3), 331-359.

INTELLIGENT TUTORING SYSTEMS IN THE MATHEMATICS CLASSROOM Murray, T., & Arroyo, I. (2002). Toward measuring and maintaining the Zone of Proximal Development in adaptive instructional systems. In S.A. Cerri, G. Gouardres & F. Paraguau (Eds.), Lecture Notes in Computer Science 2363 (pp. 133-145). Springer Berlin.

13

Razzaq, L., Feng, M., Nuzzo-Jones, G., Heffernan, N.T., Koedinger, K. R., Junker, B., Ritter, S., Knight, A., Aniszczyk, C.,Choksey, S., Livak, T., Mercado, E., Turner, T.E., Upalekar. R, Walonoski, J.A., Macasek. M.A., Rasmussen, K.P. (2005). The Assistment Project: Blending Assessment and Assisting. In C.K. Looi, G. McCalla, B. Bredeweg, & J. Breuker (Eds.) Proceedings of the 12th Artificial Intelligence In Education, 555-562. Amsterdam: ISO Press. Ritter, S., Anderson, J.R., Koedinger, K.R., & Corbett, A. (2007). Cognitive Tutor: Applied research in mathematics education. Psychonomic Bulletin & Review, 14(2), 249-255. Retrieved from www.psychonomic.org/index.html Shepard, L.A. (1991). Psychometricians beliefs about learning. Educational Researcher, 20(7), 2-16. Suppes, P. (1967). Some theoretical models for mathematics learning. Journal of Research and Development in Education, 1(1), 5-22. Uhr, L. (1969). Teaching machine programs that generate problems as a function of interaction with students. Proceedings of the 24th National Conference, 124-134. doi:10.1145/800195.805924 Vygotsky, L.S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.

INTELLIGENT TUTORING SYSTEMS IN THE MATHEMATICS CLASSROOM Wood, D., & Wood, H. (1996). Vygotsky, tutoring and learning. Oxford Review of Education, 22(1), 5-16.

14