School Based Assessment Methods Development and Implementation Dr Gavin Brown, Hong Kong Institute of Education (Invited paper

r for the First International Educational Conference on Assessment, New Delhi, India, January 2011, organised by The Central Board for Secondary Education, India (CBSE) & The Australian Council for Educational Research (ACER) ) School Based Assessment (SBA) has been carried out for a long timebut there are problems with how it has been done and new priorities that require changes. Traditionally, SBA has involved examinations and tests that mimic or mock public or school end-of-year examinations. Student performance on these kinds of SBA are usually reported as percentage scores or letter grades designed to reflect the same standards as would be applied by the examination authority in the formal, public examinations. Teachers were expected to comment on student performance in terms of effort, and likelihood of the student obtaining a pass or a high grade. It is generally assumed that public examinations (and the school-based mimics) evaluate students fairly and the consequences attached to the grades are merited and appropriate. And the consequences can be immensehigh enough grades give entry to the next level of education and highly exceptional grades lead to prestigious financial, educational, and social rewards. SBA also determines the quality of schools and teachers. Hence, public examinations and school-based versions of these assessments had great importance to the school, the teachers, let alone the students and their families. Issues with SBA include: (1) little use of alternative forms of assessments (e.g., portfolios, performances, peer, or self); (2) much trickle down of high-stakes examinations into pre-examination school years. There are some advantages to this approach to SBA. Since the stakes are so high, students are usually motivated to make significant effort. There is usually systematic and extensive coverage of the syllabus content, ensuring students and teachers pay attention to those things. There is, in most countries, a strong social acceptance that examinations are accurate, lead to valid decisions as to who is good, and have positive social consequences (i.e., exams identify talent regardless of sex, social status, ethnicity, wealth, and so on). Furthermore, there is a strong conviction that examinations are relatively robust against corruption, collusion, and cheating. There is also potential to provide diagnostic analysis of which parts of the required curriculum have yet to be or are already mastered. This is only a potential benefit as very careful curriculum analysis and mapping of test content to the curriculum map and effective reporting of performance is required. However, for teachers in schools to take advantage of this approach requires that teachers have significant professional development so as to be able to replicate the highest standards in testing. However, ordinary classroom teachers rarely have the necessary skills, which are normally available to qualifications authorities and test development companies. Hence, instead of turning teachers into testing experts, we aimed to give teachers a computer-assisted tool that helped teachers fulfill better the task which they were employed forpedagogically skilled delivery and facilitation of real learning in the real-time space of a classroom. In other words, we supported the teacher with computer assistance.

Another significant limitation of tests and examinations is that they usually generate a total score (a percentage) and/or a rank-order score such as position in class (e.g., 1st or last) or position relative to a norming sample (e.g., percentile or stanine). While these scores have some educational value, they do not lead to strong educational decision-making in the classroom. The percentage correct score is always a poor indicator of what a student knows. Real ability depends on the difficulty of the tasks answered correctly; not the proportion of questions answered correctly. Teachers grasp that hard questions should carry more weight than easy questions, but should not be expected to develop the technical skills to calculate this. Rank order scores do not indicate what each person still needs to learn and they lead to the temptation to assume that those at the bottom cannot learn and that those at the top cannot be taught anything more and thus do not help the teacher focus on who needs to be taught what next. The high-stakes use of examination scores to stream classes and/or schools can give a false message to both teachers and learners. Students may get a false impression of their ability depending on their rank. The quality of a teacher or a school cannot be fairly determined if the groups are skewed in the first placea high score does not mean high quality education. Thus, tests and examinations which only supply rank or proportion correct provide educationally weak information. For a test or an examination to have value to a teacher, it must provide more information than simply the total score or a rank-order type of score. What teachers need to know for improved teaching and student learning is who needs to be taught or learn what next. This information helps a teacher make decisions as to how to modify materials, student grouping, instructional strategies, and instructional sequences. Fundamentally, teachers need to know which objectives the students have mastered and dont need to be taught anymore; which objectives they surprisingly did well on and for which they can be praised; which objectives they did surprisingly poorly at and which they need to revise, review, and practice; and, most importantly, which objectives that were hard and that the students cannot yet do. These latter are what the teacher must include in their teaching plans, since learning hard material requires instruction. However, analysis of test questions in this way, and especially in a timely fashion, requires skills that most teachers do not have. Since the early 1990s, New Zealand has been changing its overall qualifications system towards both certifying performance in a broad range of domains and helping students learn more. The New Zealand curriculum specifies learning outcomes or objectives related to 8 levels of learning progress against which students are assessed. The revised qualifications system (the National Certificate of Educational Achievement) uses a socially moderated SBA in which teacher judgements count towards qualifications. Thus, high school teachers in New Zealand have had long experience in carrying out SBA both for school and Qualifications purposes. There is a strong acceptance that teaching is more than telling (it involves guiding and facilitating), that assessing is more than examining (though there is still a place for examinations), that all students can learn if guided with targets and feedback, and that there is an

important place for the teacher-expert in a classroom. There is wide-spread confidence in the professionalism of teachers and New Zealanders see that there are many routes to life-success other than just school-learning and high examination scores. This makes it somewhat easier to implement a rich, multi-faceted approach to SBA. Thus, in designing better SBA, we face the tension that assessment must serve multiple masters. Assessment must help improve the quality of teaching and learning; it must also assist in preparing students for external high-stakes assessments; it must contribute to helping school leaders and teachers monitor the quality of their work; and it must demonstrate to parents and others that high quality education is going on. And in some systems, SBA also contributes to official qualifications awards. New Zealand has a strong learning improvement orientation in its official goals and aims of assessment and there is a strong endorsement by teachers that the primary purpose of assessment is to improve teaching and learning. All these competing pressures require SBA that is much more sophisticated than simply rejecting tests and examinations or simply mimicking high-stakes assessments. How can teachers do this in a manageable, defensible, high-quality way? Using more than tests or exams requires effort, skill, and resourcesto categorise, analyse, diagnose individual & group needs, develop prescriptive responses, link to resources, and monitor effects. The argument we make is that the teacher still has an important role (not just learners) and that tests still have a place. This is especially the case in societies where there is great confidence in the virtues of examinations. The solution adopted in New Zealand is multi-faceted; it includes a wide variety of toolsexemplars, standardised tests, teacher professional development, and technology assisted testing systems in a culture in which teachers are respected as the people most capable of improving outcomes for all children and in which assessments are generally school-controlled and relatively low-stakes. If there is bad news, the teachers get to see it first, before any external bodies; this raises the probability that teachers will actually use assessments to improve their teaching rather than simply inflate their students test scores. For this reason, in New Zealand we developed a computer-assisted, school-controlled test system (Assessment Tools for Teaching & Learning-asTTle) that analyses student performance in this way for both individuals and groups. This system does the donkey-work, leaving the teacher free to think about how to design and deliver appropriate instruction instead of trying to figure out who got what on the test. The system generates a number of different reports that allow teachers and school leaders to evaluate, respond to, and monitor the effects of their work. Within the asTTle system, all test questions are designed to reflect both surface and deep thinking skills. The system models using the Structure of Observed Learning Outcomes taxonomy a means of understanding how to develop and assess students higher-order thinking, an essential aspect of most modern curricula. A second requirement has been ensuring that test questions within asTTle are well written and many objectively scored items are used to reduce subjectivity. Most importantly, though, improved information from tests has been devised through enhanced reporting systems which help

teachers to know: to which students does what part of the material need to be reviewed, which parts can I stop teaching, and which parts must be presented as new material? Reports were devised to help teachers explain to students and families the strengths and weaknesses of learners, to identify strengths and weaknesses of classes or cohorts of students, to identify students with similar teaching needs, and to identify appropriate teaching materials for the identified needs. For school leaders, the system reports how the school is doing relative to a variety of comparison groups, and handles classes which do tests of different levels of difficulty. Most importantly, being school-controlled, leaders do not need to wait until an external agency inspects them to know how they are doing. Our position is that SBA requires high-quality formative tests that do not just replicate summative, high-stakes examinations. The tests must align with the rich content of the curriculum, their use must make teacher workload more focused on the educational interpretation of scores, they must be but one part of a complete assessment for learning practice, they must have low-stakes consequences, teachers require professional development in the curriculum and in pedagogy to know how to teach new areas of the curriculum, and lastly technology must serve the teacher. It is possible to have tests serve the educational needs of classroom teaching. Without an appropriate resource base, policy framework, and professional context, of course, all this will NOT work. Fortunately, in public examination cultures, teachers and students and administrators are likely to accept the validity and utility of high-quality testing. All our job is now is to give teachers the tools that allow them to find out who needs to be taught what next.