An Investigation into the Implementation of Web-Based Learning for Teaching Multimedia Development in the School of Electrical & Electronic

Engineering of Singapore Polytechnic

By

Ng Hwee Kiat

Dissertation submitted in part requirement for the Master of Education Degree of the University of Sheffield September, 2003

University of Sheffield

ABSTRACT An Investigation into the Implementation of Web-Based Learning for Teaching Multimedia Development in the School of Electrical & Electronic Engineering of Singapore Polytechnic by Ng Hwee Kiat Supervisor: _____________ Department of __________ A dissertation presented on the implementation of Web Based Learning for teaching Multimedia Development in the School of Electrical & Electronics Engineering of Singapore Polytechnic. Web Based learning has grown in the higher education arena but there are still lingering doubts as to its efficacy. This research is based on a mixed approach of both face to face and Web based learning. Students are Asian vocational training students whose mother tongue is not English. One of the presumptions of Web Based Learning is mastery of English Literacy. Thus an environment of English based WBL for Asian vocational students with low proficiency of English provide an interesting study of which there are few research results. A combination of interviews with students and lecturers, student questionnaire survey and classroom observations was used as part of research instruments in 2000. The investigation included students perceptions of WBL. It also sought the views of the WBL developers/lecturers for a balanced perspective. Generally students want both the interactivity of WBL and social safety net provided by face to face teaching. An unplanned side result showed that for WBL to be successful, student and staff readiness plays an important part. This requires staff and students to be reasonably proficient in WBL technology, understands the new way and limitations of working and communication, have a good grasp of English and to be self directed.

ii

TABLE OF CONTENTS
Abstract............................................................................................................................................................ii List of Figures..................................................................................................................................................v List of Tables..................................................................................................................................................vi Acknowledgments.........................................................................................................................................vii CHAPTER 1 INTRODUCTION...................................................................................................................1 A TEST BED FOR EXPLORING WBL IN A CLASSROOM MODULE........................................................................4 GOALS OF THE PRESENT STUDY....................................................................................................................5 WHAT SETS THIS WORK APART FROM PRIOR RESEARCH?...............................................................................7 CHAPTER STRUCTURE AND OVERVIEW.........................................................................................................10 CHAPTER 2 DEVELOPMENT OF WEB BASED LEARNING...........................................................12 WHAT IS WEB BASED LEARNING? .............................................................................................................16 THE CASE FOR WBL...............................................................................................................................19 THE PROBLEMS WITH WBL......................................................................................................................21 SUMMARY...............................................................................................................................................23 CHAPTER 3 WBL IN SP AND SCHOOL OF EEE.................................................................................24 THE WBL TOOL.....................................................................................................................................25 THE MODULE..........................................................................................................................................28 THE DEVELOPMENT TEAM.........................................................................................................................28 DESIGN & DEVELOPMENT .........................................................................................................................29 IMPLEMENTATION.....................................................................................................................................34 CHAPTER 4 EVALUATION OF WBL IN MD MODULE.....................................................................38 TYPES OF EVALUATION.............................................................................................................................38 TYPE OF DATA........................................................................................................................................41 WHY AM I EVALUATING? THE PURPOSE...................................................................................................41 WHAT AM I EVALUATING? - THE QUESTIONS...............................................................................................42 HOW TO EVALUATE? THE EVALUATION PLANNING.....................................................................................45 PROBLEMS & LIMITATIONS OF EVALUATIONS................................................................................................46 EVALUATE, ANALYSE AND REPORT.............................................................................................................47 CHAPTER 5 METHODS............................................................................................................................48 STUDY SETTING AND SUBJECTS...................................................................................................................48 DATA COLLECTION...................................................................................................................................49 SURVEY QUESTIONNAIRE...........................................................................................................................50 CLASSROOM OBSERVATIONS.......................................................................................................................51 DEVELOPER/LECTURERS INTERVIEWS...........................................................................................................53 ETHICAL CONSIDERATIONS.........................................................................................................................53 SUMMARY...............................................................................................................................................54 CHAPTER 6 RESULTS...............................................................................................................................55 ON THE LEARNING RESOURCES...................................................................................................................56 ON THE DESIGN OF THE WBL PROGRAMME..................................................................................................58 ON THE STUDENTS LEARNING...................................................................................................................63 CLASSROOM OBSERVATION........................................................................................................................65 INTERVIEW OUTCOMES .............................................................................................................................67 SUMMARY OF RESULTS..............................................................................................................................69 CHAPTER 7 DISCUSSION & IMPLICATIONS.....................................................................................71 WHAT WAS LEFT OUT? ............................................................................................................................71 AREAS FOR FURTHER STUDY......................................................................................................................72 IMPLICATIONS OF FINDINGS .......................................................................................................................74 Implications on Students..............................................................................................................................74 Implications on Staff.....................................................................................................................................77

iii

Implications on Instruction Design..............................................................................................................77 CONCLUSION...........................................................................................................................................79 Bibliography..................................................................................................................................................81 References......................................................................................................................................................86 Appendix 1.....................................................................................................................................................90 SURVEY ON THE USE OF WEB BASED LEARNING FOR MULTIMEDIA DEVELOPMENT IN THE ELECTRONICS & COMMUNICATION ENGINEERING DEPARTMENT, SINGAPORE POLYTECHNIC..........................................................90 Appendix 2: Responses to Open-ended Questions.....................................................................................96

iv

LIST OF FIGURES Figure Page

FIGURE 2-1 RISING USE OF IT IN EDUCATION 1994-2002 (GREEN, 2002)..................................12 FIGURE 3-2 INTERACTIVE WEB PAGE USING YES/NO TYPE OF QUESTIONS......................32 FIGURE 3-3 INTERACTIVE WEB PAGE USING DRAG AND DROP TYPE OF INTERFACE.. .33 FIGURE 3-4 INTERACTIVE WEB PAGE USING DRAG AND DROP ORDERING.......................33 FIGURE 3-5 INTERACTIVE TESTING USING DRAG AND DROP..................................................36 FIGURE 3-6 RESPONSE PAGE PROVIDING FEEDBACK ON ASSESSMENT..............................37 FIGURE 4-7 KIRKPATRICKS 4 LEVELS OF EVALUATION..........................................................43 FIGURE 6-8 STUDENTS RESPONSE ON THE USEFULNESS OF RESOURCES.........................57 FIGURE 6-9 STUDENTS FEEDBACK ON THE USEFULNESS OF THE WBL MODULE MATERIALS DEVELOPED......................................................................................................................60 FIGURE 6-10 DISTRIBUTION OF TIME SPENT ONLINE BY STUDENTS....................................60 FIGURE 6-11 RESPONSES TO FACTORS CONDUCIVE TO WBL..................................................62 FIGURE 6-12 STUDENTS RESPONSE ON THEIR LEARNING VIA WBL ...................................64 FIGURE 6-13 ATTENDANCE PROFILE OF CLASS OF 21 STUDENTS AT FIRST WBL SESSION........................................................................................................................................................65 FIGURE 6-14 ATTENDANCE PROFILE OF CLASS OF 21 STUDENTS AT THIRD WBL SESSION........................................................................................................................................................66

LIST OF TABLES Table Page

TABLE 3-1 LECTURE/LABORATORY SCHEDULE (SEMESTER 1)...............................................35 TABLE 6-2 SURVEY QUESTIONS ON USEFULNESS OF LEARNING RESOURCES..................56 TABLE 6-3 SURVEY QUESTIONS ON THE USEFULNESS OF WBL MATERIALS DEVELOPED................................................................................................................................................59 TABLE 6-4 RANKING OF FACTORS CONDUCIVE TO WBL..........................................................61 TABLE 6-5 SURVEY QUESTIONS ON STUDENTS LEARNING THROUGH WBL.....................63 TABLE 6-6 SURVEY QUESTIONS ON STUDENTS PREFERENCES FOR WBL...........................65

vi

ACKNOWLEDGMENTS The author wishes to express sincere appreciation to Mr Dennis Sale for his assistance in the preparation of this manuscript. In addition, special thanks to members of the EEE EduOnline team: Mr Wong Chee Kong, Dr Robert Kelly and Ms Chan Yuek Wee for their valuable time and assistance in accommodating their work to the demands of this research and for taking time to conduct the survey on their classes. The author is also grateful to them for their candid views, suggestions and patience while being interviewed for this research. An acknowledgement is also due to the School of EEE management for granting permission for the conduct of this research. Finally, the author wishes to express his appreciation to his family for bearing with him for the duration of this dissertation.

vii

Chapter 1 Introduction Education is traditionally seen to the slowest moving and most conservative of any industry. However the advent of the Information Age and the World Wide Web (WWW) proliferation has driven or rather thrust the use of Information Technology (IT) onto education. There are many indisputable benefits of using IT in education as well as many ways of using IT in education. These can range from access to up-to-date information, increasing the flow of information, students not being bound by fixed schedules, sight and sound courseware that are more interesting and students learning at their own pace, There are also many ways that IT can be used in education. This spans the use of CDROMs, networked Personal Computers, and Web-Based Technology to wireless notebooks. The most prolific users of IT in education are in the distance education arena where universities faced with declining enrolment could exploit the anytime, anywhere hallmarks of the IT bandwagon. Education is no longer bound by time and space! However, the use of IT in education is not without its detractors. There are many questions left unanswered: Is the use of IT in education effective? Can the face-to-face interaction with lecturers and fellow students be replaced by online learning? Can we make use of technology to teach thinking, communications and collaboration traits desirable of the workers of the future? There has been much research in this area as the cost of investing in IT in education can be very substantial and the return on investment has to be justified in better or improved education. One of the fastest growth areas in IT in education is the use of the Internet and World Wide Web (WWW) to enhance teaching and learning. This marriage of WWW and 1

Chapter 1 Introduction education has been called e-learning, Web Based Learning, Internet Based Learning, OnLine Learning, Computer Based Learning and Distance Learning. With the exception of Internet and Web Based Learning, the others may cover varying degrees of other IT means such as CD ROMs, local area networks, recorded videos, computer programs, etc. We will use the name Web Based Learning (WBL) here to represent the use of Internet and WWW technologies in learning. What then is WBL? A typical WBL Model (McCrea, 2000, pp. 24-25) would include the three main pieces of: 1. Web Front End Learning Portal the browser user interface 2. Course Management System such as BlackBoard and TopClass 3. Content developed usually by domain experts Within the context of most educational institutes, the first two pieces are usually provided by commercial third parties such as BlackBoard, WebCT and TopClass. The content is the main bulk of work developed by the user. The question facing WBL users and developers is how can we tell if the use of this new technology really enhances pedagogy and learning? Owston suggested we examine the question of the WWWs contribution to education from three perspectives: Does the Web increase access to education? Does it promote improved learning? Does it contain the costs of learning? (Owston, 1997, pp 27-33) Does WBL make learning more accessible? The Internet is now pervasive across all spectrums of society and all countries. Students can access learning resources placed on the Internet anytime and anywhere they have a computer and an Internet connection. The beauty of the Web is that it provides an entirely new context for teaching and learning. It removes the physical and time constraints for instructors as 2

Chapter 1 Introduction well as learners. Moving a course on the Web presents the perfect opportunity to return to the core principles of teaching and learning to create a new pedagogical model for our practices. (Boettcher, 1999, p. 16) Does WBL improve learning? Computer based learning appeals to students learning mode, allows more flexible learning and encourages new kinds of learning. Consistency of content is also assured, as the material is the same wherever it is accessed. WBL also makes possible immediate distribution of content changes (Peterson, 1999, p. 16). All these have the potential to improve learning. Is it Cost effective? Cost effectiveness will still be an important consideration. If the cost of implementing WBL results in escalating the cost of education, it would not be justifiable from the point of view of all stakeholders involved. Of all the considerations, I would see the most important is still the learning aspects. A proper and useful assessment of learning in WBL is important. Owston points out: we cannot simply ask Do students learn better with the Web as compared to traditional classroom instruction? We have to realize that no medium, in and of itself, will likely improve learning in a significant way when it is used to deliver instruction. Nor is it realistic to expect the Web, when used as a tool, to develop in students any unique skills. (Owston, 1997, pp 27-33) Owston contends that the right question is: What distinct advantage does an instructional technology offer that instructors can exploit to promote improved learning? He offers at least three advantages: appealing to students learning mode, flexibility and new ways of learning.

Chapter 1 Introduction In addition to accessibility, learning and cost effectivenss, Peterson (1999, p.13) suggests that the driving factors of WBL are Exploration, Experience, Engagement, Ease of Use, and Empowerment, with the Internet as the conduit. A Test Bed for Exploring WBL in a Classroom Module Since there are few examples and publications of successful Web Based Learning (WBL) in use in Singapore Polytechnic (SP) classrooms, precisely how to successfully marry pedagogy to technology is still not clear. What do we really understand about these technologies and how they may be used to support or enhance the educational process? How can we design learning environments that foster the use of these technologies by students? To date there has been little substantive research into how they could be used or are currently being used in vocational institutions such as SP. In order to fully explore the implications of WBL in SP education, it is necessary to study WBL in use by students in the School. This research setting was established in the form of the EduOnline Project within the School of EEE of SP. EduOnline is a set-up for the development of IT resources in teaching and learning. Specialist staffs are expected to hone their skills and develop expertise in the application of IT to education. In particular, EduOnline is exploring ways to enhance, and fundamentally rethink, the way that students learn using WBL tools. SP cannot afford to pull out staff from what they are currently doing to assist in the development of an online education system. To undertake such an effort requires considerable effort. The EduOnline Project is founded on the belief that teaching staff within SP, burdened with a high teaching and administrative load will not be able to

Chapter 1 Introduction produce pedagogically sound IT materials within a limited timeframe and with little or no training. Specialist staff with multimedia expertise, instructional design background and an understanding of pedagogical issues would be in a better position to support the IT in education movement. Modules would be identified and the subject domain experts (module coordinators) are brought into the EduOnline development team. It is believed that the integration of the technology experts and the domain experts would result in WBL applications that are effective and instructionally sound. Goals of the Present Study The objectives of this study are: To evaluate the implementation of Web-based learning (WBL) for Teaching & Learning within the EEE School. To review the use of Web-based multimedia within the broader context of IT in terms of curriculum planning, development, delivery and assessment from both the lecturers and students viewpoint; To make specific recommendations on how to effectively implement Web-based Multimedia for Teaching & Learning in the School of EEE. In the Singapore Polytechnic, most technological barriers to the use of IT for learning have been removed. There is a good IT infrastructure and connectivity. All academic and support staffs are provided networked PCs. There is also a long term systematic and top-down planning for the deployment and usage of IT infrastructure. The main barriers to the adoption of WBL are therefore seen to be cultural and pedagogical. Most staffs appear satisfied with the status quo they do not see any need for IT to make their teaching more effective. Those who see the need do not have the 5

Chapter 1 Introduction means and resources to make the best use of WBL often citing lack of time or expertise to perform the exploratory work or research to produce a sound pedagogical framework for WBL development. The objective of this study is to address these cultural and pedagogical barriers to WBL implementation by evaluating a trial implementation through the EduOnline project. These barriers can basically be seen to be perception issues. A better understanding of the success factors in implementation of WBL can help to spur more staff to embark on WBL. This leads to the third objective of this study, which is to culminate in specific recommendations on effective implementations of WBL for the School. In light of this implementation challenge, this studys primary question is: Which modes or aspects of Web-based learning are most effective for the students in the school? And how do we measure the effectiveness of Web based learning? Understanding these aspects is an important first step towards the design of effective Web Based learning environments. Factors that might contribute to WBLs effectiveness include 1. the learning resources/materials 2. the design of the WBL programme 3. the students attitude/learning approach towards on-line learning A secondary question is: How will students respond to the idea of self directed independent learning through Web Based learning? Traditionally, the students in SP have been taught on a spoon-fed system and are handicapped by poor English and reading skills. The new mode of on-line learning requires the students to read on their own another hurdle to overcome. On the other hand, most of these students are well versed

Chapter 1 Introduction with the use of Web Based technology although not in the learning context. It would be interesting to see how these students adapt (if at all) to the new approach to learning. A third question is: What resources are required to develop and sustain Web based learning in the department? What are the problems that the staff as content developers faced? WBL applications take considerable time and effort to develop. Staffs in the School are already heavily loaded with teaching and administrative tasks. This question aims to determine the amount of time and effort required to develop and sustain a WBL module. The final question addressed by this study is: What are the factors that would motivate or de-motivate staff from embarking on Web based learning? Embarking on WBL is a massive task and it would not be unreasonable for most staff to expect some form of recognition or rewards other than personal satisfaction (for the more self actualised). For most SP and EEE School staff this would be considered an additional undertaking over their normal work schedule and staffs are generally reluctant to embark on additional ventures without any perceivable gain or benefit. The ultimate goal of this research is to understand both the successes and the failures of the implementation of a WBL module within the School. The lessons of this study can then be used to shape the future development of WBL, and more importantly, the forms of pedagogy and curriculum that accompany them, to ensure that the greatest benefit is derived for the most students and staff in the EEE School. What Sets This Work Apart from Prior Research? There have been many research studies on the effects of the use of IT. Most of these researches tend towards collaborative learning among students or the asynchronous 7

Chapter 1 Introduction learning in distance learning programmes. There is usually full implementation of IT in the entire course. This work is based on the SP environment where the students meet the lecturer regularly in the classroom environment and collaboration work is not required. Research findings from a distance education model or a collaborative learning model are not directly applicable here. A mixed or blended model to implement the use of IT for the purpose of promoting self-learning and continual learning in the SP environment is proposed. While there is research on the use of IT in education, it is normally conducted in a Eurocentric environment that differs significantly from the Asian or even Singaporean learning environment. This poses some difficulties in interpreting the findings of these studies for application in the local Asian settings. For instance, Asians are more prone to accept viewpoints than to criticise or argue their case in public. Dynamics of such students in discussion groups, email or forums could be very different from that of typical research. From another perspective, the students background plays a role in how they would use the IT resources for learning. Singapore students are conditioned to take examinations. They are more concerned about passing tests and examinations than to take a qualitative approach to learning. Their motivations for using IT could be very much different from that of UK or USA students. Data in this Research The methods employed in this study use both qualitative and quantitative data. This approach is specifically designed to develop a perspective on the ways that WBL is appropriated in the classroom environment. The primary measure used in this research is students usage of and attitude towards WBL. 8

Chapter 1 Introduction A variety of qualitative data sources are used to contextualize the interpretation of the quantitative data on WBL. These data include interviews with lecturers/developers and classroom observations. These qualitative accounts of behaviour illuminate the quantitative data and provide support for inferences about how student characteristics may be related to variation in WBL use. A survey instrument is used to collect information on the students perceptions about the WBL materials developed and their attitude towards learning on-line. While the above clarifies the focus of this research, it is also worth noting what this research does not address IT in education as a whole. No attempt is made to understand how IT tools, besides WBL, can change the learning outcomes of students in classrooms. The decision to focus on WBL instead of IT in education as a whole was made as the result of the authors belief that IT is not responsible for changes in learningteachers (lecturers) are. There are many plausible scenarios in which WBL can significantly improve learning and many scenarios where learning does not improve. Good and bad learning environments exist with or without the benefit of IT. Audiences for this Work The results reported from this research are intended for use by people who are responsible for the management, design and implementation of Web Based Learning in the Singapore Polytechnic context. The educational research community will find this work interesting for its evaluation of WBL within an Asian classroom context.

Chapter 1 Introduction Chapter Structure and Overview The chapters that follow describe the motivations, methods, and results of this research in detail. Chapter 2 will comprise a review of the research literature related to the use of WBL as well as the case for or against WBL. Chapter 3 will establish the designers intentions for the WBL system as well as determine the local context for WBL in the school. Chapter 4 will review the literature related to evaluation of online learning. This is to help identify a reliable and valid evaluation methodology. Chapter 5 will review the multiple methodologies used to address the issues raised in Chapters 2 to 4. These methods include a survey, interviews and classroom observations. The rationale for the methodology will be given, as well as examples of how the data were gathered and interpreted. Chapter 6 will present the results from the data. First, results from data bearing on characteristics of the WBL materials will be presented. Second, results from data relating to students usage of the WBL application. Finally, results from staff viewpoint pertaining to development issues will be presented. Chapter 7 will consider the implications of the results presented in Chapter 6. Strategies for increasing the level of WBL adoption suggested by the results will be discussed, including an overview of key findings and recommendations. The bibliography and references section lists the reading and publications used in this dissertation. 10

Chapter 1 Introduction Appendix 1 contains the survey questionnaire used by the author while Appendix 2 comprises the responses to the open ended questions.

11

Chapter 2 Development of Web Based Learning Educators has always been looking out for new tools and development that could help in the teaching and learning process. The advancement of technology has made available tools such as overhead projectors, visualisers, video, television, etc. which have been used successfully to a certain extent. Later advances such as the computer, CD-ROMs and computer networks have further opened up more possibilities and a richer range of media for educators to make use of. Computers have been used in education for over 20 years now. Computer-based training (CBT) and computer aided instruction (CAI) were the first such systems deployed as an attempt to teach using computers. While both CBT and CAI may be somewhat effective in helping learners, they do not provide the same kind of individualized attention that a student would receive from a human tutor.

Figure 2-1 Rising Use of IT in Education 1994-2002 (Green, 2002) 12

2 Development of WBL In nearly all areas of IT use in instruction, there has been a consistent rise (Green 2002) as found by Kenneth Green in his Campus Computing Project which has been tracking the use of computers in education since 1974 (see Figure 2.1). It is clear that IT use in education is here to stay and will continue to rise. The term online learning or e-learning has been used to represent computer-based training (CBT), web-based training (WBT), computer-based instruction (CBI), and technology-based training (TBT). However, each of these forms of on-line learning has specific meaning. Computer-based training (CBT), also called computer-based instruction, refers to courses presented on a computer. Such course does not provide links to learning resources outside of the course. Usually, students take a computer-based training course on a computer that is not connected to a network. Web-based training (WBT), or web-based learning (WBL), is a form of computerbased training available on an intranet, extranet, or Internet and that is linked to learning resources outside of the course, such as references, electronic mail and discussions, and videoconferencing. Technology-based training has a broader meaning; it refers to training through media other than the classroom. This may include computers, but also refer to television, audio tapes, CDROMs, video tapes, and print. The term distance learning is often also used to refer to online learning. Online learning can be considered as one form of distance learning. Distance learning is any type of educational situation in which the instructor and students are separated by time, 13

2 Development of WBL location, or both. As an example, correspondence courses and courses by television are forms of distance learning. To a lesser extent and stretching the argument, students learning in a separate building (on the same campus) from the instructor may be considered as distance learning. Hence distance learning can apply to both off-campus as well as on-campus courses. Many institutions have explored a variety of methods to deliver education at a distance arising from a need to increase intake of students. Such methods have included satellite broadcasts, home videos, audio conferencing, text based correspondence courses, television broadcasts, etc. With the growth & development of the Internet, the WWW has become one of the most widely used forms of distance learning due to its ubiquity and availability. The development of the WWW and internet has helped to facilitate changes in the way course development and learning has been traditionally envisaged. From HTML to Web Authoring Traditionally, the use of WWW for presentation of course materials requires the user to be proficient in Hypertext Mark-up Language (HTML) programming. The creation of courseware then required expert WEB developers & programmers. The situation improved with the availability of software, such as Microsoft Word and PowerPoint, which converts documents to HTML format. This allows any instructor to make use of familiar word processing or presentation software to generate WWW pages that can be posted onto the Internet. The next evolution was the development of Web Course Development & Course Management tools which allow instructors to easily develop and distribute course 14

2 Development of WBL materials. Depending on background, experience, interest, and technical sophistication, instructors can choose HTML editors, templates, authoring programs or commercial Web Course Development tools to develop and distribute online material. Some of these Web Course Development and Management tools are: BlackBoard CourseInfo Course-in-a-box Lotus LearningSpace TopClass WebCT WebMentor

From Objectivist to Constructivist Learning Traditional classrooms use the objectivist model of learning which is based on Skinner's theory of transfer of knowledge from instructor to the student. In this model, the instructor controls the material and pace of learning. Instruction is directed to the whole class as a group. The instructor controls the pace of the learning. On the other hand, constructivist model is student centred. Here the instructor acts as a facilitator primarily responsible for facilitating learning. In this model, students choose the content to be organized and learned. The internet provides both synchronous and asynchronous communication as well as file storage and retrieval capability, enabling learning to take on a more constructivist approach. The Internet provides communication access previously not possible. Almost all instructors and students in tertiary education are now provided with email and Internet accounts that allow them access to communication at unprecedented levels. With the access to large amounts of information through the WWW, the task of education will 15

2 Development of WBL have to focus on teaching students how to make sense of information. We no longer need a subject specialist to teach us. The educators new roles can now be discussion leaders, advisors, tutors, facilitators, moderators, etc. What is Web Based Learning? Internet Based Technologies such as Telnet, FTP, Email, News Groups, Chats, Conferencing and WWW (hyperlinks) allow for both synchronous as well as asynchronous interactions. Synchronous interactions can include NetMeeting where both parties can see and communicate at the same time, conferencing as well as online chats. Asynchronous interactions can include stored video and audio clips, news groups as well as emails. There are no precise definitions of the terms CAI, CAL, CBT, WBL, and WBT as there are no owners or standards body for this type of learning methodology. Therefore, WBT, WBI and WBL could possibly mean the same thing. Web Based Training can be defined as individualised instruction delivered over public or private computer networks and displayed by the Web Browser (Clark, 1996). The author would tend to disagree to this limited definition as it does not exploit the features of the WWW which is essentially its hyperlinks and multimedia rich content. A better definition is provided by Khan (1997) who defined Web Based Instruction (WBI) as a hypermedia-based instructional program which utilizes the attributes and resources of the World Wide Web to create a meaningful learning environment where learning is fostered and supported. What then is the difference between Web Based Training and Web Based Learning? The author sees the main difference as a matter of perspective. Training is to see from the instructors point of view whereas learning is to see from the students point of view. 16

2 Development of WBL Responsibility and choices for WBL will lie with the student whereas in WBT, these lie with the instructor. Since the use of the WWW opens the way for a preferable constructivist approach to learning, we will use the term WBL for this project. WBL will be used here to represent open, flexible and distributed learning activities based on the WWW. How is WBL different from traditional learning? In traditional teaching, the teacher plays the role of the subject matter expert delivering knowledge to the learners. Learners act in passive modes to complete set learning tasks. One of the affordance of the WWW is access: access to information, access to communication, access to remote computing power and access to collaboration. There have been many studies comparing WBL with traditional learning. Both positive results and negative results abound which basically highlights the difficulty in evaluating WBL. One of the most obvious differences between WBL and traditional learning is in time & space. Because of the possibility of any time any place learning, students can take lessons at anytime and anyplace. Since the WWW can be always online as compared to a lecturer, learning can be timely and response can provided immediately. Being always online allows students to go back and review the materials at any time at their convenience. Updating of information can also be immediate. Internet and WWW technologies make it easy for functions of Information Distribution, Communications, Assessment and Management. Relan and Gillani suggest the Web may be used as a: 17

2 Development of WBL 1. 2. 3. 4. resource for identification, evaluation and integration of information medium for collaboration and communication of ideas platform for expression of understandings and meanings medium for participating in simulated experiences, apprenticeships and cognitive partnerships. Relan and Gillani (1997)

In the first case, the WWW is used as a source of information. The WWW is excellent in this role as it is linked to numerous sources of information such as museums, schools, institutions and agencies throughout the world. In the second case, the WWW is a communication medium between students and teachers using email and messaging services. In the third case, the WWW is used as an electronic book to present information. With its multimedia capable presentation, materials can be presented in much more appealing and attractive formats. In the final case, the WWW is used as a teacher. Developments such as intelligent tutoring systems (ITS) and XML can help to personalise learning. WBL opens up many methods and strategies that can be used for learning. Examples are: Presentation, Exhibits, Demonstration, Drill and Practice, Tutorials, Games, StoryTelling, Simulations, Role-playing, Discussion, Interaction, Modelling, Facilitation, Collaboration, Debate, Field Trips, Apprenticeship, Case Studies, Generative Development, Motivation and others (Khan, 2001). In the Singapore Polytechnic context, the focus of the use of IT in education is to create an environment that enables students to learn in ways that best suit them (SG 2001, p. 21). With the many methods cited above, it is just left to the innovativeness of the lecturer in deciding how best of make use of WBL to create such a learning environment for the students.

18

2 Development of WBL The Case for WBL There are many good reasons for using WBL. With the developments of bigger data storage capacity, wireless & mobile WWW and higher bandwidth of broadband, the delivery and access of learning can only be better and faster. Current developments include mobility with the mobile devices such as cellular phones and Personal Digital Assistants (PDAs) supporting the WWW. Flexibility, Accessibility, Convenience - Users can proceed through a training program "at their own pace and at their own place." They can also access the training at any time, and only as much as they need - known as "Just in time and just enough." This provide the increased flexibility and convenience of taking lessons at home, school or even while travelling. WBL makes it easy for instructors to share expertise and resources research and collaboration. WBL provides access to world wide resources for learning access to huge knowledge base outside of the school museums, archives, associations, etc. Students can access learning materials when required and at their own learning pace. With WBL, the individual student has control over the speed and quantity of learning to suit their abilities and schedules. With anytime, anywhere access, students would not have many excuses for missing lessons leading to improved student attendance (Valdez, 1999). Connections - Since the WWW is platform independent, WBL can be accessed by Web browsing software on any platform: Windows, Mac, UNIX, OS/2, Amiga, etc. Courses can be delivered to any machine over the Internet or intranet without having to author a program for each platform. Web browser software and Internet connections are widely available. Most students and instructors have access to a browser, such as Internet 19

2 Development of WBL Explorer and are connected to their institutes intranet, and/or have access to the Internet. WBL can be interactive, can be collaborative, and can allow students to interact with instructors and fellow students anywhere in the world. Multimedia - WBL technologies include technologies such as Flash, Hypertext, animation, video, sound, music, etc. These help to present a more interesting mix of learning resources compared to the static word and pictures in traditional books. Such multimedia element results in improved student motivation in accessing the course materials (Valdez, 1999). A well designed WBL course can be much more engaging than simply reading printed pages. It can offer a rich mix of multimedia colour, sound and animation. Time & Cost Savings. One of the computers strength is in storing and retrieving information. From the lecturers perspective, the use of WBL to take care of mundane and repetitive tasks such as test development, administration, scoring, recording, statistical analysis and providing feedback can help to save time. If changes need to be made in the course material after the original implementation, they can be made on the server which stores the course materials and everyone worldwide can instantly access the update. Grading and feedback can be easily done through WBL technologies. Students can get immediate results & feedback when all aspects of the test are done online. This immediacy of feedback is one of the attributes of good teaching. From Administrators perspective, WBL is cost effective as it reduces delivery cost over time. Using the WWW as a tool can help streamline record keeping making it accessible to instructors and students anytime, anywhere. Progress can be easily tracked and 20

2 Development of WBL recorded. It allows instructors to monitor individual student needs and accomplishments as well as increase parental involvement (Valdez, 1999). Customised. Theoretically, individual learning paths is also possible although it is not so easy to develop. The WBL system can even be designed to adapt the material it presents to the knowledge and skills of each student. Content can be differentiated by depth or by style of presentation to suit different students. Key learning points can be made in a variety of ways to suit individual preferences. The WBL system can also provide feedback to assist with self assessment Consistency. WBL is also consistent as it is immune to instructor fatigue and variations in presentation skills. Preparation for Work. WBL is being used not only in academic learning but also by professional development training providers. Tomorrows workers need to process more information in less time. Todays students who will be tomorrows workers need to learn how to do that. The Problems with WBL WBL, like the use of any tool, can fail if not used properly. A study by Chapman (1999) listed 7 possible causes for the failure of WBL. 1. 2. 3. 4. 5. 6. 7. Poor Instructional Design Too Much Focus on Technology Lack of Mass Market Content Lack of Standards Limiting Technologies (eg. Bandwidth) The One Size Fits All Syndrome Little Effort Given to Validate Training Effectiveness (Chapman, 1999)

21

2 Development of WBL Heading the list here is Poor Instructional Design. Not all courses can be delivered well by computer some training topics are best served by with a more personal touch. It has been recognised that WBL and other technologies for learning are mainly for assisting the learning process and are not for replacing methods that already work well. In terms of instructional design and infrastructure, teachers needed to be provided the appropriate professional development, and there needed to be sufficient technology and connectivityWhere insufficient professional development was provided, student achievement and teacher satisfaction actually dropped (Valdez, 1999). Designing and maintaining a web-site is extremely costly in terms of time and expertise required. Moreover, there are many concerns among teaching staff leading to a general reluctance of such staff to develop WBL for their teaching. Some of these disincentives are: lack of time to develop the materials, lack of incentives, lack of technical expertise, concern with protection of intellectual property over the WWW, stress associated with keeping up in the field of learning technology, concerns about plagiarism, etc. (COU, 2000). Another major problem is that of lack of face to face communication. Without face to face communication, the instructor will have problems reading the non-verbal communication of students. Although Schutte (1997) demonstrated the virtual class scored an average of 20% higher that the traditional class , he also noted that the virtual students seemed more frustrated, but not entirely from the technology. Rather it stemmed from the inability to ask questions of the professor in a face to face environment. He went on to allude that the performance difference can be attributed to student collaboration as to technology, itself.

22

2 Development of WBL The WWW provides many distraction and students attention can be easily diverted when they are surfing on the internet. Finally, technology is not infallible. Designing for the Internet presents a special problem. The network may be down due to technical problems denying access when instructors/students need to make use of it. Connection speeds can sometimes be slow or intermittent, and downloads can slow down due to factors over which instructors often have no control. Summary There are three possible approaches we can take to exploit WBL in education 1. alternatives to traditional learning (distance learning, on-line course) or 2. supplement to tradition learning (additional information) or 3. blended with traditional learning (drawing on the best of both worlds) We will elaborate on the approach selected in Chapter 3. The important question that is the focus of this study is: What can WBL do that cannot be accomplished by other less expensive or more proven methods? What will we gain--and what will we lose?

23

Chapter 3 WBL in SP and School of EEE Singapores drive into the knowledge based economy demands that most jobs require a certain level of advanced education. Singapores Ministry of Education (MOE) embarked on a Master plan for IT in Education in 1977 with the following goals: 1. Enhance linkages between the school and the world around it. - Teachers and pupils will communicate and collaborate with other institutions, enabling them to acquire richer perspectives in an increasingly borderless world. 2. Generate innovative processes in education - Development of new teaching and learning strategies will open new possibilities for curricula and assessment. Schools will be given autonomy to deploy IT resources flexibly. New school designs will seek to maximise the potential of IT in education. 3. Enhance creative thinking, lifelong learning and social responsibility - IT-based learning strategies will help to develop pupils' ability to think flexibly and innovatively, to co-operate with one another and to make sound value-base judgements. 4. Promote administrative and management excellence in the education system - IT will be used to promote greater efficiency in administration and communication, thereby supporting more effective educational management. (MOE, 1997) This resulted in a urgent downstream push at tertiary institutions towards the use of IT in Education too. Singapore Polytechnic having embarked on the use of IT for education since 1994 with cautious experimentation by some forward looking lecturers (Chan & Sweeters, 1994) renewed its efforts with at least 3 different spearheads in 1997. The then Electrical Engineering (EE) Department championed Blackboard as the CMS platform. The Educational and Staff Development (ESD) Department, having invested a great deal of time and resources on an in-house developed Virtual College platform, championed its cause. The School of IT developed its own e-learning platform. The then Electronics and Communication (EC) Engineering Department was then deciding on its own approach decided to take on a more systematic evaluation resulting in this study. (Both EE and EC departments have merged into School of EEE since 2001). 24

Chapter 3 WBL in SP & SEEE To ensure that the direction of the work is aligned with the polytechnics, a review of the SP Education Model in the area of IT usage was done. The aim, as indicated, is to equip our students with basic IT knowledge and skills, teach them how to access information independently and learn collaboratively, and use IT to help develop higher order thinking and lifelong learning skills. (SG 2000, p. 14) The focus of the use of IT in education is to create an environment that enables students to learn in ways that best suit them AQU (2000, p. 14). The emphasis here is totally for the learning benefits of the students and not to ease the load of the instructor or help reduce costs for the Polytechnic. With this in mind, the objective of the project, coined EduOnline was defined as follows: a. To be the trailblazer so more teaching staff would be keen to take up the challenge to develop online learning materials for their modules; and b. To provide opportunities for students to be independent and responsible for their own learning. However, it is envisaged that there are side effects here that will benefit the instructor and the Polytechnic as well. The WBL Tool A typical WBL Model (McCrea, 2000, pp. 24-25) would include the three main pieces: 1. Web Front End Learning Portal the browser user interface 2. Course Management System such as BlackBoard and TopClass 3. Content developed usually by domain experts There are a wide variety and range of learning or course management systems that can fit into the second piece of McCreas model which can also help to manage or develop a

25

Chapter 3 WBL in SP & SEEE course. The terms learning management, learning content management and course management may be confusing as they have different meanings to different people. Learning Management Systems (LMS) help to manage learners, keeping track of progress and performance (Hall, 2003). The primary focus of a LMS is the management of learning. LMS usually do not provide content development tools, but they do support third party courseware. A Learning Content Management System (LCMS) helps to manage content or learning objects (Hall, 2003). The focus of a Course Management System (CMS) is the development and delivery of asynchronous training. CMS are typically targeted at educational institutions, offering lower costs and built-in content-creation tools. They generally offer varying levels of student and course management. They usually provide some form of communication resources for students as well as tracking and assessing student progress. Their distinguishing feature is that they enable individual instructors to develop and deliver online educational content with little or no expertise in HTML or other Web programming languages. Development tools are built in to the environments, enabling instructors to create Web pages from templates, upload documents, design online quizzes and tests. The systems also often contain management tools that include enrolment and student tracking. One of the most difficult tasks is to decide on the CMS to be used. There are many such tools available with different capabilities. The most popular includes WebCT, Blackboard CourseInfo, TopClass and Learning Space. The considerations for the system will have to be seen from the perspective of the different stakeholders: System Administrator, Course Manager, Course Designer, Instructor and Student. A detailed 26

Chapter 3 WBL in SP & SEEE presentation of the evaluation for the CMS for EduOnline is beyond the scope of this dissertation. There are many existing evaluations and comparisons of the various CMSs and LMSs tools. The Web site EduTools at http://www.edutools.info provide comprehensive comparisons between many CMSs. The consideration for choice for the EduOnline project includes usage within the local tertiary scene. At the time of evaluation, the two contenders being used by the local institutions were TopClass and Blackboard. For the purpose of the EduOnline project, TopClass was selected for use as the first 2 pieces of the WBL model the front end and CMS. The use of a CMS helps to alleviate the course management however, it was found that the CMS learning platforms are generally weakest in the area of pedagogy. They can track user access, generate tests automatically, and perform many other useful but essentially ancillary administrative tasks. They can also be used for fast prototyping of courses but lacks in the area of development of a interactive course. Content development for the EduOnline project will be WWW based making it easily portable to any other CMS. The WBL module developed was hosted on the Department Intranet Server. The TopClass CMS was adopted to provide some form of course management. Teaching staff and students are provided access to the online learning materials. Each student is given an account to use the features available in the TopClass environment. The TopClass system provides features such as mailing, discussion forums, tests, etc. Each student's progress is tracked by the system and the respective class instructor can monitor the performance of each student in his/her own class.

27

Chapter 3 WBL in SP & SEEE The Module The EduOnline team members selected the Multimedia Development module (EC1370) in October 1999 to be the pilot module to make use of information technology (IT) to explore the impact of online learning on students and staff. Web based materials were developed to replace the lecture component. Multimedia Development is a third year module in the Computer Systems Technology (CST) option in the Diploma in Electronics, Computer & Communication Engineering (DECC) Full-time Course. Its aim is to cover the concept of multimedia, and the process involved in multimedia development. In addition, it will also cover the issue of webbased multimedia to complement the material in another CST module, Client-Server Systems. Students are guided through the entire process of designing and developing a multimedia application through self paced learning using the developed WBL materials. Students are final year students (age: 19 20 years) who having completed the GCE O Level exams as a prerequisite for admission to the course. Students taking this module are expected to be IT savvy as they are also taking other IT and computer related modules. The Development Team Many different roles are required for the development team in a multimedia development project: project leader, instructional designer, writer, programmer, graphics designer, video and audio production team, editor, subject matter expert and testers (Sarliner, 1999). During the EduOnline trial, the developers will have to take on the roles of System Administrator, Course Manager, Course Designer and Instructor.

28

Chapter 3 WBL in SP & SEEE The following MD teaching staffs were assigned to develop and manage the online learning materials: a. Mr Wong Chee Kong (Module Co-ordinator) - Team leader, TopClass setup and administrator, Design and development of the following chapters: "Module Overview", "Introduction to Multimedia" and "Equipment & Interfaces" b. Mrs Chan-Tan Yuek Wee (Module Development Team Member) - Design and development of the chapter "Multimedia Resources" c. Dr Robert Kelly - Website design, instructional design, Design and development of the chapters: "Home Page", "Learning Online", "How to use this software", "Designing Multimedia Titles", "Managing a Multimedia Project" and "Web Publishing" Design & Development The development team strived to make use of educational philosophy in the development of the WBL materials with a clear intent not to be driven by technology. The MD module was designed making use of the Seven Principles & Good Practices as spelt out by Chickering and Ehrmann. These practices are: 1. 2. 3. 4. 5. 6. 7. encourages contact between students and faculty develops reciprocity and cooperation among students uses active learning techniques gives prompt feedback emphasises time on task communicates high expectations respect diverse talents and ways of learning (Chickering & Ehrmann, 1996) The single most important IT issue for all institutions arising from the Campus Computing Project for the past few years has consistently been integrating IT into 29

Chapter 3 WBL in SP & SEEE education (Green 2001, 2002). Similarly, Carnevale found that the essential feature of a good course includes interaction between instructors and students, a student centred approach and built-in opportunities for students to learn on their own (p. A46) (Carnevale, 2000). Effective learning environment should involve frequent and meaningful interactions among the learners, instructional materials, and between the learner and the instructor. This integration and interaction relates back to the first 4 good practices detailed above. There are at least 3 possibilities here in integrating on-line learning with traditional learning: - enrichments, repositories and redevelopment. The first possibility it to consider the online materials as enrichments or extras, providing information related to, but outside of the module syllabus. The second possibility is to have repositories of online materials which are exact replicates of the materials used in traditional fact to face teaching. These first two options are frequently adopted in the first flush of enthusiasm for WBL which often leads to disastrous results. The first option pushes the WBL to the periphery of the curriculum while the second is just a repository without any pedagogical application. The third option is to have a re-development of the on-line materials as a self-contained set of materials specifically for self access and independent learning. This third option deemed to be the most difficult is the option of choice. The main advantage of this is that it can be used to present pedagogically sound learning programmes to students. This can be totally different from the expository, instructor-led teaching. For investment in WBL to be justifiable, it must present a better alternative to the traditional face to face teaching alone. 30

Chapter 3 WBL in SP & SEEE There are some things that can be taught effectively using the WWW and some things that are best taught using the traditional face to face method. It is the authors and the development teams opinion that a blended approach of face to face and on-line learning would be most effective. The author envisaged that WWW will never fully replace traditional classroom instruction and human mentoring. Rather WBL will exist along other methods and tools for teaching and learning such as video-conferencing, computer based training, CD-ROMS, etc. This blended approach of weaving the best practices of new and old practices is ideal. However for the purpose of this study, the entire course contents are placed on line to evaluate the students and staffs response. Another important aspect of the design and development is the User Interface (UI) design. User Interface may be defined as the communication channel. The UI should be such that the users spend the bulk of time on the learning task itself rather than learning how to use the interface. A consistent and intuitive user interface helps to cut down learning time. Design of learning materials using interactive components such as selection (see Figure 3-1), drag and drop (see Figure 3-2) and ordering (see Figure 3-3) is to encourage the student to interact with the course materials rather than read passively.

31

Chapter 3 WBL in SP & SEEE

Figure 3-2 Interactive Web page using Yes/No type of questions. Cognitive load is the amount of information that the learner can process. Managing cognitive load is essential to effective teaching and training. One widely held and accepted practice is Information Chunking. This states that our working memory is able to hold a maximum of 72 chunks of information at any one time. The implication on the WBL design is then to ensure that any page presented should not be too cluttered with information chunks.

32

Chapter 3 WBL in SP & SEEE

Figure 3-3 Interactive Web page using Drag and Drop type of Interface.

Figure 3-4 Interactive Web page using Drag and Drop Ordering

33

Chapter 3 WBL in SP & SEEE The WBL development took about 6 months to complete: mid-October 1999 to mid-April 2000. The affected staff were given time off their teaching duties to focus on the WBL development of the module. Implementation As a result of the structure of the modular diploma system, the MD module is scheduled to be completed in 2 terms over 15 weeks. Two hours were catered for self learning via WBL and two hours laboratory time for students to complete their assignments each week. The schedule for lectures and laboratories using WBL is outlined in Table 3-1. Except for the introductory lesson and all the tests, all other learning are self-learning on WBL as indicated using shaded cells in the table. Week 1 Supervised 2 Supervised 3 4 5 Supervised 6 7 8 Learning Topic Laboratory Topic

Term 1 Module Overview/ Briefing on Lab. 1 (Intro + Script) TopClass Usage Lab. 2 (Text) Pre-Test Chapter 1 Lab. 3 (Audio)+ mini-project

9 10 11 12 Supervised 13

Chapter 2 Lab. 4 (Image) + mini-project Chapter 2 Lab. 5 (Animation)+ mini-project Progress Test 1/ Discussion Lab. 6 (Video) + mini-project Forum Chapter 3 Lab. 7 (Authoring)+ mini-project Chapter 3 Make-up Make-up Lab Test 1 & Make-up Mid-Semester Vacation (1 Week) Term 2 Mid-Semester Test Week Lab. 8 (Testing, cutting CDROM title) Chapter 4 Lab. 9 (HTML) Chapter 4 Lab. 10 (Web publishing) Progress Test 2/ Chapter 5 Chapter 5 34 Mini-project Mini-project Presentation & Interview

Chapter 3 WBL in SP & SEEE Week 14 15 Supervised Learning Topic Chapter 6 Post-Test/Evaluation Laboratory Topic Make-up & Lab Test 2 Make-up

Final-Semester Examination Table 3-1 Lecture/Laboratory Schedule (Semester 1) Each class accessed the designated computer lab for their online lecture for two hours each week. No other free-access hours were catered outside of these two hours. This was to minimise administrative workload to block access due to 25-concurrent user limit. This constraint arose from the licensing of the TopClass platform as only 25 licenses were acquired for this trial. Lecturer were only present to conduct the introductory briefing, pre-test, 2 progress tests and the post-test. The online tests allow students to evaluate themselves on the knowledge gained through web-learning. The following tests were scheduled to assist students in the evaluation exercise: 1. Pre-Test (Week 2) - to evaluate students Multimedia Development prior knowledge. 2. Progress Test 1 (Week 5) - to assess students learning in Chapters 1 and 2. (see Figure 3-4) 3. Progress Test 2 (Week 12) - to assess students learning in Chapters 3 and 4. 4. Post-Test (Week 15) to assess students knowledge upon completion of module.

35

Chapter 3 WBL in SP & SEEE

Figure 3-5 Interactive Testing using Drag and Drop. After taking each test, students will get immediate feedback on their response with detailed explanation on their answers as shown in Figure 3.5.

36

Chapter 3 WBL in SP & SEEE

Figure 3-6 Response page providing feedback on assessment The following materials were made available to students via TopClass: a. Interactive Web Based learning material; b. Self-evaluation quizzes; a. Discussion forum to share and discuss with fellow classmates and tutor; b. Important announcements/news; c. Self-correcting tests for staff and student evaluations. Laboratory sessions were scheduled to provide opportunities for students to work unsupervised on their assignments. Lecturers were scheduled to be present during the laboratory sessions and were tasked to monitor the progress of the class. All MD teaching staff participated in the electronic forum and answered students' queries. This task was rostered among the teaching staff, and was officially time-tabled, in lieu of the lecture hours.

37

Chapter 4 Evaluation of WBL in MD Chapter 4 Evaluation of WBL in MD Module Evaluating technology programs can be challenging. This is especially true in light of the many controversial results published on the usefulness of applying Web Based technology. The question here is how to plan and design an evaluation that is valid and useful. There are many guides produced for evaluating the use of technology in learning and teaching (Harvey 1998, Quinones 1998, Britain 1999). These help to jumpstart the evaluation process as they present all the relevant thoughts and resources in a single place. However, it is still left to the user to decide and pick the appropriate tools and resources for use. Evaluation is a time consuming affair and for it to be of use, needs careful thought and planning. Otherwise, the results may not be useable and the time will be wasted. Definition Evaluation is any activity that throughout the planning and delivery of innovative programmes enables those involved to learn and make judgments about the starting assumptions, implementation processes and outcomes of the innovation concerned. (Stern, 1988) Evaluation is defined as the process of determining the merit or worth of a product, process, or program. (Scriven, 1991) Types of Evaluation Evaluation can be viewed as having different dimensions. The different dimensions of evaluation have formal names: process, outcome, and impact evaluation. Process evaluation describes and assesses program materials and activities needed for project implementation such as development of software, staff training, etc. Outcome evaluation assesses program achievements and effects such as changes in knowledge, attitudes and 38

Chapter 4 Evaluation of WBL in MD skills. Impact evaluation looks beyond the immediate results of policies, instruction, or services to identify longer-term as well as unintended program effects. (Muraskin 1993, pp. 4-6) This WBL study involves all the 3 dimensions as it seeks to evaluate the development of the materials (process), the students learning (outcome) as well as the effect of WBL on the curriculum (impact). Effective WBL is a combined result of the learner, the learning materials and the implementation (Oliver et al, 1996). Thus the failure of WBL could be due to poorly designed learning materials rather than the WBL implementation itself. We can also view evaluation from the perspective of purposes or roles. This falls under the four general categories: formative, summative, illuminative and integrative (Davidson 1998). The terms "formative" and "summative" evaluation were first used by Scriven (1967). Formative evaluations aim to identify problems with resources and suggest appropriate solutions. Within the context of WBL, the aim of formative evaluation is to help improve the design of WBL. This is carried out on actual students while the materials are being developed and there are still resources to modify it. Questionnaires which ask students to give comments or ratings for presentation, content, and confidence in the learning objective, complemented by classroom observations can be used for this purpose. Summative evaluation would generally be carried out after the materials has been produced and to help users choose which components of WBL to use and for what purpose. Although this approach focuses on learning outcomes, these will need to take account of the context, as well as the educational technology which has been adopted. As

39

Chapter 4 Evaluation of WBL in MD a result, summative studies have been criticised for proving useful only in learning situations which closely resemble the evaluation. Parlett & Hamilton (1972/77/87) coined the term "illuminative" evaluation for the purpose of discovery. Illuminative evaluation is an open-ended method which aims to uncover unexpected important factors issues in a particular situation. It is a systematic focus on discovering the unexpected, instead of assessing how well an educational intervention performs on standard measures of assessment. It sets out without predefined aims and seek to identify or explain problems and issues. (Oliver, 1998) Draper et al. (1996) later used the term "integrative" evaluation for the purpose of maximising use of the evaluation. The aim of integrative evaluation is to help users make the most of a given piece of WBL. The aim of integrative evaluations is to lead to the emergence of a new role for the courseware being evaluated. They aim to improve teaching and learning by integrating CAL material into the overall situation more effectively. This leads to the conclusion that they are closely related to illuminative evaluations, which aim to identify and explain problems in learning situations, since such explanations may offer opportunities to integrate CAL more effectively. It has been argued that integrative evaluation provides the kind of feedback teachers are really seeking - not properties of CAL per se, but the delivery of effective learning and teaching using CAL (Draper et al., 1996). This study could be classified as both illuminative and formative evaluations as it seeks to answer the questions of how best WBL can be used and which components of WBL are useful and relevant for implementation within the school.

40

Chapter 4 Evaluation of WBL in MD Type of Data Quantitative evaluation comprises the collection of facts and relationships that produces quantified data such as access, cost, etc. This is relatively easy to collect analyse and report. Qualitative evaluation seeks insight and understanding perceptions rather than statistical analysis and is thus more difficult to analyse (Bell, 1993). These two types of evaluation data can be used to complement each other and to provide additional cross checking for improved confidence in results. Both these types of evaluation and associated data would be used in this study. Why am I Evaluating? The Purpose Before evaluating a program, it is critical to consider who is most likely to need and use the information that will be obtained and for what purposes. This involves identifying the beneficiaries and/or the stakeholders. Who is this for and what is it they want to find out? For this study, the main stakeholders would be the school management and the teaching staff who are interested in developing WBL for their teaching. The school management would be interested in knowing if the time spent in developing leads to either better efficiency or better quality of teaching. The teaching staff or potential developers of WBL would be interested to know about the learning gains and students attitude or response towards WBL. There are many reasons to measure the success of the WBL program such as to validate training, to justify costs, to improve design and to select training methods. Valdez (1999) propositions that evaluation should serve 2 broad purposes: 1. to provide information to guide the redesign and improvement and 2. to provide information for decision-making. 41

Chapter 4 Evaluation of WBL in MD These coincide with the two broad type of evaluation: formative and summative. The goal of formative evaluation is to improve the project whilst summative evaluation is to provide an assessment of the effectiveness. What am I Evaluating? - The Questions The evaluation for WBL includes both assessment of learners and evaluation of the instruction and learning environment. Donald Kirkpatrick first proposed a four-pronged approach to evaluating training programs in his 1959 doctoral dissertation. These are: reactions, learning, application and results (Kirkpatrick, 1994) Level I - Reactions of learners are basically a happiness index of the course. They measure user satisfaction and user perception. Reactions are important because negative reactions will likely result in poor learning. To measure reaction: surveys, observations, interviews can be used. The most commonly used measure of reaction is the questionnaire. Level II - Learning is the primary objective of any training and should be measured. Learning can be measured objectively using a test or exam at the end of this study. Examples of learning measures are completion rates, achievements of learning objectives. Learning can be measured by evaluating knowledge, skills and attitudes before and after the training. Level III - Application is measured through reflection of behaviour changes requiring measurements before and after the training. Application can be measured through observation, questionnaire, self monitoring or appraisal. This is a difficult measure as transfer of learning may not occur due to factors such as being forgotten, lack of reinforcement, lack of opportunity to apply, etc. 42

Chapter 4 Evaluation of WBL in MD Level IV - Results are the bottom line or impact on performance. The real value of training is from improved performance the results. This is traditionally the hardest to forecast or measure in an educational environment. Measures can be cost savings, time savings from faster access to information, productivity increases such as reduced duplication of effort, labour savings, etc., and even intangibles like higher levels of motivation. For the purpose of this study, results can be indirectly measured from the students ability to complete their projects and their course grades. However this can be quite complex as there could be many other mitigating factors affecting the grades. All levels of evaluation are important for different reasons but the only ones that count are those where valid, reliable results can be produced. According to Kirkpatricks model, evaluation should always begin with Level I, and then, as time and budget allows, should move sequentially through Levels II, III, and IV. Information from each prior level serves as a base for the next level's evaluation (Figure 4-1). Thus, each successive level represents a more precise measure of the effectiveness of the training program, but at the same time requires a more rigorous and time-consuming analysis.

Figure 4-7 Kirkpatricks 4 Levels of Evaluation

43

Chapter 4 Evaluation of WBL in MD In this research, the main type of evaluation is mainly for reactions of students and staff a level 1 evaluation. Learning is measured from the results of the tests while application and results (Levels III and IV) are not covered due to the difficulty and complexity of obtaining the data as well as the practicality of data analysis which would be more time consuming. Another perspective on what to evaluate by Hawkes (2001) puts forth 4 main criteria for evaluating WBL. These are the technical, instructional, organisation and ethical criteria. Technical criteria are those that are concerned with equipment. Examples are equipment requirements, specifications, and performance factors such as ease , speed of access and flexibility. This is not an issue in the SP context with the availability of broadband access and updated equipment. However it may be a concern for students accessing WBL from home using dialup modems. Instructional criteria are concerned with the delivery and access of instruction and its outcomes for learners. These include interactivity, integration, learner control, learner/instructor attitudes and learner achievement. This is the main area of concern for this study. Organisational criteria are those concerned with the day to day use of the technology. These include technical maintenance, space and time feasibility, support systems, staff development, etc. This was an area of concern previously but has since been improved with the good organisational infrastructure in place in SP now. Ethical criteria address technology availability to diverse learning audiences. Again with the free access provided by SP, this is not an issue.

44

Chapter 4 Evaluation of WBL in MD In the planning of study, questions like: What questions should I ask?, What information do I need to collect? and What measures/indicators to I use for evaluation? can be answered by a good understanding of the Kirkpatricks levels and Hawkes criteria. The systematic grouping and setting of levels and criteria can help to focus on the appropriate questions in the design of the study. How to Evaluate? The Evaluation Planning The next step in the evaluation sequence is to determine the evaluation plan. This can be seen as the most important stage as a suitable design of the study can help significantly towards the success of the evaluation. The planning stage involves the determination of the tools, data acquisition strategies to be used as well as the timeline of the evaluation. The actual evaluation methodology used in this study will be covered in Chapter 5 of this dissertation. One of the first considerations is constraints such as the size, scale and time of the evaluation. Who will be involved and what is the timescale? Due to the limitation of scope of this dissertation, an appropriate size would be a cohort of students from one of the schools Diploma programme with a timescale of a semester. What is the best time to carry out the evaluation? Manwaring suggests that the evaluator ensure that enough has happened but not when students are busy with exams (Manwaring 1998, p10). The next consideration would be the sources of information and the methods to be used to gather this information. The ideal situation is to make use of the authors current work situation without the need for additional outside party involvement. However this can only provide a very limited range of data source. A wider scope could possibly encompass resources such as students, instructors, test scores, document reviews and 45

Chapter 4 Evaluation of WBL in MD literature survey. As the information to be gathered will affect the tools and techniques to be used, this has to be decided first. The evaluator will need to consider the best way to collect the information for the study. Some of the common methods used for information gathering are: surveys, observations and interviews. There should not be any need to depend on external (outside of the school) party to carry out evaluation. A variety of methods should be used so that findings can be triangulated or be used to substantiate results obtained from other methods. The kind of analysis that is required will also affect the method data collection and type of information to be collected. If rigorous statistical analysis is not required and a straight forward frequency count is sufficient, there is no need to collect statistics or to make use of statistical analysis tools. Within the limited scope of this dissertation, straight forward frequency count is generally sufficient. Chapter 5 of this dissertation will detail the methods to used in this study. Another consideration in the planning stage is access to confidential information such as examination grades and the need to obtain permission from the school to carry out study. Finally, the evaluator needs to consider the costs involved in the study. Printing and paper costs as well as time of lecturers needs to be factored into the project. As the project was supported by the school management, the use of school resources and permission to carry out the study was also granted. Problems & Limitations of Evaluations It has to be recognised that there are problems associated with any evaluation. It may cause anxieties to staff and students involved with the evaluation. Evaluations can be costly in term of time, not just to the evaluator. Teaching time may have to be taken up to

46

Chapter 4 Evaluation of WBL in MD get students to complete questionnaires. This may interfere with the normal running of the course. Results may also be misinterpreted or misused if the context and purpose are not clear enough. Evaluate, Analyse and Report The next step after the planning would be to conduct the evaluation itself. This entails scheduling an appropriate time and place as well as preparing contingency plans in case things go wrong. The data collected will then be analysed to develop conclusions. This will be elaborated subsequently in Chapters 6 and 7 of this dissertation. The evaluator will need to continue monitoring the study and make plans to respond to unintended outcomes. To a certain extent all evaluation feeds back to influence practice. The final step, then, is to disseminate the results of the evaluation to the various stakeholders. For this study, the results were presented to the school management with the recommendations and implications. The developers were also provided a copy of the results of the evaluation.

47

Chapter 5 Methods This study was designed to explore factors that affect the use of Web Based Technologies in teaching and learning and, where possible, to make recommendations that will increase the amount of Web Technologies use in the school setting. Three different aspects of the WBL were examined in this study: structure of module management & development, aspects of the WBL components employed that encourage or discourage their use and the characteristic of student that relate to their use of WBL. This chapter describes the methods used to gather data on these different aspects of the WBL study. The setting and subjects for the study are detailed first. The description of methods then assumes a structure similar to that of the overall study, and provides details of specific instruments as they apply to the WBL study. Study Setting and Subjects This study was conducted as part of the EduOnline project during the second semester of the 1999/2000 academic year from 17 January to 21 April 2000. The entire cohort of students and teaching staff taking the Multimedia Development module were involved in this study. The subjects in this study were the 105 full time final year students and the 3 lecturers. The students were from the 5 classes taking the Computer Systems Technology (CST) option for the Diploma in Electronics, Computer and Communication (DECC). The three lecturers for this module participated in this project right from the beginning to the completion. All three lecturers were also involved as members of the development team for the WBL materials.

48

Chapter 5 Methods The students who participated in this study did so by virtue of their involvement in the Multimedia Development module. They did not know in advance that they were going to be involved in an ITTL project that employed WBL as the approach of learning. The author has the option of asking for volunteers, selecting the sample or involving the whole class. As the class size is not that big, it was decided to involve all the students and lecturers for the module. Data Collection Both quantitative and qualitative research will be carried out. Quantitative study is used for a fast and cost effective way to collect data and qualitative study is used to understand the background issues and to clarify the quantitative results. Qualitative methods considered here are classroom observations and interviews with students and staff. Whatever the methods use, the overriding objective is to maximise the validity and reliability of the data collected for the results to be useful. Reliability here refers to the consistency or dependability of the data or the extent to which a test or procedure produces similar results under constant conditions on all occasions (Bell 1993, p 64). This is achieved in the study by piloting a survey questionnaire to ensure proper question wording. Validity of the questions tells us whether the questions measures or describes what it is supposed to measure or describe. (Bell 1993, p 65) To really ensure validity may require many complex testing and measurement which runs outside the scope of this project. Validity for this investigation is based largely on literature review to study and select similar instruments used by other researchers in similar settings. 49

Chapter 5 Methods Threats to validity are selection of sample and Hawthorne effect. In a conscious effort to validate the evaluation, the selection of students and their participation in the evaluation was not explicitly made known to them at the beginning of the course. Survey Questionnaire Survey questionnaire techniques are cost and time effective for sampling opinions across a wide group of students and provide bulk information. Anonymity allows students to honestly evaluate and comment on their use of WBL. The survey was designed as a paper form for ease of use. Both closed and open ended questions were used here. Closed ended questions allow for fast and quick analysis of large quantities of data. Open ended questions were used to allow for unanticipated reactions. The main focus of the survey questions is on determining the factors that contribute to WBLs effectiveness. These were structured across 3 main categories which include the learning resources and materials, the design of the WBL programme and the students attitude and learning approach towards WBL. The questions are grouped under three parts reflecting these three factors. Respondents were given the opportunity to add in their views in every part of the questionnaire. It has to be acknowledged that the questionnaire method has its limitations. For example, students may not be willing to complete and return the questionnaires. If the response rate is too low, there will be concerns generalising of the results. The survey questions were kept to 2 pages to encourage respondents to complete them. There may also be possibilities of the questions being misinterpreted or answered by someone other than the target respondent. To counter this limitation, the lecturers taking the class were requested to administer the survey. Questionnaires are also rigid in that 50

Chapter 5 Methods questions cannot be changed quickly to respond to a change in situation or to capitalise on interesting strands of responses. Design of the questions and structure requires more careful thought. For the data to be meaningful, the survey questionnaire can only be administered at the end of the semester when the students have completed the entire WBL program In order to get maximum returns and cut down postal costs and administrative work, the survey questionnaire was administered to the students during their last lesson. The lecturers conducting the class were requested to administer the survey which the students took on the spot. An accompanying cover letter was provided to help explain the purpose of the questionnaire (as shown in Appendix 1) to the students and to establish the academic nature of this study. However since not all issues are easily quantifiable, students were provided a means within the survey to express their opinions, attitudes and ideas. The survey questionnaire was piloted 2 weeks before the actual survey to test if the questions are clear, unambiguous and easily understood by the students. The survey questionnaire was revised based on the feedback of the students and lecturer administering the survey. Classroom Observations With this approach, the author is in the classroom with the students. It is recognised that the authors presence will affect the students interactions with each other and the WBL. The goal of this observation is to watch how students interact with each other, how learner accesses WBL and how they integrate with the traditional class schedule. 51

Chapter 5 Methods Randomly selected classes involved in the study were visited at regular intervals throughout the duration of the study by the author. These visits had a broad range of objectives but all served to provide a qualitative understanding of the nature of the classroom practice and the use of WBL. The main intention is to seek an opportunity to observe the students behaviour in interacting with each other and the WBL materials. The classroom observation seeks to answer the secondary question in the study - How will students respond to the idea of self directed independent learning through Web Based learning? Some questions were prepared to aid in the classroom observations. These were as follows: How did the students use the program as advised or using a different strategy? Were there any aspects of program design or interface design that caused difficulties or confusions? Did the way the session was set up helped the students and could it be improved? Were there differences in the way the students used the program or WBL as the term progresses? The author could discuss with the students what they were doing as they engaged in the various WBL activities. These data were used to help interpret other forms of data gathered in this study, such as the interview and survey data (described above). This classroom observation is constrained by the number and timing of the scheduled classes. Permission was sought and granted from the school and course management for the classroom observations.

52

Chapter 5 Methods Developer/Lecturers Interviews Face to face interviews allow the opportunities and flexibility for probing, searching and finding explanations for discrepancies between observed and expected effects. More importantly, interviews allow follow up on incomplete answers. However interviews can be time consuming, highly subjective and subject to bias (Bell 1993, p 91). At the conclusion of the academic semester, one-on-one informal but structured interviews were conducted with each of the three developers/lecturers. This is possible due to the author being colleagues with the interviewees within the same school. These interviews were designed as a retrospective conversation with developers/lecturers about the activities and learning in their development work and the effectiveness of using WBL in the classrooms. These interviews seek to answer the third and fourth questions for this study, i.e., What resources are required to develop and sustain Web based learning in the department? What are the problems that the staff as content developers faced? What are the factors that would motivate or de-motivate staff from embarking on Web based learning? The developers were first prompted to discuss the activities they were involved in and the experiences gained. The goal of the interview exercise was to determine if there was any learning or experiences in their dual role as lecturers and developers. Questions and topics were prepared before hand to aid the author and given to the interviewees to formulate answers before the interview. Ethical Considerations The research study was carried out with the approval of the school. Although no sensitive information was sought in this study, the respondents (both staff and students) 53

Chapter 5 Methods were guaranteed anonymity. This is important as it enhanced the likelihood of the respondents being completely honest and unbiased in their responses. Summary Other methods considered but not used here included focus groups, documents review, case studies and expert reviews. The goal of this research is to evaluate the implementation of Web-based learning (WBL) for Teaching & Learning within the EEE School, to review the use of Web-based multimedia within the broader context of IT in terms of curriculum planning, development, delivery and assessment from both the lecturers and students viewpoint and to make specific recommendations on how to effectively implement Web-based Multimedia for Teaching & Learning in the School of EEE. In Chapter 6, the results of these investigations will be presented and analysed.

54

Chapter 6 Results The goal of this study is to evaluate the implementation of Web-based learning (WBL) for Teaching & Learning within the EEE School. review the use of Web-based multimedia within the broader context of IT in terms of curriculum planning, development, delivery and assessment from both the lecturers and students viewpoint; make specific recommendations on how to effectively implement Web-based Multimedia for Teaching & Learning in the EEE School The test bed is the trial WBL implementation of the Multimedia Development module. The students were surveyed on factors that the author deemed might contribute to the effectiveness of WBL. The survey questions were grouped as follows: On the learning materials/resources course management On the WBL materials developed design & content On students learning attitude & learning approach towards WBL A total of 105 students from the 5 classes from DECC 3FT CST were surveyed from 17 April to 21 April 2000 during their last scheduled class. Even then, 68 out of the 105 students responded giving a response rate 65%. The remaining 35% were mostly absent as it was the last class of the term. All the collected responses were filled in completely and there was little or no evidence of lack of care in their completion. At least half the students were willing to put in comments to supplement their responses. 55

Chapter 6 Results On the Learning Resources The first section of the survey focuses on how useful the students find the learning resources provided by the Course Management System. Because the learning resources are not functionally equivalent, it is reasonable to predict that students will find some resources more useful than others. A survey was designed asking students to rate the usefulness of each resource for their learning. Students were asked to rate the usefulness of the various learning resources on a 5-point scale (Not useful at all, Not very useful, Useful, Very Useful and Extremely Useful). The learning resources surveyed included WBL resources such as WBL module materials, email, discussion lists and announcements as shown in Table 6-2. A comparison of the usefulness is also made with the fully-supervised laboratory session.
Please the response (one only) which most closely represent your opinion of the usefulness of the each of the resources used in this module Not Not very Very Extremely Resource useful Useful useful useful useful at all 1 Supervised Laboratory 2 WBL Module Materials 3 Discussion List 4 TopClass Announcements 5 TopClass Email

Table 6-2 Survey Questions on usefulness of Learning Resources The results are presented in Figure 6.1. 51 out of 68 (75%) students responded that the supervised laboratory was useful (32), very useful (12) or extremely useful (7), as there was a tutor on hand to answer questions when they are in doubt. This indicated that face to face contact is still valued in the context of WBL.

56

Chapter 6 Results
40 35 30 25 20 15 10 5 0
Not useful at allNot very useful Supervised Laboratory W BL Module Materials Discussion List TopClass Announcements TopClass Email 4 0 7 7 9 13 14 30 26 37 Useful 32 36 23 28 17 Very useful 12 14 4 4 3 Extremely useful 7 4 3 3 2

Figure 6-8 Students response on the Usefulness of resources 79% (54 out of 68) of the students rated the WBL materials as useful or better. In other words, 21% of the students do not find the WBL materials useful. In another on-line survey conducted April-May 2000 (unpublished) on the use of Blackboard CourseInfo (with 212 respondents at the same school), 22% of the students also do not find online teaching materials useful. It appears to be consistent that about 20% of the students at the school do not find WBL useful. This could be due to the learning styles of these students. The same cohort of students was surveyed three year ago on their learning styles by the author (Ng, 1998). It was found then that 20% of the cohort prefers a more personal style of learning. Whether these are the same students would be an interesting study relating learning styles to WBL preference. Students see WBL Module materials as supplementary material to textbook or notes. It is heartening to see that both supervised laboratory and WBL are rated equally well. 57

Chapter 6 Results Emails (68%), discussion lists (54%) and announcements (49%) were generally found to be less useful than supervised lab sessions (25%) and WBL materials (21%). A significant number of students rated the discussion list (37), announcements (33) and email features (46) as not very useful or not useful at all. This is to be expected, as there was no planned conscious use made of these resources. This was unfortunate as one of the most useful ways to promote thinking is through discussion (Muilenburg 2001). More than 50% do not think much of discussion lists/forums. The results from a online BlackBoard survey of the same cohort of students confirmed the findings with 65% of students finding the discussion forums not useful. This could be due to the general reluctance of Singaporean students to put up critical discourse. On the design of the WBL programme The second section of the survey focuses on design of the WBL materials developed. Respondents were asked to indicate their level of agreement using a five-point scale (strongly disagree, disagree, neutral, agree and strongly agree) with seven statements describing the WBL materials as shown in Table 6-3.

58

Chapter 6 Results
Please the response (one only) which most closely represent your opinion of the statements (SD - Strongly Disagree, D - Disagree, N - Neutral, A - Agree, SA Strongly Agree)

To what extent do you agree with these descriptions S D N A S of the WBL programme? D A

6 It is easy to use the user interface is easily understood 7 It is enjoyable to use 8 It provides good support for my learning 9 It has clear instructions on how to work through the material 1 It helps me learn about Multimedia Development 0 module 1 It fits well with the rest of the module materials 1 1 It helps me to revise the subject 2 Table 6-3 Survey Questions on the usefulness of WBL materials developed. The responses to this section are presented in Figure 6-9. Generally the students felt positive about the WBL materials developed. This is a good reflection of the amount of thought and planning that went into the design of the WBL components in the module. Less than 20% of the students disagreed that the developed WBL materials were useful across all the factors. Although not all the students find the WBL materials useful, the 80% affirmation goes a long way to justify the efforts of the lecturers/developers who took great pains and efforts to create a pedagogically sound WBL package. Despite the amount of interactivity built in, some students commented that the program is boring, not enough interactivity giving an indication of the MTV generations higher expectations of fast, exciting interactivity.

59

Chapter 6 Results
100%

80%

60%

40%

SA A N D SD

20%

0%

It is easy to use It is enjoyable to the user interface is use easily understood 6 43 17 2 0 2 29 32 4 0

It provides good support for my learning 3 29 24 12 0

It has clear It helps me learn instructions on how about Multimedia to work through the Development 6 32 24 6 0 5 28 28 7 0

It fits well with the It helps me to rest of the module revise the subject materials 2 21 37 9 0 1 27 27 13 0

SA A N D SD

Figure 6-9 Students feedback on the usefulness of the WBL module materials developed The students were also asked to indicate the total number of hours they spent on the WBL package for the module within the duration ranges categorised as less than 6 hours, 6 to 12 hours, 13 to 18 hours or more than 18 hours.
40 No. of Students 30 20 10 0 <6 hrs 6 to 12 hrs 13 to 18 hrs >18 hrs Time Spent Online 3 19 9 37

Figure 6-10 Distribution of time spent online by students With a scheduled time table of 24 hours for WBL, a third (32%) of the students responded that they spend less than the half the timetabled hours (i.e. less than 12 hours) 60

Chapter 6 Results as shown in Figure 6-10. The classroom observation bears this out as most students were observed to finish their materials within 45 minutes to an hour although 2 hours were scheduled. Most students took the opportunity to spend less time on scheduled classes. However, we were not able to tell further if the students spend less hours because they do not need to or because they preferred to do something else with that time. This result is consistent with another study (Brown & Liedholm 2002) which found that students in virtual classes generally spent less time on the course compared to students in live classes. In attempting to determine what motivates the students to return to the WBL programme, the students were asked to rank 5 factors on a scale of A to E where A represents most important and E least important as shown in Table 6.3. Please rank the following factors in order of importance that kept you interested or would keep you interested in the WBL programme. (from A to E where A represents most important and E least important). Please use each ranking once only. Factors Rank (from A to E) 14 Highly Interactive Contents 15 High Multimedia Contents 16 Regular Online Tests/Quizzes 17 Ability to access learning materials at own time and pace 18 Scheduled WBL sessions Table 6-4 Ranking of Factors Conducive to WBL Although the students were instructed to use each ranking once only, there were many cases where the more than one ranking were repeated or omitted. The total numbers for each ranking should tally with the number of respondents if the responses are done correctly. This suggests that the students surveyed possibly had difficulties following printed English instructions or had a poor grasp of English. This will have an impact on 61

Chapter 6 Results the students learning as poor English literacy will disadvantage the Web Based Learner. Although the results of this question may be invalidated by the incorrect interpretation of the students, it is still illuminating to know what motivates the students towards WBL.

40 35 30 25 20 15 10 5 0
Highly Interactive Contents High Multimedia Contents Regular Online Tests/Quizzes Ability to access learning materials at ow n time and pace Scheduled WBL sessions A 28 20 15 37 B 25 28 21 20 C 22 21 26 12 D 14 19 15 22 E 9 8 19 7

18

16

15

38

Figure 6-11 Responses to Factors Conducive to WBL There were 37 responses which cited the ability to access learning at their own time and pace as most influential in keeping them on-line (Figure 6-4). However results from Question 13 indicated otherwise as the anytime, anyplace seduction turned out to be less time and less place when a third of the students reported they accessed the WBL materials less than the required hours. These could be students who did their work faster or already had some knowledge of the materials. However, there was no correlation done to this effect. It would be interesting to make another study to assess if there is any correlation between the time students spent on-line and their learning. 62

Chapter 6 Results Contrary to our expectations, there were 38 responses that scheduled lessons are of the least importance in influencing them to go on-line. The responses seem to indicate the students desire rather than the actual situation. During classroom observations, it has been noted that many students came late and/or left early. It was previously established that most of the students accounted for less than the timetabled hours even with scheduled lessons. It could be possible that students might go online more often without scheduled lessons but this needed to be further investigated or verified. It was however felt by the author that despite the students response, scheduled WBL lessons do keep the students on-line. On the Students Learning
Please the response (one only) which most closely represent your opinion of the statements (SD = Strongly Disagree, D = Disagree, N = Neutral, A = Agree, SA = Strongly Agree)

19 20 21 22 23 24

As you worked your way through the WBL SD D N A SA sessions, did you feel that the information was related to your existing knowledge you were being encouraged to think more about the subject area your interest was being maintained through use of a range of strategies the courseware was responsive to your own particular learning needs material was structured in a way that facilitated an overall understanding you are able to get useful feedback on your progress Table 6-5 Survey Questions on Students Learning through WBL

Generally the students were very positive about the WBL introduced. Most either agree or are neutral about the usefulness of the WBL materials developed (see Figure 6-5).

63

Chapter 6 Results
45 40 35 30 25 20 15 10 5 0
t inform ion was relat t your he at ed o exist knowledge ing you were being encouraged t t o hink m about t subject area ore he your int erest was being m ained aint t hrough use of a range of st egies rat t courseware was responsive t he o your own part icular learnin needs g m erial was st ured in a way t at ruct hat facilit ed an overall underst at anding you are able t get u o seful feedback on your progress 0 8 29 27 3 0 2 32 32 2 0 7 26 34 1 0 4 29 34 1 0 2 30 30 6

SD 0

D 4

N 21

A 40

SA 3

Figure 6-12 Students response on their learning via WBL The students were finally asked to indicate what they would like to see as improvement for the WBL. Most students would like to see more interactivity in WBL and more faceto face time with the lecturer as reflected in Table 6-5. A study at another department at the same institution found blending e-learning and face-to-face teaching as being the best learning arrangement (Fan, 2001). The respondents preferences and effective e-learning fits nicely in this case into the blended e-learning model! 64

Chapter 6 Results
This module provides a mix of both self-paced WBL and face-to-face learning with your lecturer as part of the module. To improve the module, would you prefer More self-paced WBL No More face to face time with 25 17 11 22 learning time change the lecturer No 26 Reduce WBL content 6 19 Increase WBL content 12 change No 27 Make WBL more interactive 25 11 change

Table 6-6 Survey questions on students preferences for WBL Generally students welcome the on-line quizzes as self-assessment and revision resources. 40 of the 68 respondents commented that the most useful component within the WBL is the online tests and quizzes which is not surprising considering the examination-oriented education system they have been put through. Classroom Observation During the initial classroom observations, it was seen that most students had difficulty remembering their login ID and password. Attendance was kept at 50 to 75% during the initial session (see Figure 6.6).

20 No. of Students 15 10 5 0 1 1.5 2 Tim e 2.5 3 4 12 14 15 12 10 12 8

Figure 6-13 Attendance profile of class of 21 students at first WBL session Half of the students appear to read faster than normal, scanning through the on-line materials in a page surfing mode. They appear to be just scanning for interesting materials and not learning as they read. When asked if they are able to learn from such 65

Chapter 6 Results scanning, the students replied that they would return to the materials when they need to do their project work. Most students finished the online materials within half an hour using this mode of learning. A quarter of the students came 90 minutes late, presumably to mark their attendance? There was much moving around and off-line discussions. This probably is the reason why the on-line discussions were hardly used. Attendance dwindled to about 50% by the third WBL scheduled session (see Figure 6.7). Students who turned up were observed to work on their own most of the time with a few off-line discussions. They took longer to finish the materials ranging from 45 minutes to an hour compared to the first session. Again page surfing was the main mode of interaction as most students worked quickly to finish reading the materials.

20 No. of Students 15 10 5 0 8 8.5 9 Tim e 9.5 10 6 7 12 9 8 9 8

Figure 6-14 Attendance profile of class of 21 students at third WBL session It was observed that although there were some students who ventured to the bulletin boards, the usage of email, bulletin boards and announcements was low. This could be due to the on-campus mode for this implementation of WBL.

66

Chapter 6 Results Interview Outcomes In this study, interviews with staffs and students were conducted to help clarify the results of the classroom observations and survey questionnaire. It was hoped that this will help answer some of the issues brought up in the survey. Interviews with the students were conducted during the classroom observations while interviews with staff developers were conducted at the end of the semester after the completion of the entire WBL trial. One of the outcomes from this evaluation was the clarification of the views of the staff developers. Like faculty in other higher education institutions (Wilson 2001), staff reported they were under time pressure, unrewarded for work in WBL and underprepared for the wide variety of roles expected of them. The staffs involved were generally concerned over the wide diversity of the roles they have to play. In addition to their normal curriculum development and delivery, time had to be spent on additional roles spanning from administration of the WBL platform, programming, development of media resources, planning and integration of the WBL materials. Development of the multimedia materials was extremely time consuming. One developer quoted 200 hours of time to develop 4 hours of training materials. Much time was also required to read up on the subject domain - you need to know the content well before you can plan to present the material. Another issue brought up by staff developers included the perceived lack of institutionalwide policy or direction on WBL. As there was more than one WBL platform (Blackboard, TopClass and in-house developed Virtual College) being used within SP among the various departments and schools, staff were concerned that any subsequent institutional-wide direction towards a different WBL platform could render their work 67

Chapter 6 Results obsolete. The author deems this a valid concern which SP should address by exercising its leadership in setting the direction in the choice of a WBL platform. The consequence would be a waiting game while staffs wait for SP management to make a decision and SP management wait for staff to come out with a de-facto decision. Other issues include lack of recognition for development work in WBL during staff review or promotion, lack of time to develop skills or technology, no support for media creation, limited access to technology resources, not being allowed to focus on WBL development, unavailability of instructional designer to help jumpstart design and development. All of these are real WBL support issues which the institute is not prepared to spend resources on. These may inhibit other staff from getting into WBL if not addressed. In spite of the perceived poor institutional support, staffs involved were generally highly motivated to try out WBL. Although the lecturers involved in this study have strong technical expertise in using computers, WWW and multimedia, they still experienced difficulty when trying to integrate computers into their teaching. The most notable was lack of time to develop pedagogically sound WBL materials. This finding was also confirmed by other research (Jacobsen 1998). Despite all these issues, staff still found incentives to continue developmental work on WBL. The incentives the developers cited are personal gratification and personal desires make their subject more interesting. The pride in their work was apparent from a developer who was concerned about having to deliver the WBL module when it was not completed to her satisfaction. However, it may not be realistic to expect all other staff to find the same motivation to work on WBL.

68

Chapter 6 Results The developers also observed that students were not ready or primed to go on-line. They felt that students do not want to be responsible for their own learning. They also observed that students exploit the anytime, anywhere mode of learning to spend time on more demanding modules or projects. In terms of course administration, staffs see the use of a WBL platform as helping to cut down administration and saves time. Time was saved in administering and marking of quizzes and tests. Staff also cited the convenience of all module related information such as syllabus, schedule, course materials, assignments, discussions being located in one secure place. On the other hand, administration of WBL platform turned out to be extremely time consuming. This included liaising with the vendors for licensing of tool, setting up student accounts within appropriate classes, teaching accounts for staff, etc. The staff developers suggested that full time staff may need to be hired to take care of this administrative aspect if WBL is to be implemented institutional wide or even school wide. One of the staff developers cited the most challenging tasks as deciding on the right tool to use, instructional design to attract students to use the program and integration of the module. These are important decisions which may affect the outcome of the WBL as well as its effectiveness for students learning. Summary of Results A fifth of the students in this study as well as in another unrelated study do not find WBL useful. Coincidentally, an earlier study done by the same author showed that a fifth of this cohort of students is the NF MBTI type. This type is the people-oriented type of students who preferred face-to-face communication work rather than working on the 69

Chapter 6 Results computer. There could be some correlation between these 2 factors which could form the basis of a further study on how learning styles can influence the effectiveness of WBL. The majority of the students indicate they feel positive about the use of developed WBL. Even with the amount of interactivity built in, the students would like to have even more interactivity! The students desire for higher interactivity is consistent with established literature on the on-line instructional design. One interesting unintended finding was the poor level of English literacy demonstrated by the students in their survey feedback (see Appendix 2) and response to the survey questions. This is not surprising given the educational background of these vocational students whose mother tongue is not English. The poor English literacy will indirectly affect the effectiveness of WBL as it is fully communicated in English. It would be expected that given the findings of the evaluation, implementation of WBL by itself would not be of much use to improve learning.

70

Chapter 7 Discussions & Implications Chapter 7 Discussion & Implications A central motivation for this research is to determine how to best implement Web-based Learning within the EEE School and to gauge both the staff and students response to WBL. This study takes as a basic assumption that IT infrastructure is not a problem and both students and staff are WWW savvy as staffs are teaching and students are learning computer systems as part of their course work. It was also assumed that the students knew how to make use of the WBL tool or the WBL tool (TopClass) was simple enough to be used without prior training. In other words, the results present a best case scenario. The same study with a less WWW savvy group of lecturers and students may possibly produce a less positive result. The questions addressed by this study were formulated within a particular set of constraints. These include the authors ability to conduct the study within the limited timeframe and the scope of this dissertation. It is evident that there are many questions which could be delved further into but was omitted due to the limitation of the scope. What was left out? As this is a limited study, there are many areas which have been left out due to time, space and other constraints. There was no study on the outcomes (course grade). This was due to the confidentiality of the examination results. No analysis was done on the suitability of the course contents for on-line delivery. How do we know if the WBL material developed is appropriate? A rubric or way to measure the quality of on-line learning has to be devised. This can be difficult as it can be very subjective and is beyond the scope of this limited study. However, it is the authors view that every attempt has 71

Chapter 7 Discussions & Implications been made to ensure that the WBL contents designed are sound pedagogically. Students are generally happy with the materials as seen from the results of the study. Areas for Further Study The results from this study also opened up new questions that would be interesting and illuminating to explore further. This includes the idea of learning styles conducive towards WBL, a typical Singaporean or Asian student response to WBL as well as ways to measure staff or student readiness for WBL Is there a suitable learning style for WBL? There was also no study on how the delivery mode, course contents and students learning styles interact. It was found from this study as well as another unpublished study that a fifth of the students did not welcome WBL. This could be due to a difference in learning styles. Is it possible to design WBL so that this group of students can also be accommodated? Or is keeping the traditional face-to-face learning the only option for this group of students? Is there a typical Asian/Singaporean student response to WBL? The survey results showed that nearly all students do not see discussions and emails as a learning tool. This is in contradiction with most Euro-centric studies that highlighted the importance of discussion as a form of learning. The traditional Asian learning model has been the one-way (teacher to student) objectivist view that the teacher knows all. Learning through collaboration, though seen as desirable, has never been actively emphasized within the SP or EEE context. This attitude will need to be changed for

72

Chapter 7 Discussions & Implications WBL model to be effective. Is this attitude peculiar to SPs vocational students or is it prevalent throughout Singapore or even in Asia? How do we measure staff or student readiness for WBL? Observations from the study showed that some students may not be ready for WBL. It was seen that factors such as computer and language literacy, motivation, attitudes and independence affect the successful outcome of WBL. What are the factors that are necessary or conducive to WBL and how do we measure them? What are the factors affecting staff? What are the factors affecting students? Chapnick (2000) defines a readiness model within eight categories of factors: Psychological, Sociological, Environmental, Human Resource, Financial, Technological Skill, Equipment and Content. With different stakeholders such as staff or students, the combination of factors would have to be examined differently as the interests and impact would be different. Intangible Results There are many measures and results that are not tangible or easily measurable. One of the more obvious is an improvement in the quality of the learning experience, and a shift from passive to active learning. A successful implementation of WBL is also reflected by a change in the institutional culture, especially in the ability to exploit information technology. From a pedagogy perspective, intangible measures can be improvements in transferable skills such as independent study or IT knowledge, improvements in teaching material quality, increasingly flexible student access to learning materials, from both on-site and off-site (via computer connections).

73

Chapter 7 Discussions & Implications Implications of Findings The best WBL should leverage the unique advantages of the WWW with tried and true pedagogical practices. This discussion addresses the findings from this WBL evaluation on from 3 perspectives affecting the quality of WBL: 1. students, 2. staff and 3. instructional design. The students perspective covers the possible effects of students learning styles on WBL, student readiness for effective learning via WBL as well as providing a local perspective from an Asian and Singaporean student viewpoint. From the staff perspective, what are the changes required in the roles, curriculum development and presentation for effective delivery of WBL? The instructional design viewpoint covers considerations for WBL within the context of the whole learning experience. Implications on Students How do the students view WBL? A study by Goldfinch and Davidson has shown that factors influencing students view of Computer or Web Based Learning are imminence of assessments, enthusiasm of staff, perception of match with lectures, speed of equipment, amount of feedback and provision of accompanying paper based support material. (Harvey 1998, p81). These factors are generally similar to those for the traditional class except for the equipment related factors. However, there is a clear difference between WBL and the traditional class and we need to account for this difference. In a WBL setting, there is no captive audience. Students control the pace and sequence of learning and instructors are not present to answer questions. 74

Chapter 7 Discussions & Implications The research surrounding the characteristics of an effective on-line learning student generally agrees: students should be mature, assertive, self-disciplined, and independent (Buchanan 1999), should be able to shape and manage change (Rogers 2000), and, most importantly, must be highly motivated and possess well-developed, self-directed learning skills (Carlson and Repman 2000). This was also evident from the results of this study which indicated that students generally put in less effort when using WBL and there is a need to explicitly motivate learners in WBL through the online materials. It can thus be seen that students are not always ready for WBL. A study by IHEP has identified some student characteristics that correlate to successful on line learning. These characteristics are: 1. Students who rated themselves highly on various measures of persistence related to taking on new projects 2. Students who rate the consequences of not passing as serious 3. Students who rated their chances of succeeding in their studies higher than non-completers 4. Students who do not need support from others to complete difficult tasks and did not find it important to discuss course work with other students 5. Students with high literacy levels 6. Students who rated themselves as well organized in terms of time management skills and said they generally had the time to do what they intended to do 7. Students who rated their formal and informal learning as high in terms of preparing them for university studies and 8. Female students. (p 14, IHEP, 1999) The author can identify with the fifth characteristic above. Within the local context, students here have poor English Literacy skills (as evidenced from personal experience and responses to open ended survey questions see Appendix B). In the classroom 75

Chapter 7 Discussions & Implications environment, poor English can be overcome by the lecturer through good classroom practices but in an online environment, the students are left very much on their own. There is thus a need to build up literacy: English and Reading skills. This study also showed that students here generally do not want to take responsibility for own learning, need face to face support, needs direction, lacks learning maturity, lacks independence, lacks discipline and lacks responsibility. Again this is supported from personal experience and conversations with other teaching staff. From the study it was observed that students tend to put away learning tasks to the last minute as they perceive the WBL material is always there. Materials are also skimmed through as students expect that they will be able to return to the information source when they need it. Many of the above characteristics from the IHEP study are motivational and self-directive factors. The question is how we can train students to be independent and self directed when going onto WBL. Conrad (1999) proposes there are 4 stages towards self direction, i.e., from Dependent to Interested to Involved to Self-directed, and proposes different strategies for developing each stage. How then can we prepare the students for WBL? Motivational factors are different, concentration levels change, pacing is modified, integration of skills or learning into the workday varies, and even note-taking is altered. If we accept how new and different e-learning can be, we need to give learners information on how to cope with this new environment. (SVI 2002) One lesson learnt from this study is the need to prepare students how to learn through WBL. The author observed a need to instil responsibility and self direction for learning within students mindset for WBL to be effective. This student preparation is essential as a pre-requisite for effective WBL. 76

Chapter 7 Discussions & Implications Implications on Staff A paradigm shift in the attitudes and mindsets of lecturers is required for WBL to be successful. The role is no longer that of lecturing alone but more of a facilitator, mentor, advisor or tutor. Other paradigm shifts include that of education from Mass Lectures to Individualised learning, from students as passive consumers of knowledge to active participants and creators of knowledge, and from emphasising classroom teaching methods to Instructional Design. Staffs as WBL developers want to see a clear direction/policy being set by the institution. They would like to see recognition for work done as well as time to focus on development work. However, even in institutions with a clear WBL policy and

recognition for WBL work, success rate of WBL still depends on a lot more intangible factors Implications on Instruction Design Traditional teaching is different from WBL. Shortcomings in teaching materials are overcome by the lecturers response. WBL does not have the luxury, as it must anticipate all possible responses. The students do not have anyone to ask at that instant. WBL materials have to anticipate the different responses of the students and cater to them. LaMonica (2001) listed from prior findings the best practices in WBL from other research 1. 2. 3. 4. 5. State expectations of students and the course clearly and early Structure content so that it is easy to follow Provide regular feedback and guidance to the students Provide opportunity for students to give feedback Strive for participation by all students 77

Chapter 7 Discussions & Implications 6. Promote collaborative learning She made a study to determine which of these practices was seen by instructors and students as most important. The results showed that all instructors and students agreed that the best practice course content is structured so it is easy to follow is the most important. WBL is then no different from traditional teaching in the sense that appropriate instructional design is necessary for instructional effectiveness. The crucial difference is perhaps the WBL technology which affords much wider pedagogical impact compared to traditional learning media such as chalkboard and overhead projectors. Instructional Design has been, is now, and will continue to be the primary solution to successfully moving into instructional effectiveness. Briggs (1977) described instructional design as the entire process of analysis of learning needs and goals and the development of a delivery system to meet the needs. He also included the development and try out of all instruction and learner assessment activities. In the SP teaching environment, it was observed that WBL is usually added as an afterthought after the curriculum for the modules have been designed and developed. This solution is always supplemental or remedial in nature. Generally, there has been no comprehensive redesign to assign appropriate portions of instruction to the most effective delivery or evaluation component. Schacter (1999) concluded from his studies of that learning technologies will be less effective when objectives are unclear and focus of technology is diffuse. It is true in the case here when developers are not clear of the objectives of WBL; the materials developed will not be effective.

78

Chapter 7 Discussions & Implications Conclusion The goals of IT, and hence, WBL in the Singapore Polytechnic as spelt out in the SP Education Model is to equip our students with basic IT knowledge and skills, teach them how to access information independently and learn collaboratively, and use IT to help develop higher-order thinking and lifelong learning skills (p. 21, SG 2001) The success or failure in meeting these goals would be a measure of the effectiveness of WBL within the SP context. The goals of WBL for an educator are to find new ways for students to experiment and solve problems while providing a safety net. It is also to find ways to offload menial parts of instruction to the computer. When all is said and done, it was realized that the adoption of WBL technology is similar to the adoption of other educational innovations; it is just more time consuming, labour intensive, and expensive. Is it worth it? Probably. Do we have a choice? Not really. Schools and universities all over the world have jumped onto the WBL wagon and it is not longer a question of Is it worth it to embark on WBL? but how do we make WBL worth it? Web based technology does not eliminate the work or cost. It merely changes where and when we spent our resources. There are advantages and disadvantages to WBL, compared to traditional face-to-face learning. Each learning model has its strengths and weaknesses. The challenge is to use each medium -- books, games, television, classrooms, computers, etc. -- in the best way possible to enrich the learning experience.

79

Chapter 7 Discussions & Implications Idea is to take the best from each medium to provide the best learning experience for the students. The problem is there is no one best for all the different learning styles of the students. WBL may never fully replace printed paper as paper is an extremely effective and flexible medium. The blended approach using a combination of WBL and conventional books appears to work best as it can exploit the best of both traditional and WBL. (Jones & Wright, 1999) We have to be able to recognize what WBL can or cannot do effectively with or without human assistance. It will be worth it if the use of WBL enables the student to do something they could not do before or they could do before but better. Putting all the curriculum up on-line with bells, whistles and interactivity will provide a great learning experience for some students - but will not be the right answer for students who are not able to take responsibility for their own learning and need face-to-face help and support. In conclusion, we have to bear in mind that technology does not teach, it is the effective teachers who do teach. Excelling in teaching should be first priority because adopting technology such as WBL will not improve poor teaching (Kearsley, 1996). Its the People who make the difference.

80

BIBLIOGRAPHY Boyle, T. (1997) Design for Multimedia Learning, London, Prentice Hall. Bracewell, R. & Laferriere, T. (1996) The Contribution of New Technologies to Learning & Teaching in Elementary and Secondary Schools, Universite Laval & McGill University. [Online]http://www.education.mcgill.ca/fedwww/cils/Telelearning/TICe.htm [Online]http://www.tact.fse.ulaval.ca/fr/html/impactnt.html Brahler, C. J. (1999) Developing on-line learning materials for higher education: An overview of current issues, Educational Technology & Society, Vol 2, No. 2, 1999. [Online]http://ifets.gmd.de/periodical/vol_2_99/jayne_brahler.html Brightman, H. J. (1997) GSU Master Teacher Program: On Learning Styles, Georgia State University.[ONLINE] http://www.gsu.edu/~dschjb/wwwmbti.htm Britain, S. & Liber, O. (1999) A Framework for Pedagogical Evaluation of Virtual Learning Environments, JISC Technology Applications Programme (JTAP) Report, Joint Information Systems Committee, UK. [Online]http://www.jtap.ac.uk/reports/htm/jtap041.html Bruce, B. C. & Levin, J. A. (1997) Educational Technology: Media for Inquiry, Communication, Construction, and Expression, Journal of Educational Computing Research, Vol. 17(1), pp. 79-102. [ONLINE] http://www.ed.uiuc.edu/facstaff/chip/taxonomy Byrom, E. () Review of the Professional Literature on the Integration of Technology into Educational Programs, The South East & Islands Regional Technology in Education Consortium, [Online] http://www.serve.org/seir-tec/publications/litreview.html Caviedes, J. (1998) A Technological Perspective of Anytime, Anywhere Education, ALN Magazine, Vol. 2, Issue 1, March 1998. [Online]http://www.aln.org/alnweb/magazine/vol2_issue1/jorge.htm Daniel, J. S. (1997) Why Universities Need Technology Strategies: Twenty-seven Ways of Looking at a Classroom, Change, The Magazine of Higher Learning, Vol. 29, No. 4, July/Aug 1997, pp. 10-17. Davidson, K. & Goldfinch, J. (1998) How to Add VALUE, in Mogey, N. (ed) LTDI Evaluation Studies, Learning Technology Dissemination Initiative, Edinburgh. pp. 4-12. Dearing, R. (1997) Higher Education in the Learning Society, National Committee of Inquiry into Higher Education, Middlesex, NCIHE Publications. [ONLINE]http://www.leeds.ac.uk/educol/ncihe EdWeek (1998) Technology Counts 98, Putting School Technology to the Test, Education Week on the Web, October 1998, [ONLINE]http://www.edweek.org/sreports/tc98 81

Bibliography Eliot, T. S. (1948) Chorus from The Rock, I, Selected Poems, London, Faber & Faber. [ONLINE] http://www.rjgeib.com/thoughts/information/information.html Gates, W. H. (1996) The Road Ahead, (Rev. Ed.), London, Penguin Books. Gould, L. & Gould S.L. (1998) "Education and the Net Generation: Lead, Follow, or Get Out of the Way?" On the Horizon, 6(3), pp 9-10. Green, K. (1998) The 1998 National Survey of Information Technology in Higher Education, The Campus Computing Project, November 1998, Encino, CA, USA. [Online] http://www/campuscomputing.net/summaries/1998/index.html Harris, R. (1997) Evaluating Internet Research Sources, Southern California College. [ONLINE]http://www.sccu.edu/faculty/R_Harris/evalu8it.htm Hillis, W.D., (1998) The Pattern on the Stone: The simple ideas that make computers work, NewYork, Basic Books. Jones, B. F., Valdez, G., et al (1995)Plugging In, choosing and using educational technology, EdTalk, Council for Educational Development and Research, North Central Regional Educational Laboratory, Washington DC. [Online]http://www.ncrel.org/sdrs/edtalk.toc.htm Landauer, T. K. (1988) Education in a World of Omnipotent and Omniscient Technology in Nickerson, R. S. & Zodhiates, P. P. (Eds) (1988) Technology in Education: Looking Towards 2020, New Jersey, Lawrence Erlbaum Associates, pp. 1124. Lelouche, R. (1998) The Successive Contributions of Computers to Education: A Survey, European Journal of Engineering Education, September 1998. Mary L. McNabb, M. L., et al (1999 ) Technology Connections For School Improvement: Planners' Handbook, North Central Regional Educational Laboratory, U.S. Department of Education MOE (1997) Information Literacy Guidelines, Languages & Library Branch, Curriculum Planning & Development Division, Singapore, Ministry of Education. MOE (1998) The Desired Outcomes of Education, Singapore, Ministry of Education. [Online] http://www1.moe.gov.sg/desired.htm Mogey, N. & Watt, H. (1996) The Use of Computers in the Assessment of Student Learning, Implementing Learning Technology, Learning Technology Dissemination Initiative. [ONLINE]http://www.icbl.hw.ac.uk/ltdi/implementing-it/using.htm Morrison, J. (1999) "Transforming the Role of Students and Teachers in the Information Age." On the Horizon, 7(2), pp2-3.

82

Bibliography Muraskin, L. (1993) Understanding Evaluation: The Way to Better Prevention Program, Westat Inc. National Research Council (1999) Being Fluent with Information Technology, (prepublication copy), National Academy Press, Washington DC. [Online]http://bob.nap.edu/readingroom/books/BeFIT/ NCES (1997) Technology @ Your Fingertips: a guide to implementing technology solutions for education agencies and institutions, National Center for Education Statistics, U.S. Department of Education, Washington, DC. Neal, E. (1998) Does Using Technology in Instruction Enhance Learning?, The Technology Source, June 1998. [Online]http://horizon.unc.edu/TS/commentary/199806.asp Negroponte, N. (1995) Being Digital, New York, Alfred A. Knopf. Opie, C. & Wellington, J. (1997) IT A Help or Hindrance to Education?, Module 6, Unit 1, Information Technology & Education, Sheffield, University of Sheffield Division of Education, pp. 12-15. Oppenheimer, T. (1997) The Computer Delusion, The Atlantic Monthly, July 1997, Vol. 280, No. 1, pp. 45-62, Boston, Massachusetts. [Online]http://www.theatlantic.com/atlantic/issues/97jul/computer.htm [Online]http://www.virtualschool.edu/mon/Academia/ComputerDelusion.html OTA (1995a), Teachers and Technology: Making the Connection, Office of Technology Assessment, OTA-EHR-616, April 1995, Washington, DC, US Government Printing Office. [Online]http://www.wws.princeton.edu/~ota/disk1/1995/9541.html OTA (1995b), Education and Technology: Future Visions, Office of Technology Assessment, OTA-BP-EHR-169, September 1995, Washington, DC, US Government Printing Office. [Online]http://www.wws.princeton.edu/~ota/disk1/1995/9522.html Papert, S. (1993) The Childrens Machine - Rethinking School in the Age of the Computer, NewYork, Harvester Wheatsheaf. Papert, S. (1996) The Connected Family - Bridging the Digital Generation Gap, NewYork, Basic Books. Perelman, L. J. (1993) School's Out: Hyperlearning, The New Technology, and the End of Education, New York, Avon Books. Postman, N. (1992). Technopoly: The surrender of culture to technology. New York: Alfred A. Knopf, Inc. Rawlings, G. (1997) Slaves of the machine: the quickening of computer technology, London, The MIT Press. 83

Bibliography Russell, T. L. (1997) The "No Significant Difference Phenomenon, 4th Edition, [Online]http://tenb.mta.ca/phenom/phenom.html Ruttenbur, B. W., Spickler, G. C. & Lurie, S. (2000) Elearning, the Engine of the Knowledge Economy, Morgan Keegan & Company, Inc., New York. Schn, D. (1983) The Reflective Practitioner: How Professionals Think in Action, New York, Basic Books. Schutte, J. G. (1997) Virtual Teaching in Higher Education: The New Intellectual Superhighway or Just Another Traffic Jam? [Online]http://www.csun.edu/sociology/virexp.htm Shute, V. J. & Psotka, J, (1996) Intelligent Tutoring Systems: Past. Present, and Future, Armstrong Laboratory. [Online] http://www.brooks.af.mil/AFRL/HEJ/icai/its.htm SPA (1996) Report on the Effectiveness of Technology in Schools, 95-96: Executive Summary, Software Publishers Association. [Online]http://www.spa.org/project/edu_pub/summary.htm Spady, W. (1999) "School Reform: Rushing Backward toward the Future." On the Horizon, 7(2), pp 1,4-7. Stahlke, H. F. W. & Nyce, J.M. (1996) Reengineering Higher Education: Reinventing Teaching and Learning, CAUSE/EFFECT, Vol. 19, No. 4, Winter 1996, pp 44-51. [Online]http://www.educause.edu/ir/library/html/cem9649.html Steiner, P. (1993) On the Internet, nobody knows you are a dog, The New Yorker, July 5 1993, New York, The New Yorker Magazine Inc. Stoll, C. (1995) Silicon Snake Oil: Second Thoughts on the Information Highway, New York, Doubleday. Tapscott, D (1999) The Net Generation and the School Milken Exchange on Education Technology [Online] http://www.milkenexchange.org/feature/tapscott_full.html Teo, C. H. (1997) Speech at the Launch of the Masterplan for IT in Education, 28 Apr 97, Singapore. [Online]http://www1.moe.edu.sg/Speeches/280497.htm Teo, C. H. (1998) Educating Tomorrow's Global Citizen : What Should We Be Teaching Today's Youth? 1998 World Economic Forum Annual Meeting, 3 Feb 98, Singapore. [Online]http://www1.moe.edu.sg/Speeches/030298.htm the NODE (1999) The Laptop College, Learning Technologies Report, April 1999, the NODE Learning Technologies Network. [Online]http://node.on.ca/ltreport/april1999/ Underwood, J. & Underwood, G. (1990) Computers & Learning - Helping children acquire thinking skills, Oxford, Blackwell Publishers. 84

Bibliography Urdan, T. A. & Weggen , C. C. (2000) Corporate e-learning: Exploring a new frontier, WR Hambrecht + Co. Valdez, G. (1999) Technology, A Major Catalyst for Educational Improvement, Milken Exchange on Education Technology [Online]http://www.milkenexchange.org/feature/valdez_full.html WBEC (2000) The Power Of The Internet For Learning: Moving From Promise To Practice, Report Of The Web-Based Education Commission, December 2000, Washington DC.

85

REFERENCES Bell, J. (1993) Doing Your Research Project: A guide for First Time Researchers in Education and Social Science. (2nd Ed) Open University Press, Buckingham. Briggs, L.J. (1977) Instructional Design Principles and applications, Englewood Cliffs, M.J.: Educational Technology Publications Boettcher, J. & Conrad, R. (1999) Faculty Guide for Moving Teachers and Learning to the Web, Los Angeles, CA: League for Innovation in the Community College, p.16. quoted in IHEP (2000) Quality on the Line, Benchmarks for Success in Internet-Based Distance Education, Institute for Higher Education Policy, Washington, DC. Brightman, H. J. (1997) GSU Master Teacher Program: On Learning Styles, Georgia State University. [Online] http://www.gsu.edu/~dschjb/wwwmbti.htm Britain, S. & Liber, O. (1999) A Framework for Pedagogical Evaluation of Virtual Learning Environments, JISC Technology Applications Programme (JTAP) Report, Joint Information Systems Committee, UK. [Online]http://www.jtap.ac.uk/reports/htm/jtap041.html Brown, B W & Liedholm, C E (2002) Can Web Courses Replace the Classroom in Principles of Microeconomics? American Economic Review, May 2002, 92(2), pp. 44448. Carliner, S. (1999) An Overview of Online Learning, VNU Business Media, HRD Press. [Online] http://www.lakewoodconferences.com/wp/ Carnevale, D. (2000). Study assesses what participants look for in high-quality online courses. Chronicle of Higher Education, 47, 9, A46. Chan, K. K. & Sweeters, B. (1994) Comparison of Tutorial Computer-Assisted Instruction and the Lecture/tutorial Method for the Delivery of a Lesson in Construction Technology, Journal of Teaching Practice 1994, Singapore Polytechnic, Singapore. Chapman, B (1999) Through the Eyes of Visionaries: A State of the Industry Report, Macromedia UCON99 [Online]http://www.internettime.com/itimegroup/visionary_files/index.htm Chapnick, S. (2000) Are You Ready for E-Learning, ASTD Learning Circuits, American Society for Training and Development, Nov 2000, USA, Virginia. [Online]http://www.learningcircuits.org/nov2000/chapnick.html Chickering, A. & Ehrmann, S. C. (1996) Implementing the Seven Principles: Technology as Lever AAHE Bulletin, October, pp3-6 Conrad, R. M. (1999) "Developing Learner Self-Direction in a Webcentric Learning Environment." Proceedings 15th Annual Conference on Distance Teaching and Learning, August 4-6, 1999. 86

References COU (2000) A Time To Sow, Report From The Task Force On Learning Technologies, COU No. 671, March 2000, Council of Ontario Universities, Toronto, [ONLINE]http://www.cou.on.ca/publications/briefs_reports/online_pubs/ATS.pdf Davidson, K. & Goldfinch, J. (1998) How to Add VALUE, in Mogey, N. (ed) LTDI Evaluation Studies, Learning Technology Dissemination Initiative, Edinburgh. Pp. 4-12. Ehrman, S. C. 1997, Asking the Right Question: What Does Research Tell Us about Technology and Higher Learning? [Online]http://www.learner.org/edtech/rscheval/rightquestion.html Fan, K S, et al (2001) Students Perception on Online Learning: A Pilot Study on Teaching Construction Materials in Singapore Polytechnic, Journal of Teaching Practice (2000/2001) Singapore, Singapore Polytechnic. Green, K C (2001) Campus Computing 2001 The 12th National Survey of Computing and information Technology in US Higher Education, Oct 2001 EDUCAUSE Conference Session Handouts. [Online]http://www.campuscomputing.net/archive/GreenCC2000.PDF Green, K C (2002) Campus Computing 2002 The 13th National Survey of Computing and information Technology in US Higher Education, Oct 2002 EDUCAUSE Conference Session Handouts. [Online]http://www.campuscomputing.net/archive/educause-2002Session.pdf Hall (2003), Learning Management Systems and Learning Content Management Systems Demystified, [Online]http://www.brandonhall.com/public/resources/lms_lcms/ Harvey, J (1998) LDTI Evaluation Cookbook, Learning Technology Dissemination Initiative, Heriot-Watt University, Edinburgh. [Online]http://www.icbl.hw.ac.uk/ltdi/cookbook/ Hawkes, M. (2001) Criteria for Evaluating School-Based Distance Education Programs, Learning Through Technology: A Planning and Implementation Guide, NCREL [Online]www.ncrel.org/tandl/eal4.htm Hazari, S. I. (1998) Evaluation and selection of web course management tools. [Online]http://sunil.umd.edu/webct IHEP (1999) Whats the Difference, A Review of Contemporary Research on the Effectiveness of Distance Learning in Higher Education, The Institute for Higher Education Policy, Washington, DC. IHEP (2000) Quality on the Line, Benchmarks for Success in Internet-Based Distance Education, The Institute for Higher Education Policy, Washington, DC. [Online]http://www.ihep.com/quality.pdf Jacobsen, D. M. (1998) Adoption patterns of faculty who integrate computer technology for teaching and learning in higher education, Proceedings of the ED-MEDIA and ED87

References TELECOM 98: World Conference on Educational Multimedia and Hypermedia & World Conference on Educational Telecommunications, Freiburg, Germany, June 20-25. Jones, R. & Wright, C. (1999) Online Guided Learning, Educational Technology and Society Vol. 2 No. 3, July 1999, Journal of International Forum of Educational Technology & Society & IEEE Learning Technology Task Force [online]http://grouper.ieee.org/groups/ltsc/ifets/periodical/vol_3_99/ray_jones.html Kearsley, G. (1996) Teaching Excellence: The Foundation for Technology Effectiveness. Educational Technology Review, 6, 24-27. Khan, B.H. (Ed.) (1997) Web-Based Instruction, Educational Technology Publications, 1997. Khan, B. H. (2001). A framework for Web-based learning. In B. H. Khan (Ed.), Webbased training. Englewood Cliffs, NJ: Educational Technology Publications. Kirkpatrick, D.L. (1994). Evaluating Training Programs: The Four Levels. San Francisco, CA: Berrett-Koehler. Manwaring, G. (1998) Directing your Evaluation, in Harvey (1998) McCrea, F., Gay, R. & Bacon, R. (2000) Riding the Big Waves, A White Paper on the B2B e*Learning Industry, January 18, 2000, Thomas Weisel Partners, San Francisco. MOE (1997) Masterplan for IT in Education: Summary, Singapore, Ministry of Education. [ONLINE] http://www1.moe.gov.sg/iteducation/masterplan/ Morgan, G. (2003) Faculty use of Course management System, ECAR Key Findings, May 2003, EDUCAUSE Center for Applied Research. [ONLINE]http://www.educause.edu/ir/library/pdf/ecar_so/ers/ers0302/ekf0302.pdf Muraskin, L. (1993) Understanding Evaluation: The Way to Better Prevention Program, Westat Inc. Muilenburg, L. & Berge, Z. L. (2000) A Framework for Designing Questions for Online Learning, DEOSNEWS, Vol 10, No. 2, The American Center for the Study of Distance Education at The Pennsylvania State University, [Online]http://www.ed.psu.edu/acsde/deos/deosnews/deosnews10_2.asp Ng H. K. (1998) Learning Styles of New Engineering Students, unpublished paper for the Diploma/M.Ed. course, Sheffield University and Singapore Polytechnic, January 1998. Oliver, M. (1998) Evaluating communication and information technologies: a toolkit for practitioners, Active Learning 8, July 1998.

88

References Oliver, R, et al (1996) Creating Effective Instructional materials for the World Wide Web, AusWeb96 - Second Australian World Wide Web Conference, Australia. [Online]http://wwwdev.scu.edu.au/sponsored/ausweb/ausweb96/educn/oliver/ Owsten, R. (1997) The World Wide Web: A Technology to Enhance Teaching and Learning? Educational Researcher, Vol. 26, No. 2, March 1997, American Educational Research Association, Washington, DC. [Online]http://www.edu.yorku.ca/~rowston/article.html Peterson, R., Marostica, M. & Callahan, L. (1999) e-learning: Helping Investors Climb the e-Learning Curve, November 1999, U.S. Bancorp Piper Jaffray Equity Research, Minnesota. Quiones, S. & Kirshtein, R. (1998) An Educators Guide to Evaluating the Use of Technology in Schools and Classrooms, US Department of Education, Office of Education Research and Improvement, Washington DC. [Online]http://www.ed.gov/pubs/EdTechGuide/ Ragan L. C. (1998) Good Teaching is Good Teaching: An emerging set of guiding principles and practices for the design and development of distance education, DEOSNEWS, Vol 8 No 12. American Centre for the Study of Distance Education, Pennsylvania State University, USA Relan, A. & Gillani, B. (1997) Web-Based Instruction and the Traditional Classroom: Similarities and Differences, Ch. 4 in Khan 1997 Schacter, J. (1999) The Impact of Education Technology on Student Achievement: What the Most Current Research has to Say, Milken Exchange on Education Technology. Sherry, L. (1996). Issues in Distance Learning International Journal of Educational Telecommunications, 1 (4), AACE 337-365. SG (2001) The Singapore Polytechnic Education Model (Release 2), Strategic Unit, Document No. SP-DOC-M01-001, 2 Feb 2001, Singapore, Singapore Polytechnic. Sivin-Kachala, J. (1998) Report on the effectiveness of technology in schools, 19901997, Software Publishers Association. SVI (2002) The Culture of e-Learning, E-Learning Bytes, Strategic Visions International [Online]http://www.stratvisions.com/e-learning-bytes/17-culture.html Valdez, G. (1999) Evaluation Standards and Criteria for Technology Implementation, Learning Through Technology: A Planning & Implementation Guide, North Central Region Educational Laboratory, Illinois. [Online]http://www.ncrel.org/tandl/eval1.htm Wilson, C. (2001). Faculty attitudes about distance learning. EDUCAUSE Quarterly, Volume 24, Number 2, 2001, p70-71. [Online]http://www.educause.edu/ir/library/pdf/eqm0128.pdf 89

APPENDIX 1 Survey on the use of Web Based Learning for Multimedia Development in the Electronics & Communication Engineering Department, Singapore Polytechnic 17th April 2000 Dear DECC 3FT CST students, As part of the IT in Teaching & Learning plans for the EC Department, we are conducting a small scale study on the effectiveness of Web Based Learning (WBL) for the Multimedia Development module in the Electronics & Communication Engineering Department. The aims of this survey are to solicit your views on the effectiveness of WBL and the materials developed for your learning. We are also interested in your experience in the current WBL implementation for Multimedia Development using TopClass. Your response is important as it can help us to improve on the current module as well as our future WBL developments. Please be assured that all responses will be kept confidential. Please complete the survey and return it to me by 21st April 2000. You can contact me at 7721448 or email hweekiat@sp.edu.sg if you have any queries regarding this survey. Thank you for your co-operation.

_______________ Ng Hwee Kiat Electronics & Communication Engineering Department Singapore Polytechnic NOTICE
Please read all questions and instructions carefully. Answer the questions by the appropriate box or writing in the answer if requested to do so. It is important to limit yourself to one response for questions that direct you to one.

90

Appendix 1
Your participation in this survey is voluntary, and you may choose not to reply to any question. We ask, however, that you fully and accurately complete this survey. All answers will be used for statistical purposes only.

91

Appendix 1
Module Class: ________________ On the Learning Materials
Please the response (one only) which most closely represent your opinion of the usefulness of the each of the resources used in this module. Resource 1 2 3 4 5 Supervised Laboratory WBL Module Materials Discussion List TopClass Announcements TopClass Email Not useful at all Not Usefu Very Extremel very l usefu y useful l useful

Date: ______________________

Please add any other comments you wish

On the WBL programme

Please the response (one only) which most closely represent your opinion of the statements (SD - Strongly Disagree, D - Disagree, N - Neutral, A - Agree, SA - Strongly Agree) To what extent do you agree with these descriptions of the WBL programme? 6 It is easy to use the user interface is easily understood 7 It is enjoyable to use 8 It provides good support for my learning 9 It has clear instructions on how to work through the material 10 It helps me learn about Multimedia Development module 11 It fits well with the rest of the module materials 12 It helps me to revise the subject Please add any other comments you wish SD D N A SA

13 How many hours did you spent on the TopClass WBL package in total for this module? <6 hrs 6 to 12 hrs 13 to 18 hrs >18 hrs Please rank the following factors in order of importance that kept you interested or would keep you interested in the WBL programme. (from A to E where A represents most important and E least important). Please use each ranking once only. Factors 1 4 1 Highly Interactive Contents High Multimedia Contents Rank (from A to E)

92

Appendix 1
5 1 6 1 7 1 8 Regular Online Tests/Quizzes Ability to access learning materials at own time and pace Scheduled WBL sessions

Please add any other comments you wish

93

Appendix 1
On your Learning
Please the response (one only) which most closely represent your opinion of the statements (SD = Strongly Disagree, D = Disagree, N = Neutral, A = Agree, SA = Strongly Agree) As you worked your way through the WBL sessions, did you SD D N A SA feel that 19 the information was related to your existing knowledge 20 you were being encouraged to think more about the subject area 21 your interest was being maintained through use of a range of strategies 22 the courseware was responsive to your own particular learning needs 23 material was structured in a way that facilitated an overall understanding 24 you are able to get useful feedback on your progress Any other general comments ?

On the Future
Please the response (one only) that most accurately reflects your preference This module provides a mix of both self-paced WBL and face-to-face learning with your lecturer as part of the module. To improve the module, would you prefer 25 More self-paced WBL learning time time with the lecturer 26 Reduce WBL content content 27 Make WBL more interactive No change No change No change More face to face Increase WBL

Open Questions
28. What are the problems, if any, you experienced using this approach for learning (WBL)?

29. In what ways do you think the WBL programme might have done more to help you?

30. Would you want to use the WBL programme again? Please say why or why not;

31. What aspects of the WBL program do you find most useful in your learning?

94

Appendix 1

32. What aspects of the WBL program do you find least useful in your learning?

Please add any other comments here:

Thank you for completing this survey. I may need to contact you to clarify your response. Please include your name here if you do not mind being contacted. Please be assured that all responses will be kept confidential. Name: ______________________

95

APPENDIX 2: RESPONSES TO OPEN-ENDED QUESTIONS 28. What are the problems, if any, you experienced using this approach for learning (WBL)? No problems (14) System slow (5) Unable to get help from lecturer (2) Unable to get free access (2) Not being able to access it from home (2) Not enough time to complete quiz (2) Certain concepts difficult to understand Some contents are information are too lengthy Server down frequently Not very interesting, can not concentrate Sometimes the materials are hard to understand The system logs you out after 30 min and if you accidentally click refresh, you got to log in again Not able to do a instance search on the topic or technical jargon we dont know Sometimes I find the interface or fonts too colourful. Difficulty in getting around the tutorial Not able to go to the last page I accessed Tiring eyes Unable to store information on storage media Short of explanation Confusing web page 29. In what ways do you think the WBL programme might have done more to help you? Can access at home (8) More quiz (4) Increase interactivity (5) More face to face interaction with lecturer (3) More attractive graphics Force one to read The quiz I could learn at my own pace Get a picture of what is multimedia like Sometimes too tired to learn I could have been more interesting More animation Help me better because the materials provided are more than the textbook Provide more information Allow a printable version of the materials, quizzes, etc More notes Making the contents more interesting Keep me entertain Help me better understand the subject 96

Appendix 2 By going through it, it helps me understand better the module than just going for lectures It enhance my knowledge and understanding Yes, It aids in understanding of he modules in greater depth It keeps the interest high Prints out the pages of text which are useful to my learning which not found on text, to improve my understanding Saves the last page I accessed Learn with ease 30. Would you want to use the WBL programme again? Please say why or why not; No it is boring & not enjoyable (3) No it is a very dry programme No, not enough interactivity No, not very portable and waste of electricity No waste time No, I would use it only if it improves the WBL programme No, need more notes No unless it become more informative and better structure No, will be boring to go through again No, Bored to death! Most of the time have to wait for it to slowly appear prefer a book given such situation. Yes (5) Yes, because it is useful (4) Yes it is more interesting than from reading the book (8) Yes easy to use (3) Yes it is quite fun using it Yes, I feel that I learn more from WBL than from normal lectures Yes, because the quiz acts as a form of revision Yes, it is because it is quite user friendly and easy to retain information Yes, it allows interaction to take place Yes, quite knowledgeable Yes, its informative Yes, it is more colourful than a textbook, thus increase interet in learning Yes, its fun and well interacted (2) Yes, learn things at my own pace (3) Yes, it is very interactive and makes me learn, unlike lectures Yes, because its easy to navigate through the programme For easy subject only. Most subject can be learn better with face to face with the lecturer Yes, for own reference Yes, cause it help Yes because we can review the content again and again (2) Yes it is interactive Maybe Maybe, there should be more focus on contents that are important Neutral coz very useful of WBL program in resources. On the other hand hard for free access when behind schedule 97

Appendix 2 31. What aspects of the WBL program do you find most useful in your learning? Test & Quiz (40) The contents (3) graphics (1) Interaction (3) The content that is not found in the lecture notes (2) That I can access learning materials at own time and pace (4) The presentation method of the whole module Notes Information found is sufficient Frankly could find any especially useful aspects The wealth of interactive information The interactive info The text HTML Just for reading 32. What aspects of the WBL program do you find least useful in your learning? e-mail (4) When there is too much text (3) Not enough face to face time with the lecturer (2) Notes (2) Repeating questions & topics Could be better if we could access it from home. Discussion forum The bulleting form where it doesnt contribute much The quiz part where you have to add your own answers and have to place your answers into the selected box Many Tests Not sure Too much emphasis on text based learning. After all, the subject we learn is multimedia development. It should have to be more "eye-catching" Prequiz and fill in the blanks which no answers would be written Multimedia The contents were too boring Introduction everything

98