Escolar Documentos
Profissional Documentos
Cultura Documentos
Educational Technology
2008, 24(1), 30-41
Introduction
The past decade has seen enormous growth in the use of learning management
systems (LMS) in higher education institutions, with varying levels of support
provided to staff and students during the implementation phases. This has, in theory
at least, provided the potential for rich learning environments built on social
constructivist theories and available to all students, both on campus and those
studying at a distance (Hawkes & Terry, 2003; Papasergiou, 2005: Ng, 2007). Much has
been published about what constitutes good online teaching (Chickering & Ehrmann,
1996; Oliver, 2003; Bates, 2005), and this literature has expanded with institutional
interests in quality online learning environments (see for example Evans & Nation,
2000; Oliver, 2003; McLoughlin & Visser, 2003; Bates 2005). Holt & Challis (2007)
recent paper investigated one institutions experience of implementing strategic
change through an institutional policy imperative couched in terms of integrating
wholly online units in various undergraduate curricula. This is an interesting paper
and is relevant to the research reported here for the insights it provides in
demonstrating that institutional policy initiatives do not necessarily lead to
straightforward, predictable, and unproblematic teaching and learning outcomes
(Holt & Challis 2007, p. 130). We discuss this paper subsequently.
31
32
However, despite all these services being freely available and widely advertised
through the Universitys central teaching and learning support centre, it was up to the
academic teaching staff to approach the relevant providers, and the overall number of
staff utilising the full range of these services remained relatively low. Given the size of
Monash (approximately 2500 teaching staff), it was logistically impossible that more
staff could be supported. Participation in staff development related to WebCT and
online learning prior to commencing online teaching was voluntary, although one
Faculty mandated staff development.
Similarly, students were required to use the new system, usually with little or no
introduction. Induction of students was left to the teaching staff and the expected
disparity in the range of tools and instructional designs used by different lecturers was
evident some chose to demonstrate their sites in a lecture environment, but for others
who were teaching to predominantly off campus students, this was not an option, and
any induction for students was conducted within the LMS environment.
During the implementation phase, no recommendations of teaching or design
standards or quality control mechanisms were instituted, with the unsurprising
outcome of online units of highly variable quality; the implications of this are
discussed subsequently.
Method
There has not been any cross-Faculty investigation of the use of WebCT at Monash
University since the initial project implementation, and only limited evaluation of
online teaching within one or two Faculties.
The objectives of the project reported in this paper were to:
1. conduct institution-wide surveys of staff and students actively using WebCT to
determine what use was being made of the LMS;
2. investigate which areas of online teaching with WebCT needed improvement or
further investigation;
3. inform developments in the pedagogical applications of WebCT, including quality
improvement strategies in online learning environments and;
4. inform staff development and training initiatives related to WebCT specifically and
online learning more broadly.
The principal method was an anonymous online survey. We acknowledge that
quantitative surveys alone provide limited information, and the findings therefore
may not include all perspectives (Hammond & Wiriyapinit, 2005; Gilead, 2006);
however this project was intended as a preliminary study, and (at the time of the
study) further exploration into key areas of interest identified by the research was
planned as an ongoing investigation.
Student and staff perceptions of WebCT were measured to establish what features of
the LMS were being used in teaching and learning, and the experiences of the staff and
students using these features. Two surveys were administered, one for students and
the other for staff. The student survey comprised 56 quantitative items organised into
three parts. The first part sought biographical information. Part two of the survey
measured student usage of WebCT, and the final part investigated the kind of support
respondents sought in order to use WebCT successfully. The student survey also
33
34
academic users of WebCT were concerned about support for new versions of WebCT (at
the time of the surveys, the University was preparing to move to WebCT Vista).
Importantly, staff also identified their interest in seeking advice on good online
teaching practices.
Table 1: Ongoing help required to support use of WebCT
Specific training as I require it
Online technical assistance
Assistance in migration to new versions of software
Advice on good practice in online teaching and learning
Support to ease my workload
Specific advice on particular learning issues
Support in managing large classes
Other
Number
58
57
48
44
39
27
27
13
Percentage (%)
60.4
59.4
50.0
45.8
40.6
28.1
28.1
13.5
Other ongoing support needs, identified apart from those listed in Table 1, were help
specifically with automatic assessment, quality online help facility, local support, the
availability of synchronous chat lines (at the time of the survey, the LMS Chat tool was
disabled) and decision makers taking into consideration pedagogical concerns.
Number
59
334
675
240
Percentage (%)
4.5
25.5
51.6
18.4
With respect to resources used by students to learn WebCT, half of all students
reported learning how to use WebCT by themselves (Table 3), and a further 23%
received an introduction from their lecturer or tutor. Only 31% of students were aware
of the existence of the CeLTS helpdesk (Table 4), with very few of these utilising this
service, and even fewer being aware of the online student manuals. While most
students were aware of the existence of ITS and Faculty helpdesks, a large proportion
of students were unaware of the available support services.
Table 3: Resources utilised to learn WebCT
Worked it out myself as I went
My lecturer or tutor provided an introduction to the unit
Other students helped me to get started
Consulted online resources
Lecturer or tutor helped me to get started
Completed a preliminary online exercise
Consulted helpdesk staff
Other
Number
969
439
162
91
89
70
33
49
Percentage (%)
50.9
23.1
8.5
4.8
4.7
3.7
1.7
2.6
35
Yes
No
Number Percent (%) Number Percent (%)
1007
77.5
292
22.5
791
60.8
511
39.2
731
56.4
565
43.6
402
31.1
891
68.9
327
25.2
969
74.8
Students in the study who experienced a unit rich with interesting and relevant
resources that reflected sound educational design, including opportunities for timely
feedback and, importantly, interaction with their teachers, reported their experience
with the technology positively. A substantial proportion reported sites that had
minimal feedback, limited opportunities for interaction with their teachers, hyperlinks
that did not work, and old or outdated information (These factors were indicated in
response to the open ended questions on the survey).
The qualitative data revealed students considerable dissatisfaction with the majority
of the sites they were exposed to for learning support. Students believed that it [online
teaching] works well when the lecturer wants to make it work well; however if
content was not always up to date or complete, online teaching was perceived as
poor. Further, students recognise good teaching and can gauge the potential of the
existing learning tools in WebCT; as one stated most of the tools mentioned would be
useful, but have not been used by our course designers and, for example, the calendar
is only useful if it is current. If the calendar is from the previous year, then it is
extremely unhelpful! Furthermore the qualitative data indicated students valued and
wanted easy access to online lecture notes (both for printing and saving files), and
appropriate feedback on their progress in a timely manner. Analysis of the qualitative
data also revealed that students who underwent some form of introduction to their
particular WebCT site were more successful in using the technology and clearer about
what was expected of them in the unit.
The student responses raise major issues about the quality of the online sites - both
issues of teaching quality (appropriate teaching activities, level of interaction and
feedback) and either poor use of technology (little use of available features,
inappropriate file types uploaded, large file sizes) or lack of appropriate maintenance
(broken links, outdated information). At the time of this study, no routine checks of
WebCT sites were conducted prior to release to students. Following on from this study,
a service was implemented where the WebCT helpdesk staff offered to conduct
technical and maintenance evaluations of sites on request - however very few teaching
staff took advantage of this opportunity. There is a growing evidence based literature
(see for example Oliver, 2003; Laurillard, 2002; Armatas, Holt & Rice, 2004) which
supports our view that improving the quality of online teaching is a major challenge
for Monash University and is a major imperative across the sector in Australia and
internationally.
In analysing responses to the open ended questions from both staff and students, it
was apparent that many staff had difficulty in maintaining their WebCT sites with
current and relevant information. Student feedback indicated the primary areas of
concern were with the many broken links and outdated materials, including extreme
cases of calendar entries referring to dates from two years previous. Staff comments in
36
this area were focused either on the lack of local IT support to help them maintain
these sites, or lack of time to spend in online teaching.
Discussion of findings
In this study students and staff appeared to have contrasting priorities: academic
teachers were more concerned with technical aspects and workload issues, and
students were more concerned with the quality of the online teaching, which was
reflected in their perception that their teachers were not engaged with them in what
they believed ought to have been an interactive learning environment. The student
survey data revealed that students were dissatisfied with poorly designed and
maintained sites, rather than the lack of a site; this suggests that quality online learning
approaches are an important adjunct to student learning, and if quality cannot be
assured then online learning may be of little value.
Our study revealed that academic staff perceive that online teaching necessarily makes
demands on workload and that staff require assistance with the technical and
administrative tasks associated with using WebCT. This presumes that teaching staff
may then be able to spend more time developing their skills and expertise in online
learning and teaching. The provision of various forms of support for staff who are
less comfortable with using the technology may be the only way to encourage more
engagement with their online students.
At Monash University at the time of our study a number of Faculties and Schools had
mandated the inclusion of an online component for units of study; this however is not
institutional policy. Student responses identified that in these circumstances, the online
experience was frequently less satisfactory than when online teaching was a choice of
the lecturer.
In contrast to our very broadly based survey research, Holt & Challis' (2007) insightful
paper presented a qualitatively informed case study report of an institutional initiative
to integrate wholly online units across a number of diverse curriculum areas. Their
research was undertaken in a well-established, internationally recognised distance
education institution which differs considerably from Monash as the research intensive
institution with a more traditional culture of campus based pedagogies described
earlier. Holt and Challis (2007) have gone further in theorising and intellectualising the
pedagogies associated with the implementation of wholly online units at their
university. While we do not wish to compare our results with theirs, perhaps we can
say that the Monash experience reported in this paper supports, to some degree, their
findings; that policy imperatives have meaning only in the way they are enacted in
practice.
In this study the strong student response regarding their perceptions of the poor
quality of much of the online teaching suggests that further strategies to encourage
improved teaching must be considered. Initiatives such as local development grants,
institutional showcases of exemplar sites (e.g. Designing Electronic Learning &
Teaching Approaches (DELTA), Brack et al, 2005), and awards recognising excellence
in online teaching can encourage some staff to become more engaged with their online
teaching, but may not reach the majority of staff. These initiatives are more likely to
attract interest from the early adopters of technology, whereas the major challenge is
encouraging more interaction from the mainstream majority of staff (Weaver, 2006).
37
Moreover, a more determined policy initiative such as that described by Holt & Challis
(2007) may be needed for Monash to effect the kind of dramatic improvement in the
quality of online pedagogies that this study seems to indicate is needed.
Ongoing seminars, presentations and staff development workshops can help
demonstrate different strategies used by staff to engage students in online teaching,
but these events need to be relevant to the targeted audience, and to be offered at times
and venues convenient to the staff. Previous offerings of these events have been poorly
attended at Monash, and often, attract interest largely from the early adopters - this is
problematic for it becomes a case of preaching to the converted. Staff requests for just
in time support must be accommodated, either by face to face sessions, by appropriate
online resources, or a combination of both (Weaver, 2006; Wilson & Stacey, 2004).
Issues of student training were also raised. Students are often unaware of features in
WebCT or how to use the existing tools effectively, and are largely unaware of existing
support services available to them. The WebCT Helpdesk staff offer to conduct brief
introductions on using WebCT in lectures or other classes, but it appears that few staff
are either aware of this service, or are using it for their students.
Arising from these findings, a series of recommendations was submitted to senior staff
and policy making bodies within the University (including all Faculty Deans). These
recommendations suggested further and ongoing evaluation of online teaching within
Faculties, greater encouragement for staff to actively interact with their students
online, and more appropriate use of the online technology. The key message was that
online teaching should be more than a static WebCT site. In particular, Faculties and
Schools were encouraged to provide additional technical support for staff, particularly
where online teaching is mandated, and existing staff and student support services
should be promoted more widely.
It is important to note that the research which informed this study was unfunded and
conducted largely out of the personal interests of the co-authors. While a formal report
was submitted to various stakeholders, it was our sense that in the absence of a
committee structure through which the survey was conducted, we had little real
expectation that recommendations might be systematically implemented. Since the
study however, there has been considerable change in the governance and other
structures that manage and foster e-learning at Monash. At the time of publication a
new Educational Technology Framework has been developed through a newly formed
Educational Technology Committee which is a Sub-Committee of the Universitys
Learning and Teaching Quality Committee (http://www.calt.monash.edu.au/
Quality/ETC). It may be that now is the time for Monash to address more
systematically its commitment and support for e-learning pedagogies institutionally.
Given these developments and the findings reported here, other avenues for research
in the field present themselves for Monash. Potential research questions might include:
What is the impact of policy initiatives on e-learning and teaching pedagogies in a
research intensive university?
How does blended learning influence student learning outcomes?
What influences academic staff to sustain their use e-learning strategies?
38
Conclusion
Our findings illustrate that online learning and teaching presents many challenges to
Monash University specifically and the higher education sector broadly. These issues
are not new to those of us teaching and researching the field of online learning (see for
example Evans & Nation, 2000; Spratt, 2003; Naidu, 2004; Oliver, 2005; Holt & Challis
2007). Conole (2004) in a paper titled E-learning: The hype and the reality suggests that if
we believe that e-learning is transforming education, as some would have it, then we
are sadly deluded. She argues that e-learning is still marginal in the lives of most
academics with technology being used for little more than acting as a content
repository. While they may make some contribution to student learning, online
learning environments that restrict themselves to delivering static resources do not
characterise quality online learning environments (Oliver, 2003) nor do they take
account of learner characteristics which may influence learning (Armatas, Holt & Rice,
2003).
We believe that to be effective, tertiary education should engage learners as active
participants in their learning. Achieving this means offering learners opportunities for
interaction in ways that can promote change and growth in the learners conception of
knowledge. Such pedagogies aim to encourage learners to become autonomous
lifelong learners, capable of problem solving and critical thinking, and to move them
from being passive recipients of information and knowledge to being active,
enthusiastic learners and knowledge creators. Moreover, tertiary pedagogy is
concerned with building meaningful learning relationships between learners and
teachers, and learners and their peers. It involves encouraging collaboration in
learning as well as cooperation in learning; the appropriation of technology for
teaching suggests great opportunities for the promotion of innovative and interactive
quality e-learning environments.
While the major findings of this study indicated evidence of a teacher centred, content
focused e-learning approach, both staff and students indicated willingness to learn and
engage more in the online environment. Since the completion of this study, the policy
discourse at Monash has become less concerned with the appropriation of educational
technologies to promote Monash as a provider of virtual education where student
centred flexible and online learning might differentiate Monash from its competitors,
and more concerned with developing educational policies that develop Monashs on
campus expertise. While there remains an emphasis on internationalisation and
improving the campus based experience of students, information technologies are
perceived more as supporting administration functions and supporting fac to face
teaching.
In a review of institutional AUQA quality audits, Martin (2003, p. 19) noted that in the
main, there was comparatively little attention paid to pedagogy and in relation to
flexible online learning, that the Audit Reports did not reveal any particular focus
apart from a need to ensure overall co-ordination of developments and parity of
standards across different modes of delivery (p.18). Further of relevance for this paper
was Martins (2003) finding that institutional support for new teaching staff across the
reviewed institutions needed improvement, as did opportunities for professional
development for teaching for permanent academic staff. While appreciating that the
AUQA audits are concerned broadly with quality assurance issues in complex
institutions, the apparent lack of attention to teaching related indicators of quality is
39
notable. Moreover, Martin (2003, p. 33) highlights what she calls outreach teaching
and describes as off shore campuses, partnerships and external delivery as a key area
that presents challenges for maintaining consistency of standards, of resourcing and
of planning.
Information technologies are now routinely called upon to support Martins (2003)
outreach teaching. Further, the policy documentation of higher education institutions
includes information technologies as a key strategic goal to support initiatives in
advancing student centred flexible learning, and improving the quality of teaching.
However, if the way in which the majority of WebCT sites are perceived by students in
this study is any indication, this objective is not being achieved. Students demand
more than a repository dump they want and need active and enthusiastic
engagement from their teachers. Our results indicate that, due to a perceived lack of
institutional support and adequate resourcing, many staff are forced to adopt a teacher
centred approach in their online teaching. Universities need to pay more attention to
the institutions key stakeholders, students, and support academic staff and to advance
the widely recognised potential of online learning. In other words, if academic
teaching staff are to engage with technology in ways that encourage them to innovate
then institutions must make such efforts to enhance the learning of their students a
high priority and back this in practice as well as in their rhetoric (Hannan, 2005, p.
984).
References
Armatas, C., Holt, D. & Rice, M. (2003). Impacts of an online resource-based learning
environment - Does one size fit all? Distance Education, 24(2), 141-158.
Bates, T. (2005). Technology, e-learning and distance education. London: Routledge.
Brack, C., Samarawickrema, G. & Benson, R. (2005). Technology advances: Transforming
university teaching through professional development. In A. Brew & C. Asmar, C. (Eds),
Higher Education in a changing world: Research and Development in Higher Education, Volume 28.
Sydney: Thunder Press. pp. 50-59.
http://www.itl.usyd.edu.au/herdsa2005/pdf/refereed/paper_306.pdf
Chickering, A. & Ehrmann, SC (1996). Implementing the seven principles: Technology as lever.
AAHE Bulletin, October, 3-6. [viewed Oct 2006, verified 23 Dec 2007]
http://www.tltgroup.org/programs/seven.html
Conole, G. (2004). E-learning: The hype and the reality. Journal of Interactive Media in Education,
2004(12). http://www-jime.open.ac.uk/2004/12/
Dommeyer, C.J., Baum, P., Hanna, R.W. & Chapman, K.S. (2004). Gathering faculty teaching
evaluations by in-class and online surveys: Their effects on response rates and evaluations.
Assessment & Evaluation in Higher Education, 29(5), 611-623.
Evans, T. & Nation, D. (2000). Understanding changes to university teaching. In T. Evans & D.
Nation (Eds.), Changing university teaching: Reflections on creating educational technologies.
Kogan Page, London, pp.160-175.
Garrote, R. & Pettersson, T. (2007). Lecturers' attitudes about the use of learning management
systems in engineering education: A Swedish case study. Australasian Journal of Educational
Technology, 23(3), 327-349. http://www.ascilite.org.au/ajet/ajet23/garrote.html
Gilead, Y. (2006). Beyond the classroom: WebCT and learning Modern Hebrew. Australasian
Journal of Educational Technology, 22(3), 416-438.
http://www.ascilite.org.au/ajet/ajet22/gilead.html
40
41