eLearning & eAssessment

Strategy

Examples

Dr Mathew Hillier Lecturer in Higher Education, TEDI, University of Queensland Visitor, CLPD, University of Adelaide Web edition update June 2011 Contact: mathew.hillier at gmail.com

eLearning Strategies and e-Assessment Examples

Environment Problems Solutions Examples

eLearning Environment

Consideration of the multifaceted environment in which eLearning is to take place.

Multiple Perspectives (Linstone) Systems Thinking (Checkland, Ackoff, Churchman) Worldviews (Weltanschauung) / Point-of-view Paul Ramsden: A focus on several different levels of the system
To achieve change in the quality of teaching and learning, we ought rather to look carefully at the environment in which a lecturer works and the system of ideas which that environment represents. Ramsden (2003) http://bit.ly/8XSV0I Multi leveled: Lecturer, Program teams, School/department group, Head, Faculty/division, Institution leaders

eLearning Environment

Prezi: http://bit.ly/9Jl8Ir

eLearning Potential Problems from an Academics Perspective

Prezi: http://bit.ly/bG066g

eLearning Potential Solutions

Prezi http://bit.ly/bmXA7F

Communities of Practice Build capacity

Local discipline groups Networked across the institution National/international Face to face and/or online Mentoring Teaching with virtual worlds First year co-ordinators Sessional staff RHD supervisors Work-integrated Learning Blended Learning / e-learning CoP ALTC winners, grant writing?...

Possible CoPs....

An example: http://transformingassessment/community/

Social Networks
Facebook

A range of services can be used and set-up to be internal only or externally accessible. Crosses functional boundaries.
LinkedIn

YouTube

Yammer messenger Wordpress Blog

Twitter

Social Networking

Facebook:
For teaching Where the students are! For PD Integration with LMS

First Year Use groups. Students dont need to friend each other. For local / foreign student integration. Assessed on quality of critiques.

Acknowledgement: Josh McCarthy (Architecture UofA) Public facebook group.

Transforming Assessment FB http://on.fb.me/ta-fb Imaging Our World 2010 http://on.fb.me/cv0AqC

E-Marking / Rubrics

Based on MS Excel (not tied to net). Efficient rubric based assessment. Each assessment set-up using templates. Scripted generation of student sheets. Auto print functions to PDF.

Course workbook collates marks and calculates final grades. Each assessment item has a work book listing all students in the course. Names source from course summary workbook. Each student has a marking sheet for individual feedback. Breakdown. Grad quals. Comment banks can be shared between markers. Insert auto comments - editable on-the-fly. Evaluate by sliders. Enter data only once.

e-Assessment Examples
Acknowledgements
Director: Professor Geoffrey Crisp Project Officer: Shamim Joarder

Statistics
9705 visits Jan 2010 to June 2011 69 Countries
Australia (AU) 4,062 United Kingdom (GB) 1,503 United States (US) 1,134 New Zealand (NZ) 512 Saudi Arabia (SA) 433 Hong Kong (HK) 277 Singapore (SG) 183 South Africa (ZA) 154 Canada (CA) 141 India (IN) 102 Spain (ES) 81 Germany (DE) 68 Ireland (IE) 61 Netherlands (NL) 59 Malaysia (MY) 54 Italy (IT) 47 Portugal (PT) 44 United Arab Emirates (AE) 41 Russian Federation (RU) 36 Bahrain (BH) 36 Mexico (MX) 32 France (FR) 32 and more!

Main site http://www.transformingassessment.com Secondary http://transformingassessment.adelaide.edu.au (SBLi / Open Sim) YouTube http://www.youtube.com/user/transformassessment

Learning Management Systems

Integration with...
Applets (Molecule Editor / Jmol ) Social Network (Facebook) Virtual Worlds (Sloodle / SL) Audio (Nanogong) Mind Maps (Compendium) Remote Labs Virtual Reality View (QuickTime VR) Augmented Reality (Cam markers) Virtual Classrooms (BigBlueButton)

Uni wide standard

Common Features:
Activities Resources Questions Assignments Wiki Blogs

Question types are extensible. Requires installation of scripts into the Moodle server.

http://www.transformingassessment.com/moodle/

Assignment Types
Multiple Types Single file submission Online text Offline activity Peer review Voice submission Requires installation on Moodle server. Once installed teachers can add it via the add activity menu

Voice submission
Nanogong without Save and Speed Buttons The NanoGong applet can also load a prerecorded voice file from a website.

Students use the NanoGong applet to record an audio response and submit it for assessment.

Wimba Voice Tools in Blackboard

http://gong.ust.hk/nanogong/

Quizzes: Applets
Moodle Quiz

Jmol: Open-source Java viewer for chemical structures in 3D

Students interact with online tools to obtain data to construct an answer.

http://www.transformingassessment.com/moodle/file.php/27/jmol/jmol01.html

Quizzes: Linked Java Applet

To create this type of question: 1. Create question text as normal in the LMS 2. Create another web page containing the applet.
<applet code=TitratePh.class width=450 height=370>

3. Add link to applet in the question text.

Use tools to construct answer

Student work flow Read question

Open applet using link provided in the question.

Submit Answer

More applets http://chemmac1.usc.edu/bruno/java/javaap.html

Answers recorded in grade book (Moodle / Blackboard)

Remote Labs Physical hardware can be connected to the internet. Students can use the equipment 24/7 thus increasing use time (although not to replace actual in lab time). Increases return on equipment investments.

Quizzes: Augmented Reality Mix physical and virtual. Use for complex visualisations. Overlay of data on the real world. Interactive activities and quizzes.

Quizzes: Augmented Reality Examples of augmented reality using markers. Music drum sequences. Music band / instruments

Engineering exploded models

Quizzes: Assertion-Reasoning

Multiple choice quizzes can be much more than recalling facts. For example, using an assertion and reasoning format we can test understanding of why an answer is right or wrong.

Quizzes: Certainty-Based, or Confidence-Based Marking (CBM)

Confidence based approaches penalise guessing. Students need to choose a response and declare their level of certainty. Certainty v Mark Expected Certainty levels and consequences

University College London

http://bit.ly/asa9mu

3D Virtual Worlds
Simulations Interactivity Role Plays 3D Modelling Reconstruction Data visualisation

http://www.transformingassessment.com/secondlife.php (instructions) http://slurl.com/secondlife/transforming%20assessment/254/254/23/

3D Virtual Worlds SLOODLE

As if the student was doing the activity in the LMS Set up Quiz in the LMS. Results are stored in the in grade book. A set of scripts for Moodle and SL that acts as a bridge between the two. Student undertakes assessment in the virtual world

or

Modules: Quiz, Tracker, Backpack, Blogger, Presenter, Glossary, Drop box... QuizHUD....

3D Virtual Worlds Interactivity Interactive Q&A example.

3D Virtual Worlds Interactivity by branching Scripted prims / avatars for interactive Q&A, role plays and simulations. Branching Example

Integration using web form to set-up.

3D Virtual Worlds: Touch to answer activities

QuizHUD facilitates touch to answer activities/quiz, standard MC Quiz, touch to

explore (extra info). Creates interest for students beyond a static text book! Able to test knowledge in context. HUD object Wear the HUD object. This controls and presents the quiz. HUD communicates from inside the virtual world to a web server. Touch objects in the environment to answer the questions. Basic feedback is provided in the HUD http://www.youtube.com/watch?v=ZgkUtSSEpBc

Additional scripted visual feedback after touch

Web component stores quizzes, results and information pages.

3D Virtual Worlds: Touch to answer activities

QuizHUD examples for Chemistry, Colour Wheel, Midwifery, Model Solar

system. Showing explore mode.

3D Virtual Worlds Spaces and Equipment

(Another QuizHUD example).

Virtual Lab and Equipment Orientation

Reduces time required to orientate to real world physical layouts and equipment. Processes can be established using scripting to assess steps and procedures. Beakers, flasks, crucible, watch glass, mortar and pestle, evaporating dish, pipette, funnel ring, stand, Bunsen burner, test tubes, microscope .... In fact any equipment can be simulated from the basic to the very complex.
See also iGEM lab in SL: http://maps.secondlife.com/secondlife/LINDSAY%20Virtual%20Medicine/63/71/57

Audience Response Systems

VotApedia (Urvoting.com)
Needs an account. 1. Set up via webform: 2. Choose phone numbers for each response (avoid first 5). 3. Start survey 4. To vote: a. mobile phone: - unanswered call (free) - SMS (costs) b. Web form (free) 5. Stop survey and view results

http://www.urvoting.com/

Audience Response Systems: Team Based Learning / Large Classes

TBL process for lecture sessions: Pre-reading -> readiness test -> team test -> discussion/mini lectures. TBL uses a mix of procedures some of which involve students using scratchies to uncover answers (with reducing marks the more they scratch on a given question). This provides instant feedback to students as they work towards a group response and stimulates ingroup discussion. A scanner is used to collate marks from students in the room to be feedback to the lecturer for further attention (JIT lecturing style). Problem! collating takes time in large classes interrupting the flow of the session. Potential solution: Replace physical elements with digital ones that leverage what students already have (mobile phones, laptops, ipads etc) to provide more efficient and faster collation/feedback to the lecturer saving admin time.

Wiki Collaborative document editing, discussion, detailed audit trail of edits. Dates, versions and authors. Versions can be compared.

Integrated eassessments can be computer marked.

Note: Wiki built into LMS are often rudimentary!
Some e-assessment examples using Media Wiki http://www.transformingassessment.com/wiki/

Manual marking. Potential for auto summary and data mining.

MediaWiki is a free software wiki package, originally for use on Wikipedia

e-Portfolios

Build a portfolio over time. Demonstrate competencies. Create groups. Critique and reflect. Can be integrated with Moodle. http://cterfile.ed.uiuc.edu/mahara/view/view.php?id=327 http://www.transformingassessment.com/mahara/

Blogs
Blogs can be used by students and by teachers alike.
Students can record their reflections and experiences about a particular issue, question, topic, event or objects that you may have made available. Entries are date/time stamped and can be commented upon by others.

Assessments can be embedded into a blog entry too.

You could use these as discussion points so that students could comment on the questions and the associated feedback. You could also use this format to encourage students to reflect on their level of understanding of key concepts. Students could comment on their approach to solving a problem or how they went about tackling a task that you set.

Note: blogs built into LMS are often rudimentary! WordPress is one of the best blogging platforms. See an example http://transformingassessment.com/wordpress/

Social Video Media

For Students or for PD Multimedia Generate, Share Communicate Argue Instruct Express Critique

Statistics
(May 2010 June 2011)

http://www.youtube.com/user/transformassessment

Channel Views: 2,120 Video views 17,464

Game based learning / Serious games Simulations, role plays. Business, science, history, language/communication.

Scenario Based Learning Scenarios entail branching, contingencies, multiple resources. Data is collected on time taken, path taken, resources used (efficiencies) which can be contrasted against the effectiveness of answers.

Location Window Environment Window

Action Window

Content Window

http://www.sblinteractive.org/ http://transformingassessment.adelaide.edu.au/SBLiServer/

Scenario Based Learning

Timer

Location Window Action Window

Clickable Image Part

Environment Window

Configurable by Teachers Question Release date Question Feedback Release date Moodle Compatible Maximum Attempt Number

Content Window

The scenario presented on this slide is from SBLi.

Acknowledgement : Shamim Joarder

Scenario Assessment

(high) 5 Few resources used and problem solved Efficient, Effective Many resources viewed and problem solved Not Efficient, Effective

Question Score

Few resources used And missed solution Guessing

Many resources viewed and miss the solution Not Efficient, Not effective

Resources used

100 (many)

Acknowledgement : Shamim Joarder

Scenarios for Diagnostic Assessment Instructions to student: Use the least amount of information as you can to answer the questions. If a student is able to successfully answer the (reasonably sophisticated) questions without needing additional information then we can say (all else being equal i.e not guessing or cheating) that their prior knowledge or familiarity with the topic is high. Q 20 E
Definition + Description

Example: Evaluating a students prior knowledge of a topic.

Provide the definition Provide further descriptive information Provide one or more examples

Definition

S Q 30
Definition + Description + Example

40 Q

S Start E End Q Multiple Question(s) to be answered Numbers represent weight of resources used.

Acknowledgement : Shamim Joarder

Scenario data reporting

Acknowledgement : Shamim Joarder

End!

Questions?

For further information or a copy of these slides: mathew.hillier at gmail.com

Appendix
ALTC Projects relating to eLearning ALTC Fellows relating to eLearning Researchers of interest

Appendix
Australian Learning and Teaching Council Projects of interest (as at 2010) relating to e-learning
(2010) Newcastle: The eOSCE: advancing technology to improve students (2010) UNSW: An adaptive e-learning community of practice for mechanics courses in engineering (2010) UNSW: Extending the science curriculum: teaching instrumental science at a distance in a global laboratory using a collaborative electronic laboratory notebook (2010) UNSW: Learning and teaching technical competence in the built environment using serious video game technology (2010) UQ: A national approach to education in advanced microscopic characterisation through integrated learning tools (2009) Adelaide: Getting a MUVE on: developing Web 2.0 curricula in the humanities (2009) Adelaide: The Medici project: developing a multi-disciplinary, sustainable resource for blended learning initiatives in tertiary medical education (2009) Newcastle: Facilitating WIL through skills-enabled e-portfolios in the disciplines of construction and nursing (2009) Melbourne: Web 2.0 authoring tools in higher education learning and teaching: new directions for assessment and academic integrity (2009) Murdoch: Remix, mash-up, share: authentic Web 2.0 assessment scenarios and criteria for interactive media, games and digital design (2009) UNE: e-Teaching leadership: planning and implementing a benefits-oriented costs model for technology enhanced learning (2009) UNE: The ReMarks PDF - Mark up Editor Stage 2 (2009) UNSW: Learning to teach online: developing high-quality video and text resources to help educators teach online (2009) UNSW: New media to develop graduate attributes of science students (2009) UQ: IS-IT learning? Online interdisciplinary scenario-inquiry tasks for active learning in large, first year STEM courses (2008) Deakin: Building academic staff capacity for using eSimulations in professional education for experience transfer (2008) Newcastle: Examining the impact of simulated patients and information and communication technology on nursing students' clinical reasoning (2008) UniSA: Enriching student learning experience through international collaboration in remote laboratories (2008) UNSW: Physclips II - Waves and sound: an integrated set of multi-level multimedia resources and laboratory experiments (2008) UQ: An integrated system for online clinical assessment of practical skills (eCAPS) for web-based courses (2007) LaTrobe: Encouraging benchmarking in e-learning (2007) Newcastle: Assessing and improving spatial ability for design-based disciplines utilising online systems (2008) UniSA: Facilitating flexible, enquiry-based experiential learning through an accessible, three-dimensional virtual learning environment (3DVLE) (2007) CSU: On-line student supervision training - accessible and cooperative learning in social work (2007) UniSA: Peer Review of Online Learning and Teaching (2007) UQ: Developing and disseminating TEAM SKILLS capacities using interactive online tools for team formation, learning, assessment and mentoring (2007) UQ: A national curriculum for entomology: Capacity building through collaborative, web-based delivery (2007) UQ: The virtual slidebox (2007) UTS: Embedding peer review of learning and teaching in e-learning and blended learning environments

Appendix
ALTC Fellows of interest re e-learning
Professor Des Butler 2009 ALTC Teaching Fellow Queensland University of Technology Using cost-effective multimedia to create engaging learning experiences in law and other disciplines Professor Geoffrey Crisp 2009 ALTC National Teaching Fellow The University of Adelaide Rethinking assessment in the participatory digital world assessment 2.0 Professor Geoffrey Meyer 2009 ALTC Teaching Fellow The University of Western Australia Building a network of academics who use, contribute to and disseminate an online, cost-effective histology learning and teaching resource for students in Australia and overseas Professor Matthew Allen 2008 ALTC Teaching Fellow Curtin University of Technology Learning in Networks of Knowledge (LINK) - improving student educational outcomes in online learning, using Web 2.0 concepts and a knowledge-networking approach Professor Michael Christie 2008 ALTC National Teaching Fellow Charles Darwin University Increasing the participation of Indigenous knowledge holders in tertiary teaching through the use of emerging digital technologies Dr Sandy O'Sullivan 2008 ALTC Teaching Fellow Batchelor Institute of Indigenous Tertiary Education Promoting strategies and creating opportunities for inter/multimedia practice as a culturally appropriate dissemination tool for Indigenous postgraduate research training Professor David Boud 2007 ALTC Senior Fellow University of Technology, Sydney Student assessment for learning in and after courses Professor Peter Goodyear 2007 ALTC Senior Fellow The University of Sydney Teaching, technology and educational design: the architecture of productive learning environments Professor Ron Oliver 2006 ALTC Associate Fellow Edith Cowan University Promoting the uptake of re-usable ICT based learning designs

Appendix
Researchers of interest
Caroline Steel. UQ AU. VP Ascilite, Res Fellow Lang & culture, Fmr Lecture Higher Ed, TEDI. Current and emerging Ed technologies. Teacher beliefs about tech, pedagogy, teaching and affordances theories. Catherine McLoughlin. ACU AU. A/Prof Sch Ed. Social media/web 2.0 for T&L. Cross cultural issues. Learning spaces, e-assessment. David Boud. UTS AU. Assessment futures. Student assessment in and after courses. Assessment elements. Longer term. Self assessment of performance, capabilities, work and others. David Kennedy. Lingnan HK. Director T&L centre. T&L technologies. Mobile learning. Problem based learning. Visual and information literacy. Outcomes based approaches. ePortfolios (customised). David Nicol. Strathclyde UK. Fmr Director Centre Academic practice and learning enhancement. eLearning. Graduate qualities. Diana Laurillard. London Knowledge Lab UK. Conversational approach. Interaction between T & SS. Multifaceted learning needs an array of technical capabilities. Geoffrey Crisp. Adelaide AU. Director Centre for Learning and Professional Development. Assessment, e-assessment, ed technologies, web 2.0 and social media. Exmplars. Interactive assessments. Grinne Conole. OpenU UK. Chair elearning. e4innovation.com Learning design via designed approach v beliefs based approach. Open course design (George Siemens, Stephen Downer itu.umanitoba.ca) James Dalziel. Macquarie AU. Director of eLearning Centre of Excellence. Learning activity management system (LAMS),and Research AMS. Paul Ramsden. Consultant AU/UK. Fmr PVC T&L USyd. Higher Ed Academy UK, Uni London, Maquarie. Teaching and student experience. Teaching quality. Policy. Role of leadership. Learning to teach in higher ed: learning from our students. Argues for holistic, multi leveled approach top T&L improvement lecturer, department, head, program teams, faculty, uni management. Peter Goodyear. USyd AU. CoDirector Centre Computer Supported Learning & Cognition (CoCo). Learning technology and new media. Learning cognition and motivation. Learning through inquiry. Productive learning environments. Teaching, technology and learning design. Ron Oliver. Edith Cowan AU. PVC T&L. Share/reuse of learning designs. Toolbox learning object repository. Authentic learning in web environments. Online learning communities (Chris Brook, Curtin). Terry Anderson. Athabasca CA. Chair distance ed. Social networking, online conferences, self paced learning (non-schedule based). Terrya.edublogs.org