CORRE: An evaluation

framework for transforming teaching materials into OERs

Matthias Mehldau's http://www.flickr.com/photos/wetterfrosch/130493381/

By

Dr Samuel Nikoi
BDRA - OER Evaluator samuel.nikoi@le.ac.uk

Tania Rowlett
Copyright Administrator tania.rowlett@le.ac.uk

Presentation outline
1. Context of the CORRE evaluation framework 2. Issues regarding transforming teaching materials into Open Educational Resources (OER). 3. Use of indicative evidence in the CORRE evaluation framework 4. Questions and comments on CORRE

The evaluation models available are fairly limited and use approaches that have been traditionally developed for evaluating web-based learning despite the inherent differences in the learning objects approach .
Source: Daniel, B.K and Mohan, P. A model for evaluating Learning Objects

Issues regarding the transformation of teaching materials into OERs

Staff attitudes to open access ( King et al., 2008) Degrees of openness (Hodgkinson-Williams & Eve Gray, 2009) Design for openness (Boyle, 2006; McAndrew and Weller, 2005) Open teaching (Laurillard, 2008) Evaluating learning objects for re-usability (Schoonenboom et al., 2009) Open licensing for educational resources (Bissell, 2009) Sustainability (Downes, 2006)

Frameworks, tools and services

Frameworks

Creating and sharing content (UNESCO TOOLKIT 2009) The integrity and non-integrity (remix) model (Lane, A. 2006) Models for sustainable OERs (Downes, S. 2006) A model for evaluating learning objects (Daniel, B.K and Mohan, P. 2004) The DNER & LO Evaluation Criteria (Currie, S. & Campbell, L. 2002)

Tools and services

Authoring tools e.g. LAMS, GLO Maker II, Xerte and COMPENDIUM; Open licenses e.g. Creative Commons; GNU Repositories e.g. ITunes U and OER Commons

Background to CORRE
CORRE is an OER evaluation framework in development Emerged from the OTTER (Open, Transferable and Technology-enabled Educational Resources) project OTTER is funded by JISC and the Higher Education Academy Project started in April 2009 and due to end in April 2010 OTTER works with 13 departments in the University of Leicester and one international partner, to release the equivalent of approximately 500 credits of OERs into a range of repositories.

The CORRE evaluation framework

C is for CONTENT

Gary Hayes' http://www.flickr.com/photos/garyhayes/2973684461/

The CORRE evaluation framework

CONTENT refers to teaching materials from academic partners. There are two aspects: 1. Gathering 2. Screening Indicative evidence for gathering CONTENT.

Has the teaching material been used in an educational context? Are there gaps in the material? e.g. missing units Has the credit weighting been checked? Has partner agreement been agreed?

The CORRE evaluation framework

Indicative evidence for screening CONTENT

What type of content is it e.g. lecture? Are there editorial issues? Is the language offensive? Is the language formal (e.g. jargons) or informal? Are changes required to the learning design?

The CORRE evaluation framework

O is for OPENNESS

Justin Marty's http://www.flickr.com/photos/jmarty/128010935/

The CORRE evaluation framework

OPENNESS involves transforming teaching materials to make them publicly (re)usable learning objects. There are three aspects of OPENNESS . 1. Pedagogical 2. Legal 3. Technical

The CORRE evaluation framework

Indicative evidence for assessing the pedagogic dimension of OPENNESS :

Are changes required to the learning context? Are changes required to learning goal(s)? Are changes required to learning activity(ies)? Are changes required to learning outcome(s)? Are changes required to the assessment? Is learning support required to use this material? What level of users is the material aimed at?

The CORRE evaluation framework

Indicative evidence for assessing the legal dimension of OPENNESS

Does the resource contain 3rd party material? Have 3rd party materials/IPR been duly acknowledged? Is 3rd party material still in copyright or available under a re-usable licence? Has written permission been obtained from rights holders? Is it cost effective to negotiate a quote or reject the material? Has an appropriate Creative Commons license been assigned to the learning object?

The CORRE evaluation framework

Indicative evidence for assessing the technical dimension of OPENNESS

Is the resource available in a range of formats? Is the resource standalone or does it refer to related resources? Are other tools/software required by end-user to use the resource? Will the resource be compatible with other repositories? Does the resource have the potential to evolve as technology develops? Does the end-user require further technical help to use the resource?

The CORRE evaluation framework

RR is for REUSE/ REPURPOSE

The Truth About : http://www.flickr.com/photos/thetruthabout/2665376274/

The CORRE evaluation framework

REUSE/REPURPOSE are focused on adaptability and modification of the OER This is done through reality checking and validation:
INTERNALLY

OER team Academic partners Other stakeholders e.g. students

EXTERNALLY

Students e.g. distance learners Educators e.g. Librarians

The CORRE evaluation framework

Indicative evidence for assessing REUSE/RE-PURPOSE:

Is the CC license appropriate? How clear is the learning goal? How engaging or interactive is the learning activity? How clear is the learning outcome? How easy is it to navigate through the learning material? Is the OER ready to be released to various repositories? What further improvements are required, if any?

The CORRE evaluation framework

E is for EVIDENCE

w0LD's photostreamhttp://www.flickr.com/photos/wold/486919641/

The CORRE evaluation framework

EVIDENCE is focused on tracking use, reuse, adoption and impact of the OER. There are two aspects of this:
1. 2. Quantitative evidence Qualitative evidence

Quantitative EVIDENCE

Number of of views Number of downloads Google analytics Web bugs Star ratings

The CORRE evaluation framework

Qualitative EVIDENCE i.e. User feedback on:

What changes or modifications did you make to the OER? What difficulties did you encounter using or accessing the OER? In what specific way did the OER benefit your teaching or learning? How can the OER be improved?

The CORRE evaluation framework

Teaching material Publicly usable teaching material OERs

CONTENT

OPENNESS

REUSE/REPURPOSE

EVIDENCE

Gathering

Transformation

Internal validation

Tracking

Screening

Rights Clearance

Release to repository

Formatting

External validation

Questions & comments on

CORRE
Thank You.
www.le.ac.uk/otter

References
1. Bissell, A. N., (2009). Permission granted: open licensing for educational resources. Open learning: The journal of open and distance learning. vol 24, No. 1. pp. 97 106. 2. Boyle, T., (2006). An Agile method for developing learning objects. In L. Markauskaite, P. Goodyear, & P. Reimann (Eds.) Proceedings of the 23rd Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education: Who s Learning? Whose Technology? (pp. 91-99). Sydney: Sydney University Press. 3. Currier, S. and Campbell, L., (2002). Evaluating Learning Resources for Reusability: The DNER & Learning Objects . ASCILITE. 4. Daniel, B. K., and Mohan, P., (2004). A Model for Evaluating Learning Objects. Proceedings of the IEEE International Conference on Advanced Learning Technologies. 2004. 5. Downes, S., (2006). Models for Sustainable Open Educational Resources. Interdisciplinary Journal of Knowledge and Learning Objects. vol 3. 2007. pp. 29 44. 6. Hodgkinson-William, C., and Gray, E., (2009) Degrees of openness: The emergence of Open Educational Resources at the University of Cape Town. International journal of education and development using ICT. Vol. 5. No. 5. 7. King, M., etal (2008). Analysis of academic attitudes and existing processes to inform the design of teaching and learning material repositories: A user-centred approach. Active learning in higher education. Vol 9. No. 2. 8. Laurillard, D., (2008).Open Teaching: the key to sustainable and effective open education. In Opening up education: the collective advancement of education through open technology, open content and open knowledge. Iiyoshi, T., and Kumar, M.S.V., (eds). MIT Press. Pp. 329 335. 9. Lane, A., (2006). From Pillar to Post: exploring the issues involved in repurposing distance learning materials for use as Open Educational Resources. Found at: http://kn.open.ac.uk/public/document.cfm?docid=9724 [ Accessed: 22 October 2009] 10.McAndrew, P., and Weller, M., (2005). Applying learning design to supported open learning In Learning Design: A handbook on modeling and delivering networked education and training, Koper, R. and Tattersal, C., Berlin: Springer-Verlag. pp. 281-290. 11.Schoonenboom, J., Sligte, H., Kliphuis, E., (2009). Guidelines for support re-use of existing digital learning materials and methods in higher education.ALT-J Research in learning technology. Vol. 17, no. 2. pp. 131 141. 12.UNESCO (2009). OER Toolkit : A guide for participating in the international open education commons. Found at: http://oerwiki.iiep-unesco.org/index.php?title=UNESCO_OER_Toolkit