Evaluation of Science in Your World

Evaluation of Science in Your World
A report by Consulting Inplace

February 2011
Contents
1. Executive summary 2. Introduction 3. Overview of the programme 4. Evaluation of the successes/progress of the networks 5. The impact and outcomes of the programme 6. The extent to which the programme met its objectives 7. Conclusions and recommendations
2 8 10 18 37 48 54
Appendix I: Logic Model Appendix II: Online survey analysis Appendix III: Activity observations Appendix IV: Research tools and questionnaires Appendix IV Full list of documents reviewed
1.
1.1
Executive summary
Introduction In 2010, Consulting Inplace was commissioned by the Museums Libraries and Archives Council (MLA) to evaluate the MLA managed strand of the Science in Your World programme, part of the Department for Culture, Media and Sport (DCMS) and Department for Education (DfE) funded Strategic Commissioning Programme. Science in Your World forms part of a national partnership between MLA, the Natural History Museum (NHM) and the Science Museum. The MLA managed programme supports museums to deliver science learning opportunities through a system of regional networks. This report summarises the evaluations findings, conclusions and recommendations. About Science in Your World
1.2
The two year MLA Science in Your World Programme started in 2009. It had the following aims: To deliver new science learning opportunities in English museums To build skills and capacity in museum educators and curators to develop and deliver science learning opportunities To share knowledge about science education and success between museums To build demand in schools for science learning opportunities in museums To demonstrate that museums and out of classroom learning have a major role in delivering the Science Technology Engineering Maths (STEM) agenda To maximise the advocacy benefits for the partners and the museums sector from delivering science learning opportunities These aims are delivered through a system of regional networks that cover the South East (SLIME), West Midlands (SMELT), North West (SMILE), North East (STEAM) and Yorkshire and Humber (YES-NET). These networks of museums share knowledge, ideas and experiences through online forums, meetings and events. They also deliver capacity building activities such as continuous professional development training, and work together to create and share resources such as workshop planning. Alongside the network development, the programme provided small grants and regional grants that encouraged museums to develop their own science learning opportunities, and make connections with local science professionals, teachers and schools.
1.3
About the study Consulting Inplace was commissioned by MLA to conduct this evaluation, which included developing a logic model, stakeholder interviews, an online survey with museums and activity observations which included teachers, young people and science professionals. The aim of this research was to complete a robust and independent evaluation of the Science in Your World programme that measured the impact of MLA support of the regional networks, and the impact of the regional networks on the programmes outcomes.
1.4
Conclusions
This evaluation has identified strong outcomes for museums, educators and young people as a result of Science in Your World, and provides an indication of the future impact of the programme. The conclusions based on these findings include: Demand from museums has been demonstrated: The offer by regional networks of opportunities to develop STEM (Science, Technology, Engineering, Mathematics) learning has been welcomed by local and regional museums. Furthermore, there is a need expressed by museums for ongoing support of this kind. The regional networks offer varying levels of support: The size and success of support appears to grow with the length of time the network operates, as the membership lists of the older SLIME and STEAM networks demonstrate, although SMILE and YES-NET in particular are growing quickly. The support that has been particularly valued includes: small grants, preferably linked to partnership with other local museums and schools; facilitation of face to face peer to peer support, delivered through events; and facilitation of relationships with other relevant professionals, including teachers and scientists. Networks have built capacity in museum staff: Science in Your World funded regional networks have built capacity in museum staff to deliver science learning opportunities. This capacity was developed through increasing confidence and the skills of museum staff, and providing resources and networking opportunities with peers and other professionals, such as teachers. Teacher-museum partnerships have been developed: Many one-on-one teacher-museum partnerships have been developed. These relationships have changed teachers opinions on using museums for learning, and museum staffs ability to offer appropriate learning opportunities. Partnerships with STEM professionals have been developed: Other STEM professionals, such as scientists, engineers, university students and staff and bodies such as universities, aquariums, STEMNET (STEM network) have been involved in the project. Museums, other professionals, and schools work extremely well in partnership, as they bring artefacts, real world STEM knowledge and teaching and curriculum expertise together to produce higher quality opportunities for pupils. More and better STEM opportunities for young people have been developed: Over 3,800 young people and 500 family members have participated in programmes developed through Science in Your World funded opportunities, over 450 teaching staff, and over 350 museum and archive staff have participated in CPD programmes, learning links placements or other Science in Your World funded opportunities
1.5
Recommendations The recommendations emerging from this evaluation are: Regional networks should be supported to continue where possible, and where there is demand: There is demand for the regional networks, and they are an appropriate mechanism for delivering support to the sector. The outcomes and impacts outlined in this report also support the governments STEM skills commitments, and support schools to deliver on the Department for Educations plans for science and maths as core to the curriculum, delivered through more autonomous schools. All co-ordinators and national partners are exploring alternative methods of funding for the region, including sharing administrative tasks between larger regional and national museums, working with local authority Museum Development Officers, and merging with other networks. In some regions, sub-regional events and knowledge sharing may be more appropriate: For example, the North West. It may be more appropriate to have smaller, sub-regional events and sub-networks. This will enable museums to easily travel to events and training. There are resource implications with this approach, along with less ability to deliver shared outcomes. One solution may be to target events at sub-regions, but keep a wider regional network as an umbrella. However, there is value in wider knowledge sharing regionally, and nationally. A national online resource bank would be useful: Each regional network has produced or shared resources through the internet. A national online resource bank would maximise their accessibility, and ensure that they are available beyond the funding period. A national means of sharing information and learning should be developed: A national means to share information and learning amongst peers in the museum sector would give the existing regional and local work a focus. This could take the form of an annual conference, newsletter or forum. Given the DfEs focus on innovative delivery in more autonomous schools and the governments commitment to STEM skills, national information sources, such as this recommendation and the national online resource bank, may be in line with departmental priorities. Cross-sector links with schools, universities or science professionals should be encouraged: One of the clear outcomes of the current funding is the bringing together of professionals from related fields. This has considerable benefits in terms of improving practice and outcomes for young people. There is a danger that without this small seed money, these links will not continue to
grow. Future funding is recommended to continue this work. This also supports the Governments commitment to STEM skills policy through supporting all professionals with an interest in this area to work together for more and better outcomes. Similarly, there is scope for further collaboration with national organisations who also aim to increase science engagement, including but not limited to the British Science Association, National STEM centre, National Science Learning Centre, Association of Science and Discovery Centres Further research is needed to track young peoples outcomes for programmes such as this: Further research is needed to gain an understanding of the long term impact of programmes such as this on young people. University partnerships could be used in future to support further research into the impacts of the programme. Increased advocacy for science education in museums to build upon the good individual relationships: There is a need to build upon the good work done by the programme in developing one on one relationships with the education sector through a widespread awareness raising campaign. This would enable the benefits of this approach to learning to go beyond Museum Champions into the mainstream. Relationships between national and regional museums should be continued and strengthened: There is a widespread recognition among partners and stakeholders that national museums have an important role to play in science education in museums. This should continue, and be strengthened where possible. Partnership structure for further collaborative programmes, for developing external funding bids: The network has laid the groundwork for museums to collaborate on further projects or funding bids. For example, the regional project led by Museum of Science and Industry (MOSI) is a good example of a project that has a broad enough impact that it could attract the support that one museum could not alone. Resources and learning to be distributed beyond museums: Schools are currently reacting to recent changes in the Department for Education, and are integrating new guidelines for learning as part of the National Curriculum Review. This programme can and should support schools by highlighting innovative teaching and learning outside the classroom experiences. It can also be used to influence how schools teach science by highlighting the successes of the programme. The resources created by the programme should be accessible to a wider audience, including teachers, perhaps through an existing mechanism such as the Times Educational Supplement website that can already be reached.
Regional and local partnerships should be used to support schools develop new curricula and programming: Government policy for more control to be devolved to schools and academies suggests that there is an opportunity for museums to deliver high quality targeted learning opportunities to schools. This programme has left a body of experience, case studies and resources that should be used to support schools to develop new curricula to support high quality out of school learning.
2.
2.1
Introduction
Overview This report summarises the findings of the evaluation of the MLA managed Science in Your World Programme, which was conducted by Consulting Inplace. The Science in Your World programme was part of the Department for Culture, Media and Sport (DCMS) and Department for Education (DfE) funded Strategic Commissioning Programme. It funded regional networks that support museums to offer science educational opportunities. This report will summarise the findings, conclusions and recommendations of the evaluation. The chapters focus on an overview of the programme, an overview of the regional networks, a review of the performance of the programme against its own objectives, a summary of its impact and outcomes, and conclusions and recommendations arising from the research.
2.2
Aims and objectives of the evaluation The aim of this research was to complete a robust and independent evaluation of the Science in Your World programme that measures the impact of MLA support of the regional networks, and the impact of the regional networks on the programmes outcomes. We have found out to what extent the MLA-supported networks are achieving their aims, and how this was achieved in each region. The evaluation has the following specific objectives: Develop an evaluation framework providing a logical structure for determining the success of the Science in Your World programme Evaluate the successes/ progress of the networks in each region Demonstrate the impact and outcomes of the programme Assess the extent to which the programme has met its objectives
2.3
Evaluation methodology Two of the four phases of this evaluation were concerned with data collection. The methodology of these data collection phases were designed to meet the objectives above. We have summarised the methodology in this section. Data review and Logic Model development Data review: We undertook a desk-based review of documents relevant to the Science in Your World programme. This included a review of relevant policies and strategies, including MLA policy briefings; programme documentation such as the
2.3.1
application forms for grant funding, MLA end of project reports; and quantitative data such as data capture reports . We also spoke to some stakeholders to build our understanding of the programme. These included regional network coordinators and national partners. This informed our evaluation framework, through scoping the size and type of network activity. Logic Model development: We developed a logic model and corresponding draft research tools for the fieldwork phase of this assignment, eg interview topic guides, questionnaires, satisfaction forms etc. These tools are included in Appendix V. 2.3.2 Evaluation fieldwork Management review: We conducted a management review of the programme. This involved speaking to relevant individuals and reading through all the necessary files near the start and at the end of the evaluation. The monitoring information for the programmes was reviewed, and collated into the inputs, outputs, and any outcomes that were noted at the time. This process has fed into the evaluations of the regional networks, and informed the outcomes and impacts measured by the evaluation. Network observation: We observed the activities of the networks supported by the programme; in total we observed five networks during the evaluation period. This involved signing up to the internet groups of the networks and regularly monitoring the outputs of the programme. We aimed to understand how the regional coordinator used the online networks, how the members interacted, and the type and frequency of communication, to inform our evaluation of the different regional networks. Activity review: We examined five activities run by the networks in more detail. We attended events and gathered information from the project leaders, and also from participants, including school children when possible. This enabled us to collect more in depth qualitative evidence that could demonstrate some of the softer outcomes of the programme. The findings are summarised in Appendix III. Online survey: We created an online survey using SurveyMonkey software and distributed it through the networks via e-mail. This survey aimed to capture the views of museum staff that worked with the Regional Networks on how they use their networks, alongside any effects it may have on them. We used this tool to understand how the regional museums used the networks, and what impacts the network activity was having on them. The findings are summarised in Appendix II.
3.
3.1
Overview of the programme

Logic Model The first task for the evaluation was to create a logic model for the programme. The logic model is a systematic and visual way to present and share the understanding of a programmes outcomes, activities/processes and the theoretical assumptions/principles that underpin it. We created this model drawing on evidence from programme documentation and stakeholder consultations. The Science in Your World logic model can be found at Appendix I.
3.2
Rationale A Programmes rationale defines the fundamental reasons behind the creation of the programme. It includes evidence of demand or need, and the identification of the problem the programme was created to solve. The Science in Your World Logic Model, which was agreed with the MLA and formulated by Consulting Inplace, states the rationale of the programme to be as follows: Museums were not fulfilling their potential to make a contribution to science education. There was a need: a. For museums to put school science in a real world context, enriching the curriculum in and out of school hours. This followed the Out of School Learning and Extended Schools agenda. b. To extend offers by outstanding science education museums such as National History Museum (NHM) and National Science Museum across the regions by developing regional partnerships. There was also a lack of existing engagement between museums to share learning etc. Research showed:
10
Museums lacked confidence in the education arena, training and investment Lack of Continuing Professional Development (CPD) for teachers and museum educators in Science Learning.1 This rationale was developed in discussion with stakeholders and based in part on research commissioned by the MLA and the NHM that was completed in March 20082. This research scoped the potential for a national network of museums to deliver science learning, building upon a previous programme, Real World Science. The reports recommendations and outcomes include the following: There was a clear demand for a series of regional networks and co-ordinators, linked by a national advisory group. This model for delivery would allow the programme to respond to regional and local need, while still retaining some national coordination. The regional networks would be free to develop in an organic way; however, they would work together to achieve national outcomes and targets. The focus should be on secondary age students, with curriculum-linked activity such as practical science workshops, meet the scientist sessions, trails etc. This on-site activity would be supplemented by online and outreach activities that link museums, schools and students to produce and deliver learning experiences. 3 There was a gap of CPD for museum staff or teachers to engage in natural science-based projects. This would require much closer partnership working between providers that work in sciences and also museums.
Science in Your World Logic Model from the Science in Your World Evaluation Framework, Consulting Inplace, November 2010 2 Alison Porter, Tim Caulton, Ian Blackwell, Natural Science Network: Draft Report on Consultation and Scoping;; 31st March 2008 3 It should be noted that the recommendations were combined with stakeholders views, and it was decided that in order to allow widest access to the network and to encourage museums who may have no experience creating STEM programmes to not put a limit on the audience age or area of stem provision.
11
The research found a clear demand for high quality science activities among teachers and education partners. Evaluation is important to ensure consistency and learning into this programme. The national co-ordinator has a role working with the Science, Technology, Engineering and Mathematics Network (STEMNET), examination bodies, producers of textbooks and others nationally, and Science Learning Centres regionally to ensure that the profile of museums as science learning destinations is recognised and supported,
This report provided clear evidence of need and demand for Science in Your World. 3.3 Policy summary Since the start of the programme there have been shifts in policy with the change in government. The need and demand for the programme is still there, but there are new challenges to delivery including the MLA being closed as part of the reorganisation of arms length bodies,4 and the transfer of some of its functions to the Arts Council5. In addition, changes to and a reduction in funding for the museum sector6 and potential changes to the education system, with a focus on the core subjects may impact the Learning Outside the Classroom agenda7. There are opportunities in this new environment, however. STEM subjects are recognised as being vital for the economic recovery and the economic health of
Review of arms length bodies by Department of Culture, Media and Sport, MINISTERIAL WRITTEN STATEMENT Department for Culture, Media and Sports arms length bodies, The Secretary of State for Culture, Olympics, Media and Sport (Rt Hon Jeremy Hunt), Monday 26th July th 2010, (http://www.culture.gov.uk/news/news_stories/7280.aspx, accessed on 10 February 2011) 5 Included in an announcement by Ed Vaizey, Minister for Culture on Tuesday 23 November 2010 th (http://www.mla.gov.uk/news_and_views/press_releases/2010/mlafuture_ace, accessed on 10 February 2011) 6 For example, the Comprehensive Spending Review outlines cuts to the public sector, sets out th some of the effects in the Museums Journey, Patrick Steel, 20 October 2010, (http://www.museumsassociation.org/museums-journal/news/20102010-comprehensive-spendingreview-museums , access on 10th February 2011) 7 White Paper: The Importance of Teaching, Department for Education, November 2010
12
the UK in the future.8 In addition, science and maths remain in the core subjects in the National Curriculum and are part of the focus of the current National Curriculum Review.9 This emphasis means schools need to find new ways to both support science learning and inspire young people to go on to science based careers. Finally science learning is important to many large companies, as they depend on young people learning the skills to become scientists and engineers10. The regional networks and museums could develop partnerships to leverage funding for their activities with these companies. 3.3.1 Schools and education policy There has been a recent shift in schools and education policy, following the evolution of the DCSF into the DfE. The priorities for the Department as set out in their Business Plan,11 and expanded upon in the White Paper The Importance of Teaching include: Greater autonomy to schools, through the expansion of the academies programme. This may impact on how schools choose to deliver the curriculum, removing barriers to science teaching in museums. The introduction of the pupil premium for disadvantaged pupils, which will be administered by the schools. These are to be used by the schools to improve attainment and aspiration, which museums are well placed to support through delivery of opportunities such as those promoted by Science in Your World. Improvement of the system with support for teachers, which was further developed in the White paper The Importance of Teaching. Museums are recognised in the White paper as having particular value to teachers when delivering education to students12.
8 9
http://www.bis.gov.uk/policies/science/science-and-society/stem-skills (accessed in February 2011) Review of the National Curriculum in England: Remit, (http://www.education.gov.uk/schools/teachingandlearning/curriculum/b0073043/remit-for-review-ofth the-national-curriculum-in-england/ accessed on 10 February 2011) 10 http://www.bis.gov.uk/policies/science/science-and-society/stem-skills 11 Business Plan 2011-2015, Department for Education, November 2010 12 White Paper: The Importance of Teaching, Department for Education, November 2010, p 46
13
Changes to the National Curriculum are included in the business plans. The National Curriculum Review was announced by Michael Gove, Secretary of State for Education, on 20 January 2011.13 Science and Mathematics are seen as core subjects and the teaching of the new programmes of study for English, Mathematics, Science and Physical Education becomes statutory in September 2013.14 This is a key time for the museum sector to influence how science and mathematics are taught, and so the evaluation of Science in Your World is timely.
http://www.education.gov.uk/schools/teachingandlearning/curriculum/a0073149/nationalth curriculum-review-launched (accessed 10 February 2011) 14 Timetable for curriculum review is at http://www.education.gov.uk/schools/teachingandlearning/curriculum/a0073092/timetable-for-thenational-curriculum-review (accessed 10th February 2011)
13
14
3.3.2
Arts and Culture Although currently the cultural policy landscape is in a state of flux, there are indications from the DCMS, as well as current events that affect the museum sector: A greater focus on exploring new ways of finding funding and delivering experiences, learning from the private sector, supported by NESTA (National Endowment for Science, Technology and the Arts) and the Arts Council.15 Science in Your World currently plays a part in disseminating information and advice about funding arrangements, and could support this agenda in the future The continued uncertainty of funding sources as a result of reductions in public sector funding16 have a clear impact on the continuation of activity of Science in Your World. This includes both the reduction in national strategic funding organisations that could support this programme of activity, and the reduction in funding for local authorities to support local or regional museums.
3.3.3
STEM policies In 2004, the then DCSF commissioned a mapping review of STEM provision across the UK. This uncovered around 470 separate initiatives delivered by a myriad of government departments, Higher and Further Education Institutions, professional and sector bodies, non-departmental government bodies, local education authorities, education business partnerships, charities and foundations. These findings demonstrated the vast proliferation of provision at the national, regional and local levels, a lack of coordination between providers and as a result overlapping provision.
http://www.culture.gov.uk/news/news_stories/7745.aspx The Comprehensive Spending Review outlines cuts to the public sector, sets out some of the th effects in the Museums Journey, Patrick Steel, 20 October 2010, (http://www.museumsassociation.org/museums-journal/news/20102010-comprehensive-spendingreview-museums , access on 10th February 2011)
16
15
15
These findings influenced the Science and Innovation Investment Framework 2004-2014 and the STEM Programme Report (2006), which tried to rationalise the number of programmes and increase coordination between providers. It did so through creating a high level STEM Strategy Group to join up STEM across all phases of education and make recommendations to Ministers about national STEM priorities. Consequently, STEMNET (previously SETNET) played a greater role in coordinating provision for primary and secondary provision. In particular, activity is coordinated through sub-regional brokerage contracts called STEM Points. These are awarded to various different types of institutions and aim to improve access, provide advice and guidance and organise events. The Higher Education STEM programme provides a similar role, alongside funding opportunities, within the HE sector. More recent initiatives are likely to include a greater focus on outcomes and evaluations are currently taking place of these initiatives like STEMNET (which has a strong regional focus) and East Midlands Development Agencys STEM Programme. These are as yet unpublished or not yet complete, however. In terms of future STEM policy, the Science Cluster of Sector Skills Councils is undertaking a strategic planning and review of activity to provide a fit between STEM skills provision and skill needs. The National Curriculum Review recognises the core importance of Science and Mathematics. There is a renewed commitment to expanding apprenticeships, being recognised as a major contributor to the nation's STEM skills. In further education, the Learning and Skills Improvement Service is working to improve the content and delivery of STEM skills. The Government has outlined its support for initiatives such as STEMNET, the Big Bang Fair, and HEFCE's strategically important and vulnerable subjects, and National HE STEM programmes.17 In addition Lord Browne, in his independent review of Higher Education funding and student finance, signals that public investment should be targeted on the teaching of priority subjects including
17
Hansard HC vol 513, Part No. 31, c678W (13 July 2010) (Hansard records of House of Commons debate)
16
science.18 Government rhetoric therefore suggests that there will be a renewed commitment to STEM skills policy. This emphasises the need for more outcome focused, robust evaluation.
18
Lord Browne, October 2010 Securing a sustainable future for higher education An independent review of higher education funding and student finance
17
3.4
Overview of the programme Science in Your World was managed by MLA, as part of the DCMS/DCSF funded Strategic Commissioning Programme, and is a two year programme that started in 2009. It has the following aims: To deliver new science learning opportunities in English museums To build skills and capacity in museum educators and curators to develop and deliver science learning opportunities To share knowledge about science education and success between museums To build demand in schools for science learning opportunities in museums To demonstrate that museums and out of classroom learning have a major role in delivering the STEM agenda To maximise the advocacy benefits for the partners and the museums sector from delivering science learning opportunities
It does this through: Developing regional museum networks to share knowledge Building capacity through continuing professional development for museum staff and volunteers Resourcing the development and delivery of science learning opportunities in regional museums
The development of the regional networks and associated activity was part of a National Partnership with the NHM and the Science Museum, who managed their own strategic commissioning programmes. 3.4.1 The first year (2009-2010) The programme commissioned three regional networks in its first year to deliver the Science in Your World objectives: SLIME (Science Links in Museum Education) in the South East, STEAM (Science and Technology Around Museums) in the North East, and SMELT (Science, Museums & Archives Encouraging Learning Together) in the West Midlands. SLIME and STEAM both existed before the programme and had a number of members already. SMELT was a new network set up for the programme itself. The funding allocated to these networks was 15,000 each, in total. They were to recruit members from across the science learning providers; deliver networking events; link members through online activities; promote and support members to access MLA funded opportunities including Science in Your World Learning Links and small grants; to
18
support the dissemination of the learning and resources created through these strands; and to deliver training. The networks supported the delivery of 17 small grants administered by the MLA of up to 3,000 each. These were designed to allow museums and archives to take on small, discrete projects that would develop new science learning opportunities and build partnerships across science learning providers. The total value of small grants was 47,340. Networks were also offered opportunities brokered through the national partnership, including three Talk Science programmes delivered in each region from the Science Museum. 3.4.2 The second year (2010-2011) This year refined the delivery model of the networks, as well as expanding across the North area. In the North West, Science in Your World has been developed using a partnership approach; SMILE (Science in Museums Inspiring Learning and Engagement) has been developed and led by the Learning Manager at Bolton Museums and Archives, and the regional project developed and led by MOSI in Manchester. Leeds Museums and Galleries took on the development of the network and regional project for Yorkshire, in partnership with Space Connections (YES-NET). This partnership model seeks to utilise the existing network and expertise developed by Space Connections across Yorkshire, which already has a small number of museum and archive members, as well as a number of additional partners across the Science Education field in Yorkshire, using established mechanisms and relationships. In addition to the previous activities building the capacity of the network and its members, all of the networks this year were given funding for a regional project. Each region had a different focus. This was designed as both an opportunity for the networks to deliver a programme engaging local museums in a regional project, focusing on regional need, supported and delivered through the network, but also for case studies and resources covering different audiences and areas of the STEM curriculum to be developed and shared nationally. This is important both as a learning tool and as an opportunity to demonstrate the breadth of engagement that museums and archives can contribute to the STEM agenda. The MLA project manager worked with the regions to plan programmes across the other Strategic Commissioning national workforce development programme funding streams, including Initial Teacher Training Grants, the Learning Links programme and in one case, the Talk for Writing Programme. This allowed the networks to be much more strategic in how they used available funds, as well as
19
planning a more coherent approach to the provision for their networks. In addition, the Talk Science programme, delivered by the Science Museums training team, delivered the Talk Science programme again in part during a SMILE CPD day in Bolton, as well as to interested non-network members in the East of England, one of the regions targeted for expansion should the programme have run beyond the current funding round. Total funding available directly to network leads from the Science in Your World programme is summarised in the table below.
Regional Project SLIME STEAM SMELT SMILE Network lead costs Learning Links Management costs Learning Links Placements Initial Teacher Training Total
10,000 10,000 10,000 10,000 10,000
2000 2000 2000 2000 2000
2000 2000 2000 2000 2000
6500 6500 6500 6500 6500 4000
20,500 24,500 20,500 20,500 20,500
YES-NET
The Learning Links programme is designed to offer opportunities for museum and archive educators, teachers, trainee teachers and other cultural sector and children's and young people's professionals to initiate, build and sustain learning partnerships through experiencing each other's working environment and practice. STEM themed Learning Links placements were offered to network partners through the regional leads. The Initial Teacher Training (ITT) grants programme builds learning partnerships between museums, archives and organisations offering initial teacher training. The National Literacy Trust worked in partnership with Renaissance teams, to run a series of training events offering the exciting 'Talk for Writing'19 approach. This range of tools used by schools across England can be used effectively with museum collections and archives to support literacy development. In the North East, STEAM used the Talk for Writing approach as part of their Science in Your World programme.
19
http://www.literacytrust.org.uk/projects_networks/talk_for_writing (Accessed 14 February 2011)
th
20
21
4.
4.1
Evaluation of the successes/progress of the networks

Introduction The regional networks are the backbone of this project. Each network was encouraged to develop models developed on local need, capacity and experience in response to the aims of the Science in Your World programme. They have each developed in a different way and are at different stages. This section of the report summarises the key features of each network. Each network is listed in turn, in order of their establishment. SLIME, (Science Links in Museum Education), South East Overview of the network SLIME has been running since 2006, and was one of the models for the Science in Your World programme20. SLIME exists to support museums to share practice, work together and offer more resources and programmes linked to science. It is the largest network in Science in Your World, and it had 78 members in 2009 when it received support through Science in Your World. As this network receives other funding, we are concentrating on the substantial Science in Your World funded activities in this evaluation. For the second year of the programme the regional project was GreenSLIME, a sustainability themed project. It worked with eight museums (selected from SLIME members), and three members took up Learning Links funding. Museums had the flexibility to decide the nature of their projects but each involved a school and link to schools with an environmental purpose. Each project helped improve the museums own environmental sustainability. Each project produced a case study to be shared across the network. The network was originally established with three aims: To support members to develop science learning from their collections
4.2 4.2.1
20
Alison Porter, Tim Caulton, Ian Blackwell, Natural Science Network: Draft Report on Consultation and Scoping;; 31st March 2008
22
4.2.2
To raise members confidence in developing science learning programmes To advocate the use of museums to teachers to support science learning
Outputs of the network 2009-2010 SLIME promoted and supported members access to the small grants and Science in Your World Learning Links, produced ten case studies, and collected data for the Initial Teacher Education (ITE) providers across the South East21, including contact details for those with a remit for science and information about how they plan to deliver to students. This was with funding from Science in Your World, although they also delivered other activity such as online forums, and CPD events. The small grants programme involved four museums, and reached 1,008 children and young people, 90 teachers and 140 adults22 over the time period of the funding. 2010-2011 The Regional Network SLIME used Science in Your World funding to hold the following: In addition to the GreenSLIME project the network has held the Ruth Borthwick Award for Excellence in Science Education in Museums in September and also held a network meeting for those interested in Medical Matters in Museums. A meeting was held on 29 September 2010 to share ideas, discuss a toolkit and plan a final event five museums were able to attend. An article has been submitted to a forthcoming publication Sustainable Museums: Strategies for the 21st Century A final event ran at Brooklands Museum on 3 February 2011. 40 people attended.
21 22
SLIME Data Capture Report, March 2010 This included 120 participants of all ages in family groups.
23
A workshop session on sustainability and museums was given to the Association of Science Educators national event
They delivered eight small grants23, and at the time of writing, these reached a total of 263 children and young people, 46 museum/archives staff taking part in CPD activity as well as seven other environmental organisations.
4.2.3
Successes of the network Small grants in the region were used effectively to work together to create new resources: These contributed to SLIMEs aims of giving members support to, and raising their confidence in, developing science learning programmes, for example: o Canterbury City Museum developed a resource that was included in the A History of the World Kent launch at the Museum of Canterbury on 20 February 2010 which attracted over 1,100 people Six museum trails with science themes were created in Lewes House, and seven contacts made with local schools,24 which will be used long beyond the funding period.
Promotion and development of cross-sector relationships to share best practice: which contributes to all three aims, for example: o Powell Cotton Museum involved a creative practitioner at an early stage. The museum cited the enthusiasm and willingness to participate with the students of this practitioner as enhancing student experience25, Fort Nelson/ Royal Armouries used the Education Business Partnerships and Extended Schools to market their programme, and felt that this helped to endorse and promote their work to a wider audience. 26
23 24
Some are still ongoing at the time of writing. Data capture report, end of project report, and SLIME case study write ups 25 Data capture report, end of project report 26 RA Fort Nelson Science in Your World Summative Evaluation, June 2010, Liz Denton
24
Links to national or existing events or schemes: The SLIME network has an outward looking approach, with their activities including marketing to nonmuseums, attending and delivering workshops at conferences aimed at science educational professionals and engaging with other potential stakeholders such as STEMNET. Museums such as Fort Nelson/ Royal Armouries cite links to national events such as National Science Week as an ideal way to market and attract schools for new science programmes27. This linking to existing events helped SLIME to contribute to all three aims, using the expertise of non-museum staff and networks to support its members, and also making links that could be used to advocate for museum involvement in science education. Interesting or unusual approaches trialed: SLIME and GreenSLIME have used the grants available to them to trial interesting approaches, such as student mentoring by secondary pupils in Rochester Guildhall. GreenSLIME itself, a regional focus on sustainability, is a new approach for the national programme. Large themed projects increased cross museum learning and development: Stakeholder interviews, including national partners suggest that using a theme, such as sustainability, for projects held museums to share learning, resources and advice between them: A slightly higher scale project with less overall partners, can have a longer legacy in those museums, whereas small grants funding one off events can have a greater audience, but cannot be embedded into the museum, Stakeholder, National Partner. The programme has shown that small one off grants and projects can leave a legacy within the institution (such as partnerships and resources) but the evidence does suggest greater learning and impact from themed projects.
4.2.4
Learning for the future Networks take time to build: The SLIME network has been established since 2006, and has developed in that time a large and active membership. However, interviews with the regional co-ordinator suggest that this success
27
RA Fort Nelson Science in Your World Summative Evaluation, June 2010, Liz Denton
25
did not happen in twelve, or eighteen months, and only now are the longer term impacts beginning to become apparent. Changing opinions and raising awareness of science learning in museums takes time: Although relationships with individual schools and teachers have been developed, the regional co-ordinator does not feel that there has been a large scale change in the awareness of museums as science learning venues. Networks need active support: The experience of SLIME suggests that networks do not build themselves, but needs a co-ordinator or co-ordinators to ensure that they grow.
4.3 4.3.1
STEAM, (Science and Technology Around Museums), North East Overview of the network The STEAM network is led by the North East Renaissance and was established before the Science in Your World Programme began. It aims to develop: confidence in delivering STEM based activities in museums increased subject knowledge in science and STEM related activities in the North East an increased awareness of the opportunities for STEM learning in museums across the North East
The Science in Your World funding was used to develop further partnerships and to raise the profile of science in museums across the North East, through network events and promoting and supporting access to the Science in Your World small grants and Learning Links. In the second year of funding STEAM built upon its success, with CPD and networking meetings, a presence at the Regional Annual Science Conference and continued links with STEMNET, STEMPOINT, Durham Science Learning Centre, the BASE, and Northumbria University. It also engaged with teachers and trainee teachers to develop their skills in delivering science learning outside the classroom, in addition to promoting museums as places that can be utilised in the delivery of the science curriculum. STEAMs regional project focused on developing a resource kit for museums that is aimed at secondary school pupils. This kit explores behind-the-scenes science. This was supported by a programme of Learning Links placements based on Talk for Writing aimed at improving scientific literacy.
26
4.3.2
Outputs of the network 2009-2010 The regional network delivered a science development day and regional science conference. It worked with 50 museums as members, and of those 18 were new contacts. Nine organisations received small grants funding, and reached children and 1033 young people, 129 teachers and 223 adults. 2010-2011 STEAM has delivered: STEAM developed and finalised a resource kit with enormous potential for long term work and attracted new network members, including teachers, venues and students. At the time of writing this has not gone live, but it will be cited at the North East museum Hub website: http://www.twmuseums.org.uk/hub/ A Regional Science Conference Five Learning Links using the Talk for Writing approach 20 year three science Bachelor of Education students acted as science curriculum advisors to 20 host venues, and completed a programme of work aimed to develop the science offer in these museums
At the time of writing, the regional project and learning links projects worked with 584 children and young people and the CPD elements included 148 teachers and trainee teachers and 22 museum/archive staff. 4.3.3 Successes of the network Large and varied membership: In the first year of funding STEAM has increased its membership to 50 individuals from a range of organisations delivering science learning, including museums, archives and heritage sites. The varied membership allows learning to pass between the different educators.28 The membership continued to grow, and added further venues, teachers and teaching students. 29 The ability for museums to learn from
28 29
STEAM End of Year Report 2009-2010 STEAM End of Year Report 2010-2011
27
different organisations and individuals, along with the understanding that develops when knowledge and learning is shared, contributed to the networks aims. Inclusion of national partners: STEAM has been able to use national partners to enhance its offer, including the Talk for Writing approach in partnership with the National Literacy Trust. This has increased opportunities for knowledge sharing and developing confidence for the museum members. Use of conferences to disseminate resources: STEAM has been closely integrated with the NE Renaissance activity, and used this to disseminate more widely the resources and learning developed by the network. For example, Monkwearmouth Station Museum has developed a toolkit Feel the Force that was utilised by other museums and at least once by another region30. This was disseminated during the Hub Science Conference in February 2010, and the workshop attracted 44 participants from science museums and schools.31 The Conference in 2011 was also well attended, and included the two universities to add further to the opportunities to learn from different organisations.
4.3.4
Learning for the future Benefiting from national support: Links to national partners such as Natural History Museum and Science Museum have brought benefits in advocacy, networking, and best practice.32 There were also benefits in linking to these national organisations for CPD events and the annual science conference. Integration into other activities in the region: STEAM has been successful at operating with other activities and Strategic Commissioning strands, partly through its base at the NE Museum Hub. For example, it held a combined workshop with the History Learning network to be of benefit to members of both, that was a Talk for Writing approach. Membership included strategic local organisations that can support advocacy: These strategic organisations include the National Draft Centre, the Blue Aquarium, STEMNET at Newcastle college, Science City in Newcastle (a
30 31
Evidence from the Online Survey Feel the Force Case study produced by STEAM 32 Stakeholder interview with STEAM Network Co-ordinator
28
partnership between the local authority, the university and business), North East Field Officer for Association for Science Education (ASE), universities of Newcastle, Durham and Sunderland, and Robert Stevenson Trust. 33 These links enable museums to advocate their value to science learning beyond the sector, particularly importantly to teacher trainees.
4.4
SMELT, (Science, Museums & Archives Encouraging Learning Together), West Midlands Overview of the network SMELT was a new regional network for the West Midlands established during the first year of Science in Your World and led by the Thinktank museum in Birmingham. Its delivery model was based on that of SLIME, with outreach to local museums, an online Google group to share news and resources, and CPD meetings and events. The aims in 2009-2010 were to establish a network that would make contact between local museums, develop its ways of working, support best practice, and work towards a greater understanding of the needs of the museum members. The aims of the 2010-2011 programme were to build on the learning from the first year, and to use the network and the regional project to meet the needs of network members that were identified in year one of the programme. SMELT built upon its initial gains through maintaining its online presence, and developing partners to support members in developing science learning in relation to their collections.
4.4.1
33
Stakeholder interview with STEAM regional co-ordinator
29
4.4.2
Outputs of the network 2009-2010 The SMELT network delivered one networking events and one CDP meeting, with an attendance of 55 adults. SMELT promoted and supported access to the Science in Your World grants and Learning Links programme. Four small grants were delivered that reached a total of 496 children and young people, 56 teachers and 28 adults. 2010-2011 The SMELT Network ran one CPD session and one showcasing event. It delivered one regional grant and worked with a range of partners to produce a virtual toolkit. It also delivered three learning links grants. At the time of writing, SMELT network activity included 31 children and young people, five teachers taking place in CPD and 27 museum/archive staff.
4.4.3
Successes of the network Need and demand for network established: The first year established a need and demand within the region to support science learning within museums and archives. This was developed through meetings and consultation with museums that identified a lack of confidence in developing and delivering science learning. There was also consultation with teachers and schools that determined that they would take advantage of opportunities to learn outside the classroom.34 The second year built on this, and although take up of 2010-2011 grants has not been to the extent expected, the showcasing event did indicate that there was a demand for the network to continue.35 Grants were successful: The grants contributed to the aims of supporting the overall Science in Your World aims: o The Herbert delivered a very popular workshop programme Fun with Forces, two other activity sessions, developed new relationships in the region, and pupils put up a display in the Natural History Museum
34 35
SMELT End of Year Report 2010-2011, Stakeholder interview SMELT End of Year Report 2010-2011
30
RAF Cosford delivered workshops to a large number of students and adults (over 380 pupils), that included outcomes of intergenerational contact and learning. The Pen Room engaged with a local school to develop a museum detective tour, which museums volunteers identified as being important for their future sustainability as a museum.
Online toolkit developed: SMELT engaged in a partnership approach to developing an online toolkit to support museums to deliver science learning. The virtual toolkit, https://sites.google.com/site/smeltnetwork is aimed at museum professionals from across the country and uses experiences gained from all the network grants and other professional organisations to give museum staff greater confidence in developing secondary science activities. This has contributed to the aim of building on the learning from the first year.
4.4.4
Learning for the future Regional networks may need time to develop their own needs and identity: The network has grown through the funding period, however it is not yet sustainable. There has also been a lack of take up of the available grants. There is no obvious reason for this, however, as the network developed using the templates of SLIME and STEAM, it may be that the network needs more time to embed and develop its own regional identity. Museum Development Officers and sub-regional networks: SMELT have developed good links with the Museum Development Officers working in the Local Authorities. This offers another perspective, and there is scope to further increase this partnership in the future. Specialist science learning museums have an important role to play: The leadership of the Thinktank museum enabled the membership to access a high level of science knowledge and support around delivering science learning in museums. Google groups is not accessible for all: Local authority based museums and members are not able to access Google during work time. As this was the basis of the online network, this had ramifications for its suitability. It is noted that this is contrary to the experience of STEAM in the North East.
4.5
SMILE, (Science in Museums Inspiring Learning and Engagement), North West Overview of the network Started in 2010, SMILE is a North West based network that is led by Bolton Museum and Archive Service. It has established an online network group with
4.5.1
31
Google Groups; delivered an awareness raising campaign by using existing online education networks like GEM; planned, advertised and co-ordinated three CPD events for museum staff with interests in science learning; worked with the MOSI Regional Science Project; promoted & co-ordinated the Science Learning Links project and ensured that appropriate case studies are identified and collected from the SMILE network. The objectives of the network are: To increase opportunities to access training to deliver Science Learning in museums and archives through CPD events and access to Learning Links placements. To build partnerships across the Science Learning sector in the North West. To share best practice across the region, and to raise the profile of museum, library and archive learning.
The regional project for the network was developed by a partnership led by Museum of Science and Industry, in Manchester. The project: Engaged students directly through a laboratory science programme that uses museum objects, narratives and activities that are learner centred. This learning experience fuses the themes of natural history, chemistry and social history by engaging Key Stage 3 students with a real life case study which looks at science in the 19th century. Students examined the case study of William Perkin, the scientist who discovered the first synthetic dye. Trains partners how to deliver a practical science workshop that features a social history narrative to illuminate collections and highlight the industrial history of the Northwest. CPD enabled smaller venues to successfully deliver science learning opportunities to hard to reach audiences at Key Stage 3.
It aims are to: Enhance scientific skills, literacy and understanding for chemistry units at Key Stage 3. Interpret natural history collections and textile collections in a manner that is relevant to art/design and science today. Give students insight into original practice in textile manufacture which was imperative to the development of the North West.
32
4.5.2
Outputs of the network SMILE has delivered three CPD days and one Learning Links placement. They used the rest of the available learning links grants to recruit 10 museums for mini Learning Links that involved a shorter one day meeting with teachers to develop the science learning. Nine of these occurred within the timescale. They have also set up a SMILE science online network. Plans have been put in place to make the SMILE network sustainable and beneficial to the region beyond March 2011. MOSI designed and tested the workshops, including science lab workshop. Resource material completed, including detailed social history. It worked with three partners, and presented at a SMILE CPD day The workshop resources will be made available to all Science in Your World partners. The SMILE Network delivered CPD to 50 museum/archive professionals, and the MOSI regional project included 113 KS3 students and 17 museum/archive staff taking part in CPD activity.
4.5.3
Successes of the network Membership has grown quickly: Despite the late start of the network, it has 20 members so far all connected to SMILEs Google groups. The first CPD event was successful with 18 people attending. It was used to define the regional need for the programme and the parameters of the network.36 The following events reached similar levels of people, including new members as well as repeat attendees. CPD days were identified as the most successful area of the project: In the End of Year Report the three CPD days were identified as being particularly successful, with sharing of ideas, methods of teaching and engagement. These events were held in Liverpool event on 28 September 2010, Bolton on 7 December 2010; and Blackburn on 14 January 2011. MOSI overcame setbacks to deliver in a short timescale: There was cancellation of a local programme that would have been used to recruit schools
36
SMILE Interim Report 2010-2011; Stakeholder Interview with SMIKE Network Co-ordinator
33
for the MOSI project. However, this has not had a significant impact on the progress of the project, and schools were recruited through other means.37
4.5.4
Learning for the future Flexibility with Learning Links helped to achieve outcomes and overcome barriers: There was a difficulty in recruiting museums and teachers to take part in Learning Links programme. In partnership with the MLA, it was decided to offer more, smaller mini Learning Links that would allow museums to have one meeting with a teacher to collaborate on their science offer. This was held to be successful by the regional lead, and participants valued them38. Transport issues are a consideration in rural regions: The events all had good attendance from local museums. However, it was noticeable that museums were more reluctant to travel away from their sub-region. In this case it is likely that the size of the North West was a contributing factor.
4.6
YES-NET, Yorkshire and Humber
4.6.1
Overview of the network This Yorkshire based network started very late in 2010. It is led by Leeds Museums and Galleries, and aims to integrate a science learning aspect to the existing museum network YES-NET. YES-NET has 27 members, and is an established network of extra curricula science providers in the Yorkshire and the Humber region. The members include national and independent museums, professional bodies, independent associations, universities and other relevant organisations who wish to participate in a virtual network. Leeds Museums and Galleries integrated and embedded a science based learning in museums strand of activity to this existing network to deliver the aims objectives of Science in Your World.
37 38
MOSI Interim Report 2010-2011 From Miini- Learning Links End of Project Reports
34
4.6.2
Outputs of the network Yes-Net delivered one CPD event, a conference that attracted teachers and museums. They produced quarterly e-newsletters. Their regional project is still being delivered, and the majority of the activity will occur during half term. This worked with four partners to deliver science based activities. There are four learning links planned, although not all have been completed at the time of writing. At the time of writing, 320 children and young people and 335 family visitors have been reached through the regional project, and 21 Teachers and 43 museum/ archive organisations took part in CPD activity
4.6.3
Successes of the network Hit the ground running: This programme was delivered using an existing network of 27 members, so they were able to hit the ground running with the outreach and promotion, helping them to meet their targets. Although they did not deliver all five learning links planned, they did deliver four, and similarly worked with four regional partners and delivered a large and successful CPD event in an extremely tight timescale. This network started later than the others, and so has had less time to achieve its goals of embedding overall Science in Your World aims in the Yorkshire region. This has been compounded by the end of the programme being moved forward, making the delivery of the programme challenging. However, its approach has meant that it has grown rapidly. Successful conference with high teacher attendance: The conference held in Leeds Museum on 7 February 2011 was extremely successful in attracting a wide audience, with over 60 attendees, including teachers and museums. It aimed at ways of sharing innovative science learning opportunities in museums. A full description can be found in Appendix III.
4.6.4
Learning for the future Piggybacking on existing networks is a good way to build momentum for a project quickly: The fact that YES-NET already existed and was successful meant that the programme was able to market its activities to a wide audience from the start. It could use established channels of communication to museums, schools and stakeholders, and could take advantage of the high regard of these audiences for the network. Practical sessions for innovate teaching: The event was a success, especially in its ability to attract teachers. The teachers at the event highlighted the desire to learn practical ideas and skills for teaching science, as well as keeping abreast with innovative teaching practice.
35
36
5.
5.1
The impact and outcomes of the programme

Outcomes from the programme
5.1.1
For museum staff The programme has delivered impacts for museum staff throughout the country. These are summarised here: Developed confidence and skills to develop and deliver science education in museums: The regional networks have developed confidence in their members. This is shown by the Science in Your World evaluation online survey of network members, which cites gains in confidence as a clear impact on them when attending events: I also gained confidence in using the equipment needed to teach secondary science safely. This meant that I now feel more confident in teaching secondary school children within the museum setting. The activity observations for the SMELT conference, SMILE CPD day and YES-NET one day conference also observed gained confidence in delivering science learning from all non-science specialist museums we spoke to, including history based museums. For example please see the SMILE CPD day activity observation in Appendix III. This confidence was developed through: o Increasing knowledge of science based workshops or programmes for example39, the MOSI project created a workshop that was used by other museums as part of the project40, the conference with YES-NET included sharing of workshops and activities that were found to be useful by other museums for example, at one demonstration by the Science Museum there were museum staff of another museum who stated I will steal that idea about demonstrating explosions. links to other museums delivering similar work that could be contacted for support, as cited by participants in CPD day for SMILE, for example
37
all 18 participants agreed strongly with the statement I have made new contacts today and I will contact people I met today following this meeting. o Links to national and regional science based museums highlighted by stakeholder interviews41 and as demonstrated by projects such as MOSI42. The participants cited these in the online survey (for example This gave me confidence in teaching the lesson dying to change the world )and in the activity observation as being confidence raising (At SMILE CPD day there was a comment about a presentation by Science Museum I feel I can have a go [at delivering science based learning], now). Links to teachers, students, university staff, and other professional scientists to work with the museum in developing these programmesi. This was highlighted as being particularly important by participants in the SMELT celebration conference, where particularly close relationships had been built in projects such as a new workshop developed for the Pen Museum by teachers, university students and the museum volunteers. In addition small grants that involved professionals also developed links such as Canterbury City Museum used a creative practitioner and students from Canterbury Christ Church University to develop a programme.
Personal networks between museum educators: These were developed throughout the programme, particularly through the events and the regional projects, although it is notable that respondents also used the SLIME online network for peer to peer activity, and valued it for helping them gain advice from other members. This included observation of details being exchanged during the activity observation at the CPD day at SMILE, and during the YESNET conference. Stakeholders also suggested: The network was the strongest elementwe were able to share different ideas from museums around the
42 42
Interim Grant Report Form, SLIME, October 2010. Activity Observations with organ
38
regionit was particularly useful for those museums without a science offer. It improved their confidence and able to work more with science in their venues. We know through informal discussion at network meetings, and hard copy evaluations. Personal networks developed with individual teachers: These were developed, particularly through the small grants programme, and the Science in Your World Learning Links programme. As one museum stated I know now that I can pick up the phone if I have a question about the national curriculum, and she can at least point me in the right direction43 Evaluations of the small grants also support this, with statements about individual teachers being supportive, or suggestions of future activities to be done with teachers: Our meetings enabled an opportunity to brainstorm ideas and forge new links between schools and museums to enhance future learning opportunities44. The stakeholder interviews also cited the importance of changing minds one at a time, and suggested that museum champions in schools are the key to engagement with the wider institution. SMILE CPD day, Bolton Building networks between museums and museum educators A CPD day was held at Bolton Museum and attracted 18 attendees from museums throughout the North West. The day held participative workshops showcasing science learning approaches from regional and national museums, including a hands-on Talk Science Session led by the Science Museum, looking at using objects to engage in contemporary science discussion. All participants agreed strongly that: they had made new contacts and that they would contact people from the CPD day in the future, they felt more confident delivering science workshops, and that they had learnt something new.
5.1.2
For museums Increased funding opportunities were supported through the programme. The largest funding leveraged using the Science in Your World grant for
43 44
Attendee at SMELT conference 2010 Mini Learning Links Report, Bolton Museum
39
537,000 for George Marshal Medical Museum. During the second phase of the grant making process, the museum lead on this bid used the innovative project as evidence of their ability to both attract outside funds, and deliver interesting science programming. While it was not the only factor in the decision of the Heritage Lottery Fund, the museum lead felt that it did significantly strengthen their overall bid45. Other funding sources have been disseminated over the internet by all the regional networks, and the online survey identified a successful bid for a Public Engagement Grant from the Institute of Physics to run an Astronomy Evening for Brownies in conjunction with the Mary Rose Museum. Networking with other museums was done through staff time, as summarised above. We have not collected evidence that this resulted in any formal partnerships between museums, beyond those attached to funded projects such as the MOSI project, or the YES-NET regional project. Links with schools were made by museums, although it was felt by stakeholders that these were often between individuals, and concern were raised that they were vulnerable to changes of staff at either the museum or the school. A stakeholder stated There were many one to one relationships developed, although I worry that these may change is the teacher changes jobs regarding the SLIME network. Development of new ways of working: In some cases there was evidence of a more strategic impact on the programming of individual museums that were influenced by the programme. This included instances of focusing more attention to new audiences, (for example, Activity Observation with MOSI suggested that the museum could attract secondary pupils, a new market for them), rethinking existing provision (Booth Museum of Natural History response to our Online Survey) and working more collaboratively.
SMELT Showcase event Building Links with schools The event included presentations from recipients of Science in Your World
45
SMELT End of Project Report 2010-2011
40
funding, feedback on the SMELT regional online resource kit and discussion about the future of the network. Museum staff discussed the value of the funding and the additional opportunities available through the network. One museum was struggling to attract schools, but the resource and relationships developed will help us with this. This is vital for our sustainability and another needed the expertise of [the Special Educational Needs (SEN) teacher] to develop appropriate workshops. We are a small museum and the smaller class sizes make SEN schools a good market for us. Teachers in attendance added valuable insight around the future of the network and how it could continue to build partnerships with and demand from schools.
5.1.3
For teachers New teaching ideas and skills have been disseminated: Teachers from the YES-NET conference came for the practical demonstrations of science learning along with a desire to understand what museums could offer their students. There was a particular desire for education activities that could be used by teachers to teach concepts or aspects of the curriculum that were not being understood in the classroom. We spoke to over ten teachers about the day during lunch and after the demonstrations, and they were positive about the practical ideas they had learned, and would use them in the classroom. One Primary School teacher was teaching about the human body and was pleased to learn a way of modelling it for young children in a fun way. This was corroborated by evidence from case studies and grants paperwork, for example seeing pupils allowed to investigate and experiment gave me confidence to be more creative in my approach.46 Individual awareness of museums as good venues for science learning was raised: Learning Links, small grants, and the events such as YES-NET conference in February 2011 were very successful at making the case for science learning in museums. This was clear in the end of project reports (for example, Powell Cotton Museums small grant included the note that the principal of the school that they worked with has become an advocate of the
46
Case Study for Feel the Force in Monkwearmouth produced for SLIME in 2009-2010
41
project and is looking to find further links to the museum in other subject areas including other science subjects and maths), interviews with teachers during activity observations such as the SMELT case study showcase, (I hadnt considered using museums to teach science to be honest) and the YES-NET conference confirmed that the awareness for science based learning in museum was changed as a result of the programme (I thought museums were for history) Stakeholder interviews with both regional co-ordinators and national partners confirmed the view that individual teachers were becoming more aware through the programme: The idea of using teachers to help museums to develop science opportunities is a good one, and the individual teachers will learn more about using museums in this way. Wider awareness of museums as venues for science learning opportunities has been less successful: The programme has had a stall at the Association of Science Educations Annual Conferences, and SLIME was mentioned in the Telegraph this year for promoting science learning in museums at this event47. Despite these activities, stakeholders all feel that there is not a sufficient general level of awareness amongst teachers of the value of science learning in museums: One to one relationships have been built, but it is usually those teachers that have been involved that value us it does not go beyond these one to one relationships . This has been recognised as a slow process which requires changing the overall culture in schools, and this is the next step for many of the networks: We have not yet scratched the surface of the sector.
YES-NET Conference, Leeds Museum New teaching ideas and skills have been disseminated This event attracted over 60 attendees, including a high number of teachers (mainly primary).. It comprised of talks by museums and educators presenting ways of delivering science learning in museums as well as hands on workshops demonstrating a range of techniques used to teach science in museums. This aspect of the day appealed to teachers, who commented that [they] wanted practical ideas to teach science and that they attended
47
http://www.telegraph.co.uk/science/steve-jones/8252195/Carry-on-camping-to-improve-science-education.html
42
because [they] are redesigning our curriculum at the moment, and are focusing on projects or themes that we can explore. They need to be cross-curricular, but we are struggling to include science. Teachers were inspired by the sessions and came away with a book of practical ideas. During a practical session on using skulls to teach evolution, one teacher commented I wish they would come out to our school.
5.1.4
For schools In general outcomes for schools were harder to measure. Teachers, heads of department and head teachers developed relationships with museums. We collected little evidence that this went beyond individual networking to a more systemic and strategic impact. Individual interviews seem to suggest that it is necessary for a Museum Champion to be present in a school who will support teachers to both take children on trips in spite of the expense of transport and loss of school based teaching time, and who will also get involved in working with museums to design new workshops and programming. For young people It was difficult for this programme to prove outcomes for children and young people in the timescale available. This is partially due to the long term nature of some of the outcomes desired (such as changing career paths, or attainment in science based projects) and also due to much of the delivery to young people falling outside the timescale of the funding. There is also the problem of attributing changes in attainment to one project, as there are many different influences on childrens attainment. However, there have been some identifiable outcomes: A high number of young people accessing resources and programming: Data capture and initial data from the regional projects shows a high number of young people who engaged in the resources and programmes created in the period of time during and immediately after the period of the small grants programme, and during the period of the regional projects. Over 3800 young people have been reached directly at the time of writing, although as some projects are ongoing the final output will be greater.
5.1.5
43
Some projects used young people to support the design of resources: For example, Worcestershire History Centre developed a relationship with an engineering club in a local secondary school, which not only fed into this project, but are also likely to use the museum for future initiatives48. Activity observation showed young people engage well with museum based learning; We observed activity at both SEARCH in Gosport and in MOSI, and these events showed that the young people were particularly engaged with the projects, and comments from teachers at these events supported the use of museums to enable science to come alive for the young people. Other outcomes of the activity observations on young people: The sample sizes were small, however, indicatively there were the following outcomes gained: o At MOSI: Fun, engagement with science, learning about science and history At SEARCH: Fun, practical learning, working as a team.
These outcomes have been true for many activities: For example: o Some students were motivated to learn more: Some pupils who struggle in the formal classroom setting were motivated in the museum setting, Head Teacher - , Primary School49 The museum let students see science in a practical, real world setting: The students spent lots of time investigating and thinking about other variables that they could test and many students could see applications to everyday circumstances and It has opened up further discussion and students are seeing where they can apply this elsewhere and where they can go next with the materials50
48 49
End of Project Report 2009- 2010, Worcestershire History Centre Segundum End of Project Report 2009-2010 50 Tyne and Wear Museum, End of Project Report 2009-2010
44
Creativity was encouraged: lots of enjoyment and inspiration. Some of the pupils showed much more creativity than has been observed in normal class situations51. Learning was seen to be more enjoyable: it was more funner than science at school 52 pupil.
Dyeing to Change the World, MOSI Key Stage 3 students inspired by Science in Museums A workshop session of the Dyeing to Change the World project at MOSI was delivered to young people, and attended by a partner museum Fifteen young people were consulted at the end of the event, and some said they were more interested in Science as a result of this session, and that the best bits were the practical activities, ie crushing the membrane/using the centrifuge, and learning about the history of Manchester and its links with the cotton industry. The teacher commented that students were keen to go home and show their parents/tell their parents about some of the stuff they had done and that seeing a real Scientist in practice promotes Science as a tangible idea to young people.
Biodiversity Session, SEARCH, Gosport Museums supporting schools to better engage young people in biology Hampshire County Council received funding through the SLIME network and Science in Your World to develop a new approach to teaching habitats for Key Stage 2 pupils. The project was devised as a result directly of requests from local schools to combine hands on museum based activity with creative methods, to better engage pupils in biology and sciences. Project officers believed that this project would build stronger links with other Schools in the area, once they were engaged and particularly through use of creative methods. There was clear enthusiasm from the observed year four group. Pupils attention and interest was held throughout the 2 hour session. Pupils commented that its more fun than learning at school and you get to see
51 52
Feel the Force, Monmouthshire From a self completed MOSI evaluation form
45
real things, not pictures.
5.2 5.2.1
Impact Increased use of museums as a resource for science education by schools and teachers Science in Your World has increased both the quality and quantity of science education opportunities in museums, and these have been used by schools and teachers. For example: Through the network we have been able to increase capacity to offer more science based sessions, as well as those who have a strong offer to develop new sessions, and we are able to enrich existing workshop a comment from a stakeholders from STEAM, and from the online survey in response to a question about what impact their network had on them New sessions were developed [via support from the programme] that were not history based. [the project] Brought into the museum five schools which had not previously visited; very positive feedback. Furthermore the teachers that have taken part in these activities have benefited from the interaction, through increased skills, ideas, and resources. For example, a stakeholder interview stated that teachers involved in [STEAM projects were] given confidence and skills to work with pupils to develop learning journeys; for multiple quotes from teachers, please see the activity observation for YES-NET in Appendix III. This suggests that the programme has increased use of museums as resource for science education.
5.2.2
Widespread recognition of museums value to the STEM agenda Stakeholder interviews have not agreed that this impact was reached, for example We are achieving pockets of recognition, but it is not widespread, we are working towards this. However, this was identified as a longer term objective by all networks, and the length of the programme has not been sufficient to prove this impact: There has been increased recognition of museums value to the STEM agenda within the museum sector itself, as evidenced by the number of museums who have seen the relevance of with own collections to science based learning for the first time, for example the online survey included comments such as Made us do activities we have never done before and use subjects we have not done before. Also the programme has increased understanding of museums value in the school sector, for example by the new relationships developed as a result of the Learning Links programme and evidenced in their evaluation reports.
46
5.2.3
Improved educational attainment in STEM through better teaching This impact is a long term aspiration, and as such is cannot be measured by this evaluation. However, the evaluation does suggest that teachers value the networks and projects to give them new ideas for teaching practice, for example a teacher involved in Learning Links in Guildhall Rochester stated the workshop as a Very effective way of connecting with the overall aims. Children got a lot out of it. Other evidence from teachers included It was useful to get another perspective from the teaching staff [meaning the museum educators] at the museum from a teacher present at the SMELT conference, and more widely teacher feedback from the YES-NET science conference. This suggests that Science in Your World is moving in the direction of this impact, through improving teaching practice. Young people inspired by museums to follow a career in science Although this impact cannot be measured in the timescale available for this evaluation, there have been positive indications by observers such as teachers and museums themselves. Museum visits can impact on young people to become more interested in science, and this evaluation suggests that Science in Your World projects have this impact on young people, as we found in the MOSI activity observation (there is a potential to achieve this as placing pupils in a different context and seeing a 'real Scientist' in practice promotes Science as a tangible idea to young people), and in some evaluations such as those provided by SLIME and the small grants, and suggested as possible by stakeholders. For example, [the teacher] could see several ways in which the work and approach of the featured artists could inspire students and allow them to consider science from a new perspective.53
5.2.4
53
From Evaluation Report of Mini learning links from Bury St Edmunds, Suffolk
47
6.
6.1
The extent to which the programme met its objectives

To deliver new science learning opportunities in English museums More and better science learning opportunities were delivered by this programme: The programme overall has directly funded 17 projects in the first year, and more than 12 in the second year54. Over 3,800 young people and 500 family members have participated in programmes developed through Science in Your World funded opportunities, over 450 teaching staff55, and over 350 museum and archive staff have participated in CPD programmes, learning links placements or other Science in Your World funded opportunities The evidence from regional partner stakeholder interviews is that these small grants gave the museums space and time to collaborate with appropriate people to develop new programmes (although in some cases this may have happened anyway),: Before there were not many science learning opportunities tailored for schools, we have increased supply, and the sessions are booked up., we have increased capacity to offer more science based sessions, as well as helping those who have a strong offer to develop new sessions, [we are] able to enrich existing workshops. This was confirmed by interviews with museums at SMELT Activity Observation: The project allowed us to bring in teachers...making a better workshop the Pen Room detectives workbook is much better and more glossy than anything we could have done before. Better science based programme delivered by museums: Although it is difficult to establish the scale of this, Science in Your World supported museums to deliver projects that were higher quality than they would otherwise have been, and science programming that was better than it would otherwise have been: They have improved the sessions we offer Respondent from Survey Questionnaire. There are a few possible mechanisms for this including: More interaction with teachers, for example, the involvement of a Special Educational Needs teacher with a museum in the West Midlands (Erasmus Darwin House) using a Science in Your World Learning Links placement meant
54 55
The final reports are not yet in at the time of writing. Including classroom assistants
48
that activities could be designed throughout the museum that would teach science to this group56; the YES-NET conference allowed museums to talk to teachers about their needs. Dissemination of best practice, for example, Booth Museum of Natural History responded to the Online Survey claiming that they revamped their science offer as a result of Science in Your World activities. A stakeholder suggested that If you wanted to develop a resource about something you dont need knowledge internally, bring in expert advice.
6.2
To build skills and capacity in museum educators and curators to develop and deliver science learning opportunities Science in Your World has delivered this objective through achieving outcomes for museum educators and curators that include57: Increased confidence in science based programmes Increased skill in science based programme design and delivery Increased networking with other curators and teachers (and science learning professionals) to support development of science delivery
There is also evidence from stakeholder interviews with regional co-ordinators and the online survey that resources such as case studies help museums to develop and deliver science learning opportunities that they would otherwise not be able to, or which would take considerably longer for them to develop. For example, Science programmes that involve laboratory practice for KS3 students which we have not had the capacity to do before Delivering workshops developed by other museums and shared through the network were two comments from the online survey. The extent to which this has occurred has been more difficult to evidence. Those who have attended CPD events, or delivered projects, have been mostly positive about the experience, as evidenced by regional level evaluations and the end of year reporting, activity observations and the online survey. More established
56 57
Erasmus Darwin House End of Project Report See Chapter 4 on Outcomes and Impact for details
49
networks such as SLIME and STEAM have higher membership numbers, and a high level of support for resources produced, events run and grants available. 6.3 To share knowledge about science education and success between museums The regional networks have developed strong methods of sharing knowledge and success within the network. These include online networks and collaboration, case study production, resource development and face to face training and events. These all foster the ability of members to engage in peer to peer learning, as well as access training and advice. For example: 23 out of 37 of our Online Survey respondents had accessed the network online. Of those, 100 per cent had received information online from their networks, gaining funding information, new ideas and resources, and general news. SMELT has developed and tested an online resource, which has been contributed to by its members. Feedback from its members show that the showcasing days and chance to find out what other museums are doing are the most valued part of the network.58 STEAM has held conferences and delivered workshops as part of the North East Hub development programme.59 This included attending the Regional Science conference which was attended by a range of practitioners, museums, ITT students and teachers. YES-NET has held an event that attracted a large audience to share ideas and programmes to innovative teach science. The audience included universities, teachers and museum staff.60
There was a need identified by some stakeholders to increase the amount of sharing between regional networks. Some networks have used the opportunity to access national partners knowledge and expertise (for example, YES NET had both Science Museum and the National History Museum attend and deliver talks
58 59
Draft End of Project Report February 2011 STEAM End of Project Report February 2011, 60 YES-NET End of Project Report February 2011
50
or workshops at the conference). However, other stakeholders would like more input from the partners, and more sharing of information and advice: I was hoping for more contact with the nationals, stakeholder. There have been suggestions of a national online resource that can facilitate this sharing from stakeholders in the programmes. STEAM has been a good example of a network that has used national partner input, hosting Talk Science programmes61, inviting partners to host workshops in the region and using the Talk for Writing approach developed by the National Literacy Trust and offered through other Strategic Commissioning programme strands.
61
Talk Science was used by all networks in the first year
51
6.4
To build demand in schools for science learning opportunities in museums The take up of the Learning Links programme, small grants projects, and the large teacher attendance of the YES-NET conference suggest that the programme has fulfilled existing demand in schools for science learning opportunities in museums. However, it is clear from interviews with teachers during activity observation, and stakeholder interviews that this demand has also been built through making the case to individuals, who then understand its value. Those [teachers] we have worked with became very enthusiastic Stakeholder interview, SMELT. Museums are still working very hard to market their offer to schools and teachers, and while the programme has created individual museum champions, there is no evidence that it has not gone beyond this to a shift in awareness with teachers generally. The programme has started to develop this broader shift, for example by having stalls at the Association of Science Education conference, but the evidence does not yet show the results of this activity. To demonstrate that museums and out of classroom learning have a major role in delivering the STEM agenda [This workshop has]highlighted how science can be used in the world outside of school. Too often, pupils cannot make the connection between what they do in the classroom and what happens outside of school, post education. Key Stage 4 teacher62 The networks and associated Science in Your World opportunities have demonstrated that there is need and demand for science learning to take place in museums. Quotes include: Before not many science learning opportunities tailored for schools, not there are, have increased supply, and the sessions are booked up then there is a demand. We were only beginning to skim the surface of the need for support with science teaching that is clearly in evidence from this event both from the excellent take-up at short notice to the point that we were having to turn delegates away at the last moment to the fact that we only partially
6.5
62
RA Fort Nelson Science in Your World Summative Evaluation, June 2010, Liz Denton
52
met so many needs. The delegates were mostly from the museum sector but closely followed by primary school teachers.63 The programme has demonstrated that this learning can add value to the delivery of science education in schools as illustrated by the quote from the Key Stage 4 teacher above. It also developed a building body of case studies and projects that have demonstrated the value of museums as tools for learning about a range of STEM subjects, for a range of audiences, including primary, secondary and families. This suggests that Science in Your World is working well towards this objective. 6.6 To maximise the advocacy benefits for the partners and the museums sector from delivering science learning opportunities On a national level, Science in Your World has developed partnerships between national museums and bodies such as STEMNET and Association for Science Education. Project reports and interviews show that regional networks have largely focused on developing partnerships between museums and schools, with the exceptions of SLIME and STEAM. SLIME and STEAM, as established networks, are also starting to work more strategically in their region64 as summarised in Section 4. However, the extent to which this has increased benefits for advocacy has not been demonstrated by this evaluation
63 64
From the Yes-Net End of Project Report, 2010-2011 Stakeholder Interviews, End of Project Reports 2010-2011
53
7.
7.1
Conclusions and recommendations

Conclusions The Science in Your World programme has clearly met a need in the museum sector for support in delivering science based learning. This evaluation suggests strong outcomes for museums, educators and young people, and an indication of the future impact of the programme. The key findings from the evaluation are outlined in this section. Demand from museums has been demonstrated The offer by regional networks of opportunities to develop STEM learning has been welcomed by local and regional museums. Furthermore, there is a need expressed by museums for ongoing support of this kind. The regional networks offer appropriate support The size and success of support seems to grow with the length of time that the network operates, as the member list of SLIME and STEAM demonstrates, although SMILE and YES-NET in particular are growing quickly. This suggests that the support offered by the networks, and the way in which it is offered is valued by museum educators. This conclusion is strengthened by the feedback from museums. The support that is identified as being particularly valued is: small grants, preferably linked to partnership with other local museums and schools new resources, such as toolkits, or workshop outlines produce by small grants and learning links which can be accessed throughout the region facilitation of face to face, peer to peer support, delivered through events facilitation of relationships with other relevant professionals, including teachers and scientists.
7.1.1
7.1.2
7.1.3
Networks have built capacity in museum staff The evaluation has demonstrated that Science in Your World has built capacity in museum staff to deliver science learning opportunities. This capacity was developed through increasing confidence and the skills of museum staff, and providing resources and networking opportunities with peers and other professionals.
54
7.1.4
Teacher-museum partnerships have been developed Many one on one teacher-museum partnerships have been developed. These relationships have changed teachers opinions on using museums, and museum staff ability to offer appropriate learning opportunities. These partnerships are affected by the following factors: Head teacher support is vital for these partnerships. A museum champion in a position of influence, such as head teachers, assistant heads, or heads of department can be lynchpin of these relationships. Timing of the offer to integrate with the school timetable and yearly cycle Making curriculum links of the proposed links clear to teachers External factors such as the availability of time to take children out of school, transport costs, or other commitments in the school that take up childrens time for visits etc. can have a profound impact on schools ability to develop these partnerships Teacher trainees are also a valuable source of support, advice, and mutually beneficial partnerships. They often have less time constraints to support museums, and can take the learning from the museum to the schools they work for after graduation.
There has not been a sea change of opinion and teachers often must be first shown or persuaded of the value of museums. There is also a danger that this one on one conversion may be limited if the teacher moves on. 7.1.5 Partnerships with STEM professionals have been developed Other STEM professionals, such as scientists, engineers, university students and staff and bodies such as universities, aquariums, STEMNET have been involved in the project. Museums, other professionals, and schools work extremely well in partnership, as they bring artefacts, real world STEM knowledge and teaching and curriculum expertise together to produce higher quality opportunities for pupils. More and better science learning opportunities have been developed for young people. Over 3,800 young people and 500 family members have participated in programmes developed through Science in Your World funded opportunities, over 450 teaching staff, and over 350 museum and archive staff have participated in CPD programmes, learning links placements or other Science in Your World funded opportunities
7.1.6
55
7.2
Recommendations The recommendations emerging from this evaluation are: Regional networks should be supported to continue where possible, and where there is demand: There is demand for the regional networks, and they are an appropriate mechanism for delivering support to the sector. The outcomes and impacts outlined in this report also support the governments STEM skills commitments, and support schools to deliver on the Department for Educations plans for science and maths as core to the curriculum, delivered through more autonomous schools. All co-ordinators and national partners are exploring alternative methods of funding for the region, including sharing administrative tasks between larger regional and national museums, working with local authority Museum Development Officers, and merging with other networks. In some regions, sub-regional events and knowledge sharing may be more appropriate: For example, the North West. It may be more appropriate to have smaller, sub-regional events and sub-networks. This will enable museums to easily travel to events and training. There are resource implications with this approach, along with less ability to deliver shared outcomes. One solution may be to target events at sub-regions, but keep a wider regional network as an umbrella. However, there is value in wider knowledge sharing regionally, and nationally. A national online resource bank would be useful: Each regional network has produced or shared resources through the internet. A national online resource bank would maximise their accessibility, and ensure that they are available beyond the funding period. A national means of sharing information and learning should be developed: A national means to share information and learning amongst peers in the museum sector would give the existing regional and local work a focus. This could take the form of an annual conference, newsletter or forum. Given the DfEs focus on innovative delivery in more autonomous schools and the governments commitment to STEM skills, national information sources, such as this recommendation and the national online resource bank, may be in line with departmental priorities. Cross-sector links with schools, universities or science professionals should be encouraged: One of the clear outcomes of the current funding is the bringing together of professionals from related fields. This has considerable benefits in terms of improving practice and outcomes for young people. There is a danger that without this small seed money, these links will not continue to grow. Future funding is recommended to continue this work. This also supports
56
the Governments commitment to STEM skills policy through supporting all professionals with an interest in this area to work together for more and better outcomes. Similarly, there is scope for further collaboration with national organisations who also aim to increase science engagement, including but not limited to the British Science Association, National STEM centre, National Science Learning Centre, Association of Science and Discovery Centres Further research is needed to track young peoples outcomes for programmes such as this: Further research is needed to gain an understanding of the long term impact of programmes such as this on young people. University partnerships could be used in future to support further research into the impacts of the programme. Increased advocacy for science education in museums to build upon the good individual relationships: There is a need to build upon the good work done by the programme in developing one on one relationships with the education sector through a widespread awareness raising campaign. This would enable the benefits of this approach to learning to go beyond Museum Champions into the mainstream. Relationships between national and regional museums should be continued and strengthened: There is a widespread recognition among partners and stakeholders that national museums have an important role to play in science education in museums. This should continue, and be strengthened where possible. Partnership structure for further collaborative programmes, for developing external funding bids: The network has laid the groundwork for museums to collaborate on further projects or funding bids. For example, the regional project led by Museum of Science and Industry (MOSI) is a good example of a project that has a broad enough impact that it could attract the support that one museum could not alone. Resources and learning to be distributed beyond museums: Schools are currently reacting to recent changes in the Department for Education, and are integrating new guidelines for learning as part of the National Curriculum Review. This programme can and should support schools by highlighting innovative teaching and learning outside the classroom experiences. It can also be used to influence how schools teach science by highlighting the successes of the programme. The resources created by the programme should be accessible to a wider audience, including teachers, perhaps through an existing mechanism such as the Times Educational Supplement website that can already be reached.
57
Regional and local partnerships should be used to support schools develop new curricula and programming: Government policy for more control to be devolved to schools and academies suggests that there is an opportunity for museums to deliver high quality targeted learning opportunities to schools. This programme has left a body of experience, case studies and resources that should be used to support schools to develop new curricula to support high quality out of school learning.
58
Appendix I: Logic Model

Rationale
Museums not fulfilling their potential to make a contribution to science education. There was a need: a. for museums to put school science in a real world context, enriching the curriculum in and out of school hours. This followed the Out of School Learning and Extended Schools agenda. b. to extend offers by outstanding science education museums such as NHM and NSM across the regions by developing regional partnerships. There was also a lack of existing engagement between museums to share learning etc. Research showed: museums lacked confidence, training and investment a lack of CPD for teachers.
Objectives
To deliver new science learning opportunities in English museums. To build skills and capacity in museum educators and curators to develop and deliver science learning opportunities. To share knowledge about science education and success between museums. To build demand in schools for science learning opportunities in museums To demonstrate that museums and out of classroom learning have a major role in delivering the STEM agenda To maximise the advocacy benefits for the partners and the museums sector from delivering science learning opportunities
Inputs
Financial resources, including eFunding Staffing and volunteering resources Knowledge and skills of national partners Knowledge and skills of museum partners Knowledge and skills of MLA staff
Activities
Developing regional museum networks to share knowledge: a. Holding meetings b. Hosting online forums c. Newsletters /websites developed Building capacity through CPD for museum staff and volunteers including: a. Workshops b. Resources (online and hard copy) c. talks and lectures Resourcing the development and delivery of science learning opportunities in regional museums: project funding Themed developments, i.e. physical sciences, sustainability Resources, both online and on site Outreach to schools, universities and other museums
Outputs
Number of: a. science learning opportunities b. CPD experiences for museum staff c. CDP experiences for teachers d. regional networks developed e. museums engaged with the networks f. teachers engaged with the programme g. schools engaged with the programme h. meetings between national partners and network leads Level of awareness of science learning in museums
Outcomes
For museum staff, increased: Confidence delivering science learning Ability developing programmes and resources to support science learning Partnership working For Museums, increased: Strategic added value networking and engagement with schools and other museums Engagement with national partners For teachers, increased: awareness of use of museums to support science learning understanding of how to use museums and confidence in doing so For schools, increased: awareness of museums as a resource to support science learning use of museums to increase science learning partnership and familiarity with museums For young people: Generic learning outcomes Increased interest and understanding of science
Impact
Increased use of museums as a resource for science education by schools and teachers. Widespread recognition of museums value to the STEM agenda Improved educational attainment in STEM through better teaching Young people inspired by museums to follow a career in science
59
Appendix II: Online survey

1.1 Overall The online survey consisted of 31 questions hosted on SurveyMonkey aiming to discover the impact the regional networks have on their members. The survey was distributed to members via the online networks and through email by the regional coordinators. Respondents There were 43 responses to the survey. Most were museum staff (41), and two were volunteers in museums. The majority of responses were from SLIME, which is not surprising as this is the largest network. There were seven who were not members of any network. Of the respondents, 65 per cent (27 respondents) were aware that the network was funded by Science in Your World programme, and 35 per cent (15 respondents) were not.
Network SMELT STEAM MOSI SMILE SLIME Percentage Number of responses
1.2
14.3% 0.0% 2.4% 14.3% 52.4% 16.7%
6 0 1 6 22 7
YES-NET
1.3
Events 34 (85 per cent) respondents had attended events arranged by the network.
Number of events attended 1 2 3 4
Number of responses 14
12 1 3 2
5+
When asked about the gains from events, networking was mentioned by 22 out of the 29 respondents. Other gains included new ideas, new skills, confidence, and information. I gained new knowledge of how to apply the museums collections to secondary science. I gained knowledge in the concepts needed to teach secondary science. I also gained confidence in using the equipment needed to teach secondary science safely. This meant that I now feel more confident in teaching secondary school children within the museum setting. The project also gave me chance to network
60
with people from other museums such as Macclesfield silk museum and the Manchester museum and find out about their projects. Survey respondent. 1.4 Online 23 out of 37 respondents had accessed the network online. Of those: 100 per cent had received information online from their networks, gaining funding information, new ideas and resources, and general news. Only 36 per cent had sent information out to other member using the network. They have asked for advice, guidance, information and help.
It can be isolating being the only member of education staff and having to be an 'expert' in so many areas. Being able to tap into others science expertise and experience has been hugely beneficial. Survey respondent. 1.5 Resources 18 respondents had used resources produced by the network, and 16 had not. Case studies and lesson plans were also used. The impacts of these resources were to increase ability of the museums to run these science sessions, to increase confidence and knowledge and saved time. They have improved the sessions we offer and when the museum was first starting to run education session the mini beasts was a brilliant way to contact schools. This gave me confidence in teaching the lesson dying to change the world and enhanced my knowledge of the chemical processes involved in dying. Eg that ethanol is used to denature the cells and release the dye. It gave us more ideas, gave us more training on the subject and gave us extra resources that we didn't have. I was able to run free sessions with local school for three weeks, working in the garden of another museum in local area, which greatly increased the number of school visits to the site in a traditionally quiet term. Many schools making visits to the museum who would not otherwise have come. Saved me re-inventing the wheel by giving me activities I could then adapt for my particular site.
61
Source: responses to impact question
At least three respondents cited the resources as increasing the number of school visits to the museum. Only five respondents said that they created resources to share with other members of the network. Only one detailed what these were, and the resources were around sustainability for GreenSlime. 1.6 CPD activities 14 respondents had taken part in CPD activity, and 21 had not, with nine not answering the question. These were workshops, training, and events. The responses made it difficult for us to disentangle the differences between these activities. The impact was increased confidence to use resources, a greater understanding of science activities, new skills, networking opportunity, new ideas, and increased ability to design and market activities to the education sector. 1.7 Projects 11 respondents had delivered a project funded by Science in Your World. Impacts that have been identified from these projects include: A greater awareness of sustainability issues within museums Greater involvement of young people in projects with museums that would not otherwise have occurred Developed Science programmes that involve laboratory practice for KS3 students which we have not had the capacity to do before. Students also participated in an hour long science lab to raise their aspirations and give them an opportunity to experience real life science as it relates to their locality. It also enables teachers to embed cross curricular work into their teaching. New sessions were developed that were not history based. A new relationship between a museum and a local secondary school. Large family days were run. This project gave staff confidence to develop maths activities. Growing awareness of how museums can deliver science. It has a broadened range of educational activities and so developing the audience, and it has given the museum constructive links with other museums and with schools. Impacts across eight museums so far and hopefully more when we run the sharing event
62
The clubs have been well attended so far, several of the visitors have been children who are home schooled. The project has attracted more visitors to site who are specifically interested in Science. Strengthening of a museums relationship with Malton Primary School; an increased interest in the science activities possible (from both museum staff and volunteers); and a willingness from a teacher not too sure about science to get involved.
18 respondents were inspired by the networks to develop projects not funded by Science in Your World. This included: Delivering workshops developed by other museums and shared through the network Making new collaborations with other museums to develop new workshops and programmes Gaining funding, such as a Public Engagement Grant from the Institute of Physics to run an Astronomy Evening for Brownies in conjunction with the Mary Rose Museum. One respondent revamped their whole programme of engagement, at Booth Museum of Natural History Impacts from these programmes are included in the quotations below:
New users - younger participants attending Young Nature Hunters this summer - very popular sessions Engaging visitors in different collections (eg optical toys) through science week activities Hopefully increased science visits A lot of first time visitors came in We now have a brilliant session to offer students which fuses art, design, science and history which is totally unique in the North west I have been inspired to teach key stage 3 now at the museum, when before I wouldn't have had the confidence to do so. Still under development - aim to increase engagement with this age group The project helped us develop the following:
63
Skills for working with teenagers Ideas for developing science activities for our collection. Working in partnership All activities that proved useful to other projects we have taken on since Families to archives; connection with children's library; potential to develop ideas further Brought into the museum five schools which had not previously visited; very positive feedback. encouraged teachers to see that museums are not just for history Project will be completed in spring 2011. Attracted additional families to visit our museums and local newspaper publicity Made us do activities we have never done before and use subjects we have not done before. Engaging with a new audience. Overwhelming interest in the event, leading us to organise two evenings instead of one. Working in partnership with a different museum.
64
Appendix III: Activity observation

1.1 1.1.1 Activity observation: Dyeing to Change the World, 10 December 2010, MOSI Description The event was a workshop session of the Dyeing to Change the World project at MOSI. This was delivered to young people and a museum partner in the project also attended. We spoke to the teacher from one of the schools and the partner museum, observed the session and consulted with fifteen young people at the end of the event, although this was heavily constrained by time. Observations Pupils seemed to generally enjoy the session - a few commented that it was 'okay' and some said that they enjoyed the session more than Science lessons at school as it was more practical and 'we get to do more stuff.' Some of the pupils said they were more interested in Science as a result of the session. When asked about the biggest thing they had learnt, quite a few young people said they had learned things about history - eg about Manchester's history and links with the cotton industry. However, a few also said they had learnt things about Science, e.g. that dye is natural and how many different things you can extract from natural dye and how to do this etc. Best bits for the young people: practical activities, eg crushing the membrane/using the centrifuge; and learning about the history of Manchester and its links with the cotton industry. Worst bits for the young people: the smell of the dye; too much talking in the session; and the one did not like the practical parts of the session. The teacher commented that: o Session on 10 December was their first Dyeing to Change the World session. Session was very attractive as it included a free coach trip and was fully subsidised - anything like this (ie where we can take pupils out for free) is attractive 'at a school like ours.' Also good that MOSI submitted a full risk assessment as this saves the school time. He was hoping to achieve three things: To raise aspirations of pupils - pupils that were selected are the 'academic high flyers.' He wants to encourage them to study Science further at GCSE and A-Level. Taking pupils to a different environment and doing things that they couldn't do in
1.1.2
65
school (both due to funding and health and safety) helps to do this Some educational impact for the pupils, although this was thought to be less important than raising aspirations To give MOSI a good impression of the school. o Pupils gave very positive feedback and enjoyed the day they were keen to go home and show their parents/tell their parents about some of the stuff they had done Relationship with MOSI was improved and the pupils behaved well Increased use of museums as a resource for Science education by schools and teachers - programmes such as Science in Your World have potential to do this. Own head of department was very impressed and school would do this again Young people inspired by museums to follow a career in Science - yes, there is a potential to achieve this as placing pupils in a different context and seeing a 'real scientist' in practice promotes Science as a tangible idea to young people
o o
School would definitely use MOSI/ museums in this way again but constrained by funding and curriculum pressures
1.2 1.2.1
Activity observation: SEARCH, Gosport, 25 January 2011 Description Hampshire County Council received funding through the SLIME network and Science in Your World to develop a new approach to teaching habitats for Key Stage 2 pupils. In particular, the project was developed as a result of several requests by local schools for greater use of creative methods in teaching biology. The project involves an introductory session to habitat concepts and how this applies to local habitats in particular. This involves hands-on learning at SEARCH including teaching food webs interactively, use of video microscopes and other activities to enthuse and engage pupils about biology. This introductory session is followed up by a creative session in schools that aim to cement some of the concepts taught at the introductory session. This second half of the course involves the input of a creative consultant with a specialism in creative ecology. The legacy of the Exploring Biodiversity project will be an online downloadable resource available for any teacher in Hampshire wishing to deliver creative lessons on habitats. In this sense the impact of the project will persist for some time.
66
1.2.2
Observations The school already had a strong relationship with SEARCH going back a number of years. In this sense, the specific benefit here was not in terms of building new relationships but in innovative ways of delivering of lessons as part of the curriculum. The schools involvement was brought about through the schools science coordinator. The project was devised as a result directly of requests from local schools to combine hands on museum based activity with creative methods, to better engage pupils in biology and sciences. Project officers believed that this project would build stronger links with other schools in the area, once they were engaged and particularly through use of creative methods. The effectiveness of the project was difficult to record at this stage, as the project was only in its first day of delivering the introductory sessions. Project partners and teachers stated that the benefits should include: o o greater enthusiasm for sciences and biology a useful way of communicating often difficult concepts with use of hands on and creative methods, as well as providing a lasting legacy for future year groups.
There was clear enthusiasm from the year four group we observed. Pupils attention and interest was held throughout the two hour session. For example: o o o I like the foxes, owls and beetles Its more fun than learning at school You get to see real things, not pictures.
1.3 1.3.1
Activity observation: SMILE CPD day, Bolton, 7 December 2010 Description The day was held at Bolton Museum and attracted 18 attendees from museums throughout the North West. The day was split into workshops that included: Investigating the Nature of Science Exploring the science Learning at Bolton Museum Inspiring Secondary Science in North West Museums (led by MOSI) Interactive Talk Science Workshop (led by the Science Museum).
67
1.3.2
Observations The sessions were engaging, and all participated in hands on activities. Comments such as thats a really good technique for the digging back in time concept, and really imaginative use of experiments were overheard There was a mix of knowledge in the room, including some museum staff who had never done any science based learning Participants came to the event because: o o o o Its a rare chance to have a science based CPD day Its a networking opportunity I have changed job and now I really need to understand science Its a struggle to get secondary schools in and I came for help
I asked participants to agree or disagree with a series of questions, there was unanimous strong agreement that: o o o o o I have made new contacts today I will contact people from today in the future I feel more confidence delivering science workshops I will look on a website I found out about today I learnt something new today
There was general agreement, although not strong agreement that: o o I will develop a new workshop as a result of today I will improve an existing workshop as a result of today
1.4 1.4.1
Activity observation: YES-NET Conference, 7 February 2011, Leeds Museum Description The event held at Leeds Museum attracted over 60 attendees, from schools (mainly primary), museums and universities. It consisted of a series of talks by museums and educators that present ways of delivering science learning in museums, projects already happening, and those planned for the future. In the afternoon there were hands on workshops by museums that demonstrated some techniques used. Observations There were a high number of teachers present, particularly primary school teachers. When asked why they decided to come, they said:
1.5
68
We are redesigning our curriculum at the moment, and we are focusing on projects or themes that we can explore. They need to be crosscurricular, but we are struggling to include science I want practical ideas to teach science. (this was echoed by a number of teachers spoken to, and there was agreement in a small group of six teachers)
The practical sessions were well thought of, with participants comments as follows: o The pipe cleaners are a good way to teach anatomy I will adapt that and use it next week when we look at the human body I wish they would come out to our school (in a workshop about using skulls to teach evolution)
Teachers wanted to include more museum visits, but are constrained by transport costs and the difficulty of taking students out of school: o o Its hard to justify for just one subject I have some double lessons it would be great to be able to take out students for this short time and easier than the whole day Transport costs are really difficult for us, we are in quite a deprived area and students cannot afford to contribute
Although there was not much time at the end of the day to ask how people found it, one participant (a teacher) had been part of a learning links award and was keen to tell us how much he valued that opportunity o
o
I have become a better teacher as a result of this Without this funding I would not have found out about the museum, and would not have made the partnership we have We are going to continue working together to develop workshops and visits.
1.6 1.6.1
Activity observation: SMELT showcase 4 February 2011 Description The event comprised a number of presentations from recipients of Science in Your World funding, feedback on the SMELT regional online resource kit, and discussion about the future of the network. Museums presenting included: Erasmus Darwin Museum
69
1.6.2
The Pen Room George Marshal Medical Museum
Observations Teachers that were part of the Learning Links programme were present. This added to the discussions, particularly when talking about the future of the network. Museums value the funding and the network: o I loved working with the teacher on the maths board game. It gave me a new perspective on my collection. The Pen Room was struggling to attract schools, but the resource and relationships developed will help us with this. This is vital for our sustainability. We are a volunteer run museum, and we need more volunteers Two of the teacher trainees want to stay involved with the museum. We needed the expertise of [the Special Educational Needs (SEN) teacher] to develop appropriate workshops. We are a small museum and the smaller class sizes make SEN schools a good market for us.
Teachers became more interested in museums as a result of these projects: o I had never heard of the Pen Room before. Now I am planning to take more children there. The enthusiasm and depth of knowledge of the [museum] staff surprised me. I will continue to work with Erasmus Darwin House, as I can see it would be good for the children I teach. There was a desire to continue the network, but with no funding it was uncertain how this could happen.
70
Appendix IV: List of documents reviewed

MOSI MLA Grant Proposal 2010 MOSI MLA Interim Report 2010 MOSI Project Updates 1, 2, 3 and 4 MOSI New Supplier Form MOSI Science in Your World Initial participation form MOSI - Summary Dyeing to change the world SMILE MLA Grant proposal 2010 SMILE MLA Interim Report 2010 SMELT Datacapture Science in Your World 31March 2010 SMELT Thinktank end of project report Science in Your World 2010 Datacapture and End of project reports 2009-2010 for small grants SMELT Interim Report October 2010 SLIME evaluation 2009 SLIME case studies SLIME Interim Report 2010 SLIME Datacapture and Final Reporting 2009 STEAM Interim Report 2010 STEAM Datacapture and Final Reporting 2009
71
Appendix V: Questionnaires and Proformas

1.1 Stakeholder questionnaire NOTE to interviewers: This is a semi-structured interview, so please try to address all questions, and use the italicised prompts if necessary. Leave enough time to address all questions. Some questions are not relevant to all projects, so please use your discretion. Please always ask for specific examples or evidence wherever possible.
Introduction Consulting Inplace was appointed in July 2010 to deliver an evaluation of the Science in Your World programme. The aim is to complete a robust and independent evaluation that measures the impact of MLA support of the regional networks, and the impact of the regional networks on the programmes outcomes. By 2011, we will have found out whether the MLA supported networks are achieving their aims, and how this was achieved in each region. The purpose of this interview is to help us to understand the effect of the programme on its beneficiaries, stakeholders and partners. It will be used as evidence for our final report. You have been asked to participate as stakeholders involved in delivery, or as involved in the policy context in which this programme has been operating. We expect that this interview will last around 45 minutes. This interview will be anonymised and the findings are confidential. By completing this interview they are giving their consent to use this information for the purpose of this research. Explain who you (the interviewer) are and what your role is in the evaluation. Background 1. Can you tell me a little about you and your organisation? What are its aims and objectives? What is its relationship to science education or museums What is your role?
Science in Your World 2. How did you hear about the Science in Your World programme? Has your organisation been involved in it, and if so, how? 3. Do you think that the programme has met its objectives?
72
The programme has these objectives: a. To deliver new science learning opportunities in English museums. b. To build skills and capacity in museum educators and curators to develop and deliver science learning opportunities. c. To share knowledge about science education and success between museums.
d. To build demand in schools for science learning opportunities in museums e. To demonstrate that museums and out of classroom learning have a major role in delivering the STEM agenda f. To maximise the advocacy benefits for the partners and the museums sector from delivering science learning opportunities
4. Are you aware of any further projects, funding or programmes been developed as a result of the Science in your World programme beyond those funded by Science in Your World?
Impact 5. What do you think has changed as a result of the science in your world programme? Increased use of museums as a resource for science education by schools and teachers. Widespread recognition of museums value to the STEM agenda Improved educational attainment in STEM through better teaching Young people inspired by museums to follow a career in science
Wider context 6. What lessons should be learnt from the programme, and how should they be disseminated? 7. How does the science in your world programme fit within the current policy context? Young people and education STEM and innovation Arts and culture
Thanks and close.
73
1.2 1.3
Guide for Science in Your World evaluation case studies Introduction The case studies for this evaluation will allow us to take an in depth look at some of the activities funded by the programme. Each case study will comprise: A semi-structured interview with the project manager Semi-structured interviews with teachers / schools (these can be held as small focus groups if appropriate) Focus group/ interviews with young people
This guide includes pro-formas for the interviews with the Project Manager and with teachers/ schools. We also aim to delver focus groups with young people when appropriate. Given the variation of ages, and the type of activity we will be able to undertake within the constraints of the activity being observed, we suggest that we devise these on a case by case basis. Key questions to be asked will include: What have you been doing in the museum? What was the best bit? What was the worst bit? Have you become more interested in science as a result of your visit? Did you find this visit helped you to understand science? Would you like to be a scientist as your career? Was this decision influenced by this visit?
1.3.1
Project manager interview proforma NOTE to interviewers: This is a semi-structured interview, so please try to address all questions, and use the italicised prompts if necessary. Leave enough time to address all questions. Some questions are not relevant to all projects, so please use your discretion. Please always ask for specific examples or evidence wherever possible.
Introduction Consulting Inplace was appointed in July 2010 to deliver an evaluation of the Science in your world programme. The aim is to complete a robust and independent evaluation that measures the impact of MLA support of the regional networks, and the impact of the regional networks on the programmes outcomes. By 2011, we will have found out whether the MLA supported networks are
74
achieving their aims, and how this was achieved in each region. In your region the network supported is: [insert name of network] This interview is part of a case study of your project. We want to find out how the programme has affected those involved in it. The cases study will be used as evidence for our final report. You have been asked to participate as the project manager of this case study. We expect that this interview will last around 45 minutes. This interview will be anonymised, and the findings are confidential. By completing this interview they are giving their consent to use this information for the purpose of this research. Explain who you (the interviewer) are and what your role is in the evaluation. Background 1. Can you tell me a little about your organisation? What are its aims and objectives? Who funds it? How many staff and volunteers does it have?
2. What is its relationship with Science in Your World? How long it has been involved? How many projects has it delivered? What is your role within the organisation? If the interviewer does not know it is Science in Your World funded, mention their network
Science in Your World Project 3. Tell me about your Science in Your World project. What needs were you trying to address? What were you intending to achieve?
4. What are the main activities of the project? Who else has been involved? Who are the beneficiaries of the project?
75
What have you delivered so far?
Impact 5. What impact do you think your project has had so far? (By impact we mean what change your project has had on those it involves?) What is the impact on museum staff, teachers, schools and young people? Has there been any increased partnership and engagement with museums or schools as a result of this programme? How can you evidence/ measure that impact? Have any further projects, funding or programmes been developed as a result of the Science in your World programme beyond those funded by Science in Your World?
6. What impacts are you expecting to occur as a result of the programme overall? Increased use of museums as a resource for science education by schools and teachers. Widespread recognition of museums value to the STEM agenda Improved educational attainment in STEM through better teaching Young people inspired by museums to follow a career in science
Lessons learnt 7. If you could do the project again, what would you do differently, and what would you like to change? 8. To what extent have you incorporated the lessons learnt in the rest of the project? Thanks and close
1.3.2
Schools/ teacher interview proforma NOTE to interviewers: This is a semi-structured interview, so please try to address all questions, and use the italicised prompts if necessary. Leave enough time to address all questions. Some questions are not relevant to all projects, so please use your discretion. Please always ask for specific examples or evidence wherever possible.
Introduction
76
Consulting Inplace was appointed in July 2010 to deliver an evaluation of the Science in your world programme. We aim to find out the impact of the programme. This interview is part of a case study. You have been asked to participate as a beneficiary of the project. This interview will be anonymised, and the findings are confidential. By completing this interview they are giving their consent to use this information for the purpose of this research. Explain who you (the interviewer) are and what your role is in the evaluation. Background 1. Can you tell me a little about yourself and your school? What is your role? Is your school primary or secondary? State school, academy or private school? How many pupils do you have? Do you have any specialisms?
Science in Your World 2. What is your schools relationship with Science in Your World? How did you hear about the project? If the answer is no, ask about their relationship with the museum
3. What were your motivations to take part? What did you hope to achieve?
4. What are you doing with the programme? Impact 5. What do you hope to gain by taking part in the programme? What change will happen to you? What change will happen to your school? What change will happen to your pupils? Do you think that you or your schools relationship with the museums has got better or worse as a result of this programme?
77
Have any further projects, funding or programmes been developed as a result of the Science in your World programme beyond those funded by Science in Your World? What longer term impact/ changes you think will happen as a result of this programme? Increased use of museums as a resource for science education by schools and teachers. Widespread recognition of museums value to the STEM agenda Improved educational attainment in STEM through better teaching Young people inspired by museums to follow a career in science
6.
Future use of museums in Science Learning 7. How do you use museums to support science learning? How often? What are the benefits / challenges of using museums in this way? Has this changed? How much of this change is due to the science in your world programme? Is there a difference between how you use it and how your school as a whole uses it?
Thanks and close
78
Museums, Libraries & Archives Council Grosvenor House 14 Bennetts Hill Birmingham B2 5RS
T +44 (0)121 345 7300 F +44 (0)121 345 7303 info@mla.gov.uk www.mla.gov.uk
Leading strategically, we promote best practice in museums, libraries and archives, to inspire innovative, integrated and sustainable services for all.
MLA 2010-03-03 Registered Charity No: 1079666

Evaluation of Science in Your World

Enviado por

Dados do documento

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Evaluation of Science in Your World

Enviado por

Direitos autorais:

Formatos disponíveis

Evaluation of Science in Your World

A report by Consulting Inplace