Technical Competency Framework for Evaluation

Competency Definitions

Title Best Practice in Evaluation Approaches and Methods

Definition Understands and makes appropriate use of evaluation concepts and best practice in evaluation design, methods and approaches in line with departmental policy, and relevant international and professional technical standards. Gathers and analyses relevant evidence, data and information for evaluation purposes from a range of sources, identifying relevant material, assessing its quality, spotting gaps and presenting information in a structured logical and useful way. Raises awareness and use of evaluations through effective communication and dissemination, high quality advice, training and guidance to stakeholders.

Gathering and Using Evidence for Evaluation

Communicating and Sharing Evaluation Findings, Knowledge & Expertise

Acts professionally, achieves and models high standards of Upholds Evaluation integrity, independence and quality in dealing with internal Independence, Quality and external stakeholders, applying judgement and ensuring & Standards the highest evaluation standards are upheld. Leading, Managing and Delivering Evaluations Manages evaluation resources and relationships with stakeholders including external partnerships to deliver high quality evaluations and related objectives on time and to departmental standards.

Applicants should be able to demonstrate skills at Skilled level or above in all competences.

07/02/2013

Technical Competency Framework for Evaluation

Best Practice in Evaluation Approaches and Methods Understands and makes appropriate use of evaluation concepts and best practice in evaluation design, methods and approaches in line with departmental policy, and relevant international and professional technical standards. Expert
Provides expert and strategic guidance to others within the department and to key external stakeholders on all aspects of evaluation design, methods and approaches. Provides specialist advice on departmental evaluation policy and procedures to others within the organisation and helps to develop those procedures. Pushes forward the frontiers of best practice in work with partners, including through engagement with evaluation researchers. Demonstrates expert knowledge from diverse perspectives when evaluating programmes and, where necessary proposing and testing innovative approaches. Ensures key stages of evaluation best practice are followed by others. Demonstrates highly developed skills in developing evaluation recommendations and applying them to key areas of the departments work, relating them to programmes and policies in a practical and effective way.

Skilled
Able to develop and consult on high quality terms of reference to deliver complex evaluations incorporating all relevant aspects of evaluation design. Demonstrates skilled knowledge of methods and approaches for evaluating programmes and policies from diverse perspectives, drawing on established approaches. Demonstrates skilled knowledge in planning & conducting an evaluation, bringing practical experience to managing the various stages of the evaluation cycle, and critical success factors including effective stakeholder engagement. Demonstrates skilled knowledge in evaluation design including relevant and recognised approaches and models from the evaluation literature and current best practice in evaluation units in government in the UK and internationally. Is familiar with and makes appropriate use of standard evaluation guidance criteria and frameworks from departmental policy and guidance and elsewhere (eg OECD-DAC, UK, European and US evaluation association guidelines, World Bank and UNEG standards). Demonstrates skilled knowledge of quantitative methods used for evaluation such as standard data presentation techniques and spreadsheet modelling, survey design and sampling, statistical and econometric methods, experimental and quasi-experimental methods for impact evaluation, cost-benefit assessment. Understands the importance and relevance of mixed methods in evaluation and is able to bring a range of multi-disciplinary approaches to bear in evaluation design, depending on context. Demonstrates a good understanding of qualitative methods, including for example participatory methods, case histories, appreciative enquiry. Demonstrates ability to prioritise and focus evaluation questions (and negotiate these with partners) to produce relevant and useful evaluation. Recognises ethical dilemmas and seeks advice or support to resolve them.

07/02/2013

Page 1

Technical Competency Framework for Evaluation

Competent
Able to develop appropriate terms of reference for straightforward evaluations and review with support from skilled evaluation managers where required. Understands the knowledge base of evaluation terms, concepts, theories and assumptions. Participates and helps to drive forward joint evaluations of projects, programmes and topics, working with others with more experience and knowledge. Demonstrates a basic understanding of various methodologies in evaluation, such as mail and telephone surveys, record reviews, focus groups and case studies and the principles of theory-based evaluation. Demonstrates a basic level of understanding of evaluation, planning and design. Demonstrates an awareness of data collection techniques, eg document review from programme records, structured and semi-structured interviews, focus groups and workshops.

Foundation
Understands the purpose and role of evaluation and basic concepts of evaluation monitoring, audit and research and how they relate to each other. Participates in joint and internal evaluations of projects, programmes and topics, providing support and working with others with more experience and knowledge.

07/02/2013

Page 2

Technical Competency Framework for Evaluation

Gathering and Using Evidence for Evaluation Gathers and analyses relevant evidence, data and information for evaluation purposes from a range of sources, identifying relevant material, assessing its quality, spotting gaps and presenting information in a structured logical and useful way. Expert
Demonstrates expert knowledge in interpreting and analysing the policies and working practices of others e.g. World Bank, NGOs, International Agencies and Governments, and in working collaboratively to identify relevant evaluation topics. Demonstrates expert skills in communicating and describing the programmes for evaluation, quickly comprehends the objectives of new programmes, policies and initiatives to which they are exposed, the theoretical assumptions on which they are based and the context in which they operate. Demonstrates expert influencing and negotiating skills ensuring evaluation recommendations are properly utilised and implemented in a timely fashion and that evaluation findings are linked to future activities and reported to relevant stakeholders. Reviews evaluation policies, processes and procedures to ensure they support departmental objectives. Demonstrates expert skills in interpreting management information, using this data when reporting on progress and adherence of quality standards to relevant Government bodies.

Skilled
Reviews evaluation findings and ensures results are fed into future policy. Demonstrates the ability to assess the quality of evaluations, including their relevance and the use of theory and evidence. Applies skills to formulate appropriate evaluation questions, following analysis of the evidence base. When analysing information, knows and applies professional evaluation practice. Demonstrates skill in analysing data from diverse sources, identifying key trends and gaps in information. Uses analytical skills to synthesise data and present well founded results/recommendations to support higher level strategic decision making. Uses interpersonal skills to work with others to share trends and information, feeding this into the design of future projects and initiatives.

Competent
Demonstrates a practical knowledge of internal systems and documentation to provide evaluation data. Demonstrates good attention to detail and analytical skills by reviewing data quality for content and accuracy. Uses analytical skills to contribute to evaluation process. Works on preparing data for analysis, such as testing for data accuracy, with minimal guidance. Demonstrates objectivity, fairness and balance when evaluating programmes, policies and initiatives, striving to ensure that the information gathered is factual and complete.

Foundation
Demonstrates an understanding of policy and process in order to gather relevant and accurate information for evaluation purposes from internal systems. Uses attention to detail by checking data for errors in spelling, grammar and making sure that the layout of information is in accordance with departmental standards before submitting to print. Uses appropriate information sources when dealing with requests for information. Presents information in a clear and usable format.

07/02/2013

Page 3

Technical Competency Framework for Evaluation

Communicating and Sharing Evaluation Findings, Knowledge and Expertise Raises awareness and use of evaluations through effective communication and dissemination, high quality advice, training and guidance to stakeholders. Expert
Demonstrates expert knowledge when working to promote the benefits and importance of evaluation within the department and to external partners. Maintains expert knowledge of evaluation process and procedures and uses this to provide strategic evaluation direction to the department, ensuring latest practices and techniques are introduced. Uses consultative skills when liaising with Government ministers, giving advice and guidance on policy and evaluation findings. Demonstrates gravitas when representing the evaluation department at Government and other meetings with external partners, sharing expertise of evaluation policy, process and procedure. Uses lessons learnt information to support and shape future strategy, communicating key trends and recommendations within the department and to external partners. Presents difficult evaluation and evidence results with authority and confidence. Demonstrates ability to present balanced interpretations of evaluation findings in a neutral fashion.

Skilled
Demonstrates expert knowledge regarding the evaluation processes used within the department, by other relevant stakeholders and other best practice procedures. Demonstrates effective communication skills when conveying information on the evaluation process and procedure, ensuring the message is clear, concise and suitable for the recipients level of knowledge and skill. Demonstrates skilled knowledge of evaluation process and procedure in order to provide advice to other parts of the department and external partnerships. Demonstrates sound judgement and empathy when advising other parts of the department and partners on evaluation policy and procedure, taking into account complex and sensitive issues. Demonstrates commitment to ensuring own knowledge is up to date and pro-actively seeking opportunities to share evaluation best practice throughout the department. Works jointly with partners to achieve shared evaluation outcomes. Confidently defends evidence based results, stands their ground and has conviction when delivering the message.

Competent
Demonstrates an understanding for the need to convey knowledge and expertise to others within the department and to external partners. Demonstrates a good understanding of departmental evaluation practice and approaches and provides basic level advice to other parts of the department and external partnerships. Uses listening and questioning skills to fully gather information before giving advice. Demonstrates an ability to tailor advice ensuring it is pragmatic, realistic and does not exceed parameters of own knowledge. Proactively seeks new ways and media to convey new information and knowledge, using e-mail, intranet, video conferencing and other available media. Demonstrates enthusiasm for supporting others within the evaluation department to manage the evaluation process, seeking opportunities to further own knowledge. Supports others in usage of evaluation guidance protocols. Demonstrates a good level of understanding of the evaluation mandate and how this applies to all aspects of information and evidence base.

07/02/2013

Page 4

Technical Competency Framework for Evaluation

Foundation
Uses basic knowledge of departmental evaluation approaches and principles to provide day-to-day level advice, and appropriately refers queries inline with limitations of own knowledge. Provides basic guidance on retrieval and storing of data on internal systems to other parts of the department, to assist development of the evidence base for evaluations. Demonstrates commitment to provide an excellent service to others by proactively looking for opportunities to share information and knowledge, support others in the evaluation process and deal with queries in a professional and timely manner. Uses IT skills to ensure internal systems (such as intranet, financial and electronic document management systems) are up to date and all evaluation information is stored correctly.

07/02/2013

Page 5

Technical Competency Framework for Evaluation

Upholds Evaluation Independence, Quality and Standards Acts professionally, achieves and models high standards of integrity, independence and quality in dealing with internal and external stakeholders, applying judgement and ensuring the highest evaluation standards are upheld. Expert
Uses highly developed influencing skills to encourage others within the department and external partners to build in evaluation processes at the inception of a project or policy, and to continually monitor performance throughout the project cycle. Demonstrates innovative, forward thinking skills in designing and updating evaluation process in-line with international standards and current best practice. Introduces and takes ownership of strategic evaluation policy to ensure compliance and continual updating of quality standards within the department. Champions and advances professional evaluation standards, including ethical standards and demonstrates integrity. Defends the evaluation mandate set out in the departments Evaluation Policy when dealing with senior level policy makers, practitioners and other stakeholders. Manages a wide range of stakeholders to engage them and to manage expectations throughout an evaluation process.

Skilled
Demonstrates skills in upholding evaluation policy, procedure and standards, monitoring own work and the work of others to ensure high standards and professionalism. Demonstrates influencing and negotiating skills to ensure other parts of the department and partners promote and role model evaluation process and procedure throughout the duration of projects and programmes. Knows and applies professional evaluation standards, including ethical and integrity standards. Leads by example, ensuring the quality of evidence, data and information used in evaluation is appropriate, accurate and up to date. Demonstrates attention to detail by ensuring information published for distribution reflects and upholds quality standards and accuracy. Demonstrates commitment to upholding of standards by continually looking for new ways to improve quality, efficiency and accuracy within the evaluation department.

Competent
Demonstrates skills in ensuring own information and advice communicated internally & externally upholds quality standards. Uses attention to detail in checking own work, ensuring all information is quality checked before going to press. Uses influencing skills in order to encourage other parts of the department or external parties and consultants to follow departmental quality standards and relevant processes. Proactively checks data and information from other departments to ensure parity with internal systems. Uses communication skills to provide constructive feedback on the quality and standard of information, challenging practices that need to be improved and providing clear, concise guidance on quality standards.

07/02/2013

Page 6

Technical Competency Framework for Evaluation

Foundation
Uses attention to detail by checking data for errors in spelling, grammar and making sure that the layout of the information is in accordance with departmental standards before submitting to print. Demonstrates a basic understanding of quality and compliance standards, applying these to own roles and responsibilities. Demonstrates team working skills by quality checking the work of others. Uses spelling and grammar checking on IT systems to ensure simple errors are eradicated. Uses internal guidance on use of templates and e-mail in correspondence and publishing of documentation.

07/02/2013

Page 7

Technical Competency Framework for Evaluation

Leading, Managing and Delivering Evaluations Manages evaluation resources and relationships with stakeholders including external partnerships to deliver high quality evaluations on time and to departmental standards. Expert
Demonstrates project management skills to a Senior and strategic level, ensuring appropriate resource use to achieve and monitor evaluation policy. Works effectively to establish annual and long term evaluation programme strategies, set objectives and standards, and negotiate key milestones and budget. Demonstrates expert leadership and management skills by ensuring that all members of staff are supported and feel confident to use their level of authority effectively to achieve results. Demonstrates high standards of ownership, drive and commitment to achieve results and maintains expert levels of knowledge within evaluation field. Demonstrates highly developed negotiation, influencing, conflict resolution and chairing meeting skills at Senior Management, Board, departmental and international fora level. Highly skilled in assimilating and communicating the needs of others, building strong cross functional alliances. Identifies and nurtures key relationships, pro-actively promoting the evaluation department, internally and externally. Displays expert skills in managing and negotiating expectations of multiple constituencies (e.g. World Bank, NGOs, International Governments).

Skilled
Skilled application of the project management cycle to all aspects of evaluation work. Establishes and assesses risk, builds and applies contingency plans in order to mitigate risk in evaluation study cycle. Demonstrates people management, relationship management and motivational techniques in order to maintain momentum and complete deadlines within timescales set by the evaluation departments work plan. Establishes relevant stakeholder groups, and consults, informs and manages a range of expectations. Demonstrates commitment and energy to overcome adversity and challenges decisions or actions of individuals or partners when appropriate. Demonstrates strong influencing, meeting facilitation and negotiating skills. Works collaboratively with third parties (eg NGOs and external consultants) ensuring all parties are fully aware of their responsibilities and key goals/deadlines are met. Skilled in adapting personal style taking into account the audience, cultural styles, values and message being conveyed or received. Uses evaluation process knowledge and financial and documentary information when planning and implementing evaluations.

Competent and Foundation

The competent and Foundation level of this competency are the same as those for generalist core competences.

07/02/2013

Page 8

Technical Competency Framework for Evaluation

Key to Levels: Expert Has prolonged and intense experience through practice and education of a particular competency. Is a mentor and role model for the competency i.e., is widely recognised across the organisation as a reliable source of knowledge, technique and or skill and whose judgement and application of the competency is accorded authority and status by their peers. Uses the competency consistently to maximum effect across a range of functions and situations at all levels in the organisation. Has acquired a great deal of content knowledge that when utilised represents a deep understanding of the competency. Is able to retrieve and use important aspects of their competency knowledge/skills/technique with little attentional effort. Skilled Has a thorough knowledge of a competency gained through education, training and practice. Applies the competency consistently across a range of functions at most levels in the organisation to facilitate desired outcomes. Exercises considerable independent judgement on the correct use and appropriate application of the competency in varied or diverse situations. Will share knowledge of competency with others and may be viewed as the skills transfer resource for the competency within the department. Competent Has good practical knowledge of the competency gained through proper or sufficient education, training and regular use. Is capable and/or efficient in using the competency consistently and demonstrates appropriate and timely use of the competency to resolve or address various business situations within their area of responsibility. May require advice & guidance on using advanced aspects of the competency to deal with more intricate or involved situations. Foundation Has limited experience and practical understanding of the competency gained through basic introductory training or education. Has adequate knowledge of the competency to allow limited practice and application but may need to seek advice and guidance.

Note The term education in this context, does not relate exclusively to academic education or qualifications but also to on the job learning and knowledge of role/skills.

07/02/2013

Page 9