13th European Evaluation Society Biennial Conference

Pre-Conference Professional Development Workshops - October 1 & 2, 2018

Interested in attending pre-conference development workshops? Please consult the programme and register now (sessions have a limited number of participants)!

Already registered? You can manage your registration, and also register for the workshops on the conference portal.

The EES Pre-Conference workshops in Thessaloniki will take place over the two days preceding the conference, i.e. on October 1 & 2, 2018. These pre-conference sessions will again be a unique opportunity to gain knowledge and experience from stimulating and knowledgeable presenters.

24 one-day workshops are offered this year, on evaluation approaches, evaluation designs, evaluation methods, and other topics such as SDGs, gender equity, capacity building and resilience.

Pre-Conference Workshops - Day 1

Evaluation Approaches PW1 - Dealing with complexity in development evaluation - emerging issues
For: evaluators, funding agencies and program managers.
Level: some knowledge/experience in evaluation.

This workshop is designed to provide a practical, real-world approach for assessing outcomes and impacts of complex development interventions, and to trace some of the emerging issues. Participants are encouraged to bring their own evaluations of complex interventions to illustrate promising approaches and to seek suggestions from their colleagues. At the end of the workshop participants will: 1. Be familiar with the concepts of complexity and how they apply to development evaluation; 2. Understand why established evaluation methodologies are not adequate (when used on their own) to evaluate the outcomes of complex programs; 3. Be able to assess the level of complexity of any development program and to judge whether a complexity-responsive evaluation approach is required; 4. Be familiar with the different approaches to the evaluation of complex programs; 5. Be able to apply the 5 step "unpacking" methodology.

PW2 - The basics of using theory to improve evaluation practice
For: evaluators
Level: beginner

This workshop is designed to provide practicing evaluators with an opportunity to improve their understanding of how to use theory to improve evaluation practice. By the end of this session, participants will be able to: 1. Define and describe evaluation theory, social science theory, and program theory; 2. Describe how theory can be used to improve evaluation practice; 3. Describe how implicit and explicit social science theories can be used to guide evaluation decisions; 4. Describe the components and processes of several commonly used social science theories that have been used to develop and evaluate programs; 5. Describe how developing stakeholders’ theories of change can be used to improve evaluation practice.

Evaluation Design PW3 - Evaluating value for money in development programmes
For: evaluators, commissioners, donors, development practitioners
Level: some working knowledge of international development settings.

This workshop is designed to provide practical guidance, underpinned by a sound theoretical framework, for assessing VfM in international development programmes. The approach combines explicit evaluative reasoning - through the development and use of criteria and standards - with concepts from economic evaluation, to respond to donor requirements for accountability and good resource allocation, as well as to support reflection, learning, and adaptive management. Workshop participants will learn how to: 1. Frame an evaluative question about VfM; 2. Identify criteria of merit and worth (dimensions of VfM), and set context-appropriate standards (levels of performance) to provide a transparent basis for making sound judgements about performance and VfM; 3. Combine multiple sources of evidence of performance and VfM including quantitative and qualitative evidence; 4. Incorporate economic evaluation within a VfM framework (where feasible and appropriate), without limiting the analysis to economic methods alone; 5. Interpret the evidence, using criteria and standards, to make sound evaluative judgements; 6. Present a clear and robust performance story, guided by the criteria and standards.

PW4 - Mixing methods for evaluation quality and results
Catherine ELKINS
For: any participant
Level: some knowledge/experience in evaluation.

This workshop is designed to provide a hands-on guided experience to build participant skills in working collaboratively through key decisions and trade-offs, with a methodologically open mind, to improve evaluation design and execution for useful and productive results. At the conclusion of this workshop: 1. Participants will be able to analyze a potential evaluation assignment in order to identify evaluability gaps. Addressing evaluability gaps strengthens the evaluation design toward actionable findings. 2. Participants will be able to develop or improve evaluation questions. Well-framed and clear evaluation questions help ensure each evaluation element is tied to producing meaningful evidence. 3. Participants will be able to select and justify methods through defining clear links to data that will help answer questions with valid and reliable evidence. 4. Participants will be able to strategize for more effective implementation and delivery – with a clear and coherent framework in use – to improve activities and initiatives and/or enhance results.

PW5 - Finding out what works: An introduction to impact evaluation
Line Renate HANSSEN, Sanni N. BREINING
For: any participant
Level: beginner and intermediate.

The workshop is designed to provide participants with: 1. an introduction to the purpose of impact evaluation; 2. an understanding of the type of questions the method can answer; 3. learning about the different approaches to impact evaluation and about their respective advantages and drawbacks, 4. the possibility to discuss and explore which impact evaluation approaches are best suited to your own projects.

Evaluation Methods PW6 - An introduction to prediction modelling, using EvalC3
For: evaluators, donors, managers and implementers.
Level: some knowledge of simple quantitative and qualitative methods.

This workshop is designed to provide a practical introduction to prediction modelling using EvalC3 (an Excel application). Prediction/predictive modelling is the process of creating, testing and validating a model to best predict the probability of an outcome; it can be done with small and simple data sets (e.g. 30 cases with 10 binary attributes), and it can be done with readily available and easy to use software, no coding skills are needed. The workshop' participants will be provided with:1. Knowledge about how and when to use all the functions available within EvalC3; 2. Introductory knowledge of other predictive modelling software; 3. Plans to collect and analyse data in their own field of work, using EvalC3 or related software.

PW7 - Using Interviews to enhance evaluation practice
For: any evaluation stakeholder
Level: mixed levels of experience in interviewing

This workshop is designed to enable participants to: 1. Identify the value of interviews at different parts in the evaluation process, 2. Describe interview foundations; 3. Evaluate what makes a good interview, and 4. engage in good interviews. It is through the interactive and dynamic nature of this workshop that participants will grow in their knowledge and their ability to put knowledge to use in their own practice.

SDGs, Gender Equity PW8 - Theoretical propositions and practical guidance for gender & equity responsive evaluation practice in the SDGs Era
Florence ETTA, Grace OKONJI
For: commissioners, evaluators, managers, clients & right holders
Level: from intermediate

This workshop is designed to present these AGDEN (Africa Gender and Development Evaluators Network) tools, techniques and approaches for gender responsive and human rights sensitive evaluation. At the end of the workshop, partcipants will: 1. Be familiar with/understand some of the theoretical and analytical concepts central to gender responsive and rights sensitive (equity-focused) evaluation: complexity, embedded systems, gender & development, human rights, evaluation norms, standards, criteria, and competencies. 2. Appreciate and value social resilience by understanding the hazards and risks associated with social inequity and gender inequality 3. Have familiarity of the AGDEN Culturally Responsive Curriculum for Gender Transformative Evaluation Training 4. Have critiqued & suggested improvements for the AGDEN Culturally Responsive Curriculum for Gender Transformative Evaluation Training 5. Be armed with practical knowledge of criteria and competencies for commissioning, managing and conducting gender responsive and rights sensitive (equity-focused) evaluation in the SDG era.

Resilience PW9 - Measuring resilience: Designing, collecting, and analyzing data for monitoring and evaluating resilience programs
For: monitoring, evaluation, and research professionals in the field of resilience.
Level: intermediate

This workshop is designed to provide participants with learning on: 1. How sector actors are defining and engaging with ‘resilience’ as a concept and attempting to measure it through different frameworks. 2. What distinguishes M&E approaches for resilience programming and/ or resilience development outcomes, with a key focus on shocks and stresses. 3. The key elements of planning for resilience M&E, including theories of change and result framework models, resilience assessments, and resilience indicator frameworks – and their continuous or frequent adaptations due to complexity and change. 4. Tools and tips in shock and stress measurement and innovations in measuring confounding/layering factors. 5. The current trends in resilience monitoring and evaluation and what the challenges are ahead 6. How resilience measurement data is interpreted and how it is changing the field.

Capacity PW10 - Getting it right: How to make better evaluation recommendations and improve their follow-up
Osvaldo Nestor FEINSTEIN, Ian C. DAVIES, Credentialed Evaluator
For: evaluators and managers
Level: intermediate level of experience in conducting or managing evaluations

This workshop is designed to focus on evaluation recommendations that present a key challenge to evaluators and are sometimes a source of contention with those who would have to work with them. Getting recommendations right is central to use and to a successful evaluation. This training will focus on sharing with a set of diverse participants experience and reflections of more than 30 years of evaluation practice and to be exposed to their experience and the problems they are facing in dealing with evaluation recommendations and follow-up. Participants will be provided with criteria and materials to support their work on framing recommendations and they will learn how to make better recommendations and improve their follow-up. At the end of the workshops participants will understand: 1. Why recommendations?  Different views and frequent pitfalls in making recommendations. 2. How to improve recommendations?  (reversing the negatives). 3. Criteria for judging recommendations (clarity, resilience, etc.) 4. Why follow-up? How to follow-up, and good practices.

PW11 - Evaluative thinking as capacity building – resilience in action
For: commissioners, evaluators and policy and program developers and managers
Level: intermediate to advanced

This workshop is designed to support participants in learning about and exploring the use of ET in policy and program development, implementation and evaluation in order to enhance organizational capacity. On completion of this workshop, participants will: 1. Understand the key elements of Evaluative Thinking (ET) 2. Be aware of strategies and techniques for using ET to build evaluation capacity to enhance resilience in government and non-government organisations 3. Be able to apply evaluative thinking to policy and program development and implementation.

PW12 - How to deal with 'mission impossible' evaluations
Michaela RAAB, Jasmin ROCHA
For: evaluators and evaluation commissioners/managers/clients
Level: all levels

This workshop is designed to enhance participants’ skills and knowledge in (1) Identifying pitfalls in evaluation terms of reference (TOR), so that evaluators can (2) Make well-founded choices with regard to submitting an offer/expression of interest (or decide it is not worth the effort), (3) Develop robust proposals for evaluations, especially of programs and project portfolios and (4) Communicate the proposals effectively to the (potential) client. We will use concrete examples from real evaluations and distil learning from participants’ experience, to develop a framework that can guide participants in making the best of sometimes confusing, over-ambitious or excessively laconic TOR.

Pre-Conference Workshops - Day 2

Evaluation Approaches PW13 - An interactive and case-centered primer on evaluation approaches
For: any evaluation stakeholder
Level: all levels

This workshop is designed to orient participants to the importance of evaluation theory and approaches for designing, implementing, and using evaluation. Workshop activities and materials are designed so that by the end of the course, participants will be able to: 1. Make sense of terms and concepts used in the evaluation approach literature; 2. Understand why knowledge of different evaluation approaches is important for evaluation practice and advancement; 3. Evaluate the usefulness of different methods-, use-, values-, and social justice-oriented evaluation approaches in different contexts; and 4. Apply different evaluation approaches in practice.

PW14 - Opportunities and challenges of private sector evaluation
Frederik KORFKER, Raghavan NARAYANAN
For: commissioners, managers and evaluators
Level: from intermediate (at least with experience to contribute to discussions)

This workshop is designed to: 1. Start with a presentation on methodologies used in private sector evaluation and compare them with public sector evaluation methodologies and practices. 2. To make the workshop as interactive as possible, two case studies will be presented. The main themes of the case studies will be the evaluation of financial intermediaries and evaluating direct equity investments. 3. Finally, as learning from experience is one of the main features of development evaluation, the workshop will discuss adaptive learning as an institutional strategy.

Evaluation Design PW15 - Evaluability assessment for responsive theory-building, exploratory evaluation, and evaluation planning
Tamara M. WALSER; Michael S. TREVISAN
For: commissioners, managers and evaluators
Level: all levels

This workshop is designed to provide participants with learning on: 1. Current theory and uses of evaluability assessment (EA); 2. How to use EA for responsive theory-building, exploratory evaluation, and evaluation planning; 3. How EA can support culturally- and gender-sensitive evaluation; and 4. potential applications of EA in their professional practice.

PW16 - Outcome harvesting —An alternative approach to monitoring and evaluation of social change results
For: commissioners, managers and evaluators
Level: intermediate

This workshop is designed to provide the participants with: 1. An understanding the essence of Outcome Harvesting; 2. The ability to appreciate the importance of customizing the six steps of Outcome Harvesting; 3. Insights into when and why Outcome Harvesting can be a useful evaluation approach for projects, programs or organizations that aim to influence change in other societal actors.

Evaluation Methods PW17 - Case study evaluation: its utility in changing times and contexts
For: evaluation practitioners (commissioners welcome though)
Level: all levels

This workshop is designed to provide participant with: 1. An understanding the purposes, types and use of case study evaluation in different policy, programme and project evaluation contexts; 2. Insights into the kind of knowledge generated in case study and its usefulness in policy making and practice ; 3. Greater confidence in arguing the strengths of case study as methodology; 4. Knowledge of how to plan, design and conduct case study evaluation; 5. Appreciation of the political/ethical in case study evaluation practice; 6. Awareness of different modes of analyzing and interpreting evaluation data; 7. Confidence in articulating different ways of generalizing in case study evaluation; 8. familiarity with diverse ways of reporting that are useful and credible.

PW18 - Attributing development impact: the qualitative impact protocol (QuIP)
For: commissioners, managers and evaluators
Level: all levels

This workshop is designed: 1. To promote a basic understanding of the QuIP: an evaluation methodology that captures narratives of adaptation and resilience at local level by seeking to uncover stories of change and the reasons behind them in the lives of project beneficiaries. 2. To generate thought and discussion about ways of addressing the ‘attribution issue’ in evaluation. 3. To inspire and promote reflection on the way evaluations are conducted. 4. To demystify the ‘black box’ of qualitative/thematic coding and analysis.

SDGs, Gender Equity PW19 - Using UN women’s new evaluation approach: Inclusive systemic evaluation - gender equality, environments, marginalised voices for social justice (ISE4GEMS)
Ellen LEWIS, Anne STEPHENS, Shravanti REDDY
For: all
Level: all levels

This workshop is designed to benefit to anyone interested in learning a new methodology using systems thinking as its theoretical foundation which is practical and flexible. By the end of this workshop, participants will be able to: 1. List 1-2 activities that can be used to create a safe, inclusive, participatory environment for learning; 2. List qualities of systems thinker and an ISE4GEMS practitioner; 3. Describe key components of systems thinking and their relationship to conducting an ISE4GEMs evaluation; 4. Use the ISE4GEMs Chapter 5 – Planning, to develop an evaluation plan of their own chosen project; 5. Have knowledge of the ISE4GEMs data collection and analysis phases, and where to find practitioner tools and guidance to support them to use this approach in these phases of their evaluation. 6. Apply and analyse projects using the intersectionality of GEMs and the SDGs.

Resilience PW20 - Evaluative thinking for more resilient societies
For: evaluators, evaluation capacity developers, managers
Level: all levels

This workshop is designed to provide attendees with learn on: 1. What evaluative thinking (ET) is and how it pertains to their context; 2. How to promote and strengthen ET among individuals and organizations with whom they work; 3. How to use ET to identify assumptions, articulate program theory, and conduct evaluation with an emphasis on learning and adaptive management.

PW21 - Crossing personal and technical boundaries towards more resilient evaluation practices  
For: evaluation officers, commissioners and consultants
Level: intermediate

This workshop is designed to enable participants to: 1. Identify their own personal boundaries and technical boundaries in evaluation; 2. Get out of their comfort zones, both methodologically and conceptually; 3. Discuss foundational evaluation topics with colleagues and clients in more innovative and thought-provoking terms; 4. Contribute to the planning and delivery of formal capacity development programs as well as informal exchanges on evaluation with peers and clients in a more culturally competent and contextually relevant manner; 5. Make the best use of the available resources to deal with the complexity of their future M&E tasks (including SDG measurements); 6. Use evaluation to advance social justice and gender equality by establishing bridges with other disciplines.

Capacity PW22 - Integration of evaluation capacity and culture into organizations
For: program managers
Level: intermediate and advanced

This workshop is designed for participants to understand evaluation capacity and culture and understand what is necessary for integration into organizations. This will happen through use of a handout (developed and printed by facilitator) walking through 1. Explanation of evaluation capacity and culture; 2. Practices that build capacity and culture; 3. Alignment with need and likelihood of application with participants; 4. Description of capacity and culture building using a logic model format and 5. Description of implementation of capacity and culture building using a theory of change.

PW23 - Organizational monitoring and evaluation (M&E) capacity development
For: monitoring and/or evaluation specialists, managers, trainers and trainers of trainers
Level: all levels

This workshop is designed to provide participants with an understanding and knowledge of: 1. The key principles for effective organizational M&E capacity development; 2. How to adopt a systems approach to organizational M&E capacity development; 3. Practical steps to conduct an organizational M&E capacity assessment; 4. Practical steps to strategically plan for organizational M&E capacity development; 5. Key considerations for recruiting and managing an M&E capacity building consultant.

PW24 - Serving the beast? - delivering evaluations to government officials
For: evaluators, evaluation managers, commissioners
Level: intermediate

This workshop is designed to help participants in mapping, understanding and managing the conditions in which government commissioned evaluations are procured and carried out. By completing the workshop participants will 1. learn the types of challenges and constraints typically faced by government officials in the procurement process and develop strategies to tackle these 2. develop an understanding of the specific knowledge needs at various levels of governance 3. practice techniques to map and manage these knowledge needs throughout the institutional structure 4. practice presenting evaluation information tailored to knowledge needs 5. develop a portfolio of skills relevant for managing government commissioned evaluations.

For queries about the content of the pre-conference workshops please contact Albine Guitard (Albine.Guitard@europeanevaluation.org) or Ian C. Davies, Credentialed Evaluator (CE) (Ian.Davies@europeanevaluation.org).

For logistical and organisational questions, including about your stay in Thessaloniki, please contact the Conference Secretariat (pre-conference@ees2018.eu).

Albine Guitard
Ian C Davies, CE