Imagine this: you have come into some cash and you want to go on a road trip with some friends. You decide on your desired destination (which in evaluation terms, we shall refer to as the ultimate goal). For your journey to be deemed successful, various considerations need to be fulfilled before, during, and at the end of the journey.
Thus, the overall goal will help you in clearly mapping your journey (the systematic process), its different components (objectives), resources needed, and tasks (requisite activities) that need to be executed. These can be likened to different phases in a project that necessitates the execution of different types of evaluation, namely; the design (baseline) evaluation, formative (mid-line) evaluation, summative (end-line) evaluation, and finally the impact evaluation.
But what is evaluation?
The World Bank Group uses a definition of evaluation adapted from the Development Assistance Committee of the Organisation for Economic Co-operation and Development (OECD-DAC) (2002): ‘A systematic and objective assessment of an ongoing or completed World Bank Group process, project, program, theme, strategy, or policy, and of its design, implementation, and results, in relation to specified evaluation criteria.
It can also be defined as a systematic determination of the merit, worth, or value of something, or the outcome of a process. We can use the two definitions to summarize key components of evaluation such as; for an evaluation to be considered successful it ought to be 1) a systematic and objective process that considers the links between inputs and/or activities to the outputs and/or outcomes of a program or a project, 2) frequency of the assessment(s) also should be determined during planning, and finally, 3) it should enable the evaluation team to measure or compare the results against set standards (targets/benchmark) thus enhancing decision making and learning. The latter enables the evaluator(s) to establish whether the performance can be considered as valuable, worth, or having merit and what may need to be done to improve the performance (in the future) when bottlenecks have been identified.
Let’s continue on our road trip: You decide to use your car in order to reach your desired destination at your own speed and time. Certain conditions must be met before starting off- you need to ensure that the car is in good condition, that each of its components is in place and its service log is okay. You check that the spares are present and that the fuel gauge is reading well. You also need to determine the best/fastest route to your destination and if you (or a different person) are competent to handle the journey. This may be compared to a baseline evaluation, with the various items you use to assess their conditions can be regarded as indicators. It is only once you are satisfied that your car has what it takes for the journey as well as any other items you may need (inputs), that you may embark on it. During the journey, you need to keep a keen eye on these key indicators (monitoring).
In case of any mishaps or dashboard alerts, you will need to make a quick assessment to determine if you are on track or how you can improve your car’s performance. This is what is known as formative (mid-line) evaluation. There are also externally determined alerts (signage) such as speed limits, bumps, bends, etc. that will help you to operate within the law (required standards). Others could be other road users that you can use to determine the level of speed and caution so as to avoid accidents (benchmarks or targets). Your ability to do continuous monitoring and evaluation will help you determine how well you can increase your car efficiency by reducing; fuel consumption levels, collisions as well as wear and tear.
At the end of your journey, you may need to evaluate how your journey was (end-line/summative evaluation)-what best practices made a difference? What ought to be avoided or improved? As you come up with different ideas (interventions), for example, on how to minimize the fuel consumption or wear and tear, you can measure the outcomes (of the future trips) against the experience before you had introduced such interventions (baseline). This can be likened to impact evaluation.
Similarly, in a program or a project, evaluation is necessary in order to analyze why intended results were or were not achieved, to assess specific causal contributions of activities to results, to examine implementation processes, to explore unintended results, and to provide lessons learned, and hence offer recommendations for improvement. As we found out in our road trip analogy, monitoring and evaluation are complementary in nature; monitoring will help us discover ‘what’ is happening while evaluation will enable us to understand, ‘why or how’ the performance fared. Evaluation can be conducted by use of either quantitative (numerical measures or indicators), qualitative (expressed in words), or mixed methods (both quantitative and qualitative). The evaluative theory has been defined by Shadish Jr, Cook et al. (1991) as the underlying framework of evidence-based practice. Contextually (Christie and Alkin 2013) considers evaluation to be rooted in three key purposes namely; to promote accountability, to enable systematic social inquiry (applying a justifiable set of procedures for determining accountability) and for epistemology (knowledge creation).
The Organization for Economic Co-operation and Development's (OECD) Development Assistance Committee (DAC) in 2019 developed evaluation criteria which relate to the purpose of an evaluation, they include; relevance, effectiveness, efficiency, sustainability, and impact. These criteria help in the determination of how worthy an intervention being evaluated is, for instance, we can use them to assess if the interventions introduced in the above road trip analogy had merit.
Acknowledgments & Disclaimer: The author is solely responsible for the content of this article, including all errors or omissions; acknowledgments do not imply endorsement of the content. The author would like to acknowledge the support of his colleagues Michelle Mbuthia and Dr. Moussa Bagayoko in reviewing the article and providing feedback. The picture submitted was taken by the author on a section of Thika Road in Nairobi Kenya. The views expressed in published blog posts, as well as any errors or omissions, are the sole responsibility of the author/s and do not represent the views of the Africa Evidence Network, its secretariat, advisory or reference groups, or its funders; nor does it imply endorsement by the afore-mentioned parties.