In January 2020, the African Population and Health Research Center (APHRC), for the first time in its history, delved into the world of Measurement and Impact Evaluation (M&IE). Through a project led by Drs. Damazo Kadengye, Moses Ngware, and Moussa Bagayoko, was conceptualised with the aim of strengthening the Center’s position in policy-relevant evaluations. Specifically, it is driven by measurement methodologies spanning impact evaluations, performance and monitoring systems, economic evaluations (cost-benefit analysis, cost-effectiveness analysis), and the development of theories of change, through the building of strategic relationships with key policy actors at national and sub-national levels.
Pegged as a capacity-building program, the initiative is in line with APHRC’s vision to be the pan-African center of excellence by building its internal capacity on classical and emerging M&IE methodologies.
While many may be more familiar with the ‘monitoring’ aspect of project management, impact evaluation seems to be the lesser-known twin. So let us break it down a bit: while the measurement is concerned with the performance and progress of an initiative (taking into consideration its relevance, efficiency, effectiveness and even sustainability), impact evaluation is ‘the assessment of the contributions of interventions towards a specific outcome or goal’. Here, impact refers to the changes that are attributable to a particular intervention based on models of cause and effect and requires a credible counterfactual to control for factors other than the intervention.
Counterfactual?
This indicator measures what would have happened to beneficiaries in the absence of the intervention.
Therefore, impact evaluation can be positive or negative, primary or secondary long-term effects produced by a development intervention, intentionally or inadvertently, directly or circuitously. To put it succinctly, M&IE is an accountability tool.
As a continuous exercise over the life-course of a project, impact evaluation is important because it provides evidence relevant in critical decision-making during or after the project, such as scale-up, replication, or perhaps, discontinuation. It enables researchers and program implementers to establish how much-observed change is attributable to their interventions. Attribution, in this case, means showing linkage to observed changes as stemming from the intervention or study, but also how they came about. An MIE exercise would demonstrate, for example, how APHRC’s interventions in urban informal settlements have improved the lives of the residents of these areas by conducting impact evaluation and subsequent uptake of the evidence from the evaluation- findings in decisionmaking by stakeholders at different levels for policymaking and action in informal settlements.
It is imperative to consider the timing of an impact evaluation, that is, when conducted late in the life of a project, the findings are too belated to inform any sort of change. At the same time, if they are too early then they may be premature. As with all the good things in life, timing is of the essence.
Impact evaluation need not necessarily be a one-man show; it can be a participatory process depending on the lens used- experimental versus non-experimental, qualitative versus quantitative, or perhaps both á la mixed methods. The key, however, is in the selection of the participants based on the role they play as well as the timing of their involvement. Within this stakeholder group, there are those who define impact while others are end-users of it. At times, those two groups cut across each other.
It should be noted that Impact Evaluations are conducted at different levels and scales of implementation. As the world heads into the homestretch of the Sustainable Development Goals, governments can use impact evaluation to assess their progress towards the achievement of these development objectives and use the findings arising from it to inform the identification, planning, and implementation of future targets.
Acknowledgments & Disclaimer: The author is solely responsible for the content of this article, including all errors or omissions; acknowledgments do not imply endorsement of the content. The views expressed in published blog posts, as well as any errors or omissions, are the sole responsibility of the author/s and do not represent the views of the Africa Evidence Network, its secretariat, advisory or reference groups, or its funders; nor does it imply endorsement by the afore-mentioned parties