By Mary Pat Selvaggio, MPH, Director of Health and Research
Martha Muco, PhD, Senior Vice President, Monitoring and Evaluation
Khulisa Management Services
September 15, 2023
Khulisa Management Services (Khulisa) has provided monitoring, evaluation, and learning (MEL) services in the global development sector since 1993. When designing large, multi-year evaluations, we often discover a need to identify not only the existing evidence on a topic, but also the evidence gaps: subject areas where no evidence or minimal evidence currently exists. Understanding the existing evidence and evidence gaps in a specific sector is crucial for learning, adapting, resource allocation, and decision-making.
In response to this need, Khulisa has begun developing evidence gap maps (EGMs) for our clients. EGMs are graphical representations of research and evaluations conducted in a specific area, sector, or domain, highlighting areas with abundant evidence as well as areas with evidence gaps. EGMs are typically used as interactive tools that provide policymakers and program implementers with an easy way to access and review existing data and make informed decisions. EGMs are also useful for research prioritization, avoiding duplication of effort, and directing research resources where they are needed most.
A Nutrition Evidence Gap Map in Zambia
Khulisa and its team of subcontractors first developed an EGM under the USAID/Zambia-funded Scaling Up Nutrition Learning and Evaluation (SUN LE) project. Zambia’s multi-sectoral Scaling Up Nutrition (SUN) program involves activities across many different Zambian government agencies. When the Khulisa team started SUN LE in 2018, there was limited understanding of the range of evidence (and gaps in evidence) around multi-sectoral effects on nutrition in the country. Creating an EGM was a way to identify where too much — or, conversely, not enough — information had been collected.
“Conducting an EGM is useful to the SUN LE project as it brings up evidence efficiently by a click of a mouse, and all the information on research and evaluations related to multi-sectoral nutrition in Zambia is readily available,” says Mathews Onyango, Khulisa Project Director for SUN LE. “Users have access to information on past research and evaluation information, which in turn allows a user to see where the gaps are on any topic and areas needing more research.”
A screenshot of the Zambia Nutrition EGM. To review the Zambia Nutrition EGM and all of the included studies, click here.
Using the template developed by the International Initiative for Impact Evaluation (3IE), the Khulisa team created an EGM classifying the evidence by interventions in all the relevant sectors: agriculture; health and nutrition; and water, sanitation, and hygiene [WASH]. The EGM also classified the evidence by nutrition outcomes, including nutritional status, dietary/feeding practices, and WASH and health behaviors, among others.
To create the EGM, Khulisa and its partners, along with a team from the University of Zambia, undertook a systematic scoping review following guidance from JBI (formerly the Joanna Briggs Institute). The team searched both the peer–reviewed and grey literature (unpublished articles), scouring more than 3,000 abstracts and peer-reviewed articles, three dozen websites, and grey literature in the USAID Development Experience Clearinghouse (DEC). We used Covidence to screen abstracts, review full text articles, and select articles and reports to include, as well as to extract data. Tableau was used to display the results of the systematic scoping review using the 3IE template. The final EGM includes 551 articles and reports from 1951 to 2023.
The EGM was officially launched at the Zambia National Nutrition Conference in mid-September 2023. The EGM will also be published on Zambia’s National Food and Nutrition Commission website.
“The EGM benefits the government of Zambia in its effort to improve nutrition outcomes,” Mathews says. “It efficiently delivers all the available evidence for the government’s decision-making, which enhances the government’s ability to effectively utilize its resources. The government of Zambia does not need to use more resources on conducting a study when there is already available evidence on that topic. They can rather prioritize those resources to conduct a study in an area where there is currently no evidence.”
Sylvia Masebo, Zambia’s Minister of Health, addresses the Zambia National Nutrition Conference in September 2023. The Nutrition EGM is helping the Zambian government prioritize resources in their efforts to improve nutrition outcomes in Zambia. (Photo: Chibesa Mulumba)
Future EGMs
Due to the success of the Zambia nutrition EGM, Khulisa has begun incorporating the concept into other USAID-funded projects. Khulisa is currently planning an EGM for a USAID activity in West Africa in the field of democracy and governance, helping USAID to make evidence-based decisions to preserve peace and counter violent extremism. This EGM will include evidence from 2017 onward. When implemented, the EGM will guide USAID in programming and decision-making in terms of intervention and the strategy-related objectives of the West Africa Mission.
EGMs are great tools to prevent duplication of effort, reduce cost, and create solid, easily updatable evidence for interventions. Khulisa recommends EGMs for long-term development programs and looks forward to the EGM becoming a more widely recognized tool in the field of MEL.
