While USAID’s E3 Office of Education (E3/ED) focuses on the design, implementation, monitoring, and evaluation of education programs, in the United Kingdom, DFID’s Girls’ Education Challenge Fund (GEC) shares a similar mandate. Recent GEC initiatives have explored innovative ways to build capacity within governments, communities, and schools; offered training and mentoring to teachers and government and community leaders; and provided safe spaces and educational resources to help girls learn.

In 2013, DFID commissioned one of their largest ever program evaluations to assess the effectiveness, impact, and value-for-money of their Girls’ Education Challenge Fund (GEC). By assessing the effectiveness, impact, and value-for-money of these and other GEC initiatives, DFID’s program evaluation aims to support GEC grantees by providing them with robust monitoring and evaluation (M&E) data. This data will help grantees to identify the initiatives that have the greatest effect on access to education in different cultural contexts, and why.

Behind DFID’s GEC M&E framework and the answers to these questions lay a daunting data collection challenge. Coffey International, the organization tasked with overseeing the evaluation, invited Virginia-based small business Opinion Research Business International (ORB International) to join their evaluation team and undertake a multi-year, multi-country, research study assessing the obstacles to girls education.  ORB International was commissioned to collect both quantitative and qualitative data from audiences, including the girls themselves, in nine developing, fragile, and post-conflict environments – Afghanistan, Somalia, Ethiopia, Kenya, Tanzania, Mozambique, Zimbabwe, Democratic Republic of Congo, and Sierra Leone. Over the course of three waves of research (baseline, midline and endline) in each of the 9 countries, ORB International will complete over 18,000 quantitative survey interviews and 1,600 qualitative in-depth interviews.

Gathering data in such challenging environments is no small undertaking. Quantitative data collection in all countries except for Afghanistan was completed using tablets. This data collection method allowed us to gather and report on accurate data in real-time, map the location of each interview with GPS coordinates, and closely monitor fieldwork on a daily basis. Local fieldwork teams received intensive enumerator training from ORB International and then used the tablets to administer the computer aided personal interviewing (CAPI) surveys that our team had scripted specially for these devices. 

Our previous experience of running CAPI surveys in fragile environments provided us with the experience necessary to mitigate challenges associated with using technology for data collection. These challenges included limited access to electricity for tablet charging, low internet penetration for data syncing, security issues arising from local law enforcement suspicion, and difficulties bringing devices in through local customs agencies.

Despite these challenges, baseline data collection for all nine countries was successfully completed in June 2014. For the baseline alone, ORB International – with the support of its network of local fieldwork partners – completed more than 6,000 quantitative household surveys with primary caregivers and girls aged 5-15; 3,400 surveys with school administrators and teachers; and 810 qualitative in-depth interviews with households, teachers, and community leaders. In addition to the baseline surveys and in-depth interviews, ORB undertook school assessments comprising teacher and classroom observations, teacher surveys, and reading and mathematics assessments with both male and female students at selected schools in Kenya, DRC, Ethiopia, and Sierra Leone. The data was then collated, analyzed, and fed back to GEC grantees with actionable insights to help them refine their interventions and provide access to high quality education to the world’s poorest girls.