When considering the various ways in which data is produced and used in supporting and strengthening basic urban service delivery systems (water, sanitation, etc.) around the world, we at Athena Infonomics – a data-driven development consultancy – have identified two common types of data collection: 

• One-off data collection as part of donor-funded initiatives to support planning/evaluation goals.
• Data collated from local authorities and service providers by respective line ministries as part of a national management information system (MIS) reporting mechanism. In some cases, we might also see independent national statistical and planning agencies publishing datasets, often statistical estimates, on service outcomes, in intervals of 2, 3, 5, or ten years. 

But is this the best way to collect data? Deepa Karthykeyan, Athena Infonomics’ Co-founder and Director, concludes that “over time, we’ve found these approaches are unlikely to help sustain and improve results on service outcomes unless robust data systems and a strong culture of data use exist within local governments and/or local public service providers.”

Under the Citywide Inclusive Sanitation (CWIS) Monitoring, Learning, and Evaluation (MLE) Initiative, funded by the Bill and Melinda Gates Foundation, Athena Infonomics works with a set of local governments and utilities in South Asia and Sub-Saharan Africa to unpack the limitations in current approaches to data systems and discover ways in which we can re-orient data-linked investments to sustain and advance improvements in city services.

As part of this work, we conduct targeted assessments and analyses of existing data systems to determine their relevance and usefulness to service authorities. We then analyze the motivation, financing sources, and primary uses for these datasets to shed light on the sustainability of these systems and the extent to which they have effectively shaped service outcomes.

In our work with local governments and utilities, we’ve learned some key lessons:

National MISs are useful for sustained data generation at scale, but are of poor quality and ineffective when not tied to a clear accountability and resourcing framework. All of the countries analyzed have some form of MIS to collate data from local governments and utilities. Under the CWIS MLE Initiative, we sourced most collated data from these national systems. However, these systems suffer from three drawbacks: 1) they lack a clear accountability framework and/or use case; 2) aggregated estimates often serve as the reported data within these systems, which generally do not prove useful for local service planning and delivery; and 3) inadequately resourced cities, towns, and utilities cannot reliably report and generate the data required by the MIS. So even when a national MIS exists, its impact on both the city-level services and systems is limited. Adding indicators into an ineffective MIS is not useful.

The quality and sustainability of service data systems correlate directly to its utility to the data producers, and the ease and cost of the data collection. In several cities participating in the CWIS MLE Initiative, available data were actually estimates that were used for the purposes of reporting, but failed to accurately reflect the cities’ service coverage realities. For example, the categorization of sanitation systems usually looks at infrastructure assets being established (toilets, sewage treatment plants, etc.) without looking at service improvements (safely treated, managed, etc.).

In fact, we only observed timely and reliable data production when data generation occurred via a digitally enabled platform, as a by-product of city operations, and where generated data provided direct value to local authorities. Investing in smart “digital public goods” that support city service delivery offers one pathway to enabling this. For example, the Government of India’s Smart Cities Mission has made some interesting progress through its work on the National Urban Innovation Stack (NUIS), a digital public good designed to enable different cities to design and implement local city services and generate, use, and visualize data cost effectively.

Co-production with local actors is critical for sustainable outcomes. The process of designing a useful data system requires us to set up mechanisms to listen to local authorities and their needs and develop and iterate on solutions with local actors. These processes are neither simple nor linear – they require creative moderation and facilitation that allows for effective leadership and participation from local actors in shaping the systems. While these processes may be time consuming, they help establish data systems and long-term partnerships that truly work to sustain and improve services.

In a world that is increasing dependent on data, these three lessons emphasize that sustainable outcomes rely on data that is accountable, useful, and locally owned and utilized.