As investment in DPI increases, how can we meaningfully measure its impact?

From driving much-needed economic growth in low-income countries to bridging the digital divide, digital public infrastructure (DPI) offers incredible opportunity. Because good DPI includes critical components like digital IDs, payments, and data exchange, it can provide an adaptable, trusted, and sustainable path to fostering digital inclusion for people across the world.

Yet, while DPI has great promise, there are also potential risks, such as exacerbating power imbalances, excluding marginalized populations, and being used as tools for surveillance. In addition, without thoughtful governance, DPI could lead to data being accrued inequitably and without agency of the individual.

Without a shared vision of the impact of DPI and comprehensive indicators to meaningfully measure the benefits – and risks – we will fall short in understanding whether DPI has delivered on its promise. This is why, as investment in DPI accelerates, we must ask ourselves what is needed to ensure these systems are used for the benefit of all people.

And, just as importantly, how do we measure it?

While access and usage metrics are important, alone, they are insufficient in determining whether, or not, DPI is meaningfully improving quality of life for people. For example, while increased adoption may indicate successful uptake, it also attracts the possibility of fraud, exclusion, and harm.

This is why, to gauge the benefits of DPI, we must understand how these initiatives are improving people’s lives, enabling their rights, and mitigating risks of discrimination or exclusion. In a 2022 paper published in partnership with the Brookings Institution, we identified three key gaps in measuring the impact of DPI:

1. There is no comprehensive set of indicators to measure impact. Existing measurement frameworks are fragmented, focusing on the uptake of digital services or the presence of digital rights.
2. Access and usage metrics are often used as proxies for understanding outcomes. While these measurements are critical, alone, they fail to comprehensively capture DPI’s impact on people and communities.
3. Fragmented and siloed measurement frameworks lead to siloed funding and administrative efforts. This exacerbates the challenges of assessing progress holistically and identifying key interventions to improve DPI deployments and mitigate unintended consequences.

Since these findings were first published in 2022, the DPI landscape has evolved and matured. Today, we see not only increased demand for measurement tools and guidance on impact metrics, but also a growing recognition that such tools must help assess DPI’s impact on people.

Today, we’re witnessing increasing demand signals from stakeholders across the DPI ecosystem, including funders, governments, the private sector, system integrators, and civil society organizations. Many different needs have been identified under the broader umbrella of measuring DPI’s impact on people – from a comprehensive measurement framework covering the lifecycle of a DPI initiative to understanding DPI’s sectoral impact. This demand is diverse, covering a wide range of use cases and motivations, with opportunities including:

1. Countries looking to understand how specific DPI-enabled investments – like e-government portals – are impacting people.
2. Governments seeking to gauge how DPI-enabled services are supporting development outcomes for traditional sector verticals like health, education, and agriculture.
3. Opportunities to disaggregate DPI impact to understand the unique challenges to adoption being faced by traditionally marginalized populations.
4. Frameworks to measure and assess the health of DPI initiatives. The UN has further signaled this need, recently launching an initiative to develop a Universal Safeguards Framework for Digital Public Infrastructure (DPI).

At the Digital Impact Alliance, we’re working to help close the metrics gap.

We’ve received requests from countries to help shape the measurement frameworks and indicators for their digital public infrastructure initiatives. While our work is still in its early stages, we already see a need to assess impact across the DPI lifecycle, broadly categorized across three stages – design, deployment, and operationalization. A few of the initial steps we’re taking to expand on this need and to help partner organizations, governments, and funders assess the impact of DPI on people include:

Working with governments to better understand the existing environment. By collaborating with a handful of national governments, we will develop a better understanding of how countries themselves are approaching questions about measuring progress and impact of DPI, including key elements such as institutions, governance, laws and policies, market dynamics, and technology systems. By assessing DPI through each of these elements, we can better understand current approaches and gauge the needs of our government partners.

Examining how funders and governments measure the impact of their DPI initiatives. Through such efforts, we will work to foster a collective understanding and knowledge base of learnings and practices for the DPI ecosystem. These learnings will then help us create a taxonomy around understanding and articulating DPI’s impact on people. In addition, our consultations with governments and funders will help us clarify the relationship between outcome indicators and inputs towards them across the DPI lifecycle. This will in turn help us unpack the key institutional decisions made by our country partners while setting up their DPI programs, and how to understand their impact.

Engaging researchers, funders, practitioners, and governments to collaborate on this critical topic through existing channels and communities, such as our Digital Donors Exchange network. These discussions will serve as a platform to share our learnings and gather feedback from different communities. Just as importantly, this will provide opportunities for stakeholders to highlight and share their learnings as well.

As we delve into each of these different areas, we will document and share key insights, common challenges, and recommendations going forward. These learnings, we believe, can be leveraged by governments, technologists, developments actors, and civil society organizations across the world to further support the creation of effective impact metrics. Through collaboration and collective problem-solving, we can foster a future where DPI advances quality of life, wellbeing, and inclusion for people and communities globally.