In recent years, countries across the world have made important development gains thanks to digital public infrastructure (DPI). In Brazil, the digital payment system Pix has seen uptake by almost 80% of the country’s adult population – contributing to more than 3 billion transactions per month. While in Uganda, the digital UGhub platform had been adopted by over 100 public and private entities, connecting citizens to critical public services, from passport renewal to vehicle registration.
As an approach, DPI represents an opportunity for more inclusive and empowering digital transformation. Yet, to truly understand the full impact of DPI – and ensure it is maximized for the public benefit – robust, standardized metrics are needed.
Today, most available metrics tend to focus on access to and usage of digital services, as evidenced by the examples above. Extending these metrics to include elements such as trust in digital systems, economic impact, marketplace competition, and meaningful enactment of data protection can help governments and development partners better understand how to drive positive progress towards more inclusive and trustworthy digital systems and services.
Our yearlong research effort revealed two key opportunities to better measure impact.
To help bridge these gaps, our team at the Digital Impact Alliance has spent the past year working to better understand the current DPI measurement landscape.
Our efforts include, but are not limited to:
- Reviewing existing tools. We examined more than 30 digital ecosystem-related indices and measurement tools, allowing us to better understand where fragmentation and inconsistency exist in current efforts.
- Developing a series of case studies. Through these case studies, we analyzed how different governments are classifying and tracking DPI and DPI-related progress and outcomes.
- Convening cross-sector experts. We brought together a panel of experts across the digital development and policy ecosystems to examine our findings and discuss practical recommendations for addressing the current gaps.
Overall, our research and consultations have revealed two key opportunities for strengthening the measurement ecosystem.
First is standardizing new “metrics that matter.” Many of the metrics currently used today measure economic impact or system usage as proxies for other indicators. To appropriately measure and leverage metrics that matter — like people’s well-being, expectations and experiences with digital transformation initiatives, and more — greater standardization is needed.
Second is maximizing existing data to reduce fragmentation and duplication. While there have been increased investments in DPI and focus on its impact, the processes and approaches for gaining rich insights and understanding of the impact of DPI is fragmented. This can cause practitioners to duplicate efforts and overlook useful data that already exists.
Realizing these two opportunities will be necessary for DPI to reach its full potential in advancing benefits for people, governments, and markets.
A common reference framework can bridge key measurement gaps.
With these opportunities in mind, we’ve outlined our vision of a stronger and more consistent measurement ecosystem – one that would unite the digital development community around shared expectations and outcomes.
There are many investment opportunities in this space already underway, including testing new methods and instruments, cultivating diverse research networks, and improving discoverability and usability of existing tools. But without greater coordination of these projects, we risk increasing the very fragmentation and confusion we’re aiming to reduce.
That’s why, to realize this vision for a robust, unified DPI ecosystem, our efforts pointed to one immediate, pressing need: a common reference framework. This type of framework can create a clear taxonomy for impact metrics and support greater consistency among existing projects and measurement efforts.
There are several qualities of a good common framework that will be key to achieving these goals:
1. It keeps pace with rapidly changing DPI developments.
Based on our insights over the past year, we recommend a nimble and iterative approach to creating the common reference framework. Given the pace of DPI implementation, the DPI ecosystem will not pause efforts to measure impact while a framework is developed – and rightly so.
In this case, a bottom-up approach is needed. We plan to use a time-bound, highly focused consultative process to develop specific sets of metric categories, thus avoiding any further fragmentation. These categories can then be tested through ongoing measurement projects and subsequently added to the common framework. This process will involve quick prototyping and testing cycles that feed into a global framework, rather than first building a global framework that is then adapted for individual projects. These targeted cycles of designing, testing, and integrating will not only support iterative improvements to DPI deployments but also embed insights, making the framework much more robust.
2. It allows for diverse perspectives of “impact.”
Efforts to create agreed upon DPI “metrics that matter” need not prioritize one definition of impact over another. Instead, our vision of a common reference framework allows for tailored survey instruments and relative prioritization depending on local context and development priorities.
When creating new DPI systems and services, different governments prioritize different outcomes, from greater efficiency in public service delivery, to economic growth, to public health outcomes. Of course, these goals are not mutually exclusive and can certainly all be advanced in parallel through strategic investments in foundational digital systems. However, initial motivations and national development priorities influence the key performance metrics that are often tracked, which is why a common reference framework must be adaptable to such diversity.
3. It is a unifying force for existing efforts.
In addition to the common reference framework, there are a number of ongoing and planned activities that have great potential to strengthen the DPI impact measurement ecosystem. These include stronger collaboration among research networks that bring visibility to diverse perspectives, several active survey instruments examining some aspects of DPI impact, and new models emerging for improving the discoverability and usability of relevant data that does exist.
The common reference framework is not meant to compete with or replace any of these efforts, and investment would not occur in a vacuum. Instead, each of these efforts can be designed and implemented independently, while using the framework as a common reference point to enable use of shared vocabulary, metrics, and more.
Clear and defined metrics are key to understanding – and maximizing – the value of DPI across every level of society.
Building the common reference framework is an important step to align efforts around understanding DPI’s impact on people, households, governments, and markets.
For people, a common reference framework could help gauge levels of gender inclusion, citizen satisfaction with digital services, digital literacy, as well as the intangible – yet equally important – measures of good DPI, such as participation, agency, choice, and trust.
For households, DPI impact metrics could include education outcomes for children, access to affordable healthcare, financial inclusion, and disaster preparedness, just to name a few.
For governments, key metrics might include enhanced public service delivery, cost savings, and benefits to citizens, such as reduced wait times.
For markets, having a common reference framework could help measure progress toward economic growth, competition among companies of varying sizes, local digital innovation, and meaningful job growth.
Because DPI is an approach to building society-wide, foundational digital layers, its benefits can be felt across many different sectors. The common reference framework will provide a shared means of determining what these benefits look like. And, most importantly, it will enable us to differentiate the impacts of the DPI approach from other forms of digitalization.
To make these benefits a reality, investment and collective will are needed.
To realize the benefits of the DPI approach, it is critical that the global community align around the necessary support to co-create a common reference framework.
First, investment is needed to create, test, and successfully deploy the common reference framework. Key stakeholders, including both private and public funders, can play a critical role in advancing this important work.
In addition, while the framework relies on a more dynamic and initially distributed approach, convenings and information sharing will also be necessary. Ultimately, a mandated institution (or a community of practice with established governance) will be needed to manage the common reference framework and shared database of standard indicators measuring DPI impact; host working groups to design indicator categories and corresponding methodologies; and convene countries and provide support in integrating these indicators in their national impact measurement tools.
As a catalyst for greater standardization of DPI impact metrics, a common reference framework provides a much-needed means of both promoting shared understanding around DPI’s impact and helping overcome key barriers for individuals, communities, and society.
We hope you will join us in advancing this critical work.
If you’d like to learn more about the common reference framework or contribute to our efforts, we encourage you to message us at info@dial.global
