Over the past year, the Digital Impact Alliance (DIAL) conducted an in-depth analysis of the DPI ecosystem, examining the motivations, mechanisms and key factors that drive effective DPI measurement. Through our efforts, we have developed a deep understanding of the multifaceted demands, needs, and constraints faced by DPI collaborators and implementors – at both a national and global level.

Some of our collaborative research and engagement efforts included:

• Analyzing survey instruments. We examined over 30 digital ecosystem-related indices and measurement tools and spoke with a broad range of stakeholders working on DPI. Among these stakeholders, we witnessed a strong demand for more targeted tools and guidance for measuring DPI impact. These ranged from specific use cases to a national benchmarking index, to measuring how different user groups interact with DPI.
• Exploring country approaches. We analyzed how different governments are classifying and tracking DPI and DPI-related progress and outcomes. Through these efforts, we learned that while various stakeholders may have differing motivations for tracking DPI impact, they often face common challenges, including lack of tools and indices, insufficient capacity, governance challenges, and competing priorities.
• Convening cross-sector experts. We brought together a panel of leading experts in digital development and policy to examine our findings and discuss practical recommendations for addressing the current gaps. Through these discussions, stakeholders highlighted a need for a coordination mechanism to learn, share, and discuss various measurement methods and practices – including how to assess the optimum pathway for DPI impact.

Through this targeted approach, we have pinpointed several areas where the global DPI community can drive significant change. In particular, our work revealed two critical opportunities that, if solved, could unlock substantial growth in the field of DPI measurement.

As countries becomes increasingly reliant on DPI-enabled tools and services, now is the time to harness these opportunities. Otherwise, we risk perpetuating—and even exacerbating—unintended consequences and potential negative side effects of well-intentioned DPI initiatives.

To categorize and contextualize these metrics, a common reference framework would provide a necessary foundation for a more consistent approach to DPI metrics.

Through a highly consultative process, the common reference framework would create a shared taxonomy for each of the example categories above and plus others. Both the consultative process itself and the results of this consensus-building activity would meaningfully contribute reducing fragmentation, filling gaps, and meaningfully improving the current state of play.

We have identified several key elements that will ensure the common reference framework is both effective and adaptable.

• It’s not a mega instrument. By taking a nimble and iterative approach, the common reference framework can be implemented quickly and efficiently – allowing it to keep pace with the rapidly changing DPI developments.
• It allows for diverse perspectives of impact. We recognize that implementors may have different motivations – and success indicators – when creating DPI solutions. That’s why the common reference framework will allow for tailored survey instruments and prioritize the metrics that are most relevant to their local contexts and development priorities.
• It is a unifying force for existing efforts. The common reference framework is not meant to compete with or replace efforts that are already underway. Instead, each of these efforts can be designed and implemented independently, while using the framework as a common reference point to enable use of shared vocabulary, metrics, and more.

Our targeted experience and expertise at the intersection of good DPI and impact measurement well positions us – the Digital Impact Alliance – to help realize the common reference framework. The creation of the common reference framework will undoubtedly require the efforts of diverse organizations and actors – many of whom already have related projects underway. As an established global convener, we can coordinate these collaborative efforts – quickly and effectively.

Some of our initial, immediate activities would include:

• Developing a set of core questions that can be integrated into existing measurement tools and approaches. This step will be critical in helping build connections between connectivity, productive use, and DPI metrics. For example, these questions could include components around access, usage, common constraints, which could then be evaluated against more nuanced metrics like trust, satisfaction, etc.
• Improving the discoverability and usability of existing data. For the common reference framework to be viable, the DPI community must first align around a unified vision of data sharing and reuse. We will test various data governance models and work to align technical standards, helping us build consensus around the components needed for a successful data sharing approach.
• Harmonizing definitions and categories across the value chain. Having agree-upon definitions is an important prerequisite for the common reference framework. These definitions would be extraordinarily helpful – both for the DPI ecosystem and for developing meaningful connectivity metrics. In addition, this is an important step in understanding counterfactuals – for example, how outcomes might change depending on whether a DPI approach is used.

As the common reference framework matures and a community of practice grows around it, more responsibility and ultimately, control, would be handed to other stakeholders within the DPI ecosystem.

By reducing fragmentation and advancing “metrics that matter,” the common reference framework will act an essential guide for stakeholders across the DPI ecosystem, including those helping design, implement and fund DPI initiatives.

For example, publicly available, global data on DPI impact could advance better quality DPI deployments and policies – contributing to increased trust among users. In addition, readily available datasets would help researchers generate nuanced insights that will sharpen models of design, governance, and deployment of DPI.

Here are some illustrative examples of the ways in which DPI stakeholders could benefit from the common reference framework.

The benefits of the common reference framework aren’t just limited to those working on or supporting DPI design, implementation and governance. By including metrics that measure people’s lived experience of DPI-enabled tools and services, the framework could also play a key role in advancing positive outcomes for individuals and communities across the world.

Here is how the common reference framework could benefit people in concrete ways:

With the proper foundations in place, the common reference framework will be a catalyst for greater standardization of DPI metrics, allowing us to better gauge how DPI tools and solutions are affecting people across the world. And, most importantly, promoting a shared understanding around DPI’s impact can help the global community overcome key barriers – and maximize benefits – for individuals, communities, and society.

If you would like to be involved in these crucial efforts, message us at info@dial.global

We could not have undertaken this year-long effort to understand – and address – the barriers to improving the state of DPI metrics without the help of our partners. We would like to thank them for their collaboration, guidance, and input throughout this collaborative research process – and their support going forward.