DIAL’s goal for this “midline” evaluation is to further our organizational learning and stay connected with the needs of the digital ecosystem.
For the past five years, the Digital Impact Alliance (DIAL) has taken a human-centered design approach to our programming, including building, testing and conducting research with our partners, making data-driven adaptations, and sharing our learnings with actors in the larger digital ecosystem (NGOs, donors, governments, mobile networks operators, technology specialists). This approach required our Monitoring, Evaluation, and Learning (MEL) systems and frameworks to be flexible to accommodate adaptation, and rigorous to ensure that we are capturing high-quality data that can speak to our impact.
Last year, we shared the results of our Global Digital Ecosystem Study, which served as both our baseline evaluation and a snapshot of the state of the digital ecosystem. I shared our findings and some of the challenges of the study, including the struggle of finding common language that resonated with all of the diverse actors within the digital ecosystem. In the coming months, we will be conducting a follow-up evaluation that will seek to understand what has changed at the systems-level as result of DIAL and others’ investments and capture continuing challenges to reaching greater digital inclusion to support the achievement of the Sustainable Development Goals.
In the meantime, here are some early findings from an internal evaluation that the DIAL team recently completed. DIAL’s goal for this “midline” evaluation is to further our organizational learning and stay connected with the needs of the digital ecosystem. To do this, we interviewed and surveyed our partners and clients to collect feedback on our performance, identify early signs of impact, and document lessons to be shared.
Here are a few of our main findings:
- Convening communities can help build bridges to new partnerships and understanding. In 2019, DIAL hosted two workshops in Malawi and Sierra Leone. As part of these sessions, we invited the development community and the mobile network operators to voice the challenges they had experienced when attempting to work with each other. In our follow up, we heard consistently about the importance of creating a space to break down barriers. A tech specialist shared, “the main effect the event has had on us is the fact that we initially were really not looking at NGOs… So after the event we realized that there’s so much that can be done within that space.” Additionally, organizations were able to consider how to work together within their own sector. An NGO representative acknowledged that “one of the things we got to realize is that we can use economies of scale in terms of other stakeholders…most of the NGOs in Malawi are working with the same sectors, doing the same thing. Everybody’s triplicating. Therefore, working together to get the best out of the situation as a group instead of individually.” DIAL also received positive feedback from members of Digital Donors Anonymous, the community of now over 25 donor organizations, ranging from major bilateral aid agencies to small private foundations, that convenes virtually to share best practices around integrating digital into their strategies.
- More organizations than ever are living the Principles for Digital Development. As made evident by our recent celebration of the 200th endorser, the Digital Principles continue to gain traction as an industry standard for designing and deploying digital technology. In 2019 we were able to document 23 organizations who have integrated the Principles for Digital Development into their procurement and design processes. Most notably, several donors, including DFID, USAID, Enabel, and Foundation Botnar, are increasingly using the Digital Principles as criteria against which to award new funding. NGOs are responding accordingly by endorsing the Principles in greater numbers and thoughtfully considering how to make them core tenets of their work. “We believe that having [the] Principles with broad-based support is a great advocacy tool for our work with governments on improving health information systems. We believe in the content of the Principles themselves and this is a good tool for guiding our own work.”
- More evidence around impact is needed to spur greater collective investments in digital. One of the key indicators that we use to measure movement toward greater coordination and efficiency within the digital ecosystem is the number of new collective investments that result from DIAL’s work. This includes joint funding, shared investments, partnerships, or other examples of collaboration among actors, as compared to continued duplication of investments and “pilotisis”. In 2019 we counted ten new collective investments, ranging from new funding in open source software solutions to cross-governmental partnerships. But why isn’t this number higher? Partly it’s because change takes time, particularly when it comes to procurement systems. We also heard consistently from the ecosystem, that what was really needed was “more evidence of what works” in easily digestible formats. As one donor said, “In our experience, detailed case studies of the Principles in action really speak to staff and suppliers.” Since 2017, DIAL has published 24 reports of our operational research findings on topics ranging from using mobile network data for development programs to increasing women’s involvement in open source software and 38 case studies about the Principles in Action. To be responsive to the expressed needs to the ecosystem, we are now working to repackage some of our most actionable and important findings, as well as undertaking new research to develop a methodology to calculate the rate of return of investing in digital. We also want to encourage all our partners and friends in the ecosystem to publish their findings.
We are hopeful and excited about the future of DIAL’s work, but obstacles remain, with respondents mentioning topics such as connectivity issues, lack of design for digital inclusion, the existence of affordable and sustainable business models, and the lack of digital/data literacy. For more insights, we will be publishing our full report of the “midline” evaluation in early 2020. So, in the spirit of Design with User, we want to know:
- Are you surprised by any of our findings and lessons?
- What do you want to learn more about?
- What is the right format to share these findings (e.g. written report, website, slide deck, etc.)?
Thanks for your feedback and let’s keep learning together! Email mel@dial.global for comments or questions.
