The evaluation explored whether Gavi and its partners undertook the right activities to bring about the outcomes sought by the Strategy. The evaluation assessed three priority areas: design, implementation and results achieved as well as the sustainability of the Strategy. Overall, the evaluation maintained a strategic perspective, as opposed to being a detailed monitoring exercise.
Methods
The evaluation used a retrospective and utilisation-focused approach. The evaluation methodology comprised a Theory of Change (ToC) based approach, with use of mixed methods including document review, data analysis, stakeholder consultations (global and country), key vaccine market analysis and counterfactual analysis.
The full list of planned country consultations was not conducted due to reduced stakeholder availability on account of COVID-19. However, this was not viewed as a limitation per se as a large number of interviews were conducted for the country perspective in relation to the scope and objectives of the evaluation.
Main findings and conclusions
Overall, the Gavi Supply and Procurement Strategy 2016-20 has been very relevant, appropriate, and significant in the context of the evolving/maturing market shaping role of Gavi and the supply context. Building on the Alliance partners’ long-term work, Gavi has contributed to improved health across a number of vaccine markets. A number of markets have also seen improvements in supply security, though this has not been consistent. Despite these achievements, Gavi is likely to miss its overall target of achieving moderate to high levels of health across six vaccine markets. In addition, while the achievements related to healthier markets have been significant, more work is needed to ensure that such markets benefit from “true competition”. Some markets have also experienced significant price declines, particularly pentavalent, over the 2016-20 period, although the long-term impact of these price reductions on market health has divided opinion. Other markets have seen either prices remaining relatively stable or increasing, reflecting the challenging supply situation and also Gavi’s appropriate balancing of trade-offs beyond an exclusive price focus. More generally, activities to support innovation have worked well with several improved vaccine products receiving WHO Prequalification and being procured by the Alliance. At the global level, stakeholders have praised Gavi for its improved information sharing and coordination on market shaping.
Furthermore, recognising where Gavi markets are today as well as wider external developments, Gavi is well-positioned at this stage to consider the following for the 2021-2025 strategic period:
- There is a need to more actively consider demand-side issues in the 2021-2025 strategic period
- Related to the above, and while considered of much value by the Secretariat, partners and broader community involved in market shaping, there are specific suggestions for an improved second iteration of the Healthy Market Framework (HMF) for the 2021-2025 strategic period
- More widely, several vaccine and Cold Chain Equipment (CCE) markets continue to need active market shaping interventions to support improved health and sustainability over time.
The evaluation presents the following recommendations:
Strategy design & implementation
- Build up the Strategy to be truly a ‘market’ strategy, reflecting both demand and supply aspects
- Long Term considerations should be a guiding principle across all aspects of the Strategy, including planning for vaccine and non-vaccine markets and the operationalisation of the Vaccine Innovation Prioritisation Strategy (VIPS) initiative
- Adopt a more consolidated, joint-up and long-term approach to innovations in the next strategy
- Integrate approaches within the strategy that more closely consider the wider ecosystem within which Gavi’s market shaping work functions
- Incorporate key updates to the next iteration of the HFM
- Incorporate suggestions for improvements in the development of roadmaps
- Consider additional processes and metrics to improve the monitoring and evaluation of the activities and results of the Strategy.
Country capacity building & coordination
- Work with wider Secretariat teams and partners to more effectively build country understanding of and ability to input into the Alliance’s market shaping work, especially for transition countries.
Global Partnerships & Coordination
- Build on current successes in partnerships with key stakeholders, while expanding coordination with other market shaping stakeholders where relevant
- Move away from approaching vaccines as a vertical intervention, with better coordination with other global partners on key cross-cutting issues particularly with regards to the challenges posed by country regulatory requirements.
