Monitor success of LPV/r pellet rollout by identifying and tracking relevant metrics to evaluate progress for corrective action and post-rollout learning.
Smooth rollout of a new product to the national health care system requires numerous changes to policies, operations, and staff behaviors. The national Ministry of Health (MOH) can steward this transition by monitoring and assessing its progress to inform corrective actions as needed. A monitoring and evaluation plan should complement any initial strategy document by outlining a plan to determine whether the rollout meets objectives and where improvements should be made.
For the duration of the new product rollout, the national MOH and relevant stakeholder partners, such as policy-makers, clinicians, project and commodity funders, implementing partners, civil society organizations, and nongovernmental organizations, should dedicate management resources to monitor progress against timelines, and to implement surveys to capture insights into rollout successes that are not captured through the existing reporting systems. In this toolkit, an example Health Care Worker Pellet Survey [PDF, 101KB] can be used to gain insight into health care workers' and caregivers' experiences with the new formulation.
LPV/r Pellet Considerations
- Develop monitoring and evaluation approaches to gain insight into specific rollout objectives and performance barriers.
- To the extent possible, use existing reporting mechanisms (such as an HMIS or LMIS) to inform monitoring, after making sure that the LPV/r pellets have been added into paper or electronic reporting forms as necessary.
- Consider using existing supervision channels to administer special evaluation surveys.
- Funding should be allocated from the outset for developing any evaluations, analysis, documentation, and dissemination.
- At the time of rollout training, train health care workers to correctly complete any required evaluation reports or tools.
During the initial rollout of the new product, all of your planning assumptions will be "put to the test". These assumptions include:
- Rate of consumption.
- Rate of client uptake of the new product.
- Rate at which health care workers are being trained and are beginning to prescribe and dispense the new product.
- Initial supply requirements, both at the country and facility levels.
To assess assumptions, and address situations as they arise, carefully monitor consumption and stock on hand data during the initial implementation period to ensure that overstocks and stockouts are not occurring.
Strategies for more careful monitoring of logistics data might include:
- More frequent reporting on the new product (e.g., specific monthly reports in a system that normally reports quarterly) to track consumption and stock on hand at the facility level.
- More frequent supervision visits to facilities that are dispensing the new product to verify consumption and stock on hand (and to monitor dispensing practices).
- More frequent physical inventories to closely track rate of issues.
- An emphasis or strong reminder to health care workers and district supervisors who work on ordering about the need to monitor the emergency order point at the facility level.
Because you might find that the situation is changing rapidly, e.g., if clients are being switched to the new product at a faster (or slower) rate than anticipated, you must be prepared to take "emergency" distribution steps to resolve the situation as soon as possible.
Depending on the situation, special distribution steps may include:
- Redistributing products from low-consuming to high-consuming facilities.
- Filling more emergency orders than usual. At the facility level, this should be avoided by redistributing products as above and adjusting national orders accordingly.
- Delaying or reducing the quantities of an already planned order.
- More frequent resupply intervals to respond to rapid rise in consumption.
- Preventing emergency orders: With current global supply constraints, fulfillment of emergency orders from the manufacturer may not be possible and is also costly. Steps should be taken to avoid emergency orders through proactive supply chain management.
- Establishing a regular monitoring committee, if one does not exist already. This committee should monitor progress during rollout (on training, sensitization, procurements, risks and distribution) and inform stakeholders about required action.
- Carefully monitoring observed product consumption and update forecasts and supply plans as necessary—forecasting for new commodities is never 100% accurate, so monitoring observed consumption early can help refine forecasts to help ensure availability and avoid overstocking of product.
- Pipeline monitoring (the regular process of recalculating remaining and anticipated future months of stock at each inventory-holding level in the health system) within LPV/r pellet rollout monitoring should cover both LPV/r pellets and any formulation that it replaces—for example, a slower than anticipated uptake of LPV/r pellets means higher than anticipated consumption of alternative formulations.
- In accordance with the evaluation plan, determine causes of any performance gaps and remedy accordingly.
- Share rollout learnings in a global forum.
The table below presents examples of how to use indicators from monitoring and evaluation activities to inform MOH and partner actions that improve the success of LPV/r pellet rollout.
Example Applications of Monitoring and Evaluation for LPV/r Pellet Rollout
|Management Focus Area||Potential Data Source||Example Metric||Example Management Action|
|Is implementation progress proceeding per timeline?||Routine updates on shipment status and training coverage.||% health care workers trained vs target.||Delay implementation and adjust forecasted consumption to account for identified training delay.|
|Have LPV/r pellets been successfully introduced as a regimen within ART?||Health Care Worker Pellet Questionnaire
|% of caregivers expressing uncertainty about correct dosage as reported by health care workers
% of target weight band adopting pellet formulation.
|Develop additional material to provide to health care workers to address caregiver uncertainty and increase adoption.|
|Is forecasted consumption accurate?||LMIS.||Average monthly consumption.||Adjust forecast and supply plan to account for difference in observed consumption vs forecasted.|
- Health Care Worker Pellet Questionnaire: [PDF, 97KB]
- How to Introduce and Scale Up Sayana Press (DMPA-SC in Uniject) Chapter 10 (PATH 2017): [PDF, 4.8MB] Guidance for monitoring and evaluation of new product rollout, tailored to rollout of a new family planning method but also including generic guidance.
- Assessing the Adoption of Lopinavir/Ritonavir Oral Pellets for HIV‐positive Children in Zimbabwe