Monitoring of KPIs can occur at multiple levels. For example, a KPI on lead time should be tracked at each country office level, but Supply Division should also track the overall data so they can maintain a comprehensive view of the supply chain, including identifying important differences across countries or regions.
It is important to note that the success of a KPI is predicated on the availability of reliable data. The descriptions below will note where existing supply chain data might be used (or improved) for analysis. Some KPIs, however, will require new information from partners so UNICEF should take steps to incorporate data reporting requirements into future contracts with relevant supply chain partners.
Lead Time Key Performance Indicators
Overall Lead Time KPI: Overall lead time is an aggregate of three component lead times: production lead time, transportation lead time, and in-country distribution lead time. Overall lead time is an important component of RUTF supply chain as it guides order placement decisions as well as highlights areas that cause delay in the eventual distribution. Reduction in overall lead time not only leads to better delivery performance but also indirectly benefits forecasting and planning functions.
Production Lead Time KPI: This KPI measures the length of time (in days) to produce an order of RUTF. It begins when the order is received at the producer from Supply Division, and ends when the RUTF is ready for pickup by the global logistics supplier.
Global Transportation Lead Time KPI: Global transportation time represents how long it takes (in days) for an order of RUTF to travel from point of export (i.e., when it is picked up by a global logistics supplier) to port of arrival. It includes time spent at any points of transshipment, as well as obtaining necessary documents for customs clearance at the port of departure and port of arrival.
In-Country Transportation Lead Time KPI: In-country transportation time measures the number of days from when an order clears customs at the port of arrival to when it is picked up or delivered to an implementing partner. This includes transportation time to the UNICEF country or district warehouse, as well as time spent in storage and any relevant transport time to an implementing partner’s storage facility.
Warehousing Lead Time KPI: This KPI would measure the amount of time that an order of RUTF would spend in each of the UNICEF warehouses within the country from the time the order enters a country to the time it reaches its final point of delivery.
TAD Delays KPI: This KPI would measure two dimensions of target arrival date (TAD) delays: first, how many orders for RUTF are amended to extend the TAD, as a percentage of all orders amended in that time frame; and second, how the TADs requested by COs compare to true arrival dates for orders of RUTF. TAD Delays metric is an important indicator of the expected delivery performance of the supply chain. A high percentage of TAD delays would indicate a need to analyze the reason for these delays as well as help COs plan order placements with more realistic delivery estimates.
Landed Cost Key Performance Indicators
Landed cost refers to the amount of money spent to move RUTF through the supply chain. It is a composite of expenses from procurement, global transportation, warehousing and in-country distribution. Each activity adds cost, so monitoring across the supply chain is necessary to identify where cost is added as well as the financial impact of delays in the supply chain. The project team recommends that the overall landed cost KPI be included in the first stage of monitoring.
Overall Landed Cost KPI: Overall landed cost is a measure of the cost incurred in the procurement and transportation of a metric ton of RUTF from the producer to the final point of distribution. Thus, it is an aggregate of four component costs: production cost, global transportation cost, in-country distribution cost and warehousing cost.
Procurement Costs KPI: This KPI monitors how the cost paid per metric ton for RUTF (in US dollars) compares to historic cost data. This would be calculated as total cost per net weight, in metric tons.. In addition to using raw currency values in this KPI, it should include a second dimension that standardizes the price against exchange rate fluctuations.
Global Transportation Cost KPI: The Global Transportation Cost KPI measures the total cost per metric ton of RUTF (in US dollars) to transport an order of RUTF from point of export to port of entry, whether by sea freight, over-land transport, or air shipment. This KPI compares transportation cost per method over time (in other words, sea freight expenditures per MT from one annual quarter to another) as well as a cross-sectional comparison across methods (within an annual quarter, the price of shipping via air, sea or land). It would also include a metric to assess global transportation cost as a percentage of total landed cost for RUTF orders per time period.
In-Country Transportation Costs KPI: This KPI quantifies the cost in US dollars for transporting orders of RUTF from the port of arrival to UNICEF warehouses at the country or district level. It includes multiple dimensions: first, a historic comparison of standardized transportation costs (for example, the cost per kilometer for truck transport in Kenya in January 2008 compared to January 2009); second, an assessment of real variance within a period (e.g., the range of costs for truck transport in Somalia between October and December 2008); and lastly, a percentage of total landed cost per quarter for in-country transportation.
Warehousing Cost KPI: This indicator assesses the total cost (in US dollars) required to store RUTF at each UNICEF warehouse (country and district level) per length of time. UNICEF COs have LTAs with local warehousing facilities that indicate parameters for pricing. This KPI would measure how much it truly costs to store RUTF at those facilities. Similar to other landed cost KPIs, it should include several dimensions, including what percentage of total landed cost was allocated to warehousing for a fixed length of time and a variance measure for warehouse costs in that period.
Quality Key Performance Indicators
There are two categories of Quality KPIs: quality assurance related to the product (referred to as product quality) and quality of information flows (referred to as information quality). Product quality KPIs help to ensure the efficacy and safety of RUTF. Information quality assesses the accuracy and transparency of data in the supply chain.
Product Quality – Regulatory Quality KPI: The product Regulatory Quality KPI would assess progress in meeting regulatory food safety requirements to keep dangerous contaminants from polluting RUTF. It would measure what percentage of raw ingredients and of finished RUTF product were discarded due to harmful levels of contaminants.
Product Quality – Nutritional Quality KPI: The RUTF product must meet certain nutritional requirements (as outlined in the 2007 joint statement on Community-Based Management of Severe Acute Malnutrition and the inter-agency guidance document on Quality Assurance Standards for Therapeutic Foods).
Product Quality – Wastage KPI: This KPI would monitor what percentage of RUTF is wasted, from the point that production is completed until the RUTF leaves the UNICEF country warehouse, due to damage, theft, loss or expiration.
Information Quality – Customer Service Response Time KPI: The Customer Service Response Time KPI would measure how many days elapsed from when a complaint about the supply chain is launched to when it is processed. This KPI is already in use by UNICEF and applies to general operations across the organization, including the RUTF supply chain.
Information Quality – Exceptions Handling KPI: This KPI has two dimensions: it will measure the speed with which an exception is communicated to relevant parties (in days from when the information is received to when it is relayed), as well as whether or not the information was communicated to all concerned parties (as a dichotomous yes/no).
Information Quality – Forecasting Information KPI: This KPI already exists for other UNICEF products that rely heavily on forecasting such as bed nets, vaccines, ARVs and vitamin A, and it should be expanded to include RUTF. It will measure how accurate demand forecasts were in estimating orders of RUTF, as a comparison between projected demand and actual order volume.
Access Key Performance Indicators
Access quantifies both program coverage of malnourished children and the effectiveness of the treatment. The implementation of these Access KPIs requires a greater mobilization of resources than other proposed indicators. Data on lead time and landed cost are largely already collected by UNICEF, so the implementation of those KPIs requires little more than aggregating, analyzing and reporting data. The data for these Access KPIs however are not currently monitored in supply chain databases. The necessary data exist, but they must be collected from partners in the field. Effective exchange of information between CTC programs and COs may require additional staff training to ensure that the correct data are being collected and aggregated at each office. Staff may also need training in how to harvest data from CTC’s databases and how to transfer that data to UNICEF databases. The investment in this training will pay off through more effective monitoring of program effectiveness. It is recommended that the Access KPIs be included in the first stage of monitoring because they help determine if the overarching goal of the supply chain—to meet the needs of severely malnourished children—is being met.
It is also very important to note that these KPIs measure more than just the performance of the RUTF supply chain. The timely, cost-effective and efficient supply of materials is just one determinant of program coverage and efficacy. There are a host of programmatic issues that are also highly significant for access performance.
Coverage KPI: There are two key areas in the coverage KPI. First, the KPI measures how many severely malnourished children are served by a nutrition feeding program, as a percentage of the whole. This indicates overall nutrition program coverage for a certain area, and when compared with the second coverage dimension–how many severely malnourished children are actively in treatment with RUTF–indicates whether changes in RUTF demand are based on a fall in need or if other supplementary foods such as RUSF are being substituted for RUTF. On its own the second dimension will provide information on the demand increase of RUTF.
Effectiveness KPI: The effectiveness KPI looks at readmission and cure rates for CTC programs. Readmission would be calculated as the percentage of children discharged from CTC program that relapsed and required retreatment, and cure rates would monitor the percentage of children who were cured and discharged from the program. The effectiveness KPIs, when paired with coverage KPIs, provide a more detailed picture of treatment programs’ impact. A nutrition program might have near-complete coverage over a targeted area, but if many of the children are being readmitted, this sheds insight on the program’s real success. Over time, these might be expanded to also include mortality rates, or impact on health status or DALYs. This would require additional data collection and analysis, but would significantly enrich UNICEF’s understanding of the true impact of RUTF on treating childhood SAM.