Performance Based Logistics (PBL) Metrics - Thresholds vs. Objectives
PBL metrics need to include both thresholds and objectives as a part of an incentive approach. In general, thresholds represent the minimum acceptable operational value below which the utility of the system becomes questionable.
For performance, a threshold represents a minimum acceptable value, while for schedule and cost parameters, thresholds would normally represent maximum allowable values. The failure to attain program thresholds may degrade system performance, delay the program (possibly impacting related programs or systems), or make the program too costly. The failure to attain program thresholds, therefore, places the overall affordability of the program and/or the capability provided by the system into question. Objectives represent the desired operational goal associated with a performance attribute beyond which any gain in utility does not warrant additional expenditure. Generally, the objective value is an operationally significant increment above the threshold. (An objective value may be the same as the threshold when an operationally significant increment above the threshold is not useful.)
Starting with Warfighter Requirements...
Like all sustainment metrics, PBL metrics start with Warfighter requirements set by the operational commands or Service requirements offices through the Joint Capabilities Integration Development System (JCIDS) process. The Joint Requirements Oversight Council (JROC) requires a sustainment Key Performance Parameter (KPP) for all Acquisition Category (ACAT) I and select ACAT II programs. The Sustainment KPP consists of two elements: Materiel Availability (Am) and Operational Availability (Ao). The JROC also requires three sustainment Key System Attributes or Additional Performance Attributes of: Maintainability (M), Reliability (R), and Operating and Support (O&S) Cost. Formalized in the JCIDS requirements documents, they may be further articulated in the arrangement between the Program Manager (PM) and the Warfighter command(s).
After the top level Sustainment KPP and supporting KSAs/APAs are established, the PM/Product Support Manager (PSM) then needs to coordinate with Warfighter representatives to ensure product support requirements are identified/documented and threshold and objective values are established/updated appropriate to each program’s product support strategy. Many PBL arrangements are executed at the subsystem or component level, so this usually means that system-level KPPs and KSAs/APAs will need to be decomposed to lower-level metrics appropriate for the level of responsibility and risk assigned to the Product Support Integrator(s) (PSI) and Product Support Provider(s) (PSP). These lower level metrics are typically the ones that will be included in the PBL arrangement, but they must always be linked to the overall system-level requirements. It is sometimes difficult to link the availability requirements for an individual subsystem to the platform-level requirements, but with the right data (i.e., data that indicate how the subsystem is impacting system availability) these linkages can be derived.
Tailoring
One of the most critical elements of a PBL strategy is the tailoring of metrics to the operational role of the system and ensuring synchronization of the metrics with the scope of responsibility of the support provider. The level (system, subsystem or component) and specifics of the arrangement will dictate whether to use top-level outcome metrics, lower-tier metrics or both. The differences between the top-level performance outcome and the metric(s) included in the arrangement are included below:
- Performance Outcome - Requirement (KPP and KSAs/APAs typically stated by the Warfighter)
- Performance Measure - Typically a mathematical equation (e.g., “miles per gallon” or “cost per mile”)
- Performance Metric - Measure with unit and conditions (e.g., “average # miles in jungle terrain”)
Once the level and specifics have been determined, the application of metric thresholds and objectives should be constructed to encourage performance improvement, effectiveness, efficiency, and innovation. There is no perfect metric, but selecting an appropriate complementary set of metrics with appropriate baselines and goals will help promote the desired behavior and outcome while minimizing unintended consequences. An effective metric construct will ensure PSI and PSP activities are aligned with the Warfighter mission, contribute to meeting Warfighter requirements, deliver an on-time, quality product, and reduce (or avoid) cost.
Given today's Contested Logistics Environment and the need to be responsive to changing operational conditions to meet Combatant Commander needs to flex (quantity, type, and location) and surge (accelerate or increase) support where needed, PBL metrics should include, where appropriate, measures of delivering support in a manner capable of meeting these requirements.
Listed below are examples that might be used in a PBL arrangement both during a specified period and over multiple time periods:
- Measurement during a period
- Time (delivery within 36 hours in Continental United States (CONUS))
- Number (minimum of six Ready for Issue (RFI) assets from the depot per month)
- Percentage (5% Not Mission Capable Supply (NMCS))
- Improvement over multiple periods
- Product (reduce quality defects by 3% each quarter)
- Process (increase efficiency 10% over 12 months)
- Cost (reduce O&S cost from previous fiscal year)
A sound performance metric strategy is vital to the success of a PBL arrangement. The Government needs insight into program performance to determine compliance with performance requirements and level of mission success, and they need feedback to measure this performance over time against the slated thresholds and objectives.
Supply Chain Focus
One important area to gather measurement data and incentivize performance is related to supply chain activities, as this is associated with key performance indicators such as Am, R, and O&S costs. As a reminder, the PSM is always responsible for the overall performance of the product support solution and will apply Warfighter relevant metrics to monitor its performance. Metrics assigned to the PSI or PSP reflect the responsibilities assigned to them, and they should reflect outcomes that are within the PSI/PSP’s ability to influence. The selected metrics should be measurable and manageable and map back to the higher-level program metrics.
Following with our supply chain example, an appropriate metric for a PSP may be supply material availability (SMA) or logistics/supply response time (LRT/SRT). Both of these metrics should have threshold and objective values in order to incentivize supply chain improvement over the life of the PBL arrangement.
Incentives
It should be noted, however, that including thresholds and objectives won’t, in and of itself, incentivize improvement. With respect to PBL arrangements, incentives can be a term or condition that encourages the PSI/PSP to deliver the desired performance outcome (for aspects of performance that are within their control). The incentive may be related to contract type, contract length, or incentive fees (or penalties). A Firm Fixed Price (FFP) PBL contract provides the strongest incentive for the provider to control costs, although Fixed Price Incentive Firm (FPIF) are also often used to incentivize specific schedule or performance goals. Another incentive used to leverage desired positive results is the ability to receive extensions to the duration of the contract via award terms for good performance. This is seen as an attractive incentive to commercial support providers because it provides stability to the provider’s projected work orders and adds shareholder value.
Incentives that focus on profit may not be applicable for organic PSPs, but increased percentage of available workload, promotions, bonuses, and spot awards are all possible incentives along with the desire to positively impact Warfighter outcomes. And when these incentives are tied to realistic cost, schedule and performance thresholds and objectives, the impact on desired behavior can be an extremely compelling motivator.
Click here to view a video on PBL Best Practices, featuring perspectives by Ms Lisa P. Smith, DASD(PS), and two Service practitioners describing successful PBL arrangements at the subsystem/component and system (platform) level.