Life Cycle Sustainment Outcome Metrics

(Definition Note: Technology Development is now Technology Maturation and Risk Reduction).

The JCIDS Manual defines a Key Performance Parameter (KPP) as a performance attribute of a system considered critical or essential to the development of an effective military capability. Failure of a system to meet a validated KPP threshold value triggers a review by the validation authority and evaluation of operational risk and/or military utility of the associated system(s) if KPP threshold values are not met. The review may result in validation of an updated KPP threshold value, modification of production increments, or recommendation for program cancellation. The JCIDS Manual continues describing Key System Attributes (KSAs) as performance attributes of a system considered important to achieving a balanced solution/approach to a system, but not critical enough to be designated a KPP.

Sustainment outcome metrics are briefed to the Milestone Decision Authority (MDA) via the sustainment quad chart (see DoD PSM Guidebook, Appendix C) and to the Defense Acquisition Board (DAB), which review Aquisition Category ID and special interest programs prior to transition from one acquisition phase to the next. The four quadrants of the quad chart are product support strategy (PSS), sustainment schedule, metrics data, and Operating and Support (O&S) cost data. Before signing the Acquisition Decision Memorandum (ADM) approving a program's transition to the next life cycle phase, the MDA evaluates the degree to which the system is sustainable.  The life cycle sustainment outcome metrics are the prime indicators of that potential. In addition to the PSM Guidebook, the DoD O&S Cost Management Guidebook discusses the sustainment quad chart in depth.

The JCIDS Manual and DoD Instruction 5000.91, Product Support Management for the Adaptive Acquisition Framework (para. 4.8), define the following metrics:

The Sustainment KPP, consisting of the following parts:

• Materiel Availability (Number of Operational End Items / Total Population): the percentage of the total inventory of a system operationally capable, based on materiel condition, of performing an assigned mission.
• Operational Availability (Uptime/(Uptime + Downtime)): the percentage of time that a system or group of systems within a unit are operationally capable of performing an assigned mission.

and the following KSAs or Additional Performance Attributes (APAs):

• Reliability: probability that the system will perform without failure over a specific interval, under specified conditions. Reliability shall be sufficient to support the warfighting requirements, within expected operating environments. More than one reliability metric may be specified, as KSAs and/or APAs, for a system as appropriate.
  • For continuous use systems (such as an aircraft), reliability should be measured in terms of its primary usage metric (such as operating hours, miles or flight hours).
  • For discrete systems (such as a single-use munitions), reliability should be measured as a probability.
  • Mission Reliability. The measure of the ability of an item to perform its required function for the duration of a specified mission profile, defined as the probability that the system will not fail to complete the mission, considering all possible redundant modes of operation.
  • Logistics Reliability. The measure of the ability of an item to operate without placing a demand on the logistics support structure for repair or adjustment, including all failures to the system and maintenance demand as a result of system operations.
• Maintainability: The measure of the ability of the system to be brought back to a readiness status and state of normal function. (Added by the 2018 update to the JCIDS Manual.).  Consider corrective maintenance, maintenance burden, and Built in Test (BIT).
• Operating & Support (O&S) Cost: provides balance to the sustainment solution by ensuring the total O&S costs across the projected life cycle associated with availability and reliability are considered in making decisions.
  • The O&S Cost KSA/APA is to be computed using constant year dollars.
  • For consistency and to capitalize on other efforts in this area, this KSA/APA will be calculated as the "Operational" Cost metric.
  • Energy costs shall be included in O&S cost, and will use Office of the Undersecretary of Defense (Comptroller) published inflation and escalation rates for fuel to forecast and normalize energy
  • All O&S costs are to be included regardless of funding source or management control.

The reason for two methods of measuring availability, Materiel and Operational, is often questioned. Materiel availability is a very broad measurement of the total inventory of systems the combatant commander has at their disposal to prosecute a given mission. Similarly, when Congress asks about weapon system availability they aren't concerned with detailed, hour by hour status given by the operational measurement. These high-level entities only want the "big picture" of materiel availability. Unit commanders, on the other hand, need the increased granularity of operational availability to manage their readiness. So, the reason for two measurements depends heavily on who's asking the question.

The recently published update to DoDI 3110.05, Sustainment Health Metrics in Support of Materiel Availability, provides slightly different methods of calculating operational availability (Ao) and materiel availability (Am), which it describes as the two superordinate metrics that assess the effectiveness of the DoD sustainment enterprise. Am is still based on the Total Active Inventory (TAI), but in the DoDI is calculated based on available time divided by "possible time" in lieu of the JCIDS Manual calculation based on the number of available end items over TAI. For Ao, the calculation is based on time and applies to the Primary Mission Asset Inventory (PMAI), which if interpreted as the systems within an operational unit, is very much like the JCIDS Manual. However, "uptime" is renamed "available time" and "uptime + downtime" becomes "possible time."  As additional policy and guidance updates are published, the differences between these formulas should be resolved. In addition to Ao and Am, DoDI 3110.05 introduces a new metric, Cost Per Day of Availability (C/DA), which is a measure for PMAI assets and includes costs related to but not identical to the O&S Cost KSA or APA. C/DA is described as the superordinate metric that assesses the efficiency of the DoD sustainment enterprise.

The KPP, KSAs, and APAs are used as triggers to conduct further investigation and analysis into drivers of those metrics, develop "should cost targets," and develop strategies for improving reliability, availability, & maintainability at reduced cost (including the use of outcome- and performance-based life cycle product support (PBL) product support strategies and product support arrangements (PSA). Keep in mind these metrics are envisioned to be integrated, and very likely will include sub-tier supporting metrics. The O&S cost KSA/APA, for example, provides balance; sustainment solutions cannot be availability and reliability "at any cost." They also facilitate both product and process development and improvement.

Sustainment metrics are forecasted early in a program's life cycle, beginning at Milestone A with the Reliability, Availability, Maintainability - Cost (RAM-C) Rationale Report. This report assists combatant commanders, combat developers, program managers, engineers and life cycle logisticians in designing reliability, availability and maintainability into the system without losing focus on affordability. These metrics, forecasted early, drive design initiatives that ultimately determine how the system will be supported.