As most readers are no doubt already aware, the Undersecretary of Defense for Acquisition, Technology, and Logistics (AT&L) yesterday released the latest annual report on the "Performance of the Defense Acquisition System”. According to the Undersecretary’s forward, "...this fourth report in the series continues my long-term effort to bring data-driven decision making to acquisition policy. This report demonstrates that the Department of Defense (DoD) is making continuing progress in improving acquisition. The overall series presents strong evidence that the DoD has moved—and is moving—in the right direction with regard to the cost, schedule, and quality of the products we deliver. There is, of course, much more that can be done to improve defense acquisition, but with the 5-year moving average of cost growth on our largest and highest-risk programs at a 30-year low, it is hard to argue that we are not moving in the right direction.”

What follows is admittedly more detailed than standard blog etiquette normally permits, however there are numerous highlights for the life cycle logistics and product support community to ponder, including the following (quoted verbatim):

·         Principles for Improving Defense Acquisition (page v)

o    People matter most; we can never be too professional or too competent.

o    Continuous improvement will be more effective than radical change.

o    Data should drive policy.

o    Critical thinking is necessary for success; fixed rules are too constraining.

o    Controlling life-cycle cost is one of our jobs; staying on budget isn’t enough.

o    Incentives work—we get what we reward.

o    Competition and the threat of competition are the most effective incentives.

o    Defense acquisition is a team sport.

o    Our technological superiority is at risk and we must respond.

o    We should have the courage to challenge bad policy.

·         Mythbusters (page xv)

o    Myth: Swings in O&S cost estimates indicate poor program management.

o    Reality: The dynamics of cost estimates indicate that O&S costs appear to be heavily driven by external inflation factors. Analysis shows that the recent dynamics of program O&S costs estimated during acquisition correlate with the dynamics of labor, health-care, fuel, and maintenance costs. While this aligns with intuition, it also indicates that O&S cost increases involve both factors that the acquisition system cannot control (e.g., wages, health-care costs, and fuel costs) as well as some that can, in part be controlled (e.g., system reliability, fuel efficiency, and ease of maintenance). Operational tempo also affects O&S costs through many of these factors (e.g., the amount of fuel consumed and maintenance costs), and changes in forecasted tempo will affect O&S costs independent of both inflation and weapon system performance. Thus, while the acquisition system needs continued attention to the levers it can control (with full knowledge that their effects often will not be seen for decades), stakeholders need to recognize the strong influence of other factors on O&S costs. (See discussion starting on p. 95.)

·         Insights for Current and Future Leadership (page xviii)

o  We need a metric for the portion of O&S costs related solely to weapon system design and performance. Analysis shows that many of the factors that correlate with growth of O&S cost estimates reported during acquisition are outside the control of the acquisition system (i.e., wages, health-care costs, and fuel prices). Current O&S metrics do not separate acquisition program effects from these external effects. However, a new metric could be developed to measure these internal program effects by holding the external variables constant from MS B forward (solely for purposes of comparison) so that the effects solely from the acquisition system are revealed.

·         Program Level Insights (p xix)

o  Don’t neglect suitability (reliability, maintainability, etc.) in pursuing system performance. Operational tests show that major programs are often effective when they tested as operationally suitable, but the converse is not true (see p. 20). This correlation by itself does not prove causality, but it reinforces the logic that the so-called “-illities” (e.g., interoperability, availability, maintainability, reliability) are important to achieving the mission. For example, well-engineered systems that address suitability factors are probably also better positioned to be effective. Also, no matter its features, a weapon system may not serve its function if it is unreliable and unavailable to the warfighter.

o  Don’t neglect O&S cost implications in early system requirements and design. Analysis shows that many of the factors that correlate with growth of SAR O&S cost estimates (i.e., wages, health-care costs, and fuel prices) are outside of program management control. While PMs cannot control these external factors, they can affect fuel efficiency and maintenance costs (e.g., system reliability, ease of maintenance, and repair automation). Usually, these aspects must be addressed very early in the system’s design, so don’t neglect them in early program planning and management. That is why the new affordability process sets goals and caps on life-cycle costs early in the program’s life (e.g., at the point of the Materiel Development Decision (MDD) and MS A, when bigger design changes can be made). Don’t neglect them just because you cannot control the external factors and uncertainties remain.

o  Use fixed-price contracting judiciously in development (p xix). In our updated guidance on contract incentives (Director of Defense Procurement and Acquisition Policy, data from prior annual reports and experience indicate using fixed-price contracts in development can be very risky and counterproductive, while incentive contracts can yield good cost control at lower risk and lower prices. All of the following five criteria should generally be met before using fixed-price contracts in development:

-       Requirements are stable

-       Technologies are mature

-       The contractor is experienced

-       The contractor can absorb overruns

-       The contractor has a business case for absorbing any overruns that occur

·         Institutional & Policy Changes (p xxii)

o    O&S Cost Management. Published in February 2016, a new O&S Cost Management Guidebook for PMs and product-support managers provides tools and best practices for O&S cost analyses to inform early life-cycle decisions, effect reliability trades, and identify Should-Cost initiatives having the greatest effect on future O&S costs.

o    Performance-Based Logistics. A March 2016 update of our Performance-Based Logistics (PBL) Guidebook reviews common myths about PBLs, adds new guidance regarding intellectual property issues, and continues to provide best practices, selection criteria for when PBLs are appropriate, and practical examples to maximize successful outcomes (Assistant Secretary of Defense for Logistics and Materiel Readiness, 2016).

·         Other Observations (p xlvi)

o    Operationally suitable programs are fairly likely to be operationally effective, too. An examination of the coincidence between DOT&E test results shows that major programs often tested as operationally effective when they tested as operationally suitable, but the converse was not true. In other words, we have a number of effective systems that have suitability issues (e.g., safety, interoperability, availability, maintainability, and reliability), but systems that address these suitability issues tended to also be effective against threats. (See detailed analysis and discussion starting on p. 20.)

o    Labor, health-care, fuel, and maintenance costs appear to drive O&S cost estimates. DoD-wide, these factors correlate closely with O&S cost estimate changes reported in 2001–2014 SARs after adjusting for inflation. With some differences, these are also the dominant correlate types of factors for growth in O&S estimates by DoD Component and commodity. In nearly all cases, growth in system service life or quantity did not correlate with growth in O&S cost estimates. Thus, dynamics in these factors appear to be important for controlling O&S costs, and we found these dynamics can cause estimates to vary significantly from year to year (i.e., actual annual growths varied widely from about +/−15 percent since 2001). (See detailed analysis and discussion starting on p. 95.)

·         Superior Supplier Incentive Program (pp 75-80)

o    As part of BBP, the three military departments and the Defense Logistics Agency (DLA) each established a Superior Supplier Incentive Program (SSIP) to incentivize contractor performance through public recognition. The basis for SSIP designations are contract performance assessments reported by the PM (or equivalent) to CPARS (DoD, 2011). Assessments rate the quality, schedule, cost control, management, utilization of small businesses, and regulatory compliance of the contractor’s performance on a specific contract for a specified period. The contractor is allowed to review and comment on each assessment before it is finalized.

·         Sustainment Costs (pp 95-99)

o    While much attention and data focus on the efficient development and procurement of defense systems, O&S costs can outweigh earlier acquisition costs. As a result, we are expanding our analyses to provide insights on how effective the acquisition system is at understanding and controlling total life-cycle costs. This includes understanding the major drivers of O&S cost growth and (in part) our ongoing affordability policy and process.

·         Appendix D Analytic Details: Correlates of O&S Cost Estimates (pp 155-156)