Sign In
App icon


Available Now!
Get the App
Click Here to Continue Browser Session   ❯



Contract Award Protest Rulings—Highlights From the GAO Report for 2021 Award Protest Rulings—Highlights From the GAO Report for 20212022-08-31T16:00:00Z,<div class="ExternalClass322E30326A5B4901A0ED14E92B39B621">Every year, the Government Accountability Office (GAO) reports to Congress on its most prevalent reasons for sustaining protests. It also advises Congress if any agency failed to follow GAO’s recommendations. No agency failed to follow GAO recommendations during the last fiscal year (FY), and the GAO sustain rate remained at 15 percent (also the sustain rate for FY 2020).<br> <br> When GAO sustains a protest, it means that the GAO agrees with the protester that the agency violated a procurement law or regulation in a prejudicial way. The GAO will then recommend that the violating agency handle the violation through corrective action. The violating agency is then required to keep GAO informed as to whether it will comply with GAO’s recommendation.<br> <br> (Editor’s Note: The author’s examinations of the GAO rulings have been published annually since the <a href="/library/defense-atl/p/Defense-ATandL---January-February-2018" target="_blank">January–February 2018</a> issue of the <em>Defense Acquisition Magazine</em>’s predecessor, <em>Defense AT&L</em>).<br> <br> The most prevalent reasons GAO sustained protests in FY 2021 were as follows: <ol> <li>Unreasonable technical evaluation</li> <li>Flawed discussions</li> <li>Unreasonable cost or price evaluation</li> <li>Unequal treatment</li> </ol> In the GAO annual report to Congress for FY 2021, example cases for each of the above most prevalent reasons are discussed. A short overview of each follows. <blockquote> <p style="text-align:center;"><br> THE AGENCY ANTICIPATED AWARDING A SINGLE, HYBRID, FIXED-PRICE AND LABORHOUR TASK ORDER WITH A ONE-YEAR BASE PERIOD AND FOUR ONE-YEAR OPTIONS.<br> <a href="" target="_blank"><img alt="tweet" src="/library/defense-atl/DATLFiles/Sept-Oct_2021/tweetbutton.jpg" style="width:8%;border-width:0px;border-style:solid;float:right;" /></a></p> </blockquote> <h2>Unreasonable Technical Evaluation</h2> GAO cited MetroStar Systems, Inc., B-419890, as its example of an agency performing an unreasonable technical evaluation. A protest was brought against the Defense Information Systems Agency (DISA) because of its evaluation of quotes solicited under the General Services Administration Federal Supply Schedule. The agency anticipated awarding a single, hybrid, fixed-price and labor-hour task order with a one-year base period and four one-year options. The vendors were to submit quotes for classified IT systems and secure communications support to operate and maintain the classified voice, video, and data networks for the White House. Vendors were informed through the solicitation that award would be based on a trade-off between price and the technical/management approach (referred to as “technical” hereafter), with technical considered more important than price. There were four technical subfactors for evaluation: <ol> <li>Personnel management</li> <li>Software engineering-data archival system</li> <li>Software engineering-document collaboration capability</li> <li>Network operations</li> </ol> Subfactor 1 was more important than subfactors 2 through 4. Subfactors 2 through 4 were of equal importance.<br> <br> In a trade-off, the government can award to other than the lowest-price offeror when the decision to do so is consistent with the evaluation criteria and the government reasonably determines that the perceived benefits of awarding to the higher-priced offeror warrants the additional price.<br> <br> The agency did not provide an overall adjectival rating for technical. Instead, each subfactor received its own adjectival rating. The most important technical subfactor, personnel management, required vendors to provide a transition-in plan with their quotes. This plan was to show procedures and actions that the contractor would take to staff up and obtain clearances without interrupting or degrading service.<br> <br> Vendors were required to identify and begin onboarding 50 percent of the staff within two weeks and identify and begin onboarding 100 percent of the staff within 60 days (from the date provided in the performance work statement).<br> <br> The agency rated the awardee as “good” for its personnel management approach, which was one level higher than the protester’s rating of “acceptable.” The awardee’s rating on this technical subfactor was the one discriminator making its quote technically superior to the protester’s since it was considered the most important. The awardee’s rating for its personnel management approach was based on one strength it proposed for its transition-in plan to “onboard all staff in the first 30 days and complete knowledge transfer within 60 days.”<br> <br> The agency documentation revealed the source selection authority’s agreement with the evaluation team’s finding that the awardee’s transition-in plan would benefit the government for the following reasons: <ol> <li>It reduced risk.</li> <li>It ensured that all staffing positions would be on-boarded and executing their duties well before the 60 days provided in the performance work statement.</li> <li>It provided a unique approach that exceeded the solicitation requirements by providing a notional schedule, a clear control process designed specifically for the agency and incorporating critical milestones such as knowledge transfer.</li> </ol> The price evaluation, which was less important than the technical evaluation, revealed that the protester’s price was 7 percent lower than that of the awardee. The agency determined that the awardee’s price premium was worth paying because of its “technically superior approach and its associated benefits and lower performance risk.”<br> <br> The protester filed its protest after receiving a brief explanation from the agency regarding its award decision.<br> <br> The protest filed with the GAO asserted, among other things, that the agency misunderstood the awardee’s staffing approach. The GAO did not address every issue raised by the protester but did review the contemporaneous record (the record existing when the award decision was made) and determined that the protester was correct in that the agency did misunderstand the awardee’s staffing approach. The schedule in the awardee’s quote provided that the awardee would complete knowledge transfer within the 30-day transition period but that the awardee did not commit to onboard all staff within those 30 days. The awardee’s approach met the solicitation requirements, but it was unclear whether the evaluation team and the source selection authority would have assigned the strength for the personnel management subfactor to the awardee’s quote without the flawed interpretation that all staff would be onboarded within 30 days.<br> <br> The GAO determined the trade-off evaluation was unreasonable because it was based upon the incorrectly assessed strength. In other words, the misunderstood strength was the sole discriminator in determining the award. GAO found significant prejudice to the protester and subsequently recommended that the agency reevaluate the technical proposals in accordance with the solicitation and make a new source selection decision based on that reevaluation. <h2><img alt="a pen and paperwork" src="/library/defense-atl/DATLFiles/July-Aug2022/defacq-datl_fullbook_julyaug2022_article06_image01.jpg" style="width:100%;margin-left:6px;margin-right:6px;" /><br> <br> Flawed Discussions</h2> GAO cited Ohio KePRO, Inc., B-417836, as an example of an agency conducting flawed discussions. The protest brought in the case was based on a task order award made by the Department of Health and Human Services, Centers for Medicare and Medicaid Services (CMS), for beneficiary oversight and claim review services.<br> <br> The GAO heard a series of protests brought by both the protester and the awardee over this requirement. Those earlier protests involved corrective action by the agency. The GAO can recommend corrective action, or it can be invoked by the agency. Corrective action means that the agency is taking action to resolve the protest. The solicitation was amended as a result of the agency’s last corrective action.<br> <br> The solicitation provided for awarding a 54-month period of performance, single fixed-price task order based on a best-value trade-off with price and non-price factors that were listed by importance. There were three Contract Line Item Numbers (CLINs): CLIN 1 (Higher Weighted Diagnostic Review Group [HWDRG] claim reviews), CLIN 2 (Software Specification Review claim reviews), and CLIN 3 (focused claim reviews). The offerors were required to propose a single price per review for the quantities provided in each CLIN. A chart was provided that included the minimum and maximum number of claims reviews to be performed for each claim type, and offerors were directed to provide their proposed prices per claim review type in the business volume of their proposals.<br> <br> Offerors were expected to propose a firm-fixed-price. The agency stated that it would conduct a price analysis of the business volume (an analysis of the total cost versus the individual cost elements). Offerors were also given a sample chart for including the total price for each claim type. Each chart had a spot for the offeror to enter the price per claim review.<br> <br> Discussions were held after receipt of proposals and final proposal revisions were requested afterward. The awardee’s final proposal provided two prices per claim review for each of the three CLINs. The amounts provided by the awardee were based on certain quantities. This resulted in a material omission as some quantities were not priced.<br> <br> The agency requested that the awardee provide the proposed price for the omitted claim reviews. The awardee submitted a revised proposal including two new versions of its business proposal. One version of the business proposal used tracked changes showing the revisions against the earlier proposal. There also was a second, clean version of the business proposal.<br> <br> The agency determined that awardee would provide the best value because of the higher rating for its technical proposal and its lower price. The protester filed its protest after the award, asserting that the communication was unequal since the awardee was permitted to submit two new versions of its business proposal. The GAO agreed and analyzed this case as a negotiated procurement.<br> <br> The rules in negotiated procurements dictate that an agency cannot treat one offeror more favorably than another offeror. The behavior of the parties indicates whether discussions took place. The GAO’s “acid test” in determining if there were discussions considers whether the agency provided an opportunity for a proposal to be materially changed. The GAO found that the exchange between the agency and the awardee was a discussion because the awardee was allowed to supplement its proposal. The communication was, therefore, improper and amounted to competitive prejudice.<br> <br> The GAO routinely states that competitive prejudice is a required element to be proven in a protest, and any doubt of its existence is resolved in favor of the protester. The competitive prejudice alleged here was that the awardee was given opportunities not provided to the protester. Those opportunities included additional meaningful discussions and another proposal revision. The GAO reasoned that the protester might have addressed any agency concerns and made its pricing more competitive if given the same opportunities as the awardee. The protester might then have had a substantial chance of being awarded the task order. <h2>Unreasonable Cost or Price Evaluation</h2> GAO cited DevTech Sys., Inc., B-418273 as the example case of an agency conducting an unreasonable cost or price evaluation. The GAO found for the protester on the issue of unreasonable cost or price evaluation and on the issue of unequal discussions. The GAO dismissed all of the protester’s other complaints.<br> <br> The United States Agency for International Development (USAID) issued a solicitation that anticipated a cost-plus-fixed-fee task order award for public financial management services in Indo-Pacific countries in support of the Indo-Pacific Opportunity Program.<br> <br> The solicitation provided that the award would be based on cost and five non-cost evaluation (technical) factors. Each factor was assigned a possible number of points but was also listed by descending importance. The technical factors were significantly more important than cost, with cost evaluated for realism, fairness, and reasonableness.<br> <br> While evaluating the vendors’ proposals, the contracting officer conducted exchanges with the awardee and the protester during an earlier corrective action regarding the proposed key personnel. In addition to that exchange, the agency sought what it called a clarification from just the awardee. There was no rationale included in the record to explain why the exchange was made with only the awardee.<br> <br> The agency described the exchange as a clarification; it was seeking confirmation that the awardee’s proposed subcontractors would comply with cost provisions of an underlying indefinite-delivery indefinite-quantity contract. Those cost provisions required that proposed task order rates not exceed the unburdened ceiling daily rates specified in the solicitation. The agency used the compliance confirmation it received from the awardee to find that its proposed rates were no greater than the ceiling daily rates, even though its proposal reflected higher rates. Therefore, the exchange between the agency and the awardee allowed the awardee to address and correct its proposal in an area in which it did not initially comply. This made the exchange between the agency and the awardee a discussion, not a clarification.<br> <br> <img alt="person writing with a pen" src="/library/defense-atl/DATLFiles/July-Aug2022/defacq-datl_fullbook_julyaug2022_article06_image02.jpg" style="width:100%;" /><br> <br> The protester argued that the agency failed to conduct meaningful discussions and that the discussions were unequal. The GAO disagreed with the protester on the meaningful discussion argument because the discussions with the awardee and the protester were limited to confirmation of key personnel availability and/or identification of replacement key personnel. Limiting the scope of discussion to just the issue of key personnel was reasonable. No other revisions to the proposals were permitted. Therefore, the agency was not obligated to have broader discussions on deficiencies and significant weaknesses of the proposals.<br> <br> However, the GAO did agree with the protester that the discussions were unequal. In sustaining the protest on this argument, the GAO found that the discussions were unequal because the exchange between the agency and the awardee was considered a discussion. The agency calling the exchange a clarification was not enough to make it one, even when the information sought only involved seeking a yes or no answer from the awardee. The GAO found prejudice because the protester was not afforded an equal opportunity for discussion. The GAO determined that affording the protester discussions at that time could have improved its chance of receiving the award.<br> <br> The GAO also agreed with the protester on the issue that the agency unreasonably evaluated the realism of the proposed costs. The costs paid to the contractor would be the actual and allowable costs because this was a cost-reimbursable order. A cost realism analysis considers that the proposed costs may be different from the actual costs and determines whether the proposed costs are realistic, considering the work to be performed.<br> <br> The GAO’s review standard in a protest is to determine the reasonableness of the agency’s analysis. The GAO found that adjustments were made to the awardee’s and the protester’s proposed costs after the cost realism evaluation and that the adjustment resulted in the protester’s proposed price being about $9.9 million higher than the awardee’s proposed price. The agency acknowledged that it mistakenly made adjustments to one of the protester’s proposed subcontractors.<br> <br> The protester also challenged other upward agency adjustments. The GAO found for the protester on some of the adjustments because the agency was unable to explain its basis for making them. Therefore, the GAO could not determine if the agency acted reasonably.<br> <br> <img alt="a clipboard" src="/library/defense-atl/DATLFiles/July-Aug2022/defacq-datl_fullbook_julyaug2022_article06_image03.jpg" style="margin-left:6px;margin-right:6px;width:100%;" />The agency’s contemporaneous record did not explain the final cost realism adjustment to the awardee’s proposed level of effort, which adjusted that level of effort to 100 days per year. The proposal showed a realistic requirement of 237 days of work for the awardee’s program management staff.<br> <br> Ultimately, the downward adjustments to the protester’s proposed costs and the upward adjustments to the awardee’s proposed costs reduced the difference between the proposals so that the protester’s evaluated costs were about $5.2 million greater than those of the awardee. The GAO found that, absent the adjustment errors, the protester would have had a substantial chance to be awarded the order. <blockquote> <p style="text-align:center;"><br> A COST REALISM ANALYSIS CONSIDERS THAT THE PROPOSED COSTS MAY BE DIFFERENT FROM THE ACTUAL COSTS<br> <a href="" target="_blank"><img alt="tweet" src="/library/defense-atl/DATLFiles/Sept-Oct_2021/tweetbutton.jpg" style="width:8%;border-width:0px;border-style:solid;float:right;" /></a></p> </blockquote> <h2>Unequal Treatment</h2> <p>GAO cites DigiFlight, Inc., B-419590 as its example case of unequal treatment. This case involved the protest of an award resulting from a solicitation issued by the Department of the Army, Army Contracting Command Redstone, for advisory and assistance services. The solicitation for quotes was limited to only those small business vendors for which the agency had previously established blanket purchase agreements.<br> <br> According to the solicitation, the award was to be made using a best value trade-off of price and two non-price factors: (1) technical expertise and (2) risk mitigation and management. The two non-price factors were equal and were each individually more important than price. The solicitation required that the offerors provide a “robust” risk mitigation and management approach, which required an explanation of how the offeror would mitigate and manage risk in obtaining and retaining qualified personnel, bringing together the right team to perform the performance work statement, and effectively managing the project.<br> The awardee explained in its risk mitigation and management approach for retention of personnel that its high three-year retention rate was 95 percent partly due to the awardee’s collective benefits, training opportunities, level of communication, treatment of its employees, and tuition reimbursement.<br> <br> The protester’s quote in this same area indicated that it had achieved a greater than 96 percent team retention rate by offering competitive, market-based salaries and a comprehensive benefits package. The protester indicated that its retention plan was proactive and included providing relevant training and defined advancement paths. The protester described parts of its benefits package, which included reimbursement for specialized training and academic degrees in addition to company-paid conferences and seminars.<br> <br> <img alt="a person writing with a pen" src="/library/defense-atl/DATLFiles/July-Aug2022/defacq-datl_fullbook_julyaug2022_article06_image04.jpg" style="width:100%;" /><br> <br> The agency assessed a strength to the awardee for its ability to obtain and retain qualified personnel partly because it provided opportunities for professional growth, training opportunities, and tuition reimbursement. The agency documented that the training opportunities and tuition reimbursement facilitated employee growth opportunity, career development, and greater job satisfaction and morale. The awardee’s retention rate was also recognized as demonstrating proven methods to retain qualified employees. The agency also assigned a strength to the awardee because of its low turnover rate and the resulting retention of corporate knowledge and higher productivity that would allow it to meet critical milestones and deliveries.<br> <br> The agency, on the other hand, did not likewise assess a strength to the protester for its ability to obtain and retain qualified personnel. The agency said this difference was the result of the protester providing insufficient details—specifically, its failure to provide information on what was included in its proactive retention plan or its training program. The agency justification noted that the protestor failed to refer to its tuition reimbursement in its quote.<br> <br> In making the award, the source selection authority concluded that the awardee’s quote was technically superior, had multiple strengths, was lower risk in the two non-price technical factors, and was superior enough to warrant the price premium of 44 percent. The awardee was the third-highest priced quote. The protester was the lowest priced. The awardee was rated as “good” in both non-price technical factors, and the protester was rated one notch lower as “acceptable” in both non-price technical factors.<br> <br> The protester filed its protest asserting disparate treatment. A protestor must prove disparate treatment by the agency. In order to do so, a protester must show that the agency acted unreasonably in not assessing a strength to the protester for a feature in its proposal while assessing a strength to a competing proposal for an identical or indistinguishable feature.<br> <br> The case was sustained by the GAO because disparate treatment was found in how the agency evaluated the risk mitigation and management in the area of an offeror’s ability to obtain and retain qualified personnel. The agency failed to explain in its contemporaneous documentation or in response to the protest the substantive difference between awardee’s proposed tuition reimbursement and the protester’s reimbursement of academic degrees. The record also did not explain why the agency considered the awardee’s retention rate of 95 percent noteworthy but not the protester’s retention rate of 96 percent.<br> <br> This disparate treatment led the GAO to also determine that the agency’s best value trade-off decision was unreasonable as it too was inadequately documented. The GAO concluded by finding that there was possible competitive prejudice because of the agency’s single evaluation error. It reasoned in its discussion of the case that there might have been a narrowing of the quality gap between the awardee and the protester if the protester was assessed as having similar employee-retention strength. The agency might not have then been willing to maintain its view that the awardee’s quote was superior enough to warrant the price premium.</p> <h2>Conclusion</h2> Each year we confront consistent themes on how agencies should conduct source selection evaluations and engage with offerors. It is sometimes difficult to know when we may not be complying with the procurement regulations. The case examples above are just a few scenarios for useful, instructive comparisons with our current procurements. Taking these decisions into account, we can perhaps avoid making some of the same errors in our evaluations and award decisions. Conducting reasonable technical evaluations, engaging in equal discussions, conducting reasonable cost or price evaluations, and treating offerors equally are a few of the best practices that we hope will reduce the number of future protests sustained by GAO. <hr /><em><strong>Wallace</strong> is a professor of Contract Management at Defense Acquisition University, Fort Belvoir, Virginia. She is a U.S. Air Force veteran, a former litigation attorney, and a former contract specialist. She holds a law degree from the University of North Dakota.</em><br> <br> The author can be contacted at <a class="ak-cke-href" href=""></a>. <h5>The views expressed in this article are those of the author alone and not the Department of Defense. Reproduction or reposting of articles from Defense Acquisition magazine should credit the author and the magazine.</h5> <hr /><a href="" target="_blank"><img alt="tweet" src="/library/defense-atl/DATLFiles/Sept-Oct_2021/tweetbutton.jpg" style="width:15%;border-width:0px;border-style:solid;" /></a><a href="" target="_blank"><img alt="subscribe" src="/library/defense-atl/DATLFiles/Sept-Oct_2021/suscribebutton.jpg" style="width:15%;border-width:0px;border-style:solid;" /></a><a href="/library/defense-atl/DATLFiles/July-Aug2022/defacq-datl_Wallace_julyaug2022_online.pdf" target="_blank"><img alt="print" src="/library/arj/ARJ/ARJ%20101/print_button.jpg" style="width:15%;border-width:0px;border-style:solid;" /></a></div>string;#/library/defense-atl/blog/GAOReport2021
Breaking The Supply Bottleneck The Supply Bottleneck2022-08-20T16:00:00Z,<div class="ExternalClass2FBF5E696FFE445E908E75DBCC3F3F8B">After nearly two years of disruption, supply chains across the globe remain dramatically out of sync, with congestion at seaports marking the latest in a long list of problems. Ocean shipping, facilitated by ports, makes up the lion’s share of global trade, exacerbating the scale of this most recent issue.<br> <br> In 2021, the waiting time for vessels at ports skyrocketed, with 10 percent of the world’s container capacity idling. The congestion this year has been caused by a surge in imports as U.S. consumer demand has shifted away from services to goods and home improvements, and retailers have rushed to restock inventories depleted in the pandemic. Constrained availability of truckers also has contributed to the problem.<br> <br> Experts say that the COVID-19 pandemic’s impacts on the U.S. economy are likely to create long-term problems for the ability of the Department of Defense (DoD) to keep weapons flowing. Supply chain stresses are so fraught that the Biden administration has committed to work with suppliers to execute mitigation measures at U.S. ports.<br> <br> Such effects will be felt most strongly by DoD’s second-, third- and fourth-tier suppliers that sell primarily to the civil and consumer markets, and whose military sales represent only a small piece of their revenue streams, experts said during a webinar sponsored by the Center for Strategic and International Studies.<br> <br> Data from the Defense Contract Management Agency (DCMA) shows that 106 out of 10,509 primary Pentagon contractors had to close during the pandemic. Of those, 68 companies have subsequently reopened, according to Andrew P. Hunter, Assistant Secretary of the Air Force for Acquisition, Technology and Logistics. (Hunter also temporarily had performed the duties of the Under Secretary of Defense for Acquisition and Sustainment). On the subcontractor side, 427 of the 11,413 subcontractors’ DCMA tracks closed initially, but 147 now have reopened.<br> <br> Few citizens realize how much our nation’s defense depends on small- and medium-size manufacturers that struggle to stay afloat. When it comes to defense equipment, we think of massive prime contractors building jets and ships that cost hundreds of millions of dollars. It is less well known that, for every new fighter jet that takes to the sky, there is an older bomber or refueling tanker struggling to get off the ground. In many cases, no one is making the parts necessary to keep in service the venerable B-52 bomber or the trusty KC-135 refueling tanker. Even where a small company is manufacturing the needed part, there is rarely a second optional provider.<br> <br> Since the COVID-19 pandemic began, global supply has been newsworthy. “Supply Chain Resilience” is a buzz phrase, and COVID-19, the Suez Canal closure, and other supply chain disruptions continue. It is one of President Biden’s biggest challenges. If current supply issues are not resolved soon, there is a real possibility of economic slowdown and massive layoffs.<br> <br> Most of America’s fragile supply chain centers at the Port of Los Angeles and its impact on commercial suppliers. Lost has been news about our military’s critical supply chain, and its inability to keep fighter jets, bombers, and refueling tankers operational. The sprawling network of private contract manufacturers of critical replacement parts is known as the defense industrial base (DIB). Recent reports paint a worrisome picture of the DIB’s rapidly declining ability to support our military.<br> <br> To highlight the severity, consider the events of October 2021. Headlines popped up rapidly about shortages affecting U.S. fighter jets. First there appeared an article titled “F135 Depot Rebounds, But F-35 Engine Shortage Worsens” (<em>Aviation Week</em>, Oct. 8, 2021). This followed the release of a DoD Inspector General audit that, among other problems, found a shortage of F-15 and F-16 engines. The Air Force Air Mobility Command published a request for information to the industry to determine the state of our tanker shortage following a report on the challenges faced by Boeing’s efforts to deliver the new KC-46 airborne refueling tanker. These were just three reports in less than a week. <h2>Bolstering the Industrial Base</h2> The United States must ensure that there is a domestic supply of the materials essential to U.S. defense programs, especially key munitions. Policy interventions should be tailored to the unique market failures of a given strategic and critical material market, with strong emphases on private-sector partnerships and accelerating development of diversified and reliable supply sources. The United States must consider an all-of-the-above approach, including high-risk research for advanced production processes and equipment, facilitation of business-to-business ties within the DIB and with U.S. allies, and, where appropriate, bespoke or tailored trade remedies.<br> <br> In the hopes of limiting our use of potentially unreliable or interruptible Chinese sources, the White House is directing the DoD to use more incentives to support U.S. companies accessing critical minerals. The directive is part of the Biden administration’s national <a href="" target="_blank">supply chain review</a>, which recently hit a 100-day milestone.<br> <br> The Pentagon, according to a <a href="" target="_blank">White House fact sheet</a>, will “deploy” Defense Production Act Title III incentives, including “grants, loans, loan guarantees, and offtake agreements—to support sustainably produced strategic and critical materials, including scaling proven research and development (R&D) concepts and emerging technologies from other programs such as the Small Business Innovation Research awardees.”<br> <br> Air freight rates have risen as much as 24 percent this year after more than doubling in 2020, as the COVID-19 pandemic increased demand for electronic goods such as gaming consoles and laptop computers as more people were required to work from home. The cargo market has been a rare bright spot for U.S. carriers, whose businesses have been decimated by the pandemic.<br> <br> Demand for air freight is growing in the automotive and other industrial sectors, including plastics and chemical goods, as companies restock inventories and economies rebound faster than expected, said Tim Scharwath, chief executive officer of global shipping firm DHL (“The boom of the air cargo industry after the COVID-19 pandemic” (<a href=""></a>)).<br> <br> Delays in seaborne shipping due to bottlenecks at ports and a lack of containers have also diverted some cargo to planes. Sea freight costs have surged, making air freight more competitive. The spot shipping price for a 40-foot container has increased 34 percent this year, while rates for Los Angeles have surged 49 percent, according to the Drewry World Container Index.<br> <br> “We use rail now, which is normally way too expensive. But because ocean-freight rates are so high customers demand it,” Scharwath said. “If ocean freight goes over a certain threshold, rail becomes more attractive.”<br> <br> The Navy and shipbuilding industry are monitoring the supply chain issues caused by the ongoing COVID-19 pandemic, and the industrial base is experiencing some effects, according to Navy and industry sources. While it’s too early to fully grasp the effect of the supply chain slowdown on Navy shipbuilding, some companies are starting to feel constraints, and the Navy focusing more on addressing supply chain problems.<br> <br> Vice Admiral William Galinis, the head of Navy Sea Systems Command, in October told the American Society of Naval Engineers that the Navy was experiencing longer lead times for materials (“Galinis: Navy starting to see effect of supply chain issues” [<a href="" target="_blank"></a>]). Raw material delays are affecting production lines for prime and sub-tier suppliers to Navy platforms, Service spokeswoman Jamie Koehler told Inside Defense. However, Koehler said that no programs are experiencing particularly noticeable effects. However, the prices of steel and other commodities are increasing due to the supply chain issues, an industry source told <em>Inside Defense</em>.<br> <br> Rick Giannini, chairman of the Aircraft Carrier Industrial Base Coalition and chief executive officer (CEO) of Milwaukee Valve Company, told an interviewer that his company’s reliance on U.S. suppliers had helped the company avoid supply chain problems. But other companies in the aircraft carrier supply base have experienced delays, he said, estimating that 20 percent of the supply base had been affected. “I’m certain that some of them are experiencing delays, because I know it’s a major issue in the industry,” he said.<br> <br> Huntington Ingalls Industries (HII) President and CEO Mike Petters said, during the company’s November earnings call, that the company is monitoring material availability from its supply chain and expected to have more to share at the next earnings call. HII Executive Vice President and Chief Financial Officer Tom Stiehle said that the company has a significant number of long-term contracts and material orders. Therefore, many requirements are already on order (“Huntington Ingalls Industries, Inc.’s CEO Mike Petters on Third Quarter 2021 Results—Earnings Call Transcript, ‘Seeking Alpha’ ”).<br> <br> “The preponderance of the material is coming in on time and meeting the contractual needs that we have within the yard,” Stiehle said. “I will tell you recently, in the last three to six months as we have spot orders, what we are seeing is a little bit of volatility in pricing and the validity base is shrinking a bit in terms of what we have to spot buy.”<br> <br> Giannini said that his actuator suppliers have more difficulty than usual in procuring the chips that go into electric actuators. “It is slowing us down, to some extent, from that perspective,” he said. “So, we’re seeing some impact from my supply chain.” The United States does not manufacture enough chips, so U.S. businesses rely on China and Taiwan for their supplies, he added.<br> <br> “For us, it’s a relatively small per-centage of our overall business,” Giannini said. “But nonetheless, when I hear from other suppliers in our industrial base, I know anyone in that space is having difficulty.” The issues began manifesting themselves early this year, he added. The cost of getting a container on a ship to the United States has quadrupled from last year, Giannini said, going from a spot rate of $4,000 a year ago to more than $20,000 at its highest level.<br> Giannini said that U.S. companies should lessen their dependence on China. “The whole thought of a crisis that’s hitting our supply chain as a result of China is a bit scary to me,” he said.<br> <br> Paula Zorensky, vice president, Shipbuilders Council of America, told Inside Defense that the entire economy is seeing the effects of supply chain issues. Shipyards have been affected with cost increases and disruptions, she said.<br> <br> “When you can leverage things like block buys and multiyear procurement, that stability and predictability helps. That helps mitigate potential supply chain issues, or whatever is introduced into the marketplace, because we are businesses that can adapt,” she said. <h2>Increasing Supply Chain Situational Awareness</h2> Koehler said that the Navy has multiple internal organizations working together to monitor the supply chain. “NAVSEA’s Supply Chain monitoring efforts have been in place for a number of years at the [program executive office] level,” Koehler said. “This year we intensified efforts to coordinate and communicate across the entire NAVSEA portfolio.”<br> <br> Marine Corps Commandant Gen. David Berger told the Aspen Security Forum in early November that the Corps has increased its focus on strengthening the supply chain and now has a much clearer picture of it. “Yes, we are feeling the effects,” Berger said. “No, it hasn’t in a large way affected readiness.”<br> <br> <a href="" target="_blank"><img alt="click here to see commandant's address at the aspen security forum" src="/library/defense-atl/DATLFiles/July-Aug2022/defacq-datl_fullbook_julyaug2022_article05_image01.jpg" style="border-width:0px;border-style:solid;margin-left:6px;margin-right:6px;width:50%;" /></a><br> Pentagon, White House, and congressional leaders are laser-focused on addressing our nation’s critical supply chain. The House Armed Services Committee created a Defense Critical Supply Chain Task Force. Elements of the Fiscal Year 2022 National Defense Authorization Act seek to tackle some of the challenges. National and state organizations such as the National Defense Industrial Association actively push the issue. However, many of the findings, recommendations, and actions are incremental and will not materially improve the underlying problems.<br> <br> In the face of major supply chain disruptions like port congestion, there are proactive measures contractors can take to mitigate their supply-chain risks, starting with multitier visibility. Even if you know your first-tier suppliers are mainly domestic, greatly limiting exposure to seaports, a staggering 50 percent of disruptions in supply chains occur below the first tier. The following are a few key insights into how the DoD supply chain may be impacted by the reverberations of port congestion, and what program offices can do to protect themselves. With Southern California seaports overburdened by commercial ships, pandemic, and supply-driven delays in moving cargo, the Navy has agreed to allow cargo-carrying vessels to use one of its military wharves at Port Hueneme, California.<br> <br> Port Hueneme is the only deep-water port sited between San Francisco and Los Angeles that can support the heavy, large commercial vessels with deep drafts that require deeper channels and harbors to take on or offload their cargo.<br> <br> The COVID-19 pandemic affected logistics systems due to restricted access to sea and airports, changes in manufacturing focus, and travel quarantine. Defense managers focus on meeting the cost, schedule, and performance goals set and agreed to in the acquisition program baseline (APB), sometimes referred to as the contract between the PM and the milestone decision authority. Although the APB specifies the major goals for the program, it does not explicitly discuss the many underlying factors that contribute to meeting these objectives. Therefore, the PM can become singularly focused on these areas and, unknowingly, miss other indicators of risks until they manifest as issues.<br> <br> Through analytics and data visualization tools, supply chain resilience need not be an academic abstraction. Focused investments in these tools help contractors’ supply chains become more resilient by improving the visibility, flexibility, collaboration, and control of the contractors’ various partners and suppliers. These technologies can also help a supplier prioritize its efforts on supply chain resilience, measure the results of its investments, and strengthen its ability to address vulnerabilities and respond effectively when issues inevitably arise.<br> <br> To avoid the impacts of port congestion, it’s important to first increase visibility into the supply chain—ideally with a view of multiple tiers. I believe that the following steps are great places to start: <ul> <li>Work closely with the program offices’ prime contractors and require supplier road maps to at least the fifth tier in their supply chains in order to ensure that they have adequate situational awareness.</li> <li>Require the prime contractor to maintain a risk register to assess supply chain issues and provide brief mitigation strategies just as they are required to assess technical risks.</li> <li>Review the programs’ Bill of Materials and look for sole-source suppliers that create potential vulnerabilities and develop a Plan of Action and Milestones.</li> <li>Use analytics to understand how much of your material flows through the congested port choke points and across each multitier product supply chain.</li> <li>Rank vulnerabilities of specific ports on a global level. It’s even more meaningful to assess the severity of the issue with respect to specific, long lead-time material.</li> <li>Use artificial intelligence to run delay scenarios to show when material is likely to be delayed or at risk of a supplier going out of business and look for opportunities to increase capacity.</li> </ul> Until sweeping changes occur, and advanced technologies are deployed, there must be unwavering focus by military and elected officials. Moreover, there must be tireless efforts from our policymakers to see these disruptions through. Until then, America’s DIB will remain fragile, and our country’s military potentially exposed. <hr /><em><strong>Wright </strong>is a retired U.S. Army lieutenant colonel, Logistics Branch, who served in Iraq and the U.S. Africa Command. He holds a BA in Psychology from the University of Louisiana at Monroe, an MA in Humanities from Louisiana State University, and a Post Graduate Certification in Supply Chain and Logistics from the University of Alabama at Huntsville.</em><br> <br> The author can be contacted at <a class="ak-cke-href" href="" target="_blank"></a>. <h5>The views expressed in this article are those of the author alone and not the Department of Defense. Reproduction or reposting of articles from Defense Acquisition magazine should credit the author and the magazine.</h5> <hr /><a href="" target="_blank"><img alt="tweet" src="/library/defense-atl/DATLFiles/Sept-Oct_2021/tweetbutton.jpg" style="width:15%;border-width:0px;border-style:solid;" /></a><a href="" target="_blank"><img alt="subscribe" src="/library/defense-atl/DATLFiles/Sept-Oct_2021/suscribebutton.jpg" style="width:15%;border-width:0px;border-style:solid;" /></a><a href="/library/defense-atl/DATLFiles/July-Aug2022/defacq-datl_Wright_julyaug2022_online.pdf" target="_blank"><img alt="print" src="/library/arj/ARJ/ARJ%20101/print_button.jpg" style="width:15%;border-width:0px;border-style:solid;" /></a></div>string;#/library/defense-atl/blog/Breaking-The-Supply-Bottleneck
The AUKUS Submarine Project—Planning for Country Responsibilities AUKUS Submarine Project—Planning for Country Responsibilities2022-08-06T16:00:00Z,<div class="ExternalClass16A7EED014364726AD9E72CE9CA22FAC"><h6>The Virginia-class nuclear-propulsion attack submarine USS Delaware (SSN 791) transits Connecticut’s Thames River in March. Key elements in the AUKUS project are U.S. experience with this submarine model and Britain’s with the Astute class (see page 24) whose body design will be used.<br> Source: U.S. Navy photo</h6> <hr />The joint AUKUS submarine project of Australia, the United Kingdom, and the United States is currently positioned to produce a paper tiger. A planned 18-month study is unlikely to remediate the situation without a convincing breakthrough in thinking and approach. Beyond that, AUKUS represents a real test of U.S. statecraft.<br> <br> The United States and the United Kingdom possess experience with nuclear propulsion in hunter-killer attack submarines with the Virginia class and Astute class respectively. In contrast, Australia has no nuclear experience and is committed to the proposition that it will allow no nuclear activity in country. All this sets the political and technological context for AUKUS and the allocation of country responsibilities. Australia and the United Kingdom both belong to the Commonwealth of Nations headed by Queen Elizabeth II.<br> <br> (See this writer’s earlier article, “<a href="/library/defense-atl/blog/AUKUS-Submarine-Project" target="_blank">The AUKUS Submarine Project—a Critical Assessment</a>,” in the January-February 2022 issue of Defense Acquisition magazine.)<br> <br> Is the AUKUS submarine project on target with respect to strategic vision, production capacity, and political considerations? Unfortunately, not all vectors point in the same direction.<br> <br> The AUKUS submarine project planning for country responsibilities is meant to provide a needed breakthrough in thinking and approach.<br> <br> For example, the conservative Hudson Institute sees the AUKUS partnership as more about the partnership among nations and less about submarines. More practically, Electric Boat (EB) in Groton, Conn., is operating over capacity with Virginia and Columbia class construction. Politically, Australia’s government has shown signs that it wanted to sink the AUKUS deal through the Labor Party’s demand that increasingly more AUKUS business be done in country. Then there are objections to the Adelaide site, and it is hoped that AUKUS submarines will be built in the United States despite lack of spare capacity at EB Groton. Underlying this is Australia’s antipathy to all things nuclear.<br> <br> The AUKUS submarine project planning for country responsibilities clearly could alter the landscape of the discussion. <h2>The Countries Involved</h2> As announced, the AUKUS partnership includes Australia, the United Kingdom, and the United States. France was disappointed in its bid for the Australian submarine development and was not invited to participate in the AUKUS partnership. There also is Indonesia with its submarine program that can observe AUKUS submarine operations and events in the Indo-Pacific Region.<br> <br> The key to AUKUS political relationships is technical performance. If France wants to join the partnership, it should address the application of advanced technology associated with a supercavitation supersonic torpedo to improve lethality and the unmet challenge of coating the submarine’s with anaechoic tiles to provide camouflage and improve stealth. If, as claimed, France can rebuild the fire-ravaged Notre Dame Cathedral in five years, it may be able to solve the problems associated with anaechoic tiles and supercavitation. The French Naval Group has been the aspiring French contractor for the submarine project.<br> <br> The natural U.K. role would be to provide the AUKUS submarine hull based on the Astute class, while managing space and weight as well as monitoring and mitigating noise vibration. Farnborough, England, is the site of the U.K. contractor, BAE Systems.<br> <br> <a href="/library/defense-atl/blog/AUKUS-Submarine-Project" target="_blank"><img alt="" src="/library/defense-atl/DATLFiles/July-Aug2022/defacq-datl_fullbook_julyaug2022_article04_image01.jpg" style="border-width:0px;border-style:solid;width:100%;" /></a><br> Australia has no Adm. Hyman Rickover to lead its nuclear program. Rickover was the father of the U.S. Navy’s nuclear program for three decades. Picked for its geographic location, Australia is the natural selection for berthing and refurbishment and the land-based evaluation site. Adelaide’s Osborne shipyard secured a major contract for the AUKUS submarine. Berthing is the limit of Australia’s responsibilities because it lacks a background and interest in nuclear engineering and is politically limited in pursuing a nuclear direction. Left unattended, such constraints could jeopardize the successful pursuit of an accelerated schedule to deploy AUKUS submarines.<br> <br> Australia must keep the interests of Indonesia and the Association of Southeast Asian Nations in mind when dealing with nuclear issues and the associated messaging on nuclear nonproliferation.<br> <br> The United States, having spent decades confronting a persistent Soviet adversary in the North Atlantic, acquired broad operational experience through experimentation. The United States also has a refined, nuclear-powered propulsion system that delivers speed and sustainable operations. In addition, this country can supply highly enriched uranium for nuclear propulsion and the engineering knowledge, skills, and behaviors to manage and integrate complex software-intensive subsystems. These include those associated with active and passive sonar along with the towed array, ship control and navigation, magnetic silencing, integrated radio rooms, global positioning systems, defensive weapons, cyber security defenses, and the acquisition of defensive weapons. The United States has the intellectual property and lessons learned from its deep integration engineering experience on the Virginia- and Ohio-class submarines.<br> <br> In accordance with the principle of resilience for anticipating and avoiding adversity, the AUKUS project must have a backup berthing location outside Australia in view of Australia’s political controversies about nuclear nonproliferation.<br> <br> <img alt="A Royal Navy Astute Class submarine plies the waters in sea trials off western Scotland. This submarine class body design was selected for the AUKUS project" src="/library/defense-atl/DATLFiles/July-Aug2022/defacq-datl_fullbook_julyaug2022_article04_image02.jpg" style="width:100%;" /> <h6>A Royal Navy Astute Class submarine plies the waters in sea trials off western Scotland. This submarine class body design was selected for the AUKUS project.<br> Photo used by permission of UK Ministry of Defence.<br> UK MOD © Crown copyright 2021</h6> <h2>Country Benefits</h2> The value of the AUKUS project lies in its goal of achieving a possible maritime strategic balance in the Indo-Pacific Region. Countries participating in the AUKUS project benefit in numerous ways.<br> <br> For some, it begins with eliminating technical debt. This puts a fine edge on the technical debt that the project carries from the outset. Examples of technical debt include Australia’s lack of experience and commitment to all things nuclear, the Chinese advantage in a supercavitation supersonic torpedo, the unmet challenge of coating the submarine skin with anechoic tiles as camouflage to improve stealth, Australia’s insistence that 60 percent of AUKUS expenses be made in country, and Australia’s focus on one-third of its exports going to China. The planned 18-month study is unlikely to write off this technical debt.<br> <br> <strong>France</strong>: Simply considering France for membership in the AUKUS partnership could improve relationships between France and the other nations. Besides engendering a positive French attitude, involving French scientists could well produce breakthroughs on the supercavitation supersonic torpedo challenge and the use of anechoic tiles.<br> <br> <strong>The United Kingdom</strong>: The U.K. will capitalize on its successful experience with Astute and its U.S. relationship on nuclear propulsion.<br> <br> <strong>Australia</strong>: Exploiting its geographic location to ensure membership in the prestigious partnership would help Australia reduce its dependence on China exports. Australia has the attitude that AUKUS submarines will belong to Australia and operate under its control despite the essential U.S. nuclear-propulsion contribution.<br> <br> <strong>Indonesia</strong>: So long as Indonesia stays in its lane and continues as a supporting witness to events, it will remain a member in good standing with AUKUS partners and may well offer important intelligence on activities in the Indo-China Region.<br> <br> <strong>The United States</strong>: Our country is able to lead the partnership, based on its well-earned credentials including nuclear advantages, intellectual property, and the opportunity to share its depth of experience in integration engineering. AUKUS funding could help relieve some pressure on the overtaxed U.S. maintenance and shipbuilding bases.<br> <br> <img alt="The Virginia-class fast-attack submarine USS North Carolina (SSN 777) departs Joint Base Pearl Harbor-Hickam for a regularly scheduled deployment" src="/library/defense-atl/DATLFiles/July-Aug2022/defacq-datl_fullbook_julyaug2022_article04_image03.jpg" style="width:100%;" /> <h6>The Virginia-class fast-attack submarine USS North Carolina (SSN 777) departs Joint Base Pearl Harbor-Hickam for a regularly scheduled deployment.<br> Source: U.S. Navy photo</h6> <h2>The Results</h2> The allocation of country responsibilities based on the best capabilities of each country turns the attention of AUKUS partners from politics to performance. Project risk is shared by spreading it out among partners, with each partner accepting only what its demonstrated capabilities permit. This new focus is intended to reduce the time to launch from 20 years (2040) to perhaps 10 years (2030).<br> <br> There is a tangled web of disrupted contractors in the wake of the AUKUS announcement. Some of these bit the dust when Australia terminated French involvement. Others wait apprehensively for the decisions to come from the 18-month study.<br> <br> While much attention is devoted to stealth in terms of noise and vibration, little attention is given to the disposal of two pounds per person per day of human waste. Left unattended, the trail of human waste disposal, perhaps with remnants of nuclear waste, could be tracked by an adversary.<br> <br> The use of refuse to track an adversary is not just theoretical. Decades ago, the United States tracked Russian submarines through their disposal of potato skins. Refuse is physical evidence that jeopardizes stealth. A modest personnel complement of 75 sailors, each producing two pounds of refuse per day, could accumulate 15 tons on a six-month tour. That certainly jeopardizes stealth. Why is this tolerated? When it comes to stealth, resilience lies in anticipation and avoidance.<br> <br> In addition to providing berthing and refurbishment, Australia should be assigned responsibility for disposing of human waste. Along with disposal, Australia should be responsible for checking human waste for evidence of drug use by the crew.<br> <br> In 2019, three sailors on a nuclear-armed Royal Navy submarine were caught taking cocaine. Trident nuclear missiles were involved. This incident on the HMS Vengeance followed another with the HMS Talent. Moreover, repeat offenders were involved. Clearly, drug use among submarine crew members is a clear and present danger.<br> <br> <img alt="The Virginia-class attack submarine USS New Mexico (SSN 779) is moored pierside while making a brief stop for fuel and supplies in Souda Bay, Greece" src="/library/defense-atl/DATLFiles/July-Aug2022/defacq-datl_fullbook_julyaug2022_article04_image04.jpg" style="width:100%;" /> <h6>The Virginia-class attack submarine USS New Mexico (SSN 779) is moored pierside while making a brief stop for fuel and supplies in Souda Bay, Greece.<br> Source: U.S. Navy photo</h6> <h2>A Closing Thought</h2> We need to touch on one final issue. A decades-old Navy study found that former sailors aboard U.S. submarines equipped with nuclear weapons had a greater-than-expected death rate from cancer. The study of 23,900 veterans of such submarines who were discharged between 1969 and 1981 discovered 54 cancer deaths, roughly 15 more than expected for a group of that size. However, the study also found that submarine veterans experienced fewer deaths than expected from many other diseases. <hr /><em><strong>O’Neill</strong> was president of the Center for National Software Studies from 2005 to 2008. Following 27 years with IBM’s Federal Systems Division, he completed a three-year residency at Carnegie Mellon University’s Software Engineering Institute (SEI) under IBM’s Technical Academic Career Program and has served as an SEI Visiting Scientist and a Justice Department Litigative Consultant. A software engineering manager, technologist, and independent consultant, he has a BS in Mathematics from Dickinson College.</em><br> <br> The author may be contacted at <a class="ak-cke-href" href=""></a>. <h5>The views expressed in this article are those of the author alone and not the Department of Defense. Reproduction or reposting of articles from Defense Acquisition magazine should credit the author and the magazine.</h5> <hr /><a href="" target="_blank"><img alt="tweet" src="/library/defense-atl/DATLFiles/Sept-Oct_2021/tweetbutton.jpg" style="width:15%;border-width:0px;border-style:solid;" /></a><a href="" target="_blank"><img alt="subscribe" src="/library/defense-atl/DATLFiles/Sept-Oct_2021/suscribebutton.jpg" style="width:15%;border-width:0px;border-style:solid;" /></a><a href="/library/defense-atl/DATLFiles/July-Aug2022/defacq-datl_ONeill_julyaug2022_online.pdf" target="_blank"><img alt="print" src="/library/arj/ARJ/ARJ%20101/print_button.jpg" style="width:15%;border-width:0px;border-style:solid;" /></a></div>string;#/library/defense-atl/blog/AUKUS-Sub-Project-countryresponsibilities
Bridging the Valley of Death and Thriving Beyond —Improving Transition Success With Metrics the Valley of Death and Thriving Beyond —Improving Transition Success With Metrics2022-07-30T16:00:00Z,<div class="ExternalClass18A681C923F7406591871C824281773D">For many in the field of acquisition, there is a perhaps understandable idea that a great chasm exists between science and technology (S&T) and programs of record (PoRs) in advanced development. After all, this chasm traps many transitions, defined as the movement of data, knowledge, technology, and/or capability from S&T to advanced development.<br> <br> Funding is the primary reason for S&T transitions’ success or failure. Funding is the foundation that enables S&T transitions to cross the Valley of Death to acquisition programs. But there are other best practices that bridge the chasm, mitigate adoption risks, and optimize the value of the S&T transition to PoRs and ultimately capabilities to the Warfighters. This article aims to promulgate best practices from the advanced developer’s perspective.<br> <br> The Joint Program Executive Office for Chemical, Biological, Radiological and Nuclear Defense (JPEO-CBRND), an advanced developer for acquisition programs that provide chemical, biological, radiological, and nuclear defense capabilities to the Warfighter, fully embraces optimizing the value of S&T transitions.<br> <br> Like other program executive offices, the JPEO-CBRND does not want entrapped transitions and recognizes the need to improve its transition’s value. As a result, there either is no transition or a dropped transition, resulting in squandered resources, time, and a technology gap that increases risks to program. While most 6.3 (advanced technology development) S&T projects make it to PoRs, fewer make it to advanced development prototypes, and even fewer are instantiated in capabilities fielded to the Warfighter.<br> <br> Ensuring sufficient funding to cross the “Valley of Death” is a big part of improving transition adoption and success. The JPEO-CBRND has integrated multiple alternative approaches to improve its transitions. These approaches go from lessons learned from the S&T transitions to the Integrated Early Warning PoR, to incorporating a bespoke version of Special Operations Command’s Transition Confidence Levels, to researching what other Services use (specifically the Navy) to measure and assist before, during, and after transitions. These have further contributed to adoption and success at the JPEO-CBRND.<br> <br> After realizing that neither one size nor one shape fits all applications, the JPEO-CBRND developed simple hybrid metrics to gauge and guide progress for S&T data, technology, and capabilities transitioning to PoRs. The Special Operation Command’s Transition Confidence Levels, Navy metrics, and the JPEO-CBRND’s simple hybrid metrics are explained below. <h2><img alt="a hiker in a desert" src="/library/defense-atl/DATLFiles/July-Aug2022/defacq-datl_fullbook_julyaug2022_article03_image01.jpg" style="margin-left:6px;margin-right:6px;width:100%;" /></h2> <blockquote> <p style="text-align:center;">Yea though I walk through the valley of the shadow of death, I will fear no evil.<br> —Psalm 23</p> </blockquote> <h2>What Acquisition Programs Should Include</h2> Drawing on the JPEO-CBRND’s experience with their primary transition primary partner, the Defense Threat Reduction Agency’s Joint Science and Technology Office and the Integrated Early Warning Campaign, the following suggestions helped to mitigate adoption risks and optimize the value of the S&T transitions. The adoption risks include misalignments of schedules and testing, and drafting of the transition agreement. The transition agreement codifies the understanding between S&T and the acquisition program about what is to be delivered and when.<br> <br> a. Acquisition programs should provide their transition partner with prioritized S&T needs before the Materiel Solution Analysis (MSA) phase of acquisition. “Up front, early and often” was the battle cry for Integrated Early Warning.<br> b. Acquisition programs should provide their S&T transition partner with their acquisition and test strategies at or before MSA. The acquisition program’s transition partner must be provided with these key documents as soon as possible. Forewarned is forearmed.<br> c. Acquisition programs should co-write transition agreements with their S&T partners. This compels the transition partners to share the transition responsibility equally. In the transition agreement, exit and entrance criteria such as delivery dates and expected technical maturity of the S&T deliverables are spelled out. This gives each transition partner a stake in the game.<br> d. Acquisition programs and their S&T transition partners must specify the expected maturity level of the data, technology, or capability being transitioned (often described as “Transition Readiness Levels”). They must concur on the robustness of testing (i.e., laboratory or operationally relevant), agree on delivery schedules, and clearly define the exit and entrance criteria in transition agreements. This avoids surprises involving one of the most key elements of the agreement—what actually will be delivered by the S&T transition partner to the acquisition program. Inspecting precedes expecting.<br> e. Acquisition programs should provide their S&T partner with feedback and lessons learned from their transitions. This post-transition element is absolutely essential. The acquisition program must detail to their S&T acquisition partners what worked and what did not. From the seeds of lessons learned sprout best practices.<br> f. Last, but far from least, employing metrics before, during, and after transitions is essential to gauge progress. To measure is to know.<br> <br> The JPEO-CBRND’s transition experience with the Integrated Early Warning program was enlightening. Initially targeted on the S&T transition’s technical maturity, the Integrated Early Warning program adopted Special Operation Command’s Transition Confidence Levels as metrics to gauge transition success.<br> For the Integrated Early Warning Campaign, the focus on Transition Confidence Levels augmented a typical focus on technical maturity in the transition agreement. Each of the Transition Confidence Levels 1-9 had between two and four adoptive characteristics and followed a logical arc from initiation to a completed transition. The levels enabled status scoring for a project and formed a roadmap for progress and coordination typically needed for transition success. In that sense, the Transition Confidence Level characteristics provided a scorecard and checklist for the Integrated Early Warning S&T partner and the JPEO-CBRND’s acquisition program.<br> <br> Special Operations Command has used Transition Confidence Levels to gauge the probability of transition success for more than a decade. The complete article on Transition Confidence Levels, curiously called “Bridging the ‘Valley of Death,’” can be found in the January-February 2017 edition of <em>Defense AT&L</em> magazine, pages 13-17. It is worth a read. Transition Confidence Levels from the article were tailored for applications by the JPEO-CBRND and are reproduced in Table 1. The Transition Confidence Levels are a measure of the holistic situation of a potential transition candidate. This includes funding and integration into program strategy—technical, programmatic, and operational, and tracking to completion.<br> <br> <strong>Table 1. Transition Confidence Levels​</strong> <table border="1" cellpadding="1" cellspacing="1" style="width:50%;"> <thead> <tr> <th scope="col">TRANSITION<br> CONFIDENCE<br> LEVEL</th> <th scope="col">CHARACTERISTICS</th> </tr> </thead> <tbody> <tr> <td style="text-align:center;">10</td> <td> <ul> <li>Technology fully integrated by acquisition program</li> </ul> </td> </tr> <tr> <td style="text-align:center;">9</td> <td> <ul> <li>Transition to acquisition program completed</li> <li>Transition memorandum and storyboard</li> <li>Transition success report to Chemical Biological</li> <li>Defense Program</li> </ul> </td> </tr> <tr> <td style="text-align:center;">8</td> <td> <ul> <li>Signed transition agreement between the acquisition</li> <li>program and the Science and Technology (S&T)</li> <li>transition partner</li> <li>Transition funding committed</li> </ul> </td> </tr> <tr> <td style="text-align:center;">7</td> <td> <ul> <li>Integration strategy fully defined</li> <li>Potential funding sources identified</li> <li>Coordinate assessment approach</li> </ul> </td> </tr> <tr> <td style="text-align:center;">6</td> <td> <ul> <li>Transition technical goals, exit criteria, and deliverables</li> <li>approved</li> <li>Transition schedule estimate developed</li> <li>Project included in program management plans as a potential source</li> </ul> </td> </tr> <tr> <td style="text-align:center;">5</td> <td> <ul> <li>Active communication between the S&T partner and</li> <li>the acquisition program</li> <li>Initial goals discussed</li> </ul> </td> </tr> <tr> <td style="text-align:center;">4</td> <td> <ul> <li>Target acquisition program briefed and provided</li> <li>progress updates</li> <li>Key transition stakeholders specifically named</li> <li>Relevant programs specifically named</li> </ul> </td> </tr> <tr> <td style="text-align:center;">3</td> <td> <ul> <li>Specific project technical goals established</li> <li>Target acquisition programs identified</li> </ul> </td> </tr> <tr> <td style="text-align:center;">2</td> <td> <ul> <li>Potential transition stakeholders identified</li> <li>Project initiated</li> <li>Technical maturity goals established (baseline)</li> </ul> </td> </tr> <tr> <td style="text-align:center;">1</td> <td> <ul> <li>S&T and acquisition program discussions</li> <li>Active tech discovery</li> <li>Acknowledged gap</li> </ul> </td> </tr> </tbody> </table> <h6>Source of Tables: Office of Naval Research</h6> <h2>Another Service’s Perspectives on Transition</h2> It is helpful to consider other Services’ approaches to gain insights on how they handle transitions. For one such case study, the Office of Naval Research (ONR) had issues with deployment to the operational force of technology developed within its S&T programs. Prior to its evaluation of current Future Naval Capabilities, the ONR instituted an annual review of project transitions from S&T to acquisition in order to understand the return on investment. This is particularly useful in determining why a technology would fail to deploy to the Fleet despite meeting all exit criteria, as well as to adopt lessons learned and refine best practices to improve viability of S&T projects.<br> <br> Deployed S&T transitions are defined as data, technologies, or capabilities that have made it into the Fleet, or in Army parlance, the field. The ONR recognized that while about 80 percent of efforts transitioned from S&T to acquisition programs, only about 35 percent of the transitions made it to the Fleet, which is ONR’s ultimate measure of success. To help understand the underlying cause of deployment failures, the ONR instituted annual program reviews of several S&T products and developed metrics to assess progress. A relevant example is the ONR’s review of its Future Naval Capabilities program. The Future Naval Capabilities program was initiated by the Department of the Navy in 2002 and is designed to develop and transition cutting-edge technologies to acquisition PoRs on a three-year schedule. The program delivers these technologies as products for integration into platforms, weapons, sensors, or specifications to improve Navy and Marine Corps warfighting and support capabilities.<br> <br> Future Naval Capabilities products typically begin where analytical and experimental proof-of-concept has been established, indicating the product is at Technology Readiness Level (TRL) 4. The products subsequently are matured to the point that a model or prototype can be demonstrated in a relevant environment, reaching TRL 6. Once the technology is demonstrated, the acquisition sponsor takes responsibility for any additional research, development, and test and evaluation (RDT&E) necessary to engineer and integrate the product into an acquisition program of record, or other program, that will ultimately deploy the new technological capability into the fleet or force.<br> <br> For all new technological products, S&T funding is contingent upon having a signed and negotiated Technology Deployment Agreement between the S&T program manager, relevant Chief of Naval Operations resource sponsor, and acquisition program manager to transition and deploy the technology. The inclusion of a Chief of Naval Operations resource sponsor is meant to ensure that funding to facilitate seamless financial transfer from S&T to the acquisition PoR is tracked and, where necessary, the appropriate Program Objective Memorandum issue sheets are in place to allocate funding. The Technology Deployment Agreement is reviewed, revised, and re-signed by all three sponsors each year of the Future Naval Capabilities program, and products unable to meet this requirement are subject to potential termination.<br> <br> Additionally, periodic reviews, with subject-matter experts, during program execution is another key to successful programs. This is done not only to ensure the program is executing funds but that it has not become a capability in search of a requirement. The Future Naval Capabilities program noticed that many programs either fell off the radar or existed in perpetuity because of either turnover at the oversight agency or in the program office. However, a program that is not achieving its broader objectives, does not always equate to failure. Future Naval Capabilities Leadership has found it useful to highlight key technologies that could possibly be incorporated or spun off.<br> <br> In essence, the Future Naval Capabilities program is structured to create a healthy balance of S&T “push” and acquisition “pull.” Addressing the importance of S&T and acquisition partnership in funding transitions, the ONR will fund 100 percent of the first year of a Future Naval Capabilities program, but then the acquisition program is expected to provide funding in kind for the second year at 30 percent and for the third year at 50 percent. The acquisition PoR contribution often is met by making hardware, software, and test equipment available to the Future Naval Capabilities program thereby facilitating direct integration and testing in the targeted software environment or hardware form factor. This is akin to skin in the game for the acquisition program and focuses attention on making the transition a success.<br> <br> <strong>Table 2. Integration and Test Metrics for Future Naval Capabilities<br> <br> INTEGRATION</strong> <table border="1" cellpadding="1" cellspacing="1" style="width:50%;"> <tbody> <tr> <td style="text-align:center;"><span style="background-color:#00FF00;">GREEN (Year 2/3+)</span></td> <td> <ul> <li>Validated requirement or UONs exists</li> <li>PoR system ICDs are clearly defined</li> <li>No modifications are required on PoR system or modifications</li> <li>are funded and in progress</li> <li>Only minimal product modifications are required to meet IA</li> </ul> </td> </tr> <tr> <td style="text-align:center;"><span style="background-color:#00FF00;">GREEN (Year 1)</span><br> <span style="background-color:#FFFF00;">YELLOW (Year 2/3+)</span></td> <td> <ul> <li>Requirement or UONs being considered</li> <li>POR system ICDs are not completely defined</li> <li>POR system modifications are not fully funded or delayed</li> </ul> </td> </tr> <tr> <td style="text-align:center;"><span style="background-color:#FFFF00;">YELLOW (Year 1)</span><br> <span style="color:#FFFFFF;"><span style="background-color:#FF0000;">RED (Year 2/3+)</span></span></td> <td> <ul> <li>Requirements do not exist</li> <li>ICDs are not defined</li> <li>PoR system modifications are not funded or significantly</li> <li>delayed (>3 years)</li> </ul> </td> </tr> </tbody> </table> <br> <strong>TEST</strong> <table border="1" cellpadding="1" cellspacing="1" style="width:500px;"> <tbody> <tr> <td style="text-align:center;"><span style="background-color:#00FF00;">GREEN (Year 2/3+)</span></td> <td> <ul> <li>Development testing within 1 year of delivery and is aligned to PoR on ramp</li> </ul> </td> </tr> <tr> <td style="text-align:center;"><span style="background-color:#00FF00;">GREEN (Year 1)</span><br> <span style="background-color:#FFFF00;">YELLOW (Year 2/3+)</span></td> <td> <ul> <li>Development testing within 1-2 year of delivery and is aligned to PoR on ramp</li> </ul> </td> </tr> <tr> <td> <div style="text-align:center;"><span style="background-color:#FFFF00;">YELLOW (Year 1)</span><br> <span style="color:#FFFFFF;"><span style="background-color:#FF0000;">RED (Year 2/3+)</span></span></div> </td> <td> <ul> <li>Test assets are not available or access is significantly delayed</li> <li>Tests do not align to PoR on-ramp</li> </ul> </td> </tr> </tbody> </table> <br> FOR THE OVERALL EVALUATION: <ul> <li>Evaluate the five criteria on a ten point</li> <li>scale: Green-2, Yellow-1, Red-0</li> <li>Overall evaluation guideline is: <ul> <li>GREEN 8 to 10 points</li> <li>YELLOW 5 to 7 points</li> <li>RED 0 to 4 points</li> </ul> </li> <li>The FNC DRB can consider additional factors and adjust assessment. Examples of additional factors may include: <ul> <li>Get-well plan</li> <li>Insurmountable conditions</li> <li>Alternative deployment paths</li> </ul> </li> <li>First year programs will be evaluated green or yellow only.</li> </ul> <h6>Key: FNC DRB = Future Naval Capabilities Defense Review Board, ICDs = Initial Capabilities Document,<br> PoR = Program of Record, UONs = Urgent Operational Needs</h6> Once a Future Naval Capabilities program is established, it is vital to deploy metrics to check progress delivering the data, technology, or capability specified in the technology deployment agreement between the S&T and acquisition program partners. Here again, the Navy’s approach to S&T-to-acquisition transitions can help. This includes using evaluation criteria for current and deployed transitions. This important distinction results in the use of different metrics for each transition.<br> <br> Current transitions are active and ongoing. In addition to the familiar metrics of cost, schedule, and performance, integration and test made part of the ONR evaluation criteria. Integration and testing are vitally important aspects of transitions and call for metrics. The Navy recognizes the need for different metrics in integration and test, depending on the maturity of the Future Naval Capabilities program. The integration and test metrics that the Navy uses for active Future Naval Capabilities programs are shown in Table 2, grouped by the age of the program in years.<br> <br> The evaluation criteria to track transitioned projects and specifically the five category bins for those that fail to transition are captured in Table 3.<br> <br> <strong>Table 3. Fiscal Year 2018 Future Naval Capabilities Deployment Review Board (DRB) Evaluation Criteria </strong> <table border="1" cellpadding="1" cellspacing="1" style="width:50%;"> <tbody> <tr> <td style="text-align:center;"><strong>COLOR</strong></td> <td style="text-align:center;"><strong>CRITERIA</strong></td> </tr> <tr> <td><span style="color:#FFFFFF;"><span style="background-color:#0000FF;">BLUE</span></span></td> <td>Transitioned Product deployed.</td> </tr> <tr> <td><span style="color:#FFFFFF;"><span style="background-color:#008000;">GREEN</span></span></td> <td>Fully funded and being incorporated/integrated into acquisition PoR (or alternate transition path) with a plan established for<br> delivery to the Fleet/Force.<br> <br> Note: If the Product transitioned successfully to a secondary or unplanned program, it should be marked as a successful<br> transition. The change in transition path should be noted in the comments, along with the new Program and PEO/ PM office.</td> </tr> </tbody> </table> <table border="1" cellpadding="1" cellspacing="1" style="width:50%;"> <tbody> <tr> <td style="text-align:center;"><strong>COLOR</strong></td> <td style="text-align:center;"><strong>CRITERIA</strong></td> <td style="text-align:center;"><strong>SUB-CATEGORY</strong><br> (CITE THIS AND EXPLAIN)</td> </tr> <tr> <td><span style="background-color:#FFFF00;">YELLOW</span></td> <td>Under consideration by the<br> acquisition program without a<br> definitive plan and/or adequate<br> funds aligned or planned<br> to ensure transition.</td> <td> <ul> <li>Loss of funding.</li> <li>Change in acquisition strategy.</li> <li>Technical modifications in progress. (Need to note who is paying for these).</li> <li>Other (explain).</li> </ul> </td> </tr> <tr> <td><span style="color:#FFFFFF;"><span style="background-color:#FF0000;">RED</span></span></td> <td>Failed transition or no funds planned<br> or programmed for transition.</td> <td> <ul> <li>Acquisition strategy significantly modified.</li> <li>Technology did not meet exit criteria.</li> <li>Requirements changed.</li> <li>Technology lost in acquisition competition.</li> <li>PMO continuity.</li> </ul> </td> </tr> </tbody> </table> <br> <strong>Table 4. Failed Deployed Transitions Evaluation</strong><br> <strong>VALUE OF FAILED TRANSITIONS</strong><br> Determine if value was derived from failed transitions by asking follow up questions and probing the acquisition POCs during their interviews.<br> Specifically, document any benefits that resulted from completed transition products that failed to transition in accordance with their Technology<br> Transition Agreements. For example, the product served to reduce acquisition program risk by providing important cost information or technical<br> knowledge used by the acquisition program office to make a more informed acquisition decision, confirm the best technology insertion plan, or<br> refine a request for proposal (RFP) specification. Explanations should not be dissertations, but a few sentences that succinctly identify and describe<br> the value, binned into one of the categories below. <table border="1" cellpadding="1" cellspacing="1" style="width:50%;"> <tbody> <tr> <td style="text-align:center;"><strong>CATEGORY</strong></td> <td style="text-align:center;"><strong>DESCRIPTION</strong></td> </tr> <tr> <td>REDUCED ACQUISITION PROGRAM RISK</td> <td>The product served to reduce acquisition risk by providing Important cost Information or technical knowledge used by the acquisition program office to mitigate cost/technical risk, confirm the best technology Insertion plan, or refine an RFP specification.</td> </tr> <tr> <td>TECHNOLOGY LEVERAGED FOR FOLLOW ON S&T EFFORT</td> <td>The technology was further developed in a subsequent S&T Product for another application.</td> </tr> <tr> <td>TECHNOLOGY AVAILABLE FOR FUTURE</td> <td>Placed on the shelf.</td> </tr> </tbody> </table> <br> Sixty-five percent of transitions from S&T to acquisition programs don’t make it to the Fleet. These are examined for lessons learned, in order to inform and refine best practices. The Navy’s evaluation criteria for failed transitions are shown in Table 4. <h2>JPEO-CBRND Simple Hybrid Metrics</h2> In addition to Special Operations Command’s Transition Confidence Levels and the Navy’s evaluation criteria—both established metrics for transitions—the JPEO-CBRND developed hybrid metrics. The hybrid metrics cover before, during, and after transition and are distributed among acquisition phases. Below are detailed simple metrics being developed by the JPEO-CBRND to assist acquisition programs to establish a return on investment. The metrics distill and incorporate some of what has already been discussed. Care was taken not to overburden the acquisition program, which is often replete with traditional acquisition metrics of cost, schedule, and performance. <h2>Before Transition—Materiel Solution Analysis (MSA) Phase</h2> Transition agreement benchmarks—such as stakeholders paired, technical maturity goals established, and entrance and exit criteria identified—are all established by S&T Project Manager and Advanced Developer. <ul> <li>Within one year of acquisition program initiation—Green</li> <li>Between one and two years of acquisition program initiation—Yellow</li> <li>More than two years after acquisition program initiation—Red</li> </ul> During Transition—Technology Maturation Risk Reduction (TMRR) Phase: Funding is identified for data, technology, and capability transition(s) and integration by Advanced Developer’s PoR Program Managers and S&T Project Managers <ul> <li>One year after TMRR start—Green</li> <li>Between one and two years after TMRR start—Yellow</li> <li>More than three years after TMRR start—Red</li> </ul> After Transition—Engineering and Manufacturing Development (EMD) Phase: Number of data, technology, and/or capability transitions in pre-production prototypes <ul> <li>Most of the transition(s)—Green</li> <li>Some of the transition(s)—Yellow</li> <li>None of the transition(s)—Red</li> </ul> Fielding—Production and Deployment (P&D) Phase: Number of data, technology, and/or capability transitions fielded <ul> <li>Most of the transitions—Green</li> <li>Some of the transitions —Yellow</li> <li>None of the transitions—Red</li> </ul> <h2>Conclusion: Eight Simple Techniques</h2> After distilling all the lessons learned, best practices, established procedures, innovative strategies, and aspirational thoughts, the following summarizes what a funded acquisition program can do to bridge the Valley of Death and thrive beyond. <ol> <li>Bridge the Valley of Death <ol> <li>Engage the S&T partner as soon as possible.</li> <li>Share acquisition and test strategies.</li> <li>Use metrics to gauge progress, Special Operations Command’s Transition Confidence Level or the Navy’s evaluation criteria.</li> <li>Co-write transition agreements and focus on entrance and exit criteria.</li> </ol> </li> <li>Thrive Beyond the Valley of Death <ol> <li>Use metrics to assess integration and test elements of the transition.</li> <li>Provide the S&T partner with feedback on the transition.</li> <li>Assess the return on investment.</li> <li>Document lessons learned.</li> </ol> </li> </ol> <img alt="a plant growing in sand" src="/library/defense-atl/DATLFiles/July-Aug2022/defacq-datl_fullbook_julyaug2022_article03_image02.jpg" style="margin-left:6px;margin-right:6px;width:100%;" /><br> <br> If that is too much detail, follow this Golden Rule: Use metrics. <hr /><em><strong>Newton </strong>works in the Joint Program Executive Office for Chemical, Biological, Radiological and Nuclear Defense (JPEO-CBRND) Futures Directorate. He has a Master’s degree in Systems Management from the Florida Institute of Technology and a Bachelor’s in Chemistry from McDaniel College. He is Level III certified in Engineering and in Science and Technology Management. He began his career in 1982 as a chemist at the Edgewood Arsenal in Maryland and holds patents in filter design and decontamination. He was an exchange scientist at Porton Down in the United Kingdom and later worked at the Science and Technology Center in Frankfurt, Germany, joining the JPEO-CBRND in 2005.</em><br> <br> The author can be contacted at <a class="ak-cke-href" href=""></a>. <h6>The views and opinions in this article are those of the author and do not necessarily reflect that of the JPEO-CBRND, the U.S. Army, the Department of Defense, or the U.S. Government. Reproduction or reposting of articles from Defense Acquisition magazine should credit the author and the magazine.</h6> <hr /><a href="" target="_blank"><img alt="tweet" src="/library/defense-atl/DATLFiles/Sept-Oct_2021/tweetbutton.jpg" style="width:15%;border-width:0px;border-style:solid;" /></a><a href="" target="_blank"><img alt="subscribe" src="/library/defense-atl/DATLFiles/Sept-Oct_2021/suscribebutton.jpg" style="width:15%;border-width:0px;border-style:solid;" /></a><a href="/library/defense-atl/DATLFiles/July-Aug2022/defacq-datl_Newton_julyaug2022_online.pdf" target="_blank"><img alt="print" src="/library/arj/ARJ/ARJ%20101/print_button.jpg" style="width:15%;border-width:0px;border-style:solid;" /></a></div>string;#/library/defense-atl/blog/Bridging-the-Valley-of-Death

Chat with DAU Assistant
Bot Image