The Quest for Rapid Acquisition

Lt Col Mark Massaro, USAF

One of the most recognized barriers to new technologies transitioning into usable capabilities is the “valley of death,” that gap between promising technologies and the resources required to transition them to warfighting capabilities.

Demonstrations, experimentation and prototyping combined with experienced test professionals are all paths to bridge that valley. However, as an experienced test professional who can help with transition, I have often had the impression that the Test Enterprise is viewed as the Black Knight from “Monty Python and the Holy Grail,” a comic villain whose only purpose was to stop or slow progress even when the quest was noble. Instead, I think Test is more like Tim the Enchanter from the same movie. The quest was well under way when he was brought on-board, but he quickly understood the importance of King Arthur’s quest. Once a part of the team, he provided early and valuable insight into the difficulties they faced and continued on their quest with them to provide data about their threat’s big pointy teeth.

Those reading this publication are likely aware of the 2018 National Defense Strategy (NDS) and the Department of Defense (DoD) push to restore readiness and increase lethality to achieve decisive advantages against competitive adversaries. What the audience here may be less aware of are the unique Service responses to answer the NDS’ call to “deliver performance at the speed of relevance.”

Let’s look at how the Air Force Test Enterprise is postured to increase performance and affordability. As a critical element in acquisition of these capabilities, the Air Force Test Enterprise focuses on enabling increased performance through innovation and streamlined rapid, iterative developmental and fielding approaches.

This is not the first time that the Air Force Test Enterprise has responded to the need for speed in acquisitions. In its 2004 response to the Air Force shift to evolutionary acquisition and the spiral and incremental development processes, Headquarters Air Force Test and Evaluation (AF/TE) published Air Force Instruction (AFI) 99-103 Capabilities Based Testing. AFI 99-103 consolidated into a single policy document aimed at integrating test planning and execution three separate test policy documents (AFI 99-101, Developmental Test and Evaluation; AFI 99-102, Operational Test and Evaluation; and AFI 99-105, Live Fire Test and Evaluation).

After 2018 National Defense Strategy was published, the Air Force acquisition community pivoted to rapid acquisitions, including utilizing Middle-Tier Authorities (Section 804) and tailored DoD Instruction (DoDI) 5000.02 programs. In response, AF/TE published a policy guidance memorandum that described policy guidance for Test and Evaluation of rapid acquisition programs (Middle-Tier Acquisitions, tailored DoDI 5000.02 programs, and Experiments). The Jan. 10, 2019, memo highlighted five key messages that form the basis for how the Air Force Test Enterprise can best support these programs. These five key messages are:

• Test and evaluation’s main goals are to reduce risk and provide data to inform acquisition decisions or answer warfighter capability questions.
• Rapid acquisition relies on a cycle of rapid learning; test and evaluation is the primary means of learning.
• Delivering performance at the speed of relevance still requires independent testing (“independent” refers to chain of command separation from program offices).
• Rapid does not mean reckless, nor does it mean ignoring the “ilities” that make a system suitable as well as operationally effective.
• The key to rapid testing is early and continuous tester involvement during a program.

In addition to the memorandum informing the update to AFI 99-103, it also helped shape the acquisition community’s soon-to-be published AFI 63-146, Rapid Acquisition Guidance. Rather than discuss the intricacies of policy, it is better to explore these key messages through a vignette that illustrates how early test involvement through prototyping and experimentation can enable the delivery of performance at the speed of relevance.

As I suggested earlier, one of the Test Enterprise’s methods to help transition technologies across the “valley of death” is through demonstrations or experimentations using experienced test professionals. One such way this is done by test is through the U.S. Air Force (USAF) Test Pilot School (TPS) Test Management Project (TMP) program. TMPs are part of TPS’ curriculum through which students and staff work with various customers to plan, execute and report on a short-duration test program. TMPs are a graduation requirement for students, but TPS staff sometimes conduct TMPs outside of the curriculum when a program exceeds the scope of curriculum requirements or the timeline does not synch with the academic calendar. What follows is a discussion on a TPS staff-led TMP, aimed at helping to advance technology through the valley of death in order to increase performance and lethality.

Born out of previous attempts to integrate flight control and weapons delivery functions in tactical aircraft (Tactical Data System and Integrated Flight and Fire Control programs), USAF TPS staff hatched a plan to enable the transition of a promising technology into a warfighting capability. In 2017, with DoD funding, the staff began a demonstration exploring the feasibility of a fighter aircraft-based automatic gun-tracking capability against air-to-air and air-to-ground targets named Digitally Enhanced Aiming Through Control Law (DEATH CLAW). DEATH CLAW had three objectives: (1) Demonstrate the automatic capability of the DEATH CLAW system to aim the gun in flight, (2) compare human performance against the system’s performance using air targets and both moving and stationary ground targets, and (3) identify plans to transition the system into Block 30 and Block 40/50 F-16 fighter jets.

Even though it was fast, the test demonstration applied the appropriate rigor necessary to ensure that technical and safety risk were acceptable and that the data collected would be useful for decision makers.

The demonstration kicked off in August 2017; executed 12 test, evaluation and demonstration missions from November 2017 to January 2018; and issued a final report in March 2018. An 8-month program is rapid by any measure, but unheard-of for flight control law development. Even though it was fast, the test demonstration applied the appropriate rigor necessary to ensure that technical and safety risk were acceptable and that the data collected would be useful for decision makers. TPS collaborated with Lockheed Martin Skunk Works, Calspan, the 412th Test Wing and the 416th Flight Test Squadron to plan, execute, and report on the demonstration. It was flown using the USAF TPS NF-16 Variable Stability In-Flight Simulator Test Aircraft (VISTA).

The VISTA provides a safe way to rapidly fly and update flight control law software. In essence, it takes off like a normal F-16. But, once airborne, the test aircrew members activate the test configuration control law, giving them the ability to rapidly evaluate new designs. During this demonstration, the VISTA provided the test team with the capability to fly DEATH CLAW in a controlled manner against both air and ground targets. After each mission, the team reviewed data and was able to update the control algorithm. The ability to rapidly iterate this cycle sped up the process.

The demonstration consisted of an integration and checkout phase, formal testing phase and a final demonstration phase. Integration and checkout lasted 1 week and ensured that the DEATH CLAW algorithm could be safely flown and operated as expected. The team flew three times and ensured DEATH CLAW integration with VISTA while refining some of the control laws. The formal evaluation immediately followed the integration phase. The team executed six missions over 1 week during which six different test pilots executed the same mission profile to evaluate the full air and ground target test matrix. Finally, the team executed the demonstration phase using three pilots in leadership positions from Air Combat Command (ACC), Air Force Materiel Command (AFMC) and the Air National Guard (ANG) with varying levels of experience and backgrounds. Following the successful, TPS-led demonstration, both the USAF and the ANG funded integration efforts on their F-16 fleets. The ANG plans to begin flight testing on its Block 30 F-16s in 2020 at the ANG Air Force Reserve Command Test Center (AATC).

The DEATH CLAW demonstration proved successful and achieved its three objectives, paving the way for future platform-integration efforts. Estimates showed that costs savings were 90 percent and time savings were 80 percent of what normally would be expected for this type of a program. Success depended on early involvement of the right kind of test and test support organizations. The USAF Test Pilot School, the 412th Test Wing, Calspan, and Lockheed Martin brought deep knowledge and experience in flight-control development and testing. This combined team proceeded rapidly without being reckless.

Contributing to planning early in the timeline, the test team members used their knowledge of the risk analysis process to effectively and quickly understand and mitigate safety and technical risks. Furthermore, the program’s limited scope enabled the team to focus and succeed in demonstrating a performance improvement of a lethality enhancement’s technological readiness. It also supported rapid learning during the integration and checkout phase through quick, focused iterations that included experts from the entire team. While limited in scope, the project demonstrated that the test enterprise is able to rapidly produce relevant data to deliver increased technological readiness, ultimately delivering performance to the warfighter.

The bottom line in achieving success at the speed of relevance is involving Test early and keeping Test continually engaged. As Dr. William B. Roper Jr. noted in his April 10, 2018, memo, “it takes a team to go fast,” and there is an entire Test Enterprise in the Air Force ready to assist with a shared sense of purpose. If King Arthur and his merry band of knights had brought Tim into the quest at the beginning of the movie, it probably would have been boring for the audience, but he would have reached the final castle earlier and been able to go on additional quests before the authorities put an end to it all. The Air Force Test Enterprise enables rapid acquisitions. Unlike Monty Python’s Black Knight, we promise to not “bite your legs off.”

Read the full issue of
Defense Acquisition magazine

MASSARO is Chief of the Policy, Programs, and Resources Division, Headquarters Air Force Test and Evaluation (AF/TE) in the Pentagon. He is an experimental test pilot with extensive test experience in fighter aircraft. The views expressed in this paper represent the personal views of the author and are not necessarily the views of the Department of Defense nor of the Department of the Air Force.

The author may be contacted at: [email protected].

Photos from the U.S. Air Force Test Pilot School

Illustration by Jim Gray

The views expressed in this article are those of the author alone and not the Department of Defense. Reproduction or reposting of articles from Defense Acquisition magazine should credit the authors and the magazine.