Sign In

 

 

Disasters in the Makinghttps://www.dau.edu/library/defense-atl/Lists/Blog/DispForm.aspx?ID=155Disasters in the Making2019-12-18T17:00:00Zhttps://wwwad.dauext.dau.mil/library/defense-atl/PublishingImages/DefAcqNov-Dec19_banner3.jpg, https://www.dau.edu/library/defense-atl/PublishingImages/DefAcqNov-Dec19_banner3.jpg https://wwwad.dauext.dau.mil/library/defense-atl/PublishingImages/DefAcqNov-Dec19_banner3.jpg<div class="ExternalClass1685EECEC0DB4920BFAACA97C94C4637">After almost every accident or disaster, the postmortem analysis invariably uncovers telling signs along a consequential chain-of-events leading up to the catastrophe itself. At a key point, there was a high probability that someone could have helped change the future. In two well-known cases, people spoke up but were overruled by senior management professionals exercising something far worse—their reluctance to think more critically, thereby falling victim to the likely catastrophic consequences. What if they had dug a little deeper and exercised more divergent thinking? What else did they fail to consider? This article addresses those “whys, “hows” and “whats.” <h3>Learning Through the Forensics</h3> When a major system has an accident or near accident, an investigation board convenes. Replete with subject-matter experts, this multi-discipline team hunts for the root cause and contributing factors.<br> <br> After combing through what they have at their disposal (e.g., material, processes and interviews with personnel involved with both) they pursue multiple related questions in their inquiry: Why did the space shuttle explode? Why did the plane crash? Why did the nuclear power plant leak radiation? Why did the oil rig release hundreds of thousands gallons into the ocean? If the evidence points to a manufacturing pedigree issue, why did the part fail? Was it poorly designed? Were the design tolerances exceeded and why? Was there a manufacturing abnormality? Did counterfeit parts/material sneak into the assembly line? Did an unknown design defect finally materialize? On the other hand, if the basic cause was operator error, why did that error occur? Was it due to inadequate training, lack of currency, poor judgment, poor instrumentation or simply work overload? Did the previously unknown problem that materialized pose an immediate safety concern that was prematurely dismissed or discounted? Did the affected personnel understand the severity of the situation but have little time to fully assimilate the “fix” in a way that could overcome any impending danger? Was the operator impaired due to some sort of psychological trauma or physical illness?<br> <br> Only after the possible root causes and contributing factors are fully understood along the chain-of-events can corrective measures be established to significantly reduce the risk of future recurrences. And more important, what was missing to promote deeper thinking to prevent the catastrophe in the first place? Who did what or why not? <h3><img alt="" src="/library/defense-atl/DATLFiles/Nov-Dec_2019/Def_Acq_Nov-Dec19_article3_quote1.jpg" style="margin-left:5px;margin-right:5px;float:left;width:239px;height:300px;" />The Missing Variables</h3> Those of us in the acquisition profession who find ourselves on the training side of the equation have discovered what sometimes gets overlooked, or too hastily minimized all too often—the importance of critical thinking, and its inextricable link to ethical persistence. Aside from nurturing functional expertise, the need to reinforce both of these qualities couldn’t be more vital. In their absence, how can the professional Defense Acquisition Workforce who oversee the development, production and sustainment of weapons possibly ensure that they are lethal, safe and give our war­fighters the competitive advantage they deserve in the battle sphere? Our experiences as practitioners alone constitute a rather convincing argument as far as an ideal training principle goes to meet that end. As simple as it sounds, it’s all about conditioning our students to ask the “whys” and “hows” back in their respective workplaces, and thereby more fully prepare them for unattended consequences. Rarely will you find a course or workshop at the Defense Acquisition University (DAU) without some combination of both. Critical thinking and ethical considerations are carefully woven into many of these learning opportunities along with the necessary functional focus. More and more though, DAU is driving its students to think more about their own thinking (AKA metacognition), by asking a lot of “whys” well before they ever get to the “hows” and “whats.” <h3>Striking the Learning Chords in Class</h3> In a DAU classroom, the colorful NASA videos and slides of nominal mission profiles and life on orbit demonstrate the marvel of space travel, notwithstanding the inherent risks. However, using the NASA Space Shuttle Challenger and Columbia accidents as the medium for a facilitated discussion quickly magnifies the potential perils. Traveling at speeds fast approaching 25 times the speed of sound is clearly a wonder albeit a treacherous one. For Challenger, 73 seconds after it launched on its 10th voyage in January 1986, an O-ring failed causing one of two Solid Rocket Booster struts to pivot, rupturing the external fuel tank. Tens of thousands of gallons of fuel cascaded into a white-hot exhaust. Challenger was gone.<br> <br> During Columbia’s Feb. 1, 2003, liftoff, a briefcase-sized piece of foam insulation peeled away from the external fuel tank. At about MACH 2.5, that fragment struck and shattered the carbon epoxy leading edge of Columbia’s left wing. Days later, upon re-entry, part of the thermal protection on the leading edge of the left wing vanished, enabling a jet of hot plasma gas that literally melted the critical wing structure and its embedded sensors. Columbia was gone.<br> <br> At NASA, the risks were well known. Theoretically, what could go wrong is considered in every case. Backed by empirical evidence afterward, a finding of what actually goes wrong, what could have been done and what should have been done generally leads to something more obvious as seen in both tragedies. The forensics for these disasters provide a well-documented account of the more dominating leadership decisions at all levels. Did NASA’s leadership back on Earth that these two crews trusted discount the telling signs of a potential disaster? Whatever environmental and cultural pressures reduced their natural propensity to think more critically in the context of ethical persistence helped shepherd a more heartbreaking destiny along the causal chain of events.<br> <br> During class discussions (with 20/20 hindsight), most students fervently agree that neither accident should have occurred. Students start asking, “What were they thinking? How could they ignore the clues? Why didn’t they take any action?” No one intentionally ignored the signs. However, both accidents appeared to simply represent leadership imprudence and reluctance to challenge their own beliefs or accept the recommendations from others. A shortage of critical thinking and ethical persistence sprinkled with group think, cognitive bias, conation and maybe a bit of hubris allowed these two accidents to occur. There are many ways to depict it. Figure 1 illustrates one particular way that highlights the susceptibility of our declarative knowledge in addressing “why” individuals decide to act (or not) and where ethical persistence should enter the decision loop but may not do so.<br> <br> One technique that NASA and the testing community frequently employs is the 60-Minutes Challenge by assuming the worst consequence (e.g., destruction, severe injury or death) relating to the situation or problem under consideration. Imagine a “60 Minutes” reporter with a microphone rushing toward you, demanding: What did you know, when did you know it, and what did you do about it? It’s time to defend your action or absence thereof. The 60-Minutes Challenge promotes more critical thinking, ethical persistence, and the likelihood of any accompanying regrets. <h3>Two Hypothetical Situations to Consider</h3> Nothing stirs curiosity and challenges our own behavior more like these cataclysmic disasters when it comes to influencing the action we would take next, especially when we think about our role and action in the chain-of-events. The test community is no stranger to these scenarios. They face them with regular frequency when a system finds itself at the outer edges of its operating envelope. Although the following two scenarios are fictitious, they serve to test our own resolve along these salient lines: <ul> <li>What will I do if or when I am placed in a similar situation?</li> <li>Will I take the time to think more deeply, and act accordingly to break the chain of events and prevent something like this from happening on my watch, and how far will I go?</li> <li>Is ethical persistence part of my decision equation?</li> </ul> <strong><img alt="" src="/library/defense-atl/DATLFiles/Nov-Dec_2019/Def_Acq_Nov-Dec19_article3_figure1.jpg" style="margin-left:3px;margin-right:3px;width:682px;height:400px;" /><br> Scenario 1 </strong><br> You are the Test Lead on a Foreign Military Sale (FMS) for a Fourth Generation Fighter involved in the development testing for a modification at the customer’s request. As a part of the sale, the customer wants the Fighter certified for operations at Mach 3. The aircraft can do Mach 3, but is not designed for sustained operations there. You already know that an aircraft traveling at these kind of speeds generates excessive temperatures. Its kinetic energy converts to tremendous heat through compression and friction. After the flight test, you notice that the air inlet became deformed and the canopy was still too hot to touch after the flight. Both developments raise safety concerns. In your view, the system failed the test, luckily without a catastrophic result. When test results are sent to the program office, the staff there let you know that they don’t plan to share the results since they firmly believe that the FMS customer would never fly at Mach 3. You wonder why and ask. Your program office counterpart said that he will document what you sent and that “should be good enough.” But that’s not good enough for you. You contact your 0-6 and explain your concerns. She says that she’ll follow up, and take it from here. A couple of months go by, and you later find out that the sale went through. Your curiosity compels you ask your O-6 if the FMS customer was notified of the limitations. She said they have what they need to know, and you decide that’s good enough. What do you do? <ul> <li>Stop testing because the data are good and you trust the team?</li> <li>Keep testing because the program manager needs the data and hope nothing bad happens?</li> <li>Raise your concerns to the program manager and the base safety officer to get another look at the test cases with aviation safety in mind?</li> </ul> <strong>Scenario 2 </strong><br> You start noticing signs that your chief test pilot might have a drinking problem. Occasionally, he shows up at work late smelling of spirits and seems a little unsteady. You’re concerned about his safety and that of others around him whenever he’s in the air. Until now, you’ve modeled your behavior after him. He is a highly decorated veteran of two wars and has logged more than 10,000 flight hours and 200 combat sorties. He is a legend within the fighter community and one of the humblest officers you have ever encountered. If you speak up, you might ruin his career and maybe call to attention to your own if you’re wrong. What if it’s not alcohol you smell? You are in charge and responsible for your team. You decide to speak to your supervisor, and he says he’ll check it out. A month goes by, and you haven’t seen a change. The smell of spirits is still prevalent. Your orders come through for your next assignment to the Pentagon. What do you do? Some choices: <ul> <li>Ignore the situation, keep flying and hope nothing goes wrong.</li> <li>Hold off doing anything because he is an experienced test pilot and you don’t want to jeopardize his career.</li> <li>Confront him as the concerned test lead and own the problem. Challenge his behavior and your concerns, and take any appropriate action.</li> </ul> The day before you depart, the chief test pilot has a Class A mishap. His aircraft is totaled, and he is killed. There is talk that he failed to eject. In both cases, what did you know, when did you know it, and what did you do about it? If you did nothing, or decided to transfer “ownership responsibility” to someone else or someone more senior, yet you believe they did not act as you would have, what inhibited further action on your part? Was it a lack of consideration for more critical thinking, and its inextricable link to ethical persistence when it mattered the most? When fully invigorated, these are powerful combinations to meet the challenges required by more rapid acquisition pursuits riddled with risk as well as time-urgent operational demands saturated with danger. <h3>Summary</h3> If we relentlessly employ more critical thinking along with the tenacity of ethical persistence, the decisions that warrant both will go well beyond acting on declarative knowledge alone. Without experiencing these challenges and failures through hands-on learning simulations in class, the acquisition workforce is more likely to experience them for the first time, which might be too late. NASA had no shortage of technical experts. Seemingly, they may have run short of ethical persistence. Both failures didn’t live in the technical world. They lived outside it. More recently, Boeing Aircraft Corporation faced a test of its own. When Boeing could have taken “reasonable precautionary measures” to immediately ground the 737 MAX 8 aircraft after the second crash, it chose not to. If two relatively new airplanes of the same model crash shortly after each other under what appears to be under similar circumstances, why did regulators need to wait until they knew for sure what caused the crashes before they took action? As a regulator, what would you have done to be more critical and ethically persistent? More important, what will you do next time as an acquisition professional in your role along the causal chain of events, especially when the life of others could be placed at risk? <hr />Spring, a graduate of West Point and former test pilot and NASA astronaut, and program manager, is a professor of Engineering, Test and Evaluation, plus Science and Technology Management at the Defense Acquisition University (DAU) West Region in San Diego, California. He is also a chief learning officer for all Western Ranges and a regional executive coach. Tremaine is the Associate Dean for Outreach and Mission Assistance where he is responsible for providing a wide variety of time-urgent workplace solutions for defense acquisition customers in the West Region of DAU. He holds DAWIA Level III certifications in Program Management and in Systems Planning, Research, Development and Engineering (SPRDE) functional areas, and is an instructor, course manager, mentor and leader in various capacities. He holds a Bachelor of Science degree from the United States Air Force Academy, and a master’s in Research and Development from the Air Force Institute of Technology (AFIT).<br> <br> The authors can be contacted at <a class="ak-cke-href" href="mailto:woody.spring@dau.edu">woody.spring@dau.edu</a> and <a class="ak-cke-href" href="mailto:Robert.tremaine@dau.edu">Robert.tremaine@dau.edu</a>.</div>string;#/library/defense-atl/blog/Disasters--in-the-Making
Intelligence Acquisitions - Policy Complexity Drives Critical Thinkinghttps://www.dau.edu/library/defense-atl/Lists/Blog/DispForm.aspx?ID=154Intelligence Acquisitions - Policy Complexity Drives Critical Thinking2019-12-16T17:00:00Zhttps://wwwad.dauext.dau.mil/library/defense-atl/PublishingImages/DefAcqNov-Dec19_banner2.jpg, https://www.dau.edu/library/defense-atl/PublishingImages/DefAcqNov-Dec19_banner2.jpg https://wwwad.dauext.dau.mil/library/defense-atl/PublishingImages/DefAcqNov-Dec19_banner2.jpg<div class="ExternalClass9F257651FBD1487A95B0D27275F3556E">In their Strategy+Business magazine (spring 2018) article “How to Cultivate Leadership That Is Honed to Solve Problems,” Massachusetts Institute of Technology (MIT) Professors Deborah Ancona and Hal Gregersen described the secret to generating boundary-spanning innovation through establishing an environment for solving really hard, edgy, cool problems. “Challenges are cherished at MIT because they offer opportunities to test and prove one’s skill and push the boundaries of what is possible. Presented with some barely achievable objective, people dive in to work the problem, and the more wicked the problem the better.” MIT calls this challenge-driven leadership, a focus on problem solving to push the state-of-the-art for technology.<br> <br> <img alt="" src="/library/defense-atl/DATLFiles/Nov-Dec_2019/Def_Acq_Nov-Dec19_article2_fig1.jpg" style="margin-left:3px;margin-right:3px;float:left;width:532px;height:400px;" />In the December 2018 United States Naval Institute’s Proceedings magazine article, “Naval Intelligence’s Lost Decade,” the author describes former Chief of Naval Operations ADM Gary Roughead’s November 2009 memorandum referencing the actions of Spanish explorer Hernan Cortes in motivating his men to conquer the new world by leaving no means of escape. The boats were burned and Cortes’ men had the choice of conquer the land or die trying. This action is a metaphor for bold, decisive actions required to spearhead organizations through fundamental change to achieve exceptional performance. The article explains that industry has moved forward in mass digitalization, artificial intelligence, robotics and rapid technological change; however, this fundamental change has yet to occur within Naval Intelligence. Leveraging these industry achievements requires acquisition and the ability to acquire industry efforts through a contractual agreement. The complexity of intelligence acquisition policy challenges the Department of Defense (DoD) to leverage industry’s technological achievements, particularly in a rapid response environment. <h3>Background</h3> The Office of the Director of National Intelligence (ODNI) started operations on April 22, 2005, resulting from the Sept.11, 2001, attacks and a post-9/11 investigation proposing sweeping change in the Intelligence Community (IC), including the creation of a Director of National Intelligence (DNI). The Intelligence Reform and Terrorism Prevention Act (IRTPA) of 2004 was signed into law on Dec. 17, 2004. The DNI serves as the head of the IC, overseeing and directing the implementation of the National Intelligence Program (NIP) budget and acting as the principal advisor to the president, National Security Council, and the Homeland Security Council for intelligence matters related to national security.<br> <br> As depicted on ODNI website: <blockquote> <p>The core mission of the ODNI is to lead the IC in intelligence integration, forging a community that delivers the most insightful intelligence possible. That means effectively operating as one team: synchronizing collection, analysis and counterintelligence so that they are fused. This integration is the key to ensuring national policymakers receive timely and accurate analysis from the IC to make educated decisions. The mission of ODNI is to lead and support IC integration; delivering insights, driving capabilities, and investing in the future. The vision of ODNI is a decisive national security advantage through agile leadership of the IC</p> </blockquote> <br> As outlined in IRTPA of 2004, the DNI is responsible to: <ul> <li>Ensure timely and objective national intelligence is provided to the President, the heads of departments and agencies of the executive branch, the Chairman of the Joint Chiefs of Staff, senior military commanders, and Congress.</li> <li>Establish objectives and priorities for collection, analysis, production, and dissemination of national intelligence.</li> <li>Ensure maximum availability of and access to intelligence information within IC.</li> <li>Develop and ensure the execution of an annual budget for the NIP based on budget proposals provided by IC component organizations.</li> <li>Oversee coordination of relationships with the intelligence and security services of foreign governments and international organizations.</li> <li>Ensure the most accurate analysis of intelligence is derived from all sources to support national security needs.</li> <li>Develop personnel policies and program to enhance the capacity for joint operations and to facilitate staffing of community management functions.</li> <li>Oversee the development and implementation of a program management plan for acquisition of major systems, doing so jointly with the Secretary of Defense (SECDEF)for DoD programs, that includes cost, schedule and performance goals and program milestone criteria.</li> </ul> <img alt="" src="/library/defense-atl/DATLFiles/Nov-Dec_2019/Def_Acq_Nov-Dec19_article2_fig2.jpg" style="margin-left:3px;margin-right:3px;float:right;width:407px;height:400px;" />Sixteen organizations compose the IC: Air Force Intelligence, Surveillance and Reconnaisance Agency; Army Intelligence; the CIA; Coast Guard Intelligence; Defense Intelligence Agency (DIA); Energy Department; Department of Homeland Security (DHS); Department of State; Treasury Department; Drug Enforcement Administration (DEA); the FBI; Marine Corps Intelligence; National Geospatial-Intelligence Agency (NGA); National Reconnaissance Office (NRO); National Security Agency (NSA); and Office of Navy Intelligence—as illustrated in Figure 1.<br> <br> There are six basic intelligence sources or collection disciplines: signals (SIGINT), imagery (IMINT), measurement and signature (MASINT), human-source (HUMINT), open-source (OSINT), and geospatial (GEOINT). SIGINT is derived from signal intercepts comprised of communications (COMINT), electronic (ELINT), and foreign instrumentation signals (FISINT).<br> <br> The integration of intelligence information within specific sources or disciplines, as well as across sources and disciplines, creates the ability to link actions and events that might otherwise be considered independent. Figure 2 illustrates intelligence discipline integration. This also creates complexity across the 17 intelligence organizations that have pieces of the intelligence discipline within a greater organizational structure, such as the military Services that are part of DoD and the Coast Guard that is part of DHS but maintains a role with DoD.<br> <br> The U.S. Intelligence budget has two components: NIP and Military Intelligence Program (MIP). NIP includes all programs, projects and activities of the IC to include other IC programs designated jointly by the DNI and the head of department or agency, or the DNI and the President. MIP is devoted to intelligence activity within the military departments and agencies in the DoD that support tactical U.S. military operations.<br> <br> <img alt="" src="/library/defense-atl/DATLFiles/Nov-Dec_2019/Def_Acq_Nov-Dec19_article2_fig3.jpg" style="margin-left:3px;margin-right:3px;float:left;width:544px;height:400px;" />ODNI’s role for integration creates a common picture for intelligence collection and analysis, information, and policy. ODNI influences the 16 other intelligence organizations by influencing the budget for each organization. DoD organizations receive a combination of NIP and MIP; NIP used for the national common efforts and MIP used for specific Service mission. Figure 3 depicts the integration of the IC into a common database for information. Delineating and leveraging the budgeting and execution of NIP and MIP funds affords optimal intelligence performance. <h3>Policy Paradox</h3> DoD acquisition policy regarding developing defense systems is found in DoD Directive (DoDD) 5000.01 The Defense Acquisition System and DoD Instruction (DoDI) 5000.02, Operation of Defense Acquisition System. The Defense Acquisition System is a process-dependent, decision-making system to mature technology from basic research to system disposal. The process has decision points or milestones to evaluate progress and consciously decide to invest further funding for system development. Developed systems transition from development phase into procurement phase and then operation and maintenance phase. Operational commands operate and maintain the systems that are developed and procured by the acquisition commands.<br> <br> For acquiring services, the applicable policy is DoDI 5000.74, Defense Acquisition of Services. The policy assigns responsibilities and provides procedures for defining, assessing, reviewing, and validating requirements for the acquisition of services. The policy authorizes decision authority consistent with statutory and regulatory requirements for the acquisition of services allowing tailoring of procedures to best achieve cost, schedule and performance objectives.<br> <br> Conversely, IC acquisition policy is found in Intelligence Community Directive (ICD) 800, Acquisition, Intelligence Community Policy Guidance; Intelligence Community Policy Guidance (ICPG) 801.1, Acquisition; and Intelligence Community Standard (ICS) 801-4 IC, Services Acquisition. Each armed Service/organization also has its own acquisition policy providing more specific details for execution. The relationships between these two policy foundations are not always consistent, and the overarching document addressing the overlap of DoD and IC acquisition is the Memorandum of Agreement (MOA) between Director of National Intelligence (DNI) and the SECDEF concerning the Management of Acquisition Programs executed at the DoD Intelligence Community Elements dated March 2008.<br> <br> <img alt="" src="/library/defense-atl/DATLFiles/Nov-Dec_2019/Def_Acq_Nov-Dec19_article2_fig4.jpg" style="width:698px;height:400px;" /><br> The MOA provides that ODNI and DoD will jointly conduct oversight for wholly or majority NIP-funded acquisition programs. Quarterly, the parties are to jointly review and assess program execution against Milestone Decision Authority (MDA)-approved baselines for cost, schedule and performance. As the SECDEF sets policy for DoD, the MOA flows down to military Services, requiring joint management, unless an alternative agreement has been documented. Although specific policy is not established in practice for non-Major System Acquisitions (MSA), the expectation is that MSA policy provides the guidance for best practices in implementation. Figure 4 depicts a simplified overlap of DoD and IC policy for acquisition; however, Title 50 includes DoD authority for specific purposes.<br> <br> IC MSA are equivalent to DoD Acquisition Category (ACAT) I and II programs: (MSA Research, Development, Test and Evaluation [RDT&E]) greater than $200 million (Fiscal Year [FY] 2017 base year), ACAT I RDT&E greater than $480 million (FY 2014 base year), and ACAT II RDT&E greater than $185 million (FY 2014 base year). ACAT III programs are all others not considered an ACAT I or II. DoD ACAT III programs require oversight, per the DoDI 5000.02. For the Navy, below ACAT III is further broken into ACAT IVM, ACAT IVT, and Abbreviated Acquisition Program (AAP).<br> <br> DoDI 5000.74 established oversight of service acquisitions through a Senior Service Manager (SSM) citing a program management chain-of-command from program manager to SSM. This service policy tailors the acquisition to five service categories, dollar-based.<br> <br> The MOA between DNI and SECDEF states that “wholly or majority NIP-funded acquisition programs shall be executed according to IC acquisition policy. This will be implemented through direct reference of the DNI policy in DoD 5000.” The SECDEF and DNI may delegate MDA to a DoD IC element agency head for wholly or majority NIP-funded acquisition program. The delegation would be captured in a formal memorandum, with review conducted by Deputy Director National Intelligence/Acquisition (DDNI/AQ) and Under Secretary of Defense for Acquisition and Sustainment for assurance that a mature, repeatable and fundamentally sound acquisition program is in place prior to the recommendation of the delegation of MDA. A joint assessment will be accomplished at least annually to assess need for changes of MDA delegation. <h3>Organizational System Performance</h3> The intelligence mission is unique and drives the establishment of intelligence commands that must perform both operations and acquisition functions. The same command organization develops, procures, operates, maintains and disposes of the systems. This is a paradox for military services that traditionally separate the roles into operational commands and acquisition commands. Even Combatant Commands have recognized acquisition missions. Special Operations Command (SOCOM) established an acquisition center within the command. Transportation Command and Cyber Command have an Acquisition Executive and Program Executive Offices (PEOs).<br> <br> <img alt="" src="/library/defense-atl/DATLFiles/Nov-Dec_2019/Def_Acq_Nov-Dec19_article2_fig5.jpg" style="margin-left:3px;margin-right:3px;float:left;width:441px;height:600px;" />The intelligence mission is data-oriented, requiring an information technology (IT) platform that may interface with other military Service platforms of aircraft, ships, submarines and space-enabling systems. This drives an IT-oriented workforce regardless of the primary acquisition discipline. The March-April 2018 Defense ATL magazine article, “Including Cybersecurity in the Contract Mix,” emphasized that cybersecurity crosses all acquisition disciplines: program management, IT, engineering, test and evaluation, finance, logistics and contracting, and should be included in the earliest phases of contract planning from acquisition planning to contract maintenance and closeout. It is imperative that the contracting officer understand the program’s cybersecurity requirements and construct a contracting strategy to determine whether offerors are capable of delivering those requirements. Figure 5 reflects the intelligence acquisition puzzle of balancing requirements across all policies and initiatives. <h3>IT and Chief or Command Information Officer (CIO) Role</h3> All IC elements, to include those of the DoD Armed Services, depend heavily on Information Technology, Information Management, and Cybersecurity (IT/IM/CS) capabilities to enable the prompt and sustained conduct of their assigned missions. It is imperative that those elements gain and maintain the freedom of action needed to acquire such capabilities in a manner that maximizes Intelligence mission value by meeting or beating cost, schedule and performance requirements linked to threats and/or opportunities. To an increasing degree, that necessary freedom has less to do with the traditional work of “making things” and more to do with buying services securely from the most innovative segments of the commercial marketplace. This shifts policy implementation from DoDI 5000.02 and developing systems to DoDI 5000.74 and acquiring services.<br> In order to become an effective buyer of commercial IT services, the IC elements of the DoD Armed Services must master the performance of IT/IM/CS Service Management (SM) functions based on commercial best practices. Mastery will arise from each element’s IT, Acquisition and Mission Business Owner (MBO) teams working together with commercial subject-matter experts to learn how best to balance in a hybrid fashion the operation and maintenance of specialized capabilities that must, because of mission imperatives, remain on-premises and the consumption of commoditized services provisioned by external entities, particularly commercial entities involved in the provisioning of properly secured cloud services.<br> <br> The DoD CIO is the principal staff assistant and senior advisor to the SECDEF and Deputy SECDEF for IT (including national security systems and defense business systems), information resources management (IRM) and integration efficiencies; therefore, the DoD CIO is responsible for all matters relating to the DoD information enterprise, such as cybersecurity, communications, information systems and more. This role and responsibility flows down to the military Services and organizations of the Fourth Estate (i.e., predominantly civilian, non-Service sectors of DoD). The DoD CIO role complements the role of the MDA for IT systems and Defense Business Systems.<br> In comparison, the IC CIO is the MDA for IT systems for national intelligence mission as established by National Security Act of 1947, Section 103G: <blockquote> <p>The IC CIO shall manage activities relating to IT infrastructure and enterprise architecture requirements, have procurement approval authority over all IT items related to the enterprise architectures of the IC components, direct and manage all IT-related procurement for the IC, and ensure that all expenditures for IT and research and development activities are consistent with IC enterprise architecture and the strategy of the Director for such architecture.</p> </blockquote> <h3>Critical Thinking</h3> For programs that intersect DoD and IC, specific “rules of the road” for that acquisition need to be established and managed differently. Developing an acquisition strategy accommodating the intelligence acquisition puzzle requires a network or decision-tree diagram approach with intersections leading to multiple forks. Choosing the right fork is a risk-based decision. The shift from an organization doing IT to procuring IT through service models of Infrastructure-as-a-Service (IAAS), Platform-as-a-Service (PaaS), and Software-as-a-Service (SAAS) changes the skills required for the organization. Defense Acquisition Workforce Improvement Act (DAWIA) certifications are required for personnel managing acquisitions through establishing requirements and purchasing these IT services. Leveraging commercial industry may advance technology but also increase risk to cybersecurity.<br> <br> The overlap of knowledge from multiple career fields sheds insight between the fields. In the March-April 2018 Defense AT&L magazine, the article “Interdisciplinary Competence” described the benefits of interdisciplinary knowledge: <blockquote> <p>Integrated and interdisciplinary teams achieve better problem-solving skills by leveraging common knowledge. Results from academic institutions and a 3M Company study support the development of depth and breadth in disciplines to achieve exceptional performance… Complex problems cross disciplinary fields and require the use of multiple disciplines to develop a solution… An interdisciplinary perspective requires bridging knowledge between disciplines to address complex problems. Successful teams integrate multiple disciplines to frame a problem, agree on a methodological approach, and collaboratively analyze data. Exceptional teams do a better job of integrating knowledge… Greater integration of disciplinary knowledge enables the development of more effective critical thinking and innovative ideas than are possible in traditional multidisciplinary teams.</p> </blockquote> <br> Critical thinking encompasses the process of actively conceptualizing, applying, analyzing, synthesizing and evaluating information to resolve a problem or draw a conclusion. Acquisitions for the intelligence mission benefit from critical thinking and the ability to reconcile the applicable policy between DoD and the IC. Critical thinking will springboard the Naval Intelligence mission, as well as DoD Intelligence mission, to yield the results envisioned by Roughead. <h3>Conclusion</h3> Complexity influences the ability to balance planning with the chance to anticipate and respond to changing conditions and feedback. The integration of operational and acquisition within the same command adds complexity; however, this complexity affords the opportunity to more closely align the user and developer expertise, a challenge for command structures that are separated. Agile acquisition principles emphasize the users’ involvement throughout the acquisition with feedback shaping each iteration.<br> In addition, the overlap of policy further adds integration complexity affording the workforce the opportunity to apply critical thinking. In simplistic situations, separation and reduction of the whole into smaller manageable pieces achieves optimal performance. However, under complexity where the whole is not a summation of the pieces but something different, leveraging the integration of operational and acquisition can achieve nonlinear performance. The expression, “the whole is greater than the sum of the parts” reflects a nonlinear relationship versus a linear relationship. SOCOM has experienced favorable performance with an acquisition center embedded within an operational command.<br> <br> The Intelligence acquisition mission is different from other defense acquisition missions, requiring understanding of related problems and developing different solutions. The expertise of the workforce is different, requiring an integration of knowledge across disciplines for teams, as well as individuals. Critical thinking enables identifying the critical aspects of policy integration across all policy owners to ensure appropriate policy implementation without compromising the intelligence mission.<br> <br> Intelligence acquisitions afford individuals the opportunity to work complex, as well as really hard, edgy, cool problems, and to generate boundary-spanning innovation while developing critical thinking acumen. The leaders of organizations such as MIT, Google, Microsoft, and Apple understand developing a forward-thinking workforce means challenging them at every level. In order to accomplish this effort, continuous training within their expertise and across other fields is needed. Intelligence acquisition requires elements of the fast-paced commitment of agile project management to leverage technology advances and maintain a cutting-edge mission. <hr />McIlwain is Senior Leader for Acquisition for Naval Intelligence. Page is the Command Information Officer at Naval Intelligence Activity (NIA).<br> <br> The authors can be contacted at <a class="ak-cke-href" href="mailto:carol.mcilwain@navy.mil">carol.mcilwain@navy.mil</a> and <a class="ak-cke-href" href="mailto:christopher.j.page@navy.mil">christopher.j.page@navy.mil</a>.</div>string;#/library/defense-atl/blog/Intelligence-Acquisitions----Policy-Complexity-Drives-Critical-Thinking
Learning at the Speed of Relevancehttps://www.dau.edu/library/defense-atl/Lists/Blog/DispForm.aspx?ID=156Learning at the Speed of Relevance2019-12-11T17:00:00Zhttps://wwwad.dauext.dau.mil/library/defense-atl/PublishingImages/DefAcqNov-Dec19_banner4.jpg, https://www.dau.edu/library/defense-atl/PublishingImages/DefAcqNov-Dec19_banner4.jpg https://wwwad.dauext.dau.mil/library/defense-atl/PublishingImages/DefAcqNov-Dec19_banner4.jpg<div class="ExternalClass2DCA1E9C6F4C4CFB818BE9DA433FCCAE">The National Defense Strategy (2018) encourages the acquisition communities to foster a competitive mindset. This competitive mindset requires business practice reforms for superior performance and affordability, replacing outdated business practices that have contributed to our increasing unresponsiveness. The need for speed of relevance requires many of us to rethink how we currently do things. As stated in the 2018 National Defense Strategy, “Success no longer goes to the country that develops a new technology first, but rather to the one that better integrates it and adapts its way of fighting.”<br> <br> Resuscitating existing authorities and repacking old ideas or decade-old streamlining practices may look like sufficient change for those who cannot get out of their comfort zone. If this statement causes a reaction, it was meant to do so because your attention is needed. As members of this acquisition community, we can be part of the problem. Transitioning from a complacent to a competitive mindset requires transformation or deep change. In his 1996 book, Deep Change: Discovering the Leader Within, Robert E. Quinn opined that, unlike incremental change, deep change requires a new way of thinking and behaving. It requires innovative thinking. Innovative thinking comes from learning to solve complex problems in a nontraditional way. If we change how we learn, we can solve complex problems. Innovation should not and cannot be limited to our research and development community. Innovative thinking should be inculcated into our day-to-day work life.<br> <br> <img alt="" src="/library/defense-atl/DATLFiles/Nov-Dec_2019/Def_Acq_Nov-Dec19_article4_quote1.jpg" style="margin-left:3px;margin-right:3px;width:500px;height:139px;" /><br> <br> Innovative thinking needs to be present at every level of our acquisition community…whom we recruit, whom we promote, and whom we retain. A holistic approach involves innovative thinking not only with the program manager and his or her acquisition team but with policymakers, academic community, human capital management, finance and industry. We can either continue operating as we have been, which will mean staying stuck in old ways of thinking, or we can learn how to change the equation and be part of the solution.<br> <br> Business practice reforms can start with you and your acquisition team through innovative learning at the speed of relevance. This article will give you tips on how to improve the way you learn and enhance your ability to make good business decisions that foster a competitive mindset. Being part of the solution will mean exploring new ways to learn. Improving the way you learn can change your perspective and commitment to acquisition excellence. We cannot solve wicked acquisition problems using only successful methods of the past, including our methods of teaching and learning. Deep change comes from new ideas. So, let’s get started.<br> <br> First, we need to understand how our brain works. Neuroscientists now recognize that our brain operates in two different ways—focused mode and diffuse mode (see Barbara Oakley’s 2014 book, A Mind for Numbers: How to Excel at Math and Science). Our brain’s cerebral hemispheres support these two distinct modes that are associated with learning. The focused mode (left side of brain) is a narrow thought process that filters out information that does not fit a given situation based on past experiences. The diffused mode (right side of brain) is more tolerant of ambiguities and remains adaptive, taking in more nonliteral, nonverbal clues to assess situations through sensing and perceptions.<br> As adult learners, we often tap into prior experience to guide our decision making. While this has many positive benefits, it can impede “outside of the box” thinking. We should be striving to create new neural pathways to solve new problems. Instead we continue to solve today’s problems with solutions using old neural pathways that were used to solve problems from our past. This is referred to as the Einstellung effect or tendency to habitually rely on our existing cognitive framework which results in negative consequences of innovativeness. Transformational experiences, or “ah-ha” moments, are lost by relying too heavily on existing cognitive frameworks; but when we are able to let go of stagnant ways of thinking, or when new meaning is given to prior experiences, we can see old problems in a new light (see Jack Mezirow’s 1990 book, Fostering Critical Reflection in Adulthood: A Guide to Transformative and Emancipatory Learning).<br> <br> Unfortunately, relying on existing cognitive frameworks often is why we see no real change; even with the advancement of technologies, deep change has not occurred. Past thinking often divided brain functions into hemispheres—left brain or right brain with one side playing a more dominant role. The new school of thought is that this is simply not true. In order to solve difficult problems, we must include diffuse mode thinking, and as long as we are consciously focused (i.e., using focused mode) we are blocking our abilities to use the diffuse mode. In order for us to have a competitive mindset as decision makers, policymakers and practitioners within our complex defense acquisition environment, we need to switch between these modes in performing different activities. A new way of learning is required by the acquisition workforce, with a greater emphasis on adult learning principles, as well as an overhaul of our business processes and human development programs. We need deep change in the way that we learn in order to problem solve the wicked problems our Defense acquisition community faces.<br> <br> Second, we need to understand how our brains store information. Research findings of neuroscientists and psychologists concluded that the most effective way for information to be stored in our brains is through encapsulating ideas into chunks of information (i.e., smaller pieces or sequence learning tasks), so that the information received can be properly stored in our memory systems. Using the focused mode side of our brain to chunk learning activities can help transfer information into our working memory. As these newly form chunks of information enter our brain, the diffuse mode function of the brain can retrieve these chunks to form innovative ideas and problem solving, thus avoiding the Einstellung effect.<br> <br> Learning more effective ways to learn and interleaving focused mode and diffuse mode techniques can lead to better problem solving. Hermann Ebbinhaus, considered a pioneer of memory research, published a book in 1913, titled Memory: A Contribution to Experimental Psychology, and suggested that we learn best by forgetting. This may sound counterintuitive at first, but when we return to material studied once, and then largely forgotten, the relearning of that material strengthens neural connections. If this process is repeated over time, known as spacing, the brain creates even stronger, more permanent connections. Therefore, enhancing the learning process also involves recalling newly learned chunks of information using a retrieval practice (i.e., knowledge review quiz or practice exam used in an academic settings or recalling information learned by testing yourself instead of rereading the material). Retrieval practice is a knowledge retention strategy that provides a better opportunity for storing the information gained and transferring it into our long-term memory system, especially when you are unfamiliar with the content.<br> <br> Another knowledge retention strategy involves interleaving information. Interleaving information means organizing information by intermixing rather than grouped by topic or type. Increased learning performance occurs when a subject, e.g., statistics, was being introduced, individuals will understand and retain more information when after each lesson, a set of practice exercises follows that provide a brief review of concepts that were learned several days or weeks prior. This interleaving technique creates a degree of time intervals or space between learning which enhances memory retention and learning performance. Terry Kidd and Lonnie Morris explained this process as modular learning in their 2017 book, Handbook of Research on Instructional Systems and Education Technology. Modular learning is the process of chunking new information using interleaving technique and time intervals.<br> <br> Third, we need to increase the use of technology. More than a decade of research and met-analysis led Claire Wladis, Katherine Conway and Alyse Hachey in a 2017 article, titled “Studies in Higher Education,” to conclude that the learning outcomes are the same for both online and traditional classroom learning. Integrating technology and modular learning into the Department of Defense and Armed Services academic settings will enable the speed of relevance, as current research findings suggest and improve learning outcomes. As we reform our business practices for superior performance to increase our national defense responsiveness, so must our academic and training communities embrace the use of technology in the design, development and delivery of information. Ultimately, increasing job performance through learning, memory retention and relevance of material.<br> <br> Fostering a competitive mindset, for me, is where the sense of urgency comes from … the need for protecting our democratic society and providing our warfighters with the tools and resources they need to defeat our adversaries. As leaders of the Free World, our national defense posture depends on all members of the acquisition workforce. Parts of what we do require deep change. Oakley’s salient points remind us that we need to be willing to “learn how to learn” using new and innovative methods and we need to expect this willingness from our colleagues as well as ourselves. Our future truly does reside in our brains! <hr /> <h3>Sources and Suggested Reading List</h3> Barbara Oakley, <em>A Mind for Numbers: How to Excel at Math and Science</em>, (New York, New York: Penguin Random House, 1999).<br> Hermann Ebbinhaus, <em>Memory: A Contribution to Experimental Psychology</em>, (New York, New York: Columbia University, 1913).<br> James Mattis, <em>National Defense Strategy of the United States of America: Sharpening the American Military’s Competitive Edge</em>, (National Security Strategy Archive, 2018).<br> Jack Mezirow, <em>Fostering Critical Reflection in Adulthood: A Guide to Transformative and Emancipatory Learning</em>, (New York, New York: Joey-Bass Publishers, 1990).<br> Robert Quinn, <em>Deep Change: Discovering the Leader Within</em>, (San Francisco, California: Jossey-Bass Publishers, 1996).<br> Terry Kidd and Lonnie Morris, <em>Handbook of Research on Instructional Systems and Educational Technology</em>, (Hershey, Pennsylvania: IGI Global, 2017).<br> Claire Wladis, Katherine Conway, and Alyse Hachey, <em>“Using Course-Level Factors as Predictors of Online Course Outcomes: A Multi-Level Analysis at a U.S. Urban Community College,”</em> Studies in Higher Education, Vol. 42, No. 1: 184-200, (2017). <hr />Dowling is a Learning Director in the Contracting Foundational Learning Directorate at Defense Acquisition University (DAU) Fort Belvoir, Virginia. Johnson is Instructional Systems Designer in the Enterprise Integration Directorate at DAU.<br> <br> The authors can be contacted at <a class="ak-cke-href" href="mailto:joni.dowling@dau.edu">joni.dowling@dau.edu</a> and <a class="ak-cke-href" href="mailto:ryan.johnson@dau.edu">ryan.johnson@dau.edu</a>.</div>string;#/library/defense-atl/blog/Learning-at-the--Speed-of-Relevance
The Quest for Rapid Acquisitionhttps://www.dau.edu/library/defense-atl/Lists/Blog/DispForm.aspx?ID=157The Quest for Rapid Acquisition2019-12-04T17:00:00Zhttps://wwwad.dauext.dau.mil/library/defense-atl/PublishingImages/DefAcqNov-Dec19_banner5.jpg, https://www.dau.edu/library/defense-atl/PublishingImages/DefAcqNov-Dec19_banner5.jpg https://wwwad.dauext.dau.mil/library/defense-atl/PublishingImages/DefAcqNov-Dec19_banner5.jpg<div class="ExternalClass7BB1390483914B0B961FFC87C6767D56">One of the most recognized barriers to new technologies transitioning into usable capabilities is the “valley of death,” that gap between promising technologies and the resources required to transition them to warfighting capabilities.<br> <br> Demonstrations, experimentation and prototyping combined with experienced test professionals are all paths to bridge that valley. However, as an experienced test professional who can help with transition, I have often had the impression that the Test Enterprise is viewed as the Black Knight from “Monty Python and the Holy Grail,” a comic villain whose only purpose was to stop or slow progress even when the quest was noble. Instead, I think Test is more like Tim the Enchanter from the same movie. The quest was well under way when he was brought on-board, but he quickly understood the importance of King Arthur’s quest. Once a part of the team, he provided early and valuable insight into the difficulties they faced and continued on their quest with them to provide data about their threat’s big pointy teeth.<br> <br> Those reading this publication are likely aware of the 2018 National Defense Strategy (NDS) and the Department of Defense (DoD) push to restore readiness and increase lethality to achieve decisive advantages against competitive adversaries. What the audience here may be less aware of are the unique Service responses to answer the NDS’ call to “deliver performance at the speed of relevance.”<br> <br> <img alt="The U.S. Air Force Test Pilot School’s NF-16 Variability Stability In-Flight Simulator Test Aircraft (VISTA). Photos from the U.S. Air Force Test Pilot School" src="/library/defense-atl/DATLFiles/Nov-Dec_2019/Def_Acq_Nov-Dec19_article5_image1.jpg" style="width:728px;height:400px;" /> <h5>The U.S. Air Force Test Pilot School’s NF-16 Variability Stability In-Flight Simulator Test Aircraft (VISTA).<br> Photos from the U.S. Air Force Test Pilot School</h5> <br> Let’s look at how the Air Force Test Enterprise is postured to increase performance and affordability. As a critical element in acquisition of these capabilities, the Air Force Test Enterprise focuses on enabling increased performance through innovation and streamlined rapid, iterative developmental and fielding approaches.<br> <br> This is not the first time that the Air Force Test Enterprise has responded to the need for speed in acquisitions. In its 2004 response to the Air Force shift to evolutionary acquisition and the spiral and incremental development processes, Headquarters Air Force Test and Evaluation (AF/TE) published Air Force Instruction (AFI) 99-103 Capabilities Based Testing. AFI 99-103 consolidated into a single policy document aimed at integrating test planning and execution three separate test policy documents (AFI 99-101, Developmental Test and Evaluation; AFI 99-102, Operational Test and Evaluation; and AFI 99-105, Live Fire Test and Evaluation).<br> <br> After 2018 National Defense Strategy was published, the Air Force acquisition community pivoted to rapid acquisitions, including utilizing Middle-Tier Authorities (Section 804) and tailored DoD Instruction (DoDI) 5000.02 programs. In response, AF/TE published a policy guidance memorandum that described policy guidance for Test and Evaluation of rapid acquisition programs (Middle-Tier Acquisitions, tailored DoDI 5000.02 programs, and Experiments). The Jan. 10, 2019, memo highlighted five key messages that form the basis for how the Air Force Test Enterprise can best support these programs. These five key messages are: <ul> <li>Test and evaluation’s main goals are to reduce risk and provide data to inform acquisition decisions or answer warfighter capability questions.</li> <li>Rapid acquisition relies on a cycle of rapid learning; test and evaluation is the primary means of learning.</li> <li>Delivering performance at the speed of relevance still requires independent testing (“independent” refers to chain of command separation from program offices).</li> <li>Rapid does not mean reckless, nor does it mean ignoring the “ilities” that make a system suitable as well as operationally effective.</li> <li>The key to rapid testing is early and continuous tester involvement during a program.</li> </ul> <br> In addition to the memorandum informing the update to AFI 99-103, it also helped shape the acquisition community’s soon-to-be published AFI 63-146, Rapid Acquisition Guidance. Rather than discuss the intricacies of policy, it is better to explore these key messages through a vignette that illustrates how early test involvement through prototyping and experimentation can enable the delivery of performance at the speed of relevance.<br> <br> As I suggested earlier, one of the Test Enterprise’s methods to help transition technologies across the “valley of death” is through demonstrations or experimentations using experienced test professionals. One such way this is done by test is through the U.S. Air Force (USAF) Test Pilot School (TPS) Test Management Project (TMP) program. TMPs are part of TPS’ curriculum through which students and staff work with various customers to plan, execute and report on a short-duration test program. TMPs are a graduation requirement for students, but TPS staff sometimes conduct TMPs outside of the curriculum when a program exceeds the scope of curriculum requirements or the timeline does not synch with the academic calendar. What follows is a discussion on a TPS staff-led TMP, aimed at helping to advance technology through the valley of death in order to increase performance and lethality.<br> <br> Born out of previous attempts to integrate flight control and weapons delivery functions in tactical aircraft (Tactical Data System and Integrated Flight and Fire Control programs), USAF TPS staff hatched a plan to enable the transition of a promising technology into a warfighting capability. In 2017, with DoD funding, the staff began a demonstration exploring the feasibility of a fighter aircraft-based automatic gun-tracking capability against air-to-air and air-to-ground targets named Digitally Enhanced Aiming Through Control Law (DEATH CLAW). DEATH CLAW had three objectives: (1) Demonstrate the automatic capability of the DEATH CLAW system to aim the gun in flight, (2) compare human performance against the system’s performance using air targets and both moving and stationary ground targets, and (3) identify plans to transition the system into Block 30 and Block 40/50 F-16 fighter jets.<br> <br> <img alt="" src="/library/defense-atl/DATLFiles/Nov-Dec_2019/Def_Acq_Nov-Dec19_article5_quote1.jpg" style="width:500px;height:124px;" /><br> <br> The demonstration kicked off in August 2017; executed 12 test, evaluation and demonstration missions from November 2017 to January 2018; and issued a final report in March 2018. An 8-month program is rapid by any measure, but unheard-of for flight control law development. Even though it was fast, the test demonstration applied the appropriate rigor necessary to ensure that technical and safety risk were acceptable and that the data collected would be useful for decision makers. TPS collaborated with Lockheed Martin Skunk Works, Calspan, the 412th Test Wing and the 416th Flight Test Squadron to plan, execute, and report on the demonstration. It was flown using the USAF TPS NF-16 Variable Stability In-Flight Simulator Test Aircraft (VISTA).<br> <br> The VISTA provides a safe way to rapidly fly and update flight control law software. In essence, it takes off like a normal F-16. But, once airborne, the test aircrew members activate the test configuration control law, giving them the ability to rapidly evaluate new designs. During this demonstration, the VISTA provided the test team with the capability to fly DEATH CLAW in a controlled manner against both air and ground targets. After each mission, the team reviewed data and was able to update the control algorithm. The ability to rapidly iterate this cycle sped up the process.<br> <br> The demonstration consisted of an integration and checkout phase, formal testing phase and a final demonstration phase. Integration and checkout lasted 1 week and ensured that the DEATH CLAW algorithm could be safely flown and operated as expected. The team flew three times and ensured DEATH CLAW integration with VISTA while refining some of the control laws. The formal evaluation immediately followed the integration phase. The team executed six missions over 1 week during which six different test pilots executed the same mission profile to evaluate the full air and ground target test matrix. Finally, the team executed the demonstration phase using three pilots in leadership positions from Air Combat Command (ACC), Air Force Materiel Command (AFMC) and the Air National Guard (ANG) with varying levels of experience and backgrounds. Following the successful, TPS-led demonstration, both the USAF and the ANG funded integration efforts on their F-16 fleets. The ANG plans to begin flight testing on its Block 30 F-16s in 2020 at the ANG Air Force Reserve Command Test Center (AATC).<br> <br> <img alt="" src="/library/defense-atl/DATLFiles/Nov-Dec_2019/Def_Acq_Nov-Dec19_article5_image2.jpg" style="width:500px;height:234px;" /><br> The DEATH CLAW demonstration proved successful and achieved its three objectives, paving the way for future platform-integration efforts. Estimates showed that costs savings were 90 percent and time savings were 80 percent of what normally would be expected for this type of a program. Success depended on early involvement of the right kind of test and test support organizations. The USAF Test Pilot School, the 412th Test Wing, Calspan, and Lockheed Martin brought deep knowledge and experience in flight-control development and testing. This combined team proceeded rapidly without being reckless.<br> <br> Contributing to planning early in the timeline, the test team members used their knowledge of the risk analysis process to effectively and quickly understand and mitigate safety and technical risks. Furthermore, the program’s limited scope enabled the team to focus and succeed in demonstrating a performance improvement of a lethality enhancement’s technological readiness. It also supported rapid learning during the integration and checkout phase through quick, focused iterations that included experts from the entire team. While limited in scope, the project demonstrated that the test enterprise is able to rapidly produce relevant data to deliver increased technological readiness, ultimately delivering performance to the warfighter.<br> <br> The bottom line in achieving success at the speed of relevance is involving Test early and keeping Test continually engaged. As Dr. William B. Roper Jr. noted in his April 10, 2018, memo, “it takes a team to go fast,” and there is an entire Test Enterprise in the Air Force ready to assist with a shared sense of purpose. If King Arthur and his merry band of knights had brought Tim into the quest at the beginning of the movie, it probably would have been boring for the audience, but he would have reached the final castle earlier and been able to go on additional quests before the authorities put an end to it all. The Air Force Test Enterprise enables rapid acquisitions. Unlike Monty Python’s Black Knight, we promise to not “bite your legs off.” <hr />Massaro is Chief of the Policy, Programs, and Resources Division, Headquarters Air Force Test and Evaluation (AF/TE) in the Pentagon. He is an experimental test pilot with extensive test experience in fighter aircraft. The views expressed in this paper represent the personal views of the author and are not necessarily the views of the Department of Defense nor of the Department of the Air Force.<br> <br> The author may be contacted at: <a class="ak-cke-href" href="mailto:mark.a.massaro.mil@mail.mil">mark.a.massaro.mil@mail.mil</a>.</div>string;#/library/defense-atl/blog/The-Quest--for-Rapid-Acquisition