Research Agenda 2020-2021

This Research Agenda is intended to make researchers aware of the topics that are, or should be, of particular concern to the broad defense acquisition community in the government, academic, and industrial sectors. It is compiled using inputs from Subject Matter Experts (SME) across those sectors. These topics are periodically vetted and updated as needed to ensure they address current areas of strategic interest.
The purpose of conducting research in these areas is to provide solid, empirically based findings to create a broad body of knowledge that can inform the development of policies, procedures, and processes in defense acquisition, and to help shape the thought leadership for the acquisition community. These research topics should be considered guidelines to help investigators form their own research questions. Some questions may cross topics and thus appear in multiple research areas.
Potential researchers are encouraged to contact the DAU Director of Research (research@dau.edu) to suggest additional research questions and topics, or with any questions on the topics.

Affordability and Cost Growth

• Define or bound "affordability" in the defense portfolio. What is it? How will we know if something is affordable or unaffordable?
• What means are there (or can be developed) to measure, manage, and control “affordability” at the Program Office level? At the industry level? How do we determine its effectiveness?
• What means are there (or can be developed) to measure, manage, and control “Should Cost” estimates at the Service, Component, Program Executive, Program Office, and industry levels? How do we determine their effectiveness?
• What means are there (or can be developed) to evaluate and compare incentives for achieving “Should Cost” at the Service, Component, Program Executive, Program Office, and industry levels?
• Recent acquisition studies have noted the vast number of programs and projects that don’t successfully make it through the acquisition system and are subsequently cancelled. What would systematic root cause analyses reveal about the underlying reasons, whether and how these cancellations are detrimental, and what acquisition leaders might do to rectify problems?
• Do joint programs—at the inter-Service and international levels—result in cost growth or cost savings compared with single-Service (or single-nation) acquisition? What are the specific mechanisms for cost savings or growth at each stage of acquisition? Do the data support “jointness” across the board, or only at specific stages of a program, e.g., only at Research and Development (R&D), or only with specific aspects, e.g., critical systems or logistics?
• Can we compare systems with significantly increased capability developed in the commercial market to DoD-developed systems of similar characteristics?
• Is there a misalignment between industry and government priorities that causes the cost of such systems to grow significantly faster than inflation?
• If so, can we identify why this misalignment arises? What relationship (if any) does it have to industry's required focus on shareholder value and/or profit, versus the government's charter to deliver specific capabilities for the least total ownership costs?

Industrial Productivity and Innovation
Industry insight and oversight

• What means are there (or can be developed) to measure the level of insight and/or control that government has over subcontractors?
• What means are there (or can be developed) to measure costs of enforcement (e.g., auditors) versus actual savings from enforcement?
• What means are there (or can be developed) to evaluate and compare incentives for subcontractor/supply chain competition and efficiencies?
• What means are there (or can be developed) to evaluate and compare market-based incentives with regulatory incentives?
• How can we perform institutional analyses of the behaviors of acquisition organizations that incentivize productivity?
• What means are there (or can be developed) to evaluate and compare the barriers of entry for SMEs in defense acquisition versus other industrial sectors?
• Is there a way to measure how and where market incentives are more effective than regulation, and vice versa?
• Do we have (or can we develop) methods to measure the effect of government requirements on increased overhead costs, at both government and industrial levels?
• Examine the possibilities to rationalize and balance the portfolio of capabilities through buying larger quantities of common systems/subsystems/components across Defense Agencies and Services. Are there examples from commercial procurement and international defense acquisition that have produced positive outcomes?
• Can principal-agent theory be used to analyze defense procurement realities? How?
• What means are there (or can be developed) to measure the effect on defense acquisition costs of maintaining the industrial base in various sectors?
• What means are there (or can be developed) of measuring the effect of utilizing defense industrial infrastructure for commercial manufacture, in particular in growth industries? In other words, can we measure the effect of using defense manufacturing to expand the buyer base?
• What means are there (or can be developed) to measure the breadth and depth of the industrial base in various sectors that go beyond a simple head count of providers?
• Has change in the industrial base resulted in actual change in output? How is that measured?

Independent Research and Development)

• What means do we require to measure the cost-effectiveness or Return on Investment (ROI) for DoD-reimbursed IR&D?
• Can we properly account for sales and revenues that are products of IR&D?
• Can we properly account for the barriers to entry for SMEs in terms of IR&D?
• Examine industry trends in Independent Research and Development (IRAD), for example, percentage of revenue devoted to IR&D, collaboration with academia. How do they vary by industry sector, in particular, those associated with defense acquisition?
• What means are there (or can be developed) to measure the ROI for DoD-reimbursed IR&D versus directly funded defense R&D?
• What incentive structures are required to get industry to focus on and fund disruptive technologies?
• What has been the impact of IR&D on developing disruptive technologies?

Competition
Measuring the effects of competition

• What means are there (or can be developed) to measure the effect on defense acquisition costs of maintaining an industrial base in various sectors?
• What means are there (or can be developed) of measuring the effect of utilizing defense industrial infrastructure for commercial manufacture, in particular in growth industries? In other words, can we measure the effect of using defense manufacturing to expand the buyer base?
• What means are there (or can be developed) to determine the degree of openness that exists in competitive awards?
• What are the different effects of the two best value source selection processes (tradeoff versus lowest price technically acceptable) on program cost, schedule, and performance?

Strategic competition

• Is there evidence that competition between system portfolios is an effective means of controlling price and costs?
• Does lack of competition automatically mean higher prices? For example, is there evidence that sole source can result in lower overall administrative costs at both the government and industry levels, to the effect of lowering total costs?
• What are long-term historical trends for competition guidance and practice in defense acquisition policies and practices?
• To what extent are contracts being awarded noncompetitively by congressional mandate, for policy interest reasons? What is the effect on contract price and performance?
• What means are there (or can be developed) to determine the degree to which competitive program costs are negatively affected by laws and regulations such as the Berry Amendment, Buy American Act, etc.?
• The DoD should have enormous buying power and the ability to influence supplier prices. Is this the case? Examine the potential change in cost performance due to greater centralization of buying organizations or strategies.

Effects of industrial base

• What are the effects on program cost, schedule, and performance of having more or fewer competitors? What measures are there to determine these effects?
• What means are there (or can be developed) to measure the breadth and depth of the industrial base in various sectors, that go beyond a simple head count of providers?
• Has the change in industrial base resulted in actual change in output? How is that measured?

Competitive contracting

• Commercial industry often cultivates long-term, exclusive (noncompetitive) supply chain relationships. Does this model have any application to defense acquisition? Under what conditions/circumstances?

What is the effect on program cost performance of awards based on varying levels of competition: (a) “Effective Competition” (two or more offers; (b) “Ineffective Competition” (only one offer received in response to competitive solicitation; (c) “Split Awards” versus winner take all; and (d) “Sole Source.” Improve DoD outreach for technology and products from global markets

• How have militaries in the past benefitted from global technology development?
• How/why have militaries missed the largest technological advances?
• What are the key areas that require DoD focus and attention in the coming years to maintain or enhance the technological advantage of its weapons systems and equipment?
• What types of efforts should DoD consider pursuing to increase the breadth and depth of technology push efforts in DoD acquisition programs?
• How effectively are DoD's global science and Technology (S&T) investments transitioned into DoD acquisition programs?
• Are managers of DoD's applied R&D (i.e., acquisition program) investments effectively pursuing and using sources of global technology to affordably meet current and future DoD acquisition program requirements? If not, what steps could DoD take to improve its performance in these two areas?
• What are the strengths and weaknesses of DoD's global defense technology investment approach as compared to the approaches used by other nations?
• What are the strengths and weaknesses of DoD's global defense technology investment approach as compared to the approaches used by the private sector—both domestic and foreign entities (companies, universities, private-public partnerships, think tanks, etc.)?
• How does DoD currently assess the relative benefits and risks associated with global versus U.S. sourcing of key technologies used in DoD acquisition programs? How could DoD improve its policies and procedures in this area to enhance the benefits of global technology sourcing while minimizing potential risks?
• How could current DoD/U.S. Government Technology Security and Foreign Disclosure (TSFD) decision-making policies and processes be improved to help DoD better balance the benefits and risks associated with potential global sourcing of key technologies used in current and future DoD acquisition programs?
• How do DoD primes and key subcontractors currently assess the relative benefits and risks associated with global versus U.S. sourcing of key technologies used in DoD acquisition programs? How could they improve their contractor policies and procedures in this area to enhance the benefits of global technology sourcing while minimizing potential risks?
• How could current U.S. Government Export Control system decision-making policies and processes be improved to help DoD better balance the benefits and risks associated with potential global sourcing of key technologies used in current and future DoD acquisition programs?

Comparative studies

• Compare the industrial policies of military acquisition in different nations and the policy impacts on acquisition outcomes.
• Compare the cost and contract performance of highly regulated public utilities with nonregulated “natural monopolies,” e.g., military satellites, warship building.
• Compare contracting/competition practices between DoD and complex, custom-built commercial products (e.g., offshore oil platforms).
• Compare program cost performance in various market sectors: highly competitive (multiple offerors), limited (two of three offerors), or monopoly?
• Compare the cost and contract performance of military acquisition programs in nations having single “purple” acquisition organizations with those having Service-level acquisition agencies.

Cybersecurity and Cyberanalytics
General questions

• How can we perform analyses of the investment savings associated with institution of robust cybersecurity measures?
• How can we measure the cybersecurity benefits associated with using continuous integration and continuous deployment methodologies?
• How can we cost the discrete elements of cybersecurity that ensures system operational effectiveness within the categories of system functions, mission execution, system performance and system resilience?
• How can we assess the most effective methodologies for identifying threats quickly, assessing system risk and developing countermeasures?
• How can we establish a repeatable process for incorporating a continuous ATO construct for all software-centric acquisition programs?
• How can we articulate cyber risk vs operational risk so COCOMs can be better informed when accepting new software?

Costs associated with cybersecurity

• What are the cost implications of (adding) cybersecurity to a program?
• What are reasonable benchmarks for cybersecurity cost as a percentage of Prime Mission Product (PMP)?
• What are the key cost drivers associated with cybersecurity?
• Is cybersecurity best estimated as a below-the-line common element (similar to SE/PM or Training) or a PMP element?
• How are risks associated with not incorporating cybersecurity appropriately best quantified/monetized?

Acquisition of Services
Metrics

• What metrics are currently collected and available on services acquisition:
  • Within the Department of Defense?
  • Within the U.S. Government?
  • Outside of the U.S. Government?
• What and how much do these metrics tell us about services acquisition in general and about the specific programs for which the metrics are collected?
• What are the possible metrics that could be used in evaluating services acquisition programs?
  • How many metrics should be used?
  • What is the efficacy of each metric?
  • What is the predictive power of each metric?
  • What is the interdependence (overlap) between metrics?
• How do we collect data for services acquisition metrics?
  • What is being done with the data that are currently being collected?
  • Are the data that are being collected on services acquisition reliable?
  • Is the collection process affecting the data that are collected for services acquisition?
• How do we measure the impact of different government requirements on overhead costs and rates on services contracts?

Industrial Base

• What is the right amount of contracted services for government organizations?
  • What are the parameters that effect Make, Buy decisions in government services?
  • How do the different parameters interact and affect government force management and industry research availability?
• What are the advantages, disadvantages, and impacts of capping pass-through costs, and how do they change with the value of the pass-through costs?
• For Base Operations and Support (BOS) contracts, is there a best size? Should large BOS contracts be broken up? What are the parameters that should be considered?
• In the management of large services contracts, what is the best organization? Is the System Program Office a good model? What parameters should be used in evaluating the advantages and disadvantages of an organization to manage large services contracts?
• What effect does strategic sourcing and category management have on small business if the small business is a strategic source or whether the small business is not a strategic source? ?
• Do the on-ramping and off-ramping requirements of some service contracts have an effect on the industrial base? If so, what are the impacts?

Industry practices

• What private sector business practices, other than maximizing profit, can the government effectively use to incentivize performance and otherwise improve business relationships with vendors?
• What are the best methods for evaluating different incentives to encourage small businesses to participate in government services contracts?
• What potential benefits can the government achieve from long-term supply chain relationships? What are the disadvantages?
• What benefits does industry get from the use of category managers and functional domain experts, and can the government achieve the same benefits?
• How can the government best capture, validate, and use demand management strategies?
• Are current services acquisition taxonomies comprehensive, or can they be improved?

Make/Buy

• What methods can best be used to define the cost value relationship in different classes of service contracts?
• Can we develop a method for determining the “should cost” of different services?
• Can we define and bound affordability of specific services?
• What are the characteristics of “inherently governmental” activities, and how can we evaluate the value of these services based on comparable characteristics in a competitive labor market?
• In services contracts, what are the inherent life-cycle costs, and how do we capture the life-cycle costs in make/buy decision making?
• In the case of government services contracting, what are the factors that contribute to less-than-optimum make/buy decision making?

Category Management/Strategic Sourcing

• What effect does strategic sourcing/category management have on competition?
  • Effects on short term versus long term.
  • Effects on competition outside of the strategic sourcing/category management area of consideration.
• What metrics do different industries use for measuring the effectiveness of their supply chain management?
• Would the centralization of services acquisition contracts have measurable impacts on cost performance? Why or why not?
• What are the fundamental differences between the services taxonomy and the category management taxonomy, and are there means and good reasons to align the two taxonomies?

Contract management/efficacy

• What are the best ways to address the services parts of contracts that include both services and products (goods)?
• In the management of services contracts, what are the non-value-added tasks, and are there realistic ways to reduce the impact of these tasks on our process?
• When funds for services are provided via pass-throughs (i.e., from another organization), how are the requirements tracked, validated, and reviewed?
• Do Undefinitized Contract Actions have an effect on contractor pricing and willingness, or lack of willingness to provide support during proposal analysis?
• For multi-award, Indefinite-Delivery, Indefinite-Quantity (IDIQ)-type contracts, is there a method for optimizing the different characteristics (number of vendors, timelines, on-ramping, off-ramping, etc.) of these contracts?

Policy

• What current government policies inhibit alignment of contractors’ approaches from aligning with the government’s services acquisition programs?

Administrative Processes

• What means are there (or can be developed) to measure the efficiency and effectiveness of DoD oversight, at the Component, Service, and Office of the Secretary of Defense levels?
• What measures are there (or can be developed) to evaluate and compare the costs of oversight versus the cost savings from improved processes?
• What means are there (or can be developed) to empirically establish oversight process metrics as a basis for comparison? Can these be used to establish the relationship of oversight to cost/schedule/performance outcomes?
• What means are there (or can be developed) to study the organizational and governance frameworks, resulting in successful change management?
• To what extent (investment and performance) can scenario/simulations testing improve the delivery of complex projects?
• Is there a comparative statistical divergence between organizational honesty (reality) and contractual relationships (intent) in tendering?
• How does one formulate relational contracting frameworks to better account and manage risk and liability in a collaborative environment?

Human Capital of Acquisition Workforce

• What means are there (or can be developed) to measure ROI for acquisition workforce training?
• What elements of the Professional Military Education framework can be applied to the professionalization of the civilian defense acquisition workforce?
• What factors contribute to the management and successful delivery of modern complex project management, including performance over the project life cycle?
• What behavioral leadership characteristics can be commonly observed in successful complex projects, contrasted against unsuccessful complex projects?
• What is the functional role of talent management in building organizational sustainability, performance, and leadership?
• How do we create incentives in the acquisition workforce (management, career, social, organizational) that provide real cost reductions?

Defense Business Systems
Organizational structure and culture in support of Agile software development methodologies

• At the beginning of the Business Capability Acquisition Cycle (BCAC) process, various steps are used to ensure accurate requirements are thoroughly documented and supported throughout the software development life cycle. How can these documentation requirements and processes be streamlined to support more direct-line communication between the end-user and software engineers? What are the hurdles to implement these changes and how are they overcome? What are the effects of these changes on the organization or agency?
• Regarding new starts, how can the BCAC be modified specifically to support Agile development? How are these changes advantageous or disadvantageous to the customer and organization? Would these changes be helpful or detrimental with R&D versus a concurrent design and engineering software project?
• Generally, readiness review briefings within the BCAC are used to determine if a project is at an acceptable state to go to the next step in the process. If software is developed and released to production within a single Sprint (potentially every 2 weeks), how are Test Readiness Reviews, Systems Requirements Reviews, and Production Readiness Reviews handled? How have the changes made to these events made them more or less relevant?
• How are organizations and agencies structured to support concurrent software design and development? What organizational structure would support R&D and non-R&D Information Technology (IT) capabilities?
• What steps are used to choose Agile as the default software development process versus any other software development methodology (e.g., Waterfall, Spiral, or Incremental) for your organization? What are the effects on project cost, schedule, and performance?
• Within DoD agencies and military branches, has the adaption of Agile resulted in faster deployment of new IT capabilities to the customer? How is this determined and measured?
• Industry often produces software using Agile. The DoD’s BCAC process can be a process that produces an abundance of bureaucracy counter to Agile principles. How does hiring a contractor to implement or maintain IT capabilities and introducing Agile software development methods within a BCAC non-Agile process create conflict? How are these conflicts resolved or reconciled?
• How is IT engineering investment and innovation supported throughout DoD? What organizational or cultural aspects of an agency are specific to that support?

Defense Acquisition and Society

• To what extent should the DoD use the defense acquisition process to effectuate various social policies? The existing procurement regime favors a dizzying array of private interests ranging from organized labor; domestic manufacturers and firms located in areas of high unemployment; small businesses, including disadvantaged and women-owned firms; blind, severely handicapped, and prison industries; and, most recently, environmentally friendly vendors. Affirmatively steering the government’s business from the open marketplace to preferred providers adds complexity, thus increasing transaction costs throughout the procurement process, which absorbs scarce resources. (Source: IBM Center for the Business of Government, http://www.businessofgovernment.org)
• How significant are the transaction costs resulting from the administration’s commitment to transparency (generally, and specifically in the context of stimulus or recovery spending)? In a representative democracy, transparency is critical. But transparency is expensive and time-consuming, and the additional resources required to comply with the recently enhanced disclosure standards remain an unfunded mandate. Thus, the existing acquisition workforce must devote scarce resources to an (admittedly legitimate) end other than the pursuit of value for money or customer satisfaction. Is there an optimal balance or a point of diminishing returns? In other words, at what point does the cost of developing transparent systems and measures exceed the benefits of that transparency? (Source: IBM Center for the Business of Government, http://www.businessofgovernment.org)