Market Research— Promise or Problem?
Dennis P. Longo
Imagine that your program was interrupted by a pre-award protest. You published three Requests for Information and hosted multiple Industry Days to obtain Industry capabilities for the military system development you’re procuring. You issued a competitive request for proposals (RFPs) and then a protest was lodged at the Court of Federal Claims. The protester complained that your market research for the competitive strategy was unsuitable. Now the award of that procurement has been delayed pending results of the protest.
Your colleague down the hall decided to pursue a competitive strategy for her program. As part of her market research, she contacted knowledgeable individuals within the federal government who are administering contracts similar in size and scope to her program, and decided to solicit competitive proposals to promote full and open competition. Now she is preparing to defend her decision before the Government Accountability Office (GAO) in response to a pre-award protest. You and your colleague both performed what you considered due diligence—conducting what you reasoned to be extensive market research to identify commercial capabilities that could meet the government’s minimum requirements and essential mission needs. Evidently, your diligence wasn’t sufficient.
Market research is the careful and thorough process to identify commercial products and services that will meet the government’s minimum needs. It will expand insight into the commercial marketplace to enable an understanding of how quickly technology is advancing, and to obtain information on industry capability and business practices.
When we focus too much on a strategy to procure products or services that will meet the government’s minimum needs, we may easily lose the broader view needed to identify commercial solutions.
My first illustration was derived from a protest adjudicated in 2016 by the U.S. Court of Federal Claims (Palantir USG, Inc. v US No. 16-784C). The court found that the Army was so focused on a developmental approach to a military system that it “failed to seriously consider whether commercial items were available.” Consequently, the market research for the military system development seemed designed to elicit responses that would support a predetermined conclusion.
In my second illustration, taken from a protest decided by the GAO (B-244432), the contracting officer was certain that no small businesses could perform the library services he sought. His decision to solicit with full and open competition, and not set aside the procurement exclusively for small businesses, was solely the result of contacting other agencies, even though numerous small businesses had expressed an interest.
The government in both illustrations appeared to rely on predetermined conclusions rather than adequate market research, and this resulted in unnecessary mission delays and resources spent on resolving bid protests.
Sometimes, when we don’t know what we don’t know, we rely on predetermined conclusions.
A predetermined conclusion may be more obvious, as in the protest of the award by the Labor Department (DOL) of a sole-source contract, resulting in a protest that the GAO ruled on in May 2018.
As required by Federal Acquisition Regulation (FAR) 5.207(c)(16) and 6.302-1(d)(2) DOL published notice that it intended to award a sole-source contract on Jan. 30, inviting companies to submit a capability statement by Feb. 7.
On Jan. 31, however, a day after the notice of intent, the DOL procurement officer signed the Justification and Approval (J&A) for other than full and open competition, and DOL entered into the sole-source contract.
In response to the protest, DOL maintained that the notice inviting firms to submit capability statements was a “mere formality” and that the consideration of the protester’s capability statement was “actually irrelevant” in determining whether the sole source decision was reasonable.
GAO, sustaining the protest, criticized DOL:
The agency’s responsibility to meaningfully consider (the protester’s) capability statement is not a “mere formality”. Rather, the agency’s actions in awarding the noncompetitive contract… are contrary to regulation, rendering the J&A, and the resulting sole source contract, deficient.
Reliance on predetermined conclusions may not be intentional, but it may result from a lack of conducting thorough market research. Adhering to methodical or disciplined market research may avoid the protest plague from infecting your office.
Market Research Methods
Agencies may conduct continual strategic or broad market research to gain a sense of the market and to stay informed about overall market developments, trends and capabilities.
Market research also may take the form of a focused or tactical approach relative to a specific requirement. Tactical market research is intended to provide in-depth information and answer specific questions about the capabilities, products, or services available in the market. The timing, depth, and extent of the investigation depends on the complexity of the procurement.
Market research is essential to acquisition planning. Its purpose is to take advantage of commercial methods and capabilities and to provide data on products and services, industry capabilities and practices. Market research will enable best-value acquisition, optimize potential use of commercial items, reduce military-unique requirements and provide access to advanced technology. But who is responsible for conducting proper market research?
The Acquisition Team Conducts Market Research
We need subject-matter experts who can obtain the best results of market research. Market research should be a team effort. A contracting officer may not be qualified to conduct market research for biological dysesthesia dysfunction studies. Similarly, a 12-member team may be overstaffed for researching the ventilation-filter commercial market.
The Requiring Activity should craft the capability information to be submitted by industry, identify form, fit, and function descriptions, review industry capability statements, revise government Performance Work Statement (PWS) or Statement of Work (SOW) based on industry responses, and determine applicability of commercial items and modifications to commercial items to meet the agency’s need.
The contracting officer should issue pre-solicitation notices—requests for information and sources sought, and draft requests for proposals to promote early exchanges of information, host pre-solicitation conferences to involve potential offerors early in the acquisition process, and conduct other means of stimulating industry involvement.
Subject-matter experts, such as industrial specialists and intellectual property attorneys should be part of the acquisition team as required.
In researching a market, you must gather all of the pertinent information on the capability needed in order to identify the availability and capability of commercial products or services that meet the agency’s requirements and mission needs.
As the previous illustrations indicate, market research results should determine if sources are “capable.” This may appear logical and even obvious, but at least three factors should be part of determining the capabilities of the sources considered:
1. To determine if sources are capable, you must know your requirement.
The person or acquisition team doing market research needs to understand the requirement in order to focus effectively on market research.
For example, the GAO sustained the protest by Triad Isotopes Inc. (B-411360, 16 July 2015) in finding that the agency’s market research could not have reasonably identified sources capable of responding to the requirement because it was too broad and did not align with the requirement.
The agency’s market research stated objective was to award a contract to a contractor that can provide radioisotopes research and included online searches for NAICS (North American Industry Classification System) code 325412, and located 676 concerns. The agency concluded that it was likely to receive viable quotations from at least two responsible small business concerns.
Triad Isotopes, Inc., protested the agency’s decision to issue a request for quotations for the acquisition of radiopharmaceuticals as a Small Business Set Aside, arguing that the agency’s market research was flawed because the NAICS code includes a large array of pharmaceuticals, including cold medicines and lip balms. In short, Triad asserted that the agency had not demonstrated that there was “even one small business that both meet the requirement and meet the delivery requirements” in the request for quotations.
The GAO agreed and sustained the protest.
The market research unnecessarily restricted its scope of capable offerors because it did not align properly to the requirement and effective competition was unachievable.
2. To determine if sources are capable, you must know your market.
It’s important to know your requirement, and understanding what is out there to satisfy your requirement is essential for obtaining the most efficient and cost-effective solution.
In Red River Waste Solutions, LP (B-411760.2, Jan. 20, 2016) the GAO sustained the protest because the Army’s market research focused on prior Army contract history rather than customary commercial practices. In short, the market research failed to support its conclusion that its pricing terms were consistent with customary commercial practice.
The Army’s solicitation required the contractor to collect and dispose of solid waste in designated areas in and around Fort Polk in Louisiana.
Red River protested the requirement that price proposals be submitted on a per-ton basis because the customary commercial practice for refuse collection contracts is to price such contracts on a monthly or per container basis—not on a per ton basis.
Adhering to methodical or disciplined market research may avoid the protest plague from infecting your office.
The Army explained that its market research supported the determination that requiring per ton fixed prices was customary commercial practice because other Army contracts were priced on a per ton basis. And responses solicited from industry and a local refuse company both indicated that this was customary commercial practice.
GAO rejected the Army’s claim and sustained the protest.
The Army could not require a contractor to perform a commercial practice that was not customary in the particular market; but more importantly, the conclusions drawn from market research restricted competition because commercial sources were unwilling to engage in an uncustomary practice in their particular market. Since the agency did not understand the market, the solicitation’s estimated quantities for the various contract line items were overstated.
3. The results of market research should determine if sources are capable, not “technically acceptable.”
The results of market research should determine if there is a reasonable expectation of receiving acceptably priced offers that are capable of performing the contract.
In 2014, the Air Force evaluated responses to its Request for Information (RFI) and industry day discussions and concluded that two of the small business firms that had responded were capable of performing the agency’s requirements as prime contractors. The Air Force limited competition to two small businesses under J&A. Analytical Graphics Inc. (AGI) (B-413385) submitted a protest to GAO.
AGI argued that only one firm could meet 9 of the 10 salient characteristics described in the Air Force’s RFI and that a small business set aside was improper.
GAO ruled that neither the FAR nor GAO decisions require an agency to request, or a prospective small business offeror to provide, a complete technically acceptable approach in response to market research. Agencies need only make an informed business judgment that there is a reasonable expectation of receiving acceptably priced offers from small-business concerns that are capable of performing the contract.
Making a de facto source selection decision based solely on the results of market research limits the number of qualified sources and restricts competition by eliminating the government’s opportunities to leverage commercial solutions.
Lesson from Triad: Make sure the focus of MR aligns with the requirement.
Lesson from Red River: It is not reasonable to rely on other government contracts to establish what constitutes a customary commercial practice.
Lesson from Analytical: The contracting officer must make an informed business judgment to show that the sources selected are capable of performing the work.
Market research is an enabler that will expand insight into commercial marketplace, determine how quickly technology is advancing, and obtain data on products, services, capabilities, and business practices.
The impact of hasty or superficial market research may restrict competition to sources that cannot offer the best and brightest resources toward the requirement. Knowing your requirement, knowing the market, and understanding commercial capabilities will avert the lunacy of awarding a sole-source helicopter development contract to a single-engine airplane manufacturer and avoid wasting years of resources and millions of dollars on a contractor lacking the requisite experience.
Finally, market research activities must be reasonable. For example, despite the interest expressed by six small business concerns to a pre-solicitation notice, one agency’s RFP had no small-business set aside. The RFP instead was issued for full and open competition.
Knowing your requirement, knowing the market, and understanding commercial capabilities will ... avoid wasting years of resources and millions of dollars on a contractor lacking the requisite experience.
As a result of inadequate market research, the contracting officer determined that there was no reasonable expectation that two or more small business firms could perform the work. The market research indicated, however, that the contracting officer failed to take into account available information indicating the interest of capable small business concerns in the procurement.
GAO ruled that the contracting officer did not reasonably consider a small-business set-aside and failed to take into account known information from the market research report that indicated interest from small business concerns.
Conclusion
Market research is flawed when it neglects to fully investigate possible commercially available alternatives to meet agency requirements. The lack of knowledge of the requirement, the commercial market, and industry capability each can have an impact on decisions related to full and open competition.
When we think we know what we want, or may have formed a predetermined conclusion on the product, service, and the vendor, we risk not obtaining the full value of expertise and innovation that may be available in the commercial market.
Additional information on market research and its impact on competition in contracting is available in the Training Library on the DASA P Procurement.Army.Mil (PAM) website at https://spcs3.kc.army.mil/asaalt/procurement/SitePages/NewTraining.aspx.
Read the full issue of
Defense Acquisition magazine
LONGO is an advocate for competition, task and delivery order ombudsman, and senior procurement analyst at Army Contracting Command at Aberdeen Proving Ground, Maryland. A member of the Army Acquisition Corps, he holds a bachelor’s degree from the University of Baltimore and is Level III certified in Contracting. His assignments include acquisition specialist at the Program Manager for Chemical Demilitarization within the U.S. Army Chemical Materials activity and procurement analyst at the U.S. Army Legal Services Agency. He served in the Army from 1971 to 1973 at the Southern European Task Force, Italy, and was deployed to Iraq as a civilian in 2003. He authored the Defense Acquisition University (DAU) Continuous Learning DoD Purchase Card tutorial in 2003 and DAU CON 160 Competition in Contracting course in 2020. He has been an instructor for competition in contracting since 2004.
The author can be contacted at [email protected].
The views expressed in this article are those of the author alone and not the Department of Defense. Reproduction or reposting of articles from Defense Acquisition magazine should credit the authors and the magazine.
