Operational Suitability

Operational Suitability and Testing

Operational suitability is addressed as part of Operational Test and Evaluation (OT&E). Title 10 USC 139, Director, Operational Test and Evaluation, defines OT&E as “…the field test, under realistic combat conditions, of any item of (or key component of) weapons, equipment, or munitions for use in combat by typical military users; and the evaluation of the results of such test.”

Operational Suitability and the Adaptive Acquisition Framework (AAF) 

DoD Instruction (DoDI) 5000.02, Operation of the Adaptive Acquisition Framework states: “OT&E shall be used to determine the operational effectiveness and suitability of a system under realistic operational conditions, including joint combat operations; determine if thresholds in the approved CPD [Capability Production Document] and critical operational issues have been satisfied; assess impacts to combat operations; and provide additional information on the system's operational capabilities.” (Note: bold text added for clarity.)

Quantitative Measures

The October 2005 Memorandum of Agreement (MOA) called Operational Suitability Terminology and Definitions To Be Used In Operational Test and Evaluation (OT&E) established the following quantitative measures of suitability:

• Reliability - The ability of a system and its parts to perform its mission without failure, degradation, or demand on the support system
• Availability - A measure of the degree to which an item is operable and can be committed at the start of a mission when the mission is called for at an unknown (random) point in time
• Compatibility - The capability of two or more items or components of equipment or material to exist or function in the same system or environment without mutual interference. Compatibility may apply to a specific investigation of a system’s electrical, electromagnetic, physical, and man-machine interface characteristics. Because of such applications, compatibility may also be addressed as part of the operational effectiveness evaluation in Operational Test Agencies (OTA) test plans and reports.
• Transportability-  The capability of materiel/personnel to be moved by towing, self-propulsion or carrier via any means, such as railways, highways, waterways, pipelines, oceans, and airways. Full consideration of available and projected transportation assets, mobility plans and schedules and the impact of system equipment and support items on the strategic mobility of operating military forces are required to achieve this capability.
• Interoperability -  is comprised of:
  • The ability of systems, units, or forces to provide services to and accept services from other systems, units, or forces and to use the services so exchanged to enable them to operate effectively together.
  • The conditions achieved among communications-electronics systems, or items of communications-electronics equipment, when information or services can be exchanged directly and satisfactorily between them or their users. The degree of interoperability should be defined when referring to specific cases Interoperability is often addressed as part of the operational effectiveness evaluation in OTA test plans and reports.
• Usage Rates – include the following:
  • Wartime Usage Rates - The quantitative statement of the projected manner in which the system is to be used in its intended wartime environment.
  • Peacetime Usage Rates - The quantitative statement of the projected manner in which the system is to be used in its intended peacetime environment.
  • Maintainability - The ability of an item to be retained in, or restored to, specified condition when maintenance is performed by personnel having specified skill levels, using prescribed procedures and resources, at each prescribed level of maintenance and repair. [
• Safety - Freedom from conditions that can cause death, injury, occupational illness, damage to or loss of equipment or property, or damage to the environment.
• Human Factors - The systematic application of relevant information about human abilities, characteristics, behavior, motivation, and performance. It includes principles and applications in the areas of human engineering, anthropometrics, personnel selection, training, life support, job performance aids, and human performance evaluation. [Within the context of this definition, human factors also may be addressed as part of the operational effectiveness evaluation in OTA test plans and reports.
• Manpower Supportability - The identification and acquisition of military and civilian personnel with the skills and grades required to operate and support a material system over its lifetime at peacetime and wartime rates.
• Logistics Supportability - The degree of ease to which system design characteristics and planned logistics resources (including all applicable Integrated Product Support (IPS) Elements) allow for the meeting of system availability and wartime usage requirements.
• Documentation - Documents used to determine suitability e.g., operator and maintenance instructions, repair parts lists, support manuals, and manuals related to computer programs and system software.
• Environmental Effects and Impacts -
  • Environment - Includes the air, water, land, plants, animals, and other living organisms, man-made structures, historical and cultural resources, and the interrelationships that exist among them and with people. 
  • Environmental Effects - The effects of the natural environment on the system. For example, corrosion is a natural environmental effect caused by weather, ocean conditions, etc.
  • Environmental Impacts - The system’s impact on the natural environment as a result of its operational use, maintenance, transportation, and storage. For example, impacts include pollution (noise, air, and water), threat to endangered species, threat to public health, etc.
• Training and Training Support -  The processes, procedures, techniques, training devices, and equipment used to train civilian and active duty and reserve military personnel to operate and support a materiel system. This includes individual and crew training; new equipment training; initial, formal, and On-The-Job (OJT) training; and Product Support (PS) planning for training equipment and training device acquisitions and installations.

Other Suitability Terminology

Suitability considerations defined above may be aggregated to give a higher level determination of the system’s capability to be placed in field use. When doing so, other terminology related to suitability is used. These other terms are:

• Readiness - State of preparedness of forces or weapon system or systems to meet a mission or to warfight. Based on adequate and trained personnel, material condition, supplies/reserves of support system and ammunition, numbers of units available, etc. Sustainability - The ability to maintain the necessary level and duration of operational activity to achieve military objectives. Sustainability is a function of providing for and maintaining those levels of ready forces, materiel and consumables necessary to support military effort.
• Diagnostics - The ability of integrated diagnostics (automated, semi-automated, and manual techniques taken as a whole) to fault-detect and fault-isolate in a timely manner.

DOT&E Initiatives

As documented in the 2007 DOT&E Annual Report, Reliability is the chief enabler of operational suitability, and failure to achieve reliability requirements typically results in a system being assessed "not suitable"; consequently, its independent evaluation is pivotal to OT&E. As such, DOT&E has established Initiative 4 called Substantially Improve Suitability Before Initial Operational Test & Evaluation (IOT&E). To accomplish this initiative, DOT&E staff will

• Assess at appropriate milestones whether programs meet the requirement to have a reliability growth program and identify for action by DOT&E leadership cases where this requirement is not met;
• Work with developmental testers to incorporate in the Test Evaluation Master Plan (TEMP) a reliability growth curve or software failure profile, reliability tests during development, and evaluation of reliability growth and reliability potential during development;
• Work with developmental testers to assure data from the test program are adequate to enable prediction with statistical rigor of reliability growth potential and expected IOT&E results. The rigor should be sufficient to calculate the probabilities of accepting a bad system and rejecting a good system and those probabilities should be used to plan IOT&E. For new or restructured programs DOT&E will not approve TEMPs lacking a reliability growth curve or software failure profile.