Is there DoD policy/guidance/best practice documents that plainly and emphatically state that Developmental Test objectives should be driven by, and trace to, the specification requirements.
The short answer to your question is, “not exactly.” Your IPT will have to interpret and implement the guidance available to you and come to consensus on the depth and breadth of your traceability analysis. Certainly the intent of the published guidance is that all requirements be described in clear and measurable terms to enable verification. Therefore, bi-directional traceability between requirements and verification is a powerful tool for both the Systems Engineer and the Tester. Below are select excerpts from three sources:
1. MIL-STD-961EC3 “DEPARTMENT OF DEFENSE, STANDARD PRACTICE, DEFENSE AND PROGRAM-UNIQUE SPECIFICATIONS FORMAT AND CONTENT” dated 27 October 2015.
Para 5.8 Describes Section 3 of the system specification covering requirements and states:
“The following criteria shall apply for stating requirements:
a. Each requirement shall be stated in such a way that an objective verification can be defined for it.
b. Each requirement should be cross-referenced to the associated verification.”
2. IEEE 15288.2 “IEEE Standard for Technical Reviews and Audits on Defense Programs” dated 10 December 2014
6.7 Test readiness review (TRR) detailed criteria Table 21 – TRR technical review products acceptability criteria System Technical Documentation:
l) The test procedures for each test case together with the planned test input data and drivers are correct, complete, and sufficiently robust to verify all of the requirements allocated to the test case.
q) Bi-directional traceability that is correct, complete, and consistent is provided between the requirements to be verified by the formal test event and the test procedures and test cases in which the requirements will be verified.
Table 22 —TRR technical review preparation actions Systems Engineer:
b) Ensure verification methods and success criteria are defined for all decomposed and allocated requirements.
Table 23 —TRR conduct elements
System technical documentation
a) Requirements planned to be verified during this test event
b) Any changes to these requirements that have been approved since the previous technical review
c) Required verification methods, and verification level(s) for each requirement planned to be verified during this formal test event
j) Bi-directional traceability among the test cases, test procedures, design documentation of the system element(s) under test, and the requirements documentation covering the requirements to be verified
6.8 Functional configuration audit (FCA) detailed criteria Table 25 —FCA technical review products acceptability criteria Functional, allocated and product baseline documentation
b) Adequately detailed requirements verification traceability documentation exists outlining the method for verification for each requirement in the CI specification.
a) Each requirement listed in the requirements verification traceability documentation has been successfully verified in accordance with the identified methodology based on all available test data, analysis, or inspection.
6.9 System verification review (SVR) detailed criteria
6.9.1 SVR review products acceptability criteria Table 29 —SVR technical review products acceptability criteria
System functional and product baseline documentation
b) Bi-directional traceability has been documented among the CDD, the CPD, and the system specifications.
6.9.2 SVR preparation
Table 30 —SVR preparation actions
c) Ensure all requirements in the system specification have been verified through the appropriate verification method(s) and have been appropriately documented.
6.9.3 SVR conduct
Table 31 —SVR conduct elements
System functional and product baseline documentation
b) Bi-directional requirements traceability, methodology and completeness to the CI level between the functional baseline and the verification test results
3. Defense Acquisition Guidebook Chapter 3 “Systems Engineering:
CH 3–2.5 Engineering Resources
“… The Systems Engineer is responsible for planning and overseeing all technical activity within the program office and for managing effective SE processes. The Systems Engineer should ensure the PM has sufficient and clear information for scheduling and resource-allocation decisions. In addition, the Systems Engineer implements and controls the technical effort by:
“… Reviewing requirements traceability matrix and cross reference matrix (verification)”
CH 3–4.1.6 Configuration Management Process Configuration Management activities support:
• Traceability of designs to requirements.
Critical Design Review (CDR) and verified later at the Physical Configuration Audit. In accordance with DoDI 5000.02, Enc 3, sec. 8, the PM assumes control of the initial product baseline at the completion of the system-level CDR to the extent that the competitive environment permits. This does not necessarily mean that the PM takes delivery and acceptance of the Technical Data Package. Attributes of the product baseline include:
o Requirements Traceability Matrix (RTM) is complete.
o Traceability from design documentation to system and system element verification requirements and methods is complete.