Technical Performance Measurement (TPM)

Technical Performance Measurement (TPM) involves a technique of predicting the future value of a key technical performance parameter of the higher-level end product under development based on current assessments of products lower in the system structure. TPM's are used to continuously measure growth of a measure toward meeting the required goal at the end of development. Continuous verification of actual versus anticipated achievement for selected technical parameters confirms progress and identifies variances that might jeopardize meeting a higher-level end product requirement. Assessed values falling outside established tolerances indicate the need for management attention and corrective action. A well thought out TPM program provides early warning of technical problems, supports assessments of the extent to which operational requirements will be met, and assesses the impacts of proposed changes made to lower-level elements in the system hierarchy on system performance.

Technical project management is the key focus of systems engineering. Technical management includes the direction of a totally integrated effort of system design, test and evaluation, production, and logistics support over the system life cycle. The goal of technical management is timely deployment of an effective system, sustaining it, and satisfying the user’s need at an affordable cost. Technical management involves balancing a system’s cost, schedule, and performance while controlling risk.

As illustrated in Figure 1, technical management integrates a variety of technical assessment tools and metrics to measure progress in meeting stakeholder and technical requirements; to assess performance against technical plans, schedules, and estimated costs; and to assess the effectiveness of technical work completed. Key tools used in technical assessment include testing, technical reviews, earned value management (EVM), and technical performance measurement. In particular, by judiciously selecting and tracking Technical Performance Measures (TPMs), the technical manager can assess progress throughout development and make design decisions that ensure the system will meet technical performance objectives that are included in the Acquisition Program Baseline.

Technical Performance Measurement (TPM), as defined in one industry standard (EIA-632), involves a technique of predicting the future value of a key technical performance parameter of the higher-level end product under development, based on current assessments of products lower in the system structure. Continuous verification of actual versus anticipated achievement for selected technical parameters confirms progress and identifies variances that might jeopardize meeting a higher-level end product requirement. Assessed values falling outside established tolerances indicate the need for management attention and corrective action. A well thought out TPM program provides early warning of technical problems, supports assessments of the extent to which operational requirements will be met, and assesses the impacts of proposed changes made to lower-level elements in the system hierarchy on system performance.

Selecting Technical Performance Measures
Parameters to be tracked are typically based on the combined needs of the government and the contractor. The contractor generally tracks more items than are reported to the government, as the contractor needs information at a more detailed level than does the government program office. The government program office requires visibility into the technical performance of key elements of the work breakdown structure, especially those which are needed to meet system key performance parameters (KPPs), are cost drivers on the program, lie on the critical path, or which represent high risk items. The work breakdown structure makes an excellent starting point for consideration of parameters for TPM tracking, as shown in Figure 2.

Figure 2. Shipboard Fire Control System (Partial)

Parameters selected for reporting should be traceable to the needs of the user, and they must be concrete technical parameters that can be projected and tracked. Furthermore, there should be consistency between TPMs and the Critical Technical Parameters associated with formal testing, although the TPM program will not normally be limited just to those parameters identified as critical for test purposes. If a critical requirement allocated to a particular element of the design has been identified as a risk area, then parameters should be selected that enable the manager to track progress in that area.

The level of the system at which parameters are selected is based on how readily the information supports timely design decisions. For example, the timely identification and neutralization of targets and threats are essential to both the operational effectiveness and survivability of a ship. If there is a risk associated with meeting the detection range requirement allocated to the ship’s fire control system (Figure 2), then the technical manager will want to have data that supports design decisions related to achieving both the system and subsystem performance during the design process, such as predicted (and actual) radiated power and data processor speed. These are metrics that can be measured under laboratory conditions before costly and time-consuming fabrication, integration, and conformance testing of higher assemblies; and continued monitoring assures that required values are met under actual environmental conditions and system loading. Thoughtful selection of the parameters to measure can minimize unpleasant surprises in formal developmental and operational testing.

TPM reporting to the government is a contractual issue, and those TPMs reported to the government are defined as deliverables on the contract data requirements list (CDRL). The number of parameters selected for reporting depends on several considerations, including management overhead costs, availability of people to review and follow items, complexity of the system, technological risk, phase of development, and the contractor’s past experience with similar systems. While the parameters may focus on the technical performance of lower-level elements, only those that contribute to top-level objectives or pose risk to the program should be selected. If too many parameters are being reported, management will lose visibility of what is really important to project success. Finally, TPM reporting is generally reserved for those parameters in which developmental work is required, and the contractor has a time-phased plan (profile) for achieving the required value. Parameters that cannot be profiled, measured, or are not critical to program success should not be selected for TPM reporting.

Reporting Technical Progress
TPM reports are used to (1) project the probable performance of a selected technical parameter over a period of time, (2) record the actual performance observed of the selected parameter, and (3) through comparison of actual vs. projected performance, assist the technical manager in decision-making. TPM reports generally take the form of both graphic displays and narrative explanations. A typical TPM graphic has a form similar to that shown in Figure 3. The initial graphic shows the projected behavior of the selected parameter as a function of time. Tolerance bands are an acknowledgement of estimating errors and help to alert managers when the degree of expected variation is exceeded. The actual form of the projected performance profile and tolerance bands is a function of the parameter selected, the activities planned during development, and the needs of the program office. The narrative should describe the activities to be taken in progressing to the final value.

Figure 3. Conceptual TPM Graphic

As development proceeds, actual measures of the parameter are plotted, and any deviations from the projected profile are assessed. The narrative portion of the report explains the work that has been accomplished, addresses the reason for deviations from the planned profile, explains actions underway to correct the situation if required, and projects future performance given the current situation. Government review and follow up of TPM reports are appropriate on a periodic basis when submitted by the contractor, and at other major technical events such as at technical reviews, test events, and program management reviews. A substantial effort has taken place in recent years to link TPMs with EVM reporting so that cost and schedule performance truly reflect the progress (and risks) associated with achieving technical performance required to meet the users’ needs. If linked appropriately, the earned value reported by the contractor should reflect real value in terms of achieving technical performance.

Summary
Technical Performance Measurement is an important Technical Assessment tool. A sound TPM program sets expectations of technical progress and how it is measured. TPM reports track design progress against predictions in meeting technical requirements, providing early warning of deviations that can impact system success in meeting user needs. TPMs should be an integral part of periodic program reporting and management follow-up, including elements for discussion in technical reviews and program management reviews. Thoughtful selection and monitoring of key technical performance parameters allows the technical manager to make informed decisions during system design and to identify the need for corrective actions when deviations from planned technical progress occur.