Browse by Subject
Browse by Category
Choose from the following:
Success Factor #1 Alignment
Success Factor #2: Contract Structure
Success Factor #3: Performance Management
References & Detailed Information Behind These Three Success Factors & Ten Foundational PBL Tenets
Primary Source: The Tenets of PBL, Second Edition, A Guidebook to the Best Practices Elements in Performance-Based Life Cycle Product Support Management (©2012 The University of Tennessee, and Supply Chain Visions), was prepared by the University of Tennessee under contract to the United States Air Force. Used with Permission.
Success Factor #1: Alignment
Before starting down the path to Performance Based Life Cycle Product Support (also referred to as Performance Based Logistics or PBL), it is important to ensure that both the government and the support provider have synchronized around the idea that PBL is truly a business model shift. The goal of PBL is achieving desired outcomes—not simply negotiating Product Support Arrangements (PSA) or squeezing the support provider for the lowest cost per transaction. While it is possible to have a PBL business arrangement without a respectful and trusting relationship, this is similar to building a house with no foundation: during the first major storm, the entire structure is likely to collapse.
The foundational Alignment factor includes five tenets. These tenets are designed to help programs build strong linkages between PBL constituents and program outcomes and include the following:
Each tenet is discussed below.
The most successful PBL programs are those where both the government organization and the support provider have a comprehensive knowledge of and experience in performance-based concepts, tenets, business models, and implementation strategies at the beginning of their program efforts. The very best programs tend to assemble a team from both the government and the support provider with at least 1-2 people on the PBL IPT who have successfully managed a PBL program before.
Having a team with representatives from both sides with PBL experience enables these leaders to guide the rest of the team through challenges more easily than teams without PBL experience. The teams that do not include PBL-experienced staff at the onset often struggle with issues that seem to “take two steps forward and one step back” every time they move forward.
The Tenets of PBL, Second Edition, A Guidebook to the Best Practices Elements in Performance-Based Life Cycle Product Support Management (©2012 The University of Tennessee, and Supply Chain Visions), was prepared by the University of Tennessee under contract to the United States Air Force. Its purpose was to provide touchstones and operating principles for the successful implementation of PBL, and thus to help deliver affordable weapons system support to the warfighter. Their research uncovered a secret weapon for successful PBL organizations—establishing a PBL Knowledge Base. A knowledge base of PBL resources and information can be an effective tool in helping the organization and the PBL IPTs ramp up on PBLs in an effective and rapid manner. Unlike in the past, there is now a broad set of reference materials available from DoD.
DAU advocates care in the construction of a program culture that taps into the most skilled in PBL to help lead and teach those on the team that are new to PBL programs. Unfortunately, it is not unusual for PBL teams to be assembled with little practical experience, foundation and/or resources such as reference documentation, guidebooks, lessons learned, etc.. Program offices and support providers alike have the ability to leverage the experience they have garnered with previous programs and should actively seek to use it.
The most successful programs make a concerted effort to get the team smart on the fundamentals of PBL. In fact, most of the largest support providers invest heavily in educating their employees about the basics of PBL by encouraging their employees to complete the DAU on-line or classroom training, or and other available training, such as PBL training offered by the University of Tennessee.
To be successful, PBL education must be approached from a cross-functional perspective. The teams that achieve the highest return on investment from PBL training are those that approach it from a team perspective. Having an entire program team participate in PBL training in a hands-on group “workshop” environment tends to provide the biggest bang per training buck.
The very best organizations have a “PBL Center of Excellence” where a formal process exists to collect and leverage knowledge and resources across all PBL programs. Often the companies with PBL programs have more than one—and the best practice organizations have a formal PBL benchmarking program that exists to facilitate speedy ramp up of new PBL teams. Repeatable and measurable processes enable companies to have a more efficient approach in terms of cost and time to implement their PBL programs.
Unfortunately, many organizations reinvent the wheel for every new PBL excursion. There is a need to tailor every PBL strategy to fit the circumstance, but that does not mean starting at the beginning with each new start. This problem compounds for many large organizations both - private and public – because the organizations are “siloed” and there is little proactive sharing of PBL knowledge across programs. The Army, Navy and Air Force all have de facto centers of excellence at the Army Aviation and Missile Life Cycle Command (AAMCOM), the Naval Inventory Control Point (NAVICP) and at Wright Patterson Air Force Base respectively.
After deciding to proceed with a PBL oriented business model, it is important that the cross-function IPT identify all stakeholders and the PBL champions throughout the organization, and to involve them in PBL activities as early in the process as possible. The goal of the IPT should be to drive strong consensus and participation across all stakeholders toward common support strategy objectives. Research found that not having the appropriate stakeholders and champions on board was one of the biggest challenges to an efficient PBL implementation.
Research has found that many programs attempting to implement PBL product support strategies do not do an adequate job of developing a comprehensive stakeholder analysis, and do not have a solid plan to leverage known champions or build new ones, as the case requires. The very best programs focus heavily on identifying both stakeholders and champions.
StakeholdersThe first step in a rigorous stakeholder analysis is to identify the relative “power” of each of the stakeholders. After the team identifies all possible stakeholders, the team should determine the power of the stakeholders using the two defined variables below:
InfluenceThe first variable the team should consider is the relative amount of influence a stakeholder has. Influence is defined as “the extent to which a stakeholder is able to act on project operations and therefore affect project outcomes.” Each stakeholder is given an influence rating, a measure of the power of the stakeholder. Factors include:
ImportanceThe second variable is importance, or the extent to which a stakeholder’s problems, needs, expectations and interests will be affected by the PBL project. As with influence, the stakeholder is given a rating on their potential importance.Using these two factors, the IPT can determine the ‘Power Score’ of each stakeholder by simply multiplying the importance and influence scores. The goal is to determine the importance and influence of certain individuals or stakeholders in order to understand whether or not they will be “key or primary stakeholders” for the project as a whole.
The second step is to develop a plan that addresses each stakeholder’s needs to the fullest extent possible, and then to get stakeholders involved with the team as early as possible. Without the explicit support or involvement of the stakeholder community, it is unlikely that a successful PBL program will result. The IPT then needs to work on a continuous basis to ensure primary stakeholders are on board with the team’s approach and strategies.
ChampionsSuccessful PBL programs rely on champions to support the PBL efforts. To succeed in PBL, senior leadership from the customer and the organic or commercial support supplier should be fully engaged with their respective organizations to drive towards a true win-win PBL business model. In addition, the champions from both organizations should be strong advocates for the need to change from legacy thought patterns and transactional logistics. It is important to remember that the PBL business model is different; often the change management is the biggest obstacle organizations face.
While the many senior officers and executives across all services support PBL approaches to weapons systems support, research has shown that there are fewer champions at the next level down and within the programs. Champions are often program-specific, with some programs having a great deal of support and others more or less marching forward in an effort to “comply” with PBL without really having a strong leadership commitment guiding their teams.
Champions need to come from the various disciplines that PBL may impact. Specifically, many PBLs push the envelope regarding contracting, supply management and financial statutes and policy and optimal PBL implementation may require changes in process and current procedures in these areas. Accordingly, champions in these areas need to support the Program Teams effort in working through the issues encountered.
Organizational alignment is a strategically focused approach that looks to synchronize from the shop floor to the top leadership across both customer and supplier organizations.Lack of organizational alignment can sometimes be found when a program looks to implement a public-private partnership. In some cases, there is “focused advocacy” where a small number of leaders within an organization are actively driving their organization toward a strategic approach for PBL. The highest level of the depot may champion PBL, while the lower levels may be less than enthusiastic. Situations like this can become emotionally charged, and a concerted effort to align all parties involved in the execution of the strategy pays big dividends in execution results in a win-win proposition for the entire team.
Neither side is “right” in circumstances like the one outlined above. There are public policy goals that must be respected and core capabilities protected in the organic structure. At the same time, work needs to go where it can be pest performed and misaligned organizational structures ultimately degrade support for the warfighter.
OSD provides very specific guidance, directing that “Sustainment strategies shall include the best use of public and private sector capabilities through government/industry partnering initiatives, in accordance with statutory requirements.” (DoD Directive 5000.01) There is no default decision: the support infrastructure should be in integrated across all sources of support and work structured to deliver best use in support of the warfighter, subject to statutory constraints.
The most successful PBL programs are those with a common vision from both organizations and were thus able to jointly drive towards a true win-win PBL business model at all levels of both organizations. Aligning incentives between customers (weapons system users) and support suppliers (OEMs, third party, organic) can lead to a higher level of performance at a lower cost of ownership. As mentioned previously, a PBL business model is based on achievement of desired outcomes, not based on performing transactions. The best practice PBL programs establish a true partnership mentality with a desire to develop a “win-win” business model based on mutual self-interest that focused on total system value proposition anchored in total ownership costs.
The PBL effort is not focused on reducing the price of the transactions, but on physically eliminating non-value added transactions, reducing support requirements, and implementing new business models. In essence the PBL business model should be deployed to reward suppliers, whether contractor or organic support, for their innovation in improving both reliability and affordability; with the government sharing in the benefits and savings through lower total ownership costs.
High performing programs using PBL develop a win-win PBL business model. In a win-win business model, the government and suppliers need to agree on price, risk premium, contract terms, and re-allocation of asset ownership/control. The business model needs to take into consideration three key drivers—contractual drivers, managerial decisions and exogenous factors. Internal and external alignment from top to bottom of the organization is part of the glue used to form a strong PBL foundation.
PBL programs that develop a strategy and propose outcomes designed to leverage the entire industrial capability have the greatest success. Workloads are distributed to the most effective providers consistent with statutory guidelines, with a conscientious effort to focus on best competencies, best value, and effective use of Public-Private Partnership (PPP) solutions.
Many programs get caught up in the oversight process and opt to tell the suppliers how support is to be performed, rather than describing the desired outcomes. In a traditional approach, the government provides a Statement of Work (SOW) that outlines in detail the various activities that the support provider should perform. Typically, many of these activities are priced per transaction and may not define performance targets.
In a PBL approach, the work is captured in a performance-based Statement of Objectives (SOO). The SOO focuses on the “what” and not the “how.” This transfers authority and flexibility to the support provider, which can foster the development of innovative solutions designed to most efficiently and effectively achieve the customer’s desired outcomes. The SOO is in essence a summary of key goals and/or outcomes which are incorporated into the PBL agreement.
Unfortunately, PBL programs are often misunderstood as “outsourced” efforts or “Contractor Logistics Support (CLS)” with minimal emphasis on best value and best competencies in placement of workloads. When a depot is performing organic workload for sustainment, a common misunderstanding is that PBL will automatically result in the redistribution of work from the depot to the contractor. In fact, PBL can result in additional work at the depots. Teaming relationships are central to PBL—and Title 10 clearly emphasizes the importance of PPPs.
PPPs are an effective tool for balancing workload allocation around best value solutions. PBL does not pre-ordain CLS or organic support structures. Rather, PBL gives all stakeholders the opportunity to compete for and earn business in line with their core competencies and value proposition.
The traditional approach for a contract has been to manage the supply chain by commodities or services; the approach has necessarily been focused on optimizing the achievement of end-to-end process effectiveness. Best practice PBL programs demonstrate development of a formal supply chain management strategy that focuses on maximum integration for end-to-end supply chain effectiveness. This approach means supply chain components need to be integrated and align to optimize the end-to-end process. Internally stove-piped supply chain processes must be reduced or eliminated. For some programs, this has led to co-location of the program team, so the support provider and the government – especially the depot - work side by to facilitate cohesive, comprehensive and coordinated customer and supplier involvement.
back to top
The Program Manager (PM) is assigned Life Cycle Management responsibility and is accountable for the implementation, management, and oversight of all activities associated with development, production, sustainment, and disposal of a system across its life cycle. As part of this, the PM has the responsibility to develop an appropriate sustainment strategy to achieve effective and affordable operational readiness consistent with the Warfighter resources allocated to that objective. The PM‘s responsibilities for oversight and management of the product support function are typically delegated to a Product Support Manager (PSM) who leads the development, implementation, top-level integration, and management of all sources of support to meet Warfighter sustainment and readiness requirements.
The contract structure is a visible manifestation of the PSM’s implementation responsibility, and nothing is more important than formulating the appropriate contract structure and the resulting contract, or the product support agreements for arrangements within the government. A PBL solution is not wed to any particular contract type or incentive plan, as long as it supports an outcome-based approach. However, good to robust best practices typically includes a form of incentive for achieving performance and cost savings targets. When a service provider meets expectations, they can be rewarded with financial incentives, such as performance bonuses, gain-sharing bonuses, or extended contract lengths. In the case of a Firm Fixed Price (FFP) contract, the contractor could harvest improved profit margins as improvements take hold during the period of performance.
Striking the right balance of contract type and incentives are discussed in the three tenets discussed below.
While this discussion centers on a government perspective on the relationship specified in a contract, the concepts are applicable to all organizations and agreement types. The tenets are applicable and adaptable to public-private partnerships and agreements between government entities serving as the PSM, PSI, or the Product Support Providers (PSP). Each tenet is discussed below.
Robust best practice PBL programs include a focus on total program risk reduction along with appropriate off-ramps exit criteria that are captured at the onset of the contract execution. These programs balance risk with a mitigation strategies that account for all parties in the relationship, while paying specific attention to harmonizing supplier accountability and authority. By moving some risk to the support provider, and aligning incentives to stimulate program effectiveness, the PBL business model can remove risk from the total system. It isn't just about moving risk to the supplier - it’s about realigning the incentives to reduce total program risk.
The importance of exit criteria that leave both sides whole cannot be overstated. Best practice PBL programs actively address off-ramps that balance the needs of the contractor, the customer, and the organic support structure. Traditional outsourcing arrangements often have termination for convenience clauses, but this has not been found to be sufficient. PBL contracts should include detailed criteria to assure that the program can continue sustainment efforts should the need to exit the arrangement arise. Some common off-ramp criteria are as follows:
Managing risk associated with asset ownership in a PBL arrangement is key concern during the development of a risk mitigation strategy, and during the development of specific contract language. Under traditional sustainment models, the government customer often owns and manages the resources associated with the program, including spares, repair facilities, etc. CLS arrangements will often shift responsibility for managing most aspects of resources to the supplier, but associated risk remains with the government customer because ownership of the asset remains with the customer. This is less than ideal, as the support provider does not have “skin” in the game. As cited in the
TheTenets of PBL, Second Edition, A Guidebook to the Best Practices Elements in Performance-Based Life Cycle Product Support Management, when support providers are asked “What are your inventory turns?”, the answer was often “I don’t know—it’s not my inventory.”
While there is much debate about whether a supplier should own the assets or the government should own the assets, research performed for the comprehensive PBL Guidebook referenced above shows that programs are more apt to achieve desired performance when the supplier owns the assets. Additionally, research at the Wharton School strongly suggests that best practice is to have full asset management control, including ownership, shifted to the supplier and the associated risks for asset performance accepted by the supplier. Their findings, published in Management Science, were that the optimal business arrangement was when the contractor owned the assets. Under a PBL program, the supplier is accountable to meet service levels at a fixed price. When a supplier owns the assets, they have an inherent incentive to reduce the cost of asset ownership and keep the level of inventory at the lowest possible level, while still allowing them to meet performance targets.
Setting a sound contractual PBL foundation requires a creative environment that allows for the development of tailored pricing models, incentives, and contract lengths that accounts for the funding types available and facilitates a win-win situation for all parties. These elements of the contracting environment are discussed below.
One of the most challenging elements of a contracting strategy is developing the pricing and incentives structure: the “pricing model.” The pricing model is made up of two key elements: contract type and incentive type. Incentives are optional, but structures that provide incentives for “good behavior” are desired in all contracts.
There are two basic types of contracts: fixed price and cost plus. Generally, the optimal contract type for a PBL effort is a fixed price vehicle, which uses a per unit or per unit throughput basis. In fact, direction from the Office of the Secretary of Defense states that the desired PBL pricing approach is a fixed price model. “When robust competition already exists, or there is recent competitive pricing history, I [USD ATL] expect components to be predisposed toward Firm-Fixed-Price (FFP) type contract arrangements. FFP should also be used to the maximum extent reasonable when ongoing competition is utilized in multiple award contract scenarios.” (BBP 2.0)
FFP contracts are a natural fit for buying designated performance outcomes as they build in an inherent incentive for the service provider to be efficient and meet profitability levels at the fixed price rate. In essence, the support provider increases their profit as they get more efficient. Having a fixed price agreement on a per unit or throughput basis allows for fluctuating volumes. In addition, pricing models also may have “volume bands” to allow for different pricing at different levels of volume.
While a fixed price model provides inherent incentives, it is usually necessary to begin with cost reimbursement (or cost plus) contracts in the early phases of PBL implementation while the appropriate cost and resource baselines are maturing. It is rare that a program matures to the point where all elements appropriate for cost plus elements are eliminated, and it is risky to implement fixed price agreements without first understanding the baseline performance and cost of the existing business model. Greater maturity allows the program to develop realistic cost and performance profiles. This maturity leads to program stability as cost uncertainty is driven down. As a result, programs can then transition to a PBL pricing model with more of a fixed price emphasis.
A key to any pricing strategy is to provide enough incentive to the PSI and, by implication, the PSPs to change status quo behavior and to encourage changes and/or investments that will improve processes and/or reliability of the system. The pricing strategy needs to accommodate these changes and/or investments and allow for the adequate rewards – otherwise support providers will not be induced to assume the investment or risk. However, the Government should also benefit from the relationship through improved performance and/or cost savings. If cost savings are anticipated, there needs to be a clear plan on how and when the Government will harvest the savings.
In a FFP example, the government can get cost transparency through the submission of Certified Cost and Pricing Data. By statute, this data captures actual PBL performance costs. Information regarding actual costs can serve as the basis for the subsequent period of performance, and, if the effort has executed as desired, can create a new and lower cost baseline. The lower costs negotiated represent the savings to be harvested by the government.
IncentivesAll PBLs implicitly use incentives. Best practice PBL programs use incentive strategies that are tightly aligned, promoting behaviors and outcomes that benefit both the customer and supplier. The incentives should include an explicit reflection of factors like program maturity, scope of agreement, complexity of the system, context of use, etc..
A PBL agreement can leverage various types of incentives. Contracts can be incentivized based on award fees (cash payments/bonuses) or extended contract terms. Award term extensions of the contract whereby the contract will be extended if desired outcomes are being achieved. Cost-cutting targets are inherent if a fixed price model is used; the more the supplier cuts costs the more margin they make; contract price adjustments are made at pre-defined timeframes to review costs and re-price the work. However, not all PBL programs are fixed price and as such, any cost plus type contract should include some form of cost savings incentive.
It is important to keep in mind that the government has wide discretion in assembling and blending contract types and incentive types, tailored to fit the circumstances of the program. There is no one-size fits-all, universally applicable contract and incentive template.
Contract LengthThe third element of a contracting strategy is the contract length. PBL contract lengths are typically “longer term,” but the actual length varies. In guidance issued in September of 2010, the Under Secretary of Defense for Acquisition, Technology, and Logistics said, “Single-award contract actions should be limited to three years (including options) unless, by exception, it is fully justified for longer periods by the senior manager for services. Contract length should be appropriate for the activity performed. Knowledge-based services readily meet the three-year limit.” (BBP 2.0)
Often, individuals cite that policy statement as a rationale for contracts necessary to implement a PBL contract length of three years. However, the sentences that follow in that policy memo are extremely important. “Other services such as Performance-Based Logistics (PBL), LOGCAP, and environmental remediation, as examples, may not. The intent is that each service requirement will be reviewed by the appropriate official and only those with a sound business rationale will contain longer contract performance provisions.”
Longer-term contracts encourage long-term investments to improve product or process efficiencies—a key desired outcome of a PBL – but there is no real guidance or “right answer” for the appropriate length of a contract. The general rule of thumb is that the contract length should be commensurate with the payback period for the supplier’s investments. There have been some successful PBL contracts of up to 10 years for the US government and up to 20 years with foreign governments – although this period of performance is more the exception than the rule.
The government gives guidance on this in
Section 2304 of title 10 USC, which reads that contract length may be “any period up to five years and may extend the contract period for one or more successive periods pursuant to an option provided in the contract or a modification of the contract. The total contract period as extended may not exceed 10 years unless such head of an agency determines in writing that exceptional circumstances necessitate a longer contract period.”
DoD guidance recognizes that effective PBL contracts are multi-year contracts (i.e., 3 to 5 years with additional option or award term years), with high confidence level for exercising options/award term years. They also typically feature provisions to recognize supplier investment and opportunities to recoup investments.
Longer-term contracts are more conducive to effective PBL implementation because of simple economics. Up-front investment drives performance improvement, and the annual “payback” for a PBL investment - as reflected in financial returns in the “out years” - is what justifies the up-front investment. Therefore, longer planning horizons justify higher up-front investments because of the higher total return opportunity for both the provider in terms of “paybacks” and the government in terms of improved performance and/or cost savings. One year contracts do not encourage support provider investment, and therefore are less effective in generating performance or cost improvements.
It is important to note the differences between a multi-year and multiple year contracts, which are terms that are easily confused.
It is also important to note that the PM generally does not have the authority to implement multi-year contracts, but exceptions do exist. Working capital fund authorities can be used for multi-year contracts, for example.
There is often confusion surrounding appropriate contract lengths and terms when developing a program’s PBL strategy. Including Legal Counsel in discussions on this subject early in the process is a key to success. Although statutory limitations – particularly related to colors of money - may exist that prevent the execution of what is viewed to be the optimum solution, there are compromise contract structures that can lead to the positive performance and cost changes desired.
One of the biggest challenges to PBL is funding stability and budget instability. Contract provisions should reflect fact-of-life funding variability and provide both the customer and supplier adequate adjustment mechanisms to mitigate risk to accommodate funding variability.
At the same time, good faith and diligent efforts to reduce the risk of funding variability should be attempted. Increasing levels of “out year” funding uncertainty leads to lower levels of up-front investment in performance improvement initiatives, which in turn drives higher life cycle costs. Funding variability undercuts the effectiveness of PBL.
While PBL programs can be optimized when funding is stable, it is important to accept budget realities. In today’s austere budget environment, there is no such thing as certainty in appropriated funds. Therefore, all parties to a PBL agreement should work to identify and codify a “funding baseline” or “floor,” i.e, the minimum funding level that will allow the program to endure, with flexible structures above the baseline given requisite funds availability.
Success Factor #3: Performance Management
At the heart of a PBL program is the identification of the desired outcomes and associated metrics tied to supporting warfighter requirements. Performance management is not only an essential component of the contract management process—it is fundamental to the success of a PBL program.
Having a strong performance management process aids in the analysis of business problems and opportunities, and offers a basis for rewarding desirable behaviors and outcomes. An aligned performance measurement and rewards program supports management in steering the organization in the right direction. Measurement transcends perception, creating the reality that solves problems and drives improvement. Smart companies know that information is the foundation for understanding and effective problem solving. The tenets of performance management are:
Measures help managers understand current performance relative to plans, to their own past, and allows them to benchmark against best practices. Key benefits are as follows:
The two key tenets associated with performance management are addressed in more detail below.
At the heart of PBL is the desired outcome. “The PM shall work with the users to document performance and support requirements in performance agreements specifying objectives, outcomes, measures, resource commitments, and stakeholder responsibilities.” (Defense Acquisition Guidebook (DAG) )
During the acquisition and sustainment phases of a program, metric use is spelled out in multiple documents to include by not limited to the following:
Joint Requirements Oversight Council (JROC) Memorandum 161-06 dtd 17 Aug 06
DUSD (L&MR) Policy Memo "Life Cycle Sustainment Outcome Metrics" dtd 15 Mar 2007
DUSD (L&MR) Policy Memo “Reporting” dtd 11 Dec 08
The life cycle metrics called out in the above policy documents include:
As programs progress from acquisition and fielding to sustainment, these four basic metrics are used to monitor and measure performance. In addition to these mandated metrics, the PSM may also adopt s a variety of metrics to provide insight into specific performance areas.
While the government and the support provider should work as a team to determine the required performance outcomes, the DoD is very specific about measuring and reporting on these performance outcomes.
What Should You Measure?
With literally hundreds of different things that can be measured, how do you know which ones you should use for your PBL program? The Supply Chain Council (SCC) lists over 100 metrics on the
SCC website that may help in getting started.
DoD's PSM Guidebook also provides some commonly used measures. But smart managers don’t just measure things because they can—they pick the factors that will have the biggest impact on improving a program’s performance.
When deciding what PBL metrics to focus on, the DoD's PSM Guidebook states that PBL metrics should support the top-level war fighter performance outcomes, be consistent with the support provider scope of responsibility, and have a direct linkage between performance and sustainment.
Rule of 5
When determining top-level desired outcomes, adopting “the rule of 5” is a good general guideline to limit the number of top-level metrics to 5 or fewer. The reason? Focus. Having too many metrics makes it hard for the support provider to focus on what is truly important.
Unfortunately, many programs do not support the OSD recommended metrics nor the rule of 5. Some programs, however, have learned this lesson over multiple contract “phases”. A classic example is the Air Force C-17 Globemaster III program. It reduced the number of metrics in their contract during each of three contract renewal periods.
Metrics are clearly aligned to desired outcomes
It’s not simply enough to have a few critical metrics. These metrics need to be focused on achieving the right things. It is tough to run a business without clear objectives. As such, the second best practice in selecting PBL metrics is to ensure that the metrics are clearly aligned to the desired outcomes. As Yogi Berra put it, “If you don’t have a goal, any road will get you there.” Picking metrics that are not aligned to your desired outcomes will take you down the wrong road.
Ideally, the metrics that the government and the support provider select should be focused on achieving warfighter requirements. However, there is more to it than that. The metrics need to be tied to requirements, to the operational role of the system, and must be synchronized with the scope of the support provider’s responsibilities. This support provider should not be held contractually liable for achieving performance against a metric over which they have no control. For example, an overall platform level metric of materiel availability (e.g., Operational Availability) is not appropriate when the support provider only controls the airframe and not the engine or other key subsystems. A support provider should not be held accountable for overall material availabilitywhen there is a workshare agreement with a depot either, unless the provider has an enforceable contractual relationship with the depot for their portion of the work.
While contractual metrics need to align with the scope of the support provider’s responsibilities, it is still important to measure them at the system level to provide insight into overall performance. This multi-tiered metrics and measurement approach should help provided the end-to-end performance insight the PM and PSM need for management purposes.
Data sources are accurate and timely
Another best practice when selecting metrics is ensuring that data sources are accurate and timely. If a metric is the right but to accurate or timely data is not readily available, however, the team might still want to use it – if it will be possible to collect the data in the future. The government may be able to identify the data needs in the contract such that the support provider will develop, collect and provide the required data downstream. In early versions of the C17 cost-plus contracts, for example, a provision was included to support a ramp-up period whereby processes and tools were built in support of data collection activities.
Achievement of metrics validated by a mutually agreed QASP
It is also important to be able to assess selected metrics with a Quality Assurance Surveillance Plan (QASP). The QASP must be mutually agreed upon, and often includes sampling or audit requirements. The support provider is responsible for ensuring the quality of all work performed and the government is responsible for surveillance and monitoring. A typical QASP addresses:
QA is a continuous activity designed to determine if the work being performed meets or exceeds the quality performance standards. The goal is to prevent substandard work, rather than catch it after the fact. The rigor of the QA process should match the needs of the program; it should be a major element in program management and control, focusing on insight rather than oversight, and the QA monitor(s) should be independent of the work being measured.
Five QA approaches can be used to validate achievement of desired outcomes. These are:
The program needs to ensure it has the resources to monitor the reporting management process because simply reporting on these measurements will not ensure the quality standards are being maintained. President Ronald Reagan’s signature phrase is appropriate here: the program must “trust but verify” QA results.
Clear understanding of financial impact of metrics
The best practice approach is to tie the supplier’s profits directly to delivering the desired performance. This can be done with various types of pricing models or incentives.
While most organizations agree performance measurement is important, performance reporting is vital to the success of a PBL agreement. Support providers need to understand their performance so they can proactively manage performance to make cost/profit tradeoffs; while the government needs to understand the program’s performance to determine if they are fulfilling their mission. One important area to measure is supply chain performance associated with key performance indicators such as material availability and costs. It is imperative that management adopt solid performance management practices.
Regular review cyclesIn addition to a formal QASP metrics and reports required by the government, a good PBL program will establish formal periodic reviews with the PBL stakeholders. Reporting is often relegated to the QA monitor or contracting officer’s representative. Ideally, however reviews should include all key stakeholders, including the program office, key “customer” representatives, and the representative from the contracting community. Likewise, the support provider should include a cross-functional mix of stakeholders in the review as well.
While a support provider should only be contractually liable to measure what is in the scope of their control, best practice programs use cross-functional reviews to measure the end-to-end performance. This means there should be reporting from all program participants, industry and government alike, in order to capture the big picture and enable sound government management. The best practice approach puts together all the pieces of the puzzle in order to obtain a landscape view of total performance.
Metrics reports are used as part of regular review meetingsBest practice organizations develop performance and cost reports regularly and frequently to help them proactively manage their programs. At a minimum, a best practice PBL team should have internal, formal performance metrics reviews monthly, and should execute working level reviews as often as necessary to exercise adequate oversight of critical operational metrics.
Drill down capabilityMetrics should be used to drill down to find the root causes of performance and to allow targeted process change. In the case of PBL, it is imperative to make sure that the warfighter requirements are translated to “shop-floor” metrics. Contributions to key performance metrics need to be identified at the working level, such that the support provider can affect positive change against those metrics. You are far more likely to experience successful process improvement if the people doing the work feel like they have ownership and become stakeholders in improving the process.
Metrics are posted and communicated to entire team
Another best practice is posting metrics reports so that all team members can read them. Performance against the support provider’s key performance indicators (KPIs) should be visible throughout the organization to foster awareness and ownership of requirements. Some examples of best practices include posting performance against KPIs on the company intranet and shared team spaces. This keeps employees in the loop and cognizant of performance status and fosters a sense of accountability.
Fully automated dashboardsA fully automated dashboard helps promote drill down functionality and root cause analysis. While root cause analysis can be done without an automated metrics reporting tool or dashboard, more and more support providers invest in automated solutions to help them collect, manage and report on metrics.
One tool that is emerging among best practice PBL programs is a “scorecard” to show performance toward the achievement of goals. As a reporting tool, a scorecard can be an important part of an overall performance management system. The concept of scorecards has been around since the 1890s when French process engineers invented the concept with their “Tableau de Bord” (dashboard). However, few companies utilized the concept until Kaplan and Norton popularized the idea with their “Balanced Scorecard” in the 1990s. Since then, a host of technology companies have made automated scorecards a reality, and some of the more progressive PBL organizations use automated dashboards to help them manage their business - with some programs even linking to key suppliers and the depot to mine data for tracking overall program performance.
The philosophy of PBL is to develop a business model that promotes performance improvement—and that means instituting a solid performance management approach aimed at achieving the desired outcomes. It’s one thing to have good performance metrics—but if you don’t have a culture to rigorously drive performance improvements, it is likely your PBL will only get so-so results.
Continuous improvement philosophies have been around for decades. Walter Shewart, the founder of
Statistical Process Controls (SPC) laid the foundations of continuous improvement in the 1920s—striving to make these techniques accessible to the first level operators. Today continuous improvement programs take many forms, ranging from rigorous approaches - such as
Lean Six Sigma - to less demandingl validation and verification approaches.
Regardless of which continuous improvement philosophy your organization chooses, some best practices are common across all good programs. These best practices are addressed below.
Supplier is clearly incentivized and authorized to use continuous improvementThe culture of an organization is a key element in determining the effectiveness of its performance management and continuous improvement program. Organizations that encourage ownership and facilitate change and improvement succeed, while companies that punish for non-performance or mistakes encourage an atmosphere of “tell management or the customer what they want to hear.” PBL programs need to encourage and give the support provider the authority to plan for and implement continuous product and process improvements.
Government PMs should recognize that a key step to a sound PBL program is the creation of a positive environment where change and improvement are rewarded. In some PBL programs, the PBL agreement incentivizes the supplier to undertake formal continuous improvement or to bring proactive ideas that would improve performance or costs. Many best practice PBL programs have used incentives linked to cost savings. For example, the Air Force F-117 Total System Support Partnership (TSSP) contract had a 50/50 cost share split. When Lockheed Martin realized cost savings for their customer from their continuous improvement efforts, they were rewarded with an incentive fee equal to 50% of the savings. While not all PBL programs can or should provide this type of incentive, all best practice PBL programs should encourage and give the support provider the authority to plan for and implement some type of continuous product and process improvements efforts.
A formal continuous improvement programAll PBL support providers should have some form of formal continuous improvement effort that effectively drives improvements against the top-level desired outcomes. For some organizations, detailed process improvements such as
Statistical Process Controls (SPC) or Lean Six Sigma may be too formal or complex or include unattainable goals. For example, in a distribution center environment a typical Six Sigma metric would be 3.4 late lines per million lines scheduled may require years of shipments without a single error—a goal so ambitious that it may not be appropriate for the order fulfillment process.
On the other hand, the overarching goals of these methods can often provide value. In the case of SPC, the goal is to analyze a process or its outputs so as to take appropriate actions to achieve and maintain a state of statistical control and to improve the process capability, while Lean Six Sigma is focused on optimizing processes by eliminating waste and reducing variation. The challenge and benefit is finding the right goals and metrics to incentivize improvement. The Air Force has adopted Lean Six Sigma approaches to great benefit.
Continuous improvement plan supported by investment plan
Continuous improvement efforts should be supported with a formal process for investing in the improvements that are identified. There is nothing less motivating to employees than asking them to come up with improvements that never get implemented. Not all improvement opportunities can be implemented due to time and cost constraints so continuous improvement plans should be supported by a formal investment planning process. This allows the best ideas to be prioritized and funded on a regular basis so that the program can effectively realize the improvement potentials. Improvements could be anything with a positive ROI, for example, meeting targets in process or product efficiencies such as increased reliability.
Metrics Aligned to Suppliers
As mentioned above, the
DAG states that the PM shall work with the users to document performance and support requirements in performance agreements that specify objective outcomes, measures, resource commitments, and stakeholder responsibilities. Performance agreements can take many forms such as PSAs for implementation. These arrangements should take the form of performance based agreements, memorandums of agreements, memorandums of understanding, and partnering agreements or contractual agreements with PSIs and PSPs, as appropriate.
Best practice dictates that PSA’s be used with all of the key PBL organizations. They serve two primary purposes:
If the government is the PSI, then this role of aligning the various entities becomes part of the government PM responsibilities. However, if the PSI is a contractor, the role of aligning the various PSP falls on the shoulders of the contractor.
Research has revealed that very few support providers attempt to cascade or flow down top level outcomes to sub-suppliers and various PSPs. Typically the sub-supplier contracts are transaction-based business models whereby payments are made based on consumed resources or specific activities performed by the supplier. Additionally, support providers often have agreements with other government agencies (i.e., DLA or a depot) that do not specify alignment with the overall performance requirements. In short, alignment of the PSI and PSPs is very important, but not yet an institutionalized practice. Developing outcome based PSAs with all PBL participants would help end-to-end requirements flow down and performance measurement efforts.
Performance management should not be treated as something nice to have: it is an essential component of a support provider’s management process and fundamental to the success of a PBL program. A good support provider should adopt better and best practices across all four of the performance management areas to succeed in their PBL programs.
Required fields marked with *
Please note that you should expect to receive a response from our team, regarding your inquiry, within 2 business days.