Before you Begin

Closed Captioning

The close captioning software we currently use is not 100% accurate. Not even close at times. It takes a while to edit the script and align it with the actual video. We are working on it. Some of you might prefer to turn off close captioning. If not you will be entertained at times .

Browser Compatibility

This site should work with any browser, to include those on smart phones. If you have any problems, let us know!

Videos and Downloads

We provide numerous videos and downloadable forms and worksheets. Almost all the videos are under 3 minutes in length, many 2 minutes or less. Formal videos are the exception rather than the rule. We create them intentionally as informal discussions, much like stepping into our offices and having a conversation. The smart phone is our camera of choice.

Feedback

The only way this evolves is through your feedback! Please provide feedback on how this works for you!

  • Is the guide clearly explained?
  • Is it usable?
  • Do the videos enhance the learning/application experience?
  • What can we do even better?

No suggestion is too small. If you see it, write us about it! Email Pat Barker at patrick.barker@dau.mil for all comments - good, bad and ugly.

Why You Care

Earned Value Management (EVM) is important, and not necessarily for the reasons many of us might think. EVM is important because:

  • it serves as a benchmark for best practices
  • it offers, through its flexibility, the ability for program offices to demand (and instill) discipline in program planning, performance measurement and forecasting

And here is another thing or two that a program manager or team leader might find real handy:

  • EVM is about behavior and culture
  • You don’t need to require “EVM stuff” to do “EVM stuff.”
  • EVM is not rocket science.

video icon EVM 101

In fact the argument that EVM can be too expensive to implement and sustain does not hold much water, as long as the Government program office does not impose unique (and often unnecessary) operation and reporting requirements upon the contractor’s EVM system implementation. For more information check out the report below:

download icon JSCC Better EVMS Implementation Study

Mindsets to Watch Out For … and What You Might Do When You See Them

One of the first things most notable about EVM in a Government program office environment is the way in which EVM is mischaracterized. Let us call these EVM Mindsets. Examples of common EVM mindsets:

  • “That EVM stuff” reflects a separation of EVM from the rest of the world such that contractors generating unreliable data and information is an oft-expected, but relatively unimportant thing. Yeah right.
  • A “BFM thing” suggests (wrongly) that EVM is the province of business financial management and nowhere else.
  • “Tail-light” refers to the fallacy that EVM is all about history and does not have predictive capabilities. Guess what? The moment you measure something it becomes historical in nature because time marches on. So much for “leading indicators.” EVM is actually robust in its predictive capabilities.
  • “Lagging” relates to “Tail-light” above and refers to the frustration about doing program management reviews with 45-day old EVM data due to the delivery cycles. Hey, here is an idea: why not pull the current data out of the database instead of relying on a report from last month?
  • “Expensive” is a common argument made by contractor organizations that don’t want to “do” EVM and/or government organizations that don’t want to require it on their contract vehicles. The “expense” in new EVM system building comes with the behavior change.
  • “Tools” & EVM is the weak notion that EVM is just a tool. Well, yes, it has tool-like qualities to it but in reality EVM is a behavior (It is called Earned Value Management vice something like Earned Value Toolkit)
  • “CDRLs” & EVM is the dangerous assumption that all EVM is about is the production of CDRLs, namely the Integrated Program Management Report (IPMR). Sadly, some active contracts still refer to the Contract Performance Report (CPR) which went away in 2012. And many “old timers” still call that CDRL a “Cost Performance Report.”
  • “IBR” & EVM is the erroneous assumption that Integrated Baseline Reviews are all about EVM, and in fact gets extended to the notion that if you don’t have EVM requirements you don’t need an IBR. Combine this notion with “BFM Thing” and “EVM Stuff” from above and you can quickly see why IBRs are the first milestones to be slid “to the right” on new contract awards even though if anything, prudence demands IBRs be shifted “to the left” and begun at contract award. IBRs are about risk and opportunity. In fact, if you want to know more about IBRs, check out another in our series of “practical advice” guides below:

guidebook icon Practical Advice for Integrated Baseline Reviews

Hey PMs and Team Leads: Why not consider the above EVM mindsets as a leadership opportunity? See the video below.

video icon EVM Mindset

Common EVM “Policy-Induced Mindset…”

Title: Cost Reimbursement & Incentive Contracts

Meter labeled on the outside: $20M and $100M. Inside 1/3: Risk-based decision with business case analysis. 2/3 EIA 48 Compliant EVMS Required + IBR + Tailored IMPR. 3/3 EIA 748 EVMS and formal Validation Required + IBR, IPMR. Needle of gauge pointing slightly below $20M labeled $19.9M.

The whole mindset thing eventually leads to the magical “is it above or below $20 Million” question. In reality the decision to use EVM must consider:

  • Dollar value
  • Risk
  • Duration
  • Schedule criticality
  • Existing system, if any
  • Repeat business
  • Information needs

The video below discusses this further:

video icon EVM Policy Mindset

Performance Measurement (and EVM) Starts with the Concept of “Done”

Upon what progress/maturity indicators do YOU make program decisions? And how do you know the information is reasonable? Have you asked yourself if the EVM reporting (what is “earned”, what “% complete”) reflects:

  • Technical review (PDR, CDR, TRR, etc.) exit criteria completion?
  • True “up-front” completion of “downstream” planning?
  • Affordability, achievability and traceability of requirements in submitted technical CDRLs?
  • Producibility of components, subsystems, systems?
  • Reliability, Maintainability and Availability estimates that meet expectations?
  • Criteria met for protection of critical technologies?
  • KPP estimates?
  • ESOH considerations met?
  • Software functionality?
  • PBL expectations planned for?
  • TPM expectations met?

This is why the Integrated Master plan (IMP) is such a critical document. Yet the vast majority of EVMS implementations do not give an IMP a second thought, and nor do their government customers. This needs to change.

Check out the next section and you will begin to understand why an EVMS implementation ought to be anchored upon the IMP.

The Integrated Master Plan is a Key EVM Anchor

Talk about one of the least appreciated, most underutilized and yet one of the most powerful and intuitive tools in program management! Yes, we are talking about the Integrated Master Plan, or IMP.

The IMP converts the Leader’s Intent into something that is integrated into the program planning and execution structure, and the includes EVM in a huge way. An IMP is not rocket science. As DAU Professor Pat Barker notes below in the video, an IMP is simple, intuitive and effective.

video icon Are We there Yet? SA and IMP

An example: An even more compelling story about the practical use of the IMP comes from Colonel Todd “Einstein” Wiest, USAF Senior Materiel Leader responsible for multiple strategic warning & surveillance systems at Peterson AFB, Colorado. Colonel Wiest not only led the development of an IMP for a portfolio of systems in sustainment, but he recognized the cultural and behavioral implications and thus integrated the effort with a framework for thinking and behavior (called Emergenetics®) used in many DAU workshops across the country.

Click HERE to hear Colonel Wiest’s 8-minute chat about his IMP and how he uses it in his command

Every program ought to be able to develop an integrated master plan (IMP) – it is not rocket science and can be done to the “80% good enough level” rather quickly. An IMP answers the question: “What does ‘done’ look like?” The two video clips below are what we call “Defense Acquisition Dialogues.” In these particular clips, which run about 20 minutes each, we have several senior acquisition professions talk about the use and value of IMPs. They are worth the watch!

video icon IMP Dialogue Part 1

video icon IMP Dialogue Part 2

The IMP articulates how metrics (measures of effectiveness, measures of performance, technical performance measures and other relevant accomplishment criteria) shape critical program decisions. The IMP will accurately define a program architecture – a frame of reference for decision-making – consistent with the program’s acquisition plan and other relevant planning documents such as those for systems engineering, test & evaluation and operations & support.

Used correctly, an IMP helps everything be consistent with one another and clearly articulates what program leadership is looking for at major decision points. It is hard to imagine a best practice EVM system implementation operating without one.

Want more on the IMP? Check out another in the series of our “practical advice” guides below:

guidebook icon Practical Advice for Integrated Master Plans

Some Quick Tips and References for Reviewing That “EVM Stuff”

Here are two videos that discuss various aspects of its use within the Government PMO environment. This first video summarizes some warning signs in EVM implementations and things you ought to pay closer attention to should they arise.

video icon EVM Warning Signs

The second video, with help from a local gorilla, looks at the most common indices associated with EVM: CPI and SPI.

video icon EVM Indices with KoKo

How do you really know what kind of situational awareness is out there in your teams when it comes to EVM information? Sometimes, it comes down to seeing what your team leads really know about EVM and, by extension, their industry counterparts (typically control account managers, or CAMs in the latter case). Here is a checklist you can use to ask the pressing questions that are useful to gage comfort levels with EVM. Note that for the most part, these questions could be use with or without EVM as a context.

download icon Government IPT Member EVM-Based SA Interview Checklist

Now comes a quick-reference table for basic indicators. Have you seen any of these?

When you see THIS …

Ask for THIS

Because …

CPI reported at the program level

Program CPI based on calculating discretely (objectively) measured tasks only (e.g. not level-of-effort tasks)

“Level of Effort” (LOE) EVM measurement can distort CPI values (often resulting in CPI>1.0) so challenge $$ value of LOE measurement covering more than 5% of the baseline $$

SPI reported at the program level

Program SPI based on calculating discretely measured tasks only AND how this compares to what the IMS says

“Level of Effort” (LOE) EVM measurement drives SPI values to 1.0 so challenge $$ value of LOE measurement covering more than 5% of the baseline $$ … also “SPI” is NOT “schedule” … Rather, the IMS is schedule

Sustained high SPI and/or CPI at any level

An explanation of how CPI and SPI are derived, to include how much “Level of Effort” percentage is in the baseline

AND

Alert DCMA to potential poor and/or non-compliant system implementation

Sustained, high performance values for CPI and SPI (near or above 1.0) while desired, are historically very difficult to sustain over a program development lifecycle. Odds are there are other dynamics influencing the high figures.

No TCPI (cumulative) versus CPI (cumulative) comparison at any level

TCPI (cumulative) versus CPI (cumulative) comparison.

TCPI is not designed to be a stand-alone metric. It needs to be compared to the CPI. If you see a TCPI > 5% of the CPI then it is time to challenge the EAC. TCPI>10% likely means EAC is unrealistic. Note that you need to ask for TCPI specific to the Estimate at completion (EAC).

Any Reported EAC

A “term by term” explanation (and discussion) explaining the impact to the EAC from each of the elements in yellow box below

An EAC results from a complex interaction of cost, risk, schedule and forecasting discipline. It is OK for your EAC to be different, but know why. Come up with YOUR “Quick” EAC (see “PM’s back of the envelope EAC” below) and ask your PM and/or the contractor how they came up with THEIRS based on your elements. Ask your staff to do the same, but don’t settle for just an “EVM Formula” answer. ALSO compare the EAC to the program office cost estimate and make sure your staff can trace one to the other and account for the differences.

SPI and SV discussions and reporting without specific reference and comparison to IMS

Proof/Evidence that the IMS is integrated with the EVM information and that the exact same tasks in the IMS correspond to specific work packages in the EVM information with identical approaches to measuring performance.

AND

Alert DCMA to potential poor and/or non-compliant system implementation

No IMS equals no clue. ‘Nuff said.

Furthermore, if an EVM system implementation is assessing and forecasting in a manner independent of the IMS (i.e. EVM is measuring one set of tasks and the IMS a different set of tasks for the same work) then all performance management information is to be considered suspect.

The best case, most likely and worst case EACs are equal to each other.

The dollarized risk and opportunity register to see if there is any connection between that register and actual control account performance information and forecasting

AND

Alert DCMA to potential poor and/or non-compliant system implementation

We aren’t buying hammers or kitchenware from a retail outlet. We are using EVM to help us deal with risk. There should ALWAYS be a range of estimates for your EAC, each couched within a context of risk and probability. Best and worst case EACs ought to straddle (in some fashion) what the BAC is.

The EAC is equal to the Contract Price

A meeting with the vendor PM to explain to you what the EAC really is

AND

Alert DCMA to potential poor and/or non-compliant system implementation

No way. This is a flat out demonstration of ignorance of how EVM is supposed to work.

No evidence of analysis that compares IPMR with Cost Funds Status Report (CFSR) reporting

A comparison of IPMR and CFSR information to see if the EAC prediction will be compatible with what you are actually planning to pay the vendor over time.

The IPMR is about cost and the CFSR is about price. The difference is fee. The two should move in parallel. If they are not, then you are headed towards a problem with obligations, expenditures and even a potential breach without knowing it. Another key difference is that the people who put together the IPMR and the people who put together the CFSR are different people, and they need to talk to each other.

For convenience, you can download the following sheets with tables and suggestions that echo much of what has been shown above, although in greater detail in some cases.

download icon EVM Cheat Sheet for Government PMs

download icon Red Flag EVM Indicators for IPTs

It is implicit in the implementation of EVM systems that a program is going to have a Work Breakdown Structure or WBS. You need one at the Government PMO as well. To help you along in that thinking, check out the below:

guidebook icon Practical Advice for Work Breakdown Structures

A PM’s “Back of the Envelope” Most Likely EAC

A couple years back, DSMC Professors Pat Barker and Roberta Tomasini co-wrote an article designed to help PM’s think about questions to ask when presented with an estimate at completion, or EAC.

download icon What’s in Your Estimate at Completion?

Here is what the equation means: This EAC is a number the Government PM can quickly calculate himself/herself “on the back of an envelope” and use to engage his/her counterpart in a meaningful EAC discussion (e.g. ‘how did you arrive at your estimate?’).

It supplements EACs derived by Gold Card formulas and does NOT replace PMO staff analysis. Term explanation as follows:

  • ACWP - your actual costs to date.
  • BCWR - the dollarized value of your remaining work. (BAC- BCWP).
  • MR - the MR dollars that still remain.
  • DOLLARIZED RISKS - the sum total of each program cost risk impact.
  • “SLIP” - the number of months your contract end date might slip based on SRA results.
  • “BURN” - the monthly burn rate in effect when your most significant schedule risk impacts the program.

Asking questions based on each term (Examples: “Does your EAC reflect the’ marching army’ (aggregate support staff) cost impact” based on schedule risk analysis and possible program schedule slip? How did you derive your MR and to what degree do think it will be “burned”?) help drive an interdisciplinary EAC discussion.

“Quick” EAC Formula

Require an Integrated EAC

Each term in the formula above is a potential conversation based on a powerful set of questions. The video below explains it further.

video icon EVM Integrated EAC

The References You Ought to Review

There are a variety of guides and artifacts that have been built to help you maintain awareness of multiple aspects of Earned Value Management implementations.

EVM information from contractors is typically delivered to Government program offices via the Integrated Program Management Report, or IPMR. Consider what you ought to look for in an IPMR by downloading the IPMR guidance below:

guidebook icon DoD IMPR Implementation Guide

EVM system implementations across the US, especially where DoD is concerned is based on the American National Standards Institute/Electronic Industries Alliance Standard 748, Earned Value Management Systems (typically referred to as EIA-748). It is copyrighted material so click HERE to go on their website to buy it. In a similar vein the Project Management Institute (PMI) offers the Earned Value Management Practice Standard.

The Department of Defense lead for EVM is PARCA. Based on EIA-748, they developed the following guide to help program offices set expectations for EVM performance and information.

guidebook icon DoD EVM System Interpretation Guide

guidebook icon PARCA EVM Application Determination

The most common source for EVM information resources tends to be the National Defense Industrial Association. They have a number of guides as follows:

guidebook icon NDIA EVM Acceptance Guide

guidebook icon NDIA EVM Application Guide

guidebook icon NDIA EVM Intent Guide

guidebook icon NDIA EVM Scalability Guide

guidebook icon NDIA EVM Surveillance Guide

guidebook icon NDIA Predictive Measures Guide

Increasingly we see program grapple with the use of EVM with Agile implementations. Below you can download a desktop guide to help navigate that space for better situational awareness.

guidebook icon PARCA Agile and EVM PM Desk Guide

Over the past decade, the US Government Accountability Office (GAO) has tackled subjects such as EVM “head-on” in what only could be called a selective crowd source approach. The GAO gathered together experts in their fields from across industry and federal government to produce best practices in both Cost Estimating and Scheduling, both core to EVM capability. The result is a pair of guides that are each outstanding sources for your PMO team.

Below is a video by the GAO Chief Scientist, Dr. Tim Persons, on the origins of these guides.

video icon The GAO Guides: Cost, Schedule Performance

These guides are downloadable below for your convenience.

guidebook icon GAO Cost Estimating and Assessment Guide

guidebook icon GAO Schedule Assessment Guide

EVM skills (and the program management and specific program knowledge to support it) can be taught, of course, but keep in mind that EVM is just as much of a mindset as it is a tool. And if you want to discern among EVM talent in terms of hard skills, one starting point is to determine what EVM courses an individual has taken at DAU. Click HERE for the DAU online training center. Just look for courses with the “EVM” designator.

Another great starting point is to examine professional certifications. The American Association of Cost Engineering offers the rigorous Earned Value Professional (EVP) certification.

Parting Shots: What EVM Really is

Policy tells us to require EVM on contract costs $20M or above and that it is a risk-based decision for contracts under than amount. Rumors tell us that EVM is difficult and expensive. When it all comes down to it, EVM is really a behavior that creates performance information capable of supporting leadership decisions. Check it out below:

EVM Means Expecting the Contractor TO:

  • Plan all work scope
  • Assign all work scope
  • Integrate scope, schedule and cost for measuring performance
  • Record actual costs incurred
  • Objectively assess accomplishment where work is being performed
  • Assess variances, implement corrective actions and periodically forecast
  • Use performance information for decision-making

So the Gov’t PMO Receives Performance Information THAT:

  • Is exactly the same as what the contractor uses
  • Is timely, accurate, reliable and auditable
  • Relates time phased budget to scope of work
  • Measures progress as objectively as practicable
  • Enables independent Gov’t predictions of future cost and schedule conditions
  • Contributes to APB trade-space decisions

This is not rocket science. Getting all this cool stuff above is not cost prohibitive. When you really look at the above statements, you have to ask yourself, EVM or no EVM: “why would we _not_ want this?”

So, go forward and make EVM – in part or in total -- a key contributor to your program success!

+Notes
+Bookmarks