Cost Realism

Competition seems like a sensible practice on the path to best value.  However, is it possible to create a truly competitive environment?

“It is a tradition that we don’t trust our business partners, people don’t have a clear understanding of how sharing information would result in better performance.  The lack of understanding induces fear and skepticism.”
Hau Lee, Graduate School of Business at Stanford University

There are two behaviors that debunk competition in regards to complex systems wrought with uncertainty: over-optimism and deception.  Government buyers share the dual burden of seeking advanced technologies to meet future capabilities while asked to determine best value of goods and products that lack historical reference or data.  The real challenge in meeting cost reduction and competitive initiatives falls in the hands of those responsible for defining and achieving best value for their organization – acquisition and purchasing professionals. 

Acquisition professionals must be armed with tools, knowledge, and methods that provide rapid, credible identification of predicted cost and risk.  Knowledge capture and estimating practices have proven to detect deceptive estimating, eliminate mistakes and anticipate the risks associated with optimistic predictions of new technologies.   From better selection of COTS, to benchmarking suppliers, to supporting real business case - our customers are working to understand the systems they are acquiring and the goods they are purchasing.  They are finding ways to come to the negotiating table from a position of knowledge - a position that promotes cost realism and ultimately honest competition.

Have you seen real competition at work in your industry?

I had expected to present my webinar,  “Best Practices for Cost Effectiveness Studies using TruePlanning” in early August. As you might know, I was planning to show a real world example from a recent engagement with a government customer. Unfortunately, since the Source Selection has not concluded with a downselect, I was not able to obtain the public release in time. However, for this month’s blog I will continue share some of the highlights of the webinar.

In last month’s blog we explored the uses of TruePlanning during Source Selection from the Supplier’s (or Contractor’s) perspective. This month, I would like to share with you some of the uses of TruePlanning cost estimating software from the Customer’s (or Government’s) perspective:

• Analysis of Alternatives (AoA) – Is proposed technical baseline cost-effective against other competing alternatives?
• Cost Realism – Are the performers bidding within an accurate range
• Data Driven Estimating – Are the performers bidding based on appropriate, traceable historical data points.
• Independent Cost Estimate (ICE) – Using the performer’s technical configuration, what does a completely independent look say about the performer’s bid?
• Risk Analysis – Is our bid over conservative, how much risk are we willing to take and how much cost exposure can we absorb?
  
• Schedule Estimating – Can we really do the job within the schedule constraints?
• Growth Estimating – What other configurations, materials or technologies might we consider?

While TruePlanning is useful in estimating project costs and all of the above types of analysis, I have found it most useful in developing AoAs. During Source Selection, TruePlanning provides a structured, repeatable framework that can rapidly develop cost estimates across a range of alternatives. These results are often incorporated directly into Operational Effectiveness models along with Measure of Effectiveness and Measures of Performance. In some cases, clients using tools such as Phoenix Model Center can directly link TruePlanning with optimization tools as well. 

Last month I blogged about the importance of cost realism, its roots and how as estimators we must always reflect the truth, no matter how unpopular. This month I want to share with you a recent experience on a Source Selection. As part of the Source Selection team, my role was to conduct a Cost Realism estimate on each of the performers submitting bids. I want to share with you a few insights from that experience.

One of the first rules I always follow is to never ask engineers to provide data that you can readily find. I rarely will use parametric data forms, as you will get to be unpopular fast with those you task with the honor of filling them out. Rather, I dig into the technical and cost volumes to derive the configuration, technology, weight statement, rates/overheads and other juicy information to populate TruePlanning.

Once I have the TruePlanning cost management software populated with all of the data I can derive from existing documents, I will then ask the subject matter experts in each area to discuss the other qualitative/quantitative factors about each performer. I can usually guide this conversation to derive inputs such as requirements stability, engineering complexity, integration and others. I find it effective to hold this meeting remotely using Go-To-Meeting so everyone can see my desktop displaying the TruePlanning model and no one has to leave their desk. When inputs are critical cost drivers, I usually pop up the wizards or generators so the engineers can see the choices available.

Once the model is fully populated and all inputs agreed to, I will then produce a very well documented estimate with all assumptions as the final deliverable. In this particular case the Source Selection team was very pleased with the result and asked if we could extrapolate a new configuration based on some technical changes proposed by the performer.

Using our well qualified TruePlanning model, we ran the extrapolation and reported to the Source Selection team that we expect to see a large impact. Initially, no one believed the results; it could not be so they said! However, several weeks later, when the performer’s new estimate arrived, we were within 1% of the revised bid. During this time period, the Source Selection team was able to successfully prepare a negotiating stance based on the expected bid coming in as predicted.

In all my years as an estimator, I have only seen estimates go up as more is known about the technical configuration. Bottom line, as estimators we are the “front line” for telling management the hard to hear truth. In this case, at least they were prepared!

With so many acquisition programs over budget and behind schedule, the term “Cost Realism” is suddenly very popular. In my experience as an estimator on many major acquisition programs, two things have remained certain over years (besides death and taxes). First, the probability of the program ever achieving the original cost estimate is exactly zero and second, the more information that is known about a program, the more it will exceed its original cost estimate.   

With that said, the move to Cost Realism is so important because it recognizes these two fundamental facts. Much of the interest in Cost Realism is driven by the Weapon Systems Acquisition Reform Act of 2009. According to US Senator Carl Levin “Report after report has indicated that the key to successful acquisition programs is getting things right from the start with sound systems engineering, cost estimating, and developmental testing early in the program cycle. The WSARA also calls for a “Director of Independent Cost Assessment to ensure that cost estimates for major defense acquisition programs are fair, reliable, and unbiased.”

If we go a bit further and look at definition of Cost Realism in the Parametric Cost Estimating Handbook we find  that “no one expects a cost estimate to precisely predict what a hardware or software project costs or a time and material service will cost. So, cost realism is not about the exact cost estimate. It's about the system of logic, the assumptions about the future, and the reasonableness of the historical basis of the estimate. That is, it's about the things that make up the foundation of the estimate.”

So while we recognize that at its core cost estimating can never be exact, cost realism seeks to ensure estimates are closely tied to the fundamental realities of the program as we know them at the time, without regard to politics or undue optimism.   As I finish this blog, I’m struck by the line in a Few Good Men “You can’t handle the truth!” As estimators, we need to always reflect the truth, and that is what cost realism is all about.