Brett Schultz is the Marketing Manager at PRICE Systems. He's trying to move from Novice to Intermediate in terms of his knowledge on estimating, so please be kind.

That’s a question that members of the cost estimating community will try to answer at next week’s  43^rd Annual DoDCAS symposium. The conference is centered around the “theory and implementation” of the WSARA 2009 (Weapons System Acquisition Reform Act). Simply put, the cost estimating community needs to first and foremost understand what the real life implications of the WSARA will be. At least in theory that is.  

According to GAO, nearly 70% of the Pentagon’s 96 biggest weapons programs were over budget in 2008 and another government report found $295 billion in waste and cost overruns in defense contracts. Clearly things need to change.

As the Deputy Secretary of Defense, William Lynn, correctly states, overly optimistic estimating is at the root of cost overruns.

“We rely on overoptimistic cost estimates. These estimates are success oriented…they assume every step in the development process will go as planned. To produce weapon systems efficiently, it is critical to have budget stability. But it is impossible to attain stability in our modernization budgets if we underestimate the cost of our weapons systems from the start.”

Lynn also states the need to bolster personnel by 20,000 positions over the next five years with a heavy focus on cost estimators, systems engineers, and acquisition managers. 

Naturally, PRICE Systems, is eager to support the acquisition community as it attempts to implement the key initiatives of the WSARA. We will have our some of our cost research experts on hand to present and discuss best practices in cost estimating and they will be reporting back on what they learned and observed at the conference.

Stay tuned.

It’s a common practice to measure failure or success of a project based on the initial functionality requirements and initial cost and schedule estimated.

 

The Standish Group publishes its Chaos report for software projects which terms a project as a "Success" if it is completed on time, on budget, and satisfying all the initial requirements. Projects are deemed a "Challenged" if functionality is achieved but cost and schedule over runs occur and "Failed" if a project is cancelled while in execution.

 

However there are other factors e.g. Tom DeMarco’s  Estimation Quality Factor  and Boehm’s Cone of Uncertainty (COU) or combination of these two (a brief analysis of these two factors can be found in the Jan/Feb IEEE Software magazine The Rise and Fall of the Chaos Report Figures: http://www.computer.org/portal/web/csdl/doi/10.1109/MS.2009.154).  

 

The questions is, does your company measure project success/failure based on initial software estimation? If yes , what methods are used at your company to establish the matrix (Standish, EQF, COU, any other)? In your opinion what is the value of software estimation accuracy?

It seems that business and government today is mired in cost-benefit analysis. The cost side is fairly easy to understand: how much money will the business (or government) have to expend to deliver a given service? Benefits are a little harder to quantify.

First, there is the difference between monetary versus non-monetary or social benefits. Some things like the office Christmas Party do not make any money but they keep morale high. Second, there is the difference between short-term and long-term costs and benefits. The company using Blackberries may bring the immediate satisfaction feeling mobile and linked into work but there may be some long-term costs.

Does our decision-making process really take these factors into consideration? Do people who live beyond their means really weigh the long-term financial cost of living on credit versus the immediate (non-monetary) benefit of feeling important? Do we consider the long-term productivity gains by investing in work-force/employee training immediately?

Unfortunately, I think the heart wants what the heart wants. Businesses and individuals do not weigh benefits in a methodical manner. So until the day when benefit analysis ceases to be in the eye of the beholder, we should focus on accurate cost estimation.

Bill Scheessele’s article in the Washington Technology, Time for a hard look at marketing strategies,  is a quick read about organization’s reacting to these new economic conditions.

He suggests that most organizations are...

"Attempting to move forward with an obsolete business development operation, reacting to fewer opportunities and shifting budgets by shedding business development personnel, sticking with an outdated business development process that everyone in the industry uses, or doing nothing while waiting on the sidelines for conditions to change are not reasonable decisions."

This is further supported by the recent flurry of acquisitions both inside and outside of the federal market space.　 At least organizations acknowledge their lack of agility and ability to evolve and are acquiring expertise and capabilities in new market areas that provide the HOPE of growth – and stability.　 Regardless of the methods, business as usual is not an option.

Mr. Scheessele proposes a few proactive ideas that organizations should consider:

Invest Resources in conducting a comprehensive assessment of your entire business development operation: plans, personnel, and processes.

Design, build and implement an up-to-date, customized busines development methodology that fits your organization's mission, culture, and offerings and provides the structure, discipine and thinking necessary for revenue growth.

Like Mr. Scheessele, PRICE Systems believes in developing processes that enable organizations to become more nimble and proactive. 　　We are constantly asked to support process improvement initiatives to do just that. 　

If you are interested in how your organization can achieve some of the practices Mr. Scheessele’s promotes, take a look at Dale Shermon’s recent webinar or go take a look at PRICE Systems Bid & Proposal Solutions 

Recently, Dale Shermon presented a webinar, "Preparing Bids Faster with Fewer Resources".  The content for the webinar was taken, as the title of this post suggests, from the recently released book Systems Cost Engineering.

The webinar discussed how parametric estimating could dramatically decrease the time and thus the cost that is required to make important business decisions about whether or not to pursue contract opportunities. There are critical activities that organizations engage in every time there is an RFP such as Bid/No Bid decions or competitive assessments.  So whether the RFP calls for estimating software costs or a system that requires hardware/software integration, parametric estimating can provide fast and accurate information for making decisions. PRICE has been able to save its clients millions of dollars by reducing the resources typically needed to make these decisions and ultimately win the contract.

The presentation also does a nice job of walking through the different applications of parametrics. So if you are unfamiliar with this type of estimation, Dale's commentary at the beginning of the presentation can be very useful. But I'll let you be the judge.

To download the recorded version of this webinar go here.

Whlie most of the books on the topic of parametric modeling take a look at detailed techniques and fundamentals, such as building parent/child relationships or the mathematics behind models, Systems Cost Engineering, takes a more practical perspective to answer a very basic question:  What can parametric estimating do for my organization and how can we implement it? 

The book covers an array of business processes that can be dramatically improved with the application of a standardized parametric cost estimating framework. These processes exist across multiple phases of a program's lifecycle  such as early concept planning through development and production. Chapters are devoted to various How To's such as preparing bids faster, better business planning, evaluating vendor quotes, estimating software costs and schedules and analyzing risk and uncertainty.

The book is a collaboration of work over the past 30 years in the field or parametric cost estimating. It was brought together by PRICE Systems executive, Dale Shermon, who was recently awarded the Frank Frieman Award the International Society for Parametric Analysts' highest honor.

The book is available for purchase at Amazon.com and from Gower Publishing.

Let’s start with a simple test. Which is greater: the number of six-letter English words that have "n" as the fifth letter or the number of six-letter words ending in "ing"?

If you are like most people you’re thinking the correct answer is six-letter words ending in "ing". But most people are wrong. And the reason is simple, people rely on what they can easily recall. Since it’s much easier to think of 6-letter words ending in "ing" the fact that people come to that conclusion isn't suprising. Psychologists refer to this as availability bias. This type of bias gives more importance to things that we can vividly remember and easily recall. All too often, bias like this plays a prominent a role in the development of estimates. (By the way, the group of six-letter words that have "n" as the fifth letter include all of the six-letter words ending in "ing")

Part of the power of parametric cost estimating is the ability to substantially diminish this bias and to fill in where data is lacking. A parametric approach uses sophisticated mathematical equations along with historical data from similar systems or projects in combination with existing organizational data to produce an estimate that is external to the types of biases that often plague estimates. Once organizations are estimating software, hardware or systems of systems using a parametric model, the accuracy of the model will increase as data is captured from other projects. This systematic collection of actuals compared to estimates over time leads to data driven decision making.

Parametric cost estimating is a widely used approach for bidding on a contract, input into a cost benefit analysis, or as the pre-planning tool for project implementation. Extensive literature reviews suggest that an effective parametric cost estimating methodology is becoming an essential tool for technology-driven organizations. The use of parametric estimating in budgeting, scheduling, and control of projects will enhance the ability of project management organizations to effectively and efficiently utilize valuable resources. The benefit of parametric cost estimating tools is its use as an estimating model for better determining potential resource requirements during the project pre-planning and conceptual phase. When software cost estimation is performed using a parametric approach with proven commercial framework, the benefits realized far outweigh the cost of doing the estimating.
 

Author Nassim Taleb describes a Black Swan as an unexpected event that ultimately leads to what best can be described as a paradigm-shift. In his book, The Black Swan, he includes 9/11, the rise of the Internet, Google and the personal computer as Black Swan events. We could not have predicted those events but they have had a huge impact on us.

Even positive Black Swans can be a source of frustration. Since so few people are prepared for them, it becomes impossible to profit from them. The group that is most frustrated by Black Swans are probably those whose jobs involve cost estimation.Cost-estimators rely on past data to forecast costs so it becomes very difficult to estimate the costs of Black Swans. So where are the new Black Swans and how should Cost-estimators prepare?
I think that the new Black Swans are those that deal with smart technology. This category includes:

• Smart Infrastructure: roads, bridges, etc that have built-in detection and analysis capacity
   
• Smart Energy: While there has been a lot of emphasis on alternative energy, the real paradigm shift will be in a grid that adjust itself to maximum efficiency. This will involve estimating software costs for new systems
• Smart Healthcare: Healthcare IT will create efficiencies in our bloated healthcare ecology. The digital exchange of records will eliminate redundancy and enable cost-effective delivery. This will usher in the need to develop effective IT project cost estimation.

Many of these technologies are captured in the stimulus. With the blessing of the federal government, we will see venture capital quickly follow suit. So how does the cost-estimator adapt?

First, we must analyze analogous systems. The ATM/Banking System is usually good place to start since there are a lot of similarities, especially with Health IT. There also have been some small-scale implementations of these technologies. Abu Dhabi is building a city of the future that includes smart technology.

Second, we must learn the effect that networks have on cost. Too often, we, cost estimators, wallow in the weeds of systems. We concern ourselves with how many laptops do we need rather than what effect do the laptops working together have on costs. We must ask ourselves repeatedly if the whole is greater than the sum of the parts.

Finally, we must view technology as a service rather than a product. We must evaluate architectures such as SOA, Cloud Computing and Grid Computing. We must analyze how such systems determine requirements and values.

So this is the challenge. The Black Swans have landed. They require estimation to be accurate and thus, proven cost estimating tools must be a part of the equation. Now, we need to shift the way we approach our work and answer the call.

In a seminal RAND Corporation report, Bureaucracy Does Its Thing, author and former CIA agent K.W. Komer promotes the idea that the mindset of America’s institutions led to problems in Vietnam. His thesis is that the bureaucracies of the U.S. were fixated on fighting the Vietnam War according to how the bureaucracies had prepared and organized instead of in manner that the situation required.

As I read the stimulus package before Congress, I see a focus on the idea that new technology will create jobs. Yet I also have read that Governor Schwarzenegger is consolidating the technology departments in the California state government to reduce redundancy. This seems to make sense since technology tends to be scalable with a low marginal cost. Therefore, technology consolidation is strong budget-cutting tool but one that inevitably leads to workforce reduction.

Here is the question: should the government rely on technology investment as a way to bring jobs to the economy or has Governor Schwarzenegger discovered a blue print for reducing costs in cash-strapped states. My answer is: it depends on the situation.

This is where business case analysts and cost estimators become important. Indeed, technology creates jobs in terms of implementation but tends to reduce them in terms of operations. Business case analysts should take that into consideration as they consider cost-benefit trade-offs. Cost estimators also must take into account total cost of ownership of a project. Often times, there are implicit tasks that are "hidden" and thus, actual job creation may not be considered.

I choose not to hazard an opinion on the merits of the stimulus’s approach versus that of the California Governor. However, I will say that strong business case analysis and cost estimation will help eliminate pre-conceived notions in a given scenario. Therefore, these skills can keep us from falling into the trap that Komer described in his report.

Dude, Where’s my Stimulus?

If you turn on the news or open the newspaper, you will definitely come across an item about the stimulus package. Obviously, there will be much debate to come over the price tag for the initiative. But I wanted to take a moment to discuss where resources will be directed and how they will be dispersed. I also would like to talk about how this bill might affect us in the "Land of Estimation."

High-tech is the theme for this stimulus. Most projects will involve technology upgrades of one sort or the other. Even shovel type projects will have a futuristic spin to them. For example, school repairs are geared toward energy efficiency. Health IT, alternative energy and high-tech manufacturing are at the top of this agenda.

An even more important point is that not a lot of new bureaucracy will be created. Instead, existing federal bureaucracy will be expanded. For example, the Manufacturing Extension Program, a Clinton-era initiative, will receive more funds. The Office of the National Coordinator, a Bush-era Health IT office, also will receive more funds.

So what does this mean for business, in general, and estimators, specifically?First, partnerships will be essential. Estimators should partner with federal agencies and programs on the dispersal side. These groups will be focused on accountability and affordability. Their goal is that no dollar is wasted. On the reception side, partnerships should be formed with states and industry-leaders. These groups will be interested in developing accurate business-cases. Their goal is to demonstrate that their projects create value.

So, the stimulus will indeed be controversial. But it also may create a golden opportunity for the estimation world.

It seems that, even in the midst of billions of dollars of bailout funds, the federal government is taking accountability a little more seriously.  Politics aside, the government is making the effort to ensure that every dollar spent is an investment rather than waste.  As we all know, industry follows the lead of government. 

We see this effort for accountability in a recently introduced Senate resolution (that will be re-introduced in the next Senate), S. 3384. The Information Technology Investment Oversight Enhancement and Waste Prevention Act of 2008 was sponsored by Sen. Tom Carper of Delaware.  The act was a result of IT cost-overuns in the Federal government.

The Act requires that Federal CIOs provide Congress a business case/baseline estimates for IT Investment Projects as well as performance reports.  Gross deviations must be reported to Congress along with a remediation plan.  All estimates must be independent.

So the implied task for Federal CIOs is to develop a framework for creating independent cost-estimates and conducting gap analysis between baseline estimates and actual cost.  Knowledge of the framework will be a pre-requisite for project managers and planners.  Eventually, these practices will trickle down to the private sector.  Parametric cost estimating based on industry data may provide a foundation for such a framework.  Ultimately, parametric cost-estimating can be the key to IT accountability!