Estimating Insight

Webster’s definition of the word Estimate is to judge tentatively or approximately the value, worth, or significance of. To me “to judge approximately the value of” is dictionary speak for a guess.  At the end of the day, an estimate is a guess.  And anyone of us can make a guess about anything.  But somehow, we value some guesses above others.  What makes one person’s “approximate judgment” better than another person’s?

Over the past 12 years of working for PRICE Systems and talking to hundreds of estimators across the world, I believe the answer is less scientific and more emotional.  Despite all the math and science that surrounds estimating there is an emotional dimension to the determination of how “good” an estimate is.  We have constructed an internal mechanism that looks at a problem, takes the information people are giving us and says “yeah, that’s about right”.  What makes up that gauge inside us that lets us know an estimate is good?  

• Is it a combination of the experience of the estimator?
• The quality, quantity and relevance of the data employed to support the decision?
• The approach and accuracy of the methods?

What makes you feel good about an estimate? Leave us a comment.

The following is a collection of some of the more common installation issues.  If you have additional questions, concerns, or issues with any upgrade or installation, please call our Technical Support line anytime at:  1-800-43-PRICE.

Default Password Override

TP’s default password may not be strong enough based on your security policy.  Please review your specific requirements for passwords and follow the directions below to change the default:  When you get to the screen, uncheck the “Use Default Password” and enter a valid password based upon your specific security settings.  Then click next and continue the installation process.

Existing TruePlanning Installations

If TruePlanning 2008 SR2 is currently installed, it is recommended that users export their projects, then uninstall TruePlanning, making sure to backup their database when prompted to do so during the uninstall. The uninstall of TruePlanning can be initiated from the Control Panel.  Also, please select the option to uninstall MSDE when prompted during the uninstall.  Once this is complete, install the new version of TruePlanning, making sure to manually import the projects which were exported from the older release once the installation is complete.  Upgrades from TruePlanning 2009 and above do not require TruePlanning  to be uninstalled as the TruePlanning 2010 installation will automatically upgrade the projects in the database; however, it is recommended that users still export their projects manually and before the upgrade to be safe. 

Administrative Rights

The user installing TruePlanning requires administrator rights on the machine.  On Vista it is advisable to launch the installation of TruePlanning by navigating the “Setup.exe” file on the TruePlanning CD and initiating the installation using the “Run as Administrator” option. This option is available from the right click menu in Windows Explorer (My Computer). 

XmlLite

XmlLite is a resource provided by Windows to perform certain XML parsing abilities. It is a prerequisite for installing TruePlanning 2010. If TruePlanning 2010 is installed on a computer that does not have XmlLite, and error will occur when TruePlanning is launched.  The installation will not produce any errors or warnings. This error is Windows XP specific. Vista and Windows 7 contain the XmlLite natively. Normally XmlLite is installed when Internet Explorer 7.0 (or newer) or Windows XP SP3 are installed. Installing either of these resources should resolve the error.  Some customers are not able to install IE 7.0 or Windows XP SP3 and should therefore install XmlLite directly. It is available from Microsoft at: http://support.microsoft.com/kb/915865. It should be noted that TruePlanning 2010 has features that rely on Internet Explorer 7.0 or higher. Some of the Calculators contained in TruePlanning will not function properly without Internet Explorer 7.0 or higher.  Therefore, if possible, upgrading Internet Explorer is the preferred option. 

MSXML6 SP2

There is a known issue installing TruePlanning 2010 on computers that have Windows XP SP3 and MSXML6.0 SP2 installed.  The issue occurs because of an incompatibility between SQL Server 2005 Express and MSXML6.0 SP2. This issue is described by the following Microsoft Knowledge Base article: http://support.microsoft.com/kb/968749. MSXML6.0 SP2 is installed by the Windows XP Service Pack 3. This is not an issue on Vista.  The above MS KB article provides a fix for the issue. It is advisable to execute the fix before attempting to install TruePlanning 2010. It is possible to identify computers that have had MSXML6.0 SP2 installed by examining the “Add/Remove Programs” feature under the Control Panel. If the issue in encountered during the installation of TruePlanning it can be identified by the failure of SQL Server to install. The failure will be identified as pertaining to MSXML6. This can be ascertained after the installation has been exited, by examining the log of the SQL Server installation. The summary of the SQL Server 2005 Express installation can be found at the location defined by the following support page: http://msdn.microsoft.com/en-us/library/ms143702%28SQL.90%29.aspx. If the error has been encountered please contact PRICE System’s Support for help resolving the issue.

This FREE workshop features government and industry speakers discussing how the current fiscal environment impacts the day-to-day estimating challenges faced by government program offices and their commercial counterparts.  These events seek to bring leaders together to share ideas and experiences about their most pressing issues.

We are looking to professionals, such as you to contribute ideas and best practices.  For more information or to register:http://www.pricesystems.com/services/predictive_modeling_schedule.asp

“I think we have an obligation to work with industry to ensure that our suppliers do not just remain world class in defence, but aspire to be world-class manufactures that can withstand comparison to other industries.”

Chief of Defence Procurement, Sir Robert Walmsley

Is this a practical proposition or is it a pipe dream?  The following excerpt from Dale Shermon’s Systems Cost Engineering attempts to make the case that this type of comparison is possible.


Many of the statements in proposals and marketing literature stating the superiority of a company are anecdotal or at best qualitative evidence. It has not been possible to quantify the productivity of industry to demonstrate that one manufacturer is more efficient than another.

In the ship building industry it is possible to benchmark commercial ship yards using a metric called Compensated Gross Tonnage (CGT). CGT measures the complexity of building a ship. It is calculated by multiplying the gross tonnage by the relevant CGT coefficient (calculated by the Organisation for Economic and Commercial Development [OECD]).  However, this is only applicable to this one industry. Other metrics exist in other industries, but what is really required is a single benchmarking technique and measure.

Benchmarking with Cost Normalization.  Forty years ago, PRICE Systems developed a metric that can be used to normalize cost and support the benchmarking practices needed to reach Sir Walmsley vision.  Manufacturing Complexity represents the normalized cost density of an item that has been manufactured. It has two dimensions: technology and productivity. Hence, more than one Manufacturing Complexity can exist. As aircraft are made using different technologies and by different companies, these have various complexity figures. However, if the technology is constant, the only dimension that changes is productivity. As a result, it is possible to compare the efficiency of companies that produce similar technologies.

Supplier Assessment.  One obvious and practical application of this technique is Supplier Assessment. This technique exploits the capability of parametric models to compare organization’s normalized cost density or Manufacturing Complexity. If, on paper, you have very similar proposals, how can you judge which organization will provide the best value for money? Heritage data will determine the level of development that is required and Operating and Support Cost will establish the Through Life Cost, but if these cannot differentiate the proposals perhaps the productivity of the companies should be considered. This is not a simple comparison, but with manufacturing complexity this can be achieved easily.

This process will lead to your suppliers knowing that you are able to control them; they will need more control on themselves. It will enable you to perfect your negotiation skill, leading to more success, as better negotiation material is utilised based on technology cost drivers. A parametric questionnaire is a possible tool to enable proposal validation in minutes.  All this information can be garnered at a very early stage in a project if required, providing better estimating accuracy for the purchased items. Normalization enables a bigger data sample as previous proposals received from the same companies can be compared. This in turn enables the establishing of trends. Are the suppliers becoming more or less efficient? Ultimately, these comparisons can be used to determine preferred suppliers.

Conclusion.  This method of benchmarking a project or company can be used for all environments. With the ability to move across environments it is possible to compare commercial and military organisations equally. Decisions and judgments can be made between Commercial and Military suppliers where no military competition exists.  It is possible to assess the productivity of an organisation or country at System, sub-system and equipment levels. The technique can be used to compare the whole cost or just the labour cost if that is the focus of your efficiency drive.

It is possible to determine preferred suppliers on a justified basis, or alternatively it is possible to demonstrate the efficiency of your organisation when the customer is skeptical about your productivity or your drive to improve productivity.

Finally, many organisations already have a license to use the PRICE models and over 8,000 people worldwide have been trained and are familiar with the PRICE methodology. This is not a new idea, PRICE has been established since 1975, but this new application of an established technique can help decision maker take the right decision regarding source selection cross industry and national companies. Alternatively, it can enable industry to demonstrate, with proof, that they are more effective than other solutions.

Competition seems like a sensible practice on the path to best value.  However, is it possible to create a truly competitive environment?

“It is a tradition that we don’t trust our business partners, people don’t have a clear understanding of how sharing information would result in better performance.  The lack of understanding induces fear and skepticism.”
Hau Lee, Graduate School of Business at Stanford University

There are two behaviors that debunk competition in regards to complex systems wrought with uncertainty: over-optimism and deception.  Government buyers share the dual burden of seeking advanced technologies to meet future capabilities while asked to determine best value of goods and products that lack historical reference or data.  The real challenge in meeting cost reduction and competitive initiatives falls in the hands of those responsible for defining and achieving best value for their organization – acquisition and purchasing professionals. 

Acquisition professionals must be armed with tools, knowledge, and methods that provide rapid, credible identification of predicted cost and risk.  Knowledge capture and estimating practices have proven to detect deceptive estimating, eliminate mistakes and anticipate the risks associated with optimistic predictions of new technologies.   From better selection of COTS, to benchmarking suppliers, to supporting real business case - our customers are working to understand the systems they are acquiring and the goods they are purchasing.  They are finding ways to come to the negotiating table from a position of knowledge - a position that promotes cost realism and ultimately honest competition.

Have you seen real competition at work in your industry?

I consistently run into this idea of data driven estimating.  Yet, there is no clear explanation of this concept.  I am not trying to provide one here, however, I am interested in is what is at the root of this growing movement.  My take is that it is an attempt to scratch an itch.  But what’s the itch?

I believe it is related to my early post (Accuracy is Risky Business).  In the struggle to answer the accuracy question people have decided that understanding the data used in the estimating process is key to understanding its accuracy.  To a certain degree this makes sense.  It is useful to gain insight into the information upon which a cost estimation model was based.

• How much data was available?
• How current was the data?
• Was it relevant to the project being estimated?
• What statistical techniques were employed to analyze the data?

In terms of data, there are two general areas considered relevant: data quality and fit. Understanding how much data was used; how old it was; where the data came from sheds additional light on the results. Once the amount, age and source of the information is established, the next concern is how the information was analyzed. 

Descriptive statistics are often used to describe a collection of data in quantitative terms. They aim to quantitatively summarize a data set. Descriptive statistics include many measures including central tendency (mean, median, mode), dispersion (standard deviation, range, quartile) and association (correlation). Additionally, there is statistical inference, which makes propositions about populations, using data drawn from the population of interest via some form of random sampling. 

The industry is fraught with estimates backed by reams of data surrounded by all the necessary statistics. But does understanding descriptive statistics and statistical inference alone improve our understanding and confidence of an estimate?  If so, why then is expert opinion one of the most widely used forms of estimating?  Why do programs based on these data driven estimates continue to perform poorly?  Is data driven the answer?  What do you think?

As is often the case, there is a difference between definition and connotation. The intended meaning of consumerism at these events was the phenomena of the consumer goods infiltrating and influencing government IT policy.  The IPad was all too visible in the audience as were the comments from CIO staff about their user base wanting to integrate IPad-like products into the work domain.

Like it or not we are all consumers; and consumers are getting more tech savvy.  As the adoption of these products or services increase in our daily lives, the question becomes unavoidable – Why can’t I use these for work? It was clear that for the CIOs attending these conferences this question was top of mind.  While most of the U.S. government is attempting to identify, inventory, consolidate and remove redundancies of IT infrastructure in response to Mr. Kundra’s IT modernization plan, CIOs are also feeling pressure (and criticism) from users about their ability to support seemingly everyday applications.

I’m sure there are many reasons why these aren’t supported; my intention is not to address them here.  Instead I’d like to posit an opinion.  There are  tools supporting new paradigms in how we communicate, at work and home, which are becoming the norm.  They are no longer ‘toys’ for the younger crowd.  For those waiting for this social media fad to pass, I’d suggest another look.  Recent events in the Middle East are real illustrations of their significant impact.  The U.S. State Department is beginning to use a nation’s access to internet and social media as a barometer for civil liberties.  Yet, there is still much resistance in adoption or more fundamentally understanding and embracing these venues. 

I’m encountering resistance and sometimes denial of these forces on a daily basis.  I don’t think you need to have 500 Facebook friends or Tweet until thumbs bleed.  Nor do I think being a LinkedIn Pro replaces human interaction.  It is important though to recognize the shift in communication.  Take just a few minutes to understand the different tools.   You’ll find it’s not unlike a conversation you may have in a coffee room or a hallway if you still have a coffee room or a hallway.  Many don’t.  For those who are time, money or resource constrained, you can access industry experts, thought leaders and practical advice on your own time at no cost.  There are plenty of free introductory videos or articles on line.  You can decide which mediums work best for you. 

At PRICE, we are raising discussions, sharing product capabilities and answering questions about cost estimation models through our social media.  Because people have different preferences for certain outlets or certain restrictions at work, we repeat similar material across our social media networks.  You can check us out on Facebook, LinkedIn, Twitter or follow our blog.

True accuracy can only be determined after the project or effort has been completed and a post-audit analysis reconciles what was expected to happen with what did happen.  This is a very expensive, time consuming process that many preach about but few actually attempt. 

In my experience, when people ask about accuracy what they are really interested in is uncertainty.  They want a quantification of any uncertainty in the facts and processes that derived the estimate.  Because naturally, the more uncertainty the less accurate the estimate.   I am using Dale Shermon’s definition of uncertainty as explained in the book, Systems Cost Engineering. Uncertainty means ‘a possible event, the probability and the consequences of which are unknown’.   

Uncertainty about system or project scope, requirements stability, cost or effort drivers, the assumptions, and the range of possible answers all contribute to the individual’s assessment of how much doubt to apply to a response.

In an effort to describe the accuracy of estimates, the industry had added entire disciplines designed to generate all comforting statistics around someone’s approximate judgment.  Risk analysis, confidence levels, technology readiness levels are all valid and useful guides to help the consumer of the estimate understand the estimators assessment of their own accuracy…uncertainty.

What techniques or methods have you found useful to convey your estimate’s accuracy/uncertainty?

Have you experience the same relationship between Accuracy and Uncertainty?