I’m on my way home from the ISPA/SCEA (International Society of Parametric Analysts, Society of Cost Estimating and Analysis) Conference held in Albuquerque this week. Attendance was very good (2nd best in the conferences history) and as the content seemed especially good this week. I attended lots of good talks on topics ranging from SEPM (System Engineering, Project Management) cost estimating, Joint Confidence Levels, Software Estimating, Affordability, Agile software development and estimating for Enterprise Resource Planning Systems. Of course, just because the topics are good and well presented doesn’t mean I have to agree with everything that gets said.
One particular talk entitled “Function Points, One Size Fits All” got me a little worked up. It seems as though there is an on-going and completely unnecessary amount of energy devoted by some to convince the software community that we need to settle on a single method of measurement for software. The author makes some really good, credible points. Function Points are a much better method of measuring software if your intent is to compare productivity across or within specific industries. They help eliminate development technologies, programmer inconsistencies and software architecture decisions from an analysis of how productive an organization is at delivering business value to their customers.
Having said this, I firmly believe that sometimes SLOC (Source Lines of Code) are a better tool to help organizations estimate the costs of the software development projects they are planning. Knowing how your productivity stacks against the industry is great information when you are looking for process improvement or identifying best practices. But if you want to know how much the next project is going to costor how many months it will be before you can deliver content, your own history is much more relevant than the industries. Not that your history can’t be in Function Points – it just doesn’t have to be. Many of the weaknesses sighted with SLOC – while they certainly can be considered weaknesses) -aren’t really noticeable when working within a certain context. Within an organization, software development projects are often similar to previous projects, targeted at similar markets and are being developed by similar teams with similar (on average) coding styles. And SLOC Counting processes can be institutionalized within an organization. When this is the case, SLOC counts are just as good, if not better, than Function Point Counts and probably much easier to count since their count can be automated and does not require certified specialists.
The author contends that SLOC is a bad option because it is hard to estimate early on. I contend the same is true with Function Points - which if the Counting Practices Manual is followed to the letter – one is required to make an assessment of not only all transaction but also all the data element types and record element types. The author contends that SLOC is a bad option because there is no standard for what a SLOC is. I contend that the same can be said for Function Points. Perusal of the International Software Benchmark Standards Groups’ (ISBSG) database shows that there are several different ‘standards’ for counting Function Points – because one standard was not deemed sufficient by many in the community to meet all software measurement needs.
Don’t get me wrong – I do not disagree with many of the assertions the author made, only the notion that one size fits all. Let’s be real – software measurement is not easy and there are many different ways to approach it depending on the reasons for the measurement and the circumstances around those needs. Function Points are a good tool to have in your software measurement tool box, so are Source Lines of Code, Use Case Points and user Stories. It’s true that if all you have is a hammer every problem is a nail. Fortunately we have many different tools and should feel empowered to choose the right tool depending on the current need.
TruePlanning’s Role in Should-Cost Management
I was recently asked by a client to provide a synopsis of what TruePlanning offers in response to the Ashton Carter Memorandum – Implementation of Will-Cost and Should-Cost Management. In the memo, the Undersecretary of Defense AT&L listed “Selected Ingredients of Should Cost Management”. It was interesting to note how much capability is provided by TruePlanning to effectively support efficient should cost management. In this month’s blog, I will share with my response to our client with you.
Selected Ingredients of Should Cost Mgmt (Ashton Carter Memorandum) | TruePlanning “Should Cost” Capability |
Scrutinize each contributing ingredient of program cost and justify it… | Can model size, technology and schedule parameters and understand the interaction of each on cost. |
Benchmark against similar DoD programs and commercial analogues… | Benchmarking using cost research knowledge databases based on both military and commercial programs. Can also include specific program history. |
Promote Supply Chain Management to encourage competition and incentivize cost performance at lower tiers. | Can model the entire supply chain to analyze and understand impact of competition and cost incentives. |
Identify opportunity to breakout GFE vs. prime contractor-produced items | Contains models for both GFE vs. CFE allowing including estimating new development vs. modification |
Identify items or services contracted through a second or third party vehicle. Eliminate unnecessary pass-through costs by considering other contracting options. | Ability to model different vendor scenarios using True Planning's System Level Cost Model. |
Identify an alternative technology/material that can potentially reduce development or life cycle costs for a program. Ensure the prime product contract includes the development of this technology/material at the right time. | Robust capability to quickly select alternative technologies/materials and quantify impact on lifecycle costs. |
Cloud Nine - Are We There Yet?
In 1961 at the MIT Centennial, John McCarthy opined “if computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility…. the computer utility could become the basis of a new and important industry” [1]. In 2006, Amazon Web Services was launched providing computing on a utility basis. Since that time the notion of cloud computing has been emerging and evolving.
Cloud computing is a paradigm that makes the notion of utility computing a reality. Instead of Information Technology (IT) organizations investing in all of the hardware, software and infrastructure necessary to meet their business needs, cloud computing makes access to hardware, software and infrastructure available through the internet, generally utilizing a pay for use model. Basically cloud computing allows an organization to adopt a different economic model for meeting IT needs by reducing capital investments and increasing operational investments, a model which is likely to offer cost savings to many organizations.
There is still a great deal of hype around cloud computing, as many vendors have their marketing engines further into the clouds then their technology supports. Despite this Gartner predicts that by 2012 one in five businesses will not own its own IT assets. [2]. In late 2010 the Office of Management and Budget (OMB) under direction from the White House told federal agencies that starting in 2012 they are expected to consider cloud first “whenever a secure, reliable, cost-effective cloud option exists.” [3]
There are certainly many reasons why an organization would consider moving at least some of their IT functions into the cloud. In addition to potential cost savings the cloud offers the possibility of increased availability, easier collaboration, lower capital costs, scalability and virtualization. There are of course concerns as well. The technology is still relatively immature with no definitive set of standards for interface or compliance with regulations. Businesses lose hands on control of their IT resources with little recourse if their IT vendor shuts down or goes out of business. Additionally, there are security and data privacy concerns. There is also the fact that not all ventures into the cloud will be cost effective for the business.
This paper introduces the concept of cloud computing and discusses the potential benefits for a business as well as those things which could be barriers to adoption. It examines the types of applications where cloud computing is an efficient cost effective solution and the types of applications where its use could be problematic or costly. Several examples of successful cloud implementations are presented and discussed.
For a white paper describing this reearch on cloud computing email info@pricesystems.com with the code word CLOUD in the subject line. To share your cloud computing experiences comment on this post
The cobbler's kids...
...wear the worst shoes. The cobbler was a master at his craft; he was just too tired to practice it when he got home from the shop. Sound familiar?
A disciplined approach to understanding (functional) requirements as well as analogous projects (with actuals) is our not-so-secret sauce. Why run the risk of creeping back up our career learning curve? There’s already enough scope creep to keep us busy. Plus, for you management types charged with prospecting, a consistent approach towards estimation is a great way to connect with people who've felt the pain of being the cobbler's kids.
I recently reconnected with a colleague who found himself immersed in an early-stage, large-scale federal software project. Discussion points ranged from parametrics utility to sizing to Earned Value Management. Not leaving my toolset at the shop translated into good conversation with followup request. A bad example is yours truly, a year ago, so immersed in a highly-featured personal project, that he neglected to find agreement on scope and cost with his team. Bad idea! Now my “shoes” (a one-off custom hot-rod) are way behind schedule and way over budget. Keeping on my professional cost estimation consulting hat would have saved on both… bigtime!
Data Driven
My previous blog discussed a “Should Cost” methodology used by PRICE Systems to complete an analysis. In the article I included a chart depicting calibration results for manufacturing complexities for each weapon system (X-Axis). Manufacturing complexities are a major cost driver within the model. This parameter can be derived from model knowledge tables, generators or from calibration. Many times the calibrated results are simply averaged and used for predicting cost for the new system. This assumes that the new system is very similar in technology and performance as the systems used for calibration. In general this is not the case. Below I discuss how data driven cost estimating relationships (CER) can be developed between the calibration results and a selected independent variable. The CER will then be used to derive the manufacturing complexity, which then matches much closer to the new technology.

Figure 1. Calibration Results for each Weapon System
Since the data was already collected and calibrated within the model, the next step was to identify an independent variable. For manufacturing complexity for structure, thrust was selected. However, other variables could have been selected. Maximum take-off weight, range, ceiling, empty weight, thrust per pound, or speed were all candidates for an independent variable. For manufacturing complexity for Electronics, frequency speed was selected as an independent variable. Again there are many potential candidates for an independent variable for electronics.
The next step was to graph and complete regression analysis. Figure 2, depicts the results for manufacturing complexity for structure, and figure 3, depicts the results for manufacturing complexity for electronics. I normally select a power function when regressing data. One could select any function except linear. Linear regression does not always work very well and as a practical matter technology advancements are rarely linear.

Figure 2. Manufacturing Complexity for Structure

Figure 3. Manufacturing Complexity for Electronics
Using the above equations now require information be collected on the aircraft thrust requirement for structure, and frequency speed for electronics. These requirements are then used to calculate the appropriate manufacturing complexity. This approach insures that the cost estimate now match up with the requirements.
The next time you need to complete a cost analysis, talk to one of our expert PRICE Solutions consultants for help.
DODCAS 2011 - An Excellent Training Experience for Cost Professionals
Last year the conference occurred shortly after Congress passed the Weapons System Acquisition Reform Act of 2009 (WSARA) - and the majority of the sessions were focused on discussions about how the services, contractors and the government leadership planned on dealing with this new law. From the agenda, it looks like this year there will be much discussion of how these plans were executed; what worked and what didn’t. Topics range from will cost and should cost analysis, cost efficiencies, earned value management, quality data, affordability analysis, and other topics of interest to the acquisition community.
There will be an address by the Honorable Christine Fox, Director of the OSD Cost Assessment and Program Evaluation office (OSD CAPE) as well as a session with the Cost Directors of each of the Services. Shortly after her appointment as CAPE Director, Ms. Fox addressed the attendees of DODCAS 2010 with her plans going forward. She described the US Budget process and illustrated where the CAPE will fit in that process. She discussed WSARA and its intended benefits and the challenges it presents to the CAPE and the cost community as a whole. And finally she addressed the issues associated with acquiring, training and retaining the necessary human resources to support the CAPE and the objectives of WSARA. I found her talk interesting, informative and hugely relevant to what I do every day. I’m looking forward to her address this year and learning about CAPE successes, challenges and lessons learned.
Have you attended DODCAS recently? What cost estimating and analysis conferences are the staples in your training budget?
Additional Thoughts on Will Cost/Should Cost
Last week I gave a webinar which detailed the PRICE perspective on Should Cost & Will Cost Management. The responses I have received have been very positive and also informing. For those of you who could not attend you can view the recorded version of that webinar here.
Below is a brief summation of that presenation and some key takeaways.
The Under Secretary of Defense issued a memo late last year. The thrust of the memo was the current need for greater efficiency and productivity in defense spending. His guidance contained 23 principal actions for improving the efficiency of the department organized into five initiatives: (1) Target Affordability and Control Cost Growth; (2) Incentivize Productivity and Innovation in Industry; (3) Promote Real Competition; (4) Improve Tradecraft in Services Acquisition; and (5) Reduce Non-Productive Processes and Bureaucracy.
One of the principals outlined in the first initiative is the idea of driving productivity growth through Will Cost / Should Cost management. In my view, Should Cost is "the realm of the possible" and Will Cost is "the domain of the probable". Simply put Will Cost is a forecast of a program’s cost …” based upon reasonable extrapolations from historical experience.”[1] Whereas, Should Cost is an analysis of what the program ought to cost given concerted efforts in economy and efficiency by the contractor. “A should cost analysis results in an, …” Approximation of a contract-price, developed by the customer’s accounting, engineering, procurement, and other costing staff.” [2]
Should Cost analysis for DoD was first applied by the US Air Force in the early 1960s. Over time this practice matured to where in 1972 it was declared “A Multimillion-Dollar Savings” in an article written by Major David N. Burt in the Air University Review (September-October 1972). Major Burt describes the evolution of the practice commencing in a thorough discussion of a “new” alternative approach. He goes on to describe a five phase approach spanning one to four or more months consisting of a very large team which includes actual on-site (contractor) investigation of contractor practices and operations. There are issues associated with both of these concepts.
First, a Will Cost estimate is a business as usual view which contains all of the characteristics of both good and bad production and management practices. Estimates based on Will Cost have the effect of perpetuating previous inefficiencies by providing a flawed benchmark. Secondly, a Should Cost analysis as described by Major Burt is both time consuming and costly to implement. Furthermore, Secretary Carter, in his November 3, 2010 memorandum to Military Departments and Directors of Defense Agencies declared his desire …”to establish “Should Cost” targets as management tools for all ACAT I programs… and to establish by January 1, 2011 …”’Should Cost’ estimates for ACAT II and III programs… for component MS decisions.”
Clearly Secretary Carter is serious in regards to achieving productivity growth and soon. However, given the timelines, the cost of a traditional “Should Cost” analysis, and the problems associated with a “Will Cost” approach, how can the Military Departments and Defense Agencies meet the “beyond objectives” outlined in the November 3rd memorandum?
One solution may be utilizing a parametric approach. A parametric estimating approach will reduce the time and resources required while also providing an external benchmark of industry standards for similar systems. Given the pace with which agencies are expected to move its probably an approach many should consider.
Bob Koury
Senior Research Analyst, PRICE Systems
Will Cost/Should Cost Webinar
Today, PRICE Systems, Senior Research Analyst, Bob Koury, will be presenting on Will Cost/Should Cost management.
The presentation will focus on two main requirements mandated in the Ash Carter memo (mentioned here several times): Developing Should Cost/Will Cost targets and establishing Affordability as a requirement. An example will be provided of how parametric estimating models were used to establish “Should Cost” targets and how they can be used by a budget authority (government or Industry) to be an informed consumer of contractor or sub-contractor bids. The demonstration portion of this webinar will focus on a process for using a parametric estimating model to produce the reasonable affordability targets.
To sign up for the webinar go here: https://www2gotomeeting.com/register/142598427
Ash Carter Memo Follow Up
In last month’s blog I wrote about Ash Carter’s (Under Secretary of Defense for Acquisition, Technology & Logistics) Memorandum for Acquisition Professionals, Better Buying Power: Guidance for Obtaining Greater Efficiency and Productivity in Defense Spending (14 September 2010).
I concluded the TruePlanning unified framework and comprehensive cost models, is a tool very well suited to provide the types of analysis outlined in the memorandum. In terms of Should Cost and Independent Cost Estimates (ICE), TruePlanning estimation software provides the industry standard capability to conduct Should Cost and calibration (actual program history) for ICE. Most interestingly and to the point of the Carter memo, TruePlanning has the capability of breaking the “self fulfilling” prophesy of business-as-usual.
On November 3rd, Ash Carter issued a follow up memorandum outlining specific actions to implement the September 14th Guidance. For this month’s post, I wanted to point out some specific comments about Milestone A and B actions and specifically how TruePlanning can help.
At Milestone A the memorandum specifics establishment of an “affordability target to be treated by the Program Manager (PM) like a Key Performance Parameter (KPP)…This analysis should show results of capability excursions around expected design performance points to highlight elements that can be used to establish cost and schedule trade pace.
At Milestone B, the memorandum requires ”a system engineering tradeoff analysis showing how cost varies as the major design parameters and time to complete are traded off against each other.” Thus, Integration between parametric cost models and performance models is needed to allow full exploration and optimization of the trade space, treating cost as another design parameter
Did you know that TruePlanning through its integration with Phoenix ModelCenter is a powerful tool to provide exactly this type of analysis? ModelCenter contains a native plug-in to interface with TruePlanning directly. Through the ModelCenter interface, analysis can easily perform Design of Experiments (DOE), Carpet Plot Analysis (trading performance KPP’s against cost) and Optimization Analysis. If you are interested in seeing some examples, especially system engineering trade off analysis, please get in contact with me and I will send you a few presentations.
Zach JasnoffSolutions Architect, PRICE Systems
25+ Years of Software Estimating
But I digress; I really wanted to share my thoughts about the lessons learned, challenges and opportunities in the software estimation world. I think the most significant lesson learned (in my 25 years at least) is that the more things change, the more they stay the same. We continue to experience project failures. We continue to let requirements grow and blame the software folks. We continue to plan based on unproven impacts of silver bullets. Mostly, we continue to disregard what history should be teaching us. And if we don’t start to take our history seriously, things will be the same 25 years from now.
I think the biggest challenge that estimators face centers not around the mathematical or scientific but rather around issues of credibility, acceptance and successful negotiations. Many project failures are the result of poor negotiations between the business and the project leadership. Reasons for this include: failure to successfully communicate the complexity of the project, unrealistic schedule expectations forced on the project team, over-optimistic predictions of the project team. And (in case I haven’t said this before) failure for the business and the project to learn from history and negotiate geometrically using the project management triangle.
OK – so far I am pretty much repeating the sentiments of a fellow panelist (Dan Ligett) “We’re doomed!” Dan, of course, did not stop there but offered some encouragement going forward. I will attempt to do the same. You might have noticed that I believe firmly that much of the problems in the software industry are rooted in our inability to learn from and/or believe the lessons that history has taught us. Data really will set us free. The opportunity is for the estimation community to collaborate and cooperate in order to provide paths for historical data to usefully and credibly inform future actions. We need to look for ways to facilitate the sharing of data and models that will help the community at large make it possible to grow the practice of well informed, thoughtful geometrical decisions based on well developed, history informed estimates.
Back to the Future: Should Cost and Parametric Estimating Models
I was recently struck by Ash Carter’s (Under Secretary of Defense for Acquisition, Technology & Logistics) Memorandum for Acquisition Professionals, Better Buying Power: Guidance for Obtaining Greater Efficiency and Productivity in Defense Spending (14 September 2010). Within this broad sweeping memo, Ash Carter outlines 23 principal actions in five major areas aimed at increasing efficiency in Defense acquisition. The first major area covered is “Target Affordability and Control Cost Growth”.
Within this major area, program managers must treat affordability as a requirement before milestone authority is granted to proceed (starting with Milestone A). This means developing affordability targets and treating cost as a Key Performance Parameter.
What I find really interesting is the critique of Will Cost vs. Should Cost. The memo is critical of Will Cost, or Independent Cost Estimates (ICE). As Dr. Carter points out, “the ICE reflecting business-as-usual management in past programs, becomes a self fulfilling prophesy. The forecast budget is expected, even required, to be fully obligated and expended.” To combat this “vicious cycle”, the memorandum now requires “each major program to conduct a Should Cost analysis justifying each element of program cost and showing how it is improving year by year or meeting other relevant benchmarks for value.”
In my career experience, parametric estimating models such as TruePlanning play a major role in targeting affordability and controlling cost growth. In terms of affordability analysis, TruePlanning contains a unified framework of all elements of program cost (hardware, software, IT and Systems) and has built-in capacity to interface with engineering optimization tools such as Model Center. Through this interface cost can be treated as a Key Performance Parameter (KPP) for optimization and engineering trade off analysis.
In terms of Should Cost and ICE, TruePlanning provides the industry standard capability to conduct Should Cost and calibrated (actual program history) for ICE. Most interestingly and to the point of the Carter memo, TruePlanning has the capability of breaking the “self fulfilling” prophesy of business-as-usual. Using a calibrated TruePlanning model for ICE, estimators can change key engineering and programmatic parameters and see the impact on cost. For example, parameters such as requirements stability, engineering complexity and team composition can be quickly changed to assess a new program’s reality while still taking into account past performance history.
As the Carter memorandum points out … “the ability to understand and control future cost from a program’s inception is critical to achieving affordability requirements.” Because of TruePlanning unified framework and comprehensive cost models, it is a tool very well suited to provide the types of analysis outlined in the memorandum.
Zach Jasnoff
Solutions Architect, PRICE Systems
Basic Needs for Successful Software Estimation
You need 3 things for your software estimates to be successful
And I will add a fourth one in after I talk about the first 3
1. You need qualified and experienced people to generate the estimates. They have to know how to estimate and they have to understand what the problem is that the project is going to solve…..at least well enough to estimate it. This can be one person or many depending on the difficulty of the business area. The harder it is, the better having more brains look at the problem. But not to the point where it can slow you down. A team of 2 to 5 people can be faster and more efficient that a team of 8 to 12 and it’s easier to reach consensus.
2. You need your own data as a reference point in the estimate. As a comparison or an analogy. Your own data makes selling and explaining the estimate easier. It provides context for it that can enable management to give a quicker and more reasonable answer.
3. The estimate must be used as part of the decision making process. If it’s not used its wasted time and effort. When things get tight the estimation will go away.
4. Automated project estimating software tools to speed up the software cost estimation process and to enable some more what if analysis can be indispensible,
David Seaver, Solutions Architect, PRICE Systems
Laws of Analysis
I have been fortunate in my career to have been associated with some great mentors. Each individual has provided me a bit of a golden nugget to carry with me as I tried to navigate my way through the professional waters. My first “civilian” manager, after I left the service and joined industry, provided me a list of the Laws of Analysis (I had just started a position as an operations research analyst). He explained that this list was a mix of serious and tongue in cheek snippets of wisdom. I looked at the list and thought … “He must be kidding, followed by, Wow this is not politically correct!” But what I realized over time was even the farcical statements were a bit of reverse truth. I have hung this list in every office I have occupied since it was given to me. I provide it here for your consideration and entertainment. Hopefully you too will recognize a golden nugget or two:
1. Never B.S. yourself
2. Never use a round number – even though only one significant figure counts
3. Never tell a small lie
4. Answers are always more credible if printed on paper with small holes on the sides
(I know this is a throw-back to early mainframe computers and printers – so I am old look beyond it)
5. When briefing always take someone along to share the blame
6. Always remind yourself it’s a science
7. Everything takes twice as long as it should plus a month
8. The farther a system is from reality the more cost effective it is
9. Large committees produce only large confusion
10. If more than one person is responsible for a miscalculation no one will be at fault
11. If someone isn’t mad at you, you’re not trying hard enough or been on vacation
12. Never enough time to do it right – always enough time to do it over
After having just worked with a client on estimating software costs for a major program, I found #9 to be especially relevant.
Bob Koury
Senior Cost Research Analyst, PRICE Systems
Estimating Costs Associated w/Porting Existing Software in TruePlanning
Over the past several weeks several users have inquired about the best way to estimate costs associated with porting existing software to a new hardware environment. Normally for this situation some of the existing software will require some amount of adaptation to operate on a new server. However, a large portion of the existing software will only require integration into the new environment.
Estimating software costs associated with the above will require the use of several cost objects:
- Systems cost object if program management, Quality Assurance, configuration, and documentation costs are to be included in the estimate
- Assembly cost object to estimate integration costs
- Software component cost object to estimate software adaptation costs
- Software COTS cost object to model all existing software requiring only integration effort
The main input parameter that should be reviewed with care in the system cost object is project complexity. The default value for this input is nominal. Recommend the low value be reviewed since the team size will be small.
The main input parameter that should be reviewed with care in the assembly cost object is system complexity. If the software is just be ported to the new environment, recommend a low value be selected, since the requirements definition system design has been completed previously.
The software component input parameters will be treated in a normal fashion. All adapted software will be modeled as adapted new design, code and test.
The main input parameters that should be reviewed with care in the software COTS cost object are software size, integration team maturity, and external integration. This cost object should be used for all software that will only be integrated in the new environment. Functional size should be used if the size is unknown. The input parameters for integration maturity and external integration must be reduced. The recommended value is 0.05, which is the lowest value accepted by the model. This value will account for much lower integration activity for the existing software.
If you have any further questions, please contact a PRICE Systems Support.
Solutions Architect, PRICE Systems
Economics in TruePlanning
One of the great features of the TruePlanning cost management software is the fact that it makes it easy to handle complications of inflation and estimating projects performed in different countries and currencies. The costs associated with doing work in different countries, and the relative value of different currencies is constantly changing. To address this, the cost research team at PRICE does an annual economic update performed by the cost research team, and this blog will introduce some of basic concepts and research that goes into maintaining this feature every year.
The price of goods and services changes over time, and this value is measured by inflation rates (if prices have gone up) or deflation rates (if prices have gone down). These inflation/deflation rates are constantly updated for many different currencies by the International Monetary Fund (IMF). In TruePlanning, we use past and predicted inflation rates published by the IMF to determine the cost in today’s currency of doing work in the future, for whatever currency you want to use. Within the TruePlanning tool, these factors can be applied to help produce a cost estimate of work that takes place over a long period of time, with the cost estimate shown in today’s currency values.
In addition to inflation/deflation rates for different currencies, we also recognize that the price of goods and services may be different from one country to the next (i.e. a U.S. Dollar exchanged and spent in India will buy more haircuts than a dollar spent in the United States). The degree of these price differences can be measured by looking at Purchasing Power Parity (PPP) data of a country. Standard PPP values are calculated or compiled and published by the Organization for Economic Co-operation and Development (OECD). The PPP rates published by the OECD are used in the TruePlanning tool to estimate costs of performing projects in different countries, as the cost of doing the same amount of work in different countries can vary significantly.
By automating the application of these and other economic complications of cost estimating, the TruePlanning tool helps create a process in which the effects of these economic factors can be applied consistently and correctly for all your cost estimates.
Gurney Thompson
Cost Research Analyst, PRICE Systems
Cost Realism, Truth or Consequences
Last month I blogged about the importance of cost realism, its roots and how as estimators we must always reflect the truth, no matter how unpopular. This month I want to share with you a recent experience on a Source Selection. As part of the Source Selection team, my role was to conduct a Cost Realism estimate on each of the performers submitting bids. I want to share with you a few insights from that experience.
One of the first rules I always follow is to never ask engineers to provide data that you can readily find. I rarely will use parametric data forms, as you will get to be unpopular fast with those you task with the honor of filling them out. Rather, I dig into the technical and cost volumes to derive the configuration, technology, weight statement, rates/overheads and other juicy information to populate TruePlanning.
Once I have the TruePlanning cost management software populated with all of the data I can derive from existing documents, I will then ask the subject matter experts in each area to discuss the other qualitative/quantitative factors about each performer. I can usually guide this conversation to derive inputs such as requirements stability, engineering complexity, integration and others. I find it effective to hold this meeting remotely using Go-To-Meeting so everyone can see my desktop displaying the TruePlanning model and no one has to leave their desk. When inputs are critical cost drivers, I usually pop up the wizards or generators so the engineers can see the choices available.
Once the model is fully populated and all inputs agreed to, I will then produce a very well documented estimate with all assumptions as the final deliverable. In this particular case the Source Selection team was very pleased with the result and asked if we could extrapolate a new configuration based on some technical changes proposed by the performer.
Using our well qualified TruePlanning model, we ran the extrapolation and reported to the Source Selection team that we expect to see a large impact. Initially, no one believed the results; it could not be so they said! However, several weeks later, when the performer’s new estimate arrived, we were within 1% of the revised bid. During this time period, the Source Selection team was able to successfully prepare a negotiating stance based on the expected bid coming in as predicted.
In all my years as an estimator, I have only seen estimates go up as more is known about the technical configuration. Bottom line, as estimators we are the “front line” for telling management the hard to hear truth. In this case, at least they were prepared!
Zach Jasnoff
Solutions Architect, PRICE Systems
The more you estimate, the smarter it gets...
From my perspective as a cost researcher, the calibration tool is one of the most powerful analysis capabilities built into the TruePlanning cost management software . One way I can use this tool is to go back to an old estimate for a project that is now completed, and analyze the correctness of the previously entered input values. With this analysis, I can find ways to improve our methods of soliciting input values from the user to ensure the best values are entered the next time. This way, the TruePlanning models keep getting “smarter” as new information becomes available.
TruePlanning models calculate a cost estimate based on a set of inputs about the project being estimated. After the project is completed and we have actual cost data available to us, we can go back to TruePlanning and explore the various reasons why an estimated cost may have been different from the actual cost.
Let’s say we are estimating a project involving a brand new electronic technology, and we are uncertain of the values we selected to describe its complexity. The calibration tool allows us to enter the actual costs of the project, and reverse-engineer the value of an input we were uncertain about (i.e. manufacturing complexity for electronics) that would lead to the correct answer. This allows us to identify incorrect input values so we can identify problems, then review and refine our methods used to pick the value in the first place.
Over the years, we have used this (and many other) TruePlanning analysis capabilities to further our understanding of our cost estimates and our cost estimating methods. There are many analysis tools built into TruePlanning, and many ways to slice and dice information about a cost estimate. This makes it quick and easy to standardize the cost estimation methods used, and continually review and refine our methods to get the most accurate cost estimate possible. With more analysis tools and more information becoming available all the time, TruePlanning will keep getting smarter.
Gurney ThompsonCost Research Analyst, PRICE Systems
Do you kick the tires on your software estimation tools?
I have been frequently asked to do crosschecks on other people’s software cost estimates which are potentially done in a variety of tools from spreadsheets to SLIM.
One of the common operator errors I see from other users is not understanding what activities and resources are included in the outputs of the particular tool that they are estimating with.
This is akin to deciding between two cars and not knowing if both come with the same sets of features (stereo, AC, heated seats). With software estimation tools you need to know what work is getting estimated by the tool. (Requirements, Design, Code, Test, Documentation, System Test) and what resources are included in the effort estimates to accomplish those task/activities. (Programmers, Analysts, Architects, Systems Engineers, Testers, Tech Writers, Project Managers, QA and CM)
A common mistake I see is generating an estimate with a tool then adding in Program Management effort and cost when the tool utilized already includes those activities and resources in the estimate.
That’s why I drive a pearl black TruePlanning 2 door coupe.
David SeaverSolutions Architect, PRICE Systems
The Power of Parametrics
Let’s start with a simple test. Which is greater: the number of six-letter English words that have "n" as the fifth letter or the number of six-letter words ending in "ing"?
If you are like most people you’re thinking the correct answer is six-letter words ending in "ing". But most people are wrong. And the reason is simple, people rely on what they can easily recall. Since it’s much easier to think of 6-letter words ending in "ing" the fact that people come to that conclusion isn't suprising. Psychologists refer to this as availability bias. This type of bias gives more importance to things that we can vividly remember and easily recall. All too often, bias like this plays a prominent a role in the development of estimates. (By the way, the group of six-letter words that have "n" as the fifth letter include all of the six-letter words ending in "ing")
Part of the power of parametric cost estimating is the ability to substantially diminish this bias and to fill in where data is lacking. A parametric approach uses sophisticated mathematical equations along with historical data from similar systems or projects in combination with existing organizational data to produce an estimate that is external to the types of biases that often plague estimates. Once organizations are estimating software, hardware or systems of systems using a parametric model, the accuracy of the model will increase as data is captured from other projects. This systematic collection of actuals compared to estimates over time leads to data driven decision making.
Parametric cost estimating is a widely used approach for bidding on a contract, input into a cost benefit analysis, or as the pre-planning tool for project implementation. Extensive literature reviews suggest that an effective parametric cost estimating methodology is becoming an essential tool for technology-driven organizations. The use of parametric estimating in budgeting, scheduling, and control of projects will enhance the ability of project management organizations to effectively and efficiently utilize valuable resources. The benefit of parametric cost estimating tools is its use as an estimating model for better determining potential resource requirements during the project pre-planning and conceptual phase. When software cost estimation is performed using a parametric approach with proven commercial framework, the benefits realized far outweigh the cost of doing the estimating.
Almighty Estimation
I have to say that my foray into blogging has been an interesting one. By definition, the Chief Scientist should be a nerdy sort of geek too high brow to pontificate on topics in such a pedestrian format. Actually I kind of like it. In part because I enjoy writing and I'm not picky about what I write - technical documents are OK but pontification works as well. And in part because I know that in order to be a good writer in a particular genre one must read extensively from that genre. In other words I now have a good excuse to surf the web for related blogs and have found great ideas that have fueled my imagination
Today I want to share an article I found on 'Quips On Software Development' by David Longstreet called "Ancient Wisdom for Software Estimating". In it Longstreet traces the need for good estmating all the way back to the Bible. From Luke 14:25-33 Jesus says to the crowd of discipleship "Which of you wishing to construct a tower does not first sit down and calculate the cost to see if there is enough for its completion? Otherwise after laying the foundation and finding himself unable to finish the work the onlookers should laugh at him and say, 'This one began to build but did not have the resources to finish'"
Of course this was posted on April 1st so my first thought was of practical jokes. After thinking about how cool it would be to get this kind of powerful buy-in on the importance of estimating I figured I would check it out. So I powered up another browser and went to Bible.com (who knew!). Sure enough it was the real deal.
So what's my point here? Regardless of your religious beliefs - it seems prophetic that a document representing life and times more than 2000 years ago recognizes the importance of estimation and resource management. And further acknowledges the fact that people will notice if you say you're going to do something and then don't finish it because you've run out of time or money. Even before there were machines or factories, computers or software; before the words process improvement had ever been uttered - people understood that estimation was important when embarking on a project.
The next time you think about starting a project with a WAG (Wild A** Guess) or with the number your boss wants to hear - first stop and ask yourself... "What Would Jesus Do?"