Software Project Costs

My June blog entry suggested the use of parametrics in real-options valuation. This month, I’d like to offer the generalized use of our type of modeling in valuing tangible assets. 

Typically, fundamental analysis evaluates the intrinsic value of securities. I won’t attempt to compete with Warren Buffet here. But it is certainly the case that a company, or portfolio of securities reflecting many companies, is based in part on the market value of its product assets and their potential for future earnings, as well as other objective and subjective considerations.

In parametric estimation, we take a top-down approach to developing models of estimating ultimate acquisition and ownership costs. In the case of the former, I’d argue that we can readily… and uniquely… estimate operating costs for companies developing and selling, say, technology-based products.  Marketing can determine pricing, based on their competitive landscape analyses and market-demand for unmet needs. 

We are in an excellent position to complete the forecasting of earnings (and hence, discounted cash flow) using our parametric methods to evaluate a product-system’s should/will costs. Anyone who does detailed bottoms-up budgeting for “go to market” expense, as well as production delivery, would find value in our modeling.  

And again, once estimated costs are subtracted from revenue forecasts, the netted estimate of earnings and cash flows will allow for traditional market value assessment of this asset or entity. For example, your organization needs to compare economic potentials of emerging new software products. There are no balance sheets. There are no income statements. Yet. 

Now your sales and marketing function has a good independent sense of revenue opportunity. So it’s left to you to estimate the costs of developing and delivering. Independent parametric estimation is the answer. No idea on sizing, either SLOC or function points? No problem. 

Data-driven True Planning has years of data modeling that allow you to run application-specific calculators. In addition, you can take advantage of calibrating actual data from similar projects. Likewise, this process of valuing assets holds true for IT infrastructure products as well. Cloud computing, SAAS and ERP are all further examples of where new IT/software products are emerging… and need estimating as well as financing via valuation.

I think parametric estimating has significant value commercially in the estimation of value of future product assets, and consequently their companies and associated securities.  What do you think?

Last week I gave a webinar which detailed the PRICE perspective on Should Cost & Will Cost Management. The responses I have received have been very positive and also informing. For those of you who could not attend you can view the recorded version of that webinar here.

Below is a brief summation of that presenation and some key takeaways.

The Under Secretary of Defense issued a memo late last year. The thrust of the memo was the current need for greater efficiency and productivity in defense spending. His guidance contained 23 principal actions for improving the efficiency of the department organized into five initiatives: (1) Target Affordability and Control Cost Growth; (2) Incentivize Productivity and Innovation in Industry; (3) Promote Real Competition; (4) Improve Tradecraft in Services Acquisition; and (5) Reduce Non-Productive Processes and Bureaucracy.

One of the principals outlined in the first initiative is the idea of driving productivity growth through Will Cost / Should Cost management. In my view, Should Cost is "the realm of the possible" and Will Cost is "the domain of the probable". Simply put Will Cost is a forecast of a program’s cost …” based upon reasonable extrapolations from historical experience.”[1] Whereas, Should Cost is an analysis of what the program ought to cost given concerted efforts in economy and efficiency by the contractor. “A should cost analysis results in an, …” Approximation of a contract-price, developed by the customer’s accounting, engineering, procurement, and other costing staff.” [2]

Should Cost analysis for DoD was first applied by the US Air Force in the early 1960s. Over time this practice matured to where in 1972 it was declared “A Multimillion-Dollar Savings” in an article written by Major David N. Burt in the Air University Review (September-October 1972). Major Burt describes the evolution of the practice commencing in a thorough discussion of a “new” alternative approach. He goes on to describe a five phase approach spanning one to four or more months consisting of a very large team which includes actual on-site (contractor) investigation of contractor practices and operations. There are issues associated with both of these concepts.
 
First, a Will Cost estimate is a business as usual view which contains all of the characteristics of both good and bad production and management practices. Estimates based on Will Cost have the effect of perpetuating previous inefficiencies by providing a flawed benchmark. Secondly, a Should Cost analysis as described by Major Burt is both time consuming and costly to implement. Furthermore, Secretary Carter, in his November 3, 2010 memorandum to Military Departments and Directors of Defense Agencies declared his desire …”to establish “Should Cost” targets as management tools for all ACAT I programs… and to establish by January 1, 2011 …”’Should Cost’ estimates for ACAT II and III programs… for component MS decisions.”

Clearly Secretary Carter is serious in regards to achieving productivity growth and soon. However, given the timelines, the cost of a traditional “Should Cost” analysis, and the problems associated with a “Will Cost” approach,  how can the Military Departments and Defense Agencies meet the “beyond objectives” outlined in the November 3rd memorandum?

One solution may be utilizing a parametric approach. A parametric estimating approach will reduce the time and resources required while also providing an external benchmark of industry standards for similar systems. Given the pace with which agencies are expected to move its probably an approach many should consider.

Bob Koury
Senior Research Analyst, PRICE Systems

In last month’s blog I wrote about Ash Carter’s (Under Secretary of Defense for Acquisition, Technology & Logistics) Memorandum for Acquisition Professionals, Better Buying Power: Guidance for Obtaining Greater Efficiency and Productivity in Defense Spending (14 September 2010).

I concluded the TruePlanning unified framework and comprehensive cost models, is a tool very well suited to provide the types of analysis outlined in the memorandum. In terms of Should Cost and Independent Cost Estimates (ICE), TruePlanning estimation software provides the industry standard capability to conduct Should Cost and calibration (actual program history) for ICE. Most interestingly and to the point of the Carter memo, TruePlanning has the capability of breaking the “self fulfilling” prophesy of business-as-usual.

On November 3^rd, Ash Carter issued a follow up memorandum outlining specific actions to implement the September 14^th Guidance. For this month’s post, I wanted to point out some specific comments about Milestone A and B actions and specifically how TruePlanning can help.

At Milestone A the memorandum specifics establishment of an “affordability target to be treated by the Program Manager (PM) like a Key Performance Parameter (KPP)…This analysis should show results of capability excursions around expected design performance points to highlight elements that can be used to establish cost and schedule trade pace.

At Milestone B, the memorandum requires ”a system engineering tradeoff analysis showing how cost varies as the major design parameters and time to complete are traded off against each other.” Thus, Integration between parametric cost models and performance models is needed to allow full exploration and optimization of the trade space, treating cost as another design parameter

Did you know that TruePlanning through its integration with Phoenix ModelCenter is a powerful tool to provide exactly this type of analysis? ModelCenter contains a native plug-in to interface with TruePlanning directly. Through the ModelCenter interface, analysis can easily perform Design of Experiments (DOE), Carpet Plot Analysis (trading performance KPP’s against cost) and Optimization Analysis.  If you are interested in seeing some examples, especially system engineering trade off analysis, please get in contact with me and I will send you a few presentations.

I was recently struck by Ash Carter’s (Under Secretary of Defense for Acquisition, Technology & Logistics) Memorandum for Acquisition Professionals, Better Buying Power: Guidance for Obtaining Greater Efficiency and Productivity in Defense Spending (14 September 2010). Within this broad sweeping memo, Ash Carter outlines 23 principal actions in five major areas aimed at increasing efficiency in Defense acquisition.  The first major area covered is “Target Affordability and Control Cost Growth”.

Within this major area, program managers must treat affordability as a requirement before milestone authority is granted to proceed (starting with Milestone A). This means developing affordability targets and treating cost as a Key Performance Parameter. 

What I find really interesting is the critique of Will Cost vs. Should Cost. The memo is critical of Will Cost, or Independent Cost Estimates (ICE). As Dr. Carter points out, “the ICE reflecting business-as-usual management in past programs, becomes a self fulfilling prophesy. The forecast budget is expected, even required, to be fully obligated and expended.” To combat this “vicious cycle”, the memorandum now requires “each major program to conduct a Should Cost analysis justifying each element of program cost and showing how it is improving year by year or meeting other relevant benchmarks for value.”

In my career experience, parametric estimating models such as TruePlanning play a major role in targeting affordability and controlling cost growth. In terms of affordability analysis, TruePlanning contains a unified framework of all elements of program cost (hardware, software, IT and Systems) and has built-in capacity to interface with engineering optimization tools such as Model Center. Through this interface cost can be treated as a Key Performance Parameter (KPP) for optimization and engineering trade off analysis.

In terms of Should Cost and ICE, TruePlanning provides the industry standard capability to conduct Should Cost and calibrated (actual program history) for ICE. Most interestingly and to the point of the Carter memo, TruePlanning has the capability of breaking the “self fulfilling” prophesy of business-as-usual. Using a calibrated TruePlanning model for ICE, estimators can change key engineering and programmatic parameters and see the impact on cost. For example, parameters such as requirements stability, engineering complexity and team composition can be quickly changed to assess a new program’s reality while still taking into account past performance history.

As the Carter memorandum points out … “the ability to understand and control future cost from a program’s inception is critical to achieving affordability requirements.” Because of TruePlanning unified framework and comprehensive cost models, it is a tool very well suited to provide the types of analysis outlined in the memorandum.

Zach Jasnoff
Solutions Architect, PRICE Systems

After some recent meetings with clients I am sensing some confusion on how to estimate software reuse. I think part of the problem is in the definition of reuse, so let's start with a definition and then address the estimating issue. Software reuse is defined as “the use of existing software, or software knowledge, to build new software.” This definition came from Wikipedia. From a estimating software costs perspective the above definition is part of the problem. The definition should read: "Use of existing software with no changes for operation in the new software program.” 

If the existing software is going to be changed, re-written or modified to operate in the new program, it should be modeled as software adaptation. In this case the software will require some amount of re-testing, and software integration. Software that is being re-used has been fully tested during the original development program. In this case the reused software does not have to go through full up integration and testing. Rather it will go through a regression testing (computer aided testing) to insure it is still operating correctly.

If the software is going to be re-used as is, then clients should use the TruePlanning  “Software COTS” cost object to model and estimate. The purpose of modeling is for integration control. External integration should be set to a value of 0 to 1.0. A value of zero implies that there is no additional integration effort. Values less than one reduce the amount of model calculated software-software integration effort. The reused software can be modeled as a software component. However, do not include software adaptation or new software with the reuse data. This will overstate the integration effort. 

Some of us remember taking the Iowa tests during our early school days. The Iowa Tests of Basic Skills (ITBS) are standardized tests provided as a service to schools by the College of Education of The University of Iowa. The tests, administered to students in grades K-8, became a national standard for measuring scholastic aptitude – I was educated in Pennsylvania.

Now out of Iowa comes another test of sorts, something called an Integrity Index Score based upon a proprietary algorithm of an organization called Iowa Live. Iowa Live calls itself, “a growing network of volunteer Citizens & professionals for improving Iowa.” The organization has addressed a variety of issues in the education, health, and government areas. In July, Iowa Live published its comparison of two commercially marketed parametric estimating software products, one of which was PRICE TruePlanning. The comparison was for both estimation of software projects and estimating hardware. If you’d like to see the results, visit the Iowa Live website, where you can also learn more about the theory and derivation of the Integrity Index.  

Bruce Fad
VP Professional Services, PRICE Systems

Ahhhh, the 80s… a challenging (but often confusing) time in an evolving computing world.  Working in 1985 as a software estimator as well as SQA engineer in a quality assurance department that “audited” real-time projects using new concepts like OOD & OOP… well, you get the picture.  It was a great time to get immersed into great work.  And the good news:  that company’s process as well as its developers were bullish on a young estimation/ quality types asking plenty of questions… as long as they were of the Yes-No variety.  And ask I did.  Writing and using those checklists was great OJT, above & beyond adding their value of verifying/ enabling healthy development activities. 

Less than two years later, SEI’s Walt Humphrey formalized his Capability Maturity Model with its five levels of software process maturity.  These “CMM” levels are still used by many organizations and estimating tools, including within True Planning’s “Organizational Productivity Calculator.”  Could the CMM assessment be supplemented to include cost-driver questions for purposes of parametric estimation?  It worked in my SQA audit days. 

In fact many estimators (if only because they can’t get on an SME’s schedule twice) take the opportunity to ask qualitative and quantitative questions.  Integrating both lines into one interview should still be received as a healthy value-add.  Whether estimation is resident within a quality or financial function is moot.  We have the entrée to support both costing/ planning and productivity assessment.  A good example of an integrated approach is the David Group’s Project Profile Worksheet, cited in the text “Function Point Analysis” by David Garmus & David Herron.  Less comprehensive checklists are likely out there too, including methodologies proprietary to individual companies. 

The thought today is that the future is now and structured approaches, called audits or otherwise, will enable good practices as well as illuminate new approaches too.  In Wall Street, regulations are only good in controlling past (known) bad behaviors.  In just as creative Software, process rules help developers maximize their productivity towards current & future good behaviors.

The key cost driver when estimating software costs is the size of the product. The problem is that there is no perfect technique available to measure and quantify the size of software. The two major techniques in use today are Source Lines of Code and Function Points.  Today we will talk about Source Lines or Code or SLOC.

Source Lines of Code measures logical lines of code. It takes some of the uncertainty out of physical line of code measures by counting only complete statements (which can cross over more than on physical line). SLOC excludes comments and blank lines.

SLOC has several advantages:

·         Easily counted by automated tools, which can be configured with an organizations definition to produce consistent counts. Link to free tool from USC is provided below.

http://sunset.usc.edu/research/CODECOUNT/download/2010/UCC_Release_Notes_v.2010.07.pdf

·         Provide visibility into technical progress as projects proceed through design, code and testing.

·         Can be used to develop historical databases of past project size, which enable the more accurate prediction of size of future similar projects.

·         Design, Code and Test change to modified software can be specifically quantified

SLOC has several disadvantages:

·         It’s very difficult to predict SLOC on new programs prior to design

·         Different programming languages will require different amounts of SLOC to implement the same functions. This makes summary measures of size on multi-programming language products difficult

·         SLOC is difficult to map back to product features or requirements. Which makes usage of EVM difficult to apply to SLOC based SW projects. 

·         SLOC is a meaningless number to non Software Engineers (without detailed explanation)

Over the past several weeks several users have inquired about the best way to model legacy software that is being modified when estimating software costs. The software component within the TruePlanning Software model has an input parameter call “adapted Code Size.” This input parameter accounts for existing or legacy software that will be modified or changed to meet a new requirement. Tied with the size input parameter is Percent Design/Code/Test adapted. Although the model will calculate a percentage for each input, I would recommend that user’s analyze the calculate values and override the calculation where required. The percentage for each parameter is to reflect the amount of modification being accomplished. For example, calculated values below five percent or above forty percent are probably unrealistic. 

Values below five percent reflect one single function being modified, which is normally not the case. Values above forty percent generally result in the entire software package being re-developed. Hopefully this will clear up this software input parameter. If you have further questions, please email me at james.otte@pricesystems.com

Jim Otte
Solutions Architect, PRICE Systems

Recently I came across the word “off-label”.  It is the term used by the medical community when a drug is used to treat a condition for which it has not been approved by the Food and Drug Administration.   We sometimes use TruePlanning for “off-label” purposes. A good example would be using the TruePlanning Calibration tool to answer such questions as, what is the maximum number of source lines of code (SLOC) I can get and remain within my budget?  I call this TruePlanning Optimization. Here is an example answering the SLOC question.

First begin by performing a normal cost estimate for your software project. If the cost is under your budget and the project contains all the software functionality you would like, you are “home free”. If you have excess budget and would like to see how much more functionality you can add or if you are over budget and need to consider eliminating some functionality to stay within budget, then the following is for you.

Let’s say that the requirements call for 10,000 SLOC of C++ code. Using the TruePlanning Software and TruePlanning Systems catalogs, the total cost is $873,299, but our budget is only $750,000. How many SLOC can we purchase given our budget? This question can be answered quickly and easily using the TruePlanning Calibration tool. Open the TruePlanning Calibration tool and select a New calibration. On the Calibration Settings screen, for the Inputs Cost Selection, choose the Software Component cost object and New Code Size. For the Output Cost Selection, choose System Folder and Estimated Cost. For the Target Value, enter the $750,000 and adjust the Tolerance and Maximum Iterations.

Now, on the Calibrations screen, click on Calibrate and the answer appears as the Calibrated Input. As shown below, we can purchase 8,594 SLOC with our $750,000 budget. Next the big question is, what functionality can we safely eliminate in going from the 10,000 SLOC to the 8,594 SLOC?

As you can see, the medical folks are not the only ones who use things “off-label”.    The more you use TruePlanning, the more likely you are to discover new, creative uses for TruePlanning. Off-label is not a bad word or concept.

In May of this year the Washington Post published an editorial article on the need to reduce waste in the Defense Department. The byline of the article was “Defense Secretary Gates’s war of necessity against wasteful spending.” In this article the writer points out that the secretary is taking on the challenge of maintaining our military force [at reasonable level of effectiveness] during a time in which the President and Congress are seeking cost savings / reductions based on the decrease in our presence in Iraq.  Mr. Gates goal is to look for efficiencies in the tooth (combat) to tail (support) ratio. In an era in which industry and private business has flattened their organizations the Defense Department has grown its bureaucratic levels of staff. In some case the levels between the secretary of defense to the line soldier- sailor – airman has grown from 17 under Secretary Rumsfeld to as much as 30 under Secretary Gates.

What are the implications of this need upon the cost estimating community? Clearly there is a requirement to make choices between competing priorities. The defense budget has nearly doubled since September 11, 2001. However, this increase has not been mostly tied to combat forces and force generation but to non-military purposes. For example military Health-care costs have gone from $19 billion to $51 billion in the same time period now accounting for almost 10% of the entire defense budget. Given this situation the department of defense will need to justify every allocation of resource both old and new. PEOs, / PMs will be required to make a business case for their programs. However, unlike the cold war era, in which this case is made only against the need to create a combat capability, these initiatives will need to show value added over such things as family quality of life, MILCON, or pay increases. This prioritization between disparate goals will require a more robust set of data and costing tools. It will also require a well defined, effective, and agile Business Case Analysis capability.

You need 3 things for your software estimates to be successful

And I will add a fourth one in after I talk about the first 3

1.       You need qualified and experienced people to generate the estimates. They have to know how to estimate and they have to understand what the problem is that the project is going to solve…..at least well enough to estimate it. This can be one person or many depending on the difficulty of the business area. The harder it is, the better having more brains look at the problem. But not to the point where it can slow you down. A team of 2 to 5 people can be faster and more efficient that a team of 8 to 12 and it’s easier to reach consensus.

2.       You need your own data as a reference point in the estimate. As a comparison or an analogy.   Your own data makes selling and explaining the estimate easier. It provides context for it that can enable management to give a quicker and more reasonable answer. 

3.       The estimate must be used as part of the decision making process. If it’s not used its wasted time and effort. When things get tight the estimation will go away.

4.       Automated project estimating software tools to speed up the software cost estimation process and to enable some more what if analysis can be indispensible,

David Seaver,  Solutions Architect, PRICE Systems

I have been fortunate in my career to have been associated with some great mentors. Each individual has provided me a bit of a golden nugget to carry with me as I tried to navigate my way through the professional waters. My first “civilian” manager, after I left the service and joined industry, provided me a list of the Laws of Analysis (I had just started a position as an operations research analyst). He explained that this list was a mix of serious and tongue in cheek snippets of wisdom. I looked at the list and thought … “He must be kidding, followed by, Wow this is not politically correct!” But what I realized over time was even the farcical statements were a bit of reverse truth. I have hung this list in every office I have occupied since it was given to me. I provide it here for your consideration and entertainment. Hopefully you too will recognize a golden nugget or two:

1.       Never B.S. yourself

2.       Never use a round number – even though only one significant figure counts

3.       Never tell a small lie

4.       Answers are always more credible if printed on paper with small holes on the sides
(I know this is a throw-back to early mainframe computers and printers – so I am old look beyond it)

5.       When briefing always take someone along to share the blame

6.       Always remind yourself it’s a science

7.       Everything takes twice as long as it should plus a month

8.       The farther a system is from reality the more cost effective it is

9.       Large committees produce only large confusion

10.   If more than one person is responsible for a miscalculation no one will be at fault

11.   If someone isn’t mad at you, you’re not trying hard enough or been on vacation

12.   Never enough time to do it right – always enough time to do it over
 

I had expected to present my webinar,  “Best Practices for Cost Effectiveness Studies using TruePlanning” in early August. As you might know, I was planning to show a real world example from a recent engagement with a government customer. Unfortunately, since the Source Selection has not concluded with a downselect, I was not able to obtain the public release in time. However, for this month’s blog I will continue share some of the highlights of the webinar.

In last month’s blog we explored the uses of TruePlanning during Source Selection from the Supplier’s (or Contractor’s) perspective. This month, I would like to share with you some of the uses of TruePlanning cost estimating software from the Customer’s (or Government’s) perspective:

• Analysis of Alternatives (AoA) – Is proposed technical baseline cost-effective against other competing alternatives?
• Cost Realism – Are the performers bidding within an accurate range
• Data Driven Estimating – Are the performers bidding based on appropriate, traceable historical data points.
• Independent Cost Estimate (ICE) – Using the performer’s technical configuration, what does a completely independent look say about the performer’s bid?
• Risk Analysis – Is our bid over conservative, how much risk are we willing to take and how much cost exposure can we absorb?
  
• Schedule Estimating – Can we really do the job within the schedule constraints?
• Growth Estimating – What other configurations, materials or technologies might we consider?

While TruePlanning is useful in estimating project costs and all of the above types of analysis, I have found it most useful in developing AoAs. During Source Selection, TruePlanning provides a structured, repeatable framework that can rapidly develop cost estimates across a range of alternatives. These results are often incorporated directly into Operational Effectiveness models along with Measure of Effectiveness and Measures of Performance. In some cases, clients using tools such as Phoenix Model Center can directly link TruePlanning with optimization tools as well. 

I  recently moved and thought about the need to do a new household budget. This got me to thinking of the budgeting capability in TruePlanning. First, you can use TruePlanning to determine a budget. The time phased output, either monthly or annually, is ideal for establishing a budget and budget profile for your project. 

TruePlanning also splits the time phased costs into development, production, and operating and support categories. Be cautious, however, in using the Phase report. The System cost object costs are assigned by schedule duration, which may not necessarily reflect the actual project cost flow. A better choice may be the Activity Type report. Use the development and production costs from this report as calculated by TruePlanning and then split the System costs between development and production as appropriate for your project.    

Second, in the project Properties, you can enter your budget (if you already have one) and then compare the TruePlanning phased output to your budget profile. This will give you a heads up as to where you may have a budget shortfall or overage. This type of information well in advance can lead to a more successful and productive project. Happy budgeting!!!  

John Long
Solutions Consultant, PRICE Systems

Over the past several weeks several users have inquired about the best way to estimate costs associated with porting existing software to a new hardware environment. Normally for this situation some of the existing software will require some amount of adaptation to operate on a new server. However, a large portion of the existing software will only require integration into the new environment.  

Estimating software costs associated with the above will require the use of several cost objects:

- Systems cost object if program management, Quality Assurance, configuration, and    documentation costs are to be included in the estimate

- Assembly cost object to estimate integration costs

- Software component cost object to estimate software adaptation costs

- Software COTS cost object to model all existing software requiring only integration effort 

The main input parameter that should be reviewed with care in the system cost object is project complexity. The default value for this input is nominal. Recommend the low value be reviewed since the team size will be small.

The main input parameter that should be reviewed with care in the assembly cost object is system complexity. If the software is just be ported to the new environment, recommend a low value be selected, since the requirements definition system design has been completed previously. 

The software component input parameters will be treated in a normal fashion. All adapted software will be modeled as adapted new design, code and test. 

The main input parameters that should be reviewed with care in the software COTS cost object are software size, integration team maturity, and external integration. This cost object should be used for all software that will only be integrated in the new environment. Functional size should be used if the size is unknown. The input parameters for integration maturity and external integration must be reduced. The recommended value is 0.05, which is the lowest value accepted by the model. This value will account for much lower integration activity for the existing software.

If you have any further questions, please contact a PRICE Systems Support.