IT Project Planning

While teaching an introductory TruePlanning for Software Estimating course this week at an Army location, I was asked to follow up with a clarification on “percent adapted” calculation. 

The official PRICE training materials definitions are:

• Percent of Design Adapted - the percentage of the existing (adapted code) design that must change to enable the adapted code to function and meet the software project requirements;

• Percent of Code Adapted - the percentage of the adapted code that must change to enable the adapted code to function and meet the software project requirements.

The former, Design, was received by the class as straightforward. Again, this adaptation includes architectural design changes, detailed design changes, and any necessary reverse engineering. Entered as a percentage of the Adapted Code size, this metric essentially captures the effort to modify/ enhance the underlying concepts for data and transaction algorithms.

As my class later reviewed a fellow-student’s modeling of a “real-world” example, we found that our jury was hung on the verdict of what’s really meant by adapting the latter, Code. Again, the instruction is to also enter this metric as a percentage of the pre-determined adapted code size.

But the legitimate question was: “Why isn’t it always 100%?” If we labored to already identify new code, deleted code and reused code… then isn’t any remaining modified code all adapted?

The answer, per our resident software Yoda, is consider the code in terms of modules, not just source lines. To answer what percentage of a target must change to meet new functional and project requirements, evaluate what sub-modules/ CSCIs must change and what are reused WITHIN each module. If an entire module is not modified, then it’s count is reflected in the percentage of reuse. But if module(s) have some components that are modified and some that are not, that information is captured in this latter percentage of code adaptation.

If you’ve followed my blogs, you know what’s coming next: another car example! 

Say I have an automobile to rebuild. The body, wheels & tires are fine. Even the suspension requires no modification. (No car guy would ever accept not spending time & money on handling too, but suspend your disbelief for now).

Only the motor needs “adaptation”… but also, only in certain areas. The ignition, fuel injection, water pump, A/C & exhaust all require little to no adaptation. But the cylinder heads, valves, camshafts, rods, pistons & crank all must be worked on. My percentage of motor component “code” that actually needs modification is clearly less than 100%. 

Of course, the changes made above are a function of what requirements we're trying to acheive, e.g., horsepower, torque, RPM, etc.  Likewise, this analogy holds true in typical software modification. What we are given, as previous deliverables, is more than just total lines of code. It’s functioning modules/ systems that may, or may not, require some re-work under the hood. 

Agree?

Additional insight into how TruePlanning defines and utilzes these inputs is detailed in the True Software White Paper.

At the 2011 ISPA Conference, I conducted a ½ day workshop How To Develop Data-Driven Cost Estimating Relationships  in TruePlanning. The attendees at the workshop learned how to import their own data into TruePlanning and develop custom Cost Estimating Relationships. We covered three case studies:

·         In the UCAS case study we demonstrated how we can build CERs at a higher level to provide a test of reasonableness to the CAPE.

·         In the SRDR case study we demonstrated how we develop a CER to estimate SLOC based on historical data and use the results to within the new code parameter of the UCAS Reconnaissance software estimate. 

·         In the Aircraft case study we demonstrated how we can build higher level trend lines across various aircraft system platforms and classes of aircraft to compare as “ball-park” estimates.

In each case, the attendees learned how to work with real data and develop CERs traceable back to the original data points that generated them. This was a “hands-on” workshop and everyone had a chance at generating a number of CERs. Overall, we had a very successful session and provided a new and exciting capability to our TruePlanning framework.

I also wanted to share with you a few takeaways from this workshop.

• By setting up the trend lines equations in the Equation Cost Object, estimators can create libraries of data-driven CERs for future projects
  • All data used to develop the equation can be attached to the CER to identify datapoints and methodology used.
• The power of this approach is that the CERs are integrated into the TruePlanning framework and can be used with other cost objects (H/W, SW, IT, etc)
• Libraries of CERs can be shared among projects.
• CERs are “white box” and auditable both in terms of the trend line equation and underlying data points.
• CERs represent an “estimating database” in addition to TruePlanning Cost Objects.

If you are interested in obtaining a copy of the workshop presentation including the data files and results, please e-mail me at Zachary.Jasnoff@pricesystems.com .

I’m on my way home from the ISPA/SCEA (International Society of Parametric Analysts, Society of Cost Estimating and Analysis) Conference held in Albuquerque this week.  Attendance was very good (2nd best in the conferences history) and as the content seemed especially good this week.  I attended lots of good talks on topics ranging from SEPM (System Engineering, Project Management) cost estimating, Joint Confidence Levels, Software Estimating, Affordability,  Agile software development and estimating for Enterprise Resource Planning Systems.   Of course, just because the topics are good and well presented doesn’t mean I have to agree with everything that gets said.

One particular talk entitled “Function Points, One Size Fits All” got me a little worked up.  It seems as though there is an on-going and completely unnecessary amount of energy devoted by some to convince the software community that we need to settle on a single method of measurement for software.  The author makes some really good, credible points.  Function Points are a much better method of measuring software if your intent is to compare productivity across or within specific industries.  They help eliminate development technologies, programmer inconsistencies and software architecture decisions from an analysis of how productive an organization is at delivering business value to their customers.

Having said this, I firmly believe that sometimes SLOC (Source Lines of Code) are a better tool to help organizations estimate the costs of the software development projects they are planning.  Knowing how your productivity stacks against the industry is great information when you are looking for process improvement or identifying best practices.  But if you want to know how much the next project is going to costor how many months it will  be before you can deliver content, your own history is much more relevant than the industries.  Not that your history can’t be in Function Points – it just doesn’t have to be.  Many of the weaknesses sighted with SLOC – while they certainly can be considered weaknesses) -aren’t really noticeable when working within a certain context.  Within an organization, software development projects are often similar to previous projects, targeted at similar markets and are being developed by similar teams with similar (on average) coding styles. And SLOC Counting processes can be institutionalized within an organization. When this is the case, SLOC counts are just as good, if not better, than Function Point Counts and probably much easier to count since their count can be automated and does not require certified specialists.

The author contends that SLOC is a bad option because it is hard to estimate early on.  I contend the same is true with Function Points - which if the Counting Practices Manual is followed to the letter – one is required to make an assessment of not only all transaction but also all the data element types and record element types.  The author contends that SLOC is a bad option because there is no standard for what a SLOC is.  I contend that the same can be said for Function Points.  Perusal of the International Software Benchmark Standards Groups’ (ISBSG) database shows that there are several different ‘standards’ for counting Function Points – because one standard was not deemed sufficient by many in the community to meet all software measurement needs.

Don’t get me wrong – I do not disagree with many of the assertions the author made, only the notion that one size fits all.  Let’s be real – software measurement is not easy and there are many different ways to approach it depending on the reasons for the measurement and the circumstances around those needs.  Function Points are a good tool to have in your software measurement tool box, so are Source Lines of Code, Use Case Points and user Stories.  It’s true that if all you have is a hammer every problem is a nail.  Fortunately we have many different tools and should feel empowered to choose the right tool depending on the current need.

Going to ISPA SCEA in New Mexico?  If so, join us for a workshop on data driven cost estimating.  Register: http://www.pricesystems.com/services/predictive_modeling_schedule.asp#/?i=3

Description:  Building transparency and traceability into your estimating process leads to more defendable estimates.  This hands-on workshop demonstrates how historical data is transformed into predictive models.   You will learn how your organization’s data can be synthesized into custom models that can be employed in support of third party models within a single analytical framework. 

Participants will learn:  (1)   To develop system level estimating relationships to provide a test of reasonableness and historical cross-check to proposed estimates. (2) To develop sub-system level relationships that estimate Software Lines of Code based on requirements. (3)  To develop historically-based relationships across classes of aircraft systems in support of “ball-park” estimates based on specific program actuals.

Location:  2011 ISPA/SCEA Joint Annual Conference /  Hyatt Regency Albuquerque - Fiesta 1-2 /  June 7^th, 2011 /  9:00 am – 1:00 pm (includes complimentary lunch).  Hope to see you there!  To register go to: http://www.pricesystems.com/services/predictive_modeling_schedule.asp#/?i=3

I was recently asked by a client to provide a synopsis of what TruePlanning offers in response to the Ashton Carter Memorandum – Implementation of Will-Cost and Should-Cost Management. In the memo, the Undersecretary of Defense AT&L listed “Selected Ingredients of Should Cost Management”. It was interesting to note how much capability is provided by TruePlanning to effectively support efficient should cost management. In this month’s blog, I will share with my response to our client with you.

Agile software development practices are predicated on the following tenets as introduced in 2001 in the Agile Manifesto [1]

•  Individuals and interactions over processes and tools
•  Working software over comprehensive documentation
•  Customer collaboration over contract negotiation
•  Responding to change over following a plan

Agile development processes rely on experienced, highly skilled people communicating with clients and each other to deliver software that provides the clients with the most value for the money they spend.  This requires both developers and consumers of software to accept the reality that things will change over the course of the project and that the software that is eventually delivered may not be the same as that was envisioned when the project was first kicked off.

At any given time, the agile software development team is only working on the feature that the customer currently feels is the most valuable.  Estimation is performed by the team (including developers, BAs, QAs and customer representatives) and is only focused on the feature that is currently on deck.  At the end of an iteration, the customer has an opportunity to review the implementation to date and can reprioritize remaining features based on changes in their requirements, market or expectations.  So estimating beyond the current feature doesn’t really make agile sense.

Unfortunately, the fact that estimation doesn’t make sense for the agile team does not mean it doesn’t make sense for the business.  The business needs to see the forest not just the trees.  Customers need an idea when their software will be delivered, businesses need to prep the market for new products and features, businesses need to be able to optimize resource allocations across many projects, etc.   This of course begs the question of how to perform an estimate for software when you really don’t know exactly what software you’re going to build.  As an industry, we’ve been pretty unsuccessful estimating software projects even when we think we have a handle on the end product – how can we be successful with so much uncertainty.

First we accept uncertainty and commit to incorporating uncertainty into our estimates.  Once we’ve made that commitment we are then free to pursue one of many top level estimating techniques to help us get our head around a likely range of values for cost and schedule based on what we currently know about the software project.  Parametric techniques are particularly suitable for this task especially for an organization that has some historical data from previous agile projects.  Parametric estimating models like TruePlanning for Software provide a repeatable framework through which an organization can study their past performances on similar projects (similar number of features, similar market, similar customer, etc) and use what they learn to perform estimates on the project at hand.   The amount and nature of the similarities should guide the amount of uncertainty in the estimate.  Organizational history can be used to map Story Points or User Stories to a size measurement such as source lines of code, function points or user points.  Analysis of performance on past projects can inform decisions about productivity and other project drivers.  This information can be used to drive the parametric algorithms to develop estimates for cost, effort and schedule.

Since we have already acknowledged that we didn’t get the question right, it’s unreasonable to assume that the answer will be ‘the right answer’.  What we do come away with is a range for cost, effort and schedule which will give decision makers realistic data to work with. If we have properly studied our history and thoughtfully assessed uncertainty, we can present these ranges with a quantifiable degree of certainty. 

What process does your organization employ to plan around agile induced uncertainties?

[1] Agile Alliance, “Agile Software Development Manifesto”, Feb 2001, available at www.agilemanifesto.org (retrieved January 2010)

The following is a collection of some of the more common installation issues.  If you have additional questions, concerns, or issues with any upgrade or installation, please call our Technical Support line anytime at:  1-800-43-PRICE.

Default Password Override

TP’s default password may not be strong enough based on your security policy.  Please review your specific requirements for passwords and follow the directions below to change the default:  When you get to the screen, uncheck the “Use Default Password” and enter a valid password based upon your specific security settings.  Then click next and continue the installation process.

Existing TruePlanning Installations

If TruePlanning 2008 SR2 is currently installed, it is recommended that users export their projects, then uninstall TruePlanning, making sure to backup their database when prompted to do so during the uninstall. The uninstall of TruePlanning can be initiated from the Control Panel.  Also, please select the option to uninstall MSDE when prompted during the uninstall.  Once this is complete, install the new version of TruePlanning, making sure to manually import the projects which were exported from the older release once the installation is complete.  Upgrades from TruePlanning 2009 and above do not require TruePlanning  to be uninstalled as the TruePlanning 2010 installation will automatically upgrade the projects in the database; however, it is recommended that users still export their projects manually and before the upgrade to be safe. 

Administrative Rights

The user installing TruePlanning requires administrator rights on the machine.  On Vista it is advisable to launch the installation of TruePlanning by navigating the “Setup.exe” file on the TruePlanning CD and initiating the installation using the “Run as Administrator” option. This option is available from the right click menu in Windows Explorer (My Computer). 

XmlLite

XmlLite is a resource provided by Windows to perform certain XML parsing abilities. It is a prerequisite for installing TruePlanning 2010. If TruePlanning 2010 is installed on a computer that does not have XmlLite, and error will occur when TruePlanning is launched.  The installation will not produce any errors or warnings. This error is Windows XP specific. Vista and Windows 7 contain the XmlLite natively. Normally XmlLite is installed when Internet Explorer 7.0 (or newer) or Windows XP SP3 are installed. Installing either of these resources should resolve the error.  Some customers are not able to install IE 7.0 or Windows XP SP3 and should therefore install XmlLite directly. It is available from Microsoft at: http://support.microsoft.com/kb/915865. It should be noted that TruePlanning 2010 has features that rely on Internet Explorer 7.0 or higher. Some of the Calculators contained in TruePlanning will not function properly without Internet Explorer 7.0 or higher.  Therefore, if possible, upgrading Internet Explorer is the preferred option. 

MSXML6 SP2

There is a known issue installing TruePlanning 2010 on computers that have Windows XP SP3 and MSXML6.0 SP2 installed.  The issue occurs because of an incompatibility between SQL Server 2005 Express and MSXML6.0 SP2. This issue is described by the following Microsoft Knowledge Base article: http://support.microsoft.com/kb/968749. MSXML6.0 SP2 is installed by the Windows XP Service Pack 3. This is not an issue on Vista.  The above MS KB article provides a fix for the issue. It is advisable to execute the fix before attempting to install TruePlanning 2010. It is possible to identify computers that have had MSXML6.0 SP2 installed by examining the “Add/Remove Programs” feature under the Control Panel. If the issue in encountered during the installation of TruePlanning it can be identified by the failure of SQL Server to install. The failure will be identified as pertaining to MSXML6. This can be ascertained after the installation has been exited, by examining the log of the SQL Server installation. The summary of the SQL Server 2005 Express installation can be found at the location defined by the following support page: http://msdn.microsoft.com/en-us/library/ms143702%28SQL.90%29.aspx. If the error has been encountered please contact PRICE System’s Support for help resolving the issue.

In 1961 at the MIT Centennial, John McCarthy opined “if computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility…. the computer utility could become the basis of a new and important industry”  [1].  In 2006, Amazon Web Services was launched providing computing on a utility basis.  Since that time the notion of cloud computing has been emerging and evolving.

Cloud computing is a paradigm that makes the notion of utility computing a reality.  Instead of Information Technology (IT) organizations investing in all of the hardware, software and infrastructure necessary to meet their business needs, cloud computing makes access to hardware, software and infrastructure available through the internet, generally utilizing a pay for use model.  Basically cloud computing allows an organization to adopt a different economic model for meeting IT needs by reducing capital investments and increasing operational investments, a model which is likely to offer cost savings to many organizations.

There is still a great deal of hype around cloud computing, as many vendors have their marketing engines further into the clouds then their technology supports.  Despite this Gartner predicts that by 2012 one in five businesses will not own its own IT assets. [2].  In late 2010 the Office of Management and Budget (OMB) under direction from the White House told federal agencies that starting in 2012 they are expected to consider cloud first “whenever a secure, reliable, cost-effective cloud option exists.” [3]

There are certainly many reasons why an organization would consider moving at least some of their IT functions into the cloud.  In addition to potential cost savings the cloud offers the possibility of increased availability, easier collaboration, lower capital costs, scalability and virtualization.  There are of course concerns as well.  The technology is still relatively immature with no definitive set of standards for interface or compliance with regulations.  Businesses lose hands on control of their IT resources with little recourse if their IT vendor shuts down or goes out of business.  Additionally,  there are security and data privacy concerns.  There is also the fact that not all ventures into the cloud will be cost effective for the business.

This paper introduces the concept of cloud computing and discusses the potential benefits for a business as well as those things which could be barriers to adoption.  It examines the types of applications where cloud computing is an efficient cost effective solution and the types of applications where its use could be problematic or costly.  Several examples of successful cloud implementations are presented and discussed.

For a white paper describing this reearch on cloud computing email info@pricesystems.com with the code word CLOUD in the subject line.  To share your cloud computing experiences comment on this post

“I think we have an obligation to work with industry to ensure that our suppliers do not just remain world class in defence, but aspire to be world-class manufactures that can withstand comparison to other industries.”

Chief of Defence Procurement, Sir Robert Walmsley

Is this a practical proposition or is it a pipe dream?  The following excerpt from Dale Shermon’s Systems Cost Engineering attempts to make the case that this type of comparison is possible.


Many of the statements in proposals and marketing literature stating the superiority of a company are anecdotal or at best qualitative evidence. It has not been possible to quantify the productivity of industry to demonstrate that one manufacturer is more efficient than another.

In the ship building industry it is possible to benchmark commercial ship yards using a metric called Compensated Gross Tonnage (CGT). CGT measures the complexity of building a ship. It is calculated by multiplying the gross tonnage by the relevant CGT coefficient (calculated by the Organisation for Economic and Commercial Development [OECD]).  However, this is only applicable to this one industry. Other metrics exist in other industries, but what is really required is a single benchmarking technique and measure.

Benchmarking with Cost Normalization.  Forty years ago, PRICE Systems developed a metric that can be used to normalize cost and support the benchmarking practices needed to reach Sir Walmsley vision.  Manufacturing Complexity represents the normalized cost density of an item that has been manufactured. It has two dimensions: technology and productivity. Hence, more than one Manufacturing Complexity can exist. As aircraft are made using different technologies and by different companies, these have various complexity figures. However, if the technology is constant, the only dimension that changes is productivity. As a result, it is possible to compare the efficiency of companies that produce similar technologies.

Supplier Assessment.  One obvious and practical application of this technique is Supplier Assessment. This technique exploits the capability of parametric models to compare organization’s normalized cost density or Manufacturing Complexity. If, on paper, you have very similar proposals, how can you judge which organization will provide the best value for money? Heritage data will determine the level of development that is required and Operating and Support Cost will establish the Through Life Cost, but if these cannot differentiate the proposals perhaps the productivity of the companies should be considered. This is not a simple comparison, but with manufacturing complexity this can be achieved easily.

This process will lead to your suppliers knowing that you are able to control them; they will need more control on themselves. It will enable you to perfect your negotiation skill, leading to more success, as better negotiation material is utilised based on technology cost drivers. A parametric questionnaire is a possible tool to enable proposal validation in minutes.  All this information can be garnered at a very early stage in a project if required, providing better estimating accuracy for the purchased items. Normalization enables a bigger data sample as previous proposals received from the same companies can be compared. This in turn enables the establishing of trends. Are the suppliers becoming more or less efficient? Ultimately, these comparisons can be used to determine preferred suppliers.

Conclusion.  This method of benchmarking a project or company can be used for all environments. With the ability to move across environments it is possible to compare commercial and military organisations equally. Decisions and judgments can be made between Commercial and Military suppliers where no military competition exists.  It is possible to assess the productivity of an organisation or country at System, sub-system and equipment levels. The technique can be used to compare the whole cost or just the labour cost if that is the focus of your efficiency drive.

It is possible to determine preferred suppliers on a justified basis, or alternatively it is possible to demonstrate the efficiency of your organisation when the customer is skeptical about your productivity or your drive to improve productivity.

Finally, many organisations already have a license to use the PRICE models and over 8,000 people worldwide have been trained and are familiar with the PRICE methodology. This is not a new idea, PRICE has been established since 1975, but this new application of an established technique can help decision maker take the right decision regarding source selection cross industry and national companies. Alternatively, it can enable industry to demonstrate, with proof, that they are more effective than other solutions.

1. A round table peer review, where we describe and defend our parametric modeling approach as well as consistency across similar objects/ systems/ alternatives. 
2. Introduction of an independent 3^rd party, not part of the estimating team, who is an experienced modeler. This resource can take the client perspective in reviewing, say, procurement versus cost-of-ownership as well as comparables from similar program experiences.  
3. Take an agile approach where your project leader is the “product owner” who reviews all interim deliverables top-down.  
4. Another option is incorporating a client representative into your pre-release reviews for similar validation testing. 
5. Finally, we pair off internally to verifying each other’s work against the real-time documentation we keep per common templates.  (Available from PRICE Systems, for both hardware and software modeling inputs). They are great visualization tools for internal parameter-tracking, plus external pre/ post model review with customers.

...wear the worst shoes. The cobbler was a master at his craft; he was just too tired to practice it when he got home from the shop.  Sound familiar?

A disciplined approach to understanding (functional) requirements as well as analogous projects (with actuals) is our not-so-secret sauce. Why run the risk of creeping back up our career learning curve? There’s already enough scope creep to keep us busy. Plus, for you management types charged with prospecting, a consistent approach towards estimation is a great way to connect with people who've felt the pain of being the cobbler's kids.

I recently reconnected with a colleague who found himself immersed in an early-stage, large-scale federal software project. Discussion points ranged from parametrics utility to sizing to Earned Value Management. Not leaving my toolset at the shop translated into good conversation with followup request. A bad example is yours truly, a year ago, so immersed in a highly-featured personal project, that he neglected to find agreement on scope and cost with his team. Bad idea! Now my “shoes” (a one-off custom hot-rod) are way behind schedule and way over budget. Keeping on my professional cost estimation consulting hat would have saved on both… bigtime! 

True Planning results have many options, including viewing Costs by Activity. While simple, this view can be quite powerful, especially when exported for re-organization manipulation. 

In a recent exercise, the WBS mapping of common objects, estimated by separate multiple scenarios, presented a non-trivial chore in Excel. “Transposition” features work fine for matrices, as do pivot tables. But how does one map object by activity grids into activity lists, similar to MIL-STD 881a, with singular “roll up” instances of all nonzero object costs? 

The secret is in how True Planning appends each activity output with the [object] tag. Once exported (via “Send to Excel” or “Copy Grid” options), a simple Data-Sort by Activity name will, by default, alphabetically group activities. From there, either then group by phase and/or group by scenario. The latter can be the original organization or an entirely new one. Next, use Excel’s Data-Group option to build subtotaling and roll up controls. This also gives you the ability to hide/view object instances, which are essentially activity subtotals. Finally, at your discretion, shred any scenario-shared costs such as SE/PM or personnel. Of course, build checksums to cross-check. 

Last piece of advice on this: for each step in your approach, document process refinements and save your interim artifacts as templates. The next time you are asked to rework the steps with a new scenario set or organization, your life will be even easier.

In my blog last week on Work Breakdown Structures, we reviewed the subtleties of using the [object] tag to your advantage in creating different sorts and roll up subtotals. As a followup, I’d like to drill down a bit on the initial step of using the “copy grid” exports. Each row number is unique, thus creating an identifying key for the vlookup function in Excel. Since all object X activity instances are allocated 100% to one of the three phases (with very rare exception), these row keys allow you to sort and re-group outputs while maintaining lookup visibility to corresponding grids. In the case of that rare exception, percentage spreads easily solve for allocation of that shared activity cost. 

The real trick to transpose into a WBS "Activity then Object" listing is to use the [object] tags as group delimiters. As long as you keep the row-key associated with the cost outputs, your options are not limited to just one activity list order. You could map to 881A format or to your accounting system. You could map your PBS transposition to 1921 or 1921-1 orders. In the end, cost will still match TruePlanning's output. Grouping by phase or by system won't change the bottom line.

One note: If you need activity subtotals immediately, True Planning’s Activity by Cost Object output works just fine. It’s a grid that is available for export to Excel, Word or XML. My process above solves for the often-demanded request of listing (and numbering, indenting, subtotaling) linearly. In the near future, look for PRICE Systems to utilize the COM layer feature of TP 2010 SR1 to give you yet another option in mapping PBS to WBS format. 

I consistently run into this idea of data driven estimating.  Yet, there is no clear explanation of this concept.  I am not trying to provide one here, however, I am interested in is what is at the root of this growing movement.  My take is that it is an attempt to scratch an itch.  But what’s the itch?

I believe it is related to my early post (Accuracy is Risky Business).  In the struggle to answer the accuracy question people have decided that understanding the data used in the estimating process is key to understanding its accuracy.  To a certain degree this makes sense.  It is useful to gain insight into the information upon which a cost estimation model was based.

• How much data was available?
• How current was the data?
• Was it relevant to the project being estimated?
• What statistical techniques were employed to analyze the data?

In terms of data, there are two general areas considered relevant: data quality and fit. Understanding how much data was used; how old it was; where the data came from sheds additional light on the results. Once the amount, age and source of the information is established, the next concern is how the information was analyzed. 

Descriptive statistics are often used to describe a collection of data in quantitative terms. They aim to quantitatively summarize a data set. Descriptive statistics include many measures including central tendency (mean, median, mode), dispersion (standard deviation, range, quartile) and association (correlation). Additionally, there is statistical inference, which makes propositions about populations, using data drawn from the population of interest via some form of random sampling. 

The industry is fraught with estimates backed by reams of data surrounded by all the necessary statistics. But does understanding descriptive statistics and statistical inference alone improve our understanding and confidence of an estimate?  If so, why then is expert opinion one of the most widely used forms of estimating?  Why do programs based on these data driven estimates continue to perform poorly?  Is data driven the answer?  What do you think?

The concept of the fuel cell was first published in 1938 by Christian Friedrich Schonbein.  Based on this publication Sir William Grove invented the precursor of the fuel cell in 1839. The Grove Cell created current by applying two acids to zinc and platinum electrodes separated by a porous ceramic pot.  In 1842 Grove developed the first actual fuel cell which produced electricity with hydrogen and oxygen, much like many fuel cells in use today.

Fuel cells remained an intellectual curiosity until the 1960’s when the US space program identified a requirement for extended life batteries for which fuel cells seem to offer a promising solution.  The focus on green technologies has increased interest in consumer uses of fuel cells for transportation, residential and commercial power supply, emergency backup power and portable power supplies for consumer and battlefield applications.  Increased usage of any technology begs the question of how to address the costs associated with that technology. 

A fuel cell is an electrochemical cell which converts some fuel, usually hydrogen, into electric current.  It does this through a reaction between the fuel and an oxidant in the presence of an electrolyte.  The waste product of this chemical process is water and heat.  Fuel cells, unlike conventional batteries, consume reactant from an external source rather than one stored in the battery. They do require a continuous supply of fuel, but given that this supply is available, they will not run out of charge like a conventional battery. 

Because fuel cells require neither flame nor combustion to convert fuel to electricity, there is much hope that they will become a viable power source of the future as we try to reduce our carbon footprint.  Fuel cells are very reliable and less likely to be effected by the environment as some more conventional power delivery systems are.  Because of this they are being adopted in industries such as the telecommunications where outages are particularly problematic.  They are often considered for power generation in remote areas where energy from the grid is expensive and outages are frequent.  Because heat is a waste product of the fuel cell electricity generation process, micro combined heat and power systems are gaining popularity for residential and small business needs.  Other interesting uses of fuel cell power include material handling, backup power systems and uninterruptable power supplies.

Despite increases in the use of fuel cells, they continue to evade wide spread use because they are expensive.  Certainly significant progress has been made through increases in efficiency and improvements in manufacturing processes, but it is still more expensive, in most domains, to get electricity from fuel cells than from more conventional methods. According to a report from the Department of Energy in May 2010, high volume automotive fuel cell stack cost has been reduced from $275/KW in 2002 to $61/KW in 2009 and appear to be on track to reach the $30/KW goal by 2015.   The same report indicates a 24% increase in system power density for stationary fuel cells making it possible to reduce the fuel stack volume, weight and cost.

PRICE recently conducted a research effort using publically available data to develop cost estimating relationships for various types of power systems utilizing fuel cell technology. For a white paper describing this project and the resulting cost estimating models email info@pricesystems.com with the code word FUEL in the subject line.

As is often the case, there is a difference between definition and connotation. The intended meaning of consumerism at these events was the phenomena of the consumer goods infiltrating and influencing government IT policy.  The IPad was all too visible in the audience as were the comments from CIO staff about their user base wanting to integrate IPad-like products into the work domain.

Like it or not we are all consumers; and consumers are getting more tech savvy.  As the adoption of these products or services increase in our daily lives, the question becomes unavoidable – Why can’t I use these for work? It was clear that for the CIOs attending these conferences this question was top of mind.  While most of the U.S. government is attempting to identify, inventory, consolidate and remove redundancies of IT infrastructure in response to Mr. Kundra’s IT modernization plan, CIOs are also feeling pressure (and criticism) from users about their ability to support seemingly everyday applications.

I’m sure there are many reasons why these aren’t supported; my intention is not to address them here.  Instead I’d like to posit an opinion.  There are  tools supporting new paradigms in how we communicate, at work and home, which are becoming the norm.  They are no longer ‘toys’ for the younger crowd.  For those waiting for this social media fad to pass, I’d suggest another look.  Recent events in the Middle East are real illustrations of their significant impact.  The U.S. State Department is beginning to use a nation’s access to internet and social media as a barometer for civil liberties.  Yet, there is still much resistance in adoption or more fundamentally understanding and embracing these venues. 

I’m encountering resistance and sometimes denial of these forces on a daily basis.  I don’t think you need to have 500 Facebook friends or Tweet until thumbs bleed.  Nor do I think being a LinkedIn Pro replaces human interaction.  It is important though to recognize the shift in communication.  Take just a few minutes to understand the different tools.   You’ll find it’s not unlike a conversation you may have in a coffee room or a hallway if you still have a coffee room or a hallway.  Many don’t.  For those who are time, money or resource constrained, you can access industry experts, thought leaders and practical advice on your own time at no cost.  There are plenty of free introductory videos or articles on line.  You can decide which mediums work best for you. 

At PRICE, we are raising discussions, sharing product capabilities and answering questions about cost estimation models through our social media.  Because people have different preferences for certain outlets or certain restrictions at work, we repeat similar material across our social media networks.  You can check us out on Facebook, LinkedIn, Twitter or follow our blog.

PRICE Systems recently accepted an assignment to complete a "Should Cost" estimate for a U.S. ally on a weapon system. The estimate included not only analysis on production costs, but also should cost on various operations and support costs. The only information provided by the client was quantity and time frame for production. A major ground rule for the estimate was that all data specific to the weapon system must come from publicly available information.  For example, mass, manufacturing process, and learning curve information must come from the public domain. 

After reviewing the scope for the estimate, we decided to also collect data on four similar weapon systems. This would allow us to provide not only a “Should Cost” estimate, but additional information to support the estimate. 

The initial task was to collect data on each weapon system. Pertinent model inputs were collected from various public sources. Flyaway costs were collected for each weapon system and used as the cost input for calibration. Manufacturing complexity for structure and electronics were the key input values solved for via calibration. The calibration results are displayed in a line graph, depicted in Figure One below.

Figure One. Calibration Results for each Weapon System

We then translated all calibration input data and calibration results into a predictive input file. Estimates were completed for each weapon system. All results were reported in the form of a graph depicting average amortized production costs, average flying hour cost, and average operations and support costs for each weapon system. 

The assignment could not have been completed without the capability of the PRICE Models. Model calibration was a critical task. Only a few pieces of data were required: quantity produced, production schedule, and mass. 

The model’s capability to solve for values for manufacturing complexity and a graphical depiction of the results enabled the client to quickly understand the results even though some of their personnel were not familiar with the PRICE models. Do you have a question about should cost?  Post it and we'll do our best to answer your question.  Need help? Contact one of our experts for assistance.

PRICE Systems is currently developing a COM interface for TruePlanning. I know, I know…  What’s COM you say? COM stands for (Component Object Model) and it's a programmable interface which exposes the TruePlanning estimating brains for integration and analysis!  I know it sounds boring but it’s really cool because it allows anyone, including our users, to build “apps” for TruePlanning similar to the way “apps” are built for the iPhone.  Let me give you some examples of some apps that you can build:  Excel solution, sensitivity analysis,  project comparison, risk simulation, total cost of ownership (TOC) solution, calibrate multiple inputs, optimize maintenance concepts and more.  The apps can be developed using a host of tools including VBA which comes with Microsoft Office.  Expect some apps to be built by PRICE but anyone can do it.  If you have an idea for a creative TruePlanning “app” let us know about it because that will help us build the right COM interface for you.

True accuracy can only be determined after the project or effort has been completed and a post-audit analysis reconciles what was expected to happen with what did happen.  This is a very expensive, time consuming process that many preach about but few actually attempt. 

In my experience, when people ask about accuracy what they are really interested in is uncertainty.  They want a quantification of any uncertainty in the facts and processes that derived the estimate.  Because naturally, the more uncertainty the less accurate the estimate.   I am using Dale Shermon’s definition of uncertainty as explained in the book, Systems Cost Engineering. Uncertainty means ‘a possible event, the probability and the consequences of which are unknown’.   

Uncertainty about system or project scope, requirements stability, cost or effort drivers, the assumptions, and the range of possible answers all contribute to the individual’s assessment of how much doubt to apply to a response.

In an effort to describe the accuracy of estimates, the industry had added entire disciplines designed to generate all comforting statistics around someone’s approximate judgment.  Risk analysis, confidence levels, technology readiness levels are all valid and useful guides to help the consumer of the estimate understand the estimators assessment of their own accuracy…uncertainty.

What techniques or methods have you found useful to convey your estimate’s accuracy/uncertainty?

Have you experience the same relationship between Accuracy and Uncertainty?