IT Project Management

I had expected to present my webinar,  “Best Practices for Cost Effectiveness Studies using TruePlanning” in early August. As you might know, I was planning to show a real world example from a recent engagement with a government customer. Unfortunately, since the Source Selection has not concluded with a downselect, I was not able to obtain the public release in time. However, for this month’s blog I will continue share some of the highlights of the webinar.

In last month’s blog we explored the uses of TruePlanning during Source Selection from the Supplier’s (or Contractor’s) perspective. This month, I would like to share with you some of the uses of TruePlanning cost estimating software from the Customer’s (or Government’s) perspective:

• Analysis of Alternatives (AoA) – Is proposed technical baseline cost-effective against other competing alternatives?
• Cost Realism – Are the performers bidding within an accurate range
• Data Driven Estimating – Are the performers bidding based on appropriate, traceable historical data points.
• Independent Cost Estimate (ICE) – Using the performer’s technical configuration, what does a completely independent look say about the performer’s bid?
• Risk Analysis – Is our bid over conservative, how much risk are we willing to take and how much cost exposure can we absorb?
  
• Schedule Estimating – Can we really do the job within the schedule constraints?
• Growth Estimating – What other configurations, materials or technologies might we consider?

While TruePlanning is useful in estimating project costs and all of the above types of analysis, I have found it most useful in developing AoAs. During Source Selection, TruePlanning provides a structured, repeatable framework that can rapidly develop cost estimates across a range of alternatives. These results are often incorporated directly into Operational Effectiveness models along with Measure of Effectiveness and Measures of Performance. In some cases, clients using tools such as Phoenix Model Center can directly link TruePlanning with optimization tools as well. 

I  recently moved and thought about the need to do a new household budget. This got me to thinking of the budgeting capability in TruePlanning. First, you can use TruePlanning to determine a budget. The time phased output, either monthly or annually, is ideal for establishing a budget and budget profile for your project. 

TruePlanning also splits the time phased costs into development, production, and operating and support categories. Be cautious, however, in using the Phase report. The System cost object costs are assigned by schedule duration, which may not necessarily reflect the actual project cost flow. A better choice may be the Activity Type report. Use the development and production costs from this report as calculated by TruePlanning and then split the System costs between development and production as appropriate for your project.    

Second, in the project Properties, you can enter your budget (if you already have one) and then compare the TruePlanning phased output to your budget profile. This will give you a heads up as to where you may have a budget shortfall or overage. This type of information well in advance can lead to a more successful and productive project. Happy budgeting!!!  

John Long
Solutions Consultant, PRICE Systems

To me the greatest strength of TruePlannning®  is its flexibility and wide range of parameters. AND, to me the greatest weakness of TruePlanning®  is its’ flexibility and wide range of parameters.

The nature of the TruePlanning® framework, the idea of cost catalogs with cost objects, and the implementation of activity based costing allows or provides for a wide range of solutions to our cost estimating requirements. TruePlanning’s ® ability to address a wide range of cost problems lies in its flexibility and extensibility. The clever use of worksheets, multipliers, and the product breakdown structure all assist the analyst in solving hard cost estimating problems.

Ahhhhh the beauty of an esoteric / elegant solution to a cost problem that uses the full range and power of TruePlanning®! Yes Yes … “I am the cost man made of steel which can leap tall cost problems in a single bound.” There is just one little problem with that self deluding thought. If no one else can understand what you did then how useful is your estimate? Fortunately TruePlanning®  has a great way to document what you have done so others can follow your methodology and process. Each input item has a note icon that can be opened which will provide a dialog box. In this dialog box there is a general description of the parameter but more importantly in regards to your own documentation there is a notes section and a file attachment section (see the figure below) These capabilities can be used by the analyst to fully document his or her approach to the value they selected or calulated for that particular parameter.

Bob Koury
Senior Cost Research Analyst, PRICE Systems 

Over the past several weeks several users have inquired about the best way to estimate costs associated with porting existing software to a new hardware environment. Normally for this situation some of the existing software will require some amount of adaptation to operate on a new server. However, a large portion of the existing software will only require integration into the new environment.  

Estimating software costs associated with the above will require the use of several cost objects:

- Systems cost object if program management, Quality Assurance, configuration, and    documentation costs are to be included in the estimate

- Assembly cost object to estimate integration costs

- Software component cost object to estimate software adaptation costs

- Software COTS cost object to model all existing software requiring only integration effort 

The main input parameter that should be reviewed with care in the system cost object is project complexity. The default value for this input is nominal. Recommend the low value be reviewed since the team size will be small.

The main input parameter that should be reviewed with care in the assembly cost object is system complexity. If the software is just be ported to the new environment, recommend a low value be selected, since the requirements definition system design has been completed previously. 

The software component input parameters will be treated in a normal fashion. All adapted software will be modeled as adapted new design, code and test. 

The main input parameters that should be reviewed with care in the software COTS cost object are software size, integration team maturity, and external integration. This cost object should be used for all software that will only be integrated in the new environment. Functional size should be used if the size is unknown. The input parameters for integration maturity and external integration must be reduced. The recommended value is 0.05, which is the lowest value accepted by the model. This value will account for much lower integration activity for the existing software.

If you have any further questions, please contact a PRICE Systems Support.

The late Norm Crosby’s “Quality is Free” taught us that an investment into quality is more than offset by prevention of defects based upon understanding of requirements. Only with the latter can lack of conformance (and subsequent costs) be captured and hence quality quantified. So how then is Parametrics relevant? 

Parametric estimating is more than cost modeling. Our craft represents an initial consulting function into the accuracy and completeness of program planning concepts. Our customers trust us to know when to ask and when to supplement. Yes, we are mathematical and financial modelers too. But I’d suggest that “Parametrics is also Free” when we deliver substantive systems engineering value as trusted advisors early-on in the development of project and program visualization. Spiral and 4GL paradigms notwithstanding, when we adequately act as “forcing functions” to get articulated requirements ready for primetime estimating, we have undeniably saved overruns and rework. 

A recommendation: ask early and often. And when in doubt (or they say “No, don’t need that” when you know better), model that object/activity and keep it ready. I’m the first to admit being on the receiving end of “You should’ve known to ask that” or “You’re here to fill in the blanks on what I don’t know to tell you.” No time for asking permission or forgiveness. Parametrics is a great job. But initiative and know-how are still the base ingredients to baking our quality cake.

John Swaren
Solutions  Consultant, PRICE Systems

The following is an extract from a paper written in 1978 from one of the founders of PRICE Systems: 

Two questions are often asked by those unfamiliar with TruePlanning’s approach to cost modeling: What is your CER (cost estimating relationship)? And what is your data base?

These questions are closely related.  Both are based on the assumption that the PRICE modeling approach is the same as that customarily used in developing cost estimating relationships.  This is not the case. 

The customary approach is to first gather as much relevant data as possible, then screen the data for consistency, reduce the data by formal statistical procedures and present the results in the form of one or more estimating relationships.

In contrast, the PRICE approach is process oriented rather than data base oriented. 
Models are designed to emulate the processes by which experienced managers, engineers and cost estimators assess the impacts of key cost and schedule drivers.

The strength of the classical approach is that it enables investigators to test hypotheses and identify significant factors that have affected past developments.  PRICE does not ignore these results.  Indeed, similar procedures are often used in preliminary model development.  As a consequence, there is nothing in the PRICE models that is inconsistent with classical results.  The difference is that these results are not an end in themselves.  When blended with quantifications of other subjective, but no less valid, perceptions by experience people, they contribute substantially to the PRICE methodology.

The weakness of the classical approach is that it is data base limited.  It does not extrapolate well when applied as a cost estimation procedure to new situations.  In fact, it usually does not even adequately describe the underlying data base.  Data definitions are often inconsistent, and data poorly recorded that fitted equations are questionable at best.  Extensive screening to eliminate inconsistent and unrepresentative data results in so few cases that the richness of single equation models is severely limited.  These models simply cannot account for all of the factors that drive costs.

The principal reason for the success of the PRICE model is that it does not depend on a single CER or on a single data base.  By focusing on the process of rational cost estimation, it preserves for the user the flexibility to tune the model to the particular data base most relevant to the estimate in question.  This will normally be cost histories of previous projects within the users’ own organization or product line. 

In effect, the PRICE model develops a new family of CERs to fit each specific application based on the data base it is tuned against.

Recently the Director of the Office of Management & Budget (OMB), Peter Orszag issued a directive that was posted on the OMB blog that outlined three specific actions for IT reform. The actions include a freeze on all new IT modernization task orders for financial systems, reviews of current high risk IT projects and require agencies to submit improvement plans to the CIO; thirdly, the OMB Deputy Director will develop recommendations within 120 days to improve the federal government’s overall IT procurement and management practices. Orszag states:

“While a productivity boom has transformed private sector performance over the past two decades, the federal government has almost entirely missed this transformation and now lags far behind on efficiency and service quality.  We are wasting billions of dollars a year, and more importantly are missing out on the huge productively improvements other sectors have benefited from.

Quite simply, we can’t significantly improve the efficiency and effectiveness of the federal government without fixing IT.”

My experience is that government IT project managers are very competent people and they want to deliver on time, on budget results.  However, as confirmed by this survey, 78% of government IT project managers feel that they are not equipped with the people, processes and tools to determine accurate project estimates and to conduct effective program affordability management.  The root cause of late, over budget IT projects are inaccurate, over-optimistic project estimates.  I and others at my company have worked hard to change this, and to ensure that government IT project management are educated about project estimating software but little has changed over the past ten years.  Fundamental change will only result from greater leadership focus on estimating accuracy throughout OMB and agency program management. 

Next month (8/4 @ 12pm EST) I am presenting a webinar to discuss using TruePlanning on Source Selections. What prompted me to develop this webinar were the many recent success stories I’ve had using TruePlanning during the Source Selection process. Going a bit further, I am going to show an actual case study where TruePlanning was used to conduct an Analysis of Alternatives (AoA) exercise – along with cost/effectiveness results. We will explore a bit about the technical side of the proposed designs, develop the modeling in TruePlanning and discuss the results.

In addition, we will explore the uses of TruePlanning during Source Selection from both the Customer and Supplier’s perspective. While they are similar, there are a few differences. Some of the uses of TruePlanning from the supplier’s perspective are listed below.

Supplier’s Perspective

• Should Cost – Are my vendors quoting us the best price?
• Ghosting the Competition – What are my competitors most likely to bid?
• Independent Cost Estimate (ICE) – What does an completely independent look say about my bid? Are we competitive?
• Risk Analysis – Is our bid over conservative, how much risk are we willing to take and how much cost exposure can we absorb.        
• Schedule Estimating – Can we really do the job within the schedule constraints?
• Trade Studies – What other configurations, materials or technologies might we consider?

“I know that you believe you understand what you think I said, but I'm not sure you realize that what you heard is not what I meant.” This is a famous quote by author and US. State Department spokesman Robert McCloskey (September 15, 1914 – June 30 2003) and was attributed to him by Marvin Kalb, a CBS reporter, in TV Guide 31 March 1984. This quote was in reference to a press briefing during the Vietnam War. What it really addresses is the ease to which the communication of the meaning / understanding of ideas can become confused or misunderstood.

I have found through-out my career that this is a constant danger. I think the danger is especially true in the area of cost estimation. What is really included in an estimate? Are we estimating the correct understanding of the project, system, or program? Did I understand the purpose of the item we are costing well enough to estimate it? Although many of you reading this would nod your head in agreement or even think well Duh! The act of communication, as part of the cost estimating process, is easily side-tracked. Because of this likelihood one must find the means to guard against confusion and to increase the probability of meaningful and accurate exchange of ideas.

To this end I have found a couple of techniques to be helpful. First and foremost, listen carefully to the person or group with which you are interacting. Ensure that you practice active listening by asking clarifying questions, providing appropriate feedback, or recording what is being said for further review later. Additionally, a picture really is worth a thousand words. Use visual products such as charts, diagrams or an interactive whiteboard session to clarify what is being discussed. BUT, don’t rely solely on pictures because each of us learn or communicate in different ways. Some do fine with visual communication others need to read it and even others are auditory and must hear the ideas out loud. Use a variety of interaction approaches to help increase the potential for good communication and the chance that you will provide a useful estimate will increase significantly.

Over the past several weeks several users have inquired about the best way to model software code translation when estimating software costs. Software code translation consists converting existing legacy software source code that was developed in a specific high order language (such as Ada or C) and translating it to a new high order language (such as C++). The translation does not add new functionality. For example, translating Ada code to C++, simply results in C++ code. The major benefit of the translation is maintenance. Ada experts are not needed to maintain C++ software for this example. The code and unit test activity is the major cost driver for a software translation.

Modeling this type of code translation is very straight-forward in the TruePlanning for Software model. Two input parameters should be evaluated for modeling this scenario: Auto-Translated Code Size and Auto-Translation Tool Efficiency. The translated code size should be entered for Auto-Translated code size. However, the translator may or may not be very efficient. This is the purpose of the translated tool efficiency input parameter. A drop-down dialog box is available.. The range is from “Very Low” to “Very High.” The efficiency of the translator should be selected from this dialog box. 

The purpose of the dialog box is to account for additional cost or effort due to a poor translator. For example, if the code translation size entered is 10,000 and the translation tool efficiency is Nominal or 80%, then the model will assume that 20% or 2,000 SLOC (source lines of code) will have to be manually translated into the new HOL. 

If you have additional questions concerning software code translation, please contact one of our PRICE Cost Research Analysts.

Jim Otte
Solutions Consultant, PRICE Systems
 

While sitting in the operatory chair yesterday, my dentist said something that made me stop. He was complaining about an increasing rate of incompetence and apathy he observes in those delivering services to him. And while I do agree with him in principal, he and I are of the age where some folks label us as grumpy old men. So, it may not be as bad as we think. Regardless, the statement he said he made to the an unfortunate poor-quality service provider was, “If you don’t have the time to do it right the first time, when are you going to have time to fix what you did?” Apparently, the excuse offered by the provider was that he didn’t have the time to do the job the correct way, so he tried a short-cut.

Wow! This hit me right between the eyes. I immediately expelled all the material and apparatus he had placed in my mouth so I could tell him how impressed I was with his logic. Plus, I had never heard that expression before. But how true it is and in it I see a glimmer of hope for realizing a key initiative of WSARA (Weapons Systems Acquisition Reform Act of 2009) - getting things right from the start with sound systems engineering, cost-estimating. The path to getting this accomplished is through the people doing the work and not through DoD leadership. As I reflect on my experience with DoD Systems Engineers and Cost Estimators, most I know want to do the job right the first time. If every one of us involved in planning and estimating software, hardware, and IT projects pledges to never take short-cuts, we can make the difference.

There are two reasons for why I don’t think this is just rhetoric: 1. getting things right from the start can’t happen if we are not committed to it; 2. I am unable to conjure up a single example of anything I have experienced in any dimension of my life that started out wrong and turned out right.

Those words describe the triangular probability distribution used by FRISK in performing risk analysis in TruePlanning. FRISK requires two major assumptions on the part of the user. The first is that the combination or convolution of a number of triangular distributions results in a log normal distribution. The second is that there is correlation between cost objects. 

The triangular distribution is completely defined by three simple inputs: an optimistic value, a pessimistic value, and a most likely value. By eliciting information from engineers, I have found that they are much more willing to commit to a range for a certain parameter, rather than a point value. I then use the limits of the range for the optimistic value and the pessimistic value and then ask the question, within the range that you just gave, what is the most likely value? This then gives me all the information I need to satisfy the requirements of the triangular distribution.

Another unique feature of the triangular distribution is that the optimistic and most likely or pessimistic and most likely values can be set to the same value yielding a right triangle. The triangular distribution is the only one with this property.

So, the next time you perform a risk analysis using TruePlanning think of the lowly triangular distribution and how simple, yet powerful it really is.

John Long
Solutions Consultant, PRICE Systems       

I recently had the opportunity to work directly for one of our clients on a high visibility, must-win proposal. The contractor was just about ready to commit to the bid number, but wanted to know the likely bids of the other two performers. We were asked to do a “Ghosting the Competition” study where we ethically collect open source data on two competing designs and combined with engineering technical data to develop a best cost estimate of the competitor’s bid positions.

Unfortunately, not much intelligence was known about the competing configurations, but the engineers recently noticed the other companies displaying their latest technology at a tradeshow event. Based on that information, plus some published marketing data, the engineers arrived at reasonable technology configurations and weight statements. We also established a cost baseline for the client by calibrating past data and using the client’s direct rates and overheads.  Using the TruePlanning estimation software we were able to complete all of the cost estimates in about ten hours including detailed reporting down to the resource and activity level.

Once we were done and happy with the results we were asked to brief the company President. Taking one glance at our estimates, he remarked that what he really needed was an Independent Cost Estimate or ICE in addition to our Ghosting study. The purpose of the ICE was to determine if they were really offering the best value possible to the government…and not just the lowest bid. He requested the ICE within four hours. This caused great concern for the engineers and finance as they had never done an ICE in this short amount of time.

However, using the same cost estimating tools and models (TruePlanning) we built for the Ghosting the Competition study, we were able to quickly and easily generate the ICE in less than four hours. This was done by removing the calibrated inputs for manufacturing and electronics complexity and replacing them with values from the PRICE knowledgebase. We also reverted back to the industry average rates contained in the worksheet sets.

Finally, we completed a producibility study on process improvement using the Manufacturing Process Index input. Using TruePlanning we quickly generated all of the reporting formats at the activity/resource level and export the results to EXCEL, WORD and Powerpoint.

The bottom line was that the versatility of the TruePlanning tool allowed the client to quickly and efficiently develop several types of analysis required for a must-win proposal with minimal impact on staff and in a very short amount of time.

"And me boss. And me boss. And me boss!"

Just like Bugs Bunny tricking the mob boss into an unfair share of the loot, who doesn’t want a piece of the action. In this case the “action” is the estimate you have just finished in TruePlanning and would like to share with your coworkers. No problem, just share your project.

Project sharing is a feature that is available to users who are using the Client/Server version of TruePlanning and it allows users to access projects that have been created on the centralized database by other users. If you are using standalone you will need to export your project to a tpprj file.

There are a few ways to set up sharing on a project. From the Project Manager dialog select a project, and then click the “Project Sharing” button on the left. In the Project Manager dialog you can also right click on any project listed and select Sharing from the right-click menu. If you have the project you would like to share already open, use File->Properties, then select the Sharing tab.

When sharing you can share your project to all users or you can share your project to specific users. If you would like to share to only specific users, select the “Share with the following users” radio button, then add them via the “Add” button.

Once a project has been shared, any user who has access to the project can open it and alter it. At this time TruePlanning does not support the concept of a ‘Read-only’ project. If a project is going to be shared, but the original estimate needs to be preserved, a copy of the original should be made and not shared.

Users who have been given access to TruePlanning via sharing have the ability to open, alter and save the project. They cannot rename, delete, or set sharing on projects they do not own. If a user would like to rename or alter sharing rights, they need to make a copy of the project. When a user makes a copy of a project that has been shared to them, they are the owner of the copy and can rename and alter sharing rights on the copy.

That’s it! You now have a piece of the action.

 Ed Merriman
Applications Engineer, PRICE Systems

Parametric modeling is excellent for all aspects of early-concept cost estimation, including go/no-go decisions downstream. So, in the spirit of bringing a transparency to (ethical) financial engineering…
why not apply our craft to pricing “real-options”?

The latter are essentially strategic opportunities for engaging resources (cost/schedule) into projects, ventures, investments, or even abandonments. The opportunity choice has value itself! 

Unlike static project Net Present Value (often, but not exclusively, approximated with Discounted Cash Flow) assuming pre-defined decisions, real-options reflect the merit of flexibility. If an R&D or proof-of-concept presents viability/ marketability learning, the option has positive value, above and beyond DCF. The more the flexibility, the higher the value. Likewise, a real-option appreciates with more uncertainty.

By now, you’re asking—“Wasn’t this a parametrics blog? I’m an engineering/ computing/ math/ science type, not a quantitative-finance geek. How could the above possibly help me any”?

Answer: In some situations, specifically go/no-go, the value of your flexibility created with the strategic choice to move forward (or not) can exceed its “option” cost. Not all options should be executed, just as all go/no-go decisions aren’t go’s. But, over time, continuing to pay less than their market value creates an opportunity to average out with total economic-value creation.

But, you say— “How do I find the cost of this flexibility/uncertainty option? It sounds great that I can make investment decisions based on buying/ executing (or not) these options, but do I really need to learn fancy finance stuff like Black-Scholes, Value Trees, Binomial-Risk Neutral Pricing… based on risk-free rates of return and (expected) discounted cashflows…. Yikes!”

Answer: (& bottom-line, for now) No. Use your parametric estimating tool! Concerned about the hardware cost of pilot-production/ tooling? “Buy” an option priced as the cost of preliminary design. 
{Note that the latter cost is your option’s premium, and the go-ahead cost is your option’s strike price.}
Interested in the nonrecurring cost of large-scale full software development? Buy an option for the
cost of first iteration increment. Concerned about COTS versus assembly? Estimate the development (and integration) of both scenarios. 

The point is take an economically-disciplined approach to valuing your strategic choices downstream. Parametric modeling works here and is “data-driven” defensible.  It is certainly applicable to strategic investment, capital -budgeting and new business decisions within both the public and private sectors. Transparency through mathematics is a good thing.

I have been working with several clients on data collection and repository creation this spring. One common theme that comes up is what data to collect and how to use it to make intelligent decisions about projects. And that’s the impetus for my little limerick above. 

The goal of data collection is wisdom. In the estimation world wisdom means knowing when it’s safe to bet your shirt on this new project you want to initiate. If you keep that perspective in mind it becomes much easier to identify what information you need to make wise decisions and what data you have to have to generate the necessary information. Every data item you collect costs time and money, transforming that data into information cost more time and money. So be frugal, figure out what you need to know to make you choices the correct ones and you will be that much closer to being happy and wise.