Arlene on Accurate Estimating

As Chief Scientist for PRICE Systems, provider of world class cost estimating tools and services, I spend my days researching new technologies, processes and techniques associated with the development and production of hardware, software and systems.  While the end goal is to develop an understanding of how various factors impact cost, the journey requires that I learn new things every day about how the world works.  I frequently publish or present these findings to the cost and measurement communities in various venues around the world.   The primary focus of my research (and my main passion) is all things related to Software Development and Information Technology although this has not precluded occasional forays into topics such as hardware manufacturing, systems engineering and composite materials. The goal of my research is arming cost estimators with the best technology available to address their estimating challenges and help them achieve estimating accuracy.  Read more about Arlene Minkiewicz

Failed software projects are always bad but there are additional complications when there is a contract in place to deliver the software.  Disputes over failed software can result in costly litigation that generally damages both the vendor and the buyer. According to observations of Capers Jones in "Conflict and Litigation Between Software Clients and Developers" (2007) , 5% of the projects his clients were involved in either had litigation pending or were currently involved in litigation over project failures.  His findings indicate that it is very large projects, over 10,000 Function Points that are most prone to litigation, which makes sense if you consider the relative increased complexity, as well as the large amount of money such a contract would involve.  Although one occasionally hears of pending litigation over software projects in the news, determining the background and outcomes of such litigation is difficult because they involve large customers and software contracting organizations, both of whom would rather keep their failures out of the spotlight.  This is likely a significant contributor to the fact that such litigations still occur since the industry is denied opportunities to examine and learn from the mistakes of others.

A report in Computer World [6] details a law suit filed by health insurer Highmark against KPGM for malpractice, fraud and brief of contract for a software system that Highmark had outsourced to KPMG.  While the article does not detail the outcome of this lawsuit, it makes it clear that the disagreement stemmed from misunderstandings and the failure of the two organizations to communicate and collaborate.

In light of this I have compiled a list of four responsible actions that both bidder and buyer should engage in when software projects are outsourced.  These may seem like common sense and the same old story, but clearly they bear repeating......

1. Create and Communicate Good Requirements ... misunderstood, constantly changing and new requirements are a common source of angst in software development projects.  Bidders need to understand the buyers requirements, document them fully, and incorporate the risk of requirements growth into their plans
2. Create and Maintain a credible cost estimate - the buyer needs to know what the effort should cost to evalute bids.  The bidder needs to properly assess the cost to them in order to make the project profitable and to avoid having the project cost them money
3. Understand and document risks; Plan up front for risk mitigation - bidders should work with buyers to understand their uncertainties.  These uncertainties need to be incorporated into the estimate and documented in the contract.
4. Create and maintain a cooperative relationship... when bidders and buyers collaborate and work nicely together, bumps in the road and set backs are handled sensibly, reducing the possiblities that things get out of hand.

Recently I was interviewed by Doug Beizer of Federal  Computer Weekly for an article about the shift of government agencies away from custom software development and towards the use of cloud computing.  The interest in this topic seemed to stem from the introduction of Apps.gov online store earlier this month.
 
Having been in the software cost estimation community for more than 25 years,  I have experienced this transition first hand but never really stopped to think about the whys and wherefores until questioned by Doug.  It was an interesting stroll down memory lane.  As an example, just think for a minute about how much communication has changed.  When Ifirst started working if I needed to discuss something with a co-worker or friend I either needed to pick up the phone or pick myself up out of my chair and go talk to them.  If I wanted to leave a paper trail – I actuallyneeded to use paper.  There was no internet, no e-mail, and certainly no social networking.  Like everyone else,government agencies needed to develop custom application to meet specific communication, networking and other needs.  Because the government had many communication needs – both on the business and mission side – government agencies became a leader in developing these types of applications.
 
With the advent of the Internet and ever evolving forms of social networking, the commercial sector has been forced to get really good at developing communication software – so good that the government agencies are nowl ooking more and more to the commercial sector and the cloud for such capabilities.  Not to say that all government needs can be solved with commercial solutions, but many more so than back in the day.  And while communication capabilities are my favorite example, let’s face the fact that there’s a lot of software out there– many of it that’s completely adequate (if not superior) to meet the needs of the government.

In an article in last weeks Harvard Business , IT Costs: Do You Speak Their Language), John Sviokla discusses the fact that as the information business continues to grow it is increasingly important for organizations to understand the impact of IT as it relates to their operating costs. This certainly rings True to us here at PRICE Systems who have recognized this reality. TruePlanning 2009 has been developed by PRICE specifically to help organizations get their heads around the true costs of Information Technology.

Application development projects can represent significant expense to an organization and tend to be the riskier items in the IT budgets. But it is important to recognize that they represent only a small part of most organizations IT budgets. According to Gartner’s "IT Spending and Staffing Report 2008",on average organizations spend about 20% of their IT budgets on application development. The rest of their budget is spent on hardware, software, networking, maintenance, as well as data center and help desk activities. The figure below indicates the typical distribution of IT costs based on this report:

From this last report we see that over the last 7 years the distribution of investment dollars between on-going operation and growth and innovation has remained steady with operational expenditures dominating. On average, organizations have spent 65% of their budgets to run the business, 21% to grow the business with a mere 14% of the budget left to transform the business.

With technologies such as SaaS and cloud computing, there is potential for organizations to make decisions to change this picture but they first need a way of identifying what in their IT organization is driving operational costs. TruePlanning for Information Technology makes it possible to model both your appliation development projects and the IT infrastructure (at a micro or macro leve) to help identify the best choices for optimizing Information Technology

Bad project estimates lower profitability.  Despite this fact many business leaders don’t invest in improving their estimating capability, buying into the fatalistic myth that this is as good as it gets.  This is patently wrong.  Project portfolios are prioritized based on the total expected Return on Investment (ROI) of projects.  Investments in the wrong project based on bad estimates could lead to lost revenue or delay of net benefit.

All around us we see reports of software projects which are over budget, delivered late or cancelled because they are taking too much time and money.  This very fact makes it easier for business leaders to throw up their hands and accept bad estimates instead of proactively looking to improve their estimating capability and their profitability.  It doesn’t have to be this way.  Any organization can turn this around with careful analysis of their own history combined with analysis of relevant industry benchmarks.  TruePlanning by PRICE Systems contains the methodology and cost estimating software  to help business leaders utilize history to improve project planning and avoid making bad decisions. 
 

It's finally Spring!  And along with the leaves on the trees, the beautiful flowers and the happy chirping birds.... it is once again Baseball Season.  Baseball season is a beautiful thing - and not just because, as a resident of South Jersey, my team is the 2008 World Champion Phillies.  I just love the game and everything about it.  I believe this is because with baseball the impossible becomes possible because anything can (and will)  happen and with a good plan in place you can still be successful.

I didn't always love baseball.  When I was a kid growing up in Baltimore - my Dad - a huge baseball fan - would take our family to Memorial Stadium to watch our beloved Orioles play. It was fun for the first few innings but after the hot dogs, sodas and peanuts were finished - it started to get a little boring (Sorry Dad but I just didn't get it).  Years later, as the mother of young little leaguers I actually detested baseball.  The only things worse than spending all your evenings at the little league field watching bad pitchers (cause they were learning) throw balls to bad batters (who were also learning) was the drive home with aforementioned (now angry) bad pitchers and/or hitters.  But as my boys matured something miraculous happened.  The kids started to play well and I started to enjoy the games.  The more enjoyable the games were the more I cared about the rules, the nuances, and the numbers of baseball.  Only after this journey did I realize what a great game baseball is.  A good team with good players, passion and a coach with a good plan can turn almost certain defeat into victory.

Project Planning is not all that different. When starting a software development project you need to start off with a plan and then you need to recognize that absolutely anything can happen.  Like a good coach you need to understand your risks - and you need to have contingency plans in case something unexpected happens.  A tool like PRICE Systems  TruePlanning for IT is a good place to start.  TruePlanning not only supplies cost and effort estimates based on descriptions of your project and your team, but it lets you indicate the uncertainty in these descriptions to add a level of confidence to the plan produced.

I have to say that my foray into blogging has been an interesting one.  By definition, the Chief Scientist should be a nerdy sort of geek too high brow to pontificate on topics in such a pedestrian format.  Actually I kind of like it.  In part because I enjoy writing and I'm not picky about what I write - technical documents are OK but pontification works as well.  And in part because I know that in order to be a good writer in a particular genre one must read extensively from that genre.  In other words I now have a good excuse to surf the web for related blogs and have found great ideas that have fueled my imagination

Today I want to share an article I found on 'Quips On Software Development' by David Longstreet called "Ancient Wisdom for Software Estimating".  In it Longstreet traces the need for good estmating all the way back to the Bible.  From Luke 14:25-33 Jesus says to the crowd of discipleship "Which of you wishing to construct a tower does not first sit down and calculate the cost to see if there is enough for its completion?  Otherwise after laying the foundation and finding himself unable to finish the work the onlookers should laugh at him and say, 'This one began to build but did not have the resources to finish'"

Of course this was posted on April 1st so my first thought was of practical jokes. After thinking about how cool it would be to get this kind of powerful buy-in on the importance of estimating I figured I would check it out.  So I powered up another browser and went to Bible.com (who knew!).  Sure enough it was the real deal.

So what's my point here?  Regardless of your religious beliefs - it seems prophetic that a document representing life and times more than 2000 years ago recognizes the importance of estimation and resource management.  And further acknowledges the fact that people will notice if you say you're going to do something and then don't finish it because you've run out of time or money.  Even before there were machines or factories, computers or software; before the words process improvement had ever been uttered - people understood that estimation was important when embarking on a project. 

The next time you think about starting a project with a WAG (Wild A** Guess) or with the number your boss wants to hear - first stop and ask yourself... "What Would Jesus Do?"

Here’s a great article I happened upon while doing research for a paper I’m writing.  “Lessons Learned: IT’s Biggest Project Failures”  In this article we are treated to stories of IT projects that “first make people laugh and then” (hopefully) “make them think.”  As a long time student of the failed software project, I was neither surprise nor disappointed with the projects relayed.  The projects noted failed for reasons such as:

• Failure to perform a should-cost analysis before selecting a supplier
• Failure to recognize an unhealthy project before it was too late
• Unrealistic and aggressive timelines
• Scope creep.

The US Department of Defense (DOD) continues to be plagued with cost overruns on major weapons systems.  Last month  Senators Carl Levin (D-Mich) and John McCain (R-Ariz) introduced the 2009 Weapon Systems Acquisition Reform Act intended to put measures in place to force the DOD to address the issues that cause overruns and schedule slippage.   Among other things,  this legislation would create the position of Director of Independent cost assessment for Major Defense Acquisition Programs (MDAPs) and require the DOD to perform trade-offs between cost, schedule and performance early in the program lifecycle. Read more about this legislation

The DOD's experience with cost overruns and schedule slippage is certainly not unique to them.  The size and complexity of the problems that they are trying to solve, along with the fact that their projects are funded with US tax dollars, makes their contract difficulties more dramatic and more public.  Everyone who builds large systems knows that the bigger their system the more risk there is that things will change between the time the project is first estimated and the time it is delivered.  And most changes that occur within a project involve adding requirements rather than removing them.  This is why cost, schedule and performance tradeoffs are so important – early and often in every project. 

And while there is no silver bullet that will solve the problems that the DOD and many other systems developers have with projects failing to meet goals  – there are tools and methodologies that will facilitate success with cost, schedule and performance tradeoffs.  TruePlanning for Products from PRICE Systems provides a framework in which systems developers can translate their own past performance to provide forward looking analyses that facilitate trade-offs between cost, performance and schedule. TruePlanning provides a trade space to help project managers create realistic expecatations for the cost and schedule of complex systems.

Last week I was asked to participate in Career Day at my son’s elementary school.  I was both honored and humbled.  Honored because the school felt that my career was something the children would be interested in and humbled because I was forced to concoct a story that would make cost estimating and analysis both understandable and interesting to children from kindergarten through grade eight.  Fortunately the format was such that I presented to each grade individually so at least I did not have to come up with one story to address all ages.

The experience turned out to be a lot more fun than I imagined.  It’s amazing how many kindergarteners in my son’s school have parents that are scientists (although to be fair their teacher informed me that thirty minutes earlier, when a policeman visited the classroom, many of their dad’s were also on the police force).  Lots of forensic scientists I suppose! 

And it wasn’t really as hard as I expected, explaining what PRICE Systems does.  I used images of the kinds of hardware and systems that TruePlanning for Products can be used to estimate. While some of the younger ones thought that I actually had flown on the Space Shuttle, the upper grades got it.  The experience actually helped me learn several new things about my job.  First of all, I learned that what I do actually is interesting to kids who are starting to think about what they might want to do with their lives. The little kids were fun but the older kids were attentive and asked good questions.  The fact that my work contains elements of science and technology exploration, with math and a little bit of detective work thrown in was actually appealing to many of them.  I was also reminded of how much I like my job and why.  Not only does my job create lots of new and interesting challenges, it also gives me opportunities daily to help people who do software project estimating, hardware project estimating and system level estimating.  I get to help these people be more successful at their jobs by providing tools and methodologies that make their estimates credible and believable.

                     
Like many others, I was astonished last Thursday by the images on my browser of those 155 extremely lucky people standing in the Hudson River.  And they certainly were very lucky last Thursday.  If you’re destined to fly on a flight bound for collision with birds, you want it to be piloted by a hero like Captain Sullenberger.  The incident made me think about what a hero is and how we all have the opportunities to be heroic in our chosen professions.

According to Wikipedia, a hero refers to a character that, in the face of danger and adversity or from a position of weakness, displays courage and the will for self sacrifice.  As a project manager you may not encounter danger daily, at least of a physical nature, but you certainly deal with adversity on many levels.  And courage is essential!

An heroic project manager is one who makes project decisions based on facts and who stands behind those decisions.  He or she defends the project team from those who wish to impose unrealistic deadlines and stands up to those who want to abuse or ignore the Project Management Triangle.  In order to be heroic, a project manager needs to have the tools necessary to plan a project right the first time as well as manage it with agility as situations change.  TruePlanning from PRICE Systems is just such a tool.  TruePlanning is the worlds leading solution for software cost estimation and IT Project Planning.  It gives project mangers the cost estimates they need to plan projects successfully and the courage necessary to defend those numbers to the naysayers.

Although the use of TruePlanning is unlikely to lead to instant fame or enable people to walk on water, a project manager who uses it regularly will be a hero to the team!
 

A recent Gartner report indicates that industry enthusiasm for SOA is waning.  The reasons cited are the lack of enough people with the proper skill sets to perform SOA deployments and the lack of a good business case for SOA.  It’s an interesting but not really unexpected direction.  SOA has been surrounded by significant hype, ensuring that organizations surveyed would be anxious to profess their desire to start a SOA project.  But as the rubber hits the road, these organizations are realizing that SOA may not be the answer to all of their organizational woes.

Many will argue that SOA is nothing new – but rather a repackaging of an object oriented approach to software reuse.  They insist that the reason for this repackaging is motivated by a strong marketing push to sell a new generation of tools to folks that were not successful achieving reuse with earlier versions of tools.  Although I agree that the notion of SOA is not new, I do believe that technology has been advanced to a level where enough abstraction can be applied to the process to make reuse a possibility.  I do agree with the cynics that the heavy marketing push for SOA has not helped businesses make thoughtful decisions about whether it is right for them.

Now that enthusiasm has been tempered, organizations are starting to look at the costs, benefits and ROI of SOA.  This is a good thing.  A more thoughtful approach across the industry will balance the effects of aggressive marketing and ensure that the right SOA projects are attempted.  Of course, the challenge of determining ROI is daunting, particularly without a lot of data. Determining the benefits of a relatively untested paradigm is problematic.  If the SOA vendors are to be believed the benefits are tremendous.  Experience of course tells us that this is optimistic.  But common sense indicates that there is a great deal of potential for benefit for businesses that meet one or more of the criteria listed in the article “Your SOA needs a business case” – in other words if your business must respond to frequent changes in the marketplace or with business rules and relationships.

Understanding, or predicting, the benefits SOA will bring to the organization is only half the challenge.  The costs associated with a deployment of capabilities in a service based framework are also an important consideration.  The TruePlanning Suite, available from PRICE Systems, can help in this area.  TruePlanning allows for IT Project cost estimation that will help address costs associated with deploying the right infrastructure for SOA, the costs associated with development of services, and the costs associated with the composition of applications using services.

Software cost estimation is hard.  I have learned this the hard way – as a software developer and later as the manager of software development projects. Today, as a builder of cost estimating software, I learn something everyday about software development that reinforces the fact that software cost estimation is hard.

I attend a lot of trade shows and I talk to a lot of software people about how they estimate software costs.  Many of them have no formal process, many don’t collect data as projects progress, many of them perform estimates off the cuff, and many of them, not surprisingly, are unhappy about the outcome of their software development projects.

Many of these same people are quick to point out, when presented with possible help in the form of TruePlanning for Software, why it will never work for them.  Nine times out of ten, the reason they give me has to do with the uniqueness of their product, their customer base, their market or their software development process.  They question the data behind the model and suggest that this data can not be successfully translated to properly model their specific situation.

And I can’t fight the uniqueness argument – it’s most likely true.  Every software project has factors that make it different from all the other software projects that have occurred in the past.  No two software projects are the same.  I can however argue about the suitability of the data behind my model. The majority of software projects are not defined by their uniqueness.  There is much more about most projects that is similar to other projects than is different.  A good software estimation tool like TruePlanning for Software capitalizes on this fact by allowing the user to take advantage of industry or organizational data for those many places where a project is not unique.  Further, it offers expert systems like guidance to transcend those areas where a software project veers from commonplace to extraordinary.

In software, as in life, we tend to put the most focus on the parts of a project that are challenging and interesting with much less focus on the parts that are ordinary.  If, in doing this, we are discounting the lessons we can learn from history, we are doing ourselves and our software projects a disservice.

According to Gartner, Service Oriented Architectures (SOAs) will be used in more than eighty percent of mission critical operational applications and business processes by the year 2010.  Perusal of the literature on SOAs leads to visions of ease of implementation and cost savings of epic proportions.  And it is likely that not only will service oriented implementations results in cost savings, but that without them many currently envisioned capabilities in the corporate world and the Department of Defense (DoD) will be impossible to deploy, maintain, and evolve as technology explodes and requirements and priorities shift.

As with all technology shifts, the cost savings are neither automatic nor are they immediate.  Up front investment is necessary before the cost savings start to roll in.  Businesses are anxious to begin deploying SOA capabilities but are wary as to what the transition may cost them. 

This is an area that we at PRICE are very interested in, not only because we want to help our clients overcome this software cost estimation challenge, but also because we are investigating ways that SOA concepts will help us better serve clients of the TruePlanning Software Cost Estimation capabilities. 

For purposes of understanding cost implications, a SOA deployment can be segmented into three general areas.  The implementation of the infrastructure necessary to host service based capabilities is a significant investment for a company.  This generally requires a search for the right set of commercial off the shelf middleware components to meet the business’s requirements for security, performance, etc.  The deployment of the infrastructure is a one-time investment with costs being driven by the specific requirements of the organization and the level of technical maturity with respect to SOA.

Once an infrastructure is in place, services need to be developed and existing capabilities need to be migrated from traditional applications to services.  As services are really just software implementations,  drivers in this category are similar to typical drivers for software development costs.

Finally there is an integration and test effort associated with getting services working within the context of the infrastructure to deliver value to end users.  In many ways this effort will be similar to other software integration and test efforts but additional factors will come into play. It's important to understand how the rigor put into developing the infrastructure will impact the cost and effort of actually using that infrastructure to deliver capability. Read more about on-going research into the costs and benefits of Service Oriented Architecture.

Martin Woodward ‘s comparison of software development to Sudoku does an excellent job of explaining why software cost estimation is hard.  We constantly hear about software development projects that are unsuccessful because they have violated one or more aspects of the Project Management Triangle  - they are either delivered late, delivered over budget or fail to deliver the capability that users require. And while I firmly believe that software development is an engineering discipline, software project estimation is often more complicated than estimation for other engineering disciplines because software project output is not nearly as easy to quantify and because there are other intangible factors that impact the software development process - the same kind of intangibles that Woodward mentions in relation to a Sudoku puzzle.

Woodward also does a great job in describing how agile software development practices help with software project estimation.  When employing agile development practices, an organization effectively embarks on a new software development project with every iteration (an iteration is a small chunk of time -  usually one or two weeks long - during which the next 'release' of software is completed). The tenets of agile require frequent releases and frequent customer reviews.  These reviews often trigger reprioritization and re-estimation. The agile team develops its own system of software measurement and quickly becomes good at estimating the work required for any given iteration. Agile also promotes good development practices to ease recovery when intangibles threaten to unhinge the Project Management Triangle.  

If every software development project is a two week venture and the development team is very good at estimating the next iteration, why would an organization require a software estimation tool like TruePlanning by PRICE Systems?  This, of course, is an excellent question.  It is true that the software development team doesn’t need a software estimation tool but the organization still does.  The organization needs to be able to put together credible a credible product roadmap.  Customers and sales people still want to know what’s up next and when it’s going to be available.  And although the nature of agile development indicates that content might change, the knowledge of the agile development team will actually help the organization use a software estimation tool successfully. 

Agile is all about measurement.  Successful software cost estimation, even with a great tool, requires good measurement practices.  The tool is the framework for success, but good metrics customize the framework for a specific organization.  Agile organizations that can combine their data with a framework like TruePlanning  can optimize their ability for successful long term planning.

Software size measurement continues to be a contentious issue within the software industry.  While the software engineering community has made significant progress in it’s quest for the right way to assign value, the journey has not been without missteps and detours.  And there is still more work to be done.

When I first started programming it never occurred to me to think about the size of the software I was developing.  This was true for several reasons.  First of all, back in the day, software had a tactile quality through the deck of punched cards required to run a program.  If I wanted to size the software I could touch, feel or eyeball the deck to get a sense of how much there was.  Secondly, I had no real reason to care how much code I was writing; I just kept writing until I got the desired results and then I moved on to the next challenge.  Finally, as an engineering student I was expected to learn how to program but I was never taught to appreciate the fact that developing software was an engineering discipline.  The idea of size being a characteristic of software was foreign to me – what did it really mean and what was the context?  And why would anyone care?

Twenty five years later, if you Google the phrase “software size” you will get more than one hundred thousand hits.  Clearly there is a reason to care about software size and there are lots of people out there worrying about it.  And still I am left to wonder – what does it really mean and what is the context?  And why would anyone care?

It turns out there are several very good reasons for wanting to measure software size.  Software size is an important component of productivity and quality metrics.  The ability to measure productivity and quality are an important part of any process improvement initiative.  The ability to measure size is essential to successful software project estimation.  Additionally, a good software size measure could conceivably lead to a better understanding of the value being delivered by a software application.  The problem is that there is no agreement among professionals as to the right units for measuring software or the right way to measure within the units we use today.  Read more about Software Measurement.