Is Open Source Software Higher Quality Software?

Monday, December 12, 2011 by Arlene Minkiewicz
Check out this paper   “The Economics of Community Open Source Software Projects: An Empirical Analysis of Maintenance Effort.”  In it the authors hypothesize that Open Source practices increase the quality of the software that gets produced and subsequently lead to code that is less costly to maintain. 

Low quality code must be refactored more frequently than high quality code and there is substantial evidence that maintenance interventions tend to lead to even more degradation of the quality of the code.  So not only are low quality applications more expensive to maintain, the unit maintenance cost also increases over time.  The authors use the term entropy to describe the phenomenon and introduce the concept of entropy-time as a means of creating a standard for measurement of this degradation over time due to subsequent refactoring activities.

The authors use this notion of entropy-time to conduct an empirical analysis comparing the maintenance costs for Open Source applications with the maintenance costs that come from a traditional maintenance cost estimating model developed through study of software maintenance efforts for proprietary software applications.  The data studied confirmed their hypotheses.  The study was rigorous and my intent in mentioning it is to pique your interest not to be comprehensive, so check it out if you’re interested.

What interests me are the conclusions the authors draw and their suggestions as to why these may be so.  The authors posit that Open Source Software may be higher quality software because it is being developed by people all over the planet where there is no (or little) direct communication requiring the development to be modular and very tightly coupled.  They further suggest that the interests of open source developers are more motivated towards quality because of the pride they take in their work or because they know that their work will be viewed by the masses.  The final factor they suggest might be responsible for this increase in quality with Open Source is the fact that schedules are self inflicted rather than in proprietary efforts where schedules are driven by customer or market demands.

I thought it was a great study with thought provoking results.  The authors ended with several examples where companies adopting a mixed public/private model has resulted in high quality software successes.  So maybe the software development community should look at where adopting open source practices might improve the quality of the software we are producing?

Taking the Pulse of the Estimation Industry

Tuesday, November 8, 2011 by Arlene Minkiewicz
Last week I attended the 26th annual COCOMO forum.  This meeting is an interesting combination of conference and working group and for me it’s a great place to take the pulse of the software and systems estimating community.  Lots of times you’ll go to a conference like this and feel as though the same old things are repeated year after year.  Not so with this conference – it is always a great mix of seasoned practitioners and graduate students with both groups providing forward looking information and inspiration on a variety of topics.  Not that some themes are not repeated year after year as there are some aspects of our industry that take a long time to change.  But there is also a preponderance of new and exciting (as exciting as math and statistics can get anyway) research.
The presentation topics were varied and included topics such as schedule estimation, requirements variability, very small satellites, design driven modeling and mobile application development to name a few.  In addition to many excellent presentations there were several workshops offered where attendees were able to participate in in-depth discussions about specific topics.  The workshop topics are very telling about what people in our industry think is important right now. The topics included Metrics and Estimating Models for Software Maintenance (this is a huge area for exploration and budget constraints are making it necessary to maintain and sustain systems for longer and longer) , Air Force Estimation Guidebook, DACS Data Repository Workshop (on-going effort to make software project benchmarks available to the industry), and Estimation Curriculum Development. 
There were also two panels where the future of domain modeling, estimation and improvement were discussed – one focused on the aerospace industry and the other on the commercial industry.  The best question I think for either of these panels was “What do you think we’ll be talking about 10 years from now?”  I think this was a great question because it speaks to the forward looking nature of the forum and it’s attendees.  What do you think we’ll be talking about in 10 years?

IT Project Failures

Thursday, October 20, 2011 by Arlene Minkiewicz
Check out this article.   “Why IT Projects May be Riskier than you Think”.  If you read through the comments you will see that this article truly resonates with many in the field.  In the article the authors discuss research of over 1471 IT projects (large projects with an average cost of $167 million) comparing budgets and expected performance with actual costs and results.  Their results were surprising in that the average overrun was only 27%.  Turns out that the average isn’t what requires study but rather the outliers.  The study found that 1 in 6 of these projects experienced average cost overruns of 200%.  This appears to represent a disproportionate number of projects with huge overruns.   

There is much about this discovery that is disturbing.  Nothing good happens when large IT projects go off the rails.  Money is lost, careers are ruined, and businesses tank.  And going forward – it’s not getting any better because projects are getting more not less complex.

OK – while this study is eye opening in some sense - unless you’ve been living in a cave it’s not really news that lots of large IT projects fail.  It seems to me the primary reasons for this are

* Failure of business leaders and IT personnel to communicate successfully about the problem to be solved and the plans for how to solve it.
* Project plans that evolve from optimism, bravado, or capitulation
* Failure to understand that change is hard and that technology alone will not effect change
* Refusal or inability to learn from history
* Inability to accept that changing or adding requirements to a software project can have far reaching effects that cost money and take time (seems like a no brainer but it happens all the time)
* Leadership that acts without introspection, self awareness, courage or good sense.

In other words there’s a very human element to most IT Project failures.  Some things that businesses can do to mitigate the likelihood of such failures:

* Business leaders should work collaboratively with IT on all aspects of a projects – conversations should be two way with both sides listening to the issues and concerns
* Organizational history on like projects should be studied.  If no history exists, look externally to learn what works and what doesn’t in your industry
* Tools and processes should be used wherever possible to support project estimation, planning and decicion making without emotion or bias
* Change needs to be championed from the top down
* Evolve the project in small achievable chunks.  Assess progress regularly.  Have a plan for how to identify problems as they arise and a criteria for when it is time to cut your losses
* Business and IT leaders need to act with knowledge of the business, knowledge of their teams, honest and realistic progress assessment, and courage to make hard decisions.

Certainly none of this is rocket science.  But it seems to me that any organization contemplating a large scale IT change initiative should first turn eyes on their organization and their past history to see how well or poorly they have addressed the issues outlined above. 

Quantifying Technical Debt

Monday, October 3, 2011 by Arlene Minkiewicz
Here’s an interesting article “Technical Debt as Metaphor for Future Cost” ().  In this the author discusses the acceptability of using the metaphor of technical debt to facilitate communications between business leaders and the software team when negotiating around the triangle  (time, money, scope).   And while the  author accepts the use of this metaphor good “short-hand” for communicating the fact that avoiding the work now is not sparing the cost but just rearranging the way the costs are incurred – and often increasing the overall costs that need to be spent. 

The caution this article carries about the use of this metaphor is not around the realization that decisions made to shortcut the process or defer functionality will cost in the future, but rather around quantification of how much they will cost in the future.   When we incur financial debt we basically know what we owe – within some uncertainty about future economic conditions.  But as the author points out in order for technical debt to successfully facilitate conversations with the business there needs to be trust established about the software team’s ability to quantify the magnitude of the debt. 

So how do software teams gain this trust and create successful negotiations around software project decisions. First of all they need a good language around which to have the conversation.  Using source lines of code or function points as a basis for describing the size of the debt is a good start.  An even better measure would be software size augmented by factors that indicate innate complexity and quality level (each of these is basically unit-less in the large but can be unitized within an organization) Good historical data is a great place to start understanding how size, complexity, quality all contribute to technical debt.  Teams that have shown successes through success software estimation are the teams who the business will listen to when the suggest that this shortcut will create this X hours of additional work effort in a future release or that the additional overhead associated with not fixing this problem now will be Y hours. 

How does your organization go about quantifying technical debt?

Some Random Musings on Technical Debt

Friday, September 9, 2011 by Arlene Minkiewicz

I have recently being following an animated thread on LinkedIn “Death of a Metaphor – Technical Debt.” It’s been live for 2 months with over 200 contributions from dozens of different people.  The discussion was launched by questioning whether continued use of this metaphor makes sense.  The discussion thread weaves and bobs around actually answering this question but it’s amazing how passionate the world is on this topic.  My personal opinion is that it’s a perfectly adequate metaphor because it helps create a discussion between IT and the business leaders in terms that both can understand – dollars and cents. 

Sometimes during software development decisions are made to forgo structural quality to meet some other objective – additional functional requirements, time to market, etc.  How many of us have been involved in a development project where it was more important to get the product out the door than it was to get it done right – we take short cuts with the intent to go back and do it right once the immediate fire drill is over.  The problem is that once the product is out the door and meeting the customers expectation - how do we convince the business that there is as much (maybe more) long term value in fixing a product that is seemingly working than in investing in new features that will make the product more appealing to more users.  What business leaders may not understand is the cost of continuing to maintain and grow a structurally questionable - sometimes brittle – application.  Technical debt is the monetization of that cost making it possible for IT to communicate the value to the business of creating and maintaining a structurally sound application.  Check out this blog for a good discussion of technical debt and how to prevent accruing it.  “Technical Debt gets the Message Across”. 

Software On the Move!

Friday, September 2, 2011 by Arlene Minkiewicz

The IEEE published “Top 11 Technologies of the Decade” in the  January 2011 editions of the IEEE Spectrum magazine.  It should come to a surprise to no one that the Smartphone was number 1 on this list.  The answer to the author’s question “Is your phone smarter than a fifth grader” was a resounding YES![1] 

 In 1983 Motorola introduced the first hand hell cellular phone.  It weighed in at two and a half pounds, had memory capacity for 30 phone numbers, took 10 hours to recharge and had a selling price of $4000 ($8045 in 2006).  The phone was the size of a man’s head and would sustain an hour of conversation before a recharge was required.  In June of 2007 Apple announced that it was launching the iPhone which would basically integrate all of the electronic gadgets teenagers carried around in their pockets into a complete package – phone, web browser, iPod and camera.  The iPhone 5 which should be available by Fall 2011 incorporates NFC technology, an upgraded operating system with cloud integration, music streaming, voice interface, 4G connectivity and an embedded social networking tool. NFC technology makes it possible to use the phone effectively as a credit card – making payments by swiping the phone near a device that can read its information. [2][3][4][5]

The proliferation of smartphones and tablets has led (and will continue to lead) to the proliferation of mobile applications.  And it seems as there are no bounds to the kinds of applications that are being developed for smartphones.  A sampling of some popular applications is listed below:

* Chase Mobile – which allows users to check account balances and review transactions
* Angry Birds – very popular gaming software
* Facebook - allows users to report their status from anywhere
* Yelp – allows users to locate places to eat, shop , etc. along with reviews from local patrons of said establishments

The list goes on but clearly if you can dream it – there’s an app for that (or at least there could be an app for that).  As practitioners in the art of estimation, all this mobile application development leads to the inevitable question of what does it cost to develop mobile apps and how is mobile application development different from more traditional forms of development? 
Mobiles apps can be categorized as either native applications – which run entirely on the device or web applications - with small clients resident on the device which interact with applications running on a remote server.  It appears that in general the web applications are less complex than the native ones, and thus less effort is required to build, test and deploy them.
While mobile application development is still in its infancy so a lot of what is going on in the industry now has a bit of the Wild West feel to it.  This stage of any technology is impacted by learning curve issues which may dampen productivity but at the same time there is the newness factor – where smart people are excited about the promise of new technologies and are willing to work extra hard to make things happen.  So while there is some effort/cost data available for mobiles apps, we must temper our enthusiasm.
Another concern when developing mobile applications is which platform(s) it is being developed for.  If an application is being developed for iPhone, Android and Blackberry the effort is significantly increased.  Although there are elements of the design that can certainly transcend platforms, each of these platforms has its own operating system and development environment. There are additional potential compatibility issues in cases such as Android where there are multiple companies manufacturing devices. Additionally, application developers need to determine which versions of OS for each of the platforms the application will work with.  They also need to be aware of and respect the user interface guidelines developed for the device(s) for which they are building apps.
Mobile applications may need to respond to various forms of external data from sensors, a real or virtual keypad, a GPS, microphones, etc.  They also may need to respond to movements of the actual device as well - so the screen adjusts when the user changes the orientation of the device.  There are also many instances where mobile apps will need to interact with other applications on the device.  Often the mobile application will need to share elements of the user interface with other applications. 
Mobile applications need to be developed in such a way as to limit the consumption of resources.  No matter how good your killer app is, if it drains the phone battery in half an hour no one is going to use it.  Along with all other applications that have access to the Internet, they should be built with a focus on security so that the users data is protected from malicious intrusions.
All of the issues listed above are likely to impact the complexity of the development effort of the mobile application – thus they have the potential to be an important part of the cost equation as well.  In many cases mobile applications are smaller than traditional applications so increased complexity may be offset because the projects and corresponding project teams are small.  Still this increased complexity is important to consider.
Another area where mobile apps are dramatically different than traditional apps is in the testing of the application.  Simulators and emulators exist and can be helpful in some circumstances but they are not always easy to use, effective or efficient.  There are also issues, with some platforms around the maturity of the technology that allows applications to be transferred from the development environment to the mobile device – further complicating the testing process.  Finally there is just the sheer magnitude of making sure that the application functions correctly on all the combinations of hardware, operating system and carriers on which it will be expected to perform.  The cost and effort associated with testing may present significant differences when being evaluated for mobile applications.
Smartphones are here to stay and more and more businesses will want (or need) to develop apps for them in order to remain competitive.  With the technology still relatively immature there is limited data that will help us develop cost models but there is feedback from the field on what issues are most likely to impact costs.  Though the technology is still emergent – there is some data available from commercially based mobile app developments – so at least we have a place to start.
Share your experiences with mobile application development by leaving a comment for this post.


[1] Ross, Philip E., “Top 11 Technologies of the Decade”, IEEE Spectrum, January 2011
[2]  http://www.motorola.com
[3] http://www.apple.com
[4] http://en.wikipedia.org/wiki/Mobile_phone
[5]http://www.ibtimes.com/articles/170418/20110628/quick-look-at-iphone-5-features-as-september-release-date-looks-probable-nfc-camera-8mp-lte-4g-displ.htm

Reused Code - what's it really buying you?

Monday, July 11, 2011 by Arlene Minkiewicz

There is a ton of code out there and we’re constantly adding more.  Gartner reported in 2009 that there were more than 310 billion lines of code in use worldwide. MS Word has grown from 27,000 lines of code in the first version to about 2 million in 2010.  The Windows Operating System grew from 3 million lines of code in 1992 to 20 Million in 1998.  You get the point – there’s lots of code out there doing lots of the same things that we may want our software to do.

One of the challenges software project leaders have in estimation and planning is successfully quantifying the functionality they can reuse and understanding and communicating the effort and cost associated with that reuse.  In projects where significant reuse is anticipated, reuse is really going to shape the project.  Reuse comes in many forms.  A firm grasp of the types of reuse and the potential pitfalls of such reuse facilities a thorough analysis of the impacts that reuse has on the project.  It is ill advised to expect to get a one to one benefit for reuse.  It is even more problematic to plan on reuse as a quick fix to schedule pressure.  

Some of the things you want to consider:

* Reusing legacy code can be a great productivity booster if the code is stable, well written and some of the original developers are on hand.  On the other hand, if you’re working with legacy code that has become spaghetti code which is only supported by a mildly deranged programmer in the corner cube – you might be better off starting from scratch
* The auto translation of existing code to newer technologies can also boost project productivity but one cannot expect the process to be transparent or without hiccups.  It is important to understand the nature of the translation tool, the amount of preparation required to use it and the expected amount of human intervention to make the translation complete
* The use of Commercial Off the Shelf Software (COTS) can significantly impact the schedule for a software project but there needs to be a thorough evaluation of the capabilities of the COTS against the requirements of the project.  There also needs to be acknowledgement with the consumer of the software system that using COTS software limits the amount of customization possible resulting in the risk that the finished product is not exactly what they expected.

Also it is often more complicated to integrate reused (not developed here) solutions since there is a steeper learning curve during the integration phase.  Code reuse can be a great way to achieve increased productivity but it is important not to assess this improvement over optimistically but rather through a careful appraisal of what the reuse activity entails.

Have a reuse story (success of horror) you want to share?  Post it as a comment to this blog.

Software Adaptation: Design and Code…

Wednesday, June 15, 2011 by John Swaren

While teaching an introductory TruePlanning for Software Estimating course this week at an Army location, I was asked to follow up with a clarification on “percent adapted” calculation. 

The official PRICE training materials definitions are:

 

Percent of Design Adapted - the percentage of the existing (adapted code) design that must change to enable the adapted code to function and meet the software project requirements;

 

Percent of Code Adapted - the percentage of the adapted code that must change to enable the adapted code to function and meet the software project requirements.

The former, Design, was received by the class as straightforward. Again, this adaptation includes architectural design changes, detailed design changes, and any necessary reverse engineering. Entered as a percentage of the Adapted Code size, this metric essentially captures the effort to modify/ enhance the underlying concepts for data and transaction algorithms.

 

As my class later reviewed a fellow-student’s modeling of a “real-world” example, we found that our jury was hung on the verdict of what’s really meant by adapting the latter, Code. Again, the instruction is to also enter this metric as a percentage of the pre-determined adapted code size.

 

But the legitimate question was: “Why isn’t it always 100%?” If we labored to already identify new code, deleted code and reused code… then isn’t any remaining modified code all adapted?

 

The answer, per our resident software Yoda, is consider the code in terms of modules, not just source lines. To answer what percentage of a target must change to meet new functional and project requirements, evaluate what sub-modules/ CSCIs must change and what are reused WITHIN each module. If an entire module is not modified, then it’s count is reflected in the percentage of reuse. But if module(s) have some components that are modified and some that are not, that information is captured in this latter percentage of code adaptation.

 

If you’ve followed my blogs, you know what’s coming next: another car example! 

Say I have an automobile to rebuild. The body, wheels & tires are fine. Even the suspension requires no modification. (No car guy would ever accept not spending time & money on handling too, but suspend your disbelief for now).

 

Only the motor needs “adaptation”… but also, only in certain areas. The ignition, fuel injection, water pump, A/C & exhaust all require little to no adaptation. But the cylinder heads, valves, camshafts, rods, pistons & crank all must be worked on. My percentage of motor component “code” that actually needs modification is clearly less than 100%. 

 

Of course, the changes made above are a function of what requirements we're trying to acheive, e.g., horsepower, torque, RPM, etc.  Likewise, this analogy holds true in typical software modification. What we are given, as previous deliverables, is more than just total lines of code. It’s functioning modules/ systems that may, or may not, require some re-work under the hood. 

Agree?

Additional insight into how TruePlanning defines and utilzes these inputs is detailed in the True Software White Paper.

How To Develop Data-Driven Cost Estimating Relationships in TruePlanning - 2011 ISPA Workshop Takeaways

Wednesday, June 15, 2011 by Zach Jasnoff

At the 2011 ISPA Conference, I conducted a ½ day workshop How To Develop Data-Driven Cost Estimating Relationships  in TruePlanning. The attendees at the workshop learned how to import their own data into TruePlanning and develop custom Cost Estimating Relationships. We covered three case studies:

·         In the UCAS case study we demonstrated how we can build CERs at a higher level to provide a test of reasonableness to the CAPE.

·         In the SRDR case study we demonstrated how we develop a CER to estimate SLOC based on historical data and use the results to within the new code parameter of the UCAS Reconnaissance software estimate. 

·         In the Aircraft case study we demonstrated how we can build higher level trend lines across various aircraft system platforms and classes of aircraft to compare as “ball-park” estimates.

In each case, the attendees learned how to work with real data and develop CERs traceable back to the original data points that generated them. This was a “hands-on” workshop and everyone had a chance at generating a number of CERs. Overall, we had a very successful session and provided a new and exciting capability to our TruePlanning framework.

I also wanted to share with you a few takeaways from this workshop.

  • By setting up the trend lines equations in the Equation Cost Object, estimators can create libraries of data-driven CERs for future projects
    • All data used to develop the equation can be attached to the CER to identify datapoints and methodology used.
  • The power of this approach is that the CERs are integrated into the TruePlanning framework and can be used with other cost objects (H/W, SW, IT, etc)
  • Libraries of CERs can be shared among projects.
  • CERs are “white box” and auditable both in terms of the trend line equation and underlying data points.
  • CERs represent an “estimating database” in addition to TruePlanning Cost Objects.

If you are interested in obtaining a copy of the workshop presentation including the data files and results, please e-mail me at Zachary.Jasnoff@pricesystems.com .

If all you have is a hammer......

Friday, June 10, 2011 by Arlene Minkiewicz

I’m on my way home from the ISPA/SCEA (International Society of Parametric Analysts, Society of Cost Estimating and Analysis) Conference held in Albuquerque this week.  Attendance was very good (2nd best in the conferences history) and as the content seemed especially good this week.  I attended lots of good talks on topics ranging from SEPM (System Engineering, Project Management) cost estimating, Joint Confidence Levels, Software Estimating, Affordability,  Agile software development and estimating for Enterprise Resource Planning Systems.   Of course, just because the topics are good and well presented doesn’t mean I have to agree with everything that gets said.

One particular talk entitled “Function Points, One Size Fits All” got me a little worked up.  It seems as though there is an on-going and completely unnecessary amount of energy devoted by some to convince the software community that we need to settle on a single method of measurement for software.  The author makes some really good, credible points.  Function Points are a much better method of measuring software if your intent is to compare productivity across or within specific industries.  They help eliminate development technologies, programmer inconsistencies and software architecture decisions from an analysis of how productive an organization is at delivering business value to their customers.

Having said this, I firmly believe that sometimes SLOC (Source Lines of Code) are a better tool to help organizations estimate the costs of the software development projects they are planning.  Knowing how your productivity stacks against the industry is great information when you are looking for process improvement or identifying best practices.  But if you want to know how much the next project is going to costor how many months it will  be before you can deliver content, your own history is much more relevant than the industries.  Not that your history can’t be in Function Points – it just doesn’t have to be.  Many of the weaknesses sighted with SLOC – while they certainly can be considered weaknesses) -aren’t really noticeable when working within a certain context.  Within an organization, software development projects are often similar to previous projects, targeted at similar markets and are being developed by similar teams with similar (on average) coding styles. And SLOC Counting processes can be institutionalized within an organization. When this is the case, SLOC counts are just as good, if not better, than Function Point Counts and probably much easier to count since their count can be automated and does not require certified specialists.

The author contends that SLOC is a bad option because it is hard to estimate early on.  I contend the same is true with Function Points - which if the Counting Practices Manual is followed to the letter – one is required to make an assessment of not only all transaction but also all the data element types and record element types.  The author contends that SLOC is a bad option because there is no standard for what a SLOC is.  I contend that the same can be said for Function Points.  Perusal of the International Software Benchmark Standards Groups’ (ISBSG) database shows that there are several different ‘standards’ for counting Function Points – because one standard was not deemed sufficient by many in the community to meet all software measurement needs.

Don’t get me wrong – I do not disagree with many of the assertions the author made, only the notion that one size fits all.  Let’s be real – software measurement is not easy and there are many different ways to approach it depending on the reasons for the measurement and the circumstances around those needs.  Function Points are a good tool to have in your software measurement tool box, so are Source Lines of Code, Use Case Points and user Stories.  It’s true that if all you have is a hammer every problem is a nail.  Fortunately we have many different tools and should feel empowered to choose the right tool depending on the current need.

TruePlanning’s Role in Should-Cost Management

Thursday, May 19, 2011 by Zach Jasnoff

I was recently asked by a client to provide a synopsis of what TruePlanning offers in response to the Ashton Carter Memorandum – Implementation of Will-Cost and Should-Cost Management. In the memo, the Undersecretary of Defense AT&L listed “Selected Ingredients of Should Cost Management”. It was interesting to note how much capability is provided by TruePlanning to effectively support efficient should cost management. In this month’s blog, I will share with my response to our client with you.

Selected Ingredients of Should Cost Mgmt

(Ashton Carter Memorandum)

TruePlanning “Should Cost” Capability

Scrutinize each contributing ingredient of program cost and justify it…

Can model size, technology and schedule parameters and understand the interaction of each on cost.

Benchmark against similar DoD programs and commercial analogues…

Benchmarking using cost research knowledge databases based on both military and commercial programs. Can also include specific program history.

Promote Supply Chain Management to encourage competition and incentivize cost performance at lower tiers.

Can model the entire supply chain to analyze and understand impact of competition and cost incentives.

Identify opportunity to breakout GFE vs. prime contractor-produced items

Contains models for both GFE vs. CFE allowing including estimating new development vs. modification

Identify items or services contracted through a second or third party vehicle. Eliminate unnecessary pass-through costs by considering other contracting options.

Ability to model different vendor scenarios using True Planning's System Level Cost Model.

Identify an alternative technology/material that can potentially reduce development or life cycle costs for a program. Ensure the prime product contract includes the development of this technology/material at the right time.

Robust capability to quickly select alternative technologies/materials and quantify impact on lifecycle costs.

 

Agile Estimation

Wednesday, April 27, 2011 by Arlene Minkiewicz

Agile software development practices are predicated on the following tenets as introduced in 2001 in the Agile Manifesto [1]

  •  Individuals and interactions over processes and tools
  •  Working software over comprehensive documentation
  •  Customer collaboration over contract negotiation
  •  Responding to change over following a plan

Agile development processes rely on experienced, highly skilled people communicating with clients and each other to deliver software that provides the clients with the most value for the money they spend.  This requires both developers and consumers of software to accept the reality that things will change over the course of the project and that the software that is eventually delivered may not be the same as that was envisioned when the project was first kicked off.

At any given time, the agile software development team is only working on the feature that the customer currently feels is the most valuable.  Estimation is performed by the team (including developers, BAs, QAs and customer representatives) and is only focused on the feature that is currently on deck.  At the end of an iteration, the customer has an opportunity to review the implementation to date and can reprioritize remaining features based on changes in their requirements, market or expectations.  So estimating beyond the current feature doesn’t really make agile sense.

Unfortunately, the fact that estimation doesn’t make sense for the agile team does not mean it doesn’t make sense for the business.  The business needs to see the forest not just the trees.  Customers need an idea when their software will be delivered, businesses need to prep the market for new products and features, businesses need to be able to optimize resource allocations across many projects, etc.   This of course begs the question of how to perform an estimate for software when you really don’t know exactly what software you’re going to build.  As an industry, we’ve been pretty unsuccessful estimating software projects even when we think we have a handle on the end product – how can we be successful with so much uncertainty.

First we accept uncertainty and commit to incorporating uncertainty into our estimates.  Once we’ve made that commitment we are then free to pursue one of many top level estimating techniques to help us get our head around a likely range of values for cost and schedule based on what we currently know about the software project.  Parametric techniques are particularly suitable for this task especially for an organization that has some historical data from previous agile projects.  Parametric estimating models like TruePlanning for Software provide a repeatable framework through which an organization can study their past performances on similar projects (similar number of features, similar market, similar customer, etc) and use what they learn to perform estimates on the project at hand.   The amount and nature of the similarities should guide the amount of uncertainty in the estimate.  Organizational history can be used to map Story Points or User Stories to a size measurement such as source lines of code, function points or user points.  Analysis of performance on past projects can inform decisions about productivity and other project drivers.  This information can be used to drive the parametric algorithms to develop estimates for cost, effort and schedule.

Since we have already acknowledged that we didn’t get the question right, it’s unreasonable to assume that the answer will be ‘the right answer’.  What we do come away with is a range for cost, effort and schedule which will give decision makers realistic data to work with. If we have properly studied our history and thoughtfully assessed uncertainty, we can present these ranges with a quantifiable degree of certainty. 

What process does your organization employ to plan around agile induced uncertainties?

[1] Agile Alliance, “Agile Software Development Manifesto”, Feb 2001, available at www.agilemanifesto.org (retrieved January 2010)


Cloud Nine - Are We There Yet?

Tuesday, April 5, 2011 by Arlene Minkiewicz

In 1961 at the MIT Centennial, John McCarthy opined “if computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility…. the computer utility could become the basis of a new and important industry”  [1].  In 2006, Amazon Web Services was launched providing computing on a utility basis.  Since that time the notion of cloud computing has been emerging and evolving.

Cloud computing is a paradigm that makes the notion of utility computing a reality.  Instead of Information Technology (IT) organizations investing in all of the hardware, software and infrastructure necessary to meet their business needs, cloud computing makes access to hardware, software and infrastructure available through the internet, generally utilizing a pay for use model.  Basically cloud computing allows an organization to adopt a different economic model for meeting IT needs by reducing capital investments and increasing operational investments, a model which is likely to offer cost savings to many organizations.

There is still a great deal of hype around cloud computing, as many vendors have their marketing engines further into the clouds then their technology supports.  Despite this Gartner predicts that by 2012 one in five businesses will not own its own IT assets. [2].  In late 2010 the Office of Management and Budget (OMB) under direction from the White House told federal agencies that starting in 2012 they are expected to consider cloud first “whenever a secure, reliable, cost-effective cloud option exists.” [3]

There are certainly many reasons why an organization would consider moving at least some of their IT functions into the cloud.  In addition to potential cost savings the cloud offers the possibility of increased availability, easier collaboration, lower capital costs, scalability and virtualization.  There are of course concerns as well.  The technology is still relatively immature with no definitive set of standards for interface or compliance with regulations.  Businesses lose hands on control of their IT resources with little recourse if their IT vendor shuts down or goes out of business.  Additionally,  there are security and data privacy concerns.  There is also the fact that not all ventures into the cloud will be cost effective for the business.

This paper introduces the concept of cloud computing and discusses the potential benefits for a business as well as those things which could be barriers to adoption.  It examines the types of applications where cloud computing is an efficient cost effective solution and the types of applications where its use could be problematic or costly.  Several examples of successful cloud implementations are presented and discussed.

For a white paper describing this reearch on cloud computing email info@pricesystems.com with the code word CLOUD in the subject line.  To share your cloud computing experiences comment on this post

So you think you are good? Can you prove it?

Tuesday, March 29, 2011 by Pete Pizzutillo

“I think we have an obligation to work with industry to ensure that our suppliers do not just remain world class in defence, but aspire to be world-class manufactures that can withstand comparison to other industries.”

Chief of Defence Procurement, Sir Robert Walmsley

Is this a practical proposition or is it a pipe dream?  The following excerpt from Dale Shermon’s Systems Cost Engineering attempts to make the case that this type of comparison is possible.


Many of the statements in proposals and marketing literature stating the superiority of a company are anecdotal or at best qualitative evidence. It has not been possible to quantify the productivity of industry to demonstrate that one manufacturer is more efficient than another.

In the ship building industry it is possible to benchmark commercial ship yards using a metric called Compensated Gross Tonnage (CGT). CGT measures the complexity of building a ship. It is calculated by multiplying the gross tonnage by the relevant CGT coefficient (calculated by the Organisation for Economic and Commercial Development [OECD]).  However, this is only applicable to this one industry. Other metrics exist in other industries, but what is really required is a single benchmarking technique and measure.

Benchmarking with Cost Normalization.  Forty years ago, PRICE Systems developed a metric that can be used to normalize cost and support the benchmarking practices needed to reach Sir Walmsley vision.  Manufacturing Complexity represents the normalized cost density of an item that has been manufactured. It has two dimensions: technology and productivity. Hence, more than one Manufacturing Complexity can exist. As aircraft are made using different technologies and by different companies, these have various complexity figures. However, if the technology is constant, the only dimension that changes is productivity. As a result, it is possible to compare the efficiency of companies that produce similar technologies.

Supplier Assessment.  One obvious and practical application of this technique is Supplier Assessment. This technique exploits the capability of parametric models to compare organization’s normalized cost density or Manufacturing Complexity. If, on paper, you have very similar proposals, how can you judge which organization will provide the best value for money? Heritage data will determine the level of development that is required and Operating and Support Cost will establish the Through Life Cost, but if these cannot differentiate the proposals perhaps the productivity of the companies should be considered. This is not a simple comparison, but with manufacturing complexity this can be achieved easily.

This process will lead to your suppliers knowing that you are able to control them; they will need more control on themselves. It will enable you to perfect your negotiation skill, leading to more success, as better negotiation material is utilised based on technology cost drivers. A parametric questionnaire is a possible tool to enable proposal validation in minutes.  All this information can be garnered at a very early stage in a project if required, providing better estimating accuracy for the purchased items. Normalization enables a bigger data sample as previous proposals received from the same companies can be compared. This in turn enables the establishing of trends. Are the suppliers becoming more or less efficient? Ultimately, these comparisons can be used to determine preferred suppliers.

Conclusion.  This method of benchmarking a project or company can be used for all environments. With the ability to move across environments it is possible to compare commercial and military organisations equally. Decisions and judgments can be made between Commercial and Military suppliers where no military competition exists.  It is possible to assess the productivity of an organisation or country at System, sub-system and equipment levels. The technique can be used to compare the whole cost or just the labour cost if that is the focus of your efficiency drive.

It is possible to determine preferred suppliers on a justified basis, or alternatively it is possible to demonstrate the efficiency of your organisation when the customer is skeptical about your productivity or your drive to improve productivity.

Finally, many organisations already have a license to use the PRICE models and over 8,000 people worldwide have been trained and are familiar with the PRICE methodology. This is not a new idea, PRICE has been established since 1975, but this new application of an established technique can help decision maker take the right decision regarding source selection cross industry and national companies. Alternatively, it can enable industry to demonstrate, with proof, that they are more effective than other solutions.

Me and quality

Wednesday, March 16, 2011 by John Swaren
In Parametrics is Free, I acknowledged receiving (too late) “you should’ve known to ask that” over the years. Quality control after-the-fact is fine; but it’s better and cheaper to take a systematic approach to quality assurance as part of your estimating process. The sheer volume of what we model can often keep us so close to the details that we are unable to step back and put on our QA hat on for a sanity check. Enter Quality! On a very large project, our team has introduced a few regular cross-checks, notwithstanding typical math check-sums.  
  1. A round table peer review, where we describe and defend our parametric modeling approach as well as consistency across similar objects/ systems/ alternatives. 
  2. Introduction of an independent 3rd party, not part of the estimating team, who is an experienced modeler. This resource can take the client perspective in reviewing, say, procurement versus cost-of-ownership as well as comparables from similar program experiences.  
  3. Take an agile approach where your project leader is the “product owner” who reviews all interim deliverables top-down.  
  4. Another option is incorporating a client representative into your pre-release reviews for similar validation testing. 
  5. Finally, we pair off internally to verifying each other’s work against the real-time documentation we keep per common templates.  (Available from PRICE Systems, for both hardware and software modeling inputs). They are great visualization tools for internal parameter-tracking, plus external pre/ post model review with customers.
To summarize QA process, we backward plan enough time from delivery to allow us to bring multiple perspectives and approaches to bear. Quality is still (almost) free; and it’s worth the time… every time! Do you have a best practice quality tip that you'd like to share?  Leave us a comment.

The cobbler's kids...

Wednesday, March 16, 2011 by John Swaren

...wear the worst shoes. The cobbler was a master at his craft; he was just too tired to practice it when he got home from the shop.  Sound familiar?

A disciplined approach to understanding (functional) requirements as well as analogous projects (with actuals) is our not-so-secret sauce. Why run the risk of creeping back up our career learning curve? There’s already enough scope creep to keep us busy. Plus, for you management types charged with prospecting, a consistent approach towards estimation is a great way to connect with people who've felt the pain of being the cobbler's kids.

I recently reconnected with a colleague who found himself immersed in an early-stage, large-scale federal software project. Discussion points ranged from parametrics utility to sizing to Earned Value Management. Not leaving my toolset at the shop translated into good conversation with followup request. A bad example is yours truly, a year ago, so immersed in a highly-featured personal project, that he neglected to find agreement on scope and cost with his team. Bad idea! Now my “shoes” (a one-off custom hot-rod) are way behind schedule and way over budget. Keeping on my professional cost estimation consulting hat would have saved on both… bigtime! 

Fuel Cells

Tuesday, March 1, 2011 by Arlene Minkiewicz

The concept of the fuel cell was first published in 1938 by Christian Friedrich Schonbein.  Based on this publication Sir William Grove invented the precursor of the fuel cell in 1839. The Grove Cell created current by applying two acids to zinc and platinum electrodes separated by a porous ceramic pot.  In 1842 Grove developed the first actual fuel cell which produced electricity with hydrogen and oxygen, much like many fuel cells in use today.

Fuel cells remained an intellectual curiosity until the 1960’s when the US space program identified a requirement for extended life batteries for which fuel cells seem to offer a promising solution.  The focus on green technologies has increased interest in consumer uses of fuel cells for transportation, residential and commercial power supply, emergency backup power and portable power supplies for consumer and battlefield applications.  Increased usage of any technology begs the question of how to address the costs associated with that technology. 

A fuel cell is an electrochemical cell which converts some fuel, usually hydrogen, into electric current.  It does this through a reaction between the fuel and an oxidant in the presence of an electrolyte.  The waste product of this chemical process is water and heat.  Fuel cells, unlike conventional batteries, consume reactant from an external source rather than one stored in the battery. They do require a continuous supply of fuel, but given that this supply is available, they will not run out of charge like a conventional battery. 

Because fuel cells require neither flame nor combustion to convert fuel to electricity, there is much hope that they will become a viable power source of the future as we try to reduce our carbon footprint.  Fuel cells are very reliable and less likely to be effected by the environment as some more conventional power delivery systems are.  Because of this they are being adopted in industries such as the telecommunications where outages are particularly problematic.  They are often considered for power generation in remote areas where energy from the grid is expensive and outages are frequent.  Because heat is a waste product of the fuel cell electricity generation process, micro combined heat and power systems are gaining popularity for residential and small business needs.  Other interesting uses of fuel cell power include material handling, backup power systems and uninterruptable power supplies.

Despite increases in the use of fuel cells, they continue to evade wide spread use because they are expensive.  Certainly significant progress has been made through increases in efficiency and improvements in manufacturing processes, but it is still more expensive, in most domains, to get electricity from fuel cells than from more conventional methods. According to a report from the Department of Energy in May 2010, high volume automotive fuel cell stack cost has been reduced from $275/KW in 2002 to $61/KW in 2009 and appear to be on track to reach the $30/KW goal by 2015  The same report indicates a 24% increase in system power density for stationary fuel cells making it possible to reduce the fuel stack volume, weight and cost.

PRICE recently conducted a research effort using publically available data to develop cost estimating relationships for various types of power systems utilizing fuel cell technology. For a white paper describing this project and the resulting cost estimating models email info@pricesystems.com with the code word FUEL in the subject line.


In an english paper its plagerism, it business its common sense

Monday, February 28, 2011 by David Seaver
Don't reinvent the wheel.  It's a waste of time and effort.  All too often I see organizations establishing measurement programs or new software estimation intiatives and they want to build everything from the ground up.  Mistake, mistake, mistake...

People have gone before you.  Learn from them.  Take their ideas and go forward from there.  In the past year, I have architected the implemention of our software cost estimation tools at two large federal agencies and two DoD programs.  Teaching people how to estimate is easy.  Teaching them how to find the data to develop estimates is the hard part.  My solution?  I reused (plagerized?) the measurement framework from the Practical Software and Systems Measurement (PSM) initiative.  This let us use a commercial off the shelf solution to estimate and an industry-based best practice approach to collect data to feed into the estimation technology.  The combination saved us several person years and many months of calendar time.  Our off the shelf approach made our clients very happy.  So, that's today's candy.  What do you think?  Tell me how you've saved time and money in your estimating practices.

Asset Valuation

Thursday, February 17, 2011 by John Swaren

My June blog entry suggested the use of parametrics in real-options valuation. This month, I’d like to offer the generalized use of our type of modeling in valuing tangible assets. 

Typically, fundamental analysis evaluates the intrinsic value of securities. I won’t attempt to compete with Warren Buffet here. But it is certainly the case that a company, or portfolio of securities reflecting many companies, is based in part on the market value of its product assets and their potential for future earnings, as well as other objective and subjective considerations.

In parametric estimation, we take a top-down approach to developing models of estimating ultimate acquisition and ownership costs. In the case of the former, I’d argue that we can readily… and uniquely… estimate operating costs for companies developing and selling, say, technology-based products.  Marketing can determine pricing, based on their competitive landscape analyses and market-demand for unmet needs. 

We are in an excellent position to complete the forecasting of earnings (and hence, discounted cash flow) using our parametric methods to evaluate a product-system’s should/will costs. Anyone who does detailed bottoms-up budgeting for “go to market” expense, as well as production delivery, would find value in our modeling.  

And again, once estimated costs are subtracted from revenue forecasts, the netted estimate of earnings and cash flows will allow for traditional market value assessment of this asset or entity. For example, your organization needs to compare economic potentials of emerging new software products. There are no balance sheets. There are no income statements. Yet. 

Now your sales and marketing function has a good independent sense of revenue opportunity. So it’s left to you to estimate the costs of developing and delivering. Independent parametric estimation is the answer. No idea on sizing, either SLOC or function points? No problem. 

Data-driven True Planning has years of data modeling that allow you to run application-specific calculators. In addition, you can take advantage of calibrating actual data from similar projects. Likewise, this process of valuing assets holds true for IT infrastructure products as well. Cloud computing, SAAS and ERP are all further examples of where new IT/software products are emerging… and need estimating as well as financing via valuation.

I think parametric estimating has significant value commercially in the estimation of value of future product assets, and consequently their companies and associated securities.  What do you think?


DODCAS 2011 - An Excellent Training Experience for Cost Professionals

Friday, February 11, 2011 by Arlene Minkiewicz
The DoD Cost Analysis Symposium (DODCAS 2011) is next week, Feb 15-18.  I’ll be there along with several of my colleagues at PRICE Systems.  This conference consistently provides an excellent source of information and shared experiences for the acquisition community and I am anxious to attend again this year. 

Last year the conference occurred shortly after Congress passed the Weapons System Acquisition Reform Act of 2009 (WSARA) - and the majority of the sessions were focused on discussions about how the services, contractors and the government leadership planned on dealing with this new law.  From the agenda, it looks like this year there will be much discussion of how these plans were executed; what worked and what didn’t.   Topics range from will cost and should cost analysis, cost efficiencies, earned value management, quality data, affordability analysis, and other topics of interest to the acquisition community.

There will be an address by the Honorable Christine Fox, Director of the OSD Cost Assessment and Program Evaluation office (OSD CAPE) as well as a session with the Cost Directors of each of the Services.  Shortly after her appointment as CAPE Director, Ms. Fox addressed the attendees of DODCAS 2010 with her plans going forward.  She described the US Budget process and illustrated where the CAPE will fit in that process.  She discussed WSARA and its intended benefits and the challenges it presents to the CAPE and the cost community as a whole.   And finally she addressed the issues associated with acquiring, training and retaining the necessary human resources to support the CAPE and the objectives of WSARA.   I found her talk interesting, informative and hugely relevant to what I do every day.  I’m looking forward to her address this year and learning about CAPE successes, challenges and lessons learned.

Have you attended DODCAS recently?  What cost estimating and analysis conferences are the staples in your training budget?