Software Estimation BlogInformation on Cost Estimation & Software Estimating

By Robert P Hunt (Galorath Incorporated), Paul J. Solomon (Performance-Based Earned Value), and Dan Galorath (Galorath Incorporated)

Often, traditional earned value approaches do not deal sufficiently with the idiosyncrasies of software intensive programs.  However, successful management of software intensive programs can be achieved by focusing on establishing the requirements, developing a reliable baseline estimate for cost and schedule, selecting effective software metrics, applying Performance-Based Earned Value® (PBEV), and using analytic processes to project cost and schedule based on actual performance.

Introduction

The Department of Defense estimates that software now accounts for 40% of all research, development, test and evaluation (RDT&E) spending[i].  Software intensive projects that achieve their original cost and schedule projections are rare.  Many information technology projects have been declared a disaster area by commercial and government managers.  These projects have been too costly, too late, and often don’t work right.  Applying appropriate technical and management techniques can significantly improve the current situation.

Inaccurate estimates can threaten project success causing poor project implementations, shortcutting critical processes, and emergency staffing to recover schedule.   The lack of well defined project requirements and specifications may result in significant growth in cost and schedule.   Symptoms of this growth may include constantly changing project goals, frustration, customer dissatisfaction, cost overruns, missed schedules, and the failure of a project to meet its objectives.

PMI published an analysis of several government defense and intelligence agency large-scale acquisition programs that experienced significant cost and schedule growth.  This analysis shows that several critical factors need to be addressed in the pre-acquisition phase of the acquisition cycle.  The principal causes of growth on these large-scale programs can be traced to several causes related to overzealous advocacy, immature technology, lack of corporate technology roadmaps, requirements instability, ineffective acquisition strategy, unrealistic program baselines, inadequate systems engineering, and work-force issues.[ii] This paper will discuss some key elements associated with:

• Establishing a process for requirements definition and developing the cost and schedule baseline
• Developing a reliable cost and schedule baseline,
• Identifying critical software management metrics,
• Applying Performance-Based Earned Value (PBEV), and
• Using an analytic process (such as SEER Control;) to project cost and schedule based on actual performance.

Read the rest of this entry »

Parametric cost estimation uses the statistical relationship existing between performance and physical factors associated with a project, and the actual historical costs incurred.  Performance and physical factors may include the fixed and variable costs associated with actual project performance, e.g. an engine fuel consumption and power output, contractor costs and value added or operational manning requirement needed. By combining predictive models to estimate what costs should be with real-world experience of what costs have been, a far more accurate estimate can be produced.

Parametric cost estimating uses a statistical regression analysis, and is most commonly utilized in the early stages of a project or product development; usually referred to as the System Development & Demonstration phase or SDD. It is widely used by government, military and industrial clients and their primary contractors, because whilst accurate costs are unknown due to a lack of detailed information (which only becomes available at the acquisition stage), the estimating solution is able to take into account top-level factors of design and performance.  In addition, the solution also provides risk analysis measures, for instance, probability of project success.

Effectively, parametric cost estimation is using two models: a forecasted projection and a historical model.  These are then combined to provide the project estimate.  It is therefore essential that the most current information is used in both models; i.e. the most recent historical cost information and the most up-to-date input and performance data. In addition, it is essential to compare like-with-like, so prior period estimations and data are standardized to take into account the time value of money, the impact of inflation in creating current pricing and the performance characteristics of the underlying technology.

An example would be a project requiring an estimate of the cost of a logistical fleet management solution, using historical information derived from a fleet management solution prior to the introduction of satellite tracking technology.  The cost estimate yielded will be a great deal higher than the real-world pricing, because the modern solution available fails to have the current cost-benefits of satellite tracking technology imputed into the estimation model.  A modern day example would be the estimation of a computer system using older cost and performance estimates to compare with.  Again, the cost estimates would be higher and the performance output estimates reduced, because of the advances in the underlying technology (computer speed, memory cost and increased functionality) bringing benefits and cost savings which would not be taken into account.

A major strength of parametric cost estimation is the ability to analyze how variable parameters affect cost and risk.  Cost Estimating Relationships (CERs) can be developed between the historical and forecast models, which can be manipulated to determine cost and performance changes if the underlying parameter is altered. Regression analysis is used to plot the CER’s impact, most frequently by using a “line of best fit” or in technical terms, the coefficient of determination.  This is why it is important that data collected to produce the CER is “normalized” in terms of time cost of money, but also in respect of performance and output characteristics of the underlying technology.  By varying the controlling parameter along the line of best fit, more accurate cost and performance estimates can be produced, provided the underlying CER database is operating within the real-world parameter values.  For instance, it would be inappropriate to generate a range of CERs for a family saloon car using NASCAR racing data, or to use CERs from gas-powered vehicles to generate cost estimates for hybrid fuel models.

In summary, parametric cost estimating provides significant advantages in cost estimating because it is using cost and performance data from different historical sources and from best-estimates of current data.  This generates a less risky estimate than one generated by analogy for instance.  Additionally, by using statistical techniques, the margin of error is better quantified which renders sensitivity analysis more meaningful.

Related articles

• IT Project Failure Warning Signs (galorath.com)

Packaged Applications – The Hidden Costs of Snake-Oil

By David DeWitt

Most of us remember the dubious “doctor” in Huckleberry Finn who proclaims to have the cure for any malady.  From the back of the crowd his shill would loudly proclaim that he himself had been healed by the magical elixir.   Oh my, but the quantity of snake oil they sold before the townsfolk discovered they had been duped – and the doctor long gone.   From the tone of many internet blogs it seems the same buyer’s remorse is lingering in the “Packaged Application” world.

Before we get into the discussion of packaged applications let’s take a step back and remember how these wonder cures manifested.   In the late 1980′s – (when disco was dying) manufacturing companies wanted to shape supply chain management and enterprise resource planning.  In short, they wanted to encourage the concept of a “process in a box.” (my quotes).  Somehow over the last ten years the industry has forgotten that part about the need for the organizational “process” to change; and as we shall see, that is a key cause of hidden costs.

To belabor the point of “snake oil” for just a moment longer – what exactly is a packaged application?

1.       You can’t see the ingredients (What is the quality of the code?)

2.       You don’t know the recipe (How was it developed?)

3.       You don’t know how it’s to be served (Is it designed for my architecture?)

4.       When done – you hope it won’t upset your stomach (Will it run on or corrupt my databases?)

But it heals the sick and will bring peace and joy to the world … if it’s configured just right!

The top ten hidden costs are – in no specific order:

Read the rest of this entry »