Software Estimation BlogInformation on Cost Estimation & Software Estimating

This is a blog post by none-other than Dan Galorath, the Chief Architect for the SEER-SEM software. He talks about the difference between the types of project management software and sheds some light on not only the various differences that exist, but also where the two different types of software can be useful.

One of the desired end results of any project is producing a seamless product. Said project could be a company manual, a piece of software, or an automobile, in the end the product should be perceived a single, cohesive entity. The entirety of your project should feel as if it were birthed whole and flawless, crafted with one brush in a single masterstroke.

Reality, however, has you painting a much different picture. Large projects take large groups of people. Large groups of people are broken down into several smaller teams. Even teams themselves can be further divided. Responsibilities trickle down through the ranks and deadlines are attached. Pieces of the project start to rely on each other as deadlines overlap and dependencies are built. A holdup in one team can literally delay an entire project. The larger the project, the more your single masterstroke starts to look a lot more like Chaos.

In an attempt to make things easier, many project managers turn towards some sort of project management software. The logic is fairly simple – obtain a single piece of software which can keep track of all the pieces of a project simultaneously. The ideal piece of software is busy monitoring deadlines and progress while doling out further responsibilities, all the while ensuring the desired end result.

Active Versus Passive Project Management

Problems arise when we think that all project management systems are created equal. The fact is that software systems can be broken down into two primary categories – passive and active. Passive systems reflect exactly what you put into them, in essence telling you what you already know, albeit in an organized and open manner. Active project management systems, often called estimation software, are able to make projections based on various algorithms and complex models. The difference between the two can be rather severe. If passive project management software reflects what you know, like a mirror, than active project management software reflects what you want to know, like a magic mirror.

Both types of software start at the same point, with the users supplying what is currently known. Both types then organize what is known and keep track of it. For large projects this typically takes into account a host of items such as individual employees, individual pieces of a project, and time allotments. Whether using passive or active project management software, the collection of this information creates an important foundation for any project.

The difference between active and passive software comes into play after this foundation is laid. With passive software, the user builds on that foundation, establishing milestones and to-do list items, creating the framework for the entire project based on the manager’s own understanding of his or her employees and their individual skill sets. The duty of then updating that software falls on either the project manager or users further down the pipeline. The software might take some proactive steps, such as emailing those with looming deadlines, but for the most part such project management software relied heavily on users to keep it updated.

With estimation software, however, the program itself starts to bear a fair amount of the management burden. Prime examples of good estimation software are able to calculate project requirements not just on the known quantities such as those involved, but also on the harder to measure areas like project complexity, available technologies, and the immeasurable factor: uncertainty. Active project management software thus builds a more complete idea of a project. From that idea, the software is able to extrapolate a mathematical understanding of a project. This grounding in math creates measurable and comparative results which are more easily justified than obtuse human guesswork.

The active aspect of this type of project management software comes from what the program can tell users about the future. Because of the amount of data implemented as a project’s foundation in these management tools, solid software estimation programs will account for time, effort, quality, and risk. From that, theoretical changes can be made within the software, reflecting the results without having to actually enact them. This grants project managers the ability to play with trial and error solutions without risking real life resources.

The need for project management software obviously differs based on the project at hand. For some projects, particularly those which have repeating and regular deadlines, a passive project manager might be ideal. For projects with multiple parts which are leading towards a single cohesive product, a more active piece of project management software might be in order. In such instances, much will obviously be gained from understanding the exact situation at hand. The determining factor is whether you want your resources checking in with a mirror which reflects what’s currently known, or if you want a mirror which can help you accomplish your goals by telling you what you don’t yet know.

By Robert P Hunt (Galorath Incorporated), Paul J. Solomon (Performance-Based Earned Value), and Dan Galorath (Galorath Incorporated)

Often, traditional earned value approaches do not deal sufficiently with the idiosyncrasies of software intensive programs.  However, successful management of software intensive programs can be achieved by focusing on establishing the requirements, developing a reliable baseline estimate for cost and schedule, selecting effective software metrics, applying Performance-Based Earned Value® (PBEV), and using analytic processes to project cost and schedule based on actual performance.

Introduction

The Department of Defense estimates that software now accounts for 40% of all research, development, test and evaluation (RDT&E) spending[i].  Software intensive projects that achieve their original cost and schedule projections are rare.  Many information technology projects have been declared a disaster area by commercial and government managers.  These projects have been too costly, too late, and often don’t work right.  Applying appropriate technical and management techniques can significantly improve the current situation.

Inaccurate estimates can threaten project success causing poor project implementations, shortcutting critical processes, and emergency staffing to recover schedule.   The lack of well defined project requirements and specifications may result in significant growth in cost and schedule.   Symptoms of this growth may include constantly changing project goals, frustration, customer dissatisfaction, cost overruns, missed schedules, and the failure of a project to meet its objectives.

PMI published an analysis of several government defense and intelligence agency large-scale acquisition programs that experienced significant cost and schedule growth.  This analysis shows that several critical factors need to be addressed in the pre-acquisition phase of the acquisition cycle.  The principal causes of growth on these large-scale programs can be traced to several causes related to overzealous advocacy, immature technology, lack of corporate technology roadmaps, requirements instability, ineffective acquisition strategy, unrealistic program baselines, inadequate systems engineering, and work-force issues.[ii] This paper will discuss some key elements associated with:

• Establishing a process for requirements definition and developing the cost and schedule baseline
• Developing a reliable cost and schedule baseline,
• Identifying critical software management metrics,
• Applying Performance-Based Earned Value (PBEV), and
• Using an analytic process (such as SEER Control;) to project cost and schedule based on actual performance.

Read the rest of this entry »

Parametric cost estimation uses the statistical relationship existing between performance and physical factors associated with a project, and the actual historical costs incurred.  Performance and physical factors may include the fixed and variable costs associated with actual project performance, e.g. an engine fuel consumption and power output, contractor costs and value added or operational manning requirement needed. By combining predictive models to estimate what costs should be with real-world experience of what costs have been, a far more accurate estimate can be produced.

Parametric cost estimating uses a statistical regression analysis, and is most commonly utilized in the early stages of a project or product development; usually referred to as the System Development & Demonstration phase or SDD. It is widely used by government, military and industrial clients and their primary contractors, because whilst accurate costs are unknown due to a lack of detailed information (which only becomes available at the acquisition stage), the estimating solution is able to take into account top-level factors of design and performance.  In addition, the solution also provides risk analysis measures, for instance, probability of project success.

Effectively, parametric cost estimation is using two models: a forecasted projection and a historical model.  These are then combined to provide the project estimate.  It is therefore essential that the most current information is used in both models; i.e. the most recent historical cost information and the most up-to-date input and performance data. In addition, it is essential to compare like-with-like, so prior period estimations and data are standardized to take into account the time value of money, the impact of inflation in creating current pricing and the performance characteristics of the underlying technology.

An example would be a project requiring an estimate of the cost of a logistical fleet management solution, using historical information derived from a fleet management solution prior to the introduction of satellite tracking technology.  The cost estimate yielded will be a great deal higher than the real-world pricing, because the modern solution available fails to have the current cost-benefits of satellite tracking technology imputed into the estimation model.  A modern day example would be the estimation of a computer system using older cost and performance estimates to compare with.  Again, the cost estimates would be higher and the performance output estimates reduced, because of the advances in the underlying technology (computer speed, memory cost and increased functionality) bringing benefits and cost savings which would not be taken into account.

A major strength of parametric cost estimation is the ability to analyze how variable parameters affect cost and risk.  Cost Estimating Relationships (CERs) can be developed between the historical and forecast models, which can be manipulated to determine cost and performance changes if the underlying parameter is altered. Regression analysis is used to plot the CER’s impact, most frequently by using a “line of best fit” or in technical terms, the coefficient of determination.  This is why it is important that data collected to produce the CER is “normalized” in terms of time cost of money, but also in respect of performance and output characteristics of the underlying technology.  By varying the controlling parameter along the line of best fit, more accurate cost and performance estimates can be produced, provided the underlying CER database is operating within the real-world parameter values.  For instance, it would be inappropriate to generate a range of CERs for a family saloon car using NASCAR racing data, or to use CERs from gas-powered vehicles to generate cost estimates for hybrid fuel models.

In summary, parametric cost estimating provides significant advantages in cost estimating because it is using cost and performance data from different historical sources and from best-estimates of current data.  This generates a less risky estimate than one generated by analogy for instance.  Additionally, by using statistical techniques, the margin of error is better quantified which renders sensitivity analysis more meaningful.

Related articles

• IT Project Failure Warning Signs (galorath.com)

Packaged Applications – The Hidden Costs of Snake-Oil

By David DeWitt

Most of us remember the dubious “doctor” in Huckleberry Finn who proclaims to have the cure for any malady.  From the back of the crowd his shill would loudly proclaim that he himself had been healed by the magical elixir.   Oh my, but the quantity of snake oil they sold before the townsfolk discovered they had been duped – and the doctor long gone.   From the tone of many internet blogs it seems the same buyer’s remorse is lingering in the “Packaged Application” world.

Before we get into the discussion of packaged applications let’s take a step back and remember how these wonder cures manifested.   In the late 1980′s – (when disco was dying) manufacturing companies wanted to shape supply chain management and enterprise resource planning.  In short, they wanted to encourage the concept of a “process in a box.” (my quotes).  Somehow over the last ten years the industry has forgotten that part about the need for the organizational “process” to change; and as we shall see, that is a key cause of hidden costs.

To belabor the point of “snake oil” for just a moment longer – what exactly is a packaged application?

1.       You can’t see the ingredients (What is the quality of the code?)

2.       You don’t know the recipe (How was it developed?)

3.       You don’t know how it’s to be served (Is it designed for my architecture?)

4.       When done – you hope it won’t upset your stomach (Will it run on or corrupt my databases?)

But it heals the sick and will bring peace and joy to the world … if it’s configured just right!

The top ten hidden costs are – in no specific order:

Read the rest of this entry »

There are two types of Project Managers out there, those who get estimation software, and those who think it’s a magical box which turns hard working developers into robots. Here’s a list of the most cruel and unusual questions posed by the latter.

5. Two hours of sleep is plenty, right?

4. How much will productivity increase if we institute a “2 Devs Enter, 1 Dev Leaves” policy?

3. How much does the cost for desk shackles affect our bottom line?

2. Yes, but what if we had a thousand monkeys?

1. Great, now how do we finish the project with half the people in half the time for twice the money?

Related articles

• Seven Habits of Highly Effective Parametric Model Use (galorath.com)
• Early Estimating Is Like Martial Arts (galorath.com)
• Estimating Best Practices (galorath.com)

So, you’ve checked over all of your initial data, and nothing seemed out of the ordinary. You’ve double checked with the client and you know exactly what they think that they want. And you’ve plugged all of that carefully measured data into your software estimation program of choice and hit the enter key.

Congratulations and welcome to the baseline!

You’re now at the moment where if nothing changes (they will) and everything goes according to plan (they won’t), this is exact estimate of cost, team-size, and time relating to your project.

It’s a sound idea to take a good look at the baseline estimate before releasing it either internally or to the client on the offhand chance that there was an actual mistake on the initial data collection. If you’ve worked on similar projects before, and you have previous baseline estimates, these can be helpful for a quick comparison.

If everything appears to be in order, then the fun begins – the collection of “What if”s that can skillfully whittle down a baseline estimate into a lean, productive, completed project that lands under-budget and ahead of time.

Related articles

• Early Estimating Is Like Martial Arts (galorath.com)
• React Faster and Better: Before the Attack (securosis.com)
• DoD Affordability: Implementing Directive For Ash Carter Initiative (galorath.com)

Hey, welcome to 2009! It’s taken a while, but we’re finally at the point where we can go ahead and make our first estimate.

But first, a recap of the first four steps in software estimation process.

• Understanding the problem – not just what you want to estimate, but why you want to estimate it.
• Know the rules – Estimates are bound by rules and measurements, and you need to know what you’re measuring before you rush off and start.
• Data Collection – Make use of organized work sheets so that you can keep everything organized. If you need to know what to put on said worksheet, look back to steps 1 and 2.
• Identify Work Already Done – Sometimes, it’s more cost effective to look elsewhere for solutions.

By the time you get here, to step number 5, you’ve already done a great deal of work regarding your estimate. You’ve pondered, you’ve measured, you scribbled down numbers, you’ve looked at the entirety of the problem and you’ve broken it down into measureable parts. Now, it’s time to plug it all into a machine.

If you haven’t already, it’s a good idea to have a familiar set of eyes look over your work before handing it off to the computer, just to weed out any potential for human error. If you’re working on a project for a client, show them the numbers too.

After that, it’s time to run your first software estimation, establishing the baseline from which all other estimates and changes will be measured.

Keeping up with the idea that we’re doing software estimating to create a software package, let’s go ahead and give a long look at our intended project. The first question you’re going to have to ask yourself is how big is this project going to be?

If this is code, how much is necessary? How much can be recycled?

If you’re writing new software, one of the key determining factors for all other estimates is going to be the sheer size of the project as measured in Source Lines of Code (SLOC). This sounds like an obvious place to start, but whether your under the influence of inspiration or the crush of deadlines, a project can easily bloat out of control.

Though you’re likely going to want to restrain your SLOC to the lowest number of lines possible, you’re going to want to give yourself a cushion in your initial SLOC estimate. One popular estimation method is to draw up three numbers, the least possible number of lines, the probable or likely number, and the outside largest number.

Once you know the likely sizes of your project, you can then start to whittle down the involved labor. Is there Commercial-off-the-shelf  (COTS) software that can be incorporated to help achieve the desired solution? Have you developed prior pieces of code, modules that can be incorporated? These pre-existing solutions will have an impact on your SLOC, but also the cost and time of the project, proving that while SLOC might grow or contract, the relationship between SLOC and the total cost of the software is not a solely dependent relationship.

Coming up with these answers ahead of time helps to prepare the overall software estimate by providing a more complete picture of the desired project. Understanding the source before you start it, and determining the best courses of action regarding COTS, FOSS, and pre-existing code solutions is a good method for creating shortcuts and cost savings well ahead of when they’ll actually be needed. Doing so is a lot like plotting a vacation on a map before you start the trip. Even though you have never been where you’re going, you’ll have a fleshed out idea of where you’re going, how long the trip is going to take, and whether anyone has been nice enough to already build bridges for you.

And, once you have that knowledge, you’re much better suited to answer the proverbial question, “Are we there yet?”

Better hope it's pointed exactly right

There’s a popular cliché amongst rocket scientists “a launch that is off by one degree will miss the moon by miles.” Okay, so maybe the authenticity of that phrase is leaves a lot to be desired, but the spirit of the statement – the accuracy of a rocket is only as sound as the accuracy of the initial trajectory – makes a lot of sense. Even a target as big as the moon can easily be missed if the foundation is shoddy.

How does this relate to software estimation? Easy, your estimate is built on data collection, and your initial data collection creates your foundation. As you collect your data, the results compile into the launch structure that will put your estimation rocket on the correct path.

Cheesy metaphors aside, there are some best practice tips for ensuring that your data collection is accurate and you don’t miss the moon.

Customized Worksheets are a must. A standard worksheet is good, as it allows for data to be collected in a uniform fashion. A custom worksheet allows for a more complete and tailored set of data to be collected. Doing so allows you to flesh out the project more so before hand.

Break the worksheet down into components of the project. If you’re working on a large piece of software, this means a separate worksheet for each CSCI. For other project estimates, say for manufacturing, you’ll need to make worksheets for each step of the fabrication process.

Make sure your collected data jives. Double check your numbers. Make sure the vision of your estimate is on par with what the customer envisioned. Check to make sure that any necessary resources (contractors, developers, etc) are available.

Once you have these issues sorted out, you should have a nice foundation for your project. You have the forest and you can move to sketching out the trees.

Related articles

• Data Driven Estimating Part 1: How Data Feeds The Estimation Modeling Paradigm at Galorath / SEER (galorath.com)
• FCC Investigating Google Street View Wi-Fi Data Collection (yro.slashdot.org)
• New Study by Krux Digital Shows Data Leakage on Popular Websites (drdianehamilton.wordpress.com)

It helps with the metaphor

If you recall, last week I talked about the first step that anyone doing anything with software estimation needs to do – that is to establish the ‘what’ and the ‘why’. And I said that these two aspects are your cornerstone, the start of your foundation. Now it’s time to move to step two and lay the rest of that foundation. Don’t worry, I’m not going to continue with home construction metaphor much longer, it’s just that for this instance it’s fitting.

So, now that we know what we want to estimate, and why we want to estimate it, it’s to start getting to the ‘how’. To do so, we’re going to take that simple question and break it down into three main areas.

The first of these three areas is the technical baseline, which as the name suggests, is where everything is measured from. To make your baseline perform best, and the rest of your estimation follows suit, it’s best to follow the kitchen sink approach. Working with software? Incorporate how each piece of the software interacts with each other piece and how that software would integrate with the outside world. Toss in every binary build, every delivery time. When every step is due, and how much they’re supposed to cost.

Then, move to the second part of the ‘how’ – establish your ground rules. Special considerations, technology involved, everything have to be done in blue? Yeah, those go here.

Which brings us to the last part, assumptions. Go ahead and forget the cliché your mother used to repeat, you know that that so cleverly starts “Every time you assume, you…” Because now I want you to do the opposite. Kinda. Here’s where you’ll make a list of things you’ll need to complete the project, but that you don’t already have. So, you’re not making assumptions about performance (as your mother so warned)but rather assumptions about supplies (which I’m telling you is cool.)

Related articles

• Data Center Consolidation: Avoiding Surprises and Returning Business Value (galorath.com)
• Data Driven Estimating Part 1: How Data Feeds The Estimation Modeling Paradigm at Galorath / SEER (galorath.com)
• Estimating Best Practices (galorath.com)
• Seven Habits of Highly Effective Parametric Model Use (galorath.com)