Software Estimation BlogInformation on Cost Estimation & Software Estimating

By Robert P Hunt (Galorath Incorporated), Paul J. Solomon (Performance-Based Earned Value), and Dan Galorath (Galorath Incorporated)

Often, traditional earned value approaches do not deal sufficiently with the idiosyncrasies of software intensive programs.  However, successful management of software intensive programs can be achieved by focusing on establishing the requirements, developing a reliable baseline estimate for cost and schedule, selecting effective software metrics, applying Performance-Based Earned Value® (PBEV), and using analytic processes to project cost and schedule based on actual performance.

Introduction

The Department of Defense estimates that software now accounts for 40% of all research, development, test and evaluation (RDT&E) spending[i].  Software intensive projects that achieve their original cost and schedule projections are rare.  Many information technology projects have been declared a disaster area by commercial and government managers.  These projects have been too costly, too late, and often don’t work right.  Applying appropriate technical and management techniques can significantly improve the current situation.

Inaccurate estimates can threaten project success causing poor project implementations, shortcutting critical processes, and emergency staffing to recover schedule.   The lack of well defined project requirements and specifications may result in significant growth in cost and schedule.   Symptoms of this growth may include constantly changing project goals, frustration, customer dissatisfaction, cost overruns, missed schedules, and the failure of a project to meet its objectives.

PMI published an analysis of several government defense and intelligence agency large-scale acquisition programs that experienced significant cost and schedule growth.  This analysis shows that several critical factors need to be addressed in the pre-acquisition phase of the acquisition cycle.  The principal causes of growth on these large-scale programs can be traced to several causes related to overzealous advocacy, immature technology, lack of corporate technology roadmaps, requirements instability, ineffective acquisition strategy, unrealistic program baselines, inadequate systems engineering, and work-force issues.[ii] This paper will discuss some key elements associated with:

• Establishing a process for requirements definition and developing the cost and schedule baseline
• Developing a reliable cost and schedule baseline,
• Identifying critical software management metrics,
• Applying Performance-Based Earned Value (PBEV), and
• Using an analytic process (such as SEER Control;) to project cost and schedule based on actual performance.

Read the rest of this entry »

So, you’ve checked over all of your initial data, and nothing seemed out of the ordinary. You’ve double checked with the client and you know exactly what they think that they want. And you’ve plugged all of that carefully measured data into your software estimation program of choice and hit the enter key.

Congratulations and welcome to the baseline!

You’re now at the moment where if nothing changes (they will) and everything goes according to plan (they won’t), this is exact estimate of cost, team-size, and time relating to your project.

It’s a sound idea to take a good look at the baseline estimate before releasing it either internally or to the client on the offhand chance that there was an actual mistake on the initial data collection. If you’ve worked on similar projects before, and you have previous baseline estimates, these can be helpful for a quick comparison.

If everything appears to be in order, then the fun begins – the collection of “What if”s that can skillfully whittle down a baseline estimate into a lean, productive, completed project that lands under-budget and ahead of time.

Keeping up with the idea that we’re doing software estimating to create a software package, let’s go ahead and give a long look at our intended project. The first question you’re going to have to ask yourself is how big is this project going to be?

If this is code, how much is necessary? How much can be recycled?

If you’re writing new software, one of the key determining factors for all other estimates is going to be the sheer size of the project as measured in Source Lines of Code (SLOC). This sounds like an obvious place to start, but whether your under the influence of inspiration or the crush of deadlines, a project can easily bloat out of control.

Though you’re likely going to want to restrain your SLOC to the lowest number of lines possible, you’re going to want to give yourself a cushion in your initial SLOC estimate. One popular estimation method is to draw up three numbers, the least possible number of lines, the probable or likely number, and the outside largest number.

Once you know the likely sizes of your project, you can then start to whittle down the involved labor. Is there Commercial-off-the-shelf  (COTS) software that can be incorporated to help achieve the desired solution? Have you developed prior pieces of code, modules that can be incorporated? These pre-existing solutions will have an impact on your SLOC, but also the cost and time of the project, proving that while SLOC might grow or contract, the relationship between SLOC and the total cost of the software is not a solely dependent relationship.

Coming up with these answers ahead of time helps to prepare the overall software estimate by providing a more complete picture of the desired project. Understanding the source before you start it, and determining the best courses of action regarding COTS, FOSS, and pre-existing code solutions is a good method for creating shortcuts and cost savings well ahead of when they’ll actually be needed. Doing so is a lot like plotting a vacation on a map before you start the trip. Even though you have never been where you’re going, you’ll have a fleshed out idea of where you’re going, how long the trip is going to take, and whether anyone has been nice enough to already build bridges for you.

And, once you have that knowledge, you’re much better suited to answer the proverbial question, “Are we there yet?”

Better hope it's pointed exactly right

There’s a popular cliché amongst rocket scientists “a launch that is off by one degree will miss the moon by miles.” Okay, so maybe the authenticity of that phrase is leaves a lot to be desired, but the spirit of the statement – the accuracy of a rocket is only as sound as the accuracy of the initial trajectory – makes a lot of sense. Even a target as big as the moon can easily be missed if the foundation is shoddy.

How does this relate to software estimation? Easy, your estimate is built on data collection, and your initial data collection creates your foundation. As you collect your data, the results compile into the launch structure that will put your estimation rocket on the correct path.

Cheesy metaphors aside, there are some best practice tips for ensuring that your data collection is accurate and you don’t miss the moon.

Customized Worksheets are a must. A standard worksheet is good, as it allows for data to be collected in a uniform fashion. A custom worksheet allows for a more complete and tailored set of data to be collected. Doing so allows you to flesh out the project more so before hand.

Break the worksheet down into components of the project. If you’re working on a large piece of software, this means a separate worksheet for each CSCI. For other project estimates, say for manufacturing, you’ll need to make worksheets for each step of the fabrication process.

Make sure your collected data jives. Double check your numbers. Make sure the vision of your estimate is on par with what the customer envisioned. Check to make sure that any necessary resources (contractors, developers, etc) are available.

Once you have these issues sorted out, you should have a nice foundation for your project. You have the forest and you can move to sketching out the trees.

It's the one in the lower corner

There’s a running gag between myself and a few of my friends and associates – if possible, how much information would you cede over to software estimation? Theoretically, software that was smart enough, and fed enough information could serve as a nice expert opinion, specifically when involved in an argument with a significant other. After all, it’s hard to refute a computer that gives a quantifiable reason against turning the television away from the game, or not wearing that specific tie. Granted, the battle between spouse and computer would be settled quickly (with the edge going to the spouse), but the comic examples do serve an actual purpose – the first rule of software estimation is you need to know what you’re estimating.

As good as software estimation is (and it can be really good), you need to know what you wish to estimate before you start feeding in data. To that end, it helps to establish an estimate scope, purpose, and starting point for your estimate.

The first part, the purpose, is perhaps the most important. What do you wish your estimation to do? The more specific you are in your purpose, the better prepared you’ll be when you move through the process and more useful the estimate will be. Scope’s pretty important. That’s the general goal of your estimate – will this estimate be the completion of your project, or will the estimate impact how your project proceeds? And finally, before you even begin your estimate, you need to establish your base points of contact. This is the general areas where you’ll be culling your data from.

It takes a while to really get the hang of these three areas, but they’re important for creating the foundation of a solid estimate. Once you’ve got them down, your estimates can work better for you, and maybe, just maybe, you can software estimation to settle household disputes and convince the significant other that watching the game is the best decision possible.

Here lies the difficult question. How does software estimation actually work? Conceptually speaking its simple, in practice however, that’s where things become more difficult. I’m sure some of you have a book somewhere by a fellow named Steve McConnel. The book, Software Estimation, carries the subtitle “Demystifying the Black Art” which paints software estimation in a light that it doesn’t really deserve, addressing software estimations reputation as being complex and uncertain.

From a conceptual basis, and certainly when done right, software estimation is not complex at all. Why? Because the computer does all the heavy lifting. The human aspect of software estimation largely relies on regular data entry, supplying the software with as complete a data set as possible.
Why would you need a large data set? That reasoning is the difference between a census and a sample. A sample survey asks a set of questions from a portion of a population, and then extrapolates the results of that survey to make estimates regarding the population at large. A census survey asks a set of questions to the entire population, removing the need to extrapolate.

So, once you plug in all of the relevant information for whatever you wish to estimate, that’s when the computer goes to work, and here’s McConnel’s ‘black art’ stuff. Depending on what you wish to solve, the software estimation tool of your choosing is going to rely on any number of technologies including, but not limited to, parametric algorithms, simulation-based probability techniques, grouping, and historical conclusions. And, of course, each software estimation tool uses different combinations to perform estimations.