Capability Maturity Levels and Implications to Software Estimating

An estimate is the most knowledgeable statement you can make at a particular point in time regarding cost/effort, schedule, staffing, risk, and the ...ilities of the product or service.[1]

Immature versus Mature Software Organizations [3]

Setting sensible goals for improving the software development processes requires understanding the difference between immature and mature organizations. In an immature organization, processes are generally improvised by practitioners and their management during the course of the project. Even if a process is specified, it must be rigorously followed and enforced.

Immature organizations are reactionary, with managers focused on solving immediate crises. Schedules and budgets are routinely exceeded because they are based on something other than realistic estimates. When hard deadlines are imposed, product functionality and quality are often compromised to meet the schedule.

Immature organizations have no objective basis for judging product quality or solving product or process problems. The result is that product quality is difficult to predict. Activities intended to enhance quality, such as reviews and testing, are often curtailed or eliminated when projects fall behind schedule.

Mature organizations possess guide the organization-wide ability to manage development and maintenance processes. The process is accurately communicated to existing staff and new employees, and work activities are carried out according to the planned process. The processes mandated are usable and consistent with how the work is done. These defined processes are updated when necessary, and improvements are developed through controlled pilot tests and/or cost-benefit analyses. Roles and responsibilities within the defined process are clear throughout the project and across the organization.

Let's look at the landscape of maturity in estimating the work for those providing the funding for the work.

1. Initial

Projects are small and short, and while important to the customer, they likely need to be more critical to the success of the business in terms of cost and schedule. 

• Informal or no estimating
• When there are estimates, they are manual, without any processes, and likely considered guesses.

The result of this level of maturity could be better forecasting of the cost and schedule of the planned work. And a surprise for those paying for the work.

2. Managed

Projects may be small, short, and important. Some form of estimating, either from experience or from the decomposition of the planned work, is used to make linear projects of future cost, schedule, and technical performance.

This past performance is usually not adjusted for past variances, just an average. The linear average usually doesn't consider changes in the demand for work, technical differences in the work, and other uncertainties in the future for that work.

This is the Flaw of Averages approach to estimating. The effort needed to decompose the work into the same-sized chunks is the basis of all good estimating processes. In the Space and Defense business, the 44-day rule is used to bound the duration of work. This answers how long you will wait before discovering you're late. For us, the answer is no more than one accounting period. In practice, project status - physical percent complete is done every Thursday afternoon.

3. Defined

There is an estimating process using recorded past performance and the statistical adjustments of that past performance. Reference Classes are used to model future performance from the past. Parametric estimates can be used with those reference classes or other estimating processes. Function Points are common in enterprise IT projects where interfaces to legacy systems, database topology, user interfaces, and transactions are the basis of the business processes. 

The notion that we've never done this before, so how can we estimate, begs the question do you have the right development team? This is a past performance issue. Why hire a team with no understanding of the problem and then ask them to estimate the cost of the solution? You would only hire a plumber to install a water system if she had done this before in some way.

4. Quantitatively Managed

We use measures, Metrics, and Architecture assessments - Design Structure Matrix is one we use - to construct a future model. External Databases are referenced to compare internal estimates with external experiences.

5. Optimized 

There is an estimating organization that supports development, starting with the proposal and continuing through project close out. Also, there is a risk management organization helping inform the estimates about possible undesirable outcomes in the future.

Resources

[1] Improving Estimate Maturity for More Successful Projects, SEER/Tracer Presentation, March 2010.

[2] Software Engineering Information Repository, Search Results on Estimates

[3] The Capability Maturity Model for Software