| GUIDANCE | ||||||||||||||||||||||||
| The Pricing Handbook
19. Software Pricing 19.5 Software Cost Estimating Methodologies
19.5 Software Cost Estimating Methodologies Once the basic system being estimated has been defined, planned and sized, the next step is to select an estimating methodology. There are five basic methodologies for estimating software.
Table 19-8 provides a brief description of these methodologies along with their advantages and limitations. Each of the methodologies will be defined and summarized below. An extensive appendix (Appendix 19D) is devoted to parametric software estimating models due to their frequent use in estimating software development costs. Table 19-8. Software Cost Estimating Methodologies
According to NASA’s Handbook, estimating by analogy is the act of comparing the proposed project to previously completed similar projects where project development information is known. Actual data from the completed projects are extrapolated to estimate the proposed project. Estimating by analogy can be done either at the system level or the component level. For example, consider the situation in which an agency wanted to develop a new payroll program that serves 5,000 people and contains 100,000 lines of COBOL code. A second agency had developed a similar 100,000-line program for two million dollars. It could be expected that the second agency's program, ignoring inflation, would also cost two million dollars. The advantage of the analogy method is that it is based on actual experience. However, it is limited because, in most instances, no truly similar software programs exist. For example, the cost of 100,000 lines of C++ code for a radar program would not be the same as that of 100,000 lines of COBOL code for payroll software. The main strength of this method is that the estimates are based on actual project data and past experience. Differences between completed projects and the proposed project can be identified and impacts can be estimated. One problem with this method is identifying those differences. This method is also limited because similar projects may not exist, or the accuracy of available historical data are suspect. The analogy or comparative technique uses parametric approaches including the use of CER's. The analogy methodology should not normally be used alone to estimate most software costs, except as a last resort, although it can be useful in checking other estimates for reasonability.
19.5.2 Grass Roots, Engineering Estimate or Bottoms-up Grass roots estimating, also called "engineering estimating" or "bottoms-up", provides a more sophisticated means of analyzing costs. It involves estimating software costs by a detailed analysis of the cost of each unit (or CSU), then summing unit costs to determine the cost (or effort) for each CSC, each CSCI, and then the software cost for the overall system. Grass roots estimating is sometimes used during proposal preparation and, subsequently, for tracking costs of software projects. An advantage of this method is that it provides a detailed basis for cost estimating and, consequently, tends to be more accurate and stable than other methodologies because it deals with low-level components. Also, since unit costs are assessed by personnel responsible for the development of the unit, this type of estimating is a good managerial tool for promoting individual responsibility for keeping units within the estimated costs. It supports project tracking more directly than other methods because its estimates usually address each activity within each phase of the software development life cycle. However, grass roots estimating has several disadvantages. Since detailed information about each unit is required, it may be difficult to use in certain circumstances, especially during the early phases of the life cycle where detailed information is often unavailable. Using grass roots techniques may not be appropriate until detailed design has been completed. Also, grass roots estimating is usually very time-consuming, which makes it inappropriate when either time or available estimating personnel are limited. For these reasons, grass roots estimating is infrequently used within Government agencies where time and personnel are usually scarce. Another limitation of grass roots estimating is that it does not automatically capture the costs associated with integrating units to form higher-level components and CSCIs. Integration costs must be separately estimated.
NASA’s Handbook summarizes the top-down method of estimation, which is based on overall characteristics of the software project. The project is partitioned into lower-level components and life cycle phases beginning at the highest level. This method is more applicable to early cost estimates when only global properties are known. Advantages include consideration of system-level activities (integration, documentation, project control, configuration management etc.), that may have been ignored in other estimating methods. The top-down method is usually faster, easier to implement and requires minimal project detail. However, it can be less accurate and tends to overlook lower-level components and possible technical problems. It also provides very little detail for justifying decisions or estimates.
The STSC Report and SSCAG Handbook summarize this method. Expert judgement uses the experience and knowledge of experts to estimate the cost of a software project. An advantage of this method is the experience from past projects that the expert brings to the proposed project. The expert also can factor in project impacts caused by new technologies, applications, and languages. Examples of popular expert judgment techniques include the Delphi and Wideband Delphi methods. Expert judgment techniques are suitable for assessing the differences between past and future programs; and are especially useful for new or unique programs for which no historical precedent exists. However, the expert's biases and sometimes insufficient knowledge may create difficulties. It can be hard to document the factors used by the expert who contributes to the estimate. Although Delphi techniques can help alleviate bias problems, experts are usually hard-pressed to accurately estimate the cost of a new software program. Therefore, while expert judgment models are useful in determining inputs to other types of models, they are not frequently used alone in software cost estimating.
19.5.5 Parametric Modeling or Algorithms Parametric modeling, as defined here, is a combination of algorithmic and top-down models as described by Boehm. The algorithmic method involves the use of equations to perform software estimates. The equations are based on research and historical data and use such inputs as SLOC, number of functions to perform, and other cost drivers such as language, design methodology, skill-levels, risk assessments etc. Parametric models and techniques for software projects generally estimate overall system or CSCI costs based on design characteristics, or parameters, of a software program. The overall costs can later be partitioned among lower-level components (CSCs) or among life cycle phases. Advantages of this method include being able to generate repeatable results, easily modifying input data, easily refining and customizing formulas, and better understanding of the overall estimating methods since the formulas can be analyzed. Also, they are usually fast and easy to use, require little detailed information, and capture system or CSCI-level costs. The disadvantages are that they tend to be less accurate and less stable than the grass roots methodology; and they do not inherently foster individual responsibility. The STSC Report adds that the results can be questionable when estimating future projects that use new technologies. Also, the formulas generally are unable to deal with conditions such as exceptional personnel, exceptional teamwork and exceptional matches between skill levels and tasks. Parametric modeling is usually the methodology of choice for Government agencies because of the minimal time and effort required and because it is useful in early phases of a software development program when little is known about the program. Also, attempts are continually being made to calibrate available models to insure greater accuracy. Because of their widespread use, parametric models will be discussed in general here and some currently popular software parametric cost models will be discussed in Appendix 19D. All parametric models use one or more design parameters to estimate cost or level of effort. The earliest parametric models, some of which are still in use, were simple models that estimated cost based only on the size of the software in SLOC, machine instructions, or other measures. They were often of the form E = AIB where E is the effort required in dollars, man-months, or similar measure; I is the size of the program; A is a coefficient; and B is an exponent. The coefficient and exponent are usually computed using mathematical regression analysis on historical size and effort data. This simplistic model generally shows considerable variance, even within the database from which the model was formulated, which is one reason why simple single-parameter parametric models, such as those based solely on program size, are not generally considered to be accurate. Another limitation of these models is that the coefficients and exponents vary significantly with the database used, indicating that the models are highly database dependent. This suggests that other factors, besides the size of the software program, affect the effort or cost required. Also, other methods besides regression may be used in model development. Popular parametric cost models are discussed in the Appendix 19D. |