| GUIDANCE | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| The Pricing Handbook
19. Software Pricing
Software size is a key input to most software cost estimating methodologies. However, it is not always easy to determine size, especially during the early stages of a program. Currently, there are numerous tools available for size estimation; many of which are included in, or related to, currently available software cost models. Current software cost models usually accept input from more than one sizing approach. An SEI requisite is that there needs to be a structured approach for estimating size and the amount and complexity of existing software that can be reused. This section will describe the various methods for software size estimating, including reused and COTS software. The Cost Xpert User’s Manual recommends that for small projects (generally, less than $50K) typically either a bottoms-up or top down estimate is used to generate a size estimate. For medium sized projects ($50K to $1M), probably one of the metric based approaches should be employed (lines of code, function points, or object metrics). Analysts can supplement this estimate with one of the two heuristic approaches (bottoms-up or top down) in order to validate the numbers. For larger projects, it is recommended to use two or more of the metric based approaches and correlate the results.
19.4.1 Heuristic Approaches ("Rule of Thumb") Although this section will concentrate on metric based approaches to size estimation, the two heuristic approaches will be briefly discussed first for completeness. The Cost Xpert User’s Manual is referred to for the remainder of the Heuristic subsection. Heuristic approaches to estimating the program size rely on the fact that the same basic activities will be required for a typical software development project and that these activities will require a predictable percentage of the overall effort. For example, traditional projects typically include activities that can be categorized as requirements analysis, design, coding, testing, and documentation. The percentage of effort required for these activities is predictable, varying based on the type of project and the size of the project. This knowledge is the basis of heuristic based estimating, whereas metric based approaches rely on a correlation between the metric being employed and the ultimate effort involved in building the software. Bottoms-Up For the bottoms-up approach, divide the program into modules. For each module, directly estimate the level of effort required in terms of person months or person hours. This provides the estimated total effort for the coding portion of the development. Quite often, this portion of the estimate relies heavily on input from the actual programming team. Then look at the other life cycle activities that are required. This list of activities will vary based on the life cycle standard being used during this development effort, plus there may be some project specific activities. For each of these activities, estimate the effort required. Finally, compare the percentages of effort for each activity in the project plan with standard percentages. If the percentages roughly match the standard percentages, then there is a relatively high degree of confidence that the allocation of effort between activities is reasonable. If the percentages deviate from the standard percentages significantly, this may indicate an unreasonable amount of effort, either too high or too low, for one or more activities. In summary for bottoms-up cost estimating, start with a list of all required activities, estimate the effort for each, compare these estimates with typical percentages, then roll the costs up to a total project cost. Top Down For top down estimating, begin with an estimate of the total effort for the entire project. Then use typical percentages for each life cycle activity to calculate out the effort for each of those activities. Then compare the predicted effort for each activity with the required work to determine if the numbers seem reasonable. If not, the total effort can be adjusted up or down as required.
Three currently popular-metric based approaches for estimating software size are source lines of code (SLOC), function points and object points. The Cost Xpert User’s Manual provides a good summary of the strengths for each of the different approaches for metric based size estimates. Lines of code are particularly valuable for real time and embedded systems with little user interface. This was the original metric based approach and was popularized by COCOMO. The basic COCOMO model has been incorporated into literally dozens of costing tools, and remains the most common approach. Function points were introduced by IBM in the early 1980s. This approach employs user interface features to estimate program size. It is the most common technique for estimating management information system (MIS) application size. Object metrics have become feasible only with the popularization of object oriented development and use objects as a predictor of program size. For these metric based approaches, there are several estimating models and techniques available. A summary of software sizing approaches with appropriate examples follows. Source Lines of Code (SLOC) Per STSC’s Report, estimating SLOC is the traditional and most popular technique used to estimate the size of a software project. The appeal of estimating SLOC is that it is a simple technique, at least in concept, and often is automated. However, because determining which lines of code to count can be ambiguous, rules are needed for which types of lines to count in a programming language. Two of the difficulties with using SLOC as an estimate in project planning are:
Boehm in his briefing "An Overview of the COCOMO.II Software Cost Model" stated that defining a line of code is difficult due to conceptual differences involved in accounting for executable statements and data declarations in different languages. To minimize these problems, the SEI definition checklist for a logical source statement was used in COCOMO.II for defining the line of code measure. (COCOMO.II refers to version 2.0 released in 1997.) Figure 19-8 shows a portion of the definition checklist as it is being applied to support the development of the COCOMO.II model. Each check mark in the "Includes" column identifies a particular statement type or attribute included in the definition, and vice-versa for the excludes. Other sections in the definition clarify statement attributes for usage, delivery, functionality, replications and development status.
Figure 19-8. Source Lines of Code Sizing
Some examples of SLOC estimating models in each model category are summarized in the following sections. Analogy Models These models estimate program size by comparison with one or more similar programs. Currently popular are database analogy models, such as the Aerospace Software Size Estimator as discussed by Marilee J. Wheaton in her article on "Functional Software Sizing Methodology". This model contains over 1,000 historical programs, primarily in the military space, avionics, and ground-based environments. The user can specify his or her specific application category and obtain a list of historical projects with sizes and languages. The user can further specify a subset of an application, such as mission control. This will result in fewer similar programs, but the programs will be more relevant. Analogy sizing models possess the advantage of using historical data for comparison. They are also useful very early in a program. However, they are of limited use for any programs outside of the historical database. For example, the Aerospace model would be ineffectual for business applications. Functionality (Grass Roots) Models These models compute SLOC from components of known size from a database. If the user knows what functions a program will perform, they can obtain a size estimate for each function and sum the function estimates to estimate the size of a program. The Software Architecture, Sizing, and Estimating Tool (SASET), as discussed in the SASET 3.0 User's Guide by Robert W. Ratliff, et. al., allows size to be input based on functionality. The user specifies the functions performed by the program, and the model obtains a SLOC estimate for each function from the database. For example, if the user specifies a telemetry processing function, the model provides an estimate of 5,950 HOL statements. The Quantitative Software Management (QSM) Size Planner, defined in QSM Corporation’s Size Planner User's Manual, has a "standard components" option which also estimates the size of each function specified, primarily for business applications. For example, if the user specifies a screen function, the model provides an estimate of 2,535 lines of COBOL code. Like analogy models, functionality sizing models estimate size based on historical data. In contrast to traditional bottoms-up models, they are often useful early in a program since many functions are defined during software requirements analysis or preliminary design. However, like analogy models, these models are database dependent; similar functions with historical sizes must be available. A further limitation of models such as SASET is that the language is not specified. This can be problematic since there are significant differences among HOLs. Expert Judgment Models These models estimate SLOC based on the opinions of one or more experts. A simple example of an expert judgment technique is a Program Evaluation and Review Technique (PERT) for sizing used in several cost models. A more refined example is the SEER Software Sizing Model as discussed in the SEER-SSM User’s Guide, which is based on an expert judgment model developed by George Bozoki. In SEER-SSM, the user specifies at least one "reference" program, then performs a number of "structured judgments" to determine the size of the new program(s). The user performs a pair-wise comparison of program sizes, inputs a PERT estimate for all new programs, selects the most probable size range from a set of ranges generated by the model, and performs a ranking to verify the pair-wise comparison made initially. In the sizing arena, expert judgment models are either used alone or as aids in determining inputs for other categories of models. Expert judgment models have a disadvantage in that the views of experts are often biased. Despite their subjectivity, however, expert judgment models are often the only choice for new or unique situations. Parametric Models These models use inputs consisting of numerical or descriptive values to compute program size in SLOC. They are developed using regression analysis and other methods, such as numerical analysis or management surveys. One subset of parametric sizing models are regression-based models. These are similar to database analogy models in that historical data are used in model development. Instead of using direct comparisons, however, these models use equations, sometimes called size estimating relationships (SERs) developed from regression analysis of the historical data. Minoru Ikatura and Akio Takayanagi, in their paper "A Model For Estimating Program Size and Its Evaluation" developed a regression-based SER for business programs which has the necessary high correlation for a valid equation. While regression-based parametric sizing models transcend the need for direct comparison, they are usually limited to the scope of the database used. Furthermore, as illustrated by Mark J. Whetstone’s research and later by Frank Albanese Jr., it is often difficult to obtain the requisite high correlation for model validity. The Ikatura and Takayanagi SER is one of the few successful efforts. Another example of a parametric sizing model is the PRICE-S Sizer, which is included in the PRICE-S Reference Manual. The PRICE-S Sizer was not developed solely from regression analysis; other techniques were also used. This model has multiple inputs plus a size calibration factor, SICAL, which can be used to adapt the model to the user's environment. Parametric sizing models generally possess the advantages noted for the PRICE-S Sizer model and for parametric models in general. They are fast and easy to use and are generally useful early in a program, but may be challenging to use for new or unique programs. Function Point Function point is a method of estimating size during the requirements phase based on the functionality to be built into the system. Referring to the STSC Report, using function points as a basis for size estimates grew out of the difficulties with SLOC mentioned above. Function points are derived by:
The FPC is independent of the programming languages used on the project. Tables of source statements per function point have been developed through research for most programming languages and major dialects. This technique, known as Function Point Analysis (FPA), grew out of estimating the size of Information System (IS) software development projects. FPA has proven useful because:
Criticisms of using FPA are:
Function point sizing models were popularized by the extensive research of A. J. Albrecht and J. E. Gaffney for data processing programs written in COBOL and PL/1 programming languages as documented in their article "Software Function, Source Lines of Code, and Development Effort Prediction: A Software Science Validation." They are parametric models in that they use design parameters to estimate size; however, the parameters were not developed from regression analysis. Instead, Albrecht hypothesized that five program functions; inputs, outputs, inquiries, interfaces, and master files; could be used to estimate size. The researchers performed function point analysis on more than thirty data processing programs and concluded that function points are not only a valid predictor of software size, but were also superior to SLOC as a predictor of software development cost or effort. The CHECKPOINT model, as discussed in Software Productivity Research, Inc.’s CHECKPOINT For Windows User's Guide, employs function points as a primary input for effort and schedule estimation. Other models such as PRICE-S, Cost Xpert and QSM Corporation’s Software Life Cycle Model (SLIM) allow function points as an alternative to SLOC for program size. A brief definition of the five attributes used for function points, as described by Donald J. Reifer in the ASSET-R Version 2.0 User's Manual, is shown in Table 19-5. Table 19-5. Function Point Attributes
CASE STUDY 19-1, extended from SSCAG’s Software Methodology Handbook shows how traditional function points (sometimes called Albrecht function points) are computed. The user must determine the number of external inputs (EI), external outputs (EO), external inquiries (EQ), internal files (ILF), and external interfaces (EIF) in the program. This will give a measure of "basic" function points. The user can refine this measure by considering the complexity level of each function point and fourteen complexity adjustments factors related to the overall program, as shown in CASE STUDY 19-1.
CASE STUDY 19-1. FUNCTION POINT SIZING EQUATIONS
A marked strength of function points is the excellent results obtained from the research of Albrecht and Gaffney. Ongoing research is being performed by the International Function Points User's Group (IFPUG), which meets twice a year and periodically publishes a guide to counting and using function points. Proponents of function points models also state that function point counts can be made early in a program; during requirements analysis or preliminary design. Another strength, according to Capers Jones, is that they give a more realistic measure of productivity; SLOC-per-person-month measures tend to penalize HOLs. However, function points have disadvantages compared to SLOC as a measure of size. They are often harder to visualize since a SLOC can be seen (e.g., on a code listing) while a function point is only a concept. Furthermore, function points have only been extensively studied for business or data processing applications, although attempts to adapt the function point concept to real-time and scientific environments have been made. Some function points issues are variants, real-time adaptations, and function point-to-SLOC conversion. Function Point Variants There have been several attempts to modify the traditional or IFPUG function points shown in CASE STUDY 19-1. In the CHECKPOINT model, for example, the fourteen complexity adjustment factors are not normally used (although the user has the option of using them). Instead the user rates the overall complexity of the problem, code, and documentation from "l " to "5", and these complexities are used to compute adjusted function points. The function points used in the QSM Size Planner also do not use complexity adjustment factors; however, they use five attribute ranges instead of three. In SEER-SEM, the user can use either IFPUG or Galorath function points instead of SLOC. Galorath function points add a sixth attribute, internal functions, to the five attributes used in traditional function points. Probably the most radical departure from traditional function points is Symon's Mark II Function Points. Mark II function points use only three attributes; inputs, outputs, and entities. An entity is any item about which the system provides information. Mark II function points were standardized in England. Real Time Adaptation There have been several attempts to adapt function points to scientific and real-time programs. Some of these endeavors are now briefly described. Capers Jones' Feature Points For some time, function point practitioners recognized that some classes of applications did not profit from function point analysis as much as they hoped. These application classes included real-time process control, mathematical optimization and various embedded systems. Feature points add a sixth attribute, algorithms, to the five used in traditional function points. According to Jones, an algorithm is a "set of rules which must be completely expressed in order to solve a computational problem". The basic feature points (BFP) equation is: BFP = 3AL + 4EI + 5EO + 4EQ + 7ILF + 7 EIF. Note that, compared to the basic function points equation in Case Study 19-1, the coefficient for ILF has been decremented by three; all others remain the same. The CHECKPOINT model includes a complexity adjustment for algorithms. ASSET-R Function Points The ASSET-R model uses three additional attributes to those used in traditional function points: operating modes, rendezvous, and stimulus/response relationships. Operating modes are time-dependent end-to-end processing flows to which software performance can be related. Rendezvous function points are a measure of concurrency in real-time systems; and stimulus/response relationships measure the amount of sequencing and control in real-time systems. For scientific programs, ASSET-R only adds operating modes to the five additional attributes. Unlike traditional function points and feature points, ASSET-R does not weigh attributes differently; it assigns a coefficient of "l" to all attributes and uses other inputs to adjust the total for complexity. The ASSET-R manual shows that, for a sample of 34 data processing, scientific, and real-time programs, the model was accurate to within 20% of actual SLOC values based on software requirements specification data. Real-time SLOC from Externals Gaffney and Wehling studied the feasibility of estimating SLOC for real-time programs with the four "external" attributes of traditional function points: EI, EO, EQ, and EIF. They also studied this issue with only three of the external attributes: EI, EO, and EQ. Their sample involved nineteen military real-time programs, mostly written in FORTRAN. The resultant equations relating "externals" to SLOC showed a correlation of .89 when four externals were used, and .87 when three externals were used. According to the "Function Point FAQ" (Frequently Asked Questions) homepage, when the function point technique was originally developed, it was not applied to systems with Graphical User Interfaces (GUIs), since there were no GUIs. In 1995, IFPUG completed Release 4.0 of their Function Point Counting Practices Manual. It contains the official rules for, and extensive examples of, counting GUI based systems. Subsequently, IFPUG produced a case study that contained the count of an entire GUI-based system. Function points can be used to measure object oriented development applications and their size is the same as if object oriented development had not been used. This is because the function point count for an application is independent of the technology used to develop the application. Function Point-to-SLOC Conversion Referring to the SSCAG Handbook, it is sometimes necessary to convert from SLOC to function points, or vice-versa. For instance, consider that a user of SLIM only knows the number of function points in his or her program but does not know SLOC. According to QSM Corp’s SLIM 3.0 for Windows User Manual, since the SLIM algorithms are based on SLOC that user must specify a ratio or "gearing factor", to convert the number of function points to the number of SLOC. The opposite situation occurs in CHECKPOINT where SLOC inputs must be converted to function points. To help in this conversion, sets of SLOC-to-function point ratios have been developed for various languages based on research by Jones, Reifer, and Dan Galorath and Karen McRitchie in their paper "Software Size Measures: Source Lines of Code and Functions." Jones also specifies language levels that, in effect, show the number of equivalent assembler language SLOC generated by one SLOC of the specified language. Jones further states that language levels are useful for converting size from one language to another, and for assessing relative language productivities (although the relationship between language level and productivity is not linear). While function point-to-SLOC conversion ratios are useful, and sometimes necessary, they must be used with caution. While the researchers cited above agree on the ratios for some languages such as Ada, they differ, sometimes significantly, on the ratios for others such as Pascal and PL/l. For the COBOL language, research by Garland S. Henderson, in his thesis The Application of Function Points to Predict Source Lines of Code for Software Development showed that ratios are inconsistent both within and among databases. For two databases maintained at the Air Force Standard Systems Center, Henderson found that, for the military database of 26 COBOL programs, the average SLOC-to-function point ratio was 14:1, while for a database of 31 commercial COBOL programs, the average SLOC-to-function point ratio was 165:1. Furthermore, there was considerable variance for these ratios within the databases, and the results differ significantly with the ratios developed by Jones, Reifer, and Galorath. For some languages, therefore, it appears that ratios should not be used and, for cost estimation, it may be best to use a model for which the algorithms are based on the user's size measure. Table 19-6 compares the FP to SLOC ratios for some sample languages.
Table 19-6. Function Point to SLOC Conversion
Object Points With the advent of Object Oriented (OO) programming/development (Software programs are built from prefabricated, pretested building blocks (objects) combined together to produce a fully written software application. The idea of OO programming is to make the writing of complex computer software easier.), there has been a growing demand for new ways to measure software size in an OO environment. Per the Cost Xpert User’s Manual, object metric based estimating is seeing an increased following among companies using object oriented techniques throughout the software life cycle (e.g., object oriented analysis, design, and programming). Although still relatively new, it was applied on a massive scale within Hewlett Packard with great success. One very nice thing about the object metric approach is that it works equally well with both MIS type applications and with the embedded, scientific, and system level programming projects that are not well served by function points. Of course, a disadvantage is that it only works if the software development is done using object oriented techniques. According to the STSC Report, objects typically are screens, reports, and third generation modules. (By definition, they are not directly related to "objects" in an object-oriented methodology.) Getting the Object Point Count is very similar to getting the FPC described above; adjustments are made to the raw count for complexity and summed to get a final count. The advantage of counting object points is that it usually takes less effort than function points and the results are comparable. CASE STUDY 19-2 (extended from the "COCOMO.II Software Cost Estimation Model" abstract) presents the baseline COCOMO.II Object Point procedure for estimating the effort involved in prototyping projects. Definitions of terms are provided in Table 19-7.
Table 19-7. Terms
Note that the use of the term "object" in "Object Points" defines screens, reports, and 3GL modules as objects. This may or may not have any relationship to other definitions of "objects", such as those possessing features such as class affiliation, inheritance, encapsulation, message passing, and so forth. Different counting rules for "objects" of that nature are required when used in languages such as C++.
CASE STUDY 19-2. Steps for Object Point Estimate
According to NRaD’s Manual reused code is considered in most current software cost models. The only way to estimate the amount of code to be reused is for the programmer/analysts to examine the existing code in detail. The examination should derive estimates for three factors to help estimate the effort: 1) the percentage of code that will be redesigned, 2) percentage to be recoded or modified, and 3) percentage to be retested. Cost models use these estimated percentages to arrive at an equivalent line of code estimate. The equivalent lines of code estimate represents the amount of new code that could be developed for the same effort it would take to modify the existing code. Some models allow the user to enter these three percentages while other models assume a standard percentage of effort to develop equivalent lines of new code. The REVIC and SEER models allow the user to enter the three factors. SASET uses a default of 73%, i.e., it assumes any reused code will consume 73% of the effort that it would take to develop equivalent new code. When the three percentages are used, the equivalent new lines of code is calculated as follows: ([%Redesign + %Recode + %Retest]/3) * Existing Code = Equiv LOCs For example, equivalent LOCs for a program that had 10,000 LOCs and was assumed to require 40% redesign, 50% recode, and 60% retest would be calculated as: ([40% + 50% + 60%]/3) * 10,000 = 5,000 Equiv. LOCs According to the "COCOMO.II Software Cost Estimation Model" abstract, the COCOMO.II model uses a nonlinear estimation model for estimating size in reusing software products. Analysis of reuse costs across nearly 3000 reused modules in the NASA Software Engineering Laboratory indicates that the reuse cost function is nonlinear in two significant ways. It does not go through the origin. There is generally a cost of about 5% for assessing, selecting, and assimilating the reusable component. Small modifications generate disproportionately large costs. This is primarily due to two factors: the cost of understanding the software to be modified, and the relative cost of interface checking. The results of a study conducted by Parikh and Zvegintzov indicates that 47% of the effort in software maintenance involves understanding the software to be modified. Thus, as soon as one goes from unmodified (black-box) reuse to modified-software (white-box) reuse, one encounters this software understanding penalty. Also, research by Gerlich and Denskat shows that, if one modifies k out of m software modules, the number N of module interface checks required is N = k * (m-k) + k * (k-1)/2. The size of both the software understanding penalty and the module interface checking penalty can be reduced by good software structuring. Modular, hierarchical structuring can reduce the number of interfaces that need checking. Software that is well structured, explained, and related to its mission will be easier to understand.
Software developers are using increasing amounts of commercial-off-the-shelf (COTS) components to construct new systems. Their motivation is to decrease development costs, schedule and technical risks. Richard D. Stutzke’s paper "Cost Factors for COTS Integration" refers to a study done comparing how three different models estimate the size and cost of COTS software. Two models, COCOMO.II and SEER-SEM employ algorithms similar to those used to estimate the costs of reusing software. The COCOMO.II model addresses the case where one has the source code for the COTS product and represents the nonlinear dependence of adaptation costs with the amount of existing code that will be modified. For those cases without the source code, it is believed that the size can be measured in function points and then converted to SLOC for the computations. A LORAL COTS model used function points as a basis for its sizing. It used standard function point counting rules to size the functionality that is incorporated, including the interfaces between the COTS components and the custom code. Only the portions of the COTS components that perform functions that are either used or interface with the custom code are counted. Of the three models evaluated in the study, only SEER-SEM addressed the full range of COTS products. SEER-SEM was also the only model to address differences in code type (custom code (developmental), glue code and COTS code) and three levels of reusability for COTS components [code not reused (new code), existing code not designed for reuse, and existing code designed for reuse.] Depending upon the category of the COTS type, either SLOC or function point is used for sizing. The bottom line is that COTS software must be evaluated and any coding required must be sized during the estimation process, just like any other software. Appendix 19E, "COTS Software" discusses the peculiar aspects of using COTS software components in a system.
19.4.5 Summary of Metric Based Sizing Models The Cost Xpert User’s Manual compared the strengths of metric based sizing models as follows: SLOC sizing methods are particularly useful for real time and embedded systems applications. Function point based estimates are one of the best approaches for a wide range of MIS applications that have significant interaction with the environment and where this environmental interaction is the major cost driver. Function points are significantly less effective for applications where the environmental interactions are secondary to the algorithmic requirements of the program. If object oriented software development is being used on the program, the object metric approach works equally well with both MIS type applications and with the embedded, scientific, and system level programming projects that are not well served by function points. Current sizing models usually include more than one of the metric based approaches mentioned in this section. Other sizing models are already available or currently being developed in one or more of the categories of models discussed. For example, the QSM Size Planner is a composite model which contains an expert judgment model ("Fuzzy Logic"), a functionality model ("Standard Components"), and a function points model for size estimation. The user can select any combination of models or use all three and compute a weighted size estimate. Undoubtedly, more sizing models will become available in the future. No single sizing model has been shown to be accurate, or even useful, for a wide range of programs. Each model has a unique set of advantages and limitations.
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
