Dan Horvath explores useful metrics that utilize function point analysis (FPA) that you can apply to agile projects. Dan shows how to measure productivity and quality as well as how to estimate for agile projects.
Over the past year, I have written articles on function point analysis (FPA) and agile methodology. I first explored some history and definitions of agile and other software development methodologies and discussed how FPA can be used to for the sizing measurement of agile projects and how the definition of the project is critical to this process. In my second article, I demonstrated the use of FPA in agile development through hands-on examples.
The Agile Manifesto teaches us that individuals and interactions are more important than processes and tools. In order to ensure that the individuals receive the productivity and quality they require, even in a mature agile environment, proper measurement and analysis are essential.
Let’s now explore useful metrics that utilize FPA and can be applied to agile projects. I’ll show you how to measure productivity and quality as well as discuss estimation for agile projects.
Measurement Questions
Any measurement program is an attempt to answer basic questions like:
- How productive are we?
- How good is our software quality?
- How much effort is required?
- How much should it cost?
- How long should it take?
- How many resources should be assigned?
- How is a project impacted due to changes in scope?
Function point analysis is used to provide the base measurement of several metrics, including productivity, quality, estimation, and change of scope. These, in turn, may be used to answer the types of questions noted above. Four of these major metrics will be examined further, as outlined in table 1.
Measurement Question | Agile Measures (by Sprint) | Final Delivered Project |
How productive are we? | Sprint FPs ÷ Sprint Hours | Delivered FPs ÷ Total Project Hours |
How good is our software? | Sprint Production Defects ÷ Sprint FPs | Total Production Defects ÷ Project FPs |
How much effort should it take (how many hours to deliver)? | Sprint ROM FPs ÷ Historical Sprint productivity rate | Project ROM FPs ÷ historical project productivity rate |
How much should it cost? | Sprint ROM FPs * Historical $/Sprint FP | Project ROM FPs * Historical $/Project FP |
Table 1. Measurement Questions and Approaches
The examples illustrated in the FPA counting for the agile project yielded the following function point counts as shown in table 2.
Example | Sprint | FPA |
Example 1 | 1 | 21 |
| 2 | 4 |
Example 2 | 1 | 21 |
| 2 | 25 |
Table 2. Summary of FPA counted by sprint for the two examples
The first sprint added new functions in both examples. Sprint number two of the first example simply added another. Sprint number two of the second example modified the functions developed in the first sprint while adding another. Let’s use these numbers to help answer some measurement questions we may be having.
Productivity
Productivity measurement involves function point analysis along with determining the amount of effort expended. A typical calculation for productivity is function points delivered per effort hour (FP/Hr). When using waterfall methodology, you typically measure the function points delivered and the effort hours at the final implementation phase of the project. When you use an agile methodology, you can take such measurements during the implementation phase, but you can also gather them at the end of a specific sprint. In the former case, the productivity rate reflects the delivered functionality, as it is for a waterfall project. In the latter case, the productivity measurement reflects the function points developed for that sprint.
Productivity may be impacted when functions are enhanced in more than one sprint for a given project. However, the trade-off is that the projects developed using agile methodology may be able to deliver at least some functionality at an earlier point in time, which in turn can increase customer satisfaction.
Let’s say that for the two examples, the effort for the first sprint was 1,000 hours in each case. Additionally, for the second sprint of the first example, the effort was 250 hours, and the effort for the second sprint of the second example was 1,450 hours. You can attribute the difference to the amount of rework for the second example. Now, let’s calculate the productivity.
Example | Sprint | Productivity Measure |
Example 1 | 1 | 21 FPs / 1,000 hours = 0.021 FP/hr. |
| 2 | 4 FPs / 250 hours = 0.016 FP/hr. |
| Total | 25 FPs / 1,250 hours = 0.020 FP/hr. |
Example 2 | 1 | 21 FPs / 1,000 hours = 0.021 FP/hr. |
| 2 | 25 FPs / 1,450 hours = 0.017 FP/hr. |
| Total | 25 FPs / 2,450 hours = 0.010 FP/hr. |
Table 3. Productivity Measurements
Notice that the overall productivity for the second example is lower than the first example even though the same functions are delivered. This is due to the rework required for the second example.
Quality
Quality is defined as the number of delivered defects per function point (defects/FP). Related measurements include the defect removal efficiency, which is the ratio of the defects resolved before delivery to the total number of the defects you found, and defect cost, which is the cost of correcting the defects. Quality metrics are often evaluated at the same project phase as productivity metrics. When using waterfall methodology, you can evaluate quality metrics during the implementation phase of projects, and when using an agile methodology, you can evaluate either at the end of a story or sprint or during the time of when you should deliver your projects.
The second example shows that some functions counted more than once for the same project since one agile sprint creates functions and the next one changes them. The effort for changing the functions in the second sprint may be planned, as part of the entire project, or unplanned, due to scope changes. If unplanned, it may be due to either new or changed requirements or defects. You must take care to ensure that the function point count as well as the related quality and productivity metrics accurately reflect the measurement objectives.
To further develop our examples, we will now say that there were five defects for sprint number one in each case. Sprint number two introduced one defect for the first example, and seven for the second example. Once again, the difference is due to the amount of rework required for the second example. Now let’s calculate the quality.
Example | Sprint | Quality Measure |
Example 1 | 1 | 5 defects / 21 FPs = 0.24 defects/FP |
| 2 | 1 defect / 4 FPs = 0.25 defects/FP |
| Total | 6 defects / 25 FPs = 0.24 defects/FP |
Example 2 | 1 | 5 defects / 21 FPs = 0.24 defects/FP |
| 2 | 7 defects / 25 FPs = 0.28 defects/FP |
| Total | 12 defects / 25 FPs = 0.48 defects/FP |
Table 4. Quality Measurements
We can see that our quality is reduced in example number two when compared to the first example. Again, this is due to the amount of rework involved to deliver the same functionality.
Estimation
Project estimation may use any of several methods, with one of the most rigorous and accurate methods employing an estimate of the number of function points to be delivered. From this, as well as additional data, the expected productivity, effort, cost, schedule, and staffing can be derived. For both waterfall and agile projects, these estimation metrics would typically be determined prior to the beginning of project development. For agile projects, however, this means that objectives are determined at the beginning of a sprint rather than at the beginning of development because too little is known, by the development team or even the system owner, about the requirements until each sprint begins.
Estimating agile projects at a complete project level may be done using rough order of magnitude (ROM) techniques. These techniques are used when little else is known about the detailed requirements. If the approximate number of internal logical files is all that is known up front, a ROM count can be done using only that information. You can refine the ROM count as you learn more information, such as maintenance or reporting requirements.
For example, an enhancement project has requirements that five logical files are to be added. If little else is known, you can apply some assumptions (such as an average complexity for most functions) to develop a ROM count as illustrated here:
- If you have five average internal logical files (ILFs), you have fifty unadjusted function points (UFP).
- With ten average external input transactions (EIs) (add and change functions for each), this results in forty UFP.
- With five low EIs (delete function for each): fifteen UFP.
- If you have five average external output (EOs) transactions (one report related to each ILF): twenty-five UFP.
- Total = 130 UFP
With this ROM function point count, you can now estimate effort using the productivity measurements from past projects. You can use the effort to determine any resource requirements and expected quality. Additionally, you can use the estimate to break the project up into manageable sprints, depending on what organizational standards and guidelines are imposed on you.
‘Time boxing” may be an additional consideration for you on agile projects and you can use it in different situations. You can use time boxing when the requirements for the sprints, or the sprints themselves, are not initially well defined. In this case, you can break the project up into time segments, each of which you can estimate. If historical data is available, such as the function point counts of previous projects, the organization will be able to determine the number of FP that can be delivered in a certain period of time so that you can plan time boxes. Alternatively, when the requirements are well-defined and a good function point count exists, you can use time boxing to determine how much functionality will be delivered within each sprint.
Change of Scope
According to A Guide to the Project Management Body of Knowledge (PMBOK Guide) , the project scope is defined as “The work that needs to be accomplished to deliver a product, service, or result with the specified features and functions.” Any changes to project scope can result in changes to the work that had originally been defined, and these changes may result in changes to the project cost or quality. The amount of change can be expressed as a function point difference, or an effort to make the change or changes.
You can measure the change of scope at the time the changes occur, at specific points within the project development, or at the completion of the project. Since scope changes may occur more frequently during agile projects, it may be better for you to consider scope changes at a higher level for the entire set of software delivered rather than at project intervals, because it is best to define upfront what constitutes a scope change. Doing so will allow you to distinguish any changes to functions that are inherent in agile projects.
As in the second example, if a function is added in one sprint and changed in a later one, you can consider this a change in scope and you can measure this change if each sprint is counted. Once again, because the data may be available at various stages, agile development provides the opportunity to gather this additional information in an accurate and timely manner.
Conclusion
Although the function point counting process itself is the same for the various development methodologies, there are some additional considerations you must account for when counting projects developed using an agile methodology. For example, the sprint artifacts may be different than the documents you use when counting projects using other development methodologies. In order to be properly utilized for the counting process, these artifacts must describe in detail the functionality added, changed, or deleted by the project. You also need to consider that in the case when multiple sprints are considered part of the same project, it’s possible that some application functions may be affected more than one time.
As noted, you can perform the function point analysis at various stages during the agile development effort. These cases would be different than the function point counts performed at various stages of development using waterfall methodologies, because the stages themselves are self-contained. In the case of waterfall development, there is a defined point in the project to perform the count, such as after the analysis or functional design phases. Any counts performed during the course of agile development would measure an intermediate work product at the completion of one or more stories or sprints. This may be for the purpose of assisting the customer or project manager in monitoring or controlling the project, or measuring the functionality of a sprint as part of the overall project.
By analyzing the data to determine the impact of quality, productivity, schedule, and cost, software development organizations may be better able to choose the most appropriate development methodology for their projects. The development methods you utilize for projects should not impede you from measuring the project. As demonstrated in the examples presented above, you can use the function point analysis measure an agile projects, however, several considerations need to be addressed depending on how the project is defined. These considerations include the goals and objectives of the organization as well as the supporting measures. Using FPA and the other software metrics I discussed will provide you new opportunities to apply metrics to your projects.
