Automated Estimation using Enterprise Architect August 2012 Laurence White Abstract.

Transcription

1 Predictive Current Retrospective Automated Estimation using Enterprise Architect Abstract. This paper details an approach for creating automated measures of the scale and complexity of an enhancement, based on artifacts stored in Enterprise Architect (EA) from Sparx Systems. These scale and complexity measures can then be translated into level-of-effort estimates. Existing functionality in EA is based on Karner s Use Case Point (UCP) method (1993); it can render approximate estimates at the project level. The method described here formalizes the Planning Poker method and, along with quantitative measures of the number of artifacts and links within a given boundary, provides an automated capability for estimating the work associated with implementing an enhancement. While this method for automated estimation is based on using Sparx EA, it may be implemented in any tool that captures UML artifacts in a structured data store. Sparx EA is especially well suited to the method as it supports structured specification of the individual steps within a use case. Introduction. Metrics may be used to quantify the following: Scale. Complexity. Productivity. Quality. Metrics may be predictive, current, or retrospective. Scale. Complexity. Productivity. Quality. Table 1. Predictive metrics are the most difficult to generate accurately, but most valuable for weighing effort against priorities & resources. The following outlines an approach for generating predictive metrics. 1

2 In the present case, predictive metrics in EA are built out in three phases: Use Case. Customer Request. Work Package. Use case is first because (1) it gives a good point of comparison with other estimation methods; (2) because it is the most difficult of the three and allows us to iron out issues likely to emerge elsewhere. Methodology. The chart below shows which measures affect Scale & Complexity (or both). The remainder of this document will focus on the Use Case portion of this chart. Table 2. 2

3 Use Case Metrics. Table 3. Weighting. As part of determining weighting, we could go through existing projects in EA and determine, for example, a range for number of functional requirements per use case. Alternatively, we can set ranges which we are confident will cover the values we will encounter in actual use cases. Example: We find that number of functional requirements per use case has a min. value of 3 and a max. value of 20. Set the effective range for functional requirements per use case at 1 through 50. Set a formula that maps this range to the modified Fibonacci series (shown on the next page.) As this parameter directly affects Scale (in a linear fashion), set its weight to 1.0 for Scale. This parameter indicates Complexity but is not a direct measure of it, so set its weight to 0.5 for Complexity. The goal is to map each Scale & Complexity factor to the modified Fibonacci series, then use these data points to create an overall measure of both the Scale and Complexity of the use case. Finally, we create a level-of-effort figure for the use case as a whole. 3

4 Next, we use a Modified Fibonacci to measure Scale and Complexity. The series values are 0.5, 1, 2, 3, 5, 8, 13, 20, 40, 100. The chart on the next page shows min. and max. figures for each of the use case factors. 4

5 Ranges. The min./max. figures shown below are meant to reflect a 100% Confidence Interval i.e. we should never encounter a use case with a value outside these ranges. Table 4. It is entirely possible that repeated calibration runs will cause the max. figures to be modified for some factors, but this should only be done on the basis of repeated observation. 5

6 Next we replace the Y/N indicators in the Scale & Complexity columns with weightings for each factor. Table 5. Screen shots on the following pages show how EA renders some of the factors listed above. 6

7 Structured Specification. 7

8 Links to/from Use Case. Constraints. 8

9 Taking a use case chosen at random from a project in EA, we add columns for the observed values and their related scores. Table 6. 9

10 Measurement. Using the unmodified scores for this use case (20,27), we can plot a point for the use case (see red data point below.) Normalizing the score to the modified Fibonacci series, we get (20,20), which is plotted on the next page. (The Fibonacci is used as a filter to push scores into discrete bands. This creates a clearer signal for both Scale & Complexity.) 10

11 The algorithm for mapping to the Fibonacci series works as follows: Let x = Score. Complexity, y = Score.Scale. Let F represent the [modified] Fibonacci series (0.5, 1, 2, 3, 5, 8, 13, 20, 40, 100). Let n represent the integer range 1 through 9. So, for example, F(1) = 0.5, F(6) = 8; when n = 9, F(n+1) = F(10) = 100. Then the algorithm becomes: Do for x = 1,9 until x assigned a value If x >= F(n) If (x F(n)) < (F(n+1) x) x = F(n) Else x = F(n+1) End-If End-If End-Do 11

12 In simple terms, we scan the Fibonacci series to find a closest match to the x or y value. If the x y value is closer to F(n) than it is to F(n+1), we use the F(n) value; otherwise, we use the F(n+1) value. Uncertainty. The margin of error on these estimates, like the factor weightings, can be calibrated over time as we collect more data points allowing us to compare estimates with actuals. To begin with, we can assign uncertainty values to zones within the Scale/Complexity coordinate system. The median point of the red zone highlighted below is (70,70). The margin of error for both the vertical & horizontal interval is +/- 30. Similarly, we can assign margins of error as shown in the chart on the next page. 12

13 Initial uncertainty values for Scale/Complexity. Automated Estimation using Enterprise Architect Table 7. 13

14 Conclusion. 1. As long as the factors listed in Table 3 above can be extracted, we have the raw data needed to feed & calibrate the automated metrics process. 2. As with weightings, we can make a best guess at how the scores relate to level-of-effort, then calibrate for improved accuracy. 3. This document does not address technical implementation; it outlines the logical design as a basis for discussion. 4. Once it is decided at which points in the development cycle to generate use case metrics, a history of successive estimates may be stored with the use case or at the work package level in the form of instances of custom element types, or possibly as list-type tagged values. These details are beyond the scope of this document. Possible Future Extensions. The method outlined here could be made more sophisticated, but this raises the risk of over-engineering the solution. Possible enhancements include: 1. Determine which factors have a non-liner influence on Scale & Complexity; for example, the number of possible paths through a use case could affect Scale linearly but have an exponential effect on Complexity. 2. Allow the weighting of a factor to be influenced by the number or score of other factors; for example, if a user story gives rise to a large number of use cases, this may indicate a high degree of complexity. It might be appropriate to increase the Complexity weighting for some of the use case factors. 3. Perform trend analysis on automated metrics generated over time. Use this information to tune the weightings for factors and possibly suggest significant factors which have been missed. 4. Use some of the out-of-the-box TCF s & ECF s (Technical Complexity Factors & Environmental Complexity Factors) to modulate the signal created by automated metrics. 14