EmpAnADa Project. Christian Lange. June 4 th, Eindhoven University of Technology, The Netherlands.

Transcription

1 EmpAnADa Project June 4 th, 2004 Eindhoven University of Technology, The Netherlands Outline EmpAnADa introduction Part I Completeness and consistency in detail Part II Background UML Empirical research Measurement Directions of the project Metrics Refactoring and changes in an artifact Completeness and Consistency Visualization of Metrics EmpAnADa and LaQuSo 2

2 EmpAnADa Empirical Analysis of Architecture and Design Quality Development of quantitative methods to predict attributes of early artifacts, mainly s Quality (maintainability, changeability ) Effort estimation Identification of fault-prone elements Empirical studies to Validate methods Investigate UML usage and problems in practice 3 Part I Empirical Assessment of Completeness in Industrial UML Designs 4 2

3 Completeness Overview of Part I Decomposition Consistency Techniques Developed Case Studies Observations Conclusions 5 Why Completeness? Interviews and surveys investigating industrial use of UML Uncertainty about precision and accuracy of design metrics for prediction of quality attributes Miscommunication Integration problems Uncertainty about completeness of model 6 3

4 Completeness & Consistency Problem: (Early) models allow several solutions/interpretations (dark grey) Source code describes exactly one solution Misinterpretations amplified by (light grey) lack of completeness presence of conflicts UML specific: overlapping diagram types, no formal semantics, degrees of freedom interpretations Misinterpretations Allowed solutions Goal: X X X X time Model Model 2... Model N Implementation Investigate the magnitude of completeness of s in industrial practice 7 Decomposition of Completeness Completeness Client s interest Requirements Completeness Well- Formedness Model Completeness Consistency Diagram Completeness Developer s interest Well-Formedness: property of individual diagram Consistency and Diagram Completeness: relation between different diagrams 8 4

5 Scope of Analysis Requirements R R2 R3... Use Cases? UC UC2 UC3?... Model Requirements can be traced across multiple diagram types Examples of incompleteness Scope of analysis Sequence Charts Class Class2 SC SC2 SC3 SC4... Class3 Class4?? Class5 SD?? Class6... SD2 Classes State Diagrams Implementation Class Class3 Class5 Class2 Class4 9 Relational Meta Model Use Case Diagram Use Cases Actors Message Sequence Chart precedes Scenarios consists of Messages 0..n changes to has callee caller calls 0.. Class Diagram Objects Methods States StateMachines 0.. Classes Attributes State Chart Diagram 0 5

6 Well-formedness Examples of Rules Objects in SD must have a role name Classes must have methods Consistency Messages in SD must correspond to method in class diagram Completeness Interaction of classes must be described in SD Methods must be called in SD Rule-checking is implemented in a tool Case Studies Classes 00,00% Class Associations 50,00% 0,00% Methods A A2 B B2 C C2 Inheritance Relations Messages A, A2 Image processing Subsystems, same version B, B2 Embedded Controller Subsequent versions, redesign C, C2 Embedded Controller Subsequent version, inspection in between 68 classes, 853 messages >00 man years Real-world 69 classes, 705 messages >00 man years Real-world 46 classes, 29 messages,5 man years Post-grad students 2 6

7 Results: Well-formedness Rules A A2 B C Objects without name 52.00% 6,58% 9.92% 25.24% Classes without methods 60.9% 5.9% 00.00% 20.00% Interfaces without methods 8.82% 9.38% N/A 60.00% Abstract classes in MSC 5.56% 0.60% 0.00% 0.00% Public Attributes 67.23% 5.08% 0.00% 0.00% 00,00% 80,00% 60,00% 40,00% A A2 B C 20,00% 0,00% Objects w ithout name Classes w ithout methods Interfaces w ithout methods Abstract Public classes in Attributes MSC 3 Results: Consistency Rules A A2 B C Messages Without Name 0.00% 0.00% 0.28% 0.00% Messages without Method 58.73% 7.62% 00.00% 27.4% Messages between unrelated Classes 7.94% 4.03% 77.73% 8.90% 00,00% 80,00% 60,00% 40,00% 20,00% A A2 B C 0,00% Messages without Name Messages without Method Messages between unrelated Classes 4 7

8 Results: Completeness Rule A A2 B C Classes not called in MSC 6.% 59.52% 35.29% 42.22% Interfaces not called in MSC 00.00% 87.50% 00.00% 70.00% Methods not called in MSC 67.65% 77.59% N/A 40.4% 00,00% 80,00% 60,00% 40,00% 20,00% A A2 B C 0,00% Classes not called in MSC Interfaces not called in MSC Methods not called in MSC 5 Observations in Case Studies Strong differences in the habits of designers even within a single project miscommunication may be reduced by modeling standards Degradation between C and C2 wrt. SD completeness: rules helped identify missing SD s Odd results for SD-rules in B2: identified that wrong version of model was in project repository. Case C has best scores: off critical-path project (?) Results of assessments can be used for focusing of improvements monitoring progress benchmarking: comparison between different models (versions, sub-models, processes) 6 8

9 Contributions & Conclusions Industrial-strenght techniques for quantitative assessment of incompletenesses and inconsistencies. Case studies provided insight into use of the UML Absolute number of rule violations quite large Industrial projects move into implementation while there are still many incompletenesses Incomplete spots identified by tool were confirmed by designers and led to improvements in the design 7 Future Work Which degree of completeness is needed? for high accuracy and precision of estimates to reduce miscommunication Do incompleteness and inconsistencies of the model result in flaws in the implementation? Can Modeling standards (enforced by tooling) improve completeness? 8 9

10 Part II Background Topics & Future Directions 9 Background - UML Unified Modeling Language Booch, Rumbaugh, Jacobson : 997 Purpose: visual modelling of different aspects of the system Higher abstraction level than programming Diagrams are views on the model 9 diagram types Structural Class diagram Package diagram Object diagram Use case diagram Deployment diagram Behavioral Sequence diagram Collaboration diagram State diagram Activity diagram Extensible by stereotypes, tagged values, meta-model changes Lack of formal semantics 20 0

11 Class Diagram OrderDetail * -quantity -taxstatus +calcsubtotal() +calcweight() Payment -amount Background UML Order -date -status +calctax() +calctotal() +calctotalweight() Sequence Diagram SAAT.BAT Preprocessor Parser Analyser DB Creator DB Filler Inheritance Relator DB Checker Statistic Filter Statistic Calculator Credit -number -type -expdate +authorized() Cash -cashtendered Check -name -bankid +authorized() State Diagram Start Use Case Diagram Invalid Reject Retry Get SSN AbideSys Error message submit Run Report «extends» Budget Analyst Valid Validate SSN & PIN submit Get PIN Logon Get Online Help Run Report with managed Options 2 Background Empirical Research SW engineering is cross-disciplinary Technical issues Networks, databases, operating systems Language issues Syntax, semantics Social issues and psychology Why empirical SW engineering research? Quantitative studies give objective and statistically significant results regarding Understanding } Controlling Predicting Of Improving Empirical strategies Surveys Experiments Case Studies the SW Development process 22

12 Background Empirical Research Strategies + Case Studies Factor -Risk Surveys Survey Experiments - Scale + Experiment Case study Execution control No Yes No Measurement control No Yes Yes Investigation cost Low High Medium Ease of replication High High Low 23 Empirical Research in EmpAnADa Survey Web-questionnaire about UML usage Experiment Classroom experiment is planned Case Studies Always looking for case studies Currently contacts to Philips (PMS, Optical Storage) Siemens Corporate Research LogicaCMG / Thales Oce CGE&Y CBS 24 2

13 Background - Measurement Measurement is a mapping from the empirical world to the formal, relational world (Fenton) Types of metrics Product Lines of Code (LOC), Depth of Inheritance, number of defects Process Cost per number of errors found, coverage of a test Resource Team size, experience of staff, age of staff Most product metrics research is about finished products 25 Measurement in EmpAnADa Metrics are objective and automatically collectable Implementation metrics Measure the exact product Rather late Design Metrics Early feedback in the analysis and design phase Not all implementation metrics can be used Opportunity for new metrics (multi-view) 26 3

14 Measurement in EmpAnADa II Relation between design metrics and properties of product Quality properties Maintainability Comprehensibility Changeability Defects Finding relationships between metrics and test data, bugreports Metrics to predict fault-prone classes Prediction of process data Development effort Resource usage Design metrics as predictions: Accuracy and Precision Depend on Correspondence of model and source code Completeness and Consistency of model 27 Completeness and Consistency Define completeness and consistency for s Assess the magnitude of violations in industrial practice What are the effects of C&C violations for the implementation? Which degree of C&C is needed for precise and accurate predictions based on s? Part I 28 4

15 Refactoring: Refactoring and Evolution Changing a SW system to improve its structure without altering the external behavior (Fowler) Most literature considers source code refactorings Approaches: Defining higher level refactorings (architecture, design) Investigating the role of metrics as design smells Evolution How does the quality of designs change over subsequent versions? 29 Visualization of Metrics and Artifacts For large systems metrics tools produce large amount of data Which techniques are useful to present the measurement data in an intuitive way? normal perspective original Enriched with colors representing metrics reduced perspective Showing only inheritance Showing only depencency Showing only associations Showing only classes with metric larger than specific value extended perspective Class diagram extended with related use cases 30 5

16 Summary of topics leads to Implementation triggers change measurement triggers change measurement Design Metric predicts Implement. Metric relates to relates to Quality Attribute 3 Activity within LaQuSo Architecture Repository Group Developing a repository to store SW architectures and designs in different formalisms Repository UML Analysis Tools SAAT Transformation Tools Petri nets µcrl 32 6

17 Discussion? 33 7