Ingegneria del Software II, a.a. 2004/05. V.Cortellessa, University of L Aquila

Transcription

1 1

2 2

3 3

4 4

5 5

6 6

7 Non-functional validation of software systems Vittorio Cortellessa Ingegneria del Software II (a.a ) 7

8 Programma della seconda parte del corso Introduction Non-functional validation today Extending UML Reliability Performance Risk New trends in software modeling: Metamodeling, MOF/QVT, UML 2 8

9 NON-FUNCTIONAL VALIDATION TODAY 9

10 Software requirements Functional (FR): statements of services the software system should provide, how it should react to particular inputs and behave in particular situations Non-functional (NFR): constraints on the services offered by the software system affecting the software quality Examples : functional : a password is required to be input after the user logs in non-functional: the operation f must end, on average, 2 seconds after user input 10

11 Early Software Validation Evaluating in the upper lifecycle phases whether the software model does has the ability of doing what the customer wants what the customer wants is (more or less completely and correctly) specified in the software requirement specification 11

12 Software Validation of NFR A crucial issue It is a more common practice to validate a software model vs FR rather than vs NFR Why?! 12

13 Motivations from market laws Different (and often not available) skills required for NFR modeling and validation Short time to deliver, i.e. quickly available low-quality software products seem to be very attractive nowadays! 13

14 Motivations from notation Software developers vocabulary is intrinsically distant from non-functional analysts one architecture implementation specification design requirement response time workload throughput error propagation operational profile failure probability Software development Non-functional analysis 14

15 Motivations from notation Stochastic Process Algebra Bayesian Model Stochastic Petri Net Simulation Markovian Model Queuing Network How can we help a software designer to fill the gap between the worlds? 15

16 Additional intrinsic motivation in case of early software models: Missing information related to NFRs early in the lifecycle because non-functional-related decisions are taken later req. specs possible instance of the final software product modeling decision 16

17 then why pursuing early validation of NFRs? In the early phases of the lifecycle a validation of NFRs may prevent late inconsistencies hard to fix NFRs are ever more critical in modern (possibly distributed and mobile) component-based software systems 17

18 Filling the gap between software development and NFR validation Amount and type of additional info to be embedded into a basic software model in order to perform NFR validation Algorithms to automate the step: basic software model + additional info ready-to-validation model Upon generated a ready-to-validation model has to be evaluated 18

19 What type of additional info? An example : the operational profile How to assign values to the operational profile How this info may affect the NFR validation 19

20 What type of validation? Software performance Software reliability Software risk 20

21 A general scheme Basic Software Model (original notation) Validation of Functional Requirements Additional Information: software annotations Ready-to-validation Model (possibly new notation) Validation of Non-Functional Requirements 21

22 an example MSCs and Statecharts Validation of Functional Requirements Queueing Network Operational profile + Platform features Performance Validation 22

23 an yet another example UML model Validation of Functional Requirements Bayesian model Operational profile + Failure probabilities Reliability Validation 23

24 Suitable attributes of approaches to NFR validation TRANSPARENCY: minimal influence on the adopted software notation and production process (annotations play an important role!) EFFECTIVENESS: (possibly automated) methodologies to transform the software model into a model ready to be validated From art to routine : need of systematic approaches 24

25 EXTENDING UML 25

26 Success of UML as original notation It is suitable to represent the software model in different phases of the lifecycle Different views of the software product provide more than one angle to catch (possibly hidden) non-functional features As a graphical notation, it is open to annotations Annotations may be structured to build profiles (UML extensions) 26

27 27

28 28

29 Need of extending UML In some domains a specialization or extension of UML basic concepts would be extremely useful to refine the rendering of domain-specific concepts and techniques 29

30 A predefined set of : UML profile Stereotypes Tagged Values Constraints that collectively specialize and tailor the UML for a specific domain (e.g., security, WEB applications) by extending either UML metamodel classes or other profile classes Examples of stereotypes, tags and constraints in various domains : navigational class in WEB, detailed host in Performance, etc. 30

31 31

32 32

33 33

34 Programma della seconda parte del corso Introduction Reliability Performance Risk Software reliability A methodology Notations Quality of Service New trends in software modeling: Metamodeling, MOF/QVT, UML 2 34

35 Software system S Ф(C1) C1 Non-functional validation of component-based systems Ф(C2) C2 Somehow interacting Ф : non functional attribute (e.g., reliability) Ф(Cn) Cn COMPOSITION OF NON- FUNCTIONAL ATTRIBUTES Modeling a non-functional attribute of each component and build a model of the non-functional attribute at system level Ε Ф(S) = [Ф(C1) Ф(C2) Ф(Cn)] f(c1,c2, Cn) 35

36 SOFTWARE RELIABILITY VALIDATION 36

37 Reliability informal definition: The probability that a software product (or model) does not fail within a certain interval of time! 37

38 Some useful concepts Fault feature that precludes the software from operating according to its specifications Error the value of the software state differs from the expected one Failure the actual software output (for some input) differs from the expected one 38

39 and an example program modthreeofsquare begin read(s); s := 2*s; s := s mod 3; write(s); end s=2 : no Error s=3 : Error! Fault! s=3,s=2 : no Failure s=4 : Failure! Specification: a function that computes the remainder by 3 of the square of the input value y = (s 2 mod 3) 39

40 Reliability of component-based software Most software reliability techniques treat a program as a monolith. Component-based software is expected to have high reliability as a result of deploying trusted components (think about COTS!). The claims of high reliability need further investigation. 40

41 FROM UML MODELS TO RELIABILITY MODELS: A METHODOLOGY 41

42 An approach based on UML: motivations Studying the sensitivity of the system reliability to changes in reliabilities of single components allows the system architect to select components with suitable reliability characteristics in cases where alternative reusable assets are available (tradeoff cost/reliability) 42

43 An approach based on UML: ideal set of parameters Reliabilities of components Operational profile Reliabilities of hardware devices Error propagation 43

44 Methodological steps 1. Provide annotations for application s UML diagram(s). 2. Use annotations as inputs to reliability calculations. 3. Design level analysis (prediction): 1. The algorithm predicts expected system reliability from provided (assumed, hoped for) component reliabilities. 2. Algorithm supports system-wide sensitivity analysis (what if I provide a more reliable component?). 44

45 UML annotations Annotated Use Case Diagram Annotated Sequence Diagram Actors, or external entities, or user types P ( x ) = q * m i = 1 P(executing use case x) i P ix (interaction of components within a use case) Annotated Deployment Diagram 45

46 Failure probability of component i within scenario j Failure probability of connector k between components l and i within scenario j ψ System failure probability Θ = 1 j= 1 UML annotations Θ i : failure probability of component i (one invocation) Ψ k : failure probability of link k (one message) S K p j θ ij = 1 (1 θi ) = 1 (1 lij ψ k ( N i= 1 (1 θ i ) bp ij bp ij ) Interact( l, i, j) k ( l, i, j) (1 ψ k ) Interact( l, i, j) ) 46

47 Assumptions no error propagation Component failure rates available. Failure Independence. A component s failure probability does not depend on the failure probabilities of the other components. Regularity. A component s failure probability is the same across all the busy periods of a component. Pessimism (fail-and-stop). Component failure results in a system failure. Can be corrected in a corroboration phase. 47

48 An Example A WEB-based transaction processing system (WBTPS) 48

49 Use Cases Local Operations : 1 scenario. Remote Read : 2 scenarios. Remote Write : 1 scenario. 49

50 Sequence Diagram 50

51 Deployment Diagram 51

52 From Annotations to Reliability 52

53 Connector usage table 53

54 Component Reliabilities Component Failure Probabilities as PDFs (Beta Distributions) 54

55 System Reliability Prediction 95% confidence interval of system failure probability is (0.13, 0.17). Reliability range (0.83, 0.87) Plot of Prior Probability Density Function of the System Failure Probability Θ S fitted to the normalized histogram from simulation observations 55

56 Sensitivity Analysis Increase / decrease reliabilities of individual components and observe the impact on overall system reliability. Improve Web servers C 5 : C 6 : Θ S :0.13 Θ S : 0.11 Worse remote servers C 11 : C 12 : Θ S : about 2% worse 56

57 Ф(C1) C1 about the error propagation Ф(C2) Somehow interacting Ф(Cn) Cn Reliability of each component may not suffice C2 component fail-and-stop A new parameter : error permeability 57

58 NOTATIONS FOR SOFTWARE RELIABILITY: An UML profile 58

59 The reliability domain model REuser REhost * * 1...* REservice * REcomponent * 1...* REconnector 1 1 No class has been introduced from scratch All the concepts relevant for the domain are expressed as attributes of these classes 59

60 The reliability domain model Inheritance relationships between concepts introduced in the UML profile for SPT and our classes Usage Demand (from Resource Usage Model) Active Resource (from Resource Types) REuser REcomponent Passive Resource (from Resource Types) Resource Instance (from Core Resource Model) REhost REconnector REservice 60

61 UML new stereotypes and tags Stereotype Base Class Tags Stereotype Base Class Tags «REuser» Classifier ClassifierRole Interactor Instance REaccessprob REserviceprob «REcomponent» Classifier ClassifierRole Component Instance REcompfailprob REbp Stereotype Base Class Tags «REhost» Node Classifier ClassifierRole REindexHost Stereotype Base Class Tags «REservice» Classifier REprob Stereotype Base Class Tags «REconnector» Message Stimulus AssociationRole REconnfailprob REnummsg CONSTRAINTS In case of nested components, REcompfailprob of outer components is a function (that depends on the type of nesting) of REcompfailprob of inner components 61

62 UML new stereotypes and tags Stereotype Base Class Tags Stereotype Base Class Tags «REuser» Classifier ClassifierRole Interactor Instance REaccessprob REserviceprob «REcomponent» Classifier ClassifierRole Component Instance REcompfailprob REbp Stereotype Base Class Tags «REhost» Node Classifier ClassifierRole REindexHost Stereotype Base Class Tags «REservice» Classifier REprob Stereotype Base Class Tags «REconnector» Message Stimulus AssociationRole REconnfailprob REnummsg CONSTRAINTS For each pair of components (or hosts), once fixed a direction, REconnfailprob assumes the same value over all the services 62

63 UML new stereotypes and tags Stereotype Base Class Tags Stereotype Base Class Tags «REuser» Classifier ClassifierRole Interactor Instance REaccessprob REserviceprob «REcomponent» Classifier ClassifierRole Component Instance REcompfailprob REbp Stereotype Base Class Tags «REhost» Node Classifier ClassifierRole REindexHost Stereotype Base Class Tags «REservice» Classifier REprob Stereotype «REconnector» Base Class Message Stimulus AssociationRole Tags REconnfailprob REnummsg CONSTRAINTS (Optionally but suitably) due to the low reliability of remote connections, REconnfailprob assumes a larger value over remote interactions (i.e., interactions among components on different hosts) than over local interactions 63

64 UML new stereotypes and tags Stereotype Base Class Tags Stereotype Base Class Tags «REuser» Classifier ClassifierRole Interactor Instance REaccessprob REserviceprob «REcomponent» Classifier ClassifierRole Component Instance REcompfailprob REbp Stereotype Base Class Tags «REhost» Node Classifier ClassifierRole REindexHost Stereotype Base Class Tags «REservice» Classifier REprob Stereotype Base Class Tags «REconnector» Message Stimulus AssociationRole REconnfailprob REnummsg CONSTRAINTS REaccessprob values overall the types of users sum up to 1 64

65 UML new stereotypes and tags Stereotype Base Class Tags Stereotype Base Class Tags «REuser» Classifier ClassifierRole Interactor Instance REaccessprob REserviceprob «REcomponent» Classifier ClassifierRole Component Instance REcompfailprob REbp Stereotype Base Class Tags «REhost» Node Classifier ClassifierRole REindexHost Stereotype Base Class Tags «REservice» Classifier REprob Stereotype Base Class Tags «REconnector» Message Stimulus AssociationRole REconnfailprob REnummsg CONSTRAINTS REserviceprob values overall the services sum up to 1 65

66 UML new stereotypes and tags Stereotype Base Class Tags Stereotype Base Class Tags «REuser» Classifier ClassifierRole Interactor Instance REaccessprob REserviceprob «REcomponent» Classifier ClassifierRole Component Instance REcompfailprob REbp Stereotype Base Class Tags «REhost» Node Classifier ClassifierRole REindexHost Stereotype Base Class Tags «REservice» Classifier REprob Stereotype Base Class Tags «REconnector» Message Stimulus AssociationRole REconnfailprob REnummsg CONSTRAINTS (Optionally but suitably) for each REservice, REprob is obtained as the sum (over all REuser) of the products between REaccessprob and REserviceprob 66

67 Using new stereotypes in UML diagrams: a case study ELEVATOR CONTROL SYSTEM schedule elevators to respond to requests from users at various floors control the motion of the elevators between floors «REservice» Select Destination {REprob = 0.24} ««Include» «Include» «REservice» Stop Elevator at Floor {REprob = 0.4} «REuser» Elevator User {REaccessprob = 0.6, REserviceprob = (0.4, sd), (0.6, re)} «REservice» Request Elevator {REprob = 0.36} ««Include» «REuser» Dispatch Arrival Sensor «Include» {REaccessprob = 0.4, Elevator REserviceprob = (1, sef)} USE CASE DIAGRAM 67

68 Using new stereotypes in UML diagrams: a case study : Elevator User << REuser >> {REaccessprob = 0.6, REserviceprob = (0.4,sd) (0.6,re)} : Elevator Button Interface << REcomponent >> {REcompfailprob = 0.009, REbp = 1} : Elevator Manager << REcomponent >> {REcompfailprob = 0.001, REbp = 2} : Elevator Status & Plan << REcomponent >> {REcompfailprob = 0.001, REbp = 2} : Scheduler << REcomponent >> {REcompfailprob = 0.002, REbp = 1} : Elevator Control << REcomponent >> {REcompfailprob = 0.001, REbp = 1} Elevator Button Request Elevator Request Update Acknowledge << REconnector >> {REconnfailprob = 0.0, REnummsg = 1} Elevator Commitment [is idle] Up or Down << REconnector >> {REconnfailprob = 0.0, REnummsg = 1} D1: Up (or Down) Request << REconnector >> {REconnfailprob = 0.003, REnummsg = 1} << REconnector >> {REconnfailprob = 0.0, REnummsg = 3} Collaboration Diagram: Stop Elevator at Floor Use Case A2: Approaching Floor ( Floor # ) SEQUENCE DIAGRAM OF SELECT DESTINATION USE CASE 68

69 Using new stereotypes in UML diagrams: a case study «REconnector» {REconnfailprob = 0.003, REnummsg = 1} «REconnector» {REconnfailprob = 0, REnummsg = 1} : Elevator User «REuser» {REaccessprob= 0.6, REserviceprob = ((0.4,sd) (0.6,re)} E1: Elevator Button Request : Elevator Button Interface «REcomponent» {REcompfailprob = 0.009, REbp = 1} E2: Elevator Request : Elevator Manager «REcomponent» {REcompfailprob = 0.001, REbp = 2} E3: Update E5: Elevator Commitment : Scheduler «REcomponent» {REcompfailprob = 0.002, REbp = 1} «REconnector» {REconnfailprob = 0, REnummsg = 3} E5a: Up or Down E4: Acknowledge «REconnector» {REconnfailprob = 0, REnummsg = 1} : Elevator Status & Plan «REcomponent» {REcompfailprob = 0.001, REbp = 1} E6: Up or Down : Elevator Control «REcomponent» {REcompfailprob = 0.001, REbp = 1} COLLABORATION DIAGRAM OF SELECT DESTINATION USE CASE 69

70 Using new stereotypes in UML diagrams: a case study Motor Server Manager Server C16 «REcomponent» {REcompfailprob = 0.001} C18 «REcomponent» {REcompfailprob = 0.002} C17 «REcomponent» {REcompfailprob = 0.001} C19 «REcomponent» {REcompfailprob = 0.001} «REconnector» {REconnfailprob = 0.002} C11 C12 «REcomponent» «REcomponent» {REcompfailprob = 0.002} {REcompfailprob = 0.003} C13 «REcomponent» {REcompfailprob = 0.003} C14 «REcomponent» {REcompfailprob = 0.004} C15 «REcomponent» {REcompfailprob = 0.003} «REconnector» {REconnfailprob = 0.005} «REconnector» {REconnfailprob = 0.003} C5 C6 «REcomponent» «REcomponent» {REcompfailprob = 0.009} {REcompfailprob = 0.009} C7 «REcomponent» {REcompfailprob = 0.008} C8 «REcomponent» {REcompfailprob = 0.009} C1 C2 «REcomponent» «REcomponent» {REcompfailprob = 0.009} {REcompfailprob = 0.008} C9 C10 «REcomponent» «REcomponent» {REcompfailprob = 0.008} {REcompfailprob = 0.008} C3 «REcomponent» {REcompfailprob = 0.003} C4 «REcomponent» {REcompfailprob = 0.007} Lamp Server DEPLOYMENT DIAGRAM Sensor Server 70

71 Automated construction of a reliability model from annotated UML diagrams This type of model Ф(S) = Ф(C1) Ф(C2) Ф(Cn) can become as follows for reliability modeling FP(S) = 1 - Σ j=1 K p(j) ( Π i=1 N (1-compfp(i)) bp(i,j) Π ("(l,i)) (1-connfp(l,i)) nms(l,i,j) ) REservice(j).REprob REcomponent(i).REcompfailprob REcomponent(i).REbp(j) REconnector(l,i).REnummsg(j) REconnector(l,i).REconnfailprob 71

72 Automated construction of a reliability model from annotated UML diagrams FP(elev_contr_sys) = 1 ( p(select_destination) ( (1-compfp(But_Int)) (1-compfp(Manager)) 2 (1-compfp(Stat&Plan)) 2 (1-compfp(Scheduler)) (1-compfp(Control)) (1-connfp(But_Int, Manager)) (1-connfp(Manager, Stat&Plan)) 3 (1-connfp(Manager, Scheduler)) (1-connfp(Stat&Plan, Control)) ) + p(request_elevator) REL RE + p(stop_elevator_at_floor) REL SEF ) Reliability of the Stop Elevator at Floor use case Reliability of the Select Destination use case Reliability of the Request Elevator use case 72

73 Automated construction of a reliability model from annotated UML diagrams and the numerical result upon taking values of tags from UML diagrams: FP(elev_contr_sys) = 1 ( 0.24 ( ( ) ( ) 2 ( ) 2 ( ) ( ) ( ) (1-0.0) 3 (1-0.0) (1-0.0) ) REL RE REL SEF ) = = 1 ( REL RE REL SEF ) 73

74 A WIDER CONCEPT: Quality of Service 74

75 Quality of Service and Fault Tolerance QoS deals with the non-functional aspects of a system that determine the satisfaction level of its users QoS may be intended as a multi-attribute resulting from the combination of basic non-functional attributes such as performance and usability Fault Tolerance is a very strictly related attribute that assesses the capability of a system to deliver continuous and failure-free service 75

76 Quality of Service and Fault Tolerance Multimedia applications Heterogeneous hardware platforms Distributed software systems Joint modeling and analysis of QoS and FT issues in order to develop QoS- and FT-aware software UML profile for modelling Quality of Service and Fault Tolerance characteristics and mechanisms OMG Adopted Specification, ptc/ , June

77 Relationships among UML profiles in the field of non-functional validation Meta-meta-model MOF P R O F I L E S Meta-model Model Other QoS and FT Component-based Reliability UML Other... Schedulability, Performance and Time Other... 77