Performance Evaluation of Complex Systems: from Large Software Architectures to the Cell Processor

Transcription

1 Performance Evaluation of Complex Systems: from Large Software Architectures to the Cell Processor Moreno Marzolla INFN Sezione di Padova Università di Bologna

2 Talk Outline Motivations Model-Based Performance Evaluation Software Architectures Web Services Cell Processor Conclusions Moreno Marzolla , Università di Bologna 2

3 Motivations Large software systems are complex and expensive artifacts Their success depends on whether their functional and non-functional requirements are satisfied Performance is one of the most important non-functional requirements Moreno Marzolla , Università di Bologna 3

4 How can performances be evaluated? Measurement-based approach Perform direct measures on a running system or a prototype; use these measurements to identify bottlenecks Model-based approach Develope a performance model of the software system; use the model to mimic the behavior of the system and predict its performances Moreno Marzolla , Università di Bologna 4

5 The fix-it-later approach 1.Define the system architecture 2.Build and deploy the system 3.Evaluate performance through direct measurements on the running system 4.If performance requirements are not satisfied, go back to step 1 Moreno Marzolla , Università di Bologna 5

6 Model-Based System Performance Evaluation System Model Results and Feedback Performance Model Model Evaluation Moreno Marzolla , Università di Bologna 6

7 Performance Evaluation of Software Systems Moreno Marzolla , Università di Bologna 7

8 Waterfall Software Development Model Requirements Definition System and Software design Measurement-based Performance Evaluation Model-based Performance Evaluation Implementation and unit testing Integration and system testing Operation and maintenance Moreno Marzolla , Università di Bologna 8

9 Simulation-Based Software Performance Evaluation UML-PSI ArgoUML argouml.tigris.org UML SPT Profile Marzolla and Balsamo, proc. QEST'04 Annotated System UML Model Model Results Performance and Results Feedback Performance Simulation Model Simul. Model Model Evaluation libcppsim Marzolla, proc. ESM'04 Moreno Marzolla , Università di Bologna 9

10 The modeling cycle The UML model is annotated according to the UML Profile for Schedulability, Performance and Time Specification The annotated UML model is automatically translated into a simulation model Use Case Diagrams Workloads Activity Diagrams Simulation Processes Deployment Diagrams Resources The simulation model is executed Simulation results are inserted back into the UML model as annotations Moreno Marzolla , Università di Bologna 10

11 Example Annotated Use Case Diagram <<ClosedWorkload>> Request Video PApopulation = 10 PAextDelay = [ assm, dist, [ exponential, 1.0/50.0]] Moreno Marzolla , Università di Bologna 11

12 Example Annotated Activity Diagram PAhost = ClientWorkstation PAdemand = [ assm, dist, [ exponential, 1.0/10.0]] PAhost = WebServerNode PAdemand = [ est, dist, [ constant, 2.0]] Select Service Handle Selection Initiate Playout PAhost = VideoServerNode PAdemand = [ assm, dist, [ exponential, 1.0/0.5]] PAhost = ClientWorkstation PAdemand = [ msrd, dist, [ constant, 0.5]] PAhost = ClientWorkstation PAdemand = [ msrd, dist, [ exponential, 1.0/0.5]] Confirm Initialize Player Send Video Send Terminate PAhost = Internet PArep = 100 PAdemand = [ assm, dist, [ exponential, 1.0/0.5]] PAhost = VideoServerNode PAdemand = [ msrd, dist, [ constant, 0.5]] Terminate Player PAhost = ClientWorkstation, PAdemand = [ assm, dist, [ constant, 1.0]] Moreno Marzolla , Università di Bologna 12

13 Example Annotated Deployment Diagram PArate = 2.0 PAschedPolicy = FIFO PArate = 2.0 PAschedPolicy = FIFO Parate = 2.0 PAschedPolicy = FIFO ClientWorkstation: WebServerNode: VideoServerNode: Internet: PArate = 1.0 PAschedPolicy = PS Moreno Marzolla , Università di Bologna 13

14 Model development using ArgoUML Moreno Marzolla , Università di Bologna 14

15 Model analysis using UML-PSI UML-PSI (UML Performance SImulator) is a discrete-event simulator written in C++ Command-line tool Parses annotated UML diagrams in XMI format Builds a simulation model Executes the model Performance results (resources utilization, actions response times,...) are inserted back into the XMI file as additional tagged values Moreno Marzolla , Università di Bologna 15

16 Using a different performance ArgoUML argouml.tigris.org UML SPT Profile model Balsamo, Mamprin, Marzolla proc. ESMc'04 Balsamo and Marzolla, proc. WOSP'05 Annotated System UML Model Model Results Performance and Results Feedback Queueing Performance Network Model Model QN Model Evaluation QNetworks Moreno Marzolla , Università di Bologna 16

17 Translating Use Case Diagrams <<PAopenLoad>> PAoccurrence = λ <<PAclosedLoad>> λ N PApopulation = N PAextDelay = Z Z Moreno Marzolla , Università di Bologna 17

18 Translating Activity diagrams: the easy case A1 R1 R1 A2 R2 R2 A3 R3 R3 Moreno Marzolla , Università di Bologna 18

19 Translating Activity diagrams: the not-so-easy case A1 R1 A2 R1 R2 A3 R2 Moreno Marzolla , Università di Bologna 19

20 Evaluation of the QN Model Annotated System UML Model Model Queueing Performance Network Model Model Results Performance and Results Feedback Model Evaluation Bound Analysis MVA Simulation Moreno Marzolla , Università di Bologna 20

21 Evaluation of the QN Model QNetworks: a Queueing Networks analysis package for GNU Octave Free and Open Source, GPLv3 Another good QN tool: Java Modelling Tools Moreno Marzolla , Università di Bologna 21

22 The QNetworks package Single-station queueing systems M/M/1, M/M/m, M/M/1/k, M/M/m/k, M/M/ Bounds on performance Balanced System Bounds, Asymptotic Bounds, Geometric Bounds Product-form QN MVA for single/multiclass QN, closed or mixed networks with load-dependent service centers Jackson networks Convolution algorithm Non product-form QN MVABLO Miscellaneous Continuous- and Discrete-time Markov Chains Computation of visit ratios Moreno Marzolla , Università di Bologna 22

23 References S. Balsamo and M. Marzolla, Simulation Modeling of UML Software Architectures, Proceedings of the European Simulation Multiconference, Nottingham - UK, 2003 S. Balsamo and M. Marzolla, A Simulation-Based Approach to Software Performance Modeling, Proceedings of ESEC/FSE 2003 M. Marzolla, S. Balsamo, UML-PSI: The UML Performance SImulator, Proc. of the First International Conference on the Quantitative Evaluation of Systems (QEST 2004) S. Balsamo, R. Mamprin, M. Marzolla, Performance Evaluation of Software Architectures with Queuing Network models, in Proc. of ESMc'04 S. Balsamo, M. Marzolla, Performance Evaluation of UML Software Architectures with Multiclass Queueing Network Models, in Proc. Fifth Int. Workshop on Software and Performance (WOSP'05) S. Balsamo, M. Marzolla, R. Mirandola, Efficient Performance models in Component-Based Software Engineering, in Proc. 32nd Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Moreno Marzolla , Università di Bologna 23

24 Performance Evaluation of Web Services Moreno Marzolla , Università di Bologna 24

25 Context: the glite Middleware Grid middleware developed by the EU project EGEE-3 Different interacting components Compute Elements Storage Elements Authentication Services Accounting Services... Moreno Marzolla , Università di Bologna 25

26 Context: the glite Middleware Many of the middleware services are exposed as Web Services Standardization efforts are leading towards different interchangeable implementations of the same services M. Riedel et al., Interoperation of World-Wide Production e-science Infrastructures, Concurrency and Computation: Practice and Experience, 21(8), 2009 P. Andreetto et al., Standards-Based Job Management in Grid Systems, submitted to the Journal of Grid Computing Moreno Marzolla , Università di Bologna 26

27 BPEL The Service-oriented Architecture (SOA) paradigm foresees the creation of business applications from independently developed services. Providers offer similar competing services these offerings can differ significantly in some QoS attributes like performance. The key point is to build applications through the composition of available services The application can be specified using BPEL notation Moreno Marzolla , Università di Bologna 27

28 <receive> BPEL / 1 The executing process waits for a specific incoming message to be received; <reply> Sends a message in reply to a message which was received through a receive tag; <invoke> Invokes a one-way or request-response operation on a partner; <wait> Waits for a given time period, or until a certain time has passed; <sequence> Denotes a set of activities which should be executed sequentially; for each activity it is possible to specify additional dependencies, that is, other activities which must complete before executing the current one. <If> Selects one activity from a set of choices; <while> Repeats an activity until a certain predicate is no longer true; Moreno Marzolla , Università di Bologna 28

29 BPEL / 2 <repeatuntil> Repeats an activity until a condition becomes true; <foreach> This activity repeats its child activity for a number N of times; the child activity instances can be executed sequentially, or in parallel; <pick> The process blocks until a certain message is received, or a timeout goes off. When one of these events occurs, the associated activity is executed and the pick completes; <flow> Denotes a set of concurrent activities; <switch> Allows the process to choose exactly one branch Moreno Marzolla , Università di Bologna 29

30 BPEL performance modelling BPEL2qnbound Marzolla and Mirandola, proc. QoSA'07 System Model Annotated WSDL and BPEL Results Performance and Results Feedback Queueing Performance Network Model Model QN Model Model Evaluation Evaluation (bound analysis) Moreno Marzolla , Università di Bologna 30

31 BPEL model to QN model Moreno Marzolla , Università di Bologna 31

32 Example / BPEL Closed Workload <bpws:process> <perf:workload type="closed" thinktime="120"/> <bpws:import importtype=" location="casestudy.wsdl"> <bpws:sequence> <bpws:invoke operation="authenticate"/> <bpws:flow> <bpws:invoke operation="transferisb"/> <bpws:invoke operation="transferdata"/> </bpws:flow> <bpws:while> <bpws:condition perf:prob="0.7"/> <bpws:invoke operation="jobstart"/> </bpws:while> <bpws:invoke operation="transferosb"/> <bpws:invoke operation="analyze"/> </bpws:sequence> </bpws:process> External Delay (think time) Branch probability Moreno Marzolla , Università di Bologna 32

33 Example / WSDL <!-- Interface for Storage Element --> <definitions> <porttype name="datafactory"> <operation name="transferdata"> <perf:pademand resource="df:cpu" value="1"/> <perf:pademand resource="df:disk" value="120"/> <perf:pademand resource="network" value="80"/>... </operation> </porttype> </definitions> Resource Name Operation Name Average Service Time Moreno Marzolla , Università di Bologna 33

34 Example / WSDL <!-- Interface for Computing Element --> <definitions> <porttype name="jobfactory"> <operation name="authenticate"> <perf:pademand resource="ce:cpu" value="10"/> </operation> <operation name="transferisb"> <perf:pademand resource="ce:cpu" value="2"/> <perf:pademand resource="network" value="10"/> <perf:pademand resource="ce:disk" value="120"/> </operation> <operation name="jobstart"> <perf:pademand resource="ce:cpu" value="4"/> </operation> <operation name="transferosb"> <perf:pademand resource="ce:cpu" value="1"/> <perf:pademand resource="network" value="80"/> <perf:pademand resource="ce:disk" value="30"/> </operation> </porttype> </definitions> Moreno Marzolla , Università di Bologna 34

35 Example / WSDL <!-- Interface for Analysis Element --> <definitions> <porttype name="dataanalysis"> <operation name="analyze"> <perf:pademand resource="da:cpu" value="100"/> <perf:pademand resource="da:disk" value="30"/> </operation> </porttype> </definitions> Moreno Marzolla , Università di Bologna 35

36 BPEL2qnbound Command-line tool Given annotated BPEL and WSDL, we compute the visit counts, and then the service demands Bottleneck Moreno Marzolla , Università di Bologna 36

37 BPEL2qnbound We can study the effect of the bottleneck removal Moreno Marzolla , Università di Bologna 37

38 Performance Evaluation of the Cell Processor Moreno Marzolla , Università di Bologna 38

39 The Cell/BE Developed by IBM/Sony/Toshiba Used inside Sony's Playstation 3 Power Processor Element PowerPC architecture Synergistic Processor Elements 128 registers, each 128bit wide 256KB local store Efficient vector processing f/pages/r.arch.innovation.html Moreno Marzolla , Università di Bologna 39

40 Cell Architecture Moreno Marzolla , Università di Bologna 40

41 Challenges Irregular architecture SPEs and PPE need different compilers Non-shared memory DMA transfer main memory SPE local stores Difficult to get it right Overlap SPE computations with DMA transfers Vectorize SPE code Data alignment and packing for efficient DMA transfer/vector manipulation Moreno Marzolla , Università di Bologna 41

42 Cell performance modeling Annotated System UML Model Model Results Performance and Results Feedback Queueing Performance Network Model Model (?) Model Evaluation Moreno Marzolla , Università di Bologna 42

43 Example: Multibuffering Fetch/Compute Memory SPE Double-buffering: computations overlapped with fetch of next input buffer Memory SPE B1 B2 Proc. B1 B1 Proc. B2 B2 Proc. B1 Moreno Marzolla , Università di Bologna 43

44 S1 = Mean DMA transfer time QN model of the Cell/BE Multibuffering Mem S1 SPE S2 S2 = Mean SPE computation time Multibuffering level = population size N Moreno Marzolla , Università di Bologna 44

45 Conclusions Model-based system performance evaluation can be applied at different levels, in different scenarios From Software Systems......to Web Services......down to CPU programming Results are promising, BUT... Tools are lacking Model-based PE is poorly integrated into CASE tools Difficult to find meaningful case studies Moreno Marzolla , Università di Bologna 45

46 Thank you! Moreno Marzolla , Università di Bologna 46