Visual Mapping of Program Components to Resources Representation: a 3D Analysis of Grid Parallel Applications

Transcription

1 Visual Mapping of Program Components to Resources Representation: a 3D Analysis of Grid Parallel Applications Lucas Mello Schnorr, Guillaume Huard, Philippe Olivier Alexandre Navaux Federal University of Rio Grande do Sul, Brazil Grenoble Institute of Technology, France SBAC-PAD São Paulo, Brazil October / 22

2 Introduction Highly distributed systems Grids Network characteristics Topology and Hierarchical Organization Performance: latency and bandwidth Influence the communications of parallel applications They must be taken into account in the analysis Otherwise, misguided performance analysis Leads to wrong performance optimizations Existing Visualization Approaches Structural Representations (e.g. graphs, hierarchies) Behavioral Views (e.g. space-time, gantt-charts) 2 / 22

3 Introduction 3 / 22 Existing 3D Approach [GRID 2008] Application components (2D) Evolution through time (1D) Interaction Techniques Spatial/Time Selection This paper Applications components resources representation Apply the matching technique in the 3D Approach Network Topology vs App. Communication Pattern Logical Organization vs App. Communication Pattern Implementation and Results Analysis

4 Outline 4 / 22 1 The Resource Mapping Model 2 Triva Implementation 3 Results 4 Conclusion

5 Resource Mapping Model Objectives Merge application behavior with resources utilization Create visual representation of the resources Matching: Placement of processes/threads Input Data Application Traces with Communications Composed of timestamped events Record the behavior of processes/threads Resources Description: graph or hierarchy 5 / 22

6 Resource Mapping Model Objectives Merge application behavior with resources utilization Create visual representation of the resources Matching: Placement of processes/threads 6 / 22

7 Resource Mapping Model Entity Matcher 7 / 22 Detect application components in flow of objects Obtain resource location for them Responsible for the Visualization Base layout Logical Organization (load balancing,...) Network Topology (network utilization, routes,..)

8 Resource Mapping Model (Network Graph) 8 / 22 Network Topology vs App. Communication Pattern Resource location is necessary for processes/threads Static: the resource graph from input Dynamic: communication pattern Output: graph drawing data

9 Resource Mapping Model (Network Graph) Network Topology vs App. Communication Pattern Resource location is necessary for processes/threads Static: the resource graph from input Dynamic: communication pattern Output: graph drawing data Visual Representation in 3D 9 / 22

10 Resource Mapping Model (Logical Hierarchy) 10 / 22 Logical Organization vs App. Communication Pattern Resource location is also necessary Static input: hierarchy describing resources Dynamic: application traces Output: treemap customized with application data

11 Resource Mapping Model (Logical Hierarchy) Logical Organization vs App. Communication Pattern Resource location is also necessary Static input: hierarchy describing resources Dynamic: application traces Output: treemap customized with application data Visual Representation in 3D 11 / 22

12 Triva Implementation Developed in Objective-C and C++ Implementation using existing tools DIMVisual library Pajé Components (the Simulator) Graphviz, Ogre3D, wxwidgets Squarified treemap implemented from scratch Tracing experiments in Grid / 22

13 Results 13 / 22 Applications developed with KAAPI Library Load-balancing through random work-stealing Work-stealing activity traced during execution Results are composed of Triva screenshots

14 Results (Network) 110 processes application, 3 Grid 5000 sites Interconnection represented by red lines Work stealing (WS) requests by orange lines 14 / 22

15 Results (Network) 110 processes application, 3 Grid 5000 sites Interconnection represented by red lines Work stealing (WS) requests by orange lines 15 / 22

16 Results (Network) 16 / processes application, 5 Grid 5000 clusters Size represents number of processes per clusters

17 Results (Hierarchy) 17 / 22 Squarified Treemap on the visualization base Rectangles indicate machine allocation Their size represents the amount of WS requests

18 Results (Scalability + View switching) 18 / 22 Dealing with visualization scalability 2900 processes, 13 clusters, 310 machines Network Graph + Communication Pattern

19 Results (Scalability + View switching) 19 / 22 Dealing with visualization scalability 2900 processes, 13 clusters, 310 machines Logical Hierarchy + Communication Pattern

20 Conclusion 20 / 22 Successful performance analysis of grid applications Take into account the execution environment Network characteristics / Resource utilization Resource Mapping Technique Application analysis considering network topology Performance Visualization with the 3D Approach Structural and Behavioral Representation are mixed Main results Correlation with network topology Lack of locality in work stealing requests Work stealing distribution using squarified treemaps

21 Conclusion Future Work Triva Improve representation of resources Bandwidth utilization for every network link Show (dynamic) latency information / 22

22 Conclusion Future Work Triva Improve representation of resources Bandwidth utilization for every network link Show (dynamic) latency information Tutorial this afternoon 2:50 PM, Building 40, Room / 22