A prototypical tool to discover architecture changes based on multiple monitoring data sources for a distributed system

Transcription

1 A prototypical tool to discover architecture changes based on multiple monitoring data sources for a distributed system Patrick Schäfer, , Munich Advisor: Martin Kleehaus Chair of Software Engineering for Business Information Systems (sebis) Faculty of Informatics Technische Universität München wwwmatthes.in.tum.de

2 Outline Motivation Modeled & Runtime Architecture Problem Statement Research Questions Prototypical Implementation Operating Environment Discovery Concept Implementation Live Demonstration Limitations sebis 2

3 Motivation Modeled Enterprise Architecture VM VM VM VM sebis 3

4 Motivation Runtime architecture Necessity of knowledge about runtime architecture Detecting deviations from modeled architecture Compliance violations / SLA violations Semi-automated root cause analysis Semi-automated Impact analysis State of the art solutions Application Runtime Architecture Discovery Architecture discovery VM VM VM VM sebis 4

5 Motivation Problem statement Problem: Current discovery solutions don t own knowledge about the entire architecture including dependencies between and within the architectual layers Consequences comparison of modeled and runtime architecture not possible across all layers root cause analysis is challenging and time consuming Incomplete impact analysis? Objective a system, that owns knowledge about all realtime architecture components as well as their inter- and intralayer dependencies sebis 5

6 Research questions 1 2 How to discover the Microservice Architecture by applying distributed tracing? What are component relationship types and how to discover them automatically? 3 How to discover concurrency and synchronization? 4 How to recognize changes in the Microservice Architecture? 5 How to provide a smart user interface for adding business semantic on top of business services? sebis 6

7 Prototypical Implementation Operating Environment Required components of the prototypes operating environment: Instrumented s Repository Eureka Zipkin Server Zipkin Server Client API Gateway Eureka Repository Instrumented Distributed System Legend: Required Component Optional Component Instrumented Distributed Tracing Spans sebis 7

8 Prototypical Implementation Discovery Concept: Architecture Model Business Layer Application Layer Technology Layer Processes Process Activities Activity 1 Activity 2 s 1 2 -Instances Instance 1.1 Instance 1.2 Instance 2 Hardware Hardware 1 Hardware 2 sebis 8

9 Prototypical Implementation Discovery Concept: Architecture Model - Relations Business Layer Application Layer Technology Layer Implicit meaning of relations Processes Process Activities is followed by Activity 1 Activity 2 is executed by is executed by s 1 uses 2 is composed of is composed of is composed of -Instances Instance 1.1 Instance 1.2 Instance 2 runs on runs on Hardware Hardware 1 Hardware 2 sebis 9

10 Prototypical Implementation Discovery Concept: Architecture Model - Relations Business Layer Application Layer Technology Layer Relation classification: Interlayer- and Intralayer-Relations Processes Process Activities Activity 1 Activity 2 s 1 2 -Instances Instance 1.1 Instance 1.2 Instance 2 Hardware Hardware 1 Hardware 2 Hierarchical Interlayer-Relation Non-Hirarchical Intralayer-Relation sebis 10

11 Prototypical Implementation Discovery Concept: Architecture Model - Relations Business Layer Application Layer Technology Layer Relation classification: Direct and Indirect Relations Processes Process Activities Activity 1 Activity 2 s 1 2 -Instances Instance 1.1 Instance 1.2 Instance 2 Hardware Hardware 1 Hardware 2 Direct Relation Indirect Relation sebis 11

12 Prototypical Implementation Discovery Concept: Architecture Model - Relations Business Layer Application Layer Technology Layer Relation classification: Direct and Indirect Relations Processes Process Activities Activity 1 Activity 2 s 1 2 -Instances Instance 1.1 Instance 1.2 Instance 2 Hardware Hardware 1 Hardware 2 Direct Relation Indirect Relation Path of Indirect Relation sebis 12

13 Prototypical Implementation Discovery Concept: Architecture Model - Relations Indirect Relations are necessary Activity 1 Activity 2 API Gateway sebis 13

14 Prototypical Implementation Discovery Concept: Architecture Model - Relations Indirect Relations are necessary Activity 1 Activity 2 API Gateway sebis 14

15 Prototypical Implementation Discovery Concept: Business Layer Modelling via Web UI: Process and Activity entities Process-flow Added to the architecture model via REST API Processes Activities s Instances Hardware sebis 15

16 Prototypical Implementation Discovery Concept: Application / Technology Layer Detection of s, Instances, Hardware Interlayer-Relations of these types Processes Activities s In Realtime by analysing Distributed Tracing Detects new entities only Instances Hardware Cyclic by processing Eureka API Detects new, updated or removed entitites sebis 16

17 Prototypical Implementation Discovery Concept: intralayer Relations Detection of relations between s in Realtime by analysing Distributed Tracing Spans Processes Activities s Instances Hardware Distributed Tracing Span production* *Source: sebis 17

18 Prototypical Implementation Discovery Concept: Business / Application Relations Detection of Interlayer-Relations between Activities and s in Realtime Set of regular expressions mapping to activities Set defined via Web UI and REST API Distributed Tracing Spans applied to ruleset Processes Activities s Instances Hardware sebis 18

19 Prototypical Implementation Implementation: System Components REST API Microservice Architecture Prototype Eureka Repository Distributed Tracing Architecturediscovery service Webapplication Business Layer Modelling Relation Ruleset Architecture Visualization Kafka Stream Websocket Bridge sebis 19

20 Prototypical Implementation Implementation: Environment Zipkin Server replaced by Prototype Prototype extends Zipkin implementation Zipkin Server Features stay complete and unmodified Zipkin Server Client API Gateway Eureka Repository Instrumented Distributed System sebis 20

21 Prototypical Implementation Implementation: Environment Zipkin Server replaced by Prototype Prototype extends Zipkin implementation Zipkin Server Features stay complete and unmodified Prototype and Zipkin Server Client API Gateway Eureka Repository Instrumented Distributed System sebis 21

22 Live Demonstration Scenario: Business Process Start Login Select Provider Search Route Book Route Logout Ende sebis 22

23 Live Demonstration Scenario: Microservice Architecture Accounting Core Start Login DeutscheBahn Mobility Select Provider Search Route Zuul API Proxy Travelcompanion Maps Helper Book Route Logout DriveNow Mobility Ende Business Core sebis 23

24 Live Demonstration Scenario: Microservice Architecture Accounting Core Start Login DeutscheBahn Mobility Select Provider Search Route Zuul API Proxy Travelcompanion Maps Helper Book Route Logout DriveNow Mobility Ende Business Core sebis 24

25 Live Demonstration Scenario: Microservice Architecture Business Core Start Login DeutscheBahn Mobility Select Provider Search Route Zuul API Proxy Travelcompanion Maps Helper Book Route Logout DriveNow Mobility Ende Accounting Core sebis 25

26 Live Demonstration Scenario: Microservice Architecture Business Core Start Login DeutscheBahn Mobility Select Provider Search Route Zuul API Proxy Travelcompanion Maps Helper Book Route Logout DriveNow Mobility Ende Accounting Core sebis 26

27 Live Demonstration Scenario: Microservice Architecture Business Core Start Login DeutscheBahn Mobility Select Provider Search Route Zuul API Proxy Travelcompanion Maps Helper Book Route Logout DriveNow Mobility Ende Accounting Core sebis 27

28 Live Demonstration Scenario: Microservice Architecture Business Core Start Login DeutscheBahn Mobility Select Provider Search Route Zuul API Proxy Travelcompanion Maps Helper Book Route Logout DriveNow Mobility Ende Accounting Core sebis 28

29 Live Demonstration Scenario: Microservice Architecture Business Core Start Login DeutscheBahn Mobility Select Provider Search Route Zuul API Proxy Travelcompanion Maps Helper Book Route Logout DriveNow Mobility Ende Accounting Core sebis 29

30 Live Demonstration Scenario: Microservice Architecture Business Core Start Login DeutscheBahn Mobility Select Provider Search Route Zuul API Proxy Travelcompanion Maps Helper Book Route Logout DriveNow Mobility Ende Accounting Core sebis 30

31 Live Demonstration Scenario: Microservice Architecture - Deployment Eureka Zuul API Proxy Travelcompanion Travelcompanion Config Business Core DriveNow Mobility DeutscheBahn Mobility DriveNow Mobility DeutscheBahn Mobility Prototype Accounting Core Maps Helper Maps Helper Host 0 ( ) Host 1 ( ) Host 2.1 ( ) Host 2.2 ( ) sebis 31

32 Live Demonstration sebis 32

33 Limitations Requires Zipkin-compatible instrumentation as well as Eureka Amount of metainformation depends on instrumentation Raw communications like Database-Access is not detected Virtual Hosts are not detected Business layer is modelled sebis 33

34 B.Sc. Patrick Schäfer Technische Universität München Faculty of Informatics Chair of Software Engineering for Business Information Systems Boltzmannstraße Garching bei München Tel Fax patrick.schaefer@tum.de wwwmatthes.in.tum.de

35 Backup sebis 35

36 Live Demonstration Scenario: Microservice Architecture Zuul API Proxy Business Core DriveNow Mobility Travelcompanion DeutscheBahn Mobility Accounting Core Maps Helper Synchronous communication Asynchronous communication sebis 36

37 Prototypical Implementation Discovery Concept: Architecture Model - Relations Business Layer Application Layer Technology Layer Relation classification: Direct and Indirect Relations Processes Process Activities Activity 1 Activity 2 s 1 2 -Instances Instance 1.1 Instanz 1.2 Instanz 2 Hardware Hardware 1 Hardware 2 Direct Relation Indirect Relation Path of Indirect Relation sebis 37

38 Prototypical Implementation Implementation: Zipkin Server Extension Span Span Span Span Architecturediscoveryservice Collektor REST Collektor Kafka Collektor MySQL Storage MySQL DB REST API Zipkin UI sebis 38

39 Prototypical Implementation Implementation: Zipkin Server Extension Span Span Span Span Architecturediscoveryservice Collektor REST Collektor Kafka Collektor Architecture Discovery Storage (Storage-Proxy) MySQL Storage Architecturediscoveryalgorithm MySQL DB Datamodel REST API REST API MySQL DB Stream Zipkin UI Webapplication sebis 39

40 sebis 40

41 sebis 41

42 sebis 42

43 sebis 43