Making Sense of your Data

Transcription

1 Making Sense of your Data Building A Custom DataSource for Grafana with Vert.x Gerald Mücke DevCon5

2 About me 3 IT Consultant & Java Specialist at DevCon5 (CH) Focal Areas Tool-assisted quality assurance Performance (-testing, -analysis, -tooling) Operational Topics (APM, Monitoring)

3 The Starting Point 4 Customer stored and keep response time measurement of test runs in a MongoDB Lots of Data Timestamp & Value No Proper Visualization

4 What are timeseries data? a set of datapoints with a timestamp and a value value time

5 What is MongoDB? 6 MongoDB NoSQL database with focus on scale JSON as data representation No HTTP endpoint (TCP based Wire Protocol) Aggregation framework for complex queries

6 What is Grafana? A Service for Visualizing Time Series Data Open Source Backend written in Go Frontend based on Angular Dashboards & Alerts

7 Grafana Architecture 8 Browser Angular UI Data Source Data Source Plugin... Grafana Server Implemented in GO Persistence for Settings and Dashboards Offers Proxy for Service Calls Proxy DB DB

8 Datasources for Grafana 9 Browser Angular UI Data Source Plugin Angular JavaScript Grafana Server Implemented in GO Persistence for Settings and Dashboards Offers Proxy for Service Calls HTTP Datasource

9 Connect Angular Directly to 10 Mongo?

10 From 2 Tier to 3 Tier 11 Grafana (Angular) Mongo DB Grafana (Angular) HTTP Datasource Service Mongo Wire Protocol Mongo DB

11 Start Simple 12 SimpleJsonDatasource (Plugin) 3 ServiceEndpoints /search Labels names of available timeseries /annoations Annotations textual markers /query Query actual time series data

12 /search Format 13 Request { "target" : "select metric", "refid" : "E" } Response [ "Metric Name 1", "Metric Name2", ] An array of strings

13 /annotations Format 14 Request { "annotation" : { "name" : "Test", "iconcolor" : "rgba(255, 96, 96, 1)", "datasource" : "Simple Example DS", "enable" : true, "query" : "{\"name\":\"timeseries A\"}" }, "range" : { "from" : " T12:23:47.387Z", "to" : " T12:24:19.217Z" }, "rangeraw" : { "from" : " T12:23:47.387Z", Response [ { "annotation": { "name": "Test", "iconcolor": "rgba(255, 96, 96, 1)", "datasource": "Simple Example DS", "enable": true, "query": "{\"name\":\"timeseries A\"}" }, "time": , "title": "Marker", "tags": [ "Tag 1", "Tag 2" ] } ] "to" : " T12:24:19.217Z" } }

14 /query Format 15 Request { "panelid" : 1, "maxdatapoints" : 1904, "format" : "json", "range" : { "from" : " T12:23:47.387Z", "to" : " T12:24:19.217Z" }, "rangeraw" : { "from" : " T12:23:47.387Z", "to" : " T12:24:19.217Z" }, "interval" : "20ms", "targets" : [ { "target" : "Time series A", "refid" : "A" },] } Response [ { "target":"timeseries A", "datapoints":[ [1936, ], [2105, ], [4187, ], [30001, ] }, { "target":"timeseries B", ] "datapoints":[ ] }

15 Structure of the Source Data 16 { "_id" : ObjectId("56375bc54f3c4caedfe68aca"), "t" : { "edesc" : "KW001_Partner_Zahlungsprofil-erfassen_Sammelkonto_Smoketest", "eid" : "56375ae24f3c4caedfe68a07", "name" : "KW001_Partner_Zahlungsprofil-erfassen_steppage1_oeffnen_Task", "profile" : "I01", "rnid" : "56375b694f3c4caedfe68aa0", "rnstatus" : "PASSED", "uid" : "anonymous" }, "n" : { "begin" : NumberLong(" "), "value" : NumberLong(283) } }

16 Custom Datasource 17 Should be Lightweight Fast / Performant Simple

17 Microservice? 19 Options for implementation... Java EE Microservice (i.e. Wildfly Swarm) Springboot Microservice Vert.x Microservice Node.js

18 The Alternative Options 21 Node.js Single Threaded Child Worker Processes Javascript Only Not best-choice for heavy computation Spring / Java EE Multithreaded Clusterable Java Only Solid Workhorses, cumbersome at times

19 Why Vert.x? 22 High Performance, Low Footprint Asynchronous, Non-Blocking Actor-like Concurrency Event & Worker Verticles Message Driven Polyglott Java, Groovy, Javascript, Scala Scalable Distributed Eventbus Multi-threaded Event Loops

20 Asynchronous non-blocking vs 23 Synchronous blocking

21 Event Loop 24 Photo: Andreas Praefcke

22 Event Loop and Verticles 25 3rd Floor, Verticle A 2nd Floor, Verticle B 1st Floor, Verticle C Photo: RokerHRO

23 26

24 27

25 Event Loop 28 Verticle I/O Event Verticle Verticle

26 Event Bus 30 Verticle Verticle Message Eventbus Verticle

27 Multi-Reactor 31 Verticle Verticle Other Vert.x Instance Eventbus Browser Core Core Core Core CPU

28 Event & Worker Verticles 32 Event Driven Verticles Worker Verticles Thread Pool Verticle Verticle Verticle Verticle Verticle Verticle Verticle Verticle Thread Pool

29 Implementing the datasource 33 Http Routes Pre-Processing Optional, for example: Split Request Query Database for Time Data Points Post Processing Optional, for example: Merge, Calculations

30 Step 1 The naive approach 34 Find all datapoints within range

31 Step 2 Split Request 35 Split request into chunks (#chunks = #cores) Use multiple Verticle Instance in parallel (#instances = #cores)? CPU

32 Step 4 Aggregate Datapoints 37 Use Mongo Aggregation Pipeline Reduce Datapoints returned to service

33 Step 5 Percentiles (Service) 38 Fetch all data Calculate percentiles in service CPU

34 Step 6 Percentiles (DB) 39 Build aggregation pipeline to calculate percentiles Algorithm, see DB

35 Datasource Architecture 40 HTTP Request HTTP Response DB HTTP Service Split Request Query Query Database Query Database Query Database Database Merge Result Post Process Post Process Post Process Post Process Eventbus CPU Eventbus CPU

36 41 Remember the CELL BE? CPU of the PS3 Central Bus (EIB) 1 General Purpose CPU 8 SPUs... Runs the Game (Event) Loop Special Processing Sound Physics AI

37 Adding more stats & calculation 42 Push Calculation to DB if possible Add more workers / node for complex (post-) processing Aggregate results before post-processing DB performance is king

38 Takeaways 47 Grafana is powerful tool for visualizing data Information becomes consumable through visualization Information is essential for decision making Vert.x is Reactive, Non-Blocking, Asynchronous, Scalable Running on JVM Polyglott Fun Source code on:

39 Gerald Mücke DevCon5