Grideye Architecture

Transcription

1 Grideye Architecture Contents 2. Cloud monitoring Virtualization Migration Comparison Baselining Application emulation Agents Controller Web GUI Model Extensibility Tests Conclusions... 9

2 Grideye has drawn experience from many projects in end-to-end performance measurements over IP networks. This includes work in academia at KTH, SUNET, and NorduNET as well as commercial systems at Prosilient, and others. The previous systems focused on end-to-end network measurement between networking equipment, such as servers, probes and base stations, and focus on voice and mobile networks, the Grideye system focusses on aspects of cloud monitoring. 2. Cloud monitoring The plots above show some cloud monitoring plots from the Grideye GUI. What is special with cloud monitoring? The Grideye system catches some key takeaways for cloud monitoring as follows. 2.1 Virtualization Clouds are virtual in the sense that services implemented in clouds are implemented in virtual machines or containers. This means that the execution environment is virtualized by a hypervisor or namespaces. Virtualization has existed for a long time with VMware and other hypervisors, but clouds add to the virtualization level with network virtualization and deployment on unlocalized and often unknown hardware.

3 A potential drawback with virtualization is the sharing of resources which means that behavior is less deterministic than for reserved resources. This means that monitoring is even more important to ensure the performance of a service that is less deterministic than one deployed on a known hardware and at a known location. Grideye makes continuous measurements of a set of metrics important to understand how a given application behaves in a virtualized cloud environment. 2.2 Migration Clouds provide a means for migration from a fixed, bare metal (or virtualized) environment to a more cost-effective cloud deployment. Such a migration should be verified in advance and during migration to ensure that the services work as expected and that the charge corresponds to the services provided. Grideye has been used successfully in cloud migration, most recently in the move by SUNET from an on-premise VMware environment to a public cloud service. 2.3 Comparison Placing a service in a cloud means one can compare different clouds that have different characteristics and price models. Grideye can compare the behavior of example applications, being placed on bare metal, VMs, public cloud, or container environments.

4 The screenshot shows comparison pie charts. The comparison shows that the agents 'se-east-.gep', 'controller' and 'se-tug' show some problems in the metrics 'roundtrip' and 'I/O read latency'. The heatmap below shows that most of these problems occurred early on May 31. A drill-down on I/O read latency shows more detail statistics, namely that 'se-eastgep' has temporary disk read I/O problems on up to 350ms.

5 2.4 Baselining All comparative measurements are made by defining a baseline, and then defining warning and error levels based on the baseline. Baselines are typically made on a well-defined or wanted architecture. Using baselines and spinning up the same example applications in the different environments and monitoring them gives a direct view using plots, heat maps, pie charts and alarms, gives a clear comparison between different environments.

6 The screenshot above shows the baselining of the I/O read latency graphs by a segment of a VMware plot. The warning and error level are defined as 5 and 10 times the average, ie 18 and 37ms. 2.5 Application emulation Cloud applications vary in terms of networking, compute and storage requirements. Grideye emulates applications and performs continuous tests for networking (roundtrip and anomalies), compute (cycles and memory) and storage (read/wrote latency for sequential and random access. You specify a service by profiling it in terms of these components.

7 The figure above illustrates three different applications which are modelled differently using varying processing, storage and access patterns. 2.6 Agents The basic architecture of Grideye is a controller, a set of agents, and a web GUI. The agents run on remote machines as VMs, processes, or containers. Agents are stand-alone example applications and do not interfere with actual applications, thus not causing any security breach concerns. Agents report periodically to the controller and contain NAT traversal techniques for UDP and TCP. Agents are typically started using a 'call home' mechanism, thus self-registering themselves with the controller. Agents run on most Linux platforms, including ARM and i386 and x86 architectures. There is also a docker container. 2.7 Controller The controller periodically communicates with remotes agents and collect statistics. The controller uses a key-value configuration database and is accessed using CLI, Netconf or Rest. The controller periodically pushes aggregated data to an influxdb time-series database used by the Web GUI. 2.8 Web GUI The Web GUI is implemented using client-side scripting and grafana. Pie charts, heat maps and plots are used to present analytics data. Configuration of profile data is also made in the GUI. More detailed configurations are made using the controller CLI. 2.9 Model grouping metrics{ leaf tior{ description "I/O read latency"; type uint32; units "µs"; } leaf tiow{ description "I/O write latency"; type uint32; units "µs"; }

8 } Grideye uses a YANG model to model agents, metrics, user profiles, plots, etc. The YANG model generates a command-line interface, a netconf and REST interface for external interaction. The example above shows an example of two metrics Extensibility Users may want to make their own specialized tests. Grideye therefore has an extensible plugin interface so that users can define their own tests. The example above shows how the Nordic Eduroam network added WLAN specific tests to monitor their wireless connectivity with Grideye, in the example a SUNET raspberry PI. The new plugin polls the status of the Linux /proc/net/wireless file and extracts the values: > cat /proc/net/wireless Inter- sta- Quality Discarded packets Missed WE face tus link level noise nwid crypt frag retry misc beacon 22 wlan0: Tests The currently defined continuous tests are:

9 BogoMIPS - simple cycle performance tests measuring simple loop latency. This essentially provides a baseline metric on basic compute. Cycle count can be inhibited by load, hyper-threading, cycle stealing, etc. A Dhrystones test is also provided. Memory bandwidth - the test measures random memory read latency on a memory chunk significantly larger than the largest cache size. Memory bandwidth is a primary limitation for compute. Read I/O - for file system read. The test ensures that no cache effects are visible, in principle by allocating a large file using random reads. Write I/O which writes sequentially and randomly on a file system disk. System counters including memory, load, clock rate, etc. Networking latencies, including round-trip, variances, los, duplication, reorder, TTL and ToS change, etc. WLAN tests including signal level, quality and noise level. 4. Conclusions Grideye is a cloud monitoring system tailor-made for cloud applications. It incorporates a set of useful tests for cloud monitoring and is extensible for easy addition of metrics. It has proven very useful in the migration of applications to a cloud environment from traditional on-.premise environment, and has also been used to compare different cloud and deployment environments. Copyright Grideye 2016