LOG AGGREGATION. To better manage your Red Hat footprint. Miguel Pérez Colino Strategic Design Team - ISBU

Transcription

1 LOG AGGREGATION To better manage your Red Hat footprint Miguel Pérez Colino Strategic Design Team - ISBU

2 Agenda Managing your Red Hat footprint with Log Aggregation The Situation The Challenge The Solution

3 THE SITUATION

4 Cloud Deployments They do really scale... Higher scalability More workloads per physical machine (multi-tenant) Network and Storage also Software Defined Containers and Microservices providing more granularity

5 Cloud Deployments Act as one single thing and need to be managed and operated as one Source:

6 THE CHALLENGE

7 Data (What) Data + Information flow in Log Aggregation Generate Ingest Collect Derived from: Process Store Query View

8 Personas (Who) That can use Log Aggregation Developer User / Marketing App Analysis & Debug Access to stats Cloud Ops Security Engineer Log Aggregation Root Cause Analysis Sec Analysis, Audits Service Designer IT Manager Access to aggregated data, i.e. SLA, usage Monitoring Provides Events, Consumes Logs

9 Personas (Motivation) That need Log Aggregation Application (multi-tiered) launched from CloudForms returns error Cloud Suite User I want to proactively know about active or potential degradation of service Cloud Ops (Apps) User reports that their VM request failed and returned error Cloud Ops (OpenStack) My recent commit resulted in Jenkins test failure Developer (OpenShift)

10 Situational Awareness (Why) Or the need of it! Source:

11 THE SOLUTION

12 Architecture Proposed General Architecture C Host C M C M C General Visualization Real Time Analytics and Response Data Store N Storage N Bus Archive Slide Credit: Tushar Katarki [@tkatarki] N Legend C Collector M C Message Client N Normalizer

13 Implementation Introduction to EFK Log Source TCP/UDP HTTP File: Text Stdout: CSV, JSON, MessagePack syslog/journal Fluentd Parsing Filtering Enriching Deleting Output Buffering Slide Credit: Tushar Katarki ElasticSearch Kibana Index and store data and metadata making search fast and reliable User Interface for: Search Graph Dashboard

14 Current Status Multi-Tenant Access Being delivered and supported OpenShift Container Platform 3.5 Kuberentes Services Full EFK stack provided as containers Log files Journal host logs Fluentd Kibana Kibana Syslog Master Nodes OpenStack Platform 10 Fluentd as log collector Elastic Elastic Elastic search search search App inside container Log files Journal Red Hat Virtualization Coming Soon! App inside container Curator Fluentd Syslog... Application Nodes Diagram Credit: Tushar Katarki Infra Nodes

15 BEYOND...

16 Common Data Model Ingestion pipeline To ensure integration and interoperability What Is It? A Data Model for Logs (and other data) to identify and tag data (i.e. log fields) Indexing and Storage Why? Alignment/Correlation with different RH products Improved maintainability of Data Better presentation/data consumption Enables 3rd party ecosystem Facilitates deep learning analysis of data Consumption pipeline

17 Common Data Model Example... Data extracted: Container name Pod name Namespace name Docker container ID K8S data queried: CDM A 1 B 2 C 3 Pod UID Pod labels Pod host Namespace UID. All merged into output log in JSON Format Images Credit: Anton Sherkhonov [@peatz]

18 User Experience Prototyping and validating dashboards for users Slide Credits: Peter Portante & Vince Conzola

19 Exploring different approaches Prototyping with alternative toolsets with partners Slide Credits: Luca Rosellini (Keedio)

20 ACTION!

21 How are you doing it? Please, provide your feedback...

22 THANK YOU plus.google.com/+redhat facebook.com/redhatinc linkedin.com/company/red-hat twitter.com/redhatnews youtube.com/user/redhatvideos

23