How to Troubleshoot Databases and Exadata Using Oracle Log Analytics

Transcription

1 How to Troubleshoot Databases and Exadata Using Oracle Log Analytics Nima Haddadkaveh Director, Product Management Oracle Management Cloud October, 2018 Copyright 2018, Oracle and/or its affiliates. All rights reserved.

2 Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, timing, and pricing of any features or functionality described for Oracle s products may change and remains at the sole discretion of Oracle Corporation. Copyright 2015, Oracle and/or its affiliates. All rights reserved. 2

3 Agenda Oracle Management Cloud (OMC) Log Analytics (LA) Why Log Analytics for Oracle Databases? LA Values for Oracle Databases and Exadata Troubleshooting Key LA Use Cases for Oracle DB & Exadata Copyright 2015, Oracle and/or its affiliates. All rights reserved. 3

4 Oracle Management Cloud END USER EXPERIENCE / ACTIVITY APPLICATION MIDDLE TIER DATA TIER VIRTUALIZATION TIER INFRASTRUCTURE TIER Global threat feeds Cloud access Identity Real users Synthetic users App metrics Transactions Server metrics Diagnostics logs Host metrics VM metrics Container metrics Configuration Compliance Tickets & Alerts Security & Network events Infrastructure Monitoring Log Analytics Configuration & Compliance Application Performance Monitoring Security Monitoring & Analytics Orchestration IT Analytics Comprehensive, Intelligent Management Platform Zero-effort Operational Insights Automated Preventative & Corrective Actions Copyright 2015, 2018, Oracle and/or its affiliates. All rights reserved.

5 Oracle Management Cloud - Log Analytics Data Center Application Storage Database EMCC Repository Monitor, aggregate, analyze, search, explore, correlate All Log Data from your applications and infrastructure (on-premise and cloud) in real-time Private Cloud Application Storage Logs + Oracle Public Cloud Application Oracle Operational Data Storage Other Public Cloud Application Database A Cloud Service that leverages a modern, secure, big-data platform Storage Copyright 2015, Oracle and/or its affiliates. All rights reserved.

6 Key use cases in LA IT Operation Operational Intelligence Troubleshooting Root-cause Analysis Business Process Analysis Product Analysis Digital Marketing Customer Experience OMC Log Analytics Copyright 2015, Oracle and/or its affiliates. All rights reserved.

7 Why LA for Oracle Databases and Exadata Troubleshooting? Oracle Databases: Are part of many high-visibility business application in many organizations Are highly instrumented by EM and other Oracle tools BUT No solution to aggregate, analyze and visualize DB related logs/data Copyright 2015, Oracle and/or its affiliates. All rights reserved. 7

8 LA Values for Oracle Databases and Exadata Troubleshooting Complement EM in troubleshooting and root-cause analysis Categorize problems/errors in addition to performance issues Database crashes, Connection failures, Delays in shipping archive logs Backup/recovery failures, Runaway jobs, reports or ETLs, Patching/upgrade failures Application upgrades and behavior changes Credential changes Correlate logs across the DB, Exadata ecosystem Database components: DB, listener, ASM, Clusterware, Storage cells, Infiniband, host Database infrastructure components: Host, VM, Database Firewall, Network & Storage components Database ecosystem: Golden Gate, Backup/recovery, Reporting, ETL, Security etc. Correlate DB problem to other parts of the Application stack Copyright 2015, Oracle and/or its affiliates. All rights reserved. 8

9 Agenda Oracle Management Cloud (OMC) Log Analytics (LA) Why Log Analytics for Oracle Databases? LA Values for Oracle Databases and Exadata Troubleshooting Key LA Use Cases for Oracle DB & Exadata Copyright 2015, Oracle and/or its affiliates. All rights reserved. 9

10 Key Log Analytics Use Cases for DBAs Centralized log collection Collect, aggregate, store logs from across all databases (single instance, RAC, ASM, Clusterware, Exadata, ADWCS) Collect logs from files, databases, syslog, on-demand upload Minimal configuration log ingestion for all database logs Out-of-box log sources and parsers Auto-associated with database entities, parameterized log location based on properties Copyright 2015, Oracle and/or its affiliates. All rights reserved. 10

11 Automated Target Discovery Leverage your investment in Oracle Enterprise Manager Data Collector automates discovery of targets from Enterprise Manager Target definitions Target properties Associations Groups and System definitions Exadata: Hosts, Databases, Cells, etc. All or subset of entities can be harvested Changes synchronized 15 minutes Data Collector extracts target metadata, group and system definitions from EM Oracle Enterprise Manager Cloud Control 12c Repository Cloud Agent on log analytics targets or on Syslog Host Copyright 2015, Oracle and/or its affiliates. All rights reserved. 11

12 Log Source Copyright 2015, Oracle and/or its affiliates. All rights reserved. 12

13 Single-Touch Ingestion of EM-Managed Database Logs Files Trace Logs ASM Logs Clusterware Logs Syslog Alert Logs Listener Logs Audit Tables Exadata ExaWatcher Message Logs OS Tables OSWatcher Secure Logs Custom Cell Alert Logs Copyright 2015, Oracle and/or its affiliates. All rights reserved. 13

14 Key Log Analytics Use Cases for DBAs Out of Box deep Oracle database knowledge All logs automatically classified into commonly known/used error categories Search and Explore logs using familiar error categories memory issues, connection issues, storage issues Dashboards to show overall health of database systems Database Audit logs Dashboard: I.e. Login failures happened (who and how many), monitor all connections Database Alert logs Dashboard: I.e. system crashes, error occurring Database Listener logs Dashboard: I.e. Instance start or shutdown, which client did connect to the Instance? With which user? Which port? Copyright 2015, Oracle and/or its affiliates. All rights reserved. 14

15 OOTB Error Categories Copyright 2015, Oracle and/or its affiliates. All rights reserved. 15

16 Rich OOTB Oracle DB Knowledge Copyright 2015, Oracle and/or its affiliates. All rights reserved. 16

17 How to find errors during RMAN backup or backup failed/stopped? Copyright 2015, Oracle and/or its affiliates. All rights reserved. 17

18 Log Analytics Dashboard Example Database Health Overview Copyright 2015, Oracle and/or its affiliates. All rights reserved. 18

19 Key Log Analytics Use Cases for DBAs Specialized ML algorithms enable smart clustering of log events based on common signature How can I find unique log events? Clustering log events What are my top SQL statements and their trends? SQL clustering Deep analytics on log events Linked by common attribute value(s) Audit log object analysis identify anomalies in database object access patterns, error conditions, SQL execution time Copyright 2015, Oracle and/or its affiliates. All rights reserved. 19

20 Machine Learning - Cluster Cluster log events by physical structure and analyze the variable data that LA clustering has extracted Clusters: Cluster events based on similarity in their patterns Potential Issues: These are events with different variants of severity like Error, Fault, Fatal, Warning or set of terms that are semantically similar to these Outliers: Number of outliers within the total cluster, which has only 1 occurrence Trends: Show trend of each clustered group; Correlate clustered events that show similar trends Copyright 2015, Oracle and/or its affiliates. All rights reserved. 20

21 Cluster Copyright 2015, Oracle and/or its affiliates. All rights reserved. 21

22 Cluster Copyright 2015, Oracle and/or its affiliates. All rights reserved. 22

23 Cluster Copyright 2015, Oracle and/or its affiliates. All rights reserved. 23

24 Cluster Copyright 2015, Oracle and/or its affiliates. All rights reserved. 24

25 Key Log Analytics Use Cases for DBAs Specialized ML algorithms enable smart clustering of log events based on common signature How can I find unique log events? Clustering log events What are my top SQL statements and their trends? SQL clustering Deep analytics on log events Linked by common attribute value(s) Audit log object analysis identify anomalies in database object access patterns, error conditions, SQL execution time Copyright 2015, Oracle and/or its affiliates. All rights reserved. 25

26 Top SQL Trends Copyright 2015, Oracle and/or its affiliates. All rights reserved. 26

27 Top SQL Trends Copyright 2015, Oracle and/or its affiliates. All rights reserved. 27

28 SQL Clusters Copyright 2015, Oracle and/or its affiliates. All rights reserved. 28

29 Machine Learning - Link Events Deep analytics on log events Linked by common attribute value(s) What Link does: It links events from millions of log records from across log sources which share some common attribute(s) like Transaction ID, ECID, Flow ID or User Name Compute statistics on linked events and analyze for outlier Copyright 2015, Oracle and/or its affiliates. All rights reserved.

30 When to use Link? Log events from multiple applications/tiers or hosts which are related and span time, can be linked together Example: SOA use case that captured an order, notified a user, notified UPS, decremented inventory Use cases with database: Who exactly accessed or changed data in the systems? When was the first access, the last access and duration of access? Is there any anomalous access? Who are the anomalous users? Is there any anomaly in data access pattern? Copyright 2015, Oracle and/or its affiliates. All rights reserved. 30

31 DB SQL Statement Analysis Copyright 2015, Oracle and/or its affiliates. All rights reserved. 31

32 DB SQL Statement Analysis Copyright 2015, Oracle and/or its affiliates. All rights reserved. 32

33 Database Audit Logs Session Analysis Copyright 2015, Oracle and/or its affiliates. All rights reserved. 33

34 DB SQL Event Trend Analysis Copyright 2015, Oracle and/or its affiliates. All rights reserved. 34