TAKE CONTROL OF LOGS WITH ELASTICSEARCH

Transcription

1 TAKE CONTROL OF LOGS WITH ELASTICSEARCH

2 AGENDA Benefits of Collec;ng Log Data Why Use Elas;csearch (and the Elas;c Stack) Using the Elas;c Stack to Collect Logs Learning about your System

3 Why Collect Log Data?

4 WHAT LOGS TELL US Local error messages Isolated data Point-in-;me data

5 LOG SOURCES PIA_Access Servlet Logs (IDDA) APPSRV Anything you want

6 BENEFITS Search for log data in one loca;on Correlate events between servers See trends in log events Collect data for future decisions Capture overall system health Make presy graphs!

7 Why Elasticsearch?

8 ELASTICSEARCH ADVANTAGES Control your own data You already (or will soon) use Elas;csearch Collect only the data you want Does not require PeopleSoW Performance Monitor Collect more than PeopleSoW data Easily build your own metrics Free and open source

9 ELASTICSEARCH VERSIONS PeopleTools uses Elas;csearch Kibana 4.5, Logstash 2.4 Elas;c Stack 5.3 is latest Can I use my PeopleSoW Elas;c install for logs? orcl_acl plugin breaks Kibana License Restrictions

10 Using the Elastic Stack

11 ELASTIC STACK ARCHITECTURE Collect Enrich Index Search

12 INSTALL ELASTIC STACK Elas%csearch Kibana 1. Download Elasticsearch 1. Download Kibana 2. Unzip Elasticsearch 2. Unzip Kibana 3. Run bin\elasticsearch.bat 3. Run bin\kibana.bat

13 INSTALL ELASTIC STACK Logstash 1. Download Logstash 2. Unzip Logstash 3. Configure Filters 4. Run bin\logstash.bat

14 INSTALL ELASTIC STACK Filebeat Topbeat/Metricbeat 1. Download Filebeat 1. Download Topbeat 2. Unzip Filebeat 2. Unzip Topbeat 3. Configure Files and Output 3. Configure Output 4. Run bin\filebeat.bat 4. Run bin\topbeat.bat

15 INSTALL FILEBEAT Install Filebeat with Puppet puppet module install pcfens-filebeat Configure Logstash output with Puppet class { 'filebeat': outputs => { 'logstash' => { 'hosts' => [ 'elastic.psadmin.io:5044', ], }, }, }

16 CONFIGURE FILEBEAT filebeat: prospectors: - paths: - /psoft/logs/hrapp003p/pia/pia_access.log fields: domain: hr92prd server_type: webapp region: PRD host: hrapp003p input_type: log document_type: access_log scan_frequency: 10s tail_files: true

17 CONFIGURE FILEBEAT $pia_domain_list.each $domain_name, $pia_domain_info { filebeat::prospector {"${domain_name}-web": paths => [ } } "${pia_domain_info['ps_cfg_home_dir']}/webserv/${domain_name}/servers/pia/logs/pia_access.log", ], doc_type => 'access_log', input_type => 'log', ignore_older => '24h', fields_under_root => 'true', tail_files => 'true', fields => { domain => "${domain_name}", server_type => hiera('server_type'), region => hiera('region'), }

18 LOGSTASH CONFIGURATION Give structure to incoming data Collect, Enrich, Transport input{} filter{} output{}

19 CONFIGURE LOGSTASH input { beats { type => beats port => 5044 } } filter { } output { elasticsearch { hosts => [ elastic.psadmin.io:9200 ] workers => 4 index => "logstash-%{+yyyy.mm.dd}" } }

20 LOGSTASH FILTERS filter { grok { match => { "message" => %{WL_IO_EXTENDED}"} } grok { match => { "request" => %{PS_URI_REQUEST}"} } date { } match => [ "timestamp", "MMM dd yyyy HH:mm:ss","MMM d yyyy HH:mm:ss", "ISO8601" ] } useragent { source => useragent, target => agent }

21 GROK EXPRESSIONS /psc/hr92prd/employee/hrms/c/role_manager.tl_mss_ee_srch_prd.gbl PS_URI_REQUEST %{WORD:servlet}(/%{WORD:site_name})?(/% {WORD:portal}/%{WORD:node}/)?(%{WORD:content_type}/(% {PS_WEBLIB:iscript} %{WORD:menu}\.%{PS_COMPONENT:component})?)? (\?%{GREEDYDATA:query_string})? PS_COMPONENT %{WORD:componentName}\.%{WORD:market} hsp://grokdebug.herokuapp.com

22 Explore Log Data!

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40 OPERATIONS DASHBOARD

41 OPERATIONS DASHBOARD

42 OPERATIONS DASHBOARD

43 OPERATIONS DASHBOARD

44 TRAFFIC DASHBOARD

45 TRAFFIC DASHBOARD

46 SERVER DASHBOARD

47 SERVER DASHBOARD

48 SERVER DASHBOARD

49 SPONTANEOUS STRESS TEST

50 SHARE DATA

51 LOG CORRELATION

52 LOG CORRELATION

53 DATA RETENTION Elas;c Curator Define reten;on ;mes per index curator --host servername --port 9200 delete indices --older-than 21 --time-unit days --timestring %%Y.%%m.%%d

54 psadmin.io/reconnect psadmin.io Community

55 Please Complete Your Session Evaluation Evaluate this session in your COLLABORATE app. Pull up this session and tap "Session Evaluation" to complete the survey. Session ID: