Infrastructure at your Service. Elking your PostgreSQL Database Infrastructure

Transcription

1 Infrastructure at your Service.

2 About me Infrastructure at your Service. Arnaud Berbier Senior Consultant Page 2

3 Agenda 1.Playground Infrastructure 2.Elastic Search Stack 3.PostgreSQL Monitoring 4.Conclusion Page 3

4 Playground Infrastructure > Big Picture > Use case and deployment process Page 4

5 Playground Infrastructure Big Picture Version SSH /24 Version r (Qt5.6.2) Postgres Clusters 10.0 Beats Elastic Node 0 Visualize Ingest Ship Ship PGPool II vmtestpgsql /32 vmtestpgpool /32 PG Primary Node PGPool Node 0 Health Check Beats Heartbeat vmtestelastic /32 Streaming Replication Elastic Stack Kibana Elastic Search Beats PG Standby Node PGPool Node 1 vmtestpgsql /32 vmtestpgpool /32 Page 5

6 Playground Infrastructure Use case and deployment process Database Clusters Elastic Stack Server Dashboard import Metric Agent Deployment PGPool Metric Agent Deployment Page 6

7 Elastic Search Stack > What does ELK mean? > What s a Beat? > Is all the stack Open Source? > How much costs the cloud offering? Page 7

8 Elastic Search Stack What does ELK means? > ELK is a complete monitoring platform - Elastic Search Stack > At the beginning Elastic Stack was commonly known as ELK > E - Elastic Search > Store, Search and Analyze > L Logstach > Collect, Enrich and Transport > K Kibana > Explore, Visualize and Share DATA DATA DATA INFORMATION DATA DATA DATA TRANS INFORMATION Page 8

9 Elastic Search Stack What does ELK means? > It evolved to be a full Monitoring Stack User Interface Visualization Store, Index & Analyzse Kibana Elastic Search X-Pack Security Alerting Monitoring Ingest Logstash Beats Reporting Graph Elastic Cloud Page 9

10 Elastic Search Stack What s a Beat? > Lightweight Data Shippers to ingest data into Elastic Page 10

11 Elastic Search Stack Is all the stack open source? Page 11

12 Elastic Search Stack Is all the stack open source? > Components support over subscriptions Elastic Search X-Pack X-Security Elasticsearch Kibana Beats Logstash Monitoring Multicluster Configurable Data retention Authentication + Encryption Role-Based access ctrl Audit Encryption at rest support Open Source Basic Gold Platinum Page 12

13 Elastic Search Stack Is all the stack open source? > Components support over subscriptions X-Pack Alerting via Watcher Reporting Graph Analytics & Visualization Dev Tools Search Profiler Open Source Basic Gold Platinum Support Support Coverage BH 24x7 Response Times Unlimited # of Incidents Crit. 4h L2: 1d L3: 2d Crit. 1h L2: 4h L3: 1d > Details : Page 13

14 Elastic Search Stack How much cost the cloud offering? > Elastic Cloud Offering Host Elastic Search and Kibana Page 14

15 PostgreSQL Monitoring > Postgres Cluster Settings Page 15

16 PostgreSQL Monitoring Postgres Cluster Settings > What s need to be done on the Postgres Clusters > Filebeat log harvesting > The elastic stack process owner needs access to the Postgres log files > log_file_mode in pg_settings parameters postgres=# select name, setting, unit from pg_settings where name like '%log_file%' or name like '%log_dir%'; name setting unit log_directory /app/postgres/admin/dbixchange01/pg_log log_file_mode 0776 log_filename postgresql-%a.log (3 rows) > This correspond to file mode 666 Page 16

17 PostgreSQL Monitoring Postgres Cluster Settings > What s need to be done on the Postgres Clusters > Metric Beat postgresql > A dedicated user for DB connection > Allowed access to the database > Elastic servers need to be accessible by the deployed beat agents > Firewall > Network Page 17

18 Conclusion Pro & contra > Easy to deploy > Ingest lots of data of any log type > Already have predefined metric such as > Most of the required features needs subscription Page 18

19 Any questions? Please do ask Infrastructure at your Service. Arnaud Berbier Senior Consultant We look forward to working with you! Page 19