collectd An introduction

Transcription

1 collectd An introduction

2 About me Florian "octo" Forster Open-source work since 2001 Started collectd in 2005

3 Agenda collectd Aggregation of metrics Alerting with Icinga

4 Agenda collectd Aggregation of metrics Alerting with Icinga

5 collectd Daemon collect metrics mangle / transport metrics store metrics (no retrieve)

6 collectd Open-source project MIT and GPL licensed Platform independent Linux, BSD, Solaris, AIX, HP-UX, Windows via SSC Serv (non-free)

7 collectd Agent based design Runs on each host Extensible via plugins Language bindings (Perl, Python, Java) "exec" plugin, e.g. shell scripts

8 collectd 95+ "read" (input) plugins System metrics (e.g. CPU, memory) Application metrics (e.g. MySQL) Other (Xeon Phi, SNMP, OneWire)

9 collectd 15+ "write" (output) plugins Graphite RRDtool RRDCacheD Riemann MongoDB HTTP (generic)

10 collectd Example configuration # Input # Output LoadPlugin cpu LoadPlugin write_graphite LoadPlugin memory <Plugin write_graphite> <Node "default"> LoadPlugin df <Plugin df> MountPoint "/" ValuesPercentage true </Plugin> Host "graphite.example.com" </Node> </Plugin>

11 collectd collectd's write_graphite plugin Sends metric to Graphite TCP or UDP transport Metric names somewhat adjustable Monitoring mit Graphite (15:30 in this room, German)

12 Agenda collectd Aggregation of metrics Alerting with Icinga

13 Aggregation Aggregates often more useful for alerting e.g. sum over CPUs, minimum RTT Metric storage often I/O bound Dashboards require "sane" amount of information

14 Aggregation CPU Disk collectd Memory Aggregation Graphite

15 Aggregation Load the Aggregation plugin Select (filter) applicable metrics Group by metric type and other fields Aggregate functions (e.g. sum)

16 Aggregation Load the aggregation plugin LoadPlugin aggregation example.com/battery/percent-charged <Plugin aggregation> example.com/cpu-0/cpu-idle <Aggregation> example.com/cpu-0/cpu-user example.com/cpu-0/cpu-wait example.com/cpu-1/cpu-idle example.com/df-root/df_complex-free example.com/df-root/df_complex-used </Aggregation> </Plugin> example.com/df-root/df_complex-rsvd

17 Aggregation: Selection Five fields usable for selection Host Plugin PluginInstance Type (mandatory) TypeInstance

18 Aggregation: Selection Select metrics LoadPlugin aggregation example.com/cpu-0/cpu-idle <Plugin aggregation> example.com/cpu-0/cpu-user <Aggregation> example.com/cpu-0/cpu-wait Plugin "cpu" example.com/cpu-1/cpu-idle Type "cpu" example.com/cpu-1/cpu-user example.com/cpu-1/cpu-wait example.com/cpu-2/cpu-idle example.com/cpu-2/cpu-user </Aggregation> </Plugin> example.com/cpu-2/cpu-wait

19 Aggregation: Grouping Four fields usable for selection Host Plugin PluginInstance TypeInstance One field unspecified (or more)

20 Aggregation: Grouping Configure grouping LoadPlugin aggregation example.com/cpu-???/cpu-idle <Plugin aggregation> example.com/cpu-???/cpu-user <Aggregation> Plugin "cpu" Type "cpu" GroupBy Host GroupBy TypeInstance </Aggregation> </Plugin> example.com/cpu-???/cpu-wait

21 Aggregation: Functions Up to six aggregate functions Count Sum Minimum Maximum Average Standard deviation

22 Aggregation Select aggregate function(s) LoadPlugin aggregation example.com/cpu-sum/cpu-idle <Plugin aggregation> example.com/cpu-sum/cpu-user <Aggregation> Plugin "cpu" Type "cpu" GroupBy Host GroupBy TypeInstance CalculateSum true </Aggregation> </Plugin> example.com/cpu-sum/cpu-wait

23 Aggregation Creates additional metrics Use chains to filter out unwanted "raw" metrics. Usable on client and/or server.

24 Agenda collectd Aggregation of metrics Alerting with Icinga

25 Alerting Load the Unixsock plugin Query and check values with collectd-nagios Both come with collectd

26 Alerting Load the Unixsock plugin LoadPlugin unixsock <Plugin unixsock> SocketFile "/var/run/collectd-unixsock" SocketGroup "collectd-nagios" SocketPerms "0660" DeleteSocket true </Plugin>

27 Alerting Query values with the Unixsock plugin -> GETVAL example.com/cpu-average/cpu-wait <- 1 Value found <- value= e00

28 Alerting collectd-nagios queries and checks metrics Ranged -w (warn) and -c (critical) options Conforms to Icinga's best practices

29 Alerting Example: collectd-nagios $ collectd-nagios -s /var/run/collectd-unixsock \ > -n cpu-average/cpu-wait -H example.com \ > -w '0:10' -c '0:25' OKAY: 0 critical, 0 warning, 1 okay value= ;;;;

30 Alerting commands.cfg services.cfg define command{ command_name check_cpuio_collectd command_line collectd-nagios \ define service{ use generic-service host_name example.com service_description I/O wait check_command \ -H $HOSTNAME$ \ -n cpu-average/cpu-wait \ -w $ARG1$ -c $ARG2$ } check_cpuio_collectd!10:!5: }

31 Alerting What's next? Use "passive checks" Let collectd push metrics to Icinga 2? Bring on the patches!

32 Thank you! Thank you!

33 Questions? It's time for Questions