X-road v6 monitoring. Toomas Mölder May 4, 2017 X-Road Community Spring 17

Transcription

1 X-road v6 monitoring Toomas Mölder May 4, 2017 X-Road Community Spring 17

2

3

4

5 Contents Benefits, why? X-Road monitoring options Environmental Operational Usage statistics

6 Monitoring in general Ensures that disruption of operational processes are found specified / localized prevented

7 Some benefits from monitoring and service usage log of X-Road Management decisions on different levels based on X-Road usage log System is up and running properly? Peaks of usage => capacity planning Process map, processing patterns => understanding of e-state mechanisms Anomaly detections => security improvements Usage reports => Know your customer

8 Monitoring options Environmental monitoring fetching security server metrics, monitoring of the X-Road environment: details of the security servers such as operating system, memory, disk space, CPU load, running processes and installed packages, etc. Operational monitoring monitoring of operational statistics such as which services have been called, how many times, what is the average response time, etc.

9 Environmental monitoring getsecurityservermetrics. Available starting from proxyversion 6.8.6, PR-TARGETSS Xroad-monitor sensors: SystemMetricsSensor DiskSpaceSensor OsInfoLister ProcessLister XroadProcessLister PackageLister TODO: CertificateLister The Java Management Extensions Messaging Protocol JMXMP ( sample link)

10

11 Operational monitoring getsecurityserveroperationaldata - downloading operational data of the specified time period of the security server getsecurityserverhealthdata - downloading health data of the security server As a regular X-Road service and JMXMP (local network only) PR-OPMON, PR-OPMONJMX Available starting from proxyversion 6.9.5

12 getsecurityserveroperationaldata Timestamp Parties Service Additional data Stats Parties - Client or Producer: XRoadInstance, MemberClass, MemberCode, SubsystemCode, SecurityServerAddress Service: Code, Version Additional data RepresentedParty, MessageID, MessageUserID, MessageIssue Stats: Timestamps of different steps (1-8), performance metrics Size SoapSize, MimeSize, AttachmentCount Status succeeded, or SoapFault Code and String

13 CS monitoring Data request - response Monitoring data requestints Client-specific monitoring data requestoutts 1 Client SS responseoutts responseints responseoutts Producer-specific monitoring data requestints Producer SS 5 responseints 4 requestoutts Client DB Client monitoring Producer monitoring Producer DB CS Central Server monitoring application SS Security Servers of Client and Producer DB adapter servers / databases of Client and Producer Request/Response + In/Out + TS timestamps from different steps 1-8 Request and Response sizes include SoapSize, MimeSize and AttachmentCount Monitoring systems of Client and Producer can request data on their own from data exchange partner security server

14 getsecurityserveroperationaldata JSON (gzipped) Store in DB Process with Python, R,

15 Service usage monitoring and statistics, timetable 1. Will be published as Opendata Jul 2017 NB! There is only metadata, no content of queries/responses NB! All auth.user-specific data, ie MessageuserID removed NB! Timestamps only with hour accuracy NB! At start available only for members 2. Statistics, reports to members Aug Anomalies, algorithm to find errors, misusage, disruptions from service usage logs Sep 2017

16 Data Quality issues, please ensure that Contact data in RIHA incl correct of businees and tech contact WSDL is WSDL and uploaded/available in RIHA Subsystem names also in english Service names also in english MessageID should be UUID! MessageIssue considered as reference into your Document Management System, might be hashed

17 X-Road usage stats Jan 2003 Apr #Queries #Producers #Services #Consumers Aste (#Queries)

18 Number of Queries Number of Producers, Services and Consumers Number of Queries, Producers, Services and Consumers, Jan - Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec #Query #Producer #Service #Consumer

19 Producers, Month and Rank Filter Count, Jan - Apr 2017 aar; ; 6% ehis; ; 6% arireg; ; 6% kirst; ; 6% liiklusregister; ; 7% digilugu; e-toimik; 616; 3% ; 2% emtav5; ; 3% emta; ; 29% rets; ; 19% emta rets dhl liiklusregister arireg ehis aar kirst emtav5 digilugu e-toimik dhl; ; 13%

20 Services, Month and Rank Filter Count, Jan - Apr 2017 ametlikuddokumendid.getsendstatus; adsv5.adsaadrotsing; arireg.detailandmed_v5; rets.soodustuse_tapsustamin rets.retsepti_vaatamine_apte ; 2% 099; 1% 529; e; 664 1% 797; 1% ek; ; 4% kirst.kindlustatus; ; 5% emta.vpt; ; 36% ehis.sais_oppekavad; ; 5% emtav5.torik; ; 5% rets.koostoime_list; ; 6% digilugu.hl7; ; 6% emta.vpt dhl.receivedocuments aar.taitmised digilugu.hl7 rets.koostoime_list emtav5.torik ehis.sais_oppekavad kirst.kindlustatus rets.retsepti_vaatamine_apteek ametlikud-dokumendid.getsendstatus adsv5.adsaadrotsing arireg.detailandmed_v5 rets.soodustuse_tapsustamine aar.taitmised; ; 8% dhl.receivedocuments; ; 20%

21 Kohtutäiturite Majandus- ja Pankrotihaldurite Kommunikatsiooniministeeriu aktsiaselts Tallinna Andevis; kohtutäitur ; Elin Koda; Hariduse Infotehnoloogia 021; m; 2% ; Vilippus; Ridango 1% 864 AS; 072; 7801% 820; 1% Sihtasutus; ; 4% Riigi Infosüsteemi Amet; ; 5% Eesti Metsa- ja Puidutööstuse Liit; ; 6% CN Kodanik; ; 8% Politsei- ja Piirivalveamet; ; 8% Consumers, Month and Rank Filter Count, Jan - Apr 2017 KREEDIX OÜ; ; 27% KREEDIX OÜ Eesti E-tervise Sihtasutus Registrite ja Infosüsteemide Keskus Politsei- ja Piirivalveamet CN Kodanik Eesti Metsa- ja Puidutööstuse Liit Riigi Infosüsteemi Amet Hariduse Infotehnoloogia Sihtasutus Kohtutäiturite ja Pankrotihaldurite Koda Majandus- ja Kommunikatsiooniministeerium aktsiaselts Andevis Tallinna kohtutäitur Elin Vilippus Ridango AS Registrite ja Infosüsteemide Keskus; ; 11% Eesti E-tervise Sihtasutus; ; 25%

22 Thank you! Toomas Mölder ria.ee/ee/x-tee-kontakt.html X-teed tutvustavad materjalid on valminud EL struktuuritoetuse toetusskeemist Infoühiskonna teadlikkuse tõstmine Euroopa Regionaalarengu Fondi rahastusel.