Maximizing Your MarkLogic and Java Investments Scott A. Stafford, Principal Sales Engineer, MarkLogic

Transcription

1 Maximizing Your MarkLogic and Java Investments Scott A. Stafford, Principal Sales Engineer, MarkLogic COPYRIGHT 13 June 2017MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

2 Photo attributed to smittenkitchen.com SLIDE: 2

3 Photo attributed to smittenkitchen.com SLIDE: 3

4 Photo attributed to disgruntled husband who had to clean oven SLIDE: 4

5 MarkLogic Java Recipes Build and Deploy Run automated testing Load data from a relational database Demo BAM! SLIDE: 5

6 SLIDE: 6

7 REST API SLIDE: 7

8 JAVA CLIENT API REST API SLIDE: 8

9 Build and Deploy

10 MarkLogic v4.1 v5 v6 v7 v8 v9 Mgmt API 1.0 (Read) Client API 1.0 Mgmt API 2.0 (Read/Write) Mgmt API 2.0 (Read/Write) Java Client API v1.0 v2.0 v3.0 v4.0 Mgmt API 2.0 (Read/Write) Content Pump v1.0 v7 v8 v9 Roxy v1.0 v1.5 v1.75 CORB v1.0 v2.0 v2.2 v2.3.2 ml-app-deployer v1.0 v2.6 ml-gradlle v1.0 v2.6 SLIDE: 10

11 HTTP Wrapper Management REST API SLIDE: 11

12 Configuration Coordinator HTTP Wrapper Management REST API SLIDE: 12

13 #!/bin/bash Management REST API SLIDE: 13

14 MarkLogic Python API Management REST API SLIDE: 14

15 Configuration Coordinator ml-app-deployer <java> Management REST API SLIDE: 15

16 ml-app-deployer <java> Management REST API SLIDE: 16

17 java application maven ml-gradle ml-app-deployer Management REST API SLIDE: 17

18 Run automated testing

19 Single Deploy Configuration cookbook:8200 cookbook-modules cookbook-content SLIDE: 19

20

21 Single Deploy Configuration cookbook:8200 XML cookbook-modules cookbook-content SLIDE: 21

22 Single Deploy Configuration cookbook:8200 XML XML cookbook-modules cookbook-content SLIDE: 22

23 Multiple Deploy Configuration Local Deploy cookbook:8200 CI Deploy cookbook-test:8201 cookbook-modules TEST DELETE REPEAT cookbook-content cookbook-test-content SLIDE: 23

24 Use Spring to inject DatabaseClient Remove documents from database before test execution SLIDE: 24

25 Load data from a relational database

26 CONTENT PUMP SLIDE: 26

27 SLIDE: 27

28 Performance Status Reliable Failover SLIDE: 28

29 Basic Job Life-Cycle DataMovement-Based Java Application MarkLogic Cluster Create, Configure, Start Job Threads ML Host Batcher Work Batcher Work Response ML Host Notifications Respose ML Host Listeners-sepcific action Listeners SLIDE: 29

30 SLIDE: 30

31 Demo Time!

32 RECAP MarkLogic Java Recipes Build and Deploy Run automated testing Load data from a relational database SLIDE: 32

33 Scott Stafford SLIDE: 33

34 Backup/Brainstorm

35 MarkLogic Architectures Three Tier - Spring Boot Powered Front End, Spring MVC, MarkLogic Orchestration Framework ingestion/streaming - MarkLogic Spring Batch Microservices - Apache Kafka + MarkLogic SLIDE: 35

36 ROADMAP Continuous Innovation MarkLogic 2 Clustering Document collections Enhanced search (stemming, thesaurus, wildcard) WebDAV support Document locking Backup and restore MarkLogic 3.1 Wildcard queries Directories Forward compatibility XML Contentbase Connector (XCC) 2005 MarkLogic Replication Failover Database Rollback Compartment Security Search Optimizations Search API REST capabilities SSL support Schema Validation Japanese added MarkLogic 6 Accessibility SQL/BI Java/REST/JSON UDFs/Analytics mlcp Hadoop updates MarkLogic 8 JSON storage Server-side JavaScript Semantics Bitemporal Samplestack Java Client API Node.js Client API Management API Incremental backup Flexible replication Enhanced HTTP Server 2015 Cerisent XQE Server 1 ACID transactions Text-based search Linux support Web-based protocols HTTP and XDBC XQuery XDBC support Role-based security MarkLogic 3 Advanced Search Content Processing (including PDF, Word, Excel, PPT) HTTP calls Support for Linux, Windows Server,.NET Facets, co-occur., value lexicons, bucketing Failover MarkLogic 4 Alerting Entity enrichment Geospatial Modular documents Security auditing HA: forest-level failover MarkLogic 5 Database Replication Multi-statement and distributed transactions Point-in-time recovery Search API enhanced Monitoring API/Dashboard Hadoop Integration MarkLogic 7 Semantics foundation Next-gen infrastructure support Elasticity Tiered storage Run on HDFS SLIDE: 36

37 CI Process 1. Deploy container with fresh MarkLogic install 2. Run gradle commands for MarkLogic initialization (mlinit, mlinstalladmin) 3. Load application system data (taxonomies, lookups, system data) 4. Deploy container with middle tier (i.e. Spring Boot, nginx, nodejs) 5. Run tests (gradle test) 6. Run user interface tests SLIDE: 37

38

39 One HTTP request per document to a single host

40 DMSDK

41 Application Context Reader Processor Writer Application Context

42 INITIALIZATION READER PROCESSOR WRITER POST PROCSESING

43 SLIDE: 43

44 [ hip, hip ]

45 SLIDE: 45

46 Get a gallon of milk If (eggs == true) { null; } Get a dozen SLIDE: 46

47 Some people, when confronted with a problem, think, 'I know, I'll use threads' - and then two they hav erpoblesms.

48 Immutability changes everything

49 Knock knock. Race condition. Who's there?

50 There are only two hard problems in distributed systems: 2. Exactly-once delivery 1. Guaranteed order of messages 2. Exactly-once delivery