Building scalable service-based applications Wicked Fast

Transcription

1 Building scalable service-based applications Wicked Fast Using Lumada Foundry to build Hitachi Content Intelligence Jonathan Chinitz Product Manager, Content & Data Intellligence September 2017

2 So What s the big deal?

3 Why? Composability Reusability Flexibility Focus Agility Time to Market The ability to build new things from existing pieces [Pratt, 1999] it takes Reusability just 16 minutes implies to some move explicit our cloud management resiliency and maintenance of designs build, service, packaging, that Janitor can adapt distribution, Monkey, when from installation, external code check-in changes to a multi-region configuration, occur. deployment. [Wikipedia] deployment, maintenance and software may appear to be reusable from design upgrade issues. If these issues are not considered, point "ability of of view, a business but will not system be reused to rapidly in practice. respond to change [Wikipedia] [1] [Wikipedia]

4 Lumada Foundry

5 What is Foundry? A platform for the rapid development of service-based applications Foundry Service 1 Service 4 Service 2 Application 1 Service 3 Application 2 Used to build, compose, package, and run enterprise products

6 What is Foundry? A platform for the rapid development of service-based applications My Hitachi Product Service 1 Service 4 Service 2 Application 1 Service 3 Application 2 Used to build, compose, package, and run enterprise products

7 What is Foundry? Foundry provides the common infrastructure My Foundry Hitachi Product Network Service Proxy 1 Service Logging4 Authentication Service 2 Application Metrics 1 Service Scale 3 Application Update 2 Keeping the focus on the solution, not technology hurdles

8 Why Foundry? Product 1 Product 2 Product 3 Service 1 Service 4 Service 1 Service 19 Service 1 Service 23 Service 2 Application 1 Service 2 Application 2 Service 2 Application 3 Foundry Foundry Foundry Build flexible enterprise grade solutions - faster

9 Why Foundry? Product 1 Product 2 Product 3 Service 1 Service 4 Service 1 Service 19 Service 1 Service 23 Service 2 Application 1 Service 2 Application 2 Service 2 Application 3 Service 6 Service 7 Service 13 Service 15 Service 17 Catalog Service 5 Service 6 Service 7 Service 8 Service 9 Service 10 Service 11 Service 12 Service 13 Service 14 Service 15 Service 16 Service 17 Service 18 Build and leverage an expanding catalog of services and plugins

10 Why Foundry? Product 1 Product 2 Product 3 Service 1 Service 4 Service 1 Service 19 Service 1 Service 23 Service 2 Application 1 Service 2 Application 2 Service 2 Application 3 Service 6 Service 7 Service 13 Service 15 Service 17 Customize, Package, Deliver

11 Why Foundry? Health Care Transportation Financial Service 1 Service 4 Service 1 Service 19 Service 1 Service 23 Service 2 Application 1 Service 2 Application 2 Service 2 Application 3 Service 6 Service 7 Service 13 Service 15 Service 17

12 Why Foundry? Health Care Transportation Financial Service 1 Service 4 Service 1 Service 19 Service 1 Service 23 Service 2 Application 1 Service 2 Application 2 Service 2 Application 3 Service 6 Service 7 Service 13 Service 15 Service 17 host OS host OS host OS Physical Server Deploy consistently in physical, virtual, and cloud environments

13 Why Foundry? Health Care Health Care Health Care Service 1 Service 4 Service 2 Application 1 Service 6 Service 7 host OS host OS host OS Physical Server Physical Server Physical Server Seamlessly deploy and scale clustered systems

14 Why Foundry? Health Care Health Care Health Care Service 1 Service 4 Service 2 Application 1 Application 1 Application 1 Service 6 Service 7 host OS host OS host OS Physical Server Physical Server Physical Server Scale and load balance services

15 Why Foundry? Health Care Health Care Health Care Service 1 Service 4 Service 2 Application 1 Service 2 Application 1 Service 2 Application 1 Service 6 Service 7 host OS host OS host OS Physical Server Hybrid deployments can be used to distribute load

16 Why Foundry? Health Care Patient Records Mayo Clinic Records Service 1 Service 1 Service 7 Service 1 Service 7 Service 2 Application 1 Service 2 Application 1 Service 2 Application 1 Service 6 Service 6 Service 6 Custom UI host OS host OS host OS Physical Server Physical Server Physical Server Every Foundry-based product is also a Foundry

17 Foundry Platform Features Foundry Service SDK & Test Harness Branding Scale-out / Clustering Business Object Management Deployment Authentication / Authorization Monitoring / Logging / Metrics Clustered Filesystem Persistence Services Data Connectors Packaging Update Framework Network Proxy And much more

18 What is a Service? Foundry Database Network Proxy Processing Engine A Service is one or more containerized processes running in the Foundry. Services are individually deployable and provide a single capability. Message Queue host OS hw or hypervisor Internal External Many services are internal components, and may not be used directly by end users. Can be proprietary, or open source.

19 What is an Application? Foundry Applications are services exposed outside the cluster. Database Network Proxy Processing Engine Admin App Search App API CLI UI These services provide end-user access to publicly facing UI, CLI, and APIs. Message Queue host OS hw or hypervisor Internal External They are typically secured to authenticated users and are not generally used by code within the cluster.

20 Foundry Instances and Clusters

21 Foundry Instance A Foundry Instance is a machine (server or node) running a collection of Foundry services

22 Foundry Cluster A Foundry Cluster is made up of Master instances and Worker instances A Foundry Cluster is a a collection of Foundry Instances

23 Sample Instance

24 Foundry Monitoring and Alerting

25 Foundry Cluster Monitoring Foundry Monitoring can be done at a variety of levels Cluster metrics

26 Foundry Instance Monitoring Instance metrics Instance services Instance system services

27 Foundry Services Monitoring Foundry services metrics No. of instances State Cost Usage

28 Foundry Individual Service Monitoring Foundry service metrics Foundry Service instances Foundry service events

29 Foundry System Events Foundry system events Are searchable by date, severity, user, and message type

30 Foundry Alerting Foundry alerts

31 Foundry Security

32 Foundry Authentication / Authorization Permissions Roles Groups admin:read admin:write admin:delete Data Scientist admin:read admin:write admin:delete Customer Group 1 Data Scientist Security

33 Foundry Authentication / Authorization Many system components are pluggable, including Identity Providers

34 Foundry Authentication / Authorization Configure an Identity Provider to build a Security Realm

35 Foundry Authentication / Authorization Register groups from the Identity Provider

36 Foundry Authentication / Authorization Choose permissions to build custom roles. Assign one or more roles to groups.

37 Foundry Authentication / Authorization Log into any available security realm, with single sign on. Users are granted permissions from the groups they belong to.

38 Foundry Service Management

39 Foundry Service Management

40 Foundry Service Management

41 Foundry Service Management

42 Foundry Update Framework Update framework integrated with Connector framework Product-specific update framework integrates service updates

43 Foundry Update Framework Each product is responsible for loading, verifying, pre-checking, and applying updates

44 Plugins, Plugins, Plugins

45 Foundry Plugins

46 Foundry Plugins

47 Foundry Plugins

48 Foundry Plugins

49 Foundry Plugins

50 Foundry Plugins

51 Plugin SDK The ultimate in Extensibility Sample Source Code Test/Debug them using our Test Harness. HCI Plugin SDK - Build your own connectors and pipeline stages. Start with our sample source code. Incorporate your own libraries or 3 rd party SDKs. CONNECTOR When they are ready upload them to HCI and use them in workflows. STAGE

52 Building Hitachi Content Intelligence (HCI) with Lumada Foundry

53

54

55 solr index-plugin.jar hds content-connector-plugins.jar hci search-app-service.tar.gz hci workflow-designer-service.tar.gz solr service.tar.gz

56 solr index-plugin.jar hds content-connector-plugins.jar hci search-app-service.tar.gz hci workflow-designer-service.tar.gz solr service.tar.gz

57

58