Reactive Integrations - Caveats and bumps in the road explained

Transcription

1 Reactive Integrations - Caveats and bumps in the road explained

2 @myfear

3 Why is everybody talking about cloud and microservices and what the **** is streaming?

4 Biggest Problems in Software Development High Infrastructure Cost 11% People Methodology Code Systems Developer Velocity 39% Meeting Demand 21% Awful Downtime 9% Release Frquency 20%

5 More Users

6 More Data Rather than acting on data at rest, modern software increasingly operates on data in near real-time. Shortened time-frames for putting changes into production New business models evolve from existing ones New questions need to be answered by existing applications

7 Can t really be served with these CENTRALISED MAINFRAME DECENTRALIZED APPSERVERS

8 That s why we have MICROSERVICES DISTRIBUTED CLOUD

9 That includes CENTRALISED MAINFRAME DISTRIBUTED CLOUD DECENTRALIZED APPSERVERS

10 And ultimately means + + = MORE USERS MORE DATA MORE SYSTEMS MORE CONNECTIONS

11 Scaling the most common programming task INPUT Transform Output

12 to work in a data intensive, world. highly asynchronous,

13 But isn t that _just_ integration?

14 Now playing: «Why SOA failed (Eisele remix)» WS Schema evolution Service Discovery Testing ESBs

15 Synchronous vs. Asynchronous Consumer Producer Consumer Producer Make request Make request Wait for response Continue Work Do something with response

16 But there are already APIs for that?! InputStream and OutputStream in the JDK NIO Channel s Servlet 3.1 ReadListener and WriteListener extensions JDBC ResultSet JSR annotations, Message Driven Beans and JMS, CDI events Java collection Stream and Iterator based APIs JDK9 Flow API.

17 Let s add integration to it Service A Possibly asynchronous boundaries Service C Transform

18 Let s add integration to it Service A Service C 10 msg/s 1 msg/s Transform

19 Let s add integration to it java.lang.outofmemoryerror Service A Service C 10 msg/s 1 msg/s Transform

20 What is needed is Flow Control

21 Searching for a stream library that can handle backpressure

22 map,flatmap,filter,groupby Stream all the things!

23 Reactive Streams to the rescue Reactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments (JVM and JavaScript) as well as network protocols.

24 Towards reusable shapes for integration Source<Integer> Flow<Integer, String> Sink<String> Integer Integer String String Source<Integer,NotUsed>source = Source.range(0, ); Flow<Integer,String,NotUsed>flow = Flow.fromFunction( (Integern) ->n.tostring() ); Sink<String,CompletionStage<Done>>sink = Sink.foreach(str-> System.out.println(str) );

25 What are the real usecases?

26 1) Modernization / Distributed Systems Mainframe Application Storage Cloud Services DB Application Security Search Reactive Microservices

27 2) Moving ETL/Batch style applications Batch DB Sqoop Spark Persistence DB Resource Managers YARN

28 2) to Streaming Architectures. Persistence Microservices DB Events Message Broker Akka Streams Microservices Low Latency

29 Wanna learn more about Streaming Architectures?

30 Mainframe Application Storage Cloud Services DB Application Security Search Reactive Microservices REST JMS SSE JDBC WebSocket?????

31 It depends

32 DB Application REST WebSocket JDBC Reactive Microservices SSE

33 The important parts are streaming and backpressure

34 Meet Alpakka!

35 Reactive Enterprise Integration - Alpakka Project AMQP Connector Apache Geode connector AWS DynamoDB Connector AWS Kinesis Connector AWS Lambda Connector AWS S3 Connector AWS SNS Connector AWS SQS Connector Azure Storage Queue Connector Cassandra Connector Elasticsearch Connector File Connectors FTP Connector Google Cloud Pub/Sub HBase connector IronMq Connector JMS Connector MongoDB Connector MQTT Connector Server-sent Events (SSE) Connector Slick (JDBC) Connector Spring Web Unix Domain Socket Connector File IO Azure Camel Eventuate FS2 HTTP Client MongoDB Kafka TCP

36 An example.. Finally :-D Alpakka H2 Slick Alpakka Slick Akka Streams Akka HTTP

37 <dependency> <groupid>com.lightbend.akka</groupid> <artifactid>akka-stream-alpakka-slick_${scala.version}</artifactid> <version>${akka.alpakka.version}</version> </dependency> <dependency> <groupid>com.typesafe.akka</groupid> <artifactid>akka-stream_${scala.version}</artifactid> <version>${akka.version}</version> </dependency> <dependency> <groupid>com.typesafe.akka</groupid> <artifactid>akka-http_${scala.version}</artifactid> <version>${akka.http.version}</version> </dependency> <scala.version>2.12</scala.version> <akka.version>2.5.9</akka.version> <akka.http.version> </akka.http.version> <akka.alpakka.version>0.16</akka.alpakka.version>

38 mvn compile exec:java

39

40 Integration Pattern Example //Sample Source Source<String,NotUsed>source = Source.from(Arrays.asList("1-2-3","2-3","3-4")); CompletionStage<List<Integer>> ret = source.map(s->arrays.aslist(s.split("-"))).mapconcat(f->f) //Sub-streams logic.map(s->integer.valueof(s)).runwith(sink.seq(),materializer); //Verify results List<Integer>list = ret.tocompletablefuture().get(); assertlist.equals(arrays.aslist(1,2,3,2,3,3,4));

41 What can Reactive Streams offer EE4J?

42 May the Source[T, _] be with you!

43 Next Steps! Learn more Akka and Alpakka Project Site: GitHub Repo: Documentation: Example:

44 Written for architects and developers that must quickly gain a fundamental understanding of microservice-based architectures, this free O Reilly report explores the journey from SOA to microservices, discusses approaches to dismantling your monolith, and reviews the key tenets of a Reactive microservice: Isolate all the Things Act Autonomously Do One Thing, and Do It Well Own Your State, Exclusively Embrace Asynchronous Message-Passing Stay Mobile, but Addressable Collaborate as Systems to Solve Problems

45 The detailed example in this report is based on Lagom, a new framework that helps you follow the requirements for building distributed, reactive systems. Get an overview of the Reactive Programming model and basic requirements for developing reactive microservices Learn how to create base services, expose endpoints, and then connect them with a simple, web-based user interface Understand how to deal with persistence, state, and clients Use integration technologies to start a successful migration away from legacy systems

46 Understand the challenges of starting a greenfield development vs tearing apart an existing brownfield application into services Examine your business domain to see if microservices would be a good fit Explore best practices for automation, high availability, data separation, and performance Align your development teams around business capabilities and responsibilities Inspect design patterns such as aggregator, proxy, pipeline, or shared resources to model service interactions

47