lean enterprise middleware High Volume Web API Management with WSO2 ESB

Transcription

1 lean enterprise middleware High Volume Web API Management with WSO2 ESB Paul Fremantle, CTO and Co-Founder Hiranya Jayathilaka, Associate Technical Lead. Not for redistribution.

2 WSO2 Offerings WSO2 Carbon On premise product platform Consistent products sharing the same OSGi core WSO2 Stratos A Cloud-Enabled Application Platform (CEAP) Multi-tenant, Elastic, Metered/Billed deployment An extension of the Carbon platform Same SVN code tree WSO2 StratosLive A running Platform-as-a-Service (PaaS) An instance of Stratos

3 ForresterWaveTM ESB Q Leader

4 WSO2 ESB 4.0 Click to edit Master text styles Second level Third level Fourth level Fifth level

5 WSO2 ESB Features The WSO2 Enterprise Service Bus offers: Easy configuration through an intuitive graphical interface Lean, high-performance design conserves resources through maximum efficiency. Built on the WSO2 Carbon OSGi platform: adding new capabilities as you need them is a snap. Integrates with your existing systems, with support for EDI, CSV, Files, FIX, HL7, SAP, JMS, etc Asynchronous architecture supports very high loads with fixed resources Routing, mediation and transformation are modular, extensible but provide most use-cases simply by sequencing built-in mediators Full support for SOAP/WS-* and REST

6 Key Features for High Volume Web API Load Balancing and Failover built-in Pluggable models can include affinity based LB Full REST Support (XML and JSON) Especially used in Mobile environments Can bridge into SOAP or other enterprise systems Message Relay Mode and Relay Transport Excellent performance for 100% streaming Priority Execution, Cache and Throttling Manage workloads within the ESB as well as throttle backend Cache content locally if appropriate Service Chaining Asynchronous lightweight orchestration

7 ebay Case Study High Volume Requirements: The solutions we were using no longer met our needs, so we considered whether to build a new internal system or to adopt third-party technology, Abhinav Kumar, ebay Needed enhanced service mediation and orchestration capabilities Scalability and performance to sustain increasing loads Evaluation: ebay evaluated several industry leading hardware and software ESB solutions, looking at open source, commercially licensed ESBs and hardware Test production during 2009 holiday season proved that our system was rock solid 1m messages/day with zero downtime, zero memory growth Results Shared-nothing architecture on RedHat Linux on x86 hardware Currently running at more than 1 billion messages / day Low resource utilization and flat memory usage The benefits Using the WSO2 ESBs, we've been able to provide customers and partners with the quality experience they expect on ebay, even as our global customer base has grown, Abhinav Kumar

8 Memory Usage at ebay Click to edit Master text styles Second level Third level Fourth level Fifth level

9 Deployment

10 Configuring for High Volume Operating System JVM ESB Management

11 Tuning the OS Optimize core network settings for high throughput network activity /etc/sysctl.conf Increase the number of open file descriptors allowed by the OS /etc/security/limits.conf

12 Tuning the JVM Allocate sufficient memory for the heap Consider reducing the new ratio -XX:NewRatio=n Consider using the concurrent mark and sweep collector -Xms256m -Xmx2048m -XX:MaxPermSize=256m -XX:+UseConcMarkSweepGC Read more on JVM and GC tuning

13 Tuning the ESB Configure transport thread pools Configured through nhttp.properties file IO dispatcher threads carry out network IO at the wire level Recommended to have one IO dispatcher per CPU core Server workers and client workers mediate the messages More the merrier (But keep memory usage and load in mind)

14 Message Relay Mode WSO2 ESB uses Apache AXIOM and the StAX API for processing XML XML payloads are streamed through the ESB Pull parsing model But for pure routing, load-balancing and headeronly mediation we can avoid even this step Message relay mode works around this glitch and enables 100% pure streaming of messages Works regardless of size and format of messages

15 Enabling Message Relay Enable the binary relay builder (axis2.xml) Instructs the ESB to stream the incoming messages through without touching their payloads Enable the expanding message formatter (axis2.xml) Allows the ESB to send messages that were received through the binary relay builder <messagebuilder contenttype="application/xml" class="org.wso2.carbon.relay.binaryrelaybuilder"/> <messageformatter contenttype="application/xml" class="org.wso2.carbon.relay.expandingmessageformatter"/> Above configuration tells the ESB to process all application/xml messages in message relay mode.

16 HTTP Relay Transport Brand new in WSO2 ESB 4.0 release Non blocking HTTP transport implementation specially designed for streaming messages Doesn t require the binary relay builder and expanding formatter To enable, simply uncomment the relevant entries in the axis2.xml file

17 Error Handling In a large scale, high throughput deployment, errors are not unusual Expect the unexpected Configure endpoints to gracefully handle the common errors Connection timeouts Connection close/reset Connection refused HTTP protocol violations!!!!

18 More on Error Handling Configure sequences to clearly log errors and if needed notify system administrators In case the back end server fails to respond, send detailed fault responses to clients Makes your application more appealing to the customers Pay attention to HTTP error codes

19 Monitoring In general keep an eye on: CPU usage Memory usage Thread counts Fault counts Latency/Response time Active connections

20 Monitoring Tools Utilities Provided by OS top netstat Mediation statistics JMX Clients Jconsole

21 Connections Click to edit Master text styles Second level Third level Fourth level Fifth level

22 Endpoints Click to edit Master text styles Second level Third level Fourth level Fifth level

23 Latency Click to edit Master text styles Second level Third level Fourth level Fifth level

24 Listeners Click to edit Master text styles Second level Third level Fourth level Fifth level

25 Memory and Garbage Collection Click to edit Master text styles Second level Third level Fourth level Fifth level

26 S2S Latency Click to edit Master text styles Second level Third level Fourth level Fifth level

27 Threads Click to edit Master text styles Second level Third level Fourth level Fifth level

28 Questions? 28

29 Resources WSO2 ESB 4.0 Download and Library Message Relay Mode message_relay.html