Designing for Scalability. Patrick Linskey EJB Team Lead BEA Systems

Size: px

Start display at page:

Download "Designing for Scalability. Patrick Linskey EJB Team Lead BEA Systems"

Luke Leonard Thomas
5 years ago
Views:

1 Designing for Scalability Patrick Linskey EJB Team Lead BEA Systems 1

2 Patrick Linskey EJB Team Lead at BEA OpenJPA Committer JPA 1, 2 EG Member 2

3 Agenda Define and discuss scalability Vertical Horizontal Examine ways to make software scale Code / Algorithms Asynchronous Libraries Other Languages 3

4 Scalability Ability to increase the total number of operations performed in a unit of time Vertical Scalability: Make the machine bigger Horizontal Scalability Add more machines 4

5 Bottlenecks Limit the scalability of a system Intrinsic bottlenecks Artificial bottlenecks 5

6 Example Problem Domain Financial fund management Multiple in-house engineering needs Trade Execution Trade Settlement Strategy Definition Strategy Simulation Portfolio Risk Analysis 6

7 Vertical Scalability Translated into Java: Scaling Within a Machine 7

8 Vertical Scale Factors In Your Control Improve code efficiency Memory CPU Optimize I/O between physical tiers Web 2.0: beware! Make code scale across multiple cores / CPUs 8

9 Code Optimization Possibilities Performance and scalability are linked Scalability: more operations per time unit time time Architectural time Quick and dirty 9

10 Scale Vertically via Code Optimization Reduce copying, looping, etc. Write good code SQL statement batching PreparedStatement.addBatch() ORM frameworks Transaction batching Especially powerful in XA environments JMS message batching 10

11 Write-Once Shared Memory class SlowTradeManager { private Set types; public synchronized Set gettradetypes() { if (types == null) types = loadtypedata(); return types; } } class FastTradeManager { private Set types; public Set gettradetypes() { if (types == null) types = loadtypedata(); return types; } } loadtypedata() might be called more than once 11

12 Fund Risk Balancing Problem Multiple traders act on the same security Solution Maintain fund-global position data Mutable shared state! 12

13 Multi-machine solution (circa 1998) time 13

14 Synchronization synchronized is for asynchronous execution Execute this block of code in its entirety before others that share this lock Modern computers handle high* concurrency synchronized is often a bottleneck Avoid synchronization at runtime at all costs uncontended synchronization is cheap 14

15 Multi-core / CPU synchronization sync sync sync sync sync time 15

16 Mutable Shared Memory import java.util.concurrent.atomic.atomicdouble; class AggregateFundPosition { private AtomicDouble totalexposure = new AtomicDouble(0); public double incrementby(double amount) { while (true) { double old = totalexposure.get(); double next = old + amount; if (counter.compareandset(old, next)) return next; } } } 16

17 Synchronization-free shared state CAS CAS CAS CAS CAS CAS time 17

18 Horizontal Scalability Translated into Java: Scaling Across Machines 18

19 Horizontal Scaling: Add More Servers All doing the same thing Partitioned by infrastructure layer Partitioned by application role Partitioned along data graph boundaries 19

20 Build a Farm App App App Load Balancer App App App App 20

21 Slow Down App Web EJB 237ms 983ms 21

22 Divide and Conquer Old as `time` itself mail, news, telnet all on different servers You use partitioning every day Telephone call routing ATM card transactions Stock markets Elevator banks 22

23 Break Up Stateful Services Worldwide Trade Execution, Clearing, Position Analysis Apps Apps Apps Apps 23

24 Partition Along Application Boundaries Trade Clearing Trade Execution Position Analysis Apps Apps Apps Apps Apps Apps 24

25 Partition along data set fault lines US Europe Asia Apps Apps Apps Apps Apps Apps 25

26 Asynchrony in Java Java is a mostly synchronous environment Business algorithms often aren t Take advantage of this where possible JMS message queues java.util.concurrent.executorservice commonj.work.workmanager Scheduled jobs 26

27 Async Tasks and Resource Utilization Good JMS servers / ExecutorServices / WorkManagers do resource tuning and optimization Limit threads allocated to async processing Configure priority of async vs. sync (i.e., HTTP request) async tasks throttled async task backlog handled 100 Trade Settlement Strategy Analysis Trade Execution and Strategy Definition

28 Adapt Requirements to Concurrency Identify slow-running / expensive parts of the user experience Work with requirements team to replace these with asynchronous processes Website usage statistics generated nightly instead of on-demand Dynamic PDF delivery via instead of embedded web content 28

29 Starting from Scratch 29

30 Choose Your Toolset Java makes synchronization easy... but synchronization!= scalability Other languages avoid shared state Rely on message-passing instead 30

31 Erlang: Functional, Asynchronous, Mature Designed for concurrency in the language Parallel execution Intrinsic hot-redeploy State can only be assigned once Communication happens via message-passing between actors No threads no shared state! JMS-like behavior; language-native syntax 31

32 Scala: Functional Programming for the JVM Java-integrated Designed by Java stalwart Martin Odersky JVM-optimized Supports Erlang-style concurrency 32

33 Compute Grids Federate your data around a cluster Decompose your algorithm into serializable work items Let the compute grid send your work items to the data 33

34 Decision Factors What are your application requirements? How many concurrent operations? How big of a workload? What sorts of SLAs? Tolerance of deployment complexity? How about your operations, QA teams? 34

35 Recap Concepts Scalability Bottlenecks Synchronization Asynchrony vs. concurrency Compare-and-set Application Partitioning Synchronous tasks vs. asynchronous tasks Technology java.util.concurrent j.u.concurrent.atomic Operation batching Transactions SQL JMS; Executor; WorkManager Scala and Erlang Hibernate Shards OpenJPA Slice 35

36 Questions Patrick Linskey 36

Craig Blitz Oracle Coherence Product Management

Software Architecture for Highly Available, Scalable Trading Apps: Meeting Low-Latency Requirements Intentionally Craig Blitz Oracle Coherence Product Management 1 Copyright 2011, Oracle and/or its affiliates.