How to build a European scale instant payments platform

Transcription

1 How to build a European scale instant payments platform SIA and Red Hat Giovanni Fulco, Giuseppe Bonocore, Ugo Landini May 2018

2 Giovanni Fulco Software Architect Ugo Landini Principal Solution Architect Mattia Ronchi Senior Technical Analyst Giuseppe Bonocore Solution Architect

3 SIA in a nutshell CARDS 6.1 billion operations PAYMENTS 3.3 billion transactions CAPITAL MARKETS 56.2 billion financial transactions NETWORK 784 terabytes of data carried

4 SIA Vision Red Hat Vision Everyone will be able to use their own money anytime, anywhere, simply and securely. Open unlocks the world s potential.

5 SIA & Red Hat Partnership to develop JDG Library New Trading platform on RHEL Instant Payment Platform Real Time Kernel Adoption First application delivered on linux OS

6 Wire transfer without Instant Payments

7 My Bank New Wire Transfer

8 My Bank 1 2 Select Recipient: Giovanni Fulco Mattia Ronchi Giuseppe Bonocore Ugo Landini 3

9 My Bank 1 2 Select amount: 100 SEND 3

10

11

12 My Homebanking Search... Wire Account Received: 100 OK _ x

13 Wire transfer with Instant Payments

14 My Bank New Wire Transfer

15 My Bank 1 2 Select Recipient: Giovanni Fulco Mattia Ronchi Giuseppe Bonocore Ugo Landini 3

16 My Bank Select amount: 100 Instant Payment: SEND

17 < 50mS

18 My Homebanking Search... Wire Account Received: 100 OK _ x

19 What s under the hood

20 What s under the hood JDG, In Memory Data Grid Scalable and elastic grid for exceptional performances AMQ High performance messaging FIS, Camel on OpenShift Superdynamic integrations CASSANDRA, NoSQL DB Store transaction history for non repudiation / antifraud

21 Sender 1 Bank 2 3 Fuse JDG 7 Receiver Fuse 1 Request sent to bank 2 Bank calls Instant Payments 3 Request is stored in JDG 4 JDG replicates cross site 4 JDG 5 Req is stored in Cassandra 6 Tx is propagated to bank Receiver asked for confirm Bank Site A Site B Instant Payments

22 Sender 6 Bank 5 2 Fuse 2 Confirmation is stored in JDG 3 JDG replicates cross site 4 JDG Req is stored in Cassandra 5 Tx is propagated to bank 6 Sender is notified 1 Receiver Bank Confirms Payment < 50mS! Fuse 3 JDG 1 4 Bank Site A Site B Instant Payments

23 JDG, In Memory Data Grid

24 Memory is the new disk

25 JDG: JBoss Data Grid Polyglot Extreme Performance Linear Scalability Fault Tolerant Event driven See "memory" across machines as a unified data store Read-through, write-through, write-behind In-memory data = all data needed is to be kept in memory Grid = too big for one node, data is distributed in cluster

26 Memory cluster

27 Distributed mode (typically 1 replica)

28 Adding a value PUT(K1,V)

29 Adding a value PUT(K1,V) K1 K1

30 Values are distributed PUT(K2,V) K1 K1

31 Values are distributed PUT(K2,V) K1 K1 K2 K2

32 Keys are distributed consistently through the cluster K4 K1 K5 K3 K5 K1 K2 K4 K3 K2

33 Consistent Hashing Given the same topology (i.e. number of nodes), same keys are always hashed on the same nodes. Consistent hashing can be calculated on clients too, so they can reach the right node for any given key

34 So reads go directly on the right node... GET(K2) K4 K1 K5 K3 K5 K1 K2 K4 K3 K2

35 ... thx to consistent hashing GET(K5) K4 K1 K5 K3 K5 K1 K2 K4 K3 K2

36 If you lose or add a node (topology change)... K4 K1 K5 K3 K5 K1 K2 K4 K3 K2

37 keys are automatically redistributed K5 K4 K1 K5 K1 K2 K4 K3 K3 K2

38 The client gets the new topology too... GET(K5) K5 K4 K1 K5 K1 K2 K4 K3 K3 K2

39 Data Affinity Data affinity means co-locating data together to improve performance and scalability Data affinity means co-locating computing code with data too

40 Data Affinity Grouping together all the affine data, for example: All Customer data All Credit card data Whatever partitioning criteria is better Gives: Highest possible performance Lowest possible round trips

41 Same color == same group K8 K4 K1 K6 K8 K3 K7 K1 K5 K4 K2 K5 K6 K7 K3 K2

42 Project Requirements

43 Euro Banking Association - RT1 RT1 is an instant payment system that will provide the European payments industry with a pan-european infrastructure platform for real-time payments in euro from day one of the SEPA Instant Credit Transfer Scheme.

44 Constraints 5,000 tx/sec Active/Active geo sites 27 Mln Payments/day 0 Message loss < 900 ms Roundtrip 24/7/365 Availability

45 Challenges and Solutions

46 Challenge Due to low latency and replica requirements, we could not adopt a traditional db-centric application

47 Solution Non DB Centric application We adopted a full in-memory solution relying on Jboss DataGrid

48 Challenge Cross-site replica needs to be synchronous, and the number of remote operations must be low

49 Solution Minimize remote communication DataGrid key affinity: Each transaction is handled by the node which owns the relevant data.

50 Challenge Incoming messages need to be managed in an highly available, cross site infrastructure

51 Solution Highly available messaging AMQ Network of brokers allow highly available, zero message loss topologies (site disaster resilient)

52 Challenge Multi site active-active increases the risks of split brains

53 Solution Avoid split brain Datagrid owner distribution between sites allow to operate in case of network split

54

55

56 Next Steps

57 Next Steps EnMasse Adoption of EnMasse (Messaging As A Service in OpenShift), to streamline the management of queues

58 Next Steps Release and scale, without service disruption Containerization with OpenShift to allow safer releases of newer version and instances

59 Next Steps Broader Circuit Federate other payment circuits, in order to reach more customers

60 THANK YOU plus.google.com/+redhat facebook.com/redhatinc linkedin.com/company/red-hat twitter.com/redhat youtube.com/user/redhatvideos