Erlang and VoltDB TechPlanet 2012 H. Diedrich

Transcription

1 TechPlanet 2012 H. Diedrich 1

2 Your Host Henning Diedrich Founder, CEO CTO Freshworks CTO, Producer at Newtracks Team Lead, Producer at Bigpoint OS Maintainer Emysql, Erlvolt 2

3 What makes Them Special? Are They For Me? How They Combine Getting Started!

4 Erlang Erlang may be to Java what Java was to C++ C++ pointers = Java Java deadlocks = Erlang 4

5 Erlang was Built For 5 Reliability Maintenance Distribution Productivity

6 Erlang Poster Childs Klarna AB Financial Services for E-Commerce 30 seconds downtime in 3 years Distributed Databases Membase Riak BigCouch 6

7 Sweet Spots Stateful Servers with High Throughput Cluster Distribution Layers 7

8 The Magic Microprocesses Pattern Matching Immutable Variables * * Not your familiar Regex string matching 8

9 The Actor Model Carl Hewitt 1973 Behavior State Parallel Asynchronous Messages Mailboxes No Shared State Self-Contained Machines Actor Data Data Code Code Object Process Benefits More true to the real world Better suited for parallel hardware Better suited for distributed architectures Scaling garbage collection (sic!) Less Magic 9

10 Thinking Processes What should be a Process? It's easy! Joe Armstrong Processes Don t share State Communicate Asynchronously Are Very Cheap to create And keep Monitor Each Other Provide Contention Handling Constitute the Error Handling Atom 10

11 Objects and Threads Objects sharing Threads, Object Lifetime, Idle Threads 11

12 Erlang Actors Erlang Actors: State + Code + Process 12

13 Processes are Transactional Do X1 for me! Funnel Doing X1 Doing X2 Do X2 for me! One actor is one process and so, cannot race itself. Mandating a job kind to an actor creates a transactional funnel. Only one such job will ever be executing at any one time. 13

14 Thinking Parallel The Generals Problem Lamport Clocks No Guarantees It's not easy. Robert Virding 14

15 Thinking Functional Small Functions + Immutable Variables Don t assign variables: return results! Complete State in Plain Sight Awful for updates in place. Awsome for debugging & maintenance. Erlang is not side-effect free at all. 15

16 Let It Crash! No Defense Code On Error, restart Entire Process Built-In Process Supervision & Restart Missing Branches, Matches cause Crash Shorter, Cleaner Code Faster Implementation More Robust: handles All Errors 16

17 Syntax * Small * Easy * Stable * Declarative * Inspired by Prolog and ML * Obvious State, Implicit Thread fib(0) -> 0; fib(1) -> 1; fib(n) when N>1 -> fib(n-1) + fib(n-2). 17

18 Hello, World! -module(hello). -export([start/0, loop/0]). start() -> Pid = spawn(hello, loop, []), Pid! hello. loop() -> receive hello -> io:format("hello, World!~n"), loop() end. From Edward Garson's Blog at 18

19 Immutable Variables Can t assign a second time: A = A + 1. A = 1, A = 2. * Prevent Coding Errors * Provide Transactional Semantic * Allow for Pattern Matching Syntax * Can be a Nuisance 19

20 Pattern Matching This can mean two things: A = func(). The meaning depends on whether A is already assigned. 20

21 Pattern Matching The common, mixed case: {ok, A} = func(). ok is an assertion AND A is being assigned. 21

22 Proven Productivity Motorola Study of : Erlang shows 2x higher throughput 3x better latency 3-7x shorter code than the equivalent C++ implementation. 22

23 VoltDB The free, scaling, SQL DB MySQL + scale = VoltDB 23

24 CAP Distributed Consistent Highly-Available Partition-Tolerant really? all of it! Brewer on CAP 2012: 24

25 ACID Atomicity Consistency Isolation Durability for granted? 25

26 Double Bookkeeping 26 Not Every App needs It Requires ACID Transactions Neigh Impossible to emulate Impossible With BASE (Eventual Consistency)

27 VoltDB 27 VoltDB, Inc commercial developer, support Open Source 100% dictatorial Made for OLTP fast cheap writes, high throughput CA of CAP 100% consistent & highly available Simple SQL subset of SQL '92 ACID transactions double bookkeeping In-memory 100x faster than MySQL Distributed painless growth Linear scale predictable, low cost Replication, Snapshots disk persistence, hot backup More SQL than SQL clean separation of data

28 In-Memory 28 Today good for 100s of GB of data The Redis of clusters Sheds 75% of DBM activity Full disk persistence

29 Snowflake Structures Data Data Data Data 29

30 Partitions State # ABC a-z 30 EFG a-z HIJ a-z KL M a-z NOP a-z

31 Erlvolt Erlang VoltDB Driver Open Source Asynchronous Insert Connection = erlvolt:createconnection("localhost", "program", "password"), erlvolt:callprocedure(connection, "Insert", ["안녕하세요", "세계", "Korean"]), Select Response = erlvolt:callprocedure(connection, "Select", ["Korean"]), Row = erlvolt:fetchrow(table, 1), io:format("~n~n~s, ~s!~n", [ erlvolt:getstring(row, Table, "HELLO"), erlvolt:getstring(row, Table, "WORLD") ]); 31

32 Benchmark 32 Amazon EC2 64 core node.js clusters + 96 core VoltDB cluster 695,000 transactions per second (TPS) 2,780,000 operations per second 100,000 TPS per 8 core client 12,500 TPS per node.js core Stable even under overload Pretty much linear scale

33 Benchmark // Check if the vote is for a valid contestant SELECT contestant_number FROM contestants WHERE contestant_number =?; // Check if the voter has exceeded their allowed number of votes SELECT num_votes FROM v_votes_by_phone_number WHERE phone_number =?; // Check an area code to retrieve the corresponding state SELECT state FROM area_code_state WHERE area_code =?; // Record a vote INSERT INTO votes (phone_number, state, contestant_number) VALUES (?,?,?); 33

34 Resources Erlang VoltDB Web Download List Books References Forum Voter Example Benchmark Blog Post Volt s Magic Sauce Post Mortems Erlvolt 34

35 Questions 35 hdiedrich*eonblast.com IRC: #erlounge List: