MULTICORE IN DATA APPLIANCES. Gustavo Alonso Systems Group Dept. of Computer Science ETH Zürich, Switzerland

Transcription

1 MULTICORE IN DATA APPLIANCES Gustavo Alonso Systems Group Dept. of Computer Science ETH Zürich, Switzerland SwissBox CREST Workshop March 2012

2 Systems Group = Enterprise Computing Center = Gustavo Alonso - Systems Group - ETH Zürich 2

3 The SwissBox project Build an open source data appliance Hardware Software Gustavo Alonso - Systems Group - ETH Zürich 3

4 Goals Robust by design Scalable by design Fully predictable Behavior immune to peak loads (read or write) Efficient use of modern hardware Cost efficiency Power/space/management complexity Gustavo Alonso - Systems Group - ETH Zürich 4

5 DATA APPLIANCE Database in a box Funny database Funny box Examples: Exascale (Oracle) Twin-Fin (Netezza IBM) NewDB (SAP) Teradata Gustavo Alonso - Systems Group - ETH Zürich 5

6 ORACLE EXADATA Intelligent storage manager Massive caching RAC based architecture Fast network interconnect Gustavo Alonso - Systems Group - ETH Zürich 6

7 The Multicore Challenge Gustavo Alonso - Systems Group - ETH Zürich 7

8 Data parallelism Relational model is highly parallel Independent tables Orthogonal operators Intra- and Inter-query parallelism Most successful commercial parallel systems And yet Gustavo Alonso - Systems Group - ETH Zürich 8

9 Throughput (TPS) Database engines and multicore Postgres TPC-WB 20GB DB cores cores cores TPCW-S Clients PG 48 PG-24 PG 8 Salomie, Subasu, Giceva, Alonso, EuroSys 2011 Gustavo Alonso - Systems Group - ETH Zürich 9

10 Thorughput(TPS) Database engines and multicore MySQL TPC-WB 20 GB DB 8 cores cores cores Clients MYSQL-48 MYSQL-24 MYSQL-12 Salomie, Subasu, Giceva, Alonso, EuroSys 2011 Gustavo Alonso - Systems Group - ETH Zürich 10

11 CLAIM #1 Adding resources to a troubled application does not necessarily lead to improvements Gustavo Alonso - Systems Group - ETH Zürich 11

12 Size matters The challenge of appliances is the unprecedented power available 64 cores AMD,256 GB memory, 10 Gb network + 3 TB NAS: ~14k CHF Imagine a rack full of those Is your job large enough? Gustavo Alonso - Systems Group - ETH Zürich 12

13 CLAIM #2 Large scale parallelism sensible only when looking at aggregated loads Gustavo Alonso - Systems Group - ETH Zürich 13

14 Load interaction In a highly parallel system, multiple jobs will get on each other s way: Synchronization Data movement Resource arbitrage Resource capping Heavy vs. light jobs Management and coordination Gustavo Alonso - Systems Group - ETH Zürich 14

15 Giannikis, Alono, Kossmann, PVLDB 2012 Load interaction in practice System X Gustavo Alonso - Systems Group - ETH Zürich 15

16 CLAIM #3 Robustness and performance can only be obtained by minimizing interaction Gustavo Alonso - Systems Group - ETH Zürich 16

17 Locality in the XXI st century Each package has: 12 cores 4HT ports 4 memory channels P0 P4 P2 P6 Each die has: 6 cores 4HT ports 2 memory channels P1 P3 P5 P7 Gustavo Alonso - Systems Group - ETH Zürich 17

18 CLAIM #4 Robustness and definitely performance can only be achieved on fixed data paths Gustavo Alonso - Systems Group - ETH Zürich 18

19 Architecture other data centers RACK RACK RACK RACK RACK multicore Parallel hardware accelerator Gustavo Alonso - Systems Group - ETH Zürich 19

20 CLAIM #5 Traditional layered architectures (HW/OS/VM/App) do not work in these environments Gustavo Alonso - Systems Group - ETH Zürich 20

21 CLAIM #6 Strong notions of consistency (serializability) and atomicity of complex programs no longer feasible Gustavo Alonso - Systems Group - ETH Zürich 21

22 SWISSBOX Alonso, Kossmann, Roscoe, CIDR 2011 Gustavo Alonso - Systems Group - ETH Zürich 22

23 SwissBox: the project Great opportunity for research Rethink the entire system software stack Redesign the operating system, database, and storage system architecture Software hardware co-design Gustavo Alonso - Systems Group - ETH Zürich 23

24 SwissBox: the product Direct collaboration and input from industry Great demand for tailored systems Big data Highly demanding applications Low power / high efficiency Gustavo Alonso - Systems Group - ETH Zürich 24

25 Claim # 1 => Deterministic behavior Adding resources to an application does not necessarily lead to a performance improvement System performance completely determined at design time through simple parameters Gustavo Alonso - Systems Group - ETH Zürich 25

26 Clock Scan QUERIES UPDATES Unterbrunner, Giannikis, Alonso, Kossmann, PVLDB 2009 BUILD QUERY INDEX FOR NEXT SCAN READ CURSOR WRITE CURSOR DATA IN CIRCULAR BUFFER (WIDE TABLE) Gustavo Alonso - Systems Group - ETH Zürich 26

27 Claim # 2 => Batch processing Large scale parallelism makes sense only when considering aggregated loads Execution proceeds in batches (1000 s of queries per batch) Gustavo Alonso - Systems Group - ETH Zürich 27

28 Giannikis, Alono, Kossmann, PVLDB 2012 Shared join Crescando runs selection and projections in one set of cores SharedDB runs joins on the streams from Crescando, thousands of queries at a time Gustavo Alonso - Systems Group - ETH Zürich 28

29 Claim # 3 => No load interaction Robustness and performance can only be obtained by minimizing interaction Operators are orthogonal and work on clearly delimited resources Gustavo Alonso - Systems Group - ETH Zürich 29

30 Giannikis, Alono, Kossmann, PVLDB 2012 Predictability, robustness Gustavo Alonso - Systems Group - ETH Zürich 30

31 Claim # 4 => No dynamic scheduling Robustness (and definitely performance) can only be achieved on fixed data paths No dynamic scheduling, operators are always on and at fixed locations Gustavo Alonso - Systems Group - ETH Zürich 31

32 Single plan: operator per core Gustavo Alonso - Systems Group - ETH Zürich 32

33 Giannikis, Alono, Kossmann, PVLDB 2012 Raw performance Gustavo Alonso - Systems Group - ETH Zürich 33

34 Claim # 5 => Open stack Traditional layered architectures (HW/OS/VM/App) do not work in these environments Operating System / Database co-design Gustavo Alonso - Systems Group - ETH Zürich 34

35 Claim # 6 => Consistency Strong notions of consistency (serializability) and atomicity of complex programs no longer feasible Snapshot isolation, multiversions, eventual consistency Gustavo Alonso - Systems Group - ETH Zürich 35

36 SWISSBOX Gustavo Alonso - Systems Group - ETH Zürich 36

37 Where are we? Fully predictable performance Accurate analytical model Easily tunable / scalable Tolerates high peaks of reads and updates without compromising SLA Intelligent storage engine Gustavo Alonso - Systems Group - ETH Zürich 37

38 Next steps Hardware acceleration Query optimizer Parallel operators OS / DB interfaces Gustavo Alonso - Systems Group - ETH Zürich 38

39 In the future Virtualized operators Flexible, elastic deployment through OS interaction Scalability through operator/plan replication across cores and machines Hardware acceleration by operator offloading and in-network data processing Operator parallelism Gustavo Alonso - Systems Group - ETH Zürich 39

40 SwissBox in a nutshell A new way to process data Parallel, predictable by design Not optimal but good enough Co-design at all levels Great opportunity for research Redo everything from scratch Gustavo Alonso - Systems Group - ETH Zürich 40