Scalable Data Models with the Transactional Key-Value Store Scalaris

Transcription

1 Scalable Data Models with the Transactional Key-Value Store Scalaris Nico Kruber Michael Berlin Zuse Institut Berlin Parallel and Distributed Systems 20th November INGI 2012 Doctoral School Day in Cloud Computing The research leading to these results has received funding from the European Union s Seventh Framework Programme (FP7/ ) under grant agreements n (4CaaSt) and n (Contrail). Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 1/23

2 Outline Scalaris Introduction Scalability Wiki on Scalaris MediaWiki Introduction Data Model Optimisations Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 2/23

3 Scalaris - A Transactional Key-Value Store Web 2.0 Application Transaction Layer: Paxos Commit layer implements......strong consistency, atomicity, isolation Replication Layer: Symmetric Repl....availability Structured Overlay Protocol: T-Man, Chord, Chord #...scalability... many standard internet nodes in datacenters or P2P Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 3/23

4 Scalaris - Operations supported in core: read(key) write(key, Value) convenience operations in APIs: add_del_on_list(key, ToAdd, ToRemove) add_on_nr(key, ToAdd) test_and_set(key, OldValue, NewValue) as single operations or as part of a transaction support for bundling requests Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 4/23

5 Scalaris - Scalability Quorum-Read Benchmark (10 clients/node, quorum-reads/key) quorum read, min quorum read, max Increment Benchmark (10 clients/node, 1000 incs/key) increment, min increment, max throughput (operations / s) throughput (operations / s) number of nodes (dual quad-core AMD Opteron 2370, 8 GB RAM, GbE) number of nodes (dual quad-core AMD Opteron 2370, 8 GB RAM, GbE) Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 5/23

6 Wikitext A-Z Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 6/23

7 Wikitext A-Z page title including namespace Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 6/23

8 Wikitext A-Z page title including namespace link to another page Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 6/23

9 Wikitext A-Z page title including namespace link to another page template inclusion Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 6/23

10 Wikitext A-Z page title including namespace link to another page template inclusion category the page belongs to Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 6/23

11 Wiki Operations Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 7/23

12 Wiki Operations show page: read revisions Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 7/23

13 Wiki Operations revision history overview show page: read revisions Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 7/23

14 Wiki Operations search for articles revision history overview show page: read revisions Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 7/23

15 Wiki Operations search for articles revision history overview show page: read revisions redirect to random article Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 7/23

16 Wiki Operations search for articles revision history overview show page: read revisions redirect to random article backlinks ( what links here ) Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 7/23

17 Wiki Operations search for articles revision history overview show page: read revisions redirect to random article backlinks ( what links here ) save new revision Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 7/23

18 Editing Pages Do atomically: 1. store new revision 2. update page object 3. add new backlinks 4. remove old backlinks 5. add to new categories 6. remove from old categories 7. if new page: a) add to page list b) increase page counter Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 8/23

19 Editing Pages Do atomically: 1. store new revision 2. update page object 3. add new backlinks 4. remove old backlinks 5. add to new categories 6. remove from old categories 7. if new page: a) add to page list b) increase page counter Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 8/23

20 Editing Pages Do atomically: 1. store new revision 2. update page object 3. add new backlinks 4. remove old backlinks 5. add to new categories 6. remove from old categories 7. if new page: a) add to page list b) increase page counter Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 8/23

21 Editing Pages Do atomically: 1. store new revision 2. update page object 3. add new backlinks 4. remove old backlinks 5. add to new categories 6. remove from old categories 7. if new page: a) add to page list b) increase page counter Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 8/23

22 Editing Pages Do atomically: 1. store new revision 2. update page object 3. add new backlinks 4. remove old backlinks 5. add to new categories 6. remove from old categories 7. if new page: a) add to page list b) increase page counter Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 8/23

23 Editing Pages Do atomically: 1. store new revision 2. update page object 3. add new backlinks 4. remove old backlinks 5. add to new categories 6. remove from old categories 7. if new page: a) add to page list b) increase page counter Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 8/23

24 Wiki on Scalaris - Data Layout (1) <title>:page Main Page:page page information (JSON object) {"title": "Main Page", "id": 1, "redirect": false, "currev": <rev>} <title>:rev:<id> Main Page:rev:1 single page revision (JSON object) {"text": <text>, "id": 1, "minor": false, "comment": "", "timestamp": <date>, "contributor": <user>} Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 9/23

25 Wiki on Scalaris - Data Layout (2) <title>:revs Main Page:revs list of revision info for history [{"id": 2, "size": 10, "minor": false, "comment": "", "timestamp": <date>, "contributor": <user>}, {"id": 1, "size": 4, "minor": false, "comment": "", "timestamp": <date>, "contributor": <user>}] Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 10/23

26 Wiki on Scalaris - Data Layout (3) <title>:blpages European Union:blpages list of page titles linking to <title> ["Berlin"] <cat_title>:cpages list of page titles inside a category Category:States of Germany:cpages ["Berlin"] <cat_title>:cpages:count number of pages inside a category Category:States of Germany:cpages:count 1 Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 11/23

27 Wiki on Scalaris - Data Layout (4) pages:<namespace> pages: list of pages in <namespace> ["Main Page", "Berlin"] pages:<namespace>:count pages::count number of pages in <namespace> 2 stats:pageedits total number of page edits 3 Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 12/23

28 Data Model - Scalability (read) Quorum-Read Benchmark (10 clients/node, quorum-reads/key) quorum read 3000 Wiki Read Benchmark (32 clients/node, 4 bench workers, ops) es wiki average throughput (operations / s) average throughput (HTTP requests / s) number of nodes (dual quad-core AMD Opteron 2370, 8 GB RAM, GbE) number of nodes (dual quad-core AMD Opteron 2370, 8 GB RAM, GbE) Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 13/23

29 Data Model - Scalability (edit) average throughput (operations / s) Increment Benchmark (10 clients/node, 1000 incs/key) increment average throughput (HTTP requests / s) Wiki Edit Benchmark (32 clients/node, 4 bench workers, ops) es wiki number of nodes (dual quad-core AMD Opteron 2370, 8 GB RAM, GbE) number of nodes (dual quad-core AMD Opteron 2370, 8 GB RAM, GbE) Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 14/23

30 Data Model - Value Sizes type count avg min max articles_count backlink_pagelist category_count category_pagelist page pagelist pagelist_count revision_list siteinfo template_pagelist Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 15/23

31 Data Model - Solution: Partitioning! a) spread randomly over partitions unpredictable partition position suitable for e.g. counters stats:pageedits random placement stats:pageedits: stats:pageedits:0 stats:pageedits:2 Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 16/23

32 Data Model - Solution: Partitioning! b) spread list contents with hashing allows efficient lookup and delete suitable for e.g. lists mycat:cpages page1, page2, page3, page4, page5, page6, page7, page8 hashing page4, page1, page5 page8, page6 page7, page2, page3 mycat:cpages:0 mycat:cpages:1 mycat:cpages:2 Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 17/23

33 Data Model - Partitioning Schemes es plain es wiki es-p25 split all lists, counters into 25 es-p50e max pagelist size 100k, article counter 3, edit stats 50 es-p250e max pagelist size 20k, article counter 3, edit stats 50 Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 18/23

34 Optimisation Results (page edit) average execution time in ms es es-p25 es-p100k es-p20k Scalaris Tomcat Other Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 19/23

35 Optimisation Results (page create) average execution time in ms es es-p25 es-p100k es-p20k Scalaris Tomcat Other Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 20/23

36 Optimisation Results - Scalability (read) Quorum-Read Benchmark (10 clients/node, quorum-reads/key) quorum read 3500 Wiki Read Benchmark (32 clients/node, 4 bench workers, ops) es wiki es wiki, p100k average throughput (operations / s) average throughput (HTTP requests / s) es wiki, p25 es wiki, p20k number of nodes (dual quad-core AMD Opteron 2370, 8 GB RAM, GbE) number of nodes (dual quad-core AMD Opteron 2370, 8 GB RAM, GbE) Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 21/23

37 Optimisation Results - Scalability (edit) average throughput (operations / s) Increment Benchmark (10 clients/node, 1000 incs/key) increment average throughput (HTTP requests / s) Wiki Edit Benchmark (32 clients/node, 4 bench workers, ops) es wiki es wiki, p25 es wiki, p100k es wiki, p20k number of nodes (dual quad-core AMD Opteron 2370, 8 GB RAM, GbE) number of nodes (dual quad-core AMD Opteron 2370, 8 GB RAM, GbE) Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 22/23

38 Questions Nico Kruber, Michael Berlin Scalable Data Models with the Transactional Key-Value Store Scalaris 23/23