MongoDB Chunks Distribution, Splitting, and Merging. Jason Terpko

Transcription

1 Percona Live 2016 MongoDB Chunks Distribution, Splitting, and Merging Jason Terpko NoSQL DBA, Rackspace/ObjectRocket

2 My Story Started out in relational databases in public education then financial services Next came online media distribution combined with a paywall For analytics, started working with columnar databases and engines with compression Made the switch to NoSQL at ObjectRocket by Rackspace

3 Overview MongoDB and Sharding What is a chunk in MongoDB? Chunk Distribution and Scaling MongoDB Use Cases For Splitting Use Cases For Merging Reconsidering Your Shard Key

4 Documents Common MongoDB BSON Types ObjectId String Integer Double NumberLong Date Boolean Null Array { } "_id" : ObjectId("570680d891442f6efaff2005"), "n" : "John Doe", "a" : 45, "h" : 5.9, "w" : NumberLong(165), "u" : ISODate(" T15:46:32.085Z"), "ac" : true, "nn" : null, "z" : [ "10013", "10018" ]

5 MongoDB Sharded Cluster s1 s2

6 What is a Chunk? config.chunks { } "_id" : "mydb.mycoll-uuid_\"00005cf b-cf1bde09cc77\"", "lastmod" : Timestamp(1, 5), "lastmodepoch" : ObjectId(" f2bf94777a62155"), "ns" : "mydb.mycoll", "min" : { "uuid" : "00005cf b-cf1bde09cc77" }, "max" : { "uuid" : "7fe c0-4e51-8eed-ab6b411d2b6e" }, "shard" : s1"

7 Use Cases For Splitting (Case 1) 80% 20% s1 s2 Unbalanced Operations

8 Use Cases For Splitting (Case 1) Profiler Example : db.system.profile.aggregate([ {$match: { $and: [ {op:"update"}, {ns : "mydb.mycoll"} ] }}, {$group: { "_id":"$query.uuid", count:{$sum:1}}}, {$sort: {"count": -1}}, {$limit : 5 } ]); { "result" : [ { "_id" : "00005cf b-cf1bde09cc77", "count" : },... ], "ok" : 1 }

9 Use Cases For Splitting (Case 1) Unbalanced Operations { } "_id" : "mydb.mycoll-uuid_\"00005cf b-cf1bde09cc77\"", "lastmod" : Timestamp(1, 5), "lastmodepoch" : ObjectId(" f2bf94777a62155"), "ns" : "mydb.mycoll", "min" : { "uuid" : "00005cf b-cf1bde09cc77" }, "max" : { "uuid" : "7fe c0-4e51-8eed-ab6b411d2b6e" }, "shard" : s1" { uuid : 7800e10e-273d-4186-bca7-9e3f6647d33a, { uuid : 7801a55e-e326-4cdb-a5fb-a5d133016f13, { uuid : 7801c3d b-ad66904ae01e, { uuid : 7801ff57-50b cc8ff80, sh.splitat()

10 Moving the Chunks Example Command: db.runcommand({ "movechunk" : "mydb.mycoll", "bounds" : [ { "uuid" : "7fe bf-9a00-462a0284c982" }, { "uuid" : "ffffe5bc-d04d-4fcf-b8e2-283fa " } ], "to" : "s2", "_secondarythrottle" : true });

11 Use Cases For Splitting (Case 2) s1 s2 Unbalanced Resources s1 s2

12 Use Cases For Splitting (Case 2) Unbalanced Resources Jumbo Chunks Requirements Larger than db.settings.find({"_id" : "chunksize"}) Or contains 250,000 or more documents Causes Importing of data Config metadata lock Compound Shard Key Identifying sh.status(true) movechunk() count() or Aggregation Pipeline ObjectRocket Utility : ChunkHunter.py* Rectifying sh.splitat() and sh.splitfind() ObjectRocket Utilities : ChunkManager.py* Data reduction Re-sharding the collection *

13 Splitting : Identifying and Rectifying Aggregation Example: db.mycoll.aggregate([{$group: {"_id": {"uuid": "$uuid"}, "count": {$sum: 1}}},{$sort: {"count": -1}},{$limit:5}]) Data Size: db.admincommand({ datasize: "mydb.mycoll", keypattern: { "uuid" : 1 }, min: { "uuid" : "7fe c0-4e51-8eed-ab6b411d2b6e" }, max: { "uuid" : "7fe bf-9a00-462a0284c982" } }) ObjectRocket Utils : ChunkManager.py ChunkHunter.py Cnt ChunkSplitter.py exit

14 Use Cases For Splitting (Case 3) 100% s1 s2 New and Small Collections

15 Use Cases For Splitting (Case 3) New and Small Collections Considerations Workload Shard Key Shard Count Current Collection Size Expected Collection Size Command db.runcommand( { shardcollection: "mydb.mycoll", key: { uuid": "hashed" }, numinitialchunks : 1024 } ) Calculating The Value (size_in_mb / chunk_size) * 2

16 Use Cases For Merging Empty Chunks 80% 20% s1 s2 Unbalanced Resources s1 s2 Unbalanced Operations { "uuid" : "00005cf6-..." } -->> { "uuid" : "7fe " } on : s1 100K 45MB { "uuid" : "7fe " } -->> { "uuid" : "7fe " } on : s2 0 0MB

17 Use Cases For Merging (Deletes) Unexpected remove() Operations s1 s2 s3 s4 insert() -> split -> remove() insert() -> split -> remove() insert() & split insert() & split

18 Merging Process How we have resolved this with JavaScript in past: 1. Check Balancer State 2. Read in empty chunks from ChunkHunter.py results collection 3. Locate adjacent chunk 4. If empty and adjacent chunks reside on the same shard, merge 5. Else move chunk to the shard with the adjacent chunk, then merge What have we learned from the current process?

19 Use Cases For Merging (TTL) TTL : db.mycoll.ensureindex( { "ca": 1 }, { expireafterseconds: } ) splitvector & split s1 s2 s3 s4 movechunk()

20 Chunk Size and Splits How frequently are chunks being split? Global Change split & splitvector Decreasing the size Increased the size

21 Review Your Shard Key How frequently does this occur? Is this a re-occurring problem? What impact does it have to your business? Re-analyzing your structure, workload, and access patterns What method will you use to re-shard a sharded collection?

22 ObjectRocket by Rackspace

23 Questions? MongoDB Sharding Chunks Splitting Chunks Merging Chunks ObjectRocket Rackspace