MongoDB Index Types How, when and where should they be used?

Transcription

1 MongoDB Index Types How, when and where should they be used? 26 April - 1:30 PM - 1:55 PM at Room 209

2 { me : '@adamotonete' } Adamo Tonete I've been working for Percona since late 2015 as a Senior Technical Services Engineer. I live in São Paulo/Brazil 2

3 Agenda What is an index? How do indexes work? Costs of maintaining an index Types of Indexes in MongoDB and differences them. Q&A

4 What is an index? Here the library is a table and each bookshelf is a file in the disk. 4

5 What is an index? The execution plan: Could you please find all the books written in 1977 and 1978? 5

6 What is an index? The execution plan: The only option is to read the entire bookshelf and check the book publication date. 6

7 How do indexes work? The Index. This is an illustration, in the real world, of what an index looks like. If you were the guy in the last picture, what would you do? Would you read all the books or simply check the indexes? 7

8 Indexes in MongoDB bigcollection (~ documents) - No Index db.bigcollection.find({name : 'PERCONA'}).explain(true) "executionstats" : { "executionsuccess" : true, "nreturned" : 0, "executiontimemillis" : 350, "totalkeysexamined" : 0, "totaldocsexamined" : , "executionstages" : { "stage" : "COLLSCAN", "filter" : { "name" : { "$eq" : "PERCONA" 8

9 Indexes in MongoDB smallcollection (100 documents) - No Index db.smallcollection.find({name : 'PERCONA' }).explain(true) "executionstats" : { "executionsuccess" : true, "nreturned" : 0, "executiontimemillis" : 0, "totalkeysexamined" : 0, "totaldocsexamined" : 100, "executionstages" : { "stage" : "COLLSCAN", "filter" : { "name" : { "$eq" : "PERCONA" } 9 },

10 Indexes in MongoDB bigcollection (~ documents) With Indexes db.bigcollection.find({name : 'PERCONA' }).explain(true) "executionstats" : { "executionsuccess" : true, "nreturned" : 0, "executiontimemillis" : 0, "totalkeysexamined" : 0, "totaldocsexamined" : 0, "executionstages" : { "stage" : "FETCH", "nreturned" : 0, "executiontimemillisestimate" : 0, "inputstage" : { "stage" : "IXSCAN", "nreturned" : 0, "executiontimemillisestimate" : 0, 10

11 Indexes in MongoDB smallcollection (100 documents) - With Indexes db.smallcollection.find({name : 'PERCONA' }).explain(true) "executionstats" : { "executionsuccess" : true, "nreturned" : 0, "executiontimemillis" : 0, "totalkeysexamined" : 0, "totaldocsexamined" : 0, "executionstages" : { "stage" : "FETCH", "nreturned" : 0, "executiontimemillisestimate" : 0, "inputstage" : { "stage" : "IXSCAN", "nreturned" : 0, "executiontimemillisestimate" : 0, "works" : 1, 11

12 How do indexes work? A-Z PERCONA M-Z A-L PERCONA A-G M-S H-L T-Z Q-S M - (percona) P 12

13 Costs of maintaining indexes: Each Index adds cost to the writing process. Writes will occur in real time there is no "delay" when updating indexes. A lot of indexes will probably slow the write performance down. 13

14 Costs of maintaining an index A-Z PERCONA M-Z A-L PERCONA A-G M-S H-L PERCONA T-Z Q-S M-P 14

15 Indexes in MongoDB The index types are: _id GeoIndexes (geohaystack, 2d spherical, flat 2d indexes) Single Field MultiKey indexes Compound Indexes Hash indexes Text Sparse TTL indexes Unique Partial 15

16 Types of Indexes _id is the default index in mongodb. _id is not clustered and the engine must perform another operation so as to read all the values of the document. 16

17 Types of Indexes Single field is also called simple index and it indexes only one field at a time. Creating a single field index will help the query optimizer find the desired document quickly. 17

18 Types of Indexes When single field is not enough by itself, the compound indexes help the execution plan to filter as many documents as possible. If we filter by name and lastname and only the name field is indexed, we may need to open a lot of documents to read the lastname. On the other hand, if lastname is already indexed, there is no need to do so. 18

19 Types of Indexes Multikey Indexes are used when the field value is an array; For each key in the array there is an entry with its values. For the document { likes : ['mongodb', 'mysql'] }, the multikey will generate 2 different index keys pointing to the same document. 19

20 Types of Indexes Unique keys are unique! There isn't much to say here... except that unique keys don t work in a sharded environment. 20

21 Types of Indexes There are plenty of GeoIndexes. MongoDB offers rich geo algorithms; For flat calculations, we can use 2d indexes; It supports 2d sphere; We can also use geohaystack in order to get responses - This will compare small squares instead of reading whole collection: It is very similar to geohashes. 21

22 Types of Indexes Text indexes behave similarly to the single field index, but there are some extended properties such as collations and case insensitive. We can also give weight to words when running a search. 22

23 Types of Indexes Sparse and Filtered indexes. Although they sound very similar, sparse indexes only create index keys in documents that do have the field. The filtered index will create an index only for a field value that matches the argument. E.g { 'age' : {$gte : 10 }} 23

24 Types of Indexes Hash indexes are commonly used in shards to create random keys for the writes and increase the write performance. 24

25 Types of Indexes TTL indexes are very useful. Every minute mongodb will run a "TTL round", where all the documents matching time to live will be removed from the database. It is very useful, but when you are creating a TTL in a big collection, it is better to first delete the expired documents, otherwise the database load will increase significantly. 25

26 Quick Review Indexes are good for you if well used. Avoid creating more indexes than required. Use the right index for your application. Indexes use space in the disk and memory. Indexing all the fields in one collection is useless in 90% of the cases. There is no magic, fast processors and memory will help you. 26

27 Q&A 27

28 Rate this session 28