MongoDB Distributed Write and Read

Transcription

1 VICTORIA UNIVERSITY OF WELLINGTON Te Whare Wananga o te Upoko o te Ika a Maui MongoDB Distributed Write and Read Lecturer : Dr. Pavle Mogin SWEN 432 Advanced Database Design and Implementation

2 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 1 Plan for Distributed Write and Read Distributed Write Write on Sharded Cluster Write on Replica Sets Write Concern Bulk() Method Distributed Queries MongoDB and Transaction Processing Reedings: Have a look at Readings on the Home Page

3 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 2 Write Operations on Sharded Clusters For sharded collections in a sharded cluster, the mongos directs write operations from applications to shards that are responsible for the portion of the data set using the sharding key value The mongos gets needed metadata information from the config database residing on config servers

4 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 3 Sharded Cluster Application Server Shard (replica set) Router (mongos) Data Driver Shard (replica set) Writes Metadata Config Server Config Server Config Server

5 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 4 Write Operations on Replica Sets In replica sets, all write operations go to the set s primary The primary applies the write operations and then records the operations on its operation log (oplog) Oplog is a reproducible sequence of operations to the data set Secondary members of the set continuously replicate the oplog by applying operations to themselves in an asynchronous process

6 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 5 Replica Set Operations Client Application Writes Primary Secondary Secondary

7 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 6 New insert Syntax MongoDB V2.6 and later support a new insert syntax: db.runcommand( { ) insert: <collection>, documents: [<document>, <document>,... ], ordered: <boolean>, writeconcern: { <write concern>}, bypassdocumentvalidation: <boolean> }

8 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 7 An insert Example db.runcommand( { insert: mycollection", documents: [ doc1, doc2, doc3 ], ordered: false, writeconcern: { w: "majority", wtimeout: 5000 } } ) We focus on the optional writeconcern option

9 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 8 Write Concern (1) Write concern describes the guarantee that MongoDB provides when reporting on the success of a write operation The strength of the write concerns determines the level of guarantee When inserts, updates and deletes have a weak write concern, write operations return quickly In some failure cases, write operations issued with weak write concerns may not persist With stronger write concerns, clients wait longer after sending a write operation, for MongoDB to confirm the write operations

10 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 9 Write Concern (2) MongoDB (version 2.6 and later) provides different levels of write concern: Unacknowledged (lowest level), Acknowledged (default), Journaled, and Replica Acknowledged (highest level) Clients may adjust write concern to ensure that the most important operations persist successfully to an entire MongoDB deployment For other less critical operations, clients can adjust the write concern to ensure faster performance rather than ensure persistence to the entire deployment

11 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 10 Specification of writeconcern Write concern can include the following fields: { w: <value>, j: <boolean>, wtimeout: <number> } The w option requests acknowledgement that the write operation has propagated to a specified number of mongod instances The j option requests acknowledgement that the write operation has been written to the journal, and The wtimeout option to specify a time limit in miliseconds to prevent write operations from blocking indefinitely w = 0 means no acknowledgement of the write operation, w = 1 is the default write concern and requests acknowledgement that the write operation has propagated to the standalone mongod or the primary in a replica set w = majority requests acknowledgement that write operations have propagated to the majority of voting nodes, including the primary

12 Write writeconcern: {w: 0} Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 11 Write Concern: Unacknowledged If {w: 0}, MongoDB does not acknowledge the receipt of a write operation Driver mongod Apply

13 Write writeconcern: {w: 1} Response Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 12 Write Concern: Acknowledged If {w: 1}, MongoDB confirms that it applied a change to the in memory data Driver Data persisting on disk is not confirmed mongod Apply

14 Write writeconcern: {w: 1, j: true} Response Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 13 Write Concern: Journaled If {w: 1, j: true}, MongoDB confirms that it committed data on (master s) disk Driver mongod Apply Journaling latency Journal

15 Replicate Replicate Write Concern: {w: 2} Response Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 14 Write Concern: Replica Acknowledged If {w: 2}, the first secondary to finish in memory application of primary s oplog operation, returns acknowledgment Driver Primary Journaling latency Apply Secondary Apply

16 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 15 Distributed Queries Applications issue operations to one of mongos instances of a sharded cluster Read operations are most efficient when a query includes the collection s shard key Otherwise the mongos must direct the query to all shards in the cluster (scatter gather query) and that might be inefficient By default, MongoDB always reads data from a replica set s primary

17 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 16 Reading From a Secondary Reading from a secondary server is possible and justified if there is a need : To balance the work load, To allow reads during failover, but Eventual consistency can be guaranteed, only To allow reading from a slave server, one of the following set-ups are needed: Modifying the read preference mode in the driver, which results in a permanent change, or Connecting to a slave server shell and issuing the following commands : db.getmongo().setslaveok() use <db_name> db.collection.find()

18 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 17 Read Concern To use readconcern with the find() method: db.collection.find().readconcern(<level>) The level parameter of the readconcern() method has the following values: local is default The query returns the most recent copy of data Provides no guarantee that the data has been written to a majority of the replica set members majority The query returns the most recent copy of data confirmed as written to a majority of members in the replica set

19 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 18 Read Isolation MongoDB allows clients to read documents inserted or modified before committing modifications to disk, regardless of write concern level MongoDB performs journaling frequently, but only after a defined time interval If the mongod terminates before the journal commits, even if a write returns successfully, queries may have read data that will not exist after the mongod restarts This is a read uncommitted transaction anomaly. When mongod returns a successful journaled write concern ( j: true ), the data is fully committed to disk and will be available after mongod restarts

20 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 19 Atomicity A write operation is atomic on the level of a single document, even if the operation modifies multiple embedded documents within a single document When a single write operation modifies multiple documents, the modification of each document is atomic, but the operation as a whole is not atomic and other operations may interleave There exists the $isolated operator that can isolate a single write operation But it does not work on sharded clusters

21 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 20 Transaction Like Semantics Since a single document can contain multiple embedded documents, single-document atomicity is sufficient for many practical use cases For cases where a sequence of write operations must operate as if in a single transaction, a two-phase commit can be implemented in an application However, the two-phase commit can only offer transaction-like semantics Using two-phase commit ensures data consistency, but it is possible for applications to return intermediate data during the two-phase commit or rollback

22 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 21 Concurrency Control In relational databases, concurrency control allows multiple applications to run concurrently without causing data inconsistency or conflicts MongoDB does not offer such mechanisms Instead, there are techniques to avoid some sorts of inconsistencies: Unique indexes used with certain methods like findandmodify() prevent duplicate insertions or updates Also, there are certain programming patterns that can be applied to avoid concurrency control anomalies, like the lost update anomaly

23 Advanced Database Design and Implementation 2018 MongoDB_Distributed_WR 22 Summary Routers direct client read and write operations to shards and their replica sets using meta data from config servers All writes go to the master server By default, all reads also go to the master server Write Concern is the guarantee that MongoDB provides when reporting on the success of a write operation Week write concern: fast, but not very reliable Strong write concern: slower, but more reliable By default, queries are of the type read uncommitted Queries based on the shard key value are the fastest Transaction like behavior is achievable to some extent