MongoDB - Replication & Sharding

MongoDB - Replication & Sharding Masterprojekt NoSQL Mirko Köster Universität Hamburg Fachbereich Informatik Arbeitsgruppe VSIS 29. November 2013 Mirko Köster MongoDB - Replication & Sharding 29.11.2013 1 / 26

Agenda 1 Replication 2 Sharding Mirko Köster MongoDB - Replication & Sharding 29.11.2013 2 / 26

Replication Mirko Köster MongoDB - Replication & Sharding 29.11.2013 3 / 26

Motivation Redundancy Failover Maintanance Load Balancing Mirko Köster MongoDB - Replication & Sharding 29.11.2013 4 / 26

Overview Master - Slave - Replication deprecated more than 12 nodes (ReplSet) up to 12 nodes consists of 1 master and some slaves (and/or arbiter(s)) write to master read from master consistent read from slave eventually consistent Mirko Köster MongoDB - Replication & Sharding 29.11.2013 5 / 26

ReplSet States State Vote State Description All members start up in this state. The mongod STARTUP parses the replica set configuration document while in STARTUP. PRIMARY + The primary is the only member to accept write operations. SECONDARY + The secondary replicates the data store. RECOVERING + Members either perform startup self-checks, or transition from completing a rollback or resync. FATAL Has encountered an unrecoverable error. STARTUP2 Forks replication and election threads before becoming a secondary. UNKNOWN Has never connected to the replica set. ARBITER + Arbiters do not replicate data and exist solely to participate in elections. DOWN Is not accessible to the set. ROLLBACK + Performs a rollback. SHUNNED Was once in the replica set but has now been removed. Mirko Köster MongoDB - Replication & Sharding 29.11.2013 6 / 26

Heartbeat Every node pings every other node every 2 seconds n (n 1) pings every 2 seconds worst case: 12 (12 1) = 132 pings every 2 seconds Mirko Köster MongoDB - Replication & Sharding 29.11.2013 7 / 26

Failover If the master becomes unresponsive, a new master has to be elected. Majority Vote Initial number of nodes defines size of majority. Majority of n nodes: n/2 + 1 In most cases the most up to date secondary will be elected master. After recovery the former failed master may rejoin the ReplSet as secondary. Missing data will be replicated. If too far behind: complete resync No majority no master The current master will step down and become a secondary. Writing to the ReplSet is now impossible. Mirko Köster MongoDB - Replication & Sharding 29.11.2013 8 / 26

Replication oplog - master records all changes to the data capped collection on 64 Bit Linux: the greater of 1 GB and 5% discsize entries (transactions) are idempotent and identified with a BSON timestamp. entries are replicated to secondaries (and applied locally). In case the master crashed: entries from the oplog, which have not been replicated, will be rolled back. the oplog does NOT replace journaling. journaling is still advised to minimize downtime in case of node crash. Mirko Köster MongoDB - Replication & Sharding 29.11.2013 9 / 26

Commits & Write Concerns a change is considered to be commited, once it is replicated to a majority of nodes. In most cases the consistency one gets by writing to the master is good enough. If it is not, one can use write concerns. controlled via two parameters w: total number of nodes the change must have been replicated to (can be a number or the value "majority") timeout: how long (time in ms) the application should wait (block) Mirko Köster MongoDB - Replication & Sharding 29.11.2013 10 / 26

ReplSet Config (excerpt) priority votes arbiteronly slavedelay Mirko Köster MongoDB - Replication & Sharding 29.11.2013 11 / 26

MongoDB Drivers available for most common languages e.g. C, C++, C#, Erlang, Java, Perl, PHP, Python, Ruby, Scala connect to specific node master secondary (read only) set parameter :slave_ok to true no automatic failover connect to whole ReplSet aware of all nodes automatic failover but error not hidden see driver s documentation Mirko Köster MongoDB - Replication & Sharding 29.11.2013 12 / 26

Scenarios - Heartbeat Mirko Köster MongoDB - Replication & Sharding 29.11.2013 13 / 26

Scenarios - Arbiter Mirko Köster MongoDB - Replication & Sharding 29.11.2013 14 / 26

Scenarios - Two Datacenters Mirko Köster MongoDB - Replication & Sharding 29.11.2013 15 / 26

Scenarios - Three Datacenters Mirko Köster MongoDB - Replication & Sharding 29.11.2013 16 / 26

Read Scaling a ReplSet with the maximum of 12 Servers means majority = 7 If your cluster is available, that is the lower bound of nodes you can count on for read performance. not enough? you have basically 2 options (Master - Slave - Replication) Sharding Mirko Köster MongoDB - Replication & Sharding 29.11.2013 17 / 26

Sharding Sharding Mirko Köster MongoDB - Replication & Sharding 29.11.2013 18 / 26

Sharding Motivation distribute large volumes of data evenly accross nodes add capacity increase write and/or read throughput keep working data + indexes in RAM RAM is limiting factor reduce load on node(s) production-level sharding since August 2010 (v1.6) Mirko Köster MongoDB - Replication & Sharding 29.11.2013 19 / 26

Sharding Sharding Components shards each shard is a ReplSet routers ( mongos ) no state often run on the same server as application config servers 3 config servers - on 3 different machines (redundancy) may be run on same server with other MongoDB services persistently store cluster state (shard cluster meta data) Mirko Köster MongoDB - Replication & Sharding 29.11.2013 20 / 26

Sharding Scenarios - Sharding Mirko Köster MongoDB - Replication & Sharding 29.11.2013 21 / 26

Sharding Sharding a collection (1) MongoDB shards data on a per collection basis. Every database has a primary shard that holds all the un-sharded collections in that database. define a shard key per sharded collection (basically the main index used to distribute the data among the shards). The data is split into non-overlapping ranges (chunks). distributed evenly accross cluster not guaranteed to be contiguous per shard. Mirko Köster MongoDB - Replication & Sharding 29.11.2013 22 / 26

Sharding Sharding a collection (2) Balancing automatic splitting and migrating of chunks splitting (logical): default max chunk size is 64 MB or 100,000 documents migrating (physical): per default when number of chunks diverge by 8 or more Sharding logic separated from application little to no need to change app code after migrating from single server (or ReplSet) to sharded cluster Indexes on a per-shard basis unique index only for shard key and/or _id otherwise it would require inter-shard-communication. Mirko Köster MongoDB - Replication & Sharding 29.11.2013 23 / 26

Sharding Query / Query Performance shard key part of query? yes: execute on right shard(s) no: hit all shards scatter / gather - query merge result on router use prefix subset of compound shard key analyze queries (e.g. performance, how many shards involved etc) Mirko Köster MongoDB - Replication & Sharding 29.11.2013 24 / 26

Summary use replication in production commodity hardware sufficient easy to set up highly customizable enable sharding when needed analyze needs of your application analyze queries to find a good shard key enable it early - not when it s too late Mirko Köster MongoDB - Replication & Sharding 29.11.2013 25 / 26

Literatur [Ban12] Banker, Kyle: MongoDB in Action. 20 Baldwin Road, PO Box 261, Shelter Island, NY 11964, USA : Manning Publications Co., 2012. 288 S. ISBN 9781935182870. ebook edition (pdf) [Cho13] Chodorow, Kristina: MongoDB: The Definitive Guide. Second Edition. 1005 Gravenstein Highway North, Sebastopol, CA 95472, USA : O Reilly Media, Inc., 2013. 410 S. ISBN 9781449344689. ebook edition (pdf) [Mon13] MongoDB Documentation. Version: Release 2.4.8, 2013. http: //docs.mongodb.org/v2.4/mongodb-manual.pdf. ebook edition (pdf) Mirko Köster MongoDB - Replication & Sharding 29.11.2013 26 / 26