MySQL Replication Tips and Tricks

Transcription

1 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Replication Tips and Tricks Dr. Mats Kindahl Lead Developer, Replication mats@sun.com mysqlmusings.blogspot.com Dr. Lars Thalmann Development Manager, Replication and Backup lars.thalmann@sun.com larsthalmann.blogspot.com Conference and Expo April 23 th, 2009

2 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Tricks and Tips L Some old, some new Some easy, some more advanced Meant to be food for thoughts

3 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Slave stopped now what? Check position it stopped on SHOW SLAVE STATUS Next event to transfer from master: Master_Log_File, Read_Master_Log_Pos Next event to apply: With respect to master log: Relay_Master_Log_File, Exec_Master_Log_Pos With respect to relay log: Relay_Log_File, Relay_Log_Pos Use mysqlbinlog to read the contents mysqlbinlog master-log mysqlbinlog relay-log Investigate the problem, and possibly: Remove rows in database; START SLAVE SET SQL_SLAVE_SKIP_COUNTER=1; START SLAVE M

4 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Examining binlog mysqlbinlog --hexdump master-bin Statement-based INSERT1/2: Query event header L $ mysqlbinlog --hexdump master-bin # at 235 # :16:02 server id 1 end_log_pos 351 # Position Timestamp Type Master ID # eb e2 cf # Size Master Pos Flags # f

5 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Examining binlog mysqlbinlog --hexdump master-bin Statement-based INSERT 2/2: Query event data # fe # a # e std # e test.INSE # e RT.INTO.t1.VALUE # e S...A...B...X # e 27 2c......Y...X...X. # e 29. # Query thread_id=2 exec_time=0 error_code=0 SET TIMESTAMP= ; INSERT INTO t1 VALUES ('A','B'), ('X','Y'), ('X','X'); L

6 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Examining binlog mysqlbinlog --verbose master-bin Reconstructing INSERT from row-based events L $ mysqlbinlog --verbose master-bin BINLOG ' qznvsrmbaaaakqaaaaycaaaaabaaaaaaaaaabhrlc3qaanqxaaedaae= qznvsrcbaaaajwaaac0caaaqabaaaaaaaaeaaf/+awaaap4eaaaa '/*!*/; ### INSERT INTO test.t1 ### SET ### INSERT INTO test.t1 ### SET

7 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Replication to get online backup Replicate to a slave server So that all data is there Stop replication To stop changes to the slave Do backup on the slave It does not matter if this is blocking since the master is not blocked M This technique is used by the Time Machine Backup

8 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB L Extra slave to initiate new slaves Extra slave New slave

9 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Using mysqldump to inititialize slave M Dump data from a server master and restore it on another server slave : mysqldump --host=master uuser -master-data --lock-all-tables --databases test mysql --host=slave uuser This locks the database!

10 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Using mysqldump to inititialize slave M To avoid locking the entire table: mysqldump --host=master uuser -single-transaction - master-data --databases test mysql --host=slave uuser Note: only works for tables that can handle consistent snapshots, such as InnoDB

11 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Using replication to repopulate a table M Write a script that, for each table t1: SELECT * INTO OUTFILE t1.txt FROM t1; DROP TABLE IF EXISTS t1; Works with statementbased replication CREATE TABLE t1 ; LOAD DATA INFILE t1.txt INTO TABLE t1; A version using INSERT-SELECT (works for row-based): CREATE TEMORARY TABLE t1_tmp LIKE t1; INSERT INTO t1_tmp SELECT * FROM t1; Works with rowbased replication DROP TABLE IF EXISTS t1; CREATE TABLE t1 SELECT * FROM t1_tmp; For row-based replication, the temporary table is not transferred to the slave, so this will only send the data that is necessary to bootstrap the table.

12 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB L Point-in-time recovery using mysqlbinlog For point in time recovery, you can take the replication log and apply it into the. mysqlbinlog \ --read-from-remote-server \ --host=master -uroot --position=192 \ --to-datetime= :36:56 \ master-bin.00002[2-4] mysql -uroot --host=slave

13 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB M Proxy to do multi-master Proxy Proxy writes a new binary log

14 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Use a default engine for a server CREATE TABLE ( uid INT AUTO_INCREMENT PRIMARY KEY, name VARCHAR(64) ) ENGINE=INNODB; SET GLOBAL STORAGE_ENGINE = INNODB; CREATE TABLE ( uid INT AUTO_INCREMENT PRIMARY KEY, name VARCHAR(64) ); Not replicated, so remember to set default engine on slaves!

15 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB M Split replication stream on table replication do table=east_coast replication do table=west_coast Relay Slave ALTER TABLE xxx ENGINE = BLACKHOLE SET GLOBAL STORAGE_ENGINE = BLACKHOLE Relay Slave

16 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB L Cluster multi-source replication Cluster Cluster Cluster Let masters update different data

17 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB L Multi-channel replication with fail-over Cluster Cluster

18 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB M Slave having time-sharing masters Clocked switching 1. Stop slave 2. Save position 3. Change master 4. Start slave

19 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Stop the slave and save position using sh(1) socket="mysql-test/var/tmp/$1.sock" mysql_exec () { # Read commands from stdin client/mysql --vertical --socket="$1" --batch \ --skip-column-names --batch user=root } stop_slave () { echo STOP SLAVE mysql_exec "$1"; } fetch_pos () { echo SHOW SLAVE STATUS mysql_exec "$1" grep '\<Master_\(Host\ Port\ Log_File\)\ \<Read_Master_Log_Pos' cut -f2 -d: } stop_slave "$socket" echo `fetch_pos "$socket"` >$1.savepos M

20 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Restore the slave position and start slave using sh(1)... start_slave () { echo START SLAVE mysql_exec "$1"; } M change_to_pos () { mysql_exec "$1" <<EOF # Pass command using here-is CHANGE MASTER TO MASTER_HOST='$2', MASTER_PORT=$3, MASTER_LOG_FILE='$4', MASTER_LOG_POS=$5 EOF } name cat $1.savepos { read host port file pos change_to_pos $socket "$host" "$port" "$file" "$pos" start_slave "$socket" }

21 cnt= Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Rotate between several slaves using sh(1) while true; do save_pos_and_stop_slave mysqld.$cnt cnt=`expr $cnt % 5 + 1` restore_pos_and_start_slave mysqld.$cnt } M names are of the form mysqld.x savepos files are of the form mysqld.x.savepos

22 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB M Encrypted replication CHANGE MASTER TO MASTER_SSL = 1, MASTER_SSL_CA = 'cacert.pem' MASTER_SSL_CERT = mycert.pem MASTER_SSL_KEY = mykey.pem'

23 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Using current database to filter In STATEMENT format, you can set current db to filter statements: If --binlog-ignore-db=db1 is used on server, then the following INSERT is not logged: USE foo; INSERT into bar.t1 VALUES(1); If --replicate-ignore-db=db1 is used on slave, then the following INSERT is not applied: USE foo; INSERT into bar.t1 VALUES(1); M In ROW format, you can not use this trick, since row-based always use the actual database: The following INSERT is logged and applied if db1 is not filtered: USE foo; INSERT into db1.t1 VALUES(1);

24 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB M More columns on slave than master You can add a column on the slave: Master: CREATE TABLE t1 (a INT, b INT); Slave: ALTER TABLE t1 ADD ts TIMESTAMP; Extra columns have to have default values! Master: INSERT INTO t1(a,b) VALUES (10,20); Row-based replication need extra column last

25 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB M Less columns on slave than master In row-based replication, you can remove columns on the slave: Master: CREATE TABLE t1 (a INT, b INT, comments TEXT); Slave: ALTER TABLE t1 DROP comments; Only possible to remove columns from the end Master: INSERT INTO t1 VALUES (1,2, Do not store this on slave );

26 New in 6.0/ Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Replication Heartbeat Master Replication I/O thread Slave SE1 SE2 Binlog Relay Binlog SE1 SE2 Binlog Storage Engines

27 New in 6.0/ Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Replication Heartbeat Master Slave Automatic checking of connection status No more relay log rotates when the master is idle Detection of master/slave disconnect configurable in millisecs CHANGE MASTER SET master_heartbeat_period= val; SHOW STATUS like 'slave_heartbeat period' SHOW STATUS like 'slave_received_heartbeats'

28 New in 6.0/ Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Ignoring s in Circular Replication A B D C Circular replication If server A is removed from the circle, server B can be set to terminate A's events in the new circle B> CHANGE MASTER TO MASTER_HOST=C... IGNORE_SERVER_IDS=(A)

29 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB L Replication database test c1 c2 c copy rpl_db c1 c2 c master replication c1 c2 c3 rpl_db Slave

30 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB L 1. Two options to only replicate this part: 1. On master: Replication database [mysqld]... binlog-do-db=rpl_db Note that this filters the binlog on master On slave: [mysqld]... replicate-do-db=rpl_db This filters the replication execution on the slave Note that you need row-based for this to work In statement-based replication, the source tables are need to execute the statement.

31 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB L Replication database CREATE DATABASE rpl_db; USE test; CREATE TABLE t1 (a INT, b INT, c INT); USE rpl_db; CREATE TABLE t1 (a INT, b INT, c INT); USE test; # Not replicated INSERT INTO t1 VALUES (1,2,3), (2,5,9); USE rpl_db; # Replicated INSERT INTO t1 VALUES (3,4,5), (4,6,11);

32 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Less columns on slave than master L a b c a c trigger master rpl_db replication a c slave rpl_db

33 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB L Less columns on slave than master Master: CREATE DATABASE rpl_db; USE test; CREATE TABLE t1 (a INT, b BLOB, c INT); CREATE TRIGGER tr_t1 AFTER INSERT ON test.t1 FOR EACH ROW INSERT INTO rpl_db.t1_v(a,c) VALUES(NEW.a,NEW.c); USE rpl_db; CREATE TABLE t1_v (a INT, c INT); Use like this: USE test; = REPEAT('beef',100); INSERT INTO t1 VALUES (1,@blob,3), (2,@blob,9);

34 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB M Only replicate some rows to slave a b c trigger rpl_db a b c master replication a b c Slave rpl_db

35 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB M Only replicate some rows to master # Just replicate rows that has odd numbers in first column. USE rpl_db; CREATE TABLE t1_h (a INT, b BLOB, c INT); --delimiter // CREATE TRIGGER slice_t1_horiz AFTER INSERT ON test.t1 FOR EACH ROW BEGIN IF NEW.a MOD 2 = 1 THEN INSERT INTO rpl_db.t1_h VALUES (NEW.a, NEW.b, NEW.c); END IF; END// --delimiter ;

36 Provides: Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Execution of replication events on slave is to be N seconds behind master User interface: Feature Preview Time-delayed replication CHANGE MASTER TO MASTER_DELAY= N N is a non-negative less than MAX_ULONG number of seconds to wait by the slave. (Attempts to set higher is rejected with error.) Controlling of a delayed setup is via SHOW SLAVE STATUS Seconds_behind_master Kudos: Kay Röpke, original patch; Sven, modifications L

37 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Scriptable Replication High-level structure Client Master Slave Dump I/O SQL function befo local q = eve slave:query(q end function befo local q = eve slave:query(q end Binary Log function befo local q = eve slave:query(q end Extension modules Relay Log function befo local q = eve slave:query(q end

38 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB The oracle algorithm Paul Tuckfield s (Google/YouTube) module(..., package.seeall); require luasql.mysql pattern = { ["UPDATE%s+(%w+).*%s(WHERE.*)"] = "SELECT * FROM %1 %2", ["DELETE%s+FROM%s+(%w+).*%s(WHERE.*)"] = "SELECT * FROM %1 %2", } env = luasql.mysql() con = env:connect("test", root,, localhost, mysql.port) function before_write(event) local line = event.query if not line then return end for pat,repl in pairs(pattern) do local str = string.gsub(line, pat, repl) if str then con:execute(str); break; end end end

39 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Semi-synchronous Replication Original patch: Mark Callaghan and Wei Li, Google Adoptions: Zhenxing He, Sun Microsystems Application Logging/ Replication Semi-Sync Replicator Semi-Sync Receiver Relay Log/ Applier Application Master Replication L R R A Slave Ack SE1 SE2 Binlog Relay Binlog SE1 SE2 Storage Engines Storage Engines

40 On master: New in 6.0/ Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Install Plug-ins and Play INSTALL PLUGIN 'rpl_semi_sync_master' SONAME 'libsemisync_master.so'; SET rpl_semi_sync_master_enabled=1; SET rpl_semi_sync_master_timeout=1000; On slave: milliseconds INSTALL PLUGIN 'rpl_semi_sync_slave' SONAME 'libsemisync_slave.so'; SET rpl_semi_sync_slave_enabled=1; START SLAVE;

41 Lars Thalmann & Mats Kindahl Replication Tricks and Tips AB Look into the DevZone L Replication DevZone Dr. Mats Kindahl Lead Developer, Replication Technology mats@sun.com Dr. Lars Thalmann Development Manager, Replication and Backup Technology lars.thalmann@sun.com