<Insert Picture Here> New MySQL Enterprise Backup 4.1: Better Very Large Database Backup & Recovery and More!

Transcription

1 <Insert Picture Here> New MySQL Enterprise Backup 4.1: Better Very Large Database Backup & Recovery and More! Mike Frank MySQL Product Management - Director

2 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle.

3 Agenda Brief Overview of Backup & Recovery Options Backups and Performance 101 What s New, When to Use, How to Use Optimistic Differential Backup Improved Redo Log Copying Improved Log Messages Quick Refresh Fast Compression High Compression Encryption Enhanced Zero Space Streaming Enhanced Filtering And More And How to Use Questions Please Ask Them Anytime during the Webinar using Q&A Panel

4 BRIEF OVERVIEW OF BACKUP AND RECOVERY OPTIONS

5 Types of Backup and Tools for these types Logical Mysqldump Mysqlpump Next Generation for logical backup Physical MySQL Enterprise Backup (MEB) Other Snapshots Replicas

6 Logical Backup Collection of SQL Queries to recreate the database Advantages Easy of use - simple commands allow you to easily backup and restore Good for small database or tables minimum impact on backup and restore performance Flexibility logical backup allows you to choose what you want to backup and not backup. Change the scripts to restore partially etc. Readability - good assurance that database files are not corrupt - all the data is read and it is read using standard SQL queries. Portable use across various OS and MySQL versions

7 Logical Backup Collection of SQL Queries to recreate the database Disadvantages Not an online solution - write operations are locked while performing the backup, thereby blocking use of the database. Poor performance for larger databases backup and especially restore times are very slow for larger databases. Not consistent database won t necessarily be restored to a consistent state. No incremental backup all backups are full backups, can be time consuming and require more storage.

8 Physical Backup (MEB) Copy of the internal files that constitute the MySQL database Advantages Performance faster for backups and faster for restore Flexible support for incremental backups, partial backups, backup compression, point in time recovery and more. Archival Backups suitable archival format for historical purposes Scalable performance is near linear for larger databases. Consistent delivers consistent point in time recovery

9 Backup Options Comparison/Summary MySQL Enterprise Backup mysqlpump mysqldump LVM Snapshot MySQL HA GR/Replication Full Backup Y Y Y Y Y Incremental Backup Y N N Y N Partial Backup Y Y Y N N Compression Support Y N N N N Encryption Support Y N N N N Allows Updates Y N N N Y Point in time consistent Y N N Y Y Backup Speed Very Fast Slow Very Slow Fast Very Fast Recovery Speed Very Fast Very Poor Very Poor Fast Very Fast Partial Restore Y Y Y N N Corruption Detection Y Y Y N N Meet Regulatory Archive Requirements Y Y Y N N Supports DDL Y Y Y N N Streaming Support Y Y Y N N Direct Media Manager/Tape Support Y N N N N

10 Quick Lesson Backup Performance 101

11 To consider performance think of structure of a running backup one way is like this.

12 To consider performance think of structure of a running backup or like this.

13 To consider performance think of structure of a running backup or like this.

14 What determines backup speed? The longest time

15 What determines backup speed? The longest time

16 What determines backup speed? The longest time

17 What determines backup speed? The longest Time

18 Too much Sharing slows things down Mostly IO Contention, At times CPU

19 Improving backup speed

20 Does Highest Compression=Higher Speed Not typically High Compression Impacts on CPU

21 MySQL Enterprise Backup is Truly Parallel Read, Write, Processing Parallel Processing Is Faster

22 MySQL Enterprise Backup is Buffered

23 Image or Directory backup and Performance MySQL Enterprise Backup supports Backup to a single image a single file or streamed Backup to a directory In our testing confirms Single image can be a bit better Performance advantage comes from combining different steps that you might otherwise have to perform in sequence Focus on using image mode Only supported for encryption for instance Image is simpler to manage Image can be converted to directory

24 Understanding your system CPU load Patterns look at times of day Load level is it low or high often DBs are low = 30% IO load Patterns look at times of day Architecture IO rate How fast can you read database files? How fast can you write? Run DD or other tools whats you MB/sec?

25 WHAT S NEW, WHEN TO USE, HOW TO USE

26 Optimistic Backup Overview Takes advantage of Database table usage patterns Tables that are infrequently updated Tables that are typically only update during certain periods of time Uses that information to optimize the backup pattern Results in Faster, Less Time Smaller Less overhead backups Quicker recovery, Less Time

27 Optimistic Backup Overview Hows it work You provide information about tables Which are active or inactive Based on names or timing etc. Use this info to perform a 2 phase backup

28 Why is optimistic better faster in some cases Normally we assume every table could change Start collecting the REDO log at start of the backup All of it Thus if the backup takes 1 hour we backup 1 hour of REDO changes for all tables If we know many tables don t change then We don t start copying the REDO till we get to active tables Its may be that 80% of data isn t changing We copy that 80% Then start active 20%. REDO may be ~80% smaller On restore REDO brings to a consistent state Now has 80% fewer pages to go through and apply

29 Phase 1 copy inactive files Phase 2 copy active files

30 Optimistic Parameters optimistic-time This option specifies the cut-off date-time to identify tables to be skipped in first phase. Several options - easiest is {Number}{Unit} for example --optimistictime=23hours optimistic-busy-tables This option specifies the InnoDB tables (i.e. busy tables) to be skipped in the first phase. Defined using a regular expression Note: optimistic-busy-tables gets higher precedence than the optimistictime option

31 Optimistic Differential Backup We already have FULL optimistic now Incremental completes the feature - even greater value Without incremental optimistic Current algorithm checks all tables to find changed pages With optimistic (more efficient) Scans only those tables that have been modified since the previous backup In case of optimistic incremental backup, no additional parameters are necessary No need for --optimistic-time or --optimistic-busy-tables options Busy tables are already clearly defined and identified

32 How s it optimistic incremental works Optimistic incremental backup starts Compares the modified time of each table against the consistency time. If the modification time of a table is greater than the consistency time Table has been modified after consistency time was recorded Optimistic incremental backup copies changed pages

33 How s it work Example Visual Table 1 Table 2 Table 4 Table 5 CreateTable 7 Table 3 Table 6 Update Table 1 CreateTable 8 Table 1 Table 2 Table 4 Table 5 Table 7 Table 8 Table 3 Table 6 There are 6 tables to be scanned MEB notes the consistency time Table7 is created while backup is still copying the meta files. After the backup is finished, Table1 is updated and Table8 is created. Incremental starts, looks for the consistency time from the prior Compares the consistency time with the modification time of all the finds that only three tables that have been modified after consistency time. Three tables are scanned for changed pages. Remaining tables are unchanged tables and are ignored. Any unchanged tables are modified during optimistic incremental backup, changes recorded in the redo log file. And will be copied and applied during a restore.

34 Example of Optimistic Backups FULL and Incremental FULL this uses a datetime format could also use hours, days, weeks, etc. since change format # A full optimistic backup performed on 2017/02/04, Sat, at 1130 PM. mysqlbackup --defaults-file=/home/admin/my.cnf \ --optimistic-time= \ --backup-dir=/home/admin/temp_dir --backup-image=/home/admin/bkups/mydb_full_ bi \ --with-timestamp \ backup-to-image # A sequence of optimistic incremental backups are then performed on each the following six days at 1130 PM # On Sunday, 2017/02/05 mysqlbackup --defaults-file=/home/admin/my.cnf \ --incremental=optimistic --incremental-base=history:last_backup \ --backup-dir=/home/admin/temp_dir \ --backup-image=/home/admin/bkups/mydb_incremental bi \ --with-timestamp \ backup-to-image

35 Example of Optimistic Restore FULL and Incremental # Restore the database to its state at Tuesday, 2017/02/07, at 11:30 PM # First, restore the full optimistic backup taken on the Saturday before, which was 2017/02/04: mysqlbackup --defaults-file=/etc/my.cnf --backup-image=/home/admin/backups/mydb_full_ bi \ --backup-dir=/home/admin/temp_dir --datadir=/var/lib/mysql \ --with-timestamp \ copy-back-and-apply-log # Next, restore the optimistic incremental taken on the Sunday, Monday, and Tuesday that follow: mysqlbackup --defaults-file=/etc/my.cnf --backup-image=/home/admin/backups/mydb_incremental bi \ --incremental-backup-dir=/home/admin/temp_dir --datadir=/var/lib/mysql --incremental \ --with-timestamp \ copy-back-and-apply-log mysqlbackup --defaults-file=/etc/my.cnf --backup-image=/home/admin/backups/mydb_incremental bi \ --incremental-backup-dir=/home/admin/temp_dir --datadir=/var/lib/mysql --incremental \ --with-timestamp \ copy-back-and-apply-log mysqlbackup --defaults-file=/etc/my.cnf --backup-image=/home/admin/backups/mydb_incremental bi \ --incremental-backup-dir=/home/admin/temp_dir --datadir=/var/lib/mysql --incremental \ --with-timestamp \ copy-back-and-apply-log

36 Various ways to report on Last Update for a Table Here using MySQLWorkbench Schema Inspector Note: Update Time is Nulled on a Restart SELECT TABLE_SCHEMA, TABLE_NAME, CREATE_TIME, UPDATE_TIME FROM information_schema.tables order by UPDATE_TIME desc; SELECT * FROM mysql.innodb_table_stats; On OS - Go to data_dir > stat %y

37 Improved Redo Log Copying Fixes issue with Redo Log overwrites If redo log records were written faster than they can be processed by MEB, the backup operation fails Log scan was only able to reach <LSN1>, but a checkpoint was at <LSN2>. Means that the database server has overwritten a part of the circular REDO log files before the backup was able to read it. To solve this problem either: Re-run the backup when the database has less load, or Re-configure the database with bigger InnoDB log files. Improved now this problem is highly unlikely

38 Improved Redo Log Copying Prior to 4.1 The redo log thread does a lot of work for a single thread

39 Improved Redo Log Copying New in 4.1 Now REDO processing split into multiple threads Could replicate the redo log overwrite issues easily with the older version, but we are yet to see the problem in the 4.1 release.

40 Improved Log Messages Logging is a very important diagnostic tool for understanding behavior troubleshooting

41 Improved Log Messages Prior to 4.1 release, the log messages are printed as shown below. Previously messages were a bit interlaced And no way to determine which thread printed the message. crucial information

42 Improved Log Messages Trace Levels The trace levels of the log messages, in the order of increasing detail Setting the trace level --trace=level 0 - INFO (information, warnings, errors) 1 - FINE (more information given than at trace level 0) 2 - FINER (finer level of information given than at trace level 1) 3 - FINEST (finest level of information that can be given)

43 Where is the log file? By default, Creates the log file in the default <backup_dir>/meta Name is MEB_<Date.time>_<operation_name>.log. User may specify any other existing directory to create the log file using the -- messages-logdir= <directory_ path> option. Users may also skip the log file creation using the --skip-messageslogdir option However that is not recommended.

44 Additional Adds New options --skip-final-rescan Potentially shortens the duration for the lock and reducing the impact on the server's normal operation Review details before using --lock-wait-timeout specify the timeout in seconds for the FLUSH TABLES WITH READ LOCK. If exceeded, the statement is failed, the lock is released, so that queries held up by the lock can then be executed. Then retry again. mysqlbackup then retries the statement and continues with the backup. Default is 60 [seconds].

45 Additional Adds New command, print-message returns an exit message for any given exit code supplied with the new option --errorcode. See Exit codes of MySQL Enterprise

46 QUICK REFRESH

47 Compression options LZ4 Fastest and Most Efficient Slightly Lower compression rate ZLIB Medium Compression Medium Cost LZMA High Compression rate Huge Dictionary High cost

48 LZ4 default compression (older versions were zlib) IF To save on write IO on backup For backup to run faster/in less time Most cases faster than uncompressed If write bound less data to write is faster And makes CPU overhead even lower On a busy database LZ4 is low overhead database CPUs not saturated. Lz4 compression doesn t side affect DB performance If zlib compression was to much CPU overhead Try LZ4 - better than LZO/QuickLZ and many others NOT IF Database is already compressed or blob data is compressed Can t compress what is already compressed

49 LZMA new option for compression at all costs IF Really want to save on space Running off hours or cold backup Have strong CPUs Have horribly Slow IO NOT IF You want the backup to run faster Typically time is slower You have a busy database LZMA is high overhead and will saturate CPUs If zlib was to much overhead previously, LZMA is more Instead try LZ4

50 Test these new Options Recommendation Use previous guidance BUT Trying various options out Your results may vary Its why we have options Pick what works best for you Let us know what worked best

51 New Encryption option Secure your sensitive backup data with encryption Note only supported for image backup --backup-image Works very well with compression Compress then encrypt you can t compress encrypted files. Uses symmetric Encryption AES Advanced Encryption Standard 256 bit Using very latest OpenSSL Library Data is encrypted prior to hitting disk or other storage

52 New - Streaming Restore of Partial Backup Single step restore of partial backup taken with use-tts Can take selective backups of Innodb tables And now hot copy it to another running server seamlessly Backup Zero storage Restore No Big Write IO No Big Staging InnoDB Table (File per Table) MEB SSH - Streamed Image Can Compress or Encrypt MEB InnoDB Table (File per Table) Nice for very efficiently copying especially large tables Works for FULL backup since 3.9

53 References MySQL Enterprise Backup: Product Information MySQL Enterprise Backup Team Blog MySQL Enterprise Backup: Documentation dev.mysql.com/doc/mysql-enterprise-backup/4.1/en/index.html How Do I get it Support and patches (My Oracle Support = MOS)

54 Thanks for Attending You ll get the slides Please don t hesitate to send us ideas, feedback, doc requests, blog requests, etc. Directly through support Blog Comments mike.frank@oracle.com priya.jayakumar@oracle.com