Monday, September 15, 14

Size: px

Start display at page:

Download "Monday, September 15, 14"

Brian Goodman
6 years ago
Views:

2 MySQL Server Performance Tuning 101 Ligaya Turmelle Principle Technical Support Engineer - ligaya.turmelle@oracle.com We will be covering some of the most basic and common areas you should consider. Who is on 5.6? 5.5? 5.1? 5.0? lower?

THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY, AND MAY NOT BE INCORPORATED INTO ANY CONTRACT.

3 THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY, AND MAY NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY, AND SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISIONS. THE DEVELOPMENT, RELEASE, AND TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED FOR ORACLE'S PRODUCTS REMAINS AT THE SOLE DISCRETION OF ORACLE. 3 Copyright 2013, Oracle and/or its affiliates. All rights reserved. safe harbor - basically if I tell you about any upcoming products it is for informational purposes only. Oracle makes no guarantee.

4 MySQL in a Nutshell Worlds most popular open source database M of LAMP Main Site: mysql.com Developer Zone: dev.mysql.com OR mysql.org Downloads and Labs Manuals and Bugs Forums and Worklogs 4 Copyright 2013, Oracle and/or its affiliates. All rights reserved.

6 Step 0 General No easy answers Benchmark and test Under allocate 6 Copyright 2013, Oracle and/or its affiliates. All rights reserved. This is a soft skill with no absolute answers Change 1 thing at a time Under allocate rather then over allocate or you swap There is such a thing as over-tuning the system

Step 0 The Server OS Network Filesystem 7 Copyright 2013, Oracle

Step 0 The MySQL Server Optimize the queries Database Schema 8 Copyright 2013, Oracle

9 MySQL and Memory Global Server Start Large Values Allocated Once Per Connection As Needed Small Values Allocated 0-N Times Global Memory + (Max_Connections * Session Buffers) 9 Copyright 2013, Oracle and/or its affiliates. All rights reserved. To tune the server you have to understand how MySQL uses memory.

10 Current Settings mysql> SHOW GLOBAL VARIABLES; 10 Copyright 2013, Oracle and/or its affiliates. All rights reserved. mysql> SHOW GLOBAL VARIABLES; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 autocommit ON automatic_sp_privileges ON back_log 80 basedir /usr big_tables OFF bind_address * binlog_cache_size binlog_checksum CRC32 binlog_direct_non_transactional_updates OFF binlog_format STATEMENT binlog_max_flush_queue_time 0 binlog_order_commits ON binlog_row_image FULL binlog_rows_query_log_events OFF binlog_stmt_cache_size What are the current settings for the server?

11 Current Settings my.cnf / my.ini [client] socket = /var/run/mysqld/mysql.sock [mysqld] # These settings are for this specific box. server-id = 1 log-bin = /var/lib/mysql/binlogs/xxxxxxxx-bin pid-file = /var/run/mysqld/mysql.pid socket = /var/run/mysqld/mysql.sock performance_schema max_connections = 200 # The ft_min_word_len is set to 3 instead of the default # 3 letter acronyms in the tables with full text ft_min_word_len = 3 11 Copyright 2013, Oracle and/or its affiliates. All rights reserved. everything else

12 Anything Else? RAM? Dedicated? 32 or 64 bit OS? 32 or 64 bit MySQL? Workload? Storage Engines? 12 Copyright 2013, Oracle and/or its affiliates. All rights reserved. 32 bit addressing limit read/write heavy? short or report queries Mostly InnoDB? MyISAM? Memory? Etc.

13 Current Status mysql> SHOW GLOBAL STATUS; mysql> show global status; Variable_name Value Aborted_clients 9363 Aborted_connects Binlog_cache_disk_use Binlog_cache_use Bytes_received Bytes_sent Com_admin_commands Com_assign_to_keycache 0 Com_alter_db 0 Com_alter_db_upgrade 0 Com_alter_event 0 Com_alter_function 0 Com_alter_procedure 0 Com_alter_server 0 Com_alter_table 37 Com_alter_tablespace 0 Com_analyze 0 13 Copyright 2013, Oracle and/or its affiliates. All rights reserved. Counters - what is done, not what we think is done. Issued twice to find the Delta. Ok during normal operations but best during peak times.

19 COM_* Counters Each command Used to calculate the Delta 19 Copyright 2013, Oracle and/or its affiliates. All rights reserved. What you are actually doing. Com_select Com_set_option Com_signal 0 Com_show_authors 0 Com_show_binlog_events 0 Com_show_binlogs Com_show_charsets 1 Com_show_collations 39 Com_show_contributors 0 Com_show_create_db 3 Com_show_create_event 0 Com_show_create_func 1120 Com_show_create_proc Com_show_create_table Com_show_create_trigger 0 Com_show_databases 435 Com_show_engine_logs 0 Com_show_engine_mutex 0 Com_show_engine_status

20 Et Al. Connections Queries Questions Slow_queries Sort_merge_passes 20 Copyright 2013, Oracle and/or its affiliates. All rights reserved. Connections - # of attempts to connect - successful or not Queries - # of statements executed - including stored procedures Questions - # of statements sent to the server by clients and executed Slow_queries - # of queries that took longer then long_query_time sec to run Sort_merge_passes - # of merge passes that sort algorithm had to do

22 How is InnoDB doing? Space Innodb_buffer_pool_pages_data Innodb_buffer_pool_pages_total Innodb_buffer_pool_pages_free 22 Copyright 2013, Oracle and/or its affiliates. All rights reserved. - Pages data number of pages containing data (clean and dirty) - Pages total total size in pages (innodb_page_size compiled in, default is 16KB) - Pages_free some are fine but if you see a lot for a while, you over allocated - Dirty page = a page that has changes in the buffer that has been saved to the log files

23 How is InnoDB doing? Efficiency Innodb_buffer_pool_read_requests Innodb_buffer_pool_reads Formula: 1 - (Innodb_buffer_pool_read / Innodb_buffer_pool_read_requests) = buffer pool hit ratio 23 Copyright 2013, Oracle and/or its affiliates. All rights reserved. reads - reads from disk read_requests - disk or memory

24 innodb_buffer_pool_size Global Caches data and indexes Larger values reduces IO Self-contained 80% max 24 Copyright 2013, Oracle and/or its affiliates. All rights reserved. - Size in bytes - InnoDB uses clustered indexes remember - Larger the value the more it acts like an in-memory database. - Handles and buffers *everything* for itself - Setting it too high can lead to swap === bad

25 innodb_log_file_size Size of file on disk Larger the file, the less checkpoint activity Size for each log file Max combined log file size 4G <= = 512GB Larger the log file longer recover time 25 Copyright 2013, Oracle and/or its affiliates. All rights reserved. - Less checkpoint activity means less IO - Typically there are 2 log files in a log group (innodb_log_files_in_group) - crash recovery code was optimized in 5.5. So the recovery time will be significantly lower in 5.5 then in Start test using the full 4G and modify as needed for your requirements and limitations.

26 innodb_log_buffer_size Buffer for writing to log files on disk Large transactions? Yes - increasing may help with IO Default is 8MB 26 Copyright 2013, Oracle and/or its affiliates. All rights reserved. - If you have large transactions, try to have them fit in here. This reduces the need to write the log to disk before the transactions commit.

27 innodb_file_per_table.ibd files Default: Off <= > On Easier to reclaim space TRUNCATE for a table is faster Can monitor table size at the file system Can store specific tables on different storage devices (5.6) Transportable tablespaces 27 Copyright 2013, Oracle and/or its affiliates. All rights reserved. - for each newly created table the data and indexes will be placed in a separate ibd file rather then in the shared tablespace - reclaiming space from the shared tablespace is not trivial. - reasons for separate devices: IO Optimization, space management, backup purposes - Transportable tablespaces allow the moving of the.ibd files from 1 database server to another. Manual has limitations and usage notes as well as a page covering how to do it.

29 How is MyISAM doing? Space Key_blocks_unused Formula: key_buffer_size - (Key_blocks_unused * key_cache_block_size) = amount actually in use Key_blocks_used High water mark 29 Copyright 2013, Oracle and/or its affiliates. All rights reserved. - Block size can be found in key_cache_block_size - Key_blocks_used indicates the max number of blocks that have ever been in use at one time.

30 How is MyISAM doing? Efficiency Key_read_requests Key_reads Formula: Key_reads/Key_read_requests = key cache miss rate Key_write_requests Key_writes 30 Copyright 2013, Oracle and/or its affiliates. All rights reserved. - Key_read_requests requests to read a key block - Key_reads number of physical reads from disk. Large value may mean your key_buffer_size is too small * Key cache miss rate the lower the value, the better - Key_write_requests # of requests to write a key block - Key_writes - # of physical writes of a key block to disk

31 How is MyISAM doing? Locking Table_locks_immediate Table_locks_waited 31 Copyright 2013, Oracle and/or its affiliates. All rights reserved. - MyISAM uses table level locking. That means that every time you have to change it, the whole table will be locked for the duration. (Yes - concurrent inserts but that only works under specific circumstances (If a MyISAM table has no holes in the data file (deleted rows in the middle), an INSERT statement can be executed to add rows to the end of the table at the same time that SELECT statements are reading rows from the table.)) - Want to watch Table_locks_waited since this iterates every time you had to wait to get a table lock. If you have high values for this, and you do not use the MyISAM specific features, you may want to consider changing to another storage engine for better concurrency.

32 key_buffer_size AKA key cache Global Index blocks only 25% - maybe 32 Copyright 2013, Oracle and/or its affiliates. All rights reserved. - The index blocks are buffered and shared by all threads. - MyISAM uses the OS file caching system to hold the data files. This means that we have to leave space for the OS file system otherwise we can start swapping. And swapping can be bad with MySQL. - For this reason, on a dedicated machine heavy on MyISAM, 25% of RAM is a place to start. Notes: - The maximum size is 4G for each key cache. - You can have more then one key cache see the manual for more information on that.

33 QUERY CACHE 33 Copyright 2013, Oracle and/or its affiliates. All rights reserved. Ask if people understand how the query cache works. If they do not understand - explain it. Also explain that this can become a bottle neck for performance with high concurrency since it is single threaded.

34 How is the QC doing? Qcache_total_blocks Qcache_free_blocks Not what you think Qcache_lowmem_prunes 34 Copyright 2013, Oracle and/or its affiliates. All rights reserved. - Free_blocks think this would be a good thing, but isn t. The higher the value, the greater the fragmentation in the query cache. * As Qcache_free_blocks approaches Qcache_total_blocks/2 the more severe the fragmentation * To defragment the query cache use the FLUSH QUERY CACHE statement - lowmem prunes number of queries deleted from the QC because of low memory

36 QC Data into Info Hit rate Higher the better Formula: Qcache_hits / (Qcache_hits + Com_select) = Query Cache Hit Rate Invalidating queries Bigger the diff - the better Comparison: Qcache_inserts << Com_select 36 Copyright 2013, Oracle and/or its affiliates. All rights reserved. Keep in mind warming up the cache

37 - Minor side note - (5.1) Query cache mutex still acquired even with query cache size set to 0 query_cache_size Global Allocated all at once Default is disabled Default size 0 < => 1M 37 Copyright 2013, Oracle and/or its affiliates. All rights reserved. - The default of 0 disables the query cache. Generally speaking it is recommended that you start with the query cache off, and only if you think it can help you, do you turn it on. Be sure to check the performance of the system when you turn it on to verify that it does increase. There is potential for substantial performance improvement, but do not assume that it will happen. With some query cache configurations or server workloads, it is possible to see a performance decrease. Before default size was 0 - now 1M with query_cache_type=0.

38 query_cache_type Global - but can be session 3 options 0 Off 1 On 2 On Demand Default: 1 < => 0 38 Copyright 2013, Oracle and/or its affiliates. All rights reserved. 0 - Do not cache results or pull results from the QC 1 - Cache all you can unless told not to with SQL_NO_CACHE 2 - Cache only those queries that have SQL_CACHE Turning it Off (0) - now saves a significant amount of overhead since it will no longer try to acquire the query cache mutex at all. However that means you can not turn on the query cache at runtime (dynamically).

Gotcha! 39 Copyright 2013, Oracle and/or its affiliates. All rights reserved.

41 Threads Threads == Connection Thread Cache Threads_cached Threads_connected Threads_created Formula: Threads_created /Connections = Thread cache miss rate 41 Copyright 2013, Oracle and/or its affiliates. All rights reserved. This is more for those using an older system. - In regards to this discussion that is. - Thread cache * threads can be expensive to create and destroy for every connection. - explain how the thread cache works. (starts out empty and as new connections are made it checks the cache to see if one is there. If so it uses it. If not it makes a new thread. When done it puts the threads back into the thread cache until it has its max value. Watch Threads_created if large consider increasing the thread_cache

42 thread_cache_size Global but grows as needed Number to be cached until needed May increase performance with lots of new connections or may not Formula: ((Threads_created/Connections) * 100) = Thread cache efficiency 42 Copyright 2013, Oracle and/or its affiliates. All rights reserved. Because of modern thread implementations be sure to test to see if you get any improvement.

43 SESSION LEVEL 43 Copyright 2013, Oracle and/or its affiliates. All rights reserved. - These values should in the global scope be set to a small value since they are used for each session. Only if you have a session that is doing something out of the ordinary should you consider dynamically changing these values. Keep in mind that each query can potentially have 1 N instances of a session level buffer as well (think multiple temp tables), so if you increase these values for a specific session you have to be sure the space is available.

44 Temporary Tables max_heap_table_size Dynamic Sets maximum size of all MEMORY tables tmp_table_size Dynamic In-memory - MEMORY table Max size of internal in-memory temp tables Becomes on-disk MyISAM table 44 Copyright 2013, Oracle and/or its affiliates. All rights reserved. - Other reasons temp tables can automatically go to disk: * BLOB * column in a GROUP BY or DISTINCT clause larger than 512 bytes * any column larger than 512 bytes in the SELECT list, if UNION or UNION ALL is used

45 sort_buffer_size Dynamic Doing a sort - allocates a buffer this size Helps with ORDER BY and/or GROUP BY Watch Sort_merge_passes 45 Copyright 2013, Oracle and/or its affiliates. All rights reserved. - Whole buffer is allocated even if it is not used prior to 5.6.4

46 bulk_insert_buffer_size Dynamic Cache to assist with MyISAM bulk inserts Set to 0 to disable 46 Copyright 2013, Oracle and/or its affiliates. All rights reserved. - Bulk inserts == INSERT SELECT, INSERT VALUES ( ), ( ), ( ), LOAD DATA INFILE when adding data to non-empty tables.

47 join_buffer_size Use with care Used for plain index scan range index scan joins that do not use indexes - full table scan One buffer for EACH full join between tables 47 Copyright 2013, Oracle and/or its affiliates. All rights reserved. - I am always hesitant to show this slide. I do not want people to think this can be used to help poorly optimized queries. Should be used as a band aid or if adding the appropriate indexes is not possible. Make it very large - and you may have to wait for the the allocation of the memory

49 MySQL Server Performance Tuning 101 Ligaya Turmelle Principle Technical Support Engineer - ligaya.turmelle@oracle.com

MySQL Performance Tuning 101

MySQL Performance Tuning 101 Ligaya Turmelle MySQL Support Engineer ligaya@mysql.com 1 1 MySQL world's most popular open source database software a key part of LAMP (Linux, Apache, MySQL, PHP / Perl /