MySQL 5.1 Configuration Files & Options Part II. Presented by: Sheeri K. Cabral

Transcription

1 MySQL 5.1 Configuration Files & Options Part II Presented by: Sheeri K. Cabral

2 A bout P ythia n Recognized Leader: Global industry-leader in database infrastructure services for Oracle, Oracle Applications, MySQL and SQL Server 150 current multinational companies such as Forbes.com, Fox Sports and Western Union to help manage their complex IT deployments Expertise: One of the world s largest concentrations of dedicated, full-time DBA expertise. Global Reach & Scalability: 2 24/7/365 global remote support for DBA and consulting, systems administration, special projects or emergency response 2009/2010 Pythian

3 INSERT DELAYED max_delayed_threads (20) 3 if 0, INSERT DELAYED is disabled Delayed_insert_threads (0) Delayed_writes (0), Delayed_errors (0) Not_flushed_delayed_rows (0)

4 INSERT DELAYED 4 delayed_insert_limit (100) delayed_insert_timeout (300) delayed_queue_size (1000) max_insert_delayed_threads (20) max_delayed_threads Delayed_insert_threads

5 Timeouts back_log (50) net_read_timeout (30) 5 net_retry_count (10) net_write_timeout (60)

6 More Timeouts 6 connect_timeout (10) wait_timeout (28800) interactive_timeout (28800)

7 Non-InnoDB Disk I/O 7 flush (OFF) Sync changes to disk after every SQL statement If ON, write changes to disk; let OS handle sync flush_time (0) Close tables and sync data to disk every x seconds 0 = disabled Enable for systems with very few resources

8 OS-Related Variables 8 skip_external_locking (ON) thread_handling (one-thread-per-connection) Open_streams (0)

9 Linux Parameters large_files_support (ON) large_pages (OFF) Use pages >4Kb large_page_size 9 Use files >2Gb Size of memory pages if large pages are used

10 File Descriptors table_open_cache (8,128) 10 Independent of innodb_open_files Open_tables (224) Opened_tables (519)

11 File Descriptor Usage 11 Open_files (152) Opened_files (974,785) open_files_limit (25,000)

12 TCP/Networking skip_networking (OFF) max_allowed_packet (24M) 12 net_buffer_length

13 Locking Status Variables 13 Table_locks_immediate (52,016,680) Table_locks_waited (4) Com_lock_tables (0) Com_unlock_tables (400)

14 Locking System Variables sql_buffer_result (OFF) low_priority_updates (OFF) 14 sql_low_priority_updates max_write_lock_count (4,294,967,295)

15 Storage Engines 15 have_csv (YES) have_innodb (YES) have_ndbcluster (NO)

16 MyISAM-specific 16 concurrent_insert (2) delay_key_write (ON) keep_files_on_create (OFF)

17 MyISAM-specific 17 myisam_data_pointer_size (6) myisam_stats_method (nulls_unequal ) myisam_use_mmap (OFF)

18 Repairing MyISAM Tables myisam_recover_options (FORCE,BACKUP) myisam_repair_threads (1) 18 multi-threaded repair by sorting is beta-quality

19 MyISAM Index Sorting 19 Done with REPAIR TABLE, ALTER TABLE, LOAD DATA INFILE, CREATE INDEX myisam_sort_buffer_size (56M) myisam_max_sort_file_size (10G)

20 FULLTEXT search 20 MyISAM only ft_min_word_len (4) ft_max_word_len (84) ft_stopword_file (built-in)

21 FULLTEXT search 21 ft_boolean_syntax (+ -><()~*:""& ) ft_query_expansion_limit (20)

22 MyISAM Key Cache Can have more than one 22 CACHE INDEX IN... key_buffer_size (16M) key_cache_block_size (1024)

23 MyISAM Key Cache 23 Uses LRU; hot/warm sub-chains key_cache_age_threshold (300) key_cache_division_limit (100)

24 MyISAM Key Cache Sizing Percentage used for all key caches: 24 1 (13360 * 1024)/16*1024*1024 = bytes Key_blocks_used(36) 1 (Key_blocks_unused * key_cache_block_size) / key_buffer_size) max used at any time Key_blocks_not_flushed (0)

25 MyISAM Key Cache Efficiency 25 Cache miss % Key_reads (from disk) / Key_read_requests * / *100 = 0.01 % Key_write_requests (10) Key_writes (10)

26 Transaction Basics 26 autocommit (ON) tx_isolation (READ-COMMITTED) completion-type (0)

27 Transaction Basics foreign_key_checks (ON, session only) transaction_prealloc_size (4 Kb) transaction_alloc_block_size (8 Kb) 27 Starting/ending size of a shared per-transaction memory pool if a per-transaction memory pool needs more memory, increase by this much

28 InnoDB Basics 28 innodb_data_file_path (ibdata_primary:128m:autoextend) innodb_autoextend_increment (8) innodb_data_home_dir ()

29 InnoDB Flags 29 ignore_builtin_innodb (OFF) innodb_flush_log_at_trx_commit (1) innodb_fast_shutdown (1)

30 InnoDB Flags 30 innodb_file_per_table (ON) innodb_locks_unsafe_for_binlog (OFF) innodb_stats_on_metadata (ON) innodb_use_legacy_cardinality_algorithm (ON)

31 InnoDB Flags 31 innodb_adaptive_hash_index (ON) innodb_checksums (ON) innodb_autoinc_lock_mode (1)

32 InnoDB Flags 32 innodb_force_recovery (0) innodb_table_locks (ON) innodb_support_xa (ON)

33 InnoDB Flags innodb_lock_wait_timeout (240) 33 innodb_rollback_on_timeout (OFF) timed_mutexes (OFF) unique_checks (ON)

34 InnoDB Doublewrite 34 innodb_doublewrite (ON) Innodb_dblwr_writes (592,341) Innodb_dblwr_pages_written (4,537,224)

35 InnoDB Concurrency Variables 35 innodb_thread_concurrency (16) innodb_commit_concurrency (0) innodb_concurrency_tickets (500) innodb_sync_spin_loops (20)

36 InnoDB Buffer Pool innodb_buffer_pool_size (8G) 36 innodb data/index cache Innodb_page_size (16 Kb) Innodb_buffer_pool_pages_total (524,288) Innodb_buffer_pool_pages_data (46,905) Innodb_buffer_pool_pages_free (477177) Innodb_buffer_pool_pages_misc (206)

37 Dirty Pages innodb_max_dirty_pages_pct (90) Innodb_buffer_pool_wait_free (0) Innodb_buffer_pool_pages_dirty (9) 37 after this % is reached, wait while dirty pages are flushed to disk Included in the Innodb_buffer_pool_pages_data

38 Flush / Sync Innodb_buffer_pool_pages_flushed (4,537,224) Innodb_data_fsyncs (10,201,211) 38 counter of # times flushed Innodb_data_pending_fsyncs (0)

39 InnoDB Row Locks Innodb_row_lock_waits (269) 39 Innodb_row_lock_current_waits (0) Innodb_row_lock_time (31,637 ms) Innodb_row_lock_time_avg (117 ms) Innodb_row_lock_time_max (4,599 ms)

40 InnoDB Reads Innodb_buffer_pool_read_requests (1,360,965,036) Innodb_buffer_pool_reads (27,482) Innodb_data_reads (28,754) 40 reads buffer pool could not do & single page read Innodb_data_pending_reads (0)

41 InnoDB Reads 41 Innodb_data_read ( Mb) Innodb_pages_read (44,377) Innodb_rows_read (2,721,225,514)

42 InnoDB Writes Innodb_buffer_pool_write_requests (47,307,349) Innodb_data_writes (12,468,811) 42 Innodb_data_pending_writes (0)

43 InnoDB Writes 43 Innodb_data_written (153 Gb) Innodb_pages_written (4,537,224) Innodb_pages_created (3,664)

44 InnoDB Rows Written 44 Innodb_rows_inserted (32,681) Innodb_rows_deleted (1,555,539) Innodb_rows_updated (3,908,062)

45 InnoDB Log Basics innodb_log_file_size (1G) innodb_log_files_in_group (3) 45 innodb_mirrored_log_groups (1) innodb_log_group_home_dir (./)

46 InnoDB Logs innodb_log_buffer_size (8M) Innodb_os_log_fsyncs (8,464,144) 46 Innodb_log_waits (0) Innodb_os_log_pending_fsyncs (0)

47 InnoDB Log Writes Innodb_log_write_requests (26,175,407) Innodb_log_writes (8,166,171) 47 Innodb_os_log_pending_writes (0) Innodb_os_log_written (14.42 Gb)

48 Other InnoDB Variables 48 innodb_additional_mem_pool_size (16Mb) innodb_open_files (300) innodb_thread_sleep_delay (10,000 =.01 sec)

49 Advanced InnoDB Variables 49 innodb_flush_method () innodb_max_purge_lag (0)

50 InnoDB Status for Performance 50 Innodb_buffer_pool_read_ahead_rnd (16) Innodb_buffer_pool_read_ahead_seq (286)

51 Percona patch extensions XtraDB Percona build of MySQL 51 (5.x.xx-percona-log) Standard and InnoDB Plugin Slides are current up to percona-log

52 I/O Extensions 52 innodb_read_io_threads innodb_write_io_threads innodb_io_capacity innodb_use_purge_thread

53 More I/O extensions 53 innodb_ibuf_accel_rate innodb_ibuf_active_contract innodb_ibuf_max_size

54 More I/O extensions 54 innodb_enable_unsafe_group_commit innodb_flush_neighbor_pages innodb_read_ahead none random linear both

55 Checkpointing 55 innodb_adaptive_checkpoint innodb_checkpoint_age_target

56 SHOW ENGINE INNODB STATUS BUFFER POOL AND MEMORY Total memory allocated ; in additional pool allocated Internal hash tables (constant factor + variable factor) + Adaptive hash index ( ) + Page hash Dictionary cache ( ) + File system ( ) + Lock system ( ) + Recovery system 0 (0 + 0) + Threads ( ) Buffer pool size Buffer pool size, bytes Free buffers Database pages 3963 Modified db pages 1510 Pending reads 0 56

57 Statistics/Info 57 innodb_show_locks_held innodb_show_verbose_locks innodb_stats_auto_update innodb_stats_method innodb_stats_update_need_locks

58 New in INFORMATION_SCHEMA 58 innodb_table_stats innodb_index_stats

59 Recovery 59 innodb_fast_recovery innodb_recovery_stats

60 Recovery Stats RECOVERY STATISTICS Recovery time: 18 sec. (1 turns) Data page IO statistics Requested pages: 9126 Read pages: 9126 Written pages: 7957 (Dirty blocks): 1156 Grouping IO [times]: number of pages, read request neighbors (in 32 pages chunk), combined read IO, combined write IO 1, 32, 335, 548 2, 0, 121, 97 3, 7, 49, 44 4, 4, 43, , 0, 2, 25 60

61 Recovery Stats Recovery process statistics Checked pages by doublewrite buffer: 128 Overwritten pages from doublewrite: 0 Recovered pages by io_thread: 9145 Recovered pages by main thread: 0 Parsed log records to apply: Sum of the length: Applied log records: Sum of the length: Pages which are already new enough: 93 Oldest page's LSN: Newest page's LSN:

62 Other Extensions innodb_thread_concurrency_timer_based innodb_relax_table_creation innodb_extra_rsegments 62 INFORMATION_SCHEMA.innodb_rseg

63 Other Extensions innodb_dict_size_limit 63 Innodb_dict_tables innodb_expand_import innodb_extra_undoslots innodb_overwrite_relay_log_info

64 T ha nk Y ou. Win a signed copy of Sheeri s book. Leave your business card and you could win a book. We ll invite you to read our blog posts, follow us on twitter, and join our next webinars. Drawing will be immediately after the talk once all cards are collected. 2009/2010 Pythian

65 T ha nk Y ou Questions, Comments, Feedback? Sheeri Cabral cabral@pythian.com Blog: Ask me about saving 15% on our Adoption Accelerator for MySQL while at MySQL Conference 2010! /2010 Pythian

66 u fg o C 1 5 L Q S w lt.c ik y:sh b td sn e rtip a P p O s& F

67 A bout P ythia n Recognized Leader: Global industry-leader in database infrastructure services for Oracle, Oracle Applications, MySQL and SQL Server 150 current multinational companies such as Forbes.com, Fox Sports and Western Union to help manage their complex IT deployments Expertise: One of the world s largest concentrations of dedicated, full-time DBA expertise. Global Reach & Scalability: 2 24/7/365 global remote support for DBA and consulting, systems administration, special projects or emergency response 2009/2010 Pythian - S uc c e s s ful g row ing bus ines s for more than 10 years - Served many customers with complex requirements/infrastructure just like yours. - Operate globally for 24 x 7 always awake services 2

68 INSERT DELAYED max_delayed_threads (20) if 0, INSERT DELAYED is disabled Delayed_insert_threads (0) Delayed_writes (0), Delayed_errors (0) Not_flushed_delayed_rows (0) 3 3 DELAYED keyword is ignored on InnoDB; basically just a way to get around table-level locking on MyISAM tables. This is not durable ( D in ACID compliant)

69 INSERT DELAYED delayed_insert_limit (100) delayed_insert_timeout (300) delayed_queue_size (1000) max_insert_delayed_threads (20) max_delayed_threads Delayed_insert_threads 4 4 delayed_insert_limit: When doing a bulk insert from INSERT DELAYED, process any SELECTs after this many records have been inserted delayed_insert_timeout: How long the handler waits for another INSERT DELAYED before terminating delayed_queue_size: Per-table queue size for INSERT DELAYED. If this fills, an INSERT DELAYED statement will hang max_delayed_threads: Max # of threads to handle INSERT DELAYED. If all threads are in use, further INSERT DELAYED statements ignore DELAYED. Thus, 0 disables DELAYED. max_insert_delayed_threads same as max_delayed_threads Delayed_insert_threads: How many handler threads are in use

70 Timeouts back_log (50) net_read_timeout (30) net_retry_count (10) net_write_timeout (60) 5 5 back_log=size of TCP listen queue, how many outstanding requests MySQL can have before it stops answering new requests net_read_timeout, net_write_timeout only for TCP connections net_retry_count is for interrupted read connections.

71 More Timeouts connect_timeout (10) wait_timeout (28800) interactive_timeout (28800) 6 6 Increasing the connect_timeout value might help if clients frequently encounter errors of the form Lost connection to MySQL server at 'XXX', system error: errno. If this is too low, Aborted_connects status variable will be higher (but is not the only reason) wait_timeout is how long to wait before killing sleeping non-interactive timeouts.

72 Non-InnoDB Disk I/O flush (OFF) Sync changes to disk after every SQL statement If ON, write changes to disk; let OS handle sync flush_time (0) Close tables and sync data to disk every x seconds 0 = disabled Enable for systems with very few resources 7 7

73 OS-Related Variables skip_external_locking (ON) thread_handling (one-thread-per-connection) Open_streams (0) 8 8 skip_external_locking: If 1, Do not use system locking; see turn to 0 if you need to lock tables externally, as for myisamchk. Note that if you use this option on a system on which lockd does not fully work (such as Linux), it is easy for mysqld to deadlock. thread_handling: valid values: one-thread, one-thread-per-connection (useful for debugging linux, also the default)

74 Linux Parameters large_files_support (ON) large_pages (OFF) Use files >2Gb Use pages >4Kb large_page_size Size of memory pages if large pages are used 9 9

75 File Descriptors table_open_cache (8,128) Independent of innodb_open_files Open_tables (224) Opened_tables (519) if Opened_tables increases rapidly, the table_open_cache should be increased (if possible).

76 File Descriptor Usage Open_files (152) Opened_files (974,785) open_files_limit (25,000) open_files_limit can be set, but the server-reported value is the # of files the OS allows MySQL to open. If 0,. MySQL can't set this value. Not really anything to optimize here, just knowledge of how many files have used a certain library call (my_open) to open a file. Open_files shouldn't be close to open_files_limit, that isn't a problem here.

77 TCP/Networking skip_networking (OFF) max_allowed_packet (24M) net_buffer_length net_buffer_length = Per-client start buffer length, max_allowed_packet is max, shrinks down to net_buffer_length after each query

78 Locking Status Variables Table_locks_immediate (52,016,680) Table_locks_waited (4) Com_lock_tables (0) Com_unlock_tables (400) Table_locks_immediate # times a table lock request was granted immediately Table_locks_waited # times a table lock request had to wait (big # is a problem) Com_lock_tables Com_unlock_tables

79 Locking System Variables sql_buffer_result (OFF) low_priority_updates (OFF) sql_low_priority_updates max_write_lock_count (4,294,967,295) sql_buffer_result (OFF) If ON, forces results to be stored in temp tables, releases locks earlier than if not enabled. Session variable only. Useful for, say, if it takes a long time for a result to be sent to a client. low_priority_updates (OFF) If on, writes are lower priority than reads for table-level locking tables. ie, MyISAM, not InnoDB. called sql_low_priority_updates before, both show up in SHOW GLOBAL VARIABLES currently. max_write_lock_count (4,294,967,295) After this many write locks, allow a read lock. Basically by default writes are higher priority than reads, but if you want x writes to happen, then allow a read, set this variable.

80 Storage Engines have_csv (YES) have_innodb (YES) have_ndbcluster (NO) That's all, nothing for MyISAM, federated, archive, etc. CSV is used in system tables (general log, slow log) by default.

81 MyISAM-specific concurrent_insert (2) delay_key_write (ON) keep_files_on_create (OFF) ALSO - (sql_)low_priority_updates from slide 100 concurrent_insert. 1=default, concurrent_insert if no fragmentation. 2, concurrent_insert even with fragmentation. 0 = no concurrent insert. delay_key_write ON=default, key buffer not flushed for tables created with DELAY_KEY_WRITE keyword. Other values = OFF and ALL, which treats all MyISAM tables as having DELAY_KEY_WRITE on. keep_files_on_create = overwrite existing MYI and MYD files when a.frm file isn't present. Set to on or always explicitly use INDEX_DIRECTORY and DATA_DIRECTORY to not overwrite MYI and MYD files. If a.frm file is present, mysqld returns an error that the table exists.

82 MyISAM-specific myisam_data_pointer_size (6) myisam_stats_method (nulls_unequal ) myisam_use_mmap (OFF) myisam_data_pointer_size = The default pointer size in bytes, to be used by CREATE TABLE for MyISAM tables when no MAX_ROWS option is specified. This variable cannot be less than 2 or larger than 7. The default value is 6. A value of 4 allows tables up to 4GB; a value of 6 allows tables up to 256TB. If you get Table is full you may need to increase this, or set MAX_ROWS when doing CREATE TABLE. myisam_stats_method = How the server treats NULL values when collecting stats about distribution of index values for MyISAM tables. Values are: nulls_equal all NULL index values are considered equal and form a single value group that has a size equal to the number of NULL values nulls_unequal NULL values are considered unequal, each NULL forms a distinct value group of size 1. nulls_ignored, NULL values are ignored. myisam_use_mmap = added in 5.1.4, use memory mapping for reading/writing MyISAM tables.

83 Repairing MyISAM Tables myisam_recover_options (FORCE,BACKUP) myisam_repair_threads (1) multi-threaded repair by sorting is beta-quality recover_options=recovery mode. Values = any combination of the values of DEFAULT, BACKUP, FORCE, or QUICK, separated by commas. Specifying with no argument is the same as specifying DEFAULT, and specifying with an explicit value of "" disables recovery (same as not giving the option). If recovery is enabled, each time mysqld opens a MyISAM table, it checks whether the table is marked as crashed or wasn't closed properly. If this is the case, mysqld runs a check on the table. If the table was corrupted, mysqld attempts repair. The following options affect how the repair works: BACKUP If the data file was changed during recovery, save a backup of the tbl_name.myd file as tbl_name-datetime.bak. FORCE Run recovery even if we would lose more than one row from the.myd file. QUICK = Don't check the rows in the table if there aren't any delete blocks. DEFAULT = Recovery w/out backup, forcing, quick checking. Before the server automatically repairs a table, it writes a note about the repair to the error log. If you want to be able to recover from most problems without user intervention, you should use the options BACKUP,FORCE. This forces a repair of a table even if some rows would be deleted, but it keeps the old data file as a backup so that you can later examine what happened.

84 MyISAM Index Sorting Done with REPAIR TABLE, ALTER TABLE, LOAD DATA INFILE, CREATE INDEX myisam_sort_buffer_size (56M) myisam_max_sort_file_size (10G) myisam_sort_buffer_size = buffer size for sorting indexes during REPAIR TABLE or adding a new index to an existing table w/data. max is 4Gb before , 16,384 petabytes and later, assuming 64-bit. 64-bit windows limit is 4Gb no matter what version, larger values are truncated. myisam_max_sort_file_size = max temp file when recreating a MyISAM index during repair, ALTER TABLE or LOAD DATA INFILE. If bigger than this, index is created using key_cache instead, which takes longer.

85 FULLTEXT search MyISAM only ft_min_word_len (4) ft_max_word_len (84) ft_stopword_file (built-in) ft_stopword_file defined in the source code in the storage/myisam/ft_static.c, or specify a file. Comments (C style) aren't honored, but all words in the file are used. If you set this to '', there is no stopword filtering. ie, is, the, etc. common words.

86 FULLTEXT search ft_boolean_syntax (+ -><()~*:""& ) ft_query_expansion_limit (20) ft_boolean_syntax = list of operators supported by boolean full-text searches (using IN BOOLEAN MODE). See Section , Boolean Full-Text Searches The default variable value is '+ -><()~*:""& '. The rules for changing the value are as follows: Operator function is determined by position within the string. The replacement value must be 14 characters. Each character must be an ASCII nonalphanumeric character. Either the 1st or 2nd character must be a space. No duplicates are allowed except the phrase quoting operators in positions 11 and 12. These two characters are not required to be the same, but they are the only two that may be. Positions 10, 13, and 14 (which by default are set to :, &, and ) are reserved for future extensions. ft_query_expansion_limit = The number of top matches to use for full-text searches performed using WITH QUERY EXPANSION. It works by performing the search twice, where the search phrase for the 2nd search is the original search phrase concatenated with the few most highly relevant docs from the first search. Thus, if one of these docs contains the word database and the word MySQL, the second search finds the documents that contain the word MySQL even if they do not contain the word database. see

87 MyISAM Key Cache Can have more than one CACHE INDEX IN... key_buffer_size (16M) key_cache_block_size (1024) key_buffer_size, key_cache_block_sizecan be set per named cache using SET GLOBAL cachename.variable=value, ie SET GLOBAL session.key_buffer_size=10240;

88 MyISAM Key Cache Uses LRU; hot/warm sub-chains key_cache_age_threshold (300) key_cache_division_limit (100) key_cache_division_limit and key_cache_age_threshold can be set per named cache using SET GLOBAL cachename.variable=value. age_threshold = how fast something gets demoted from hot to warm sub-chain division_limit % of key cache to use in warm (vs. hot) subchain of the cache

89 MyISAM Key Cache Sizing Percentage used for all key caches: 1 (13360 * 1024)/16*1024*1024 = bytes Key_blocks_used(36) 1 (Key_blocks_unused * key_cache_block_size) / key_buffer_size) max used at any time Key_blocks_not_flushed (0) 24 24

90 MyISAM Key Cache Efficiency Cache miss % Key_reads (from disk) / Key_read_requests * / *100 = 0.01 % Key_write_requests (10) Key_writes (10) 25 25

91 Transaction Basics autocommit (ON) tx_isolation (READ-COMMITTED) completion-type (0) completion-type Best left alone default 0 = COMMIT if 1, COMMIT = COMMIT AND CHAIN (open new txn), same with ROLLBACK. If 2, COMMIT = COMMIT RELEASE (close session), same with ROLLBACK

92 Transaction Basics foreign_key_checks (ON, session only) transaction_prealloc_size (4 Kb) Starting/ending size of a shared per-transaction memory pool transaction_alloc_block_size (8 Kb) if a per-transaction memory pool needs more memory, increase by this much 27 27

93 InnoDB Basics innodb_data_file_path (ibdata_primary:128m:autoextend) innodb_autoextend_increment (8) innodb_data_home_dir () 28 28

94 InnoDB Flags ignore_builtin_innodb (OFF) innodb_flush_log_at_trx_commit (1) innodb_fast_shutdown (1) ignore_builtin_innodb as opposed to plugin innodb_flush_log_at_trx_commit 0 = log buffer written once per second and flushed 1 = log buffer written at each commit, flush to disk 2 = log buffer written at each commit, flush to disk once per second fast_shutdown 0 = full purge, insert buffer merge before shutdown. can take a long time. 1 = skips at shutdown 2 = flushes logs and shuts down cold, as if a crash. No commit txn will be lost, but crash recovery will be done on startup.

95 InnoDB Flags innodb_file_per_table (ON) innodb_locks_unsafe_for_binlog (OFF) innodb_stats_on_metadata (ON) innodb_use_legacy_cardinality_algorithm (ON) innodb_stats_on_metadata When on, as default and before , InnoDB updates statistics during metadata statements such as SHOW TABLE STATUS or SHOW INDEX, or when accessing the INFORMATION_SCHEMA tables TABLES or STATISTICS. When disabled, InnoDB does not updates statistics during these operations. Disabling this variable can improve access speed for schemas that have a large number of tables or indexes. innodb_use_legacy_cardinality_algorithm (5.1.35) With OFF, better randomization of index cardinality

96 InnoDB Flags innodb_adaptive_hash_index (ON) innodb_checksums (ON) innodb_autoinc_lock_mode (1) checksums are a safety feature. The manual says you may want to turn off for benchmarks. lock mode of 1 ( consecutive), ensures that, in the presence of INSERT statements where the number of rows is not known in advance (and where autoincrement numbers are assigned as the statement progresses), all auto-increment values assigned by any INSERT-like statement are consecutive, and operations are safe for statement-based replication. lock mode of 2 = interleaved, not good for statementbased replication lock mode of 0 = way it was before , only use for testing/backwards compatibility.

97 InnoDB Flags innodb_force_recovery (0) innodb_table_locks (ON) innodb_support_xa (ON) force recovery only set if you're actually trying to recover from a crash and the InnoDB table is corrupt. then, a value from 1-6.

98 InnoDB Flags innodb_lock_wait_timeout (240) innodb_rollback_on_timeout (OFF) timed_mutexes (OFF) unique_checks (ON) Innodb_lock_wait_timeout how long InnoDB will wait for a row-level lock. rollback on timeout if ON, entire txn is rolled back when there's a timeout. Otherwise, only the previous statement is rolled back. timed_mutexes whether or not SHOW ENGINE INNODB MUTEX shows time in ms or just shows 0. unique_checks session variable only. if set to 0, large table imports in InnoDB can be speeded up, but uniqueness violations may occur. If set to 1 (the default), uniqueness checks for secondary indexes in InnoDB tables are performed. Note that even if set to 0, a storage engine is still allowed to check for them and issue duplicate-key errors if it detects them.

99 InnoDB Doublewrite innodb_doublewrite (ON) Innodb_dblwr_writes (592,341) Innodb_dblwr_pages_written (4,537,224) Doublewrite means that before writing pages to a data file, InnoDB first writes them to a contiguous tablespace area called the doublewrite buffer. Only after the write and the flush to the doublewrite buffer has completed does InnoDB write the pages to their proper positions in the data file. If the operating system crashes in the middle of a page write, InnoDB can later find a good copy of the page from the doublewrite buffer during recovery.

100 InnoDB Concurrency Variables innodb_thread_concurrency (16) innodb_commit_concurrency (0) innodb_concurrency_tickets (500) innodb_sync_spin_loops (20) innodb_thread_concurrency Max value= disables the checking of thread concurrency. After x OS threads are in innodb, any more are put into a waiting queue. innodb_commit_concurrency 0 is for unlimited innodb_concurrency_tickets # times a thread can enter InnoDB without having to be queued if the threads exceed the value of innodb_thread_concurrency sync_spin_loops how many times to wait for a mutex to be freed before suspending the thread

101 InnoDB Buffer Pool innodb_buffer_pool_size (8G) innodb data/index cache Innodb_page_size (16 Kb) Innodb_buffer_pool_pages_total (524,288) Innodb_buffer_pool_pages_data (46,905) Innodb_buffer_pool_pages_free (477177) Innodb_buffer_pool_pages_misc (206) According to this, you are under-utilizing your InnoDB buffer pool. misc pages are pages used for admin overhead, ie row locks & adaptive hash

102 Dirty Pages innodb_max_dirty_pages_pct (90) after this % is reached, wait while dirty pages are flushed to disk Innodb_buffer_pool_wait_free (0) Innodb_buffer_pool_pages_dirty (9) Included in the Innodb_buffer_pool_pages_data wait free is a counter for the # of times the buffer pool had to wait for pages to be freed by cleanup

103 Flush / Sync Innodb_buffer_pool_pages_flushed (4,537,224) counter of # times flushed Innodb_data_fsyncs (10,201,211) Innodb_data_pending_fsyncs (0) if buffer_pool_pages_flushed is a lot, there's a lot of disk I/O. Writes may not be done in batches that correspond to pages.

104 InnoDB Row Locks Innodb_row_lock_waits (269) Innodb_row_lock_current_waits (0) Innodb_row_lock_time (31,637 ms) Innodb_row_lock_time_avg (117 ms) Innodb_row_lock_time_max (4,599 ms) 39 39

105 InnoDB Reads Innodb_buffer_pool_read_requests (1,360,965,036) Innodb_buffer_pool_reads (27,482) reads buffer pool could not do & single page read Innodb_data_reads (28,754) Innodb_data_pending_reads (0) 40 40

106 InnoDB Reads Innodb_data_read ( Mb) Innodb_pages_read (44,377) Innodb_rows_read (2,721,225,514) 41 41

107 InnoDB Writes Innodb_buffer_pool_write_requests (47,307,349) Innodb_data_writes (12,468,811) Innodb_data_pending_writes (0) 42 42

108 InnoDB Writes Innodb_data_written (153 Gb) Innodb_pages_written (4,537,224) Innodb_pages_created (3,664) 43 43

109 InnoDB Rows Written Innodb_rows_inserted (32,681) Innodb_rows_deleted (1,555,539) Innodb_rows_updated (3,908,062) 44 44

110 InnoDB Log Basics innodb_log_file_size (1G) innodb_log_files_in_group (3) innodb_mirrored_log_groups (1) innodb_log_group_home_dir (./) 45 45

111 InnoDB Logs innodb_log_buffer_size (8M) Innodb_log_waits (0) Innodb_os_log_fsyncs (8,464,144) Innodb_os_log_pending_fsyncs (0) innodb_log_buffer_size sensible values range from 1Mb 8Mb according to the manual. A large log buffer allows large transactions to run without a need to write the log to disk before the transactions commit. Thus, if you have big transactions, making the log buffer larger saves disk I/O. If log_buffer_size too small, there will be Innodb_log_waits

112 InnoDB Log Writes Innodb_log_write_requests (26,175,407) Innodb_log_writes (8,166,171) Innodb_os_log_pending_writes (0) Innodb_os_log_written (14.42 Gb) 47 47

113 Other InnoDB Variables innodb_additional_mem_pool_size (16Mb) innodb_open_files (300) innodb_thread_sleep_delay (10,000 =.01 sec) additional_mem_pool_size additional memory for things like the data dictionary. If this value is too small, warnings will be written to the error log and more memory will be allocated from the OS. Default is 1 Mb. innodb_open_files file descriptors for.ibd files. do not affect table_open_cache, variable is independent of other open files limits. innodb_thread_sleep_delay how long (microseconds) a thread waits before joining the InnoDB queue

114 Advanced InnoDB Variables innodb_flush_method () innodb_max_purge_lag (0) fsync() by default to flush data files & logs O_DSYNC = O_SYNC to open and flush logs, fsync() for data files. O_DIRECT (some GNU/Linux versions, FreeBSD, and Solaris), O_DIRECT (or directio() on Solaris) to open the data files, fsync() to flush data files and logs. This variable is relevant only for Unix. On Windows, the flush method is always async_unbuffered and cannot be changed. purge lag = if # of txns that UPDATE or DELETE is > this, then delay a while before proceeding. 0 means no delay ever.

115 InnoDB Status for Performance Innodb_buffer_pool_read_ahead_rnd (16) Innodb_buffer_pool_read_ahead_seq (286) Innodb_buffer_pool_read_ahead_rnd: Number of random read-aheads, for when a large part of the table is scanned, in random order Innodb_buffer_pool_read_ahead_seq: Number of sequential read-aheads, for sequential full table scan High rates of these are both bad.

116 Percona patch extensions XtraDB Percona build of MySQL (5.x.xx-percona-log) Standard and InnoDB Plugin Slides are current up to percona-log Up to percona-log All are global and can be set in my.cnf Many are dynamic, see the wiki for more info Percona did a great job of keeping the variable defaults so they'd work the same as InnoDB does. We've found improvements in performance even when none of these variables are tweaked!

117 I/O Extensions innodb_read_io_threads innodb_write_io_threads innodb_io_capacity innodb_use_purge_thread read_io_threads: The number of background I/O threads for read prefetch requests. Default 1, range write_i/o threads. The number of background I/O threads for writing dirty pages from the buffer pool. Default 1, range you can see them in the FILE I/O part of SHOW ENGINE INNODB STATUS on xtradb io_capacity - # of disk I/O operations InnoDB assumes the server can perform per second. The patch adjusts InnoDB's behavior to consume more I/O resources when it's told they are available. Default 100 how innodb works Allowed values 100 ~ 999,999,999 use_purge_thread, default is 0 through InnoDB plugin and 1 starting Range is 0-64

118 More I/O extensions innodb_ibuf_accel_rate innodb_ibuf_active_contract innodb_ibuf_max_size Ibuf = insert buffer innodb_ibuf_accel_rate if innodb_io_capacity isn't enough to tune the insert buffer, you can also fine-tune it using this. Default 100, 0-999,999,999 innodb_ibuf_active_contract - Default 0(~1.0.5), 1(1.0.6~). 0 means do not contract the size of the insert buffer until you reach max size. 1 makes each user thread contract the insert buffer as possible in asynchronous. innodb_ibuf_max_size default is ½ size of innodb buffer pool. Range is less than that, starting with 0. 0 = very small, and very small values aren't good for performance. If you use very fast storage, small value (like several MB) may show better performance. very fast storage means much faster than HDD based storage (any RAID storages). It is like the battery-backed huge dynamic memory array etc So, it is not recommended to set the value for performance, if the storage doesn't have ram-level speed/response, the performance will drop down.

119 More I/O extensions innodb_enable_unsafe_group_commit innodb_flush_neighbor_pages innodb_read_ahead none random linear both innodb_enable_unsafe_group_commit bug fixed in XtraDB 1.0.6, so not needed. Default is 0, 1 forces group commit even if with binlog. If 1 is set, InnoDBtransaction log order and binlog order may not be same. In this case, if you use some hot-backup of InnoDB, the snapshot may not correspond to any binlog position. WHAT BUG WAS FIXED BY THIS? Flush neighbor pages If a dirty page is flushed, should you also flush neighbor pages in the same data file? Default 1, but 0 is better for disks that don't have delays due to head seeking, like SSD's, or if you have lots of write-buffering. innodb_read_ahead Control native read-ahead behavior of InnoDB. 'none':disable readahead, 'random':enable read-ahead for random access only, 'linear':enable read-ahead for sequential access only, 'both':enable both of read-ahead feature. Default 'both' read-ahead for random access is removed from Plugin So also XtraDB ignores it after I think the idea is better performance, you probably usually set this to linear 'cause that makes sense.

120 Checkpointing innodb_adaptive_checkpoint innodb_checkpoint_age_target Adaptive checkpointing changes when blocks are flushed to disk. Instead of flushing when reaching max_dirty_pages_pct, flushed based on the age of the block. Default value is none, how InnoDB works. Reflex, If there is a block that is ½ the max age, start weak flushing of all blocks. If a block is ¾ max age, start strong flushing. Strength is based on io_capacity, so if you use reflex you need to tune that. 'estimate': If the oldest block age > 1/2 of the max age capacity, flush a # of blocks every second. The number of the blocks is determined by [number of modified blocks], [LSN progress speed] and [average age of all modified blocks]. independent from innodb_io_capacity. experimental innodb_checkpoint_age_target 0 (default): disable the feature. max_checkpoint_age_async can be controlled by this value. It is not needed to shrink innodb_log_file_size to tune recovery time.???? and the LOG section of SHOW ENGINE INNODB STATUS - max checkpoint age, modified age and checkpoint age

121 SHOW ENGINE INNODB STATUS BUFFER POOL AND MEMORY Total memory allocated ; in additional pool allocated Internal hash tables (constant factor + variable factor) + Adaptive hash index ( ) + Page hash Dictionary cache ( ) + File system ( ) + Lock system ( ) + Recovery system 0 (0 + 0) + Threads ( ) Buffer pool size Buffer pool size, bytes Free buffers Database pages 3963 Modified db pages 1510 Pending reads Added extended information of InnoDB internal hash tables sizes (in bytes) in BUFFER POOL AND MEMORY section. It also adds Buffer pool size in bytes information.

122 Statistics/Info innodb_show_locks_held innodb_show_verbose_locks innodb_stats_auto_update innodb_stats_method innodb_stats_update_need_locks innodb_show_verbose_locks - Whether to show records locked in SHOW INNODB STATUS innodb_show_locks_held - Number of locks held to print for each InnoDB transaction in SHOW INNODB STATUS. (see bugs.mysql.com/bug.php?id=29126s innodb_stats_method similar to myisam_stats_method (see slide 17) ('nulls_equal', 'nulls_unequal', 'nulls_ignored'). But InnoDB doesn't have several patterns of statistics currently. So, though this option able to be changed dynamically, we need re-calculate statistics to change the method for the table. (HUH?) innodb_stats_update_need_locks - (default 1) - If you meet contention of &dict_operation_lock, setting 0 reduces the contention. But 0 disables to update Data_free: of show table status. innodb_stats_auto_update - (default 1) - InnoDB updates the each index statistics automatically. Setting this option 0 can stop these automatic recalculation of the statistics except for first open and ANALYZE TABLE command.

123 New in INFORMATION_SCHEMA innodb_table_stats innodb_index_stats GET EXAMPLES!!! table_name table name in InnoDB internal style ('database/table') rows estimated number of all rows clust_size cluster index (table/primary key) size in number of pages other_size other index (non primary key) size in number of pages modified internal counter to judge whether statistics recalculation should be done If the value of modified column exceeds rows / 16 or , the statistics recalculation is done when innodb_stats_auto_update == 1. We can estimate the oldness of the statistics by this value. innodb_index_stats - shows index statistics information of dictionary cached. Field Notes table_name table name in InnoDB internal style ('database/table') index_name index name fields How many fields the index key has. (it is internal structure of InnoDB, it may be larger than the 'CREATE TABLE') row_per_keys estimate rows per 1 key value. ([1 column value], [2 columns value], [3 columns value], ) index_size index size in pages leaf_pages number of leaf pages

124 Recovery innodb_fast_recovery innodb_recovery_stats innodb_fast_recovery buf_flush_insert_sorted_into_flush_list() hack not to sorting (new variable innodb_fast_recovery (default false(~1.0.5), true(1.0.6~)) - if set true, the hack is enabled.) MORE WORK NEEDED More recovery stats (default, I think.)

125 Recovery Stats RECOVERY STATISTICS Recovery time: 18 sec. (1 turns) Data page IO statistics Requested pages: 9126 Read pages: 9126 Written pages: 7957 (Dirty blocks): 1156 Grouping IO [times]: number of pages, read request neighbors (in 32 pages chunk), combined read IO, combined write IO 1, 32, 335, 548 2, 0, 121, 97 3, 7, 49, 44 4, 4, 43, , 0, 2, innodb_fast_recovery buf_flush_insert_sorted_into_flush_list() hack not to sorting (new variable innodb_fast_recovery (default false(~1.0.5), true(1.0.6~)) - if set true, the hack is enabled.)

126 Recovery Stats Recovery process statistics Checked pages by doublewrite buffer: 128 Overwritten pages from doublewrite: 0 Recovered pages by io_thread: 9145 Recovered pages by main thread: 0 Parsed log records to apply: Sum of the length: Applied log records: Sum of the length: Pages which are already new enough: 93 Oldest page's LSN: Newest page's LSN: innodb_fast_recovery buf_flush_insert_sorted_into_flush_list() hack not to sorting (new variable innodb_fast_recovery (default false(~1.0.5), true(1.0.6~)) - if set true, the hack is enabled.)

127 Other Extensions innodb_thread_concurrency_timer_based innodb_relax_table_creation innodb_extra_rsegments INFORMATION_SCHEMA.innodb_rseg Relax table creation default 0, values are 0 or 1. Dynamic, global. extra_rsegments - (default 0) - The number of extra user rollback segments created when new db is created. Information schema added innodb_rseg GET EXAMPLE shows information about all rollback segments Field Notes rseg_id rollback segment id space_id space where the segment placed zip_size compressed page size in bytes if compressed otherwise 0 page_no page number of the segment header max_size max size in pages curr_size current size in pages

128 Other Extensions innodb_dict_size_limit Innodb_dict_tables innodb_expand_import innodb_extra_undoslots innodb_overwrite_relay_log_info dict_size_limit: for memory. default is 0, unlimited. Data dictionary can grow large. Innodb_dict_tables is status variable with # entries in the cache. expand_import: convert.ibd using.exp info when doing IMPORT TABLESPACE, when this is 1. default is 0. Use this to change indexes quickly. extra_undo_slots Only for err Warning: cannot find a free slot for an undo log. If true, up to 4072 undo slots. If false, MySQL default is 1024 undo slots, default is false. overwrite_relay_log_info: The position of replication is updated after InnoDB commit of the transaction on the slave. If a crash occurs after the commit but before the replication position update, the committed transaction is executed again after the crash recovery. If this is set to true, the relay log file will be overwritten with the value (later position). Default is false, because you might end up skipping statements from other storage engines.

129 T ha nk Y o u. Win a signed copy of Sheeri s book. 64 Leave your business card and you could win a book. We ll invite you to read our blog posts, follow us on twitter, and join our next webinars. Drawing will be immediately after the talk once all cards are collected. 2009/2010 Pythian - S uc c es s ful g row ing bus ines s for more than 10 years - Served many customers with complex requirements/infrastructure just like yours. - Operate globally for 24 x 7 always awake services 64

130 4/12/10 T ha nk Y ou Questions, Comments, Feedback? Sheeri Cabral cabral@pythian.com Blog: Ask me about saving 15% on our Adoption Accelerator for MySQL while at MySQL Conference 2010! /2010 Pythian