MySQL Cluster Student Guide

Transcription

1 MySQL Cluster Student Guide D62018GC11 Edition 1.1 November 2012 D79677

2 Technical Contributor and Reviewer Mat Keep Editors Aju Kumar Daniel Milne Graphic Designer Seema Bopaiah Publishers Sujatha Nagendra Joseph Fernandez Nita Brozowski Copyright 2012, Oracle and/or its affiliates. All rights reserved. Disclaimer This document contains proprietary information and is protected by copyright and other intellectual property laws. You may copy and print this document solely for your own use in an Oracle training course. The document may not be modified or altered in any way. Except where your use constitutes "fair use" under copyright law, you may not use, share, download, upload, copy, print, display, perform, reproduce, publish, license, post, transmit, or distribute this document in whole or in part without the express authorization of Oracle. The information contained in this document is subject to change without notice. If you find any problems in the document, please report them in writing to: Oracle University, 500 Oracle Parkway, Redwood Shores, California USA. This document is not warranted to be error-free. Restricted Rights Notice If this documentation is delivered to the United States Government or anyone using the documentation on behalf of the United States Government, the following notice is applicable: U.S. GOVERNMENT RIGHTS The U.S. Government s rights to use, modify, reproduce, release, perform, display, or disclose these training materials are restricted by the terms of the applicable Oracle license agreement and/or the applicable U.S. Government contract. Trademark Notice Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

3 Contents 1 Introduction to MySQL Cluster Course Goals 1-2 Course Lesson Map 1-4 Student Introductions 1-6 Classroom Environment 1-7 MySQL: Overview 1-8 MySQL Cluster Carrier Grade Edition 1-9 MySQL Cluster Users and Applications 1-10 Key Questions to Ask When Considering MySQL Cluster 1-11 Key Benefits of MySQL Cluster 1-12 How Does the MySQL Cluster Manager Help? 1-13 If You Are Not Using the MySQL Cluster Manager 1-14 MySQL Tools 1-15 MySQL Connectors and APIs 1-16 MySQL Services 1-18 MySQL Community Support 1-19 Oracle Lifetime Support for MySQL Rely on the Experts, Get Unique Benefits 1-20 MySQL-Supported Operating Systems 1-21 MySQL Websites 1-22 Download MySQL Cluster 1-23 Evaluate MySQL Cluster Carrier Grade Edition 1-24 Bootstrap Single Host Cluster 1-25 MySQL Curriculum Footprint 1-26 MySQL Certification 1-27 MySQL Online Documentation 1-28 Lesson Summary 1-29 Practice 1-1 Overview: MySQL Resources 1-30 Practice 1-2 Overview: Installing MySQL Server 1-31 Practice 1-3 Overview: Installing the world Database MySQL Cluster Concepts Objectives 2-2 What Is a Cluster? 2-3 Shared-Nothing Versus Shared-Disk 2-4 MySQL Cluster Hardware 2-5 NDB Storage Engine 2-7 Partitioning 2-8 iii

4 Horizontal and Vertical Partitioning 2-9 Partitioning and Fragments 2-10 Partitioning and Node Groups 2-11 Distribution Aware Applications 2-12 Distribution Aware: Multiple Tables 2-13 Validate Whether Partition Pruning Is Working 2-14 Parallel Operations for Throughput 2-15 MySQL Cluster Architecture 2-16 MySQL Extreme Resilience 2-18 MySQL Cluster Processes 2-19 API Flexibility SQL and NoSQL Combined 2-20 Replication 2-21 Two-Phase Commit 2-22 Two-Phase Commit: Phase Two-Phase Commit: Phase Geographical Replication 2-25 Split-Brain Scenario 2-27 The Arbitrator Decides 2-28 Failure Detection 2-29 A Missed Heartbeat 2-30 Checkpoints and Logging 2-32 In-Memory and Disk-Based Data 2-34 Summary Install and Start MySQL Cluster Objectives 3-2 Initiate MySQL Cluster 3-3 config.ini -[NDBD DEFAULT] Section 3-5 config.ini -[NDB_MGMD] Section 3-6 config.ini -[NDBD] Section 3-7 config.ini -[MYSQLD] Section 3-8 Node Identification 3-9 Management Daemon 3-10 Management Console 3-11 SHOW Command 3-12 Starting the Data Nodes 3-14 Data Node Startup Options 3-17 Connecting the SQL Nodes: my.cnf 3-19 Putting It All Together 3-20 Creating a Clustered Table 3-21 Startup Phases 3-23 iv

5 Monitoring the Startup Phases 3-25 Restarting a Cluster 3-26 Restarting on Failure 3-27 Data Node Delays 3-28 Testing Management Node Failure 3-29 Testing Data Node Failure 3-31 Upgrading Within Major Releases 3-32 Upgrading to a New Major Release 3-34 Summary 3-36 Practice 3-1 Overview: Setting Up and Configuring a Cluster 3-37 Practice 3-2 Overview: Restarting the Cluster 3-38 Practice 3-3 Overview: Testing High Availability Cluster Configuration File Objectives 4-2 Cluster Configuration File 4-3 Configuration Options for All Nodes 4-5 Management Node Configuration 4-8 Data Node Configuration 4-11 SQL Node Configuration 4-17 Summary 4-20 Practice 4-1 Overview: Editing the config.ini File NDB Storage Engine Objectives 5-2 NDB Storage Engine 5-3 NDB s Storage Method: In-Memory Storage 5-4 NDB s Storage Method: Disk-Based Storage 5-6 NDB s Storage Method: Undo Logging 5-8 NDB s Storage Method: Fixed-Width Columns 5-9 NDB Indexing 5-10 Hash Index: Overview 5-11 Ordered Index 5-12 Storage of Indexes 5-14 NDB Communication Interconnects 5-15 NDB s Transactional Capabilities 5-17 Durability 5-19 Global Checkpoint 5-21 Local Checkpoint 5-22 Summary 5-23 v

6 6 Designing a Cluster Objectives 6-2 CPU Considerations 6-3 Network Connection Considerations 6-5 Disk-Based Storage Considerations 6-7 Memory Considerations 6-9 Memory Allocation 6-10 Memory Alignment 6-12 Sizing Up Memory 6-13 Memory Pages 6-14 Other Memory Issues: One Record, Multiple Pages 6-15 Other Memory Issues: BLOBs of Memory 6-16 Other Memory Issues: Configuration Parameters 6-18 Putting It All Together 6-20 Load Balancing and Failover: Data Nodes 6-22 Load Balancing and Failover: Management Nodes 6-23 Load Balancing and Failover: SQL Nodes JDBC Driver Support 6-25 Load Balancing and Failover: SQL Nodes Round-Robin DNS 6-27 Load Balancing and Failover: SQL Nodes Hardware Load Balancing 6-29 Load Balancing and Failover: SQL Nodes Software Load Balancing 6-30 Physical Cluster Setups: Basic Setup 6-32 Physical Cluster Setups: One-to-One MySQL Farm 6-33 Physical Cluster Setups: Separate Application and MySQL Farm 6-34 Network Redundancy 6-36 Summary 6-38 Practice 6-1 Overview: Running the ndb_size.pl Perl Script 6-39 Practice 6-2 Overview: Logical Data Layout Handling Data in MySQL Cluster Objectives 7-2 Disk Data Objects 7-3 Creating a Log File Group 7-4 Adding Undo Files to a Log File Group 7-5 Verifying UNDO Files Are Created 7-6 Creating the Tablespace 7-7 Creating a Disk Data Table 7-9 Creating a Disk Data Column 7-11 Monitoring Free Space 7-14 Exporting Data from and Importing Data to Cluster Tables 7-15 Converting an Existing Table 7-17 Importing Table: Typical Errors 7-19 vi

7 Dropping Cluster Tables 7-21 Summary 7-23 Practice 7-1 Overview: Handling Data in MySQL Cluster MySQL Cluster Management Objectives 8-2 MySQL Cluster Management Console: Cluster Status 8-3 MySQL Cluster Management Console: Cluster Control Commands 8-5 MySQL Cluster Management Console: Cluster Logging Commands 8-7 MySQL Cluster Management Console: Single User Commands 8-11 MySQL Cluster Management Console: Cluster Backup Commands 8-13 MySQL Cluster s Native Backup Tool 8-15 mysqldump 8-19 Restoring Backups 8-21 Restoring mysqldump Files 8-26 Troubleshooting Backups 8-27 Modifying Table Definitions 8-29 Optimizing Cluster Tables 8-32 Summary 8-34 Practice 8-1 Overview: MySQL Cluster Management 8-35 Practice 8-2 Overview: Classroom Cluster MySQL Cluster Security Objectives 9-2 Cluster Security Best Practices: Cluster Communication 9-3 Cluster Security Best Practices: Authentication 9-4 Cluster Security Best Practices: Privileges 9-5 Cluster Security Best Practices: Putting It All Together 9-6 Physically Segregated Architecture 9-7 Software Firewall 9-9 Mixed Architecture 9-11 Recommended Options 9-12 Summary MySQL Cluster Performance Objectives 10-2 Performance Terms: Throughput and Response Time 10-3 Performance Terms: Latency and Service Time 10-4 Performance Terms: Scalability, Scalable System and Capacity Planning 10-5 Benchmarking 10-6 Indexes 10-8 vii

8 Creating Indexes 10-9 First Node/First Fragment Second Node/Second Fragment Primary Key Lookup Unique Key Lookup Ordered Index Range Full Table Scan EXPLAIN Index Statistics Query Cache Horizontal Partitioning Data Size and Usage Issues: Column Types Data Size and Usage Issues: Page Sizes Data Size and Usage Issues: Buffer Manager Query Design Query Design: Using Batching and Transactions Lesson Summary Practice 10-1 Overview: Using EXPLAIN Practice 10-2 Overview: Classroom-Wide Cluster Resolving Cluster Problems Objectives 11-2 Management Client 11-3 Cluster Logs 11-5 Cluster Log Output 11-6 Data Storage Warnings 11-7 Missed Heartbeat Warnings 11-8 Starting ndbd Errors 11-9 Missing Directory Error Handler Restart Communication Errors Solving the Split-Brain Problem Arbitrator Process Table Is Full Errors Error Where to Get Additional Help Summary viii

9 12 MySQL Cluster Replication Objectives 12-2 MySQL Replication 12-3 Cluster Replication Logging: binlog 12-4 Cluster Replication Logging: Replication Servers and Row-Based Replication 12-6 Cluster Replication Logging: Enabling Binary Logging 12-8 MySQL Cluster Replication: System Tables 12-9 MySQL Cluster Replication System Tables: NDB Injector Thread Known Issues Replication Preparation MySQL Cluster Replication: Preparation Steps Single Replication Channel Batched Updates Redundant Replication Channels Implement Failover Using MySQL Cluster Replication for Backups: From the Management Client Using MySQL Cluster Replication for Backups: From the Operating System Shell Restore a Cluster Replication Backup Summary Practice 12-1 Overview: Setting Up Binary Logging Practice 12-2 Overview: Multiple Classroom Clusters for Replication Conclusion Course Goals 13-2 MySQL Curriculum Path 13-4 MySQL Resources 13-5 We Need Your Evaluation! 13-6 Thank You 13-7 Q&A Session MySQL Cluster 7.x Features MySQL Cluster 7.1: Key Enhancements 14-2 ndbinfo Database 14-3 ndbinfo: memoryusage 14-4 ndbinfo: TABLE_SCANS 14-5 ndbinfo: logbuffers 14-6 ndbinfo.counters 14-7 ndbinfo.logbuffers 14-8 ndbinfo.logspaces 14-9 ndbinfo.memoryusage ix

10 ndbinfo.nodes MySQL Cluster Connector for Java ClusterJ ClusterJ: Interfaces ClusterJ Configuration: Properties Example ClusterJ: Annotated Interface Example ClusterJ: Application Code Example ClusterJPA ClusterJPA: Annotated Class Example ClusterJPA: Application Code Example MySQL Cluster 7.2: Key Enhancements Multithreaded Data Node Extensions Adaptive Query Localization (AQL): Scaling Distributed Joins Adaptive Query Localization (AQL): How to Use Memcached API: Most Widely Adopted Key-Value, NoSQL API API Flexibility: SQL and NoSQL Combined MySQL 5.5 Server Integration MySQL 5.5 Feature Applicability to Cluster Virtualization Support Multisite Clustering Multisite Clustering: STONITH Issue Addressed in MySQL Cluster Multisite Clustering: New STONITH Algorithm Multisite Clustering: WAN Recommendations Geographic Replication Enhancements Geographic Replication: Conflict Resolution Ease of Use: User Privilege Consolidation Ease of Use: Pushing Privilege Data into Data Nodes MySQL Cluster Manager Features MySQL Cluster Manager: Glossary MySQL Cluster Manager: Example Configuration MySQL Cluster Manager Resources x