MySQL Enterprise Solutions

Size: px

Start display at page:

Download "MySQL Enterprise Solutions"

Esther Young
5 years ago
Views:

1 MySQL Enterprise Solutions Alexander Sasha Pachev Wiley Publishing, Inc.

3 MySQL Enterprise Solutions Alexander Sasha Pachev Wiley Publishing, Inc.

4 Publisher: Robert Ipsen Editor: Robert M. Elliott Managing Editor: Vincent Kunkemueller Book Producer: Ryan Publishing Group, Inc. Copyeditors: Elizabeth Welch and Tiffany Taylor Proofreader: Nancy Sixsmith Compositor: Gina Rexrode Designations used by companies to distinguish their products are often claimed as trademarks. In all instances where Wiley Publishing, Inc., is aware of a claim, the product names appear in initial capital or ALL CAPITAL LETTERS. Readers, however, should contact the appropriate companies for more complete information regarding trademarks and registration. This book is printed on acid-free paper. Copyright 2003 by Wiley Publishing, Inc. All rights reserved. Published by Wiley Publishing, Inc., Indianapolis, Indiana Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) , fax (978) Requests to the Publisher for permission should be addressed to the Legal Department, Wiley Publishing, Inc., Crosspoint Blvd., Indianapolis, IN 46256, (317) , fax (317) , permcoordinator@wiley.com. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services please contact our Customer Care Department within the United States at (800) , outside the United States at (317) or fax (317) Trademarks: Wiley, the Wiley Publishing logo and related trade dress are trademarks or registered trademarks of Wiley Publishing, Inc., in the United States and other countries, and may not be used without written permission. All other trademarks are the property of their respective owners. Wiley Publishing, Inc., is not associated with any product or vendor mentioned in this book. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Library of Congress Cataloging-in-Publication Data: Pachev, Alexander, MySQL enterprise solutions / Alexander Pachev. p. cm. Wiley Computer Publishing. Includes index. ISBN SQL (Computer program language) I. Title. QA76.3.S67 P dc Printed in the United States of America

5 CONTENTS Acknowledgments About the Author Introduction ix xi xiii Part I Bringing MySQL into Your Enterprise Chapter 1 Overview of MySQL 1 How Is MySQL Most Commonly Used in the Enterprise? 2 Database Backend for a Web Site 2 Usage Logger 3 Data Warehousing 3 Integrated Database 3 Embedded Database 4 Strengths and Weakness of MySQL 4 Strengths 5 Weaknesses 9 MySQL from the Application Developer s Perspective 12 Overview of MySQL Integration with Other Industry-Standard Software 13 Getting Help with MySQL 14 Online Documentation 14 Mailing List 15 Local Linux User Groups 16 Commercial Support from MySQL AB 16 Chapter 2 Selecting a Platform for MySQL Server 19 Platform Criteria 19 Size of the User Base 20 Amount of Usage under High Load on Mission-Critical Servers 20 Maturity of the C/C++ Compiler 20 Number of MySQL AB Developers Regularly Using the Platform 20 Degree of Standard Compliance in the System Libraries 20 Maturity of the Thread Library Available on the System 21 Platform Comparison 21 Linux 21 Windows 22 Solaris 23 FreeBSD 24 Other Systems 24 Operating System Tuning Tips 25 Hardware Tips 26 iii

6 iv Contents Chapter 3 Installing MySQL 27 Method of Installation 27 The Need for Transactional Table Support 28 Version Issue 29 Installation Process 31 Binary Installation 31 Source Installation 35 Basic Post-Installation Checks 37 Post-Installation Setup 38 Proxy Database Access 39 Hosting Provider 40 Single User 40 Direct Multiple-User Database Access 41 Troubleshooting 42 mysqld ended 43 Installation of grant tables failed! 43 ERROR 1045: Access denied 44 ERROR 2002: Can t connect to local MySQL server through socket '/tmp/mysql.sock' (111) 45 Other Problems 46 Chapter 4 Testing Your MySQL Installation 49 The Standard MySQL Test Suite (mysql-test-run) 49 The Server Limit Test (crash-me) 55 The One-Threaded Standard MySQL Benchmark 68 The Basic Multithreaded Benchmark (mysqlsyseval) 74 Your Own Tests 78 Chapter 5 Access Control and Security 83 MySQL Access Privilege System 83 Granting Privileges 85 Revoking Privileges 86 Removing Users 86 System Security 87 Check Setuid Binaries 87 Run Only Necessary Services 88 Set Up a Firewall 89 Monitor the System Daily 89 Database Application Security 89 Server Configuration Security 90 Data Transfer Security 91 Dealing with Especially Sensitive Data 92 Conclusion 92

7 Contents v Part II Developing MySQL Applications Chapter 6 Choosing the Client Language and Client-Server Network Architecture 93 Choosing a Client Language 93 Performance 93 Development Time 94 Code Integration 94 Portability 95 Developer Skills and Preferences 95 Client Language Performance Comparison 96 Network Architecture 99 Client and Server on the Same Host 99 One Server Host and One Remote Client Host 100 One Server Host and Many Remote Client Hosts 100 Data Distributed Across Several Server Hosts and Queried by One Client Host 101 Data Distributed across Several Server Hosts and Queried by Several Client Hosts 101 Estimating Load from Clients 101 Client Programming Principles 102 Chapter 7 MySQL Client in a Web Environment 105 Choosing a Web Server 105 Server-Application Integration Methods 106 Web Optimization Techniques 107 Avoiding Long Queries 107 Avoiding Unnecessary Queries 108 Avoiding Unnecessary Dynamic Execution 109 Using Persistent Connections 109 Stress-Testing a Web Application 110 Using ApacheBench 111 Other Approaches 112 Chapter 8 C/C++ Client Basics 115 Preparing Your System 115 Structures and Functions of the API 116 API Overview 122 A Sample Application 124 Tips and Tricks 138 Chapter 9 PHP Client Basics 139 Preparing Your System 140 API Functions 141 API Overview 146 Sample Code 148 Tips and Tricks 162

8 vi Contents Chapter 10 Perl API Basics 165 System Preparation 166 DBI Methods and Attributes 167 API Overview 169 Sample Code 170 Tips and Tricks 178 Chapter 11 Java Client Basics 181 System Configuration 182 JDBC Classes and Methods 182 API Overview 189 Sample Code 192 Chapter 12 Writing the Client for Optimal Performance 203 Query Caching 203 Replication-Aware Code 204 Improving Write-Dominant Applications 205 Reducing Network I/O 206 Understanding the Optimizer 207 Part III Maintaining and Optimization Chapter 13 Table Design 223 Column Types and Disk Space Requirements 223 Variable-Length versus Fixed-Length Records 226 Normalization 227 The Need for Proper Keys 230 Data Wisdom 233 The Proper Table Type 234 Chapter 14 Configuring the Server for Optimal Performance 237 Optimizing the Schema 237 Optimizing Server Variables 238 Variable Descriptions 242 Additional Variables for MySQL Verifying Support for Variables 258 Operating System Adjustments 263 Hardware Upgrades 264 Chapter 15 Analyzing and Improving Server Performance 265 Staying Out of Trouble 265 Dealing with Slow Performance 267 Using the EXPLAIN Command 270 Using the mysqldumpslow Script 275 Monitoring Server Activity Patterns with SHOW STATUS 277

9 Contents vii Chapter 16 Replication 293 Replication Overview 293 Uses of Replication 294 Setting Up Replication 296 Configuring the Master 296 Configuring the Slave 299 Replication Maintenance 302 Measuring How Far the Slave Is Behind the Master 303 Replication Errors 304 Stopping Replication 304 Replication Caveats 304 Improperly Replicated Queries 305 Replication of Temporary Table Activity 305 Replication of LOAD DATA INFILE 305 Bidirectional Replication 306 Replication Internals 306 Masters and Slaves 307 Server IDs 307 Binary Logs 307 Conclusion 309 Chapter 17 Backup and Table Maintenance 311 Physical Backup 311 Logical Backup 313 Incremental Backup 316 Backup through Replication 316 Table Maintenance 317 Chapter 18 Exploring MySQL Server Internals 323 Getting Started 323 Tour of the Source 327 Execution Flow 328 General Guidelines for Extending the Source 340 Adding a New Native SQL Function 341 Adding a UDF 350 Adding a New Table Handler 353 Maintaining Your Code Modifications 360 Conclusion 362 Part IV Appendices Appendix A Migration Notes 363 Appendix B Troubleshooting Notes 367 Functionality 367 Stability 368 Performance 369

10 viii CONTENTS Appendix C SQL Problem Solving Notes 371 Problem Problem Problem Problem Problem Problem Problem Problem Problem Problem Appendix D Online Resources 379 Index 381

11 ACKNOWLEDGMENTS Dedication This book is dedicated to my wife Sarah, and my children Benjamin, Jenny, and Julia. Acknowlegments This book could not have been written without the participation of several parties who I would like to thank. The acknowledgments are not given in any particular order of importance as I regard everyone as an equal participant, a part of the big whole without whom the whole would not be the same, or not even exist in the first place. Tim Ryan saw a potential author in me, approached me with the idea to write this book, and guided me through refining the rough edges of my writing. Special thanks to Monty Widenius for creating MySQL and to David Axmark for convincing Monty to make it available to everyone, which eventually permitted me to get involved with it. MySQL AB allowed me to take some time off to write the book. Special thanks to Marten Mickos, CEO of MySQL AB, Tom Basil, the Director of Customer Support, Larry Stefonic, the VP of Sales, and other employees of MySQL AB for their support. Special acknowledgments to Joe Gradecki, who reviewed several chapters in the book and contributed some additional content. My wife and children have given me a lot of support and deserve more credit than I could possibly express in writing. My wife Sarah was pregnant with our third child Julia throughout most of the work, and had to take care of Benjamin (3), and Jenny (2) on her own while I was locked up in the office writing. She went to great lengths to make sure the little things that took me a long time to do got done so I could have enough time to write the book. Every night, Benjamin and Jenny patiently waited for Daddy to finish his work for the day before he could get out of the office and play with them. ix

13 ABOUT THE CONTENTS AUTHOR Alexander Sasha Pachev has been working with MySQL AB since His most significant contribution was the development of the master-slave replication functionality for MySQL server. He has added several other features to the server; and provided high-level , login, and phone support for MySQL customers, presales support, and on-site consulting. During his work with MySQL, he has established a reputation for being able to quickly track down and fix difficult bugs. He enjoys solving difficult problems, and his motto is Let s make it work! Prior to his work at MySQL AB, Sasha was employed by several Internet startups. There, he developed his love and appreciation for MySQL as he used it to create solutions that allowed the company to meet its needs without spending a lot of money in software licensing fees. Raised in Moscow, Russia, Sasha came to Provo, Utah in 1993 to study at the Brigham Young University. After taking a two year break to serve a mission for the LDS church between , Sasha graduated from BYU with a Bachelor s degree in Computer Science in While at BYU, Sasha married his wife, Sarah Matthews, and they are parents of three children: Benjamin, Jenny, and Julia. Sasha and his family live in Provo, Utah. Sasha is an avid distance runner, and his favorite distance is the full-length marathon (26.2 miles). He has won the Boise Marathon twice, placed in the top in many other races, and has a personal best of 2:33:20. His next goal is to qualify for the U.S. Olympic Trials, which would give him a shot at making the U.S. Olympic Team. xi

15 Introduction Several months ago, Tim Ryan approached me with an invitation to write a book for experienced database programmers and administrators who were in the process of adopting MySQL. I initially hesitated, as my hands were already full with my job at MySQL AB, family responsibilities, serving in my church, and training in an attempt to qualify for the U.S. Olympic trials in the marathon. Nevertheless, after some prayer followed by a discussion with my wife, Sarah, I knew that writing a book would be the right thing to do. I accepted the offer, and made the attempt to communicate in writing the knowledge and experience I have accumulated working with MySQL as a user and client programmer, and later on as a server programmer and support engineer. This book is meant to be a practical guide for anyone deploying a missioncritical application that uses MySQL as a database backend. No book can answer every question on a subject, but it is my hope that for the questions it does not answer, it will give you enough background and direction to enable you to find the answer quickly through your own research. As such, this book is not a substitute rather it is a companion for the MySQL online manual ( Although I am an employee of MySQL AB, which undoubtedly affects my perception of the subject, the views and opinions stated in this book are my own and do not necessarily represent the official position of the company. The advice in this book is based on my own experience working with MySQL and with other MySQL users. I have made every effort to stick as close as possible to the facts available to me. It is my hope that you will be able to see the issues as they really are, so that you can decide whether to adopt and deploy MySQL, and if so, the best way to go about it.

MySQL Enterprise Solutions

MySQL Enterprise Solutions Alexander Sasha Pachev Wiley Publishing, Inc. MySQL Enterprise Solutions Alexander Sasha Pachev Wiley Publishing, Inc. Publisher: Robert Ipsen Editor: Robert M. Elliott Managing