Scaling with Postgres. Robert Treat Percona Performance Conference 2009

Transcription

1 Scaling with Postgres Robert Treat Percona Performance Conference 2009

2 Disclaimer Every really big site uses custom solutions. Don t expect to get there using out of the box software. You don t have to reinvent the wheel. But you may have to build the car.

3 Who am I?

4 Who am I? OmniTI TB+ databases top 100 websites Oracle/MySQL/Postgres Not necessarily in that order :-)

5 Scaling Databases 101

6 Scaling Databases 101 Implementation

7 Scaling Databases 101 Implementation Tuning Optimize Queries Optimize Schema

8 Scaling Databases 101 Implementation Tuning Optimize Queries Optimize Schema Throw Hardware At It Caching

9 Scaling Databases 101 Implementation Tuning Optimize Queries Optimize Schema Throw Hardware At It Caching Replication

10 Scaling Databases 101 Implementation Tuning Optimize Queries Optimize Schema Throw Hardware At It Caching Replication Federation

11 Implementation Normalization Foreign Keys Relational Modeling

12 Implementation Normalization Foreign Keys Relational Modeling Mostly Irrelevant

13 Implementation

14 Implementation Monitoring nagios, munin, etc.. check_postgres

15 Implementation Monitoring nagios, munin, etc.. check_postgres Trending cacti, noit watch your hardware!

16 Tuning

17 Tuning Get the Basics Right Tuning_Your_PostgreSQL_Server

18 Tuning Get the Basics Right Tuning_Your_PostgreSQL_Server Someday Xlogs, Checkpoints Vacuum Strategy Costs and other Minutia

19 Optimize Queries explain explain analyze explain analyze verbose

20 Optimize Queries writing queries tuned to your databases planner

21 Optimize Schema

22 Optimize Schema indexing partial, functional fix data types add/remove tables de-normalize (maybe)

23 Why is this important? Postgres is good at scaling vertically Flawed database schema hides the real problems

24 Why is this unimportant? Optimizing is often a trade off of time vs. money It is not an ultimate solution

25 Throw Hardware At It Find your bottlenecks Memory / RAM CPU I/O, DISKS usually the biggest problem

26 Can you take me higher? 32 Cores 128 GB+ RAM Spindles out the wazoo tablespaces

27 Caching

28 Caching Internal materialized views

29 Caching Internal materialized views External memcached, et al. static files

30 Oh, the places you ll go Nothing Fancy or Exciting Solid Engineering Understanding needs / limits

31 Oh, the places you ll go Nothing Fancy or Exciting Solid Engineering Understanding needs / limits Top Website

32 Replication Ignore the hype Triggers won t destroy your site Scripting solves a lot of config issues Telling your DBA to do his!@#$ job solves config issues too Add-on is FUD for Modular

33 Replication Truth in advertising Oracle does it better MySQL does it simpler

34 Replication

35 Replication Scaling Reads Master, many slaves Slony, Londiste, Bucardo, PGPoolII

36 Replication Scaling Reads Master, many slaves Slony, Londiste, Bucardo, PGPoolII Scaling Writes Mostly impossible Bucardo, PGPoolII

37 What is Federation? A process for eliminating performance and/or scaleability as a requirement for your database

38 Federation

39 Federation MySQL software has no federation advantage

40 Federation MySQL software has no federation advantage MySQL community has large federation advantage

41 Federation Things to consider Horizontal Partitioning Application Based Partitioning BASE golconde, gearman

42 El Fin

43 Postgres BOF 7:30 PM Camino Real