5/7/13. Mission Critical Databases. Introduction AOBD

Transcription

1 Mission Critical Databases AOBD 30 April Introduction Paulo Ferro Graduated in Information Systems and Computer Engineering (LEIC) by Instituto Superior Técnico Solutions Novabase paulo.ferro@novabase.pt 2 1

2 Goals Bridge academic knowledge with real life needs How Requirements affect Architecture What hides behind high-performance systems Understand the several system types and their characteristics. How and when to deal with performance issues in a project 3 Architecture Guidelines 2

3 Architecture Guidelines. Always be ruled by keep it simple System design should focus on functionality rather than performance Design and create easy to maintain systems. Denormalize data focused on functionality. Ex.: Number of items in an order Database access is an external access and should be respected as such. Logical data model is different from physical model and should focus on functionality. Transformation layer between models should be isolated. Ex.: No SQL in the code 5 Architecture Guidelines. Avoid solutions that lead to dead-ends. Ex.: Database Isolation layers that don t allow SQL manipulation. Deny dogmas and anything that implies this is the best approach for everything. Ex.: Policies like always use stored procedures. Avoid excessive layers or tiers. Don t rely on the database for concurrency management. Ex.: Pessimistic models for concurrency management (explicit locks). Don t rely too much on the database to manage data integrity 6 3

4 Role of the DBA Role of the DBA DBA in a project. Identify project category Basic Normal. Mission critical. From volume and/or performance requirements and/or usage (Ex: load). Evangelize team and customer about project category. Identify associated trade-offs. Functional, technical and/or architecture. Be pedagogic in conception phases Alert for common mistakes. Raise awareness of critical areas 8 4

5 5/7/13 Role of the DBA DBA in a project..... Define database access guidelines and rules Participate in the data-model definition as a consultant Be a pillar of trust for the client and the team during the tuning phases (Pre Go live and Go live) Refrain unfocused optimization attempts in stressful moments Ex.: hardware upgrade, index creation, etc. 9 Critical Milestones 5

6 Critical Milestones Pre Go Live. Constant Monitoring during Pre Go Live (and Go Live). Ex.: Profiler. Refrain unfocused optimization attempts. Ex.: hardware upgrade, index creation, etc.. Always optimize based on objective analysis 11 Critical Milestones. Avoid maintenance tasks during rush hours. Ex.: backups, re-indexing, maintenance tasks Go Live. Everyone tends to underestimate the impact of such tasks in mission critical systems Ensure updated statistics at the right moments (DB with relevant data). This is the crucial phase of tuning 12 6

7 Mission Critical Projects Formula 1 Highly demanding in terms of performance (response time) Lower demands in terms of load. Behaved regarding data Ex.: Archiving systems, Great Warehouses Usually have small variations in usage (INSERT/SELECT) Are considered Mission Critical for the high data volumes (up to Millions). Demands focus on query performance 14 7

8 Mission Critical Projects Rally Highly demanding in terms of load (Concurrency & Usage) Data Volume not very high. Misbehaved regarding data Ex.: Assembly line supporting systems, massive information acquisition systems Usually complex in terms of DB usage Considered Mission Critical for the heavy load it is subjected to. Demands focus on concurrency reliability (Ex.: locks/deadlocks) 15 Optimizing Systems. Two distinct phases n secs to 1sec. 1 sec to 0,x sec Distinct actions to each phase. Actions vary with the project type. Operations (Ex.: DELETEs, etc.) have different impacts in different project types 16 8

9 Optimizing from n sec to 1 sec Mistakes easy to identify quick wins with low impact in terms of effort. Highly addicting for all team members, including manager and client. Once started it is hard to know when to stop 17 Optimizing from n sec to 1 sec Common Mistakes (1/2) Indexing errors Explicit Locks Deadlocks. Execution plans impossible to define. Ex.: column=isnull(@parm, column). Procedural programming in a relational engine IFs inside procedures. Cursors Everything to be processed by the DB can be done without cursors. Everything that requires cursors must not be processed by the DB 18 9

10 Optimizing from n sec to 1 sec Common Mistakes (2/2) Poor optimizer usage (or lack of confidence in it J ) Excessive usage of temporary tables. Poor programming Ex.: Access to the database within cycles. Not detectable on the profiler, but the lack of trace becomes its evidence. Transaction time too high 19 Optimizing from 1 sec to 0,x sec Great effort and impact for small wins. Hard to manage expectations of the all team members, including manager and client Only for mission critical systems in terms of performance and load. Greater DBA involvement 20 10

11 Optimizing from 1 sec to 0,x sec Examples (1/2) Replicate columns to reduce the number of JOINs Denormalization beyond the obvious to reduce JOINs and inner queries Remove triggers Do not DELETE Minimize UPDATEs. Avoid OUTER JOINs 21 Optimizing from 1 sec to 0,x sec Examples (2/2) Remove relations (when extremely necessary). Assume dirty reads and manage data integrity in the application. Ex.: Hints like WITH (NOLOCK) in SQL Server. Separate the physical model in different storages to minimize the I/O impact 22 11

12 Conclusions Building high-performance systems demands a strong methodology A good system qualification becomes essential There are no silver bullets (one size fits all) Optimization is a reactive process In order to react, you need a simple and adaptable system Optimizing without evidences might result in non-desirable effects. Optimizing at the wrong timing might result in fat systems. 23 Thank you all 12