Real-World Performance Training Core Database Performance

Real-World Performance Training Core Database Performance Real-World Performance Team

Agenda 1 2 3 4 5 6 Computer Science Basics Schema Types and Database Design Database Interface DB Deployment and Access Options Application Algorithms Resource Management

Agenda 1 2 3 4 5 6 Computer Science Basics Schema Types and Database Design Database Interface DB Deployment and Access Options Application Algorithms Resource Management

Some Computer Science Basics

Database Performance Core Principles The Oracle database is a process based architecture To perform efficiently each process requires, at least: Idle CPU to schedule a running process System services such as network, Disk I/O and inter-process coordination should be fast and efficient

Database Performance Core Principles To determine acceptable CPU utilization take a probabilistic approach to the subject. If a CPU is 50% busy the chance of getting scheduled is 1 in 2 If a CPU is 66% busy the chance of getting scheduled is 1 in 3 If a CPU is 80% busy the chance of getting scheduled is 1 in 5 If a CPU is 90% busy the chance of getting scheduled is 1 in10 If the probabilities are used as indicator of the predictability of user response time, then the variance in user response time becomes noticeable at about 60-65% This has been observed in production and laboratory conditions for many years.

Database Core Principles Impact of Too Many Processes Tx/s 16000 14000 12000 10000 8000 6000 4000 2000 0 4 8 12 16 20 24 28 32 #of CPUs 1 Proc/Core 10 Proc/Core Avg 50 Proc/Core Avg 10 Proc/Core Max 50 Proc/Core Max 10 Proc/Core Min 50 Proc/Core Min

Response Time What it means Response time defines your users (customers) experience Response time is a measure of performance quality Consistency of response time is an equally important measure of performance quality If response time is not consistent, bad things happen!

Response Time By Numbers 10 Frequency X 1,000,000 9 8 7 Multi hump indicating inconsistent response times 6 5 Single distinct spike of the majority of transactions 4 GOOD BAD 3 2 1 0 0 1 2 3 4 5 6 7 8 9 10 Response Time ms Short tail Long tail

Response Time v DB Time v Latency Network Application Network Database Server Server End User Time Line Total User Response Time

Database Time Total time spent in database ON IDLE SYSTEM Recorded wait time User 1 Db file sequential read Run-queue On CPU Actual wait time ON DEGRADED SYSTEM Recorded wait time Recorded wait time Recorded wait time User 2 Run-queue Lock Wait Run-queue On CPU Db file sequential read Actual wait time On CPU On CPU On CPU Lock Wait On CPU On CPU On CPU Latch On CPU On CPU Wait Actual wait time Actual wait time

Latency - Some Important Numbers Best Block Access Speeds Block Location Access Time L2 CPU cache ~ 1 nano sec ( 10-9 ) Virtual Memory ~ 1 micro sec ( 10-6 ) NUMA Far Memory ~ 10 micro sec ( 10-6 ) Flash Memory (PCI) ~ 0.01 milli sec ( 10-3 ) Flash Memory (Networked) ~ 0.1 milli sec ( 10-3 ) Disk I/O ~ 1-10 milli sec ( 10-3 )

Response Time - Demo Observations Users experiencing poor response time Low overall system throughput Wait events observed in the database Culture of blame: Blame the database for all performance issues Development blames the DBA The DBA blames the SW/HW or system administrators

Response Time Performance Data Response Time > 4 seconds Transaction rate at 3,400 TPS

Response Time Application Server Performance Data Majority of time spent in application logic Application Server CPU at 95%

Bad Response Time Resolution Response time ~ 2 ms, Transaction rate ~ 34,000 TPS Application server changes applied here

Response Time Demo Application Server Bottleneck Data analysis shows: Small proportion of the actual response time is in the database Majority of response time spent in application logic CPU is overloaded on the application servers Potential root cause: Capacity planning mistake? Application code change last week?

Performance is always and only about time Human time is critical to the enterprise Systems performance affects business goals Human time + technology resource time Time is money Performance improvement means doing things faster

Database Time (DB Time) Total time in database calls by foreground sessions Includes CPU time, IO time and non-idle wait time DB Time <> response time Common currency for Oracle performance analysis Database time is total time spent by user processes either actively working or actively waiting in a database call.

Where is DB Time used? ADDM EM Performance page and drill downs EM Express ASH report AWR and AWR compare periods reports

Avg Active Sessions and DB Time Active sessions ASH sample count is value of active sessions function at sample times DB time is area under curve t = 1 sec DB Time t0 time t1