Tuesday, April 6, Inside SQL Server

Transcription

1 Inside SQL Server

2 Thank you

3 Goals What happens when a query runs? What each component does How to observe what s going on

4 Delicious SQL Cake

5 Delicious SQL Cake

6 Delicious SQL Cake

7 Delicious SQL Cake

8 Delicious SQL Cake

9 The Query

10 What happens now?

11 The Command Parser checks for proper syntax creates a parse tree

12 The Query Optimizer Takes parse tree from command parser uses a Cost Based Optimizer to determine the best plan

13 The optimizer only cares about optimizable statements things that can t be optimized are compiled the optimizer really only worries about things that can happen multiple ways

14 Query Plan Generation

15 Normalize & Optimize The parse tree is broken down into many smaller queries Each mini-query is optimized

16 Choosing the Query Plan In simple queries - this is frequently very fast and there is only one choice In complex queries, multiple plans are examined and the best plan is chosen based on the cost

17 How do we find the cost? If you REALLY want details, refer to Joe Chang s article The SQL Server Cost Based Optimizer

18 Otherwise, accept that it s a combination of available statistics memory requirements CPU projected disk I/O number of rows affected existing indexes

19 The optimizer is good enough. The optimizer isn t exhaustive, it just does a good enough job to produce a roughly optimized query. This is to make sure that too much time isn t spent on optimizing the queries

20 Multiple Passes Multiple passes are done with different cost thresholds - simple plans, parallelism, etc

21 After all of this...

22 An execution plan is produced from the optimized, normalized tree The plan details the tables to query, indexes to use, search predicates, triggers to call, constraints to check. It can be far more complex that the original query.

23 The Query Executor Executes the query plan step by step until the batch is complete

24 The Storage Engine Multiple duties for the Storage Engine: Provide access to data Provide transactional support Locking Backups, bulk loads, full-text index population, other fun things like that

25 Data Access One of the duties of the storage engine is managing access to data

26 Row & Index Operations Actual data access code Maintains and supports B-trees Performs actions within each row - get column first_name, write last_name Interfaces with locking code Handles LOB access Balances b-tree (balanced tree) indexes so that a search for any given record traverses the same number of levels and requires same amount of page accesses as any other read.

27 Page Operations Writes both data and log pages in 8KB chunks Tracks page use Data and log pages are written 8KB at a time (8 pages to an extent) The Page Operation code keeps track of used pages, purpose of each page, and how much space is available on each page. All data is stored on data pages, LOB pages, or index pages.

28 Versioning Operations Row versioning Online index rebuilds Snapshot Isolation

29 Transactional Operations Atomicity Write ahead logging Cross DB transactions Concurrency Supports the atomicity of data - only fully complete transactions are committed Write ahead logging ensures that the record of each transactions changes are captured before a transaction is acknowledged as committed. Delineates the boundaries of transactions. Manages concurrency through pessimistic (locking) and optimistic (row-versioning) concurrency

30 Locking Coordinates with Transactional code Resolves deadlocks Lock escalation Correctness and consistency

31 SqlOS Think relational.

32 Schedulers There s 1 scheduler per CPU, think of it like a CPU used by SQL Server Worker processes Every CPU has a scheduler associated with it, even if SQL Server is not associated with that CPU (

33 Workers A worker is thread bound to a scheduler. Workers are assigned on creation and can t move between schedulers. Created when a scheduler receives a request and there are no idle workers. Destroyed when idle for 15 minutes or when there is memory pressure. Use at least half an MB on 32-bit systems or 2MB on 64-bit systems

34 Observing Schedulers sys.dm_os_schedulers sys.dm_os_workers sys.dm_os_threads sys.dm_os_tasks sys.dm_os_waiting_tasks one row per scheduler one row per worker list of all currently running threads one row per active task one row per tasking that is waiting on a resource

35 Memory Management

36 Buffer Pool All unused memory Data cache Plan cache 8KB pages All memory not used by any other memory component Data cache and plan cache live here

37 Managing Buffers Hashing for fast look up LRU-K Periodic scans Data is quickly located by hashing dbid, file number, and page number Old data is expired using an LRU-K algorithm (least recently used - k values) K is the last time the page was accessed LRU-K can differentiate the types of pages accessed Data cache is periodically scanned from start to finish to check for dirty pages Scanning of the buffer pool, writing dirty pages, and maintaining buffer list is done by individual workers

38 Checkpoints

39 What causes checkpoints? Database owner issues a CHECKPOINT The log is > 70% full and the database is in autotruncate mode (FULL recovery, but never backed up) A shutdown of SQL Server has been requested

40 What happens during a checkpoint? SQL Server writes a checkpoint to the transaction log Checkpoint process scans the pages in the puffer pool When a dirty page is found, other physically contiguous pages are checked and written if they are dirty

41 Memory Broker Analyzes SQL Server s memory use Dynamically improves memory distribution between: buffer pool query optimizer query executor other cache structures

42 Observing Memory Anything that starts with sys.dm_os_memory... is a good place to look dm_os_memory_clerks - shows 1 row per memory clerk currently active - different types of memory in use inside SQL Server (e.g. plan cache, buffer pool, SQL CLR) dm_os_memory_cache_counters - health of each cache, provides runtime info about the cache entries

43 DMV Overview dm_exec_* dm_os_* dm_tran_* dm_io_* dm_db_* dm_exec_ : execution of user code dm_os_ : low-level system info like locking, blocking, schedulers dm_tran_ : current transactions dm_io_ : i/o activity dm_db_ : database and database objects, including indexes

44