Banner Oracle Advanced Oracle Database Administration Training Workbook

Transcription

1 Banner Oracle Advanced Oracle Database Administration Training Workbook January 2005 Using Oracle 9i for Banner 6 HIGHER EDUCATION What can we help you achieve?

2 Confidential Business Information This documentation is proprietary information of SunGard Higher Education and is not to be copied, reproduced, lent or disposed of, nor used for any purpose other than that for which it is specifically provided without the written permission of SunGard Higher Education. Prepared By: SunGard Higher Education 4 Country View Road Malvern, Pennsylvania United States of America SunGard. All rights reserved. The unauthorized possession, use, reproduction, distribution, display or disclosure of this material or the information contained herein is prohibited. In preparing and providing this publication, SunGard Higher Education is not rendering legal, accounting, or other similar professional services. SunGard Higher Education makes no claims that an institution's use of this publication or the software for which it is provided will insure compliance with applicable federal or state laws, rules, or regulations. Each organization should seek legal, accounting and other similar professional services from competent providers of the organization s own choosing. Without limitation, SunGard, the SunGard logo, Banner, Campus Pipeline, Luminis, PowerCAMPUS, Matrix, and Plus are trademarks or registered trademarks of SunGard Data Systems Inc. or its subsidiaries in the U.S. and other countries. Third-party names and marks referenced herein are trademarks or registered trademarks of their respective owners.

3 Table of Contents Section A: Introduction...9 Overview...9 Section B: Oracle Architecture Review...11 Overview...11 Oracle Architecture Review...13 Oracle Instance Review...14 Dynamic Parameters...15 Memory Buffer Cache...16 Redo Log Buffers...17 Shared Pool...18 Library Cache...19 PL/SQL and Memory...20 Memory Review PGA...21 Questions # Oracle Instance Review...23 Background Processes...24 Commonly Seen Processes...25 DBWR...26 Buffer Cache...27 LGWR...28 SMON...29 PMON...30 ARCH...31 Archiver Process...32 CKPT...33 Server Processes...34 Questions # Oracle Database Review...36 Tablespaces...37 Create Tablespace Dictionary Managed...38 Tablespace Datafiles...39 Redo Log Files...41 Control Files...42 Parameter File...43

4 Table of Contents (Continued) Section C: Managing Data...44 Overview...44 Data Management Basics...45 Storage Parameters...46 PCTFREE and PCTUSED...47 More Basics...48 Physical Requirements...50 Questions...51 Fragmentation...52 Common Methodologies...53 DBMS_SPACE_ADMIN...54 Indexes...55 Temporary Segments...56 Summary...57 Section D: Creating a Database...58 Overview...58 Creating a Database...60 New System vs. Established...61 Prepare for a Database...62 Parameter File...63 Oracle User and Directories...64 Name a database...65 Defining the Database Characteristics...66 Defining the Database...67 Choose the Method...68 Manual example...69 Database Configuration Assistant...70 Datafiles...72 Example of a Manual Creation...73 Manual Creation Steps...74 Summary...75

5 Table of Contents (Continued) Section E: Security...76 Overview...76 Securing a Database...77 Account Security...78 Password Management...79 Database System Resources...80 System Management...81 Kernel Management...82 Profile Password Options...83 OEM Password...84 User Profiles...85 Other Resource Allocation Limits...86 Viewing Profile Information...87 OEM Profiles...89 Password Encryption...90 Creating DBA Users...91 Limit SQL Commands...93 Resource Consumer Groups...94 Summary...95 Section F: Auditing the Oracle Database...96 Overview...96 Auditing the Oracle Database...97 Auditing Options Overview...98 Statement Auditing: Enabling/Disabling...99 Statement Auditing: Options Statement Auditing: Shortcuts Examples: Statement Auditing Privilege Auditing: Enabling/Disabling Example: Privilege Auditing Object Auditing: Enabling/Disabling Example: Object Auditing The Audit Trail Managing the Audit Trail Examples Viewing the Audit Trail Viewing the Audit Trail Examples Auditing with Triggers Examples Fine-Grain Object Auditing Summary...118

6 Table of Contents (Continued) Section G: Oracle Locks Overview Oracle Locks Why Locks? Preventable Phenomena Transaction Isolation Data Concurrency Model Rollback Segments and Data Consistency Concurrency/Consistency - How It Works Snapshot Too Old Errors Consistency via Locks Types of DML Locks Types of Table Locks Deadlocks Lock Waits Detecting Lock Waits Deadlock Behavior Determining Locking OEM Section H: Tuning and Monitoring Overview Tuning Performance Memory Issues OS Paging and Swapping Redo Log Buffers Private SQL Areas Shared Pool Examining Library Cache Data Dictionary Cache Pinning Objects in Memory Viewing Executing SQL Database Buffer Cache Sorting I/O Bottlenecks Chaining and Migration Redo Logs UNDO vs. Rollback Segments CPU Bottlenecks Oracle Optimization Testing SQL Efficiency Summary...177

7 Table of Contents (Continued) Section I: Referential Integrity and Constraints Overview Referential Integrity Parent Table Child Table Maintaining Referential Integrity Constraints Enabling Constraints Resizing Tables Resizing Tables with RI Export Issues Import Issues How Does RI Affect Recovery? RI and Data Manipulation RI and Table/Row Locking Section J: Oracle Net Overview Oracle Net Oracle Internet Directory Local Net Service Name LISTENER.ORA LSNRCTL Command Multiple Listeners No Listener Entry Resolving service_name Errors Log and Trace Files Tuning listener.ora, tnsnames.ora, sqlnet.ora SNMP Oracle Connection Manager Connection Filtering Summary...209

8 Table of Contents (Continued) Section K: Backup and Recovery Overview Introduction Loss of Data Redo Logs Perform Online Database Backup Types of Media Recovery Applying Archived Redo Logs Performing Incomplete Recovery Creating a New Control File Tablespace Point In Time Recovery (TSPITR) Other Methods Summary...233

9 Section A: Introduction Overview Workbook goal This course is a follow-up to the Oracle Database Administration course. It focuses on the Oracle server architecture. Intended audience Technical personnel who are responsible for the daily maintenance, support, and modification of SCT Banner. Section contents Overview...9 Page 9

10 Page 10

11 Section B: Oracle Architecture Review Overview Introduction This section provides a general review of the components of Oracle architecture. Intended audience Technical personnel who are responsible for the daily maintenance, support, and modification of SCT Banner. Objectives At the end of this section, you will be able to Review and describe the components of Oracle architecture Review and describe Oracle processes Prerequisites To complete this section, you should have completed OR101 (Introduction to Oracle) exposure to SQL and SQL* Plus (OR102 Advanced PL/SQL and Database Objects is recommended) completed OR103 (Oracle Database Administration I) Page 11

12 Section B: Oracle Architecture Review Overview (Continued) Section contents Oracle Architecture Review...13 Oracle Instance Review...14 Dynamic Parameters...15 Memory Buffer Cache...16 Redo Log Buffers...17 Shared Pool...18 Library Cache...19 PL/SQL and Memory...20 Memory Review PGA...21 Questions # Oracle Instance Review...23 Background Processes...24 Commonly Seen Processes...25 DBWR...26 Buffer Cache...27 LGWR...28 SMON...29 PMON...30 ARCH...31 Archiver Process...32 CKPT...33 Server Processes...34 Questions # Oracle Database Review...36 Tablespaces...37 Create Tablespace Dictionary Managed...38 Tablespace Datafiles...39 Redo Log Files...41 Control Files...42 Parameter File...43 Page 12

13 Section B: Oracle Architecture Review Oracle Architecture Review Overview In this section, we will review the RDBMS part of an Oracle system. What makes up an Instance What makes up a Database Instance Memory structures are allocated. Oracle background processes are started. Database Physical files. Logical collection of data treated as a unit. Page 13

14 Section B: Oracle Architecture Review Oracle Instance Review Memory Structures There are two types of Oracle structures that are used in memory when an Oracle instance is running: SGA - System Global Area PGA - Program Global Area SGA The SGA is a memory structure allocated at instance startup. It contains many sub-structures that are used to handle various database tasks such as read/write buffers, transaction history, and code execution. The characteristics of the SGA are defined to Oracle by setting parameters in the init.ora file. Dynamic SGA As of 9i, the SGA can be dynamically managed. A maximum size is set in init.ora, sga_max_size All SGA parameters can be manipulated so long as sga_max_size is not met The SGA will be tracked in granules. Granules The granule size will be based on the sga_max_size. Less than 128 mb, granule = 4 mb Greater than 128 mb, granule = 16mb All SGA components can be modified so long as sufficient granules are left. Oracle will round up if values are not on granule boundaries. Page 14

15 Section B: Oracle Architecture Review Dynamic Parameters Non-modifiable parameters Not all SGA values can be dynamically modified. The following cannot be modifed dynamically: Large Pool Java Pool Redo Log buffer Note: More in Section H (Tuning and Monitoring). Memory structures - diagram System Global Area (SGA) Page 15

16 Section B: Oracle Architecture Review Memory Buffer Cache Buffer cache Divided into blocks the same size as the blocks in the tablespaces datafiles. Size and number defined in the initialization Parameter File. Holds copies of data blocks read from disk. Each block size has its own buffer cache Contents As of 9i, can be defined for each block size up to 5 different sizes. Contains the DIRTY, FREE and PINNED buffers. Contains two (2) lists, WRITE and LRU (Least Recently Used) Page 16

17 Section B: Oracle Architecture Review Redo Log Buffers Redo log buffers Stores the REDO ENTRIES (changes to the database) Changed data is moved from the database buffer cache to the redo log buffer then to the redo log files. Page 17

18 Section B: Oracle Architecture Review Shared Pool Contents Library Cache Data Dictionary Cache Control structures Reusable Runtime Memory Diagram Page 18

19 Section B: Oracle Architecture Review Library Cache Library cache SHARED by all users. Shared and Private SQL areas for executed statements. PL/SQL Procedures and Packages. Allocation and Reuse of Memory Shared SQL PARSE TREES for SQL statements. EXECUTION plans for SQL statements Size dependent on complexity Private SQL Each session has one. Persistent and Runtime areas. Cursors. Page 19

20 Section B: Oracle Architecture Review PL/SQL and Memory PL/SQL Program Units Procedures Functions Packages Anonymous blocks Database triggers Allocation and Reuse of Memory Every SQL statement submitted, Oracle automatically allocates memory. Checks for existence. Allocate private SQL area. May flush the shared pool. Page 20

21 Section B: Oracle Architecture Review Memory Review PGA PGA The PGA is established at the time that a user makes a connection. It will differ in its contents depending on whether the method of connection is through a shared server process (MTS) or a dedicated server process. Can be limited by automated PGA Page 21

22 Section B: Oracle Architecture Review Questions #1 Questions What is the name of the Main memory structure? What are the contents of it? Which part of this memory structure is the most affected by changing the block size? What considerations must be made when resizing Oracle's memory? Page 22

23 Section B: Oracle Architecture Review Oracle Instance Review Memory structures diagram Page 23

24 Section B: Oracle Architecture Review Background Processes Background processes Oracle manages these memory structures with several background processes Depending on the Oracle system installed, the numbers and types of these processes will vary. Exclusive verses shared (Parallel-8i,RAC-9i). Distributed verses Replicated (DD, AdvRep). Others (Standby, Parallel query..) Utilization Background processes are utilized by Oracle to maximize performance, tune, recovery and accommodate many users. These processes are made to handle the common overhead tasks that must be performed to read data in, write data out, maintain data consistency, and provide a mechanism for recovery. They are the foundation of Oracle Page 24

25 Section B: Oracle Architecture Review Commonly Seen Processes Standard DBWR Database Writer LGWR - Log Writer SMON - System Monitor PMON - Process Monitor CKPT Checkpoint ( as of 9i) Optional Dnnn - Dispatcher (MTS) Snnn - Server (dedicated) RECO - Distributed Recovery LCKn - Lock Manager SNPn - Job Queue Processes Pnnn - Parallel Query Page 25

26 Section B: Oracle Architecture Review DBWR Functions It performs all writes to the datafiles from the buffers. If it is not configured properly, data can be swapped out and written to disk. It contains lists that can be tuned for optimal performance, dirty and LRU Dirty list is used by DBWR, LRU is used by the server process When does DBWR write? Every three seconds Shutdown normal, immediate or transactional When the server process says, 'wake up and write, I need a buffer!' An imposed threshold has been met. DBWR diagram Page 26

27 Section B: Oracle Architecture Review Buffer Cache Organization Buffers are organized by two lists: Write list Holds the dirty buffers that have not been moved to disk yet. LRU list Free buffers available for use Pinned buffers currently in use Dirty buffers not yet moved to the write list. Page 27

28 Section B: Oracle Architecture Review LGWR Log Writer The Log Writer is responsible for managing the Redo log buffer. It writes all redo entries since the last time it wrote. The LGWR by default records the checkpoint entries in the header file of the datafiles (except if CKPT is going). There is only one redo allocation latch. Only one user process at a time is serviced. LGRW diagram Page 28

29 Section B: Oracle Architecture Review SMON Functions The System Monitor has a variety of tasks that it does on behalf of Oracle. At instance startup, SMON will perform any needed instance recovery. SMON also handles space management duties such as cleaning up unused Temp segments and coalesces contiguous free extents. SMON recovers dead transactions in the event of file-read or offline errors with files. SMON - diagram Page 29

30 Section B: Oracle Architecture Review PMON Functions The Process Monitor performs process recovery whenever a user process terminates. These include cleaning up any buffer cache and freeing resources the process was using such as locks. It also is responsible for restarting failed dispatcher processes. Stays asleep until it is needed PMON - diagram Page 30

31 Section B: Oracle Architecture Review ARCH Functions The Archiver is responsible for getting the online redo log files copied to a designated storage device. This process is an optional Oracle process, but must be configured for any database that contains data that must be recovered if lost. ARCH must be turned on by entries in the init.ora file and the database must be put into ARCHIVELOG mode for this process to run. ARCH - diagram Page 31

32 Section B: Oracle Architecture Review Archiver Process Destinations Multiple destinations can be configured Multiplexed Standby Other locations ARCHIVE LOG LIST Archive log mode NOARCHIVELOG Only can do instance recovery RMAN can still be used ARCHIVEMODE Can recover all committed transactions Can do an online backup Utilize LOGMINER utility RMAN utility Standby database can be used. Page 32

33 Section B: Oracle Architecture Review CKPT Functions Optional process to help the LGWR. Writes to the control file every three (3) seconds the position of the redo logs. Updates the control file and each datafile's header file the checkpoint and SCN numbers. As of 9i, it is automatically started Without CKPT - diagram With CKPT - diagram Page 33

34 Section B: Oracle Architecture Review Server Processes Functions Server processes, sometimes called shadow processes, run on the database server on behalf of the users of the database. These processes can be either dedicated or shared. The tasks include reading data from datafiles into cache parsing and executing SQL returning results to the application. Diagram Page 34

35 Section B: Oracle Architecture Review Questions #2 Questions Which background processes are required? Which background process(es) only start when users log on? Which background process is involved in instance recovery? Which background process updates the header files? Page 35

36 Section B: Oracle Architecture Review Oracle Database Review Database definition The database part of an Oracle instance deals with the storage of the data. Types There are two ways one needs to look at the storage of Oracle data. Logical Storage Tablespaces Objects Physical Storage Datafiles Redo Logs Control Files Parameter File Page 36

37 Section B: Oracle Architecture Review Tablespaces Types System defined at database creation Temporary primarily for sorting Rollback (UNDO-9i) Read write Read only Transportable Functions Oracle uses this structure to divide up the database logically so the DBA can: Control disk space allocation Assign quotas Control grouping of objects for: o Performance o Maintenance o Recovery o Accessibility Page 37

38 Section B: Oracle Architecture Review Create Tablespace Dictionary Managed Syntax and diagram CREATE TABLESPACE DATAFILE DEFAULT STORAGE Big_indx '/u01/oradata/sid/big_indx.dbf' SIZE 500M storage clause... Page 38

39 Section B: Oracle Architecture Review Tablespace Datafiles Types Dictionary managed Default storage clause used Locally managed Uniform extent Autoallocate Oracle managed files (OMF) Oracle manages the allocation, growth and removal Default file location defined in the init.ora Dictionary Managed Datafile These datafiles require monitoring not only for space, but for segment and extent usage. Oracle maintains in the data dictionary several tables to monitor storage. Default storage clause will need to be reviewed and monitored for extent management. Locally Managed File Oracle creates a bit-mapped header file in the first 64 kb of the first datafile of a tablespace to manage the datafile. These files can be in two different formats: Uniform extent size Autoallocate You cannot use the default storage clause. Oracle Managed File A new feature as of 9i. Oracle will name and create as per init.ora parameters: DB_CREATE_FILE_DEST DB_CREATE_ONLINE_LOG_DEST_n where _n is the location for each member. If a control file location is not listed, it uses the online redo log location. Page 39

40 Section B: Oracle Architecture Review Tablespace Datafiles (Continued) Datafiles Although a database may have a single datafile, a more usual configuration will be many datafiles, organized into many tablespaces, all associated with the database. Page 40

41 Section B: Oracle Architecture Review Redo Log Files Redo log files Redo log files are used to store the change history of all transactions as they occur against the database. These can either be committed transactions or non-committed transactions. Since they contain a copy of all changes made to the database they can be used to reconstruct the database in cases of failure. The redo entries are grouped by transaction and a System Change Number (SCN) is assigned to each set of redo entries. Diagram Page 41

42 Section B: Oracle Architecture Review Control Files Functions Oracle uses control files to keep track of the database during normal operation and as such must be available for writing at all times. The database name and timestamp is recorded at startup. The physical structure of the database, the datafiles and the redo logs, are recorded in the control file also. Finally, the control file is where the checkpoints are recorded. Diagram Page 42

43 Section B: Oracle Architecture Review Parameter File Parameter file The parameter file, the initsid.ora file is the only file that the DBA edits directly. As of 9i, the parameter file can be managed by Oracle, called spfilesid.ora. Usage The parameter file is used by Oracle to define and configure the database instance. Before Oracle 9i, changes required the database be restarted. There are a multitude of dynamic parameters. $ORACLE_ADMIN/<SID>/pfile/initSID.ora Parameters DB_NAME DB_BLOCK_SIZE DB_CACHE_SIZE DB_#K_CACHE_SIZE PROCESSES LOG_BUFFER CONTROL_FILES COMPATIBLE OPTIMIZER_MODE SGA_MAX_SIZE SHARED_POOL_SIZE SORT_AREA_SIZE SORT_AREA_RETAINED_SIZE PGA_AGGREGATE_TARGET DISPATCHERS SHARED_SERVERS OPEN_CURSORS SERVICE_NAMES INSTANCE_NAME DB_DOMAIN LOG_ARCHIVED_DEST_# LOG_CHECKPOINT_INTERVAL LOG_CHECKPOINT_TIMEOUT Page 43

44 Section C: Managing Data Overview Introduction This section discusses techniques of managing and maintaining data. Intended audience Technical personnel who are responsible for the daily maintenance, support, and modification of SCT Banner. Objectives At the end of this section, you will be able to describe: Basics of Space Storage Storage Parameters Default Storage Parameters Fragmentation Methodologies for maintaining Index management Temporary segments Prerequisites To complete this section, you should have completed OR101 (Introduction to Oracle) exposure to SQL and SQL* Plus (OR102 Advanced PL/SQL and Database Objects is recommended) completed OR103 (Oracle Database Administration I) Section contents Data Management Basics...45 Storage Parameters...46 PCTFREE and PCTUSED...47 More Basics...48 Physical Requirements...50 Questions...51 Fragmentation...52 Common Methodologies...53 DBMS_SPACE_ADMIN...54 Indexes...55 Temporary Segments...56 Summary...57 Page 44

45 Section C: Managing Data Data Management Basics Basics First and foremost, tune memory. The amount of memory will drive the amount of I/O that can be handled. Effective management of any system requires strategic planning, quality control, and action to resolve out-of-control parts of the system. Establish a command center database to monitor the other databases. This will minimize the impact while collecting statistics. Command database Must be planned and sized to fit your needs Create it after you have fully defined what it is you want to monitor Start by monitoring the most common monitored objects and thresholds o Extents o Segments o Rate of change Basic components Database blocks small space element of a database. Extent - Contiguous allocation of database blocks. Segment - Logical collection of extents that make up a table, cluster, index, temporary segment, or rollback segment, etc. Page 45

46 Section C: Managing Data Storage Parameters Parameters These parameters define storage attributes of database segments. Initial - allocates the first extent of space Next - size of the extent allocated after the initial Pctincrease - controls the rate of growth of extents beyond the second one. Minextents - Number of extents allocated during the initial creation of the segment Maxextents - Limits the total number of extents the segment can create Optimal - sets the optimal size in bytes for a rollback segment Page 46

47 Section C: Managing Data PCTFREE and PCTUSED PCTFREE Reserved portion of the data block that is used for later updates to rows in that block Tied to the way updates occur If it is too low, the number of chained rows will steadily increase Should be set large enough to hold several records Defaults value is 10% PCTUSED Used to determine when a block is added to and retained on the free list. A block will not be back on the free list until it's % used falls below PCTUSED. Default value is 40% If PCTUSED is too high and inadequate space is available for updates row migration. PCTUSED + PCTFREE should = 80% Page 47

48 Section C: Managing Data More Basics Partitioning Tablespace - The tablespace to store the segment Partitioning - Feature of Oracle8 that allows a segment to be separated into multiple segments, each existing in a single or multiple tablespaces Example Create table EMP ( EMPID NUMBER, F_Name VARCHAR2(25), L_Name VARCHAR2(25), MI VARCHAR2(1), SSN VARCHAR2(11) )Pctfree 10 Pctused 40 storage (Initial 2K Next 2K Pctincrease 0 Maxextents unlimited) Tablespace USERS; Default storage Maintained at the tablespace level Used in the absence of segment level storage parameters Values include initial, next, minextents, maxextents, pctincrease Only available with dictionary managed tablespaces. Segment parameters override default storage parameters with the exception of locally managed tablespace in Oracle8i Locally managed tablespaces force the use of uniform extent sizes o Does not enforce it with imports Example Create tablespace USERS datafile '/u04/oradata/devl/users_devl_01.dbf' size 100M default storage (initial 100K next 100K maxextents 99 pctincrease 1); ** Note : pctincrease is nonzero to allow for automatic coalescing by smon. Page 48

49 Section C: Managing Data More Basics (Continued) Autoextend vs. adding Both can be executed while the database is up. Both can extend the tablespace and be ready for use immediately. You can Autoextend an existing datafile autoextend on next 50M maxsize 500M Your disk space will shrink dynamically. Adding a datafile Adding a datafile is under your control. Alter tablespace add datafile '/u01/oradata/seed/newfile_seed_02.dbf' size 500M; Page 49

50 Section C: Managing Data Physical Requirements RAID Important to have for an Oracle system. Jbod Lowest data cost low overhead. 0 low data cost load balancing. 1, 0+1 high data cost most expensive. 5 Highest data cost safe. RAID info It takes 4 I/Os for 1 write (read data, read parity,write data, write parity). Minimum of 3 disks plus 66-87% of space. The more disks, the higher the gain. RAID vs RAW RAID provides a safety net, but not without a cost: Mirror one whole drive is wasted Stripping/parity partial amount wasted RAW bypasses the UNIX buffer cache, data is transferred directly from the buffer cache Required for RAC Cannot be used for archive logs Standard backup strategies cannot be used. Page 50

51 Section C: Managing Data Questions Questions Row migration is caused by What? What is the default value for PCTUSED? Can a locally managed tablespace have the autoallocate option? Is the only time to coalesce is when pctincrease is non-zero? You are setting up a RAC system, what is required to do this? Page 51

52 Section C: Managing Data Fragmentation Free space vs. Segment fragmentation Free space fragmentation is determined by the number of contiguous free space chunks within a tablespace Segment fragmentation is determined by the number of extents relative to the number of segments within a tablespace. Monitoring Space Usage Monitoring any system for any changes takes time. Start a simple rate of change in free space and extent size. Develop a rate of change in these values that will be acceptable at your location. Free Space Fragmentation Caused by dropping or truncating segments. Categorized as usable or unusable o a usable free space chunk is one large enough to be used by the growth of an existing segment o unusable is a free space chunk too small to be used by an existing segment Monitor DBA_FREE_SPACE for usage and rate of change in the datafile and/or tablespace Segment Fragmentation Caused by segments requiring multiple extents Using a uniform extent size (ie. Locally managed tablespaces) Monitor dba_segments for segment size and management. Purge any unwanted segments Coalesce free space. Page 52

53 Section C: Managing Data Common Methodologies Appropriate sizing of segments often complicated and time consuming bound to a finite period of time for growing tables. Rebuilds (imp/exp) eliminate existing fragmentation by using imp/exp Uniform Extent sizes eliminates unusable free space fragmentation does not eliminate segment fragmentation minimizes maintenance efforts extent sizes should be carefully chosen to achieve best performance supported by locally managed tablespaces in Oracle8i Page 53

54 Section C: Managing Data DBMS_SPACE_ADMIN Functions Provided PL/SQL package to monitor locally managed tablespaces. Provides defect analysis and repair functionality Can be used to migrate a dictionary managed tablespace to a local one. Best tool to manage the bit-map header file. Migrating Local/dictionary You can convert dictionary managed to local managed. Caution it will not convert existing objects to local Execute dbms_space_admin: Execute sys.dbms_space_admin.tablespace_migrate_to_local('tablespace'); Page 54

55 Section C: Managing Data Indexes Functions Should be separated out of data tablespaces. Easily created with import command: Imp system/manager full=y direct=y indexfile=d:\oracle\admin\seed\exp\seed_index.sql file=seed_full.dmp Must be rebuilt to salvage unused leaves Index Rebuild Use the alter index command to rebuild the index. Easy way to move a single index from one tablespace to another Reclaims wasted space caused by deleting rows, but it does not reduce the high water mark. Can use the deallocate unused clause of the alter index command Page 55

56 Section C: Managing Data Temporary Segments Functions Used by Oracle to house sort data from queries that have large results Degrades performance because lots of I/O Create the temporary tablespace with temporary segments. o Prevents users from storing permanent data in temporary tablespace Types Temporary segments in permanent tablespaces Hard on SMON coalescing the segments Purged when the transaction completes Temporary segments in temporary tablespaces All segments are released except for first segment Set uniform size with initial extent = next extent. Page 56

57 Section C: Managing Data Summary Summary Basics of Space Storage Storage Parameters Default Storage Parameters Fragmentation Methodologies for maintaining Index management Temporary segments Page 57

58 Section D: Creating a Database Overview Introduction This section discusses the process of creating an Oracle database. Intended audience Technical personnel who are responsible for the daily maintenance, support, and modification of SCT Banner. Objectives At the end of this section, you will be able to perform the steps to create an Oracle database: Prepare the O/S Determine the ORACLE SID Create a parameter file Start the instance Execute the CREATE DATABASE SQL command Build the data dictionary and PL/SQL packages Prerequisites To complete this section, you should have completed OR101 (Introduction to Oracle) exposure to SQL and SQL* Plus (OR102 Advanced PL/SQL and Database Objects is recommended) completed OR103 (Oracle Database Administration I) Page 58

59 Section D: Creating a Database Overview (Continued) Section contents Creating a Database...60 New System vs. Established...61 Prepare for a Database...62 Parameter File...63 Oracle User and Directories...64 Name a database...65 Defining the Database Characteristics...66 Defining the Database...67 Choose the Method...68 Manual example...69 Database Configuration Assistant...70 Datafiles...72 Example of a Manual Creation...73 Manual Creation Steps...74 Summary...75 Page 59

60 Section D: Creating a Database Creating a Database Summary Requires a lot of forethought One of the most unprepared aspects Once created, several options cannot be changed unless the database is re-created. To name only a few: o Block size o System tablespace format Steps Six general steps to creating a database Prepare the Operating System Create a parameter file Start the instance Create the database Build the data dictionary Default PL/SQL procedures Creation methods New Oracle Install verses established. OEM Database configuration assistant Manually, either with scripts or by command line. Page 60

61 Section D: Creating a Database New System vs. Established New vs. established New database systems will be the template for future ones. Oracle creates a test database with software installation Good source of parameter changes. Good reference for supporting file(s) Page 61

62 Section D: Creating a Database Prepare for a Database Prepare the Operating System An Oracle database consists of datafiles, redo log files, and control files Prior to creating a database, the location(s) of these files should be defined The systems administrator should create the appropriate directories for these files Operating System Block size is a multiple of the OS block size Spread the database across devices Separate high access datafiles from others o Temp and UNDO (rollback) o Redo and Archives o Data and Index Datafile sizing Page 62

63 Section D: Creating a Database Parameter File Parameter File Another part of creating a database that is not reviewed well. Two options as of 9i, ASCII vs. binary If a parameter is not stated, Oracle will define system defaults Several new options will greatly affect performance New 9i Performance Parameters Cache and buffers will greatly affect performance. Several new parameters should be reviewed. (covered in Performance and Tuning) Cursor_sharing Pga_aggregate_target Sga_max_size Statistics_level Workarea_size_policy Db_cache_size Db_keep_cache_size Db_recycle_cache_size Db_cache_advice Db_##k_cache_size Page 63

64 Section D: Creating a Database Oracle User and Directories Directory owner The owner of the directories should be the Oracle software owner (typically "oracle") On Unix systems, the DBA group should also be the group owner of the directories VMS granted rights identifier ora_sid_dba NT member of ora_dba group The Oracle software owner should be the only user that has write access to the directories. Page 64

65 Section D: Creating a Database Name a database Set the name of the database Most operating systems can run more than one database environment concurrently. Each database should be labeled with a unique identifier. Known as the ORACLE SYSTEM IDENTIFIER Referred to as the ORACLE SID Defining the Oracle SID Identifies which database a user will connect to when executing an Oracle application. Unix/NT - ORACLE SID is designated by an environment variable, $ORACLE_SID VMS - ORACLE SID is designated by a logical ORA_ROOT Page 65

66 Section D: Creating a Database Defining the Database Characteristics Parameter file Defines the database type and behavior. Contains attributes of the SGA that are customizable by the DBA Can be either ASCII or Binary. OEM creates the binary version, spfile Page 66

67 Section D: Creating a Database Defining the Database DBA considerations The DBA must evaluate all parameters, defined or not. Memory is the most critical. It usually causes most of the database errors Determine the purpose of the database, read only, warehouse, standby, OLAP or DSS. Page 67

68 Section D: Creating a Database Choose the Method OEM OEM will generate everything, even creating the system objects. Will create all users and tablespaces Create and populate the System Tablespace Manually create the database with scripts. All options must be well defined. Page 68

69 Section D: Creating a Database Manual example Example Once the Oracle instance is started, execute a CREATE DATABASE SQL command. Below is an abridged syntax: CREATE DATABASE <database name> DATAFILE '<full path to datafile>' size <datafile size>m LOGFILE GROUP <group number> '<full path to logfile>' size <logfile size>m Manual with OMF Init.ora DB_CREATE_FILE_DEST Default location for all datafiles. Also used for UNDO, temporary, online redo and control files if no other parameter used. o SYSTEM 100mb auto extensible o Redo 100mb auto extensible o UNDO 10mb auto extensible All normal tablespace and database commands can be used at any time. The FILE_DEST can be altered dynamically. Page 69

70 Section D: Creating a Database Database Configuration Assistant DBCA DBCA has grown up! Can create a database Configure database options Delete an entire database Manage templates (New in 9i). Can save your creation scripts to be used for another database. DBCA Templates Four templates are provided: Data warehouse General purpose New database Transaction processing Purpose Provides a simple mechanism to modify Initialization parameters. List all of them, default values are provided and option to include it or not in the init.ora Memory can be easily configured Default file locations can be defined Page 70

71 Section D: Creating a Database Database Configuration Assistant (Continued) Diagram Page 71

72 Section D: Creating a Database Datafiles Purpose The first datafile defined is for the SYSTEM tablespace. As of 9i release 2, the SYSTEM tablespace can now be locally managed Care in to size and type must be considered Storage locations are defined. Types Dictionary managed Default storage clause used Locally managed Uniform extent Autoallocate Oracle managed files (OMF) Oracle manages the allocation, growth and removal Default file location defined in the init.ora Page 72

73 Section D: Creating a Database Example of a Manual Creation Example code SVRMGR> create database SEED 2> datafile '/oradata/seed/system_seed_01.dbf' size 200M 3> logfile group 1 '/oradata/seed/log_seed_01.rdo' size 3M, 4> group 2 '/oradata/seed/log_seed_02.rdo' size 3M 5> / Statement processed. Page 73

74 Section D: Creating a Database Manual Creation Steps Steps Once the CREATE DATABASE command is executed, a new database is created. The control files associated with this database are also created in the location designated by the parameter file The system tablespace is created. Scripts The scripts catalog.sql and catproc.sql create the data dictionary and system PL/SQL packages. The former script MUST be run first. These scripts are located under: $ORACLE_HOME/rdbms/admin (unix/nt) ORA_ROOT:[rdbms] (VMS) First Users Created Once these scripts have run, two default users are created in the database: SYS (initial password CHANGE_ON_INSTALL) SYSTEM (initial password MANAGER) DBCA will require that you use different passwords than MANAGER and CHANGE_ON_INSTALL. Page 74

75 Section D: Creating a Database Summary Summary Steps to create an Oracle database Prepare the O/S Determine the ORACLE SID Create a parameter file Start the instance Execute the CREATE DATABASE SQL command Build the data dictionary and PL/SQL packages Page 75

76 Section E: Security Overview Introduction This section discusses maintaining security over an Oracle database. Intended audience Technical personnel who are responsible for the daily maintenance, support, and modification of SCT Banner. Objectives At the end of this section, you will be able to Monitor Passwords by several authentication methods Limit resources with Profiles, consumer groups and Product User Profile table. Manage users' accesses by several means Prerequisites To complete this section, you should have completed OR101 (Introduction to Oracle) exposure to SQL and SQL* Plus (OR102 Advanced PL/SQL and Database Objects is recommended) completed OR103 (Oracle Database Administration I) Section contents Securing a Database...77 Account Security...78 Password Management...79 Database System Resources...80 System Management...81 Kernel Management...82 Profile Password Options...83 OEM Password...84 User Profiles...85 Other Resource Allocation Limits...86 Viewing Profile Information...87 OEM Profiles...89 Password Encryption...90 Creating DBA Users...91 Limit SQL Commands...93 Resource Consumer Groups...94 Summary...95 Page 76

77 Section E: Security Securing a Database Objectives Understand Oracle system resources Password management Limit resource usage through creation and assignment of user profiles Limit commands Monitor system resource usage and assignment Develop user grouping Topics Account Security System-level Privileges Object Security Auditing (Discussed in Section F) Limiting Resources Using OEM Page 77

78 Section E: Security Account Security Concerns First and foremost, change all the passwords on your system on a regular schedule Change all accounts that have high levels of access (role DBA). Lock accounts that are not accessed regularly. Users are your worst offenders. If it is there, they will try it. Limit resources and privileges Page 78

79 Section E: Security Password Management Purpose To manage direct or indirect access Standards can be enforced with Profiles Usage can be managed with password aging, limiting and history Can be defined internally and externally Encrypt password passage from client to database server. Profiles Manage expiration, reuse and complexity of passwords Place limits on the amount of system and database resources However, profiles are reactive no action takes place until the user has exceeded the limit. Enabling Resource Limits Add the Initialization parameter to the SID_INIT.ORA file RESOURCE_LIMIT=TRUE Alter the system in a live database SVRMGR> ALTER SYSTEM SET RESOURCE_LIMIT=TRUE; System altered. Note: ALTER SYSTEM system privilege must be granted to the issuing user for this command. Page 79

80 Section E: Security Database System Resources Kernel Parameters CPU TIME I/O OPERATIONS IDLE TIME CONNECT TIME PRIVATE MEMORY (PGA) CONCURRENT SESSIONS Page 80

81 Section E: Security System Management Session Level limits CPU_PER_SESSION Total CPU time measured in hundredths of seconds SESSIONS_PER_USER Number of concurrent session allowed per username Page 81

82 Section E: Security Kernel Management Session Level limits CONNECT_TIME Elapsed connect time measured in minutes IDLE_TIME Periods of inactive time measured in minutes LOGICAL_READS_PER_SESSION Number of data blocks (physical and logical reads) PRIVATE_SGA Private space in SGA measured in bytes (MTS only) Call Level limits CPU_PER_CALL CPU time per call in hundredths of seconds LOGICAL_READS_PER_CALL Number of data blocks read per call Page 82

83 Section E: Security Profile Password Options Password Parameters FAILED_LOGIN_ATTEMPTS Specifies the number of failed attempts to log in to the user account before the account is locked. PASSWORD_LIFE_TIME Limits the number of days the same password can be used for authentication. The password expires if it is not changed within this period, and further connections are rejected. PASSWORD_REUSE_MAX Specifies the number of password changes required before the current password can be reused. If you set PASSWORD_REUSE_MAX to an integer value, then you must set PASSWORD_REUSE_TIME to UNLIMITED. PASSWORD_LOCK_TIME Specifies the number of days an account will be locked after the specified number of consecutive failed login attempts. PASSWORD_GRACE_TIME Specifies the number of days after the grace period begins during which a warning is issued and login is allowed. If the password is not changed during the grace period, the password expires. PASSWORD_VERIFY_FUNCTION Allows a PL/SQL password complexity verification script to be passed as an argument to the CREATE PROFILE statement. Oracle provides a default script, but you can create your own routine or use third-party software instead. (utlpwdmg.sql) o Function is the name of the password complexity verification routine. o NULL indicates that no password verification is performed. PASSWORD_REUSE_TIME Specifies the number of days before which a password cannot be reused. If you set PASSWORD_REUSE_TIME to an integer value, then you must set PASSWORD_REUSE_MAX to UNLIMITED. Page 83

84 Section E: Security OEM Password Screen image Page 84

85 Section E: Security User Profiles Creating a user profile SQL> CREATE PROFILE my_profile LIMIT 2 SESSIONS_PER_USER 5 3 CPU_PER_CALL UNLIMITED 4 IDLE_TIME 60; Profile created. Alter a user profile SQL> ALTER PROFILE my_profile LIMIT 2 SESSIONS_PER_USER 2 3 CPU_PER_SESSION IDLE_TIME 30 5 LOGICAL_READS_PER_CALL 1000; Profile created. Delete a user profile SQL> DROP PROFILE my_profile; Profile dropped. Assign a user profile with CREATE USER SQL> CREATE USER new_user IDENTIFIED BY 2 new_passwd 3 DEFAULT TABLESPACE users 4 TEMPORARY TABLESPACE temp 5 PROFILE my_profile; User created. Assign a user profile with ALTER USER SQL> ALTER USER new_user PROFILE 2 my_profile; User altered. Page 85

86 Section E: Security Other Resource Allocation Limits Composite limit Weighted sum of four resource limits: CPU_PER_SESSION CONNECT_TIME PRIVATE_SGA (only available with shared server enabled) LOGICAL_READS_PER_SESSION Composite limit example SQL> CREATE PROFILE my_comp_limit LIMIT 2 COMPOSITE_LIMIT ; Profile created. Resource Costs Used to weight CPU_PER_SESSION, CONNECT_TIME, PRIVATE_SGA, and LOGICAL_READS_PER_SESSION Resource cost example SQL> ALTER RESOURCE COST 2 CPU_PER_SESSION LOGICAL_READS_PER_SESSION CONNECT_TIME 10; Resource cost altered. Page 86

87 Section E: Security Viewing Profile Information View assigned profiles for all users SQL> select username,profile 2 from sys.dba_users; USERNAME PROFILE SYS DEFAULT SYSTEM DEFAULT FIMSARC DEFAULT GENERAL DEFAULT SATURN DEFAULT ALUMNI DEFAULT FIMSMGR DEFAULT PAYROLL DEFAULT View all profile data First, set some column widths to stop the wrap. SQL> COL profile SQL> COL resource_name SQL> COL limit FORMAT A20 FORMAT A30 FORMAT A20 SQL> select profile, resource_name, limit 2 from sys.dba_profiles 3 where resource_type = 'KERNEL' 4 order by profile 5 / PROFILE RESOURCE_NAME LIMIT DEFAULT COMPOSITE_LIMIT UNLIMITED DEFAULT SESSIONS_PER_USER UNLIMITED DEFAULT CPU_PER_SESSION UNLIMITED DEFAULT CPU_PER_CALL UNLIMITED DEFAULT LOGICAL_READS_PER_SESSION UNLIMITED DEFAULT LOGICAL_READS_PER_CALL UNLIMITED DEFAULT IDLE_TIME UNLIMITED DEFAULT CONNECT_TIME UNLIMITED DEFAULT PRIVATE_SGA UNLIMITED Page 87

88 Section E: Security Viewing Profile Information (Continued) View all profile data (cont.) SQL> select profile, resource_name, limit 2 from sys.dba_profiles 3 where resource_type = 'PASSWORD' 4 order by profile 5 / PROFILE RESOURCE_NAME LIMIT DEFAULT FAILED_LOGIN_ATTEMPTS UNLIMITED DEFAULT PASSWORD_LIFE_TIME UNLIMITED DEFAULT PASSWORD_REUSE_TIME UNLIMITED DEFAULT PASSWORD_REUSE_MAX UNLIMITED DEFAULT PASSWORD_VERIFY_FUNCTION UNLIMITED DEFAULT PASSWORD_LOCK_TIME UNLIMITED DEFAULT PASSWORD_GRACE_TIME UNLIMITED View resource cost information SQL> select * from resource_cost; RESOURCE_NAME UNIT_COST CPU_PER_SESSION 0 LOGICAL_READS_PER_SESSION 0 CONNECT_TIME 0 PRIVATE_SGA 0 Page 88

89 Section E: Security OEM Profiles Screen image Page 89

90 Section E: Security Password Encryption Password encryption When connection is made to the database server from via Oracle Net, Oracle passes the password in unencrypted format Set ORA_ENCRYPT_LOGIN to true in the SQLNET.ORA file This is a performance hit. Each password must be altered before transmitting it to the server. Page 90

91 Section E: Security Creating DBA Users Methods These users can be created one of three ways: OS-authenticated Password-file authenticated Oracle authenticated OS-Authenticated DBA First create user using operating system commands Start administrative tools User Manager in Windows /etc/passwd file in UNIX (or admin tool in CDE) Authorize facility in VMS OS-Authenticated Groups Assign the OS user to one of these groups/rights ID: DBA UNIX ORA_DBA Windows ORA_DBA OpenVMS (rights identifier) OS-Authenticated setup Create the user as o Create user OPS$... Identified externally Set the REMOTE_LOGIN_PASSWORD to NONE in the init.ora file Grant OSDBA or OSOPER role to the user. Password File Authenticated Create the user: o Create user identified by Grant the appropriate privileges SYSDBA or SYSOPER to the user Create the password file Change REMOTE_LOGIN_PASSWORDFILE for the appropriate login Page 91

92 Section E: Security Creating DBA Users (Continued) ORAPWD Create the password file $ORACLE_HOME/bin/orapwd file=<fname> password=<password> entries=<users> Fname is the name of the password file. It is created in $ORACLE_HOME/database Password is the password for sys Entries is the maximum number of distinct DBAs and OPERs ORAPWD - setup Edit the init.ora file and set: REMOTE_LOGIN_PASSWORDFILE to EXCLUSIVE for users connecting as themselves REMOTE_LOGIN_PASSWORDFILE to SHARED for users connecting as SYS or SYSTEM. Grant SYSOPER or SYSDBA privileges to each user who needs to perform database administration. Page 92

93 Section E: Security Limit SQL Commands Limitations Maybe you do not want your users HOSTing out to the underlying operating system or CONNECTing as another user. Run pupbld.sql to set up several tables and views for Product User Profiles. Enter the product, userid, command and DISABLED in PRODUCT_USER_PROFILE TABLE. Example of Command Limit Let's disable the command host from a user: Insert into product_user_profile values ('SQL*Plus', 'USERNAME', 'HOST', NULL, NULL, 'DISABLED', sysdate, NULL); Note: notice the case in the word 'SQL*Plus' Page 93

94 Section E: Security Resource Consumer Groups Resource consumer groups For the most part, this is a way to group users by consumption. Everything except CPU and memory limits require Cost-based Optimizer Must have the ADMINISTER_RESOURCE_MANAGER privilege You may need to create DBMS_RESOURCE_MANAGER package Create consumer groups and assign to users. Consumer Groups To be able to dynamically change a resource, execute DBMS_RESOURCE_MANAGER.CREATE_PENDING_AREA(); Create a plan, grant privileges and plan directives with the DBMS_RESOURCE_MANAGER. For more information, use the Oracle 9i DBA handbook Page 94

95 Section E: Security Summary Summary Monitor Passwords by several authentication methods Limit resources with Profiles, consumer groups and Product User Profile table. Manage users' accesses by several means OEM Console is a good tool! Page 95

96 Section F: Auditing the Oracle Database Overview Introduction This section discusses the process of auditing an Oracle database and the tools used to do so. Intended audience Technical personnel who are responsible for the daily maintenance, support, and modification of SCT Banner. Objectives At the end of this section, you will be able to Audit statements, privileges and objects in an Oracle database View and manage the audit trail Prerequisites To complete this section, you should have completed OR101 (Introduction to Oracle) exposure to SQL and SQL* Plus (OR102 Advanced PL/SQL and Database Objects is recommended) completed OR103 (Oracle Database Administration I) Section contents Auditing the Oracle Database...97 Auditing Options Overview...98 Statement Auditing: Enabling/Disabling...99 Statement Auditing: Options Statement Auditing: Shortcuts Examples: Statement Auditing Privilege Auditing: Enabling/Disabling Example: Privilege Auditing Object Auditing: Enabling/Disabling Example: Object Auditing The Audit Trail Managing the Audit Trail Examples Viewing the Audit Trail Viewing the Audit Trail Examples Auditing with Triggers Examples Fine-Grain Object Auditing Summary Page 96

97 Section F: Auditing the Oracle Database Auditing the Oracle Database Objectives Understand auditing options Log statement and system level accesses to DB objects Monitor the audit trail Manage the audit trail Protect your database Page 97

98 Section F: Auditing the Oracle Database Auditing Options Overview Audit for a reason Investigate suspicious activity Monitor database activity Gather data about database activity Fine-grain access of object values System, object, value access Auditing levels Statement auditing Privilege auditing Object auditing Fine-grain auditing (as of 9i) Audit trail destination Add the initialization parameter to the initsid.ora file AUDIT_TRAIL = value [NONE DB OS] DB all entries go into sys.aud$ OS all entries go into the OS. Note: The database must be stopped and restarted for change to be effected Page 98

99 Section F: Auditing the Oracle Database Statement Auditing: Enabling/Disabling Enable auditing Enable auditing by user; by session, or by access recording Example AUDIT option1, option2 AUDIT option1, option2 BY user1, user2 BY SESSION [ WHENEVER [NOT] SUCCESSFUL] AUDIT option1, option2 BY user1, user2... BY ACCESS [WHENEVER [NOT] SUCCESSFUL] Page 99

100 Section F: Auditing the Oracle Database Statement Auditing: Options Options Option SQL Statement Audited Include in ALL ALTER SEQUENCE ALTER SEQUENCE sequence N ALTER SYSTEM ALTER SYSTEM Y ALTER TABLE ALTER TABLE table N COMMENT TABLE COMMENT ON table, view, snapshot, column N DELETE TABLE DELETE FROM table, view N EXECUTE PROCEDURE Calls to procedures and functions N GRANT PROCEDURE GRANT privilege ON procedure N REVOKE privilege ON procedure N GRANT TABLE GRANT privilege ON table, view, snapshot N REVOKE privilege ON table, view, snapshot N INDEX CREATE INDEX Y ALTER INDEX Y DROP INDEX Y INSERT TABLE INSERT INTO table, view N LOCK TABLE LOCK TABLE table, view N NOT EXISTS NOT EXISTS All SQL statements that return an Oracle error because the specified structure or object does not exist. Y PROCEDURE CREATE [or REPLACE] FUNCTION Y CREATE [or REPLACE] PACKAGE Y CREATE [or REPLACE] PACKAGE BODY Y CREATE [or REPLACE] PROCEDURE Y DROP FUNCTION Y DROP PACKAGE Y DROP PROCEDURE Y PUBLIC SYNONYM CREATE PUBLIC SYNONYM Y DROP PUBLIC SYNONYM Y Page 100

101 Section F: Auditing the Oracle Database Statement Auditing: Options (Continued) Options (cont.) Option SQL Statement Audited Include in ALL ROLE CREATE ROLE Y ALTER ROLE Y SET ROLE Y DROP ROLE Y ROLLBACK SEGMENT CREATE ROLLBACK SEGMENT Y ALTER ROLLBACK SEGMENT Y DROP ROLLBACK SEGMENT Y SELECT TABLE SELECT FROM table, view, snapshot N SESSION Connects and disconnects Y SYNONYM CREATE SYNONYM Y DROP SYNONYM Y SYSTEM AUDIT AUDIT Y NOAUDIT Y SYSTEM GRANT GRANT system privilege/role TO user/role Y REVOKE system privilege/role FROM user/role Y TABLE CREATE TABLE Y DROP TABLE Y TRUNCATE TABLE Y TABLESPACE CREATE TABLESPACE Y ALTER TABLESPACE Y DROP TABLESPACE Y TRIGGER CREATE [or REPLACE] TRIGGER Y ALTER TRIGGER ENABLE or DISABLE ALTER TABLE with ENABLE, Y DISABLE and DROP clauses Y USER CREATE USER Y ALTER USER Y DROP USER Y UPDATE TABLE UPDATE table, view N VIEW CREATE [or REPLACE] VIEW Y DROP VIEW Y ALTER VIEW COMPILE Y Page 101

102 Section F: Auditing the Oracle Database Statement Auditing: Options (Continued) Options (cont.) This is not a complete list. Please review Oracle 8i concepts, 9i Administrators guide and any reference manual. Page 102

103 Section F: Auditing the Oracle Database Statement Auditing: Shortcuts Shortcuts Option shortcut CONNECT RESOURCE DBA ALL Statement option included SESSION ALTER SYSTEM, CLUSTER, DATABASE LINK, PROCEDURE, ROLLBACK SEGMENT, SEQUENCE, SYNONYM, TABLE, TABLESPACE, VIEW SYSTEM AUDIT, PUBLIC DATABASE LINK, PUBLIC SYNONYM, ROLE, SYSTEM GRANT, USER See appropriate Oracle manual for statement options included in ALL Page 103

104 Section F: Auditing the Oracle Database Examples: Statement Auditing Examples Example: Audit CREATE/ALTER/DROP statements (included in the audit option user ) regardless of success or not SQL> AUDIT user BY ACCESS; Audit succeeded. Example: Audit all connections to the database SQL> AUDIT connect; Audit succeeded. Example: View statement audit options SQL> COL user_name FORMAT A15 SQL> COL audit_option FORMAT A30 SQL> SELECT user_name, audit_option, success, failure 2 FROM sys.dba_stmt_audit_opts; USER_NAME AUDIT_OPTION SUCCESS FAILURE USER BY ACCESS BY ACCESS CREATE SESSION BY ACCESS BY ACCESS Example: Audit unsuccessful attempts to connect with the database SQL> AUDIT connect 2 BY ACCESS 3 WHENEVER NOT SUCCESSFUL; Audit succeeded. Page 104

105 Section F: Auditing the Oracle Database Examples: Statement Auditing (Continued) Examples (cont.) Example: Disabling statement auditing for create user and alter user but leaving drop user SQL> NOAUDIT create user, alter user; Noaudit succeeded. Example: Disabling statement auditing for connections to the database SQL> NOAUDIT connect; Noaudit succeeded. Example: Review statement auditing options SQL> COL user_name FORMAT A15 SQL> COL audit_option FORMAT A30 SQL> SELECT user_name, audit_option, success, failure 2 FROM sys.dba_stmt_audit_opts; USER_NAME AUDIT_OPTION SUCCESS FAILURE USER BY ACCESS BY ACCESS SYSTEM AUDIT NOT SET BY ACCESS PUBLIC SYNONYM NOT SET BY ACCESS PUBLIC DATABASE LINK NOT SET BY ACCESS ROLE NOT SET BY ACCESS SYSTEM GRANT NOT SET BY ACCESS 6 rows selected. Page 105

106 Section F: Auditing the Oracle Database Privilege Auditing: Enabling/Disabling Enable auditing Enable auditing by user, by session or by access recording AUDIT sys_priv1, sys_priv2. AUDIT sys_priv1, sys_priv2. BY user1, user2 BY SESSION [WHENEVER [NOT] SUCCESSFUL] AUDIT sys_priv1, sys_priv2. BY user1, user2... BY ACCESS [WHENEVER [NOT] SUCCESSFUL] Privilege audit options For available privilege audit options, see the Oracle SQL Reference manual under GRANT system_privileges and roles for the list of system privileges. Page 106

107 Section F: Auditing the Oracle Database Example: Privilege Auditing Examples Example: Enable auditing of the AUDIT SYSTEM, ALTER TABLESPACE, GRANT ANY PRIVILEGE, and MANAGE TABLESPACE privileges SQL> AUDIT system, 2 alter tablespace, 2 grant any privilege, 3 manage tablespace; Audit succeeded. Example: Enable auditing of every create table or index by new_user SQL> audit create table 2 by new_user 3 by access; Audit succeeded. Example: Enable auditing of alter any table, alter any procedure by new_user only when successful SQL> AUDIT alter any table, 2 alter any procedure 3 BY new_user 4 BY ACCESS 5 WHENEVER SUCCESSFUL; Audit succeeded. Example: Disable privilege auditing for create table SQL> NOAUDIT create any table 2 BY new_user; Noaudit succeeded. Page 107

108 Section F: Auditing the Oracle Database Example: Privilege Auditing (Continued) Examples (cont.) Example: Disable the privilege auditing feature for AUDIT SYSTEM, ALTER TABLESPACE, GRANT ANY PRIVILEGE, and MANAGE TABLESPACE SQL> NOAUDIT audit system, 2 alter tablespace, 3 grant any privilege, 4 manage tablespace; Noaudit succeeded. Example: Review privilege auditing options SQL> SELECT * FROM sys.dba_priv_audit_opts; USER_NAME PRIVILEGE SUCCESS FAILURE NEW_USER CREATE TABLE BY ACCESS BY ACCESS NEW_USER ALTER ANY TABLE BY ACCESS NOT SET NEW_USER ALTER ANY PROCEDURE BY ACCESS NOT SET Page 108

109 Section F: Auditing the Oracle Database Object Auditing: Enabling/Disabling Object auditing syntax AUDIT object_opt1, object_opt 2. ON schema.object [DEFAULT] AUDIT object_opt1, object_opt 2. ON ALL [DEFAULT] AUDIT object_opt1, object_opt 2 ON schema.object [DEFAULT] BY SESSION [WHENEVER [NOT] SUCCESSFUL] AUDIT object_opt1, object_opt 2 ON schema.object [DEFAULT] BY ACCESS [WHENEVER [NOT] SUCCESSFUL] Available audited objects Tables Views Sequences Packages Stand-alone stored procedures Stand-alone stored functions By object type Object audit options vary by object type Tables - Select, Update, Insert, Delete, Index, etc. Views - Select, Update, Insert, Delete, etc. Sequences - Select, Grant, etc. Packages - Execute, Grant, etc. Stand-alone stored procedures Stand-alone stored functions Page 109

110 Section F: Auditing the Oracle Database Example: Object Auditing Examples Example: Enable auditing of EXECUTE on procedure P_MESS_REMINDER SQL> AUDIT EXECUTE 2 ON p_mess_reminder 3 BY SESSION; Audit succeeded. Example: Enable auditing of DELETE on table SPRIDEN SQL> AUDIT DELETE 2 ON spriden 3 BY ACCESS; Audit succeeded. Example: Review object auditing SQL> COL object_name FORMAT A15 SQL> SELECT * 2 FROM user_obj_audit_opts 3 WHERE object_name like 'SP%'; OBJECT_NAME OBJECT_TY ALT AUD COM DEL GRA IND INS LOC REN SEL UPD REF EXE SPBPERS TABLE -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- SPRADDR TABLE -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- SPRAPIN TABLE -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- SPRCMNT TABLE -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- SPRCOLR TABLE -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- SPREMRG TABLE -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- SPRHOLD TABLE -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- SPRIDEN TABLE -/- -/- -/- A/A -/- -/- A/A -/- -/- -/- A/A -/- -/- SPRINTL TABLE -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- SPRMEDI TABLE -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- SPRTELE TABLE -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- SPTADDR TABLE -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- SPTIDEN TABLE -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- SPTPERS TABLE -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- -/- 14 rows selected. Page 110

111 Section F: Auditing the Oracle Database Example: Object Auditing (Continued) Examples (cont.) Example: Disable object auditing on table SPRIDEN SQL> NOAUDIT INSERT, UPDATE, DELETE 2 ON spriden; Noaudit succeeded. Page 111

112 Section F: Auditing the Oracle Database The Audit Trail Components User who executed the statement Action code indicating the type of statement executed or privilege used (Look in AUDIT_ACTIONS) Objects referenced in the statement Date and time the statement was issued Monitoring the audit trail Watch audit trail growth enable auditing only when necessary Make it count - be selective about which options are specified Control database auditing restrict auditing privilege to security personnel Managing the audit trail Removing all records from the audit trail Removing selected records from the audit trail Archiving audit records to different table Archiving audit records out of the database Page 112

113 Section F: Auditing the Oracle Database Managing the Audit Trail Examples Examples Example: Removing all records from the audit trail SVRMGR> TRUNCATE TABLE aud$; Statement processed. Example: Removing selected records from the audit trail SQL> DELETE FROM sys.aud$ 2 WHERE timestamp < '01-JAN-98'; 100 rows deleted. Example: Archiving audit records to different table SQL> CREATE TABLE audithist AS 2 SELECT * FROM sys.aud$ 3 WHERE timestamp > '01-JAN-98'; Table created. Example Example: Archiving audit records out of the database $ exp username/password tables=sys.aud$ file=audit1.dmp Export: Release Production on Mon Jun 22 11:31: Copyright (c) Oracle Corporation 1979, All rights reserved. Connected to: Oracle7 Server Release Production Release With the distributed, replication and Spatial Data options PL/SQL Release Production Export done in US7ASCII character set About to export specified tables via Conventional Path... Current user changed to SYS.. exporting table AUD$ 10 rows exported Export terminated successfully without warnings. Page 113

114 Section F: Auditing the Oracle Database Viewing the Audit Trail Predefined views Predefined views for displaying auditing information: USER_AUDIT_TRAIL DBA_AUDIT_TRAIL USER_AUDIT_STATEMENT DBA_AUDIT_STATEMENT USER_AUDIT_OBJECT DBA_AUDIT_OBJECT USER_AUDIT_SESSION DBA_AUDIT_SESSION USER_OBJ_AUDIT_OPTS DBA_OBJ_AUDIT_OPTS AUDIT_ACTIONS DBA_STMT_AUDIT_OPTS ALL_DEF_AUDIT_OPTS DBA_PRIV_AUDIT_OPTS DBA_AUDIT_EXISTS Page 114

115 Section F: Auditing the Oracle Database Viewing the Audit Trail Examples Examples Example: View database connections SQL> COL username FORMAT A10 SQL> COL action_name FORMAT A6 HEADING ACTION SQL> COL returncode FORMAT HEADING RCODE SQL> SELECT username, timestamp, action_name, 2 logoff_time, logoff_lread, logoff_pread, 3 returncode 4 FROM sys.dba_audit_session; USERNAME TIMESTAMP ACTION LOGOFF_TI LOGOFF_LREAD LOGOFF_PREAD RCODE USER1 20-JUN-98 LOGOFF 20-JUN USER2 21-JUN-98 LOGON 21-JUN USER2 21-JUN-98 LOGOFF 21-JUN USER3 22-JUN-98 LOGON 1017 USER3 22-JUN-98 LOGON 22-JUN Example: View deletes from tables SQL> SELECT username, 2 TO_CHAR (timestamp, 'DD-MON-YY HH:MI:SS'), 3 owner, obj_name, action_name, 4 returncode, priv_used 5 FROM sys.dba_audit_trail 6 WHERE action_name = 'DELETE'; Page 115

116 Section F: Auditing the Oracle Database Auditing with Triggers Examples Examples Example: Install value-based auditing CREATE TRIGGER audit_employee AFTER INSERT OR DELETE OR UPDATE ON EMP FOR EACH ROW BEGIN IF auditpackage.reason IS NULL THEN raise_application_error (-20501, 'Must specify reason for update before performing update; use auditpackage.set_reason() '); END IF; INSERT INTO audit_employee VALUES (:OLD.ssn, :OLD.name, :OLD.class, :OLD.sal, :NEW.ssn, :NEW.name, :NEW.class, :NEW.sal, auditpackage.reason, USER, SYSDATE); END; CREATE TRIGGER audit_emp_cleanup AFTER INSERT OR DELETE OR UPDATE ON EMP BEGIN auditpackage.reason :=NULL; END; Page 116

117 Section F: Auditing the Oracle Database Fine-Grain Object Auditing Fine-grade auditing One failing of object auditing up to now, you can see what object and who did it, but not what values. Nor could you audit select statements In 9i a new package, DBMS_FGA, enables fine-grain auditing. Create 'policies' on what to audit DBMS_FGA Options are, ADD_POLICY, DROP_POLICY, ENABLE_POLICY, DISABLE_POLICY You must create error handlers, such as triggers, to activate when the audit policy condition is met. Note: Use only with Cost-based optimization. Rule-based can cause unexpected results when row filtering is applied Banner is currently Rule-based. Page 117

118 Section F: Auditing the Oracle Database Summary Summary The database has the ability to audit all actions Monitor usage of sys.aud$ Protect this file. Audit the auditors. Option to audit in DB or OS Secure your database. Page 118

119 Section G: Oracle Locks Overview Introduction This section discusses Oracle locks and how to handle them. Intended audience Technical personnel who are responsible for the daily maintenance, support, and modification of SCT Banner. Objectives At the end of this section, you will be able to Describe types of Oracle locks and their usage Describe Oracle's handling of deadlocks Prerequisites To complete this section, you should have completed OR101 (Introduction to Oracle) exposure to SQL and SQL* Plus (OR102 Advanced PL/SQL and Database Objects is recommended) completed OR103 (Oracle Database Administration I) Section contents Oracle Locks Why Locks? Preventable Phenomena Transaction Isolation Data Concurrency Model Rollback Segments and Data Consistency Concurrency/Consistency - How It Works Snapshot Too Old Errors Consistency via Locks Types of DML Locks Types of Table Locks Deadlocks Lock Waits Detecting Lock Waits Deadlock Behavior Determining Locking OEM Page 119

120 Section G: Oracle Locks Oracle Locks Objectives Why locks? How Data Concurrency works How Oracle locks data How to monitor locks Page 120

121 Section G: Oracle Locks Why Locks? Data Concurrency Multiple users can access and manipulate the same data at the same time. Data Consistency Data should be consistent within the timeframe of the transaction start. Transaction Isolation Model Ensures that transactions maintain consistency Serial execution means slow performance Transaction Throughput Parallel transaction execution for performance Lost updates Updates from one transaction are over-written by another transaction. Destructive DDL Data structures are not removed while another transaction is using them. Data structures are not modified while another transaction is using them. Page 121

122 Section G: Oracle Locks Preventable Phenomena Dirty Read A transaction reads data written by another transaction but not yet committed Non-repeatable Read A transaction re-reads data it has previously read and finds another committed transaction's modified data. Phantom Read A transaction re-executes a query and finds another committed transaction has inserted additional rows satisfying the query. Page 122

123 Section G: Oracle Locks Transaction Isolation Levels SQL92 (SQL Standard) defines four levels of transaction isolation. Read uncommitted Read committed Repeatable read Serializable Oracle support Oracle supports: Read committed dirty read - not possible non-repeatable read - possible phantom read - possible Serializable dirty read - not possible non-repeatable read - not possible phantom read - not possible Read-only (Not part of SQL92 standard) SCT Banner support SCT Banner was built to support: Read committed Transactions are not likely to conflict and throughput is high. Both non-repeatable and phantom reads are possible It was the only isolation available previous to Oracle 7.3 Page 123

124 Section G: Oracle Locks Data Concurrency Model Characteristics Users can always read and never wait for any other transaction(s) A user's DML statements never wait for users reading the same data A user's DML statement only wait for other users' DML statements on the same data rows Statements are not affected by the work of other active transactions during the execution of the statement All statements have a point in time view of the data at statement execution time Page 124

125 Section G: Oracle Locks Rollback Segments and Data Consistency Rollback segments Rollback segments are used to store the before image of data to provide consistent views of data. Example 10:00 User 1: UPDATE spraddr SET spraddr_stat_code = 'NY' WHERE spraddr_stat_code = 'Ny'; 10:01 User 2: SELECT spraddr_pidm, spraddr_city, spraddr_stat_code FROM spraddr; 10:02 User 1: COMMIT; 10:06 User 2: /* Select is still going */ 10:08 User 2: SELECT spraddr_pidm, spraddr_city, spraddr_stat_code FROM spraddr; Note: System Commit Numbers determine whether a data element needs to be rebuilt. More recent SCNs indicate a change to data. Page 125

126 Section G: Oracle Locks Concurrency/Consistency - How It Works Determining SCN As a query enters the execution stage, the current system change number (SCN) is determined Writing blocks with observed SCN As data blocks are read on behalf of the query, only blocks written with the observed SCN are used. Reconstructing changed blocks Blocks with changed data (more recent SCNs) are reconstructed from data in the rollback segments, and the reconstructed data is returned for the query. Diagram Page 126

127 Section G: Oracle Locks Snapshot Too Old Errors Snapshot Too Old errors Occur when Oracle cannot reconstruct data via the rollback segments because the image of the data at has been overwritten Usually happens to transactions containing long running queries A rollback segment which contains undo entries needed to reconstruct the query snapshot for the current transaction has wrapped around and overwritten data Page 127

128 Section G: Oracle Locks Consistency via Locks Categories of Oracle locks DML Locks Data level locks for a multi-user environment DDL Locks On object structure to prevent transaction problems Internal Locks/Latches SGA locks protect memory and database structures Note: All implicit locks are handled automatically by Oracle and require no user actions. Modes of Oracle locks Exclusive Lock Mode Does not allow the resource to be shared. This lock is obtained to modify data. Share Lock Mode Allows sharing of the resource, depending on the operations involved. Many readers can hold share locks on a single resource, preventing a writer from getting an exclusive lock on the resource. Lock Duration All locks are held for the duration of the transaction. They are released only when: Transaction is committed Transaction is rolled back Lock conversion Usually not required as the lock is at the row level, the highest level of restrictiveness. Automatic if needed... SELECT FOR UPDATE will convert from a shared lock to an exclusive lock when the rows are actually modified. Oracle never escalates locks. Page 128

129 Section G: Oracle Locks Types of DML Locks Row level locks (TX) Generated by: o Insert o Update o Delete o Select For Update Exclusive to the row being modified No limit to the number of DML locks a transaction may hold Table level locks (TM) Automatically generated for the table when a row level lock is obtained. Used to restrict other user's access to the table on behalf of the current transaction. Used to prevent other DDL operations that would conflict with the current transaction. Can be done manually with a Lock Table statement. Page 129

130 Section G: Oracle Locks Types of Table Locks Row Share (RS) Locks rows and intends to update the rows Selects are still allowed Other DML statements can update, delete, or insert into other rows in the table Prevents other transactions from exclusive write locks on the table Commands giving (RS): SELECT. For Update; Lock Table xyz in Row Share Mode; Row Exclusive (RX) Locks and changes rows in a table Select are still allowed Other DML statements can update, delete, or insert into other rows in the table Prevents other transactions from locking the table in Share, Share Exclusive, or Exclusive Commands giving (RX): Insert, Update, and Delete; Lock Table xyz in Row Exclusive Mode Share (S) Locks the table to prevent write locks Select are still allowed Only Select For Update statement is allowed, though the transaction cannot update data until all other share locks are released Multiple transactions can have share locks on a table Commands giving (S): Lock Table xyz In Share Mode Page 130

131 Section G: Oracle Locks Types of Table Locks (Continued) Share Row Exclusive (SRX) Locks the table to prevent either read or write locks Selects are still allowed Other transactions cannot issue update statements against table, though a SELECT for UPDATE can be issued Only one SRX lock on a table at a time Commands giving (SRX): Lock Table xyz In Share Row Exclusive Mode Exclusive (X) Locks the table and allows NO other locks Selects are still allowed Only SELECT statements are permitted against the table locked in exclusive mode No update statements allowed against table, including SELECT for UPDATE Commands giving (X): Lock Table xyz In Exclusive Mode Page 131

132 Section G: Oracle Locks Deadlocks Definition Deadlocks occur when two or more transactions are waiting for data locked by each other. Diagram Response Oracle automatically detects deadlocks and rolls one of the transactions back. Occur most often when overriding default Oracle behavior. Can be avoided by following an access order in related tables. Master table then detail table for all applications accessing these tables. Page 132

133 Section G: Oracle Locks Lock Waits Definition Similar to deadlocks, except the situation never reaches the point where Oracle can detect the deadlock. Occurs when one user has locks on data that the second transaction is waiting for, but the first user is not completing the transaction. Data Dictionary Views DBA_WAITERS Shows all the sessions waiting for locks and the session that holds the lock DBA_LOCKS Shows each lock that is being held by the instance DBA_BLOCKERS Contains sessions someone waiting on a lock they hold, but they are not waiting DBA_DDL_LOCKS Contains each DDL lock that is being held DBA_DML_LOCKS Contains each DML lock that is being held Note: ORA_ROOT:[rdbms.admin]catblock.sql creates the above DBA locking views Page 133

134 Section G: Oracle Locks Detecting Lock Waits Example User 1 updates rows but does not commit User 2 attempts to update the same rows and is kept in a lock wait state. SQL> select username,sid from v$session; USERNAME SID FIMSMGR 9 SATURN 11 SYSTEM 12 FIMSMGR does the following: SQL> update spriden 2 set spriden_ntyp_code = 'T' 3 where spriden_pidm = '120'; 1 row updated. A look at DBA_LOCKS shows: SQL> select * from dba_locks where session_id in ('9','11'); SESSION_ID LOCK_TYPE MODE_HELD MODE_REQUESTED LOCK_ID1 LOCK_ID2 BLOCKING_OTHERS Transaction Exclusive None Not Blocking 9 DML Row-X (SX) None Not Blocking SATURN next does: SQL> update spriden 2 set spriden_ntyp_code = 'U' 3 where spriden_pidm = '120'; Page 134

135 Section G: Oracle Locks Detecting Lock Waits (Continued) Example (cont.) A look at DBA_LOCKS now shows: SQL> select * from dba_locks where session_id in ('9','11'); SESSION_ID LOCK_TYPE MODE_HELD MODE_REQUESTED LOCK_ID1 LOCK_ID2 BLOCKING_OTHERS Transaction Exclusive None Blocking 9 DML Row-X (SX) None Not Blocking 11 DML Row-X (SX) None Not Blocking 11 Transaction None Exclusive Not Blocking A look at DBA_BLOCKERS shows: SQL> select * from dba_blockers; HOLDING_SESSION A look at DBA_WAITERS shows: SQL> select * from dba_waiters; WAITING_SESSION HOLDING_SESSION LOCK_TYPE MODE_HELD MODE_REQUESTED LOCK_ID1 LOCK_ID Transaction Exclusive Exclusive FIMSMGR now does the following: SQL> rollback; Rollback complete. A look at DBA_LOCKS now shows: SQL> select * from dba_locks where session_id in ('9','11'); SESSION_ID LOCK_TYPE MODE_HELD MODE_REQUESTED LOCK_ID1 LOCK_ID2 BLOCKING_OTHERS Transaction Exclusive None Not Blocking 11 DML Row-X (SX) None Not Blocking Page 135

136 Section G: Oracle Locks Deadlock Behavior Deadlock behavior User 1 updates Table1 row 1a User 2 updates Table2 row 2a User 1 updates Table2 row 2a (lock wait) User 2 updates Table1 row 1a (deadlock) Example SQL> select username,sid from v$session; USERNAME SID SATURN 8 SYSTEM 9 FIMSMGR 11 SATURN does the following: SQL> update spriden 2 set spriden_ntyp_code = 'T' 3 where spriden_pidm = '120'; 1 row updated. A look at DBA_LOCKS shows: SQL> select * from dba_locks; SESSION_ID LOCK_TYPE MODE_HELD MODE_REQUESTED LOCK_ID1 LOCK_ID2 BLOCKING_OTHERS Transaction Exclusive None Not Blocking 8 DML Row-X (SX)None Not Blocking FIMSMGR next does: SQL> update spraddr 2 set spraddr_to_date = sysdate 3 where spraddr_pidm = '120'; 1 row updated. Page 136

137 Section G: Oracle Locks Deadlock Behavior (Continued) Example (cont.) Now a look at DBA_LOCKS shows: SQL> select * from dba_locks where session_id in ('8','11'); SESSION_ID LOCK_TYPE MODE_HELD MODE_REQUESTED LOCK_ID1 LOCK_ID2 BLOCKING_OTHERS Transaction Exclusive None Not Blocking 8 DML Row-X (SX) None Not Blocking 11 Transaction Exclusive None Not Blocking 11 DML Row-X (SX) None Not Blocking SATURN now does the following: SQL> update spraddr 2 set spraddr_to_date = sysdate 3 where spraddr_pidm = '120'; * The row updated message does not appear because we are lock waited. Look at DBA_LOCKS to see: SQL> select * from dba_locks where session_id in ('8','11'); SESSION_ID LOCK_TYPE MODE_HELD MODE_REQUESTEDLOCK_ID1 LOCK_ID2 BLOCKING_OTHERS Transaction Exclusive None Not Blocking 8 DML Row-X (SX) None Not Blocking 8 DML Row-X (SX) None Not Blocking 8 Transaction None Exclusive Not Blocking 11 Transaction Exclusive None Blocking 11 DML Row-X (SX) None Not Blocking A look at DBA_BLOCKERS shows: SQL> select * from dba_blockers; HOLDING_SESSION Page 137

138 Section G: Oracle Locks Deadlock Behavior (Continued) Example (cont.) A look at DBA_WAITERS shows: SQL> select * from dba_waiters; WAITING_SESSION HOLDING_SESSION LOCK_TYPE MODE_HELD MODE_REQUESTED LOCK_ID1 LOCK_ID Transaction Exclusive Exclusive FIMSMGR now attempts the following: SQL> update spriden 2 set spriden_ntyp_code = 'U' 3 where spriden_pidm = '120'; A quick look at DBA_LOCKS before Oracle figures out it has a deadlock: SQL> select * from dba_locks where session_id in ('8','11'); SESSION_ID LOCK_TYPE MODE_HELD MODE_REQUESTED LOCK_ID1 LOCK_ID2 BLOCKING_OTHERS Transaction Exclusive None Blocking 8 DML Row-X (SX) None Not Blocking 8 Transaction None Exclusive Not Blocking 11 Transaction Exclusive None Not Blocking 11 DML Row-X (SX) None Not Blocking 11 DML Row-X (SX) None Not Blocking 11 Transaction None Exclusive Not Blocking SATURN now receives the following: SQL> update spraddr 2 set spraddr_to_date = sysdate 3 where spraddr_pidm = '120'; update spraddr * ERROR at line 1: ORA-00060: deadlock detected while waiting for resource Page 138

139 Section G: Oracle Locks Deadlock Behavior (Continued) Example (cont.) A quick look at DBA_LOCKS after Oracle figures out it has a deadlock: SQL> select * from dba_locks where session_id in ('8','11'); SESSION_ID LOCK_TYPE MODE_HELD MODE_REQUESTED LOCK_ID1 LOCK_ID2 BLOCKING_OTHERS Transaction Exclusive None Blocking 8 DML Row-X (SX) None Not Blocking 11 Transaction Exclusive None Not Blocking 11 DML Row-X (SX) None Not Blocking 11 DML Row-X (SX) None Not Blocking 11 Transaction None Exclusive Not Blocking Status Notice that there is still a block in place. Oracle rolled back SATURN's first transaction's update on SPRIDEN, but FIMSMGR's transaction is still blocked by the second part of SATURN's transaction, which has not been rolled back. Example (cont.) SATURN now issues the manual rollback: SQL> update spraddr 2 set spraddr_to_date = sysdate 3 where spraddr_pidm = '120'; update spraddr * ERROR at line 1: ORA-00060: deadlock detected while waiting for resource SQL> rollback; Rollback completed Page 139

140 Section G: Oracle Locks Deadlock Behavior (Continued) Example (cont.) A quick look at DBA_LOCKS after the manual rollback: SQL> select * from dba_locks where session_id in ('8','11'); SESSION_ID LOCK_TYPE MODE_HELD MODE_REQUESTED LOCK_ID1 LOCK_ID2 BLOCKING_OTHERS Transaction Exclusive None Not Blocking 11 DML Row-X (SX) None Not Blocking 11 DML Row-X (SX) None Not Blocking FIMSMGR now is cleared to update: SQL> update spriden 2 set spriden_ntyp_code = 'U' 3 where spriden_pidm = '120'; 1 row updated. Page 140

141 Section G: Oracle Locks Determining Locking OEM Diagram Page 141

142 Section H: Tuning and Monitoring Overview Introduction This section discusses the tuning and monitoring of an Oracle database. Intended audience Technical personnel who are responsible for the daily maintenance, support, and modification of SCT Banner. Objectives At the end of this section, you will be able to Tune an Oracle database for increased efficiency Monitor the performance of an Oracle database Prerequisites To complete this section, you should have completed OR101 (Introduction to Oracle) exposure to SQL and SQL* Plus (OR102 Advanced PL/SQL and Database Objects is recommended) completed OR103 (Oracle Database Administration I) Page 142

143 Section H: Tuning and Monitoring Overview (Continued) Section contents Tuning Performance Memory Issues OS Paging and Swapping Redo Log Buffers Private SQL Areas Shared Pool Examining Library Cache Data Dictionary Cache Pinning Objects in Memory Viewing Executing SQL Database Buffer Cache Sorting I/O Bottlenecks Chaining and Migration Redo Logs UNDO vs. Rollback Segments CPU Bottlenecks Oracle Optimization Testing SQL Efficiency Summary Page 143

144 Section H: Tuning and Monitoring Tuning Purpose Issues with system resources and limitations Effects of Demands Response time and throughput OLTP define performance in terms of throughput Tuning Steps There are actually several steps to an effectively and efficiently tuned database. This section will concentrate on the common DBA tuning steps: Memory Resource Monitoring You may ask, 'My system is working fine. Why do I need to monitor it?' DBAs must develop proactive mechanisms instead of reactive ones. Prepare for future growth with database trends. Page 144

145 Section H: Tuning and Monitoring Performance Areas to tune and monitor Tuning and monitoring are essential parts of a system's performance. Areas to watch the most are: Memory Issues I/O bottlenecks Resource contention CPU bottlenecks Network bottlenecks Page 145

146 Section H: Tuning and Monitoring Memory Issues SGA The purpose of the System Global Area (SGA) is to store data in memory for fast access. Mismanaged systems will cause the database to be busy on handling resources, not requests (monitoring). If memory is adequate, issues with the way Oracle is using the memory may need to be review (tuning). Page 146

147 Section H: Tuning and Monitoring OS Paging and Swapping Tasks Monitor the operating system usage daily. Excessive trashing (paging and/or swapping) can quickly reduce the performance. Load any production database first into non-paged, non-swapped main memory. Oracle will degrade quickly if it is utilizing fragmented memory. Paging -SGA Pages of a process's memory allocation are written out to disk instead of in memory. Pages of a process's sort allocation may be written out to disk. Uncommitted redo entries could be written to the online redo logs. Usually a LRU algorithm Some amount of paging is acceptable Swapping - SGA Process of moving an entire process out of memory to disk to free up memory for current processes. Tune similarly written SQL statements. Tune indexes and sequel that are not efficiently created. Oracle will have to swap a statement out of memory to make room for a new one. SGA Tuning Defined by the Parameter file and tuned by several parameters. This is one of the biggest areas of tuning for the DBA. Coordination by the DBA with System administration is critical. Overall, increase if available memory, decrease if not. If decreasing is necessary, the system will have to be tuned extensively. PGA Tuning Allocated by Oracle when a user connects. Operating system specific, defined by the Oracle Server software. Each process requires statement parsing and sorting regions. In 9i, there is a new feature called Automated PGA. Page 147

148 Section H: Tuning and Monitoring OS Paging and Swapping (Continued) Automated PGA Oracle 9i offers the option of automated PGA space management. You can set a target amount of memory available to each process Pga_aggregate_target Another parameter has two options of manual or auto: workarea_size_policy When set to auto, Oracle will tune the PGA. Page 148

149 Section H: Tuning and Monitoring Redo Log Buffers Bottlenecks As transactions are changing data, the LGWR process is writing entries from the LOG BUFFER into the ONLINE REDO LOGFILES. With fast processors and slow disks, the process may become bottlenecked at this point. Examining Redo Log Buffer Use The goal is zero waits for Log Buffers. SQL> select name, value from v$sysstat where name in ('redo log space requests', redo entries'); NAME VALUE redo log space requests 828 redo entries Note: If the redo logs space requests / redo entries is > 1:5000, increase the size of the LOG_BUFFER parameter. Redo Log Buffer Tuning Size is defined in the Init.ora file by the LOG_BUFFER parameter. LOG_BUFFER should be set a multiple of the DB_BLOCK_SIZE (GOAL: value = 0). Page 149

150 Section H: Tuning and Monitoring Private SQL Areas Parse calls Parse calls (reparsing) are the issue with tuning the private SQL areas. Tradeoff is between memory and reparsing. More memory = fewer parse calls. Reducing reparsing creates more client demands. The number of open cursors go up Examining Private SQL - Method 1 Identify reparsing issues. Method 1 - Check the SQL area for any statement doing a lot of re-parsing: SQL> select substr(sql_text,1,50) 2 parse_calls, 3 executions 4 from v$sqlarea 5 where parse_calls = executions; Examining Private SQL - Method 2 Identify reparsing issues. Method 2 - Check the session information. First, find the proper statistic number from the V$STATNAME view. SQL> select statistic#, 2 name 3 from v$statname 4 where name in ('parse count (hard)', 5 'execute count'); Page 150

151 Section H: Tuning and Monitoring Private SQL Areas (Continued) Method 2 results Identify reparsing issues: Method 2 - Check the session information. (The previous query would return something similar to the following.) STATISTIC# NAME parse count (hard) 166 execute count Find the session Identify reparsing issues: Check the session information. Now, take the statistic number returned from the previous query and look at the counts in the V$SESSTAT view. SQL> select * from v$sesstat 2 where statistic# in (164,166) 3 order by value,sid; Check the Session Information Identify reparsing issues: Method 2 - Check the session information. The query to V$SESSTAT would rerun the SID's the processes which could then be joined to the V$SESSION view to find the heavy reparsing process. SQL> select username from v$session 2 where sid in = 8; Private SQL Area Tuning Controlled on the server by the Init.ora parameter OPEN_CURSORS. Can also be controlled with Oracle Precompilers by setting three clauses: HOLD_CURSOR=yes RELEASE_CURSOR=no MAXOPENCURSORS=desired value Page 151

152 Section H: Tuning and Monitoring Shared Pool Shared pool tuning Tuning the shared pool consists of tuning two areas: Library Cache Data Dictionary Cache Oracle's algorithm tends to keep Dictionary cache in memory longer than Library cache, so tuning Library cache to an acceptable level usually works for Dictionary cache as well. If using MTS, session specific information will be kept in the shared pool also. Library Cache Contains the fully parsed executable forms of SQL cursors and PL/SQL programs, the application logic. Ideally, these executables would be parsed once and remain in memory to be reused. 'Cache Misses' occur when the logic calls for execution of one of these objects and it is not found in memory. These misses can occur on two different steps: Parse Execute Parse miss Application makes parse call and the parsed representation of the statement does not exist in the shared SQL area in library cache. Oracle must first parse the statement Oracle then allocates a shared SQL area Fix by making sure SQL statements can be shared by insuring that the statements are coded properly. Match objects and syntax exactly Use variables Page 152

153 Section H: Tuning and Monitoring Shared Pool (Continued) Execute miss Application makes execution call and the parsed representation of the statement has been deallocated from library cache to make room for another SQL statement. Required steps: Oracle reparses the statement Oracle allocates a shared SQL area Oracle executes the statement Fix by allocating more memory to the library cache. Page 153

154 Section H: Tuning and Monitoring Examining Library Cache Examining library cache activity Use the view V$LIBRARYCACHE, which records all library cache activity since instance startup. Each row contains statistics for one type of item kept in library cache. For SQL and PL/SQL statement activity by using the following NAMESPACE values: SQL AREA TABLE/PROCEDURE BODY TRIGGER Other NAMESPACE values reflect library cache activity for dependency maintenance. Pins and Reloads The following two columns reflect library cache misses on execution calls: PINS - The number of times an item in library cache was executed. RELOADS- The number of times the call missed finding the executable form of the SQL statement in library cache on execution steps. View statistics Use the following query to view the statistics for Library Cache. SQL> select sum(pins) Executions 2 sum(reloads) Cache Misses while Executing, 3 round(sum(reloads)/sum(pins) 4 * 100,2) Miss Ratio 3 from v$librarycache; Executions Cache misses while Executing Miss Ratio Page 154

155 Section H: Tuning and Monitoring Examining Library Cache (Continued) Analyzing statistics From the returned values, one can see that for executions, 549 calls resulted in a cache miss. 549 / = = 0.18% miss ratio. The ideal would be no library cache misses, but up to 1% would still be acceptable. Library Cache Tuning If the execution miss ratio for Library Cache is too high, then adjust by increasing the Init.ora parameter SHARED_POOL_SIZE. When increasing the shared pool, you may also need to increase the Init.ora parameter OPEN_CURSORS to take advantage of the extra memory. Use identical SQL statements and using stored procedures to reduce library cache parse calls Page 155

156 Section H: Tuning and Monitoring Data Dictionary Cache Definition A collection of database tables and views containing reference information about: The database itself Database structures Database users Ideally, this information would be loaded once into 'Row' cache and remain available for re-use 'Cache Misses' occur when access to a database object and the needed data dictionary data is not available in data dictionary cache. Examining Dictionary Cache Use the view V$ROWCACHE, which records all data dictionary cache activity since instance startup. Each row contains statistics for one type of data dictionary item kept in data dictionary cache. To examine the effectiveness of data dictionary cache, the following columns are of interest: PARAMETER - Groups statistics by type of data dictionary item. GETS - Total number of requests for data dictionary information for the particular type of item. GET MISSES - Total number of requests resulting in cache misses for the particular type of item. View statistics SQL> select sum(gets) Data Dictionary Gets, sum(getmisses) Data Dictionary Cache Misses, round(sum(gets)/(sum(gets)+sum(getmisses)) * 100,2) Hit Ratio from v$rowcache; Data Dictionary Gets Data Dictionary Cache Misses Hit Ratio Page 156

157 Section H: Tuning and Monitoring Data Dictionary Cache (Continued) Analyze statistics From the returned values, one can see that for 1,439,044 dictionary gets, 3120 calls resulted in a dictionary cache miss / = = 0.22% hit ratio. The ideal would be no data dictionary cache misses, but 5% - 10% would still be acceptable. Data Dictionary Cache Tuning If the hit ratio for Data Dictionary Cache gets too low, then adjust by increasing the Init.ora parameter SHARED_POOL_SIZE. Do not increase the shared pool to the point that excessive paging or swapping results. Page 157

158 Section H: Tuning and Monitoring Pinning Objects in Memory KEEP option Reexecution of packages that are used over and over is not efficient use of the SGA. There is a procedure (dbms_shared_pool) that has KEEP option to pin packages into memory The package dbms_shared_pool is created by running the dbmspool.sql script as user sys 'Keeping' a package does not guarantee its being pinned in memory. It must first be executed. Determining the Objects to Pin Pin objects in the shared pool soon after instance startup, since at this point the sga has not been fragmented First determine which ones by running the following SQL after the instance has been up for a while On the following slide, an example to collect names of package, package body, function and procedure that have a memory usage greater than Script for Active Packages REM Display used SGA memory for triggers, packages & procedures. SPOOL USED_MEM.LIS COLUMN EXECUTIONS FORMAT 999,999,999 COLUMN mem_used FORMAT 999,999,999 select substr(owner,1,10) Owner, substr(type,1,12) Type, substr(name,1,20) Name, executions, sharable_mem Mem_used, SUBSTR(KEPT ' ',1,4) "KEPT?" from v$db_object_cache where type in('trigger','procedure','package BODY','PACKAGE') order by EXECUTIONS DESC / SPOOL OUT Page 158

159 Section H: Tuning and Monitoring Pinning Objects in Memory (Continued) 'Keep' Scripts Besides the obviously large ones, there are several others that need consideration for systems like SCT Banner with web applications. Keep_it_ias.sql As an example, the following SQL will look for all packages currently in the sga and generate a keep script. 'Keep' Example SET PAGESIZE 999 SET HEAD OFF SET FEEDBACK OFF SPOOL pin.sql SELECT 'EXECUTE dbms_shared_pool.keep(''' name ''');' sql_stmt FROM v$db_object_cache WHERE type='package' / SPOOL OFF Page 159

160 Section H: Tuning and Monitoring Viewing Executing SQL Viewing executing SQL It's sometimes important to know the type of statements in the shared pool. define access_level = 10 (number) spool shared_cache select parsing_user_id, executions, sorts, command_type, disk_reads, sql_text from v$sqlarea where disk_reads > &access_level order by disk_reads desc; PARSING_USER_ID EXECUTIONS SORTS COMMAND_TYPE DISK_READS SQL_TEXT SELECT FARINVC_PART_PMT_IND FROM FIMSMGR.FARINVC X WHERE FARINVC_INVH_CODE=:b1 AND FARINVC_ITEM=TO_NUMBER(:b2)AND EXISTS(SELECT'x' FROM FIMSMGR.FPRPODT WHERE FPRPODT_POHD_CODE=X.FARINVC_POHD_CODE AND FPRPODT_ITEM=X.FARINVC_PO_ITEM AND(FPRPODT_CLOSED_IND<>'Y' OR FPRPODT_CLOSED_IND IS NULL)OR EXISTS(SELECT'x' FROM FIMSMGR.FGBENCD WHERE FGBENCD_NUM=X.FARINVC_POHD_CODE AND FGBENCD_ITEM=X.FARINVC_PO_ITEM AND(FGBENCD_STATUS<>'C' OR FGBENCD_STATUS IS NULL))) PARSING_USER_ID EXECUTIONS SORTS COMMAND_TYPE DISK_READS SQL_TEXT SELECT'1' INTO:b1 FROM SYS.DUAL WHERE EXISTS(SELECT NULL FROM GENERAL.GURMAIL WHERE GURMAIL_SYSTEM_IND=:b2 AND GURMAIL_DATE_PRINTED IS NULL AND GURMAIL_AIDY_CODE=:b3AND(GURMAIL_DATE_INIT+NVL (GURMAIL_WAIT_DAYS,0)) <=SYSDATE) PARSING_USER_ID EXECUTIONS SORTS COMMAND_TYPE DISK_READS SQL_TEXT delete from general.glbextr where glbextr_user_id = 'FAISMGR' and glbextr_selection in ('NWDPNS','NWDPS','NWINNS','NWINS', 'FA_NEW_INV_FALL','FA_INVITE_READMI','FA_NEW_INVITE') Page 160

161 Section H: Tuning and Monitoring Database Buffer Cache Purpose Performed after the tuning step for the other memory issues. The remaining memory may then be dedicated to Buffer Cache. Goal is to provide enough memory for data I/O that physical reads from disk are minimized. Examining Buffer Cache Use the view V$SYSSTAT, which records many different statistics about the database. Each row contains statistics for a particular kind of statistic. To examine the effectiveness of buffer cache, the following columns are of interest: NAME - Type of statistic. Of interest are the types: o DB BLOCK GETS - Total number of requests for data. o CONSISTENT GETS - Gets satisfied by access to buffers in memory. o PHYSICAL READS - Total requests resulting in access to disks. VALUE - The value for the particular statistic View statistics SQL> select name, value 2 from v$systatt 3 where name in ('db block gets', 'consistent gets', 'physical reads'); Name Value db block gets consistent gets physical reads Page 161

162 Section H: Tuning and Monitoring Database Buffer Cache (Continued) Analyze statistics From the returned values, one can see that a total of 364,680 requests for data resulted in a total of 23,182 physical reads. Hit ratio = 1 - (physical reads / (db block gets + consistent gets) o 1 - (23182 / ( )) o 1 - (23182 / ) o 1 - (0.064) = = 94% Hit ratio Buffer Cache Tuning Data Block requests should be from memory at least 90% of the time. If the read hit ratio is less than 90%, increase the Init.ora parameter DB_BLOCK_BUFFERS. Total Buffer Cache size = DB_BLOCK_BUFFERS * DB_BLOCK_SIZE. Page 162

163 Section H: Tuning and Monitoring Sorting Sorts Sorts involve memory structures called sort areas. Part of the runtime area In PGA in dedicated server mode In SGA in shared server mode (MTS) Must be balanced between: Large sorts Memory needs Increasing the size of the sort area will reduce the total number of disk sorts, thus reducing disk I/O. Swapping can occur if too little memory is left. Sort Issues Statements that will perform sorting: CREATE INDEX Select with Order By, Distinct, Group By, Union Unindexed Joins Some Correlated Subqueries Examining Sorts Use the view V$SYSSTAT, which records many different statistics about the database. Each row contains statistics for a particular kind of statistic. To examine the effectiveness of sorts, the following columns are of interest: NAME - Type of statistic. Of interest are the types: SORTS (MEMORY) - Sorts contained within memory. SORTS (DISK) - Sorts requiring temporary segments on disk. VALUE - The value for the particular statistic Page 163

164 Section H: Tuning and Monitoring Sorting (Continued) Example SQL> select name, value 2 from v$sysstat 3 where name in ('sorts (memory)', 'sorts (disk)'); NAME VALUE sorts (memory) 965 sorts (disk) 8 Sort tuning SORT_AREA_SIZE will allocate memory for sorting (Per process): Used to determine the space allocated in main memory for each process to perform sorts. If the sort cannot be performed in memory, temporary segments are allocated on disk to hold intermediate runs. SORT_AREA_RETAINED_SIZE can be used to release sort memory once the sort has completed. Page 164

165 Section H: Tuning and Monitoring I/O Bottlenecks Causes Controllers May be a source of contention if the activity is too great for the disks it is managing Disk Contention Due to more than one process requests data from the same disk at the same time Spread out different types of segments over different disks can help (i.e., Data and Indexes) Query to find the datafiles and number of reads and writes to each SQL> select df.file#, name, 2 phyrds, phywrts 3 from v$filestat fs, v$datafile df where fs.file# = df.file# FILE# NAME PHYRDS PHYWRTS /u01/oradata/prod/devl_prod_01.dbf /u02/oradata/prod/syst_prod_01.dbf /u03/oradata/prod/devl_prod_02.dbf /u04/oradata/prod/rbs_prod_01.dbf /u05/oradata/prod/temp_prod_01.dbf /u06/oradata/prod/user_prod_01.dbf /u01/oradata/prod/tool_prod_01.dbf /u02/oradata/prod/syst_prod_02.dbf /u03/oradata/prod/rbs_prod_02.dbf /u06/oradata/prod/trnd_data_prod_01.dbf /u06/oradata/prod/trnd_indx_prod_01.dbf Tuning I/O Query returns the username and the first 1000 bytes of the SQL statement having the highest number of disk reads per execution Designed to help determine the user generated SQL causing a large number of disk reads per statement execution Page 165

166 Section H: Tuning and Monitoring I/O Bottlenecks (Continued) Example select username, sql_text from v$sqlarea, v$session where address = sql_address and username is not null and disk_reads/executions = (select max(disk_reads/executions) from v$sqlarea, v$session where address = sql_address and username is not null); Note: This will also include such things as Job Submission and sleepwake jobs. To not see those, add this line to the both where clauses: and username not in ('GENLPRD','FIMSMGR') Script This script lists the disks, file names, reads, writes, and block reads and writes. Look for an even distribution across the disks. set term off timi off echo off feedback off pagesize 60 select c.tablespace_name, b.name, a.phyblkrd + a.phyblkwrt total, a.phyrds, a.phywrts, a.phyblkrd, a.phyblkwrt, ((a.readtim / decode(a.phyrds, 0, 1, a.phyblkrd)) / 100) avg_rd_time, ((a.writetim / decode(a.phywrts, 0, 1, a.phyblkwrt)) /100) avg_wrt_time from v$filestat a, v$datafile b, sys.dba_data_files c where b.file# = a.file# and b.file# = c.file_id order by tablespace_name; Note: System Manager must take into consideration other software running on the system. Page 166

167 Section H: Tuning and Monitoring I/O Bottlenecks (Continued) Balancing I/O If not in a RAID-5 environment, balance I/O by placing different types of segments on different drives. System Temporary Data Indexes Rollback Segments Redo Files Page 167

168 Section H: Tuning and Monitoring Chaining and Migration Spanning Caused when data from one row will not fit into a single Oracle block, causing it to chain data across several blocks More common with the datatypes of long, raw, and long raw Can only be fixed by recreating the database with a larger block size Chaining Caused when data is added to one row causing it to chain data across several blocks It does happen in Banner, so this must be monitored Row Migration Caused when a row grows and can't fit in the existing data block, it may migrate to another block Performance decreases due to additional I/O Can be fixed by correctly setting PCTFREE, PCTUSED Analyzing Chaining and Migration Use the Analyze command. Analyze [Table Index] objectname [compute estimate] statistics; SQL> analyze table spriden compute statistics; Table Analyzed Review Statistics in DBA_TABLES and/or DBA_INDEXES Only available if OPTIMIZER_MODE=CHOOSE or COST Page 168

169 Section H: Tuning and Monitoring Chaining and Migration (Continued) DBA_TABLES SQL> describe dba_tables; Name Null? Type OWNER NOT NULL VARCHAR2(30) TABLE_NAME NOT NULL VARCHAR2(30) TABLESPACE_NAME NOT NULL VARCHAR2(30) :: NUM_ROWS NUMBER BLOCKS NUMBER EMPTY_BLOCKS NUMBER AVG_SPACE NUMBER CHAIN_CNT NUMBER AVG_ROW_LEN NUMBER NUM_ROWS Total Rows in table BLOCKS Number of Oracle Blocks EMPTY_BLOCKS Blocks containing no data AVG_ROW_LEN (Bytes) AVG_SPACE Average Free Space (bytes) per block CHAIN_CNT Number of chained rows Note: Make sure OPTIMIZER_MODE is set to RULE before gathering statistics. Page 169

170 Section H: Tuning and Monitoring Redo Logs Redo logs Put on a separate drive from datafiles Multiplex If I/O is an issue, then rotate every other one on different drives (due to I/O of archiver fighting log writer) Distributing I/O: Archive Logs Archive logs should be on a separate spindle from all other datafiles/logfiles A must in production Archive to multiple locations: o By duplexing o By location Page 170

171 Section H: Tuning and Monitoring UNDO vs. Rollback Segments UNDO management As of Oracle 9i, Oracle has provided an automated Read consistency, UNDO management. Both provide same result. Can only have one defined at a time. UNDO provides retention. Note: The remainder of this topic is for historical purposes. Rollback Segments Oracle recommends the following: Number of Concurrent Transactions (X) Recommended Rollback Segments X<= 16 4 X > 16 < <= n X/4, but no more than 50 Usage Use large rollback segments for large transactions o Write programs with set transaction statement o Avoids Snapshot too old errors. If you set ROLLBACK_SEGMENTS, all rollback segments must be available. To ensure that a particular rollback is used, only use the ROLLBACK_SEGMENTS parameter. OLTP systems For OLTP systems, use smaller and more rollback segments. Use TRANSACTIONS_PER_ROLLBACK < 20 o If this is set in the init.ora, the ROLLBACK_SEGMENTS clause is ignored Use the OPTIMAL clause and set MAXEXTENTS to unlimited when creating them. Page 171

172 Section H: Tuning and Monitoring UNDO vs. Rollback Segments (Continued) V$ROLLSTAT and V$ROLLNAME Use the views V$ROLLSTAT and V$ROLLNAME. SQL> select name, waits, gets 2 from v$rollname v, v$rollstat s 3 where v.usn = s.usn; NAME WAITS GETS SYSTEM RS RS RS UNDO The only disadvantage, you can only have one UNDO tablespace per database. UNDO_RETENTION is new and can be set as to how long to retain undo data. OLTP systems demands are small Long running queries will require it most This parameter is in seconds. Review consumption in V$UNDOSTAT. Page 172

173 Section H: Tuning and Monitoring CPU Bottlenecks Causes CPU bottlenecks are often caused by ill-tuned memory and CPUs faster than controllers can handle Memory and I/O problems should be analyzed and corrected before replacing or upgrading the CPU Long-running resource intensive queries are run during normal business hours. CPU Bottleneck Reduction Review hardware and bus speed. Submit batch jobs on off hours Review memory requirements by collecting system statistics Handle user load by running in Shared server mode. Tune and optimize application code Page 173

174 Section H: Tuning and Monitoring Oracle Optimization Data access methods Oracle must decide how best to access the data: RULE based 20 predefined conditional rankings. COST based More sophisticated. Uses database information like table size, number of rows, key spread and several others Banner uses Rule. Enforce this in the init.ora! Optimizer_mode = rule Should I Use Optimizer? COST is not necessarily the only way to go. If every SQL is tuned, you do not need optimization. Do not analyze for the sake of analyzing. New thresholds may be met and the new plan is worse, not better. Use the DBMS_STATS package with the EXPORT_SCHEMA_STATS. You can revert back to the previous statistics. Optimizer Misconceptions RULE tuned SQL will run well in COST false. Hints cannot be used with the RULE based optimizer false. SQL tuned for COST will run well in rule highly unlikely You cannot run both RULE and COST mode false. If set to CHOOSE, only the tables analyzed will run cost. If COST or CHOOSE is set, COST will be used false. If no table has been analyzed, it defaults to RULE! Common RULE Problems Incorrect driving table 40% Incorrect Index 40% Incorrect driving index 10% Using the ORDER BY index, not the WHERE index 10% Page 174

175 Section H: Tuning and Monitoring Oracle Optimization (Continued) Driving Table If the driving table of a join is not optimal, longer elapsed time can result. It is important that the table that is listed last in the from clause returns the fewest rows. Index rankings is what Oracle uses for basing the driver table. If both tables are the same, the last one is used. Driving table example The parser processes the tables from left to right. (first 19 secs, second 1.9 secs) Select count(*) from acct a, trans b where b.cost_center = 'MASS' and a.acct_name = 'MGA' and a.acct_name = b.acct_name; Select count(*) from trans b, acct a where b.cost_center = 'MASS' and a.acct_name = 'MGA' and a.acct_name = b.acct_name; Incorrect Index and Driver If a WHERE clause item is a leading item in an index, that index will be used. If the item has a function on it, NO index will be used If multiple indexes exist AND satisfy the condition, the last one created is used. Cost-Based Tools Analyze creates execution plans that Oracle will use each time Dbms_stats new package Hints Outlines Page 175

176 Section H: Tuning and Monitoring Testing SQL Efficiency SQL trace facility and TKPROF SQL trace facility and TKPROF let you accurately assess the efficiency of the SQL statement. For best results, use in conjunction with EXPLAIN PLAN. SQL trace facility provides performance information and generate statistics. TKPROF formats the contents of the trace output and reports each statement with resource consumption. SQL Trace Several parameters need to be set: TIMED_STATISTICS MAX_DUMP_FILE_SIZE USER_DUMP_DEST Enable it per statement, session or system. Setting SQL_TRACE to true will cause a severe performance hit use wisely TKPROF Reads the output and formats it into a readable report. Each report will show: The statement Parse, execute and fetch Library cache utilization Execution plan Will not report commits or rollback Other tools STATSPACK extension of UTLBSTAT AND UTLESTAT. Collects more data. Creates a new schema and objects to store the information in. Will need 100MB for the PERFSTAT schema objects Script to run is $ORACLE_HOME/rdbms/admin/spcreate.sql TIMED_STATISTICS must be set to true Page 176

177 Section H: Tuning and Monitoring Summary Summary Tuning and monitoring are not tasks to take lightly. They will be the major part of a DBA's job. This section packed in a week's course with highlights. Nothing can replace taking a Performance and Tuning course. Page 177

178 Section I: Referential Integrity and Constraints Overview Introduction This section discusses the concept of referential integrity and how it is used. Intended audience Technical personnel who are responsible for the daily maintenance, support, and modification of SCT Banner. Objectives At the end of this section, you will be able to Manage parent and child tables via referential integrity Prerequisites To complete this section, you should have completed OR101 (Introduction to Oracle) exposure to SQL and SQL* Plus (OR102 Advanced PL/SQL and Database Objects is recommended) completed OR103 (Oracle Database Administration I) Section contents Referential Integrity Parent Table Child Table Maintaining Referential Integrity Constraints Enabling Constraints Resizing Tables Resizing Tables with RI Export Issues Import Issues How Does RI Affect Recovery? RI and Data Manipulation RI and Table/Row Locking Page 178

179 Section I: Referential Integrity and Constraints Referential Integrity Objectives What is Referential Integrity? Enabling constraints Resizing Tables Reorganizing Tables and Indexes Export/Import Issues Recovery Locking Issues What is Referential Integrity? Data Integrity Automatically Enforced Table Relationships Centralized Rules Data Integrity No corruption of data Relationships b/w data in tables are true Correct data is allowed Automatically Enforced Table Relationships No coding involved. SQL> delete from stvterm; * ERROR at line 1: Constraint (fk1_xxxxxx_in_stvterm_code) violated child record found Page 179

180 Section I: Referential Integrity and Constraints Referential Integrity (Continued) Centralized Rules Enforced in the database Keeps network traffic down All applications will abide by the rules Page 180

181 Section I: Referential Integrity and Constraints Parent Table Primary Keys SQL> ALTER TABLE saturn.stvschd ADD CONSTRAINT pk_stvschd PRIMARY KEY (stvschd_code); Naming standard: PK_'tablename' Creating and dropping When a primary key is created: primary key index is automatically created will always enforce unique values When a primary key is dropped: primary key index is automatically dropped will not enforce unique values Page 181

182 Section I: Referential Integrity and Constraints Child Table Foreign Keys SQL> ALTER TABLE saturn.saradhs ADD CONSTRAINT fk1_saradhs_inv_stvterm_code FOREIGN KEY (saradhs_term_code) REFERENCES saturn.stvterm(stvterm_code); Naming standard: FK#_'child table'_inv_'parent table'_code FK#_'child table'_inv_'parent table'_key Creating and dropping When a foreign key is created and enabled: will always enforce constraint 'bad' data will not be inserted/updated When a foreign key is dropped or disabled: will not enforce constraint 'bad' data could be entered into the table Page 182

183 Section I: Referential Integrity and Constraints Maintaining Referential Integrity Constraints Constraints in Banner Referential integrity adds new twists to the maintenance chores of a Banner DBA. Banner 4.x Constraints Primary Foreign Banner 5.x Constraints Primary Foreign Data Dictionary Tables DBA_CONSTRAINTS stores ALL constraints for ALL tables All Constraints All Tables Not Null SPRIDEN Primary Key STVTERM Referential APBCONS Check : SYS.TAB$ Page 183

184 Section I: Referential Integrity and Constraints Maintaining Referential Integrity Constraints (Continued) DBA_CONSTRAINTS SQL> describe dba_constraints Name Null? Type OWNER NOT NULL VARCHAR2(30) CONSTRAINT_NAME NOT NULL VARCHAR2(30) CONSTRAINT_TYPE VARCHAR2(1) TABLE_NAME NOT NULL VARCHAR2(30) SEARCH_CONDITION LONG R_OWNER VARCHAR2(30) R_CONSTRAINT_NAME VARCHAR2(30) DELETE_RULE VARCHAR2(9) STATUS VARCHAR2(8) DBA_CONS_COLUMNS Defines columns referenced in constraints SQL> describe dba_cons_columns Name Null? Type OWNER NOT NULL VARCHAR2(30) CONSTRAINT_NAME NOT NULL VARCHAR2(30) TABLE_NAME NOT NULL VARCHAR2(30) COLUMN_NAME NOT NULL VARCHAR2(30) POSITION NUMBER Page 184

185 Section I: Referential Integrity and Constraints Enabling Constraints Alter table enable alter table spraddr enable constraint fk1_spraddr_inv_code exceptions into exceptions; Exceptions table Column Name Datatype ROW_ID rowid OWNER varchar2(30) TABLE_NAME varchar2(30) CONSTRAINT varchar2(30) During data loads Where do you want to fix the problems? Before they're in the database After they're in the database What to do Delete the 'orphaned' data Insert missing data into parent table Update the 'orphaned' data Page 185

186 Section I: Referential Integrity and Constraints Resizing Tables Without Referential Integrity no defined table relationships With Referential Integrity possible defined table relationships Page 186

187 Section I: Referential Integrity and Constraints Resizing Tables with RI DROP restrictions Can't: DROP A TABLE SQL> DROP TABLE stvterm; Error at line 1: ORA-02266: table has some unique/primary keys that are referenced by some foreign keys TRUNCATE restrictions Can't: TRUNCATE A TABLE SQL> TRUNCATE TABLE stvterm; Error at line 1: ORA-02266: table has some unique/primary keys that are referenced by some foreign keys Note: A table can be truncated if: No referencing tables have data referencing the table. Referencing constraint are disabled. Dropping with CASCADE Can: DROP WITH CASCADE ( This can be dangerous ) SQL> DROP TABLE stvterm CASCADE CONSTRAINTS; Table Dropped. Table is dropped. Referencing foreign keys are dropped. No way to dynamically rebuild foreign keys A method of resizing * Use the GURRDDL script provided with SCT Banner's General software. Page 187

188 Section I: Referential Integrity and Constraints Export Issues Exporting Using constraints=n $ exp saturn/u_pick_it file=stvterm.dmp table=stvterm constraints=n Exported: Table definition Table data Index definition(s) * Not Exported: Primary key definitions * Referential constraints * Primary key index is not part of the index definitions. Page 188

189 Section I: Referential Integrity and Constraints Import Issues Order of Table import Table is created Table data is loaded Primary Key constraint created * Indexes created Foreign Key constraints created ** * Primary key index is created at this time. ** Data is verified before constraint is enabled. Order of Full database import Table is created Table data is loaded Primary Key constraint created * import repeats steps 1-3 for all tables, then Indexes created Foreign Key constraints created ** * Primary key index is created at this time. ** Data is verified before constraint is enabled. Errors Errors Importing Primary Key * definition of primary key not imported primary key index not created Errors Importing Foreign Key * definition of foreign key not imported * Usually caused by data existing prior to import. Using ignore=y on import will not remove existing data. Page 189

190 Section I: Referential Integrity and Constraints Import Issues (Continued) Indexes Using indexfile=filename.sql does not include primary key indexes Using indexes=n imports the constraints still imports the primary key index Page 190

191 Section I: Referential Integrity and Constraints How Does RI Affect Recovery? Recovery types Full database recovery o offline o online o archiving Partial database recovery Table/Data recovery Page 191

192 Section I: Referential Integrity and Constraints RI and Data Manipulation DML processing Oracle checks RI after the entire statement completes. sql> update spraddr set spraddr_atyp_code = 'MA'; Page 192

193 Section I: Referential Integrity and Constraints RI and Table/Row Locking Index considerations With indexes on foreign key columns Row level locking is performed Without indexes on foreign key columns Table level locking is performed Page 193

194 Section J: Oracle Net Overview Introduction This section discusses Oracle Net and its components. Intended audience Technical personnel who are responsible for the daily maintenance, support, and modification of SCT Banner. Objectives At the end of this section, you will be able to Describe Oracle Net and its components Tune configuration files Prerequisites To complete this section, you should have completed OR101 (Introduction to Oracle) exposure to SQL and SQL* Plus (OR102 Advanced PL/SQL and Database Objects is recommended) completed OR103 (Oracle Database Administration I) Section contents Oracle Net Oracle Internet Directory Local Net Service Name LISTENER.ORA LSNRCTL Command Multiple Listeners No Listener Entry Resolving service_name Errors Log and Trace Files Tuning listener.ora, tnsnames.ora, sqlnet.ora SNMP Oracle Connection Manager Connection Filtering Summary Page 194

195 Section J: Oracle Net Oracle Net Description Originally released as SQL*Net, renamed Net8, and now known as Oracle Net The Oracle networking facility that allows client workstations to communicate with a database server through a network Facilitates the sharing of data between databases regardless of the machines' architecture Overview It is backward compatible to only Allows a client to send database requests over a network protocol Now supports the multi-tier architecture Servers can now support client operations to allow implementation of a thin client The end result is the ability to communicate with all databases that are accessible Structures Oracle Net relies on configuration files on the client and server to manage database connectivity Several methods are used and utilize different files depending on if it is a client or server Page 195

196 Section J: Oracle Net Oracle Internet Directory Description Provides a central repository for the network The most common directory form is Lightweight Directory Access Protocol (LDAP) Oracle supports both the Oracle Internet Directory (OID) and Microsoft Active Directory Functionality Uses ldap.ora for its configuration file Resolves a connect descriptor as follows: o Oracle Net contacts the directory to obtain the connect identifier o The directory server takes the connect identifier, locates the descriptor and returns it to Oracle Net o Oracle Net uses the resolved descriptor and makes a connection to the correct listener Common LDAP attributes are used OID Setup Oracle Net Configuration Assistant or Oracle Net Manager can be used for initial setup Oracle Net Manager is used to register the names A server is setup utilizing the Database Configuration Assistant (DBCA) Oracle Names Since Oracle intends to desupport this option soon, it will not be covered. Basic premise was using GLOBAL Names for managing distributed database Page 196

197 Section J: Oracle Net Local Net Service Name TNSNAMES.ORA TNSNAMES.ORA is the configuration file Resides on any Oracle-based machine that will utilize client connections, even the server Resides on the client computer or on a fileserver accessible to the client user Has aliases and destination addresses for all databases accessible to the user with access to this file TNSNAMES.ORA example ################ # Filename...: tnsnames.ora # Node...: bpx2.xyz.edu # Date...: 16-SEP-97 14:04:18 ################ IND215_DB= (DESCRIPTION= (ADDRESS_LIST= (ADDRESS= (PROTOCOL=TCP) (HOST=mickey.xyz.edu) (PORT=1526) ) ) (CONNECT_DATA=(SID=IND215) ) ) Oracle Net connection Client computer TNSNAMES.ORA NETWORK Database Server LISTENER.ORA Page 197

198 Section J: Oracle Net LISTENER.ORA LISTENER.ORA Resides on the database server Can be configured to contain dedicated servers with a SID Can be unconfigured ( more later) LISTENER.ORA example ################ # Filename...: listener.ora # Name...: xyz.edu # Date...: 03-JAN-98 ################ LISTENER = (ADDRESS_LIST= (ADDRESS= (PROTOCOL=TCP) (HOST=mickey.sctcorp.com) (PORT=1526) ) ) STARTUP_WAIT_TIME_LISTENER=0 CONNECT_TIMEOUT_LISTENER=10 TRACE_LEVEL_LISTENER=ON SID_LIST_LISTENER= (SID_LIST= (SID_DESC= (SID_NAME=IND215) (PROGRAM=sys$sysdevice:[oracle7.715_2_3.network.admin]orasrv_v2_ind2 15.com) (TIMEOUT=0) ) ) Page 198

199 Section J: Oracle Net LSNRCTL Command Functions The LSNRCTL command is used to administer the SQL*Net listener on the database server. There are four basic functions that the DBA performs with the LSNRCTL tool: start the listener stop the listener status of the listener reload the listener LSNRCTL Command - Help To start the command, log in as the oracle user account and type lsnrctl $ lsncrtl LSNRCTL> help The following operations are available An asterisk (*) denotes a modifier or extended command: start stop status services version reload trace spawn dbsnmp_start dbsnmp_stop dbsnmp_status change_password quit exit set* show* LSNRCTL Command - Start LSNRCTL> start Connecting to (ADDRESS=(PROTOCOL=TCP)(HOST=sctrs1)(PORT=2337)) The command completed successfully *This command starts the listener so that users may access the database. Page 199

200 Section J: Oracle Net LSNRCTL Command (Continued) LSNRCTL Command - Stop LSNRCTL> stop Connecting to (ADDRESS=(PROTOCOL=TCP)(HOST=sctrs1)(PORT=2337)) The command completed successfully *This command stops the listener so that no further connections can be made to the database. LSNRCTL Command - Reload LSNRCTL> reload Connecting to (ADDRESS=(PROTOCOL=TCP)(HOST=sctrs1)(PORT=2337)) The command completed successfully *This command reads the listener.ora file and starts all services added to the file without altering the services that are currently running. Page 200

201 Section J: Oracle Net Multiple Listeners Multiple listeners The DBA can define as many listeners as he/she wants, each with its own set of attached databases, provided that each listener runs from a separate TCP port. Starting With multiple listeners, each must be started individually under lsnrctl. This allows the DBA greater control over database access. Example Example of a listener called 'firstlistener' follows. FIRSTLISTENER = (ADDRESS_LIST= (ADDRESS = (PROTOCOL = TCP) (HOST=my.test.edu) (PORT=1526) ) ) STARTUP_WAIT_TIME_FIRSTLISTENER=0 CONNECT_TIMEOUT_FIRSTLISTENER=60 SID_LIST_FIRSTLISTENER (SID_LIST = (SID_DESC = (SID_NAME=SEED) (ORACLE_HOME=/u01/oracle/product/8.1.6.#) ) ) Page 201

202 Section J: Oracle Net No Listener Entry service_name Oracle 8i introduced 'service_name' Advantages: If instance is not up, no listener defined No configuration needed Disadvantage: Older versions of SQL*Net may have connect problems. Requirements initsid.ora Service_names Instance_name Db_domain E.g.: instance_name = trng service_names = (trng_db, trng_ms, trng_web) Tnsnames.ora Sid_name replaced by service_name Note: It is "SERVICE_NAME" and not "SERVICE_NAMES" as defined in "init<sid>.ora". E.g.: TRNG_DB = (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP) (HOST = oracle8i) (PORT = 1521) ) ) (CONNECT_DATA = (SERVICE_NAME = trng_db) ) ) Page 202

203 Section J: Oracle Net Resolving service_name Errors Errors The new error: "ORA :TNS:listener could not resolve SERVICE_NAME given in connect descriptor" is similar to the the error: "ORA :TNS: Could not resolve service name" InitSID.ora One of the values is missing: service_names db_domain Check what is in the listener.ora Global_dbname (?) Resolution If both db_domain and service_names are set: Tnsnames.ora entry (service_name = service_names.db_domain) If only service_names: Use service_names If neither are set: Use db_name Page 203

204 Section J: Oracle Net Log and Trace Files sqlnet.log sqlnet.log will be very beneficial in troubleshooting connections. To identify the relevant part of the problem, you will need to turn on tracing. There are three main areas to trace: The SQL*NET client The 'listener' process The SQL*NET 'server' Trace File Levels Establishing a connection Client---- Listener --- Server An established connection Client --- Server 1 3 Client Level Tracing Add in the file sqlnet.ora Output sent to /tmp/filename_<pid> Trace_level_server=## Trace_file_server =filename Trace_directory_server =/tmp Listener Level Tracing Add in the file Listener.ora Trace_level_listener=## Trace_file_listener=filename Trace_directory_listener=/tmp This defines FULL tracing in /tmp/filename.trc Server Level Tracing Add in the file sqlnet.ora Output sent to /tmp/filename_<pid> Trace_level_server=## Trace_file_server =filename Trace_directory_server =/tmp Page 204

205 Section J: Oracle Net Tuning listener.ora, tnsnames.ora, sqlnet.ora Listener.ora Prespawn_max timeout Pool_size queuesize Tnsnames.ora Packet transferring SDU Sqlnet.ora Names.default_domain Names.directory_path Page 205

206 Section J: Oracle Net SNMP SNMP Support integrates the management of Oracle products into a number of existing, widelyused management systems Enables key Oracle products running anywhere on an enterprise's network to be located, identified, and monitored by a management station running at a centrally located node Any network that supports SNMP can use this management station Page 206

207 Section J: Oracle Net Oracle Connection Manager Oracle Connection Manager Acts as a router used to establish database communication links between otherwise incompatible network protocols Can be used as a firewall for the Oracle Net Traffic Page 207

208 Section J: Oracle Net Connection Filtering Rules The rules can be based on any of the following: Destination host names or IP addresses for servers Destination database service name Source host names or IP addresses for clients Used to enhance your firewall security Page 208

209 Section J: Oracle Net Summary Summary Oracle Net provides a multitude of network solutions to protect your database The type and use are tunable and changeable The underlying issue of what is more important will drive your Oracle Net setup survivability versus performance Page 209

210 Section K: Backup and Recovery Overview Introduction This section provides a brief overview of Backup and Recovery techniques and tools. Intended audience Technical personnel who are responsible for the daily maintenance, support, and modification of SCT Banner. Objectives At the end of this section, you will be able to Ensure that you have a workable Disaster and Recovery plan Practice scenarios Review dynamic views and alert_sid.log for any problems Prerequisites To complete this section, you should have completed OR101 (Introduction to Oracle) exposure to SQL and SQL* Plus (OR102 Advanced PL/SQL and Database Objects is recommended) completed OR103 (Oracle Database Administration I) Section contents Introduction Loss of Data Redo Logs Perform Online Database Backup Types of Media Recovery Applying Archived Redo Logs Performing Incomplete Recovery Creating a New Control File Tablespace Point In Time Recovery (TSPITR) Other Methods Summary Page 210

211 Section K: Backup and Recovery Introduction Why Develop a Disaster and Recovery Plan? To guard against failures of media, operating systems, software, and any other serious failures that may cause loss of vital data The better the backup plan, the more recovery choices you have A proper plan will require discipline and practice Incomplete recovery is the number one cause for total loss of data What Causes Failure? Most sites will have a plan and don't test it. IEEE groups failure into four groups: Physical Hardware failure Design Software bugs Operations Human intervention, poor DBA skills, user errors, inappropriate or inadequate system setup of backup Environmental 'Act of GOD!' Prevention You can control 'Operations', but not easily control the others. Poor DBA Skills Train, certify, and practice, practice, practice User Error Increase security, bulletproof software Inadequate backup procedures create a plan that can be tested Objectives Learn the Backup/Recovery mechanisms of Oracle Differentiate between complete and incomplete recoveries When to use cancel-based and time-based incomplete recovery methods Using RMAN Learn what LogMiner can do for you Learn other mechanisms to detect and repair data block corruption Page 211

212 Section K: Backup and Recovery Loss of Data Loss of Data Failures Media failure prohibits the reading from and/or the writing to a file or disk Memory failure prohibits the writing of dirty buffers or redo entries to disk. Can crash the database hard and corrupt it File systems become corrupt and data blocks become unusable Users delete the wrong table/record or value The computer crashes and no system is available Common Sense items First and foremost, create a backup and recover plan. Ask the operative questions: How much are you willing to lose? If the system is unusable, how do we replace it? When can the system be down for backups? What Do I Need to Know? BACKUP, BACKUP, and then BACKUP! Have a knowledge of items like: Standby database Multi-master /Advanced Replication RAC Log Miner RMAN Page 212

213 Section K: Backup and Recovery Redo Logs Redo logs All changes made to the Database are recorded in the online redo logs. Current redo log information is stored in Every control file owned by the database Every data file owned by the database The redo log files owned by the database These files can be overwritten! Common method is archiving the online redo logs. Redo Log Structures All of the database files are updated by the LGWR process at the completion of a checkpoint The log sequence number is stored in block 1 of each data file When the current log fills up, a log switch occurs and the sequence number information is updated In ARCHIVELOG mode, the filled log file is immediately archived to LOG_ARCHIVE_DEST Note: Production databases should always be run in ARCHIVELOG mode Archive Log Method Example: View log file and archivelog mode from the data dictionary SQL> CONNECT / as sysdba Connected. SQL> ARCHIVE LOG LIST; Database log mode Automatic archival Archive destination Oldest online log sequence Next log sequence to archive Current log sequence Archive Mode Enabled /usr04/oradata/archives/archivelog_ Page 213

214 Section K: Backup and Recovery Redo Logs (Continued) Log File Information Example: View log file details and archive status SQL> COL group# FORMAT SQL> COL thread# FORMAT SQL> COL members FORMAT SQL> COL status FORMAT A9 SQL> COL first_change# FORMAT SQL> SELECT * FROM V$LOG ; GROUP# THREAD# SEQUENCE# BYTES MEMBERS ARC STATUS FIRST_CHANGE# FIRST_TIME NO CURRENT /21/98 13:11: NO INACTIVE /19/98 12:54: NO INACTIVE /19/98 15:09:10 Archiving Redo Logs NOARCHIVELOG mode means database can only be recovered from last full cold backup ARCHIVELOG mode means database can be recovered by Complete recovery Incomplete recovery Offline tablespace recovery Offline tablespace individual data file recovery Page 214

215 Section K: Backup and Recovery Perform Online Database Backup Steps Database MUST be in ARCHIVELOG mode Determine files needed to be backed up Alter tablespace(s) to be backed up Backup tablespace data files Alter tablespace(2) after backup [END BACKUP] Backup control file Note: Oracle does not recommend backing up multiple tablespaces at one time! RMAN reads each block such as a query. No manual intervention necessary Check Archivelog Mode Example: Check to ensure database is in ARCHIVELOG mode SQL> CONNECT / as sysdba Connected. SQL> ARCHIVE LOG LIST; Database log mode Automatic archival Archive destination Oldest online log sequence Next log sequence to archive Current log sequence Archive Mode Enabled /usr04/oradata/archives/archivelog_ List Current Tablespaces Example: Obtain file names owned by tablespace(s) to be backed up SQL> COL file_name FORMAT A40 SQL> COL tablespace_name FORMAT A15 SQL> SELECT file_name, tablespace_name, status, bytes 2 FROM sys.dba_data_files 3 ORDER BY tablespace_name, file_name; Page 215

216 Section K: Backup and Recovery Perform Online Database Backup (Continued) Tablespace List FILE_NAME TABLESPACE_NAME STATUS BYTES /u04/oradata/e300/devl_e300_02.dbf DEVELOPMENT AVAILABLE /u05/oradata/e300/devl_e300_01.dbf DEVELOPMENT AVAILABLE /u04/oradata/e300/rbs_e300_01.dbf RBS AVAILABLE /u04/oradata/e300/syst_e300_01.dbf SYSTEM AVAILABLE /u04/oradata/e300/temp_e300_01.dbf TEMP AVAILABLE /u04/oradata/e300/tools_e300_01.dbf TOOLS AVAILABLE /u04/oradata/e300/users_e300_01.dbf USERS AVAILABLE rows selected. Start Online Backup Example: Alter tablespace to reflect the beginning of a backup sequence SQL> ALTER TABLESPACE development BEGIN BACKUP; Statement processed. Backup Underlying Datafiles Example: Perform backup of ALL data files owned by this tablespace (UNIX illustrated) find /*/oradata/e300 -name '*devl*' print cpio ocvb > /dev/rmt/0n /u04/oradata/e300/devl_e300_02.dbf blocks /u05/oradata/e300/devl_e300_01.dbf blocks Put Tablespace Back Online Example: Alter tablespace to reflect completion of backup sequence SQL> ALTER TABLESPACE development END BACKUP; Statement processed. Page 216

217 Section K: Backup and Recovery Perform Online Database Backup (Continued) Do Not Forget the Control File The control file should be backed up at the same time Always backup the control file if any changes to the database occur: SQL> ALTER DATABASE BACKUP CONTROLFILE TO '/u01/cntrle300_01.ctl' REUSE; Database altered. Verify Backup V$BACKUP lists all online files and gives their backup statuses Select file#, status from v$backup; A status field of active means being backed up. Caution! if you have restored a file and/or control file, this view will be inaccurate! DBVERIFY utility Page 217

218 Section K: Backup and Recovery Types of Media Recovery Types Uses redo records, or you can perform incremental if you use RMAN. The main ones are: Complete Recovery Incomplete Recovery Opening the Database with the RESETLOGS Option Complete Recovery Uses redo data or incremental backups combined with a backup of the database Called complete because Oracle applies all of the redo changes Minimizes amount of work lost Can be restored to the point of failure Incomplete Recovery Recovery will not necessarily be to the point-of-failure. Media Recovery options: Recover until termination by user entering CANCEL Recover until all committed changes up to a specified TIME have been applied* Recover until all committed changes up to a named SCN have been applied* Recover until a specified log sequence number* * RMAN TYPES Opening with RESETLOGS Whenever a complete or incomplete recovery is performed using a backup control file, you must reset the redo logs All archives from this point back are no longer viable You essentially have a new incarnation Page 218

219 Section K: Backup and Recovery Applying Archived Redo Logs Applying Archived Redo Logs After restoring data files from backup, the sequence numbers in the header blocks of the data files will not coincide with the current sequence number in the control file Recovery is required Once recovery begins, the necessary redo log files are applied to reconstruct the data files Automatic application Oracle will suggest the next redo log to apply. To automatically apply redo log files, issue the SET AUTORECOVERY ON before beginning media recovery. Page 219

220 Section K: Backup and Recovery Performing Incomplete Recovery Steps Shutdown the database Execute a full database backup o If media failure has occurred, correct the failure Restore control file if necessary o Control files should be multiplexed to ensure an up-to-date copy of the control file Restore data files Restore necessary archived redo logs o If the archived redo logs will not fit in LOG_ARCHIVE_DEST, place log files in alternate path and log into Oracle SQLPLUS / AS SYSDBA to SET LOGSOURCE parameter Connect to the database and startup the database with the MOUNT option Place data files online Attempt recovery After successful recovery, bring database online Shutdown the database Example: Shutdown the database SQL> SHUTDOWN [IMMEDIATE ABORT] Database closed. Database dismounted. ORACLE instance shut down. SQL> Page 220

221 Section K: Backup and Recovery Performing Incomplete Recovery (Continued) Perform a full database backup Example: Perform a full database backup [Repair media failure if necessary] find /*/oradata/e300 -depth print cpio ocvb > /dev/rmt/0n /u01/oradata/e300/cntrl_e300_01.dbf xxx blocks /u02/oradata/e300/cntrl_e300_02.dbf xxx blocks /u03/oradata/e300/cntrl_e300_03.dbf xxx blocks /u04/oradata/e300/devl_e300_02.dbf xxx blocks /u04/oradata/e300/rbs_e300_01.dbf xxx blocks /u04/oradata/e300/temp_e300_01.dbf xxx blocks /u04/oradata/e300/users_e300_01.dbf xxx blocks /u04/oradata/e300/syst_e300_01.dbf xxx blocks /u04/oradata/e300/tools_e300_01.dbf xxx blocks /u04/oradata/e300/log_e300_01a.dbf xxx blocks /u04/oradata/e300/log_e300_02a.dbf xxx blocks /u04/oradata/e300/log_e300_03a.dbf xxx blocks /u05/oradata/e300/devl_e300_01.dbf xxx blocks Restore control file if necessary Example: Restore control file if necessary [Should not be necessary if control files are multiplexed] cpio icvdb < /dev/rmt/0n /u01/oradata/e300/cntrl_e300_01.dbf Restore data files to original locations Example: Restore data files to original locations cpio icvdb < /dev/rmt/0n /*/oradata/e300/* /u04/oradata/e300/devl_e300_02.dbf xxx blocks /u04/oradata/e300/rbs_e300_01.dbf xxx blocks /u04/oradata/e300/temp_e300_01.dbf xxx blocks /u04/oradata/e300/users_e300_01.dbf xxx blocks /u04/oradata/e300/syst_e300_01.dbf xxx blocks /u04/oradata/e300/tools_e300_01.dbf xxx blocks /u04/oradata/e300/log_e300_01a.dbf xxx blocks /u04/oradata/e300/log_e300_02a.dbf xxx blocks /u04/oradata/e300/log_e300_03a.dbf xxx blocks /u05/oradata/e300/devl_e300_01.dbf xxx blocks Page 221

222 Section K: Backup and Recovery Performing Incomplete Recovery (Continued) Restore necessary archived redo logs Example: Restore necessary archived redo logs cpio icvdb < /dev/rmt/0n /usr04/oradata/archives/archivelog_* /usr04/oradata/archives/archivelog_arce300_13064.arc /usr04/oradata/archives/archivelog_arce300_13065.arc /usr04/oradata/archives/archivelog_arce300_13066.arc /usr04/oradata/archives/archivelog_arce300_13067.arc /usr04/oradata/archives/archivelog_arce300_13068.arc /usr04/oradata/archives/archivelog_arce300_13069.arc Connect to and startup database Connect to the database, startup database with MOUNT option /> SQL SQL> CONNECT INTERNAL Connected to an idle instance. SQL> STARTUP MOUNT ORACLE instance started. Total System Global Area bytes Fixed Size bytes Variable Size bytes Database Buffers bytes Redo Buffers bytes Database mounted. SQL> Page 222

223 Section K: Backup and Recovery Performing Incomplete Recovery (Continued) Make sure data files are online Example: Make sure data files are online If a datafile has a status of OFFLINE, you must alter the database to bring the data file online. SQL> SELECT * FROM V$DATAFILE; FILE# STATUS ENABLED CHECKPOINT BYTES CREATE_BYT NAME SYSTEM READ WRITE /u06/oradata/e300/syst_e300_01.dbf 2 ONLINE READ WRITE /u05/oradata/e300/rbs_e300_01.dbf 3 ONLINE READ WRITE /u05/oradata/e300/temp_e300_01.dbf 4 ONLINE READ WRITE /u06/oradata/e300/tools_e300_01.dbf 5 ONLINE READ WRITE /u05/oradata/e300/users_e300_01.dbf 6 ONLINE READ WRITE /u02/oradata/e300/devl_e300_01.dbf 7 OFFLINE READ WRITE /u01/oradata/e300/devl_e300_02.dbf 7 rows selected. SQL> SQL> ALTER DATABASE DATAFILE '/u01/oradata/e300/devl_e300_02.dbf' ONLINE; Statement processed. SQL> Note: The status of the file will be set to ONLINE, whether or not the file exists. It is only the status that is updated in the control file and the file will need to be recreated if it does not exist before recovery can be performed Note: If a tablespace is brought offline, all files belonging to the tablespace will have a status of OFFLINE Attempt an incomplete recovery Example: Attempt an incomplete recovery SQL> ALTER DATABASE RECOVER DATABASE AUTOMATIC UNTIL TIME ' :16:30:00'; or SQL> ALTER DATABASE RECOVER DATABASE AUTOMATIC UNTIL CANCEL or SQL> ALTER DATABASE RECOVER DATABASE AUTOMATIC UNTIL CHANGE 600; Page 223

224 Section K: Backup and Recovery Performing Incomplete Recovery (Continued) Bring the database online Example: If recovery is successful, bring the database online SQL> ALTER DATABASE OPEN RESETLOGS; Statement processed. Note: If recovery was actually complete, do not reset the log sequence number Note: When recovery is finished, Oracle records the completion in the alert_sid.log Page 224

225 Section K: Backup and Recovery Creating a New Control File Why Create a New Control File To recreate the control file when all control files are damaged and there are no backups available To change the name of the database (cloning) To change one of the permanent database settings Note: You must know the full path names to all of the data files and redo log files to recreate the control file Steps Shutdown the database Backup all database files Start the new instance and create the control file Mount and recover the database Open the database Login and shutdown the database Example: Login and shutdown the database SQL> SHUTDOWN [IMMEDIATE ABORT] Database closed. Database dismounted. ORACLE instance shut down. SQL> Page 225

226 Section K: Backup and Recovery Creating a New Control File (Continued) Generate a full database backup Example: Generate a full database backup in case of problems find /*/oradata/e300 -depth print cpio ocvb > /dev/rmt/0n /u01/oradata/e300/cntrl_e300_01.dbf xxx blocks /u02/oradata/e300/cntrl_e300_02.dbf xxx blocks /u03/oradata/e300/cntrl_e300_03.dbf xxx blocks /u04/oradata/e300/devl_e300_02.dbf xxx blocks /u04/oradata/e300/rbs_e300_01.dbf xxx blocks /u04/oradata/e300/temp_e300_01.dbf xxx blocks /u04/oradata/e300/users_e300_01.dbf xxx blocks /u04/oradata/e300/syst_e300_01.dbf xxx blocks /u04/oradata/e300/tools_e300_01.dbf xxx blocks /u04/oradata/e300/log_e300_01a.dbf xxx blocks /u04/oradata/e300/log_e300_02a.dbf xxx blocks /u04/oradata/e300/log_e300_03a.dbf xxx blocks /u05/oradata/e300/devl_e300_01.dbf xxx blocks Start the database using NOMOUNT Example: Start the database using the NOMOUNT option and create the new control file SQL> CONNECT INTERNAL Connected to an idle instance. SQL> STARTUP NOMOUNT ORACLE instance started. Total System Global Area bytes Fixed Size bytes Variable Size bytes Database Buffers bytes Redo Buffers bytes SQL> START createe300_ctl.sql Statement processed. Page 226

227 Section K: Backup and Recovery Creating a New Control File (Continued) Create the new control file # createe300_ctl.sql create controlfile reuse set database E300 logfile '/u04/oradata/e300/log_e300_01a.dbf', '/u04/oradata/e300/log_e300_02a.dbf', '/u04/oradata/e300/log_e300_03a.dbf' noresetlogs datafile '/u04/oradata/e300/devl_e300_02.dbf', '/u04/oradata/e300/rbs_e300_01.dbf', '/u04/oradata/e300/temp_e300_01.dbf', '/u04/oradata/e300/users_e300_01.dbf', '/u04/oradata/e300/syst_e300_01.dbf', '/u04/oradata/e300/tools_e300_01.dbf', '/u05/oradata/e300/devl_e300_01.dbf' MAXLOGFILES 10 MAXDATAFILES 40 ARCHIVELOG; Mount the database and recover Example: After successful creation of the control file(s), mount the database and recover. SQL> ALTER DATABASE MOUNT; Statement processed. SQL> RECOVER DATABASE; Media recovery complete. Note: If the RESETLOGS option was used in creating the control file, perform an incomplete cancel-based recovery SQL> ALTER DATABASE MOUNT; Statement processed. SQL> RECOVER DATABASE USING BACKUP CONTROLFILE UNTIL CANCEL; Bring the database online Example: Bring the database online SQL> ALTER DATABASE OPEN; Statement processed. Page 227

228 Section K: Backup and Recovery Tablespace Point In Time Recovery (TSPITR) TSPITR Can be done by either an O/S method or with transportable tablespaces Fundamental difference is that with transportable, you can recover a dropped tablespace Each require a restore or another cloned database to recover from Steps Restore the clone to another system Take the bad tablespace offline Restore the required archived redo logs (for O/S method) Connect to the database and recover tablespace or data file as needed Bring the tablespace online Connect to database and bring the tablespace offline Example: Connect to database and bring the tablespace offline SQL> ALTER TABLESPACE development OFFLINE; Statement processed. Restore old copies Example: Restore old copies of the damaged data files (Assumption: replace all data files for DEVELOPMENT tablespace) cpio icvdb < /dev/rmt/0n /*/oradata/e300/devl_e30_* /u04/oradata/e300/devl_e300_02.dbf xxx blocks /u04/oradata/e300/devl_e300_01.dbf xxx blocks Page 228

229 Section K: Backup and Recovery Tablespace Point In Time Recovery (TSPITR) (Continued) Restore copies of necessary archive logs Example: Restore copies of necessary archive logs cpio icvdb < /dev/rmt/0n /usr04/oradata/archives/archivelog_* /usr04/oradata/archives/archivelog_arce300_13064.arc /usr04/oradata/archives/archivelog_arce300_13065.arc /usr04/oradata/archives/archivelog_arce300_13066.arc /usr04/oradata/archives/archivelog_arce300_13067.arc /usr04/oradata/archives/archivelog_arce300_13068.arc /usr04/oradata/archives/archivelog_arce300_13069.arc Reconnect to database and initiate the recovery Example: Reconnect to database and initiate the recovery SQL> RECOVER DATAFILE '/u04/oradata/e300/devl_e300_01.dbf', '/u04/oradata/e300/devl_e300_02.dbf'; Bring tablespace back online Example: Bring tablespace back online SQL> ALTER TABLESPACE development ONLINE; Statement processed. Page 229

230 Section K: Backup and Recovery Other Methods Methods RMAN Transportable tablespaces Enterprise Manager LogMiner Third Party options RMAN Has been available since 8i If you are using 9i or moving to 9i, use 9i version to manage 9i and 8i databases Capability to work multi-vendor backup software (Veritas, Tivoli, etc.) Can be scheduled to run in batch and off hours Fully explained and used in BACKUP and RECOVERY course RMAN Methods Two ways to implement RMAN: In the associated database This takes up precious space and resources In a separate 'RMAN' or command center database Database-independent Can be used for 9i down to databases o Install 9i version only Simple Steps for RMAN Create a RMAN tablespace Create the RMAN user Grant recovery_catalog_owner to RMAN Log into the database and create the RMAN catalog: rman catalog rman_901/rman_901 RMAN> create catalog; Page 230

231 Section K: Backup and Recovery Other Methods (Continued) LogMiner Can be used to recover dropped, deleted and other unwanted DML actions Can be set to automatically record SQL DDL statements (alter/create/drop) Can be used against online or archived logs of the 'current' or 'foreign' database Uses an external dictionary file LogMiner limitations LogMiner does not: Trace application SQL Replicate an application Work in noarchive log mode LogMiner Exports LogMiner has three methods of exporting the changes To a flat file To redo logs Online data dictionary Additional methods DBMS_REPAIR DBVERIFY ANALYZE DB_BLOCK_CHECKING Page 231

232 Section K: Backup and Recovery Other Methods (Continued) Preventable Options STANDBY MULTI-MASTER PARALLEL SERVER ADVANCED REPLICATION MIRRORING TRANSPORTABLE TABLESPACES ENTERPRISE MANAGER CONSOLE Dynamic Views V$DATABASE displays archivelog mode V$ARCHIVE_LOG control file info on log files V$LOG displays all online redo log groups V$LOG_HISTORY SCN ranges and which logs have been archived V$DATAFILE Lists datafile header information V$BACKUP Backup status of online datafiles Page 232

233 Section K: Backup and Recovery Summary Summary Ensure that you have a workable Disaster and Recovery plan Practice scenarios Review dynamic views and alert_sid.log for any problems Page 233

234 Release Date This workbook was last updated on 11/18/2005.