Oracle Database 10g Extensibility Framework:

Transcription

1 Oracle Database 10g Extensibility Framework: Building, Deploying, and Managing Data Cartridges Geoff Lee Principal Product Manager, Oracle Corporation

2 Agenda Overview Oracle Database 10g Extensibility Framework Demo Summary Q & A

3 Specialty Data in Life Sciences Genomic Proteomic Chemical Microarrays Virtual Screening

4 Managing All Your Data Consistent, universal access Easy to search and find any content Easy to develop applications Easy to deploy with enterprise Quality of Service Easy to manage Documents Static Web Pages Dynamic Web Pages Structured Data Legacy Data Text Audio Images Video Locations Oracle10g

5 Oracle Extensible Platform Text Spatial Utilities Tools Text Spatial Image Image Data Cartridges Optimizer Query Engine Index Engine Type Manager Extensibility T e x t S p a t I a l I m a g e

6 Specialty Data Cartridges Data Cartridge Oracle8i Server Server Oracle10g Service Interfaces Extensibility Interfaces Type System Server Query Processing Execution... Data Indexing Database Extensibility Services

7 Evolution of the Extensibility Framework Extensible Type system Extensible Server Execution Env. Range Partitioned Local Domain Indexes ODCIEnv Type Alter Indextype DBMS_ODCI Package Pipelined Parallel Table Functions User-Defined Aggregates Generic SQL Types Multi-Threaded Extproc Array Inserts Alter Operator Debug Info (29891) Trans. Tablespace for Domain Indexes Parallel Index Create/ Index Rebuild Extensible Indexing Extensible Optimizer Parallel Domain Indexe Scan DBMS_STATS integration Oracle8/8i Oracle9i/9i R2

8 Agenda Overview Oracle Database 10g Extensibility Framework Building a data cartridge Deploying a data cartridge Managing data cartridges Demo Summary Q & A

9 Building Data Cartridges Designing the schema (data types, LOBs, etc.) User defined types, operators, indexes, and optimizer Using the right languages (pl/sql, Java, C/C++) Boosting performance Array insert Parallel pipelined table functions Parallel (inter-partition) local domain indexes Create, alter rebuild, query User defined aggregates Debugging trace events

10 Key Steps to Building a Data Cartridge 1. User-Defined Types 2. User-Defined Operators 3. User-Defined Indexes 4. Extensible Optimizer

11 User-Defined Types (Step 1 of 4) Type System Object Types (inheritance, attributes, methods, etc.) Methods (dynamic method dispatch, PL/SQL, Java, or C/C++) Object References (Object ID, dot notation) Collection Types LOBs Physical Model Column objects and row objects Object views and object tables Type evolution, replication, import/export, SQL*Loader, etc. Versatile PL/SQL, Java, C/C++, OLE DB and XML access SQL 2003 standard

12 LOBs Table BFILE BLOB/ CLOB LOBs LOBs can can be be stored stored in in a separate separate tablespace tablespaceor or in in operating operating system system files files LOBs help capture unstructured data Three types of LOBS supported Binary LOB (BLOB) Character LOB (CLOB, NCLOB) Binary File (BFILE) LOBs replace LONG,LONG RAW Multiple LOBs per table or UDT Piece-wise access of LOB data Streaming Interface LONG to LOB API

13 LOB Performance Improvements 5x performance gain for accessing inline (< 4KB) LOBs Temporary LOBs use reference counting to provide orders of magnitude performance gain Reference on Read Copy on Write Temporary LOBs use aggressive garbage collection at row level

14 Native Floating-Point Data Types Two new numeric data types BINARY_FLOAT, BINARY_DOUBLE IEEE 754 Standard for binary floating point arithmetic Part of numerous other standards (e.g, Java, XML Schema) and hardware platforms Prevalent in Business Intelligence, Life Sciences, Engineering/Scientific Computation, etc.

15 User-defined Operators (Step 2 of 4) Domain specific operation Evaluated using functions or indexes CREATE FUNCTION SssMatchBlob (a BLOB, b VARCHAR2) RETURN NUMBER AS / CREATE OR REPLACE OPERATOR sss BINDING (BLOB, VARCHAR2) RETURN NUMBER WITH INDEX CONTEXT, SCAN CONTEXT cdc_ixmdl USING cdcsss.sssmatchblob, (BLOB, CLOB) RETURN NUMBER WITH INDEX CONTEXT, SCAN CONTEXT cdc_ixmdl USING cdcsss.sssmatchblob ;

16 User-Defined Indexes (Step 3 of 4) Registers a new indexing scheme to support a set of operators Implements domain indexing scheme as ODCIIndex interface functions CREATE OR REPLACE TYPE cdc_ixmdl AS OBJECT { STATIC FUNCTION ODCIIndexCreate(..) RETURN NUMBER, STATIC FUNCTION ODCIIndexInsert(..) RETURN NUMBER, STATIC FUNCTION ODCIIndexUpdate(..) RETURN NUMBER, }; CREATE OR REPLACE INDEXTYPE ixmdl FOR sss(blob, VARCHAR2), sss(blob, CLOB) USING cdc_ixmdl; CREATE INDEX acd2d_mol_mdlix ON acd2d_mol(ctab) INDEXTYPE IS c$mdlimol2.ixmdl PARAMETERS ('RCG_DB = ACD2D');

17 Extensible Indexing Architecture User App. CREATE INDEX DMLs DBMS_STATS SELECT ODCIIndexCreate ODCIIndexInsert, etc. ODCIStatsCollect ODCIIndexStart ODCIIndexFetch ODCIIndexClose Userdefined indexes/ stats Logical index Physical index table

18 Simple Query SELECT cdbregno as cdbregno, chime_string(ctab) structure FROM acd2d_mol WHERE sss(ctab, querystring )=1; Operation Options Object SELECT STATEMENT Cost=1229 TABLE ACCESS BY INDEX ROWID ACD2D_MOL DOMAIN INDEX ACD2D_MOL_MDLIX

19 Extensible Optimizer (Step 4 of 4) SELECT /*+ ORDERED */ to_char(acd.cdbregno) as ACD_regno, ch.ch_mdlnum as MDL_number, ch.ch_casrn as CAS_Number, ch.ch_prdcnt as Product_Count, chime_string(ctab) as structure FROM acd991.acd2d_moltableb acd, acd991.acd2d_mol mol, acd991.chemical ch WHERE sss(ctab,<chime String>)=1 AND ch.ch_prdcnt > 1 AND acd.cdbregno = mol.cdbregno AND mol.mdlnumber = ch.ch_mdlnum Aware of user-defined indexes User can specify cost & selectivity User can collect domain specific statistics Simple defaults or comprehensive functions

20 Cost-Based Optimizer Query Hints CBO C(P1) C(P2) C(P3) C(Pn) (min) Execution Plan: join order, Join methods, access methods, Statistics System State

21 Estimating Costs I/O cost: number of data blocks fetched DBMS_STATS package CPU cost: number of machine instructions SQL> DECLARE 2 a INTEGER; 3 BEGIN 4 a := DBMS_ODCI.ESTIMATE_CPU_UNITS(10); 5 DBMS_OUTPUT.PUT_LINE ('CPU units = ' a*1000); 6 END; 7 / CPU units =

22 Complex Query SELECT /*+ ORDERED */ to_char(acd.cdbregno) as ACD_regno, ch.ch_mdlnum as MDL_number, ch.ch_casrn as CAS_Number, ch.ch_prdcnt as Product_Count, chime_string(ctab) as structure FROM acd991.acd2d_moltableb acd, acd991.acd2d_mol mol, acd991.chemical ch WHERE sss(ctab,<chime String>)=1 AND ch.ch_prdcnt > 1 AND acd.cdbregno = mol.cdbregno AND mol.mdlnumber = ch.ch_mdlnum Query Plan SELECT STATEMENT Cost = NESTED LOOPS NESTED LOOPS TABLE ACCESS BY INDEX ROWID ACD2D_MOLTABLEB DOMAIN INDEX ACD2D_MOLTABLEB_MDLIX TABLE ACCESS BY INDEX ROWID ACD2D_MOL INDEX RANGE SCAN IX_ACD2D_MOL TABLE ACCESS BY INDEX ROWID CHEMICAL INDEX UNIQUE SCAN CH_MDLNUM

23 Agenda Overview Oracle Database 10g Extensibility Framework Building a data cartridge Deploying a data cartridge Managing data cartridges Demo Summary Q & A

24 Deploying Cross platform transportable tablespaces Collecting statistics Runtime architecture (J2EE, RAC, Grid, etc.) User privileges (invokers rights) Multi-threaded agents

25 Transportable Tablespaces Copy database subsets (Tablespaces) between databases Operating system file copy for data Managed transfer of metadata (indexes, constraints, stats, etc.) between databases Plug and go Introduced in Oracle8i Enhanced in Oracle9i to support different block sizes Result: extremely fast bulk data transport between databases

26 Oracle10g Cross Platform Transportable Tablespace Little Endian Platforms Linux (Intel IA32/64) Windows (Intel IA32/64) Open VMS (Intel IA64) Tru64 UNIX (Alpha) Simple File Copy Within These RMAN High Speed File Convert Big Endian Platforms Solaris (SPARC) HP-UX (Intel IA64) HP-UX (PA-RISC) AIX (PowerPC) Simple File Copy Within These

27 Automatic Statistics Gathering DBMS_STATS.GATHER_SCHEMA_STATS( SCOTT, options => GATHER AUTO ) This simple statement gathers statistics for the entire schema, and implicitly determines: what tables need to be sampled how much sampling is needed which columns need histograms what degree of parallelism to use when sampling

28 Dynamic and Scalable Parallel Execution SQL query server query server query coordinator query server query server query coordinator query server query server results Query coordinator allocates query servers, and divides query into separate subtask Each set of query servers perform different task (e.g. scanning, sorting, joining) Results are pipelined from one set of query servers to the next

29 RAC Configuration Order Entry High Speed Inter-connects Supply Chain High Speed Switches

30 Agenda Overview Oracle Database 10g Extensibility Framework Building a data cartridge Deploying a data cartridge Managing data cartridges Demo Summary Q & A

31 Managing Data Cartridges Data cartridge upgrades Alter type/operator/indextype/index Performance Tuning Current statistics, Explain plan, Trace event Inlined LOBs, Persistent Data interface, chunk sizes, loading, compression Data Management Partitioning, Import/export, SQL*Loader, data pump ALL_TYPES, ALL_TYPE_ATTRS, ALL_TYPE_METHODS ALL_OPERATORS, ALL_INDEXTYPES, ALL_INDEXTYPE_OPERATORS, ALL_INDEXES views ALL_IND_STATISTICS, ALL_TAB_STATISTICS, ALL_TAB_COL_STATISTICS Space Mgmt ASSM/Locally Managed Tablespaces Inlined LOBs storage

32 Type Evolution Non-structural changes adding, dropping or modifying methods Structural changes adding, dropping or modifying attributes ALTER TYPE person_t ADD/MODIFY/DROP attribute (ssn VARCHAR2(11)) CASCADE;

33 Alter Operator Add bindings to or drop bindings from an existing operator ALTER OPERATOR Ordsys.Contains ADD BINDING (music.artist, music.artist) RETURN NUMBER USING music.contains_func;

34 Alter Indextype Associate the added operator binding to the indextype ALTER INDEXTYPE TextIndexType ADD lob_contains(clob, CLOB);

35 Alter Index ODCIIndexAlter( ia ODCIIndexInfo, parms IN OUT VARCHAR2, alter_option NUMBER, env ODCIEnv) RETURN NUMBER alter_option is one of the following options: AlterIndexNone: ALTER INDEX [PARTITION] PARAMETERS AlterIndexRename: ALTER INDEX RENAME [PARTITION] AlterIndexRebuild: ALTER INDEX REBUILD [PARTITION] [PARALLEL (DEGREE deg)] [PARAMETERS] AlterIndexUpdBlockRefs: ALTER INDEX [schema.]index UPDATE BLOCK REFERENCES

36 Agenda Overview Oracle Database 10g Extensibility Framework Demo Summary Q & A

37 D E M O N S T R A T I O N A Data Cartridge in Action

38 Agenda Overview Oracle Database 10g Extensibility Framework Demo Summary Q & A

39 Summary Manage All of Your Data with Oracle10g Database Consolidate all your data Simplify application development, deployment, and management Explore versatile capabilities Ensure RAS, manageability, and security Managing All of Your Data with Oracle10g Database!

40 Q U E S T I O N S A N S W E R S

41