Oracle Database In-Memory By Example

Transcription

1

2 Oracle Database In-Memory By Example Andy Rivenes Senior Principal Product Manager DOAG 2015 November 18, 2015

3 Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle.

4 Program Agenda Getting Started Configuring & Populating In-Memory Column Store Querying the In-Memory Column Store Joins & Aggregation with In-Memory Column Store

5 In-Memory By Example - Schema Overview Star schema based on TPCH benchmark Lineitem& Orders tables combined to create fact table Schema built with a 3GB scale factor

6 In-Memory By Example Demonstration Details Virtual Box on my laptop Set of SQL scripts to show how Database In-Memory works SGA size = 4G IM column store size = 1504M _small_table_threshold = Tables have "cache" attribute set Each script connects as a new session

7 Program Agenda Getting Started Configuring & Populating In-Memory Column Store Querying the In-Memory Column Store Joins & Aggregation with In-Memory Column Store

8 Row Format Databases vs. Column Format Databases Rows Stored Contiguou sly SALES Transactions run faster on row format Example: Query or Insert a sales order Fast processing few rows, many columns Columns Stored Contiguou sly SALES Analytics run faster on column format Example : Report on sales totals by region Fast accessing few columns, many rows Until Now Must Choose One Format and Suffer Tradeoffs

9 Breakthrough: Dual Format Database Normal Buffer Cache SALES Row Format SALES New In-Memory Format SALES Column Format BOTH row and column formats for same table Simultaneously active and transactionally consistent Analytics & reporting use new in-memory Column format OLTP uses proven row format

10 Configuring : In-Memory Column Store System Global Area (SGA) Buffer Cache Large Pool Shared Pool Other Log Buffer In-Memory Area New Component of SGA Static Area (Not a cache!) Controlled by INMEMORY_SIZE parameter - Minimum size of 100MB SGA_TARGET must be large enough to accommodate

11 Populating : In-Memory Column Store ALTER TABLE sales INMEMORY; ALTER TABLE sales NO INMEMORY; CREATE TABLE customers PARTITION BY LIST (PARTITION p1 INMEMORY, (PARTITION p2 NO INMEMORY); ALTER TABLE sales INMEMORY NO INMEMORY (PROD_ID); New INMEMORY ATTRIBUTE Segment types eligible - Tables - Partitions - Subpartitions - Materialized views Possible to populate only certain columns from a table or partition

12 Populating : In-Memory Column Store Population is completed by a new set of background processes ora_w001_orcl Number of processes controlled by parameter INMEMORY_MAX_POPULATE_SERVERS

13 Configuring & Populating In-Memory Column Store Examples

14 Configuring & Populating In-Memory Column Store Summary Size of the column store is controlled by INMEMORY_SIZE What size is the column store on your systems? To add a table to the column store set IN MEMORY attribute alter table lineorder inmemory; Monitor the column store using v$im_segments v$im_segments also shows you the MEMCOMPRESS ratios What was the largest compression ratio you saw?

15 Program Agenda Getting Started Configuring & Populating In-Memory Column Store Querying the In-Memory Column Store Joins & Aggregation with In-Memory Column Store

16 Querying: Why is an In-Memory scan faster than the buffer cache? Buffer Cache SELECT COL4 FROM MYTABLE; X X X X X RESULT Row Format

17 Querying: Why is an In-Memory scan faster than the buffer cache? IM Column Store SELECT COL4 FROM MYTABLE; RESULT Column Format X X X X X 17

18 Querying : Oracle In-Memory Column Store Storage Index Example: Find all sales from stores with a store_id of 8 Memory Min 1 Max 3 Min 4 Max 7 Each column is the made up of multiple column units Min / max value is recorded for each column unit in a storage index SALES Column Format Min 8 Max 12 Min 7 Max 15? Storage index provides partition pruning like performance for ALL queries

19 Querying : Orders of Magnitude Faster Analytic Data Scans Memory CPU Load multiple region values STATE Vector Register CA CA CA CA Example: Find all sales in state of CAlifornia Vector Compare all values an 1 cycle > 100x Faster Each CPU core scans local in-memory columns SIMD vector instructions used to process multiple values in each instruction - Originally designed for graphics & science Billions of rows/sec scan rate per CPU core - Row format is millions/sec

20 Querying : Determining If In-Memory Is Used By A Query Examine the execution plan

21 Querying : Determining If In-Memory Is Used Session level statistics are the best way to determine if In- Memory was used Also indicate what the benefits were The demo will show the IM session level statistics

22 Querying: In-Memory Column Store Examples

23 Querying: In-Memory Column Store Summary Control the use of the column store using INMEMORY_QUERY alter session set INMEMORY_QUERY=disable; INMEMORY & NO_INMEMORY hints Simple query shows column store is 10X faster than buffer cache Use session statistics to check how little of the column store is accessed Column store is 40X faster than the buffer cache for a single row access with no index Index is just as fast as the column store, but you have to maintain it Storage index MIN/MAX pruning reduces data to be scanned significantly Use session statistics to check how much

24 Program Agenda Getting Started Configuring & Populating In-Memory Column Store Querying the In-Memory Column Store Joins & Aggregation with In-Memory Column Store

25 Joining : Combining Data Also Dramatically Faster Example: Find total sales in outlet stores Stores Type Store ID Bloom Filter StoreID in 15, 38, 64 Sales Converts joins of data in multiple tables into fast column scans Type= Outlet Store ID Amount Bloom filter pushdown - Filtering pushed down into scan Joins tables 10x faster Sum

26 Joining : Bloom Filter How to identify a bloom filter in the execution plan Why is this significant? Join converted to additional filter applied to the fact table Columnar format very efficient at processing filters

27 Joining : Multiple Bloom Filters Select d.d_year, s.s_nation, sum(lo_revenue - lo_supplycost) profit From LINEORDER l, DATE_DIM d, PART p, SUPPLIER s Where l.lo_orderdate = d.d_datekey And l.lo_partkey = p.p_partkey And l.lo_suppkey = s.s_suppkey And (p.p_mfgr ='MFGR#12' or p_mfgr='mfgr#2') And s.s_region = 'AMERICA Group by d.d_year, s.s_nation Order by d.d_year, s.s_nation; Analytics is about finding patterns & trends by aggregating data Possible to apply multiple bloom filters to the fact table due to sophisticated Optimizer Simultaneously joining to all of the dimension tables while scanning the fact table

28 Joining : Multiple Bloom Filter Execution Plan

29 Aggregation : In-Memory Aggregation Select p.product_name, st.store_name, sum(amount_sold) profit From SALES S, STORES st, PRODUCTS p, Where s.prod_id = p.prod_id And s.store_id = st.store_id And s.type = OUTLETS And p.type = FOOTWARE Group by p.product_name, st.store_name Order by p.product_name, st.store_name; New Vector Group-By enables extremely efficient in-memory array based aggregation Simultaneously accumulates aggregate values into inmemory arrays during fact table scan

30 Aggregation : In-Memory Aggregation Example: Report sales of footwear in outlet stores Products Footwear Stores Outlets Outlets In-Memory Report Outline Footwear $ $$ $$$ $ Sales Sales Dynamically creates in-memory report outline Then report outline filled-in during fast fact scan Reports run much faster - Without predefined cubes Also offloads report filtering to Exadata Storage servers

31 Aggregation : Vector Group by Step Id Operation Name SELECT STATEMENT 1 TEMP TABLE TRANSFORMATION 2 LOAD AS SELECT 3 PX COORDINATOR 4 PX SEND QC (RANDOM) :TQ BUFFER SORT 6 VECTOR GROUP BY 7 KEY VECTOR CREATE BUFFERED :KV PX RECEIVE 9 PX SEND HASH :TQ PX BLOCK ITERATOR * 11 TABLE ACCESS INMEMORY FULL PRODUCTS 12 LOAD AS SELECT 13 PX COORDINATOR 14 PX SEND QC (RANDOM) :TQ HASH GROUP BY 16 PX RECEIVE 17 PX SEND HASH :TQ VECTOR GROUP BY 19 HASH GROUP BY 20 KEY VECTOR CREATE BUFFERED :KV PX BLOCK ITERATOR * 22 TABLE ACCESS INMEMORY FULL STORES Scan each of the dimension tables and create a Key vector Key vector is used to complete the join and enable the vector group by

32 Aggregation : Vector Group by Step Id Operation Name SELECT STATEMENT 1 TEMP TABLE TRANSFORMATION 2 LOAD AS SELECT 3 PX COORDINATOR 4 PX SEND QC (RANDOM) :TQ BUFFER SORT 6 VECTOR GROUP BY 7 KEY VECTOR CREATE BUFFERED :KV PX RECEIVE 9 PX SEND HASH :TQ PX BLOCK ITERATOR * 11 TABLE ACCESS INMEMORY FULL PRODUCTS 12 LOAD AS SELECT 13 PX COORDINATOR 14 PX SEND QC (RANDOM) :TQ HASH GROUP BY 16 PX RECEIVE 17 PX SEND HASH :TQ VECTOR GROUP BY 19 HASH GROUP BY 20 KEY VECTOR CREATE BUFFERED :KV PX BLOCK ITERATOR * 22 TABLE ACCESS INMEMORY FULL STORES Create a temporary table for the payload columns from the dimension tables This will allow a more efficient join back later

33 Aggregation : Vector Group by Step 3 * 23 HASH JOIN * 24 HASH JOIN 25 TABLE ACCESS FULL SYS_TEMP_0FD9D662E_2D VIEW VW_VT_80F HASH GROUP BY 28 PX RECEIVE 29 PX SEND HASH :TQ VECTOR GROUP BY 31 HASH GROUP BY 32 KEY VECTOR USE :KV KEY VECTOR USE :KV PX BLOCK ITERATOR * 35 TABLE ACCESS INMEMORY FULL SALES 36 TABLE ACCESS FULL SYS_TEMP_0FD9D662F_2D TABLE ACCESS FULL SYS_TEMP_0FD9D662D_2D Scan the SALES table with the help of the Key Vectors Join the results back to the temporary tables created with the payload columns

34 Joins & Aggregation with In-Memory Column Store Examples

35 Joins & Aggregation with In-Memory Column Store Summary Bloom filters allow joins to be converted to filters on fact table Bloom filter on the column store is 10x faster than on buffer cache Vector group by used when number of rows processed is large Vector group by allows aggregation to occur during scan of fact table Vector group by is 3x faster than In-Memory column store alone

36 In-Memory Recap SUMMARY

37 Full table Scan Performance In-Memory Buffer Cache 700 X FASTER Elapse Time (sec)

38 Single Record Lookup In-Memory Buffer Cache Index Access 700 X FASTER Elapse Time (sec)

39 Multiple Table Joins X FASTER Elapse Time (sec) In-Memory Buffer Cache

40 Multiple Column Aggregation X FASTER 15 Elapse Times (sec) In-Memory No VG Vector Group

41 Summary Size of the column store is controlled by INMEMORY_SIZE Compression Advisor can tell you how much space you need To add a table to the column store set INMEMORY attribute alter table lineorder inmemory; Control the population of the column store via the PRIORITY attribute Resource manager can also be used to control the work processes To disable the use of the column store use INMEMORY_QUERY or hint Monitor the column store using v$im_segment & v$mystat All additional Information you need is on the In-Memory blog

42 Additional Resources Related White Papers Oracle Database In-Memory White Paper Oracle Database In-Memory Best Practices Oracle Database In-Memory Aggregation Paper When to use Oracle Database In-Memory Oracle Database In-Memory Advisor Join the Conversation Related Videos In-Memory YouTube Channel Managing Oracle Database In-Memory Database In-Memory and Oracle Multitenant Industry Experts Discuss Oracle Database In-Memory Software on Silicon Any Additional Questions Oracle Database In-Memory Blog My

43

44