Exadata with In-Memory Option the best of all?!? Konrad HÄFELI Senior Solution Manager Infrastructure Engineering BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA 1
Unser Unternehmen Trivadis ist führend bei der IT-Beratung, der Systemintegration, dem Solution-Engineering und der Erbringung von IT-Services mit Fokussierung auf und Technologien im D-A-CH-Raum. Unsere Leistungen erbringen wir aus den strategischen Geschäftsfeldern: B E T R I E B Trivadis Services übernimmt den korrespondierenden Betrieb Ihrer IT Systeme. 2
Mit über 600 IT- und Fachexperten bei Ihnen vor Ort Hamburg 12 Trivadis Niederlassungen mit über 600 Mitarbeitenden 200 Service Level Agreements Düsseldorf Frankfurt Mehr als 4'000 Trainingsteilnehmer Forschungs- und Entwicklungsbudget: CHF 5.0 Mio. / EUR 4.0 Mio. Stuttgart Finanziell unabhängig und nachhaltig profitabel Freiburg Basel Brugg Bern Zürich Lausanne München Wien Erfahrung aus mehr als 1'900 Projekten pro Jahr bei über 800 Kunden 3 Stand 12/2013 3 3
Trivadis an der DOAG Ebene 3 - gleich neben der Rolltreppe Wir freuen uns auf Ihren Besuch. Denn mit Trivadis gewinnen Sie immer. 4
A year ago DOAG 2013 5
Chose another topic Flash Cache 6
finaly in July 2014 Previews Betas Versions Announcments: 10th of June Releasing 22nd of July 12.1.0.2 full release 7
It s an option, is it worth it? On a normal Database Server? Hopefully, if requirements accordingly But what about Exadata Database Machine? Are the «expensive» Storage Server Feature Smart Scan Flash Cache a benefit for IMCS (In Memory Column Store)? Resp. vice versa? 8
AGENDA 1. Exadata Feature 2. In-Memory Overview 3. In-Memory Configuration 4. In-Memory Management 5. In-Memory with Exadata Tests 6. Conclusion 9
Exadata Feature 10
Architecture Features Overview Smart Scan Capabilities Predicate Filtering Column Filtering Join Processing Scans on encrypted Data Scans on compressed Data Create Tablespace Storage Server formats blocks Source: Oracle Corporation Backup and Recovery I/O for incremental Backup is much more efficient, only changed Blocks are returned to Server 11
Exadata Hybrid Columnar Compression (HCC) A compression unit is a logical structure spanning multiple DB blocks Each row is self-contained within a compression unit Data is organized by column during data load Each column is compressed separately Smart Scan is supported 12
In-Memory Overview 13
Columnar Storage: Example for Columnorientation 14
In-Memory Option What is it? Oracle Databases traditionally stores Data in Row Format Oracle Database In-Memory Option uses an In-Memory Column Store A new memory component in the SGA called In-Memory Area Data are stored in column format in the In-Memory Column Store (IMCS) Both Memory Structure (Buffer Cache and IMCS) will be populated Dual Format Architecture does not double the Memory requirement The Optimizer is fully aware about the In-Memory Column Store Source: Oracle 15
In-Memory Option Database Tiering With the In-Memory Area not all data must fit in Memory Less performance sensitive can reside on lower cost Disk IM Option acts transparently against the applications On Exadata acts on the top of all existing Storage tiers It is also fully compatible with the new Multitenant architecture Source: Oracle 16
In-Memory Option SIMD In-Memory Column Store are scanned with SIMD Vector Processing SIMD = Single Instruction Multiple Data SIMD Processing are evaluated in a single CPU Instruction IM Column Store are specifically designed to maximize the number of column entries that can be loaded into Vector Registers Theoretically Scan of billions of rows per second are possible 17
Summary: Oracle Database In-Memory Powering the Real-Time Enterprise Extreme Performance: Analytics & OLTP Extreme Scale-Out & Scale-Up Extreme Availability Extreme Simplicity All In-Memory Benefits With No Application Changes 18
In-Memory Configuration 19
In-Memory Option The way to use it On Exadata there are some prerequisites Storage Server Min. 12.1.1.1.0* Database Server Min. 12.1.1.1.0 Grid Infrastructure 12.1.0.2 * Full Offload Support for 12.1.x RDBMS RDBMS 12.1.0.2 20
In-Memory Option Configuration Configuration is very simple The Initializations Parameters INMEMORY_SIZE controls the In- Memory Area SQL> alter system set inmemory_size=16g scope=spfile sid= * ; SQL> select * from v$sga; NAME VALUE CON_ID -------------------- --------------- ---------- Fixed Size 5304248 0 Variable Size 11274289224 0 Database Buffers 108716359680 0 Redo Buffers 263131136 0 In-Memory Area 17179869184 0 21
In-Memory Option Configuration Initialization Parameter inmemory_clause_default inmemory_force inmemory_max_populate_servers inmemory_query inmemory_size inmemory_trickle_repopulate_ser vers_percent optimizer_inmemory_aware Comment enables you to specify a default In-Memory Column Store (IM column store) clause for new tables and materialized views allows you to specify whether tables and materialized view that are specified as INMEMORY are populated into the In-Memory Column Store or not specifies the maximum number of background populaserverste INMEMORY_QUERY is used to enable or disable in-memory queries for the entire database at the session or system level INMEMORY_SIZE sets the size of the In- Memory Column Store (IM column store) on a database instance limits the maximum number of background populate servers used for In-Memory Column Store (IM column store) repopulation enables or disables all of the optimizer cost model enhancements for in-memory 22
In-Memory Option Configuration Tables or Tablespace must be enabled to use IM Columnn Store Depending on configuration the IMCS will be populated Activation at Tablespace and Table Level At Table Level configuration for compression and priority are possible 23
In-Memory Option Configuration Compression Types Compression Type No Compress Memcompress for DML Memcompress for Query Low Memcompress for Query High Memcompress for Capacity Low Memcompress for Capacity High Description No Compression Compression optimized for DML Compression optimized for Query (default) Compression optimized for Query and Space saving Compression optimized with orientation towards Space saving Compression optimized for Space saving Priority Levels Priority Level Critical High Medium Low None Description Objects are populated immediately Objects are populated after all critical objects are populated, if remains space Objects are populated after all critical and high objects are populated, if remains space Objects are populated after all critical, high and medium objects are populated, if remains space Objects are populated only after they are scanned 24
In-Memory Option Configuration Activation at Tablespace Level SQL> alter tablespace SOE inmemory; At Table Level configuration for compression and priority are possible memcompress for query low and priority none are the default settings SQL> alter table orders inmemory; Table activation with compression and priority settings SQL> alter table orders inmemory memcompress for query low priority critical; 25
In-Memory Management 26
In-Memory Option Licensing License is need for using the In-Memory Option ~ 18000 / per Processor The Option is installed with the RDBMS Installation, however not activated Bug 19308780 Wrong Feature Tracking, fixed in PSU October 2014 Very lovely discussion between Maria Colgan (Oracle IM Option Product Manager) and Kevin Closson ( independent Oracle/Exadata specialist) 27
In-Memory Option Management Few v$ Views are in place for In-Memory Column Store management Not all v$ Views are documented (only red ones) SQL> select table_name from dict where table_name like 'V$IM_%'; TABLE_NAME ------------------------------ V$IM_COLUMN_LEVEL V$IM_COL_CU V$IM_HEADER V$IM_SEGMENTS V$IM_SEGMENTS_DETAIL V$IM_SEG_EXT_MAP V$IM_SMU_CHUNK V$IM_SMU_HEAD V$IM_TBS_EXT_MAP V$IM_USER_SEGMENTS Only V$ Views in red are documented New IM Memory Option PL/SQL Package SQL> desc dbms_inmemory.. PROCEDURE DISABLEFASTSTART PROCEDURE ENABLEFASTSTART PROCEDURE POPULATE PROCEDURE REPOPULATE Only Procedure in red are documented 28
In-Memory Option Management New dba_/all_/user_ views attribute are been implemented SQL> desc user_tables Name Null? Type ----------------------------------------- -------- ---------------------------- TABLE_NAME NOT NULL VARCHAR2(128). INMEMORY VARCHAR2(8) INMEMORY_PRIORITY VARCHAR2(8) INMEMORY_DISTRIBUTE VARCHAR2(15) INMEMORY_COMPRESSION VARCHAR2(17) INMEMORY_DUPLICATE VARCHAR2(13 Monitoring of In-Memory Segment SQL> select owner, segment_name, segment_type, inmemory_size, bytes_not_populated, populate_status from v$im_segments; OWNER SEGMENT_NAME SEGMENT_TYPE INMEMORY_SIZE BYTES_NOT_POPULATED POPULATE_ ---------- -------------- --------------- ------------- ------------------- --------- EXA IM2 TABLE 1179648 0 COMPLETED EXA IM4 TABLE 8519680 0 COMPLETED 29
In-Memory with Exadata Tests 30
Dedicated Query SQL> SELECT SUM(amount_sold) FROM sales; 147 Mio Rows, 10GB Cell offload_processing FALSE TRUE CACHE FALSE TRUE INMEMORY_QUERY DISABLE DISABLE DISABLE ENABLE ENABLE Elapsed time 56.82 13.51 1.17 0.13 013 Factor 1 4 48 437 437 IM is factor 10 faster then jus Buffer cache First population of IM data factor 4 faster with offloading 80% offload efficiency 31
Dedicated Query SQL> SELECT t.time_id, TO_CHAR(SUM(amount_sold), '9,999,999,999') AS sales, TO_CHAR(AVG(SUM(amount_sold)) over(order BY t.time_id range BETWEEN INTERVAL '2' DAY preceding AND INTERVAL '2' DAY following), '9,999,999,999') AS centered_5_day_avg FROM sales s, times t WHERE s.time_id = t.time_id GROUP BY t.time_id ORDER BY t.time_id; Cell offload_processing FALSE TRUE CACHE FALSE TRUE INMEMORY_QUERY DISABLE DISABLE DISABLE ENABLE ENABLE Elapsed time 01:30.14 29.49 05.95 05.49 05.48 Factor 1 3 15 16 16 32
In-Memory Option Exadata Test Enviroment Eighth Rack (2 DB Server 256 Gb RAM, 3 Storage Server) Swingbench (charbench) used as Benchmarking Tool (2 Min) OLTP Profile with 100 concurrent Users - 60/40% Read/Write ratio DSS Profile with 16 concurrent Users - 100/0% Read/Write ratio (Rollup and Cube Function) In-Memory Area with 16G Memory (SGA 128GB) All Columns of involved Tables (full Schema) are stored in the IM Column Store (~ 10GB) and populated Duplicate Feature intended for RAC Environment was not used This Feature should be considered in HA or Scalability Environments Our first steps in this way, was not so encouraging in terms of scalability 33
In-Memory Option Test Cases 1. Smart Scan off, no In-Memory 2. Smart Scan on, no In-Memory 3. Smart Scan off, In-Memory compression for DML 4. Smart Scan on, In-Memory compression for DML 5. Smart Scan on, In-Memory compression for query low 6. Smart Scan on, In-Memory compression for query high 7. Smart Scan on, In-Memory compression for capacity low 8. Smart Scan on, In-Memory compression for capacity high 34
In-Memory Option Test Results OLTP Total completed Transaction 180000 160000 140000 120000 100000 80000 60000 40000 20000 0 Total Completed Transaction 35
In-Memory Option Test Results OLTP Average CPU Utilization 90 CPU Usage 80 70 60 50 40 Avg User CPU Avg Sys CPU 30 20 10 0 36
In-Memory Option Test Results OLTP Total completed Transaction 18000090 16000080 14000070 12000060 10000050 8000040 6000030 4000020 2000010 0 0 37
In-Memory Option Test Results DSS Total completed Transaction (Reports) 120 Total Completed Transactions 100 80 60 40 20 0 38
In-Memory Option Test Results DSS Average CPU Utilization 60 CPU Usage 50 40 30 Avg User CPU Avg Sys CPU 20 10 0 39
In-Memory Option Test Results DSS Total completed Transaction (Reports) 12060 10050 8040 6030 4020 2010 0 0 40
In-Memory Option on Exadata Result discussion OLTP factor 12 :-o No offloading due to no direct path functionality All in cache with high CPU load 40% read profit of IMCS OLTP less CPU with IMCS due to SIMD benefit DSS No Offloading/No IMCS as fast but a lot of CPU Buffer Cache Offloading, as fast but less CPU work done in Storage Server IMCS 25% plus work, less CPU The dataset and ressources are not in a good balance to test Much more data needed 41
Conclusion 42
In-Memory Option on Exadata Conclusion Exadata Database Machine works fine Oracle IMCS Option works fine Testing and interpretation is not trivial The option costs 50% of EE core price A serious evaluation is needed PoC to find the best of all, unless you need a: 43
Questions and answers... Konrad HÄFELI Tel: +41 58 459 59 69 konrad.haefeli@trivadis.com BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA 44