Oracle on HP Storage

Size: px

Start display at page:

Download "Oracle on HP Storage"

Dora Miles
5 years ago
Views:

1 Oracle on HP Storage Jaime Blasco EMEA HP/Oracle CTC Cooperative Technology Center Boeblingen - November Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice Agenda Oracle recommendations on HP Storage 0g Automatic Storage Management tests 3TB/hour Online Backup page 2

2 EMEA HP/Oracle Competence Center Located at HP in Boeblingen/Germany, Sophia Antipolis/France and Oracle Reading/UK Oracle and HP employees in one team Founded in spring 994 Technology consulting for partners / customers Evaluation and tests of new products/features Technology transfer to and from US labs Technical pre-sales assistance Customer specific Oracle Database, Oracle AS and Oracle E-Business Suite benchmarks page 3 Agenda Oracle recommendations on HP Storage 0g ASM tests 3TB/hour Online Backup page 4

3 the usual database layout sort database objects by size and expected I/O volume implement high-volume objects in striped volumes with various stripe-widths squeeze other objects in where they fit monitor performance of individual disks and objects play chess with file placement to find best performance page 5 SAME technique (proposed by Oracle s Juan Loaiza) Stripe And Mirror Everything stripe all files across maximum # disks use MB stripe size use mirroring for high availability place hot files on outer edge of disks keep it simple page 6

SAME advantages large I/Os minimize impact of disk head movement outer edges of disk have larger capacity and greater transfer speed very wide stripe-set allows full

4 SAME advantages large I/Os minimize impact of disk head movement outer edges of disk have larger capacity and greater transfer speed very wide stripe-set allows full I/O throughput capacity to help all transactions no need to consider characteristics of individual files/tables/transactions page 7 SAME Methodology on HP Storage page 8

5 SAME Methodology on HP Storage page 9 PV mapping to the hp xp array tablespace tablespace tablespace oracle LV LV LV PV PV PV PV PV LVM vol. group... xp array L U N L U N L U N Array (RAID) group L L L U U U N N N Array (RAID) group L L L U U U N N N Array (RAID) group page 0

6 implementing SAME decide stripeset width (# s) four to eight AGs recommended span ACP pairs same disksize/raid create one volume group per stripeset use LUSE to coalesce LUNs (reduce # devs) create multiple logical volumes per VG (create filesystems on top of LVs) allocate Oracle objects among stripesets map Oracle files to LVs or to filesystem files page RAW or File System raw performance double-buffering asynchronous writes (one DBWR process) no unnecessary file overhead no unnecessary file locking - File System: - hp-ux buffer cache size - Modified VXFS mount options: + Undo segments and Temp datafiles: mincache=dsync, convosync=dsync + Datafiles and Redo log files: mincache=direct, convosync=direct page 2

7 Performance considerations Configure enough LUNs to allow I/O load balancing Redo logs: written sequentially and in a flush should be written to the array cache Use a dedicated LUN (different VG) for the redo logs to avoid the log writer waiting on the same SCSIqueue with other I/O requests Archive logs physically separated from dbf and redo logs (security and performance) Oracle colleagues approach: 0 spindles/cpu Stripe size: 64k, 256k, M, 4M,6M,32M (?) page 3 SAME: Benchmarks best practices 5 steps approach. VG over at least 4 I/O channels 2. VG2 for redo logs (queue depth) over same channels 3. VG2 for redo logs over new I/O channels 4. Put the redo logs on a JBOD (5 disks) 5. Use Cache LUN for redo logs page 4

8 EVA layout: one-disk-group configuration bay All database components are in the same disk group Multiple Oracle installations are in the same disk group HSV A 4shelf HSV B 3 2 page 5 EVA layout: two-disk-group configuration, parallel installations bay Oracle shelf 4 3 HSV A HSV B Archived redo Mirrored online redo Executables Database data Original online redo 2 page 6

Agenda Oracle recommendations on HP Storage 0g ASM tests 3TB/hour Online Backup page 7 Oracle 0g ASM Automatic Storage Management Dynamically Provision and Tune Storage Portable,

9 Agenda Oracle recommendations on HP Storage 0g ASM tests 3TB/hour Online Backup page 7 Oracle 0g ASM Automatic Storage Management Dynamically Provision and Tune Storage Portable, high performance storage system Eliminates need for conventional file system and volume manager Automatic mirroring Automatic I/O tuning Stripes data across disks to balance load page 8

10 The Operational Stack TODAY ASM Tables Tables Tablespace Files File System Logical Vol Tablespace Files File System Logical Vol Oracle ASM Disks Disk Group The best way to lower mgmt costs is to remove complexity page 9 Traditional vs ASM - Setup. Determine required storage capacity 2. Install Volume Manager, File System 3. Architect data layout to avoid hot spot 4. Create logical volumes 5. Create file systems 6. Install database 7. Create database. Determine required storage capacity 2. Install ASM 3. Create Disk Groups 4. Install database 5. Create database page 20

11 Oracle 0g ASM test 3 different environments for the tests. Database using ASM 2. Database using RAW devices and HP s Logical Volume Manager (LVM) 3. Database using File System with default and modified mount options Hardware Setup: HP server rp740 8 CPU s (750Mhz), 8GB RAM HP Storage Virtual Array VA740 with 90x73GB disks Software Setup: Oracle Database 0g beta 2 ( ) on HP-UX i page 2 Oracle 0g ASM test: setup For each database we used 4 LUN s with 50GB LVM stripe size = ASM default template: MB for datafiles / temp files / undo 28KB for control files and redo logs loaded data was TPC-C like in a 300 GB tablespace Goal: 3 environments as comparable as possible, not necessarily tuned Test queries: select count (*) with parallel degree 8 and full scan hint create table with parallel degree 8 as select degree 8 page 22

12 Oracle 0g ASM test: results RAW Peak MB/s Time Min:sec Select full scan Create table full scan :37 0:5 page 23 Oracle 0g ASM test: results RAW FILE default Peak MB/s Time Min:sec Peak MB/s Time Min:sec Select full scan 80 4:37 0 5:40 Create table full scan 200 0:5 45 :02 page 24

13 Oracle 0g ASM test: results RAW FILE default FILE opt Peak MB/s Time Min:sec Peak MB/s Time Min:sec Peak MB/s Time Min:sec Select full scan 80 4:37 0 5: :52 Create table full scan 200 0:5 45 :02 40 :2 page 25 Oracle 0g ASM test: results RAW FILE default FILE opt ASM Peak MB/s Time Min:sec Peak MB/s Time Min:sec Peak MB/s Time Min:sec Peak MB/s Time Min:sec Select full scan 80 4:37 0 5: : :34 Create table full scan 200 0:5 45 :02 40 : :50 page 26

14 Oracle 0g ASM test: summary Impressive performance & managebility ASM Performance equal to RAW ASM much better than file system (optimized) Additional unique ASM features page 27 Agenda Oracle recommendations on HP Storage 0g ASM tests 3TB/hour Online Backup page 28

HP achieves world record backup and restore 3.

achieved with the following equipment: Superdome with

15 HP achieves world record backup and restore 3.6TB/hr Oracle online backup.23tb/hr Oracle restore New! achieved with the following equipment: Superdome with 32CPU's XP024 HP Tape library (ESL9595) with 6 Ultrium2 drives Oracle 9iR2 ( ) for recovery catalog and target database OpenView Data Protector TB database on file system using the TPC-H standard setup: 8 tables partitioned over ~50 Tablespaces and ~ 260 datafiles. page 29 The team page 30

Test lab - system diagram HP Superdome server 32 x 2Gbps fibre channels XP024 disk array HP fc6b Fibre Switch X 2 storage routers HP m2402 storage router SCSI FC SCSI SCSI FC SCSI m2402

LDEV2 LDEV3 LDEV4 LDEV5 LDEV6 LDEV7 physical volumes LDEVn HP-UX device files device files device files device files device files device files LVM lvol n lvol n lvol n lvol n lvol n lvol n

16 Test lab - system diagram HP Superdome server 32 x 2Gbps fibre channels XP024 disk array HP fc6b Fibre Switch X 2 storage routers HP m2402 storage router SCSI FC SCSI SCSI FC SCSI m2402 m2402 m2402 m2402 m2402 m2402 Ultra-3 SCSI Ultra-3 SCSI m tape drive tape drive tape drive robotics 6 Ultrium 460 tape drives in ESL9595 library page 3 HP-UX/Oracle9i XP024 LDEV LDEV2 LDEV3 LDEV4 LDEV5 LDEV6 LDEV7 physical volumes LDEVn HP-UX device files device files device files device files device files device files LVM lvol n lvol n lvol n lvol n lvol n lvol n volume group volume group vxfs filesystem(s) vxfs filesystem(s) vxfs filesystem(s) data files data files data files archived redo logfiles control file Oracle database files redo logfiles page 32

17 Generating the test database Test databases are available from the Transaction Processing Council TPC Used TPC-H dbgen tool Limit of 535GB per mount point as a result of the 8MB extent size. This is a limitation set by the HP-UX logical volume manager. page 33 Test database structure 9TB of space allocated with 4.3TB of real data loaded 8 tables within the TPC schema Line item 5billion records partitioned over 45 tablespaces 3 datafiles per tablespace Order : 4.5 billion records over 5 tablespaces Partsupp : 2.4 billion records over 20 tablespaces Part: 600 million records 0ver 20 tablespaces Customer : 400 million records over 20 tablespaces Supplier: 30 million records over 20 tablespaces Nation: 25 records in a single tablespace Region: 5 records in single tablespace Total: 259 datafiles and 46 tablespaces page 34

Test Database structure partitioned data line item table orderkey partkey supp_key linenumber shipdate comment July orders Aug orders partition partition 4 tablespaces: datafiles L L2 L3 L43 L44 L45

18 Test Database structure partitioned data line item table orderkey partkey supp_key linenumber shipdate comment July orders Aug orders partition partition 4 tablespaces: datafiles L L2 L3 L43 L44 L45 line.dbf line2.dbf line3.dbf line33.dbf line34.dbf line35.dbf mount points /u05 /u06 /u07 /u6 /u7 /u35 /u8 page 35 RMAN Data Protector integration Edit window Include control file Full backup filesperset = Channel allocation page 36

19 RMAN & Data Protector tuning concurrency filesperset blocksize maxopenfiles tape I/O slaves disk I/O slaves contolfile autobackup page 37 Results: 3TB/hr backup # tape drives # processors CPU idle Data protector parameters RMAN settings backup performance 6 in HP ESL9595 tape library 32 34% concurrency= ( RMAN channel/tape drive) disk I/O slaves=6 filesperset=no set maxopenfiles= tapeblocksize=256k backup_tape_io_ slaves=disabled controlfile autobackup=on 3.6 TB/hr (including tape load/unload time) 3.62 TB/hr (excluding tape load/unload time) page 38

20 Results: backup (6 way CPU) # tape drives # processors CPU idle Data protector parameters RMAN settings backup performance 6 in HP ESL9595 tape library 6 5% concurrency= ( RMAN channel/tape drive) disk I/O slaves=6 filesperset=no set maxopenfiles= tapeblocksize=256kb backup_tape_io_ slaves=disabled controlfile autobackup=on 2.87 TB/hr (excludes tape load/unload time) page 39 Results: restore # tape drives # processors CPU idle Data protector parameters RMAN settings backup performance 6 in HP ESL9595 tape library 32 45% concurrency= ( RMAN channel/tape drive) disk I/O slaves=6 filesperset=no set maxopenfiles= tapeblocksize=mb backup_tape_io_ slaves=enabled controlfile autobackup=on.23 TB/hr (including tape load/unload time).29 TB/hr (excluding tape load/unload time) Restore times subsequently improved to 2.46TB/hr! page 40

21 Lessons Learnt Backup performance determined in the main by disk subsystem performance. Use SAME for best Oracle file system performance Storage Router configuration could be simplified. Memory usage was low 9GB Max (of 256GB) Could have been achieved with 6 Processors (5% idle) page 4 Lessons Learnt In a high performance environments set RMAN channel per tape drive, don t use external multiplexing Restore performance subsequently increased to 2.4 TB/HR restore 40-60% of backup rate. Data Protector 5. integration with Oracle is now even better restore GUI. Only Ultrium technology has adaptive tape speed to optimize performance and increase media life. page 42

How HP delivered a 3TB/hour Oracle TM backup & 1TB/hour restore. Andy Buckley Technical Advocate HP Network Storage Solutions

How HP delivered a 3TB/hour Oracle TM backup & 1TB/hour restore Andy Buckley Technical Advocate HP Network Storage Solutions Why Oracle? RDBMS Market share 2002 (%) NCR 3.5 Microsoft 22.8 Others 7.1 Oracle