Customer Coffee Corner for SAP IQ Using sp_iqrebuildindex()

Transcription

1 Customer Coffee Corner for SAP IQ Using sp_iqrebuildindex() Customer SAP Product Support February, 2017

2 Agenda Objectives sp_iqrebuildindex() usage FAQs Useful scripts Closing remarks Open discussion 2016 SAP SE or an SAP affiliate company. All rights reserved. Customer 2

3 Objectives Proactive outreach based on feedbacks Target audience IQ DBAs Novice/Beginner Awareness about key infrastructure IQ procedure, which may help in better planning and execution avoiding how to IQ incidents SAP SE or an SAP affiliate company. All rights reserved. Customer 3

4 sp_iqrebuildindex() usage o sp_iqrebuildindex() rebuilds column indexes. o Typical variations - FP call sp_iqrebuildindex ( 'mytable', 'column c1 1024' ) HG call sp_iqrebuildindex ( 'mytable', HGindexname RETIER' ) Other call sp_iqrebuildindex ( 'mytable', otherindexname' ) o Toggle - Nbit to Flat ( IQ UNIQUE(0) or Large number > 2,147,483,647 in index clause ) Flat to nbit ( If number of unique values are < 2,147,483,647 ) HG to THG ( RETIER ) THG to HG ( RETIER, MERGEALL ) o Post upgrade - Upgrade FP indexes from pre IQ versions. o Performance - Implement IQ query optimizer hints in html query plans as data changes over period of time. o Troubleshooting - Sometimes helps working around problems like non-fatal stack traces, inconsistent query results etc. o Miscellaneous - Reducing fragmentation, redistribution across dbfiles o Cannot rebuild if data is inconsistent or has unrecoverable errors. o Cannot rebuild text indexes. Text indexes need to be dropped and recreated SAP SE or an SAP affiliate company. All rights reserved. Customer 4

5 FAQs - Execution How to track sp_iqrebuildindex() progress? There is no specific database option which will print progress messages in IQ diagnostic files. Will sp_iqrebuildindex() cause versioning? No. Following dbisql sequence shows that no version retained because of sp_iqrebuildindex() Session 1 select columns from table Session 2 - select columns from table sp_iqversionuse() will show 2 versions for 2 sessions Session 1 sp_iqrebuildindex() Successful completion Session 2 - will show that session 1 version go away. Can sp_iqrebuildindex() repair FP indexes if underlying data has inconsistencies? No. SAP IQ has no functionality to repair FP indexes. Index rebuild is possible only when there are no issues with underlying data. Can end-user run sp_iqrebuildindex() while other users are active? Yes. Users working on other tables will not be affected SAP SE or an SAP affiliate company. All rights reserved. Customer 5

6 FAQs - Execution What type of lock sp_iqrebuildindex() acquires? Procedure acquires write lock on table. Can I execute sp_iqrebuildindex() on different columns of same table in separate sessions? No. Session 1 acquires write lock. Session 2 trying to acquire write lock on same table will get error similar to this. User 'another user' has the row in table' locked SQLCODE=-210, ODBC 3 State="40001" Can I execute sp_iqrebuildindex() on different tables in separate sessions? Yes. If there are no previous write locks on target tables, session 1 and 2 will be acquire separate write locks on different tables and execute procedure successfully. Can end-user run sp_iqrebuildindex() on any multiplex writer node? Yes. Direct coordinator run may avoid communication overhead between secondary node and coordinator. Which other stored procedures can be used to check sp_iqrebuildindex() execution effects? Pre and post snapshots of sp_iqindexfragmentation(), sp_iqrowdensity(), sp_iqindexsize(), sp_iqindexmetadata() 2016 SAP SE or an SAP affiliate company. All rights reserved. Customer 6

7 FAQs - Execution Are there any space considerations before running sp_iqrebuildindex()? Yes. The procedure is not building index in place. It will need space while it is rebuilding index. If database runs out of space, end-user may see error like this rolling back transaction. Exception Thrown from slib/s_blockmap.cxx:5332 O/S Err#: 0, ErrID: 2096 (s_nodbspaceexception); SQLCode: , SQLState: 'QSB66', Severity: 14 You have run out of space in iq_main DBSpace. Can end-user abort sp_iqrebuildindex()? Will it cause any data corruption? It can be aborted without any corruption. Abort should rollback transaction without any data corruption. I have nbit FP index. Do I need to run sp_iqrebuildindex() to reflect increase/decrease of unique values? No. Rollover within nbit levels and between nbit to Flat is automatic. It is transparent to end-user. Should I be concerned about sp_iqrebuildindex() speed? Speed depends on amount of data, and target index structure complexity. Users can expect to see a temporary performance drop when sp_iqrebuildindex runs on a large HG index SAP SE or an SAP affiliate company. All rights reserved. Customer 7

8 FAQs - Upgrade Do I really need to run sp_iqrebuildindex() on all the indexes to upgrade from pre 16 IQ versions? It is highly recommended to run sp_iqrebuildindex() on all FP indexes due to change in FP index structure. 15.x had 1/2/3 byte optimized FP whereas 16.x has nbit dictionary. If there are huge number of columns and hence FP indexes, end-user should do it in maintenance window before releasing IQ to users after upgrade. IQ can do the same when column is opened for modification for the first time but it may impact performance. Do I really need to run sp_iqrebuildindex() on all the indexes to upgrade between IQ 16 service packs? No. sp_iqrebuildindex() run is generally not required to upgrade from IQ 16 lower service pack to IQ 16 higher service pack. Check If covering letter accompanying service pack suggests to run this procedure for any known issue. If lower service pack is SP 01 or earlier, sp_iqrebuildindex() run is required. Why sp_iqindexrebuildwidedata() does not rebuild index? sp_iqindexrebuildwidedata() does not really rebuild index. It merely identifies wide columns which must be rebuilt after IQ 16 upgrade SAP SE or an SAP affiliate company. All rights reserved. Customer 8

9 FAQs - Repair I have table abc with columns ( a, b, c ). I cannot rebuild index on column b and sp_iqcheckdb on column b returns error. Can support salvage column b data directly from disk and repair index? No. IQ is columnar database. Individual column has complex index structure. It gets compressed further when written to disk. It is impossible to salvage raw data from disk. But variation of following strategy may help salvage some data from indexes if there are absolutely no alternate resources to retrieve data from. 1. Identify good columns. Good columns are those on which sp_iqcheckdb does not return any problems. 2. Identify unique key if any on problem table. If there is none or problem with key column itself, treat rowid as unique key 3. Create empty staging table. The table will hold unique keys, data from good columns and just bad column definition. select * into abc_new from abc where 1=2 ; select rowid("abc") as mykey, * into abc_new from abc where 1=2 ; 4. Copy unique keys and data from good columns to staging table. Use null or some unique value for filling bad column. insert into abc_new select a, null, c from abc ; insert into abc_new select rowid("abc"), a, null, c from abc ; 2016 SAP SE or an SAP affiliate company. All rights reserved. Customer 9

10 FAQs - Repair 5. Create another empty staging table. This table will hold unique keys and only bad column definition. select a, b into abc_salvage from abc where 1=2 ; select rowid("abc") as mykey, b into abc_salvage from abc where 1=2 ; 6. Now salvage as much data as you can. This process will be iterative and potentially trial and error with predicates. insert into abc_salvage select rowid("abc"), b from abc where a between.. and.. insert into abc_salvage select rowid("abc"), b from abc where rowdid( "abc" ) between.. and.. 7. Now populate salvaged data in table in step 3. Remaining data can only come from alternate resources. update abc_new set t1.b = t2.b from abc_new t1, abc_salvage t2 where t1.a = t2.a ; update abc_new set t1.b = t2.b from abc_new t1, abc_salvage t2 where t1.mykey = t2.mykey ; 8. Move old table for Root Cause Analysis and future drop. alter table abc rename abc_old ; 9. Move new table as original. If rowid() was used, drop mykey column before move. alter table abc_new drop mykey ; alter table abc_new rename abc ; 2016 SAP SE or an SAP affiliate company. All rights reserved. Customer 10

11 FAQs - Repair -- Prepare create table abc ( a int, b int, c int ) ; insert abc values ( 1, 10, 100 ) ; insert abc values ( 2, 20, 200 ) ; insert abc values ( 3, 30, 300 ) ; -- Assume column b has problems per sp_iqcheckdb, select * from abc where a = 3 does not work giving errors. Now Salvage remaining data. select rowid("abc") as mykey, * into abc_new from abc where 1=2 ; insert into abc_new select rowid("abc"), a, null, c from abc ; select rowid("abc") as mykey, b into abc_salvage from abc where 1=2 ; insert into abc_salvage select rowid("abc"), b from abc where rowid( "abc" ) between 1 and 2 ; update abc_new set t1.b = t2.b from abc_new t1, abc_salvage t2 where t1.mykey = t2.mykey ; alter table abc rename abc_old ; alter table abc_new drop mykey ; alter table abc_new rename abc ; select * from abc ; 2016 SAP SE or an SAP affiliate company. All rights reserved. Customer 11

12 Useful scripts o Find columns names and index names for specified table select column_name + index_name from sp_iqindex( table ) ; o Generate Individual index rebuild sql statements for specified table owned by owner select 'sp_iqrebuildindex owner.table'', ''column ' + column_name + ''' ;' from sp_iqcolumn( table', owner' ) ; o Generate Individual index metadata sql statements for specified table select 'sp_iqindexmetadata ''' + index_name + ''' ;' from sp_iqindex( table' ) ; 2016 SAP SE or an SAP affiliate company. All rights reserved. Customer 12

13 KBA s - specific to today s topic Script to identify and rebuild nbit indexes - SAP IQ IQ got crashed when sp_iqrebuildindex executed in the process of upgrading to IQ IQ16 SP11.01 : rebuildindex on FP encountered server crash with error msg 's_bufman.cxx:5313' Query causes server crash. SAP IQ16 SP sp_iqrebuildindex causes crash at slib/s_vdo.cxx:733 - SAP IQ Data corruption after upgrading from sp01 to sp03 - SAP IQ NLS Compression in IQ vs. Oracle - SAP IQ 2016 SAP SE or an SAP affiliate company. All rights reserved. Customer 13

14 KBA s - product specific Customer Virtual Coffee corner for ASE, IQ, Replication Server, Software Developers Kit Customer Coffee Corner for SAP IQ Americas SAP IQ restore database returns 'Failed to undo index delete' error create database doesn't use symbolic links on filesystem - SAP IQ Cockpit issues an explicit "set chained on command and leaves locks opened. Cockpit IQ Is SUSE Linux 12.1 Certified or Supported in IQ 16? - SAP IQ Load table command failed to load the column with NULL value when the CONVERSION_ERROR is off "The temporary directory name is limited to 64 characters" error - SAP IQ dbisql crashes when loading incorrect version of libodbcinst binary SAP IQ 16 Restrictions on the use of new reserved words is not enforced Invalid setting for option 'Mpx_Idle_Connection_Timeout' - SAP IQ 2016 SAP SE or an SAP affiliate company. All rights reserved. Customer 14

15 Closing Remarks What s next? Please provide your feedback to IQ VCC coordinators on Did you learn something new/useful? Did this outreach help understanding sp_iqrebuildindex() better? 2016 SAP SE or an SAP affiliate company. All rights reserved. Customer 15

16 Any Questions?

17 Thank you