TABLE DISTRIBUTION IN HANA HANA SAP Active Global Support, June 2012
Table Distribution : Why Load Balancing Parallelization Table Partitioning - A non-partitioned table can support only 2 billion rows. - Same table split across multiple index servers reduces delta merge processing time as modifications are restricted to a sub-set of tables and merge is spread across multiple nodes. 2011 SAP AG. All rights reserved. Internal 2
Table Redistribution : Concept SAP HANA Supports two ways of distributing data.. HANA Slave Hana Master * Different tables assigned to different index servers (Database Partitioning) HANA Slave HANA Slave Hana Master * Same table split across multiple index servers (Table Partitioning) HANA Slave 2011 SAP AG. All rights reserved. Internal 3
Table Redistribution : Concept ~ Master Redistribution Master Slave Slave Slave Slave ~ During Redistribution the link to the table and not the physical table, is moved Prior to Rev54, Index Server crashes if the link is moved without moving the physical table. SAP Note 1847236. The physical table is moved after table re-distribution whenever the next merge is triggered By default tables are distributed in a round-robin manner. However It is also possible to specify explicitly that a table or partition be created on a specific index server Manually move physical tables after redistribution prior to rev 54, before restarting the indexserver 2011 SAP AG. All rights reserved. Internal 4
Table Redistribution : Pre-Requisites You must have the system privilege RESOURCE ADMIN and at least the object privilege ALTER for all schemas involved to be able to run table Re-Distribution. To Modify table distribution manually from one host to another following system privileges are required : * System Privilege DATA ADMIN or * System Privilege CATALOG READ and SQL privilege ALTER Changing how tables are distributed across hosts of a distributed HANA system is a critical operation. Therefore before executing redistribution, it is recommended to save the current distribution of table data so that It can be restored later if needed. [HANA Studio ->Table Redistribution ->Landscape- >Redistribution->Save->Execute] Preferable be on HANA Revision 54 or higher (From Previous Slide). 2011 SAP AG. All rights reserved. Internal 5
Table Redistribution : Redistribution Scenarios Redistributing table before removing a host [HANA Studio->Administration Editor->Landscape-> Configuration->Remove Host] The system marks the host for removal and executes the required redistribution operation. This results in the data on the index server of the host being moved to the index servers of the remaining hosts in the system. Redistributing table after adding a host [HANA Studio]->Administration Editor->Landscape->Configuration->Redistribution-> Redistribution Operations->Redistribute tables after adding host(s)->execute]. The system redistributes the tables in your system across all available index servers. The associated redistribution operation appears in the list of executed operations. Restoring Previous Table Redistribution [HANA Studio]->Administration Editor->Landscape->Configuration->Redistribution-> Executed Operations->Show Plan->Restore]. Optimizing Table Redistribution [HANA Studio]->Administration Editor->Landscape->Configuration->Redistribution-> Redistribution Operations->Optimize table distribution->execute]. The system redistributes and repartition the tables and partitions in your system. 2011 SAP AG. All rights reserved. Internal 6
Table Distribution : What resides where Master Node : Handles OLTP Load. ABAP system tables and general operational data of BW are stored on the master node. DDL statements are executed on this node, global locks are acquired here. Slave Node : Handles OLAP load exclusively. BW Data (master data + cubes/dso/psas) is distributed evenly across all slaves. This ensures a balanced utilization of the available CPU and memory resources. SAP note 1659383 contains the current list of tables that are usually created as row store tables. All BW Master data is distributed across the slave servers using round robin. Fact, DataStore Object (DSO) and Persistent Staging Area (PSA) tables are partitioned with the predefined rule and distributed to the slave servers. There are no BW data tables on the master server. Other BW tables, including row-store tables, ABAP system tables and General Operation tables like REPOSRC are created on the master. 2011 SAP AG. All rights reserved. Internal 7
Table Redistribution : Configuration * In a BW/HANA deployment you want the table location to be decided by a pre-defined BW table distribution rule. In order for the pre-defined rule to take effect, the method parameter in the table_placement section of global.ini must be set to 2. The parameter is applied only when creating new tables. * Parameter bw_schema in the table_placement section of the global.ini controls which schema is allowed on the master server. * Tables in all other schemas are created on slave servers. * All tables in BW that start with a / are always created on the slave servers. All other tables are only created on the master server if they are in one of the BW schema specified by parameter global.ini/table_placement/bw_schema. 2011 SAP AG. All rights reserved. Internal 8
Table Redistribution : Monitoring SQL to Verify where the tables or partitions of a split table physically reside : select * from M_TABLE_PERSISTENCE_LOCATIONS where schema_name='your_schema' and table_name='your_table' SQL to check if there are tables in the system which have there persistence on a different host : "select schema_name, table_name, part_id, count(*) from "SYS"."M_TABLE_PERSISTENCE_LOCATIONS_" group by schema_name, table_name, part_id having count(*) > 1;" If the result is empty, there are no tables with persistence on different host. If there are table returned, there persistence can be physically moved by running the following command : Alter table <TABLE> move partition <PART_ID Physical; 2011 SAP AG. All rights reserved. Internal 9
Table Redistribution : FAQ Tables can also be redistributed from within SAP BW. - SAP Note 1792096 HANA Scale-out landscape, BW table partitions - SAP Note 1695778 Partitioning BW tables in SAP HANA Database RSDU_TABLE_CONSISTENCY report will, in first run, identify tables need to be partitioned; in second run it will partition the tables Once a table is split, the entire table cannot be moved as a whole in one shot. Once the table is split, each partition has to be moved one at a time. Alter table <table_name> move part i to location <host:port> For non-split table, the entire table can be moved in one shot : Alter table name Move To Location <new_host:port> The port is index server internal port, which ends with 03. For example, if your instance is 24, then the port will be 32403 and the query for moving the whole table would be : alter table <table_name> Move To Location new_host:32403 2011 SAP AG. All rights reserved. Internal 10
2012 SAP AG. All rights reserved No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation. IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, System z9, z10, z9, iseries, pseries, xseries, zseries, eserver, z/vm, z/os, i5/os, S/390, OS/390, OS/400, AS/400, S/390 Parallel Enterprise Server, PowerVM, Power Architecture, POWER6+, POWER6, POWER5+, POWER5, POWER, OpenPower, PowerPC, BatchPipes, Center, System Storage, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, Parallel Sysplex, MVS/ESA, AIX, Intelligent Miner, WebSphere, Netfinity, Tivoli and Informix are trademarks or registered trademarks of IBM Corporation. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries. Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks or registered trademarks of Adobe Systems Incorporated in the United States and/or other countries. Oracle is a registered trademark of Oracle Corporation. UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group. Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems, Inc. HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C, World Wide Web Consortium, Massachusetts Institute of Technology. Java is a registered trademark of Sun Microsystems, Inc. JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape. SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer, StreamWork, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries. Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. Business Objects is an SAP company. Sybase and Adaptive Server, ianywhere, Sybase 365, SQL Anywhere, and other Sybase products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Sybase, Inc. Sybase is an SAP company. All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary. The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without the express prior written permission of SAP AG. This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intended strategies, developments, and functionalities of the SAP product and is not intended to be binding upon SAP to any particular course of business, product strategy, and/or development. Please note that this document is subject to change and may be changed by SAP at any time without notice. SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, or other items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitation shall not apply in cases of intent or gross negligence. The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages. 2011 SAP AG. All rights reserved. Internal 11