opensap TEXT ANALYTICS WITH SAP HANA PLATFORM WEEK 1 Version: January 20, 2016 Exercises / Solutions Anthony Waite / SAP Labs, LLC. Bill Miller / SAP Labs, LLC.
Contents Desktop in SAP Cloud Appliance Library... 4 Preconfigured front-end client... 4 Front-End Client Configuration... 6 Getting Help... 6 Adding a HANA Studio Perspective... 6 Create a connection to the HANA server... 8 Create a Repository Workspace... 11 Create a HANA Project... 13 Important HANA Studio customizations... 15 (Mandatory) Set permissions for course exercises... 17 Exercise 1 Solution... 21 In SAP HANA Studio... 21 Exercise 2 Solution... 25 In SAP HANA Studio... 25 Exercise 3 Solution... 28 In SAP HANA Studio... 28 Exercise 4 Solution... 30 In SAP HANA Studio... 30 2
BEFORE YOU START System Host: HANA IP address System Instance Number: 00 System User ID: SYSTEM Password: Master Password you entered for the solution when creating the instance in the SAP Cloud Appliance Library 3
DESKTOP IN SAP CLOUD APPLIANCE LIBRARY Preconfigured front-end client Steps Screenshot 1) Click on the SAP Dev Tools for Eclipse icon on your desktop in the SAP Cloud Appliance Library (CAL) to open the front-end client. Note: If you have SAP HANA Studio installed on your computer, you can use it instead of the preconfigured front-end client in CAL. If so, bypass this section and jump to the following Adding a HANA Studio Perspective. 2) Click on the Systems view. Right-click on HDB (SYSTEM). 3) Select Log On from the pop-up menu. 4
4) Log on as: User Name: SYSTEM Password: Master Password you specified when creating the instance in SAP CAL Click the OK button to close the dialog box. 5) You are now logged on to your SAP HANA database. 5
FRONT-END CLIENT CONFIGURATION Getting Help If you need additional help resources beyond this document: http://help.sap.com/hana/sap_hana_developer_guide_en.pdf Adding a HANA Studio Perspective Steps Screenshot 1) To support the developercentric workflow, there is an additional Eclipse perspective which has been added to SAP HANA Studio. This may not displayed by default. In the upper right corner of your front-end client, click the Open Perspective button. 2) Add the SAP HANA Development perspective. This is the perspective you should be using for the entire workshop. 3) After adding the SAP HANA Development perspective, you may also still see other perspectives such as SAP HANA Administration Console. If that is the case, you may want to right-click and choose Close as we will not be using this or any 6
other perspectives. 7
Create a connection to the HANA server Steps Screenshot 1) Make sure you are in the SAP HANA Development perspective by clicking on the button. Note: This section only applies to you if you are using your own SAP HANA Studio installation on your computer. For those of you using the SAP Dev Tools for Eclipse in CAL, bypass this section and jump to the following Create a Repository Workspace. 2) Click on the Systems view. Right-click on the white space below this tab and choose Add System. 8
3) Enter the server Host Name: HANA IP address Enter the Instance Number: 00 Optionally, you can enter a meaningful description of your choice. Click the Next button. 4) Specify the user ID and password. User Name: SYSTEM Password: Master Password specified when creating the instance in SAP CAL Select the Store user name and password in secure storage option. Click the Finish button. 9
5) You should now have a new connection with your specific user ID for the HANA system. Please make sure to use this connection for the rest of the course exercises. Note: The System ID and users shown in these screenshots might be different than the ones you are working with and may be blurred to avoid any confusion. 10
Create a Repository Workspace Steps Screenshot 1) Switch to the Repositories view. Find your system entry and the default workspace. Right-click and choose Create Repository Workspace. Note: In this and the following screenshot, your system naming conventions and workspace paths may be different. 2) Confirm the file system location on your local machine which will hold the local copy of this workspace. Click the Finish button. 11
3) You should now see the local workspace mapped to the remote workspace in the Repositories view. Note: Your System ID, User ID, Host Name, and System Numbers will be different than those displayed in subsequent screenshots. For this reason we may have blurred this information in screenshots to avoid confusion. 12
Create a HANA Project Steps Screenshot 1) Under the Repositories tab, navigate to (Default) / student00. Right-click student00 and choose Check Out and Import Projects from the popup menu. 2) Click the Next button. Note: It may take several minutes to download the package contents to your local machine. 13
3) Click the Finish button to complete the process. 4) Switch to the Project Explorer view. You should see the student00 project that you created in the previous step. 14
Important HANA Studio customizations Steps Screenshot 1) Open Preferences from the Window menu. 2) In the Preferences dialog box, under SAP HANA / Runtime / Result, select the Enable zoom of LOB columns option. Click the Apply button. 15
3) Continuing in the Preferences dialog box, navigate to General / Appearance / Content Types. Under the Content Types tree, navigate to Text / XML. Click the Add button and enter: *.hdbtextconfig *.hdbtextdict You will see these two new content types under File associations. 4) Continuing in the Preferences dialog box, navigate to General / Editors / File Associations. Under File types, highlight *.sql. Under Associated editors, highlight SAP HANA SQL Console, then click the Default button. Note: This particular setting may apply to you if you are using the SAP Dev Tools for Eclipse in CAL. Click the OK button to close the Preferences dialog box. 16
(Mandatory) Set permissions for course exercises Steps Screenshot 1) Under the Repositories view, navigate to (Default) / student00 / data. Doubleclick on prepare.sql. 2) If there is No connection to database displayed in the SQL console, click on the Choose Connection icon, which is found right of the green circle with an arrow (Execute) icon. 17
3) In the Choose Connection dialog, select the appropriate database. Click the OK button. 4) In the SQL console, you will notice the following SQL syntax from the prepare.sql file: call "_SYS_REPO"."GRANT_ACTIVAT ED_ROLE"('student00.securi ty::ta_developer','system' ); Click on the Execute (green circle with an arrow) icon or hit the F8 key. 5) Select the Don t show this message again during this editor session option and click the Yes button. Note: Disregard this warning message throughout the course exercises. 18
6) Afterwards you should see a similar confirmation in the SQL console: Statement 'call "_SYS_REPO"."GRANT_ACTIVAT ED_ROLE"('student00.securi ty::ta_developer','system' )' successfully executed in 17 ms 256 µs (server processing time: 15 ms 968 µs) - Rows Affected: 0 Please proceed to the exercises. 19
EXERCISE 1 CREATE FULLTEXT INDEX WITHOUT SPECIFYING LANGUAGES Objective In this exercise, you will execute a SQL statement to create a fulltext index for your copy of the workshop data. Intentionally, we will omit the LANGUAGE DETECTION parameter. Exercise Description Create fulltext index without specifying possible languages Search for restaurant terms in the conte Observe that English is assumed for all inputs 20
EXERCISE 1 SOLUTION In SAP HANA Studio Steps Screenshot 1) Under the Repositories tab, navigate to (Default) / student00 / solutions / week-1. Double-click on exercises.sql. 2) If there is No connection to database displayed in the SQL console, click on the Choose Connection icon, which is found right of the green circle with an arrow (Execute) icon. 21
3) In the Choose Connection dialog, select the appropriate database. Click the OK button. 4) In the SQL console, highlight the following SQL syntax: SET SCHEMA OPENSAP_TA_WORKSHOP; Click on the Execute (green circle with an arrow) icon or hit the F8 key. Note: If you close this session at any point while working on Week 1 exercises, you will need to re-execute this command at the start. 5) In the SQL console, highlight the following SQL syntax: CREATE FULLTEXT INDEX PRODUCT_REVIEWS_IDX ON "student00.data::product_r EVIEWS"(CONTENT) FAST PREPROCESS OFF; 22
Click on the Execute (green circle with an arrow) icon or hit the F8 key. 6) In the SQL console, highlight the following SQL syntax: SELECT FILE_NAME, LANGUAGE(CONTENT), SNIPPETS(CONTENT) FROM "student00.data::product_r EVIEWS" WHERE CONTAINS(CONTENT, 'meal OR menu OR entree OR appetizer OR food', LINGUISTIC); Click on the Execute (green circle with an arrow) icon or hit the F8 key. 7) As you can see, these are the results of the search for some restaurant terms in the fulltext content. Notice in the query results that English is assumed for all inputs. en = English 23
EXERCISE 2 RECREATE FULLTEXT INDEX SPECIFYING LANGUAGES Objective In this exercise, you will execute a SQL statement to create a fulltext index for your copy of the workshop data. We will specify the LANGUAGE DETECTION parameter. Exercise Description Drop fulltext index from previous exercise Create fulltext index specifying possible languages Search for restaurant terms in the content Language will be automatically determined for all inputs 24
EXERCISE 2 SOLUTION In SAP HANA Studio Steps Screenshot 1) In the SQL console, highlight the following SQL syntax: DROP FULLTEXT INDEX PRODUCT_REVIEWS_IDX; Click on the Execute (green circle with an arrow) icon or hit the F8 key. 2) In the SQL console, highlight the following SQL syntax: CREATE FULLTEXT INDEX PRODUCT_REVIEWS_IDX ON "student00.data::product_r EVIEWS"(CONTENT) FAST PREPROCESS OFF LANGUAGE DETECTION ('EN', 'DE', 'FR'); Click on the Execute (green circle with an arrow) icon or hit the F8 key. 3) In the SQL console, highlight the following SQL syntax: SELECT FILE_NAME, LANGUAGE(CONTENT), SNIPPETS(CONTENT) FROM "student00.data::product_r EVIEWS" WHERE CONTAINS(CONTENT, 'meal OR menu OR entree OR appetizer OR food', LINGUISTIC); Repeating the same search in EXERCISE 1, click on the Execute (green circle with an arrow) icon or hit the F8 key. 4) As you can see, these are the results of the search for some restaurant terms in the fulltext content. Notice 25
in the query results that the correct language is detected for all inputs. en = English de = German fr = French 26
EXERCISE 3 RECREATE FULLTEXT INDEX SPECIFYING LANGUAGES WITH TEXT ANALYSIS ON Objective In this exercise, you will execute a SQL statement to create a fulltext index for your copy of the workshop data. We will specify the LANGUAGE DETECTION and TEXT ANALYSIS ON parameters. Exercise Description Drop fulltext index from previous exercise Create fulltext index specifying languages and text analysis output Notice new table with text analysis annotations for each input 27
EXERCISE 3 SOLUTION In SAP HANA Studio Steps Screenshot 1) In the SQL console, highlight the following SQL syntax: DROP FULLTEXT INDEX PRODUCT_REVIEWS_IDX; Click on the Execute (green circle with an arrow) icon or hit the F8 key. 2) In the SQL console, highlight the following SQL syntax: CREATE FULLTEXT INDEX PRODUCT_REVIEWS_IDX ON "student00.data::product_r EVIEWS"(CONTENT) FAST PREPROCESS OFF LANGUAGE DETECTION ('EN', 'DE', 'FR') TEXT ANALYSIS ON; Click on the Execute (green circle with an arrow) icon or hit the F8 key. 3) In the SQL console, highlight the following SQL syntax: SELECT * FROM "$TA_PRODUCT_REVIEWS_IDX" WHERE TA_LANGUAGE = 'en' ORDER BY ID, TA_COUNTER; Click on the Execute (green circle with an arrow) icon or hit the F8 key. 4) In the query results, notice the text analysis annotations for each input, in order of their appearance in the original text. 28
EXERCISE 4 MONITOR TEXT ANALYSIS PROCESSING Objective In this exercise, you will learn how to monitor the progress and status of your text data processing. Exercise Description Run a command to monitor the progress for text processing Run a command to report any errors for text processing 29
EXERCISE 4 SOLUTION In SAP HANA Studio Steps Screenshot 1) In the SQL console, highlight the following SQL syntax: SELECT * FROM SYS.M_FULLTEXT_QUEUES WHERE SCHEMA_NAME = 'OPENSAP_TA_WORKSHOP' AND TABLE_NAME = 'student00.data::product_r EVIEWS'; Click on the Execute (green circle with an arrow) icon or hit the F8 key. 2) Notice the queue status is shown at the table level. 3) In the SQL console, highlight the following SQL syntax: SELECT ID, INDEXING_STATUS(CONTENT), INDEXING_ERROR_CODE(CONTEN T), INDEXING_ERROR_MESSAGE(CON TENT) FROM "student00.data::product_r EVIEWS" ORDER BY ID; Click on the Execute (green circle with an arrow) icon or hit the F8 key. 4) Notice this shows the errors for each job (document a.k.a. row of the input table). Note: This command will be useful later when we start modifying the configuration settings for text analysis. 30
31
www.sap.com 2015 SAP SE or an SAP affiliate company. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company. SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. Please see http://www.sap.com/corporate-en/legal/copyright/index.epx#trademark for additional trademark information and notices. Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors. National product specifications may vary. These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty. In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation, and SAP SE s or its affiliated companies strategy and possible future developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.