Informatica BCI Extractor Solution

Transcription

1 Informatica BCI Extractor Solution Objective: The current BCI implementation delivered by Informatica uses a LMAPI SDK plugin to serially execute idoc requests to SAP and then execute a process mapping for the requested data source. SAP logical systems have the capability to process multiple extractors concurrently. I have developed a solution with new mappings and workflows that will enhance the BCI process to gain significant performance improvements, availability and monitoring. Key features: 1. Execute multiple concurrent extracts from the same Logical system. 2. Run parallel extract jobs for full or delta initialization extracts and process sessions. 3. Ability to execute datasource extracts from a 3 rd party schedule. 4. Includes an SAP RFC Ping utility that is integrated in the request session so it will not start an extract request if the BCI listener is not actively connected to the SAP server. 5. A more detailed SAP error message is returned in session log of a send request for RQState 5 RSINFO return. 6. Database views for information and monitoring: a. BCI_REQUEST_STATS: Tracks statistics of current and past BCI extracts from RSINFOSTAGING table. b. BCI_REQUEST_ERR_MSG: Displays all errors in RSINFOSTAGING table for each extractor. Requirements: This solution is currently built on Oracle, but should be transferrable to DB2 or SQL server with some minor changes to the SQL in the mapplet. You will need to configure your source SAP system for Informatica BCI (per Informatica s documentation) and a relational connection to the Informatica metadata views. The zip file is located at Contents of zip file: bci_solution.xml Informatica XML file for all objects needed for solution. database objects.sql SQL to create tables and views. sap_bci_paramfile.ipf Sample parameter file to use for the send_request session and contains the parameters needed for the java transformation. JTX jar files folder Two jar files that need to exist on the integration server. These are used by the java transformation in the send_request session. encryptstring.zip This is a small program that will encrypt a string and is needed for the parameter file and java transformation. It is similar to the pmpasswd program. sap_rfc_ping.zip This is a small program that will ping the RFC server on SAP to ensure that a connection between Informatica and SAP is active.

2 Installation steps: 1. Import PowerCenter objects from the bci_solution.xml file 2. Copy JTX jar files to $PMRootDir/ExtProc folder on Integration server. 3. Copy sap_bci_paramfile.ipf parameter file to $PMRootDir/ParamFiles folder integration server. 4. Execute database objects.sql on target database. 5. Extract sap_rfc_ping.zip to $PMRootDir/ExtProc folder on Integration server. Follow instructions in the SAP RFC Ping User Guide.pdf document to download the correct SAPJCO library for your system 6. Extract encryptstring.zip on your desktop client machine. Configuration: 1. Edit the Router transformation in the BCI_Listener mapping to verify that it matches the basic IDoc type in the SAP system. 2. Edit the Java Classpath entry under the properties tab in the send request session of the processing workflow. Ensure the path is the same as on the server where the above jar files were copied to. Alternatively, you can set this path on the Integration server level instead. 3. Modify the sap_bci_paramfile.ipf parameter file with the correct schema names, and Informatica connection names. 4. Execute Encrypt string with database password where Indicator table is located and put in sap_bci_paramfile.ipf parameter file. 5. Create folder $PMRootdir/SrcFiles/bci_request_files. Use this folder to put generated send_request files that are generated from the Generate BCI Mapping wizard in the designer tool.

3 New BCI Extract Process The below process is to show how one data source is requested and processed. With this new solution you can execute multiple processes concurrently to speed up extract/load time for your data loads. 1. Start the BCI Listener workflow (wf_bci_listener). The only difference between the new Listener mapping and the one delivered by Informatica is the removal of Indicator target table and the BCI_Scheduling_Target LMAPI target table. 2. Follow the standard practice of activating the 0ARTICLE_TEXT datasource on the source SAP server and creating a send_request file to be placed on Integration server. There is no difference in the process mapping other than normal customization to your environment (ie database type and modification of the SQL if needed). 3. Execute the wf_process_0article_text workflow. The send_request and processing workflows are combined into one processing workflow.

4 4. The Send Request Session/mapping is executed: The first pipeline in the mapping sends an IDoc request to SAP. The second pipeline reads the command generating Data Command echo 0ARTICLE_TEXT in SQ_ff_input and passes that string into the mplt_get_rqst_status mapplet. mplt_get_rqst_status The mapplet has a java transformation that executes a query on the database and loops until the RQState of the request ID reaches an 8 or 9 status. Once the entry is found in the RSINFO Staging table, it inserts a record into the indicator table and continues to the next transformation. If an RQState of 5 is received the Java transformation fails the session and workflow indicating there is an error.

5 5. The process mapping/session executes, there is no change to what is normally being done. 6. The BCI_check_idoc_error mapping executes This mapping runs a query against a view that parses the RSINFO data and a query on the OPB_SWIDGET_LOG view in the Informatica metadata and compares the row count. If the count does not match, an abort function is executed to fail the workflow. There are two views available that parse the RSINFO Staging data and joins to other tables to show reporting/real time monitoring as well as SAP error reporting on Informatica BCI processing. Sample output for the BCI_REQUEST_STATS view. Sample output for the BCI_REQUEST_ERR_MSG view.

6 Parallel job Example for Full and Initial extracts The process below is to show how one data source can be executed in parallel to gain better performance when extracting. When you request an extract from SAP, a single job is started to extract data from the tables and send to Informatica. With this new process you can execute multiple jobs concurrently to speed up extraction and processing time for your loads. This example shows how to extract 0FI_GL_12 in parallel. 1. Execute the wf_process_0fi_gl_12_full_part workflow. 1) This workflow is using four different send_request sessions with source files that are filtered to only extract a certain range of profit centers in each. Requests are spaced 5 seconds apart to help the java program pick a request ID to track. 2) Once all of the send_request sessions complete, the s_m_create_bci_parameter_file session is executed. This session reads a template parameter file, bci_parmfile_template.txt, (containing connection info), and creates a new parameter file which merges the template and the requestid s from the SOURCE_FOR_BCI table. 3) The process mappings use an override SQL in the session to filter on one $$REQUESTID per session. The final parameter file would look similar to this.