Publishing and Subscribing to Cloud Applications with Data Integration Hub

Transcription

1 Publishing and Subscribing to Cloud Applications with Data Integration Hub Informatica LLC. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise) without prior consent of Informatica Corporation. All other company and product names may be trade names or trademarks of their respective owners and/or copyrighted materials of such owners.

2 Abstract This article describes how to set up Data Integration Hub to publish from and subscribe to cloud applications through Informatica Cloud, including tasks that you perform both in Informatica Cloud and in Data Integration Hub. Supported Versions Data Integration Hub Table of Contents Publishing and Subscribing to Cloud Applications with Data Integration Hub Overview Publishing and Subscribing to Cloud Applications with Data Integration Hub Process Creating a Data Integration Hub Connection Creating a Data Synchronization Task for a Publication Step 1. Define Task Details Step 2. Select Publication Source Step 3. Select Publication Target Step 4. Configure Field Mapping Creating a Data Synchronization Task for a Subscription Step 1. Define Task Details Step 2. Select Subscription Source Step 3. Select Subscription Target Step 4. Configure Field Mapping Configuring Cloud Connectivity System Properties Creating Publication and Subscription Workflows Creating a Custom Mapping Publication with a Cloud Task Step 1. Access the Custom Cloud Publication Wizard Step 2. Define Basic Publication Properties Step 3. Select Publication Workflow Step 4. Define Publication Schedule Step 5. Review Publication Settings and Save the Publication Creating a Custom Mapping Subscription with a Cloud Task Step 1. Access the Custom Cloud Subscription Wizard Step 2. Define Basic Subscription Properties and Select Topics Step 3. Select Subscription Workflow Step 4. Define Subscription Schedule

3 Step 5. Review Subscription Settings and Save the Subscription Publishing and Subscribing to Cloud Applications with Data Integration Hub Overview With Data Integration Hub, you can publish from and subscribe to any cloud-based application that Informatica Cloud supports using Informatica Cloud tasks. To set up Data Integration Hub to work with cloud publications and subscriptions, you need to set up Informatica Cloud to work with Data Integration Hub connector. In Data Integration Hub, you need to set up connectivity to Informatica Cloud. After you set up both systems, you create Data Synchronization tasks in Informatica Cloud, and then create workflows, cloud publications, and cloud subscriptions in Data Integration Hub, as required. Publishing and Subscribing to Cloud Applications with Data Integration Hub Process Setting up publishing from and subscribing to cloud applications through Informatica Cloud includes the following tasks: In Informatica Cloud Creating and maintaining an Informatica Cloud organization. For more information, see the Cloud Administrator Guide. Installing the cloud Data Integration Hub connector. The Data Integration Hub connector is an add-on connector that you must download and install. You cannot install the Data Integration Hub connector as a free trial. For more information, see the Cloud Administrator Guide. Note: If you do not download and install the Data Integration Hub connector, it does not display on the Add-On Connectors page and is not available for selection when you create a connection. Creating a cloud Data Integration Hub connection that uses the Data Integration Hub connector. Creating Data Synchronization tasks for publications and subscriptions. In Data Integration Hub Configuring cloud connectivity system properties. Creating publication and subscription workflows that use the Data Synchronization tasks that you created in Informatica Cloud. Creating cloud publications and subscriptions. Creating a Data Integration Hub Connection 1. In Informatica Cloud home page, click Configure and then select Connections. The Connections page appears. 2. Click New. The New Connection page appears. 3. Specify the following details: Connection Name Enter a unique name for the connection. 3

4 Description Type Provide a relevant description for the connection. Select Data Integration Hub (Informatica LLC) from the list. Runtime Environment The name of the runtime environment where you want to run the tasks. Data Integration Hub Console URL The Data Integration Hub Operation Console URL. The default is Data Integration Hub Console Username User name to connect to the Data Integration Hub Operation Console. Data Integration Hub Console Password Password to connect to the Data Integration Hub Operation Console. Data Integration Hub Server URL The Data Integration Hub server URL. The default is Must be identical to the URL in the dx-configuration.properties file that resides in the Data Integration Hub installation. Data Integration Hub Internal Username User name to connect to the Data Integration Hub server. Must be identical to the username in the dx-configuration.properties file that resides in the Data Integration Hub installation. The default value is servicedefaultusername. Data Integration Hub Internal Password Password to connect to the Data Integration Hub server. Must be identical to the password in the dx-configuration.properties file that resides in the Data Integration Hub installation. Publication Repository JDBC Connection URL Optional. The Data Integration Hub publication repository URL. Must point to the Data Integration Hub publication repository. For Oracle the format must be jdbc:informatica:oracle://<host>:<port>;sid=<sidname> For Microsoft SQL Server the format must be jdbc:informatica:sqlserver:// <host>:<port>;databasename=<dbname> Publication Repository JDBC Jar Directory The path where the Data Integration Hub publication repository JAR files reside. The default is <DIHInstallationDir>/shared/lib. Must be local to the Secure Agent that resides in the location that you specify in Runtime Environment. If you do not specify a directory path here, the Secure Agent obtains the JAR files from the directory that is specified in the CLASSPATH system variable. Publication Repository Schema Microsoft SQL Server only. Data Integration Hub publication repository schema. 4

5 Publication Repository Username User name to connect to the Data Integration Hub publication repository. Publication Repository Password Password to connect to the Data Integration Hub publication repository. 4. Click Test to evaluate the connection. A message informs you that the test is successful. 5. Click OK. Creating a Data Synchronization Task for a Publication To create a Data Synchronization Task for a publication, perform the following tasks: Define task details. Select the publication source. The source is the cloud application from which you want to publish data. Select the publication target. The target is the topic table in the Data Integration Hub publication repository, into which the cloud application publishes the data. Optionally, define data filters. Configure field mapping. Map source fields to topic fields. Save and close the task. Step 1. Define Task Details Define task properties in the Definition page of the Data Synchronization Task Wizard. 1. Click Apps > Data Synchronization. The Data Synchronization Task Wizard appears. 2. Specify the following details and then click Next: Task Name Enter a name for the Data Synchronization task. The name of a Data Synchronization task must be unique within the organization. A Data Synchronization task name can contain alphanumeric characters, spaces, and the following special characters:_ A Data Synchronization task name is not case sensitive. Tip: Indicate the type of the task in the task name. This will ensure that when you select a task to use in a Data Integration Hub publication workflow, you select a publication task. For example, name the task Pub_<TaskName>. Description Optionally, enter a description for the task. The description can have a maximum length of 255 characters. Task Operation Choose Insert. The Source page appears. 5

6 Step 2. Select Publication Source Select the publication source in the Source page of the Data Synchronization Task Wizard. u Specify the following details and then click Next: Connection Select a source connection that connects to the source from which you want to publish data. Source Type The source type depends on the number of tables that you want to publish: To publish a single table, select Single. To publish multiple tables, select Multiple and then create a relationship between the tables. Source Object Select the source from which you want to publish data. The Tareget page appears. Step 3. Select Publication Target Select the publication target in the Target page of the Data Synchronization Task Wizard. 1. Specify the following details and then click Next: Connection Select the Data Integration Hub connection. Target Object Select the topic table to which you want to publish data. The format of the target object is TopicName.tableName. The Data Filters page appears. 2. Optionally, configure data filters. You configure data filters for Data Integration Hub publications in the same way that you configure data filters for other Informatica Cloud tasks. Click Next. The Field Mapping page appears. Step 4. Configure Field Mapping Map source fields to topic fields in the Field Mapping page of the Data Synchronization Task Wizard. 1. Map fields in the Source column to fields in the Target column and then click Next: The Schedule page appears. 2. Verify that the option Do not run this task on a schedule is selected and then select Save > Save and Close to save the task and close the Data Synchronization Application. Creating a Data Synchronization Task for a Subscription To create a Data Synchronization Task for a subscription, perform the following tasks: Define task details. Select the subscription source. The source is the topic table in the Data Integration Hub publication repository, from which you want to consume data. 6

7 Select the subscription target. The target is the cloud application that you want to consume the data. Optionally, define data filters. Configure field mapping. Map topic fields to target fields. Save and close the task. Step 1. Define Task Details Define task properties in the Definition page of the Data Synchronization Task Wizard. 1. Click Apps > Data Synchronization. The Data Synchronization Task Wizard appears. 2. Specify the following details and then click Next: Task Name Enter a name for the Data Synchronization task. The name of a Data Synchronization task must be unique within the organization. A Data Synchronization task name can contain alphanumeric characters, spaces, and the following special characters:_ A Data Synchronization task name is not case sensitive. Tip: Indicate the type of the task in the task name. This will ensure that when you select a task to use in a Data Integration Hub subscription workflow, you select a subscription task. For example, name the task Sub_<TaskName>. Description Optionally, enter a description for the task. The description can have a maximum length of 255 characters. Task Operation Choose Insert. The Source page appears. Step 2. Select Subscription Source Select the subscription source in the Source page of the Data Synchronization Task Wizard. u Specify the following details and then click Next: Connection Select the Data Integration Hub connection. Source Type The source type depends on the number of tables that you want to consume and on the subscription type: To consume a single table, select Single. To consume multiple tables, or when the subscription is a compound subscription, select Multiple Source Object and then create a relationship between the tables. Select the topic table from which you want to consume data. The format of the object is TopicName.tableName. The Tareget page appears. 7

8 Step 3. Select Subscription Target Select the subscription target in the Target page of the Data Synchronization Task Wizard. 1. Specify the following details and then click Next: Connection Select a target connection that connects to the target into which you want to consume data. Target Object Select the target into which you want to consume the data. The Data Filters page appears. 2. Optionally, configure data filters. You configure data filters for Data Integration Hub subscriptions in the same way that you configure data filters for other Informatica Cloud tasks. Click Next. The Field Mapping page appears. Step 4. Configure Field Mapping Map topic fields to target fields in the Field Mapping page of the Data Synchronization Task Wizard. 1. Map fields in the Source column to fields in the Target column and then click Next: The Schedule page appears. 2. Verify that the option Do not run this task on a schedule is selected and then select Save > Save and Close to save the task and close the Data Synchronization Application. Configuring Cloud Connectivity System Properties Configure cloud connectivity system properties in the Data Integration Hub Operation Console. 1. In the Navigator, click Administration > System Properties. The System Properties page appears. 2. Click the Edit icon next to the following system properties enter the values for cloud connectivity: dih.ics.url URL of the Informatica Cloud server. The following URL is the default value: dih.ics.username User name for the Informatica Cloud server. dih.ics.password Password for the Informatica Cloud server. Creating Publication and Subscription Workflows Configure a publication workflow and a subscription workflow in the Data Integration Hub Operation Console. 1. In the Navigator, click Hub Management > Workflows and then click New Workflow. The Create Workflow page appears. 2. Specify the following details and then click Save: 8

9 Workflow Name Name of the Data Integration Hub workflow. The name can contain up to 60 characters and can include spaces and special characters. Description Flow Type Type Workflow Optional description of the workflow. Select Cloud Data Synchronization Task. Type of process for which to use the workflow. For a publication workflow, select Publication. For a subscription workflow, select Subscription. Click the ellipses to open the Add a Cloud workflow dialog box. Choose the appropriate task from the list and then click OK. 3. In the Create Workflow page click Save. Creating a Custom Mapping Publication with a Cloud Task To create a publication with a custom mapping and a cloud task, perform the following tasks: 1. Access the create publication wizard. 2. Define basic publication properties. 3. Select the publication workflow. Only workflows that are based on an Informatica Cloud task are available for selection 4. Define the publication schedule. 5. Review the publication settings and save the publication. Step 1. Access the Custom Cloud Publication Wizard Access the create publication wizard in the Data Integration Hub Operation Console. 1. In the Navigator, click Hub Management > Applications. The Applications page appears. 2. Click the name of the application from which you want to publish content. The Edit Application page appears. 3. Select the Publications tab, and then click New Publication > Custom mapping > Cloud. The Create Custom Cloud Publication wizard appears. Step 2. Define Basic Publication Properties Define publication properties in the General page of the publication wizard. 1. Enter the publication name. 2. Optionally, enter a description of the publication. 3. Select a topic from the Topic list. 9

10 The Topic Structure area shows the structure of the topic to which the application publishes the data. You can view the structure of all the tables in the topic, or choose a table for which to show the structure. 4. Click Next. The Mapping page appears. Step 3. Select Publication Workflow Select a Data Integration Hub publication workflow in the Mapping page of the publication wizard. 1. Select a workflow from the Custom Mapping list. Only publication workflows that are based on an Informatica Cloud task are available for selection. 2. Click Next. The Schedule page appears. Step 4. Define Publication Schedule Define the method and the frequency of data publishing in the Schedule page of the publication wizard. 1. Select the method and the frequency of data publishing. Manually or by an external trigger No schedule. You can use the following methods to run the publication: Run manually. Click the Run arrow on the Publications page. Run by an API. Call a command-line API or a web service API that starts the publication. Run by a PowerCenter workflow. Start a publication transformation. For file publications that use this scheduling option and that publish multiple files, all the files must be present in the source location when the publication starts. By schedule 2. Click Next. Runs the publication according to the defined schedule. Select one of the following options: Every n minutes. Runs the publication in intervals of up to 60 minutes. You select the number of minutes from the list. Hourly. Runs the publication in intervals of up to 24 hours. You select the number of hours from the list. The publication runs at the beginning of the hour. For example, if you enter 2, the publication runs at 00:00, 02:00, and at consecutive two-hour intervals. Daily. Runs the publication at the same hour every day. Weekly. Runs the publication every week on one or more days at the same hour. Monthly. Runs the publication every month on a specific date or a specific day at the same hour. Define the publication intervals in the Repeat running area. For file publications that use this scheduling option and that publish multiple files, all the files must be present in the source location when the publication starts. The Summary page appears. Step 5. Review Publication Settings and Save the Publication Review the publication settings and save the publication in the Summary page of the publication wizard. 1. Review the publication settings. 10

11 2. Click Finish. The publication wizard closes. The Publications tab of the Edit Application page shows the publication you created. The publication is listed in the Publications catalog. If you configured a publication schedule, the publication publishes data according to the defined schedule. Creating a Custom Mapping Subscription with a Cloud Task To create a subscription with a custom mapping with a cloud task, perform the following tasks: 1. Access the create subscription wizard. 2. Define basic subscription properties and select the topic to which you want to subscribe. 3. Select the subscription workflow. Only workflows that are based on an Informatica Cloud task are available for selection. 4. Define the subscription schedule. 5. Review the subscription settings and save the subscription. Step 1. Access the Custom Cloud Subscription Wizard Access the create subscription wizard in the Data Integration Hub Operation Console. 1. In the Navigator, click Hub Management > Applications. The Applications page appears. 2. Click the name of the application to which you want to deliver content. The Edit Application page appears. 3. Select the Subscriptions tab, and then click New Subscription > Custom mapping > Cloud. The Create Custom Cloud Subscription wizard appears. Step 2. Define Basic Subscription Properties and Select Topics Define subscription properties and select topics in the General page of the subscription wizard. 1. Enter the subscription name. 2. Optionally, enter a description of the subscription. 3. Select the topic to which you want to subscribe and then click Add Topic. The topic appears in the topic table. 4. Optionally, repeat step 3 to subscribe to additional topics and create a compound subscription. 5. For a compound subscription, in the Wait for all topics to be available for consumption for... hours field, specify the number of hours to wait until all published data is available from the time the first publication is ready to consume. If all of the publications in all topics finish publishing the data before the maximum time, the subscription process runs immediately after the last publication is ready to consume. If some publications are not ready to consume within the maximum time, the subscription process does not run. An error event is created, and no data is delivered. 6. Click Next. The Mapping page appears. 11

12 Step 3. Select Subscription Workflow Select a Data Integration Hub subscription workflow in the Mapping page of the subscription wizard. 1. Select a workflow from the Custom Mapping list. Only subscription workflows that are based on an Informatica Cloud task are available for selection. 2. Click Next. The Schedule page appears. Step 4. Define Subscription Schedule Define the method and the frequency of the subscription in the Schedule page of the subscription wizard. If you create a compound subscription, where the subscription consumes data sets from multiple topics, you can only choose to consume data when it is published. 1. Select the method and the frequency of the subscription. When published data is ready Runs the subscription immediately after the published data is ready. Manually or by an external trigger No schedule. You can use the following methods to run the subscription: Run manually. Click the Run arrow on the Subscriptions page. Run by an API. Call a command-line API or a web service API that starts the subscription. By schedule Runs the subscription according to the defined schedule. Select one of the following options: Every n minutes. Runs the subscription in intervals of up to 60 minutes. You select the number of minutes from the list. Hourly. Runs the subscription in intervals of up to 24 hours. You select the number of hours from the list. 2. Click Next. Daily. Runs the subscription at the same hour every day. Weekly. Runs the subscription every week on one or more days at the same hour. Monthly. Runs the subscription every month on a specific date or a specific day at the same hour. Define the delivery intervals in the Repeat running area. The Summary page appears. Step 5. Review Subscription Settings and Save the Subscription Review the subscription settings and save the subscription in the Summary page of the subscription wizard. 1. Review the subscription settings. 2. Click Finish. The subscription wizard closes. The Subscription tab of the Edit Application page shows the subscription you created. If you configured a subscription schedule, the subscription consumes data according to the defined schedule. 12

13 Authors Lior Mechlovich Manager, Development Rachel Aldam Principal Technical Writer 13