IBM ANALYTICS SOLUTIONS FOUNDATION

Transcription

1 IBM ANALYTICS SOLUTIONS FOUNDATION Developer Guide Release 1.5 1

2 Copyright and Trademarks Licensed Materials - Property of IBM. Copyright IBM Corp IBM and the IBM logo are trademarks or registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at While every attempt has been made to ensure that the information in this document is accurate and complete, some typographical errors or technical inaccuracies may exist. IBM does not accept responsibility for any kind of loss resulting from the use of information contained in this document. The information contained in this document is subject to change without notice. 2

3 Introduction This guide describes the functionality and APIs supported by the Analytics Solutions Foundation. The APIs are used to configure and extend the supplied functionality. Typically, solutions created with Analytics Solutions Foundation consume events and make recommendations. They use profiles and key performance indicators to aggregate the events. These aggregations are used to provide scores using predictive models. Decision management makes recommendations based on the values of the scores. Events are most often generated by devices. But events may also have pre-aggregated measurements, or they may be synthetic events generated periodically to trigger scoring and recommendations. Events may also be created to record the results of scoring and recommendations. Analytics Solutions Foundation provides the following APIs: API's implemented by the orchestration engine - Event Processing API's implemented to support an orchestration engine - Event Storage - KPI and Profile Storage - Profile Calculations - Scoring and Decision Service - Master Data Management The orchestration engine is metadata driven. Event processing and orchestration is specified through metadata. The overall processing and orchestration in response to an event is based on the event's "orchestration key". This key is used to map to one or more orchestrations that will be performed to process the event. Orchestration Definition An orchestration_definition may include a list of event_orchestration_mappings and a list of orchestrations. Mapping Events to Orchestrations Each event passed to the orchestration engine for processing must implement the com.ibm.analytics.foundation.orchestration.api.event interface. The interface allows the orchestration engine to obtain a key value from the event to map it to a set of one or more orchestrations to perform. The mappings are defined as event_orchestration_mappings. Each mapping includes an event_orchestration_key_cd value, which is the key value obtained from an incoming event, and an orchestration_cd value that identifies the orchestration to perform. <event_orchestration_mapping> <event_orchestration_key_cd>com.ibm.pmq.key</event_orchestration_key_cd> <orchestration_cd>com.ibm.pmq.orchestration</orchestration_cd> </event_orchestration_mapping> 3

4 Orchestration Overview The diagram below provides a high-level overview of a possible event processing orchestration, showing the flow of an event into the orchestration engine. The contents of the event are processed and then stored in an event store. The contents of the event may be used to update Key Performance Indicators and Profiles. The key performance indicators provide a history of performance over time. The profiles typically contain only the current aggregated values. KPIs and Profiles may also record information, such as recommended actions determined by a predictive model, and the results of decision management. Event Source Event Source Event Source Orchestration Engine Database SPSS/BPM/Any 3rd party service Event Store Key Performance Indicators Profiles The event processing using a configured orchestration engine can be divided into two stages. In the first stage, external events are received and mapped into the Java event format required by the configured recommendation engine. Receiving and mapping external events can be achieved using an integration product such as IBM Integration Bus. The events are optionally inserted into an event store. The data values in the events are used to immediately update measures in key performance indicators for a KPI period. A historical record is maintained of the KPI values for each period (a period is typically one day). In addition, the event data is used to immediately update profile measures which contain information about the current state of the event source. The values in the event, KPI's, and profile tables can be used as input to a predictive statistical model and decision management rules that can result in a recommended action. Orchestration Identifier The orchestration must contain the same orchestration identifier. As in the example above, to identify the correct orchestration to perform, the orchestration should have the identifier 'com.ibm.asf.t1' defined as the orchestration_cd. <orchestration> <orchestration_cd>com.ibm.asf.t1</orchestration_cd>.... </orchestration> 4

5 Defining an Orchestration A typical event processing application built using the Analytics Solutions Foundation involves storing the event, updating profiles, performing scoring and decision management, and acting on recommendations. As mentioned previously, an orchestration is comprised of a number of steps, one step for each of these tasks. Each step should define one adapter 'adapter_class' to perform the required task. An adapter may or may not require configurations. If an adapter requires any sort of configurations, such configurations should be provided under 'adapter_configuration'. 1. Event Store Step Typically, the first step in an orchestration is defined to store the event by identifying the fully qualified class name of the event store adapter. <step> <adapter_class>com.ibm.analytics.foundation.adapter.eventstore.eventstoreadapter </adapter_class> </step> The event store adapter is added to an orchestration to insert the event being processed into a database. The adapter stores the event along with any observations it contains. A database connection must be present in the orchestration context (accessed via the key OrchestrationEngineConstants.ORCHESTRATION_DATABASE_CONNECTION_KEY). Notice that this adapter does not require any configuration. The adapter will automatically store all the events it receives. 2. Profile Update Step The second step in the orchestration updates profiles. This step uses the profile adapter and identifies it using its fully qualified class name. <step> <adapter_class>com.ibm.analytics.foundation.adapter.profile.profileadapter </adapter_class> </step> The profile adapter updates profile tables by using the event data. A database connection must be present in the orchestration context. The database connection is accessed by using the OrchestrationEngineConstants.ORCHESTRATION_DATABASE_CONNECTION_KEY key. The profile adapter requires a configuration, and this configuration must be identified with the correct type. <step> <adapter_configuration xsi:type="q1:profile_adapter_configuration"> </adapter_configuration> </step> For configuring the profile adapter, see the section Profile Adapter Configuration. 3. Scoring BPM Service Adapter Steps 5

6 Further steps in the orchestration usually involve passing the selected updated profiles and events onto a scoring service which performs predictive modeling and analytics, and provides a score that can also be saved as an event in the database. The BPM adapter is a specialized adapter to start IBM Business Process Management processes as part of the orchestration. For further details on each of the above steps, see the detailed sections on each of them. In addition to defining the steps needed to perform different tasks in an orchestration, a solution may also provide other general purpose configurations, such as pool configurations 'pool_configuration'. A pool is required for each shared data object used by the profile and service adapters. For instance, the profile adapter maintains a pool of ProfileUpdateData shared objects that may be used by multiple threads attempting to process new events. The ProfileUpdateData object maintains the list of event and observation row objects to be reused. The configurations of the pool include (1) the maximum number of data objects that can be allocated by the pool at a given time, (2) the minimum number of data objects that can remain idle in the pool without having to create new instances, and (3) the maximum number of data objects that can be idle in the pool without releasing new instances. If not defined, the default values for these parameters are 8, 0, and 8 respectively. <orchestration> <pool_configuration max_pool_size="10" min_pool_size="1" max_idle_size="5" /> </orchestration> For more information about pooling event and observation rows, see the section on Pooling. 6

7 Master Data This section describes the support for master data included with the Analytics Solutions Foundation. This support includes the ability to define master data tables, and to load data into the tables. Table Definition Master data tables are defined in XML and loaded by the master data manager. The manager will then provide SQL scripts to create the tables. This includes user defined tables and standard support tables such as language, tenant, and calendar tables. The scripts also initialize the tables with any required default data, such populating the calendar table. Master Data Definition XML Master Data Manager Table Creation, Configuration, and Initialization Scripts Database SQL The Master Data table definitions in XML allow the Analytics Solutions Foundation to be aware of the content of the tables and to generate the scripts required to add the tables to the database. These scripts follow certain patterns to provide the functionality to include support for multiple languages and multiple tenants (organizations). 7

8 Tables are defined in XML according to a master data definition schema. Here is an example definition <table table_cd="employee" has_last_updated_timestamp="true" > <column column_cd="employee_code" is_key="true" type="string" size="10" /> <column column_cd="employee_name" type="string" size="10" /> <reference reference_cd="department_id" table_reference="department" is_nullable="false" /> <surrogate_key_column column_cd="employee_identifier" type="int"/> </table> 8

9 Business Key vs. Surrogate Key The concept of a key is important when dealing with tables. A key is used to identify a single row in the table. Usually, there is a column that holds an identifier value that can be used to identify a single row. However, in some cases, it may be a combination of identifiers in different columns that is required to identify a single row. In some cases, the value of a reference to a row in another table may form part of the key of the table to which the reference belongs. Normally, a user does not need to be concerned about surrogate keys. They are not used in any of the Analytics Solutions Foundation APIs. However, when browsing database tables, the surrogate key used for references will be seen rather than the business keys that were used to establish the reference. When the indicator is set for the use of a surrogate key, Analytics Solutions Foundation APIs assume that it is stored in a column named <Table_Cd>_ID. If the Analytics Solutions Foundation APIs are being used on top of an existing database, and the name of the surrogate key column does not match the above convention, a custom surrogate key column may be defined in the table definition. 9

10 Components of the Master Data Definition Type Attribute or Element Description table table_cd A unique code that identifies the table and will be used as the table name in the database. Required. is_surrogate_primary_key An indicator to automatically create a column to hold an integer surrogate primary key for the table. The name of the column will be the table_cd with "_ID" appended. This column is populated automatically. Optional. Default is false. is_multilanguage An indicator for adding a column to identify the language for a row. The name of the column is LANGUAGE_ID. Optional. Default is false. is_multitenant An indicator for adding a column to identify the tenant for a row. The name of the column is TENANT_ID. Optional. Default is false. is_row_deactivateable An indicator for adding an IS_ACTIVE column to identify if a row is still active. Optional. Default is false. is_missing_member_insert An indicator to automatically create a new row in the table when looking up a reference. If the row in this table being referenced does not exist, it will be created. Columns in the table that are not nullable must have a default value. max_surrogate_key_cache_size The maximum number of entries in the surrogate key cache for this table. validator_class The fully qualified class name of a validator for rows in this table. has_last_updated_timestamp An indicator for tracking when a record in this table was last updated. When set to true, a column called LAST_UPDATED_TIMESTAMP is added. The column will be updated on successful upserts even if the field values do not actually change. Default is false. column_cd A unique code that identifies the column and will be used as the column name in the database. Required. type The data type of the column. Can be int, bigint, decimal, double, string, boolean, date, time, or timestamp. Required. column 10

11 reference self reference surrogate_key_column size A size may be provided for string (the number of characters - ex. 100) and decimal (the number of digits before and after the decimal - ex. 9,2). Optional. Default for string size is 50 and decimal is 9,2. is_key An indicator that this column is part of the business key of a row. Key columns may not have null values. Optional. Default is false. is_nullable An indicator that a non-key column may contain null values. Optional. Default is false. default_value A default value that is assigned to the column. reference_cd A unique code that identifies the reference and will be used as the name of the column in the database used to hold the foreign key reference. table_reference The table_cd of the table being referenced. The table being referenced must have a surrogate primary key. is_key An indicator that this reference is a part of the business key of a row. Key references may not have null values. Optional. Default is false. is_nullable An indicator that the column used for a non-key reference may contain null values. Optional. Default is false. column_mapping A column mapping is used when more than one column has the same identifier. It maps a custom column identifier for the reference to a column in the referenced table. reference_cd A unique code that identifies the reference and will be used as the name of the column in the database used to hold the foreign key reference. number_of_levels The number of levels in the hierarchy table created for the self reference. column_mapping This element is required to provide column names that differ from the key columns of the table. It maps a custom column identifier for the self reference to a column in the table. duplicate_column_cd A column in the table to duplicate into the hierarchy table. The column is duplicated at every level in the hierarchy. column_cd A unique code that identifies the column and will be used as the surrogate key column name in the database. Required. type The data type of the surrogate key column. Can 11

12 be int or bigint. Required. References References allow a row in one master data table to refer to a row in another table. References may only be made to tables that have a surrogate primary key. 12

13 Support Tables Tables with the following table_cd values will be defined automatically by the Analytics Solutions Foundation, and do not need to be defined by the client. LANGUAGE TENANT CALENDAR EVENT_TIME KEY_LOOKUP Using the Row API - Obtaining a row - Setting and retrieving values - Automatic type conversions - Content providers - Business keys for references Row API The Row is a Java API for loading and retrieving Master data. It is also used to load Event data and retrieve Profile data. A Row instance represents a record in the database table. The primary purpose of Row based APIs is to allow clients to use business keys and hide the Surrogate key. Rows support fields with the following data types: int, decimal, double, string, boolean, date, time, and timestamp. Rows should be reused to avoid the cost of repeatedly creating them before each use. Obtaining a row A Java API is provided for loading and modifying master data. A master data manager provides the ability to obtain a master data row object for a given table. The table is identified by the table_cd value supplied in its definition. The row values to be loaded are then assigned to this row object. The row object is then passed to the upsert (insert/update) API of the master data manager. For example, a row for the table MASTER_PRODUCT can be obtained as: 13

14 For example, a row for the table MASTER_PRODUCT can be obtained as: MasterDataManager masterdatamanager = MasterDataManagerFactory.getMasterDataManager("path_to_solution_def_xml"); MasterDataRow productrow = masterdatamanager.preparerow("master_product"); Setting and retrieving values The method preparerow will always return a new object, but users should reuse a row many times by changing the values and resubmitting to the upsert method. The methods used to assign values to fields in the row are type-specific. The MasterDatatRow interface contains various type-specific setter methods that can be used to set field values. setstring(string fieldname, String value) setdate(string fieldname, Date value) setdecimal(string fieldname, BigDecimal value) setint(string fieldname, int value) settime(string fieldname, Time value) settimestamp(string fieldname, Timestamp value) setboolean(string fieldname, boolean value) setbigint(string fieldname, long value) setnull(string fieldname) The following are the valid fieldnames for a row: 1. The column_cd for all the column elements included in the table definition are valid field names. 2. For each reference in the table definition, the following convention is used to determine the fieldnames. The column_cd for each column included in the business key of the referenced table is used for a field name. If this would cause more than one field with the same name, then a column mapping must be used. A column mapping maps a custom column identifier for the reference to a column in the referenced table. This is only supported to one level (the referenced table cannot have a reference as part of its key). References allow a row in the table to refer to a row in another table. References may only be made to tables that have a surrogate primary key. 3. If is_multilanguage is true, then LANGUAGE_CD is a valid field name. 4. If is_multitenant is true, then TENANT_CD is a valid field name. 5. If is_row_deactivateable is true, then IS_ACTIVE is a valid field name. As an example, below is a code snippet that sets field values and upserts a MASTER_PRODUCT row. MasterDataRow productrow = masterdatamanager.preparerow("master_product"); 14

15 productrow.setstring("product_cd", "101"); productrow.setstring("product_name", "GEAR"); productrow.setdate("inception_date", date; productrow.setstring("tenant_cd", "IBM"); productrow.setstring("language_cd", "EN"); masterdatamanager.upsertrow(productrow); The values set on the row can be cleared by invoking the clearallvalues() method on the row instance. For example: productrow.clearallvalues(); The same row can then be used to set a different set of field values to be upserted or retrieved. The field values on the rows can retrieved via the various type specific getter methods on the row. String getstring(string fieldname) int getint(string fieldname) Date getdate(string fieldname) boolean getboolean(string fieldname) BigDecimal getdecimal(string fieldname) long getbigint(string fieldname) double getdouble(string fieldname) java.sql.time gettime(string fieldname) java.sql.timestamp gettimestamp(string fieldname) boolean isnull(string fieldname) For example, String productcd = productrow.getstring("product_cd"); In addition to these type specific getter methods, the Row API also includes a method getvalueasstring()that returns the value as a String. Content providers Rather than setting the values in a row object directly, a content provider can be used to update the row. This technique can be used to effectively insert multiple rows with a single call to insertrow / upsertrow. If the row has a content provider, then next() will be called to populate the row so it can be upserted. This is repeated until next returns false. public interface RowContentProvider<R extends Row> { boolean next(r row) throws RowContentProviderException; void releaseresources() throws RowContentProviderException; } 15

16 The next() method updates the content of the row to the next in a series of content. To insert multiple rows, the user can implement a RowContentProvider that populates a row upon calling next(). For more information, refer to the Java documentation. An example of a RowContentProvider can be a CSVContentProvider that reads contents of a row from a line of the CSV and populated the Row object. Calling next() on the Row will populate the contents of the Row with those from the next line on the CSV file. The RowContentProvider interface also has a releaseresources() method that can be overridden by the client to release resources at the end of the batch operation. Business Key vs. Surrogate Key (and use of business keys for references) The concept of a key is important when dealing with tables. A key is used to identify a single row in the table. Usually, there is a column that holds an identifier value that can be used to identify a single row. However, in some cases, it may be a combination of identifiers in different columns that is required to identify a single row. In some cases, the value of a reference to a row in another table may form part of the key of the table to which the reference belongs. Normally, a user does not need to be concerned about surrogate keys. They are not used in any of the Analytics Solutions Foundation APIs. However when browsing database tables, the surrogate key used for references will be seen rather than the business keys that were used to establish the reference. Similarly for each reference in the table definition, the column_cd for each column included in the business key of the referenced table is present as a field on the Row. If this causes more than one field with the same name, then a column mapping must be used. A column mapping maps a custom column identifier for the reference to a column in the referenced table. This is only supported to one level (the referenced table cannot have a reference as part of its key). Consider the below code snippet where a table MASTER_PRODUCTION_BATCH refers to a PRODUCT_ID of MASTER_PRODUCT table <table table_cd="master_production_batch" > <column column_cd="production_batch_cd" is_key="true" type="string"/> <column column_cd="production_batch_name" type="string/> <reference reference_cd="product_id" table_reference="master_product" /> </table> A Row prepared for the MASTER_PRODUCTION_BATCH table would contain a String field PRODUCT_CD, which is the only key field on MASTER_PRODUCT. Master Data Loading A Java API is provided for loading and modifying master data. A master data manager provides the ability to obtain a master data row object for a given table. The table is identified by the table_cd value supplied in its definition. The row values to be loaded are then assigned to this row object. The row object is then passed to the upsert API of the master data manager. 16

17 Master Data Manager preparerow(tablecd) Master Data Row upsert(row) The method preparerow will always return a new object, but users should reuse a row by changing the values and resubmitting to the upsert method. If an error occurs during the upsert, an exception will be thrown. It is the user's responsibility to ensure that all the required values as specified in the table definition have been assigned to the row. The methods used to assign values to fields in the row are type-specific. The following API is available on the MasterDataRow to assign type-specific values to fields in the row. The following are the valid field names for a row: 1. The column_cd for all the column elements included in the table definition are valid field names. 2. For each reference in the table definition, the following convention is used to determine the fieldnames. The column_cd for each column included in the business key of the referenced table is used for a field name. If this would cause there to be more than one field with the same name, 17

18 then a column mapping must be used. A column mapping maps a custom column identifier for the reference to a column in the referenced table. This is only supported to one level (the referenced table cannot have a reference as part of its key). If is_multilanguage is true, then LANGUAGE_CD is a valid field name. If is_multitenant is true, then TENANT_CD is a valid field name. If is_row_deactivateable is true, then IS_ACTIVE is a valid field name. If has_last_updated_timestamp is true, then LAST_UPDATED_TIMESTAMP is a valid field name for reading the value. The settimestamp() method for this field will throw an exception. Insert vs Upsert The Master data loading API supports two operations: insertrow and upsertrow. insertrow(): Inserts the supplied row into the database. This method does not check if the row being inserted already exists in the database. If it already exists in the database, the JDBC exception resulting from the attempt to insert will be wrapped in an InsertException and thrown up to the caller. upsertrow(): Persists the supplied row to the database. If the key for the entry does not exist in the database, the row will be inserted. If the key for the entry exists in the database, the row will be updated. If an error occurs during the upsert/insert, an exception will be thrown. It is the user's responsibility to ensure that all the required values as specified in the table definition have been assigned to the row. Supplying Multiple Rows Rather than setting the values in a row object directly, a content provider can be used to update the row. This technique can be used to effectively insert multiple rows with a single call to insertrow / upsertrow. Once a content provider has been set on the row, the user may invoke next(), which populates the next row, and then insert/upsert it. This has to be repeated until next() returns false. The below code snippet shows an example of inserting multiple master data rows using a content provider. // CSVReader is a RowContentProvider that provides rows from a CSV file CSVReader csvreader = new ("MASTER_PRODUCT.csv"); MasterDataRow mdr = mdmanager.preparerow("master_product"); //Upsert as long as the content provider supplies a row. while (csvreader.next()) { mdr = buildmasterdatarow(csvreader); mdmanager.upsertrow(mdr, dbconnect.getconnection()); } 18

19 Retrieving Rows The retrieve API of MasterDataManager (and ProfileManager/EventManager) supports the retrieval of the current values for a Master data row (or Profile/Event row correspondingly). Normally, all of the key values are specified for the row before passing it to the retrieve API. If all of the key values for a row are specified and a matching set of values are found, then those values will be set into the row and the retrieve method will return true. The field values in the row may be obtained by calling one of the type-specific get methods. If there are currently no values that match the key values, then the row will not be updated and the retrieve method will return false. Retrieving Multiple Rows If not all of the key values for a row are specified, then a call to the retrieve API may produce more than one result. Note that all of the key values for a given reference must be supplied together. For example, if a reference requires three key values, and only one of those is specified (not null), an exception will be thrown. If there is more than one result, the row is populated with the first set of values, including missing key values. A call to next() on the row will populate it with the next set of values. After the last result has been reached, a call to next will return false and the row will not be updated. Consider the following example that retrieves a MASTER_PRODUCT row with a given PRODUCT_CD. MasterDataRow productrow = masterdatamanager.preparerow("master_product"); productrow.setstring("product_name ", "Gear"); MasterDataRow retrievedrows = masterdatamanager.retrieve(productrow); if(retrievedrows!=null){ //Row contains missing keys as well productcd = retrievedrows.getstring("product_cd"); //There could be more than 1 result since not all keys were specified while(retrievedrows.next(){ //Upon calling next() the row is populated with the next record productcd = retrievedrows.getstring("product_cd"); } } It returns 0 or more MASTER_PRODUCT rows that match the applied field filters. The field values on the rows can be retrieved via the various type-specific getter methods on the row. String getstring(string fieldname) int getint(string fieldname) Date getdate(string fieldname) boolean getboolean(string fieldname) BigDecimal getdecimal(string fieldname) 19

20 double getdouble(string fieldname) java.sql.time gettime(string fieldname) java.sql.timestamp gettimestamp(string fieldname) boolean isnull(string fieldname) In addition to these type-specific getter methods, the MasterDataRow API also includes a method getvalueasstring()that returns the value as a String. Releasing resources Upon retrieving Master data rows, the caller is required to release the row resources. The caller invokes the releaseresources() method on the MasterDataRow. This method is invoked by the caller to indicate that no further rows in the series of rows are going to be retrieved by the solution. This can be effectively used by the implementation to release any database and JDBC resources that this Row or its associated RowContentProvider holds. It is generally good practice to release resources as soon as you are finished with them to avoid tying up database resources. Calling the releaseresources() method on a Row object that is already released has no effect. Row Reuse The values set on the row can be cleared by invoking the clearallvalues() method on the row instance. For example: productrow.clearallvalues(); The same row can then be used to set a different set of field values to be upserted or retrieved. Validation A Java API for validating master data is also provided. A Master Data Validator provides the capability to do cross field validation as well as any other custom validation that needs to be performed on a row before the upsert operation is executed. Basic validations, such as the integrity of field types, are checked by the Row API. A Validator can be supplied to validate the values in the row upserted to a master data table. The following snippet shows the Validator interface in Analytics Solutions Foundation. The validator is specified using the validator_class attribute of the master table definition in the solution definition file. This validator class must implement the com.ibm.analytics.foundation.validation.validator interface. The custom validator class files need to be present in the application class path. 20

21 <table table_cd="master_resource" is_surrogate_primary_key="true" is_multilanguage="true" is_multitenant="true" is_row_deactivateable="true" validator_class="com.ibm.analytics.foundation.samples.masterdata.validation.masterresourcevalidator"> It is expected that a validator will completely validate a row and will return a set of validation violations. This is recommended to allow for the full set of violations to be corrected before resubmitting a row. Here is an example of a custom validation violation. The constraints to be validated are defined by the implementation. For example, a validation can be designed to check that the value for a specific column, RESOURCE_NAME in this example, starts with a valid prefix from a predefined list of prefixes. 21

22 Another use case for this validator is cross field validation, where one field s value determines the value or constraint on another column. 22

23 Profiles The support for aggregation includes the ability to define Profile and KPI tables and to aggregate data into those tables using a configured profile adapter. KPI tables are considered to be a kind of Profile table that includes an interval column. A profile adapter configuration identifies a row in the Profile or KPI table to update and the calculations to perform to implement the update. For each of the calculations, the inputs and outputs of the calculations are specified. Using a mapping in the configuration, the input to a calculation can come from the event being processed, from a Profile or KPI row, from a constant value, or from the orchestration context. The output can be written to a Profile or KPI row again as defined by a mapping in the configuration. The behavior of the profile adapter is implemented as a series of steps within the execute method. First, the rows to update are obtained, then the inputs for the calculations are obtained, and the calculations performed. Table Definition Profile tables are defined in XML. This allows the Analytics Solutions Foundation to be aware of the contents of the tables and to generate the scripts required to add and update the tables in the database. These scripts follow certain patterns to provide common profile data functionality to include support for multiple tenants (organizations). Tables are defined in XML according to a profile definition schema. 23

24 Example Profile Definition <profile_definition> <table table_cd="audio_profile"> <column column_cd="profile_date" is_nullable="false" type="timestamp" /> <column column_cd="last_profile_date" type="timestamp" /> <column column_cd="period_average" type="double" /> <column column_cd="period_min" type="double" /> <column column_cd="period_max" type="double" /> <reference reference_cd="audio_id" table_reference="audio" is_nullable="false" is_key="true" /> <interval_column column_cd="kpi_date" type="date" /> </table> </profile_definition> Components of the Profile Definition Type Attribute or Element Description table table_cd A unique code that identifies the table and will be used as the table name in the database. Required. has_last_updated_timestamp An indicator for tracking when a record in this table was last updated. When set to true, a column called LAST_UPDATED_TIMESTAMP is added. The column will be updated on successful upserts, even if the field values do not actually change. Default is false. column_cd A unique code that identifies the column, which will be used as the column name in the database. Required. type The data type of the column. Can be int, bigint, decimal, double, string, boolean, date, time, or timestamp. Required. size A size may be provided for string (the number of characters - ex. 100) and decimal (the number of digits before and after the decimal - ex. 9,2). Optional. Default for string size is 50 and decimal is 9,2. is_key An indicator that this column is part of the business key of a row. Key columns may not have null values. Optional. Default is false. reference_cd A unique code that identifies the reference and will be used as the name of the column in the database used to hold the foreign key reference. table_reference The table_cd of the table being referenced. The table being referenced must have a surrogate primary key. is_key An indicator that this reference is a part of the business key of a row. Key references may not column reference 24

25 have null values. Optional. Default is false. interval column column_mapping A column mapping is used when more than one column has the same identifier. It maps a custom column identifier for the reference to a column in the referenced table. column_cd A unique code that identifies the column, which will be used as the column name in the database. Required. type The data type of the column. Can be date, time, or timestamp. Required. Profile Data Content: - Obtaining a row object - Setting key values on the row - Setting non-key values on the row - Upserting the row - Populating a row for retrieval - Retrieval - Obtaining the next row - Releasing resources - Setting measure values on the row Profile data is updated using a Java API. A ProfileRow object for a particular table is obtained from the ProfileManager using a set of key values. This object is updated by calculations and submitted back to the ProfileManager using an upsert API. ProfileManager API The ProfileManager interface supports interactions for populating a profile row and persisting it to the database. A summary of the available methods can be found below: Return Type Method and Description ProfileRow public ProfileRow preparerow(string tablecd); Returns a new instance of a ProfileRow object for the given table. boolean public boolean retrieverows(profilerow profilerow, Connection conn) throws RetrieveException; 25

26 Retrieves profile rows that match the filters specified. Returns true if at least one matching record was found, and false otherwise. boolean public boolean retrieverows(profilerow profilerow, int maxrows, Connection conn) throws RetrieveException; Retrieves profile rows that match the filters specified. Returns true if at least one matching record was found, and false otherwise. boolean public boolean retrieverowsforupdate(profilerow profilerow, Connection conn) throws RetrieveException; Retrieves profile rows that match the filters specified. This retrieval also locks the profile rows so that the data integrity is maintained throughout. Returns true if at least one matching record was found, and false otherwise. void public void upsertrow(profilerow row, Connection conn) throws UpsertException; Persists the supplied row to the database. If the key for the entry does not exist in the database, the row will be inserted. If the key for the entry exists in the database, the row will be updated. An UpsertException will be thrown if there is any error in this process. void public void incrementupsertprofilerow(profilerow row, Connection conn) throws UpsertException; Adds the values from the supplied row to the row in the database. If the key for the entry does not exist in the database, the row will be inserted. If the key for the entry exists in the database, the row will be updated in an incremental manner which means that the row needs not be retrieved for update, rather the fields will be updated at the database level. An UpsertException will be thrown if there is any error in this process. This method is thread-safe. When a row does not exist in the database and two threads are trying to insert the same row, only one inserts the row while the other updates it. Obtaining a row object A ProfileRow object for a particular table is obtained using the ProfileManager API and a set of key values. The general pattern is that a request is made for a row object through a call to preparerow(tablecd). This method always returns a new object that has been configured for the given table. The caller can reuse this row object for efficiency, particularly when performing multiple retrievals from or inserts to the table. ProfileRow retrieverow = profilemanager.preparerow(table_name + "_PROFILE"); Setting key values on the row 26

27 Setting key values on the row requires a row object. In order to set key values on the row, simply call a set method on the row object and pass the desired key values. For example: retrieverow.setstring(key, value); retrieverow.settimestamp(key, value); retrieverow.setdate(key, value); Upserting the row The upsertrow(row) method inserts the row, if it does not yet exist, and updates the row if it does exist. All of the key values must be set prior to calling the upsert method. In the following example, profilerow is upserted using a connection to the database. profilemanager.upsertrow(profilerow, (Connection) dbconnection); Note that in the above example, the database connection is cast as a Connection object. A common approach in Analytics Solutions Foundation is to configure the database connection as an object in the orchestrationcontext for later use. For more information, see the orchestrationcontext section under Defining an Orchestration. Populating a row for retrieval Prior to retrieving a row, first populate the row using the retrieverows method in the ProfileManager API as follows: CSVReader reader = new CSVReader("path_to_csv_file"); ProfileRowImpl profilerow(profilerowimpl)profilemanager.preparerow("resource_kpi"); String[] csv_row_data = reader.getdata(); profilerow.setstring(csv_row_data[0]); profilemanager.retrieverows(profilerow, dbconnection); Retrieval The retrieve API of ProfileManager supports the retrieval of the current values for a profile row. Normally, all of the key values are specified for the row before passing it to the retrieve API. If all of the key values for a row are specified and a matching set of values are found, then those values will be set into the row and the retrieve method will return true. The field values in the row may be obtained by calling one of the type-specific get methods. If there are currently no values that match the key values, then the row will not be updated and the retrieve method will return false. Two items to note: 1.) If any of the key values for a row are not specified, a call to the retrieve API could produce more than one result. If there is more than one result, the row is populated with the first set of values, including missing key values. 2.) All key values for a given reference table must be supplied together. For example, if a reference requires three key values, and only one is specified (not null), an exception will be thrown. 27

28 I. Retrieving a Single Row An example of retrieving a single row can be found below: ProfileRow retrieverow = profilemanager.preparerow(table_name + "PROFILE"); profilemanager.retrieverows(retrieverow, conn); RetrieveProfileCommand retrieveprofilecommand = new RetrieveProfileCommand(); retrieveprofilecommand.execute(retrieverow, profilemanager, map, false); II. Retrieving Multiple Rows To retrieve multiple rows, the retrieverows method provides the option of passing an additional parameter: int maxrows, which will return the maximum number of rows to be retrieved. Passing -1 will retrieve all rows. Obtaining the next row If a call to the retrieve API produces more than one result, a call to retrieverow.next(); on the profile row will populate it with the next set of values. After the last result has been reached, a call to next will return false, and the row will not be updated. Releasing Resources To close the resources on a particular retrieval row, use releaseresources(). A call to this method will release the JDBC resources that this particular provider holds. Note that this method will only close the ResultSet object. It should not close the connection object, as the lifecycle of the connection is not owned by this provider. You should not use this method on an object that has already been released. ProfileRow retrieverow = (ProfileRowImpl) profilemanager.preparerow(table_name + "_PROFILE"); retrieverow.setstring(tenant_col, tenant_cd); Field resultsetrowcontentprovider = BaseRowImpl.class.getDeclaredField("rowContentProvider"); resultsetrowcontentprovider.setaccessible(true); ResultSetRowContentProvider rowcontentprovider = (ResultSetRowContentProvider)resultSetRowContentProvider.get(retrieveRow); Field resultsetfield = ResultSetRowContentProvider.class.getDeclaredField("resultSet"); resultsetfield.setaccessible(true); ResultSet resultset = (ResultSet) resultsetfield.get(rowcontentprovider); retrieverow.releaseresources(); Setting measure values on the row 28

29 Prior to upserting a profile row, a solution might set any measure values or calculations on that particular row. To set measure values on the profile row, simply use any of the set methods in the ProfileManager API. For example: profilerow = (ProfileRowImpl) profilemanager.preparerow(table_name + "_PROFILE"); profilerow.setstring(measure_name, measure_value); profilerow.setdouble(average, avg); profilerow.setdouble(minimum, min); profilerow.setdouble(maximum, max); profilerow.settimestamp(timestamp, new Timestamp(System.currentTimeMillis())); Events The support for events includes the ability to define Events tables with or without observations. An event store adapter is typically used to store events rather than calling the event insert API directly. Table Definition Events and Observation tables are defined in XML. This allows the Analytics Solutions Foundation to be aware of the content of the tables and to generate the scripts required to add the tables in the database. These scripts follow certain patterns to provide common event data functionality to include support for multiple tenants (organizations). Tables are defined in XML according to an event definition schema. 29

30 30

31 Example Event Definition The following is an example of an Event Definition <event_definition> <table table_cd="event"> <column column_cd="event_start_time" type="timestamp" /> <column column_cd="event_end_time" type="timestamp" is_nullable="true" /> <column column_cd="incoming_event_cd" type="string" size="200" is_nullable="true" /> <reference reference_cd="event_type_id" table_reference="master_event_type" /> <reference reference_cd="source_system_id" table_reference="master_source_system" /> <reference reference_cd="process_id" table_reference="master_process" /> <reference reference_cd="production_batch_id" table_reference="master_production_batch" /> <reference reference_cd="location_id" table_reference="master_location"/> <observation table_cd="event_observation"> <column column_cd="observation_timestamp" is_key="true" type="timestamp" /> <column column_cd="observation_text" type="string" size="800" is_nullable="true" /> <column column_cd="measurement" type="double" /> <reference reference_cd="measurement_type_id" is_key="true" table_reference="master_measurement_type" /> <reference reference_cd="value_type_id" is_key="true" table_reference="master_value_type" /> <reference reference_cd="event_code_id" is_key="true" table_refece="master_event_code" /> <reference reference_cd="material_id" table_reference="master_material" /> <event_interval_column column_cd="observation_date" type="date" /> <event_interval_column column_cd="observation_time" type="time" /> </observation> </table> </event_definition> 31

32 Components of the Event Definition Type Attribute or Element Description table table_cd A unique code that identifies the table and will be used as the table name in the database. Required. has_last_updated_timestamp An indicator for tracking when a record in this table was last updated. When set to true, a column called LAST_UPDATED_TIMESTAMP is added. The column will be updated on successful upserts, even if the field values do not actually change. Default is false. column_cd A unique code that identifies the column and will be used as the column name in the database. Required. type The data type of the column. Can be int, bigint, decimal, double, string, boolean, date, time, or timestamp. Required. size A size may be provided for string (the number of characters - ex. 100) and decimal (the number of digits before and after the decimal - ex. 9,2). Optional. Default for string size is 50 and decimal is 9,2. is_key An indicator that this column is part of the business key of a row. Key columns may not have null values. Optional. Default is false. reference_cd A unique code that identifies the reference and will be used as the name of the column in the database used to hold the foreign key reference. table_reference The table_cd of the table being referenced. The table being referenced must have a surrogate primary key. is_key An indicator that this reference is a part of the business key of a row. Key references may not have null values. Optional. Default is false. column_mapping A column mapping is used when more than one column has the same identifier. It maps a custom column identifier for the reference to a column in the referenced table. column_cd A unique code that identifies the column and will be used as the column name in the database. Required. type The data type of the column. Can be date or time. Required. column reference event interval column 32

33 observation table_cd A unique code that identifies the observation table and will be used as the table name in the database. Required. column as above reference as above event_interval_column as above Loading event and observation data (using the event manager) Content: - obtaining a row (event, observation) - setting values on a row - adding observations to an event - inserting the row - using a content provider Obtaining a row A Java API is provided for inserting event and observation data into the database. An event data manager provides the ability to obtain an event or observation row object for a given table. The table is identified by the table_cd value supplied in its definition. The row values to be loaded are then assigned to this row object. The row object is then passed to the insert API of the event manager. The event manager can insert an event row into the database, along with all the observations that have been associated with that event. As defined in the event definition, each event may have observation tables associated with them, or inline observations. 33

34 For example, a row for the EVENT table and one for the EVENT_OBSERVATION table can be obtained as: eventmanager = EventManagerFactory.getEventManager("path_to_solution_def_xml"); EventRow eventrow = EventManager.prepareEventRow("EVENT"); EventObservationRow observationrow = EventManager.prepareEventObservationRow("EVENT_OBSERVATION"); Setting values on a row The method preparerow will always return a new object, but users should reuse a row many times by changing the values and the observations and resubmitting to the insert method. The methods used to assign values to fields in the row are type-specific. The EventRow and EventObservationRow interfaces contain various type-specific setter methods that can be used to set field values. setstring(string fieldname, String value) setdate(string fieldname, Date value) setdecimal(string fieldname, BigDecimal value) setint(string fieldname, int value) settime(string fieldname, Time value) settimestamp(string fieldname, Timestamp value) setboolean(string fieldname, boolean value) setnull(string fieldname) The following are the valid fieldnames for a row: 34

35 1. The column_cd for all the column elements included in the table definition are valid field names. 2. For each reference in the table definition, the following convention is used to determine the fieldnames. The column_cd for each column included in the business key of the referenced table is used for a field name. If this results in more than one field with the same name, then a column mapping must be used. A column mapping maps a custom column identifier for the reference to a column in the referenced table. This is only supported to one level (the referenced table cannot have a reference as part of its key). References allow a row in table to refer to a row in another table. References may only be made to tables that have a surrogate primary key. Adding Observations to an Event Rather than inserting observations directly into the database, each observation is added to an event and inserted alongside the event. EventObservationRow observationrow = eventmanager.prepareeventobservationrow("event_observation_type_1"); eventrow.addobservation(observationrow); As an example, below is a code snippet that sets field values and inserts an EVENT row, along with its observations, and inserts to the database. EventRow eventrow = (EventRow) eventmanager.prepareeventrow("event"); Date start_time = new Date(); eventrow.settimestamp( EVENT_START_TIME, new Timestamp(start_time.getTime())); Date end_time = new Date(start_time.getTime() + (35000)); eventrow.settimestamp( EVENT_END_TIME, new Timestamp(end_time.getTime())); Date planned_end_time = new Date(start_time.getTime() + (60000)); eventrow.settimestamp( EVENT_PLANNED_END_TIME, new Timestamp(planned_end_time.getTime())); eventrow.setstring( INCOMING_EVENT_CD, ""); eventrow.setstring( EVENT_TYPE_CD, "ALARM"); eventrow.setstring( SOURCE_SYSTEM_CD, "PROD23"); eventrow.setstring( TENANT_CD, "IT3"); EventObservationRow observationrow = eventmanager.prepareeventobservationrow("event_observation_type_1"); Timestamp observationtimestamp = new Timestamp(System.currentTimeMillis()); observationrow.settimestamp( OBSERVATION_TIMESTAMP, observationtimestamp); 35

36 String observationtext = "dummy text"; observationrow.setstring( OBSERVATION_TEXT, observationtext); double measurement = 19.0; observationrow.setdouble( MEASUREMENT, measurement); String measurementtypecode = "QTY"; observationrow.setstring( MEASUREMENT_TYPE_CD, measurementtypecode); java.sql.date observationdate = new java.sql.date(system.currenttimemillis()); observationrow.setdate( OBSERVATION_DATE, observationdate); Time observationtime = new Time(System.currentTimeMillis()); observationrow.settime( OBSERVATION_TIME, observationtime); observationrow.setstring( TENANT_CD, "IT3"); eventrow.addobservation(observationrow); eventmanager.insertrow(eventrow, connection); insertrow(): Inserts the supplied row into the database along with all its observation rows. This method does not check if the row being inserted already exists in the database. If it already exists in the database, the JDBC exception resulting from the attempt to insert will be wrapped in an InsertException and thrown up to the caller. If an error occurs during the insert, an exception will be thrown. It is the user's responsibility to ensure that all the required values as specified in the table definition have been assigned to the row. Supplying Multiple Rows Rather than setting the values in a row object directly, a content provider can be used to set the values in the row. This technique can be used to effectively process multiple events with a single call to processevent. If the row has a content provider, then next() will be called to populate the row so it can be processed. This is repeated until next returns false. For more information about implementing the next method, see the subsection Content Providers in the topic Using the Row API. Row Reuse The values set on the row can be cleared by invoking the clearallvalues() method on the row instance. For example: eventrow.clearallvalues(); 36

37 The same row can then be used to set a different set of field values to be inserted. Event Data Validation The EventManager API supports validating the master data references on the Event. The validatereferences method on the EventManager can be invoked to preemptively validate the master data references prior to insertion of the event row. EventManager.validateReferences(EventRow eventrow, Connection connection) throws ValidationException; If the referenced master data supports missing member inserts, then calling this method may result in triggering an insert on the referenced table. The method throws a ValidationException upon encountering the first unresolvable reference. 37

38 Calculation Definition A calculation definition is used to define the input and output of a calculation. A calculation is used by the profile adapter to update values in a profile row. The definition of a calculation includes the calculation_cd attribute, which uniquely identifies the calculation. The is_increment attribute identifies if the calculation can be performed without knowing any current values. The result of an is_increment calculation is simply added to the current value when updating a profile. Count and Total are examples of calculations where is_increment should equal true. Setting is_increment to false when the calculation does not require current values will not cause incorrect behavior, but it may result in slower performance. Setting is_increment to true when the calculation does require current values will cause incorrect behavior, because the calculation will not have the values that it requires. The calculation_class attribute is used to provide the fully qualified name of the class, which implements the calculation. This class must implement the interface com.ibm.analytics.foundation.calculation.api.calculation The following is an example of a calculation definition <calculation calculation_cd="minimum" calculation_class="com.ibm.analytics.foundation.calculation.calculations.minimum" is_increment="false"> <input> <field_name>measure_value</field_name> <type>double</type> </input> <input> <field_name>current_min</field_name> <type>double</type> </input> <output> <field_name>updated_min</field_name> <type>double</type> </output> </calculation> A calculation must implement the calculate method. This method should return true if the result of the calculation should be used to update a profile row, and false otherwise. Returning false allows for faster performance by avoiding an update to a profile row that does not result in a change. 38

39 Service Definition A service definition is used to define the input and output of a service. A service definition is used by the service adapter to perform a call to the service. The definition of a service includes the service_cd attribute, which uniquely identifies the service. The following is an example of a service definition. <service_definition> <service service_cd="rpm" > <input> <field_name>rpm</field_name> <type>double</type> </input> <input> <field_name>rpm_above_limit</field_name> <type>int</type> </input> <input> <field_name>rpm_below_limit</field_name> <type>int</type> </input> <output> <field_name>score</field_name> <type>double</type> </output> </service> </service_definition> 39

40 Event Store Adapter The event store adapter is added to an orchestration to insert the event being processed into a database. The adapter stores the event along with any observations it contains. A database connection must be present in the orchestration context (accessed via the key OrchestrationEngineConstants.ORCHESTRATION_DATABASE_CONNECTION_KEY). Event Store Adapter Configuration No configuration is required for the event store adapter. It will automatically store all the events it receives. 40

41 Profile Adapter The profile adapter updates profile tables based on event data. A database connection must be present in the orchestration context (accessed via the key OrchestrationEngineConstants.ORCHESTRATION_DATABASE_CONNECTION_KEY). Profile Adapter Configuration The configuration for this adapter provides a list of profile updates to be executed when an event is received. A profile update specifies when it should be triggered, what profile rows should be updated, and the calculations that will perform the update. Profile Updates There are two types profile updates provided. event_profile_update is used when the event itself contains the data needed to update the profile. observation_profile_update is used when the event has observations that contain the data needed to update the profile. The update contains three components: 1) an event or observation selector that determines if the update applies to a given event or observation, 2) one or more profile row selectors that determine the set of profile rows involved in the update, and 3) calculation invocations that perform the work of updating. Observation Profile Update Event Selector An event_selector determines if the update applies to a particular event. The event_selector has a list of fields and values that can be used to select a particular event. For example, the below sample illustrates a selector that matches an event row that has EVENT_TYPE_CD value set to MEASURMENT. <event_selector> <event_field_value> <field_name>event_type_cd</field_name> <value>measurement</value> </event_field_value> </event_selector> Observation Selector An observation_selector determines if the update applies to a particular observation on an incoming event. The observation_selector specifies the table_cd of the observation. It may also specify a list of fields and values that can be used to select a particular observation or group of observations. For example, the following sample illustrates a selector that selects that observation with table_cd EVENT_OBSERVATION, and having the value of RPM for MEASUREMENT_TYPE_CD and the value ACTUAL for VALUE_TYPE_CD. <observation_selector table_cd="event_observation"> <observation_field_value> <field_name>measurement_type_cd</field_name> <value>rpm</value> </observation_field_value> <observation_field_value> <field_name>value_type_cd</field_name> <value>actual</value> </observation_field_value> </observation_selector> 41

42 Profile Row Selector A list of profile_row_selectors determines what profile rows will be updated. Each profile_row_selector specifies the table_cd or shared_selector_cd of the profile or shared_profile_row_selector, and a list of key_field_value to specify a single profile row. A shared_profile_row_selector can be defined in the solution definition file and referenced by an observation profile update. Each key_field_value identifies a profile_field_name and a value for that field. The value may be a literal value, or it may be determined by referring to a field in the event or the selected observation. Normally, all the selected profile rows are updated. If a profile row is selected only to provide input to a calculation, an alias attribute must be set to a unique value within the scope of the update. A profile_row_selector can be shared rather than defining the same selector many times for various updates. For example, the below profile_row_selector will select the profile rows in RESOURCE_KPI that have the value OPHR Delta for PROVILE_VARIABLE_CD. <profile_row_selector> <shared_selector_cd>resource_kpi</shared_selector_cd> <key_field_value> <profile_field_name>profile_variable_cd</profile_field_name> <value>ophr Delta</value> </key_field_value> </profile_row_selector> Read Only Profile Row Selector A list of read_only_profile_row_selectors determines what profile rows will be read. Each read_only_profile_row_selector specifies the table_cd (or shared_selector_cd) of the profile, and a list of key_field_value to specify a single profile row. Each key_field_value identifies a profile_field_name and a value for that field. The value may be a literal value, or it may be determined by referring to a field in the event or the selected observation. A read only profile row is selected only to provide input to a calculation. For example, this read only selector selects the rows in RESOURCE_PROFILE that have the value of OPHR Delta for PROFILE_VARIABLE_CD. <read_only_profile_row_selector alias="resource_profile_last_value"> <shared_selector_cd>resource_profile</shared_selector_cd> <key_field_value> <profile_field_name>profile_variable_cd</profile_field_name> <value>ophr Delta</value> </key_field_value> </read_only_profile_row_selector> Calculation Invocation A list of calculation_invocations is responsible for updating the values of a profile row. Each invocation specifies a calculation_cd to identify the calculation to perform. An input_field_value identifies the field_name of a calculation input and a value for that field. The value may be a literal value, or it may be determined by referring to a field in the event, the selected observation, or another observation. The value may also be obtained from a field in the profile row being updated, another selected profile row (by using the alias for that row), or from an orchestration context variable. Here are some examples of value references: <value_ref> <event_field_name>rpm</event_field_name> </value_ref> <value_ref> <context_variable_name>profile_variable_cd</context_variable_name> </value_ref> 42

43 <value_ref> <selected_observation_field_name>observation_timestamp</selected_observation_field_name> </value_ref> <value_ref> <observation_field> <table_cd>event_observation</table_cd> <key_field_value> <field_name>measurement_type_cd</field_name> <value>temp</value> </key_field_value> <field_name>value_type_cd</field_name> </observation_field> </value_ref> An update_field_value specifies what fields to update after a calculation is performed. Either a field in the profile row or an orchestration context variable may be updated. The value is specified to come from one of the output fields of the calculation. For example, the below update_field_value specifies the row in the profile table to be updated: FORECAST_VALUE. <update_field_value> <profile_field_name>forecast_value</profile_field_name> <value>1.99</value> </update_field_value> A list of calculation_invocations can be grouped together and shared for reuse. A shared_calculation_invocation_group can be defined in the solution definition file and referenced by an observation profile update, as in the example below, where the shared_calculation_invocation_group VALUE was defined in the solution XML. <calculation_invocation invocation_cd="value"> <input_field_value> <field_name>threshold</field_name> <value>200</value> </input_field_value> </calculation_invocation> Event Profile Update The configuration of an event_profile_update is similar to the configuration of an observation_profile_update. The event_profile_update is normally used when all of the data to update a profile row is part of the event itself rather than in an observation. <event_profile_update> <event_selector> <event_field_value> <field_name>rpm</field_name> <value>*</value> </event_field_value> </event_selector> <profile_update_action> <profile_row_selector> <shared_selector_cd>resource_kpi_inline</shared_selector_cd> <key_field_value> <profile_field_name>profile_variable_cd</profile_field_name> <value>rpm</value> </key_field_value> </profile_row_selector> <shared_calculation_invocation_group_cd>kpi.measurement_above_limit.actual.inline</shared_calculati on_invocation_group_cd> <calculation_invocation invocation_cd="kpi.measurement_above_limit.actual.value"> <input_field_value> 43

44 <field_name>measure_value</field_name> <value_ref> <event_field_name>rpm</event_field_name> </value_ref> </input_field_value> <input_field_value> <field_name>threshold</field_name> <value>100</value> </input_field_value> </calculation_invocation> </profile_update_action> </event_profile_update> Type Conversions Normally, values are obtained from fields of the same type. However the following automatic data type conversions are provided: int bigint decimal double string boolean date time timestamp > > > > > > > > > decimal, double, string, boolean decimal, double, string, boolean, int int, bigint, double, string int, bigint, decimal, string int, bigint, decimal, double, boolean, date, time, timestamp int, bigint, string string, timestamp string string, date, time Data Mapping During execution of the Profile Adapter, data is mapped in various settings. Profile Row Selection This setting provides the required input to the calculation. The event and any observations are available. If a specific observation was selected for the update, then that observation is available for input without requiring selection. Destination profile field Source literal event field selected observation field (only valid in an observation_profile_update) other observation field (requires selecting an observation) Input to Calculation This setting provides the required input to the calculation. The event and any observations are available. If a specific observation was selected for the update, then that observation is available for input without requiring selection. Destination calculation input field Source literal 44

45 event field selected observation field (only valid in an observation_profile_update) other observation field (requires selecting an observation) profile field other profile field (using an alias) context variable Update after Calculation In this setting the goal is to use the results of the calculation to update the profile row or a context variable. The output of the calculation is available. Destination profile field context variable Source literal calculation output field Input to Service In this setting, the required input for the service call is provided. The incoming event and the selected profile rows are available, along with any context variables. Destination service input field Source literal event field observation field (requires the observation be specified) profile field (requires the profile row alias be specified) context variable Update after Service In this setting, the goal is to deal with the results of the service call. A service event in the context can be used to hold the result. Results can also be recorded directly in the orchestration context. The output of the service call is available. The incoming event is also available, along with any observations it contains. Therefore, all of the master data references can be copied from the incoming event to the service event. A shared update group can be used to perform this copying. Destination event field observation field (requires the observation be specified) context variable Source literal service output field incoming event field incoming observation field (requires the observation be specified) 45

46 Service Invocation The service invocation is designed to work in a context that has the ability to perform the actual call to the service. The role of the adapter is to provide the required input for this call and to store the result. Taking IBM Integration Bus (IIB) as an example, the orchestration engine can be called from a Java Compute node to perform an orchestration that includes a call to a web service. However, IIB has the capability to actually perform the call to the web service with a dedicated node. The orchestration engine can be configured to take advantage of capabilities of IIB through the following mechanism. 1. The Orchestration node is a Java Compute node that performs the call to an Orchestration Engine s processevent() method. The process event method takes an Event and a Map of orchestration context values. Each of the values is associated with a key that identifies how it will be used by the orchestration (ex. a key of _DBConnection is used to supply a JDBC database connection). 2. The Orchestration includes an instance of ServiceInvocationHandler (typically an anonymous inner class) as one of the orchestration context values. The ServiceInvocationHandler implements the invokeservice method to actually perform the web service call. 3. In IIB, the ServiceInvocationHandler is going to create the message structure and explicitly propagate to the SOAP Request node. 4. When the SOAP request has returned, the Process Result node records the result in the IIB execution context. 5. Control now returns the ServiceInvocationHandler, which retrieves the results from the IIB execution context and returns them as the result of the invokeservice method. Using this technique allows the orchestration engine to take advantage of the capability of IIB to perform the web service call. Handlers and Scoring Events The following are attributes of the service_invocation element in an orchestration definition. service_cd The attribute service_cd is used to identify the service being invoked. Services are defined in the solution definition file. handler_context_variable_name This is the name used to access the service invocation handler in the orchestration context. Users are responsible for populating the orchestration context with this handler and passing it to the processevent API. The handler must implement the ServiceInvocationHandler interface. The following is an example of the declaration of a service invocation handler and a reference to it in a service invocation element. // Create and populate the context Map<String, Object> map = new HashMap<String, Object>(); 46

47 map.put(orchestrationengineconstants.orchestration_database_connection_key, dbconnect.getconnection()); map.put("rpm_score", new ServiceInvocationHandler() public void invokeservice(servicerow input, ServiceRow output, Map<String, String> configproperties) throws ServiceInvocationException { try { // Demo implementation - a real implementation would use the input and call scoring output.settimestamp("current_timestamp", new Timestamp(System.currentTimeMillis())); output.setdouble("score", 10.0); } catch (FieldAccessException e) { throw new ServiceInvocationException("Exception setting score value", e); } } }); engine.processevent(eventrow, map); <service_invocation service_cd="spss" handler_context_variable_name="rpm_score"> event_table_cd The result of a service invocation can be stored in an event by the service adapter. Specifying the event_table_cd attribute allows the service adapter to assign values to the event after scoring. event_context_variable_name This name accesses the event used to hold the result of scoring in the orchestration context. Users should populate the context using this name with an event row matching the event_table_cd. The following is an example of supplying an event and a reference to it in a service invocation element. // Create and populate the context Map<String, Object> map = new HashMap<String, Object>(); map.put(orchestrationengineconstants.orchestration_database_connection_key, dbconnect.getconnection()); EventRow scoreevent = eventmanager.prepareeventrow("event"); EventObservationRow observation = eventmanager.prepareeventobservationrow("event_observation"); scoreevent.addobservation(observation); observation.setstring("measurement_type_cd", "HS"); scoreevent.setorchestrationkeycd(score_event_orchestration_key_cd); map.put("score_event", scoreevent);... engine.processevent(eventrow, map); <service_invocation service_cd="spss" handler_context_variable_name="rpm_score" event_table_cd="event" event_context_variable_name="score_event"> event_handler_context_variable_name This is the name used to access the event handler in the orchestration context. The event handler is used to process the event that holds the result of a service invocation. Users are responsible for populating the orchestration context with this handler and passing it to the processevent API. The handler must implement the ServiceEventHandler interface. Normally, this handler is only used if the event being processed has a content provider. Using this handler allows the service event to be processed after each scoring invocation. The following is an example of the declaration of an event handler and a reference to it in a service invocation element. map.put("score_event_handler", new ServiceEventHandler() public void handleevent(eventrow scoreevent, Map<String, Object> orchestrationcontext) throws ServiceEventException { try { engine.processevent(scoreevent, orchestrationcontext); } catch (OrchestrationEngineException e) { throw new ServiceEventException("Exception processing score value", e); } 47

48 } }); <service_invocation service_cd="spss" handler_context_variable_name="rpm_score" event_table_cd="event" event_context_variable_name="score_event" event_handler_context_variable_name="score_event_handler" > enablement_handler_context_variable_name This is the name used to access the adapter enablement handler in the orchestration context. Users are responsible for populating the orchestration context with this handler and passing it to the processevent API. The handler must implement the AdapterEnablementHandler interface. The following is an example of the declaration of a service invocation handler and a reference to it in a service invocation element. // Create and populate the context Map<String, Object> map = new HashMap<String, Object>(); map.put("scoring_enablement_handler", new AdapterEnablementHandler() public boolean isenabledforevent(eventrow event, Map<String, Object> orchestrationcontext, AdapterConfiguration adapterconfiguration) throws AdapterException { String event_type_code = ""; try { event_type_code = event.getstring("event_type_cd"); } catch (FieldAccessException e) { e.printstacktrace(); } if(event_type_code.equals("measurement")){ return true; }else{ orchestrationcontext.put("offer", "No health tip!"); return false; } } } ); engine.processevent(eventrow, map); <service_invocation service_cd="scoring" enablement_handler_context_variable_name="scoring_enablemnent_handler"> Service Adapter The service adapter calls a service and optionally updates an event with the result of the service. Service Adapter Configuration The configuration for this adapter provides a list of service invocations. A service invocation specifies when it should be triggered, what inputs will be used for the service, and how to process the result of the event. service_adapter_listener_context_variable_name This is the name used to access the service adapter listener in the orchestration context. Users are responsible for populating the orchestration context with this listener. The handler must implement the ServiceAdapterListener interface. The preserviceinvocation is done after the input mapping, and the postserviceinvocation is done before the update mapping. The following is an example of how it would be named and referenced in the orchestration and the declaration of a service adapter listener. <adapter_configuration xsi:type="q1:scoring_adapter_configuration" service_adapter_listener_context_variable_name="service_adapter_listener"> 48

49 map.put("service_adapter_listener", new BaseServiceAdapterListener() public boolean preserviceinvocation(eventrow event, ServiceRow input, Map<String, Object> orchestrationcontext, AdapterConfiguration adapterconfiguration) { return true; public boolean postserviceinvocation(eventrow event, ServiceRow output, Map<String, Object> orchestrationcontext, AdapterConfiguration adapterconfiguration) { return false; } } ); Service Invocation Configuration The event selector and observation selectors are combined to determine if a particular event should trigger a service invocation action <event_selector> <event_field_value> <field_name>event_type_cd</field_name> <value>measurement</value> </event_field_value> </event_selector> <observation_selector table_cd="event_observation"> <observation_field_value> <field_name>measurement_type_cd</field_name> <value>rpm</value> </observation_field_value> <observation_field_value> <field_name>value_type_cd</field_name> <value>actual</value> </observation_field_value> </observation_selector> Service Invocation Action Service Profile Row Selector A list of service_profile_row_selectors determines what profile rows will be selected as inputs for the service. Each service_profile_row_selector specifies the table_cd or shared_selector_cd, and a list of key_field_value to specify a single profile row. Each key_field_value identifies a profile_field_name and a value for that field. The value may be a literal value, or it may be determined by referring to a field in the event or an observation. Because the profile rows are being selected from multiple profile tables and are not being used for updates, it is required that all the profile_row_selectors provide an alias. This alias is used when defining an input for the service. A profile_row_selector can be shared rather than defining the same selector many times for various updates. A shared_profile_row_selector can be defined in the solution definition file and referenced by a service_profile_row_selector. <profile_row_selector alias="rpm"> <shared_selector_cd>resource_kpi</shared_selector_cd> 49

50 <key_field_value> <profile_field_name>profile_variable_cd</profile_field_name> <value>rpm</value> </key_field_value> </profile_row_selector> Service Invocation A service invocation determines the input values for the service. The input_field_value are a list of input fields that can be either values or field references from a profile, observation, or event row, as in the example below. <input_field_value> <field_name>rpm_below_limit</field_name> <value_ref> <profile_field> <alias>rpm_below_limit</alias> <field_name>actual_value</field_name> </profile_field> </value_ref> </input_field_value> <input_field_value> <field_name>current_value</field_name> <value_ref> <observation_field> <table_cd>event_observation</table_cd> <key_field_value> <field_name>measurement_type_cd</field_name> <value>rpm</value> </key_field_value> <field_name>measurement</field_name> </observation_field> </value_ref> </input_field_value> The update_field_value are a list of fields from either the service event or the context_variable_name that will be updated by the service output. <update_field_value> <event_field_name>event_start_time</event_field_name> <value_ref> <service_output_field_name>current_timestamp</service_output_field_name> </value_ref> </update_field_value> <update_field_value> <observation_field> <table_cd>event_observation</table_cd> <key_field_value> <field_name>measurement_type_cd</field_name> <value>hs</value> </key_field_value> <field_name>measurement</field_name> </observation_field> <value_ref> <service_output_field_name>score</service_output_field_name> </value_ref> </update_field_value> 50

51 BPM Adapter The BPM adapter extends the service adapter, but is specialized to start IBM Business Process Management processes. The BPM adapter accepts inputs as the service adapter does. Although it is an extension of the Service Adapter, it does not use outputs because BPM processes are started and finished asynchronously. As a result, it does not use update fields. BPM Adapter Configuration The BPM Adapter Configuration is similar to the Service Adapter Configuration, but its type is different. It is a remote service adapter. In addition, it requires two extra parameters: your BPM server host name, and its port number. <adapter_configuration xsi:type="q1:remote_service_adapter_configuration"> <host_name> <port_number>9080</port_number> <remote_invocation_configuration> <event_selector> <event_field_value> <field_name>event_type_cd</field_name> <value>measurement</value> </event_field_value> </event_selector> Host name and port number can still be overridden at runtime by using the reserved words HOST_NAME and PORT_NUMBER, like pictured below in a sample orchestration definition and sample solution definition: <remote_invocation_action> <bpm_invocation service_cd="bpm"> <process_name>pump Inspection Test</process_name> <input_field_value> <field_name>host_name</field_name> <value_ref> <context_variable_name>bpm_host_name </context_variable_name> </value_ref> </input_field_value> <input_field_value> <field_name>port_number</field_name> <value_ref> <context_variable_name>bpm_port_number </context_variable_name> </value_ref> </input_field_value> <input_field_value> <field_name>process_name</field_name> <value_ref> <context_variable_name>bpm_process_name </context_variable_name> </value_ref> </input_field_value> <input_field_value> <field_name>resource_id</field_name> 51

52 <value_ref> <context_variable_name>resource_id</context_variable_name> </value_ref> </input_field_value> </bpm_invocation> </remote_invocation_action> </remote_invocation_configuration> In addition, as you can see below, a default process name can be defined for each bpm_invocation. This process name can also be overridden at runtime, like the host name and port number, by using the reserved keyword PROCESS_NAME. Below is a sample solution definition snippet: <service_definition> <service service_cd="bpm" service_name=""> <input> <field_name>host_name</field_name> <type>string</type> </input> <input> <field_name>port_number</field_name> <type>string</type> </input> <input> <field_name>process_name</field_name> <type>string</type> </input> <input> <field_name>resource_id</field_name> <type>string</type> </input> Scoring Adapter Configuration The Scoring Adapter Configuration is very similar to the BPM adapter. Here is an example: <adapter_configuration xsi:type="q1:remote_service_adapter_configuration"> <host_name> <port_number>9080</port_number> <remote_invocation_configuration> <event_selector> <event_field_value> <field_name>event_type_cd</field_name> <value>measurement</value> </event_field_value> </event_selector> The content of the invocations is also similar. For example, the host name and port number can be overridden like in the BPM adapter. However, the Scoring Adapter requires a score request id, and not a BPM process name, which can be overridden at runtime. In addition, an input table can be specified. Here is an example: <remote_invocation_action> <scoring_invocation service_cd="scoring"> 52

53 <input_field_value> <field_name>host_name</field_name> <value_ref> <context_variable_name>cds_host_name </context_variable_name> </value_ref> </input_field_value> <input_field_value> <field_name>port_number</field_name> <value_ref> <context_variable_name>cds_port_number </context_variable_name> </value_ref> </input_field_value> <input_field_value> <field_name>score_request_id</field_name> <value_ref> <context_variable_name>score_request_id </context_variable_name> </value_ref> </input_field_value> <input_field_value> <field_name>input_table_name</field_name> <value_ref> <context_variable_name>input_table_name </context_variable_name> </value_ref> </input_field_value> <input_field_value> <field_name>bp_sys</field_name> <value_ref> <context_variable_name>bp_sys</context_variable_name> </value_ref> </input_field_value> <input_field_value> <field_name> BP_DIAS</field_name> <value_ref> <context_variable_name> BP_DIAS</context_variable_name> </value_ref> </input_field_value> <update_field_value> <context_variable_name>campaign</context_variable_name> <value_ref> <service_output_field_name>campaign </service_output_field_name> </value_ref> </update_field_value> <update_field_value> <context_variable_name>offer</context_variable_name> <value_ref> <service_output_field_name>offer </service_output_field_name> </value_ref> </update_field_value> <score_request_id>health_tips</score_request_id> 53

54 <input_table> <input_table_name>customer</input_table_name> <request_input_row> <input> <name> BP_DIAS</name> <value>90</value> </input> <input> <name>bp_sys</name> <value>120</value> </input> </request_input_row> </input_table> </scoring_invocation> </remote_invocation_action> The values of the input table given can be overridden at runtime. The parameter must have the same name as the input specified. If there is more than one table, the parameter must follow the format <input_table_name>.<input name>. The input table name can be overridden only if there is only one input table. In that case, the parameter name uses the reserved keyword INPUT_TABLE_NAME. Below is a sample solution definition snippet: <service service_cd="scoring" service_name=""> <input> <field_name>host_name</field_name> <type>string</type> </input> <input> <field_name>port_number</field_name> <type>string</type> </input> <input> <field_name>score_request_id</field_name> <type>string</type> </input> <input> <field_name>input_table_name</field_name> <type>string</type> </input> <input> <field_name>bp_sys</field_name> <type>string</type> </input> <input> <field_name>campaign</field_name> <type>string</type> </input> <input> <field_name>offer</field_name> <type>string</type> </input> The response received after the score request is also parsed, and can be mapped from the outputs defined in the solution definition, as shown, like it would be done with a service adapter. 54

55 Event Validation Adapter This adapter can be used to validate the master data references on events. This adapter can be used when a user does not want to store the events, but only validate the event to verify if it contains only valid master data references. In the case where the EventStoreAdapter is configured, the references are validated prior to storing the event. The class com.ibm.analytics.foundation.adapter.validation.eventvalidationadapter represents the Event Validation Adapter. The EventValidationAdapter may trigger a referenced data insertion if missing member inserts is enabled on the references master table. Any validation error will result in an AdapterException being thrown during the orchestration. Processing events as a batch - Content provider - Handling a service event Analytics Solutions Foundation supports processing a group of events as a batch. This allows for more efficient database interactions. Events can be processed as a group by using an event content provider. The orchestration engine will automatically update the event using the content provider. It is the user's responsibility to call commit on the database connection after each batch is processed. Batches of events can be processed in parallel only when the batches do not update the same profile rows. The reason for this is that there is a high probability of deadlock. When a profile row is selected for updating, a database lock is obtained on the row. This would prevent another thread from selecting the same row. The lock is released only when the transaction is committed/ rolled-back; that is, once the entire batch of events is processed. If batches do not update the same profile rows - for example batches from different devices that use different materials, these batches can be processed in parallel. For details on how to supply a batch of events using a content provider, see the section Loading event and observation data (using the event manager). The code snippet below demonstrates batch event processing: //Assume the events.csv file contains multiple events CSVReader event_reader = new CSVReader("events.csv"); //Create an Event content provider 55

56 EventsListContentProvider eventrowcontentprovider = new EventsListContentProvider(event_reader, eventmanager); EventRow eventrow = eventmanager.prepareeventrow("event"); eventrow.setcontentprovider(eventrowcontentprovider); //Process the batch orchestrationengine.processevent(eventrow, context); In this example, the EventsListContentProvider is a content provider which allows events to be read from a CSV file. Handling a service event The result of a service invocation can be stored in an event by the service adapter. As described in the section Service Adapter, the score event is populated in the orchestration context against the key defined via the property event_context_variable_name within the service_invocation element. The score event can be processed in one of the below mechanisms: 1. Using the orchestration engine to process the event inline via an OrchestrationEngine.processEvent() call from the Service Event handler. In this technique, the scoring event is processed synchronously in the same thread. The processing of the original event resumes after the score event is processed. It is the responsibility of the user to use a different orchestration context to process the score event. Since the orchestration engine holds various orchestration related caches and other artifacts within the orchestration context, it cannot be used simultaneously by two different orchestrations. In this technique, the scoring event is processed in same transaction that the incoming event is processed. 2. Sending the event to a queue for processing in a different thread/flow. In this mechanism, the client may choose to send the scoring events to another queue/event repository, from where they can be processed by a different event processing flow. This approach is suitable when the client does not intend to block the processing of incoming event batch while processing the score events. With this approach unless the score events processing update the same profile rows, they can run in parallel with the incoming event processing batch. As described in earlier sections, a ServiceEventHandler instance handles the scoring event and processes it via one of the above mechanisms defined above. Example of inline/synchronous score event processing (technique 1): 56

57 The code snippet below defines a ServiceEventHandler that processes the scoring event inline within the same transaction as the incoming event. new BaseServiceEventHandler() public void handleevent(eventrow scoreevent, Map<String, Object> orchestrationcontext) throws ServiceEventException { try { engine.processevent(scoreevent, orchestrationcontext); // The incoming event processing blocks until the scoring //event is processed }catch (OrchestrationEngineException e) { //handle exception } }}; Example of asynchrounous score event processing (technique 2): List<EventRow> scoreeventlist = new ArrayList(); new BaseServiceEventHandler() public void handleevent(eventrow scoreevent, Map<String, Object> orchestrationcontext) throws ServiceEventException { try { scoreeventlist.add(scoreevent); //The incoming event processing resumes upon adding the //score event to a list or to another queue. }catch (OrchestrationEngineException e) { //handle exception } }}; The events in scoreeventlist are processed in a different transaction. Instead of adding to the scoreeventlist, the scoreevent could be sent to another queue for processing. Profile row caching 57

58 One of the merits that batch event processing provides is that the profile row caches are re-used across the batch of events. The profile row cache's lifetime is scoped to that of the orchestration. Hence, if a profile row is loaded once from the database while processing an event, it is always fetched from the cache should subsequent events need it. When profile row cache is enabled, all changes to profile rows during event processing are updated in the cache. At the end of the orchestration, the cache changes are written back to the database. This significantly reduces the number of database round-trips during batch event processing. Processing events from multiple threads - Locking - Limitations - Guidelines Multithreading Analytics Solutions Foundation supports multi-threaded event processing. The orchestration engine can be called to process events from different threads. This allows multiple events to be processed in parallel. It is the responsibility of the user to fork multiple threads and execute orchestrations on those threads. This is typically achieved by configuring multiple JMS message listener threads. Multithreaded event processing can happen most efficiently when the events are updating different profile rows. Deadlocks can occur if two separate threads are updating at least two similar profile rows, but in a different order. If each thread obtains the lock on a row that the other requires, then there will be a deadlock, as neither orchestration can complete. Therefore, an orchestration that is being performed in parallel should include only one step that uses a profile adapter (this is the typical case). Multithreaded event processing is most suited for real-time event processing use-cases, where events arrive one after another, and there is a need to process them as and when they arrive. To reduce the scope of profile row locks, a single event is processed within an orchestration (as opposed to a batch of events). They also guarantee optimal utilization of system resources by using them when another thread is waiting for a database response. This could increase the throughput. Limitations When events are processed one at a time, the scope of profile row caching is also significantly reduced. If a single event is processed in an orchestration, the cache's lifetime is also scoped to a single event. Therefore, performance merits gained due the profile row caching is largely lost. There is a trade-off between batch and real-time processing, and the user will have to choose the one that suits the given use case. See previous sections for guidelines on Batching and Multithreading. Improving Performance 58

59 There are two main approaches to increasing the performance of event processing. Events can be processed in multiple threads, and events can be processed as a batch. Processing events as a collections results in improved performance the most when the events in the collection are updating the same profile rows. Thus, if possible, it is best to sort the events so that similar events are processed together. For example, sorting them by device, time, and measurement. Events processed as a collection may only be processed by a single thread. The only exception to this is the case where the collections that are processed in separate threads do not update any of the same profile rows. Processing single events using multiple threads will improve performance when the events are generally updating different profile rows. If the events are all updating the same profile rows, then there is little advantage using multiple threads, since a thread will lock the profile rows it is updating, and the other threads will have to wait until the lock is released (when the transaction is committed). Calculations that are identified as is_increment will also generally improve performance because they are able to update a profile row in the database without first having to retrieve it and lock it. Pooling Foundation APIs provide the capability to reuse data objects such as event and observation rows by creating a pool for each data object. Pools are created during initialization of the orchestration engine, where each pool is uniquely identified by a pre-defined key. All pre-defined keys are declared in 'PoolConstants'. The 'EVENT_ROW_POOLED_RESOURCE_KEY' and 'OBSERVATION_ROW_POOLED_RESOURCE_KEY' keys are used to create the event row and observation row pools respectively. Prior to processing events, an event or observation row can be borrowed from the pool. The 'PoolManager' provides a borrowobject method for this purpose. For example, this code is used to borrow an event row from the event row pool: PoolManager.getInstance().borrowObject(PoolConstants.EVENT_ROW_POOLED_R ESOURCE_KEY, EVENT ); EVENT here is the key used to pull events from the event pool. When a solution is finished with the event or observation row, the row instance should be returned back to the pool. The 'returnobject' method in the 'PoolManager' can be used to do so. For example, the following code is used to return an observationrow back to the pool: 59

60 PoolManager.getInstance().returnObject(PoolConstants.OBSERVATION_ROW_PO OLED_RESOURCE_KEY, EVENT_OBSERVATION, observationrow); EVENT_OBSERVATION is the key that was used to pull observation rows from the observation row pool. Orchestration engine configuration options The keys for various orchestration engine configuration parameters are defined within the enum: com.ibm.analytics.foundation.engine. OrchestrationEngineConfigurationParameter.java An OrchestrationEngineConfigurationParameter can be set as: orchestrationengineprops.put(orchestrationengineconfigurationparameter.<prop_ Value>, <Prop_Value>); For example: orchestrationengineprops.put(orchestrationengineconfigurationparameter.orches TRATION_DIR_KEY, orchestrationxmllocation); Below is the list of keys that are orchestration engine configuration options. ORCHESTRATION_DIR_KEY: Defines the directory that contains the orchestration XMLs. Can be an absolute or a relative path. ORCHESTRATION_FILE_NAME_PREFIX_KEY: Prefix to use to filter the orchestration XMLs to load into the engine SOLUTION_DEFINITION_XML_KEY: Defines the directory that contains the orchestration XMLs. Can be an absolute or a relative path. ENABLE_XML_CONTENT_VALIDATION_KEY: Key used to enable/disable XML content validation of the solution definition and orchestration XMLs. Default: Enabled ENABLE_PROFILE_ROW_CACHING_KEY: This key is used to enable/disable the profile row cache. Analytics Solutions Foundation supports caching of profile rows. When this cache is enabled, the profile rows retrieved from the database during event processing are added to an in-memory cache. Subsequent requests for the same profile row are retrieved from this cache, thereby reducing the number of database roundtrips significantly. The cache's lifetime is that of the orchestration. A profile row retrieved during a profile adapter step is re-used during a Service invocation, if it is an input to the service. The benefits are greater during batch event processing, when the profile row cache is reused across the batch of events. Hence, if a profile row is loaded once from the database 60

61 while processing an event, it is always fetched from the cache should subsequent events need it. When the profile row cache and batch profile updates are enabled, all changes to profile rows during event processing are updated in the cache. At the end of the orchestration, the cache changes are written back to the database in a single query. Default: Enabled (Set to 'false' or a non-true value to disable it) e.g. orchestrationengineprops.put( OrchestrationEngineConfigurationParameter.ENABLE_PROFILE_ROW_CACHING_KE Y, "false"); ENABLE_PROFILE_UPSERT_BATCHING: This key is used to enable/disable batch profile updates. When batch profile updates is enabled, all profile rows modified in the orchestration are persisted back to the database in a single batch update query. This happens at the end of the orchestration, after all events are processed. When the profile row cache is disabled by the user, the user-specified batch upsert setting is ignored and the feature is disabled by the engine. Default: Enabled. Set it to 'false' or a non-true value to disable. ENABLE_PROFILE_SELECT_BATCHING_KEY: This key is used to enable/disable batch selection of profile rows required for a Service invocation. Typically, service invocation requires a set of profile rows as input. Some of them may not be present in the profile row cache. When batch selection is enabled, a single batch select query is fired to retrieve all such profile rows from the database. Default: Enabled. Set to 'false' to disable. ENABLE_PREPARED_STATEMENT_CACHING_KEY: To prevent frequent query parsing and prepared statement creation, Analytics Solutions Foundation supports caching of PreparedStatement instances. Default: Enabled. Set it to 'false' to disable. Below are a set of pool-related configuration options. For more details on these, see the section on Pooling. Profile Row pool configuration options: When profile row caching is enabled, the number of profile rows cached is determined by the profile row pool configuration. A pool of stateless profile row objects is created, and upon request for a profile row, an entry from the pool is selected and its state is populated. This mechanism enables having control over the number of active in-memory profile rows. If a stateless profile row object isn't available in the pool, and if a new instance cannot be created due to the maximum active rows in memory cap being reached, the thread has to wait until a profile row is 61

62 returned back to the pool at the end of the orchestration. At the end of the orchestration, the Orchestration engine clears the state of the loaded profile row objects and releases them back to the pool. PROFILE_ROW_POOL_MAX_ACTIVE: This key is used to configure the maximum number of profile rows that can be active in the orchestration engine at any given time. Default: No limit, Set to a non-negative value to enforce a limit. PROFILE_ROW_POOL_MIN_IDLE: This key is used to configure the target for minimum number of idle stateless-profile rows to be maintained in the profile row pool. Default: 5, set to a non-negative value to enforce a limit. PROFILE_ROW_POOL_MAX_IDLE: This key is used to set a cap on the number of "idle" instances in the profile row pool. This property enables the pool to shrink. If maxidle is set too low on heavily loaded systems, it is possible you will see profile row objects being destroyed and almost immediately new profile row objects being created. This is a result of the active threads momentarily returning objects faster than they are requesting them, causing the number of idle profile row objects to rise above maxidle. The best value for maxidle for heavily loaded systems will vary, but the default is a good starting point. Default: 50, set to a non-negative value to enforce a limit. Setting a negative value indicates an unlimited number of idle instances (that is, the pool never shrinks). Event Row and Event Observation Row pool configuration options: As described in section on Pooling, the Orchestration Engine creates pools of Event rows and Event observation rows. The user can use these pools to fetch event and observation row instances and populate them. This enables re-use of row objects, thereby resulting in performance improvements. These pools can be fine tuned further by using the following configuration keys: EVENT_ROW_POOL_MAX_ACTIVE: This key is used to configure the maximum number of event rows that can be active (per Event type) in the orchestration engine at any given time. Default: No limit, Set to a non-negative value to enforce a limit. EVENT_ROW_POOL_MIN_IDLE: This key is used to configure the target for minimum number of idle event rows (per Event type) to be maintained in the event row pool. Default: 3, set to a non-negative value to enforce a limit. EVENT_ROW_POOL_MAX_IDLE: This key is used to set a cap on the number of "idle" instances (per Event type) in the event row pool. This property enables the pool to shrink. If maxidle is set too low on heavily loaded systems, it is possible you will see event row objects being destroyed and almost immediately new event row objects being created. This is a result of the active threads momentarily returning objects faster than they are requesting them, causing the number of idle event row objects to rise above maxidle. Default: 10, set to a non-negative value to enforce a limit. Setting a negative value indicates unlimited number of idle instances (that is, the pool never shrinks). 62

63 OBSERVATION_ROW_POOL_MAX_ACTIVE: This key is used to configure the maximum number of event observations rows (per event observation type) that can be active in the orchestration engine at any given time. Default: No limit, set to a non-negative value to enforce a limit. OBSERVATION_ROW_POOL_MIN_IDLE: This key is used to configure the target for minimum number of idle event observation rows (per event observation type) to be maintained in the event row pool. Default: 10, set to a non-negative value to enforce a limit. OBSERVATION_ROW_POOL_MAX_IDLE: This key is used to set a cap on the number of "idle" instances (per event observation type) in the event observation row pool. This property enables the pool to shrink. If maxidle is set too low on heavily loaded systems, it is possible you will see event row objects being destroyed and almost immediately new event row objects being created. This is a result of the active threads momentarily returning objects faster than they are requesting them, causing the number of idle event observation row objects to rise above maxidle. Default: 100, set it to a non-negative value to enforce a limit. Setting a negative value indicates unlimited number of idle instances (that is, the pool never shrinks). 63

64 Support Alternate Databases Analytics Solutions Foundation is configured for IBM DB2, but can be configured to support alternate databases. This can be done by customizing the following artifacts for your specific database. sql.properties file sql.properties file contains vendor specific SQL that is used in Analytics Solutions Foundation. Use the db2.sql.properties as a template, and replace the DB2 SQL with SQL specific to your database. xml2ddl_transformer specification The xml2ddl_transformer XSL transform specification is used to generate the DDL script to create the Analytics Solutions Foundation database tables. The output DDL syntax includes CREATE TABLE ALTERNATE TABLE to add PRIMARY KEY and UNIQUE KEY constraints ALTERNATE TABLE to add FOREIGN KEY constraints and the concept of an identity column. Use the xml2ddl_transformer.xsl as a template, and modify the XSL specification to generate DDL specific to your database. In order to generate a DDL file from the solution XML, the XML2DDL.java class has been provided in the Analytics Solutions Foundation API. The XML2DDL class is passed the location of the solution definition XML and output DDL location when it is instantiated. The transform method will perform the XML transformation and produce a DDL file at the specified location. Below is an example of how to execute the DDL generation. XML2DDL ddltransformer = new XML2DDL(solutionDefinitionXML, output/solution_definition.ddl ); ddltransformer.transform(); narowinserts specification The narowinserts XSL transform specification generates a stored procedure to populate the Master Data tables with NA rows. The output creates a New_NA_LG stored procedure. Stored Procedures The populate_calendar_and_event_time.sql creates the Calendar_pop stored procedure to load the Calendar and Event_Time support tables. The stored procedure syntax will have to be modified for your database. Add custom XSL transform specifications to the classpath for Analytics Solutions Foundation to use. The sql.properties file could be added to the classpath, or use the dbpropfile system property. System Properties Set the following system properties for Analytics Solutions Foundation to locate the sql.properties file. dbvendor is a prefix that identifies the sql.properties file for your database. For example, set dbvendor system property to 'ora' for Analytics Solutions Foundation to use ora.sql.properties from the classpath. dbpropfile is an absolute path to the custom sql.properties file. It is an alternative to having a.sql.properties name prefixed with dbvendor system property in the classpath. 64

65 Generating NA Row Script A class XML2NARowSql is provided that will generate an NA row insert script from the solution definition XML. The purpose of the NA row script is to put a single row with -NA- values for the business keys of each master data table. This allows for references to that master data table to be unassigned. For example, there is a table, MASTER_RESOURCE, that has a location as one of its business keys. If there was one resource that was a mobile resource that did not have an assigned location, then it is still possible to enter that resource into the MASTER_RESOURCE table without violating the SQL key constraint of the MASTER_RESOURCE table by using the NA value for location. In order to generate the NA row insert script, pass the same XML file that was used to generate the DDL to the XML2NaRowSql. The constructor for XML2NARowSql takes a file path just like XML2DDL. The transform() method can be called with or without parameters. There are two parameters that can be passed into XML2NaRowSql. The string value that the user wants to be used as the NA value that is used for the business keys is passed into a parameter called NA_STRING. The value that the user wants to be the size is called NA_NAME_SIZE. xmltransformer = new XML2NaRowSql(solutionDefinitionXML, output/narowsql.sql ); Map<String, Object> params= new HashMap<>(); params.put("na_string", "'-NA-'"); params.put("na_name_size", "300"); xmltransformer.transform(params); is an example of using params in the case of the XML2NaRowSql. Once a.sql script has been generated, the stored procedure must be created/deployed on the database. To do this, run the SQL script generated in the previous step. Once created, you can call the procedure with a call like: CALL NEW_NA_LG('EN', 'English', 'IBM','INTERNATIONAL BUSINESS MACHINES','NOT AVAILABLE',1,1) 65

66 Orchestration Listener The OrchestrationListener interface provides an analytic solution with the facility to tap into the data updates caused by an orchestration during event processing. The OrchestrationListener interface currently provides a callback hook at the end of event processing; that is, after all the steps in an orchestration have completed executing. An implementation of the OrchestrationListener is passed into the engine via the OrchestrationContext on a processevent() call. The listener must be set into the OrchestrationContext using OrchestrationEngineConstants.ORCHESTRATION_LISTENER_KEY as the key. If an implementation of com.ibm.analytics.foundation.engine.orchestrationlistener is not passed, the processevent() call will abort with an OrchestrationEngineException. The OrchestrationFacade allows access to the by-products of the orchestration. Currently, if OrchestrationEngineConfigurationParameter.ENABLE_PROFILE_ROW_CACHING_KEY is set to true, the interface provides access to cached profile rows that have either been updated or retrieved during the orchestration. If OrchestrationEngineConfigurationParameter.ENABLE_PROFILE_ROW_CACHING_KEY is set to false, but and OrchestrationListener is passed into processevent(), getcachedprofilerows() will return null. 66

67 Analytics Solutions Foundation UI Web Application The Analytics Solutions Foundation UI web application provides a browser-based user interface for users of the Analytics Solutions Foundation to author the solution definition and the orchestration XML files. This web application is delivered as a war (web application archive) file and should be deployed on an application server by the user to use the web-based authoring capabilities. Installing and starting the Analytics Solutions Foundation Web Application This section describes the steps needed to install the Analytics Solutions Foundation UI web application on an existing WebSphere Application Server instance. Installation Procedure: 1. Login to the WebSphere Application Server console as the administrator. For example, Default username/password: admin/admin 2. The Analytics Solutions Foundation UI web application stores the contents of the solution definition and the orchestration XML files in the database. Therefore, a JDBC data source needs to be configured prior to deploying the web application. Launch the JDBC data source creation screen in WAS as follows: Click on to create a new data source. 3. Enter basic data source information and click Next. 67

68 4. Select Create new JDBC provider and click Next. 5. Create the new JDBC provider and click Next. 68

69 6. Enter database driver class path information and click Next. 69

70 7. Enter the database-specific properties for the data source. 70

71 8. Skip the security alias setup below for the time being, and click Finish on the confirm screen. 9. Save the changes to the configuration. 10. The Foundation UI Datasource will display in the list of configured data sources. 71

72 11. The data source needs to be edited to set the schema to be used on connections. Click on the data source to edit it and find the Custom Properties link on the right. 12. Select the currentschema property to edit it. 72

73 13. Specify the schema name to use, click OK, and save the configuration changes. 73

74 14. Configure the authentication credentials to connect to the data source. Locate the JAAS J2C Authentication Data link on the right. 15. Click New to create a new authentication configuration. Specify the credentials to use and save the configuration changes. 74

75 16. Go back to the Foundation UI Datasource and scroll down to the Security Settings section to associate the authentication data just created to the data source. Save the changes to the configuration. 75

76 17. Click on the Test connection button to test the connection to the configured data source. If the test connection was successful, click OK to save the data source. 76

77 18. To deploy the Analytics Solutions Foundation UI web application, click on the New Application link on the left, then select New Enterprise Application in the right panel. 19. Browse and locate the war file on the local or remote file system. 20. Select the Fast Path mode of install. 77

78 Click Next on the Select Installation Options and Map Modules to Servers screen (no changes needed). 21. Map the JDBC data source that was created earlier to the jdbc/tenx resource reference, and click Next. 22. Map to default_host as the virtual host and click Next. 78

79 23. Specify the context root for the web application and click Next. 24. Click Next on the Metadata for Modules screen (no changes needed) and click Next. 79

80 25. Click Finish on the summary screen, and Save the changes to the configuration. 80

81 26. The web application will now display on the list of Enterprise Applications. Click on it to edit it. 81

82 27. Click on Manage Modules, then the Foundation Web UI module to change the classloader policy. 28. Change the Classloader order property to load with the local classloader first, parent last. Click OK and Save the changes to the configuration. 82

83 29. Select the war application just deployed and click start. 83

84 30. Access the web application from a browser by using the following URL: You should land on the following main UI page: 84

85 UI Components The Analytics Solutions Foundation web interface consists of a number of tabs. This section presents each tab. Data Model Tab The data model tab allows Solutions to create definitions of master data tables, profile tables, events, and event observations. When loading the UI for the first time, it lands on the Data Model tab as shown above. Managing Columns and Reference of a Table Forms for all tables (master data tables, profile tables, event tables, and observation tables) have a table of columns and a table of references to allow users to add columns and references: The process of creating, editing, and deleting columns and references is identical among the four types of tables (master data table, profile table, event table, and observation table). Managing Columns To add a new column to the table of columns, click the Add Column button. An Add Column form will appear: 85

86 Enter the name of the column, the column type, and the column size as desired: Users could also specify if the column is part of the key, if the column allows nulls, or if the column has a default value. To add a default value, check the Has a default value check-box and type the default value in the field adjacent to it: When done, click the Save Column button to add the column to the table: Users may edit the column by clicking on the column name hyperlink: To delete a column, select the column from the table by clicking on any of the column attributes except the column name, and then click the Delete... button: Notice that adding, editing, or deleting a column from the table is not reflected in the database until the Save button is clicked on the main Table Form (for example, Master Table Form, or Profile Form). Managing References To add a new reference to the table of references, click the Add Reference button. An Add Reference form will appear: 86

87 Enter the name of the reference in the Reference name field: A user then selects the master table that this table is referencing by clicking on the Select... hyperlink. A Select... form should appear, which allows you to select any of the existing master tables: In addition to the user-defined master tables, the list above also shows a list of the system-owned master tables or support tables (for example, CALENDAR, EVENT_TIME, KEY_LOOKUP, LANGUAGE, and TENANT tables). The latter tables do not appear on the UI, but will be present in the solution definition XML once exported. Click on the table to be selected. You could also filter out the list of tables by typing the first letters of the table to be selected: To make the reference as part of the key, the user can check the Is part of the key check-box: 87

88 Finally, the Add Reference form has a table where you can define a column mapping: To add a new column mapping, click on the Add Column Mapping button. An Add Column Mapping form will appear: Type in the key column name of the referenced table in the Entity key column name field, and the column name of the reference in the Reference column name field. You can also provide a default value for this column mapping in the Default Value field: Click the Save Column Mapping button to add the new column mapping to the table: Users may edit a column mapping by clicking on the Entity key column name hyperlink: To delete a column mapping, select the column mapping from the table by clicking on any of the column mapping attributes except the Entity key column name, and then click the Delete... button: 88

89 Note that adding, editing, or deleting a column mapping from the table is not reflected in the database until the Save button is clicked on the main Table Form (for example, Master Table Form, or Profile Form). Click the Save Reference button to save the new reference to the table of references on the main Table form: Deleting Tables Each table, regardless of its type (master data, profile, event, or observation) can be deleted once created for the first time. When you save a table for the first time or when selecting a table from the left tree, a Delete button appears on the corresponding table form. To delete a table, click the Delete button. Depending on the type of table you are trying to delete, a warning message is displayed. For example, if the table is a master table, the following message will be displayed: Clicking OK will confirm the deletion of the table from the database. Editing Tables To edit a table of any type (master data, profile, event, or observation), select the table from the left tree: 89

90 The corresponding form will show in the center: You may edit details of the table and then hit Save to persist the changes in the database. Master Data Table To create a new master data table, click the Add Master Table button. Initially, this will create a blank form. Type a desired name for the master data table you wish to create in the Table name field: To assign a custom surrogate key which is different from the default surrogate key '<MASTER_TABLE_NAME>_ID', check the Surrogate primary key check box and replace the default key 90

91 name with the desired name of the surrogate key. You can also change the type of the surrogate key to big integer by choosing BigInteger from the Surrogate key type drop-down: Users can also configure the master data table to support multiple languages, multiple tenants, and/or missing member inserts. In addition, users can set a default indicator, mark rows as deactivateable, and/or add a timestamp column to the table to indicate the last time a record in the table was updated: Finally, users can set a custom size to be the maximum size of the surrogate key cache and provide a validator class to validate rows in the table: Managing Columns and References To manage columns and references of a Master Table, follow similar instructions as in the 'Managing Columns and Reference of a Table' section. Creating a Hierarchy Table A user can define a hierarchy table by checking the Create a hierarchy table check box: The table above shows all key columns defined in the columns table, and their corresponding reference columns names as they will appear in the hierarchy table. Enter the number of levels in the hierarchy and the reference name accordingly, and select the list of duplicate columns by starting to type the first letters of the column names and then selecting from the dropdown: 91

92 To store the new master data table in the database, click the Save button. When saved, the master table should be added to the list of Master Tables on the left tree: For example, creating and exporting the below master data table will produce the following XML table definition: <table is_surrogate_primary_key="true" is_multilanguage="true" is_multitenant="true" is_row_deactivateable="false" is_missing_member_insert="false" is_default_indicator="false" has_last_updated_timestamp="false" table_cd="master_group_dim"> <column column_cd="group_member_cd" type="string" size="50" is_key="true" is_nullable="false"/> <column column_cd="group_type_cd" type="string" size="50" is_key="true" is_nullable="false" default_value="group_type_cd_1"/> <surrogate_key_column column_cd="group_dim_id" type="bigint"/> </table> Creating and exporting a master data table with the configurations shown below will produce the following XML table definition: 92

93 <column column_cd="resource_name" type="string" size="104" is_key="false" is_nullable="false"/> <column column_cd="serial_no" type="string" size="50" is_key="true" is_nullable="false"/> <column column_cd="model" type="string" size="50" is_key="true" is_nullable="false"/> <column column_cd="operator_cd" type="string" size="100" is_key="true" is_nullable="false"/> <reference reference_cd="group_dim_id_2" table_reference="master_group_dim" is_key="false"> <column_mapping reference_column_cd="group_member_cd_2" table_column_cd="group_member_cd"/> <column_mapping reference_column_cd="group_type_cd_2" table_column_cd="group_type_cd"/> </reference> <reference reference_cd="group_dim_id_1" table_reference="master_group_dim" is_key="false"> <column_mapping reference_column_cd="group_member_cd_1" table_column_cd="group_member_cd"/> <column_mapping reference_column_cd="group_type_cd_1" table_column_cd="group_type_cd" default_value="group_type_cd_default"/> </reference> <self_reference reference_cd="parent_resource" number_of_levels="10"> <column_mapping reference_column_cd="parent_operator_cd" table_column_cd="operator_cd"/> <column_mapping reference_column_cd="parent_serial_no" table_column_cd="serial_no"/> <column_mapping reference_column_cd="parent_model" table_column_cd="model"/> <duplicate_column_cd>resource_name</duplicate_column_cd> </self_reference> Profile Table To create a new profile table, click the Add Profile button: Initially, this will create a blank form: Type a desired name for the profile table you wish to create in the Table name field: 93

94 You can choose to include an interval column by checking the Include an interval column (supports Key Performance Indicators) check box. To change the default name of the interval column, replace the content of the Interval column name field with the desired name. You can also choose the type of the interval column from the Interval column type drop-down. Finally, to include an additional timestamp column on the profile table, check the Update timestamp check box: Managing Columns and References To manage columns and references of a Profile Table, follow similar instructions as in the 'Managing Columns and Reference of a Table' section. To store the new profile table in the database, click the Save button. When saved, the profile table should be added to the list of Profiles on the left tree: For example, creating and exporting the below profile table will produce the following XML table definition: <profile_definition>... <table has_last_updated_timestamp="true" table_cd="resource_kpi"> <column column_cd="actual_value" type="double" is_key="false" is_nullable="true"/> <reference reference_cd="resource_id" table_reference="master_resource" is_key="true"/> <interval_column column_cd="kpi_timestamp" type="timestamp" is_key="true" is_nullable="false"/> </table>... </profile_definition> Event Table 94

95 To create a new event table, click the Add Event button: Initially, this will create a blank form: Type a desired name for the event table you wish to create in the Event name field: To include an additional timestamp column on the event table, check the Create timestamp check box. You can also link the event table with the calendar and/or event time tables (that is, create event_interval_columns for calendar and/or event time). For example, to link the event table to the systemdefined calendar table 'CALENDAR' and a time table called 'EVENT_TIME', check both Link to Calendar table and Link to Time table check boxes. You may keep the event interval column name as default (that is, 'EVENT_DATE' and 'EVENT_TIME') or replace it with a desired column name: Managing Columns and References To manage columns and references of an Event Table, follow similar instructions as in the 'Managing Columns and Reference of a Table' section. To store the new event table in the database, click the Save button. When saved, the event table should be added to the list of Events on the left tree: For example, creating and exporting the below event table will produce the following XML table definition: 95

96 <event_definition>... <table has_last_updated_timestamp="true" table_cd="event"> <column column_cd="incoming_event_cd" type="string" size="200" is_key="false" is_nullable="true"/> <reference reference_cd="resource_id" table_reference="master_resource" is_key="false"/> <event_interval_column column_cd="event_date" type="date"/> <event_interval_column column_cd="event_time" type="time"/> </table>... </event_definition> Observation Table The Add Observation button is usually disabled unless a user selects an event from the list of events previously created. To create a new observation table to be associated with the event table, select the desired event from the left tree first: The Add Observation button at that time should be enabled: Click the Add Observation button. Initially, this will create a blank form: 96

97 Type a desired name for the observation table you wish to create in the Observation name field: Similar to event tables, a user may link the observation table with the calendar and/or event time tables (that is, create event_interval_columns for calendar and/or event time). For example, to link the observation table to a system-defined calendar table 'CALENDAR' and a time table called 'EVENT_TIME', check both Link to Calendar table and Link to Time table check boxes. You may keep the event interval column name as default ('OBSERVATION_DATE' and 'OBSERVATION_TIME'), or replace it with a desired column name (e.g., OBS_DATE and OBS_TIME): Managing Columns and References To manage columns and references of an Observation Table, follow similar instructions as in the 'Managing Columns and Reference of a Table' section. To store the new observation table in the database, click the Save button. When saved, the observation table should be added to the list of observations under the selected event in the left tree: For example, creating and exporting the below observation table will produce the following XML table definition: 97

98 <event_definition>... <table has_last_updated_timestamp="true" table_cd="event">... <observation table_cd="event_observation"> <column column_cd="measurement" type="double" is_key="false" is_nullable="false"/> <reference reference_cd="event_code_id" table_reference="master_event_code" is_key="true"/> <event_interval_column column_cd="obs_time" type="time"/> <event_interval_column column_cd="obs_date" type="date"/> </observation> </table>... </event_definition> 98

99 Calculations Tab The calculations tab allows Solutions to view pre-defined calculations or create custom calculations that can be invoked later at run time by a profile adapter. When clicking the Calculations Tab for the first time, the following appears on the UI: In the center is a Calculation Form that allows you to create a new user-defined or custom calculation. On the left side is a list of all calculations, including both pre-defined calculations and user-defined calculations. Note that all the calculations shown above are pre-defined uneditable calculations generated by Analytics Solutions Foundation, and can be directly used. You may select a pre-defined table from the list and observe its properties. You may also edit any of its properties, but you cannot save any of the changes (the Save button is missing for such calculations): To create a new custom calculation, click the Add Calculation button. A blank calculation form will appear. As mentioned previously, you may also use the default form when landing on the calculations tab for the first time. Type a desired name for the calculation you wish to create in the Calculation Name field: Then type the full name of the calculation class: You can indicate that the custom calculation allows incremental calculation by checking the Is Incremental check box: 99

100 Recall that incremental calculations are those calculations that can be performed directly on the database side without the need to retrieve any inputs, and save the results back into the database (e.g., ASSIGN, INCREMENT_COUNT). Managing Inputs and Outputs Both the calculation form and the service form have a table of inputs and and a table of outputs: The process of creating, editing, and deleting inputs and outputs is identical for both calculations and services. Managing Inputs To add a new input to the table of inputs, click the Add Input Parameter button. An Add Input Parameter form will appear: Enter the name and type of input as desired: When done, click the Save Parameter button to add the new input parameter to the table of input parameters: Users can edit the input by clicking on the parameter name hyperlink: 100

101 To delete an input, select the input from the table of inputs by clicking on the type of the input in the table, and then click the Delete... button: Note that adding, editing, or deleting an input from the table of inputs is not reflected in the database until the Save button is clicked on the main Calculation/Service form. Managing Outputs To add a new output to the table of outputs, click the Add Output Parameter button. An Add Output Parameter form will appear: Enter the name and type of output as desired: When done, click the Save Parameter button to add the new output parameter to the table of output parameters: Users can edit the output by clicking on the parameter name hyperlink: 101

102 To delete an output, select the output from the table of outputs by clicking on the type of the output in the table, and then click the Delete... button: Note that adding, editing, or deleting an output from the table of outputs is not reflected in the database until the Save button is clicked on the main Calculation/Service form. To store the new calculation in the database, click the Save button. When saved, the new calculation should be added to the list of calculations on the left: For example, creating and exporting the below calculation will produce the following XML representation: <calculation calculation_cd="mean" is_increment="false" calculation_class="com.ibm.analytics.solution.calc.meancalculation"> <input> <field_name>measure_value2</field_name> <type>double</type> </input> <input> <field_name>measure_value1</field_name> <type>double</type> </input> <output> <field_name>calculated_value</field_name> <type>double</type> </output> </calculation> 102

103 Services Tab The services tab allows Solutions to create services that can be invoked at run time by orchestrations that employ service adapters and/or remote service adapters. When clicking the Services Tab for the first time, the following appears on the UI: To create a new service, click the Add Service button. A blank service form will appear. As mentioned previously, you can also use the default form when landing on the services tab for the first time. Type a desired name for the service you wish to create in the Service Name field: Managing Inputs and Outputs To manage inputs and outputs of a service, follow similar instructions as in the 'Managing Inputs and Outputs' in the Calculations Tab section. To store the new service in the database, click the Save button. When saved, the new service should be added to the list of services on the left: For example, creating and exporting the below service will produce the following XML representation: 103

104 <service_definition>... <service service_cd="scoring_service"> <input> <field_name>temp_above_limit_sum</field_name> <type>double</type> </input> <input> <field_name>temp</field_name> <type>double</type> </input> <input> <field_name>resource_id</field_name> <type>int</type> </input> <output> <field_name>recommendation</field_name> <type>string</type> </output> </service>... </service_definition> Deleting Calculations/Services A calculation or service can be deleted once created for the first time. When you save a calculation or service for the first time or when selecting a calculation/service from the list on the left of the corresponding tab, a Delete button appears on the corresponding form. To delete a calculation/service, click the Delete button Depending if it is a calculation or service you are trying to delete, a warning message is displayed. For example, if it is a calculation, the following message will be displayed: 104

105 Clicking OK will confirm the deletion of the calculation/service from the database. Editing Calculations/Services To edit a calculation or service, select the calculation/service from list on the left side of the corresponding tab: The corresponding form will show in the center: You can edit details of the calculation/service and then hit save to persist the changes in the database. 105

106 Orchestrations Tab The orchestrations tab is where Solutions can define new orchestrations or modify existing orchestrations. Adding New Nodes and Panels The list of defined orchestration configuration details are represented as nodes in the tree for the orchestrations tab. Clicking a node will display the contents of the configuration and allow users to edit and delete the node. Clicking the Add button will reveal the possible nodes that can be added based on the user's current node selection. The hierarchy of possible nodes that can be added to the orchestrations tree are as follows: Note: These options are contextual, and therefore will only be displayed when the currently selected node's sibling or parent is of a valid type, as shown above. Reordering Nodes and Panels For many of the defined configurations and processes, ordering matters in order to ensure the correct jobs are ran during the orchestration. Nodes and panels that track ordering will display a set of navigation arrows to update their relative sequence order. Ordering is displayed in a chronological order. For example, pressing the up arrow for the displayed Update node will cause it to be run first among the group of profile update actions. The figures below reference a before and after view of the reordering. 106

107 107

108 Orchestration To create a new store orchestration, select New Orchestration from the drop-down in the Add button. Initially, this will create a blank form: To select an event type for the orchestration, choose any of the available events in the Event drop-down. If there are no available events, then events must be defined in the Data Model tab. A table is provided to define orchestration event key mappings. To create an event key code, click the <click to specify a code> hyperlink to begin typing in the in-line editor. To remove the row, leave the editor empty or click the delete icon in the right-most column. To define additional pool properties for the orchestration, check the Advanced configuration check box and enter the appropriate values. 108

109 For example, exporting the below orchestration form will produce the following XML: <?xml version="1.0" encoding="utf-8" standalone="yes"?> <ns3:orchestration_definition xmlns:q1=" xmlns:ns3=" xmlns:xsi=" xsi:schemalocation=" orchestration_definition.xsd"> <event_orchestration_mapping> <event_orchestration_key_cd>com.ibm.sample.orchestration.key </event_orchestration_key_cd> <orchestration_cd>foundation.sample</orchestration_cd> </event_orchestration_mapping> <orchestration> <pool_configuration max_pool_size="20" min_pool_size="5" max_idle_size="10" /> <event_table_cd>event</event_table_cd> <orchestration_cd>foundation.sample</orchestration_cd> </orchestration> </ns3:orchestration_definition> 109

110 Store Event Step To create a new store event step for the orchestration, select the Store Event Step from the drop-down in the Add button. No configuration is required for the event store adapter. It will automatically store all events it receives. Give the step a descriptive name and save the form. 110

111 Custom Step To create a new custom adapter step for the orchestration, select the Custom Step from the drop-down in the Add button. Initially, this will create a blank form with a default custom step name, which can be modified to be more descriptive: You can specify the custom adapter class to be used during the orchestration, as well as any property key and values the adapter will require at run-time. To add a key and value, select the <click to specify a key> or <click to specify a value> hyperlinks and begin typing into the in-line editor that appears. For example, the custom step defined above would produce the following XML when exported: <step step_name="custom Step > <adapter_class>com.abc.mycustomadapter</adapter_class> <adapter_configuration> <config_property> <key>adapter_key</key> <value>value</value> </config_property> </adapter_configuration> </step> 111

112 Profile Adapter To create a new orchestration step and profile adapter, select the Profile Update Step from the drop-down in the Add button. All selectors and profile update actions will be nested under the created step. Event and Observation Profile Update The event and observation profile updates are managed on the Selector form, nested under the Profile Update Step in the Add button drop-down. Initially, this will create a blank form with a default selector name, which can be modified to be more descriptive: If the Selector form contains an observation selector definition, then it will be exported as an observation profile update. Otherwise, if only an event selector exists, the Selector will be exported as an event profile update. Event Selector 112

113 To add an event selector, begin by clicking the Add an event selector hyperlink. This will add a new table so that you can select events based on specified event field values. To enter values, click the <click to specify a value> hyperlink to launch a dialog to set constant values. Enter the value, and then click Ok to place the value in the table. For example, the XML below would be created for the event selector defined above: <event_selector> <event_field_value> <field_name>event_type_cd</field_name> <value>measurement</value> </event_field_value> </event_selector> 113

114 Note: If an Event was not associated and saved for the parent orchestration, then the Event Selector will display a reminder to set this value. This can be done by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. 114

115 Observation Selector To add an observation selector, begin by clicking the Add an observation selector hyperlink. This will add a new table so that you can select an event's observations based on specified field values. To enter values, click the <click to specify a value> hyperlink to launch a dialog to set constant values. Enter the value, and then click Ok to place the value in the table. For example, the XML below would be created by the export of the observation selector above: 115

116 <observation_selector table_cd="event_observation"> <observation_field_value> <field_name>value_type_cd</field_name> <value>actual</value> </observation_field_value> <observation_field_value> <field_name>measurement_type_cd</field_name> <value>rpm</value> </observation_field_value> </observation_selector> Note: If an Event was not associated and saved for the parent orchestration, then the Observation Selector will display a reminder to set this value. This can be done by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. A list of available observations will now be provided based on the specified Event. 116

117 Profile Update Action The profile update action and the associated profile row selectors, read only profile row selectors, and calculations are managed on the the Update form, nested under Selector in the Add button drop-down. Initially, this will create a blank form with a default update name, which can be modified to be more descriptive. In order to save a profile update action, you must specify at least one profile row selector that will be used to trigger the update action. Profile Row Selectors To add a profile row selector, begin by clicking the hyperlink Add Profile to Update. This will add a new table so that you can select a profile based on the specified field values. 117

118 It will also populate a common table on the form to hold the profile fields that can be updated after performing a designated calculation. Selecting any of the <click to specify a value> hyperlink buttons will display a dialog to specify calculations, which is detailed in the Calculation Invocations section. Users can also select a shared profile selector created in the Reusable Components tab from the dropdown. This will apply the shared values to the profile row selector, and allow any values defined in the selector key fields table to override those defined in the shared profile selector. Clicking the shared profile selector's View button will allow the user to view the defined fields that can be overridden locally. These values can be changed in the Reusable Components tab to apply across all references. 118

119 Note: Additional profile row selectors can only be selected if they share the same non-key fields that can be updated based on the first profile row selector defined. Therefore, if there are no additional profiles sharing common fields that can be updated, the Profile to Update drop-down will be blank. To delete a profile row selector, click the Delete hyperlink located beneath the panel. To specify a local profile row selector field, select the <click to specify a value> hyperlink button to launch a dialog containing the valid value types available. The following options are provided: Value Type Explanation Constant A valid constant value. Context Variable A valid context variable name that will be used. Event Field The orchestration's selected event field value will be used for matching. The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration as shown below. Observation Field The value from the chosen observation field name in the selected observation will be used for matching. The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used for matching in the profile. 119

120 Note: The following message may be shown on selecting an Observation Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. Selected Observation Field The value in the chosen observation selector field will be used for matching. 120

121 The drop-down will be populated each of the valid observation selector fields. Note: The following message may be shown on selecting a Selected Observation Field. This can be fixed by navigating to the parent Selector form, clicking the Add an observation selector hyperlink button to add an observation selector, and then saving the Selector, as shown below. After clicking Ok in dialog to specify values for the profile row selector form, the table will be filled with a summary of the values: 121

122 For example, the XML below would be created by the export of the above profile row selector: <profile_row_selector> <shared_selector_cd>resource_kpi</shared_selector_cd> <key_field_value> <profile_field_name>profile_variable_cd</profile_field_name> <value>ophr Delta</value> </key_field_value> </profile_row_selector> Read Only Profile Row Selectors To add a read only profile row selector, begin by clicking the hyperlink Add Read-Only Profile Row to Supply Values. This will add a new table so that you can select a profile row based on the specified field values, but will not update the actual row with values produced in a calculation. Instead, these values can be referenced for input to the calculations based on the specified Row Selector Alias name. Users can select a shared profile selector created in the Reusable Components tab from the drop-down. This will apply the shared values to the read only profile row selector, and allow any values defined in the selector key fields table to override those defined in the shared profile selector. 122

123 Clicking the shared profile selector's View button will allow the user to view the defined fields that can be overridden locally. These values can be changed in the Reusable Components tab to apply across all references. Note: To delete a read only profile row, click the Delete hyperlink located beneath the panel. To specify a local read only profile row selector field, select the <click to specify a value> hyperlink button to launch a dialog containing the valid value types available. The following options are provided: Value Type Explanation Constant A valid constant value. Context Variable A valid context variable name that will be used. Event Field The orchestration's selected event field value will be used for matching. The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. 123

124 Observation Field The value from the chosen observation field name in the selected observation will be used for matching. The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used for matching in the profile. Note: The following message may be shown on selecting an Observation Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. 124

125 Selected Observation Field The value in the chosen observation selector field will be used for matching. The drop-down will be populated for each of the valid observation selector fields. Note: The following message may be shown on selecting a Selected Observation Field. This can be fixed by navigating to the parent Selector form, clicking the Add an observation selector hyperlink button to add an observation selector, and then saving the Selector, as shown below. 125

126 After clicking Ok in dialog to specify values for the read only profile row selector form, the table will be filled with a summary of the values: For example, the XML below would be created by the export of the above read only profile row selector: <read_only_profile_row_selector alias="resource_profile_last_value"> <shared_selector_cd>resource_profile</shared_selector_cd> <key_field_value> <profile_field_name>profile_variable_cd</profile_field_name> <value>ophr Delta</value> </key_field_value> </read_only_profile_row_selector> 126

127 Calculation Invocations Once a profile row selector has been specified for the profile update action, you can begin to create calculations that will be performed to take input values provided by the configuration and selectors, as well as assign update values whose destinations will be set. To access the form to define local calculations, select the <click to specify a value> hyperlink button on the top update fields table. Initially, the calculations will be empty and the option to choose a shared calculation group is provided. Selecting a shared calculation group will populate the form with a list of previously defined calculation invocations. These calculation invocations cannot be reordered or changed to different calculations locally, as running an orchestration requires that shared calculations run first and are followed by any locally created calculations. Clicking the shared calculation group's View button will allow the user to view the defined fields that can be overridden locally. These values can be changed in the Reusable Components tab to apply across all references. 127

128 Alternatively, you can define completely new calculations by clicking the Add Calculation hyperlink at the bottom of the page. These calculations can be reordered relatively. Note: To delete a locally created calculation, click the Delete hyperlink located beneath the panel. To specify a calculation input value, regardless of whether the calculation invocation is a new local calculation or a local override of values specified in the shared calculation, select the <click to specify a value> hyperlink button to launch a dialog containing the valid value types available. The following options are provided: Value Type Explanation Constant A valid constant value. Context Variable A valid context variable name that will be used. Event Field The orchestration's selected event field value will be used. The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. 128

129 This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. Observation Field The value from the chosen observation field name in the selected observation will be used. The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used for matching in the profile. Note: The following message may be shown on selecting an Observation Field. 129

130 This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. Selected Observation Field The value in the chosen observation selector field will be used. The drop-down will be populated for each of the valid observation selector fields. Note: The following message may be shown on selecting a Selected Observation Field. This can be fixed by navigating to the parent Selector form, clicking the Add an observation selector hyperlink button to add an observation selector, and then saving the Selector, as shown below. 130

131 Selected Profile Field The selected profile's current field value will be used. The drop-down will be populated with a list of the valid profile fields. Read-Only Profile Field The selected read-only profile's current field value will be used. The Select an alias drop-down field will be filled with all the read-only selectors that can be used for providing values. Once an alias is selected, the Select a profile field drop-down will be populated with all valid profile fields. 131

132 Note: The following message may be shown on selecting a Read-Only Profile Field. This can be fixed by navigating to the main Update form and adding a new Read-Only Profile Selector by clicking the Add Read-Only Profile Row to Supply Values hyperlink located near the bottom of the page. Be sure to give the selector an alias name for identification and profile. Similarly, to specify a calculation output value, select the <click to specify a value> hyperlink button in the Destination column to launch a dialog for defining multiple mappings. 132

133 Multiple output destinations can be specified for updates using the produced calculation values. The following value type options are provided when clicking the <click to specify a value>: Value Type Explanation Context Variable A valid context variable name that will be used. Selected Profile Field The selected profile's current field value will be used. The drop-down will be populated with a list of the valid profile fields. Note: These mappings can be deleted by clicking the trash can icon to the right of the definition. Finally, each calculation invocation can specify additional updates by checking the Additional updates after the calculation invocation check box. Constant values can be used for input Value, while Constant Variable and Selected Profile Field can be used for Destinations. These are the same as described earlier. For example, the calculation mapping below for a local override of a calculation within the shared calculation group would correspond to the following XML on export. 133

134 <event_profile_update>... <profile_update_action> <profile_row_selector>... </profile_row_selector> <shared_calculation_invocation_group_cd>kpi_measurement_above_limit </shared_calculation_invocation_group_cd> <calculation_invocation invocation_cd="value"> <input_field_value> <field_name>threshold</field_name> <value>100</value> </input_field_value> </calculation_invocation> </profile_update_action> </event_profile_update> 134

135 Service Step To create a new service step for the orchestration, select the Service Step from the drop-down in the Add button. All selectors will be nested under the created step. Initially, this will create a blank form with a default step name, which can be modified to be more descriptive: To add service configurations, check the Include additional configuration check box to display a panel for entering values: You can specify the handler context variable to be used during the orchestration, as well as any property key and values the adapter will require at runtime. To add a key and value, select the <click to specify a key> or <click to specify a value> hyperlinks and begin typing into the in-line editor that appears. 135

136 For example, the service step defined above would produce the following XML when exported: <step step_name="service Step"> <adapter_class>com.ibm.analytics.foundation.adapter.service.serviceadapter</adapter_class> <adapter_configuration xsi:type="ns3:service_adapter_configuration" enablement_handler_context_variable_name="service_context"> <config_property> <key>key</key> <value>value</value> </config_property>... </adapter_configuration> </step> 136

137 Event and Observation Profile Update The event and observation profile updates are managed on the the Selector form, nested under the Service Step in the Add button drop-down. Initially, this will create a blank form with a default selector name, which can be modified to be more descriptive: Event Selector To add an event selector, begin by clicking the Add an event selector hyperlink. This will add a new table so that you can select events based on specified event field values. 137

138 To enter values, click the <click to specify a value> hyperlink to launch a dialog to set constant values. Enter the value, and then click Ok to place the value in the table. For example, the XML below would be created for the event selector defined above: <service_invocation_configuration name="selector"> <event_selector> <event_field_value> <field_name>event_type_cd</field_name> <value>measurement</value> </event_field_value> </event_selector> </service_invocation_configuration> Note: If an Event was not associated and saved for the parent orchestration, then the Event Selector will display a reminder to set this value. This can be done by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. 138

139 Observation Selector To add an observation selector, begin by clicking the Add an observation selector hyperlink. This will add a new table so that you can select an event's observations based on specified field values. Multiple observations can be used for selecting by clicking the Add an observation selector hyperlink to create additional panels as needed. To enter values, click the <click to specify a value> hyperlink to launch a dialog to set constant values. Enter the value, and then click Ok to place the value in the table. 139

140 For example, the XML below would be created by the export of the observation selector above: <service_invocation_configuration name="selector"> <observation_selector table_cd="event_observation"> <observation_field_value> <field_name>value_type_cd</field_name> <value>actual</value> </observation_field_value> <observation_field_value> <field_name>measurement_type_cd</field_name> <value>rpm</value> </observation_field_value> </observation_selector> </service_invocation_configuration> Note: If an Event was not associated and saved for the parent orchestration, then the Observation Selector will display a reminder to set this value. This can be done by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. A list of available observations will now be provided based on the specified Event. 140

141 Invocation Configuration To create a new invocation configuration, select Invocation Configuration from the drop-down in the Add button. All selectors and invocations will be nested under the created configuration. Initially, this will create a blank form with a default invocation configuration name, which can be modified to be more descriptive: 141

142 Profile Selector To create a new Profile Selector, select Profile Selector from the drop-down in the Add button. These selectors will supply read-only values to be used by the sibling invocations. Initially, this will create a blank form with a default selector name, which can be modified to be more descriptive. This name will serve as the reference alias for invocations. Users can also select a shared profile selector created in the Reusable Components tab from the dropdown. This will apply the shared values to the profile row selector, and allow any values defined in the selector key fields table to override those defined in the shared profile selector. Clicking the shared profile selector's View button will allow the user to view the defined fields that can be overridden locally. These values can be changed in the Reusable Components tab to apply across all references. 142

143 Note: Once a shared profile is selected, you must choose none of the shared references in the drop-down in order to select a different profile directly. Otherwise, the profile drop-down field will be locked. To specify a field value, select the <click to specify a value> hyperlink button to launch a dialog containing the valid value types available. The following options are provided: Value Type Explanation Constant A valid constant value. Context Variable A valid context variable name that will be used. Event Field The orchestration's selected event field value will be used for matching. The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. 143

144 This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. Observation Field The value from the chosen observation field name in the selected observation will be used for matching. The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used for matching in the profile. Note: The following message may be shown on selecting an Observation Field. 144

145 This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. After clicking Ok in dialog to specify values for the profile row selector form, the table will be filled with a summary of the values: For example, the XML below would be created by the export of the profile row selector above: <step step_name="service Step"> <adapter_class>com.ibm.analytics.foundation.adapter.service.serviceadapter</adapter_class> <adapter_configuration xsi:type="ns3:service_adapter_configuration" enablement_handler_context_variable_name="service_context">... <service_invocation_configuration name="selector">... <service_invocation_action name="invocation Configuration"> <profile_row_selector alias="rpm"> <shared_selector_cd>resource_kpi</shared_selector_cd> 145

146 </step> <key_field_value is_value_required="false"> <profile_field_name>profile_variable_cd </profile_field_name> <value>rpm</value> </key_field_value> </profile_row_selector> </service_invocation_action> </service_invocation_configuration> </adapter_configuration> 146

147 Service Invocation To create a new service invocation, select Service Invocation from the drop-down in the Add button. Invocations will be run in the relative order that they are defined, using values supplied from other defined read-only profile selector siblings. Initially, this will create a blank form: Start by selecting one of the defined services from the Service to invoke drop-down. These are created in the Services tab if edits to existing instances or new services are required. On selection, the table of inputs and updates will be refreshed for assigning mappings: 147

148 The service handler context variable can be specified, or an event can be used to record the results with the appropriate configuration: Users can select a shared service update created in the Reusable Components tab from the drop-down. This will apply the shared values to the update, and allow any values defined in the selector key fields table to override those defined in the shared service update. To view the current assignments, simply click the View button next to the shared update drop-down: 148

149 To overwrite or create new local input values for the invocation, select the <click to specify a value> hyperlink button to launch a dialog containing the valid value types available. The following options are provided: Value Type Explanation Constant A valid constant value. Context Variable A valid context variable name that will be used. Event Field The orchestration's selected event field value will be used for matching. The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. Observation Field The value from the chosen observation field name in the selected observation will be used for matching. 149

150 The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used. Note: The following message may be shown on selecting an Observation Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. Read-Only Profile Field The selected read-only profile's current field value will be used. The Select an alias drop-down field will be filled with all the read-only selectors that can be used for providing values. Once an alias is selected, the Select a profile field drop-down will be populated with all valid profile fields. 150

151 Note: The following message may be shown on selecting a Read-Only Profile Field. This can be fixed by creating new Profile Selector as a sibling of the current invocation. Refer to the Profile Selector section on how to create new selectors. 151

152 To overwrite or create new local output values for the invocation update, select the <click to specify a value> hyperlink button in the Destination column to launch a dialog for defining multiple mappings. Multiple output destinations can be specified for updates using the produced service values. The following value type options are provided when clicking the <click to specify a value>: Value Type Explanation Context Variable A valid context variable name that will be used. Event Field The orchestration's selected event field value will be used for matching. The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. This can be fixed by navigating to the event record configuration on the form, checking the Use an event to record the results check box, and selecting a generated event type from the Event field drop-down, as shown below. Observation Field The value from the chosen observation field name in the selected observation will be used for matching. 152

153 The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used. Note: The following message may be shown on selecting an Observation Field. This can be fixed by navigating to the event record configuration on the form, checking the Use an event to record the results check box, and selecting a generated event type from the Event field drop-down, as shown below. 153

154 Additional mappings can be made after a service has been run. These mappings are specified in the table located at the bottom of the form. To choose a destination update field value, select the <click to specify a value> hyperlink in the Destination column. Destination types include Context Variables, Event Fields, and Observation Fields exactly as described above. To choose an input value to use for assignment, select the <click to specify a value> hyperlink in the Value column. The following input options are provided: Value Type Explanation Constant A valid constant value. Event Field The orchestration's selected event field value will be used for matching. The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. Observation Field The value from the chosen observation field name in the selected observation will be used for matching. The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used. 154

155 Note: The following message may be shown on selecting an Observation Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. For example, performing an export on the invocation below would produce the following XML during export. <service_invocation_action>

156 <service_invocation service_cd="maximo_wo_create" handler_context_variable_name="maximo_wo_create_handler" name="service Invocation"> <input_field_value> <field_name>siteid</field_name> <value_ref> <event_field_name>model</event_field_name> </value_ref> </input_field_value> <input_field_value> <field_name>assetnum</field_name> <value_ref> <event_field_name>serial_no</event_field_name> </value_ref> </input_field_value> <input_field_value> <field_name>maximowo</field_name> <value_ref> <profile_field> <alias>maximowo</alias> <field_name>profile_date</field_name> </profile_field> </value_ref> </input_field_value> <input_field_value> <field_name>woperiodmsrcount</field_name> <value_ref> <profile_field> <alias>maximowo</alias> <field_name>period_msr_count</field_name> </profile_field> </value_ref> </input_field_value> <input_field_value> <field_name>ihsrc</field_name> <value_ref> <context_variable_name>ihsrc</context_variable_name> </value_ref> </input_field_value> <input_field_value> <field_name>spssrc</field_name> <value>ihs101</value> </input_field_value> <input_field_value> <field_name>maximorc</field_name> <value>301</value> </input_field_value> </service_invocation> </service_invocation_action> 156

157 Process Step To create a new process step for the orchestration, select the Process Step from the drop-down in the Add button. All selectors will be nested under the created step. Initially, this will create a blank form with a default step name, which can be modified to be more descriptive: A host name and port should be provided when using a remote process. To add other process configurations, check the Include additional configuration check box to display the panel: You can specify the handler context variable to be used during the orchestration. 157

158 For example, the step defined above would produce the following XML when exported: <step step_name="process Step"> <adapter_class>com.ibm.analytics.foundation.adapter.bpm.bpmadapter </adapter_class> <adapter_configuration xsi:type="ns3:remote_service_adapter_configuration" enablement_handler_context_variable_name="bpm_context_handler"> <host_name> <port_number>9080</port_number> </adapter_configuration> </step> 158

159 Event and Observation Profile Update The event and observation profile updates are managed on the the Selector form, nested under the Process Step in the Add button drop-down. Initially, this will create a blank form with a default selector name, which can be modified to be more descriptive: Event Selector To add an event selector, begin by clicking the Add an event selector hyperlink. This will add a new table to be able to select events based on specified event field values. To enter values, click the <click to specify a value> hyperlink to launch a dialog to set constant values. Enter the value, and then click Ok to place the value in the table. 159

160 For example, the XML below would be created for the event selector defined above: <remote_invocation_configuration name="selector"> <event_selector> <event_field_value> <field_name>event_type_cd</field_name> <value>measurement</value> </event_field_value> </event_selector> </remote_invocation_configuration> Note: If an Event was not associated and saved for the parent orchestration, then the Event Selector will display a reminder to set this value. This can be done by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. Observation Selector 160

161 To add an observation selector, begin by clicking the Add an observation selector hyperlink. This will add a new table so that you can select an event's observations based on specified field values. Multiple observations can be used for selecting by clicking the Add an observation selector hyperlink to create additional panels as needed. To enter values, click the <click to specify a value> hyperlink to launch a dialog to set constant values. Enter the value, and then click Ok to place the value in the table. 161

162 For example, the XML below would be created by the export of the observation selector above: <remote_invocation_configuration name="selector"> <observation_selector table_cd="event_observation"> <observation_field_value> <field_name>value_type_cd</field_name> <value>actual</value> </observation_field_value> <observation_field_value> <field_name>measurement_type_cd</field_name> <value>rpm</value> </observation_field_value> </observation_selector> </remote_invocation_configuration> Note: If an Event was not associated and saved for the parent orchestration, then the Observation Selector will display a reminder to set this value. This can be done by navigating to the root orchestration, selecting an event table, and then saving the orchestration as shown below. A list of available observations will now be provided based on the specified Event. 162

163 Invocation Configuration To create a new invocation configuration, select the Invocation Configuration from the drop-down in the Add button. All selectors and invocations will be nested under the created configuration. Initially, this will create a blank form with a default invocation configuration name, which can be modified to be more descriptive: 163

164 Profile Selector To create a new Profile Selector, select Profile Selector from the drop-down in the Add button. These selectors will supply read-only values to be used by the sibling invocations. Initially this will create a blank form with a default selector name, which can be modified to be more descriptive. This name will serve as the reference alias for invocations. Users can also select a shared profile selector created in the Reusable Components tab from the dropdown. This will apply the shared values to the profile row selector, and allow any values defined in the selector key fields table to override those defined in the shared profile selector. Clicking the shared profile selector's View button will allow the user to view the defined fields that can be overridden locally. These values can be changed in the Reusable Components tab to apply across all references. 164

165 Note: Once a shared profile is selected, you must choose none of the shared references in the drop-down in order to select a different profile directly. Otherwise, the profile drop-down field will be locked. To specify a field value, select the <click to specify a value> hyperlink button to launch a dialog containing the valid value types available. The following options are provided: Value Type Explanation Constant A valid constant value. Context Variable A valid context variable name that will be used. Event Field The orchestration's selected event field value will be used for matching. The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. 165

166 This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. Observation Field The value from the chosen observation field name in the selected observation will be used for matching. The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used for matching in the profile. Note: The following message may be shown on selecting an Observation Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. After clicking Ok in dialog to specify values for the profile row selector form, the table will be filled with a summary of the values: 166

167 For example, the XML below would be created by the export of the profile row selector above: <step step_name="process Step"> <adapter_class>com.ibm.analytics.foundation.adapter.bpm.bpmadapter</adapter_class> <adapter_configuration xsi:type="ns3:remote_service_adapter_configuration" enablement_handler_context_variable_name="bpm_context_handler">... <remote_invocation_configuration name="selector">... <remote_invocation_action name="invocation Configuration"> <profile_row_selector alias="rpm"> <shared_selector_cd>resource_kpi</shared_selector_cd> <key_field_value is_value_required="false"> <profile_field_name>profile_variable_cd </profile_field_name> <value>rpm</value> </key_field_value> </profile_row_selector> </remote_invocation_action> </remote_invocation_configuration> </adapter_configuration> </step> 167

168 BPM Invocation To create a new BPM invocation, select BPM Invocation from the drop-down in the Add button. Invocations will be run in the relative order that they are defined, using values supplied from other defined read-only profile selector siblings. Initially, this will create a blank form: Start by selecting one of the defined services from the Service to invoke drop-down. These are created in the Services tab if edits to existing instances or new services are required. On selection, the table of inputs and updates will be refreshed for assigning mappings: 168

169 The process name can be specified and a table is provided for invocation input values. To enter any required names or values, simply click the <click to specify a key> or <click to specify a value> hyperlinks to begin typing in an inline editor. Users are also given the option to use an event to record the results with appropriate configuration: Users can select a shared service update created in the Reusable Components tab from the drop-down. This will apply the shared values to the update, and allow any values defined in the selector key fields table to override those defined in the shared service update. To view the current assignments, simply click the View button next to the shared update drop-down: 169

170 To overwrite or create new local input values for the invocation, select the <click to specify a value> hyperlink button to launch a dialog containing the valid value types available. The following options are provided: Value Type Explanation Constant A valid constant value. Context Variable A valid context variable name that will be used. Event Field The orchestration's selected event field value will be used for matching. The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. 170

171 Observation Field The value from the chosen observation field name in the selected observation will be used for matching. The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used. Note: The following message may be shown on selecting an Observation Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. 171

172 Read-Only Profile Field The selected read-only profile's current field value will be used. The Select an alias drop-down field will be filled with all the read-only selectors that can be used for providing values. Once an alias is selected, the Select a profile field drop-down will be populated with all valid profile fields. Note: The following message may be shown on selecting a Read-Only Profile Field. This can be fixed by creating new Profile Selector as a sibling of the current invocation. Refer to the Profile Selector section on how to create new selectors. 172

173 To overwrite or create new local output values for the invocation update, select the <click to specify a value> hyperlink button in the Destination column to launch a dialog for defining multiple mappings. Multiple output destinations can be specified for updates using the produced service values. The following value type options are provided when clicking the <click to specify a value>: Value Type Explanation Context Variable A valid context variable name that will be used. Event Field The orchestration's selected event field value will be used for matching. The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. This can be fixed by navigating to the event record configuration on the form, checking the Use an event to record the results check box, and selecting a generated event type from the Event field drop-down, as shown below. Observation Field The value from the chosen observation field name in the selected observation will be used for matching. 173

174 The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used. Note: The following message may be shown on selecting an Observation Field. This can be fixed by navigating to the event record configuration on the form, checking the Use an event to record the results check box, and selecting a generated event type from the Event field drop-down, as shown below. 174

175 Additional mappings can be made after an invocation has been run. These mappings are specified in the table located at the bottom of the form. To choose a destination update field value, select the <click to specify a value> hyperlink in the Destination column. Destination types include Context Variables, Event Fields, and Observation Fields, exactly as described above. To choose an input value to use for assignment, select the <click to specify a value> hyperlink in the Value column. The following input options are provided: Value Type Explanation Constant A valid constant value. Event Field The orchestration's selected event field value will be used for matching. The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. Observation Field The value from the chosen observation field name in the selected observation will be used for matching. The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used. 175

176 Note: The following message may be shown on selecting an Observation Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. 176

177 For example, performing an export on the invocation below would produce the following XML during export. 177

178 <step step_name="process Step"> <adapter_class>com.ibm.analytics.foundation.adapter.bpm.bpmadapter </adapter_class> <adapter_configuration xsi:type="ns3:remote_service_adapter_configuration" enablement_handler_context_variable_name="bpm_context_handler">... <remote_invocation_configuration name="selector"> <remote_invocation_action name="invocation Configuration">... <bpm_invocation service_cd="ihs" name="bpm Invocation"> <input_field_value> <field_name>location_id</field_name> <value_ref> <event_field_name>location_cd</event_field_name> </value_ref> </input_field_value> <input_field_value> <field_name>tenant_id</field_name> <value>150</value> </input_field_value> <update_field_value> <context_variable_name>context_rec </context_variable_name> <value_ref> <service_output_field_name>recommendation </service_output_field_name> </value_ref> </update_field_value> <update_field_value> <context_variable_name>context_material_cd </context_variable_name> <value_ref> <observation_field> <table_cd>event_observation</table_cd> <key_field_value> <field_name>measurement_type_cd </field_name> <value>201</value> </key_field_value> <field_name>material_cd</field_name> </observation_field> </value_ref> </update_field_value> <process_name>bpm_process</process_name> <param> <name>key2</name> <value>value2</value> </param> <param> <name>key1</name> <value>value1</value> </param> </bpm_invocation> </remote_invocation_action> </remote_invocation_configuration> </adapter_configuration> </step> 178

179 Scoring Step To create a new scoring step for the orchestration, select the Scoring Step from the drop-down in the Add button. All selectors will be nested under the created step. Initially, this will create a blank form with a default step name, which can be modified to be more descriptive: A host name and port should be provided when using a remote process. To add other process configurations, check the Include additional configuration check box to display the panel: You can specify the handler context variable to be used during the orchestration. 179

180 For example, the step defined above would produce the following XML when exported: <step step_name="scoring Step"> <adapter_class>com.ibm.analytics.foundation.adapter.scoring.scoringadapter </adapter_class> <adapter_configuration xsi:type="ns3:remote_service_adapter_configuration" enablement_handler_context_variable_name="scoring_context_handler"> <host_name> <port_number>9080</port_number> </adapter_configuration> </step> 180

181 Event and Observation Profile Update The event and observation profile updates are managed on the the Selector form, nested under the Scoring Step in the Add button drop-down. Initially, this will create a blank form with a default selector name, which can be modified to be more descriptive: Event Selector To add an event selector, begin by clicking the Add an event selector hyperlink. This will add a new table to be able to select events based on specified event field values. To enter values, click the <click to specify a value> hyperlink to launch a dialog to set constant values. Enter the value, and then click Ok to place the value in the table. 181

182 For example, the XML below would be created for the event selector defined above: <remote_invocation_configuration name="selector"> <event_selector> <event_field_value> <field_name>event_type_cd</field_name> <value>measurement</value> </event_field_value> </event_selector> </remote_invocation_configuration> Note: If an Event was not associated and saved for the parent orchestration, then the Event Selector will display a reminder to set this value. This can be done by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. Observation Selector 182

183 To add an observation selector, begin by clicking the Add an observation selector hyperlink. This will add a new table so that you can select an event's observations based on specified field values. Multiple observations can be used for selecting by clicking the Add an observation selector hyperlink to create additional panels as needed. To enter values, click the <click to specify a value> hyperlink to launch a dialog to set constant values. Enter the value, and then click Ok to place the value in the table. 183

184 For example, the XML below would be created by the export of the observation selector above: <remote_invocation_configuration name="selector"> <observation_selector table_cd="event_observation"> <observation_field_value> <field_name>value_type_cd</field_name> <value>actual</value> </observation_field_value> <observation_field_value> <field_name>measurement_type_cd</field_name> <value>rpm</value> </observation_field_value> </observation_selector> </remote_invocation_configuration> Note: If an Event was not associated and saved for the parent orchestration, then the Observation Selector will display a reminder to set this value. This can be done by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. A list of available observations will now be provided, based on the specified Event. 184

185 Invocation Configuration To create a new invocation configuration, select the Invocation Configuration from the drop-down in the Add button. All selectors and invocations will be nested under the created configuration. Initially, this will create a blank form with a default invocation configuration name, which can be modified to be more descriptive: Profile Selector To create a new Profile Selector, select Profile Selector from the drop-down in the Add button. These selectors will supply read-only values to be used by the sibling invocations. Initially, this will create a blank form with a default selector name, which can be modified to be more descriptive. This name will serve as the reference alias for invocations. 185

186 Users can also select a shared profile selector created in the Reusable Components tab from the dropdown. This will apply the shared values to the profile row selector, and allow any values defined in the selector key fields table to override those defined in the shared profile selector. Clicking the shared profile selector's View button will allow the user to view the defined fields that can be overridden locally. These values can be changed in the Reusable Components tab to apply across all references. Note: Once a shared profile is selected, you must choose none of the shared references in the drop-down in order to select a different profile directly. Otherwise, the profile drop-down field will be locked. 186

187 To specify a field value, select the <click to specify a value> hyperlink button to launch a dialog containing the valid value types available. The following options are provided: Value Type Explanation Constant A valid constant value. Context Variable A valid context variable name that will be used. Event Field The orchestration's selected event field value will be used for matching. The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. Observation Field The value from the chosen observation field name in the selected observation will be used for matching. The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used for matching in the profile. 187

188 Note: The following message may be shown on selecting an Observation Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. After clicking Ok in dialog to specify values for the profile row selector form, the table will be filled with a summary of the values: For example, the XML below would be created by the export of the profile row selector above: <step step_name="scoring Step"> <adapter_class>com.ibm.analytics.foundation.adapter.scoring.scoringadapter</adapter_class> <adapter_configuration xsi:type="ns3:remote_service_adapter_configuration" enablement_handler_context_variable_name="scoring_context_handler">... <remote_invocation_configuration name="selector">... <remote_invocation_action name="invocation Configuration"> 188

189 </step> <profile_row_selector alias="rpm"> <shared_selector_cd>resource_kpi</shared_selector_cd> <key_field_value is_value_required="false"> <profile_field_name>profile_variable_cd </profile_field_name> <value>rpm</value> </key_field_value> </profile_row_selector> </remote_invocation_action> </remote_invocation_configuration> </adapter_configuration> 189

190 Scoring Invocation To create a new BPM invocation, select BPM Invocation from the drop-down in the Add button. Invocations will be run in the relative order that they are defined, using values supplied from other defined read-only profile selector siblings. Initially, this will create a blank form: Start by selecting one of the defined services from the Service to invoke drop-down. These are created in the Services tab if edits to existing instances or new services are required. On selection, the table of inputs and updates will be refreshed for assigning mappings: 190

191 The scoring request ID can be specified and a table is provided for assigning input row parameters to a specific table. To designate a table, click the <click to specify a table name> hyperlink to type the table name. To enter any required names or values as input row parameters, simply click the <click to specify request input row parameters> hyperlink to launch a dialog. Multiple input row parameters can be defined. Then click either the <click to specify a key> or <click to specify a value> hyperlinks to begin typing the constant values. 191

192 Users are also given the option to use an event to record the results with the appropriate configuration: Users can select a shared service update created in the Reusable Components tab from the drop-down. This will apply the shared values to the update, and allow any values defined in the selector key fields table to override those defined in the shared service update. To view the current assignments, simply click the View button next to the shared update drop-down: To overwrite or create new local input values for the invocation, select the <click to specify a value> hyperlink button to launch a dialog containing the valid value types available. The following options are provided: Value Type Explanation Constant A valid constant value. Context Variable A valid context variable name that will be used. Event Field The orchestration's selected event field value will be used for matching. The drop-down will be populated with a list of the valid event fields. 192

193 Note: The following message may be shown on selecting an Event Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. Observation Field The value from the chosen observation field name in the selected observation will be used for matching. The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used. Note: The following message may be shown on selecting an Observation Field. 193

194 This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. Read-Only Profile Field The selected read-only profile's current field value will be used. The Select an alias drop-down field will be filled with all the read-only selectors that can be used for providing values. Once an alias is selected, the Select a profile field drop-down will be populated with all valid profile fields. Note: The following message may be shown on selecting a Read-Only Profile Field. This can be fixed by creating new Profile Selector as a sibling of the current invocation. Refer to the Profile Selector section on how to create new selectors. 194

195 To overwrite or create new local output values for the invocation update, select the <click to specify a value> hyperlink button in the Destination column to launch a dialog for defining multiple mappings. Multiple output destinations can be specified for updates using the produced service values. The following value type options are provided when clicking the <click to specify a value>: Value Type Explanation Context Variable A valid context variable name that will be used. Event Field The orchestration's selected event field value will be used for matching. The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. This can be fixed by navigating to the event record configuration on the form, checking the Use an event to record the results check box, and selecting a generated event type from the Event field drop-down, as shown below. Observation Field The value from the chosen observation field name in the selected observation will be used for matching. 195

196 The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used. Note: The following message may be shown on selecting an Observation Field. This can be fixed by navigating to the event record configuration on the form, checking the Use an event to record the results check box, and selecting a generated event type from the Event field drop-down, as shown below. 196

197 Additional mappings can be made after an invocation has been run. These mappings are specified in the table located at the bottom of the form. To choose a destination update field value, select the <click to specify a value> hyperlink in the Destination column. Destination types include Context Variables, Event Fields, and Observation Fields, exactly as described above. To choose an input value to use for assignment, select the <click to specify a value> hyperlink in the Value column. The following input options are provided: Value Type Explanation Constant A valid constant value. Event Field The orchestration's selected event field value will be used for matching. The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. Observation Field The value from the chosen observation field name in the selected observation will be used for matching. The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used. 197

198 Note: The following message may be shown on selecting an Observation Field. This can be fixed by navigating to the root orchestration, selecting an event table, and then saving the orchestration, as shown below. 198

199 For example, performing an export on the invocation below would produce the following XML during export. 199

200 <step step_name="scoring Step"> <adapter_class>com.ibm.analytics.foundation.adapter.scoring.scoringadapter </adapter_class> <adapter_configuration xsi:type="ns3:remote_service_adapter_configuration" enablement_handler_context_variable_name="scoring_context_handler">... <remote_invocation_configuration name="selector">... <remote_invocation_action name="invocation Configuration">... <scoring_invocation service_cd="ihs" name="scoring Invocation"> <input_field_value> <field_name>location_id</field_name> <value_ref> <event_field_name>location_cd</event_field_name> </value_ref> </input_field_value> <input_field_value> <field_name>tenant_id</field_name> <value>150</value> </input_field_value> <update_field_value> <context_variable_name>context_material_cd </context_variable_name> <value_ref> <observation_field> <table_cd>event_observation</table_cd> <key_field_value /> <key_field_value> <field_name>value_type_cd </field_name> <value>201</value> </key_field_value> <key_field_value /> <key_field_value /> <field_name>material_cd</field_name> </observation_field> </value_ref> </update_field_value> <update_field_value> <context_variable_name>context_rec</context_variable_name> <value_ref> <service_output_field_name>recommendation </service_output_field_name> </value_ref> </update_field_value> <score_request_id>scoring_req_id</score_request_id> <input_table> <input_table_name>customer</input_table_name> <request_input_row> <input> <name>bp_dias</name> <value>90</value> </input> <input> <name>bp_sys</name> <value>120</value> </input> </request_input_row> </input_table> </scoring_invocation> </remote_invocation_action> </remote_invocation_configuration> </adapter_configuration> </step> 200

201 Reusable Components Tab The reusable components tab is where shared definitions can be defined for use in existing orchestration components. Adding New Nodes and Panels The list of defined configurations are represented as nodes in the tree for the reusable components tab. Clicking a node will display the contents of the configuration and allow the user to edit and delete the node. Clicking the Add button will reveal the possible nodes that can be added based on the user's current node selection. The hierarchy of possible nodes that can be added to the reusable components tree are as follows: Note: These options are contextual, and therefore will only be displayed when the currently selected node's sibling or parent is of a valid type, as shown above. 201

202 Profile Selector To create a new shared profile selector, select Profile Selector from the drop-down in the Add button. These selectors will supply read-only values. Initially, this will create a blank form. To assist in making input value assignments for the selector, the shared profile selector allows choosing an optional incoming event, an event observation, and a required profile to load the appropriate fields. All events and profiles are available for construction, while observations will be filtered based on the chosen event. Note: Assigning specific event, observation, or profile attributes to the shared profile selector assumes that these will be available in each profile selector context that the shared profile selector is used. Be sure to check mappings where referenced for correct assignments. Otherwise, provide local overrides to fix the issue. To specify a field value, select the <click to specify a value> hyperlink button to launch a dialog containing the valid value types available. The following options are provided: Value Type Explanation Constant A valid constant value. Context Variable A valid context variable name that will be used. Event Field The orchestration's selected event field value will be used for matching. 202

203 The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. This can be fixed by navigating to the shared profile selector form and selecting an Incoming Event table from the drop-down. Observation Field The value from the chosen observation field name in the selected observation will be used for matching. 203

204 The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used for matching in the profile. Note: The following message may be shown on selecting an Observation Field. This can be fixed by navigating to the shared profile selector form and selecting an Incoming Event table from the drop-down. Selected Observation Field The value in the chosen observation selector field will be used for matching. The drop-down will be populated for each of the valid observation selector fields. 204

205 Note: The following message may be shown on selecting a Selected Observation Field. This can be fixed by navigating to the shared profile selector form and selecting an Observation Selector from the drop-down. After clicking Ok in the dialog to specify values for the shared profile row selector form, the table will be filled with a summary of the values: 205

206 For example, the XML below would be created by the export of the shared profile row selector above: <shared_profile_row_selector shared_selector_cd="resource_profile_sample" table_cd="resource_profile" incoming_event_table_cd="event" selected_observation_table_cd="event_observation"> <key_field_value is_value_required="false"> <profile_field_name>event_code</profile_field_name> <value_ref> <selected_observation_field_name>event_code</selected_observation_field_name> </value_ref> </key_field_value> <key_field_value is_value_required="false"> <profile_field_name>tenant_cd</profile_field_name> <value_ref> <event_field_name>tenant_cd</event_field_name> </value_ref> </key_field_value> <key_field_value is_value_required="false"> <profile_field_name>location_cd</profile_field_name> <value_ref> <event_field_name>location_cd</event_field_name> </value_ref> </key_field_value> <key_field_value is_value_required="false"> <profile_field_name>value_type_cd</profile_field_name> <value_ref> <selected_observation_field_name>value_type_cd</selected_observation_field_name> </value_ref> <key_field_value> </shared_profile_row_selector> 206

207 Calculation Group To create a new shared service update, select Service Update from the drop-down in the Add button. These selectors will supply update configurations that can be used in invocations. Initially, this will create a blank form. To assist in making value assignments for the shared calculation group, you can choose an optional incoming event, observation selector, or profile template. Note: Assigning specific components to the shared service update assumes that these will be available in the invocation context that the shared profile selector is used. Be sure to check mappings where referenced for correct assignments. Otherwise, provide local overrides to fix the issue. To define new calculations, start by clicking the Add Calculation hyperlink at the bottom of the page. These calculations can be reordered relatively. The type of calculation is selected in the Calculation to perform drop-down, while the unique invocation name, used for identifying a calculation, is specified in the Invocation Name field. Note: To delete a locally created calculation, click the Delete hyperlink located beneath the panel. To specify a calculation input value, select the <click to specify a value> hyperlink button in the Value column to launch a dialog containing the valid value types available. The following options are provided: Value Type Explanation Constant A valid constant value. Context Variable A valid context variable name that will be used. Event Field The orchestration's selected event field value will be used. The drop-down will be populated with a list of the valid event fields. 207

208 Note: The following message may be shown on selecting an Event Field. This can be fixed by navigating to the Shared Calculation Group form and selecting an Incoming Event from the drop-down. Observation Field The value from the chosen observation field name in the selected observation will be used. The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used for matching in the profile. 208

209 Note: The following message may be shown on selecting an Observation Field. This can be fixed by navigating to the Shared Calculation Group form and selecting an Incoming Event from the drop-down. Selected Observation Field The value in the chosen observation selector field will be used. The drop-down will be populated for each of the valid observation selector fields. Note: The following message may be shown on selecting a Selected Observation Field. This can be fixed by navigating to the parent Shared Calculation Group form and selecting an Observation Selector from the drop-down. If one is not displayed, an Incoming Event must be chosen for filtering. 209

210 Selected Profile Field The selected profile's current field value will be used. The drop-down will be populated with a list of the valid profile fields. Note: The following message may be shown on selecting a Selected Profile Field. This can be fixed by navigating to the Shared Calculation Group form and selecting a Profile Template from the drop-down. Read-Only Profile Field The selected read-only profile's current field value will be used. 210

211 The Select an alias drop-down field will be filled with all the read-only selectors that can be used for providing values. Once an alias is selected, the Select a profile field drop-down will be populated with all valid profile fields. Similarly, to specify a calculation output value, select the <click to specify a value> hyperlink button in the Destination column to launch a dialog for defining multiple mappings. Multiple output destinations can be specified for updates using the produced calculation values. The following value type options are provided when clicking the <click to specify a value>: Value Type Explanation Context Variable A valid context variable name that will be used. Selected Profile Field The selected profile's current field value will be used. The drop-down will be populated with a list of the valid profile fields. 211

212 Note: The following message may be shown on selecting a Selected Profile Field. This can be fixed by navigating to the Shared Calculation Group form and selecting a Profile Template from the drop-down. Note: These mappings can be deleted by clicking the trash can icon to the right of the definition. Finally, each calculation invocation can specify additional updates by checking the Additional updates after the calculation invocation check box. Constant values can be used for input Value, while Constant Variable and Selected Profile Field can be used for Destinations. These are the same as described earlier. After clicking Ok in the dialog to specify values for the shared calculation group form, the table will be filled with a summary of the values: 212

213 For example, the XML below would be created by the export of the shared calculation group above: <shared_calculation_invocation_group shared_group_cd="event_profile_assign_sample" template_profile_table_cd="event_profile" incoming_event_table_cd="event" selected_observation_table_cd="event_observation"> <calculation_invocation invocation_cd="assign" calculation_cd="assign"> <input_field_value> <field_name>measure_value</field_name> <value_ref> <selected_observation_field_name>measurement </selected_observation_field_name> </value_ref> </input_field_value> <update_field_value> <profile_field_name>material_cd</profile_field_name> <value>150</value> </update_field_value> <update_field_value> <profile_field_name>measurement_value</profile_field_name> <value_ref> <calculation_output_field_name>field_to_assign </calculation_output_field_name> </value_ref> </update_field_value> </calculation_invocation> <calculation_invocation invocation_cd="assign_string" calculation_cd="assign_string"> 213

214 <input_field_value> <field_name>input_text</field_name> <value_ref> <selected_observation_field_name>observation_text </selected_observation_field_name> </value_ref> </input_field_value> <update_field_value> <profile_field_name>measurement_text</profile_field_name> <value_ref> <calculation_output_field_name>field_to_assign </calculation_output_field_name> </value_ref> </update_field_value> </calculation_invocation> </shared_calculation_invocation_group> 214

215 Service Update To create a new shared service update, select Service Update from the drop-down in the Add button. These selectors will supply update configurations that can be used in invocations. Initially, this will create a blank form. To assist in making value assignments for the service update, the shared service update allows choosing an optional incoming event, generated event, and service. Note: Assigning specific events or services to the shared service update assumes that these will be available in the invocation context that the shared profile selector is used. Be sure to check mappings where referenced for correct assignments. Otherwise, provide local overrides to fix the issue. To specify an update value, select the <click to specify a value> hyperlink button in the Value column to launch a dialog containing the valid value types available. The following options are provided: Value Type Explanation Constant A valid constant value. Event Field The orchestration's selected event field value will be used. The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. 215

216 This can be fixed by navigating to the shared service update form and selecting an Incoming Event table. Observation Field The value from the chosen observation field name in the selected observation will be used. The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used. Note: The following message may be shown on selecting an Observation Field. 216

217 This can be fixed by navigating to the shared service update and selecting an Incoming Event table. 217

218 Selected Service Output Field The service output field whose field will be redirected to the destination. The drop-down will be populated with all the valid service output fields. Note: The service output field may not be initially listed. If missing, this can be fixed by navigating to the shared service update form and selecting a Service Name from the drop-down. 218

219 To specify the update value destination, select the <click to specify a value> hyperlink button in the Destination column to launch a dialog containing the valid value types available. The following options are provided: Value Type Explanation Context Variable A valid context variable name that will be used. Event Field The orchestration's selected event field value will be used. The drop-down will be populated with a list of the valid event fields. Note: The following message may be shown on selecting an Event Field. This can be fixed by navigating to the shared service update and selecting an Generated Event Type table. Observation Field The value from the chosen observation field name in the selected observation will be used. 219

220 The observation Table Cd drop-down will be populated with the list of observations under the chosen event. Values can be specified in the observation field table to select a specific observation. The selected Field Name value will then be used. Note: The following message may be shown on selecting an Observation Field. This can be fixed by navigating to the shared service update and selecting an Generated Event Type table. 220

221 After clicking Ok in the dialog to specify values for the shared service update form, the table will be filled with a summary of the values: For example, the XML below would be created by the export of the shared service update above: <shared_service_update shared_update_cd="event_to_event_sample" incoming_event_table_cd="event" generated_event_table_cd="event" service_cd="hs"> <update_field_value> <context_variable_name>hs_context</context_variable_name> <value_ref> <service_output_field_name>hs</service_output_field_name> </value_ref> </update_field_value> <update_field_value> <event_field_name>operator_cd</event_field_name> <value_ref> <event_field_name>operator_cd</event_field_name> </value_ref> </update_field_value> <update_field_value> <event_field_name>serial_no</event_field_name> <value_ref> <event_field_name>serial_no</event_field_name> </value_ref> </update_field_value> <update_field_value> <event_field_name>tenant_cd</event_field_name> <value_ref> <event_field_name>tenant_cd</event_field_name> </value_ref> </update_field_value> <update_field_value> <event_field_name>source_system_cd</event_field_name> <value_ref> <event_field_name>source_system_cd</event_field_name> 221

222 </value_ref> </update_field_value> <update_field_value> <observation_field> <table_cd>event_observation</table_cd> <key_field_value> <field_name>event_code</field_name> <value>10</value> </key_field_value> <field_name>event_code</field_name> </observation_field> <value_ref> <event_field_name>event_type_cd</event_field_name> </value_ref> </update_field_value> <update_field_value> <event_field_name>operator_na</event_field_name> <value_ref> <event_field_name>operator_na</event_field_name> </value_ref> </update_field_value> <update_field_value> <event_field_name>model</event_field_name> <value_ref> <event_field_name>model</event_field_name> </value_ref> </update_field_value> </shared_service_update> 222

223 Export Tab The export tab allows Solutions to export the data model, services, calculations, and orchestrations defined via the Analytics Solutions Foundation web UI to file system artifacts solution definition and orchestration definition files. The tab facilitates the selection of Analytics Solutions Foundation artifacts to be exported to the file system. It allows the user to select to export any combination of the following: 1) The solution definition 2) One or more orchestrations Clicking on the export tab opens up a form where the export configuration can be specified. The form contains a field to specify the output directory, an option to specify if the solution definition needs to be exported, and a field to specify the file name of the exported solution definition. In addition, for each orchestration, it presents an option to configure if the orchestration needs to be exported and if so, a field to specify the name of the exported orchestration definition XML file. Initially, the tab opens up as this: The following fields should be configured by the user on the export form: Output Directory: The path to the file system directory where artifacts need to be exported. Once exported, the solution definition XML is placed directly within this directory. The orchestrations are placed within a sub-directory named orchestrations within this output directory. Export Solution Definition option: If this option is enabled, the solution definition is exported to the file system. Filename: If the Export Solution Definition option is selected, the filename field can be edited by the user. By default, the field value contains the default file name solution_definition.xml. The name can be changed by the user. 223

224 Select Orchestrations to Export: This shows a table that displays the list of orchestrations (orchestration name). Besides each orchestration, there is a check box to select it for export. The filename field is then used to specify the name of export orchestration XML file. As an orchestration XML can contain a number of orchestrations, a same filename can be specified for more than one orchestration. This would mean all such orchestrations would be present in the same XML file. In the example below, the orchestrations named pmq.feature and pmq.inspection are selected to be exported, and they are exported into two different XML files. Validation Once the export form is configured, the solution definition and selected orchestrations can be validated. The validation feature verifies the solution definition and checks that selected orchestrations do not contain semantic errors. It validates the constraints on the content of the XML files, and ensures the reference values are defined. 224

225 The validation can be triggered by clicking the Validate button. A dialog with the validation result appears as below: The dialog contains expandable sections for the solution definition and for each orchestration. The corresponding section can be clicked to expand and view the success/failure messages. The sections colored red indicate validation failure, and a green section indicates a validation success. Export The export button can be clicked to start the export process. The export process validates the solution definition and each orchestration before it exports them. In the event of a validation failure, the export process completes with warnings. In the event of such warnings, the validate button has to be clicked and the validation errors have to be resolved prior to exporting. Save The export tab also provides the ability to store the specified export configuration. This is useful when adhoc exports need to be done without impacting the saved export configuration. Once the export configuration is saved, the export form is pre-filled with this configuration whenever the user clicks on the export tab. The same can be edited and optionally saved back. 225