IBM DECISION OPTIMIZATION CONNECTOR IBM COGNOS TM1 FOR IBM PLANNING ANALYTICS AND. IBM Analytic Solutions by. v

Transcription

1 IBM DECISION OPTIMIZATION CONNECTOR FOR IBM PLANNING ANALYTICS AND IBM COGNOS TM1 v July 11, 2018 IBM Analytic Solutions by

2 Contents INTRODUCTION... 3 PRODUCT REQUIREMENTS... 3 INITIAL CONSIDERATIONS PRELIMINARY STEPS... 4 {PA/TM1} SERVER CONFIGURATION DOWNLOADING THE {CONNECTOR} INSTALLING THE {CONNECTOR} CONFIGURING THE {CONNECTOR} SETTING {PA/TM1} SERVER INFORMATION ENCRYPTING CREDENTIALS: PASSWORD AND API KEY MAPPING DATA ELEMENTS BETWEEN {PA/TM1} AND {DO} MODELS HOW TO MAP {PA/TM1} STRUCTURES TO {DO} TUPLES ROLE OF MAPPED {PA/TM1} OBJECTS IN {DO} DO OBJECTS DIMENSION DO-TM1 MAPPING CUBE VIEW MAPPING CUBE SUBSET MAPPING CUBE BUILDING {PA/TM1} SUBSETS AND VIEWS {PA/TM1} SUBSETS {PA/TM1} VIEWS VERSIONS BUILDING THE {DO} MODEL RUNNING THE {CONNECTOR} USING TURBO INTEGRATOR (TI) PROCESSES QUEUE MANAGEMENT (FOR MULTIPLE USERS) ADD A DO CLOUD RUN TO THE QUEUE CHORE RUNS EACH QUEUED OPTIMIZATION TROUBLESHOOTING Run Status in the DO-TM1 Control Cube: Connector Log Optimization Log...21 APPENDIX A: EXAMPLE MODELS AND DATA A.1. Example {PA/TM1} Model...23 A.2. Example {DO} Model...23 A.3. EXAMPLE BUSINESS PROBLEM APPENDIX B: OPTIMIZING MULTIPLE DO MODELS FROM ONE TM1 MODEL APPENDIX C: FINAL {PA/TM1} SUBSETS AND VIEWS FOR THE EXAMPLE

3 3 Introduction Planning and optimization are used to develop and identify the best solutions and scenarios for many different business problems. Some of the most frequent and valuable uses include: 1) Supply Chain Management: Sales & Operations Planning (including Inventory Management, Distribution Planning/Design, Production Planning/Scheduling, and Capacity Planning) 2) Workforce Management (including Workforce Planning and Workforce Scheduling) 3) Sales Performance Management (including Sales Territory Management ) 4) Revenue Management / Pricing Planning (including Price Planning) 5) Strategic Planning/Portfolio Management (including Capital Project Planning, Assortment Management, and Product Planning) The IBM Decision Optimization Connector for IBM Planning Analytics and IBM Cognos TM1 is designed to solve and return optimized solutions using either IBM Decision Optimization (DO Cloud or IBM CPLEX Optimization Studio on-premises) from data held in either IBM Cognos Planning Analytics or in IBM Cognos TM1. For simplicity in this document, we shall collectively use {PA/TM1} for IBM Planning Analytics and for IBM Cognos TM1, {DO} for IBM CPLEX Optimization Studio and for DO Cloud, and {Connector} for the IBM Decision Optimization Connector for IBM Planning Analytics and IBM Cognos TM1 The {Connector} transmits data from {PA/TM1} to {DO}, and returns the optimized results to {PA/TM1}. This documentation explains setup, configuration and runtime steps. NOTE: Cloud Installation Tag: Please watch for this tag, PA on-cloud, for any unique steps when configuring the {Connector} for use with IBM Planning Analytics deployed on-cloud. Product Requirements Planning and Forecasting using {PA/TM1}, either: o On-premises versions: IBM Planning Analytics (2.0 or greater) IBM Cognos Configuration IBM Cognos TM1 Server IBM Cognos TM1 Architect or IBM Cognos TM1 Performance Modeler OR IBM Cognos TM1 ( or greater) IBM Cognos Configuration IBM Cognos TM1 Server IBM Cognos TM1 Architect or IBM Cognos TM1 Performance Modeler o On-Cloud versions: IBM Planning Analytics (2.0 or greater) Decision Optimization {DO} Products and Services: o For Model building: IBM CPLEX Optimization Studio (12.2 to 12.7) o For Execution: IBM Decision Optimization on Cloud [DO Cloud]

4 4 Java Runtime Environment (JRE ) 7.0 or higher installed. We recommend you use the version from IBM that is included with the {PA/TM1} installation. The On-Cloud version uses that version. Initial Considerations We have assumed that you have already built or will build matching models in {PA/TM1} and {DO}. This documentation does not cover basic model building in {PA/TM1} nor in {DO}. This documentation does help an experienced user to map the variables, metadata, and data to enable the {Connector} to move data between {PA/TM1} and {DO}, to use information in a {PA/TM1} model as the data source for a {DO} model, to execute the {DO} model and to return the optimized results in the {PA/TM1} model. 0. Preliminary Steps {PA/TM1} Server Configuration Follow these support instructions to enable your {PA/TM1} model to use the TM1 REST API. Be sure to specify the HTTPPortNumber parameter to the {PA/TM1} server configuration file, as shown in Figure 0.1. Depending on your situation, these instructions (and restarting the {PA/TM1} server) may be the only step(s) required. Figure 0.1: Example of additional {PA/TM1} REST API parameter in tm1s.cfg file. Security Modes Configuration: The {Connector} supports {PA/TM1} servers configured for IntegratedSecurityMode = 1, 4, or Downloading the {Connector} The {Connector} will reside on-premises or on-cloud, matching the location of your PA/TM1 installation. For PA On-Cloud installation Download Cloud Files.zip from TM1 Decision Optimization Community page in IBM developerworks. Submit the following support request to IBM Planning Analytics Support to request enablement of the {Connector} in your PA Cloud environment Summary: Details: Requesting enablement of DO-PA Connector for PA on Cloud. We request enablement of the DO-PA Connector on the machine <your cloud machine s path>. As part of your confirmation and response from IBM Support, you will receive a Welcome Kit describing an automated Cognos Command Center user that has been added to the LDAP namespace for your cloud environment. Follow these instructions to add this automated user to each {PA/TM1} server you wish to enable for optimization via the {Connector}. This automated user requires PA/TM1 Administrator privileges, For PA On-Premises installation For an on-premises installation, download Connector Base RestAPI.zip from TM1 Decision Optimization Community page in IBM developerworks.

5 5 2. Installing the {Connector} For each TM1 Server you wish to enable for optimization via the {Connector}, complete these five steps: 1. Stop the {PA/TM1} server. 2. Extract (or copy) the DO-TM1_Connector folder from the appropriate zip file (that you downloaded from Section 1 above and shown with details in Figure 2.1 below), into the file system location of the {PA/TM1} server data files. 3. Copy all files from within the TM1_Objects folder into the {PA/TM1} server data files directory. 4. Confirm the files in Figure 2.1 are present in the data files directory. NOTE: Operating System Short Path Names: If the drive containing the server data files has short path names disabled, then the directory name cannot contain spaces. 5. Restart the {PA/TM1} server. Figure 2.1: Folder structure of the on-premises vs. on-cloud {Connector} zip files 3. Configuring the {Connector} Open the connector.properties file (see red box in Fig 3.1) in a text editor. Figure 3.1: Properties file location. The {Connector} configuration parameters are case-sensitive and should be configured as follows: Table 3.1: Configuration file parameters for the {Connector}. tm1.adminhost PA on-premises: This is specified in the tm1s.cfg file of the {PA/TM1} server as the AdminHost parameter. If blank, then this parameter equals localhost. The {Connector} only supports one Admin Host. PA on-cloud: Use the IP address for the data tier provided in your Welcome Kit.

6 6 tm1.adminport tm1.adminssl api.url cam.namespace DO.dll.path TM1 admin server port number for the REST API. The default value is 5895, unless you changed it per the REST API instructions. PA on-premises: True or False does the TM1 Admin Server use SSL? (T or F). The default value is F. PA on-cloud: Must be True URL for the DO Cloud API. Assigned with a DO Cloud subscription along with the API key. PA on-premises: Cognos Access Manager {CAM} Namespace for {PA/TM1} servers using IntegratedSecurityMode = 4 or 5. PA on-cloud:the namespace will be LDAP, since that is the namespace to authenticate the Cognos Command Center automated user. PA on-premises: Absolute path to the CPLEX DLL files. You can find these files in the path <CPLEX Optimization Studio install path>/opl/bin/x64_win64. The CPLEX Studio install folder is dependent on the type of installation. Users with {DO} Center can find a CPLEX_Studio folder in their installation location, which contains the above folders. Note: You must use the forward slash / to separate the folders rather than the backwards slash \ (see example below). If you are not using on-premises {DO} CPLEX, you may leave this blank. PA on-cloud: Not applicable After this step your connector.properties file should contain entries similar to this: tm1.adminhost=localhost tm1.adminport=5895 tm1.adminssl=f api.url= Cloud.ibmcloud.com/job_manager/rest/v1/ cam.namespace=ldap DO.dll.path=C:/DOC38/Developer/bin/x64_win64 On-Premises: Set the TM1 installation path Edit batch files credential_encryption.bat and do-tm1_connector.bat to specify the TM1 installation path. As shown in Figure 3.2 below, set the TM1_INSTALL_PATH to the default path of your TM1 installation. Figure 3.2: Batch file lines with TM1 installation path. On-Premises: Configure {DO} CPLEX on-premises Navigate to the path <CPLEX Optimization Studio install path>/opl/lib. Copy the oplall.jar file Navigate to the DO-TM1_Connector/lib folder in your {PA/TM1} server data files (see Figure 3.1) Paste the "oplall.jar" file for use by the {Connector}

7 7 4. Setting {PA/TM1} server information Logon to your {PA/TM1} server to populate the {Connector} configuration cubes. Configure the {Connector} with information about the {PA/TM1} server o Open Connector TM1 Server Information cube. The view is shown in Figure 4.1. o Specify the Server Name and TM1 User name Server Name TM1 User Name Name of the {PA/TM1} server. PA on-premises: We recommend that you create a {PA/TM1} user name that minimally provides admin access to the {PA/TM1} server. The associated password is encrypted in section 6. PA on-cloud: Your TM1 User Name will be the Cognos Command Center automated user name in the Welcome Kit. Figure 4.1: {PA/TM1} server information configuration 5. Encrypting Credentials: Password and API key The {Connector} requires the credentials for {PA/TM1} and for DO Cloud be encrypted and stored in the file system. The encryption step(s) are activated from a Turbo Integrator (TI) process: PA on-premises: Run the TI process for each TM1 Server you wish to enable for optimization via the {Connector}. PA on-cloud: Run the TI script just once for your cloud installation. Run the Encrypt Credentials TurboIntegrator (TI) process. At the start of the TI process, you will be prompted for these two credential parameters: 1) The {PA/TM1} password is mandatory 2) The DO Cloud API key is mandatory when using DO Cloud; otherwise, empty NOTE: Getting Credentials: The {PA/TM1} password is associated with the {PA/TM1} user name defined in Section 0, above. The DO Cloud API key is provided with a valid DO Cloud subscription. If you are only solving with on-premises {DO} CPLEX, then you do not need to input an API key. NOTE: Keeping credentials/changing credentials: Once you have successfully completed the Turbo Integrator (TI) process(es), the two credential parameters will be safely stored as encrypted information. The encrypting of credentials is only done again if either of the two credential parameters change.

8 8 6. Mapping Data Elements between {PA/TM1} and {DO} Models The {Connector} lets you map metadata so it knows how to pass data between {PA/TM1} and {DO}. {PA/TM1} structures (i.e. cube views and dimension subsets) need to be passed to {DO} structures (tuples). The {Connector} requires one control dimension and three mapping cubes and views to map a {PA/TM1} model to a single {DO} model: DO Objects: Dimension containing the names of tuple variables in the {DO} model. See section 6.3. DO-TM1 Mapping: The cube and view name for general mapping. Specifies names of {PA/TM1} objects subsets/dimensions and views/cubes that map to the {DO} variables. See section 6.4. View Mapping: The cube and view name for details about each mapped {PA/TM1} cube view. Here you choose which dimensions in a view should be mapped to {DO} tuple fields, and which dimensions should have just a single element selected (single multi-dimensional slice). See section 6.5. Subset Mapping: The cube and view name for details about each mapped {PA/TM1} dimension subset. Here you set which dimension attributes should be mapped to {DO} tuple fields. See section How to Map {PA/TM1} structures to {DO} tuples Refer to Appendix A for complete information about the sample models illustrated here. In Figure 6.1, the left image shows a {DO} tuple structure (lines 20 24). This structure can be applied to any number of variables. In our sample example case, it is applied to the input variable demands (line 44). It contains three fields: product, component, and demand. The first two fields are keys, which are the same conceptually as primary keys in a database table or dimension members in a {PA/TM1} cube. Tuples in this context can be considered as flattened cube views. Figure 6.1: {DO} tuple structure and an example variable (left), corresponding cube view (right)

9 9 Figure 6.2: Visual mapping of a {PA/TM1} cube view to a {DO} tuple variable. {TM1/PA} View: demands Dimensions: Products DO-TM1 Mapping cube View Mapping cube {DO} variable: demands Tuple Fields: key string product Components key string component Version Demand Unmapped: Slice on one element for each dimension. float demand Values Table 6.1: Cube view data from Figure 6.1 in same format as {DO} tuple data. {PA/TM1}: Products (dimension) Components (dimension) demands (view data) gas nitrogen 1 gas hydrogen 3 gas chlorine 0 chloride nitrogen 1 chloride hydrogen 4 chloride chlorine 1 {DO}: product (tuple key field) component (tuple key field) demand (tuple non-key field) The mapping cubes described in the next section are used to record this information in Figure 6.2: which dimensions are mapped to which fields and, for unmapped dimensions, which elements should be sliced on (i.e. selected). For subsets within {PA/TM1}, attributes are mapped to the tuple fields in {DO} Role of Mapped {PA/TM1} Objects in {DO} In {PA/TM1}, cube views format data for analysis and planning, and for data movement. Dimension subsets in {PA/TM1} provide the context (or dimensionality) for the data. It is the same concept in {DO} models that are designed to map to {PA/TM1}. The contexts defined by the dimension subsets must be applied to all cube views that contain that dimension.

10 10 The following examples may help clarify the dimensionality and context for running optimizations. Example #1: I am a part of a company that manufactures 100 different products in 50 different production plants, and I have included optimization in this process. Both the Products and Plants dimensions in {PA/TM1} have subsets that are mapped to {DO}, and they contain the 100 products and 50 plants. Some of the data from cube views in my {PA/TM1} model that are mapped to {DO} are: the demand (the number of units of each product my company expects to sell) for each week, and the capacity of each plant in number of units of each product. Typically, I want to optimize over all 100 products and all 50 plants, since in some ways they are all interdependent. However, I ve received a request from a regional manager to run the optimization for only the 5 plants that they manage. This means I need to reduce my subset of plants from 50 to 5. I will also need to make the same changes in any views that contain the Plants dimension. Since plants are not a part of the demand, I don t need to change it there, but Plants is a part of context of the capacity. Within that cube view I need to change the plants to the 5 specified, or else the {Connector} will give an error if I leave it with all 50 included, because I m including plants that are outside the context of the 5 plants provided by the subset. Example #2: Consider a dimension Time with 52 weeks, W01 through W52. We are running an optimization problem for only the first 10 weeks: W01 through W10. Those 10 elements are being passed through a subset to {DO} as the time context for the optimization. One of the views being created has the Time dimension mapped to {DO}. Within this view, the Time dimension cannot contain any elements outside of the W01 through W10 range, or the {Connector} will give an error. The subset itself must be consistent with the cube views that also contain that dimension. The cube view does not need to have all those elements included, but any that are included must come from that subset. NOTE: Keeping Mapping Cubes Valid Do NOT move any dimensions in the mapping cube views that are being read by the Connector (i.e. the views defined in the DO-TM1 Mapping Management cube, the DO-TM1 Control cube and the DO-TM1 Mapping Management cube). Elements can be added in any of the dimensions, but not removed or hidden. The DO Objects dimension is a special case where only the {DO} variables should be added as elements. NOTE: Output to cube views: The {Connector} supports storing optimized data from {DO} using {PA/TM1} cube views, but does not support using dimension subsets.

11 DO Objects dimension Within {PA/TM1}, the DO Objects dimension is a list of the {DO} input and output variables. These variables must be put in the same order as they appear in the {DO} model as the mapping is positional within a DO tuple variable. The user must populate this dimension based on their own {DO} model. As an example, with the {DO} model in Figure A.2, the dimension would have the following elements: products components demands profits stocks productionout The {PA/TM1} element names MUST match the {DO} variable names. Since {DO} is case-sensitive, these elements must match the case as well DO-TM1 Mapping cube The cube (and view) name is DO-TM1 Mapping and it contains two dimensions: DO Objects and Mapping Parameters. The first dimension, DO Objects, is discussed in section 6.3. Rules can be added to the DO-TM1 Mapping cube, as shown in Figure 6.3 to simplify data entry in this cube. For example, automatically making the TM1 Name equal to the DO Objects element name. Figure 6.3 shows the mapping between {PA/TM1} and {DO} for the example models in Appendix A. {DO} variable names are on the Rows, mapping parameters are on the columns. Figure 6.3: Example DO-TM1 Mapping Cube Table 6.2: General DO-TM1 mapping parameters. DO Input/Output TM1 Type Is it an input or output variable in DO? {PA/TM1} View or Subset that the {DO} variable maps to. Currently only Views are supported as Output.

12 12 TM1 Name TM1 Parent Name *Data DO Field Type *Data DO Field Name Note: Name of the {PA/TM1} View or Subset. Name of the cube or dimension associated with the {PA/TM1} object. The DO data type of the field. In Figure 6.1, it is a float type. The name of the corresponding positional field in the DO tuple variable. * fields are only used for {PA/TM1} cube views. Example: Refer to Figure 6.1, lines 23 and 44, the {DO} tuple variable demands. It can be seen that demands is an input variable due to the ellipsis. Since demands requires data and is used in calculations, then the TM1 Type is a View. For simplicity the cube view name is set as the same as the {DO} object name. The name of the cube with this view is Market Demand. Since it is a View, the data field information needs to be completed as shown in Figure 6.4. In Figure 6.1, the numerical data in the view correspond to the tuple field float demand on line 23. This means the Data DO Field Type is float, and the Data DO Field Name is demand. This is also visualized in Figure 6.2, also, where Values are the view data View Mapping cube The cube (and view) name is View Mapping and it contains three dimensions: DO Objects, Dimension Number, and View Mappings. The Dimension Number dimension is used to map each dimension of a {PA/TM1} view separately, and supports cubes with as many as 10 dimensions. Modelers may add members to support cubes with more than 10 dimensions. The View Mappings dimension contains the measures for the view mapping information, which should not be changed.

13 13 Figure 6.4: Cube to define how views are mapped to {DO} tuples. Zeroes are suppressed. Table 6.3: View mapping parameters View Mapping Parameters Dimension Name DO Field? Explanation The name of the nth dimension in the cube, where n is the current Dimension Number element. Currently determined by {PA/TM1} rule. Asking if this dimension maps to a {DO} tuple field (Yes/No). TM1 Attribute DO Field Type DO Field Name TM1 Element Name Only used if DO Field? is Yes Asks which attribute should be mapped to the tuple field. Can choose Invariant Name or another attribute. Follow the attribute recommendations described in Subset Mapping section 6.6. The basic data type for the {DO} tuple field (int, float, string). This is found in the {DO} tuple structure definition. The name of the {DO} tuple field. Only used if DO Field? is No Name of the dimension element to select (slice on).

14 Subset Mapping cube The cube (and view) name is Subset Mapping and it contains three dimensions, similar to the View Mapping cube: DO Objects, Attribute Number, Subset Mappings. The Attribute Number dimension is used to map each attribute of a dimension in {PA/TM1} separately. Users can add more numbers to this dimension. The Subset Mappings dimension contains the measures for the mapping information. Figure 6.5: Cube defining how subsets are mapped to {DO} tuples. Zeroes are suppressed. The big difference between this cube and the View Mapping cube is that for views, every dimension must be addressed, whether mapped or unmapped. For subsets, the user only needs to list the attributes that are being mapped to {DO}, i.e. the attributes present in {DO} tuples. In the example above, there is only one field in each tuple, so the mapping information is added in the 1 Attribute Number dimension member. Table 6.4: Subset mapping parameters. TM1 Attribute Name DO Field Type DO Field Name The name of the attribute. See the note on what attributes to use below. The basic data type for the {DO} tuple field (int, float, string). This is directly in the {DO} tuple. The name of the {DO} tuple field. Subset Attributes Alias attributes are strongly recommended for text attributes. If text and numeric attributes map to key tuple fields in {DO}, they must contain unique values. It is also worth noting that if you use a rule to update attribute values within their respective }ElementAttributes cube, then those cells need to be fed as well. The example above is fairly straightforward with only one attribute each. In Figure 6.6 and Figure 6.7 is an example of mapping multiple attributes to a {DO} tuple. Due to the natural structure of dimensions, there is no need to have more than one key tuple field when passing a subset. Anything used as a key must be uniquely assigned for each element. Order of the attributes in the Subset Mapping cube does not matter.

15 15 Figure 6.6: Left: {DO} tuple structure and variable definition. Right: {PA/TM1} subset and corresponding attributes Figure 6.7: Subset mapping definition for more than one attribute. 7. Building {PA/TM1} Subsets and Views It is the {PA/TM1} modeler s responsibility to build the subsets and views {PA/TM1} Subsets Steps: 1) Create a subset of the name specified in the DO-TM1 Mapping cube. 2) Select the desired elements. We strongly recommend that you include only leaf elements, i.e., no consolidation elements. Perform consolidation calculations in {DO}, rather than using consolidated elements from {PA/TM1}. When the {Connector} returns solution data to {PA/TM1} from the optimization run, the results will only be stored in leaf elements. 3) Mapped dimension attributes must be created and populated. We recommend using alias attributes in DO Models over the use of invariant names to increase the cohesion between the two models. a. The {Connector} supports the Invariant Name as an attribute, but does not support any other attributes that are automatically generated by Cognos TM1 Performance Modeler (e.g. Index, Weight, etc.). 4) Subsets are primarily passed to {DO} to enumerate the dimensionality for calculations, the optimization itself, or constraints. Thus, they provide the context for the problem. Therefore include all elements that are required to optimize the model; if you don t, {DO} is likely to return an error.

16 {PA/TM1} Views Steps: 1) Create a view of the name specified in the DO-TM1 Mapping cube. 2) The only stipulation on where dimensions should be placed is that mapped dimensions (i.e. those that map to a {DO} tuple field) must be in the Rows or Columns. For consistency, as well as ease of use and best performance, we recommend placing unmapped dimensions in Context with the required element selected as the slice. Then put mapped dimensions on Rows and Columns to create a consumable view of the cube data. 3) For applying subsets within the view, use the recommendations above in section ) The {Connector} intentionally ignores null values in {PA/TM1}, requiring the {DO} model to populate tuple sets with any zero values needed, which can often be done using the subsets that are mapped in section 6.6. See section 8 for an example. 5) Some {PA/TM1} modelers may notice that there is almost enough information in the View Mapping cube to create the views automatically through TI processes. You may add members (e.g. Name of Subset) to the View Mappings measures dimension to enable automation of the view creation. 6) You can create objects called Planning Analytics Views in the Planning Analytics Workspace. However, because these views are not saved to the TM1 server, they are not currently usable by the {Connector}. Output DO Views It is important to note that the {TM1/PA} Views that receive any optimized data from {DO} will have their data overwritten. Thus, we strongly recommend adding a {TM1/PA} measure dimension element specifically for the optimized data returning from {DO}. For example, if planning Production, then create a measure dimension element called Optimized Production to ensure baseline scenario planning data is not overwritten Versions We expect that most planning and forecasting models in {PA/TM1} use versioning. Optimization will likely be an iterative process with different scenarios. We recommend retaining different optimization scenarios (input and output data) in different members in a Versions dimension. 8. Building the {DO} Model The {DO} model must be designed to work with both the {PA/TM1} model and the {Connector}. Input and output variables in the {DO} model (i.e. variables with data being transferred between {DO} and {PA/TM1}) can only be sets of tuples. The tuple structure can only contain primitive data types as fields: float,

17 17 int, or strings. More complex data types are not supported as tuple fields. Output variables must be defined in the post-processing block when using DO Cloud. When naming input and output variables, do not use case alone to distinguish between variables. {DO} is casesensitive, but {PA/TM1} is not. The {Connector} intentionally does not pass null values from {PA/TM1} to {DO}. The default null value in {PA/TM1} is zero. If the {DO} model requires zero values in variables, you will need to ensure that the {DO} model populates those zero values. For example, assume that we wish to send the time in weeks that is required to transport goods between a production plant and a distribution center (DC). From the New York plant to the Miami DC is one week, so the value of 1 is passed from {PA/TM1} to {DO}. However, from the Miami plant to the Miami DC it is zero weeks. This value of zero would not be passed to {DO}. After the input variable is defined in the {DO} model, the modeler would then add zero values for any of the DC and plant combinations that were not present. One way to populate the unspecified values would be to use an array, which automatically populates any unspecified values with zero. See Figure 8.1. Figure 8.1: Example of using an array to fill zeroes that are not passed from {PA/TM1}. ISSUE: Map two or more measures from one {PA/TM1} view into the same DO tuple. The {Connector} is only able to map a {PA/TM1} dimension (not individual elements) to a {DO} tuple field. For example, assume that there is a cube with two dimensions, a Plants dimension and a Measures dimension. This Measures dimension contains measures such as Working Days (weekly basis) and Hours per Day that the plant is operational. In the DO model, we would want three tuple fields: one for

18 18 the plant name, one for working days, and one for hours per day. However, the {Connector} does not allow us to simply map these separate elements from the Measures dimension to multiple tuple fields. SOLUTION: Use View with a Subset: See Figure 8.2. In this case, you would map the {PA/TM1} Measures dimension from the view (with a subset containing the two desired measures) to the measure tuple field in the Measure1 tuple structure. Then in the DO model, reformat it to the desired tuple structure (Measure2). The variable m1 in Figure 8.2 would be the input data from {PA/TM1}, and m2 would be the same data reformatted within the {DO} model to a more convenient tuple structure. Figure 8.2: Example of mapping multiple measures from PA to DO 9. Running the {Connector} using Turbo Integrator (TI) Processes The entry point for running the {Connector} with multiple users is the queue management system, described in section 10, specifically the DO Run History Update process. The Run Optimization Turbo Integrator (TI) process calls a batch file to run the {Connector} jar file (directly onpremises, or indirectly on-cloud). To run this TI process, you simply need a single parameter, the name of the {DO} model to be used, also found in the corresponding element name in the DO Models dimension. NOTE: Verify Success from the DO-TM1 Control cube, NOT from the TI dialog box. The TI process gives a perfunctory dialog message that tells you that it (the TI process) has completed successfully, regardless of the actual status of the optimization solution. For the status of your optimization run, please look at the Connector Run Status in the DO-TM1 Control cube (see Figure B.2). A successful optimization run will contain the value Successful in its status.

19 19 The Clean Up Optimization TI process is optional to delete the views and subsets specified in the mapping cubes. This is only recommended to use if you have also created an associated TI process to create the views and subsets prior to running the {Connector}. We generally recommend keeping the subsets and views as static as possible. This process also requires the element name in the {PA/TM1} DO Models dimension associated with the {DO} model. 10. Queue Management (for multiple users) The {Connector} is single-threaded, requiring our queue system to allow multiple users to submit requests to run the {Connector} while it may already be running. The queue system includes a chore, several processes, cubes, and dimensions. HINT: No queue needed for single user: If only one user is running the {Connector}, then the queue mechanism is not necessary, but is recommended. You may directly use the Run Optimization TI process, as discussed above in section Add a DO Cloud run to the queue It is recommended to create a cube where the user can select which model they want to run from a picklist of DO Model dimension elements. Then pass the model name and user name to the DO Run History Update process, which will add the model and user to the DO Run History cube, and indicate it is ready to run. Figure 10.1: DO Run History Update process parameter entry example Chore runs each queued optimization The chore DO Ready Run Chore will run as scheduled (default is every 5 minutes) after the chore is activated. It locates each of the queued runs in the DO Run History cube, and runs each (one at a time). The DO Queue Control cube is maintained by processes to detect when an optimization is already running. Figure 10.2: View for DO Queue Control cube.

20 20 The DO Run Control cube maintains who has run each model and shows what the next run will be for each user and model combination. See examples of these cube views in Figure Figure 10.3: View for DO Run Control cube. 11. Troubleshooting When troubleshooting, there are three logs that provide helpful information: DO-TM1 Control Cube Optimization Log Connector Log Run Status in the DO-TM1 Control Cube: After the {Connector} has been run, the DO-TM1 Control cube will show the Connector Run Status. The status can be any of the following values: 1. Empty (nothing in the cell): an error has occurred, and you should check the connector.log file. This indicates a problem with simple initialization of the {Connector} and potentially the connection to {PA/TM1}. 2. Running : an error occurred while the {Connector} was running. You should look for a failure in the Optimization Log when executing the DO Model. 3. Error : an error occurred while the {Connector} was running. Check the connector.log file, as this error can be caused by different things. 4. Successful : the {Connector} ran successfully and had no errors Connector Log When the {Connector} is run, a log file named connector.log is created and updated in the <TM1 Server data files>/do-tm1_connector directory. The connector log file contains informative error messages. As an example, if a {PA/TM1} View was not created, the error message at the end of the log file will tell the user that it cannot find a View. Any errors will generally show at the end of the log file. Note that there is a known defect in the {Connector} regarding encoding Chinese characters in the output.csv file used to load optimized data into TM1 cubes. When solving with an on-premises {DO} CPLEX engine, you may encounter an error in the connector.log file that says the DLL files cannot be found. Here are some possible reasons with suggested fixes: ISSUE: The DO.dll.path in the configuration file may not be pointing to the DLL files;

21 21 FIX: Add the correct path to the configuration file. ISSUE: The on-premises CPLEX does not return data. The files may not match the version of the oplall.jar file in the library; FIX: Check the connector.log file. It may contain the following error: ERROR [main] stderr: Native code library failed to load: ensure the appropriate library (opl1271.dll/.so) is in your path...it might also be related to missing Microsoft Visual C++ Redistributable Package. In this case, the DLL files may not match the version of the oplall.jar in the library path. Install the matching DLL file and jar file for your on-premises {DO} CPLEX installation. Another possibility, especially if the DLL files are being used separately from the original CPLEX install, is that the required Microsoft Visual C++ Redistributable is missing. Install that locally to solve the issue. Note the {Connector} compatibility statement with {DO} libraries in the Product Requirements section. ISSUE: Error during optimization. o Check connector.log file and optimization.log file for more details. These are located in the folder <TM1 server data files>\do-tm1_connector o There are several possibilities that need to be debugged: Mapping between TM1 and DO models is incorrect: Mapping Data Elements between {PA/TM1} and {DO} Models. If using DOcplexcloud, keep in mind that DOcplexcloud does NOT relax constraints and will not return values if it doesn t find an optimized solution within the constraints. Check the optimization.log file. Ensure that the DO model matches all the requirements in this document: Building the {DO} Model. o Note that you have the option to save the input data as files (JSON or.dat), and these files can then be used manually with DOcplexcloud or on-premises CPLEX to debug any mismatches between the DO model and the input data. ISSUE: Cannot login to TM1/PA. o Check that the cam.namespace parameter in the Connector s config.properties (Configuring the {Connector}) file is LDAP. o Check that the server name and username are spelled correctly in the Connector TM1 Server Information cube: Setting {PA/TM1} server information. o Check that the password for the user is spelled correctly (no spaces after it) when running the Encrypt Credentials TI process. o Check that the user has Admin access to TM1 in the security privileges. o Make sure in PA on Cloud that the automation user has been added correctly to TM1 using the AddClient function within a TI process Optimization Log The optimization.log file provides the details of the optimization from DO Cloud, including potential errors. The optimization.log file is found in these locations, depending on your installation, as follows:

22 22 If you solved via an on-premises {DO} CPLEX engine, there is no log file. If you solved via DO Cloud then the log file location depends on you {PA/TM1} installation: PA on Premises: You will find the log file in the lib folder location, the same one containing the {Connector} jar file where the {Connector} is installed. PA on Cloud: You will find the log file in the DO-TM1_Connector folder associated with your TM1 server. Support: As stated earlier in this documentation, we count on you to create and maintain your planning and optimization models. The {Connector} helps you to run optimization scenarios as part of your planning and forecasting process. IBM supports the {Connector} as part of the normal IBM support commitment to IBM products listed on page 3 of this document in the Product Requirements section. Issues with the {Connector} can be addressed to IBM Customer Support at We also value any other feedback about your use of the {Connector}, including ideas, tips, tricks and reviews and ask that you post them on our IBM developerworks community at: Licensed Materials Property of IBM Corp. Copyright IBM Corp The Samples are owned by IBM and are copyrighted and licensed, not sold. They are licensed under the terms of the International License Agreement for Non-Warranted Programs, which you accepted prior to downloading the content from IBM developerworks web site ( See the complete license terms at: By using the Samples, you agree to these terms. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates.

23 23 A.1. Example {PA/TM1} Model Appendix A: Example Models and Data Figure A.1: Cubes and dimensions of example {PA/TM1} business model. A.2. Example {DO} Model A good OPL reference is the IBM ILOG OPL Language Reference Manual. The {Connector} supports a single {DO} model (.mod) file when running an on-premises optimization, and either a single model file or an OPL project with multiple model files for DO Cloud. A revised example gas model is shown in Figure A.2. We have made minor changes to accommodate the {Connector}, such as changing all tuple fields to basic data types (int, float, string), the corresponding references to those tuple fields, and the output variable definition in the post-processing block. The original file is located in the path <CPLEX Optimization Studio install path>/opl/examples/opl/gas/gas.mod See section 8 for more guidelines on how to build a {DO} model to optimize data from {PA/TM1}.

24 Figure A.2: {DO} example gas model. 24

25 25 A.3. Example Business Problem Based on the {DO} example model in Figure A.2 above. Figure A.3: Diagram of example problem. Product Demand Component: Stock Demand Product 1 nitrogen: 50 1 gas 3 hydrogen: chloride chlorine: 40 1 Sale: $30 profit/unit Sale: $40 profit/unit Three components (nitrogen, hydrogen, and chlorine) are used to make two products (gas, chloride). Producing one unit of gas requires (demands) 1 unit of nitrogen, and 3 units of hydrogen. It nets $30 in profit per unit sold. Producing one unit of chloride requires (demands) 1 unit of nitrogen, 4 units of hydrogen, and 1 unit of chlorine. It nets $40 in profit per unit sold. With current stock levels of: 50 units of nitrogen, 180 units of hydrogen, and 40 units of chlorine, how many units of each product should be made to earn the highest profit?

26 26 Appendix B: Optimizing Multiple DO Models from one TM1 Model To be able to map a {PA/TM1} server to multiple {DO} models, all of the {PA/TM1} objects listed above in sections can be duplicated with different names. However, when building the mapping cubes, the user should keep in mind that all of the dimension structures (i.e. where they are in Rows and Columns) MUST be the same, and they MUST use all of the other same dimensions. For example, the user may create a dimension called DO Objects 2, and a corresponding DO-TM1 Mapping 2 cube. When creating this DO-TM1 Mapping 2 cube, the DO Objects 2 dimension must be on the Rows and the Mapping Parameters dimension must be on the Columns. There are several dimensions and cubes included to manage multiple DO models as well: 1) DO Models (dimension). This is the list of {DO} models with one element per DO model. The element names are up to the user s discretion. When running a model through {DO}, the name will be passed to the {Connector}. If only using one {DO} model, then only one element is needed. 2) DO-TM1 Mapping Management (cube and view name). This is where users provide the names of the mapping views/cubes and subsets/dimensions for each model. Object Mapping is the general mapping cube (covered in section 6.4), View Mapping is the mapping cube for views (section 6.5), Subset Mapping is the mapping cube for subsets (section 6.6), and DO Objects is the dimension for the list of {DO} variables (section 6.3). Figure B.1: DO-TM1 Mapping Management cube. Table B.1: Description of the Mapping Management parameters. Name Parent Name Name of the view or subset in {PA/TM1} Name of the cube or dimension in {PA/TM1}

27 27 3) DO-TM1 Control (cube). Manages the file associated with each {DO} model (Figure B.2) Figure B.2: Control cube for managing model related files. Relative paths shown. Table B.2: Control parameters for each {DO} model. All file paths can be absolute or relative to the {PA/TM1} data files directory. Model File Location Mapping File Location Save DO Input File? DO Input File Location DO data File Location Relax Labeled Constraints Optimization Method Save DO Output File? DO Output File Location Points to location of the {DO} model file. If Optimization Method = DO Cloud, then you can optionally enter the path of a folder here, where the folder contains an oplproject. The {Connector} will add all files from this folder and run the Default Run Configuration, so be sure it contains only files that are appropriate for OPL model types in DO Cloud. The mapping cube information will be converted to a json file and stored in this location. User selects whether to save the JSON-formatted data and/or.dat file input that is sent to {DO} paired with the {DO} model. If Save DO Input File? = Yes, then this is the path to the input JSON file. If Optimization Method = CPLEX-local and Save DO Input File? = Yes, then a.dat file will be saved in the specified location. If Optimization Method = CPLEX-local, then select whether to enable relaxing labeled constraints (Yes) or not (No). If using DO Cloud, no relaxation is available. For DO Cloud: The {Connector} will solve the optimization model on DO Cloud. For CPLEX-local: The {Connector} will use the on-premises CPLEX files you specify in the properties file (see section 3) to solve the {DO} model User selects whether to save the JSON-formatted results data that from the {DO} run. If Save DO Output File? = Yes then this is the path to the output JSON file.

28 28 Connector Run Status Status of the {Connector} s last run for this {DO} model. It is either empty (nothing in cell), Running, Error or Successful. See section 11 for details. NOTE: Size of Saved Input Files: Save the input file with a small amount of data, possibly for initial testing or debugging of the mapping and optimization. For best speed use of memory on production runs, we advise against saving large input files.

29 29 Appendix C: Final {PA/TM1} Subsets and Views for the Example Figure C.1: Subset for products {DO} input variable (dimensionality). Figure C.2: Subset for components {DO} input variable (dimensionality). Figure C.3: View for the demands {DO} input variable. Figure C.4: View for profits {DO} input variable. Based on the View Mapping, only the Profit values will be mapped to {DO}. Note that the {Connector} supports working subsets as well as named subsets.

30 30 Figure C.5: View for stocks {DO} input variable. Figure C.6: View for productionout {DO} output variable. The optimization has not been run yet, so no production values are available. Figure C.7: View for productionout {DO} output variable after the optimization was run. The next run of the DO {Connector} will replace these values.