LAE 5.1 Release Notes
Copyright THE CONTENTS OF THIS DOCUMENT ARE THE COPYRIGHT OF LIMITED. ALL RIGHTS RESERVED. THIS DOCUMENT OR PARTS THEREOF MAY NOT BE REPRODUCED IN ANY FORM WITHOUT THE WRITTEN PERMISSION OF. Disclaimer No representation, warranty or understanding is made or given by this document or the information contained within it and no representation is made that the information contained in this document is complete, up to date or accurate. In no event shall Lavastorm Analytics be liable for incidental or consequential damages in connection with, or arising from its use, whether Lavastorm Analytics was made aware of the probability of such loss arising or not. Page 1
Contents 1.0 2.0 3.0 4.0 5.0 6.0 7.0 Lavastorm Analytics Engine 5.1... 3 1.1 Overview... 3 1.2 Supported Platforms... 3 New Features and Enhancements... 3 2.1 Iteration... 3 2.1.1 2.1.2 2.1.3 2.1.4 Do While... 3 Do While Conditional... 3 Sleep... 4 Switch... 4 2.2 Silent Installation... 4 2.3 User-Hosted Update Server... 4 2.4 Automated Error Reporting... 4 2.5 Logistics Manager Enhancements... 4 2.6 Password Policy... 4 Lavastorm Analytics Library... 5 Compatibility Scanner... 5 Bug Fixes and Corrections... 5 Backwards Compatibility... 8 Known Issues... 8 7.1 Automated Error Reporting... 8 Page 2
1.0 Lavastorm Analytics Engine 5.1 1.1 Overview Release 5.1 of the Lavastorm Analytics Engine includes several new functional enhancements, bug fixes, and popularly-requested features. 1.2 Supported Platforms 2.0 LAE 5.1 is supported on the following operating systems: For the BRE client and Lavastorm Desktop: o Windows 8 (64-bit) o Windows 7 (32-bit and 64-bit) For the LAE Server: o Linux (x86; 32-bit and 64-bit) o Red Hat Enterprise Linux 5 o Red Hat Enterprise Linux 6 o Oracle Enterprise Linux 5 o Oracle Enterprise Linux 6 o SUSE Enterprise Linux Server 11 SP2 o Oracle Solaris o Solaris 11 (SPARC; 64-bit) o Solaris 10 (SPARC; 64-bit) o Windows Server 2008 New Features and Enhancements 2.1 Iteration The largest new feature of 5.1 is the ability to create loops over a section of a graph until a set condition is reached. The Do While, Do While Conditional and Sleep nodes expose this functionality, and users can create their own more advanced implementations using the Switch node and Event Pins. For further information, please see the Iteration section of the BRE Help documentation, including several tutorials and a detailed explanation of the functionality. 2.1.1 Do While Executes user defined BRAINScript and passes the result to an optional sub-graph, looping while the specified condition is met.. 2.1.2 Do While Conditional Provides the ability to execute a section of analytic logic one or more times while a userdefined condition is met. Page 3
2.1.3 Sleep Introduces a user-specified delay in the processing of the data flow. 2.1.4 Switch Inspects both inputs at runtime and passes the data of the second (optional) input if it contains data. Otherwise it passes the data of the first input. The node is conceptually similar to the existing "Bypass" node, however it determines which of its inputs to pass through to the output based on runtime availability of the data on its inputs. 2.2 Silent Installation The LAE 5.1 Windows Desktop installer now includes an option to run the installer silently, for desktop distribution to all user machines. Please see the Unattended Installation section of the Windows Desktop Installation Guide for more information. 2.3 User-Hosted Update Server Should you wish to distribute updates to all of your client machines on your own personalized schedule, 5.1 includes the ability to host your own update server, granting you control of releasing product updates to desktops on your network. Please see the User-Hosted Update Server section in the Enterprise and Windows Server installation guides for more information. 2.4 Automated Error Reporting Upon encountering an issue with a node in BRE, users can now automatically package and transmit an error report, including all necessary logs, to Lavastorm. Previously, submitting an error required manually finding the packaged logs and emailing them to support@lavastorm.com. Additionally, an option has been added to use a password to encrypt the package generated by the error reporting mechanism. 2.5 Logistics Manager Enhancements Several popular requests for enhancements to the Logistics Manager have been implemented, including run conditions for dependent graphs and the overriding of default parameters with user generated parameters. Please see the Logistics Manager-specific documentation for more information. Additionally, updates to the Logistics Manager have required changes to the underlying database structure. A graph to automate these database changes has been included. Please see the 5.0.0.1 to 5.1 Migration section of the Enterprise and Windows Server install guides for more information. 2.6 Password Policy LAE now allows the changing of passwords directly from BRE in addition to the existing command line utilities, and it provides support for password policy enforcement including: Password expiration Password expiration warning periods Password complexity rules (e.g. password must contain an uppercase character, number, non-alphanumeric, etc.) Page 4
3.0 4.0 5.0 The password policy can be configured via a new LAE property, ls.brain.passwdpolicy, which points to a password policy configuration file. An example of this file has been provided in the conf/brain/server directory of your installation folder named passwdpolicy. The ls.brain.passwdpolicy property is undefined by default in the ls_brain.prop file found in the conf/brain directory of your installation folder. Additional notes about this feature: Upon enabling a password policy that includes password expiration, all existing users' passwords will immediately expire. The changelaepassword utility has been modified so that providing the existing password for a user is no longer necessary. This is to facilitate administration of passwords that have been lost or forgotten under the assumption that access to this utility is restricted to administrator-type users. An expired password cannot be changed through BRE. An administrator must change the password from the server side. Lavastorm Analytics Library Installed along with LAE 5.1 is the Lavastorm Analytics Library (LAL), including Simba Technologies Hive drivers. In order for the Hive Metadata Query, Hive Sample, and Hive Join nodes to function, a reboot of the system might be required following installation of these drivers. An option to reboot the system is presented at the end of the installation. This option is not presented during silent installation, or if Visual Studio 2010 Redistributables is already present on the target machine. Please reboot the system manually following silent installation to enable the Hive nodes. Compatibility Scanner Included in the release of LAE 5.1 is a compatibility scanning graph for those upgrading from LAE 5.0, and 4.6. The scanner will identify graph elements that may encounter compatibility issues upon upgrading to 5.1. Note that the compatibility scanner must be run against the 5.0.x or 4.6.x LAE instance from which you are upgrading, not the 5.1 LAE instance. The documentation describing how to use the scanner can be located in the LAE installation under docs/upgradecompatibilityscanner.pdf. The compatibility scanner BRGs referenced in the document are located in the LAE installation under lib/brain/brg/migration/upgrade/. Bug Fixes and Corrections 1. Clarified the help content for the bool() BRAINscript function to accurately reflect its behavior when converting numeric values to booleans. Issue 1495 2. In the documentation for 'filter' the example given is: filter(¬-null, list(null, 1, null, 2, 3, null, 4)) # value: { 1 2 3 4 } not-null.filter(list(null, 1, null, 2, 3, null, 4)) # value: { 1 2 3 4 } This has been be replaced with '&isnotnull'. Issue 1395 3. When running BRXs with laecontrol or LXAs with the logistics manager, if the BRXs or LXAs contained nodes which produced output with columns names outside of the 7-bit US- Page 5
ASCII, the run would fail with an error of "Error while lexing spec format: Input length = 1". This has been fixed. Issue 1392 4. A bug with the "Copy as Excel" and "Save as Excel" functionality in the BRD Viewer has been fixed. Previously, when exporting BRD data to excel via the BRD Viewer, "\" characters were interpreted as unicode escape sequences that needed to be converted. This would lead to errors when trying to export strings like "c:\tmp\myfile.txt" to Excel using the BRD Viewer. Issue 1364 5. Logistics Manager now automatically filters out certain characters in all name fields. The disallowed characters are filtered out as the user types. The disallowed characters are the following: /, ", ', @,!, #, $, %, \, ^, &, *, ~, `, ;, :,, <, >, (, ),{, }, [, ],,,.,? Issue 1341 6. Fixed an issue where, in rare circumstances, data written to an output pin of a Java-based node containing an escape sequence could become corrupted due to the escape character being stripped out of the data. Issue 1317 7. When saving a BRX or LXA within BRE, newlines were encoded using the windows '\r\n' newline characters. However, when BRE attempts to execute a node and sends the node information to the server, multi-line parameters were sent with unix '\n' newline separators. In general this would not cause problems except for some multi-line parameters. In these cases, the server is expecting to receive the parameters with '\n' newline separators, which could result in some issues with the execution of some BRX and LXA files. This has now been fixed to ensure BRE doesn't force the use of windows newline characters. For BRX's and LXA's which have previously been saved and exhibit this problem, opening the corresponding BRG within BRE and re-saving as a BRX/LXA will resolve the problem. Issue 1149 8. A regression was introduced in the JDBC Store node in LAE 5.0.0 (and LAE 4.6.2) as a result of fixing issue 4294, which allowed for users to enter table and schema names with spaces in them. The node was modified to automatically add double-quote characters for all table and schema names in the generated SQL INSERT statement. The issue further stated that the DbTable, SchemaName, and CatalogName parameters should not contain quoting characters (e.g. "" or []) - as the node would insert the required quote characters automatically. While using the double-quote character is correct ANSI SQL syntax, not all databases accept this as a quoting character. Further, the use of quoting within table and schema names can be used to indicate casesensitivity rather than just ensuring that the table/schema name will work when it contains things such as space characters. In order to resolve this, the JDBC driver is consulted to determine which characters are allowed in identifier names. If there are any characters within the schema or table name which are not allowed in the Page 6
identifier names according to the driver, then the node will automatically insert quote characters around the identifier (schema or table) to ensure that this works correctly. Rather than simply using double-quotes, the quote character to use is obtained from the JDBC driver. There is an existing parameter on the JDBC Store node: TableAndSchemaCase. This parameter is used to determine whether all table and schema names should preserve the case, convert to upper case, or convert to lower case. Previously, this had a default of "To Upper", which would convert the case of all table and schema names provided to upper case - regardless of whether they are quoted or not. A new option has been added to the parameter: "Use Driver Settings". This option ensures that the driver is queried to determine whether such identifiers should be converted to upper case, to lower case, or have their case preserved. In general, this option should always be used as it will provide the correct interpretation of the case preservation mechanism according to the database. The use of the other options will therefore be deprecated over time. If the "Use Driver Settings" option is selected and the user enters a table or schema which is surrounded by the quote characters recognized by the JDBC driver, or enters a table or schema which needs to be quoted, then the database settings reported by the driver are used to determine what to do with the case of such quoted identifiers. Using this setting, the node will handle these differently than unquoted identifiers (which do not need to be automatically quoted). This allows the user more control of the case settings of the table and schema. Using an Oracle database, all identifiers are automatically converted to upper case unless they are quoted, where the case is preserved. As an example, if a user is created within an Oracle database, using: create user "myuser" identified by... And subsequently a table is created, using: create table "myuser".mytable Then the database will have a user "myuser" with a table "MYTABLE" - as the username was quoted, while the table was not. When entering this into the JDBC Store node, with the "Use Driver Settings" (default) option selected, if the SchemaName parameter is set to: "myuser", while the DbTable parameter is set to: mytable, the table will correctly be converted to uppercase in the insert statement, and the case of the schema will correctly be preserved. While the "Use Driver Settings" option should always provide the correct behavior, since the default of the TableAndSchemaCase parameter has been modified, this default can be reset on a server wide basis by modifying the compatibility-5.0.1.0.prop file found in the LAE server installation directory, under the conf/brain/ subdirectory. In order to revert to the previous "To Upper" default for the TableAndSchemaCase, simply open the compatibility-5.0.1.prop file and uncomment (remove the '#' at the start of the line) the following line: #ls.brain.node.java.dbloader.tableandschemacase="to Upper" Page 7
In addition to this a new parameter has been added "EmptySchemaAsNull" which specifies what to do when the SchemaName parameter is not populated. When set to true, this parameter will set the schema to null when retrieving the metadata from the database. When set to false (default),. When the default of false is used, this will set the schema to the empty string (""). This retrieves the metadata for tables without a schema. When set to null this means that the schema name should not be used to narrow the search. Generally this parameter should be left as the default - which would be the same as the existing behavior. However, you can explicitly choose for this to be null - in particular this seems to work around an issue connecting using the MS SQL Server 4.0 driver when attempting to publish to a table which was created without a schema name. Issue 962 9. The BRAINScript expression double now converts the strings NaN, Inf, and Infinity into the IEEE floating point values for Not-a-Number and Infinity. Issue 1451 10. The section of the BRE Help regarding Advanced Preferences has been updated to correctly match the changes to the dialog layout made in LAE 5.0. In addition, the Help button on each of the tabs within the Advanced Preferences dialog is now correctly linked to the corresponding sub topic for that tab in the help. Issue 969 11. Fixed a typo ("parmaeter" to "parameter") in the BRE help on special parameters. Issue 1472 12. In the Help documentation for slice: slice(list, offset, [length]) list.splice(offset, [length]) 6.0 The bottom line was changed to slice. Issue 1511 Backwards Compatibility 7.0 1. As noted in Bug Fix #8, the JDBC Store Node has been reworked. Please refer to Bug Fix #8 for details about possible compatibility issues with older LAE versions. Known Issues 7.1 Automated Error Reporting 1. Due to an error in a third party library used by LAE (Apache HttpClient 4.3.4), the error reporting feature may take an unusually long amount of time before reporting a connection error if proxy settings are configured incorrectly in BRE. Page 8