Teradata Aster Analytics Release Notes Update 2

Transcription

1 What would you do if you knew? Teradata Aster Analytics Release Notes Update 2 Release B K September 2017

2 The product or products described in this book are licensed products of Teradata Corporation or its affiliates. Teradata, Aster, BYNET, Claraview, DecisionCast, IntelliBase, IntelliCloud, IntelliFlex, QueryGrid, SQL-MapReduce, Teradata Decision Experts, "Teradata Labs" logo, Teradata ServiceConnect, and Teradata Source Experts are trademarks or registered trademarks of Teradata Corporation or its affiliates in the United States and other countries. Adaptec and SCSISelect are trademarks or registered trademarks of Adaptec, Inc. Amazon Web Services, AWS, Amazon Elastic Compute Cloud, Amazon EC2, Amazon Simple Storage Service, Amazon S3, AWS CloudFormation, and AWS Marketplace are trademarks of Amazon.com, Inc. or its affiliates in the United States and/or other countries. AMD Opteron and Opteron are trademarks of Advanced Micro Devices, Inc. Apache, Apache Avro, Apache Hadoop, Apache Hive, Hadoop, and the yellow elephant logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. Apple, Mac, and OS X all are registered trademarks of Apple Inc. Axeda is a registered trademark of Axeda Corporation. Axeda Agents, Axeda Applications, Axeda Policy Manager, Axeda Enterprise, Axeda Access, Axeda Software Management, Axeda Service, Axeda ServiceLink, and Firewall-Friendly are trademarks and Maximum Results and Maximum Support are servicemarks of Axeda Corporation. CENTOS is a trademark of Red Hat, Inc., registered in the U.S. and other countries. Cloudera and CDH are trademarks or registered trademarks of Cloudera Inc. in the United States, and in jurisdictions throughout the world. Data Domain, EMC, PowerPath, SRDF, and Symmetrix are either registered trademarks or trademarks of EMC Corporation in the United States and/or other countries. GoldenGate is a trademark of Oracle. Hewlett-Packard and HP are registered trademarks of Hewlett-Packard Company. Hortonworks, the Hortonworks logo and other Hortonworks trademarks are trademarks of Hortonworks Inc. in the United States and other countries. Intel, Pentium, and XEON are registered trademarks of Intel Corporation. IBM, CICS, RACF, Tivoli, IBM Spectrum Protect, and z/os are trademarks or registered trademarks of International Business Machines Corporation. Linux is a registered trademark of Linus Torvalds. LSI is a registered trademark of LSI Corporation. Microsoft, Active Directory, Windows, Windows NT, and Windows Server are registered trademarks of Microsoft Corporation in the United States and other countries. NetVault is a trademark of Quest Software, Inc. Novell and SUSE are registered trademarks of Novell, Inc., in the United States and other countries. Oracle, Java, and Solaris are registered trademarks of Oracle and/or its affiliates. QLogic and SANbox are trademarks or registered trademarks of QLogic Corporation. Quantum and the Quantum logo are trademarks of Quantum Corporation, registered in the U.S.A. and other countries. Red Hat is a trademark of Red Hat, Inc., registered in the U.S. and other countries. Used under license. SAP is the trademark or registered trademark of SAP AG in Germany and in several other countries. SAS and SAS/C are trademarks or registered trademarks of SAS Institute Inc. Sentinel is a registered trademark of SafeNet, Inc. Simba, the Simba logo, SimbaEngine, SimbaEngine C/S, SimbaExpress and SimbaLib are registered trademarks of Simba Technologies Inc. SPARC is a registered trademark of SPARC International, Inc. Unicode is a registered trademark of Unicode, Inc. in the United States and other countries. UNIX is a registered trademark of The Open Group in the United States and other countries. Veritas, the Veritas Logo and NetBackup are trademarks or registered trademarks of Veritas Technologies LLC or its affiliates in the U.S. and other countries. Other product and company names mentioned herein may be the trademarks of their respective owners. The information contained in this document is provided on an "as-is" basis, without warranty of any kind, either express or implied, including the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. Some jurisdictions do not allow the exclusion of implied warranties, so the above exclusion may not apply to you. In no event will Teradata Corporation be liable for any indirect, direct, special, incidental, or consequential damages, including lost profits or lost savings, even if expressly advised of the possibility of such damages. The information contained in this document may contain references or cross-references to features, functions, products, or services that are not announced or available in your country. Such references do not imply that Teradata Corporation intends to announce such features, functions, products, or services in your country. Please consult your local Teradata Corporation representative for those features, functions, products, or services available in your country. Information contained in this document may contain technical inaccuracies or typographical errors. Information may be changed or updated without notice. Teradata Corporation may also make improvements or changes in the products or services described in this information at any time without notice. To maintain the quality of our products and services, we would like your comments on the accuracy, clarity, organization, and value of this document. Please teradata-books@lists.teradata.com Any comments or materials (collectively referred to as "Feedback") sent to Teradata Corporation will be deemed non-confidential. Teradata Corporation will have no obligation of any kind with respect to Feedback and will be free to use, reproduce, disclose, exhibit, display, transform, create derivative works of, and distribute the Feedback and derivative works thereof without limitation on a royalty-free basis. Further, Teradata Corporation will be free to use any ideas, concepts, know-how, or techniques contained in such Feedback for any purpose whatsoever, including developing, manufacturing, or marketing products or services incorporating Feedback. Copyright 2017 by Teradata. All Rights Reserved.

3 Preface Summary This document describes the new features, new and revised functions, and fixed and known issues in Aster Analytics versions , , and This release works correctly only when installed on a system running Aster Database version 6.20 (AD 6.20) or later, and Aster Execution Engine version (AX ). Contact Teradata Global Technical Support (GTS) For assistance and updated documentation, contact Teradata Global Technical Support (GTS): Support Portal: International: US Customers: 877-MyT-Data ( ) Third Party Licenses An Aster installation includes a number of open source products. The license text for these products is available on the Aster queen, as a set of text files in the /home/beehive/licenses directory. About This Document This document applies to Aster Analytics releases , , and Revision History Release Product ID Date AA Updated to support AA release AA Updated to support AA release B K September 2017 B K May 2017 AA B K May 2017 Teradata Aster Analytics Release Notes Update 2, Release

4 Preface About This Document 4 Teradata Aster Analytics Release Notes Update 2, Release

5 CHAPTER 1 Release Information Who Should Install This Release? All Teradata Aster installations using AD 6.20 or later, or AX should install this release for all Aster Analytics functions they have licensed from Teradata. Teradata Aster installations using AD or earlier should Contact Teradata Global Technical Support (GTS). What's New? Teradata Aster Analytics Foundation Release AA New Functions Time Series, Path, and Attribution Analysis TimeSeriesOrders. The TimeSeriesOrders function takes an input table that represents a time series and determines the orders and whether to incorporate the drift term into a time series model. The TimeSeriesOrders output is typically input to the VARMAX function as the orders table. Ensemble Methods XGBoost_Drive. The XGBoost_Drive function takes a training data set and uses gradient boosting to produce a strong classifying model that can be input to the XGBoost_Predict function. XGBoost_Predict. The XGBoost_Predict function applies the model output by the XGBoost_Drive function to a new data set. Revised Functions GLM2 (changed output model table) GLM2Predict (changed syntax to accommodate changed GLM2 model table) VARMAX (new optional input table, orders; new IncludeDrift argument) Teradata Aster Analytics Release Notes Update 2, Release

6 Chapter 1: Release Information What's New? Release AA New Functions Statistical Analysis GLM2. The GLM2 function is similar to the GLM function except that GLM2 supports the regularization models ridge, lasso, and elastic net. GLM2Predict. The GLM2Predict function uses the model generated by GLM2 to perform generalized linear model prediction on new input data. Revised Functions Forest_Drive (new OOB argument) VARMAX (new Method argument) Teradata Aster Extensions for KNIME Teradata Aster Extensions for KNIME supports the functions that are new for this release and the revisions made to functions for this release. Saving results to a table In the previous release, to persist the results table permanently, you had to use the AsterDB Table Writer node. In this release, the AsterDB Table Writer node is deprecated (it works as it did previously, but you cannot include it in new workflows). To persist the results table permanently in a new workflow, use the Options tab in the Node Configuration dialog to specify the name of the table (and optionally, the distribution key) in which to store results. All nodes from AA are deprecated. Workflows with these nodes work as they did previously, but new workflows must use the newer versions of the nodes. Release AA New Functions Time Series, Path, and Attribution Analysis FFT. The FFT function uses a Fast Fourier Transform (FFT) algorithm to compute the discrete Fourier Transform (DFT) of each signal in one or more input table columns. A signal can be either real or complex, and can have one, two, or three dimensions. IFFT. The IFFT function uses an inverse Fast Fourier Transform (IFFT) algorithm (also called a Fourier synthesis algorithm) to reverse the Fast Fourier Transform (FFT) performed by the FFT function. Statistical Analysis LinRegPredict. The LinRegPredict function takes a model built by the Linear Regression Function and a test data set whose input attributes are the same as those in the model, and predicts the response variable for each observation in the test data set. 6 Teradata Aster Analytics Release Notes Update 2, Release

7 ROC. The ROC function takes a set of prediction-actual pairs for a binary classification model and calculates the following values for a range of discrimination thresholds: True positive rate (TPR) False positive rate (FPR) Area under the ROC curve (AUC) Gini coefficient Chapter 1: Release Information What's New? Revised Functions AdaBoost_Drive (new OutputResponseProbDist argument; new model table columns left_label_probdist, right_label_probdist, and prob_label_order) AdaBoost_Predict (new OutputResponseProbDist, ModelTableName, and Accumulate arguments) CCM (new PredictStep and PointSelectRule arguments, no PointsToTest argument, can select the best value for its EmbeddingDimensions argument) Forest_Predict (accepts model table as input, ModelFile and Forest arguments are now optional) GLMPredict (accepts model table as input, ModelTable argument is now optional) NaiveBayesPredict (accepts model table as input, Model argument is now optional) Single_Tree_Drive (new OutputResponseProbDist and ResponseProbDistType arguments) Single_Tree_Predict (new OutputResponseProbDist, ModelTableName, and Accumulate arguments) TrainNamedEntityFinder (improved log messages) Teradata Aster Extensions for KNIME Teradata Aster Extensions for KNIME lets you combine the scale of Aster Analytics with the easy-to-use visual interface of the KNIME Analytics Platform. Using the KNIME drag-and-drop interface, you can build workflows that incorporate Aster Analytics SQL-MapReduce functions. No SQL coding is required. Teradata Aster R Release AA New Functions Parallel Data Analysis Functions aa.timeseriesorders() aa.xgboost.drive() aa.xgboost.predict() R Support Teradata Aster R supports R Release AA Teradata Aster Analytics Release Notes Update 2, Release

8 Chapter 1: Release Information What's New? New Functions Parallel Data Analysis Functions aa.glm2() aa.glm2.predict() Release AA New Functions Parallel Data Analysis Functions aa.adaboost.drive() aa.adaboost.predict() aa.add.one.player() aa.allpairs.shortest.path() aa.aml.generator() aa.antiselect() aa.apache.log.parser() aa.approx.dcount() aa.approx.percentile() aa.arima() aa.arima.predict() aa.attribution() aa.basket.generator() aa.betweenness() aa.burst() aa.canopy() aa.categorize() aa.ccm() aa.ccm.prepare() aa.changepoint.detection() aa.cfilter() aa.closeness() aa.cmavg() aa.confusion.matrix() aa.cor() aa.coxph() aa.cox.predict() aa.cox.survfit() aa.crossvalidation() aa.distnmatch() aa.dtw() aa.dwt() aa.dwt2d() 8 Teradata Aster Analytics Release Notes Update 2, Release

9 aa.eigenvector.centrality() aa.emavg() aa.fellegi.sunter.predict() aa.fellegi.sunter.train() aa.fft() aa.fmeasure() aa.forest() aa.forest.evaluate() aa.forest.predict() aa.fpgrowth() aa.frequent.paths() aa.generate.combination() aa.geometry.overlay() aa.glm() aa.glm.predict() aa.gmm() aa.gmm.predict() aa.gmm.profile() aa.gtree() aa.hist() aa.hmm.decoder() aa.hmm.evaluate() aa.hmm.supervised.learner() aa.hmm.unsupervised.learner() aa.identity.match() aa.idwt() aa.idwt2d() aa.ifft() aa.interpolator() aa.json.parser() aa.kmeans() aa.kmodes() aa.kmodes.predict() aa.knn() aa.knn.recommender.predict() aa.knn.recommender.train() aa.lars() aa.lars.predict() aa.lda() aa.lda.inference() aa.lda.topicprinter() aa.ldist() aa.lm() aa.lm.predict() Chapter 1: Release Information What's New? Teradata Aster Analytics Release Notes Update 2, Release

10 Chapter 1: Release Information What's New? aa.local.clustering.coefficient() aa.loopy.belief.propagation() aa.lr.test() aa.minhash() aa.modularity() aa.multicase() aa.murmur.hash() aa.naivebayes.predict() aa.naivebayes.textclassifier.predict() aa.naivebayes.textclassifier.train() aa.naivebayes.train() aa.namedentity.find() aa.namedentity.finder.evaluate() aa.namedentity.finder.train() aa.ner.evaluate() aa.ner.predict() aa.ner.train() aa.neuralnet() aa.neuralnet.predict() aa.ngram() aa.ntree() aa.npath() aa.outlierfilter() aa.pack() aa.pagerank() aa.partition.scale() aa.path.evaluate() aa.path.generator() aa.path.start() aa.path.summary() aa.pca() aa.pca.plot() aa.percentile() aa.pivot() aa.pointin.polygon() aa.pos.tagger() aa.psalsa() aa.random.sample() aa.random.walk.sample() aa.recommender() aa.rt.changepoint.detection() aa.sample() aa.sax() aa.scale() 10 Teradata Aster Analytics Release Notes Update 2, Release

11 aa.scale.map() aa.scale.printer() aa.sentenizer() aa.sentiment.evaluate() aa.sentiment.extract() aa.sentiment.train() aa.series.splitter() aa.sessionize() aa.single.tree.drive() aa.single.tree.predict() aa.smavg() aa.sort.combination() aa.string.similarity() aa.supervised.shapelet.classifier() aa.supervised.shapelet.train() aa.svm.dense.modelprinter() aa.svm.dense.predict() aa.svm.dense.train() aa.svm.sparse.modelprinter() aa.svm.sparse.predict() aa.svm.sparse.train() aa.text.chunker() aa.text.classifier.evaluate() aa.text.classifier.predict() aa.text.classifier.train() aa.text.morph() aa.text.parser() aa.text.tagging() aa.text.tokenizer() aa.tf() aa.tfidf() aa.unpack() aa.unpivot() aa.unsupervised.shapelet() aa.uri.pack() aa.uri.unpack() aa.varmax() aa.vector.distance() aa.vwap() aa.wmavg() aa.xml.parser() aa.xml.relation() Chapter 1: Release Information What's New? Teradata Aster Analytics Release Notes Update 2, Release

12 Chapter 1: Release Information Getting and Installing Aster Analytics Foundation Teradata Aster Spark Connector Release AA There is no change to Aster Spark Connector. AA includes aster-connectorspark r64898.x86_64.rpm, the same version of Aster Spark Connector released in AA Release AA This release includes: Support for Kerberos authentication for the Hadoop/Spark cluster Teradata Aster Spark API RunOnSpark function argument changes Release AA This release introduces the Aster Spark Connector, which lets you to invoke any ready-to-use Spark function by running a simple Aster query, and the ready-to-use Spark functions. Getting and Installing Aster Analytics Foundation To get the release, obtain the Aster Analytics Foundation packages for the product bundle that you have purchased. Notifications The installation instructions in the Teradata Aster Analytics Foundation User Guide include instructions for installing Teradata Aster Extensions for KNIME. Documentation Library Changes The Aster Analytics documentation library includes: This document Teradata Aster Analytics Foundation User Guide Teradata Aster R User Guide (new to this library) This guide, which explains both installation and usage, has new examples of using Teradata Aster R with tables, data frames, Aster Analytics functions, and custom R scripts and functions. Teradata Aster Spark Connector User Guide (new to this library) 12 Teradata Aster Analytics Release Notes Update 2, Release

13 Chapter 1: Release Information Fixed Issues This guide explains installation, configuration, the RunOnSpark function, and the ready-to-use Spark functions. Fixed Issues Teradata Aster Analytics Foundation HMMSupervisedLearner and HMMUnsupervisedLearner may not consider the final symbol in an observation sequence. Issue ID: ANLY-3157 Details: HMMSupervisedLearner and HMMUnsupervisedLearner may not consider the final symbol in an observation sequence. For random sequences, this has a negligible effect on the probabilities in the models generated by these functions. Workaround: None TextTokenizer incorrectly tokenizes some words containing apostrophes. Issue ID: ANLY-3178 Details: If a word contains an apostrophe, the function might split the word into two tokens. Workaround: None NER displays misleading error message. Issue ID: ANLY-2354 Details: If the input table to the NER function was not specified with the PARTITION BY ANY clause, the function displays this misleading error message: SQL-MapReduce function NER failed: INTERNAL: The column argument TextColumn does not have a target table specified. Please contact Teradata Aster support. Workaround: Specify the input table using a PARTITION BY ANY clause. GMMFit fails when schema for output table is specified. Issue ID: ANLY-4017 Details: If the user specifies the schema for the output table, the function displays this error message:error: SQL-MR function GMMFIT failed: [AsterData][ASTERJDBCDSII](34) ERROR: schema "_temp_table_for_gmmfit_test" does not exist () Workaround: 1. Change the user's search path as follows: ALTER USER user_name SET search_path='test','public'; 2. Have the user run GMMFit without specifying the schema for the output table. Teradata Aster Analytics Release Notes Update 2, Release

14 Chapter 1: Release Information Known Issues Teradata Aster R ta.pull with table name argument fetches NULL values incorrectly. Issue ID: ANLY-3975 Details: When the ta.pull function has a table name argument, it fetches NULL values incorrectly. For example: ta.pull("empty_values_tab") -> r1 > r1 val_1 val_2 val_3 3 a c 11 4 a b 1 a b c 5 2 b c Workaround: Create a virtual data frame and use it instead of the table name in the ta.pull call. For example: ta.data.frame("select * from empty_values_tab",sourcetype="query")-> vdf > ta.pull(vdf)-> r2 > r2val_1 val_2 val_3 3 a c 11 <NA> <NA> <NA> 2 b c 4 a b 1 a b c 5 Known Issues Teradata Aster Analytics Foundation Install Script for Analytic Packages can get error with POSTaggerModel Script. Issue ID: BH-5398 Details: When installing the model file pos_model_2.0_zh_cn_ bin (which POSTagger uses), this error sometimes occurs: INSTALL FILE 'pos_model_2.0_zh_cn_ bin' SCHEMA "public"; ERROR: An internal error has occurred. Workaround: Try again to install the model file. If the error recurs, contact Teradata Global Customer Support. Some SQL-MR driver functions do not work in an SSL-enabled cluster. Issue ID: CLNT Teradata Aster Analytics Release Notes Update 2, Release

15 Details: These driver functions do not work in an SSL-enabled cluster: AdaBoost_Drive.sql CoxPH.sql CrossValidation.sql KModes.sql RandomSample.sql Workaround: None Chapter 1: Release Information Known Issues GLM2 may run slowly with a large dataset if Family is BINOMIAL or POISSON. Issue ID: ANLY-4339 Details: The GLM2 function may be very slow for large data sets if the value of the Family argument is BINOMIAL or POISSON. Workaround: Work with your Customer Support Representative to optimize performance. Teradata Aster R Some in-database R functions do not work within RStudio on Linux. Issue ID: CLNT-2635 Details: In-database R functions with the FUN.upload = script option and utility functions, such as ta.install.scripts and ta.install.files, do not work within RStudio on Linux due to compatibility issues between RStudio and the Aster ODBC driver. Workaround: Install scripts from RConsole using the ta.install.scripts or ta.install.files functions or use the ACT client to install the scripts on the cluster. In-Database R Function Runners can then be used without specifying the FUN.upload = script argument. ta.within function fails in R Issue ID: ANLY-4881 Details: The ta.within() function fails in R Workaround: The ta.within() function is a wrapper function for the ta.transform() function. Use the ta.transform() function instead in R Note: When using ta.transform() in stead of ta.within(), exclude the "{}" from the second argument. For example, for the function ta.within(df, {c=pi}), you use ta.transform(df, c=pi) instead. Teradata Aster Analytics Release Notes Update 2, Release

16 Chapter 1: Release Information Naming Convention for Teradata Aster Releases Teradata Aster Spark Connector Spark jobs running longer than default time might fail. Issue ID: SC-65 Details: A Spark job that runs longer than the default value of the MAILMANSERVER_BLOCK_TIMEOUT_SEC argument, 3600 seconds, might fail. Workaround: If you expect the job to run longer than 3600 seconds, increase the value of the MAILMANSERVER_BLOCK_TIMEOUT_SEC argument, in either the argument clause or the spark.config file. Non-ssh sanity check queries fail. Issue ID: SC-12 Details: Non-ssh sanity check queries fail. Workaround: Use ssh Spark-submit spark.config identifiers instead of non-ssh Spark-submit. nossh spark_cluster_id does not work with Cloudera CDH Using Kerberos. Issue ID: SC-144 Details:For Cloudera CDH using Kerberos for hadoop.security.authentication, neither AsterSpark*site.json-file-noSSh nor AsterSpark*site.json-socket-noSSh spark_cluster_id works in RunOnSpark queries. Workaround: Instead, use AsterSpark*site.json-file-SSh or AsterSpark*site.json-socket-SSh spark_cluster_id in RunOnSpark queries. Naming Convention for Teradata Aster Releases The Teradata Aster release naming convention is: AD: Aster Database AX: Aster Execution Engine AC: Aster Client AA: Aster Analytics A release number has the format XX.YY.ZZ.nn, where: XX is a major release. Major releases of Aster typically introduce new features. YY is a minor release. Minor releases of Aster typically provide feature enhancements and bug fixes. A minor release may also introduce new features. ZZ is a maintenance release. 16 Teradata Aster Analytics Release Notes Update 2, Release

17 Maintenance releases of Aster typically provide only bug fixes. They may also include feature enhancements. nn is an efix or hotpatch. Efixes and hotpatches address urgent customer issues. Chapter 1: Release Information Naming Convention for Teradata Aster Releases Teradata Aster Analytics Release Notes Update 2, Release

18 Chapter 1: Release Information Naming Convention for Teradata Aster Releases 18 Teradata Aster Analytics Release Notes Update 2, Release