Teradata Data Mover Installation, Configuration, and Upgrade Guide for Customers

Transcription

1 What would you do if you knew? Installation, Configuration, and Upgrade Guide for Customers Release B K December 2016

2 The product or products described in this book are licensed products of Teradata Corporation or its affiliates. Teradata, Applications-Within, Aster, BYNET, Claraview, DecisionCast, Gridscale, MyCommerce, QueryGrid, SQL-MapReduce, Teradata Decision Experts, "Teradata Labs" logo, Teradata ServiceConnect, Teradata Source Experts, WebAnalyst, and Xkoto are trademarks or registered trademarks of Teradata Corporation or its affiliates in the United States and other countries. Adaptec and SCSISelect are trademarks or registered trademarks of Adaptec, Inc. Amazon Web Services, AWS, [any other AWS Marks used in such materials] are trademarks of Amazon.com, Inc. or its affiliates in the United States and/or other countries. AMD Opteron and Opteron are trademarks of Advanced Micro Devices, Inc. Apache, Apache Avro, Apache Hadoop, Apache Hive, Hadoop, and the yellow elephant logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. Apple, Mac, and OS X all are registered trademarks of Apple Inc. Axeda is a registered trademark of Axeda Corporation. Axeda Agents, Axeda Applications, Axeda Policy Manager, Axeda Enterprise, Axeda Access, Axeda Software Management, Axeda Service, Axeda ServiceLink, and Firewall-Friendly are trademarks and Maximum Results and Maximum Support are servicemarks of Axeda Corporation. CENTOS is a trademark of Red Hat, Inc., registered in the U.S. and other countries. Cloudera, CDH, [any other Cloudera Marks used in such materials] are trademarks or registered trademarks of Cloudera Inc. in the United States, and in jurisdictions throughout the world. Data Domain, EMC, PowerPath, SRDF, and Symmetrix are registered trademarks of EMC Corporation. GoldenGate is a trademark of Oracle. Hewlett-Packard and HP are registered trademarks of Hewlett-Packard Company. Hortonworks, the Hortonworks logo and other Hortonworks trademarks are trademarks of Hortonworks Inc. in the United States and other countries. Intel, Pentium, and XEON are registered trademarks of Intel Corporation. IBM, CICS, RACF, Tivoli, and z/os are registered trademarks of International Business Machines Corporation. Linux is a registered trademark of Linus Torvalds. LSI is a registered trademark of LSI Corporation. Microsoft, Active Directory, Windows, Windows NT, and Windows Server are registered trademarks of Microsoft Corporation in the United States and other countries. NetVault is a trademark or registered trademark of Dell Inc. in the United States and/or other countries. Novell and SUSE are registered trademarks of Novell, Inc., in the United States and other countries. Oracle, Java, and Solaris are registered trademarks of Oracle and/or its affiliates. QLogic and SANbox are trademarks or registered trademarks of QLogic Corporation. Quantum and the Quantum logo are trademarks of Quantum Corporation, registered in the U.S.A. and other countries. Red Hat is a trademark of Red Hat, Inc., registered in the U.S. and other countries. Used under license. SAP is the trademark or registered trademark of SAP AG in Germany and in several other countries. SAS and SAS/C are trademarks or registered trademarks of SAS Institute Inc. Simba, the Simba logo, SimbaEngine, SimbaEngine C/S, SimbaExpress and SimbaLib are registered trademarks of Simba Technologies Inc. SPARC is a registered trademark of SPARC International, Inc. Symantec, NetBackup, and VERITAS are trademarks or registered trademarks of Symantec Corporation or its affiliates in the United States and other countries. Unicode is a registered trademark of Unicode, Inc. in the United States and other countries. UNIX is a registered trademark of The Open Group in the United States and other countries. Other product and company names mentioned herein may be the trademarks of their respective owners. The information contained in this document is provided on an "as-is" basis, without warranty of any kind, either express or implied, including the implied warranties of merchantability, fitness for a particular purpose, or noninfringement. Some jurisdictions do not allow the exclusion of implied warranties, so the above exclusion may not apply to you. In no event will Teradata Corporation be liable for any indirect, direct, special, incidental, or consequential damages, including lost profits or lost savings, even if expressly advised of the possibility of such damages. The information contained in this document may contain references or cross-references to features, functions, products, or services that are not announced or available in your country. Such references do not imply that Teradata Corporation intends to announce such features, functions, products, or services in your country. Please consult your local Teradata Corporation representative for those features, functions, products, or services available in your country. Information contained in this document may contain technical inaccuracies or typographical errors. Information may be changed or updated without notice. Teradata Corporation may also make improvements or changes in the products or services described in this information at any time without notice. To maintain the quality of our products and services, we would like your comments on the accuracy, clarity, organization, and value of this document. Please teradata-books@lists.teradata.com Any comments or materials (collectively referred to as "Feedback") sent to Teradata Corporation will be deemed non-confidential. Teradata Corporation will have no obligation of any kind with respect to Feedback and will be free to use, reproduce, disclose, exhibit, display, transform, create derivative works of, and distribute the Feedback and derivative works thereof without limitation on a royalty-free basis. Further, Teradata Corporation will be free to use any ideas, concepts, know-how, or techniques contained in such Feedback for any purpose whatsoever, including developing, manufacturing, or marketing products or services incorporating Feedback. Copyright by Teradata. All Rights Reserved.

3 Table of Contents Preface...5 Purpose... 5 Audience...5 Revision History...5 Additional Information...5 Product Safety Information... 6 Chapter 1: Overview...7 Dependencies...7 Best Practices for Data Mover Networking... 8 Section I: Installing and Configuring Software Chapter 2: Configuring the Environment...11 Configuring the Data Mover Daemon...11 The daemon.properties File...11 Configuration Properties Configuring the Data Mover Agent Installing and Configuring the Data Mover Agent on a Linux Teradata Server...18 The agent.properties File...19 Configuring the Data Mover Command-Line Interface Configuring the Data Mover Command-Line Interface on a Linux Teradata Server...21 Installing and Configuring the Data Mover Command-Line Interface on Non-Teradata Servers...21 The commandline.properties File...23 Configuring the Data Mover REST Service...24 About Configuring High Availability Configuring Automatic Failover...25 High Availability Configuration Scenario Verifying Data Mover Package Installation Example: Verifying Data Mover Package Installation Setting Up Host Files or DNS...28 Example: Setting Up Host Files or DNS Verifying Required Ports Open...28 Example: Verifying Required Ports are Open...29 Defining Unique Data Mover Agent Names...29 Installation, Configuration, and Upgrade Guide for Customers Release

4 Table of Contents Example: Defining Unique Data Mover Agent Names Synchronizing the Master and Slave Repositories...30 Example: Synchronizing the Master and Slave Repositories Configuring Dual Active Java Message Service (JMS) Brokers Example: Configuring the Dual Active Java Message Service (JMS) Brokers Configuring the Sync Service Example: Configuring the Sync Service Configuring the Cluster and Starting the Monitoring Service Failover.properties File...35 Example: Configuring the Cluster and Starting the Monitoring Service Checking the Status of Master and Slave Components Example: Checking the Status of the Master and Slave Components Starting the Synchronization Service Example: Starting the Synchronization Service Verifying Failover Configuration Example: Verifying the Failover Configuration...38 Completing the Automatic Failover Setup...38 Configuring the Synchronization Service Without Automatic Failover...39 Configuring the Synchronization Service...39 Configuring Data Mover to Use Teradata Ecosystem Manager...39 Configuring Multiple Managed Servers...41 Configuring Data Mover to Log to Server Management...41 Enabling Logging Server Management Alerts When a Failover Occurs...41 Configuring Data Mover Managed Server to Increase Network Throughput...42 About Adding Source and Target COP Entries...42 About Defining Routes for Source and Target COP Entries Restarting the Network About Verifying the Route Changes Data Mover Log Files...44 Data Mover Properties Files Preserved During Upgrades Chapter 3: Administrative Tasks Data Mover Components Script Changing DBC and DATAMOVER Passwords on the Data Mover Server...47 Creating a Diagnostic Bundle for Support Section II: Upgrading Software Chapter 4: Upgrading Software...55 About Upgrading Data Mover Software...55 Creating an Incident...55 Upgrading the Data Mover Command-Line Interface on Non-Teradata Servers Upgrading the Data Mover Agent on a Linux Teradata Server Installation, Configuration, and Upgrade Guide for Customers Release 16.00

5 Preface Purpose This guide provides customer information and procedures for installing, configuring, and upgrading software. Audience This guide is intended for use by: System administrators Database administrators and relational database developers Customers Teradata Customer Support Revision History Date Release Description December Initial release. Additional Information Related Links URL Description Use Teradata At Your Service to access Orange Books, technical alerts, and knowledge repositories, view and join forums, and download software packages. External site for product, service, resource, support, and other customer information. Related Documents Documents are located at Title User Guide Describes how to use the portlets and command-line interface. Publication ID B Installation, Configuration, and Upgrade Guide for Customers Release

6 Preface Product Safety Information Title Parallel Upgrade Tool (PUT) Reference Describes how to install application software using PUT. Teradata Viewpoint User Guide Describes the Teradata Viewpoint portal, portlets, and system administration features. Publication ID B B Customer Education Teradata Customer Education delivers training for your global workforce, including scheduled public courses, customized on-site training, and web-based training. For information about the classes, schedules, and the Teradata Certification Program, go to Customer Support Customer support is available around-the-clock, seven days a week through the Global Technical Support Center (GSC). To learn more, go to Product Safety Information This document may contain information addressing product safety practices related to data or property damage, identified by the word Notice. A notice indicates a situation which, if not avoided, could result in damage to property, such as equipment or data, but not related to personal injury. Example Notice: Improper use of the Reconfiguration utility can result in data loss. 6 Installation, Configuration, and Upgrade Guide for Customers Release 16.00

7 CHAPTER 1 Overview Dependencies Data Mover Server Requirements Software Level Server SUSE Linux Enterprise Server 11 SP 1 and SP 3 Internal Repository Teradata Database For non-teradata servers, you must install or upgrade to JRE 7 before installing or upgrading any components. Data Mover Component Version Requirement As of Data Mover 16.00, only the major and minor versions of the Data Mover daemon and the Data Mover command line interface or portlet must match. For instance: Command Line Interface/ Portlet Version Daemon Version Supported Yes No The Data Mover agent must be the exact same version as the Data Mover daemon. External Component Requirements The versions of components that Data Mover works with are listed below. Software Level Teradata Database Versions Hortonworks Data Platform for Commodity Hardware Hortonworks Data Platform for Teradata Appliance for Hadoop Teradata Connector for Hadoop Versions 2.2.x, 2.3.x, and 2.4 Versions 2.3.x and 2.4 Version 1.5.x Teradata Aster Database Versions 6.0, 6.10, and 6.20 Teradata Viewpoint Version or Installation, Configuration, and Upgrade Guide for Customers Release

8 Chapter 1: Overview Best Practices for Data Mover Networking Software Level Cloudera Distribution of Hadoop Version 5.4.x, 5.7, and 5.8 Required Permissions You must be a root user to install and configure Data Mover components. Required Open Ports on the Data Mover Server The ports listed below must be open for incoming and outgoing traffic on the Data Mover server: Port Number 22 SSH Used By 1025 CLI and JDBC ARC access module ARC server ActiveMQ Master sync service 1080 RESTful API Best Practices for Data Mover Networking A comprehensive knowledge article about Data Mover networking best practices is available to help you understand and resolve a variety of performance issues, including test and validation procedures for suggested changes. The Data Mover Networking Best Practices knowledge article is located at /0/41a6f00b2ff7e d330008e2df. 8 Installation, Configuration, and Upgrade Guide for Customers Release 16.00

9 SECTION I Installing and Configuring Software Installation, Configuration, and Upgrade Guide for Customers Release

10 Section I Installing and Configuring Software 10 Installation, Configuration, and Upgrade Guide for Customers Release 16.00

11 CHAPTER 2 Configuring the Environment Configuring the Data Mover Daemon 1. Edit the daemon.properties file and restart the Data Mover daemon to implement the changes. For properties that can be set dynamically, the changes take effect one minute after the updated daemon.properties file is saved. There is no need to restart the daemon service if you are only updating dynamic properties. 2. Use the list_configuration and save_configuration commands to modify the other Data Mover properties. The daemon.properties File The following table shows the properties in daemon.properties. Property Description Default Value arcserver.port=port broker.port=port broker.url=url cluster.enabled=setting for cluster viewpoint.url viewpoint.port logger.usetvilogger=sett ing for TVI messages A long-lived server port on the machine running the DMDaemon, which is used for inbound socket connections from DM Agents. The port number of the machine where the Java Message Service (JMS) message broker is listening. The hostname or IP address of the machine running the Java Message Service (JMS) message broker. When set to True, establishes a connection to a secondary Java Message Service (JMS) broker in case the primary JMS broker fails. The hostname or IP address for the Viewpoint Authentication server. Example: viewpoint.url= The port number for the Viewpoint Authentication server. Example: viewpoint.port=80 The Server Management logger can be set to true or false. If set to true, fatal error messages are sent to Server Management. Dynamic property localhost False localhost 80 True jobexecutioncoordinator. maxconcurrentjobs=maximu m number of jobs The maximum number of jobs allowed to run on the daemon at the same time. Additional jobs are placed on 20 Installation, Configuration, and Upgrade Guide for Customers Release

12 Chapter 2: Configuring the Environment Configuring the Data Mover Daemon Property Description Default Value the queue and run when slots become available. Dynamic property. 1 jobexecutioncoordinator. maxqueuedjobs=maximum number of jobs allowed in queue The maximum number of jobs allowed in the job queue. Additional jobs are placed in a higher level memory queue until slots are available in the job queue. Dynamic property. 1 n 20 log4j.appender.logfile=o rg.apache.log4j.rollingf ileappender Informs the logging application to use a specific appender. It is recommended that this property value not be changed. log4j.appender.logfile.f ile=file path name log4j.appender.logfile.l ayout=org.apache.log4j.p atternlayout Relative or absolute path of the log file. If changing log file location, specify the absolute path of the file. For Windows, specify back slash instead of forward slash, for example, C:\Program File\Teradata\Log \dmdaemon.log. Both file path and file name can be set dynamically. 1 Dynamic property. 1 Do not edit. This is an internal setting for logging infrastructure. dmdaemon.log log4j.appender.logfile.m axbackupindex=<number of backup files> The number of backup logging files that are created. After the maximum number of files has been created, the oldest file is erased. Dynamic property. 1 Example If Max Backups = 3, three backup logs are created: 3 dmdaemon.log.1 dmdaemon.log.2 dmdaemon.log.3 If the current dmdaemon.log size exceeds 10MB, it rolls to become the new dmdaemon.log.1 and a new dmdaemon.log is created. The previous dmdaemon.log.2 becomes the new dmdaemon.log. 3. The previous dmdaemon.log.3 is deleted. log4j.appender.logfile.m axfilesize=<maximum size of log files> log4j.appender.logfile.l ayout.conversionpattern= <log file pattern layout> The maximum size of the logging file before being rolled over to backup files. Dynamic property. 1 The pattern of the log file layout, where: d = date t = thread p = log level 10MB %d [%t] %-5p %c{3} - %m%n 12 Installation, Configuration, and Upgrade Guide for Customers Release 16.00

13 Property Description Default Value c = class name m= message n = platform dependent line separator Dynamic property when layout is PatternLayout. 1 Information for creating a layout is at: logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/ PatternLayout.html Chapter 2: Configuring the Environment Configuring the Data Mover Daemon log4j.rootlogger=<level of logging> 6 levels of logging, TRACE DEBUG INFO WARN ERROR FATAL. From trace level to application error. LOG_LEVEL can be updated dynamically, but not logfile. 1 Value is: LOG_LEVEL, logfile Do not remove the term logfile. INFO,logfile If the Viewpoint Authentication server has HTTPS enabled, you can set the following if you want to authenticate via HTTPS instead: viewpoint.url to and viewpoint.port to Dynamic Properties For properties that can be set dynamically, the changes take effect one minute after the updated daemon.properties file is saved. There is no need to restart the daemon service if you are only updating dynamic properties. For example: If you changed the value of log4j.rootlogger from the default of INFO, logfile to DEBUG, logfile, any debug messages generated would start appearing in the log file one minute after saving the updated properties file. If you changed the value of jobexecutioncoordinator.maxconcurrentjobs from the default value of 20 to a new value of 25, the new value of 25 would take effect one minute after saving the updated daemon.properties file. Configuration Properties Property Description Default agentcollector.agentheartbeat WaitMillis blocked.job.maxallowedlimit Sets the amount of time in milliseconds to wait for an agent heartbeat before assuming it has gone out of service. The maximum number of jobs that can be marked as BLOCKED and retried. If a job is detected as blocked when the blocked.job.maxallowedlimit has already been reached, the job is added to the Job Queue Installation, Configuration, and Upgrade Guide for Customers Release

14 Chapter 2: Configuring the Environment Configuring the Data Mover Daemon Property Description Default The value cannot be greater than 25% of the maximum concurrent job limit. blocked.job.retry.enabled blocked.job.retry.interval blocked.job.retry.maxinterval daemon.default.compareddl.ena bled databasequeryservice.usebasev iewsonly deadlock.retry.enabled deadlock.retry.interval deadlock.retry.maxattempts different.session.charsets.en abled event.table.default hadoop.connector.max.task.slo t When set to True, detects any locks on the source/ target objects being moved and retries running the job after a specified interval. Sets an interval to retry running any jobs blocked because of locks on source/target objects. Time unit can be specified as HOURS or MINUTES. Sets the maximum interval for attempting to start any jobs blocked because of locks on source/target objects. Jobs are marked as FAILED after this interval is exceeded if they are still blocked. Time unit can be specified as HOURS or MINUTES. Enables/disables the default compareddl behavior at the daemon level. Sets all data dictionary queries on Teradata source and target systems to use the base views instead of X or VX views. When set to True, if an SQL query execution fails with DBS error (2631) because of a deadlock, retries executing the query after a specified interval. The interval during which to retry executing an SQL query that fails with a DBS deadlock error (2631). Time unit can be specified as SECONDS or MINUTES. The maximum number of attempts to retry executing an SQL query that fails with a DBS deadlock error (2631). Determines whether or not specifying different source and target session character sets in a job is allowed. Default value False means this is not allowed. Default event table in which to save event details. Events are sent to this event table by default when tmsm.mode is either BOTH or ONLY_INTERNAL_TMSM. Individual jobs can use a different event table by using the log_to_event_table job definition parameter. Multiple values can be set as follows: <value>event1</value> <value>event2</value> Specifies the maximum number of concurrent Hadoop Connector tasks. False 1 HOUR 1 HOUR True False 1 MINUTE 10 False NULL 2 14 Installation, Configuration, and Upgrade Guide for Customers Release 16.00

15 Property Description Default hadoop.default.mapper.export hadoop.default.mapper.import hadoop.default.mapper.type hanging.job.check.enabled Specifies the number of mappers for Hadoop to Teradata jobs. This property only used when hadoop.default.mapper.type is DataMover. Default is 8. Specifies the number of mappers for Teradata to Hadoop jobs. This property is only used when hadoop.default.mapper.type is DataMover. Determines which product will decide the default number of mappers for a Hadoop job. Possible values are TDCH and DataMover. If enabled, an internal process awakens periodically and reviews active jobs to see if any are hanging DataMover Disabled hanging.job.check.rate Rate at which to check for hanging jobs (in hours). 1 HOUR hanging.job.timeout.acquisiti on hanging.job.timeout.in.minute s hanging.job.timeout.large.app ly hanging.job.timeout.large.bui ld hanging.job.timeout.large.ini tiate hanging.job.timeout.medium.ap ply hanging.job.timeout.medium.bu ild hanging.job.timeout.medium.in itiate If the progress of a new job is not reported within this period (in hours), the job will be aborted. Timeout is specifically for the acquisition phase. Internal use only. If enabled, causes all hanging.job parameters to use minutes instead of hours. If the progress of a new job is not reported within this period (in hours), the job is aborted. Timeout specifically for the TPTAPI apply phase for a large object. If the progress of a new job is not reported within this period (in hours), the job is aborted. Timeout specifically for the ARC build phase for a large object. If the progress of a new job is not reported within this period (in hours), the job is aborted. Timeout specifically for the initiate phase for a large object. If the progress of a new job is not reported within this period (in hours), the job is aborted. Timeout specifically for the TPTAPI apply phase for a medium object. If the progress of a new job is not reported within this period (in hours), the job is aborted. Timeout specifically for the ARC build phase for a medium object. If the progress of a new job is not reported within this period (in hours), the job is aborted. Chapter 2: Configuring the Environment Configuring the Data Mover Daemon 1 HOUR Disabled 4 HOURS 4 HOURS 4 HOURS 2 HOURS 2 HOURS 2 HOURS Installation, Configuration, and Upgrade Guide for Customers Release

16 Chapter 2: Configuring the Environment Configuring the Data Mover Daemon Property Description Default Timeout specifically for the initiate phase for a medium object. hanging.job.timeout.range.lar ge.min hanging.job.timeout.range.sma ll.max hanging.job.timeout.small.app ly hanging.job.timeout.small.bui ld hanging.job.timeout.small.ini tiate job.allowcommandlineuser job.databaseclientencryption Defines the minimum size (in MB, GB, TB, or default GB if the unit is not provided) for an object to be considered a large object. Defines the maximum size (in MB, GB, TB, or default MB if the unit is not provided) for an object to be considered a small object. If the progress of a new job is not reported within this period (in hours), the job is aborted. Timeout specifically for the TPTAPI apply phase for a small object. If the progress of a new job is not reported within this period (in hours), the job is aborted. Timeout specifically for the ARC build phase for a small object. If the progress of a new job is not reported within this period (in hours), the job is aborted. Timeout specifically for the initiate phase for a small object. When set to True, the daemon will always allow CommandLine requests when the security level is Daemon. When set to True, utilities such as ARC, JDBC, and TPTAPI initiate encrypted sessions to both the source and target database systems. Performance decreases when encryption is initiated. 10 GB 5 MB 1 HOUR 1 HOUR 1 HOUR False False job.default.queryband Provides a set of name/value pairs to be used as the default query band for all jobs. Applicatio nname=dm;v ersion= job.default.queryband.enabled Enable to use of the default query band features. False job.force.direction job.never.target.system job.onlinearchive Forces the direction of data movement from source to target system. Prevents certain database systems from ever being a target system in a Data Mover job. When set to True, online archiving is used for objects that merit the use of ARC. False False 16 Installation, Configuration, and Upgrade Guide for Customers Release 16.00

17 Property Description Default Performance decreases when this setting is used for object availability. Chapter 2: Configuring the Environment Configuring the Data Mover Daemon job.overwriteexistingobjects job.securitymgmtlevel job.usegroupuseridpool job.usesecuritymgmt job.usesyncservice job.useuseridpool repository.purge.definition.e nabled repository.purge.enabled repository.purge.history.unit repository.purge.history.unit count repository.purge.hour repository.purge.minute When set to True, objects that already exist on the target database system are overwritten. The level of security management enabled. Valid choices are Daemon and Job. Defines a set of system names and credentials. When creating a job, this group user id pool can be used for the source or target in place of directly specifying credentials in the job. When set to True, some Data Mover commands require the admin username and password to be specified when executing the command. For a complete list of commands affected by this parameter, see the User Guide. Records any changes to the Data Mover repository tables (inserts/updates/deletes) in an audit log table. The value must be set to True to use the Sync service. Uses a target user from the pool of users. This enables the running of multiple arc tasks at the same time. Enables the automated purging of job definitions. Enables/disables the repository purge feature. The default value False means purging is disabled. The unit for job history data to be kept in the repository before purging should occur. The current supported values are Days, Weeks, Months, and Years. The number of units for job history data to be kept in the repository before purging should occur. This value is combined with the value for repository.purge.history.unit to determine the amount of time before purging should occur for old jobs (for example, 60 days, 3 years, or 10 months). The value of -1 disables the purging by time. The hour when the daily repository purging should start. Default value 1 means 1 am. The minute when the daily repository purging should start. False Job None False False False False Days Installation, Configuration, and Upgrade Guide for Customers Release

18 Chapter 2: Configuring the Environment Configuring the Data Mover Agent Property Description Default repository.purge.percent sqlh.max.task.slot system.default.database.enabl ed target.system.load.slots tmsm.frequency.bytes The percentage of repository permspace that needs to be available to determine when purging should occur. The default value 50 means the repository should be purged when more than 50% of the available permspace is in use. The value of -1 disables the purging by percentage. Specifies the limit for maximum number of concurrent T2H tasks. Enables/disables the default target/staging databases at the system level. The default value False means disabled. Controls the total number of load slots that Data Mover can use at one time on target Teradata systems. Controls the frequency of messages sent to Teradata Ecosystem Manager when using byte-based utilities (for example, ARC). Providing a low value can hurt performance. The recommendation is to use default value False BYTES tmsm.mode Controls how Data Mover directs Teradata Ecosystem Manager messages. Possible values are BOTH, ONLY_REAL_TMSM, ONLY_INTERNAL_TMSM, and NONE. When set to BOTH, messages are sent to the real Teradata Ecosystem Manager and written to the TDI event tables. None Configuring the Data Mover Agent The agent.properties file is located in the /etc/opt/teradata/datamover directory. 1. Edit the agent.properties file and restart the Data Mover agent to implement the changes. For properties that can be set dynamically, the changes take effect one minute after the updated agent.properties file is saved. There is no need to restart the agent service if you are only updating dynamic properties. Installing and Configuring the Data Mover Agent on a Linux Teradata Server 1. Install the appropriate DMAgent software package for your system as follows: 18 Installation, Configuration, and Upgrade Guide for Customers Release 16.00

19 Chapter 2: Configuring the Environment Configuring the Data Mover Agent Operating System Linux (for non- Teradata servers) Actions a. At the command line, type the following to install the DMAgent and TTU packages:./ dminstallupgradeagent b. Answer the prompts as needed, and press Enter to accept the defaults where appropriate. c. Type rpm -qa grep DMAgent to verify the installation. The agent.properties File The following table shows the properties in agent.properties. Property Description Default Value agent.id=id Unique identifier for this agent. Agent1 arc.port=port number cluster.enabled= setting for cluster broker.port=port number broker.url=url log4j.appender.logfile=org.apache.log4j.rollingfilea ppender log4j.appender.logfile.fil e=log file path log4j.appender.logfile.lay out=org.apache.log4j.patte rnlayout Port number that can be used by Teradata ARC to manage ARC streams. When set to True, establishes a connection to a secondary Java Message Service (JMS) broker in case the primary JMS broker fails. The port number of the machine where the Java Message Service (JMS) Message Broker is listening. The hostname or IP address of the machine running the Java Message Service (JMS) Message Broker. Informs the logging application to use a specific appender. It is recommended that this property value not be changed. Relative or absolute path of the log file. If changing log file location, specify the absolute path of the file. For Windows, specify back slash instead of forward slash, for example, C: \ProgramFile\Teradata\Log \dmagent.log. Both file path and file name can be set dynamically. 1 Dynamic property. 1 Do not edit. This is an internal setting for logging infrastructure False localhost dmagent.log Installation, Configuration, and Upgrade Guide for Customers Release

20 Chapter 2: Configuring the Environment Configuring the Data Mover Agent Property Description Default Value log4j.appender.logfile.max BackupIndex=number of backup logging files log4j.appender.logfile.max FileSize=maximum size of log files log4j.appender.logfile.lay out.conversionpattern=log file pattern layout log4j.rootlogger=level of logging The number of backup logging files that are created. After the maximum number of files has been created, the oldest file is erased. Dynamic property. 1 Example If Max Backups = 3, three backup logs are created: dmagent.log.1 dmagent.log.2 dmagent.log.3 If the current dmagent.log size exceeds 10MB, it rolls to become the new dmagent.log.1 and a new dmagent.log is created. The previous dmagent.log.2 becomes the new dmagent.log.3. The previous dmagent.log. 3 is deleted. The maximum size of the logging file before being rolled over to backup files. Dynamic property. 1 The pattern of the log file layout, where: d = date t = thread p = log level c = class name m= message n = platform dependent line separator Dynamic property. 1 Information for creating a layout is at: apidocs/org/apache/log4j/patternlayout.html 6 levels of logging, TRACE DEBUG INFO WARN ERROR FATAL. From trace level to application error. LOG_LEVEL can be updated dynamically, but not logfile. 1 Value is: LOG_LEVEL, logfile 3 10MB %d [%t] %-5p %c{3} - %m%n INFO,logfile agent.maxconcurrenttasks=m aximum number of tasks Do not remove the term logfile. The maximum number of tasks allowed to run on this agent at the same time. Note that tasks are distributed to agents using a round robin method. Task size is not currently 5 20 Installation, Configuration, and Upgrade Guide for Customers Release 16.00

21 Property Description Default Value considered, so load may not be balanced if one agent is randomly assigned larger tasks than another. Chapter 2: Configuring the Environment Configuring the Data Mover Command-Line Interface logger.usetvilogger=settin g for TVI messages The TVI logger can be set to true or false. If set to true, fatal error messages are sent to TVI. Dynamic property. 1 True 1 Dynamic Properties For properties that can be set dynamically, the changes take effect one minute after the updated daemon.properties file is saved. There is no need to restart the daemon service if you are only updating dynamic properties. For example: If you changed the value of log4j.rootlogger from the default of INFO, logfile to DEBUG, logfile, any debug messages generated would start appearing in the log file one minute after saving the updated properties file. If you changed the value of agent.maxconcurrenttasks from the default value of 5 to a new value of 6, the new value of 6 would take effect one minute after saving the updated agent.properties file. Configuring the Data Mover Command-Line Interface Configuring the Data Mover Command-Line Interface on a Linux Teradata Server The Data Mover Command-Line Interface is installed for Linux Teradata servers with PUT. Configure the commandline properties if you want to customize these settings. 1. Edit the commandline.properties file located in the /etc/opt/teradata/datamover directory. Installing and Configuring the Data Mover Command-Line Interface on Non-Teradata Servers The Data Mover Command-Line Interface must be installed for Linux on non-teradata servers, Windows, Solaris Sparc, and IBM AIX systems using the following procedure. You cannot use PUT to install the Command-Line Interface on those systems. Steps 1 through 4 do not apply to installation on Windows systems. 1. Add the following lines of code to the end of the /etc/profile file to update the JAVA_HOME and PATH environment variables for all users: export JAVA_HOME={full path of java installation location} export PATH=$JAVA_HOME/bin:$PATH 2. Run the command: source /etc/profile 3. Run the command: java -version and verify that the output shows JRE.1.7. Installation, Configuration, and Upgrade Guide for Customers Release

22 Chapter 2: Configuring the Environment Configuring the Data Mover Command-Line Interface 4. Open the.profile file of the root user and verify that the values for the JAVA_HOME and PATH environment variables are the same as those defined in /etc/profile. If the values are different, the java -version command will not produce the correct output during install time, and the installation will fail. 5. Install the appropriate DMCmdline software package for your system as follows: Operating System Linux (for non- Teradata servers) Actions a. At the command line, type export DM_INTERACTIVE_INSTALL=1 to set the environment variable for interactive install. b. At the command line, type the following: gunzip DMCmdline linux_i tar.gz tar xvf DMCmdline linux_i tar cd DMCmdline.16.00* rpm -Uvh DMCmdline linux_noarch rpm c. Answer the prompts as needed, and press Enter to accept the defaults where appropriate. d. Type rpm -qa grep DMCmdline to verify the installation. Windows a. Copy the Data Mover directory on the media to a folder on the hard drive. b. Go to DataMover/Windows and unzip tdmwindows windows_i zip. c. Go to the DISK1 directory and run setup.exe. d. Answer the prompts as needed and press Next to accept defaults where appropriate. e. Click Install when finished. f. Go to Start > Control Panel > Add or Remove Programs to verify installation. Solaris Sparc a. At the command line, type the following to install: gunzip tdm-solaris solaris_sparc tar.gz tar xvf tdm-solaris solaris_sparc tar pkgadd -d 'pwd' DMCmdline b. Answer the prompts as needed and press Enter to accept defaults where appropriate. c. Type pkginfo -l DMCmdline to verify the installation. IBM AIX a. At the command line, type the following to install: gunzip tdm-aix aix_power tar.gz tar xvf tdm-aix aix_power tar installp -acf -d./dmcmdline DMCmdline b. Answer the prompts as needed and press Enter to accept defaults where appropriate. c. Type lslpp -l "DM*" to verify the installation. 6. If the broker URL needs to be changed, edit the commandline.properties file located in the TDM_install_directory\CommandLine\commandline.properties directory after installation. 7. Specify the broker URL and broker port number for communicating with the JMS bus. 22 Installation, Configuration, and Upgrade Guide for Customers Release 16.00

23 Chapter 2: Configuring the Environment Configuring the Data Mover Command-Line Interface The broker URL value is the machine name or IP address of the machine where ActiveMQ runs. The broker port value should also be the same as the port number that ActiveMQ uses. The defaults are broker.url=localhost and broker.port= The commandline.properties File Property Description Default Value cluster.enabled= setting for cluster broker.port=<port> broker.url=<url> log4j.appender.logfile= org.apache.log4j.rollin gfileappender log4j.appender.logfile. file=<file path name> log4j.appender.logfile. layout=org.apache.log4j.patternlayout When set to True, establishes a connection to a secondary Java Message Service (JMS) broker in case the primary JMS broker fails. The port number of the machine where the Java Message Service (JMS) Message Broker is listening. The hostname or IP address of the machine running the Java Message Service (JMS) Message Broker. Informs the logging application to use a specific appender. It is recommended that this property value not be changed. Relative or absolute path of the logfile. If changing log file location, specify the absolute path of the file. For Windows, specify back slash instead of forward slash, for example, C: \Program File\Teradata\Log\dmCommandLine.log. Do not edit. This is an internal setting for logging infrastructure. False localhost dmcommandli ne.log log4j.appender.logfile. maxbackupindex=<number of backup files> log4j.appender.logfile. maxfilesize=<maximum size of log files> The number of backup logging files that are created. After the maximum number of files has been created, the oldest file is erased. Example If Max Backups = 3, three backup logs are created: dmcommandline.log.1 dmcommandline.log.2 dmcommandline.log.3 If the current dmcommandline.log size exceeds 10MB, it rolls to become the new dmcommandline.log.1 and a new dmcommandline.log is created. The previous dmcommandline.log.2 becomes the new dmcommandline.log.3. The previous dmcommandline.log.3 is deleted. The maximum size of the logging file before being rolled over to backup files. 3 10MB Installation, Configuration, and Upgrade Guide for Customers Release

24 Chapter 2: Configuring the Environment Configuring the Data Mover REST Service Property Description Default Value log4j.appender.logfile. layout.conversionpatter n=<log file pattern layout> log4j.rootlogger=<level of logging> The pattern of the log file layout, where: d = date t = thread p = log level c = class name m= message n = platform dependent line separator Information for creating a layout is at: logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/ PatternLayout.html 6 levels of logging, TRACE < DEBUG < INFO < WARN < ERROR < FATAL. From trace level to application error. Value is: <LOG_LEVEL>, logfile Do not remove the term logfile. %d [%t] %-5p %c{3} - %m%n INFO,logfil e Configuring the Data Mover REST Service During installation of Data Mover, the DM REST component and REST Container are installed and started automatically. You need to configure the tdmrest.properties file for your environment and restart the service. 1. In the directory /etc/opt/teradata/datamover, locate tdmrest.properties. 2. Configure the following properties: Property Description Default Value broker.url=url broker.port=port cluster.enabled=setting Hostname or IP address of the machine running the Java Message Service (JMS) message broker. Port number of the machine where the JMS message broker is listening. When set to true, establishes a connection to a secondary JMS broker for the cluster if primary JMS broker fails. localhost False 3. Restart the Data Mover REST service using the following command: /etc/init.d/tdrestd start 24 Installation, Configuration, and Upgrade Guide for Customers Release 16.00

25 About Configuring High Availability A High Availability configuration is the base configuration for a Data Mover system. If the primary component of a system goes down, a High Availability configuration ensures that the system continues to function with a secondary component. This configuration depends on a monitoring service, which monitors the primary, or master, components through SSH connections to see if services are running. If any of the main components are down, a failover sequence begins the process of allowing the slave component to take over for the master component. The daemon, agent, and Sync service that will be monitored by the monitoring service on the master and slave components must be run using user dmuser. The monitoring service cannot be used to monitor the daemon, agent, and Sync service components if a user other than dmuser has been set up to run these services. The following work together to maintain High Availability: SSH logons Dual Active Java Message Service (JMS) brokers Synchronization service Master and slave components Chapter 2: Configuring the Environment About Configuring High Availability Configuring Automatic Failover Data Mover provides automatic failover support when multiple Data Mover servers are configured in a dual environment. Automatic failover configuration requires that: You have two additional monitoring severs to monitor the master and slave components. It is highly recommended that you use a Viewpoint managed server for this purpose. Each monitoring server must be local to the site and ideally be attached to the same network as the components being monitored to avoid automatic failovers caused by network partitions. The DMFailover package must be installed on all servers, including the master and slave daemon, monitoring, and agent servers that are part of the cluster. If additional monitoring servers are not available, you can enable failover by using the Data Mover synchronization service. The synchronization service alone does not support automatic failover and requires manual intervention to enable failover from master to slave components. If configuring the synchronization service without configuring automatic failover, see Configuring the Synchronization Service Without Automatic Failover. When using the synchronization service with failover, use the public IP address of the system, not the localhost hostname or the IP address. The following files, located in the /opt/teradata/client/nn.nn/datamover/failover/ folder, are required when using the monitoring service. Where nn.nn in the filename refers to the major and minor version numbers of Data Mover. File /etc/opt/teradata/ datamover/ failover.properties Description Specifies the master and slave components to be monitored Installation, Configuration, and Upgrade Guide for Customers Release

26 Chapter 2: Configuring the Environment About Configuring High Availability File /opt/teradata/client/ nn.nn/datamover/ failover/dmcluster /opt/teradata/client/ nn.nn/datamover/ failover/ DMFailover.jar /etc/opt/teradata/ datamover/ monitor.properties Description Script for setting up SSH log on, configuring the servers in master and slave modes, starting and stopping the monitoring service, and checking the status of the master and slave components Executable binary file used for automatic failover Specifies if Server Management alerts need to be sent if the monitoring service detects a failure. This file is used only on the monitoring server. The following tasks must be performed to configure automatic failover: 1. Verifying Data Mover Package Installation 2. Setting Up Host Files or DNS 3. Verifying Required Ports Open 4. Defining Unique Data Mover Agent Names 5. Synchronizing the Master and Slave Repositories 6. Configuring Dual Active Java Message Service (JMS) Brokers 7. Configuring the Sync Service 8. Configuring the Cluster and Starting the Monitoring Service 9. Checking the Status of Master and Slave Components 10. Starting the Synchronization Service 11. Verifying Failover Configuration High Availability Configuration Scenario The following table defines the terms used during the failover configuration process. To configure high availability successfully, each step must be performed in order as listed in the Configuring Automatic Failover section. Following each required step is a use-case example to demonstrate the configuration process. Example Server Name Server Type Description DM1 Original-master The system primarily used as the master system before failover occurred. Typically, the server where the primary Data Mover daemon runs, along with one Data Mover agent. DM2 Designated-slave The slave system assigned to take responsibility as the master system if the original-master stops working. If the DM1 server fails, this is the server that becomes the next master. In the meantime, one Data Mover agent runs here. DM3 Additional agent Server where one Data Mover agent runs. Not a slave server. 26 Installation, Configuration, and Upgrade Guide for Customers Release 16.00

27 Chapter 2: Configuring the Environment About Configuring High Availability Example Server Name Server Type Description VP1 Orignal-master monitor Server where the primary Data Mover monitor runs. This monitor continuously checks that DM1 is running properly and initiates failover to DM2 if necessary. VP2 Designated-slave monitor Server where the secondary Data Mover monitor runs. This monitor only becomes active if failover occurs. If failover occurs, this monitor continuously checks that DM2 is running properly and initiates failover to DM1 if necessary. Verifying Data Mover Package Installation Before configuring automatic failover, verify that all servers involved in the configuration have the correct packages installed. 1. Verify the original-master, designated-slave, and slave-only servers have the following required packages installed: DMAgent-XX.XX.XX.XX-1 DMCmdline-XX.XX.XX.XX-1 DMDaemon-XX.XX.XX.XX-1 DMFailover-XX.XX.XX.XX-1 DMSync-XX.XX.XX.XX-1 Servers must be one of the following server types: Data Mover Teradata Managed Server Consolidated Teradata Managed Server with Data Mover 2. Verify the additional agent server has the DMAgent-XX.XX.XX.XX-1 package installed. Server must be one of the following server types: Data Mover Teradata Managed Server Consolidated Teradata Managed Server with Data Mover Cloud-based instance of Data Mover (for example, Teradata Managed Cloud or Amazon AWS) 3. Verify the original-master monitor and the designated-slave monitor have the DMFailover- XX.XX.XX.XX-1 package installed. Viewpoint Teradata Managed Server is recommended. Example: Verifying Data Mover Package Installation 1. On DM1 and DM2, verify that the following packages are installed: DMAgent-XX.XX.XX.XX-1 DMCmdline-XX.XX.XX.XX-1 DMDaemon-XX.XX.XX.XX-1 DMFailover-XX.XX.XX.XX-1 DMSync-XX.XX.XX.XX-1 Installation, Configuration, and Upgrade Guide for Customers Release