Hitachi Hyper Scale-Out Platform (HSP) Hortonworks Ambari VM Quick Reference Guide

Transcription

1 Hitachi Hyper Scale-Out Platform (HSP) MK-95HSP October 2016

2 2016 Hitachi, Ltd. All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or stored in a database or retrieval system for commercial purposes without the express written permission of Hitachi, Ltd., or Hitachi Data Systems Corporation (collectively, Hitachi ). Licensee may make copies of the Materials provided that any such copy is: (i) created as an essential step in utilization of the Software as licensed and is used in no other manner; or (ii) used for archival purposes. Licensee may not make any other copies of the Materials. "Materials" mean text, data, photographs, graphics, audio, video and documents. Hitachi reserves the right to make changes to this Material at any time without notice and assumes no responsibility for its use. The Materials contain the most current information available at the time of publication. Some of the features described in the Materials might not be currently available. Refer to the most recent product announcement for information about feature and product availability, or contact Hitachi Data Systems Corporation at Notice: Hitachi products and services can be ordered only under the terms and conditions of the applicable Hitachi agreements. The use of Hitachi products is governed by the terms of your agreements with Hitachi Data Systems Corporation. By using this software, you agree that you are responsible for: 1) Acquiring the relevant consents as may be required under local privacy laws or otherwise from authorized employees and other individuals to access relevant data; and 2) Verifying that data continues to be held, retrieved, deleted, or otherwise processed in accordance with relevant laws. Notice on Export Controls. The technical data and technology inherent in this Document may be subject to U.S. export control laws, including the U.S. Export Administration Act and its associated regulations, and may be subject to export or import regulations in other countries. Reader agrees to comply strictly with all such regulations and acknowledges that Reader has the responsibility to obtain licenses to export, re-export, or import the Document and any Compliant Products. Hitachi is a registered trademark of Hitachi, Ltd., in the United States and other countries. AIX, AS/400e, DB2, Domino, DS6000, DS8000, Enterprise Storage Server, eserver, FICON, FlashCopy, IBM, Lotus, MVS, OS/390, PowerPC, RS6000, S/390, System z9, System z10, Tivoli, z/os, z9, z10, z13, z/vm, and z/vse are registered trademarks or trademarks of International Business Machines Corporation. Active Directory, ActiveX, Bing, Excel, Hyper-V, Internet Explorer, the Internet Explorer logo, Microsoft, the Microsoft Corporate Logo, MS-DOS, Outlook, PowerPoint, SharePoint, Silverlight, SmartScreen, SQL Server, Visual Basic, Visual C++, Visual Studio, Windows, the Windows logo, Windows Azure, Windows PowerShell, Windows Server, the Windows start button, and Windows Vista are registered trademarks or trademarks of Microsoft Corporation. Microsoft product screen shots are reprinted with permission from Microsoft Corporation. All other trademarks, service marks, and company names in this document or web site are properties of their respective owners.

3 Contents Preface Intended audience Product version Document revision level Accessing product documentation Getting help Comments v vi vi vi vi vii vii Quick Reference 1 KVM image details 2 CentOS VM software bundle 3 Repository VM software bundle 3 Bundled components 4 Licenses used 5 Contents iii

4 iv Contents

5 Preface This document provides a quick reference for Ambari and Hortonworks Hadoop Data Platform (HDP) 2.4 VM for Hitachi Hyper Scale- Out Platform (HSP). This Preface includes the following information: Intended audience Product version Document revision level Accessing product documentation Getting help Comments Preface v

6 Intended audience Intended audience This document is intended for system administrators, Hitachi Data Systems representatives, and authorized service providers who require quick reference information for Hortonworks Hadoop Data Platform (HDP) 2.4 VM for Hitachi Hyper Scale-Out Platform (HSP). Readers of this document should be familiar with the following: KVM based Virtual Machine images Ambari administration Linux operating system and working in a restricted shell environment Site-specific network information Product version This document applies to Hyper Scale-Out Platform release or later. Document revision level Revision Date Description MK-95HSP April 2016 Initial release MK-95HSP April , supersedes and replaces MK-95HSP MK-95HSP June , supersedes and replaces MK-95HSP MK-95HSP October , supersedes and replaces MK-95HSP Accessing product documentation Product documentation is available on Hitachi Data Systems Support Connect: Check this site for the most current documentation, including important updates that may have been made after the release of the product. vi Preface

7 Getting help Getting help Hitachi Data Systems Support Portal is the destination for technical support of products and solutions sold by Hitachi Data Systems. To contact technical support, log on to Hitachi Data Systems Support Connect for contact information: Hitachi Data Systems Community is a global online community for HDS customers, partners, independent software vendors, employees, and prospects. It is the destination to get answers, discover insights, and make connections. Join the conversation today! Go to community.hds.com, register, and complete your profile. Comments Please send us your comments on this document: hsp.documentation.comments@hds.com Include the document title and part number, including the revision (for example, -01), and refer to specific sections and paragraphs whenever possible. All comments become the property of Hitachi Data Systems. Thank you! Preface vii

8 viii Preface

9 Quick Reference This chapter describes the Hortonworks Hadoop Data Platform (HDP) KVM based Virtual Machine image for HSP. KVM image details Bundled components Licenses used Quick Reference 1

10 KVM image details KVM image details Hortonworks Ambari deployment begins with a fresh CentOS 7 KVM based VM image and a repository VM image. The package consists of two VM images: CentOS-7.2 HSP-1.2 Vanilla HSP VM template Repo-with_HDP-2.4_Ambari Repository VM The following describes the basic steps involved in deploying the VMs, see the Hortonworks Ambari VM deployment guide for full instructions. 1. Add the CentOS template and repository template to the HSP cluster and deploy a single instance of the repository VM to the cluster. 2. Deploy the Ambari VMs. One or more Ambari VMs can be deployed to form the HDP cluster. 3. The install script is packaged as an RPM on the repository VM. The Ambari-prep noarch.rpm script sets up the /etc/hosts file so that all of the nodes can find each other. SSH into the Ambari VM you will use as your Ambari server and use wget to fetch the script. ssh <Ambari Server VM IP Address> wget <repository VM IP Address>/rpm/ambari-2.2.2/AmbariPrep noarch.rpm 4. Install the script package using yum: yum -y install AmbariPrep noarch.rpm 5. Run the script. /opt/escale/bin/ambariprep.py --repo-host <Repository VM IP address> When complete Ambari will have been installed, configured and started. The Ambari server GUI can take up to three minutes to start responding on port Open a browser, log into the Ambari server instance and configure the new cluster from the browser server IP address>: Quick Reference

11 KVM image details CentOS VM software bundle CentOS 7.2 x64 - Linux Operating system Installed Software Oracle JDK 1.8 jdk-8u60-linux-x64 Java virtual machine Python Python interpreter (for scripting language) PostgreSQL el7_2.x86_64 Repository VM software bundle HDP VM with Ambari Server/local HDP 2.4 repositories. Packaged Software Ambari Server HDP Hortonworks Hadoop software HDFS compatible plugin for HSP Quick Reference 3

12 Bundled components Bundled components The following Hortonworks HDP components have been bundled and are pending certification to be compatible with the Hitachi Scale-Out Platform version Apache DataFu Apache Hadoop Apache HBase Apache Hive Apache Pig Apache Spark Apache Sqoop Apache Tez Apache ZooKeeper Quick Reference

13 Licenses used Licenses used This media kit includes software covered under the following licenses: Apache License, Version 2.0 (the "License") The Platform Components comprising the Hortonworks Data Platform (HDP) are released under the Apache 2.0 License. Unless required by applicable law, software distributed under the Apache License is distributed on an AS IS BASIS. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the Apache License at: Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. Optional Add-Ons and 3rd-Party Components While HDP is provided under the terms of the Apache 2.0 License, it is also commonly used with 3rd-Party Components and Optional Add-Ons (example Hive ODBC Driver). When you choose to use those components, it is recommended you read and understand the licensing terms specific to each of those components. Also note that Add-Ons can be downloaded separately from HDP and each comes with instructions on how to install and leverage the features alongside HDP. Trademarks: Apache and the Apache feather logo are trademarks of The Apache Software Foundation CentOS is a trademark of Red Hat, Inc. Hortonworks Data Platform (HDP) is trademark of Hortonworks Inc. Python is a registered trademark of the Python Software Foundation Quick Reference 5

14 6 Quick Reference

15

16 Hitachi Data Systems Corporate Headquarters 2845 Lafayette Street Santa Clara, California U.S.A. Regional Contact Information Americas Europe, Middle East, and Africa +44 (0) Asia Pacific MK-95HSP013-03