Connect. Communicate. Collaborate. Click to edit Master title style. Installing and Configuring the perfsonar Services

Connect. Communicate. Collaborate Click to edit Master title style Installing and Configuring the perfsonar Services

COURSE OBJECTIVES By the end of this course you will be able to: Describe key perfsonar services. Install, configure and deploy the following perfsonar services: The Lookup Service The BWCTL Measurement Point The SSH / Telnet Measurement Point The RRD Measurement Archive The SQL Measurement Archive Identify how to interact with the Service Desk. 2

THE PERFSONAR ADMINISTRATION GUIDE Please note that these slides do not contain detailed instructions about how to install the perfsonar services. For detailed instructions, please refer to the perfsonar Administration Guide. 3

COURSE OUTLINE Module 1 perfsonar Technical Overview Module 2 General Installation Considerations Module 3 Installing and Configuring the Lookup Service Module 4 Installing and Configuring the BWCTL MP Module 5 Installing and Configuring the SSH / Telnet MP Module 6 The Service Desk Module 7 Installing and Configuring the RRD MA Module 8 Carrying out an Installation on Debian Module 9 Installing and Configuring the SQL MA Module 10 Feedback on perfsonar Installation and Configuration 4

Connect. Communicate. Collaborate Click to edit Master title style MODULE 1: perfsonar TECHNICAL OVERVIEW

WHAT IS PERFSONAR? perfsonar is: A project consisting of a variety of organisations and individuals A set of protocols that: Assume a set of services based on defined roles. Define their communication syntax and semantics. Allow anyone to develop an implementation of a service. A set of code Service implementations 6

WHAT ARE THE PERFSONAR SERVICES? The perfsonar services form an interoperable, distributed performance measurement middleware framework. perfsonar stands for PERformance Service Oriented Network monitoring Architecture. 7

DESIGN OBJECTIVES perfsonar is designed to be: Decentralised and Scaleable Large number of networks and services, large volume of data Each domain can set its own security policy Dynamic and Self-Discovering Add and remove components during operation Components automatically become aware of one another Secure Will not put participating networks at risk of attack or congest them Modular Allows discrete module development 8

THREE-TIER ARCHITECTURE The perfsonar framework: Is middleware. Is distributed between domains. Facilitates inter-domain performance information sharing. perfsonar services wrap existing measurement tools. 9

WHAT IS A SERVICE? A Service is a tightly defined, independent entity that has a well defined interface and can be accessed directly. 10

ARCHITECTURAL PRINCIPLES Postulate: all measurement systems contain a combination of: Measurement tools Data Storage Security and Policy implementation Topology information Visualization Services have been identified that are: Based on these roles Based on requirement for other functionality such as Service discovery, resource protection, data formatting 11

THE PERFSONAR SERVICES FRAMEWORK (1) perfsonar divides measurement system tools into generic families or services. Each service has a protocol. 12

THE PERFSONAR SERVICES FRAMEWORK (2) Client Client Authentication Service Transformation Service Lookup Service Measurement Point Point Service Resource Protector Service Measurement Archive Service 13

THE CLIENT Client Client Authentication Service Transformation For example the perfsonar Service Visualisation User Interface Lookup Service Measurement Point Point Service Measurement Archive Service Resource Protector Service 14

THE LOOKUP SERVICE Client Client Authentication Service Registers services including their Transformation capabilities. Service Lookup Service Facilitates complex searches. Participates Measurement in network of Lookup Services Point Point Service Measurement Archive Service Resource Protector Service 15

THE LOOKUP SERVICE (1) Purpose: all other services must register with the lookup service in order to participate in the framework. 16

THE LOOKUP SERVICE (2) Other services (including measurement points) register their existence with a lookup service, by delivering lookup information : Location Type of Service Each domain has an instance of the lookup service These instances (will) communicate with one another Clients find other services by querying the lookup service. All the client needs to know is the URL of a Lookup Service 17

THE AUTHENTICATION SERVICE Client Client Authentication Service Measurement Point Point Service Transformation Service Lookup Service Provides authentication for clients and protects privacy. Can be federated. Measurement Archive Service Resource Protector Service 18

THE AUTHENTICATION SERVICE Purpose: ensures client-privacy and domain security by using role-based authentication and authorisation. 19

THE MEASUREMENT POINT SERVICE Client Client Exposes measurement tools and publishes Authentication their data. Service Transformation Service Lookup Service Measurement Point Point Service Measurement Archive Service Resource Protector Service 20

MEASUREMENT POINTS Measurement Points: Belong to domains (domain = a network) Each measurement point implementation maps to a tool that provides one or several metrics Examples: One-way-loss Jitter TCP throughput Show commands on routers 21

THE MEASUREMENT ARCHIVE SERVICE Client Client Exposes Authentication measurement databases Service and file stores. Publishes measurement data Avoids queries to multiple Measurement Point Services Measurement Point Point Service Transformation Service Resource Protector Service Lookup Service Measurement Archive Service 22

MEASUREMENT ARCHIVES Purpose: expose measurement data held in databases or file systems. Wrapper for any type of storage mechanism (SQL Databases, RRD files, etc) Provides access to recent and stored data Collects information from Measurement Points, Transformation Services or other Measurement Archives. i.e. it subscribes to these other services Can also write to databases and file systems 23

THE TRANSFORMATION SERVICE Transforms data in a variety of ways (e.g., aggregation, filtering, Authentication correlation). Service For future development. Precise role needs to be defined. Measurement Point Point Service Client Client Transformation Service Resource Protector Service Lookup Service Measurement Archive Service 24

THE RESOURCE PROTECTOR SERVICE Controls the comsumption of limited resources (e.g. network bandwidth). Client Client Authentication Service Transformation Service Lookup Service Measurement Point Point Service Resource Protector Service Measurement Archive Service 25

THE PERFSONAR SERVICES FRAMEWORK (3) Each service has a specific function. Each instance of a service belongs to an administrative domain. 26

PROTOCOLS perfsonar has developed a set of protocols for sharing performance data. These: Assume the services set out in the framework. Define their communication syntax (schema) and semantics (business logic). Allow anyone to develop an implementation of a defined service. Are compliant with the Global Grid Forum s Network Measurement Working Group (NM-WG) schema specification. Are based on XML over SOAP. You can see the protocols as rules and tools for participating in the perfsonar framework. 27

GENERIC SERVICES AND SERVICE IMPLEMENTATIONS Measurement Archive Service Common functionality, common protocol Implementation: tool-specific functionality added Implementation: tool-specific functionality added RRD Measurement Archive service implementation Round Robin Database files SQL Measurement Archive service implementation SQL database Measurement Point Service Common functionality, common protocol Implementation: tool-specific functionality added Implementation: tool-specific functionality added BWCTL Measurement Point service implementation BWCTL measurement tool SSH / Telnet Measurement Point service implementation SSH / Telnet measurement tool 28

SERVICE IMPLENTATIONS The perfsonar project has also developed a set of service implementations that use the defined protocols. Some of these have been developed by JRA1, within the GEANT2 project for the perfsonar pilot: The Lookup Service The BWCTL Measurement Point The SSH / Telnet Measurement Point The RRD Measurement Archive The SQL Measurement Archive 29

DATA COLLECTION, NORMALISATION AND SHARING VIA THE FRAMEWORK 30 Standardised Syntax Standardised Syntax Tool-specific commands Measurement Data Tool-specific commands Measurement Data Data normalization / conversion to XML Data normalization / conversion to XML Data normalization / conversion to XML Data normalization / conversion to XML

FRAMEWORK FACILITATES NORMALISED END-TO-END PERFORMANCE DATA (SIMPLIFIED DEPICTION) 31

THE PERFSONAR PILOT: MEASUREMENT AND ARCHIVING SERVICES 32

THE PERFSONAR PILOT 33

THE ROADMAP Enhanced Lookup Service: Lookup Services (ideally one per domain) peer with one another for increased ease of data discovery. Authentication Service: Collaborating with JRA5 and Edugain to produce an implementation of the service. A data subscriber (client) and a data producer (server) can communicate directly across domains provided that they are authorised to do so. Further service implementations using Netflow information or packet capture features 34

TECHNICAL OVERVIEW: SUMMARY perfsonar will provide an infrastructure to: Locate data sources Authenticate and authorise clients Protect resources and ration their usage Retrieve, normalise, transform and share data Only a partial infrastructure is implemented in the pilot perfsonar is a flexible and open framework: perfsonar services can wrap around existing data collection tools 35

THE PERFSONAR SERVICES FRAMEWORK - RECAP How do we expose measurement tools? Measurement point services How do we expose measurement databases and file stores? Measurement archive services How do we transform data (aggregate, correlate, filter etc.)? Transformation services How do we locate all these services and their capabilities? Lookup Services How do we protect resources? Resource Protection Services How do we ensure a client is allowed to access a service? Authorisation and Authentication Services 36

MODULE 2 GENERAL INSTALLATION CONSIDERATIONS

GENERAL INSTALLATION PREREQUISITES (1) All of the perfsonar services require the following: Recommended operating system: Redhat Linux / Fedora. You can install on other platforms, but perfsonar has not been tested on these Installations on Windows are not supported wget command must be available on the OS Required by installers to download software Perl module: LWP Perl version 5.6.1 or higher These prerequisites must be manually installed before you begin installation of the perfsonar services. 38

GENERAL INSTALLATION PREREQUISITES (2) All of the perfsonar services except for the BWCTL Measurement Point (a non-java application) require the following: Java Developers Kit (JDK) version 1.5 or higher Already installed for you on the training server Note that the RRD MA must have JDK version 1.5 (not any other) Apache Ant 1.6.x Tomcat application server Jakarta Tomcat These prerequisites must be manually installed before you begin installation of the perfsonar services. Note: Tomcat can be automatically installed by the bundle installer, but it recommended that you download Tomcat from the Apache web-site and manually install it before running the bundle installer. 39

OTHER INSTALLATION PREREQUISITES In addition to the General Installation prerequisites, each service has one or more prerequisites that are specific to it. Some must be installed manually by you before you begin to install a service: These are listed in subsequent course modules and are also documented in the Installation Manual. Some can be installed automatically by the perfsonar Bundle Installer: These are documented in the perfsonar Administration Guide. 40

THE ROLE OF THE ADMINISTRATION GUIDE The perfsonar Administration Guide: Lists all pre-requisite software. Provides step-by-step instructions explaining how to install the perfsonar services. Will be used extensively during this course. Will be handed out to you in hard copy. Should be used as your guide when you are installing the services on your own servers. During the exercises, please point out any inconsistencies or errors in the guide to your trainer. 41

THE ROLE OF THE SERVICE DESK The Service Desk is being set up as a single point of contact for all issues relating to perfsonar installation, configuration and use. More information about the service desk is provided in module 6 of the course. 42

INSTALLATION STEPS To install the perfsonar services: Unzip and untar the bundle installer (perfsonar-2.0.tar.gz) Empty the CLASSPATH variable. Execute the bundle installer Follow the on-screen dialogue For detailed instructions, refer to the perfsonar Administration Guide. 43

HOW DOES THE BUNDLE INSTALLER WORK? The bundle installer will ask you whether you want to: Install a new service Modify or test an existing service Give feedback to the perfsonar team If you choose to install a new service: The installer will ask you to choose from the six available services It will then: Automatically download the appropriate installation files and execute the installation steps in sequence Ask you interactive questions about the installation 44

CONFIGURING THE SERVICES STITCHING Before they can be used, several of the perfsonar service implementations require a type of configuration known as stitching. Stitching: Is the process of configuring metadata for your service. Metadata is data that describes other data. E.g. data units, interface name, direction of traffic etc. Usually involves the creation of a metadata configuration file. 45

STITCHING IN CONTEXT (1) Instance of RRD Measurement Archive Service Performance data Performance data RRD file MRTG MRTG In Out POS-6/0 router Performance data Performance data RRD file Cricket Cricket In Out GEO/0 router Stitching allows a service to understand the data it is dealing with. Examples: Which router? Inbound or outbound traffic? What data units? Etc. 46

STITCHING IN CONTEXT (2) Stitching allows a service to understand the data it is dealing with. Examples: Which link? Which interfaces? Location of interfaces? Etc. 47

THE ADVANTAGES OF STITCHING Ultimately, stitched metadata tells the perfsonar framework precisely what kind of data your service sends or can receive. Why this approach? The ability to perform stitching makes service implementations flexible. E.g. the RRD Measurement Archive can handle data from RRD files in different networks that are structured in different ways. Stitching allows service implementations to deal with multiple flavours of the same kind of data. E.g. a single instance of the SQL Measurement Archive service can deal with both inbound and outbound traffic since you can mark each data source as providing information about either inbound or outbound traffic. 48

OTHER CONSIDERATIONS If you are installing multiple services in the same Tomcat instance, using the same exist XML database (as in training): Each service requiring XML database access must have a different username and password You must ensure that each service is allocated a unique name for its own collection Must be manually created for the Lookup service Ensure that you only install one instance of ant on your machine Putting multiple ant installations in the same path causes problems 49

BACKUPS Once you have successfully installed and configured a service, back it up. Backup the webapps directory related to the service Located by default within your perfsonar bundle directory Can be used to restore the service Backup your metadata configuration files 50

ACTIVITIES (1) Installing Tomcat Install Change ports in Tomcat s.conf file (Two ports - must be unique for each participant) Start up Installing Ant Install Change path variable in bash profile to include Ant location Start up 51

ACTIVITIES (2) Notes: Java Developers Kit (JDK) is already installed on the server A UNIX account has been created for each of you In your UNIX account s home directory you will find: Tomcat installation files Ant installation files The perfsonar bundle installer 52

MODULE 3 INSTALLING AND CONFIGURING THE LOOKUP SERVICE

THE LOOKUP SERVICE (1) All other services must register with the lookup service in order to participate in the framework. 54

THE LOOKUP SERVICE (2) Other services (including measurement points) register their existence with the lookup service, by delivering lookup information : Location (URL) Type of Service Service-specific information For example an Measurement Point will tell the Lookup Service what kind of measurements it can take Clients find other services by querying the lookup service. All the client needs to know is the URL of the Lookup Service 55

THE LOOKUP SERVICE (3) The LS keeps Lookup Information in Lookup Storage (LSSTORE), an XML database. Format of information described in NMWG schema. 56

THE LOOKUP SERVICE (4) 57

LOOKUP MESSAGES Other services can interact with the lookup service to: Register with it. De-register. Update registration details. Keep-alive registration details. Query the lookup store. 58

LOOKUP SERVICE-SPECIFIC PREREQUISITES The following prerequisites are necessary for Lookup Service installation: Java Developer s Kit (JDK) version 1.5 or later Apache Ant 1.6.x exist XML Database version 1.0.1 or 1.1.1 Can re-use an existing exist XML database Install new exist XML database as a webapp via Tomcat Some configuration is required after installation You must set your JAVA_HOME environmental variable to your Java directory 59

THREE-STAGE INSTALLATION The Installer will work through three stages: Pre-Install Establishes information required for the rest of the process Configure E.g. the application server port and service directory Set important parameters: Deploy E.g. Service type, name, URL, XML database username and password etc. Deploys the service on the application server 60

HINTS AND TIPS Change the password for the exist database admin user after the service installation is complete Prevents use of the exist client application to alter data Not necessary in training, but important in a live context If you change Tomcat s default port, ensure you configure the Lookup Service to use the amended port number. For any installation on Linux: Recommended that you download Tomcat from the Apache web-site Recommended that you do not use the version of Tomcat supplied with the distribution 61

TESTING In order to find out whether the service has been successfully set up, perform the following test: ant client-echo This contains an XML Database connectivity test Subsequently, you can perform the following tests: ant client-register ant client-query ant client-deregister Note that test results are not printed on the screen, but are put into an XML file. 62

INTERACTION WITH THE SERVICE DESK (1) If you encounter problems during or after installation, contact the Service Desk with the following information: A description of the problem Software versions for the following: Lookup Service exist XML DB Java Ant Continued on next slide 63

INTERACTION WITH THE SERVICE DESK (2) If you encounter problems during deployment of the service or runtime please give the service desk the following information: Log files (especially sonar.log) Configuration files service.properties const.properties log4j.properties components.properties Request / response files (if run) The result of ant client-echo (if run) 64

LOOKUP SERVICE INSTALLATION OVERVIEW Check that all of the required prerequisite software is installed. Check that you have the right version of each prerequisite. Download and install prerequisite software if necessary. Java Ant exist XML database Tomcat application Server Install the Lookup Service Follow the instructions in the perfsonar Administration Guide 5) Test your installation 65

ACTIVITIES Lookup service Installation and Configuration Demonstration Exercise Dependency Checks are all of the Required Prerequisites installed? Installing Manual Prerequisites the exist XML Database Preparing for the Installation Carrying out the Installation Testing the Installation Feedback 66

MODULE 4 INSTALLING AND CONFIGURING THE BWCTL MEASUREMENT POINT

MEASUREMENT POINTS (1) Purpose: expose measurement tools to provide three types of performance measurement data: Active measurements Passive measurements Network state information 68

MEASUREMENT POINTS (2) Measurement Points: Belong to domains (domain = a network) Each measurement point implementation maps to a tool that provides a specific metric Examples: Active delay One-way-loss Jitter Available bandwidth 69

BWCTL MP SPECIFICS The BWCTL MP: Is implemented as typical UNIX daemon. Is a wrapper for the BWCTL tool. Receives client requests to trigger BWCTL tests. Sends these requests to the BWCTL tool, which executes them. Returns test results to the client. Implementation could be adapted for use with other command line tools Change parsing of input and output parameters as necessary 70

THE BWCTL MP: USER-ADVANTAGES Using the BWCTL Measurement point offers you two major advantages: You don t have to be logged on to the machine where the BWCTL tool is installed You don t have to configure BWCTL keys Additionally, since the BWCTL MP plugs-in to the perfsonar framework, it makes measurements available to the perfsonar community, subject to local security policies. 71

FOUR STAGE INSTALLATION The Installer will work through four stages: Pre-Install Establishes information required for the rest of the process Configure E.g. the installation directory Confirms which necessary Perl modules already exist on the server and which need to be installed. Deploy Test Asks for the user and group ID the service will be started as Offers the opportunity to test your installation 72

INSTALLATION PREREQUISITES In addition to the general prerequisites required for all services, the BWCTL MP requires: BWCTL Tool version 1.1b or higher. Iperf Tool version 2.0.2 or higher. 73

HINTS AND TIPS If required Perl Modules are not found in the local Perl installation, then they will be installed into the installation directory of the BWCTL MP. You could choose to use a system tool Check your network connectivity! 74

TESTING You can use the supplied test script to check whether the service daemon and the init script have been correctly installed. The test script will only work if the init script is installed properly. You need root privileges for this. 75

ISSUES WHEN INSTALLING UNDER DEBIAN The included init script has been designed for installation on Fedora Linux, but should also work on other LSB compliant systems. It is known NOT to work on Debian 3.1 and earlier. It was not tested on Debian 4.0. The test script will not work out of the box on Debian, because it depends on the init script. 76

ADDING MEASUREMENTS WITH OTHER TOOLS The BWCTL MP source code is structured in a modular fashion. This approach makes it relatively easy to link further command-line measurement tools to the service. OWAMP functionality has already been added. 77

INTERACTION WITH THE SERVICE DESK When things go wrong: Typically you will need to supply the service desk with the following information: Which operating system you are using, including the version Which version of Perl you are using Which version of BWCTL / IPERF you are using The installation stage at which you began to experience problems The log output of the installation scripts Whether there is anything unusual about the way in which your system is configured 78

BWCTL MP INSTALLATION OVERVIEW 1) Check that all of the required prerequisite software is installed. 2) Check that you have the right version of each prerequisite. 3) Download and install prerequisite software if necessary. BWCTL Tool IPERF Perl 4) Install the perfsonar BWCTL Measurement Point Follow the instructions in the perfsonar Administration Guide 5) Test your installation 79

ACTIVITIES BWCTL Measurement Point Installation and Configuration Demonstration Exercise: Dependency Checks are all of the Required Prerequisites Installed? Preparing for the Installation Carrying out the Installation Testing the Installation Feedback 80

MODULE 5 INSTALLING AND CONFIGURING THE SSH / TELNET MEASUREMENT POINT

SSH / TELNET MP SPECIFICS (1) The SSH / Telnet Measurement Point acts as a central contact point inside a network. It: Is able to retrieve information from routers uses standard protocols such as SSH or Telnet Issues show like commands Discovers configuration information: routing tables, interface configuration etc. Can only issue pre-configured commands and parameters Prevents usage for malicious attacks 82

SSH / TELNET MP SPECIFICS (2) The SSH / Telnet MP is the back-end of the Looking Glass user interface Similar to existing Looking Glasses on the web (traceroute.org). 83

SSH / TELNET MP: AVAILABLE COMMANDS A client can issue the SSH / Telnet MP two types of command: MetadataKeyRequest: client asks the MP what its capabilities are SetupDataRequest: Client issues a command with its parameters through the MP for execution on a specific device 84

SSH / TELNET MEASUREMENT POINT USE CASES Example use cases: Retrieval of a routing table entry for a specific network Traceroute command Ping What you can do depends upon the commands and parameters that are configured for each instance of the measurement point. 85

SSH / TELNET MP PREREQUISITES The SSH / Telnet MP is a Java application that is deployed using Axis and Apache Tomcat. For communication with routers SSHTools [J2SSH], JSch [JSCH] or Telnet/SSH/Terminal for Java application [JTA] are required. 86

SUPPORTED NETWORK DEVICES Quagga (Telnet) Cisco (Telnet) Cisco (SSH) Juniper (Telnet) Juniper (SSH) 87

THREE STAGE INSTALLATION The Installer will work through three stages: Pre-Install Establishes information required for the rest of the process Configure E.g. installation path, Tomcat path Set important parameters: Deploy Give path of configuration (stitching) file or create the file interactively Deploys the service on the application server 88

STITCHING For the SSH / TELNET Measurement Point, stitching is the process of defining the available devices and the commands and parameters that users can issue to them through the measurement point. These settings are held in the service.properties file. The service.properties file can be: Automatically created via a wizard-like script that runs during installation. or: Manually created and then imported during configuration. 89

AN EXAMPLE OF STITCHING Devices: Cisco1 (10.10.3.24) JuniperBerlin (10.10.1.14) Commands: PING Ping 1 parameter Syntax (reg exp): ^[0-9]+\\.[0-9]+\\.[0-9]+\\.[0-9]+(\\/[0-9]{1,2})?$ 90

HINTS AND TIPS (1) To complete the installation and configuration you will need to know: The routers and other devices that you want to make available via the SSH / Telnet MP. The commands that you want to make available for these routers and devices. 91

HINTS AND TIPS (2) In order to protect the SSH / Telnet Webservice you should: Configure the regular expressions that can be used in requests I.e. configure acceptable commands and parameters etc. Configure a reasonable access rate for a device. The access rate is the time between two requests in which no other request can be handled. 92

HINTS AND TIPS (3) For any installation on Linux: Recommended that you download Tomcat from the Apache web-site Recommended that you do not use the version of Tomcat supplied with the distribution 93

TESTING Check in browser at the MP s URL if the service is running. If so, run the ant test command. creates a SetupDataRequest for every command of every device, and sends it to the MP. Results coming back are automatically checked. 94

ISSUES WHEN INSTALLING UNDER DEBIAN No known issues when installing under Debian. Service has been deployed and tested completely on a Debian setup. 95

INTERACTION WITH THE SERVICE DESK When problems arise during installation, please provide the following information: Java Version An URL which we can use to test remotely The service.properties file 96

SSH / TELNET MP INSTALLATION OVERVIEW 1) Check that all of the required prerequisite software is installed. 2) Check that you have the right version of each prerequisite. 3) Download and install prerequisite software if necessary. Java Ant Tomcat application Server 4) Install the perfsonar SSH / Telnet Measurement Point Follow the instructions in the perfsonar Administration Guide 5) Test your installation 97

ACTIVITIES SSH / Telnet Measurement Point Installation and Configuration Demonstration Exercise Dependency checks are all of the Prerequisites Installed? Preparing for the Installation Carrying out the Installation Examining the service.configuration file Testing the Installation Feedback 98

MODULE 6 THE SERVICE DESK

THE ROLE OF THE SERVICE DESK The Service Desk is a single Point of Contact for: The 5 MDM pilot deployers GEANT2 community NOC and PERT users Through the service desk you can: Report incidents about the installation, configuration, operation and utilisation of the web services and visualisation tools. Ask questions: About the MDM service About the installation, configuration and operation of perfsonar web-services and visualisation tools Raise enhancement Requests 100

THE DUTIES OF THE SERVICE DESK (1) The duties of the service desk will include: Logging all calls, events and requests. Acting as the first layer of incident management. Taking overall ownership of incidents Escalating and re-assigning them until they are resolved Monitoring the MDM service, the web-services and the visualisation tools. 101

INCIDENT MANAGEMENT EXAMPLE KNOWN ERRORS Several incident management processes have been defined. This is one example. 102

THE DUTIES OF THE SERVICE DESK (2) The duties of the service desk will include: Gathering feedback on the service provided. Generating regular reports on: Incidents. Lessons learned. Missing pieces (continuous improvement). Offering a managed service for FCCN and GEANT2. 103

SUCCESS FACTORS Critical success factors are: Well defined and efficient support processes Trouble ticketing system, CMDB, monitoring tools Documentation Training 104

SUPPORTED SOFTWARE The Service Desk will support the: Installation, configuration and operation use of: SQL MA, RRD MA, SSH / Telnet MP, BWCTL MP, L2 status MP, LS. Use and configuration of CNM and E2EMON. Installation and use of the perfsonar UI and of the Looking Glass. Use of: The Hades MA. The Hades Visualisation Tool. Use and operation of Hades monitoring tools. 105

L2 STATUS Please note that the L2 status MA cannot be supported until released. E2EMon visualisation will be supported at the same time. 106

INFORMATION TO PROVIDE TO THE ASD NREN MDM Contact detail Deployed web-services IP addresses, URL, locations, the GPS installation quotes When web-services are installed, so that the ASD can start monitoring it For equipment shipment When a planned maintenance will affect the MDM service 107

CONTACT DETAILS E-mail: asd@geant2.net Phone: +44 1223 371 380 (available from 18 th of June) In the meanwhile, please call +44 1223 371 3xx 108

MODULE 7 INSTALLING AND CONFIGURING THE RRD MEASUREMENT ARCHIVE

MEASUREMENT ARCHIVES Purpose: Measurement archives expose measurement data held in databases or file systems. They: Are wrappers for any type of storage mechanism (SQL Databases, RRD files, etc). Provide access to recent and stored data. Can also be used to write information to databases and file systems. 110

THE RRD MEASUREMENT ARCHIVE (1) The Round Robin Database (RRD) Measurement Archive is a wrapper for binary files of the RRDTool format. 111

THE RRD MEASUREMENT ARCHIVE (2) The RRD Measurement Archive has two main functions: Writing and storing measurement data in RRD files E.g. information from perfsonar Measurement Points collected as a result of regularly scheduled or on-demand measurements Publishing measurement data held in RRD files to client applications E.g. Measurements that have been stored in RRD files by nonperfsonar applications such as MRTG (Multi Router Traffic Grapher) or Cricket 112

THE RRD MEASUREMENT ARCHIVE IN CONTEXT 113

RRD MA INSTALLATION OVERVIEW Follow these steps: Install the perfsonar RRD Measurement Archive Service Generate and populate metadata configuration files Deploy the Web Service Test the deployed service to see if it is working 114

THE RRD MA: THREE STAGE INSTALLATION The Installer will work through three stages: Pre-Install Establishes information and performs tasks required for the rest of the process Configure Collects information such as installation path, Tomcat port, exist admin user password Compiles the RRD J tool Set important parameters: Deploy E.g. name and path of metadata configuration file, location of file-store, whether or not to automatically register with a lookup service Deploys the service on the application server 115

TESTING You can execute a test script that sends a series of test requests to the service. The metadata configuration file used by the service must be the test metadata configuration file that is supplied with the installation files. The command to run is ant test Once the test is complete, you can analyse the responses generated by the service. If there are problems, then clear error messages will be displayed in the responses. If there is no response, then there is a problem. 116

STITCHING FOR THE RRD MEASUREMENT ARCHIVE In the context of a measurement archive, stitching: Is the process of configuring metadata that underlies the performance data handled by your archive. Metadata is data that describes other data E.g. data units, interface name, direction of traffic etc. Involves the creation of a metadata configuration file. Ultimately, the metadata configuration file tells the perfsonar framework what kind of data the archive stores. 117

RRD MA STITCHING IN CONTEXT Instance of RRD Measurement Archive Service Performance data Performance data RRD file MRTG MRTG In Out POS-6/0 router Performance data Performance data RRD file Cricket Cricket In Out GEO/0 router Stitching allows a service to understand the data it is dealing with. Examples: Which router? Inbound or outbound traffic? What data units? Etc. 118

THE METADATA CONFIGURATION FILE The Metadata Configuration File: Is an xml file that will expose information describing your network s RRD files. Conforms to NMWG s XML schema. The default perfsonar installation provides samples of: A metadata configuration file An RRD file The sample metadata configuration file: Is provided as a template only and should be changed to correctly describe your own RRD archives. 119

STITCHING FOR THE RRD MA THREE STEP PROCESS Step 1 Understand the metadata configuration file structure Step 2 Create your own metadata configuration file Step 3 Apply your metadata configuration file to your instance 120

STITCHING FOR THE RRD MA STEP 1 To Understand the metadata configuration file structure you should: Refer to the metadata configuration file guide Study the sample metadata configuration file Gather information about each data source in your RRD files. A data source usually equates to a measurement of an interface s traffic in a single direction. You should create a a metadata chain for each data source. There is no limit to the number of metadata chains you configure. 121

STITCHING FOR THE RRD MA CHAINS A chain: Describes measurement data for a single interface, for one direction only Consists of two connected parts: Metadata Data Host Name (DNS entry of the router containing the interface) IPV4 interface address Interface s name Interface s description Traffic direction (in or out) Authentication realm Capacity / interface speed Name and path of the RRD file Data source within the RRD file Data storage unit (example: bps or Bps) 122

RRD MA METATDATA CHAINS EXAMPLE OF METADATA 123

RRD MA METATDATA CHAINS EXAMPLE OF DATA 124

STITCHING FOR THE RRD MA: STEP 2 Create your metadata configuration file Can be created by manual file editing. Can be generated using contribution scripts or your own scripts Contribution scripts are available for some tools E.g. MRTG If scripts do not already exist for your tool, you can create your own script based on existing contribution scripts Manual file creation for a large number of interfaces would be labourintensive and time-consuming When something changes in your network, update your file Recommended that you automate this process 125

STITCHING FOR THE RRD MA: STEP 3 Apply your metadata configuration file to your instance Three possible methods: Use the exist XML database web-based User Interface Recommended method Use the perfsonar installer Not covered in this course; refer to the installation guide for details Use the service installation scripts Not covered in this course; refer to the installation guide for details 126

MAKING THE MEASUREMENT ARCHIVE AVAILABLE TO THE VISUALISATION TOOLS A user client application needs to know the address of the Measurement Archive. This information can be taken from the Lookup Service Contact Andreas Hanemann at DFN in order to get CNM to use your service. When the Lookup Service is installed, please notify the perfsonar UI team. 127

INSTALLATION PREREQUISITES In addition to the general prerequisites required for all services, the RRD MA requires: RRDTool version 1.2.x The RRD MA can be installed on any Linux platform 128

HINTS AND TIPS During the installation, you will need to supply the location of the RRD Tool. It is better to store metadata configuration information in an XML database than in a text file as this improves performance. To run the RRD MA on a 64-bit machine, you must be consistent in your use of 32 bit or 64 bit software i.e. either all software (Java, rrdjtool, rrdtoo libs, rrd files) must be compiled for 32-bit or all software must be compiled for 64-bit. For any installation on Linux: Recommended that you download Tomcat from the Apache web-site Recommended that you do not use the version of Tomcat supplied with the distribution 129

ISSUES WHEN INSTALLING UNDER DEBIAN The librrd2-dev package must be installed. 130

INTERACTION WITH THE SERVICE DESK When problems arise during installation, please provide the following information as a minimum: Log files Configuration files Request / response messages 131

RRD MA INSTALLATION OVERVIEW 1) Check that all of the required prerequisite software is installed. 2) Check that you have the right version of each prerequisite. 3) Download and install prerequisite software if necessary. Java Ant RRDTool exist XML database Tomcat application Server 4) Install the perfsonar RRD Measurement Archive Follow the instructions in the perfsonar Administration Guide 5) Test your installation 132

ACTIVITIES RRD Measurement Archive Installation and Configuration Demonstration Exercise Dependency Checks are all of the Required Prerequisites Installed? Preparing for the Installation Carrying out the Installation Testing the Installation Stitching Editing the RRD Metadata Configuration File Testing the Results of Stitching Feedback 133

MODULE 8 CARRYING OUT AN INSTALLATION ON DEBIAN

OVERVIEW OF DEBIAN Very popular OS among the Open-Source alternatives Stability Powerful package management, Easy upgrade Huge Community Comes in 3 flavours Stable: Integrate robust package. (Not always recent but Security patched!! AKA ETCH) Testing: The next Stable version (AKA LENNY) Unstable: Bleeding edge package (AKA SID) Stable is the way to go Security Robust service due to package stability 135

RRD MA INSTALLATION: DEBIAN BASE INSTALLATION Get the ISO from www.debian.org Either download the full distribution but If possible prefer the NETINSTALL ISO (163 Meg) NETINSTALL provides up to date packages Install the minimum distribution (server/standard install) A rule of thumb for production environment is To always use the KISS method If you don t need it, don t install it 136

REMINDER: APT, DPKG, LOCATE ARE ALL FRIENDS apt dpkg Package/Distribution management tool After minimum install, update source.list, chose your favorite mirror and add non-free repository Then apt-get update; apt-get dist-upgrade apt-cache search, if you are lost! -l <pkg_name>: List all packages installed -L<pkg_name>: List all files part of a package locate Locate <file_name> Where is <file_name> Updatedb when to need to locate a file on the file system 137

PERFSONAR BUNDLE INSTALLER / RRD MA DEPENDENCIES ssh java rrdtool and librrd2-dev wget perl libwww-perl (AKA LWP) gcc ant and don t forget ant-optional! Debian Tomcat or PerfSONAR Tomcat It s up to you! 138

TOMCAT Debian Tomcat Disable security manager Enable TOMCAT but define a TOMCAT security policy Tomcat from bundle installer Preferred method Easy upgrade Security manager disabled 139

POTENTIAL ISSUES Environment variable LD_LIBRARY_PATH No RRD-GRAPH in PerfSONARUI JAVA_HOME not set TOMCAT won t start Enable system wide environment variable Put it in /etc/profile export LD_LIBRARY_PATH Export JAVA_HOME Tomcat refuse to install the services on packaged Tomcat Disable Tomcat security manager or Add security policy 140

FINAL TOUCH Starting PerfSONAR service at system startup /etc/init.d/rc.local script Run the service as perfsonar user! (Avoid root ) Diagnostic tools netstat a grep LISTEN check that Tomcat is listening at port defined and also when the MA is interrogated ps def grep perfsonar check the process is running 141

LAST BUT NOT LEAST. SECURITY PerfSONAR services provide access to sensitive infomation so General rules Permit «anyone» that wants to access the service ONLY Permit remote administration using SSH from your LAN Permit ICMP echo request from 194.141.0.9 Available tools on RRD-MA host TCPD (AKA TCP wrapper) IPTABLES Tripwire Available tools on the local LAN Router access-list Switch VACL/Private VLAN Etc. 142

QUESTIONS? Comments and suggestions are of course welcome! 143

MODULE 9 INSTALLING AND CONFIGURING THE SQL MEASUREMENT ARCHIVE

THE SQL MEASUREMENT ARCHIVE (1) The SQL Measurement Archive is a wrapper that allows perfsonar to access data stored in a database. utilisation and path status are currently supported Supports MySQL or PostgreSQL databases Theoretically other databases can be used, but these have not been tested 145

THE SQL MEASUREMENT ARCHIVE (2) The SQL Measurement Archive has two main functions: Publishing measurement data from a database to client applications Writing and storing measurement data to a database When installing, you can either: Setup a new database Schema setup scripts included in installation Use an existing database Configure the SQL MA to work with your existing database 146

STITCHING FOR THE SQL MEASUREMENT ARCHIVE In the context of a measurement archive, stitching: Is the process of configuring metadata that underlies the performance data handled by your archive. Metadata is data that describes other data E.g. data units, interface name, direction of traffic etc. Involves the creation of a metadata configuration file. Ultimately, the metadata configuration file tells the perfsonar framework what kind of data the archive stores. 147

SQL MA STITCHING IN CONTEXT Stitching allows a service to understand the data it is dealing with. Examples: Which link? Which interfaces? Location of interfaces? Etc. 148

THE METADATA CONFIGURATION FILE Your Metadata Configuration File will expose information describing your network s: Interfaces and their utilisation data. Links and link status The default perfsonar installation provides samples of: A metadata configuration file A SQL database The sample metadata configuration file: Is provided as a template only. Should be changed to correctly describe the your own SQL database. 149

STITCHING FOR THE SQL MA THREE STEP PROCESS Step 1 Understand the metadata configuration file structure Step 2 Create your own metadata configuration file Step 3 Apply your metadata configuration file to your instance 150

STITCHING FOR THE SQL MA STEP 1 To understand the metadata configuration file structure you should refer to: The metadata configuration file guide The sample metadata configuration file There are two types of SQL MA metadata: Metadata describing interface utilisation Similar to the RRD MA s metadata Metadata describing path status Unique to the SQL MA 151

SQL MA ARCHITECTURE 152

UNDERSTANDING PATH STATUS METADATA SQL MA path status metadata can be subdivided into: Node metadata Link metadata First create metadata describing each node. Then create metadata describing each link. 153

LINK METADATA AND NODE METADATA When creating link and node metadata: You need to create a metadata chain for each link that you want to collect data about. Within the link s metadata chain, associate two nodes with the link. I.e. the start and demarcation points of the link The same node can be referred to in the metadata of multiple links. 154

STITCHING METADATA FOR NODES AND LINKS Node metadata includes: Node ID and name Node s country, city and institution Node s latitude and longitude Link Metadata includes: Link name and global name Name of related nodes Roles of related nodes: I.e. end point or demarcation point 155

STITCHING PATH STATUS CHAINS A path status chain describes measurement data for a single link and refers to multiple nodes. Consists of two connected parts: Metadata For Nodes For Links Data Name of relational database configuration file Path ID 156

SIMPLIFIED LINK PATH METADATA EXAMPLE 157

NODE METADATA EXAMPLE 158

LINK CHAINS AN EXAMPLE 159

LINK CHAIN DATA AN EXAMPLE 160

SQL MA ARCHITECTURE 161

STITCHING FOR THE SQL MA INTERFACE UTILISATION CHAINS Interface utilisation chains: Describes measurement data for a single interface, for one direction only. Are the same as RRD Measurement Archive link chains However the data is different because the storage type is different Consist of two connected parts: Metadata Data Host Name (DNS entry of the router containing the interface) IPV4 interface address Interface s name Interface s description Traffic direction (in or out) Authentication realm Capacity / interface speed Name of relational database configuration file 162

STITCHING FOR THE SQL MA: STEP 2 Create your own metadata configuration file. Can be created by manual file editing. Generate your file using contribution scripts or your own scripts. When something changes in your network, update your file. Recommended that you automate the process 163

STITCHING FOR THE SQL MA: STEP 3 Apply your metadata configuration file to your instance Three methods: Use the exist XML database web-based User Interface Recommended method Use the perfsonar installer Not covered in this course; refer to the installation guide for details Use the service installation scripts Not covered in this course; refer to the installation guide for details 164

THREE STAGE INSTALLATION OF THE SQL MA The Installer will work through three stages for the SQL MA: Pre-Install Establishes information required for the rest of the process Configure E.g. installation path, Tomcat port, exist admin user password Set important parameters: Deploy E.g. setup the database username and password for database name and path of metadata configuration file location of file-store whether or not to automatically register with a lookup service Deploys the service on the application server 165

MAKING THE MEASUREMENT ARCHIVE AVAILABLE TO THE VISUALISATION TOOLS A user client app needs to know the address of MA (can be taken from LS). 166

TESTING You can execute a test script that sends a series of test requests to the service. The metadata configuration file used by the service must be the test metadata configuration file that is supplied with the installation files. The command to run is ant test Once the test is complete, you should analyse the responses generated by the service. 167

INSTALLATION PREREQUISITES In addition to the general prerequisites required for all services, the SQL MA requires: One of the following relational databases: Mysql version 5.0 Or PostgreSQL version 8.x 168

HINTS AND TIPS The SQL MA can be installed on any Linux platform. Metadata Configuration must be held in XML database. For any installation on Linux: Recommended that you download Tomcat from the Apache web-site Recommended that you do not use the version of Tomcat supplied with the distribution 169

INTERACTION WITH THE SERVICE DESK When problems arise during installation, please provide the following information as a minimum: Log files Configuration files Request / response messages 170

SQL MA INSTALLATION OVERVIEW 1) Check that all of the required prerequisite software is installed. 2) Check that you have the right version of each prerequisite. 3) Download and install prerequisite software if necessary. Java Ant MySQL (or postgresql) database exist XML database Tomcat application Server 4) Install the perfsonar SQL Measurement Archive Follow the instructions in the perfsonar Administration Guide 5) Test your installation 171

ACTIVITIES SQL Measurement Archive Installation and Configuration Demonstration Exercise Dependency Checks are all of the Required Prerequisites Installed? Preparing for the Installation Carrying out the Installation Testing the Installation Stitching Editing the SQL Metadata Configuration File Testing the Results of Stitching Feedback 172

MODULE 10 FEEDBACK ON PERFSONAR INSTALLATION AND CONFIGURATION

ACTIVITIES Please use the forms provided to give us feedback about the perfsonar installation and configuration process. This asks for your feedback about the installation process and the associated software, not about the training course You will be asked for feedback about the training course separately For both the BWCTL MP and the JAVA Services Provide three positive points and three things to enhance about the installation and configuration process No inter-personal issues No generalities. Be specific. Make suggestions about how to improve Provide answers to quantitative questions 174