DevOps examples on NonStop Tools Overview. Cor Geboers, ATC Consultant

Transcription

1 DevOps examples on NonStop Tools Overview Cor Geboers, ATC Consultant

2 About me Cor Geboers Senior Consultant in NonStop ATC, based in Belgium 35+ years in IT development and support 25+ years NonStop experience Working with middleware and new technologies OSS believer since the beginning

3 What is DevOps?

4 DevOps Interaction (simplified) Development QA Operations Time 4

5 Examples in this presentation Using Ansible on NonStop Using Jenkins for Continuous Deployment 5

6 What is Ansible? Simple IT automation engine cloud provisioning configuration management application deployment Designed for multi-tier deployments Parallel execution on different nodes at the same time Uses no agents and no additional custom security infrastructure Easy to deploy Uses a very simple language (YAML)

7 Ansible Architecture webserver webserver ssh webserver Ansible Mgmt Node Switch SOAP Host Inventory Playbook SOAP 7

8 Ansible and NonStop 8

9 Why use a Tool like Ansible? Infrastructure as Code Automation Documented Progress Deterministic Process Idem-potency Continuous Delivery 9

10 Ansible Playbook A playbook is the term that Ansible uses for a configuration management script Is written in YAML Contains list of tasks to execute sequentially Multiple nodes at once Can change user using sudo Task contains name and module Modules are built-in scripts 10

11 Ansible Module Set of scripts that come with Ansible Extensible Development kit Extensions local Version control Preferred git 11

12 Playbook relationships Playbook Play Host Task Module 12

13 Ansible Inventory Inventory contains hosts Text file DNS names or IP addresses Hosts can be grouped Configuration options for each host Authorization key Default Username TCP/IP address and port numbers 13

14 Inventory Example nsblde9 ansible_port=22 ansible_user=cgeboers nsblde6 ansible_host=nsblde6.atc-hp.com ansible_port=22 ansible_user=corg nsx09 ansible_host=nsx09.atc-hp.com ansible_port=22 ansible_user=corg [webservers] nsblde9 ansible_host=nsblde9.atc-hp.com ansible_port=22 ansible_user=cgeboers nsx09 ansible_host=nsx09.atc-hp.com ansible_port=22 ansible_user=corg 14

15 Installation Need console installation on Linux/Windows(Cygwin) system Need valid NonStop SSL installation Need private/public key Authorized_keys on managed node Use SSHCOM to install public key for operator key Need sudo for scripts requiring SUPER access Requires Python 2.5 or later Python 2.7.x provided as part of T1203 (coreutils scripts) no other Python modules needed for NonStop No need for installing agents on the NonStop 15

16 Why Ansible on NonStop? Facilitates installation and deployment of tools Think of development machines that might need additional subsystems Configure using templates Use Playbooks No real NonStop knowledge required once playbooks are written and tested Virtual NonStop Node deployments 16

17 Initial Testing ansible all -m ping nsx09 SUCCESS => { "changed": false, "ping": "pong" } nsblde6 SUCCESS => { "changed": false, "ping": "pong" } nsblde9 SUCCESS => { "changed": false, "ping": "pong" } 17

18 Example Playbook for NonStop Simple scenario New (virtual) node configured Need to deploy itp Webserver using standard configuration Need to deploy NSSOAP 4 inside created webserver Operations to be done from management node Use variables for easy customization 18

19 Playbook install_webserver.yml hosts: webservers tasks: - name: Check if distribution is already on system stat: path="{{ dist_folder }}/{{ itp_version }}" register: sym - name: Unpack itp Webserver archive command: "/bin/pax -r -s:/usr/tandem/webserver:{{ dist_folder }}: -s:/usr/lib:{{ dist_folder }}/lib: -s:/usr/include:{{ dist_folder }}/include: -f /G/system/zossutl/t8996pax" when: sym.stat.islnk is not defined - name: Setup answer file template: src=templates/itpwebserver_setup.sh.j2 dest={{ dist_folder }}/itpwebserver_setup.sh mode="u=rwx,g=rx,o=rx" - name: Install itp Webserver command: "{{ dist_folder }}/itpwebserver_setup.sh" 19

20 Playbook customization Uses YAML files which have name of groups Automatic activation from directory layout --- webserver_location: /home/hp/corg/tmp/webserver dist_folder: /home/hp/corg/dist itp_version: T8996H03_01APR15_AFA_H329_01 20

21 Playbook output (1/2) ansible-playbook install_webserver.yml PLAY [webservers] ************************************************************** TASK [setup gather_subset=all] ************************************************* ok: [nsblde9] TASK [Check if distribution is already on system path={{ dist_folder }}/{{ itp_version }}] *** ok: [nsblde9] TASK [Unpack itp Webserver archive _raw_params=/bin/pax -rv -s :/usr/tandem/webserver:{{ dist_folder }}: -s :/usr/lib:{{ dist_folder }}/lib: - s:/usr/include:{{ dist_folder }}/include: -f /G/system/zossutl/t8996pax] *** changed: [nsblde9] TASK [Setup answer file dest={{ dist_folder }}/itpwebserver_setup.sh, src=templates/itpwebserver_setup.sh.j2, mode=u=rwx,g=rx,o=rx] *** ok: [nsblde9] TASK [Install itp Webserver _raw_params={{ dist_folder }}/itpwebserver_setup.sh] *** changed: [nsblde9] PLAY RECAP ********************************************************************* nsblde9 : ok=5 changed=2 unreachable=0 failed=0 21

22 Playbook repeated output ansible-playbook install_webserver.yml PLAY [webservers] ************************************************************** TASK [setup gather_subset=all] ************************************************* ok: [nsblde9] TASK [Check if distribution is already on system path={{ dist_folder }}/{{ itp_version }}] *** ok: [nsblde9] TASK [Unpack itp Webserver archive _raw_params=/bin/pax -rv -s :/usr/tandem/webserver:{{ dist_folder }}: -s :/usr/lib:{{ dist_folder }}/lib: -s:/usr/include:{{ dist_folder }}/include: -f /G/system/zossutl/t8996pax] *** skipping: [nsblde9] TASK [Setup answer file dest={{ dist_folder }}/itpwebserver_setup.sh, src=templates/itpwebserver_setup.sh.j2, mode=u=rwx,g=rx,o=rx] *** ok: [nsblde9] TASK [Install itp Webserver _raw_params={{ dist_folder }}/itpwebserver_setup.sh] *** changed: [nsblde9] PLAY RECAP ********************************************************************* nsblde9 : ok=4 changed=1 unreachable=0 failed=0 22

23 Ansible Templates Templates are processed by the Jinja2 templating language Six additional variables can be used in templates sandboxed execution powerful automatic HTML escaping system for XSS prevention template inheritance compiles down to the optimal python code just in time optional ahead-of-time template compilation easy to debug. Line numbers of exceptions directly point to the correct line in the template. configurable syntax 23

24 Template Example #!/bin/bash # cd {{ dist_folder }}/{{ nssoap4_version }}/bin./deploy.sh << EOF 1 {{ nssoap4_location }} nssoap4 nssoap4 {{ webserver_location }} n 4 4 EOF 24

25 Template example 2 #!/bin/bash # # setup for itp Webserver requires /bin/cp! export PATH=/bin:$PATH cd {{dist_folder}}/{{itp_version}}./setup {{webserver_location}} <<EOF y n y 1 3 /G/ztc /G/w33k /G/sas1/web33k y EOF 25

26 Ansible for NonStop usage? Other usages to think of: Install database Install Pathway environment Install NSSOAP4 services 26

27 Jenkins Continuous Integration 27

28 What is Jenkins Open-source, continuous integration tool Written in Java, forked from Hudson in Very active community, releases every few months Very feature-rich Runs on NonStop servers under OSS Executes in web containers Jenkins Java is a registered trademark of Oracle and/or its affiliates 28

29 Jenkins Main Characteristics Easy to install on NonStop Deploy in NSJSP..or run it standalone, as it embeds the Jetty web container Easy to configure Web GUI is simple to use Plug-in add additional functionality into existing configuration pages Lots of plug-ins! Extensible Jenkins plugins are easy to create Distributed Jenkins can distribute builds & tests to multiple machines. Can be running different OSes. 29

30 Global Features: Jenkins Dashboard Dashboard showing status of all jobs, with links to go into details about each job Result of last run, as well as weather report shown for each job Create new jobs from Dashboard Create custom views Can run jobs on demand from dashboard Set up RSS feeds for results Manage credentials 30

31 Global Features: Credential Support Out of the box support for Username/password SSH PKCS 12 Certificate Available plug-ins LDAP Kerberos Active Directory CAS OpenID SAML Google OAuth Credentials Many more, see Jenkns Auth Plugins Java is a registered trademark of Oracle and/or its affiliates 31

32 Global Features: Plug-Ins Categories Source code management Build triggers Build tools Build wrappers Build notifiers Slave launchers and controllers Build reports Artifact uploaders Other post-build actions External site/tool integrations UI plugins List View column plugins Page decorators Authentication and user management Cluster management and distributed build CLI extensions Maven Parameters ios development.net development Android development Ruby development Library plugins Scala plugins Miscellaneous Uncategorized plugins Java is a registered trademark of Oracle and/or its affiliates 32

33 Global Features: Job Triggers On demand Scheduled Based on SCM update (polled) Based on build result Monitor folders (Files Found Trigger) Triggered from another Jenkins job, upon successful or failed completion (pipelines) Via URL, e.g. From a remote Jenkins master Others gerrit, ivy-script, groovy script, pollurl, generic script Java is a registered trademark of Oracle and/or its affiliates 33

34 Jenkins on NonStop - Installation hints - Example with Pathway Domains and 3GLs 34

35 Jenkins on the NonStop - Installation Hints Only run one copy listening on a particular port...maxservers 1 if using NSJSP multiple standalone slave copies can run on different ports CPUs/IPUs they must be configured to use different work directories Expected startup exception: No suitable implementation found for Free Swap Space monitor not an issue Give Jenkins plenty of heap space -Xms512m -Xmx512m Some plugin installations require a restart of Jenkins, stop and restart the process Turn on the auto refresh for the job status page ( ENABLE AUTO REFRESH upper right hand corner of the job status page) Java is a registered trademark of Oracle and/or its affiliates 35

36 Example: Jenkins Continuous Deployment without downtime Develop serverclass in NSDEE Use Source Control tool like git Jenkins monitors git repository Can work from sources and check build Can work with delivered binaries On successful modification deploy into a TS/MP domain Works with your COBOL, C/C++, TAL, or new technologies 36

37 Jenkins Flow NonStop Server Pathway Domain %PM00 1. Commit changes to repository Program file $PM01 server server server $PM02 server server server 2. Jenkins checks repository 3. Create local copy 4. Deploy to first Pathmon 5. Deploy to second Pathmon 37

38 Thank you 38