Automated Out-of-Band management with Ansible and Redfish Jose Delarosa Senior Linux Engineer at Dell EMC Jake Jackson Product Field Engineer at Ansible by Red Hat May 2 nd, 2017
Who are we Jose De la Rosa (@jdelaros1) Linux Engineer at Dell EMC. Systems engineer, emerging technologies evangelist. Jake Jackson (@thedoubl3j) Product Field Engineer at Ansible by Red Hat. Works with the Getting Started team helping customers with standing up and getting started with Ansible.
Before we start 1. Thank you for coming. 2. Please ask questions at any time. 3. If time runs out, we will be happy to talk to you after the session.
Content 1. Out-of-band management with PowerEdge idrac 2. Scalable out-of-band management with Redfish 3. Automated out-of-band management with Ansible
idrac Overview
Integrated Dell Remote Access Controller (idrac) Embedded chip on a PowerEdge server used to run tasks that are independent of the host OS and the rest of the server. Detects hardware failure Manage power: turn off, on, hard reset System event and lifecycle logs Has its own ethernet port, usually connected to separate management network. Referred to as out-of-band management, as opposed to inband management which is provided by the OS.
Web UI Login
Main page
Storage controller status
Power & thermal readings
System event logs
Simple Out-of-band management Management Network 1-5 servers
Redfish Overview
What is Redfish? RESTful API specification defined by the DMTF for hardware management. Aims to replace IPMI and vendor-specific interfaces like WSMAN. Schema-based but human-readable. Clients send URI requests over https to idrac, so clients can be any OS or application on a server, workstation or mobile device.
What can Redfish do? Monitor server health status Alert on server health status changes View server hardware inventory and firmware versions Reset, reboot, and power control servers Access system logs
Example: Get system health $ curl s https://<idrac_ip>/redfish/v1/systems/system.embedded.1 -k -u root:password python -m json.tool jq.status { } "Health": "OK", "HealthRollUp": "OK", "State": "Enabled"
Example: Get storage controller health $ curl s https://<idrac_ip>/redfish/v1/systems/system.embedded.1/storage/controllers/raid. Slot.8-1 -k -u root:password python -m json.tool jq.name "PERC H730 Adapter" $ curl s https://<idrac_ip>/redfish/v1/systems/system.embedded.1/storage/controllers/raid. Slot.8-1 -k -u root:password python -m json.tool jq.status { } "Health": "OK", "HealthRollUp": "OK"
Example: Get power consumption during last hour $ curl s https://<idrac_ip>/redfish/v1/chassis/system.embedded.1/power/powercontrol -k -u root:password python -m json.tool jq.powermetrics { } "AverageConsumedWatts": 152, "IntervalInMin": 60, "MaxConsumedWatts": 168, "MinConsumedWatts": 148
Example: Get system event logs
What else can Redfish do? Retrieve Telemetry - Basic server identification and asset information - Health state - Temperature sensors and fans - Power consumption and thresholds Discovery - Service endpoint (network-based discovery) - System topology (rack, chassis, server, node) Basic I/O Infrastructure Data - Host NIC MAC addresses for LOM devices - Simple hard drive status / fault reporting Security - Session-based leveraging HTTPS Common Management Actions - Reboot / power cycle - Change boot order - Configure BMC network settings - Manage user accounts Access and Notification - Serial console access via SSH - Alert / event notification - Event log access More Coming Soon! https://www.dmtf.org/standards/redfish
Redfish API tree structure
Redfish Roadmap Version 1 focused on servers. Expand over time to cover rest of IT infrastructure. Will add devices over time to cover new technologies (i.e. NVDIMMs, Multifunction Adapters) SNIA is developing Swordfish, which builds upon Redfish s local storage management to address advanced storage devices. Open source efforts: Client libraries (Python, Java, PowerShell) Command line utility redfishtool (similar to ipmitool)
Scalable Out-of-band management https:/ / <idrac-ip>/ redfish/ v1/ Managers/ idrac.embedded.1/ Logs/ Sel Management Network More than 5 servers Server data
Ansible Overview
What is Ansible? It s a simple automation language that can perfectly describe an IT application infrastructure in Ansible Playbooks. It s an automation engine that runs Ansible Playbooks. Ansible Tower is an enterprise framework for controlling, securing and managing your Ansible automation with a UI and restful API.
How Ansible works PUBLIC / PRIVATE CLOUD CMDB ANSIBLE S AUTOMATION ENGINE USERS INVENTORY API HOSTS ANSIBLE PLAYBOOK MODULES PLUGINS NETWORKING
More about Ansible Agentless minimum footprint Make repetitive tasks easy Defines a desired state, so OK to run playbook more than once Easier to use than writing shell scripts
Ansible use cases Infrastructure Automation Networking Containers Code Deployment Server / Bare Metal Want more information? Visit www.ansible.com and docs.ansible.com
Ansible + Redfish + idrac together!
Scalable & Automated Out-of-band management https:/ / <idrac-ip>/ redfish/ v1/ Managers/ idrac.embedded.1/ Logs/ Sel Management Network More than 5 servers Database Server data
Key Lifecycle Management tasks Server Power On/Off; Reboot; Hard Reset Install BIOS, Configure BIOS, Reset to Default Configure idrac (CRUD operations): User & Password Management Certificate Management (import, export, delete) Network Configuration NTP and Time Zone settings Storage (RAID, Virtual Disk, Physical Disks, PERC) System Inventory H/W, Firmware, Sensor OS Deployment remote file share, vmedia Import / Export SCP remote file share, vmedia Backup and Restore Server Profiles Upgrade using DSU (Dell Server Update) or DUEC (Dell Update Engine for Consoles) Get list of available and applicable updates Firmware Upgrade BIOS Upgrade OS Drivers Upgrade Job Management Check JOB status Create JOB Delete JOB Create JOB Queue Delete JOB Queue Get Logs Export LC logs Export System Event Logs
Ansible module for idrac Manage your entire Dell EMC IT infrastructure (servers, routers, switches, storage) from your Ansible Controller. Automated monitoring, provisioning, firmware updates at scale. Open source, so you can write your own extensions as needed and contribute back to the community. Working with Red Hat to include as core Ansible module.
Implementation (playbook) idrac.yml getdata.yml
Server inventory Use case: collect inventory data, maintain in spreadsheet or database Server idrac IP Model IP address BIOS CPU Type RAM Service Tag Status webserver-1 192.168.2.10 PowerEdge R630 10.0.1.30 2.3.4 2 webserver-2 192.168.2.11 PowerEdge R630 10.0.1.31 2.3.4 2 webserver-3 192.168.2.12 PowerEdge R630 10.0.1.33 2.3.2 2 appserver-1 192.168.2.13 PowerEdge R830 10.0.1.34 2.3.2 4 dbserver-1 192.168.3.10 PowerEdge R730 10.0.2.30 2.1.2 2 dbserver-2 192.168.3.11 PowerEdge R730 10.0.2.31 2.3.4 2 dbserver-3 192.168.3.12 PowerEdge R730 10.0.2.32 2.3.4 2 dbserver-4 192.168.3.13 PowerEdge R730 10.0.2.33 2.3.4 2 Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.60GHz Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.33GHz Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.33GHz Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.33GHz Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.33GHz 128 5WT4Q47 OK 128 5XR7Q32 OK 128 5XR7QYY OK 512 5XR7QYY OK 256 5XR7Q67 OK 256 5WT4Q37 OK 256 5WR4Q12 OK 256 5TT1Q44 OK
Source code repository https://github.com/dell/idrac-ansible-module Will continue adding features in coming weeks and months. Pull requests and contributions are welcome and encouraged!
Resources idrac with Lifecycle Controller: http://dell.to/2qdbd0y Redfish API specification: https://www.dmtf.org/standards/redfish Dell EMC PowerEdge Redfish API Overview: http://dell.to/2odsh1p idrac Redfish API Reference Guide: http://dell.to/2oyjmty Getting started with Ansible: http://docs.ansible.com/ansible/intro_getting_started.html
Q & A
Backup
idrac operation APIs Dell Redfish API URLs /redfish/v1/managers /redfish/v1/managers/idrac.embedded.1 /redfish/v1/managers/idrac.embedded.1/actions/manager.reset /redfish/v1/managers/idrac.embedded.1/networkprotocol /redfish/v1/ Managers/iDRAC.Embedded.1/SerialInterfaces /redfish/v1/ Managers/iDRAC.Embedded.1/SerialInterfaces/<Serial-key> /redfish/v1/managers/idrac.embedded.1/logservices /redfish/v1/managers/idrac.embedded.1/logservices/sel /redfish/v1/managers/idrac.embedded.1/logservices/lclog /redfish/v1/managers/idrac.embedded.1/logservices/sel/actions/logservice. ClearLog /redfish/v1/managers/idrac.embedded.1/virtualmedia /redfish/v1/managers/idrac.embedded.1/virtualmedia/<media-type> /redfish/v1/managers/idrac.embedded.1/ethernetinterfaces /redfish/v1/managers/idrac.embedded.1/ethernetinterfaces/<fqdd> /redfish/v1/managers/idrac.embedded.1/accountservice /redfish/v1/managers/idrac.embedded.1/accounts /redfish/v1/managers/idrac.embedded.1/accounts/<account-id> Comments Used to perform idrac reset Reports information about idrac's network services. Includes Web server, SNMP, vmedia, Telnet, SSH, IPMI & KVM. idrac BMC serial interface Access to server System Event Log Access to Lifecycle Controller Log Used to clear LC Log Status of idrac virtual media idrac network interface idrac user accounts
Chassis inventory APIs Dell Redfish API URLs /redfish/v1/chassis /redfish/v1/chassis/system.embedded.1 /redfish/v1/chassis/system.embedded.1/thermal /redfish/v1/chassis/system.embedded.1/sensors/fans /redfish/v1/chassis/system.embedded.1/sensors/fans/<fan-fqdd> Comments Top-level URI for server chassis Reports fan status for server and FX2 chassis /redfish/v1/chassis/system.embedded.1/sensors/temperatures /redfish/v1/chassis/system.embedded.1/sensors/temperatures/<sensor- FQDD> /redfish/v1/chassis/system.embedded.1/power /redfish/v1/chassis/system.embedded.1/power/powercontrol /redfish/v1/chassis/system.embedded.1/sensors/voltages /redfish/v1/chassis/system.embedded.1/sensors/voltages/<voltage-fqdd> /redfish/v1/chassis/system.embedded.1/power/powersupplies Reports thermal data for server and FX2 chassis <Sensor-FQDD> addresses each temperature probe Power consumption and supply status <Voltage-FQDD> addresses each voltage output /redfish/v1/chassis/system.embedded.1/power/powersupplies/<psu-fqdd> <PSU-FQDD> addresses each power supply /redfish/v1/chassis/system.embedded.1/power/redundancy/<psredundancy -FQDD> <PSRedundancy-FQDD> addresses power supply redundancy
System status APIs Dell Redfish API URLs /redfish/v1 /redfish/v1/systems /redfish/v1/systems/<servicetag+nodeid> /redfish/v1/systems/system.embedded.1/actions/computersystem.reset /redfish/v1/systems/system.embedded.1/processors /redfish/v1/systems/system.embedded.1/processors/<processor-fqdd> Comments Top-level API access Server inventory and status information access Server reset operation Details on CPUs /redfish/v1/systems/system.embedded.1/ethernetinterfaces /redfish/v1/systems/system.embedded.1/ethernetinterfaces/<ethernetinterface-fqdd> /redfish/v1/systems/system.embedded.1/ethernetinterfaces/<ethernetinterface- FQDD>/Vlans /redfish/v1/systems/system.embedded.1/ethernetinterfaces/<ethernetinterface- FQDD>/Vlans/<Vlan-FQDD> Reports NIC IP address, DHCP and DNS information. Example <EthernetInterface-FQDD> = NIC.Embedded.1-1-1 /redfish/v1/systems/system.embedded.1/storage/controllers /redfish/v1/systems/system.embedded.1/storage/controllers/<controller-fqdd> Typical <Controller-FQDD>=RAID.Slot.N-1; describes details of controller, backplane, enclosure, attached drives
Registries, Sessions, Tasks and Event APIs Dell Redfish API URLs /redfish/v1/registries/messages/en Comments PowerEdge message registry /redfish/v1/odata Enables OData clients to navigate idrac Redfish resources /redfish/v1/$metadata /redfish/v1/$metadata#<collection or a single resource> /redfish/v1/jsonschemas /redfish/v1/jsonschemas/<file> /redfish/v1/sessionservice /redfish/v1/sessions /redfish/v1/sessions/<sessionid> /redfish/v1/taskservice /redfish/v1/eventservice /redfish/v1/eventservice/actions/eventservice.submittestevent /redfish/v1/eventsubscriptions /redfish/v1/eventsubscriptions/<subscription ID> Provides a metadata document describing the resources and collections that are available at the idrac Redfish service root URI Schema descriptions for all supplied data Redfish session management Redfish internal task management Redfish event management
THANK YOU plus.google.com/+redhat facebook.com/redhatinc linkedin.com/company/red-hat twitter.com/redhatnews youtube.com/user/redhatvideos