Getting Started with Contributing to OpenStack An Introductory Crash Course on OpenStack Development

Transcription

1 Getting Started with Contributing to OpenStack An Introductory Crash Course on OpenStack Development Mike Turek mjturek on freenode

2 Who Am I? Mike Turek Software Engineer at IBM in the Linux Technology Center Started in June 2014 after finishing my Master's in Computer Science at Binghamton University Currently focused on Power enablement for OpenStack's Ironic project. Specifically CI testing using baremetal nodes.

3 What to Expect Overview of OpenStack and a handful of its integrated projects. A focus on Nova. Overview of DevStack and its role in OpenStack Overview of Gerrit and some best practices when working with the community. A demonstration of DevStack and a sample bug.

4 Main OpenStack Components OpenStack Compute (Nova) Provision and manage instances OpenStack Network (Neutron) Manages network resources. OpenStack Storage (Cinder & Swift) Block and object storage respectively Shared Services (Glance, Keystone, Heat, Trove, etc.)

5 Basic OpenStack Structure

6 Nova Manages instances, flavors, images, and network resources Spun off many of the integrated projects. Glance is the image service Neutron is the network service (Will eventually deprecate nova-net) Ironic replaces the nova baremetal driver

7 Nova Resources Instances Virtual Machines the most common usage. Several drivers accomplishing this (libvirt, vmware, hyperv). Physical Hardware Originally just the baremetal driver in Nova. Now accomplished by the Ironic project. Containers Currently accomplished via Nova-Docker and Libvirt-LXC drivers. Whatever else might come along! Nova is extensible. Flavors Specifies what combinations of resources an instance can use Images Specifies the OS image to be used to build the instance.

8 More on Shared Services Glance Storage and retrieval of images Keystone Oslo Ironic Identity service that projects talk to in order to authenticate requests. Common functions used throughout OpenStack Allows provisioning baremetal machines as simply as virtual machines

9 Governance Each project has a project team lead (PTL) Makes key decisions about the project. Priorities for a cycle, etc. Each project has core reviewers Well respected community members get higher voting power. You need two core reviewers to approve a patch.

10 OpenStack is... Open source Apache 2.0 license Written in Python For the most part On a six month release cycle Named in alphabetical order, each referring to something related to that release's summit location Not simple to deploy Vastly configurable, not simple to deploy

11 DevStack Quickly and easily deploy OpenStack Creates an all in one environment All services run on the same machine OpenStack integrated project Important to developers and the Tempest project NOT production ready Intended for creating volatile environments Great for development and testing

12 DevStack local.conf This will provide everything we need for today. [[local localrc]] ADMIN_PASSWORD=supersecret DATABASE_PASSWORD=$ADMIN_PASSWORD RABBIT_PASSWORD=$ADMIN_PASSWORD SERVICE_PASSWORD=$ADMIN_PASSWORD SERVICE_TOKEN=tokenizemecaptain The [[local localrc]] section local refers to when in the stacking process to source this information. (One of the first things to happen) localrc refers to the file using these options.

13 The DevStack Toolbox stack.sh Sets up your environment using local.conf unstack.sh Tears down the environment rejoin-stack.sh Devstack runs each component in foreground mode in a separate screen. After reboot screens must be recreated and components restarted. create-stack-user.sh Creates a user account suitable for running DevStack.

14 Gerrit A code collaboration tool Adds the concept of a store of pending changes Provides a forum for code reviews Each patch has a web page where you can review, comment, and vote on patches Integrates with git via the git-review plugin Won't be going into setup, for help see HowTo/FirstTimers#Set_up_git_and_git_review

15 Gerrit

16 The Gerrit Review System Patches come in and get voted on by Core reviewers (-2, -1, 0, +1, +2) Reviewers (-1, 0, +1) CI Systems (-1, 0, +1) Each patch needs two +2 votes to get approved Fix will then be committed A -2 from a core will always block the patch Fix will eventually be merged into the in progress branch.

17 Launchpad OpenStack uses Launchpad for bug tracking Each project has a launchpad page Where you can find, report, and discuss bugs. Also used for blueprints

18 Launchpad Blueprints A tool for planning larger features and contributions Same flow as a patch Usually submitted towards the beginning of a cycle. Specifies problem, approach, alternatives, and impacts Submitter is usually the implementer.

19 More on Finding Bugs Bugs can be organized by Priority, Status, and other qualities within Launchpad Highly recommend the advanced search Lets you filter out bugs that have people assigned Show certain priorities and tags (low-hanging-fruit) Bugs with Triage status Solution has been discussed but needs someone to implement it. Can be a good place to start

20 Best Practices Keep patches small. A larger patch may require a blueprint Make your commit messages clear. A commit message should succinctly explain What your patch does. Why your patch is needed. Keep discussions on the review page. s and IRC chats can be helpful, but in the end its best to keep discussions public and organized.

21 Nobody's Perfect Patches rarely are accepted on their first submission Expect to get -1's Just the community saying that a change needs to be made to your patch Nothing personal and not a big deal Discuss with the community how to resolve their concerns and resubmit

22 Other Tips Communities will sometimes do bug days or sprints. A good way to find and discuss bugs Can be a fun way to contribute and get to know the community Following IRC chats and mailing lists will help you find out when these are happening Code reviews are appreciated A great way to build trust within the community and learn the code base.

23 Need Help? Talk to the Community! Each project has a mailing-list Great place to discuss features and bugs Each project has a freenode IRC channel Development discussions, not usage questions. Each project has a weekly meeting. Where to follow what's important to the community Open section to bring up concerns and bugs. See each project's page for information. Example -

24 Demonstration Slides are provided for audience use and will not be shown during the presentation. If you'd like a copy of this deck, please let me know!

25 Python Clients Provide a simple interface to the OpenStack services All follow the following format <project_name> <openstack_options> <command> <command_options> <positional_arguments> nova os username mjturek boot flavor 3 image ubuntu mjturek_instance Help commands for each project and each command nova help nova help list

26 Sample Bug Let's review a sample bug. A simple usability bug. Disclaimer This demo will be on a bug from the Juno time frame. This means that I needed to use the stable/juno branch rather than the current upstream branch for this demo.

27 Create a branch Each project uses git Branch within the directory of the project you're modifying. For nova, branch within /opt/stack/nova To create a branch git checkout -b mjturek/

28 Replicate the Bug Bug report indicates that uninformative log happens when there's a NoValidHost exception Could overload the system until it can't provide a host. Let's just force the NoValidHost to occur via code.

29 Where to Force NoValidHost Notice that the bug report shows an exception occurring WARNING nova.scheduler.driver This is the class where the exception is reported So we know it's happening during scheduling. We're using the FilterScheduler Found in nova.conf

30 Where to Force NoValidHost Let's modify the FilterScheduler to throw the exception Find where it normally throws a NoValidHost Force it to happen Should now see the Exception in the logs and the instance in an error state

31 Uh oh... The 'nova boot' succeeded... It seems that our changes didn't take. Need to restart the process for changes to take effect. Enter 'screen -x'

32 screen screen is a fullscreen window manager. DevStack deploys each OpenStack service in a process within one terminal window. Some useful commands Ctrl+A D detach from screen (return to terminal) Ctrl+A N Go to the next screen Ctrl+A P Go to previous screen Ctrl+A [0-9] Go to respective screen number Ctrl+A [ Copy mode (Lets you scroll back through logs) Ctrl+A Enter a window number to switch to.

33 Restart the Process We modified nova code. Go through each screen and restart ones starting with n- Switch to each screen Ctrl+C to stop the process Up one entry (up-arrow on keyboard) Press enter. Let's try another 'nova boot'

34 There's the Exception Look through the nova screens (screens that start with n-) and you should find the exception. We're now seeing what the bug reporter is seeing. Let's improve the logging message!

35 Find Where to Modify Remember the warning we see? WARNING nova.scheduler.driver Again, this is the class where the exception is reported Found in '/opt/stack/nova/nova/scheduler/driver.py' Searching for the exception shows where the modification needs to happen.

36 The Fix It seems that when a NoValidHost exception happens, we simply set the state to error, and throw a warning at the user. NoValidHost exceptions have a 'reason' string in them. Let's report that 'reason' string!

37 Clean Up We don't want to submit that exception being forced! This would break nova! We only forced the exception for testing, so remove it. A quick test should show instances are succeeding again. It's time to submit, but first we need the gitreview plugin

38 Submit Your Patch Running 'git diff' will show your changes Now commit your changes git commit -a If after initially committing you need to make more changes, you can update your commit with git commit -a amend You will be asked to enter a commit message

39 The Commit Message More informative nova scheduler logging This patch adds a warning log entry for when a NoValidHost exception causes a scheduling error. The NoValidHost exception's message will be logged before reporting that the machine went to error. Previously, all other exceptions would show a stack trace. However, since the NoValidHost case is skipped, a user may not understand why his/her machine went to an ERROR state. This patch seeks to remove any confusion. Closes Bug: The first line has a 50 character limit The body has a 72 character limit per line Ending section has tags for Closes-Bug: <bug number>, Implements: <blueprint>, Change ID: <uuid generated by git review> For specifics, see:

40 Submit Now that you've committed your changes to your branch, you can submit it for review Make sure you're within the project's directory and type git review Upon submission you'll receive a link to your review page.

41 Debugging When you require more robust debugging Import your favorite python debugger (pdb, pudb) and set breakpoints where needed Restart the processes where you've added breakpoints Pdb will come up within the screen of the respective process