OpenStack Infrastructure tools

Size: px

Start display at page:

Download "OpenStack Infrastructure tools"

Evelyn Booth
5 years ago
Views:

1 Coordination and OpenStack Infrastructure tools you will want to borrow Thierry Carrez Carrez Thierry Release Manager, OpenStack

3 Crazy loads every day 000 patchsets are proposed 7500 Gerrit comments and votes posted 6000 test environments spawn 50 changes merged

4 What we use Gerrit in producing OpenStack Puppet Jenkins

5 Fully automated & open source Everything deployed by Puppet Only open source tools Our puppet manifests are open source Our config files are open source

9 Zuul: a scalable gate system Only merge changes if tests pass Receives events about proposed changes Triggers tests and reports back Handles cross-project dependencies

10 Serial gating

11 Serial gating

12 Serial gating 3

13 Serial gating 3

14 Serial gating 3 4

15 Parallel gating 3 4

16 Parallel gating 3 4

17 Parallel gating 3 4Thierry Carrez (@tcarrez)

18 Speculative gating 3 3 4

19 Speculative gating 3 3 4

20 Speculative gating 3 3 4

21 Speculative gating Thierry Carrez (@tcarrez) 4

22 Speculative gating Thierry Carrez (@tcarrez) 4

23 Speculative gating Thierry Carrez (@tcarrez) 4

24 Speculative vs. Serial gating Thierry Carrez (@tcarrez) 4 3 4

25 Zuul Speculative Gating Strict gating Optimize for speed, not resources At best, as fast as parallel gating At worse, as slow as serial gating

26 Visualizing Zuul

28 Jenkins-job-builder Makes handling thousands of jobs easier YAML descriptions converted to XML jobs New jobs are sent to Jenkins using the API

29 Macros and job descriptions job: name: myproject docs node: bare trusty triggers: zuul builders: git prep docs publishers: scp: site: 'scp server' files: target: 'dir/ectory' source: 'build/html/foo' keep hierarchy: true console log builder: name: git prep builders: shell: "/slave_scripts/git prep.sh" builder: name: docs builders: shell: "/slave_scripts/run docs.sh" publisher: name: console log publishers: scp: site: 'scp server' files: target: '$JOB_NAME/$BUILD_NUMBER' copy console: true copy after failure: true

30 Job templates job template: name: '{name} docs' node: '{node}' triggers: zuul builders: git prep docs publishers: scp: site: 'scp server' files: target: 'dir/ectory' source: 'build/html/foo' keep hierarchy: true console log project: name: myproject node: bare trusty jobs: {name} docs project: name: myproject node: bare centos6 jobs: {name} docs

31 Jeepyb A collection of tools to: - manage Gerrit projects & groups easily - integrate Gerrit with GitHub, Launchpad

32 Describe your Gerrit projects project: myorg/myproject description: Best project ever. remote: git://github.com/ttx/proj.git use storyboard: true project: myorg/project description: Another cool project. acl config: /path/to/project.acl [access "refs/heads/*"] label Code Review =..+ group proj core label Workflow =..+ group proj core [access "refs/heads/proposed/*"] label Code Review =..+ group proj lead label Workflow =..+ group proj lead [receive] requirechangeid = true [submit] mergecontent = true

33 Jeepyb periodic tasks Coordination and manage-projects Leadership Creates Gerrit projects challenges & groups, GitHub mirrors in producing OpenStack close-pull-requests Closes GitHub pull requests for projects managed in Gerrit expire-old-reviews Thierry Carrez Marks old Gerrit changes Release management PTLAbandoned

34 Jeepyb commit hooks Coordination and update-bug, update-blueprint Leadership Launchpad bugs & blueprint challenges integration in producing OpenStack notify-impact Send notification based on commit message trivial-rebase Detectsmanagement trivial rebase and Release PTL reapplies votes

36 git-review A helper to submit changes to Gerrit Sets up the 'gerrit' remote if missing Rebases on master by default Support hooks

37 git-review workflow git clone git://git.openstack.org/openstack/nova.git cd nova # Change a few files here ttx@mercury:~/nova$ git commit a ttx@mercury:~/nova$ git review remote: Processing changes: new:, refs:, done remote: remote: New Changes: remote: remote: To ssh://ttx@review.openstack.org:948/openstack/nova * [new branch] HEAD > refs/publish/master/branch nova/.gitreview [gerrit] host=review.openstack.org port=948 project=openstack/nova.git defaultbranch=master

38 Easier reviews with git-review git review d 4584 downloads change 4584 for review git review m 54,4 0 Thierry (@tcarrez) showscarrez changes between patchset 4 and 0

39 gertty CLI version of Gerrit web frontend /newsgroup reading workflow Lets your work offline

40 Subscribing to projects

41 Listing reviews

42 Reviewing a change

43 Side-by-side diff

44 Voting

46 Why StoryBoard Need for cross-teams coordination Launchpad Bugs is great Launchpad Blueprints is not great Limits in Launchpad design (scale, API)

47 StoryBoard base concepts Stories describe the problem to solve Stories have tasks affecting projects API server / Webclient design

49 Docs at: ci.openstack.org Join us on Freenode: #openstack infra

50 Thank you!

flickr.com/photos/jurvetson/74084534) Coordination by Doug Wheller (https://www.flickr.com/photos/doug88888/468790667) Sharpener by Andrew Kelsall (https://www.

51 on Coordination ttx on and picture credits Robot dance by Steve Jurvetson ( Coordination by Doug Wheller ( Sharpener by Andrew Kelsall ( Dam by Tim Hamilton (

The OpenStack Project Continuous Integration System. Elizabeth K.

The OpenStack Project Continuous Integration System Elizabeth K. Joseph @pleia2 Elizabeth K. Joseph Core/root member of the OpenStack Infrastructure Team Author of Common OpenStack Deployments (along with