Delivering Predictability: The Yocto Project Autobuilder, Automated Sanity Testing, License Collection, and Build Statistics Tracking

Transcription

1 It s not an embedded Linux distribution It creates a custom one for you. Delivering Predictability: The Yocto Project Autobuilder, Automated Sanity Testing, License Collection, and Build Statistics Tracking Elizabeth Flanagan Intel Corporation April 11, 2011

2 Being proactive about code quality Reproducible builds Identify bugs and fix early and often Reduce time needed for code stabilization Avoid integration headaches Build performance history Manage the chaos License compliance Deep QA Testing 2/52

3 Being proactive about code quality Maximizing your ability to respond to changes in a complex embedded ecosystem. 3/52

4 Being proactive about code quality Reduce Software Development Lifecycle Churn. 4/52

5 Complexities Our Recipe 5/52

6 Complexities qemuarm qemumips qemux86 Our Recipe qemuppc qemux86_64 6/52

7 Complexities beagleboard qemuarm routerstation qemumips qemux86 Our Recipe mpc8315e qemuppc qemux86_64 atom-pc 7/52

8 Complexities beagleboard qemuarm routerstation qemumips qemux86 Our Recipe mpc8315e qemuppc qemux86_64 atom-pc emenlow Jasperforest n450 sandybridge crownbay Crownbay -noemgd 8/52

9 Complexities One recipe 5 architectures 4 core BSP 6 non-core BSPs in meta-intel 15 x-compiles But... 9/52

10 Complexities One recipe 16 theoretical different image types per arch * 15 architectures Not all arches support all image types 240 total theoretically possible images sato, lsb, sdk... 10/52

11 Complexities If we're not proactive about code quality, lots of things can go wrong... 11/52

12 Complexities It's only going to get more complex 12/52

13 What we need Reproducible builds Basic QA License tracking Finding the pain points 13/52

14 Yocto Project Delivering Predictability Autobuilder Poky Sanity Testing License Wrangling Build Statistics 14/52

15 Yocto Project Delivering Predictability Autobuilder Poky Sanity Testing License Wrangling Build Statistics 15/52

16 Autobuilders Production Autobuilders Quickly respond to a fast changing code base Avoid Works on my machine -itis Find race conditions Find host dependent issues Find bad commits Help bisect build failures Help find dependency chain breakage 16/52

17 Autobuilders Developer Autobuilders Test cross-compilation before commit Production style builds Small OS footprint Build what you want to build 17/52

18 Autobuilders Yocto autobuilder buildbot based git://git.yoctoproject.org/pokyautobuilder.git Setup in under 5 minutes! 18/52

19 Autobuilders Prerequisites: Python 2.6 python-twisted python-jinja2 python-twisted-mail sqllite 19/52

20 Autobuilders Comes with Basic pokyabconfig.py Helper scripts Easy Installer 20/52

21 Set up your own! cd ~ git clone git://git.yoctoproject.org/poky-autobuilder.git cd poky-autobuilder./scripts/poky-setup-autobuilder both source ~/.profile; cd../poky-master; make start cd../poky-slave; make start 21/52

22 Live Demo 22/52

23 Yocto Project Delivering Predictability Autobuilder Poky Sanity Testing License Wrangling Build Statistics 23/52

24 Sanity Testing Extensible Frees up QA resources Reproducible smoke testing Multiarch/multiimage scenarios Can run automatically post build via local.conf Or via an autobuilder 24/52

25 Sanity Testing Sanity Test Bitbake Class Architecture/image based test scenario Test library/runner Test helper scripts 25/52

26 Sanity Testing meta/classes/imagetest-qemu.bbclass scripts/qemuimage-tests/scenario/${arch}/* scripts/qemuimage-testlib and runners scripts/qemuimage-tests/{sanity tools}/* 26/52

27 Sanity Testing Test suite Architecture and image based scenarios Very easy to add already existing tests 27/52

28 Sanity Testing sanity ssh sanity scp sanity dmesg sanity zypper_help sanity zypper_search sanity rpm_query sanity connman sanity shutdown 28/52

29 Sanity Testing Tests bash/expect based test scripts called via test runners in scripts/sanity Tests stored in qemuimage-testlib More secure to create tap devs with poky-gen-tapdevs. 29/52

30 Sanity Testing Gotchas: QEMU user NOPASSWD More secure to create tap devs with poky-gen-tapdevs. For headless, see wiki docs: 30/52

31 Yocto Project Delivering Predictability Autobuilder Poky Sanity Testing License Wrangling Build Statistics 31/52

32 License Wrangling Verify image compatibility to required license type non-gplv3 Provides an entire package directory tree Actual licenses Generics. Helps maintain license compliance 32/52

33 Recipe contains: LICENSE License Wrangling Tells license.bbclass the common license type Symlink from license wrangling output to a generic LIC_FILES_CHECKSUM License file URI Checksum Where we get the specific license 33/52

34 License are found in: License Wrangling ${POKYBASE}/build/tmp/deploy/images/licenses 34/52

35 Yocto Project Delivering Predictability Autobuilder Poky Sanity Testing License Wrangling Build Statistics 35/52

36 Build Statistics Build level: Host info Elapsed build time CPU usage Build failure information 36/52

37 Build Statistics Package level: List of events triggered Elapsed event time CPU usage Event failure information 37/52

38 Build Statistics Image Type Build Package Event Event Event Package Event Event Event build/tmp/deploy/images/licenses/: poky-image-minimal-qemux86 ` build_stats -- autoconf-native-2.65-r2 -- do_compile -- do_configure -- do_fetch -- do_install -- do_patch -- do_populate_sysroot -- do_setscene `-- do_unpack -- automake-native r1 -- do_compile -- do_configure -- do_fetch -- do_install -- do_patch -- do_populate_sysroot -- do_setscene `-- do_unpack 38/52

39 Build Statistics Gives us: Performance indicators Track down issues CPU/Dependancy/IO bound Visualize your build performance Patch to pybootchart 39/52

40 Build Statistics Visualization Time 40/52

41 Build Statistics Visualization 41/52

42 Build Statistics Visualization 42/52

43 Build Statistics Visualization 43/52

44 Build Statistics Visualization 44/52

45 Where do we go from here? 45/52

46 Where do we go from here? Autobuilder: Meta-targets Helper script integration into main config Continuous integration 46/52

47 Where do we go from here? License tracking: More generic license files Better LICENSE field parsing 47/52

48 Where do we go from here? Build Statistics: Collect even more data. Image size w/o free space. Better data visualization. Web based 48/52

49 Production autobuilder 49/52

50 Resources pybootchartgui patch for build statistics 50/52

51 Legal Information INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY RELATING TO SALE AND/OR USE OF INTEL PRODUCTS, INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT, OR OTHER INTELLECTUAL PROPERTY RIGHT. Intel may make changes to specifications, product descriptions, and plans at any time, without notice. All dates provided are subject to change without notice. Intel is a trademark of Intel Corporation in the U.S. and other countries. *Other names and brands may be claimed as the property of others. Copyright 2009, Intel Corporation. All rights are protected. 51/52

52 52/52