Effective packaging. Petr Uzel. L3/Maintenance ptr IRC

Transcription

1 Effective packaging Petr Uzel L3/Maintenance ptr IRC

2 About this talk collection of tips and tricks that make (some) package maintenance tasks more efficient no rocket science, no cool technologies based on experience there are definitely better ways 2/1 c Novell Inc. All Right Reserved

3 Which package is broken? given a bug (e.g. /path/to/bin crashes), sometimes it is not obvious which package should be fixed bugzilla does not track package info rpm -qf /path/to/file gives name of the RPM we need source package, not RPM $ rpm -qf /lib64/libblkid.so.1 libblkid x86_64 $ osc search -s --package blkid No matches found for blkid in packages 3/1 c Novell Inc. All Right Reserved

4 Source package name let s ask RPM: $ rpm -q --queryformat %{SOURCERPM}\n \ libblkid x86_64 util-linux src.rpm Everything combined together (file source package) $ rpm -q --queryformat %{SOURCERPM}\n \ $(rpm -qf /lib64/libblkid.so.1) \ sed -e s/\(^.*\)-.*-.*src.rpm/\1/ util-linux Where did I install the package from? $ rpm -q --queryformat %{DISTURL}\n util-linux obs://build.opensuse.org/home:puzel:staging/opensuse_11.3/6a[cut] 4/1 c Novell Inc. All Right Reserved

5 Get package source from OBS With branching: slow: osc branch opensuse:factory foo; osc co home:*:branches:prj faster: osc branch --checkout opensuse:factory foo fastest: osc getpac foo or osc bco foo hyperspeed: osc getpac + osc bash completion Without branching: function show_devel { osc meta pkg opensuse:factory ${1} fgrep devel } go directly to devel project 5/1 c Novell Inc. All Right Reserved

6 Building package osc build in package working copy usually does the right thing more control: osc build [OPTS] REPOSITORY ARCH SPECFILE REPOSITORY, ARCH and SPECFILE detected automatically if not specified list REPO-ARCH combinations: osc repos [PRJ] 6/1 c Novell Inc. All Right Reserved

7 osc build options build debuginfo and debugsource packages: osc build --debuginfo/-d skip post build checks: osc build --nochecks delete old build root before build: osc build --clean use alternative project for build: osc --alternative-project=opensuse:factory use ccache to speed up build: osc build --ccache 7/1 c Novell Inc. All Right Reserved

8 Use custom RPMs in buildroot foo depends on bar, which is not in buildservice yet but we have bar.rpm osc build -p/--prefer-pkgs DIR: prefer packages from DIR when installing buildroot $ cd foo $ osc build -p /path/to_dir_with_bar.rpm foo.spec osc build -x/--extra-pkgs PKG: add this package to buildroot 8/1 c Novell Inc. All Right Reserved

9 Buildroot tips by default, buildroot is /var/tmp/build-root OSC BUILD ROOT overrides the default $ export OSC_BUILD_ROOT=/var/tmp/build-root-temp $ osc build use when parallelly building multiple packages buildroot on tmpfs 5% speedup of parted build $ grep build-root /etc/fstab tmpfs /var/tmp/build-root tmpfs size=5g 0 0 9/1 c Novell Inc. All Right Reserved

10 Avoid password for building by default, osc build asks for root password; to avoid it set su-wrapper = sudo in ~/.oscrc add yourlogin ALL = (root) NOPASSWD: /usr/bin/build to sudoers file 10/1 c Novell Inc. All Right Reserved

11 Build log build log is stored in $OSC BUILD ROOT/.build.log slow: less /var/tmp/build-root/.build.log faster: osc lbl/localbuildlog respects $OSC BUILD ROOT from working copy osc rbl/remotebuildlog PRJ PKG REPO ARCH - get remote build log 11/1 c Novell Inc. All Right Reserved

12 Version update workflow 1 get package source 2 download tarball 3 verify tarball 4 repack tarball 5 check differences 6 adjust specfile 7 refresh patches 8 run test build 9 test the package 10 write changelog entry 11 commit 12/1 c Novell Inc. All Right Reserved

13 Downloading tarball with ncftp many projects publish tarballs on FTP sites ncftp is a command-line FTP client with bookmarking support useful to remember locations where upstream publishes tarballs supported by bash-completion $ ncftp ut<tab> -> ncftp util-linux ncftp... ls ncftp... get fil<tab> -> filename ncftpbookmarks: bookmark editor 13/1 c Novell Inc. All Right Reserved

14 Repacking tarball packaging policy requires the tarball to be bzip2-ed (or xz-ed) sometimes upstream provides only.tar.gz $ bznew parted-2.3.tar.gz $ ls *.bz2 parted-2.3.tar.bz2 14/1 c Novell Inc. All Right Reserved

15 What s new in tarball? Compare outputs from configure --help $ git clone git://gitorious.org/opensuse/pack-tools.git $ diffconf gnupg tar.bz2 gnupg tar.bz2 15/1 c Novell Inc. All Right Reserved

16 Change specfile usually just open specfile and increase the version number small helper: $ alias vs= vim -p *.spec in vim (normal mode), press Ctrl+A/Ctrl+X to increase/decrease numbers 16/1 c Novell Inc. All Right Reserved

17 Packaging with quilt about quilt Quilt allows you to easily manage large numbers of patches by keeping track of the changes each patch makes. Patches can be applied, un-applied, refreshed, and more. 17/1 c Novell Inc. All Right Reserved

18 Packaging with quilt quilt setup initialize a source tree from RPM spec file 1 unpack tarball 2 apply patches 3 un-apply patches (but remembers the series) IOW: %prep phase $ quilt setup -v foo.spec $ cd foo shortcut: $ alias qs= quilt setup -v *.spec 18/1 c Novell Inc. All Right Reserved

19 Packaging with quilt working with series of patches now we are in the directory created by quilt setup (unpacked tarball) none of the patches is applied (but quilt knows about them) quilt push applies one patch (-a: all patches) quilt pop un-applies one patch (-a: all patches) quilt top prints name of the last applied patch quilt refresh refreshes last patch (so that it applies cleanly, without fuzz and offsets) quilt series prints the names of all patches 19/1 c Novell Inc. All Right Reserved

20 Packaging with quilt inspecting patches $ quilt diff $ quilt diff --color=auto or quilt diff colordiff $ quilt files or quilt diff lsdiff $ quilt diff diffstat $ quilt diff -U num view differences in vimdiff (side by side): $ alias qdv= quilt diff -p0 -C color=never --diff=vimdiff 20/1 c Novell Inc. All Right Reserved

21 Packaging with quilt creating new patch 1 quilt push -a 2 quilt new foo-fix-bug.patch 3 quilt add src/file.c 4 vim src/file.c 5 hack, hack 6 quilt refresh steps 3 and 4 can be combined to quilt edit src/file.c the patch appears in parent directory (after quilt refresh) you have to add it to specfile 21/1 c Novell Inc. All Right Reserved

22 Packaging with quilt importing patch goal: import patch (e.g. taken from upstream) to quilt 1 quilt push -a 2 quilt import /path/to/bugfix.patch 3 quilt push 4 quilt refresh bugfix.patch appears in the parent directory -p num controls how many directory levels to strip (as with patch(1)) -P name overrides name of the patch multiple patches can be imported at once 22/1 c Novell Inc. All Right Reserved

23 Packaging with quilt rejected patches when quilt push fails, use quilt push -f applies as much as possible, leaves rejects (file.c + file.c.rej) $ quilt push Applying patch gnupg-files-are-digests.patch patching file g10/gpg.c Hunk #3 succeeded at 1998 (offset -4 lines). Hunk #4 succeeded at 2487 (offset -4 lines). patching file g10/options.h Hunk #1 FAILED at out of 1 hunk FAILED -- rejects in file g10/options.h patching file g10/sign.c Patch gnupg-files-are-digests.patch does not apply (enforce with -f) 23/1 c Novell Inc. All Right Reserved

24 Packaging with quilt rejected patches - continued failed hunks in *.c.rej have to be applied manually (vim *.c) then quilt refresh helper: vim quilt plugin (zypper install vim-plugin-quilt) (Contrib) vim foo.c - if there is also foo.c.rej - opens vertically split view easier to copy code from rejected hunks to original file the plugin can do more: :help quilt 24/1 c Novell Inc. All Right Reserved

25 Packaging with quilt make quilt more friendly $ alias q= quilt $ alias qd= quilt diff --color=always $ alias qdv= quilt diff -p0 -C color=never --diff=vimdiff $ alias qp= quilt push --color=auto $ alias qs= quilt setup -v *.spec $ cat ~/.quiltrc # show C function name in hunk header QUILT_DIFF_OPTS= -p 25/1 c Novell Inc. All Right Reserved

26 osc status, osc addremove we have to tell osc about new/deleted files example: update foo-1.0 to foo-2.0 $ ls foo*bz2 foo-1.0.tar.bz2 foo-2.0.tar.bz2 $ osc status? foo-2.0.tar.bz2 $ rm foo-1.0.tar.bz2 $ osc ar # osc addremove D foo-1.0.tar.bz2 A foo-2.0.tar.bz2 $ osc st # osc status D foo-1.0.tar.bz2 A foo-2.0.tar.bz2 osc add + osc remove = osc ar 26/1 c Novell Inc. All Right Reserved

27 Committing changes osc diff displays changes in the package more readable: $ alias osccd= osc diff colordiff less osc vc opens foo.changes file (prepares new entry header, unless -e option is given) vim can open compressed tarballs: :tabnew foo-2.0.tar.bz2 search for Changelog, NEWS,... ; open copy, paste and edit changelog entry osc commit osc sr 27/1 c Novell Inc. All Right Reserved

28 Vim and changelogs to make changelog editing with vim more convenient, add to.vimrc: set nocompatible filetype plugin on syntax on autocmd BufRead,BufNewFile *.changes.* set filetype=changes autocmd FileType changes set spelllang=en_us spell textwidth=67 28/1 c Novell Inc. All Right Reserved

29 osc bash completion supports completing: osc subcommand names (checkout, commit,... ) global long osc options (e.g. --quiet, --debug,... ) long subcommand options project names package names (in given project) file names (in given project/package) repository names (opensuse Factory,... )... git clone git://gitorious.org/opensuse/osc-bash-completion.git see README 29/1 c Novell Inc. All Right Reserved

30 osc bash completion examples osc com<tab> => osc commit osc build --a<tab> => osc build --alternative-project osc ls Ar<tab> => osc ls Archiving osc ls Base:System pa<tab> => osc ls Base:System parted osc meta a<tab> => osc meta attribute osc request <tab> => osc request accept approvenew co decline checkout list log revoke show wipe... 30/1 c Novell Inc. All Right Reserved

31 Browsing source code packager often needs to understand code written by somebody else application crashes patch does not apply to new version build fails there are tools that significantly help: ctags cscope both work with C/C++, ctags supports many other languages zypper install ctags cscope 31/1 c Novell Inc. All Right Reserved

32 Browsing source code ctags ctags creates database of symbols in the source code tree editor (or other tools) use this database to quickly search for symbols and jump to the definition to create the database (tags file): $ cd foo $ ctags -R. vim automatically uses ctags database if present Ctrl+] / Ctrl+L click jump to definition of symbol under cursor Ctrl+t / Ctrl+R click jump back :tag symbol jump to definition of symbol vim -t symbol open vim and jump directly to definition of symbol :help ctags 32/1 c Novell Inc. All Right Reserved

33 Browsing source code vim + ctags + bash completion sometimes you forget exact name of the symbol bash completion becomes handy: $ ctags -R. $ vim -t sym<tab> => vim -t symbol_name_full bash completion with ctags and Vim put it to ~/.bash completion zypper install bash completion 33/1 c Novell Inc. All Right Reserved

34 Browsing source code cscope ctags can only jump to symbol definition; cscope can do more whith cscope, it is possible to quickly lookup e.g. where is the symbol defined (same as ctags) where is the symbol used what functions call this function what functions are called by this function which files include this file... support in vim vim tutorial.html 34/1 c Novell Inc. All Right Reserved

35 Browsing source code taglist source code browser plugin for vim provides overview of the structure of source code files zypper install vim-plugin-taglist (Contrib) set up taglist binding in.vimrc: nnoremap <silent> <F8> :Tlist<CR> 35/1 c Novell Inc. All Right Reserved

36 Browsing source code taglist screenshot 36/1 c Novell Inc. All Right Reserved

37 Patches from Fedora do not reinvent wheel - before writing patch, make sure it does not already exist Fedora uses git now: # Credits: Pavol Rusnak function fedora_getpkg() { git clone git://pkgs.fedoraproject.org/${1}.git } 37/1 c Novell Inc. All Right Reserved

38 Patches from git... with tig tig is ncurses interface for git (zypper install tig) run tig from the cloned repository, press h for help to find and extract patch form git repository, add the following to /.tigrc: bind generic P!git format-patch %(commit)^..%(commit) run tig (e.g. tig v2.2..) select the patch you want to extract hit P - the patch will be saved as sth like 0001-fix-name.patch 38/1 c Novell Inc. All Right Reserved

39 RPM macros sometimes not obvious what rpm %macro does rpmbuild --showrc lists all defined RPM macros rpm -E %macro prints expanded macro supported by bash completion: $ rpm -E %la<tab> => rpm -E %lang_package %package lang Summary: Languages for package %{name} Group: System/Localization Requires: %{name} = %{version} Provides: %{name}-lang-all = %{version} Supplements: packageand(bundle-lang-other:%{name}) BuildArch: noarch %description lang Provides translations to the package %{name} 39/1 c Novell Inc. All Right Reserved

40 EOF Questions? 40/1 c Novell Inc. All Right Reserved