Helping Developers Help You. Tom Caputi

Size: px

Start display at page:

Download "Helping Developers Help You. Tom Caputi"

Cuthbert Knight
5 years ago
Views:

1 Helping Developers Help You Tom Caputi

2 The ZFS Code Base 2

3 The ZFS on Linux* Code Base: Statistics Source code 641 files lines of code ZFS Test suite (not including ztest) 1327 files lines of script Contributors 3435 git commits 239 People 15 Sponsor Companies of the 2017 Developer Summit * Not counting Illumos, FreeBSD, OSX code bases! 3

4 4

5 5

6 Mistakes Happen! We re sorry. We are trying our best No one person knows the entire code base Please help us make ZFS better for everyone 6

7 The ZoL Git Repository 7

8 What is Github? git is a source code version control software Helps to keep track of code contributions Developers can work simultaneously on different issues Github is a website built around git Acts as the central truth of what ZFS on Linux is Issue tracking and comment threads Release management Hooks for running automated tests Easy(ish) user interface to manage the project 8

9 9

10 ZFS repo: SPL repo: 10

11 Latest commit: contribution 11

12 Source code (including the last commit that touched that file / directory) 12

13 Commit log 13

14 14

15 Title and description 15

16 Authors and contributors 16

17 Diff: list of code changes made in this commit 17

18 Commit hash: unique identifier for this commit 18

19 19

20 Issues / Feature requests submitted by users 20

21 21

22 Reporting Issues and Debugging Basics 2 2

23 Search for existing issues Step 1: Check to make sure your issue hasn t already been filed Read through the ticket to make sure it matches your symptoms If the ticket is closed, read the comments for workarounds / solutions If the ticket is open, skip Step 2 23

24 Create a new issue ticket Step 2: If you don t find an issue like yours open a new ticket 24

25 Create a new issue ticket Step 3: Provide any debugging info you may have. If you think that the problem involves a certain feature, you tag the developer of that feature to get their attention 25

26 Providing Debugging Information: The Basics System information ZFS Version: cat /sys/module/zfs/version Kernel version: uname -r Operating System Distribution CPU architecture: (almost always x86_64) ZFS Configuration Pool layout: zpool status <pool> List of relevant datasets: zfs list Properties of affected datasets: zfs get all <dataset> 26

27 Identifying Problems: Crashes and Deadlocks Crashes Task never completes Commands can t be killed System may become completely unresponsive Stack trace appears in dmesg Deadlocks Very similar to a crash (commands never complete, can t be killed) System still responsive on other consoles Stack traces may appear in dmesg after 5 minutes or so 27

28 28

29 Stack trace: shows where in the code the problem occurred * include a few lines above the trace 29

30 Identifying Problems: Other Issues Performance Issues Task eventually completes, but takes too much time / system resources Commands can be killed Warnings may be printed in dmesg (probably not) Be sure to provide statistics about what isn t performing well Other Issues Errors: ZFS returns errors when commands are used properly Non-intuitive behavior: ZFS does not function reasonably UI problems: commands / docs are inaccurate or unclear 30

31 Providing Debugging Information: Extended Steps to Reproduce THE MOST HELPFUL INFO YOU CAN PROVIDE Drastically reduces time to fix Allows developer to work on a fix without waiting for you to reply Most other debug info isn t important if you have this Sequence of commands that trigger the issue If the issue is intermittent, provide approximate reproduction rate Information on ZFS workload If you can t reproduce the info provide info about ZFS usage Kinds of files stored, zfs commands being run, etc 31

32 Providing Debugging Information: Extended Recompiling with --enable-debug and --enable-debuginfo ZFS has a compilation option that includes added sanity checks Turns on ASSERTs Turns on extra debugging by default Forces debug symbols to be compiled into the binaries Try this if: You are proficient at compiling your own source code Your issue is causing your system to panic You want to ensure that you are providing as much info as possible 32

33 Providing Debugging Information: Situational ZFS debug message log Does not require debug build echo 1 > /sys/module/zfs/parameters/zfs_dbgmsg_enable echo 0xffffffde > /sys/module/zfs/parameters/zfs_flags cat /proc/spl/kstat/zfs/dbgmsg Line Number from stack trace Requires debug build Allows you to convert a stack trace to a line of code (See next slide) 33

34 34

35 Some stack traces will include a RIP: gdb -batch -ex \ 'list *(multilist_sublist_remove+0x10)' \ $(modinfo zfs grep filename awk '{ print $2 }') 35

36 Issue Examples 3 6

37 Issue Examples Hard Issues Easy Issues

Section 1: Tools. Contents CS162. January 19, Make More details about Make Git Commands to know... 3

Section 1: Tools. Contents CS162. January 19, Make More details about Make Git Commands to know... 3 CS162 January 19, 2017 Contents 1 Make 2 1.1 More details about Make.................................... 2 2 Git 3 2.1 Commands to know....................................... 3 3 GDB: The GNU Debugger