Automated Documentation Proofreading. Warren Block FreeBSD documentation committer

Transcription

1 Automated Documentation Proofreading Warren Block FreeBSD documentation committer

2 Why is Documentation Hard to Write?

3 Why is Documentation Hard to Write? Rules, rules, so many different rules!

4 Why is Documentation Hard to Write? Rules, rules, so many different rules! Text files

5 Why is Documentation Hard to Write? Rules, rules, so many different rules! Text files mdoc(7)

6 Why is Documentation Hard to Write? Rules, rules, so many different rules! Text files mdoc(7) DocBook SGML

7 Why is Documentation Hard to Write? Existing documentation is inconsistent Learning by example is difficult when the examples vary wildly in quality

8 Why is Documentation Hard to Write? Toolchains are unhelpful Format and style errors are often unreported <para>the new setting may be viewed as before, notice the <literal>s</literal> is now in the field designated for the group permission settings:<para>

9 Why is Documentation Hard to Write? Toolchains are unhelpful Format and style errors are often unreported <para>the new setting may be viewed as before, notice the <literal>s</literal> is now in the field designated for the group permission settings:</para>

10 Why Worry? If It Builds, Ship It! Quality Consistency encourages quality

11 Why Worry? If It Builds, Ship It! Maintenance Clean, consistent documents are easier to understand, maintain, and modify

12 Why Worry? If It Builds, Ship It! Conversion To Other Formats DocBook XML mandoc future formats

13 Why Worry? If It Builds, Ship It! Entropy Problems accumulate

14 Why Worry? If It Builds, Ship It! Entropy The FreeBSD Porter's Handbook 16,000 lines of DocBook SGML

15 Why Worry? If It Builds, Ship It! Entropy The FreeBSD Porter's Handbook 16,000 lines of DocBook SGML To fix whitespace required an 8,000-line commit.

16 Why Worry? If It Builds, Ship It! Entropy The FreeBSD Porter's Handbook 16,000 lines of DocBook SGML To fix whitespace required an 8,000-line commit. Followed by another 4,000-line commit.

17 What Can Be Done? Make things easier for writers! Especially for people who rarely work on documentation at all.

18 What Can Be Done? Make things easier for writers! Especially for people who rarely work on documentation at all. Encourage programmers to document their work

19 What Can Be Done? Make things easier for writers! Especially for people who rarely work on documentation at all. Encourage programmers to document their work Encourage users to improve the quality of documentation

20 What Can Be Done? Make things easier for writers! Especially for people who rarely work on documentation at all. Encourage programmers to document their work Encourage users to improve the quality of documentation Encourage writers to expand and clarify documentation

21 What Can Be Done? Automated Proofreading

22 What Can Be Done? Automated Proofreading Remember things Help those who suffer from CRS syndrome

23 What Can Be Done? Automated Proofreading Remember things Find errors Subtle errors, but also errors of inexperience

24 What Can Be Done? Automated Proofreading Remember things Find errors Help comply with standards Indirectly educate the user on standards

25 What Can Be Done? Automated Proofreading Remember things Find errors Help comply with standards Keep mistakes out of the tree

26 What Can Be Done? Automated Proofreading Remember things Find errors Help comply with standards Keep mistakes out of the tree Let the writer concentrate on the message!

27 What Tests Can Be Automated? Tests For All Files

28 What Tests Can Be Automated? Tests For All Files Spelling Use misspellings from FreeBSD text files, man pages, and DocBook source

29 What Tests Can Be Automated? Tests For All Files Spelling Repeated words Detect repeated words in a line or from one line to the next

30 What Tests Can Be Automated? Tests For All Files Spelling Repeated words Bad phrases The to, to for, again from actual FreeBSD files

31 What Tests Can Be Automated? Tests For All Files Writing Style you and your should obviously and needless to say simply and basically starting too many sentences with the e.g. and i.e. No examples! Great potential to improve readability and clarity

32 What Tests Can Be Automated? mdoc(7) Tests

33 What Tests Can Be Automated? mdoc(7) Tests Sentences begin on a new line Not enforced by the toolchain

34 What Tests Can Be Automated? mdoc(7) Tests Sentences begin on a new line Document date updated on non-trivial changes Easy to forget

35 What Tests Can Be Automated? mdoc(7) Tests Sentences begin on a new line Document date updated on non-trivial changes Structure: the eight minimum macros From the manual page template in mdoc(7)

36 What Tests Can Be Automated? mdoc(7) Tests Sentences begin on a new line Document date updated on non-trivial changes Structure: the eight minimum macros.dd.dt.os.sh NAME.Nm.Nd.Sh SYNOPSIS.Sh DESCRIPTION From the manual page template in mdoc(7)

37 What Tests Can Be Automated? DocBook SGML Tests Rules are described in the FreeBSD Documentation Project Primer

38 What Tests Can Be Automated? DocBook SGML Tests Whitespace Tabs versus spaces at the beginning of lines

39 What Tests Can Be Automated? DocBook SGML Tests Whitespace Indentation Indent level, matching open/close tags, lines wrap at 70 columns

40 What Tests Can Be Automated? DocBook SGML Tests Whitespace Indentation Tag usage style Tags like <programlisting> need special handling

41 What Tests Can Be Automated? DocBook SGML Tests Whitespace Indentation Tag usage style Title capitalization AP Style

42 igor, The Lab Assistant

43 igor, The Lab Assistant Must be easy and quick to use

44 igor, The Lab Assistant Must be easy and quick to use Auto-detect type of input file

45 igor, The Lab Assistant Must be easy and quick to use Auto-detect type of input file Handle multiple files and compressed files

46 igor, The Lab Assistant Must be easy and quick to use Auto-detect type of input file Handle multiple files and compressed files Test for conformance with the FDP Primer

47 igor, The Lab Assistant Must be easy and quick to use Auto-detect type of input file Handle multiple files and compressed files Test for conformance with the FDP Primer Be able to run one test or all

48 igor, The Lab Assistant Must be easy and quick to use Auto-detect type of input file Handle multiple files and compressed files Test for conformance with the FDP Primer Be able to run one test or all Avoid false positives

49 igor, The Lab Assistant Implementation

50 igor, The Lab Assistant Implementation Written in Perl, but whatever

51 igor, The Lab Assistant Implementation Written in Perl, but whatever It's regexes all the way down

52 igor, The Lab Assistant What does it look like?

53 igor, The Lab Assistant Checking man pages -D to skip comparing.dd with today's date

54 igor, The Lab Assistant Clarifying the output -R to produce ANSI color sequences

55 igor, The Lab Assistant Checking writing style -y for style tests

56 igor, The Lab Assistant Checking DocBook whitespace -Z for whitespace tests only

57 igor, The Lab Assistant Checking DocBook content -z for non-whitespace tests only

58 Where Is It? /usr/ports/textproc/igor

59 Lessons Learned

60 Lessons Learned Optimize regexes, short-circuit when possible

61 Lessons Learned Optimize regexes, short-circuit when possible DocBook SGML indentation is non-trivial

62 Lessons Learned Optimize regexes, short-circuit when possible DocBook SGML indentation is non-trivial Syntax highlighting is good for whitespace Example whitespace syntax highlighting for nano on the web site

63 Lessons Learned Optimize regexes, short-circuit when possible DocBook SGML indentation is non-trivial Syntax highlighting is good for whitespace Advertising

64 The Future

65 The Future Rewrite Better language, or style, or speed

66 The Future Rewrite Better DocBook indentation testing Something smarter, maybe even a full parser

67 The Future Rewrite Better DocBook indentation testing Advanced language analysis Analyze content rather than just words

68 The Future Rewrite Better DocBook indentation testing Advanced language analysis Other languages Possibly easier than it sounds

69 The End Special thanks to Glen Barber and Benedict Reuschling Thank you!