Introduction to Dragon Voice Automation

Transcription

1 Introduction to Dragon Copyright Alan Cantor Thursday 12 July :30-5:30 p.m. RESNA 2018 Arlington, VA Alan Cantor Cantor Access Inc This handout is available in alternative formats

2 Contents Overview 1 Objectives... 1 Timetable Voice Command Basics 2 Relevance... 2 Why custom voice commands?... 2 Prerequisite knowledge and skills... 2 Four modes of the Command Browser... 3 "MyCommands Editor" Built-in Commands 5 A technique for discovering built-in commands... 5 How to interpret commands in the Command Browser Auto-Text Macros 6 Overview... 6 Three Kinds of Auto-Text Macros Step-by-Step Macros 7 Overview... 7 Examples of Step-by-Step Macros Advanced Scripting 8 Overview... 8 Advanced Scripting format... 8 Essential Advanced Scripting commands... 9 AppBringUp...9 SendKeys...9 SendSystemKeys and SendDragonKeys...10 Wait...10 HeardWord...10 SetMousePosition R, x, y...11 RememberPoint and DragtoPoint...11 Debugging and testing aids List Commands 12 Overview When to create list commands Simple lists Complex lists Using two or more lists Open-ended commands Further reading 17 Books (PDF) On-line forums and resources Copyright Alan Cantor i Introduction to Dragon

3 Overview Objectives 1. Voice command basics: Describe three situations in which custom voice commands are necessary; understand two prerequisite skills; and explain all parts of the "Command Browser" and "MyCommands Editor" user interfaces. 2. Built-in commands: Know one technique for discovering built-in commands with the Command Browser. 3. Text and Graphics macros: See the development of three "Auto-Text" macros for inserting unformatted text, formatted text, and images. 4. Step-by-Step macros: See the development of three "Step-by-Step" macros for opening applications, waiting, inserting keystrokes, and typing text. 5. Advanced Scripting macros: See the development of three "Advanced Scripting" macros that use "AppBringUp," "SendKeys," "Wait," "SetMousePosition," and "HeardWord." 6. List and open-ended commands: See the development of three "list commands" and one open-ended command. Timetable Time Objective 1:30 [5] Welcome and introductions 1:35 [30] 1. Voice commands basics 2:05 [10] 2. Discovering built-in commands 2:15 [20] 3. Text and Graphics (Auto-Text) macros 2:35 [50] 4. Step-by-Step macros 3:25 [15] Break 3:40 [60] 5. Advanced Scripting macros 4:40 [45] 6. List and opened-ended commands 5:25 [5] Wrap-up and evaluation 5:30 End of session Copyright Alan Cantor Introduction to Dragon

4 Relevance 1. Voice Command Basics The more reliant an individual is on speech recognition, the more they benefit from custom commands. The ability to script voice commands is an essential skill for assistive technologists who provide Dragon support and training. Custom commands improve access to standard applications, and make it easier and in some cases, make it possible to control non-standard applications by voice. Why custom voice commands? Save time, energy, and frustration Perform tasks that are difficult, time-consuming, or impossible with built-in commands Automate repetitive processes Make voice access more reliable Make it easier to remember voice commands Sculpt commands to better fit the physical and cognitive needs of an individual Prerequisite knowledge and skills Good knowledge of built-in Dragon commands Ability to drive Windows from the keyboard Good understanding of the applications you want to voice-enable. Ability to discover hotkeys for the applications you want to voice-enable. Copyright Alan Cantor Introduction to Dragon

5 The "Command Browser" Four modes of the Command Browser Mode Commands organized by... Use to... Browse Context: e.g., "Microsoft Word 2016" Search for commands Script Manage Grammar: e.g., "Microsoft Word for Windows" Group: Usually user defined Create, edit, delete, and preview commands Import, export, delete, and change properties of commands MyCommands All groups Edit groups of commands Copyright Alan Cantor Introduction to Dragon

6 "MyCommands Editor" MyCommand Name Description Group Availability Command Type Name Editor... Script Copyright Alan Cantor Introduction to Dragon

7 2. Built-in Commands A technique for discovering built-in commands Comprehensive lists of all Dragon commands do not exist. There are too many commands hundreds of thousands, maybe millions. You may be able to guess some, but most commands must be discovered: 1. Open the "Command Browser" 2. Click "Browse." 3. Click "Context" and choose an application from the drop-down list. 4. Click "Keyword Filter." 5. Choose one (or more) keywords you think might be part of the command name. 6. Click "Done." 7. Choose a promising looking command in the right pane. 8. Choose a command in the right pane, and repeatedly click "Ellipsis" until "Ellipsis" grays out. (You might need to click many times.) 9. Click "Show All." This process is not intuitive. Trial-and-error experimentation will lead you to understand when (if ever) to click "Optional," "Or," and "Back to List." How to interpret commands in the Command Browser Many commands are cryptic-looking. For example: Clear [the] formats from ( ( that this it ) [the] selection... The ellipsis... means that part of the command name is cut off. Click "Ellipsis" to expand the view. Words in square brackets are optional. For example, you can say "Clear the formats..." or "Clear formats..." The vertical line means "OR." When different words appear between parentheses, choose one of the listed words. For example, you can say "Clear formats from that" OR "Clear formats from this" OR "Clear formats from it." The three are equivalent. Copyright Alan Cantor Introduction to Dragon

8 Overview 3. Auto-Text Macros Auto-Text macros are also called "Text and Graphics" macros. Auto-Text macros insert boilerplate text and/or graphics. You can insert three kinds of things: 1. Unformatted text 2. Formatted text 3. Bitmap images Three Kinds of Auto-Text Macros To insert... MyCommand Name What it inserts... Unformatted text My Name and Degrees S. Smith B.A., M.Sc., Ph.D. My Telephone Number Formatted text My Contact Info 123 Main Street Arlington Bitmap image Big Lion Copyright Alan Cantor Introduction to Dragon

9 Overview 4. Step-by-Step Macros This is the easiest scripting technique to: Send keystrokes and key sequences Access menus Open applications Wait for actions Control the microphone Examples of Step-by-Step Macros MyCommand Name What it does Double Underline Sends keystrokes in Word to double underline Get Rid of Stars Take a Letter Save in My Documents Deletes all asterisks in a document Opens Microsoft Word Saves a document in a particular folder Copyright Alan Cantor Introduction to Dragon

10 5. Advanced Scripting Overview "Advanced Scripting" is a technique for developing versatile and powerful commands. You do not need to be a programmer to take advantage of Advanced Scripting, as many tasks can be voice-enabled using only "HeardWord," "SendKeys," and "Wait." There are Advanced Scripting equivalents to the Step-by-Step commands introduced in the previous section. Advanced Scripting format All Advanced Scripting commands share this format: Sub Main Step 1 Step 2 etc. End Sub Sub Main means "the voice command start here." Step 1, Step 2, etc. are individual commands that make up the voice command. End Sub means "The voice command ends here." Copyright Alan Cantor Introduction to Dragon

11 Essential Advanced Scripting commands AppBringUp "AppBringUp" is similar to the Step-by-Step "Open (application)." The following command launches a calculator program: AppBringUp "c:\windows\system32\calc.exe" SendKeys To output keystrokes, use "SendKeys": SendKeys "Sandy S. Smith", True ' Type the name SendKeys "{Enter}", True ' Press Enter key SendKeys "123 Main Street", True ' Type the address "True" ensures all keystrokes are processed before proceeding to the next instruction. {Enter} is an example of a "reserved word." These "words" are equivalent to pressing certain keys. The meaning of most is obvious: SendKeys "{Backspace}" SendKeys "{Tab}" SendKeys "{Left}" SendKeys "{F1}" ' Press the "Backspace" key ' Press the "Tab" key ' Press the "Left arrow" key ' Press the "F1" key Other reserved words are less intuitive, but frequently needed: + = Shift ^ = Ctrl % = Alt Examples with Shift, Ctrl, and Alt: SendKeys "%f" SendKeys "^c" SendKeys "+{End}" ' Activate the file menu = Alt + F ' Copy to the clipboard = Ctrl + C ' Select to end of line = Shift + End To repeat a reserved word, place a repeat value before the closing brace: SendKeys "{Down 10}" ' Move cursor down 10 lines Copyright Alan Cantor Introduction to Dragon

12 SendSystemKeys and SendDragonKeys In some contexts, SendKeys outputs too quickly, or fails. There are two alternatives: SendSystemKeys and SendDragonKeys. It's hard to predict which of the three will work, so use trial-and-error. Reserved words are different than for SendKeys: Here are three ways to output Alt + F: SendKeys "%f", True SendSystemKeys "{Alt+f}" SendDragonKeys "{Alt+f}" Wait Macros execute quickly. Sometimes, you must slow down a script to allow time for a task to finish. To slow down a macro, insert Wait commands: AppBringUp "c:\windows\system32\calc.exe"' Launch the calculator Wait 1 ' Wait 1 second SendKeys "50" ' Type "50" HeardWord If you know a built-in Dragon command, mimic it with "HeardWord." For example: Built-in Dragon command "go to bottom" "select next four words" "click Save" "press tab" HeardWord equivalent HeardWord "go", "to", "bottom" HeardWord "select", "next", "four", "words" HeardWord "click", "Save" HeardWord "press", "tab" Notes about HeardWord: HeardWord is picky about case. You can usually discover the correct case by saying the command and watching the "Results Box" or checking the "Recognition History." Some HeardWord commands includes backslashes, hyphens, or underscores. The spellings change from version to version: Copyright Alan Cantor Introduction to Dragon

13 Built-in Dragon command "no caps that" "cap that" "all caps that" "go to sleep" HeardWord equivalent HeardWord "\No-Caps", "that" (Version 13) HeardWord "\no-caps", "that" (Version 15) HeardWord "\Cap", "that" (Version 13) HeardWord "\cap", "that" (Version 15)??? (Version 13) HeardWord "\all-caps", "that" (Version 15) HeardWord "go_to_sleep" (Version 13) HeardWord "go to sleep" (Version 15) SetMousePosition R, x, y Moves the mouse to coordinates (x, y) relative to an origin: R = 0 = Relative to the top left corner of the screen R = 1 = Relative to the top left corner of the active window R = 2 = Relative to the current pointer position R = 5 = Relative to the top left corner of the window interior SetMousePosition 0, 20, 30 SetMousePosition 1, 40, 50 SetMousePosition 2, 0, 1 SetMousePosition 5, 0, 0 ' Move pointer to (20, 30) relative to screen ' Move pointer to (40, 50) relative to window ' Move pointer one pixel down ' Move pointer to top-left of window interior ButtonClick B, n B = 1 = Left button B = 2 = Right button B = 4 = Middle button ButtonClick 1, 2 ButtonClick, 2, 1 n = 1 = Single click n = 2 = Double click ' Double left click ' Single right click RememberPoint and DragtoPoint Use these commands to simulate dragging the mouse. Debugging and testing aids Press F5 to run an entire script, and F8 to run a script one line at a time. Copyright Alan Cantor Introduction to Dragon

14 Overview 6. List Commands Lists make it possible to perform dozens (or millions!) of related actions with one command. "Auto-Text" and "Advanced Scripting" both support list commands. When to create list commands When a group of commands do almost exactly the same thing, and The command names vary by one or two (and maybe more) words. Here are three good candidates for list commands: What you say... Project Number 1 Project Number 2 Project Number 99 etc. What the command outputs... P P P John's phone number Mary's phone number extension 1234 My phone number etc. Section 1 Paragraph 40 s(1) p 40 Section 9 Paragraph 99 s(9) p 99 Section 22 Paragraph 2 s(22) p 2 etc. Copyright Alan Cantor Introduction to Dragon

15 Simple lists A simple list consists of words or phrases that Dragon recognizes easily. For example, <acne-stage> is a simple list with three items: mild moderate severe MyCommand Name: Acne <acne-stage> When a command contains one simple list, the variable associated with it is ListVar1. Sub Main SendKeys "The patient presented with a " SendKeys ListVar1 ' ListVar1 = mild, moderate, or severe SendKeys " case of acne on both sides of the face." End Sub or: Sub Main Let Stage = ListVar1 SendKeys "The patient presented with a " SendKeys Stage ' mild, moderate, or severe SendKeys " case of acne on both sides of the face." End Sub or: Sub Main Let Stage = ListVar1 SendKeys "The patient presented with a " & Stage & _ " case of acne on both sides of the face." End Sub Copyright Alan Cantor Introduction to Dragon

16 Complex lists A complex list consists of words (or phrases) that Dragon CANNOT easily recognize, and are hard to pronounce and/or remember. Each item consists of a written form and an easy-to-pronounce spoken form. The two parts are separated by a backslash. For example, the list <parts> consists of four items, one per line. Each line includes a hard-to-dictate part number, a backslash (which serves as a separator), and an easy-tosay number. A-1-A\1 B-2\2 C-3000\3 D-4-D\4 When a command contains a complex list, the variable associated with it is always ListVar1. The command must include instructions for extracting the written form. You do not need to understand the code to use it! Mid(ListVar1, 1, Instr(ListVar1, "\")-1) MyCommand Name: Part Number <parts> Sub Main SendKeys Mid(ListVar1, 1, Instr(ListVar1, "\")-1) End Sub or: Sub Main Let x = Mid(ListVar1, 1, Instr(ListVar1, "\")-1) SendKeys x End Sub For example, when you say "Part Number 3," Dragon will output "C-3000." Copyright Alan Cantor Introduction to Dragon

17 Using two or more lists You can include several lists in a command. This example outputs dates in this format: 01_JAN_2018. The command uses two complex lists and one simple list: <DD> <MMM> <YYYY > 01\1 02\ \31 JAN\January FEB\February... DEC\December Simple or complex? Simple or complex? Simple or complex? MyCommand Name: Insert <DD><MMM><YYYY> The variable associated with list <DD> is ListVar1 The variable associated with list <MMM> is ListVar2 The variable associated with list <YYYY> is ListVar3 Sub Main SendKeys Mid(ListVar1, 1, Instr(ListVar1, "\")-1) ' Insert day (complex list) SendKeys "_" SendKeys Mid(ListVar2, 1, Instr(ListVar2, "\")-1) ' Insert month (complex list) SendKeys "_" SendKeys ListVar3 ' Insert year (simple list) End Sub Copyright Alan Cantor Introduction to Dragon

18 Open-ended commands Beginning with NaturallySpeaking 13, you can end a custom command with a list called <dictation>, which is any word or phrase in Dragon's vocabulary. MyCommand Name: Save File As <dictation> Sub Main SendKeys "^f" Wait 0.5 SendKeys "a" Wait 0.5 SendKeys ListVar1 ' SendKeys "{Enter}" End Sub ' Press Alt + F to activate "File" menu ' Press A to activate "Save As" ' Insert dictated name for simple list <dictation> ' Save the file (commented out here) Copyright Alan Cantor Introduction to Dragon

19 Further reading Books (PDF) "Scripting for Dragon NaturallySpeaking 11" by Larry V. Allen: On-line forums and resources Keyboard Access FAQ (Alan Cantor): Keyboard Shortcuts (Microsoft): KnowBrainer Forums: Nuance Dragon Scripting Quick Reference: Copyright Alan Cantor Introduction to Dragon