halef Documentation ETS

Size: px

Start display at page:

Download "halef Documentation ETS"

Bernard Shields
5 years ago
Views:

1 ETS Apr 02, 2018

3 Contents 1 OpenVXML Without Tears 1 2 Halef Setup Process 19 i

4 ii

5 CHAPTER 1 OpenVXML Without Tears 1 Authors Vikram Ramanarayanan and Eugene Tsuprun (with inputs from the OpenVXML Setup and User Guides) 1.1 Overview In this manual, you will learn how to: Install Eclipse and the OpenVXML plugin for Eclipse Create your OpenVXML first project a Hello world chatbot application Deploy the application Use halefbot (a text interface to Halef) to test out the application Use WebGGS to add logging and semantic categorization capabilities Prerequisites This tutorial assumes some familiarity with regular expressions, which we use to search for patterns in the users responses (see RegexOne tutorial.) 1.2 Installing Eclipse and OpenVXML Eclipse is an IDE (interactive development environment) that will allow you, among other things, to design call workflows compatible with Halef. 1 The absence of tears is not guaranteed. 1

6 1.2.1 Install Java Development Kit 8 JDK 8 is a prerequisite for Eclipse and OpenVXML. Download and install it Install Eclipse Download and install Eclipse. Do not use the latest version only this one Install the OpenVXML plug-in for Eclipse Download OpenVXML 5.1. Here is the compiled OpenVXML binary Unzip the file. Start Eclipse. From the menu, select Help Install New Software.... In the Install dialog that appears, in the Work with: field at the top, hit Add.... Hit Local... and select the unzipped directory: OpenVXML_ Make sure all the check boxes in the bottom part of the screen are unchecked (Show only the latest versions of available software, Group items by category, etc.) Select All packages. 2 Chapter 1. OpenVXML Without Tears 1

7 On the Review Licenses screen, there aren t actually any licenses to review. Just click I accept the terms... and then Finish. During the installation you ll need to click OK once, to allow unsigned content to be installed. When the installation is complete, restart Eclipse. Now that you have installed OpenVXML, change your perspective (the windows and tabs you see in Eclipse). From the menu, select Window Open Perspective Other... OpenVXML. 1.3 Creating a Hello World Project In this part of the tutorial, you will create and deploy a text-based chatbot that says Hello World Import the starter project The Starter war file helps you get started designing your chatbot quickly. 1. Download the Starter war file: Creating a Hello World Project 3

8 2. Open Eclipse. Choose a new workspace.(note: it is best practice to create a new workspace for each application you are working on.) 3. Open the OpenVXML perspective: Window Open Perspective Other... and choose OpenVXML. 4. File Import... Choose General -> Existing Projects into Workspace and click on Next. Then, select the button next to Select archive file:, hit Browse... and navigate to the location of the.war file you just downloaded. In the drop-down menu with file extensions (which is located in the lower right corner of the dialog box on Windows), choose *.*. Then choose the.war file that you would like to import and select Open. 5. Hit Finish in the window that appears. 6. You may or may not get an error message. If you do, just restart Eclipse Set up a text-based Hello World canvas Every OpenVXML project consists of two components: The Voice, which stores all your audio files (for speech applications) The Workflows, a visual representation of the chatbot application you want to design. In Project Explorer, expand Deploy_Workflow and then Workflow Design. Open up Main Canvas.canvas. As the name suggests, this is the canvas on which you will be drawing a flowchart-like representation of your chatbot. You should now see something like this: The Project Explorer (on the left) is where you should see the Voice and Workflow projects in your Eclipse workspace. The Design Area (in the middle) shows the call flow of the currently selected canvas. The Voice Pallet (on the right) displays the available blocks for the application. The Starter warfile only has a Begin block (denoting the start of the callflow) and the Submit block (marking where the call ends). Drag the block called PlayPrompt onto the canvas. PlayPrompt outputs something to your user either a sound file or some text. Let s have it say Hello world! : 4 Chapter 1. OpenVXML Without Tears 1

1. Double-click the PlayPrompt on your canvas 2. Under Media, click on Not Configured. 3. Press Add Entry. 4. In Content Type, choose Text. 5. In the text box, type Hello world! 6. Hit OK three times.

9 1. Double-click the PlayPrompt on your canvas 2. Under Media, click on Not Configured. 3. Press Add Entry. 4. In Content Type, choose Text. 5. In the text box, type Hello world! 6. Hit OK three times. Now that you ve created the PlayPrompt, let s make sure your application knows to play it when you start interacting with the chatbot. To do this, click on the arrow in the lower right corner of the Begin block and drag and drop it onto the PlayPrompt block. In the ensuing dialog box, select Continue and hit OK. Now, let s connect the PlayPrompt to the Submit block to mark the end of the application. Connect your PlayPrompt to Return the same way: click the little arrow on the PlayPrompt, drag it to Submit, and hit OK. Your canvas should now look like this: 1.3. Creating a Hello World Project 5

Save the user s response into the database on our server 3. Categorize the user s response into the semantic categories of yes or no 4.

10 1.4 Creating a Branching Application Let s now create a more complex callflow. In this section of the tutorial, you will build a text-based chatbot that will: 1. Ask the user if they like pizza 2. Save the user s response into the database on our server 3. Categorize the user s response into the semantic categories of yes or no 4. Follow up with an appropriate response ( Me too! I love pizza! or I m sorry to hear you don t like pizza. ) 5. Ask for clarification, if the original response was not understood Our callflow will look like this: 6 Chapter 1. OpenVXML Without Tears 1

11 1.4.1 Configure the question block In contrast to a PlayPrompt, which plays back a message for the user and expects no response, a Question block allows us to prompt the user for a response. In this case, we ll be asking if the user likes pizza. Drag a Question block onto your canvas and connect the Begin block to it. Double click it to edit: 1. Set User Input Style to S and leave the drop-down at Voice Only. (Do this first.) 2. Set the Name for the question block. This name is arbitrary and will just help you identify the block on your canvas. 3. Set a Variable Name for the variable that will store the response for our question. Our convention is to start the variable name with A_, for instance, A_do_you_like_pizza. 4. Double-click on Not Configured next to Prompt (in the Media tab). Press Add Entry. In the dialog box that appears, set Content Type to Text. Type Do you like pizza? in the text area. That s the text that will be shown to the test taker. Hit OK twice. 5. Double-click on Not Configured next to Voice Grammar. Choose Grammar File (from the dropdown menu) and then type ignore.wfst. This is the name of the language model Halef will be using when converting the user s speech input into text. Because we are building a text-based chatbot for now, we don t need to customize a language model. We do, however, need to specify a value here, because Halef expects one. Your question block should look like this: 1.4. Creating a Branching Application 7

1.4.2 Create a Script block to handle semantic classification Script blocks allow you to use the JavaScript language to manipulate variables, communicate with external services, and control the flow

12 1.4.2 Create a Script block to handle semantic classification Script blocks allow you to use the JavaScript language to manipulate variables, communicate with external services, and control the flow of the application. In this application, we will use the script block to classify the response to the question Do you like pizza? into one of two categories: yes or no. We will also send the user s response to a back-end service, which will then store it into a database. Fortunately, you don t need to know JavaScript to achieve the above goals. We ve created a tool called WebGGS that will write the script for you. Drag a Script block and connect the Question block to it. Now, open WebGGS: In WebGGs, define the variable name: A_do_you_like_pizza 8 Chapter 1. OpenVXML Without Tears 1

13 Add some regular expressions representing potential responses to the question Do you like pizza? and corresponding semantic categories. Your WebGGS box should look like this: Now, hit the right arrow to generate the script. You should the following output: This script will do the following: Log A_do_you_like_pizza (the variable containing the response to the question block) to the server If the response contains the strings yes or yeah, set the variable SC_do_you_like_pizza to equal to yes. If the response contains the string no, set the variable SC_do_you_like_pizza to equal to no Create a Branch block Drag a Branch block from the Voice pallet and connect the Script block to it. The Branch block is what allows to route the application in accordance with the semantic category of the response to Do you like pizza? We will build exit paths to deal with three types of responses: responses that fall into the category of yes (users who like pizza) responses that fall into the category of no (users who don t like pizza) responses which did not fall into either category (i.e., what our script failed to categorize) Open the Branch block and hit Add Branch. Exit Path Name is an arbitrary name, but we recommend keeping it consistent with the name of the semantic category. Let s first make a branch for the yes category. Set Exit Path Name to yes. The Expression is a JavaScript statement that should return true or false. If the statement is true, the call will be routed through this exit path. In this case, we will want to enter: Variables.SC_do_you_like_pizza == "yes" 1.4. Creating a Branching Application 9

This so-called Default path will be triggered if the JavaScript expressions for all the other semantic categories were computed to false. 1.4.

14 Now, make another exit path for the no category. The Exit Path Name should say no, and the Expression should read: Variables.SC_do_you_like_pizza == "no" Your Branch properties should now look like this: We do not need to define an Expression for the third exit path (neither yes nor no ). This so-called Default path will be triggered if the JavaScript expressions for all the other semantic categories were computed to false Add the PlayPrompts Depending on the category of the response, we will respond with a relevant PlayPrompt Me too! I love pizza!, I m sorry to hear you don t like pizza., or Sorry, I don t understand.. Create three new PlayPrompts that say this Add the connectors Draw an arrow from your Branch to Me too! I love pizza! In the ensuing dialog box, choose the yes exit path. Now, draw an arrow from the Branch to I m sorry to hear you don t like pizza. Choose the no exit path. Draw an arrow from the Branch to I don t understand and choose the Default exit path. If the system doesn t understand the response, let s ask the user to repeat it. To do so, connect the I don t understand PlayPrompt back to the Do you like pizza? question to create a loop. 10 Chapter 1. OpenVXML Without Tears 1

15 Finally, connect the remaining two play prompts to a Submit block to mark the end of the application Save and export your project We need to export the newly created project into a Web ARchive (WAR) application that can be served by the web server as VoiceXML and then read by Halef s Voice Browser. A voice browser browses voice/speech web pages (in the VoiceXML format) much like Firefox or Chrome browse HTML pages. 1. Save the project: File Save All 2. Go to File Export Under Voice Tools, choose Web Application 4. Select Archive file. 5. Choose where you d like to save the file. We recommend saving it in a git repository for better version control Test your application on halefbot halefbot is the text-based interface to Halef. Open up halefbot URL (ask your system administrator for the URL). The Start URL is [your war file name without.war ]/Deploy_Workflow/Begin. For instance, if you called your war file helloworld.war, the Start URL is helloworld/deploy_workflow/ Begin. Open your project in halefbot and try a few different responses: 1.4. Creating a Branching Application 11

You ll note that the response in Russian ( ) was not in our grammar, and so it correctly went to the Default exit path. The other two responses ( yeah of course I love pizza!

16 You ll note that the response in Russian ( ) was not in our grammar, and so it correctly went to the Default exit path. The other two responses ( yeah of course I love pizza! and not so much ) were successfully captured by our regular expressions. 1.5 Adding Counters In our pizza callflow, there is no set limit as to how many times the user will get to the I don t understand dialog state. As long as they keep saying something we don t have in our semantic categories, the conversation can go on indefinitely. We can use simple counter logic to set a limit. Let s do this: 1. Initialize a counter and set it to Each time we hit the Default exit path (the response does not fit into the yes or no semantic categories), increment the counter by If the counter is greater than 2, exit the application. Else, keep asking the user if they like pizza. 12 Chapter 1. OpenVXML Without Tears 1

17 We ll revise our callflow to look like this: First, let s initialize the counter. Open the Begin block and add a new variable called counter_do_you_like_pizza. Set Type to Decimal, and Value to 0. Note: we strongly recommend keeping your counter variable names exactly consistent with the prompt variable names (A_do_you_like_pizza in this case) Adding Counters 13

18 In the first version of this callflow, our Default exit path was connected directly to the I don t understand PlayPrompt. Now, we ll want to change this logic so that before we end up at I don t understand, we ll first make sure that the user hasn t already been there twice. Add another Script block to your canvas. If you d like, give it a name like increment counter. Set the Default exit path of the branch to go to this new script block. In your script block, add this JavaScript to increment the counter: Variables.counter_do_you_like_pizza = Variables.counter_do_you_like_pizza + 1; Now, after we run this code, we ll want to decide whether the counter is greater than 2. To help with this, let s use the Decision block from the Voice Pallet. The Decision block helps us formulate a single JavaScript expression that evaluates to true or false. Select counter_do_you_like_pizza on the left, Greater Than (>) as the comparison operator, then Expression of 2 on the right. Just like a Branch block, a Decision block supports multiple exit paths. Connect the true exit path to another PlayPrompt that says something like: Sorry you re having trouble. Please try again later. Connect the false exit path to our I don t understand PlayPrompt, which then is connected back to the original prompt. Let s save, deploy, and try this in halefbot: 14 Chapter 1. OpenVXML Without Tears 1

19 After the third time we hit the Default exit path, the counter was greater than 2, and so we got kicked out from the application. 1.6 Appendix A: Portals: Extending Workflows to Span Multiple Canvases Using multiple canvases is a great way to separate an application into more manageable pieces. To add a new design canvas to a Workflow (and configure a portal between the new and existing canvas): 1. Right-click on the Workflow Design folder for Deploy_Workflow in the Project Explorer and select: New Other Voice Tools Wizard -> Design Document. Hit Next. This will open the new design document wizard. Give the new canvas a name and hit Finish. 2. Enter a name for the new design canvas in the input box (for example, SecondCanvas. Note: This name must be unique amongst the existing design canvases in the application. 3. When you open up the new canvas, you ll see a Portal Exit block on the new canvas Appendix A: Portals: Extending Workflows to Span Multiple Canvases 15

7 Appendix B: Creating an OpenVXML project from scratch The easiest way to get started with an OpenVXML project is to follow the instructions above and import Starter.war, a starter project.

20 4. Let s set up a Portal Entry and connect it to the second canvas. Drag and drop Portal Entry from the Voice Pallet onto the original canvas. Click on the new block and choose the Portal Exit to connect it to. 1.7 Appendix B: Creating an OpenVXML project from scratch The easiest way to get started with an OpenVXML project is to follow the instructions above and import Starter.war, a starter project. But if you really want to create your project from scratch using the Eclipse plugin, here are the instructions. First, let s create a new voice project: File New Project Voice Tool Wizards Voice Call the voice project HelloWorld_Voice. You should now see the voice project in your Project Explorer pane: Now, create the interactive workflow: File New Project Voice Tool Wizards Interactive Workflow The application name for this and all Halef workflows should be Deploy_Workflow. Hit Next. In the Branding dialog box, leave Brands at Default. Hit Next. On the Interaction Type Support dialog box, leave Voice Interaction checked. Hit Next. On the Language Support dialog box, associate your workflow with your voice by clicking Not Configured and choosing HelloWorld_Voice: 16 Chapter 1. OpenVXML Without Tears 1

You have now created your voice and your workflow, and you have associated the two. You should see both HelloWorld_Voice and Deploy_Workflow in the Project Explorer pane. 1.

21 You have now created your voice and your workflow, and you have associated the two. You should see both HelloWorld_Voice and Deploy_Workflow in the Project Explorer pane. 1.8 Appendix C: Troubleshooting Symptom: A Question dialog state does not respond to speech input Try this: 1. Look in the Tomcat log to see if there was any error 2. If the log indicates instead that the last action was fetching the grammar file: a. Ensure that the grammar file in the war has Linux line endings: Export it, remove it from the war, check the line endings in a text editor, and reimport to the war. b. Ensure that the grammar element in the vxml shown in the Tomcat log has mode= voice and there is no grammar element with mode= dtmf. If the vxml has a problem: (a) Import the war into OpenVxml and open the Question node (b) Change Settings > User Input Style = Hybrid (c) If Media > Dtmf Grammar shows anything other than Not Configured, then double-click the displayed value and set Grammar File to an empty string. (d) If Media > Voice Grammar shows Not Configured, then double-click Not Configured, set the dropdown to Grammar File, and type the filename of the grammar file (including the file extension) into the textbox. (e) Change Settings > User Input Style = Voice Only with the S shown in blue and the D shown in gray Appendix C: Troubleshooting 17

22 (f) Save and then Export. 18 Chapter 1. OpenVXML Without Tears 1

23 CHAPTER 2 Halef Setup Process How to setup Kaldi Recognizer and Test Recognition Model How to setup Cairo Speech Server How to setup JvoiceXML Voice Browser Application 2.1 How to setup Kaldi Recognizer and Test Recognition Model 1. Make sure that you have latest Ubuntu updates sudo apt-get update sudo apt-get dist-upgrade sudo apt-get upgrade 2. Install libraries required for Kaldi recognizer sudo apt-get install zlib1g-dev sudo apt-get install libatlas3-base sudo apt-get install libwebsockets-dev sudo apt-get install libssl-dev 3. Create Kaldi directory, clone custom Kaldi repository and run build script mkdir -p /export/apps/kaldi/ cd /export/apps/kaldi/ git clone git://git.code.sf.net/p/halef/cassandra halef-cassandra cd /export/apps/kaldi/halef-cassandra bash./build_kaldi.sh 4. Copy recognition models and binaries to destination directories 19

24 mkdir -p /export/apps/kaldi/ /deployment/bin cd /export/apps/kaldi/ /deployment cp /export/apps/kaldi/ /halef-cassandra/kaldi-trunk/src/online2bin/strm-asr-server /export/apps/kaldi/ /deployment/bin chown -R <your_login>:<your_group> /export/apps/kaldi/ Copy recognition models and startup scripts cd /export/apps/kaldi/ /deployment/ tar xzvf models.tar.gz tar xzvf 9404.tar.gz 6. Start Kaldi instance cd /export/apps/kaldi/ /deployment/srv/9404./server.sh & 2.2 How to setup Cairo speech server To setup Cairo instances a convenient setup script has been created. 1. Download setup script and Kaldi settings files by opening the following links and clicking Download this file link on top of each page Copy downloaded SETUP.* files into /export/apps directory on the server and edit them. You will see direcory settings and database connection settings in the top part of SETUP.sh file which you need to correct. SETUP.kaldi.properties should contain at least 1 entry defining item name, server ip and port on which Kaldi is listening. For example: default=localhost: Make sure that you have ~2GB of free space in /export/apps and server can connect to the Internet for Cairo download. 4. To install Cairo with Java, please run setup in unattended mode:./setup.sh quiet Cairo installation can take considerable time depending on your Internet connection speed. 5. You can install and run Cairo as service by running CairoService script (please review CairoService file content first to check if all directories are correct there): cd /export/apps/cairo/cairo-vm/scripts sudo CairoService install sudo service cairo start 2.3 How to setup JvoiceXML Voice Browser Application 1. For JvoiceXML we recommend using latest version directly from JvoiceXML developer repository ( github.com/jvoicexml/jvoicexml) : 20 Chapter 2. Halef Setup Process

25 mkdirs -p /export/apps/jvxml cd /export/apps/jvxml git clone 2. Configure JvoiceXML by copying DB logging file from Cairo: cp /export/apps/cairo/cairo-vm/config/halef-logging-db.xml /export/apps/jvxml/jvoicexml/main/ and edit mrcpv2 settings to point to Cairo server by replacing IP for cairoaddress and cairohost in the file: /export/apps/jvxml/jvoicexml/org.jvoicexml.implementation.mrcpv2/build.gradle 3. To compile and run JvoiceXML you need Gradle version 3 or higher. Because system apt-get will install only version 2 it is recommended to download Gradle directly from: and symply unpack it into /export/apps 4. You can build and run JvoiceXML using the following commands (or put them into shell script): cd /export/apps/jvxml/jvoicexml/main/ export JAVA_HOME=/export/Apps/jdk export PATH=/export/Apps/jdk/bin/java:/export/Apps/gradle-3.4.1/bin:$PATH nohup gradle run 2.3. How to setup JvoiceXML Voice Browser Application 21

CSCI 201 Lab 1 Environment Setup

CSCI 201 Lab 1 Environment Setup "The journey of a thousand miles begins with one step." - Lao Tzu Introduction This lab document will go over the steps to install and set up Eclipse, which is a Java integrated