CBC/Web Tutorial and Example

Transcription

1 CBC/Web Tutorial and Example Introduction This tutorial leads you step-by-step through a simple CBC study, from thinking about a marketing problem, planning a questionnaire and creating a new study in SSI Web, to entering a list of the attributes and levels, generating the questionnaire and analyzing the results. SSI Web is a powerful program, and this tutorial only touches on the basics. You ll become aware of different or more sophisticated approaches by reading other sections in this manual, by viewing on-line help and by experimenting with SSI Web. We also suggest the following two articles, available within our Technical Papers Library at : CBC Technical Paper and Getting the Most from CBC. (Note: this example and the data presented are purely fictional. The suggested steps represent one approach to this type of marketing problem, but may not necessarily reflect the single best way to conduct such a research study using the SSI Web system.) The Marketing Problem You have been hired as a consultant to a little-known startup company called Performance Plus. The brilliant design engineers at Performance Plus have developed a golf ball that flies farther than average balls. The proposed name for the ball is Long Shot. Because Performance Plus is not well known among golfers, your client has considered approaching a well-known maker of golf clubs and balls (Golfers, Inc.) to market the ball using their existing brand name for balls: Eclipse. Sales for Eclipse have been declining, and they welcome the opportunity to benefit from the new technology. A line extension brand called Eclipse+ has been proposed. However, the royalty Golfers, Inc. is proposing seems high. Two other manufacturers already market high performance balls: High-Flyer Pro (by Smith and Forester), and Magnum Force (by Durango). High performance balls command a price premium of between 20% to 50% over traditional balls, and the market is growing rapidly. One day between rounds of golf with the president of Performance Plus (you gracefully lost), you jointly draft up the following questions on the back of your score card. 1) How well could Long Shot by Performance Plus hope to compete with existing competitors Smith and Forester and Durango in the high performance ball market? 2) Should Performance Plus form an alliance with Golfers, Inc.? How much incremental value does the Golfers, Inc. brand name Eclipse add? 3) How do golfers trade off performance vs. price for high performance balls? 4) How should Long Shot be manufactured and positioned in terms of performance and price to maximize profitability? 1

2 5) Do novice or experienced golfers differ in their preferences for high performance golf balls? Could this lead to a target market strategy? Defining a List of Attributes and Levels One of the first steps in using any Conjoint technique is to define the problem in terms of attributes and levels. Back at the office, you review the points written on the back of the score card, and you write down the following: Brand High-Flyer Pro, by Smith and Forester Magnum Force, by Durango Eclipse+, by Golfers, Inc. Long Shot, by Performance Plus Performance Drives 5 yards farther than the average ball Drives 10 yards farther than the average ball Drives 15 yards farther than the average ball Price $4.99 for package of 3 balls $6.99 for package of 3 balls $8.99 for package of 3 balls $10.99 for package of 3 balls Each of these levels is mutually exclusive; golf balls are described using a single level from each attribute. The range of the attributes is based on technical specifications that Performance Plus has given you. As for prices, your research suggests that the two competitors are currently selling packages of three balls for between $7.99 to $ You select a wider range to cover the price levels you are interested in, but don t consider any prices below $4.99, having determined that such a low price cannot cover the manufacturing and marketing costs and result in a profit. A Study Plan and Sample Scripted Questionnaire Now that you have formulated some attributes and levels, you begin to think about the rest of the questionnaire, and plan a design and field strategy. You decide to invite golfing enthusiasts (by ) to visit a site on the web and take a CBC survey. You ve purchased an opt-in list of addresses from a widely-read golfing magazine. Each of the participants who complete the survey will be entered into a drawing for a free set of golf clubs from a leading manufacturer. You expect a completion rate of between 15% to 25%, resulting in a final sample size of between 300 to 500. You have scripted the following introductory text screens and questions in your word processing package: 2

3 Introduction: Thank you for your willingness to participate in this study. You are part of a special group of golfers we ve selected to ask for feedback regarding golf equipment. By completing this survey, you ll be eligible for a drawing to receive a free set of PrimoDriver clubs. (Click the Next button to continue) Frequency: First, we d like to ask you about how often you golf. Would you say that you golf... 5 times or fewer per year from 6 to 25 times per year from 26 to 50 times per year more than 50 times per year Skill: We'd like to get a feel for your skill level. Over the last year, what was your best score on a par-72, 18-hole course? 120 strokes or more Between 100 and 120 strokes Between 85 to 100 strokes Above par, but lower than 85 At or below par (72 or lower) 3

4 Venue: Do you usually golf at... Public courses Private courses IntroCBC: In the next section, we ll be asking you specifically about high performance golf balls. These balls are premium balls specially engineered to travel farther than the typical ball with good accuracy. We d like you to imagine that you are considering purchasing golf balls for your next golf outing. We ll show you some different high performance golf balls, and ask which one you would purchase. Some of the golf balls you are going to see are not currently available on the market, but we d like you to imagine that they were available today. It is important that you answer in the way you would if you were actually buying golf balls. If you wouldn t purchase any of the balls we ll show you, you can indicate that by choosing None. By choosing none, you indicate that you d buy another brand, or would continue using existing balls in your golf bag. At this point in the questionnaire, you plan to ask the Choice-Based Conjoint questions (tasks). You want to measure preference for the different levels of the attributes and their impact on choice for high performance golf balls. To accomplish this, you decide to specify 15 randomized CBC tasks (below is an example of one such task). The features will combine freely to form up to 48 (4 x 3 x 4) different product offerings. Even though there are only 48 total possible product concepts in this relatively small design, there are thousands of different ways to display multiple product concepts at a time in choice sets. Choice Task (one of 15 randomly constructed tasks): If you were considering buying golf balls for your next outing and these were the only alternatives, which would you choose? High-Flyer Pro, by Smith and Forester Drives 10 yards farther than the average ball Magnum Force, by Durango Drives 15 yards farther than the average ball Eclipse+, by Golfers, Inc. Drives 5 yards farther than the average ball None: I Wouldn t Purchase Any of These $10.99 for package of 3 balls $6.99 for package of 3 balls $8.99 for package of 3 balls 4

5 The term random tasks is easily misunderstood. Some people take it to mean that the composition of the questions is randomly (haphazardly) selected. However, the questions are carefully chosen according to design principles of balance and independence (orthogonality). We call the tasks random because respondents are randomly selected to receive one of many carefully constructed, unique versions of the CBC questionnaire. In addition to the random choice tasks, you also plan to ask a few fixed holdout tasks that will be constant across all respondents. You plan to do this for three main reasons: 1) By doing some research on the Internet and at local golf stores, you have learned the performance specifications and average prices for the two existing balls on the market (High-Flyer Pro and Magnum Force). Your client has also given you a good feel for the product specifications they think could match up well with these two brands. Therefore, you and your client can imagine a specific potential product scenario you d like to directly assess. 2) Your client has no experience with conjoint methods and seems a bit uncertain regarding the reliability and accuracy of the market simulator. Demonstrating that the market simulator can accurately predict responses to some holdout fixed scenarios should boost his confidence in the method. 3) In addition to logit, you are considering using an advanced technique such as Latent Class analysis and Hierarchical Bayes (HB) to analyze the results. You d like to have a couple of holdout observations by which to compare the predictive accuracy of alternative simulation models you might develop. These fixed tasks will not be constructed randomly, but are to be displayed exactly with the specific levels you use to specify them. These two tasks will be asked as the 7th and 9th tasks. In total, you ll ask 15 random + 2 fixed = 17 tasks. In the first fixed holdout, you plan to show the relevant competition versus your client s offering with your client s brand name. In the second fixed task, you plan to show that same relevant competition versus the Eclipse+, by Golfers, Inc. Here are those two fixed holdout concepts: Fixed Holdout Task #1 (7th overall choice task): If you were considering buying golf balls for your next outing and these were the only alternatives, which would you choose? High-Flyer Pro, by Smith and Forester Drives 15 yards farther than the average ball Long Shot, by Performance Plus Drives 10 yards farther than the average ball Magnum Force, by Durango Drives 5 yards farther than the average ball None: I Wouldn t Purchase Any of These $10.99 for package of 3 balls $6.99 for package of 3 balls $8.99 for package of 3 balls 5

6 Fixed Holdout Task #2 (9th overall choice task): If you were considering buying golf balls for your next outing and these were the only alternatives, which would you choose? Magnum Force, by Durango Drives 5 yards farther than the average ball Eclipse+, by Golfers, Inc. Drives 10 yards farther than the average ball High-Flyer Pro, by Smith and Forester Drives 15 yards farther than the average ball None: I Wouldn t Purchase Any of These $8.99 for package of 3 balls $6.99 for package of 3 balls $10.99 for package of 3 balls Notice that your client s offering is in the center position for both tasks, but the position of its competitors is rotated. The two fixed holdout tasks are separated by a random choice task so that respondents will generally not recognize that it is a repeated task with just one small difference (the brand name attached to your client s ball). Lastly, you decide to ask about gender and income. These might prove useful as a respondent filter in analysis, for bench-marking versus future waves of research, or for future target/database marketing efforts. Gender: Are you.. Male Female Refused Income: This is the last question in the survey. What is your total household income for the last calendar year, before taxes? Less than $30,000 Between $30,000 and $60,000 Between $60,000 and $100,000 More than $100,000 Refused 6

7 Signoff: That completes our survey. Thank you very much for your input. Now that you have developed a list of attributes and levels, and scripted a rough-draft of your questionnaire and the design of the choice tasks, you are ready to create the study and compose the questionnaire within the SSI Web system. Using SSI Web to Write the Questionnaire Start by opening the Sawtooth Software SSI Web program. If you have a standard installation, you start it by clicking Start Programs Sawtooth Software Sawtooth Software SSI Web. SSI stands for Sawtooth Software Inc. and Web conveys the idea that this software is for developing Web-based surveys (although SSI Web can also be used for standalone computer interviewing with its CAPI interviewing module.) CBC/Web is one component within that software program. The next step is to open a new study, and define a study name. After starting up SSI Web, choose File New Study. The New SSI Web Study dialog appears, with your cursor active in the studyname field. The Location field lists the folder (directory) that SSI Web currently is using to store studies. You can use any folder you like for this tutorial project. In general we suggest placing each new SSI Web study you create in a separate study folder. You can browse to and create new folders by clicking the button to the right of the Location field, and then (after browsing to the folder in which you want to create a new sub-folder to contain this study) by clicking the new folder icon. Each studyname in SSI Web has a maximum number of four characters (either letters or numbers). For this tutorial, you might create a name such as cbc1. From the New SSI Web Study dialog, specify cbc1 as the studyname, and make sure that the Survey Type field indicates CBC and CiW. Click the OK button. You are returned to the main menu, and a new Study Navigator window is displayed along the left-hand side of the screen. 7

8 As you work with this study, items appear in the Study Navigator window, listing the functional areas you have been using or that are now available to you. This list of items provides a Shortcut link to access those parts of SSI Web. Alternatively, you can access those same areas by clicking icon buttons on the toolbar or by using the pull-down menus. Entering the List of Attributes and Levels When composing conjoint analysis studies, one typically begins by specifying the list of attributes and levels in the software. To enter the list of attributes and levels you developed, click the A button (A for Attributes) on the toolbar, or choose Compose Conjoint Settings Add/Edit Attributes. The Specify Attributes and Levels dialog opens. To add the first attribute, click the Add button (at the bottom left of the Attributes panel). Your cursor is now active in the Attribute 1 Name field. 8

9 Type in the first attribute name: Brand. The attribute name is a label that you ll use for organizing information within SSI Web. Click OK to accept this information and close the dialog. Now that at least one attribute name is in the list, the Add button under the Levels panel becomes active. Also note that the Brand attribute is highlighted in the Attributes panel. With the Brand attribute highlighted in the Attributes panel, click the Add button under the Levels panel to add levels within the Brand attribute. The Level Text dialog is displayed. Type High- Flyer Pro, by Smith and Forester. To add the next level of Brand, press the ENTER key twice. Type in the next level: Magnum Force, by Durango. Repeat the process for the remaining two levels of brand. When you are ready to add the next attribute (Performance), click the Add button under the left Attributes panel, type the attribute label, and click OK to place that new attribute on the attribute list. With that new attribute highlighted on the attributes list, click the Add button under the Levels panel to add the three levels of that attribute. Follow the same pattern for the last attribute, Price. For your convenience, we repeat the full list of attributes below. Note that you can copy-and-paste attribute level text from this document (and other text documents) into the text fields within SSI Web. After highlighting the words to copy with your mouse, use the shortcuts Ctrl-C to copy, and Ctrl-V to paste into the desired field. Brand High-Flyer Pro, by Smith and Forester Magnum Force, by Durango Eclipse+, by Golfers, Inc. Long Shot, by Performance Plus Performance Drives 5 yards farther than the average ball Drives 10 yards farther than the average ball Drives 15 yards farther than the average ball Price $4.99 for package of 3 balls $6.99 for package of 3 balls $8.99 for package of 3 balls $10.99 for package of 3 balls To make changes to existing attributes or levels, select the attribute or level to edit by clicking the attribute or level in the Attribute or Levels lists within the Specify Attributes and Levels 9

10 dialog and by clicking the corresponding Edit button. Then, edit the item in the Level or Attribute text fields. When you are finished entering the list of attributes and levels, close the Specify Attributes and Levels dialog by clicking OK. Additional Study Parameters After you have specified your list of attributes and levels, you can specify other study parameters to govern your CBC questionnaire. Click Compose Conjoint Settings CBC Settings (or click the icon) and a message appears, stating that if you continue, CBC questions will be added to your survey. Click OK to accept and close the warning box. The Specify CBC Interview Parameters dialog is shown: Depending on your license, you will see various tabs for this dialog (if you have the Advanced Design Module, the tabs are available as shown). On the General tab, you can specify how many Random (experimentally designed) choice tasks and how many fixed (user specified) tasks you want. For this golf ball study, we want 15 random and 2 fixed tasks. Change the Number of Random Choice Tasks field to 15. As planned, the questionnaire shows three products on the screen, plus a None, for a total of four columns in the task (each available choice alternative can be fit into a column ). So, the default settings for Number of Concepts per Choice Task and Number of Columns are already correct and need no modification. Recall that we want to include a None option in this CBC questionnaire. This gives respondents the option to indicate that they would choose none of the products displayed on the screen. The Include None Option box is checked by default, but the text is not quite correct. Our planned questionnaire shows the following None text: None: I Wouldn t Purchase Any of These 10

11 You can directly edit the Text for None Option field to change this, or you can click the Pencil icon on the toolbar to access a larger edit dialog. Click the Format tab, to review the following settings: The Alternating Concept Colors are used in alternating concepts in the choice task to help make the choice task more readable, and provide a visual indicator from one choice task to another that the question has changed (sometimes choice tasks can look so similar that respondents might not notice from, say, the first task to the next that anything is different.) The Format Options on the right-hand side of this dialog control how large the choice tasks will appear on the respondent s monitor, how much white space is included between concepts and attribute levels, and whether an additional separation is added between concepts. To preview how the CBC task will look when it runs on the respondent s computer using Windows Explorer, click the Preview button. The question is displayed in Preview mode (no data are saved, and no special error checking or validation is performed for the question). 11

12 You ll note that SSI Web has used default heading text for the CBC question (and it isn t quite what we need for this questionnaire). In the next section, we ll show you the dialog for modifying that text. Click OK to close the preview window. At this point, you might try modifying some of the settings on the General and Format windows. After each change you make, you can click Preview to see the effect of the changes. Generating the Experimental Design In CBC studies, we use a carefully chosen experimental design plan. The experiment involves observing how respondents react to different golf ball designs. The design reflects the attribute combinations that make up the golf balls, and how these combinations are placed into choice sets. Ideally, each respondent would receive a unique questionnaire version, with different combinations of golf balls arranged within sets in unique ways. There are only a finite number of possible combinations, so there potentially can be some identical choice tasks across respondents, but the idea is to improve measurement of the effects of the attribute levels (including reducing order and learning effects) by ensuring a high degree of variability in the choice tasks across individuals. CBC/Web lets you generate up to 999 unique versions of the questionnaire to be uploaded to the web server. By pre-specifying the design plans in a file residing on the server, we can simply assign each new respondent to the next subsequent design version within the file, which places minimal demands on the web server. Even if you have more than 999 respondents, 12

13 once a reasonably large number of designs are allocated across respondents, the statistical gains of using even more design versions are very minimal. The default in the software is to use 300 design versions. The additional gains in precision for going beyond that are probably negligible. If you aren t already at the Specify CBC Interview Parameters dialog, go there by clicking Compose Conjoint Settings CBC Settings. Click the Design tab, and the following is displayed: Many of the features of this dialog are beyond the scope of this tutorial. You can read additional details by pressing F1. We ll cover some of the basics here. Under Experimental Design Creation Settings, we can specify the Random Task Generation Method. This indicates the strategy that CBC/Web uses to generate the random choice tasks in our study. Complete Enumeration is the default strategy, and it should work well for our study. Number of Questionnaire Versions refers to how many unique versions of the CBC questions that SSI Web will save into a file. When respondents enter the questionnaire, they are assigned the next subsequent questionnaire version. Once a respondent is assigned the 300 th questionnaire, the next respondent begins again with version #1, and so forth. Randomize Attribute Position within Concepts lets you randomize the order in which the attributes appear in the choice task. We specified the attribute list in the order Brand, 13

14 Performance and Price. If we use the default (which is not to randomize the order), Brand will always appear in the top and Price always at the bottom. This seems like a natural presentation order for this study, so we ll retain the Do not Randomize Attribute Order setting. Concept Sorting within Tasks controls how the concepts are arranged within the choice task. By default, the order of presentation is randomized. But, if we wanted the first brand always to appear in the first concept position, etc., we could specify to sort concepts based on the Natural Order for Brand. You can investigate details regarding Concept Sorting by clicking F1. For this tutorial, we ll retain the defaults. Once we are comfortable with our settings, we click Generate / Test Design. The following report is displayed: Design Efficiency for Main Effects Copyright Sawtooth Software A Priori Estimates of Standard Errors for Attribute Levels Task Generation Method: Complete Enumeration Design seed: 1 Number of versions: 300 Total Choice Tasks: 4500 Att/Lev Freq. Actual Ideal Effic (this level has been deleted) High-Flyer Pro, by Smith and Forester Magnum Force, by Durango Eclipse+, by Golfers, Inc Long Shot, by Performance Plus (this level has been deleted) Drives 5 yards farther than Drives 10 yards farther than Drives 15 yards farther than (this level has been deleted) $4.99 for package of 3 balls $6.99 for package of 3 balls $8.99 for package of 3 balls $10.99 for package of 3 balls The report indicates that 300 versions of the CBC questionnaire were created. We know that each version of the questionnaire had 15 random choice tasks, so a total of 300 x 15 = 4500 choice tasks were generated. Some of the details of the report are beyond the scope of the unit, but you can obtain more information within the Help text. The Freq. column indicates how many times each level is represented in the design. Within each attribute, you can see that CBC has perfectly balanced the presentation of levels. The Effic. Column indicates the relative efficiency of the design. Design efficiency runs from the worst (0.0) to the best (1.0), and we can see that this particular design is quite efficient with respect to main effects. If you include prohibitions in your design, sometimes designs can be quite inefficient. In some cases, you will receive a warning regarding that the design is deficient or will notice asterisks rather than data in the test design report. You should re-evaluate your design setup should this occur. Failure to correct the problem can result in unusable data. Click Close to close the test design report, and click OK to close the Specify CBC Interview Parameters dialog. 14

15 Let s now examine the questionnaire we ve built to this point. Click Compose Write Questionnaire or click the pencil icon displayed: on the toolbar, and the following dialog is You ll note that SSI Web has added 15 Random (CBCRAN1 through CBCRAN15) and 2 Fixed CBC tasks (CBCFIX1, CBCFIX2) to the survey. These questions are based on your settings and the default template within SSI Web. The CBCFIX tasks aren t yet in the proper positions (we wanted them in the 7 th and 9 th task positions in the survey). You also haven t yet specified the attribute combinations used for your user-defined Fixed tasks, so SSI Web has inserted the questions with default levels that we ll modify shortly. Before we fine tune those CBC questions, let s turn our attention to the issue of assigning passwords for respondents to enter at the opening screen of an SSI Web survey. Assigning Passwords and the Password Screen Placing a survey on the web makes it convenient for a geographically dispersed population to take surveys. However, the danger is that the survey may become available to people that have not been invited to take the survey. Also, some respondents might try to take the survey multiple times. Assigning respondent passwords is a way to deal with both of these issues. Password assignment is beyond the scope of this tutorial, so to make things simple we ll assume that no passwords are to be used. To turn off passwords for this project, click Compose Passwords Set Passwords and in the upper-right hand side of the Enter Passwords dialog, un-check both the User Names and Passwords boxes. SSI Web will ask you to confirm that you want to delete User Names and Passwords from your study. Finally, you ll be asked to specify the maximum number of respondents for this study. Since we plan to interview about 300 to 450 people in this example, 15

16 let s specify a comfortable limit such as After you have typed 1000 click Save and Close. Specifying Explanatory Text and Select Questions (If you are already familiar with adding Text/HTML Filler and Select questions within SSI Web surveys, you may choose to skip forward to the section entitled Fine-Tuning the CBC Questions ). Let s enter the introductory text and initial questions using the Write Questionnaire dialog. You can access the Write Questionnaire dialog by selecting Compose Write Questionnaire or by clicking the pencil icon on the toolbar. The introductory text and survey questions would most likely be initially developed within a word processing document. Assuming you really had such a document, you might use the Ctrl-C to copy, and the Ctrl-V shortcuts to paste the information into SSI Web. We suggest you simply copy-and-paste the text within this document into your SSI Web questions rather than re-type the text (if viewing this document with Acrobat Reader, you can use the text select icon from the Acrobat Reader toolbar.) The first question to compose is the introductory screen: Introduction: Thank you for your willingness to participate in this study. You are part of a special group of golfers we ve selected to ask for feedback regarding golf equipment. By completing this survey, you ll be eligible for a drawing to receive a free set of PrimoDriver clubs. (Click the Next button to continue) In the terminology of SSI Web, this could be a Text/HTML Filler question (it requires no specific response). But, every SSI Web questionnaire begins with an introductory <Password> question, even though there may not be passwords used in every project. The <Password> question is always the first question in the List of Questions. To edit the <Password> question (which now serves as our text-only introduction question), highlight <Password> on the List of Questions and click the Edit button. The Password Settings dialog is displayed: 16

17 Most question types in SSI Web have a Header 1, Header 2 and Footer section. These are text areas in which you can insert any text (or HTML, if you know HTML). When the question is viewed with a web browser, the sections are organized roughly as follows: For this introductory screen (that includes three paragraphs of information), it seems to make sense to place the first paragraph in the Header 1 area, the second paragraph in the Header 2 area, and the third paragraph in the Footer area. Type (or cut and paste) the following text for the Introduction question into the text 17

18 areas in the <Password> question. Put the first paragraph in Header 1, the second paragraph in Header 2, and the third paragraph in Footer. Thank you for your willingness to participate in this study. You are part of a special group of golfers we ve selected to ask for feedback regarding golf equipment. By completing this survey, you ll be eligible for a drawing to receive a free set of PrimoDriver clubs. (Click the Next button to continue) The main dialog only shows a portion of the text, but you can click the pencil icon adjacent to each text area to expand the text editing window. To preview how the question will look when it runs on the respondent s computer using Windows Explorer, click the Preview button. The question is displayed in Preview mode (no data are saved, and no special error checking or validation is performed for the question). SSI Web automatically places paragraph breaks (extra blank lines) between the Header 1, Header 2, and Footer sections. If you put all the text in a single section, you may see that when the web browser interprets the text, it all runs together without any blank lines between the paragraphs (unless you insert some HTML instructions to force blank lines between paragraphs). We ll talk about using HTML within your text to take greater control over the layout, font, and style later in this unit. Click the X icon in the upper-right hand portion of the Preview window to close that window and return to the previous SSI Web dialog. After viewing the text in preview mode, you might decide that it is really too small, or that you want to make the text bold. You can change the size and styles (bold, italic, underline) or the text justification for the three major text sections by clicking the controls on the Password Settings dialog. After changing any setting, click Preview again. 18

19 After you are happy with the layout of the introductory screen, click OK to return to the Write Questionnaire dialog. Now you are ready to specify the first Select-Type question: Frequency: First, we d like to ask you about how often you golf. Would you say that you golf... 5 times or fewer per year from 6 to 25 times per year from 26 to 50 times per year more than 50 times per year First, make sure that you highlight the <Password> question on the List of Questions (when you add a new question, it is placed directly below the highlighted question on the list). At the Write Questionnaire dialog, click Add, choose Select as the question type (click CiW, and then choose Select from the available drop-down list), and type Frequency for the question name. Click OK and the Select Question dialog is shown. Place the heading text for the question in the Header 1 field. To specify the response options, click the Response Options button. From the Specify Response Options dialog, use the Add button to add the four response options for this question. When you are finished, the question should look something like: Now that you have seen how to specify Single Select question types in SSI Web, you have the tools you need to specify the remaining four select-type questions for the golf ball questionnaire (Skill, Venue, Gender and Income please refer to the questionnaire text for these questions as presented near the front of this unit). 19

20 Remember that when you add a new question to the List of Questions, it is inserted directly below the question that was highlighted when you clicked the Add button. If you need to move a question once it has been added to the List of Questions, simply highlight the question to be moved and click the or buttons to move the questions within the list (you can also highlight a question and click Ctrl-X or Ctrl-C to cut or copy and Ctrl-V to paste questions within the current questionnaire, or even to another SSI Web study.) Formatting Text in SSI Web Questions In addition to the Select questions we added to the questionnaire previously, there are two text-only, instructional screens to add. Referring to the questionnaire we outlined earlier, these questions are named IntroCBC and Signoff. Even though the respondents aren t asked to provide specific inputs to these, we refer to these as questions in the sense that we add them to the questionnaire as if they were standard questions, and they are listed separately by their question name in the List of Questions. You may recall that we placed the very first introduction screen within the <Password> question. Following that were Select-type questions Skill and Venue. After the Venue question follows the text-only instructions: IntroCBC: In the next section, we ll be asking you specifically about high performance golf balls. These balls are premium balls specially engineered to travel farther than the typical ball with good accuracy. We d like you to imagine that you are considering purchasing golf balls for your next golf outing. We ll show you some different high performance golf balls, and ask which one you would purchase. Some of the golf balls you are going to see are not currently available on the market, but we d like you to imagine that they were available today. It is important that you answer in the way you would if you were actually buying golf balls. If you wouldn t purchase any of the balls we ll show you, you can indicate that by choosing None. By choosing none, you indicate that you d buy another brand, or would continue using existing balls in your golf bag. This text layout is a bit more challenging than we dealt with before (when we put three paragraphs of introductory text into the <Password> question). There are more than three separate paragraphs here, and we ll need to deal with the additional element of bolding selected text. Add a Text/HTML Filler question directly following the Venue question in the questionnaire. To do so, highlight the Venue question, click Add, and specify the Question Name as IntroCBC and the question type as Text/HTML Filler. 20

21 The Text/HTML Filler dialog is displayed: If the text-only instruction question was the last question in our survey (or if it were one of many possible ending points in a survey), we would indicate it was a terminating question by checking the Terminating Question box. This instructional section is an introduction to the CBC questions, so we ll not check that box. Earlier in this tutorial, when we used the <Password> question to format the opening page of the survey, we placed each paragraph of text in a separate Header 1, Header 2, or Footer section. We saw that SSI Web automatically places blank lines between text in these sections. However, with the text in the IntroCBC question, there are many more paragraph breaks. We ll take the opportunity here to introduce the concept of using a few simple HTML instructions within our survey text. HTML stands for HyperText Markup Language and provides simple ways for you to enhance the look of your surveys, such as by bolding or underlining words, or making paragraph breaks within text. Browsers know how to interpret HTML instructions when displaying the results to the screen. If you know HTML, you can use it whenever you wish within SSI Web surveys to accomplish your aims. If you don t know HTML, it isn t very difficult to learn a few HTML tricks, or you can use the toolbar in SSI Web available within the editor that appears when you click Pencil icons in SSI Web: Either type or cut-and-paste the text for the IntroCBC question into the Header 1 text 21

22 field (Any of the other three fields could be used also, though the Body field is indented). Click the Pencil icon to display the larger text editing window. Initially, the text appears something like this: Even though it appears that there are extra blank lines between the paragraphs, web browsers ignore these hard returns (as well as more than one consecutive space characters), so you ll need to provide HTML instructions to insert these paragraph breaks (hard return plus blank line). HTML instructions are placed within <> brackets, called tags. For example, the HTML instruction to create a paragraph break begins with an open paragraph tag written as <p> and optionally ends with a close paragraph tag written as </p>. The text to be formatted as a separate paragraph is enclosed within these tags. You can either directly type HTML within your document, or you can highlight text to be modified and click the icons on the HTML toolbar in the editor. If you highlight the first paragraph with your mouse: In the next section, we ll be asking you specifically about high performance golf balls. These balls are premium balls specially engineered to travel farther than the typical ball with good accuracy. And then (with the text in the first paragraph highlighted), click the Paragraph icon on the toolbar. This inserts a <p> prior to the text and a </p> after the text: 22

23 <p> In the next section, we ll be asking you specifically about high performance golf balls. These balls are premium balls specially engineered to travel farther than the typical ball with good accuracy.</p> When the browser interprets this text, it doesn t display the tags but instead separates the text enclosed within the tags as a separate paragraph. Repeat the same for each of the paragraphs in the IntroCBC question. Next, we need to bold certain words in the text. The HTML tags for bolding text are <b></b> (with the text to be bolded placed between the open and close bold tags). You can either directly type these tags within the document, or highlight the text to be bolded with the mouse and click the Bold icon on the toolbar. After you finish separating the paragraphs with <p></p> tags and bolding the appropriate text with <b></b> tags, it should look something like: Click OK to return to the Text/HTML Filler dialog and then Preview to see how the web browser displays this question. It should look like: 23

24 You should now have the tools necessary to add the other Text/Only HTML Filler question (Signoff) at the end of the survey. Make sure when adding the Signoff question at the end of the survey to mark this Text/HTML Filler question as a Terminating Question. To make it a terminating question, edit the question from the Write Questionnaire dialog and check the Terminating Question checkbox. SSI Web may warn you at this point that a terminating question must stand alone on its own page in the survey. This leads us into our next discussion on page layout. Page Layout and Global Settings It may be helpful at this point to review how SSI Web breaks the survey into separate pages, and some basic global settings that affect the look and functionality of your SSI Web questionnaire. From the Write Questionnaire dialog, click the Layout button. 24

25 The Layout dialog shows how the various questions we ve specified (or that SSI Web has automatically added to the list) are arranged across different pages. Notice that the Select questions (Frequency, Skill and Venue) and IntroCBC we ve added to the questionnaire are all currently arranged on page 2. Let s assume we wanted to break these up, one question per page. You set a new page break after the Frequency question by highlighting Skill on the list and clicking the Set Page Break button. Skill is now placed on page 3. Repeat this operation to place Venue and IntroCBC on separate pages (pages 4 and 5). Make sure to place the final text screen Signoff on its own (last) page. Click OK to return to the Write Questionnaire dialog. While we had been setting page breaks for our questionnaire, you may have noted that the two fixed CBC tasks (CBCFIX1 and CBCFIX2) are not yet in the 7 th and 9 th CBC task positions. You can move a question on the list by highlighting the question in the List of Questions and clicking the or buttons to move the question to another point in the questionnaire. Move CBCFIX1 directly after CBCRAN6 and move CBCFIX2 directly after CBCRAN7. 25

26 Specifying Fixed Tasks A fixed choice task looks just like typical Random (experimentally designed) tasks. However, rather than let the design algorithm determine the combination of attribute levels to be shown for each respondent, you specify the codes for the levels to be displayed in each fixed product concept. Recall that the purpose of specifying fixed holdout tasks for this example is to achieve a controlled and direct measure of the existing competitors offerings versus our client s proposed offerings. You ll also be able to use the results to check the ability of the market simulator you develop using the 15 random choice tasks to predict the responses to the 2 fixed holdout tasks. Recall that our list of questions already includes two fixed tasks named CBCFIX1 and CBCFIX2. Let s preview the first fixed task, CBCFIX1. From the Write Questionnaire dialog, edit CBCFIX1. From the CBC Random and Fixed Questions dialog, click Preview. You ll notice that the CBCFIX1 question looks exactly like the other CBC choice tasks we ve previewed to this point, however the same attribute levels (level 1) are used across all three product concepts. This is an initial starting point that SSI Web creates, which you need to modify for each fixed choice task in your study. Click OK to return to the CBC Random and Fixed Questions dialog. To modify the levels displayed in the fixed choice task, click the Fixed Task Designs button. The following is displayed: From this one dialog, you can specify the attribute combinations to be used in each of your fixed tasks. To change from one fixed task to another, use the Fixed Choice Task drop-down box at the upper-left hand corner of the dialog. We ll first modify CBCFIX1. 26

27 Recall that the first fixed holdout was to display the following three products: High-Flyer Pro, by Smith and Forester Drives 15 yards farther than the average ball $10.99 for package of 3 balls Long Shot, by Performance Plus Drives 10 yards farther than the average ball $6.99 for package of 3 balls Magnum Force, by Durango Drives 5 yards farther than the average ball $8.99 for package of 3 balls Use the drop-down controls provided to specify the three product concepts as shown. Click the OK button when finished. Preview the fixed holdout task to make sure it looks as you expect. Make any necessary changes. Next, you need to modify the second fixed holdout task. From the CBC Fixed Choice Task Settings, used the Fixed Choice Task drop-down control to select CBCFIX2. Modify CBCFIX2 to have the appropriate specifications for our second fixed choice task: Magnum Force, by Durango Drives 5 yards farther than the average ball $8.99 for package of 3 balls Eclipse+, by Golfers, Inc. Drives 10 yards farther than the average ball $6.99 for package of 3 balls High-Flyer Pro, by Smith and Forester Drives 15 yards farther than the average ball $10.99 for package of 3 balls Again, preview this task to make sure it looks as expected. Fine-Tuning the CBC Questions After you have finished these steps, your questionnaire should be nearly complete, though you haven t yet customized the text to display directly above each choice task. At this point, we suggest you view your survey by clicking Preview Survey under the Compose Write Questionnaire area, or by editing/previewing each question one-byone. Depending on how picky you are about the look of your survey, you could spend a good deal of time adjusting each question and adding HTML elements to achieve just the right look. Let s turn to editing the CBC tasks. Under Compose Write Questionnaire, highlight one of the CBC choice tasks (perhaps CBCRAN1) and click Edit. The following dialog is displayed: 27

28 You can modify the text place directly above the CBC task in the Header field. Note that the changes you make on this dialog apply universally to all CBC choice tasks (Random plus Fixed tasks). Therefore, if you want to somehow customize the text shown directly above or below the CBC tasks on a task-by-task basis, you may wish to delete the text from the Header or Footer field and instead add a Text/HTML Filler question directly before or after each choice task. Note that you can control the Size and Style for the Header and the Level Text used in the CBC task from this dialog. Click Preview to see the effect of any of your changes. (The in the Footer field is an HTML-special string that indicates to place a space character in the Footer field, which is needed for some browsers to add the default space beneath the choice task. You can change or delete this if you like. Click Preview to see the effect.) Testing Your Questionnaire To test run your questionnaire, we use a tool within SSI Web called Local Web Server. Local Web Server uses Apache Server technology to make your computer act like a web server, showing the questionnaire and saving the respondents answers to a data file. To test run your survey, click Field Local Web Server (or click the globe icon ). The Local Web Server dialog is displayed: 28

29 We suggest you use the default Survey Subdirectory Name (your four-character study name) for this example. When you click Upload Survey, SSI Web copies the files needed to run the survey into a local folder on your hard drive where the Local Web Server can use them. A message appears stating that the upload is finished. It is not necessary for you to know the details of where this folder is or how Local Web Server works, but if you are interested, you can investigate details in the online Help. Whenever you make a change to your survey setup and want to test run the survey, you ll need to make sure to click Upload Survey so that the new survey project is copied into the subfolder used by Local Web Server. If you don t do this, you ll end up test running the old survey, prior to your changes. Whenever you click Upload Survey, any data that you have entered in your test surveys are deleted (after an initial warning message). Click Run Survey, and a browser window opens with your questionnaire. When you are finished taking the survey, you can close that window by clicking the in the upperright hand corner of the browser window. If you took the survey under Local Web Server mode, you may have noted that there are pre-programmed error messages that are displayed if, for example, respondents forget to answer a question on a page, or type a letter into a numeric field. You can access and edit these error messages from the Global Settings dialog. Warning: We do not suggest you use Local Web Server for collecting real respondent data, as there is potential for data loss should someone accidentally click Upload Survey. There is a free software application for running SSI Web in CAPI mode (stand alone computer interviewing) called SSI Web CAPI. We suggest using this instead if you need to conduct real interviews locally (rather than on a Web server). After you have edited the survey to the point that you are pleased with the look, content, and functionality of your survey, you should examine the test data you ve collected using Local Web Server (this process is described in a separate tutorial called Getting Started with SSI Web: A 45-minute Hands-On Tour available on the Help menu), then at least 29

30 examine the CBC data using Counting analysis (described later). The results should generally reflect your preferences. If they don t, this suggests something may be wrong with your study setup. After you are convinced that the survey is functioning properly, you should also pretest your survey among your colleagues. They can give you feedback regarding the usability of the survey, and you can examine the resulting Counts data or perhaps even the part worths (assuming you have enough data) to make sure the data at least have face validity. Pretesting and Fielding Let s now return to the marketing problem and story we began at the beginning of this document. After you have tested the survey in Local Web mode, you post the survey to the Web using an account set up for you by an ISP (Internet Service Provider). (Setting up your survey on the web is beyond the scope of this unit, but is described in detail in the SSI Web help documentation. Fielding options also include hosting on your own company s server, or using Sawtooth Software s hosting services). The next day, you send an to your client, with a link to take the survey. The president of Performance Plus first takes a survey and suggests some minor wording changes. After you make those changes, you invite six other individuals at Performance Plus take the survey. You download the results and analyze the resulting data. Due to the small sample size, the results are a bit noisy (and there is an obvious bias toward your client s balls), but the data seem to feel right. Next, you recruit six golf enthusiasts within your city to come to a central site to take the survey. You watch silently as they take the survey. At the end of each session, you ask each respondent follow-up questions to ensure that there weren t any parts that were difficult to understand or just didn t make sense. You ask them specifically about the choice tasks, making sure that there wasn t too much information on the screen at once, and that they didn t feel overwhelmed with the task. After debriefing the test respondents and analyzing their data to ensure that the results looked reasonable, you make a few small adjustments and proceed to field. Using a bulk program, you send an to the list of 2000 golfing enthusiasts from the opt-in list mentioned at the beginning of this unit. After a few weeks, and a reminder sent to those who had not yet completed the survey, you achieve 300 total completes. The completed data reside on the ISP s server where you uploaded the SSI Web survey. The details for viewing and downloading the data from the server are provided in other areas of the SSI Web documentation and are beyond the scope of this CBC/Web unit. Even so, it is useful to at least describe the steps you would perform to access and download the data from the Web, estimate the part worth utilities, and begin analysis using the market simulator. 30