Module Evaluation Enterprise Survey Tool

Transcription

1 Module Evaluation Enterprise Survey Tool Enterprise Surveys Page 1

2 Before you begin YOU MUST CONTACT THE LTT AND ASK FOR A MODULE EVALUATION ACCOUNT PROVIDE A COMPLETE LIST OF COURSES THAT YOU WISH TO EVALUATE, GIVING THE CORRESPONDING CODES SPEAK TO YOUR LEARNING TECHNOLOGIST VIA THE SERVICE DESK itservicedesk@durham.ac.uk ARTS AND HUMANITIES Judith Jurowska j.e.jurowska@durham.ac.uk SOCIAL SCIENCE AND HEALTH Julie Mulvey julie.mulvey@durham.ac.uk SCIENCE AND EDUCATION Elaine Tan elaine.tan@durham.ac.uk Enterprise Surveys Page 2

3 Before you begin YOU SHOULD KNOW The Enterprise Survey tool is a new Blackboard tool which exists within duo and replaces the old and outdated Module Evaluation tool. The benefits of this tool are that the Surveys are quick and easy to set up Comparisons of Surveys, ie year on year can be achieved Surveys can be deployed to external people via uploading addresses Automatic Reminders can be sent out to only those still to complete Students do not have to be logged in to complete a survey thus helping to increase Return Rates The LTT are working to produce an alternative report that works around many of the tools issues The next upgrade to this tool is due Summer 2014 The LTT are working together with Blackboard to enhance the Enterprise Survey tool TO GET STARTED Contact your Learning Technologist to be given the role of Evaluation Manager without this you cannot access the tool. itservicedesk@durham.ac.uk They will then ask you for further details to set you up in the system. BEFORE YOU BEGIN Consider your Survey before you begin and ask yourself the following questions 1. Are you wanting to use the SAME survey (identical no questions about individual staff members) for all your modules? 2. Are you wanting to use a unique survey for each module? Enterprise Surveys Page 3

4 TIPS TO HELP YOU 1. We recommend that you use Matrix questions rather than multiple answer questions as this question type is not working in the desired manner. 2. Required questions are not enforced at this time, however, recipients are not aware of this and it doesn t seem to be hindering completion 3. The reports do not offer a mean score, instead they use a percentage. Please note probation is achieved if there is a response of 65%. A tool to perform a conversion is available for download on the LTT help blog. 4. Printable Reports with Comments do not display the question text asked when option no comparison need instead choose option response periods 5. Questions in the survey are shown in the order created. If the questions have been reordered whilst editing this will not be reflected in the final report. 6. There is no automatic numbering applied to questions and we recommend you add your own. 7. We do not recommend ticking Start section on new page. This is not enforced if you wish to copy the survey for multiple evaluation use. Instead select Add Page Break and then Add Section break. 8. Return Rates are shown individually as numbers within the Response Period. The LTT are currently working on producing a solution. 9. Do not use an apostrophe in the Survey Name 10. The No beside the checkbox for Include Not Applicable option should be ignored. Ticking this box indicates that this option should be included. 11. When creating Matrix questions, row headings actually means column headings 12. When using multiple choice questions, the Add Open Entry Field option is currently not working as expected 13. When using long text answers, no notification is displayed to respondents that they have exceeded the 1000 character limit. Answers are instead truncated upon submission to meet this imposed limit. We suggest that you a warning to this effect in the question text. 14. Module evaluations can only be deployed to Courses not to Organisations. To deploy to an organisation please use the Upload Addresses option. 15. When scheduling surveys, currently the options to deploy Later and Now are not functioning correctly. Instead we recommend that surveys are deployed upon creation. If you wish to save a draft before deploying, do so by clicking Save and Exit at the Notify stage. 16. Releasing of Survey Results for Instructors is currently not functioning. 17. There is currently no method of distinguishing the owner of a survey. We suggest that you put the name of the author in the description. Enterprise Surveys Page 4

5 1 CREATING A SURVEY 1. Setting it up 2. Headers and Footers 3. Creating Instructions 4. Question Headings and Pages 5. Question types 2 SENDING SURVEYS 1. Finalising your survey 2. Selecting Respondents 3. Setting deadlines 3 SURVEY RESULTS 1. Closing Surveys 2. Generating Results Enterprise Surveys Page 5

6 1 Setting up a new Survey Enterprise Surveys Page 6

7 1.1 Setting up a survey On the front page of duo, locate the Tools panel and select the Enterprise Surveys link. TIP - If you do not see this link please contact your designated Faculty Learning Technologist, to be granted access to this tool. Click on Create Survey. Give your survey a unique Survey Name such as Module Code and provide a description. TIP Do not put an apostrophe in the Name of your Survey TIP Please type your name in the Description (Created by Fred Bloggs) TIP We do not recommend ticking Do not allow backtracking as this means that respondents cannot click to go back to previous pages. When completed click on the Submit button and a blank survey will be created. Note you get a green Success bar at the top to indicate the survey has been created. Enterprise Surveys Page 7

8 Please look at the buttons across the top of the screen when the Survey Canvas opens. These are: Create Question, Create Visual Element and across to the right of the screen Preview. Using the Preview button is very useful to see what your survey looks like as you build it. 1.2 Headers and Footers To edit the header of your survey click on the downwards arrow next to Edit Header. You can edit the text displayed at the top of each page of your survey here. Similarly with the footer click the downwards arrow and click Add to alter the text displayed at the foot of every page of your survey. Enterprise Surveys Page 8

9 1.2 Adding instructions To add instructions or Department blurb about the surveys, click on the Create Visual Element dropdown and choose Instructions from the list. Fill this in in the same way you would any content item and hit Submit. Your instructions will appear as a block at the top of your survey. Click on the Preview button located at the top right of the menu bar to see what it looks like Instructions Survey Title Enterprise Surveys Page 9

10 1.3 Creating Section Headings and Pages To put your questions under various headings, use a Section Break. Click on Create Visual Element and choose Section Break from the list. Put in the title of the heading and choose the styling of how you wish this to be displayed. You ll see a preview of this underneath. TIP Please ignore Start Section on a new page as this currently does not work on copied surveys. If you want a section to start on a new page choose to add a Page Break and then add a Section Break Your section break will be added as a block to the survey. If you wish to see how this will be displayed, click the Preview button. Enterprise Surveys Page 10

11 1.4 Question Types Add your questions to the survey by clicking on Create Question drop down and selecting from the following list. Either/Or (One of two options- you can choose from a list or create your own). Likert Scale (You can determine the scale and weighting Note for Percentages/Means you must put in a point value). Matrix (MCQ or Likert Questions displayed in a tabular format) Multiple Choice (one answer from a list of choices) Option to turn a MCQ into a Multiple Answer question but we do not recommend this at the moment. Open Entry (Short text 255 characters, Long text 1000 characters.) NOTE - There is an option to align each question to Course Goals this is not being used at this moment in time. Either/Or Enter your question text Select Answer Choices If you require a scale add Points. Click Submit. Your question will then be included in your survey. Enterprise Surveys Page 11

12 Likert Scale Choose the scale that you wish to apply TIP - To include a Not Application option to your question - tick the box (ignore the word No) If none of the above choices is appropriate, choose Create New Click Next and create your own scale. Enter your question text. If you have selected a scale these answers will be entered as choices. If you clicked create new enter your desired answers on the scale. To add more options click on Add Answer Choice and add further options. Use the Order drop downs to reorder delivery of choices. IMPORTANT! - To do any calculations you must apply points to each Answer choice. Your question will now be added to the survey. TIP Check that the Point Values show and that they are correct Enterprise Surveys Page 12

13 Matrix Questions Select a scale for your Matrix (grid) question which will go across to top of your table. Again if none of the scales listed are suitable you can create your own by clicking Create New. TIP - To include a Not Application option to your question - tick the box (ignore the word No) TIP Please note that Row Labels actually means Column Labels Your question will be included in your survey. IMPORTANT! Remember to Add Point Values if you need Means or Percentages. Enterprise Surveys Page 13

14 Multiple Choice Questions A MCQ allows for one choice to be chosen from a list of possible answers. Enter the answer options in the scale Tip Ignore Add Open Entry field as this currently does not work Your question will be added to the survey. Multiple Answer questions TIP Please do not use the Multiple Answer question type as this is currently not working as expected. As an alternative use the Matrix Question Type See example below Create a new Matrix question and select Create New from the Select Scale options Click Next. Again ticking the Include Not Applicable Option will include this as a selection. Enter your question text and use the rows and columns to create a multiple answer question that is displayed vertically. Set your Row (Column) labels to None. The question will be displayed as follows. Enterprise Surveys Page 14

15 Open-Entry Questions TIP In the question text please put There is a 1000 character limit because at the moment it does not warn students that there is a limit and only records up to 1000 characters TIP Open Text questions should be an optional however by default this option is ticked making it a required question please resolve this by removing the tick from the Required check box! Choose whether you wish to have a Short (255 characters) or Large Text (1000 characters) entry field. Click Submit. This question will be added to your survey. Depending on the length displayed they will appear as large or small boxes to the Respondents. Enterprise Surveys Page 15

16 2 Sending your Survey Enterprise Surveys Page 16

17 2.1 Finalising your Survey When you are ready to send your survey, click on the Create Response Period this is located in the bottom right hand corner of the page. Or alternatively select Response Periods from the drop down next to the Survey name. Please note that we you do next is to go through a tabbed step by step process that will create a Response Period (aka Deployment period) for your survey. On the first tab Detailed Summary Create a name for your response Period. IMPORTANT Make the Response Period name the same as the Module Code and Name Click Save and Continue Enterprise Surveys Page 17

18 On the second tab Select Recipients Click on Choose Places. Select Specific Places and then click on the Find Courses to select courses/modules to apply the survey to. The courses listed will be those that you have had associated to you as an Evaluation Manager. TIP If you can t see any courses listed change the search criteria to Not Blank Choose the course you wish to send the Survey to and click on Submit. IMPORTANT! Do not select Multiple Course Codes! If the Survey is unique then one survey = 1 response period If the Survey is identical then one survey = multiple response periods TIP Currently Surveys cannot be deployed to Organisations if you come across this use the Upload Addresses option. All you need is a.csv file with one column with all the addresses in it. This option can also be used for surveys running outside of the University Enterprise Surveys Page 18

19 By default the course role is set to Student but note you can change this to include different roles for different kinds of surveys Choose the roles that you wish to send the evaluation to and click Calculate. Once this calculation has been done it will show you the number of users with this role (Approximate Recipients) that it has found. TIP Clicking on Calculate will help you to mentally equate the number of Students listed especially if you have added additional people as students who are not real students Click on Next Third Tab Notify On the third tab called Notify you will be able to choose how you wish to send a notification to your respondents. By default if you choose a Course Code then Course Notification and My Blackboard are set to run by default, but is also an option. TIP - The benefit of the option is that it will a link to the students and when they click on this link it will open the survey and does not require a login to duo and therefore helps to increase Return Rates Course notification will create a bar/banner that appears on the top of the corresponding duo course displayed every time the user logs in prompting them to complete the survey it only disappears when they have completed the survey. My Blackboard will create an alert notification to complete the survey but will be displayed alongside all other Blackboard alerts. prompts to complete the survey will be sent to the selected users via they do not have to log into duo to complete the survey. If you select the option you will have some additional settings to choose from. The system knows who you are so will put your address in for bounced notifications. You can alter this to a generic address if you wish by simply replacing the address. Enterprise Surveys Page 19

20 You can alter the Automated System Response messages displayed to users when they submit a survey, attempt to take a survey a second time or attempt to complete a survey after it has been closed. Clicking Edit and changing the text. The date displayed in the closed message is only a sample. The dates for survey deployment are set in the next of step of survey deployment - Scheduling. If your Survey is unique then the following will be fine the s take the Name of the Survey BUT If your survey is identical then we recommend you EDIT each of the above responses and change [ INSTRUMENT.NAME ] with [ DEPLOYMENT_NAME ] To do this click on Edit and highlight what you want removing then click on the Drop Down arrow next to each box and select Response Period Name. Change this throughout the s. You can then COPY the Response Period for the next Survey and edit it by updating the Response Period Name and select the Correct module. Enterprise Surveys Page 20

21 Fourth Tab - Scheduling Under Scheduling you choose the dates on which you wish to release and close your survey. Under Send Survey our recommended method is not to choose Now or Later as both of these options currently have problems. Instead select Automatically Send on Date and choose a date/time that you want the surveys to run from. Under Close Survey - please note that closed surveys cannot be reopened. We, therefore, recommend that you leave the closing date set to Never and close the surveys manually when you are happy with the Return Rate. You can also set reminder s to be sent out to respondents. These reminder s will only be sent if they have not already completed the survey. Click Add Reminder button in the Survey Reminders section. Remember that if you are using an identical survey and setting multiple Response Periods to edit the reminders to show Response Period Name instead of Survey Name Enterprise Surveys Page 21

22 To add a Reminder Select Choose Specific Date and select a date and time that you want the first Reminder to be sent out on. Then tick the Repeat Reminder Every box and put the number 1 in the box and set the drop down to Weeks. This will then send a reminder out every week on the same day of the week and time. Alter the text if desired. Alter [_INSTRUMENT.NAME_] to show [_DEPLOYMENT.NAME_] as this will insert the Name of the Response Period (Module Code and Module Name) into the sent. Enterprise Surveys Page 22

23 Under section 3 you have Release Survey Results. TIP This does not currently work so please ignore. Once this is working it will put a banner at the top of the course it is associated with that only Instructors can see and they will be able to view the results from a date/time designated by you. This option releases the results to Instructors in a Course only and you can select a date/time from when you wish the results to be shown. When the date/time arrives a banner will be displayed at the top of the selected course which is only visible to instructors. The results are never shown to students. Enterprise Surveys Page 23

24 3 Survey Results Enterprise Surveys Page 24

25 3.1 Return Rates & Closing Surveys In order to close the surveys you need to be able to see the Return Rates. TIP - There can be up to a 24 hour delay in returning the results to you a suggestion is to close the surveys at the end of the day and return the next morning to view the results. The Return Rates are currently shown as numbers if you access the Response Period for that survey. It will say 50/100 but does not show a percentage. The LTT have a new work in progress tool that will allow you as an Evaluation Manager to view a list of all the surveys you currently have live showing the Return Rates both in numbers and Percentages. Until we have the tool ready for you to use yourself please send an to itservicedesk@durham.ac.uk and ask for your designated Learning Technologist to run the report off for you and this to you. This report will help you to determine when to close the Surveys. The University currently has a recommended Return Rate of 50% - although Departments /Schools can, and do, specify their own desired Return Rate. To view the results - click on the number in the Response Groups. The response periods will be displayed. Click on Close Survey so that no further results are accepted and the status will be now shown as Closed. Enterprise Surveys Page 25

26 3.2 Generating Results Go back to the Enterprise Surveys page by clicking OK at the bottom right of the screen. Click on the Analyse Results button for the survey you need to analyse. To view the results tick ONE response period and then under Compare Response By select Response Periods option and then click the View Results button. TIP Under Compare Responses By change selection from No Comparison Need to the second option in the list - Response Periods. The results of your survey will be displayed. Enterprise Surveys Page 26

27 Printing Click on the Print Report button and you get two options from the drop down list. Printable View and Printable View with Comments. We recommend you select Printable View with Comments as the first option has problems. Please note that the report is produced in a PDF which you cannot edit. You can then either send this PDF to a printer or Save as an archive. 3.3 Analysing Results If your department uses means and needs to calculate these from the likert scales. A tool to convert the figures displayed from percentages to means is available for download on the LTT blog. For Probationary requirements it is noted that staff need a mean of 3.5 this simply equates to 65% in the final Reports. For access to this tool please contact the LTT via the service desk itservicedesk@durham.ac.uk Enterprise Surveys Page 27

28 3.4 Exporting Results as Excel Files On your list of evaluations, when a survey is closed, click on the drop down menu and select Export Submission Data. Choose whether you want this downloaded as an Excel (tab delimited) or a Text (comma delimited) file. Click Download. Save the file with an appropriate name, ie the Module Code which includes the year code. Sharing Files with Instructors To share any of these Reports with other Staff you can either add to a Shared Area or alternatively go into the Module itself for that survey and add a Hidden Area this will only be available to Instructors and upload the file. Enterprise Surveys Page 28