Guidelines for On- line Data E ntry and Downloading Impact of the Global Financial and Economic Crisis on Education in Selected Developing Countries (DFID RIVAF) UNESCO, Division for Planning and Development of Education Systems, Section for Sector Policy Advice and ICT in Education (ED/PDE/PAD) Introduction: UNESCO ED/PDE/PAD developed an electronic data entry interface to facilitate country s survey data entry and to generate survey databases comparable across-countries, which could be aggregated into a multi-country database. We used Qualtrics, an on-line survey provider, to design and put the three surveys on-line. 1 By entering the survey responses on-line, Qualtrics will generate a database (or spreadsheet) to be downloaded and used by the country teams for the visualization and analysis of survey findings. This guideline provides instructions for getting started with the data entry in Qualtrics and for viewing and downloading the survey data. Basically, country teams will use Qualtrics for the following research activities: 1. Enter the survey responses into the on-line survey, and 2. View and download the survey responses. 1. Enter the survey responses on-line 1.1. General instructions (for the three surveys) 1. Accessing the survey pages for your country: UNESCO ED/PDE/PAD has created a specific Qualtrics account for each Country Team. Your account shall be accessed by clicking on the three links provided by email, one for each survey. The link takes you directly to the first data entry page of the survey, which corresponds to the first page of the paper survey. The first page indicates the survey you are seeing and your country. 2. Going to the next page: After entering the survey data for each page, click the >> button at the lower right bottom of each page to go to the next page. 3. Going back to the previous page: If you notice errors or want to double check the data you have entered, you can go back to the previous pages at any time by clicking the << button at the right bottom of each page. All data entered on a page will be temporarily saved if you go back to the previous pages. 1 The UNESCO team explored using other on-line survey tools (Google surveys and Survey Monkey) but Qualtrics proved to be the best tool for the needs of the RIVAF project. 1
4. Entering responses: Each page shows one section of the paper survey (for example, Section I: Headteachers demographic and background information) and the questions in that section. Survey data are to be entered on-line in the space and format provided. To the extent possible, on-line questions and response options mimic those in the paper version of the survey, although in some cases the on-line version was adapted because of the Qualtrics specifications. Basically, there are three type of questions and responses: a. Text/numeric responses: where you should type in the response, whether it is an actual text or a number, as in the example below. b. Single-choice responses: where you are asked to tick one response only. c. Table questions: where text/numeric or single choice responses shall be entered, although in this case they are presented in a table format as in the paper survey. For example: Text/numeric: 2
Single choice: 5. Conditional questions: For some questions, additional information is asked if certain responses were given. For example, in the headteacher survey, the school ID number is only asked if the headteacher indicated the school has an ID to begin with. If schools have no ID number, principals are not asked to provide their ID. In this type of questions, the on-line survey is set up in a way where the conditional questions will only appear on your screen only if the condition has been met or if the response linked with the second question has been give. For example: Condition is met: School has ID Condition is not met: School has no ID 3
6. Blank responses: If no answer was given by the respondent, the way the response is entered on-line will depend on the type of question: a. Text/numeric, including table questions (examples a or c in paragraph 4 above): Leave it blank; in other words, no text or number will be entered if no answer was given. b. Choice questions, including table questions (example b or c in paragraph 4): Click No Response if a response to a question is blank. 7. N/A (not applicable) responses: Some questions allow respondents to respond N/A (not applicable) instead of providing an answer. The way N/A responses are entered depends on the type of questions and responses: c. Text/numeric, including table questions: you will enter -1 for N/A. For example, if the teacher did not work on any school in 2007 (a new teacher), the answer to the question below will be -1 in the 2007 column. On the other hand, if there were no delays in getting the compensation, the answer will be 0. b. Choice questions, including table questions: click N/A if the responded marked N/A in the paper survey. 4
8. Other option: few questions allow the respondent to choose the option Other when none of the choices apply to their case. Respondents are asked to specify their response. When a respondent used the Other option you will enter the response exactly as the respondent provides it (do not correct spelling and grammar). The Other response will be recorded in the space provided after each of these questions, as shown below for question 3.b. 9. Preventing data-entry errors: Most of the questions have a validation function to prevent incorrect data from being entered (most often typing errors). In these instances, Qualtrics would not allow you to proceed to the next page if an incorrect response had been entered. An error message will appear at the top of the page, in a red box, and a second error message will appear next to the error, with instructions as to which question and how it needs to be entered (see the year of birth example shown below). A correct response must be entered before Qualtrics lets you go to the next page. To enter the data correctly and avoid validation problems, please read carefully and use the instructions provided in Annex 1. 5
10. Save and Continue: Like with any on-line page, when you get to the Concluding Section of the on-line survey you will see the Going to the next page button >> button at lower right. When you click that button, Qualtrics will save all your survey responses and data entry progress. 2 If you save your session and continue, you will read a Qualtrics message saying that your response has been received, and then you will be automatically redirected to the first-survey page for you to re-initiate the data entry for another survey. 1.2. Instructions of the Parent/Guardian survey 11. Reformatted questions in the Parent/Guardian survey: The Parent/Guardian survey contains three questions (number 17, 18 and 23 although numbering might change depending on the country) whose column headers and row headers are switched in Qualtrics from the actual survey instrument. Be very careful to observe this change in format when entering the data. The difference in format is shown in the example below: Question 17 in the paper survey (actual format) Question 17 in the on-line survey (Qualtrics format) 2. View and download the survey responses 12. Once you entered the responses from the last survey on-line, you will still be automatically redirected to the first survey page. You will have to exit that first-page in your Internet Explorer or other Internet provider, by clicking: 2 If you are unsure about saving the data entered and want to double-check, you can always click the Going back button also ( << ) also at the lower right. 6
File, then Exit to close your browser or your browser tab. 13. After data-entry is finished for each of the three surveys, you can: (a) view the responses and (b) download the three datasets (one per survey) into your computer for later use in the analysis. For accessing any of the two options after exiting the initial data-entry page, you need go to the Qualtrics website: http://www.qualtrics.com/. 14. To log in, you must enter the username and password below in the corresponding fields, and then click Login : Username: hite-unesco-research Password: edpde2011 15. After login-in the RIVAF page in Qualtrics, you will see eight tabs on the top of the page. The tabs are: My Surveys, Create Survey, Edit Survey, Distribute Survey, View Results, Polls, Library, and Panels. Click My Surveys tab, as shown below. 16. After clicking My Surveys tab, you will see a drill down menu with the list of all 12 RIVAF countries (picture below). Once you select your country, you will have access to the three surveys prepared for your country (picture below). You will also be able to see how many responses have been entered for each survey, and you will have access to the Qualtrics functionalities available for each of you country surveys. 17. From this page you will be able to view the survey results and/or download the data to your computer. 7
2.1. View the data 18. You can use the View Report option in Qualtrics for an initial exploration of the survey results. However, we recommend not to rely much on this view options and, instead, download first the data and then view and analyze it in the way you are most familiar with, for example using Excel or SPSS. 19. In the My Surveys page (picture in paragraph 15 above), each survey in the list is accompanied by four buttons under the label of Tasks, as shown in the picture in paragraph 16: Results, Send, View, and Remove. You will only be using the buttons Results or View: 3 20. If you click the button Results, you are taken to the View Results tab where you can Select a Report to view. 3 Send: This button will take you to the Distribute Survey tab. In this tab you will be provided an active link to send to every person who will be entering data for that survey. Remove: You can remove a survey from Qualtrics. You don t use this task at this time. 8
21. When you click Initial Report you will see on the left the list of all survey questions (Q1, Q2, etc.). On the right, there are the basic statistics for each question, showing the frequencies, percentages and other distribution statistics. 22. Click the button Next Page on the lower right corner to see the results for the following question. If you need to jump to another question, you can also click the question number of the list of questions on the left side (shown in the picture above). 23. When looking at the responses for one selected question, if you place the mouse on the table, you will see that a menu Table Options appears on the top left corner, above the column Answer. 4 In the Table Options menu, you have different options for seeing the responses for the particular question, like sorting the responses differently, changing the number of decimals you see, and exporting the table into Word, Power Point, Excel and PDT. 24. For questions that have a table format and requested responses for the period 2007 to 2008 (for example, question 7 in the teacher questionnaire), the results are set up to show only the frequency of responses for each answer. To see percentages instead, go to the Table Options drill menu on the top left corner and select Toggle percentages. 4 Unfortunately, the menu only appears when you place the mouse there and therefore cannot be shown here. 9
25. I you are interested in viewing the results in a different visual formal, like a graph instead of the default table format, you can click the Add Graph button on the right side of the screen, below the Next Page button. 26. If you want to view the results organized by another question or characteristic, for example cross-tabulate all headteachers responses by whether schools are public/private, you can click the Drill down menu in the screen (see left-side photo on paragraph 21 above). You will get a question asking Drill down by Q? and you should reply Yes. All survey responses will be presented by the drilled down question you selected, as in the example below. To go back to viewing the results without the cross-tabulation, close down the drill down option by clicking the X in the red button. 2.1. Download or export the survey responses 16. After you have finished entering the data from all of the surveys, you can download and export the database for analysis. This is done by clicking the Results button (the same one used to view the surveys results in paragraph 16 above) and then the Download Data button, which is between the Responses and Cross Tabulation buttons as shown in the picture in paragraph 20. 10
17. The data will be downloaded into a database type that you would be allowed to determine. Regardless of the option you choose (most likely CSV or SPSS), you will be asked if you want to open or save the spreadsheet. A spreadsheet will automatically open if you click Open. a. CSV: A Comma Separated Value dataset that can be easily imported into Excel, Access or other programs. Note: Excel 2003 has a column limit of 256 columns. Some surveys may have more variables than this. The data will be stored in the csv file, but cannot be displayed in Excel 2003. If this is the case you will need to find another program to view the data (SPSS, SAS, STATA, Excel 2007, etc.). c. XML: An Extensible Markup Language (XML) format, which is a general purpose markup language. b. SPSS: A.sav file that can be used with the SPSS (Statistical Package for the Social Sciences) software. d. HTML: An HyperText Markup Language format of the raw data, showing the data in one large table when opened by a web browser. 18. Before you choose the database type, you will have to determine few other specifications on how to download the data. For example: a. Questions: make sure All Questions is selected. b. Representation: You can choose the representation format that works best for you. The option Answers as Values will give you the actual values of the response, for example for Question 1 asking about the directors gender, you would get 1, 2 and 3 (representing Female, Male and No Response respectively. On the contrary, the option Answers as Labels will give you the labels instead of the values. In the same example, you will see Female, Male and No Response instead of the numbers. c. Export tags: You can choose the representation format that works best for you. Both options will give you the Question name as we have designed it, for example, Q1 for Question 1 in the director s survey (gender). That said, the option Yes will export, in addition, the tags created by us when developing the survey (often similar to the question name but not necessarily), whereas the option No will not export the tags. In the latter option, variables will be listed as V1, V2, to V instead of showing the survey tag. 11
12
Annex 1: Instructions to enter the data correctly As stated, most of the questions have a validation function to prevent incorrect data from being entered (most often typing errors). In these instances, Qualtrics would not allow you to proceed to the next page if an incorrect response had been entered. A correct response must be entered before Qualtrics lets you go to the next page. Question number Survey number Date Table 1. Headteacher Survey: Content Validation by Question. Content validation Enter a number between 1 and25. Enter data within the specific ranges of numbers as follows: Day: Between 1 and 31 Month: Between 1 and 12 Year: 2010 or 2011 2 Enter the date between specific ranges of numbers as follows: Day: Between 1 and 31 Month: Between 1 and 12 Year: Between 1900 and 1999 4 Enter a number between 0 and 99. Write -1 for N/A. 5 Enter a number between 0 and 99. 6 Enter a number between 0 and 99. 10 Enter a number between 1850 and 2010. 14 Enter a number between specific ranges of numbers as follows: Hours per day: Between 1 and 24 Days per week: Between 1 and 7 Weeks per year: Between 1and 52 16 Enter a number between 0 and 12. 21 Enter a number between 0 and 99. 22 Enter a number between 0 and 600 000. 24 Enter a number between 0 and 5000 (Tuition) and 1000 (Non-tuition). 25 Enter a number between 0 and 999. Leave it blank when there is no response 13
26 Enter a number between 0 and 9999. 27 Enter a number between 0 and 99. 29 Enter a number between 0 and 999. Leave it blank when there is no response Table 2. Teacher Survey: Content Validation by Question Question number Content validation Survey number Enter a number between 1 and 250. Date Enter a date between specific ranges of numbers as follows: Day: Between 1 and 31 Month: Between 1 and 12 Year: 2010 or 2011 2 Enter a date between specific ranges of numbers as follows: Day: Between 1 and 31 Month: Between 1 and 12 Year: Between 1900 and 1999 4 Enter a number between 0 and 99. 5 Enter a number between 0 and 99. 9 Enter a number between specific ranges of numbers as follows: Hours per day: Between 1 and 24 Days per week: Between 1 and 7 Weeks per year: Between 1and 52 11 Enter a number between 0 and 12. Leave it blank when there is no response 14 Enter a number between 0 and 31. Leave it blank when there is no response 18 Enter a number between 0 and 99. Leave it blank when there is no response 19 Enter a number between 0 and 99. Leave it blank when there is no response 20 Enter a number between 0 and 999. Leave it blank when there is no response 14
Table 3. Parent/Guardian Survey: Content Validation by Question Question Content Survey number Enter a number between 1 and 125. Date Enter the date between specific ranges of numbers as follows: Day: Between 1 and 31 Month: Between 1 and 12 Year: 2010 or 2011 2 Enter the date between specific ranges of numbers as follows: Day: Between 1 and 31 Month: Between 1 and 12 Year: Between 1900 and 1999 4 Enter a number between 1and 10. 5 Enter a number between 1and 10. 7 Enter a number between 1 and 20. 11 Enter the date between specific ranges of numbers as follows: Standard the child attended: Between 1 and 7 Number of pupils in the child s class: Between 1 and 999 12 Enter a number between 1 and 31. Leave it blank when there is no response 15 Enter a number between 1 and 9999. 17 Enter a number between 0 and 168. 18 Enter a number between 0 and 168. 23 Enter a number between specific ranges of numbers as follows: Hours per day: Between 1 and 24 Days per week: Between 1 and 7 Weeks per year: Between 1and 52 15
Annex 2: Questions with Other response A list of the questions which contain the choice of Other (specify below): are listed below. An example of this type of question is also provided below. Table 4. Questions with Other response, by survey Head teacher Teacher Parent/Guardian 3 3 3 7 7 6 12 8 8 13 12 15 20 22 16