The Effectiveness of in Obtaining Respondents for Web-based Surveys

Transcription

1 The Effectiveness of in Obtaining Respondents for Web-based Surveys Donna M Jewell Prepared by Stuart Allen and Chris Rasmussen May 16, 2005 RTI International is a trade name of Research Triangle Institute 3040 Cornwallis Road P.O. Box Research Triangle Park, North Carolina, USA dmj@rti.org

2 NSoFaS 2004 Background National Study of Faculty and Students Web-based survey with a telephone follow-up Faculty Sample: 35,629 (Full scale ran from 02/04 09/04) Student Sample: 109,213 (Full scale ran from 03/04 08/04)

3 Why use ? access common among sample population Easier to contact larger audiences Simple survey access Reduce call center costs Less intrusive than phone calls

4 How did we obtain addresses? Faculty: Lists sent in from the schools: 976 NLC (NSoFaS List Completer) Application: Polling school websites if < 75% with s Provided ~4,300 extra addresses Student: Lists sent in from the schools: 1,364

5 messages Personalized s to help reduce SPAM filtering Importance of the study and other pleas Link to survey website Username and password to access the survey site Incentive offers Provided Help Desk phone number IRB approved content and schedule

6 schedule Early response periods: Faculty - 4 weeks Student - 3 weeks Staggered schedule based upon early response period Initial announcing study 2 weeks before early response period end 2 days before early response period end

7 Respondents with addresses All Respondents 100% 80% 60% 40% 64% Late response Early response Student 28% 72% No With 20% 0% 36% Student Respondents with Faculty 17% 83% 100% 80% 36% 0% 20% 40% 60% 80% 100% 60% 40% 20% 64% Late response Early response 0% Faculty Respondents with

8 ????? Percentage of Respondents 100% 80% 17% 28% 60% 40% 83% 72% Without With 20% 0% Faculty Student

9 respondents timeframe Respondents with 100% 80% 60% 40% 20% 36% 64% 64% 36% Late response Early response 0% Faculty Student

10 Respondent schedule results Percentage of early responders 100% 80% 60% 40% 20% 13% 25% 22% 40% 28% 36% 36% 4th 3rd 2nd 1st 0% Faculty Student

11 How effective was it? Faculty: 64% Early response (extremely effective) Students: 36% Early response (very effective) Sending multiple s is effective, last minute reminder is important. Encountered little hostility from s

12 Tool used to obtain addresses Web Application: NLC (NSoFaS List Completer) School list is sampled and entered into the DB If coverage is < 75%, school is flagged for NLC Each school within NLC is presented with sample member counts, who is currently working the school and other school data. Clicking school presents all sample member information from the school and school s contact information (website, phone number) Website pops up in another window for directory searching Sample member information (phone, ) is added into NLC for each sample member found Once information search is complete or exhausted, school is marked for sample member release and initial s are sent

13 NLC School listing School data (phone, IPEDS ID, name) Sample member counts (Dup. phone, Need work, total sampled) Status Information (Status code, current user, last updated by)

14 NLC School updating School data (phone, IPEDS ID, website) Sample member counts (# that need work, # already worked, etc.) Sample member data (current , address, phone #)

15 NLC results ~4,300 extra addresses harvested using the NLC tool

16 Tool used to send s Web application: out Setup mailout templates and bodies Customize each aspect of an (Body, subject, sender, etc.) Document-merge style of filling body contents from SQL-query based data. Ability to perform SQL actions after each is sent. Record history of each sent. API access for automatic batch processing

17 out properties Sender address DB Connection String SQL statement Callback SQL statement Primary key variable variable Information variable Body Selector variable 3 different bodies selectable Merge variables

18 out results Over 300,000 s sent to faculty and students

19 Future plans Customizing links in each to accurately measure schedule effectiveness Bounced processing to update sample member database HTML bodies will be added

20 Thank you for attending Questions? Donna Jewell Presentation at: