A Brief History of Cell vs Landline Performance Wisconsin BRFSS

Transcription

1 A Brief History of Cell vs Landline Performance Wisconsin BRFSS 2015 IFD&TC John Stevenson, August Salick, Robert Cradock, Jennifer Dykema May Materials may not be reproduced without permission of the author.

2 The problem: Landline vs. Cell Coverage 2

3 Why do we use cell samples? Error using landline numbers only is not random Who do we miss without cell (Luke & Bloomberg 2014) Young Poor Renters Hispanics Those with children 3

4 Why am I here? Look at cell vs. landline sample A few years ago Cell completes cost 2-3 times more Desire to know Who are we getting with cell sample? When should I encourage/discourage with clients? What does new reality ($) look like Look at some crude measures today 4

5 History of BRFSS cell as fielded in Wisconsin Many changes Cell use screening for eligibility Use of incentives Length of survey Calling protocol Sample size Proportion cell 5

6 BRFSS Cell Phone Sample Protocol - Wisconsin Year Cell Use Incentive Length Call limits N %CP/ LL+CP 2008 Cell only $10 post ~10 min Varied 5 & % 2009 Cell only $10 post ~15 min w/contact 261 6% 2010 Cell only none ~15 min w/contact 413 9% 2011 Cell only none ~24 min w/contact % 2012 Cell 90% + none ~22 min 6 +8 w/contact % 2013 Cell 90% + none ~26 min 6 +8 w/contact 1,514 29% 2014 Everyone none ~26 min 6 +8 w/contact 2,366 40% 6

7 BRFSS Cell Phone Sample Protocol - Wisconsin Year Cell Use Incentive Length Call limits N %CP/ LL+CP 2008 Cell only $10 post ~10 min Varied 5 & % 2009 Cell only $10 post ~15 min w/contact 261 6% 2010 Cell only none ~15 min w/contact 413 9% 2011 Cell only none ~24 min w/contact % 2012 Cell 90% + none ~22 min 6 +8 w/contact % 2013 Cell 90% + none ~26 min 6 +8 w/contact 1,514 29% 2014 Everyone none ~26 min 6 +8 w/contact 2,366 40% 7

8 BRFSS Cell Phone Sample Protocol - Wisconsin Year Cell Use Incentive Length Call limits N %CP/ LL+CP 2008 Cell only $10 post ~10 min Varied 5 & % 2009 Cell only $10 post ~15 min w/contact 261 6% 2010 Cell only none ~15 min w/contact 413 9% 2011 Cell only none ~24 min w/contact % 2012 Cell 90% + none ~22 min 6 +8 w/contact % 2013 Cell 90% + none ~26 min 6 +8 w/contact 1,514 29% 2014 Everyone none ~26 min 6 +8 w/contact 2,366 40% 8

9 BRFSS Cell Phone Sample Protocol - Wisconsin Year Cell Use Incentive Length Call limits N %CP/ LL+CP 2008 Cell only $10 post ~10 min Varied 5 & % 2009 Cell only $10 post ~15 min w/contact 261 6% 2010 Cell only none ~15 min w/contact 413 9% 2011 Cell only none ~24 min w/contact % 2012 Cell 90% + none ~22 min 6 +8 w/contact % 2013 Cell 90% + none ~26 min 6 +8 w/contact 1,514 29% 2014 Everyone none ~26 min 6 +8 w/contact 2,366 40% 9

10 BRFSS Cell Phone Sample Protocol - Wisconsin Year Cell Use Incentive Length Call limits N %CP/ LL+CP 2008 Cell only $10 post ~10 min Varied 5 & % 2009 Cell only $10 post ~15 min w/contact 261 6% 2010 Cell only none ~15 min w/contact 413 9% 2011 Cell only none ~24 min w/contact % 2012 Cell 90% + none ~22 min 6 +8 w/contact % 2013 Cell 90% + none ~26 min 6 +8 w/contact 1,514 29% 2014 Everyone none ~26 min 6 +8 w/contact 2,366 40% 10

11 BRFSS Cell Phone Sample Protocol - Wisconsin Year Cell Use Incentive Length Call limits N %CP/ LL+CP 2008 Cell only $10 post ~10 min Varied 5 & % 2009 Cell only $10 post ~15 min w/contact 261 6% 2010 Cell only none ~15 min w/contact 413 9% 2011 Cell only none ~24 min w/contact % 2012 Cell 90% + none ~22 min 6 +8 w/contact % 2013 Cell 90% + none ~26 min 6 +8 w/contact 1,514 29% 2014 Everyone none ~26 min 6 +8 w/contact 2,366 40% 11

12 BRFSS Cell Phone Sample Protocol - Wisconsin Year Cell Use Incentive Length Call limits N %CP/ LL+CP 2008 Cell only $10 post ~10 min Varied 5 & % 2009 Cell only $10 post ~15 min w/contact 261 6% 2010 Cell only none ~15 min w/contact 413 9% 2011 Cell only none ~24 min w/contact % 2012 Cell 90% + none ~22 min 6 +8 w/contact % 2013 Cell 90% + none ~26 min 6 +8 w/contact 1,514 29% 2014 Everyone none ~26 min 6 +8 w/contact 2,366 40% Cell Phone / Landline + Cell Phone X

13 Response rates Too many caveats to mention! Changes in calling structure Changes in calculation Sample makeup Non-contact and E estimator 13

14 Response Rates Wisconsin BRFSS 70% 60% 50% 40% 30% Landline Cell 20% 10% 0%

15 Landline sample call dispositions 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% * Hang-up Non-contact Non-working Not Eligible Refused Partial Complete 15

16 Cell sample call dispositions 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% * Hang-up Non-contact Non-working Not Eligible Refused Partial Complete 16

17 Who are we getting by Cell vs. Landline Who are we getting with cell sample? Looked at demographics Age Gender Hispanic/Race Education Income Household Size Not weighted, just raw counts by year 17

18 Age Landline sample completes 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% ACS & older 40 to & under 18

19 Age Cell sample completes 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% ACS & older 40 to & under 19

20 Gender - landline vs cell (% female) 80% 70% 60% 50% 40% 30% 20% Cell Landline 10% 0%

21 Ethnicity - landline vs cell (% Hispanic) 6% 5% 4% 3% 2% Cell Landline 1% 0%

22 Race, Landline vs cell Percent African American Absolute measure is problematic! 20% 15% 10% Cell Landline 5% 0%

23 Education Landline 100% 90% 80% 70% 60% 50% 40% 30% College+ Some College HS < HS 20% 10% 0%

24 Education Cell 100% 90% 80% 70% 60% 50% 40% 30% College+ Some College HS < HS 20% 10% 0%

25 Income Landline sample completes 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% K+ $35-75K $<35K 25

26 Income Cell sample completes 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% K+ $35-75K $<35K 26

27 Percent of households with children Landline 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% >

28 Percent of households with children - Cell 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% >

29 Effects on effort and cost How are we doing? How are calls per complete changing over time How are changes affecting yield of sample/amount of sample fielded Change in phone room effort Calls Sample fielded 29

30 Calls per complete by year Cell Landline 30

31 Sample points fielded per complete Cell Landline

32 Relative cost of cell complete/landline complete (*only included some of the field costs) 180% 160% 140% 120% 100% 80% 60% 40% 20% 0%

33 Discussion Cell efficiency getting better Concern about high rate of noncontact for cell Tough to reach still tough to reach Problem of attempting to oversample/target small geographic areas (add cite) Questions: Consistent with what you are seeing at other shops? Improvements in sampling/targeting? 33

34 Thank You! For copies of this presentation or more information, contact: John Stevenson Please visit us at: