Data Entry, Processing and Management

Transcription

1 Data Entry, Processing and Management Raka Banerjee Multi-Topic Household Surveys Poverty and Inequality Course March 7 th, 2012

2 Introduction Data entry, processing and management as fundamental aspects of household surveys Key consideration when planning field work Critical role in ensuring quality and timeliness of data collected Choice of protocol can have significant impact on success of field work

3 Outline Data Entry Centralized Data Entry Computer-Assisted Field Entry (CAFE) Computer-Assisted Personal Interviewing (CAPI) Database Construction Database Cleaning Database Finalization

4 Key Questions What data entry software will you use? Who will create the data entry template? How many data entry staff will you need? How will you train the data entry staff? Who will supervise the data entry process? How will errors be detected and corrected?

5 Centralized Data Entry: Features Data are entered by a data entry team at headquarters Questionnaires received and logged on rolling basis Double data entry to reduce error: Verify vs. Compare Data entry process is managed via direct oversight Immediate resolution of technical issues High-level, consistent supervision of data entry quality

6 Centralized Data Entry: Issues Questionnaires must be transported to headquarters, resulting in delay in data entry and data quality impacts Renders enumerator error more intractable Often not feasible to conduct callbacks Diminishes headquarters supervision of enumerators via rapid performance feedback Difficult to motivate enumerator improvement

7 Computer-Assisted Field Entry: Features Data are entered in the field by data entry staff embedded into mobile survey teams Enumerator errors are checked and accounted for on a timely basis, allowing for: Improvement of enumerator performance over time via incorporation of feedback Improvement of data quality via the facilitation of callbacks

8 Computer-Assisted Field Entry: Issues Direct data entry supervision by headquarters staff no longer possible High level of responsibility for survey team leaders Increased logistical cost to resolve technical issues Data must be transmitted to headquarters Intermittent access to internet and electricity Concerns regarding security and confidentiality

9 Computer-Assisted Personal Interviewing (CAPI) Integration of interviewing and data entry via the use of a handheld device with an electronic questionnaire Radically changes the entire process of conducting a household survey Decentralized data entry system should be considered an interim step towards CAPI

10 Data Compilation Final raw database compiled by headquarters upon conclusion of field work Rolling basis or after field work is complete Basic checks of the complete database include: Checking of household/community/price data Preparation of final sampling weights Database cleaning

11 Database Cleaning All variables within the database must be checked for remaining errors / discrepancies Cleaning!= Fixing Errors corrected during data collection process Range and consistency checks conducted during field work Analysts determine final approach regarding outliers, missing values, etc.

12 Database Finalization File structure should be organized by the underlying unit of analysis Household / individual / topic levels Identification information necessary for merging Constructed variables could include: Consumption/expenditure aggregate Income aggregate Z-Scores/Nutrition

13 Documentation Checklist Number of households included Reasons for exclusions Distribution of households Number of individuals included Unique identifiers Instructions for merging files Problems encountered Methodology for constructed variables

14 Centralized Data Entry Field teams visit households Questionnaires Transported to the HQ 1 st Data Entry 2nd Data Entry & Verification Data Processed, Analyzed, Errors Flagged Feedback on Team & Enumerator Performance (Rolling Basis) Questionnaires Checked & Data Processing Finalized

15 Decentralized Data Entry Mobile Survey Teams with Concurrent Data Entry & Processing Visit Households Data Sent to Headquarters Data Checked Feedback Review Reports Enter Data Questionnaires Transported to Headquarters Generate Onsite Error Error Reports Headquarters 2 nd Data Entry & Verification

16 Second Data Entry Centralized Data Entry 1 st and 2 nd Data Entry are conducted simultaneously at headquarters Daily discrepancy checks correct both enumerator and data entry error Computer-Assisted Field Entry 1 st Data Entry is conducted in the field; 2 nd Data Entry is conducted at headquarters Discrepancy checks correct data entry error

17

18

19