AEC 874 (2007) Field Data Collection & Analysis in Developing Countries. VI. Data Preparation & Processing

Transcription

1 AEC 874 (007) Field Data Cllectin & Analysis in Develping Cuntries VI. Data Preparatin & Prcessing. Richard H. Bernsten Agricultural Ecnmics Michigan State University A. Overview Need t plan fr data preparatin and prcessing at the beginning f the prject Prject wrkplan shuld identify Type f analysis anticipated Hw data will be analyzed Resurces required, including training fr data entry Data Preparatin Activities Field Activities--enumeratr, supervisr review Office Activities--lg-in, sight editing, pst-cding, file creatin, data entry, validatin, preliminary analysis

2 B. Field Data Preparatin Mst prjects limit field data preparatin t reviewing the questinnaire fr cmpleteness/legibility.. Purpses T prduce a cmplete, legible questinnaire fr ffice review T minimize need t return questinnaires t the field t cllect missing data Nte--The earlier yu find & crrect errrs, the less time it will take t crrect them later. Respnsibilities Enumeratrs must checks each questinnaires after cmpleting the interview Field Supervisr Must Read each questinnaire t identify missing/unclear answers, incrrect answers (lgical impssibilities) Cmpile a list f pssible errrs fr each questinnaire (List prblem questin numbers n p. f the questinnaire) Review prblem questins with the enumeratr If enumeratr can t clarify prblem questins, ask him/her t cnduct recall interviews t clarify/btain missing data 4

3 C. Office Data Preparatin Senir researcher must clsely supervise all ffice data preparatin activities. Early Preparatin--activities befre questinnaires arrive at ffice Develp flw chart t dcument all data prcessing steps (Flw Chart) Assign staff respnsibilities fr: recrding receipt f questinnaires, sight editing, pst-cding, file creatin, data entry, validatin, preliminary analysis Obtain a secure rm fr string questinnaires Obtain apprpriate hardware & sftware Train the editing, pst-cding & data entry staff 5. Mnitring Questinnaire Prcessing Develp a data prcessing recrd system, s yu always knw where each questinnaire is/wh has it Cunt questinnaires when they arrive at the ffice, check against sample size t see if any questinnaires are missing Require staff t check-ut any Qs that they remve frm the strage rm IF YOUR SURVEY INVOLVES SEVERAL MODULES, CARRY OUT ALL OF THE FOLLOWING ACTIVITIES FOR ONE MODULE AT A TIME

4 . Sight Editing Def. Initial ffice visual review f all questinnaires Develp written guidelines (i.e., a checklist) fr assessing cmpleteness & accuracy Instruct sight editr t flag/circle pssible prblems/errrs Site editrs must review each page in each questinnaire fr: Cmpleteness (e.g., missing pages, missing answers, legibility) Accuracy (e.g., crrect/acceptable cdes fr categrical data, extreme values fr cntinuus data, crrect branching fr skip questins) 7. Only supervisr shuld assign values t missing data, reslve incnsistencies, assign cdes fr missing data (e.g., cde male/female, based n name; enter values fr a questin/variable based n answers t anther questin) Supervisr must review/check a sample f sight editrs cmpleted questinnaires 4. Pst-Cding Def. Assigning numeric cdes t pen-ended questins, and/r cding f items entered in the Q as ther (specify) May als summarize data t reduce the amunt f data entry (e.g., Q asked fr ages f all kids, but yu nly cde ttal # f kids < 5 yrs); Q asks fr details abut all rganizatinal memberships but yu nly cde # f memberships) Well-designed/pre-tested Q shuld require minimal pst-cding 8 4

5 a) Respnsibility fr Pst-cding ) T minimize nn-sampling errr, ne persn familiar with the data/prject gals shuld be assigned t pst-cding specific Qs ) If the apprpriate pst-cde categry is unclear, the pst-cding staff must review the text with the supervisr (senir researcher) befre assigning a cde b) The Cdebk (single cpy!!!) ) Develp a cdebk fr assigning cde values t pst-cded questin, as yu prcess each questinnaire ) Add new categries t the cdebk, as they ccur in the questinnaires, and assign this categry the next cde value ) Write the cde assigned t the respndent s answer directly n each questinnaire, in far right margin 4) Pst-cding supervisr (senir researcher) must review/check sample f pst-cded questinnaires 9 c) Pst-Cding Cnventins/Guidelines ) Use the same cde value fr all questins with the same answer categries (e.g., crp planted/harvested: =maize, =rice, =wheat) ) Use 0, fr dichtmus items (e.g., 0=male, =female; 0=n, =yes) ) Avid cding answers int a few (<0?) highly aggregated categry; yu can aggregate later using SPSS (e.g., Cde variety name, recde t MV & TV later; Cde fertilizer brand, recde t type later) 4) Missing Values Use different missing value cdes t specify why data are missing (e.g., -9=nt applicable, -8=didn t knw answer) Use negative value t indicate missing data t avid using same cde fr missing, as fr a real data value 0 5

6 5) If n data value/number is entered, SPSS cnverts empty/blank cells t system missing (.). S, enter a number in every cells (e.g., zer, real number, -9=nt applicable, -8=dn t knw) ) Only use numeric cdes, unless yu plan t assign numeric cdes t string/alphanumeric after data are entered int SPSS (i.e., Enter answers t pen-ended questins as string variable, then create a new categrical variable & recde string data t numeric cdes using SPSS. Be sure t keep the riginal string variables, as yu may want t later mdify yur categries.) 5. Data Transfer a) Must Determine the File Structure Befre Entering Data Data levels (see Bernsten & Beaver) Review the questinnaire t determine units f analysis Enter data fr different units f analysis in separate files ) Single-Level Data (rectangular/symmetric data set) (i.e., ne answer per questin/variable per respndent Resp # 4 Village Age Educ 9 4 N. Kids 7 4 Gender Married Farmer 0 0

7 ) Multiple-Level Data (nn-symmetric data set) (i.e., number f answers t a questin varies frm respndent-trespndent (e.g., characteristics f each family member, field, parcel, variety, transactin) Variety example (belw) Variety is the unique variable, s must create a file: variety Can analyze data at crp/variety level r aggregate data t crp, parcel, field r HH level with SPSS aggregate cmmand Resp # Village Field Parcel Crp Variety Area Output b) Optin fr Data Entry ) Enter Data int a Spreadsheet Steps Create a spreadsheet (Excel, Quattr Pr) Enter variable names/data int the spreadsheet Imprt spreadsheet int SPSS Assign value, value labels, etc. after imprting int SPSS Disadvantages/Disadvantages Can use if SPSS is nt available t all DE staff Errrs pssible, since value cdes are nt in spreadsheet ) Use SPSS s Data Entry Prgram Allws yu t create a DE frm, define variables, enter data, define data cleaning rules, & clean yur data But DE prgram is expensive, s it s ften nt available ) Enter Data Directly int a SPSS Systems File Often best practical ptin Can view value cded while entering data 4 7

8 4) Use Census Bureau s DE Prgram Gd ptin fr big prject, as it allws yu t define valid cdes, value ranges but prgram is cmplicated c) Guidelines fr Entering Data int a SPSS Systems File ) Tasks and Respnsibilities Creating the file senir researcher Entering data--trained DE peratr r researcher Reviewing suspicius data values--researcher & DE peratr ) Create a SPSS Systems File fr Data Entry Open SPSS Select Type in Data, then click n OK Select Variable View (bttm left crner f the page) Place the cursr in rw, clumn f the blank frm G t the Edit menu, select Insert Variable t create a new variable named VAR0000 (r simply type a name in the cell) 5 Click n VAR0000 t assign it a specific variable name Define the variable s characteristics by editing the default values (i.e., type, width, decimals, label, values, missing, clumns, align, measure) Nte--always define clumns = r greater.why? Repeat the prcess until all variable are defined Save the file (i.e., assign the file name) G t File, select Display Data Infrmatin, assign a file name & view/edit/print ut file t review hw yu defined yur data Suggestins Name SPSS files with wrds/memnics that refer t the data type (e.g., HHMemr, Parcel, Trans, Equip Name variables with memnics r the survey questin number (e.g., V-., V-.,... V-.5 but can t begin name with a number) Assign cmplete variable labels (with questin #) Assign cmplete value labels t categrical variables 8

9 ) Entering Data Int the SPSS Systems File Nte--each questinnaire that yu give the DE peratr must be cmpletely cded, with either pre-cded and/r pst-cded values entered clearly in the data field (right side f questinnaire) Open SPSS Select Open an existing data surce, highlight the SPSS files that yu want t pen, and click n OK Select Data View (bttm left f frm) G t the tlbar and click n Value Labels (next t the last icn n the right) t tggle between text and numeric values. Befre entering data, turn n text t display the descriptin f the value cdes (may have t first enter/delete a data value). Enter data fr each case (respndent) Save file frequently during DE & after all data are entered 7 d) Validating/Cleaning the Data ) Reasns fr Cmputer Validatin Sight editing misses errrs Need t catch errrs befre beginning data analysis, t avid prducing errneus results If data errrs are fund during analysis, will have t rerun yur analysis ) Guidelines fr Cleaning Yur Data Review the questinnaire t identify relatinships amng variables that must be true. Then, create/run script t cnfirm relatinships, cmpare t data entered (e.g., ttal farm area must = sum f parcel areas) Fr categrical variables Run select if t find utlyers (e.g., yu specified cdes -5, s select if value is > 5) Run frequencies. Then, lk fr ut f range values 8 9

10 Fr cntinuus variables Specify a reasnable range f data values (e.g., variable age = between and 80 years) Then, run select if data values are </> reasnable values) If utlyers are fund, check questinnaire fr cding/de errrs Can assign values t missing/errneus data, but be very careful/cautius. Suspicius values ften turn ut t be OK. They may seem extreme, but are actually crrect If yu can t reslve the errr, return the questinnaire t a field fr a recall interview, especially if: Many variables in questinnaire are missing/suspicius The missing data are imprtant variables fr yur analysis Nte--If yu send Q t the field, add a cver page nting prblem Q & instructins fr reslving the prblem 9 e) Carry Out A Final Data Check After cleaning yur data, yu must carry ut preliminary analysis befre beginning serius data analysis ) Fr cntinuus variables, run descriptives t check fr extreme values (e.g., mean, minimum, maximum) ) Fr categrical variables, re-run frequencies t check fr unspecified cdes, suspicius distributins f respnses ) List cases (questinnaire ID# plus variable) & scan data t lk fr suspicius values 4) Cmpare suspicius value t value in the questinnaire, crrect if necessary 0 0

11 C. Assigning Weights t Yur Data Def. Prcedures that frce certain case t cunt mre in the statistical cmputatins, than ther cases. Is required if yu used a stratified/cluster sampling prcedure and yu are planning t make inferences abut the ppulatin Prblem--must have data/infrmatin abut the relatinship f sample t ppulatin t determine what weights t use Weighting nt required if yu are nly analyzing relatinships between respndent in each strata (i.e., estimating/cmparing yields f sampled rainfed vs. irrigated rice farmers). Can weight data t crrect fr nn-respnse, if yu knw the characteristics f nn-respndents & their percent in ppulatin (e.g., crrect sample fr lw number f female farmers, if yu knw crrect percent f wmen farmers in the ppulatin) DATA PREPARATION AND PROCESSING FLOW CHART FIELD ACTIVITIES Enumeratrs review: Supervisrs review: Enumeratr review questinnaire IF OK-----> Send Qs t field supervisr IF NO-----> Recall respndent! Supervisr reviews questinnaire IF OK-----> Send Qs t ffice fr lgging-in IF NO-----> Return Qs t enumeratr fr recall interview! > Return Qs t supervisr fr final check > Send Qs t ffice fr lgging-in

12 OFFICE ACTIVITIES Lg-in questinnaire Sight editing Pst-cding Staff lgs-in questinnaire-----> Send Qs t sight editr Sight editr reviews questinnaire IF OK-----> Send Qs t pst cder IF NO-----> Send Qs t supervisr > Is recall needed? Staff create pstcde categries, Supervisr checks categries created, Staff transfers new cde values t questinnaires IF OK-----> Send Qs t file creatr IF NO-----> Send Qs t supervisr > Is recall needed? Nw Questinnaires are Ready fr Data Transfer Create File Researcher creates file fr data entry, Supervisr reviews IF OK-----> Send file + Qs t data entry staff IF NO-----> Researcher revises frmat Data Entry Validatin DE staff enter data IF OK-----> Send Qs t researcher fr validatin check IF NO-----> Send Qs t supervisr >Is recall needed? Researcher defines ranges, relatinships, staff runs check IF OK-----> Send file& Qs t researcher fr preliminary analysis IF NO-----> Researcher check Q fr DE errrs Send Qs t supervisr >Is recall needed? 4

13 Preliminary Analysis Researcher runs frequencies, descriptives t identify suspicius values IF OK-----> IF NO-----> Send file t researcher fr data analysis Researcher checks values against Qs Send Qs t supervisr------> Is recall needed? Fr mre infrmatin, see Guide t Scial Science Data Preparatin and Archiving, at: Return t p. 5 5