Applications to support the curation of African government microdata for research purposes

Transcription

1 Statistics SA/OECD Seminar on Innovative Approaches to turn Statistics into Knowledge Applications to support the curation of African government microdata for research purposes Lynn Woolfrey, DataFirst, University of Cape Town Diego Iturralde, Statistics South Africa

2 How do we make this data available to researchers? 2 Your reference

3 Metadata editor Microdata management Toolkit s Metadata Editor Application used by DataFirst to create standardised survey documentation (metadata) Developed by the IHSN Based on NESSTAR (dev by Norwegian Social Science data services) 3 Your reference

4 Metadata editor A framework for the description of all elements of the survey data - to support data usage: 4 Your reference

5 The metadata editor uses the Data Documentation Initiative (DDI)as a standard to describe data elements Metadata editor The Dublin Core (DC) Standard is used to describe the core elements of external resources for the survey data (questionnaires, codebooks, reports) 5 Your reference

6 Metadata editor Ensures the usability of the survey data through: Uniquely identifying the survey dataset Including all possible information to support usage e.g. data provenance, data collection methods, codes and classification schemes, data errors Standardisation, which allows for data comparability 6 Your reference

7 The National Data Archive (NADA 3.0) Application used by DataFirst to support the data discovery and data dissemination Developed by the IHSN Open source, PHP application using a MySQL database Freely available to National Statistics Offices in developing countries 7 Your reference

8 National data archive Data management datasets are made available from the NADA catalogue data files (in Stata, SPSS), metadata (DDI format) and external resources (pdf) Data discovery all elements of the surveys are searchable in the NADA catalogue Data access allows for controlled access to microdata 8 Your reference

9 DataFirst s NADA catalogue National data archive 9 Your reference

10 National data archive Other features (NADA just released) Supports the collection of user statistics Citation manager provides access to citations of works based on the survey data and links to these publications A one-stop shop for microdata users 10 Your reference

11 The IHSN tools support data curation for research 11 Your reference

12 Presentation to OECD Seminar on Innovative Ways of Turning Statistics into Knowledge Data Access and Data Quality Innovation Diego Iturralde (Stats SA) Lynn Woolfrey (UCT) 10 December Your reference

13 Layout 1. Fundamental Principles of Official Statistics 2. Policy on Data Access 3. Data Quality 4. Web Dissemination at Statistics SA 13 Your reference

14 Fundamental Principles of Official Statistics In its endeavour to fulfil the purpose of providing users with quality information, Statistics South Africa has adopted the Fundamental Principles of Official Statistics developed by the Economics and Social Council Statistical Commission of the United Nations Principle 1deals with relevance, impartiality and equal access and states that statistics that meet the test of practical utility are to be compiled and made available on an impartial basis to honour citizen s entitlement to public information 14 Your reference

15 Policy on Data Access Article 4.5 on the Policy of Publication and Data Access deals with access to data Access to data already in the public domain is not restricted in any way Unpublished data is limited to Stats SA staff for the purpose of carrying out their terms of employment Stats SA staff should not make copies of data for purposes other than that of their terms of employment Access to Stats SA employees for purposes other than those of their terms of employment can only be done with the written permission of the Statistician-General Access to external parties of unpublished data is also subject to written permission of the Statistician-General 15 Your reference

16 Data Quality Statistics SA places a high premium on data quality and subjects any government department wishing to have its data certified as official to go through the SASQAF process Consequently, Statistics SA subjects its own data to quality checks. This involves ensuring that the metadata, data and publication of the household surveys are all speaking the same language. Type of checks involved include ensuring there are no logical errors, repeat observations, ensuring that data items in the metadata are consistent with the data and that tables in the publication are consistent with the data Evidently it is not possible to check to what extent responses where accurately recorded, bias of interviewer or whether the questions were understood by interviewer and respondent these are issues that research and testing of questionnaires need to address 16 Your reference

17 Data quality should not just be a operational priority but it is a critical imperative for everybody Ultimately the quality of data used impacts the quality of research produced. It is not reasonable to expect that given poor quality data, poor quality research will be conducted which will be used to make poor decision on issues of importance to our communities. Data quality also means that when comparing data we need to bear in mind that tools and methods of measurement today are better than those used 5-10 years ago and hence be careful to make pronouncements on trends that the data has followed during this time An issue for data quality to consider is that of consistency of the framework of spatial distribution. Maintaining a consistent and comparable geographic hierarchy overtime is critical and Statistics SA counts on the support provided by the Municipal Demarcation Board for such a purpose 17 Your reference

18 In the interest of ongoing data quality, Statistics SA is engaged in a data quality partnership with Data First from University of Cape Town which is aimed at enhancing data quality, particularly of the older data sets whilst ensuring that data quality and good practice is maintained Data quality requires good documentation to be produced and that accessible archiving systems be developed for these to be available for many years to come By means of sharing the type of data access Statistics SA provides the following slides highlighting what web dissemination at Stats SA looks like 18 Your reference

19 19 Your reference

20 20 Your reference

21 21 Your reference

22 Text 22 Your reference

23 Text 23 Your reference

24 Text 24 Your reference

25 25 Your reference

26 26 Your reference

27 27 Your reference

28 Text 28 Your reference

29 29 Your reference

30 30 Your reference

31 31 Your reference

32 32 Your reference

33 33 Your reference

34 Text 34 Your reference

35 Text 35 Your reference

36 Topic 36 Your reference

37 37 Your reference

38 Looking Ahead On going re-weighting of household datasets Rolling out of new surveys and economic time series Engaging with users to establish relevant products API applications FTP download site 38 Your reference