IPUMS-International: Free, Worldwide Microdata Access Now for Censuses of 62 Countries--80 by 2015

Transcription

1 IPUMS-International: Free, Worldwide Microdata Access Now for Censuses of 62 Countries--80 by 2015 Robert McCaa, Steven Ruggles, Matt Sobek and Wendy L. Thomas Session STS065 The Future of Microdata Access 58 th International ti Statistical ti ti Institute, t Dublin, Ireland, 26 August, 2011 * * * rmccaa@umn.edu for additional details, please see: 1

2 Dissemination [means] opening up the value inherent in our data --Walter Radermacher (President, Eurostat) and Pieter Everaers (Director, Eurostat) IPUMS-International: Free, Worldwide Microdata Access Now for Censuses of 62 Countries--80 by 2015 IPUMS opens up the value inherent in census microdata. Robert McCaa, Steven Ruggles, Matt Sobek and Wendy L. Thomas Session STS065 The Future of Microdata Access 58 th International ti Statistical ti ti Institute, t Dublin, Ireland, 26 August, 2011 * * * rmccaa@umn.edu for additional details, please see: 2

3 Deirdre Cullen, Senior Statistician CSO-Ireland, testimonial: Advantages of IPUMS for Ireland Bonus for CSO: as a result of this project, our historic data sets are now in a much more usable format IPUMS allows mix of Census years available in 1 file Comparability with other countries Ease of access for users Positive publicity for Census in Ireland

4 Outline IPUMS-International: International: the future is now 5 slides 2002: Integrated microdata disseminated for 7 countries, 25 samples 2015:. ~80 countries, ~250 samples Four IPUMS-International International goals and achievements: 1. Archive preserve census documentation and microdata 2. Integrate integrate census microdata and metadata 3. Access restricted to researchers who agree to conditions of license 4. Disseminate extracts: country(ies), census(es 3 Future initiatives: 1. SDA: Online data tabulation and analysis 2. TerraPop: Population, climate, land-use data 3. IRDC: International ti Research Data Center: trusted t sites Conclusion: Sustainability of IPUMS-International International 1 es), populations, variables 4

5 IPUMS Milestones» 1995: IPUMS-USA USA first release of integrated microdata» 1999: IPUMS-International International funded» st International launch: 7 countries, 25 samples.» 2007 launch (56 th ISI): 32 89» 2009 launch (57 th ISI): » ~279 million person records» ~3,000 registered users» 2011 launch (58 th ISI): » 397 million person records» 5,000 registered users» And for 2015 (Río!): ~80 ~250» 600 million person records» ~10,000 registered users

6 Fig. 1. IPUMS-International: International: a global collaboratory dark green = integrated and disseminating ** 62 countries, 185 censuses, 397 millon person records ** medium green = integrating. Light green = negotiating 2015: 80 countries, 250 censuses integrated Open Invitation to Use and Cooperate Microdata Disseminating Integrating Mollweide projection None inventoried None entrusted 6

7 Fig. 1. Cartogram of IPUMS-International International partners weighted by population dark green green = integrated and disseminating Open Invitation to Use and Cooperate Microdata Disseminating Integrating Mollweide projection None inventoried None entrusted 7

8 University of Minnesota assumes responsibilities and risks for integrating & disseminating microdata and metadata» Uniform Memorandum of Understanding with each National Statistical Office:» Founding partners (2001): France, Spain, China, Vietnam, Kenya, Colombia, Mexico, USA now almost 100 countries» Specific conditions of access: ownership of data (NSO), use, access, restrictions, ti confidentiality, security, publication, violations, sharing, jurisdiction, and precedence.» Almost 100 countries are participating.» 13 most populous countries NOT participating:» Russian Federation, Japan, Congo (DR), Myanmar, Algeria, Afghanistan, Uzbekistan, Korea (RO), Saudi Arabia, Korea (PDR), Yemen, Syria, and Australia 8

9 1. Archive census microdata and metadata: There is no future without the past See, pp 3-4: 2011: IPUMS-International: Free, Worldwide Microdata Access Now for Censuses of 62 Countries--80 by 2015, 58 th International Statistical Institute, Dublin, Ireland, August.. 9

10 Preserve microdata and metadata against accident, deterioration and technological obsolescence» Microdata: - transfer to stable media - use standard data storage protocols - repatriate copies to NSO partners» Metadata: collect, catalogue, scan, and disseminate - Census forms (preserve all versions) - Enumeration and data processing instructions/manuals - Codebooks (photocopies and scanned images) - Technical studies, evaluations, reports 10

11 Source documents (forms, instruction manuals) --limited to integrated samples only Links to Official Statistical Agency Partners Bibliography: view cites, link to publications 11

12 Mapped in DDI; compatible with IHSN Microdata toolkit 12 copies entered into the NADA catalog and archive

13 Metadata: Vietnam 1999 original and integrated In cooperation with the International Household Survey 13 Network (IHSN) with funding by Paris21

14 2. Integrate metadata and microdata: See paper, pp

15 Browse (metadata) and Select Data Source documents (forms, instruction manuals) Link to Official Statistical Agency home pages Bibliography: view cites, link to publications 15

16 Metadata are integrated, open access, dynamically constructed. Example: Marital Status (Fig. 2) Page is constructed dynamically Displays currently selected samples 16

17 Fig. 2 (cont d) Integrated IPUMS-I Metadata: Codes and Frequencies Detailed, Case-Count Count View 2 rules: 1. Retain details 2. Harmonize everything Page is constructed dynamically y Displays currently selected samples 17

18 Integrated IPUMS-I Metadata: Enumeration text View text in English for any combination of countries and censuses. 2 documents: First the form Page is constructed dynamically Displays currently selected samples 18

19 Integrated IPUMS-I Metadata: Enumeration text View text in English for any combination of countries and censuses. 2 documents: First, the form; then, the enumeration instructions scroll down for more Page is constructed dynamically Displays currently selected samples 19

20 3. Access to microdata is restricted to academics and policy makers, who must agree to stringent conditions of use Despite the P in IPUMS, access to the microdata is restricted. See paper, pp

21 NSI 1. NSI entrusts census metadata and anonymized microdata to MPC MPC MPC integrates metadata and confidentializes microdata samples IPUMS- International NSI 62+ IPUMS-International International manages access and entrusts researchers with custom- tailored <ddi>, SAS, STATA, and SPSS metadata and microdata extracts for any combination of countries, censuses, sub-populations, and variables. Trusted researcher Trusted researcher Trusted researcher receives customized extracts 21

22 User Registration, conditions of use license Browse (metadata) and Select Data Download Data Extract (and <ddi> codebook) Source documents (forms, instruction manuals) Link to Official Statistical Agency home pages Bibliography: view cites, link to publications 22

23 Thwarting intruders by legal and administrative procedures» Usage is restricted to bona-fide researchers who agree to stringent conditions of use to protect statistical confidentiality» 1,100 word application form; <5,300 word Facebook policy» Agree to 8 specific conditions of use» Supply extensive personal and institution details» Identify Portion your employer s of the IPUMS-I Office for I application Protection of Human form Subject, IRB, requesting etc. information on institutional affiliation» Describe research detailing need for access» Rogue intruders face legal and institutional sanctions» University attorney s office is obligated to initiate sanctions against both individual and the institution similar to NIH probationary status

24 Conditions of use: must agree to each one--no exceptions Data must not be redistributed without authorization. All data extracted from the IPUMS-International database are intended solely for the use of the licensee. Under IPUMS-International agreements with collaborating agencies, redistribution of the data to third parties is prohibited. Each member of a research team using the data must apply for access and be licensed individually. The microdata are intended only for scholarly research and educational purposes. These microdata are provided for the exclusive purposes of teaching and scholarly research, and may not be used for any other purposes without explicit written approval from the relevant official statistical authority. Commercial use and redistribution of the microdata is strictly prohibited. Users are prohibited from using microdata acquired from the Integrated t Public Use Microdata Series International ti or other authorized ddistributors ib t in the pursuit of any commercial or income-generating venture either privately, or otherwise. Use of the microdata must follow strict rules of confidentiality. Users will maintain the confidentiality of persons and households. Any attempt to ascertain the identity of persons or households from the microdata is prohibited. Alleging that a person or household has been identified in these data is also prohibited. Statistical results that might reveal the identity of persons or entities may not be reported or published in any form. The microdata must always be safely secured. Users will implement security measures to prevent unauthorized access to microdata acquired from Integrated Public Use Microdata Series International, its partners or authorized distributors. Upon the completion of this research, data may be retained only if they can be safely secured. If security cannot be guaranteed, the microdata must be destroyed. Scholarly publications are permitted, and must be cited appropriately. The publishing of research results based on IPUMS-International microdata is permitted in communications such as scholarly papers, journals and the like. The authors of these communications are required to cite Integrated Public Use Microdata Series-International and the relevant official statistical authority as the source of the microdata, and to indicate that the results and views expressed are those of the author. Users are requested to provide the IPUMS-International staff with a full citation for any publications resulting from their work with these data. Any violation of this license agreement will result in disciplinary action, including possible loss of employment. Violation of this agreement will lead to revocation of this license, recall of all microdata acquired, a motion of censure to the relevant professional organization(s) and civil prosecution under national or international statutes, at the discretion of the Regents of the University of Minnesota and the official statistical agencies. Sanctions likewise may be taken against the institution with which the violator is affiliated. User agrees to notify ipums@pop.umn.edu regarding errors in the data.

25 Africa Mirror Site: 25

26 Europe Mirror Site: 26

27 4. Dissemination and usage See Tables 1 and 2 (or card handout and brochure): 1. Card for list of available samples and usage statistics 2. Brochure for list of participating statistical offices, census microdata available, and info on IPUMS-InternationalInternational 27

28 Top 20 institutions using IPUMS-I I(Table2) 1 University of Michigan Columbia University Universitat de Barcelona, Spain Harvard University Inter - American Development Bank Arizona State University National University of Singapore, Singapore World Bank University of California - Berkeley Universidade idd Federal de Minas Gerais, Brazil University of Chicago Universidad del Valle, Colombia Institute for Health Metrics & Evaluation Princeton University University of Wisconsin - Madison Brown University University it of Vienna, Austria University of Pittsburgh University of Delaware El Colegio de México, México

29 Extracts by researcher s place of identity: Top 20 (see table 3 for complete list) Samples Extracts Extracted Institutions Place of Identity (N) (mean) (N) United States 14, France Spain United Kingdom Canada Colombia Brazil Mexico Singapore Germany Austria Italy Chile Argentina Switzerland Belgium Australia Netherlands China Japan

30 3 new initiatives: SDA (online data tabulator), IPUMS-I I Research Data Center, TerraPop 30

31 NSI 1. NSI entrusts census metadata and anonymized microdata to MPC MPC MPC integrates metadata and confidentializes microdata samples IPUMS- International Online Data Tabulation System NSI 62+ SDA tabulation and analysis coupled with dynamic metadata navigation tool. Trusted researcher Trusted researcher Online data tabulation and analysis Restricted to trusted researchers 31

32 NSI 1. NSI 62+ NSI entrusts anonymized full- count, long-form, or other highest precision census microdata and metadata to MPC MPC MPC integrates metadata and microdata IPUMS- International Virtual RDC Researchers analyze integrated microdata remotely using software in the Virtual RDC Certified Institution 1 Certified Institution Certified Institution N. Trusted researcher Trusted researcher IPUMS-International International Research Data Center (pilot to launch in 2013) 32

33 International Research Data Center IRDC contents: All microdata and metadata available through IPUMS-I Plus higher precision microdata and greater geographic detail entrusted by NSI partners MPC responsible for: It Integrating ti and maintaining itii microdata dt and metadata tdt Managing international access to microdata Vetting trusted sites, users, and user outputs Trusted sites maintain their facilities NSIs: Manage (or delegate) access to national (or mirror) site 33

34 NSI 1. NSI entrusts census metadata and anonymized microdata to MPC MPC MPC integrates metadata and confidentializes microdata samples IPUMS- International GIS NSI 62+ GIS aggregate census data and boundary files coupled with dynamic metadata navigation tool. Trusted researcher Trusted researcher TerraPop: aggregated g census data, climate, etc. coupled with boundary files 34

35 Conclusion: sustaining IPUMS-InternationalInternational» Sustained commitments» University of Minnesota» NSO partners (contingent upon good-will)» Sustained funding» National Science Foundation, the Hubble telescope project of the Social Sciences: 3 series of 5 years grants» National li Institutes of fh Health: 2 series of f10 year regional grants» MPC» Youthful integration team» Ambassador must retire in a year or 3» Invitation to use and cooperate» Researchers: Please use the microdata» NSO partners: Please entrust 2010 round microdata without delay NSO partners: Please entrust 2010 round microdata without delay 35

36 The IPUMS-International International ti lt team (includes National Science Foundation Board) Steven Ruggles, inventor of IPUMS, Professor of History, and Director of the Minnesota Population Center (Not present: some computer gurus, researchers, research assistants, civil service employees, and others who were not at the NSF Board meeting)

37 Dissemination [means] opening up the value inherent in our data --Walter Radermacher (President, Eurostat) and Pieter Everaers (Director, Eurostat) IPUMS-International: Free, Worldwide Microdata Access Now for Censuses of 62 Countries--80 by 2015 IPUMS opens up the value inherent in census microdata. for the 2010 round for the 2000, 1990 Session and STS065 earlier The rounds Future of (where Microdata microdata Access exist) 58 th International ti Statistical ti ti Institute, t Dublin, Ireland, 26 August, 2011 And for many countries. * * * Robert McCaa, Steven Ruggles, Matt Sobek and Wendy L. Thomas rmccaa@umn.edu for additional details, please see: 37

38 Thank you 38