Planning for a post-equinox world

Size: px

Start display at page:

Download "Planning for a post-equinox world"

Tracy Williamson
5 years ago
Views:

1 Planning for a post-equinox world Vince Gray & Caroline Patenaude DLI National Training Day June 2,

2 Why and when is Equinox being shut down? Equinox is a partnership between Western and Montreal Don t have resources to update the Equinox system to new software platform Microsoft software support ends : the Equinox server will be shut down then No files will be loaded after Local availability of metadata and data in any fashion after next June is unknown: hence, a matter of some urgency

3 What Equinox has DLI files Microdata, aggregated data, geospatial files (including.e00 files converted to shape files) Links to documentation Enhanced data and metadata CIHI Discharge Abstract Database with flag variables, counts, data that matches documentation Fixed missing values declarations and ambiguous codes Variable labels (e.g., Self-assessed amount of stress in respondent's life rather than Thinking about the amount of stress in your life, would you ) Non-DLI data & geospatial files E.g., DMTI Spatial files, selected ICPSR files, public files, private (UWO-only)

4 How DLI files were added DLI releases a file; file is downloaded by Western English SPSS code (and/or software programs) created or edited to enhance labels, decimals, missing values, fix coding problems, etc.* Data loaded into SPSS, output files created* Software programs run to extract basic metadata for Equinox* Data loaded into SQL, basic metadata loaded into Equinox* Other metadata (e.g., question text, variable universe) extracted from documentation and loaded* English metadata transferred to a second instance of the Equinox database for French metadata entry (English metadata vetted)* French metadata checked for accuracy* French metadata loaded into main Equinox database * Shows a step at which error checking and/or enhancement is done

5 Equinox design Fully bilingual database structure with parallel French and English fields for records: all files do not have bilingual metadata loaded Delivers SPSS, SAS, and Stata syntax files Delivers HTML documentation specific to dataset retrieved Delivers ASCII tab-delimited data in either decimal point or decimal comma format Metadata can be exported as field-delimited ASCII text files, XML, or custom reports DDI-compatible, but not compliant (native)

6 Canadian data delivery services Equinox 46 universities and colleges from every province in Canada Each subscribing institution must decide how will provide access to data in the future <odesi> 22 institutions: Ontario universities & USask CHASS Subscribers vary by service (e.g., Census Analyzer, SDA) SDA 21 institutions: UofT, Ryerson, York, COPPUL (West), Memorial, Saint Mary s, Moncton, Southern Alberta Institute of Technology LANDRU Institutions: Calgary, Alberta,? Abacus SFU, UBC, UNBC, UVic

7 What is DLI doing about Equinox? EAC was made aware of Equinox situation at April 2014 meeting EAC formed DLI working group tasked to determine what and if materials housed in Equinox can be retained for DLI Asking for feedback from community to take back to EAC meeting in fall 2014: committee to draft report

8 DLI Equinox working group members Amber Leahey, Scholars Portal Caroline Patenaude, Montreal Chantal Ripp, Statistics Canada Jane Fry, Carleton Michelle Edwards, Guelph Sai Chua, Dalhousie Susan Mowers, Ottawa Vince Gray, Western

9 Potential collections to harvest CANSIM cross-classified education files E.g., table : Number of full-time students enrolled in elementary and secondary schools showing age of student (18), by type of institution (5) and by sex (3), for Canada, the Provinces, the Territories, and Department of National Defense Overseas (14), to Basic summary tabulations and/or User summary tapes from pre-2001 censuses DLI-distributed geospatial files (especially those converted from.e00 to.shp) Other data sets with enhanced data and/or metadata (e.g., DAD, many others) Complete files in decimal comma format

10 Excluded from DLI consideration DMTI files ICPSR files Public files (e.g., from York University s Institute for Social Research) UWO-only files

StatsCan IP-based software to deliver data to end users Nesstar Project approved by EAC to develop an enhanced search engine to Nesstar server Could deliver microdata, field-delimited

11 StatsCan IP-based software to deliver data to end users Nesstar Project approved by EAC to develop an enhanced search engine to Nesstar server Could deliver microdata, field-delimited aggregated data (e.g., BSTs/USTs), zipped decimal comma data files with SPSS syntax, Geosuite layers Beyond 20/20 Web Server To be demonstrated at Boot Camp Could deliver downloadable geospatial layers

12 Other destinations for Equinox data and/or metadata <odesi> group is working on examining collection to see what might be harvested Happy to working with other data providers (e.g., CHASS, Abacus, LANDRU) to provide metadata, data and/or geospatial files Possible DMTI work-around (82.8 Gb)

13 Western and Montreal want to pass Equinox s content to the Canadian data community who wants to take it? (

14 Questions to consider What should be the role of DLI in preserving Equinox content? Should Equinox files be archived by DLI? What do you mean by archive: allow DLI contact download only or also allow end user access? Which version(s) of data/metadata should be made available (and to whom): As released by DLI? As enhanced by Equinox? Both formats? Does it depend on the file?

15 Additional questions What standards of data curation should be established for DLI collection? Is this a role for the DLI community? How do we (or do we) establish better practices for managing and curating DLI data moving forward? What do you do to DLI data to make it more useful to your users? Is it important to have a separate place to perpetually access DLI data after it is disseminated by DLI? (e.g., CANSIM XCL, rebased LFS, etc.)

Queen s University Library. Research Data Management (RDM) Workflow

Queen s University Library. Research Data Management (RDM) Workflow Queen s University Library Research Data Management (RDM) Workflow Alexandra Cooper Jeff Moon Data Services, Open Scholarship Services Queen s University Library February 2018 Table of Contents RDM Planning...