idigbio TCN IT Needs Assessment Results José Fortes, Shari Ellis, Andréa Matsunaga

Size: px

Start display at page:

Download "idigbio TCN IT Needs Assessment Results José Fortes, Shari Ellis, Andréa Matsunaga"

Marshall Hunt
5 years ago
Views:

1 idigbio TCN IT Needs Assessment Results José Fortes, Shari Ellis, Andréa Matsunaga idigbio Summit November 29th, 2011

2 Overall Statistics 15-question survey of institutions in 3 funded TCNs 30 returned surveys (13+8+9) Significant variety in the types and amount of data being collected, storage requirements, and the software tools and services being utilized Three areas where the TCNs may readily look to idigbio for guidance and support: Archival storage Feedback on the data Computationally intensive transformations 2

3 Type of Data Types of Data Collected Most collected: Label image and data, specimen images, geo-reference, and annotations Types of data to be collected Other types Geo-reference Stitched image Video Sound Annotations Label data Specimen image Label image Collecting Now Plan on Collecting Not Sure Number of Respondents 3

4 Metadata Collected Most often expected to be collected: Operator or data enterer GUID (globally persistent unique identifier) Other metadata included: Source of funding Instrument Light source Copyright Date of entry Institution Collection permits Accession data None (2) TBD (4) 4

5 Types of Tools Tools Used or Planned by TCNs Label Capture: OCR, OCR+NLP, Tesseract, SALIX, HERBIS Image manipulation: Capture One (2), ImageMagick, Gigapan (2), DSLR camera software (2), Epson scanner software Data Management: MS Excel (4), MySQL (1), MS SQL (1), FileMaker (2), MS Access (4), KE Emu (2), Accumulatus (1), Silver Biology (1), Specify (9), Arctos/Oracle (1), Symbiota (2) Geo-referencing : BioGeomancer (4), GEOLocate (3), Google Earth Label capture Tools Used or Planned Image manipulation Data Management Geo-referencing Number of Responses 5

6 Type of Storage Disk Storage Types Most common for primary storage, archival storage and dissemination: Central server disks Types of storage to be used Other Cloud storage Tape storage Optical storage Dissemination Archival Primary Central server disk Workstation disk Number of Respondents 6

7 Tools for Primary and Archival Data Primary: Data Management + user interface (desktop or web) Arctos, Dspace, Fedora Commons, KE Emu, Silver Biology, Silver Collections, Symbiota Data Management: FileMaker Data processing: Medici Raw data NAS, FreeNAS, Delta Cloud API (AWS/Rackspace/NCSA) Archival: CrashPlan Backup Exec Institutional SAN Off-site server and own scripts Expecting recommendations Unknown To be determined 7

8 Storage Requirements Large variation: Raw storage: 100GB ~ 50TB Number of specimens: 1,000 ~ 56 million Number of digital objects: 5,000 ~ 56 million Digital object per specimen: ~ 200 Sufficient storage: 13 yes, 10 maybe, and 3 no Storage Requirements Gigabytes # specimen objects # digital objects Number of Respondents 8

9 Replication Agreements Provider DataONE MorphBank TACC GBIF EOL Other Providers Tools, Services, Comments Why? agreed to take all images, than problems arose with too many images too fast agreed to take all images, than problems arose with too many images too fast DiGIR (2 respondents) DwC-A Files KE Emu Data (3 respondents) Maybe Species descriptions DiGIR SYMBIOTA Discoverlife.org OAI idigbio 9

10 Protocol and Standards Number of Protocol Respondents DiGIR 9 TAPIR 3 ODBC 3 JSON 2 Other Symbiota My University services Silver Image XML-RPC XML Number of Standards Respondents SQL 9 XML 8 DwC 5 KML 2 RDF 2 Other My university standards Darwin Core Archive and custom text 10

11 Data Access Desktop: SQL clients (3), Specify (3), own Web-based: Discover Life, GBIF (2), HUBZero, Silver Collection (2), Symbiota (3), own (8), TBD (2) Feedback mechanism: or comment forms (4), idigbio recommendation (2), own suite of tools (1), TBD (10) Access Control: No restrictions (6); sensitive locality data blocked (6); rolebased access (5) and TBD (4) 11

12 Type of Data Transformation Data Processing Most cited: geo-referencing Others: GIS analyses, imaging a subset of the collections, citizen scientists, custom data cleaning, and natural language processing From: In-house (7), TCN collaboration (5), idigbio (3), not an issue (2), TBD (4) others Data Transformations or Planned geo-referencing query MapReduce Currently Perform Plan to Perform image manipulation OCR Number of Respondents 12

13 Questions? Thank you! 13

Relation between Geospatial information projects related to GBIF

Relation between Geospatial information projects related to GBIF Synthesys 3.6-Synthesys 3.7-GBIF.DE- BioGeomancer The most up to date work can always be found at: http://www.biogeografia.com/synthesys