Scientific Data Management for the ATP 3. Edward J. Wolfrum, Eric Knoshaug, Lieve Laurens, Valerie Harmon, John A. McGowen

Size: px
Start display at page:

Download "Scientific Data Management for the ATP 3. Edward J. Wolfrum, Eric Knoshaug, Lieve Laurens, Valerie Harmon, John A. McGowen"

Transcription

1 Scientific Data Management for the ATP 3 Edward J. Wolfrum, Eric Knoshaug, Lieve Laurens, Valerie Harmon, John A. McGowen

2 The ATP 3 UFS Scientific Data Management in 1 slide The ATP 3 team uses commercial and open-source tools to collect, store, audit, correct, & retrieve data about samples Existence & Location (what & where they are) Composition (what they look like) History (how were they produced) Standardizing how we collect and store these data across all ATP 3 sites is absolutely necessary for success Each site uses standardized spreadsheets to collect defined primary data about samples & experiments A data quality team reviews these spreadsheets, and works with site personnel to correct errors; these spreadsheets hold the raw data The UFS data are available to the public Curated datasets are available for download at OpenEI.org A manuscript describing these data is in preparation for Nature Scientific Data

3 Outline The Scientific Data Management Workflow Data repository - OpenEi.org What are the data? Data collection & curation Using the data Conclusions

4 Scientific Data Management (n.) Scientific Data Management (SDM) is a series of discrete activities we performed to curate and store the data from ATP 3 experiments The goal is to ensure that robust and complete datasets are collected and archived to allow useful review and analysis by internal and external stakeholders SDM does not include generating the data, which is a big challenge itself!

5 The SDM Workflow (1/2) 1. DATA COLLECTION Each site records pond operation, weather, and algal compositional analysis data into standardized Excel spreadsheets 2. DATA VALIDATION Spreadsheets (1 for each site-experiment combination) are stored in Dropbox and reviewed by QA/QC team. Data include pond operational, weather, instrumental, and harvest data.

6 The SDM Workflow (2/2) 3. DATA EXTRACTION R scripts extract the data and create ASCII files for pond operational data, instrumental & weather data, and summary harvest data the flat files 4. DATA QA/QC These flat files are then examined visually by the ATP 3 data analysis team. Errors are corrected in the original spreadsheet 5. SHARING The datasets and experimental protocols are publicly available on the Open Energy Information network:

7 OpenEI.org Using the existing infrastructure of OpenEI.org provides a rapid, robust, and low-cost solution for making the ATP 3 datasets public

8 OpenEI.org -

9 Exactly what data are available? Pond Operational Data Over 15,800 unique time points Instrumentation Data Over 1,084,000 time points & daily averaged data Weather Data Over 217,500 unique time points & daily averaged data Harvest Yield Data 2150 discrete harvests

10 Spreadsheet Configuration Production data are collected in a spreadsheet - this is the primary unit of experimental information for the ATP 3 Consortium CONUNDRUM FLEXIBILITY vs. RAPID DEVELOPMENT The form of the algae pond production spreadsheet was adapted from experience at ASU Compositional analysis data are collected in a separate spreadsheet, adapted from experience at NREL All spreadsheets are under version/change control The spreadsheets have evolved over the course of the project; our SDM applications have had to evolve as well

11 Structuring the spreadsheets simply Primary Data Validation We locked down the spreadsheets and added data validation functionality to the Pond Data Discrete (PDD) page in the Excel spreadsheets to ensure proper data types and values (e.g., no ph values of really high or -4 ). Not completely robust, but solved some problems Real-time Visualization We added graphs in the collection spreadsheets and placed active spreadsheets in a shared Dropbox folder to allow each site (and the ATP 3 leadership team) to monitor the results of experiments on a daily basis

12 Spreadsheet PDD Page (1/3)

13 Spreadsheet PDD Page (2/3)

14 Spreadsheet PDD Page (3/3)

15 The challenges of data curation All primary data from the UFS spreadsheets were plotted Obviously bad or missing data removed What is an outlier?

16 Using ATP3 UFS Data (1/2) Once the primary data are curated and stored, they can be examined directly, or used to calculate things like productivity & algal growth rates 16

17 Using ATP 3 UFS Data (2/2) Cultivation Data from the ATP 3 are used in NREL s Algae State of Technology (SOT) Assessments Productivity and Growth Rate calculations can be done in multiple ways Slope of AFDW vs. time Intermediate Harvests Across Multiple Harvests We calculated the multiple harvest yield productivity using 3 different time frames to simulate different production strategies (Reference: Knoshaug, et. al. NREL/TP ) 17

18 Conclusions The Scientific Data Management process used by the ATP3 Consortium evolved over the course of the project We used open-source tools to review and QA/QC data from multiple sites and multiple experiments over multiple years Datasets that have been QA/QC d are available on OpenEI.org, a DOEfunded public-facing website We (again) emphasize that Scientific Data Management is a set of processes, not a set of software tools. If you want to manage scientific data, you need a manager; this is a discrete and critical task, particularly when the data are being generated from multiple sources

19 Questions?

20 TRUTHS OF SCIENTIFIC DATA MANAGEMENT 1. The primary unit of data collection for most researchers & projects is the spreadsheet 2. The data types being collected will likely change over the course of a project 3. It is possible to minimize these changes by careful consideration at the beginning of the project, but they cannot be eliminated 4. Whatever system and workflow used for scientific data management must be able to respond to these changes 5. The difficulty in coordinating these changes among research partners goes up as the exponential of the number of partners 6. Data curation (QA/QC) to identify and correct data entry errors is a separate activity from data collection/storage and is equally important 7. Scientific Data Management is not a collection of software tools, it is a process that can use software tools

MICROSOFT ONLINE (ONEDRIVE) VS G SUITE (GOOGLE DRIVE)

MICROSOFT ONLINE (ONEDRIVE) VS G SUITE (GOOGLE DRIVE) MICROSOFT ONLINE (ONEDRIVE) VS G SUITE (GOOGLE DRIVE) COST ONEDRIVE (MICROSOFT ONLINE) OneDrive offers three different business plans: First option: OneDrive for Business Plan 1 - $5.00/month per user

More information

Quality Assured (QA) data

Quality Assured (QA) data Quality Assured (QA) data Towards DOI quality of data generated at the UFZ Mark Frenzel (Ecologist) & Thomas Schnicke (IT) DataCite / Helmholtz Open Science Workshop Leipzig, 12.01.2016 QA + DOI: Best

More information

Streamline the Chromatographic Method Validation Process using Empower 2 Method Validation Manager

Streamline the Chromatographic Method Validation Process using Empower 2 Method Validation Manager Streamline the Chromatographic Method Validation Process using Empower 2 Method Validation Manager Table of Contents Introduction... 3 Optimize your chromatographic method validation workflow... 4 Efficiently

More information

Data Curation Profile Water Flow and Quality

Data Curation Profile Water Flow and Quality Data Curation Profile Water Flow and Quality Profile Author Profile Author Institution Name Contact J. Carlson N. Brown Purdue University J. Carlson, jrcarlso@purdue.edu Date of Creation October 27, 2009

More information

An Experimentation Workbench for Replayable Networking Research

An Experimentation Workbench for Replayable Networking Research An Experimentation Workbench for Replayable Networking Research Eric Eide,, Leigh Stoller, and Jay Lepreau University of Utah, School of Computing NSDI 2007 / April 12, 2007 Repeated Research A scientific

More information

Data Curation Profile: Agronomy / Grain Yield

Data Curation Profile: Agronomy / Grain Yield Profile Author Author s Institution Contact Researcher(s) Interviewed Researcher s Institution Data Curation Profile: Agronomy / Grain Yield Marianne Stowell Bracke Purdue University Marianne Stowell Bracke

More information

An Experimentation Workbench for Replayable Networking Research

An Experimentation Workbench for Replayable Networking Research An Experimentation Workbench for Replayable Networking Research Eric Eide, Leigh Stoller, and Jay Lepreau Repeated Research A scientific community advances when its experiments are repeated University

More information

Digital Preservation: How to Plan

Digital Preservation: How to Plan Digital Preservation: How to Plan Preservation Planning with Plato Christoph Becker Vienna University of Technology http://www.ifs.tuwien.ac.at/~becker Sofia, September 2009 Outline Why preservation planning?

More information

A Digital Preservation Roadmap for Public Media Institutions

A Digital Preservation Roadmap for Public Media Institutions NDSR Project: A Digital Preservation Roadmap for Public Media Institutions Goal Summary Specific Objectives New York Public Radio (NYPR) seeks an NDSR resident to aid in the creation of a robust digital

More information

Managing Superfund Field Data

Managing Superfund Field Data Managing Superfund Field Data Joe Schaefer Environmental Response Team 24 th NARPM Training Program Objective: Improve the Information Currency of Superfund u Translate the work that happens on your site

More information

Vector Xpression 3. Speed Tutorial: III. Creating a Script for Automating Normalization of Data

Vector Xpression 3. Speed Tutorial: III. Creating a Script for Automating Normalization of Data Vector Xpression 3 Speed Tutorial: III. Creating a Script for Automating Normalization of Data Table of Contents Table of Contents...1 Important: Please Read...1 Opening Data in Raw Data Viewer...2 Creating

More information

Chapter 3: Rate Laws Excel Tutorial on Fitting logarithmic data

Chapter 3: Rate Laws Excel Tutorial on Fitting logarithmic data Chapter 3: Rate Laws Excel Tutorial on Fitting logarithmic data The following table shows the raw data which you need to fit to an appropriate equation k (s -1 ) T (K) 0.00043 312.5 0.00103 318.47 0.0018

More information

About Knowledge Convergence. e-infrastructures Austria an interdisciplinary case study concerning research resources and their management

About Knowledge Convergence. e-infrastructures Austria an interdisciplinary case study concerning research resources and their management About Knowledge Convergence e-infrastructures Austria an interdisciplinary case study concerning research resources and their management Paolo Budroni The Munin Conference Tromsø, 27th November 2014 THE

More information

DEVELOPING, ENABLING, AND SUPPORTING DATA AND REPOSITORY CERTIFICATION

DEVELOPING, ENABLING, AND SUPPORTING DATA AND REPOSITORY CERTIFICATION DEVELOPING, ENABLING, AND SUPPORTING DATA AND REPOSITORY CERTIFICATION Plato Smith, Ph.D., Data Management Librarian DataONE Member Node Special Topics Discussion June 8, 2017, 2pm - 2:30 pm ASSESSING

More information

Research Elsevier

Research Elsevier Research Data @ Elsevier From generation through sharing and publishing to discovery IJsbrand Jan Aalbersberg SVP Journal and Data Solutions NDS, Boulder - June 12, 2014 Contributors: Anita de Waard Hylke

More information

Data Management Plan: OR Mooring - Ocean Acidification related measurements (Taken from NOAA Data Sharing Template and adapted for IOOS Certification)

Data Management Plan: OR Mooring - Ocean Acidification related measurements (Taken from NOAA Data Sharing Template and adapted for IOOS Certification) I. Type of data and information created 1. What data will you collect or create in the research? Contextual statement describing what data are collected and relevant URL (IOOS Certification, f 2) Hales

More information

Data publication and discovery with Globus

Data publication and discovery with Globus Data publication and discovery with Globus Questions and comments to outreach@globus.org The Globus data publication and discovery services make it easy for institutions and projects to establish collections,

More information

How to use the SATURN Observation Network: Endurance Stations Site: Table of Contents

How to use the SATURN Observation Network: Endurance Stations Site: Table of Contents How to use the SATURN Observation Network: Endurance Stations Site: Table of Contents Preface... 2 Introduction to the SATURN Interface... 3 Fixed station user interface... 5 The Recent Tab... 6 Reading

More information

Data Curation Profile Plant Genetics / Corn Breeding

Data Curation Profile Plant Genetics / Corn Breeding Profile Author Author s Institution Contact Researcher(s) Interviewed Researcher s Institution Katherine Chiang Cornell University Library ksc3@cornell.edu Withheld Cornell University Date of Creation

More information

Starting small to go Big: Building a Living Database

Starting small to go Big: Building a Living Database Starting small to go Big: Building a Living Database Michael Sabbatino 1,2, Baker, D.V. Vic 3,4, Rose, K. 1, Romeo, L. 1,2, Bauer, J. 1, and Barkhurst, A. 3,4 1 US Department of Energy, National Energy

More information

Enterprise Challenges of Test Data Size, Change, Complexity, Disparity, and Privacy

Enterprise Challenges of Test Data Size, Change, Complexity, Disparity, and Privacy Size, Change, Complexity, Disparity, and Privacy For simple applications, representative test data can be relatively easy What if you are testing enterprise-scale applications? In enterprise data centers,

More information

Using Excel for Graphical Analysis of Data

Using Excel for Graphical Analysis of Data Using Excel for Graphical Analysis of Data Introduction In several upcoming labs, a primary goal will be to determine the mathematical relationship between two variable physical parameters. Graphs are

More information

University at Buffalo's NEES Equipment Site. Data Management. Jason P. Hanley IT Services Manager

University at Buffalo's NEES Equipment Site. Data Management. Jason P. Hanley IT Services Manager University at Buffalo's NEES Equipment Site Data Management Jason P. Hanley IT Services Manager Structural Engineering and Earthquake Simulation Laboratory, Department of Civil, Structural and Environmental

More information

Data Curation Profile Movement of Proteins

Data Curation Profile Movement of Proteins Data Curation Profile Movement of Proteins Profile Author Institution Name Contact J. Carlson Purdue University J. Carlson, jrcarlso@purdue.edu Date of Creation July 14, 2010 Date of Last Update July 14,

More information

ISO Self-Assessment at the British Library. Caylin Smith Repository

ISO Self-Assessment at the British Library. Caylin Smith Repository ISO 16363 Self-Assessment at the British Library Caylin Smith Repository Manager caylin.smith@bl.uk @caylinssmith Outline Digital Preservation at the British Library The Library s Digital Collections Achieving

More information

ES-2 Lecture: Fitting models to data

ES-2 Lecture: Fitting models to data ES-2 Lecture: Fitting models to data Outline Motivation: why fit models to data? Special case (exact solution): # unknowns in model =# datapoints Typical case (approximate solution): # unknowns in model

More information

Information Technology Branch Organization of Cyber Security Technical Standard

Information Technology Branch Organization of Cyber Security Technical Standard Information Technology Branch Organization of Cyber Security Technical Standard Information Management, Administrative Directive A1461 Cyber Security Technical Standard # 1 November 20, 2014 Approved:

More information

Digital The Harold B. Lee Library

Digital The Harold B. Lee Library Digital Preservation @ The Harold B. Lee Library CIMA 23 May 2013 How we got here? 1. Understanding Digital Preservation 2. Search for Content 3. Maintain Optical Disc Storage 4. In House Preservation

More information

Writing a Data Management Plan A guide for the perplexed

Writing a Data Management Plan A guide for the perplexed March 29, 2012 Writing a Data Management Plan A guide for the perplexed Agenda Rationale and Motivations for Data Management Plans Data and data structures Metadata and provenance Provisions for privacy,

More information

BUSINESS-BASED VALUE IN AN MDR

BUSINESS-BASED VALUE IN AN MDR MERCK METADATA REPOSITORY: BUSINESS-BASED VALUE IN AN MDR A. Brooke Hinkson Manori Turmel Karl Konrad PhUSE Connect Conference, Raleigh NC, 4-6 June 2018 2 Business Problems to Address Current information

More information

Data Curation Profile Human Genomics

Data Curation Profile Human Genomics Data Curation Profile Human Genomics Profile Author Profile Author Institution Name Contact J. Carlson N. Brown Purdue University J. Carlson, jrcarlso@purdue.edu Date of Creation October 27, 2009 Date

More information

Strategies for Sound Internet Measurement

Strategies for Sound Internet Measurement Strategies for Sound Internet Measurement Vern Paxson Presented by Hossein Falaki Vern Paxson M.S. and Ph.D. degrees Berkeley Staff scientist at the Lawrence Berkeley National Laboratory Founder of the

More information

How a Metadata Repository enables dynamism and automation in SDTM-like dataset generation

How a Metadata Repository enables dynamism and automation in SDTM-like dataset generation Paper DH05 How a Metadata Repository enables dynamism and automation in SDTM-like dataset generation Judith Goud, Akana, Bennekom, The Netherlands Priya Shetty, Intelent, Princeton, USA ABSTRACT The traditional

More information

Open Software Standards for Next- Generation Community Satellite Software Packages June 2017

Open Software Standards for Next- Generation Community Satellite Software Packages June 2017 Atmospheric and Environmental Research www.aer.com Lexington, MA 2017 IMAP/ CSPP Users Group Meeting Open Software Standards for Next- Generation Community Satellite Software Packages June 2017 David Hogan

More information

Dynamic, Rule-based Quality Control Framework for Real-time Sensor Data

Dynamic, Rule-based Quality Control Framework for Real-time Sensor Data Dynamic, Rule-based Quality Control Framework for Real-time Sensor Data Wade Sheldon Georgia Coastal Ecosystems LTER University of Georgia Introduction Quality Control of high volume, real-time data from

More information

DATA-SHARING PLAN FOR MOORE FOUNDATION Coral resilience investigated in the field and via a sea anemone model system

DATA-SHARING PLAN FOR MOORE FOUNDATION Coral resilience investigated in the field and via a sea anemone model system DATA-SHARING PLAN FOR MOORE FOUNDATION Coral resilience investigated in the field and via a sea anemone model system GENERAL PHILOSOPHY (Arthur Grossman, Steve Palumbi, and John Pringle) The three Principal

More information

CTL.SC4x Technology and Systems

CTL.SC4x Technology and Systems in Supply Chain Management CTL.SC4x Technology and Systems Key Concepts Document This document contains the Key Concepts for the SC4x course, Weeks 1 and 2. These are meant to complement, not replace,

More information

Application of Clustering Techniques to Energy Data to Enhance Analysts Productivity

Application of Clustering Techniques to Energy Data to Enhance Analysts Productivity Application of Clustering Techniques to Energy Data to Enhance Analysts Productivity Wendy Foslien, Honeywell Labs Valerie Guralnik, Honeywell Labs Steve Harp, Honeywell Labs William Koran, Honeywell Atrium

More information

Microsoft SharePoint Server 2013 Plan, Configure & Manage

Microsoft SharePoint Server 2013 Plan, Configure & Manage Microsoft SharePoint Server 2013 Plan, Configure & Manage Course 20331-20332B 5 Days Instructor-led, Hands on Course Information This five day instructor-led course omits the overlap and redundancy that

More information

Brooke Roecker, Kristen Ward, Chris Mickle, Sarah Wright & Shauna McKellar

Brooke Roecker, Kristen Ward, Chris Mickle, Sarah Wright & Shauna McKellar Brooke Roecker, Kristen Ward, Chris Mickle, Sarah Wright & Shauna McKellar Overview of ICEDM, BMP and path forward Synergies with other organizations White paper overview Data Management Plan Valid Values

More information

Research Electronic Data Capture

Research Electronic Data Capture Research Electronic Data Capture Data Management and Survey Tool Lynn Simpson, MPH Research Analytics & Data Service Manager Partners HealthCare Enterprise Research Infrastructure & Systems Harvard Catalyst

More information

TIBCO StreamBase 10.2 Building and Running Applications in Studio, Studio Projects and Project Structure. November 2017

TIBCO StreamBase 10.2 Building and Running Applications in Studio, Studio Projects and Project Structure. November 2017 TIBCO StreamBase 10.2 Building and Running Applications in Studio, Studio Projects and Project Structure November 2017 TIBCO StreamBase 10 Experience 1. Build a StreamBase 10 Project 2. Run/Debug an StreamBase

More information

Error Analysis, Statistics and Graphing

Error Analysis, Statistics and Graphing Error Analysis, Statistics and Graphing This semester, most of labs we require us to calculate a numerical answer based on the data we obtain. A hard question to answer in most cases is how good is your

More information

CDISC Laboratory Standards Release Notes. for. Base Model Version Schema Version Microbiology Extension Review Version

CDISC Laboratory Standards Release Notes. for. Base Model Version Schema Version Microbiology Extension Review Version CDISC Laboratory Standards Release Notes for Base Model Version 1.0.1 Schema Version 1.0.1 Microbiology Extension Review Version Revision History Date Version Summary of Changes Primary Author 9-September-2003

More information

Using Excel for Graphical Analysis of Data

Using Excel for Graphical Analysis of Data EXERCISE Using Excel for Graphical Analysis of Data Introduction In several upcoming experiments, a primary goal will be to determine the mathematical relationship between two variable physical parameters.

More information

Storage Virtualization. Eric Yen Academia Sinica Grid Computing Centre (ASGC) Taiwan

Storage Virtualization. Eric Yen Academia Sinica Grid Computing Centre (ASGC) Taiwan Storage Virtualization Eric Yen Academia Sinica Grid Computing Centre (ASGC) Taiwan Storage Virtualization In computer science, storage virtualization uses virtualization to enable better functionality

More information

Building a National SGCN Dataset

Building a National SGCN Dataset Building a National SGCN Dataset Moving form 56 Disparate Plans to One Integrated Product U.S. Department of the Interior U.S. Geological Survey Presentation Overview NBII Species Mashup LIVE DEMO Species

More information

Graphical Analysis of Data using Microsoft Excel [2016 Version]

Graphical Analysis of Data using Microsoft Excel [2016 Version] Graphical Analysis of Data using Microsoft Excel [2016 Version] Introduction In several upcoming labs, a primary goal will be to determine the mathematical relationship between two variable physical parameters.

More information

Wade Sheldon. Georgia Coastal Ecosystems LTER University of Georgia CUAHSI Virtual Workshop Field Data Management Solutions

Wade Sheldon. Georgia Coastal Ecosystems LTER University of Georgia   CUAHSI Virtual Workshop Field Data Management Solutions Wade Sheldon Georgia Coastal Ecosystems LTER University of Georgia email: sheldon@uga.edu CUAHSI Virtual Workshop Field Data Management Solutions 01-Oct-2014 Georgia Coastal Ecosystems LTER started in

More information

Feed the Future Innovation Lab for Peanut (Peanut Innovation Lab) Data Management Plan Version:

Feed the Future Innovation Lab for Peanut (Peanut Innovation Lab) Data Management Plan Version: Feed the Future Innovation Lab for Peanut (Peanut Innovation Lab) Data Management Plan Version: 20180316 Peanut Innovation Lab Management Entity The University of Georgia, Athens, Georgia Feed the Future

More information

Improved Database Development using SQL Compare

Improved Database Development using SQL Compare Improved Database Development using SQL Compare By David Atkinson and Brian Harris, Red Gate Software. October 2007 Introduction This white paper surveys several different methodologies of database development,

More information

Appendix: Data Availability Policies & Replication Policies Time of Evaluation: January 2012

Appendix: Data Availability Policies & Replication Policies Time of Evaluation: January 2012 Appendix: Data Availability Policies & Replication Policies Time of Evaluation: January 2012 Table of Contents: Data Availability Policies:... 2 1) American Economic Review:... 2 2) Journal of Political

More information

WHITE PAPER. Operationalizing Threat Intelligence Data: The Problems of Relevance and Scale

WHITE PAPER. Operationalizing Threat Intelligence Data: The Problems of Relevance and Scale WHITE PAPER Operationalizing Threat Intelligence Data: The Problems of Relevance and Scale Operationalizing Threat Intelligence Data: The Problems of Relevance and Scale One key number that is generally

More information

Data Curation Profile Food Technology and Processing / Food Preservation

Data Curation Profile Food Technology and Processing / Food Preservation Data Curation Profile Food Technology and Processing / Food Preservation Profile Author Author s Institution Contact Researcher(s) Interviewed Researcher s Institution Sonia Wade Lorenz & Lisa Zilinski

More information

Tools for Data Management. Research Data Management : Session 3 9 th June 2015

Tools for Data Management. Research Data Management : Session 3 9 th June 2015 Tools for Data Management Research Data Management : Session 3 9 th June 2015 What do we mean by tools for data? A system that automates in some way the process of creating, transforming, analysing, visualising,

More information

Privacy and Security Aspects Related to the Use of Big Data Progress of work in the ESS. Pascal Jacques Eurostat Local Security Officer 1

Privacy and Security Aspects Related to the Use of Big Data Progress of work in the ESS. Pascal Jacques Eurostat Local Security Officer 1 Privacy and Security Aspects Related to the Use of Big Data Progress of work in the ESS Pascal Jacques Eurostat Local Security Officer 1 Current work on privacy and ethics in Big data Privacy Confidentiality

More information

Electronic Records Archives: Philadelphia Federal Executive Board

Electronic Records Archives: Philadelphia Federal Executive Board Electronic Records Archives: Philadelphia Federal Executive Board L. Reynolds Cahoon Assistant Archivist for HR and IT and Chief Information Officer 18 March 2004 Agenda (The Mission) Electronic Records

More information

CA Security Management

CA Security Management CA Security CA Security CA Security In today s business environment, security remains one of the most pressing IT concerns. Most organizations are struggling to protect an increasing amount of disparate

More information

Fourier Transforms and Signal Analysis

Fourier Transforms and Signal Analysis Fourier Transforms and Signal Analysis The Fourier transform analysis is one of the most useful ever developed in Physical and Analytical chemistry. Everyone knows that FTIR is based on it, but did one

More information

Excel For Algebra. Conversion Notes: Excel 2007 vs Excel 2003

Excel For Algebra. Conversion Notes: Excel 2007 vs Excel 2003 Excel For Algebra Conversion Notes: Excel 2007 vs Excel 2003 If you re used to Excel 2003, you re likely to have some trouble switching over to Excel 2007. That s because Microsoft completely reworked

More information

Tips and Guidance for Analyzing Data. Executive Summary

Tips and Guidance for Analyzing Data. Executive Summary Tips and Guidance for Analyzing Data Executive Summary This document has information and suggestions about three things: 1) how to quickly do a preliminary analysis of time-series data; 2) key things to

More information

SAP Agile Data Preparation Simplify the Way You Shape Data PUBLIC

SAP Agile Data Preparation Simplify the Way You Shape Data PUBLIC SAP Agile Data Preparation Simplify the Way You Shape Data Introduction SAP Agile Data Preparation Overview Video SAP Agile Data Preparation is a self-service data preparation application providing data

More information

Introduction to using the CALERIE Public Use Database

Introduction to using the CALERIE Public Use Database Introduction to using the CALERIE Public Use Database Outline Where to find information Data download instructions Database installation Raw and Analysis datasets Important Data usage notes Major data

More information

USE OF BASELINES. Definition, reasons and examples. RD.11/######.#

USE OF BASELINES. Definition, reasons and examples.  RD.11/######.# USE OF BASELINES Definition, reasons and examples www.ricardo.com 2 Agenda Introduction Principles and reasons for use Examples Advanced features Using Baselines Baseline Views 3 BASELINES Introduction

More information

Construction Change Order analysis CPSC 533C Analysis Project

Construction Change Order analysis CPSC 533C Analysis Project Construction Change Order analysis CPSC 533C Analysis Project Presented by Chiu, Chao-Ying Department of Civil Engineering University of British Columbia Problems of Using Construction Data Hybrid of physical

More information

Supporting Data Stewardship Throughout the Data Life Cycle in the Solid Earth Sciences

Supporting Data Stewardship Throughout the Data Life Cycle in the Solid Earth Sciences Supporting Data Stewardship Throughout the Data Life Cycle in the Solid Earth Sciences Vicki L. Ferrini, Kerstin A. Lehnert, Suzanne M. Carbotte, and Leslie Hsu Lamont-Doherty Earth Observatory What is

More information

Software Prototyping Animating and demonstrating system requirements. Uses of System Prototypes. Prototyping Benefits

Software Prototyping Animating and demonstrating system requirements. Uses of System Prototypes. Prototyping Benefits Software Prototyping Animating and demonstrating requirements Ian Sommerville 1995/2000 (Modified by Spiros Mancoridis 1999) Software Engineering, 6th edition. Chapter 8 Slide 1 Uses of System Prototypes

More information

Lidar and GIS: Applications and Examples. Dan Hedges Clayton Crawford

Lidar and GIS: Applications and Examples. Dan Hedges Clayton Crawford Lidar and GIS: Applications and Examples Dan Hedges Clayton Crawford Outline Data structures, tools, and workflows Assessing lidar point coverage and sample density Creating raster DEMs and DSMs Data area

More information

Scripting without Scripts: A User-Friendly Integration of R, Python, Matlab and Groovy into KNIME

Scripting without Scripts: A User-Friendly Integration of R, Python, Matlab and Groovy into KNIME Scripting without Scripts: A User-Friendly Integration of R, Python, Matlab and Groovy into KNIME Felix Meyenhofer Technology Development Studio 3. March 2011 4th KNIME Users Group Meeting and Workshop

More information

Harmonizing the data collection and data entry applications for longitudinal and cross-sectional surveys in social science: A metadata driven approach

Harmonizing the data collection and data entry applications for longitudinal and cross-sectional surveys in social science: A metadata driven approach Harmonizing the data collection and data entry applications for longitudinal and cross-sectional surveys in social science: A metadata driven approach Benjamin D Clark and Gayatri Singh Aim of the paper

More information

Data Curation Profile Cornell University, Biophysics

Data Curation Profile Cornell University, Biophysics Data Curation Profile Cornell University, Biophysics Profile Author Dianne Dietrich Author s Institution Cornell University Contact dd388@cornell.edu Researcher(s) Interviewed Withheld Researcher s Institution

More information

Developing a Research Data Policy

Developing a Research Data Policy Developing a Research Data Policy Core Elements of the Content of a Research Data Management Policy This document may be useful for defining research data, explaining what RDM is, illustrating workflows,

More information

Real World Data Governance- Part 1

Real World Data Governance- Part 1 Real World Data Governance- Part 1 Day in the Life of a Business Steward Jesse Lambert and Jack Spivak, TopQuadrant Inc. November 30, 2017 Today s Program TopBraid EDG: A Day in the Life of a Business

More information

A brief history of time for Data Vault

A brief history of time for Data Vault Dates and times in Data Vault There are no best practices. Just a lot of good practices, and even more bad practices. This is especially true when it comes to handling dates and times in Data Warehousing,

More information

Data Analysis for Yield Improvement using TIBCO s Spotfire Data Analysis Software

Data Analysis for Yield Improvement using TIBCO s Spotfire Data Analysis Software Data Analysis for Yield Improvement using TIBCO s Spotfire Data Analysis Software Andrew Choo, Thorsten Saeger TriQuint Semiconductor Corporation 2300 NE Brookwood Parkway, Hillsboro, OR 97124 Andrew.Choo@tqs.com

More information

PharmaSUG China Big Insights in Small Data with RStudio Shiny Mina Chen, Roche Product Development in Asia Pacific, Shanghai, China

PharmaSUG China Big Insights in Small Data with RStudio Shiny Mina Chen, Roche Product Development in Asia Pacific, Shanghai, China PharmaSUG China 2016-74 Big Insights in Small Data with RStudio Shiny Mina Chen, Roche Product Development in Asia Pacific, Shanghai, China ABSTRACT Accelerating analysis and faster data interpretation

More information

PREDICT RA Workshop. Trial Data Management

PREDICT RA Workshop. Trial Data Management PREDICT RA Workshop Luke Stevens Data Management Coordinator Clinical Epidemiology and Biostatistics Unit Murdoch Childrens Research Institute www.mcri.edu.au luke.stevens@mcri.edu.au Topics Primary Principles

More information

Making the most of DCIM. Get to know your data center inside out

Making the most of DCIM. Get to know your data center inside out Making the most of DCIM Get to know your data center inside out What is DCIM? Data Center Infrastructure Management (DCIM) is the discipline of managing the physical infrastructure of a data center and

More information

Yield Statistics (YST) AN 48

Yield Statistics (YST) AN 48 Yield Statistics (YST) AN 48 Application Note to the KLIPPEL QC SYSTEM v5.0 The yield is the overall criteria of the production process. Using Klippel QC the quality of DUTs can be ensured, however, the

More information

LEADING WITH GRC. Approaching Integrated GRC. Knute Ohman, VP, GRC Program Manager. GRC Summit 2017 All Rights Reserved

LEADING WITH GRC. Approaching Integrated GRC. Knute Ohman, VP, GRC Program Manager. GRC Summit 2017 All Rights Reserved LEADING WITH GRC Approaching Integrated GRC Knute Ohman, VP, GRC Program Manager Agenda 1. Organization Overview: Vision, Key Facts and Needs 2. GRC Program Governance, Challenges and Community 3. Implementation

More information

Data Analysis and Validation for ML

Data Analysis and Validation for ML Analysis and for ML Neoklis (Alkis) Polyzotis, Google Research Collaborators: Eric Breck, Sudip Roy, Steven Whang, Martin Zinkevich Outline ML in production is hard, and a big part of hardness is related

More information

USER DEFINED OBJECTS - UDOS

USER DEFINED OBJECTS - UDOS USER DEFINED OBJECTS - UDOS GENERAL SECURITY RECOMMENDATIONS We make JD Edwards work better for people and harder for business TABLE OF Contents 03 04 05 05 06 06 07 08 08 Section 01 INTRODUCTION Section

More information

Data Analyst Nanodegree Syllabus

Data Analyst Nanodegree Syllabus Data Analyst Nanodegree Syllabus Discover Insights from Data with Python, R, SQL, and Tableau Before You Start Prerequisites : In order to succeed in this program, we recommend having experience working

More information

Visualizing the World

Visualizing the World Visualizing the World An Introduction to Visualization 15.071x The Analytics Edge Why Visualization? The picture-examining eye is the best finder we have of the wholly unanticipated -John Tukey Visualizing

More information

COMPETITION SUMMARIES Page 1

COMPETITION SUMMARIES Page 1 Computer Science (CSCI) Competition COMPETITION SUMMARIES Page 1 A competition for teams of 1 or 2 undergraduate student programmers, challenged to solve specific programming problems and evaluated for

More information

NRAO VLA Archive Survey

NRAO VLA Archive Survey NRAO VLA Archive Survey Jared H. Crossley, Loránt O. Sjouwerman, Edward B. Fomalont, and Nicole M. Radziwill National Radio Astronomy Observatory, 520 Edgemont Road, Charlottesville, Virginia, USA ABSTRACT

More information

Polynomial Models Studio October 27, 2006

Polynomial Models Studio October 27, 2006 Polynomial Models Studio October 27, 26 A. Download the data spreadsheet, open it, and select the tab labeled Murder. This has the FBI Uniform Crime Statistics reports of Murder and non-negligent manslaughter

More information

Course Microsoft Dynamics 365 Customization and Configuration with Visual Development (CRM)

Course Microsoft Dynamics 365 Customization and Configuration with Visual Development (CRM) Course 822716 Microsoft Dynamics 365 Customization and Configuration with Visual Development (CRM) Length 3 days Prerequisites Working knowledge of: Dynamics 365 (CRM) features and functionality; development,

More information

PLC Training - Intermediate

PLC Training - Intermediate PLC Training - Intermediate Contact us Today for a FREE quotation to deliver this course at your company?s location. https://www.electricityforum.com/onsite-training-rfq This Intermediate PLC Training

More information

3 Tips for Your Woes: Streamline. Simplify. Cloud.

3 Tips for Your  Woes: Streamline. Simplify. Cloud. Singtel Business Product Brochure Email Archiving 3 Tips for Your Email Woes: Streamline. Simplify. Cloud. Secure and flexible email archival and e-discovery with Singtel Email Archiving Services. Email

More information

NDSA Web Archiving Survey

NDSA Web Archiving Survey NDSA Web Archiving Survey Introduction In 2011 and 2013, the National Digital Stewardship Alliance (NDSA) conducted surveys of U.S. organizations currently or prospectively engaged in web archiving to

More information

Performance of Virtual Desktops in a VMware Infrastructure 3 Environment VMware ESX 3.5 Update 2

Performance of Virtual Desktops in a VMware Infrastructure 3 Environment VMware ESX 3.5 Update 2 Performance Study Performance of Virtual Desktops in a VMware Infrastructure 3 Environment VMware ESX 3.5 Update 2 Workload The benefits of virtualization for enterprise servers have been well documented.

More information

Web Applications Testing. Ario Nejad, Christopher Choi

Web Applications Testing. Ario Nejad, Christopher Choi Web Applications Testing Ario Nejad, Christopher Choi What is a Web Application? Though the boundaries of what constitute a web application are vague, it is commonly perceived to be an application that

More information

Representing LEAD Experiments in a FEDORA digital repository

Representing LEAD Experiments in a FEDORA digital repository Representing LEAD Experiments in a FEDORA digital repository You-Wei Cheah, Beth Plale Indiana University Bloomington, IN {yocheah, plale}@cs.indiana.edu IU-CS TR666 ABSTRACT In this paper, we discuss

More information

How to use WISKI for CCRN data. Kevin Shook Centre for Hydrology, University of Saskatchewan

How to use WISKI for CCRN data. Kevin Shook Centre for Hydrology, University of Saskatchewan How to use WISKI for CCRN data Kevin Shook Centre for Hydrology, University of Saskatchewan What is WISKI? Water Information System KIsters http://www.kisters.net Commercial environmental time series database

More information

Examining Rescue Studies

Examining Rescue Studies White Paper Examining Rescue Studies Introduction The purpose of this White Paper is to define a Rescue Study, outline the basic assumptions, including risks, in setting up such a trial based on DATATRAK

More information

Trimble Connect Overview

Trimble Connect Overview Trimble Connect Overview Trimble Connect reduces costs and improves efficiency for buildings and infrastructure projects. It does this by consolidating and seamlessly exchanging information throughout

More information

SYSPRO s Fluid Interface Design

SYSPRO s Fluid Interface Design SYSPRO s Fluid Interface Design Introduction The world of computer-user interaction has come a long way since the beginning of the Graphical User Interface, but still most application interfaces are not

More information

Dutch View on URN:NBN and Related PID Services

Dutch View on URN:NBN and Related PID Services Dutch View on URN:NBN and Related PID Services Arjan Hogenaar DANS PID-workshop Cologne 1 Dutch view DANS-view Agreement to some extent Minor differences Mainly the view of the institute DANS PID-workshop

More information