TRANSPORTATION RESEARCH BOARD. Data Management and Governance Practices at Transportation Agencies. Monday, February 26, :00-3:30 PM ET

Transcription

1 TRANSPORTATION RESEARCH BOARD Data Management and Governance Practices at Transportation Agencies Monday, February 26, :00-3:30 PM ET

2 The Transportation Research Board has met the standards and requirements of the Registered Continuing Education Providers Program. Credit earned on completion of this program will be reported to RCEP. A certificate of completion will be issued to participants that have registered and attended the entire session. As such, it does not include content that may be deemed or construed to be an approval or endorsement by RCEP.

3 Purpose Discuss NCHRP Synthesis 508. Learning Objectives At the end of this webinar, you will be able to: Describe key concepts in data management and governance List current data management and governance practices

4 NCHRP Synthesis Report 508: Data Management and Governance Practices NCHRP Project 20-05/ Topic 47-05

5 NCHRP is a State-Driven Program Sponsored by individual state DOTs who Suggest research of national interest Serve on oversight panels that guide the research. Administered by TRB in cooperation with the Federal Highway Administration.

6 Practical, ready-to-use results Applied research aimed at state DOT practitioners Often become AASHTO standards, specifications, guides, syntheses Can be applied in planning, design, construction, operations, maintenance, safety, environment

7 Synthesis Objective Develop a collection of transportation agency data management practices and experiences Demonstrate how agencies currently access, manage, use and share data

8 Today s Speakers Nasir Gharaibeh, Texas A&M University Johanna Zmud, Texas A&M Transportation Institute Jared Causseaux / Josh Whitaker, Florida Department of Transportation Moderator: Peggi Knight, Iowa Department of Transportation

9 Data Management & Governance Practices at Transportation Agencies NCHRP Synthesis 508 Research Team Nasir Gharaibeh, Johanna Zmud, Isaac Oti, David Schrank, Texas A&M Transportation Institute Texas A&M University February 26, 2018

10 Project Panel Members (Topic 47-05) CHAD T. BAKER, California Department of Transportation JAMES P. HALL, University of Illinois at Springfield DAVID HURST, VHB PEGGI KNIGHT, Iowa Department of Transportation ARLENE MATHISON, University of Minnesota Centre for Transportation Studies BENITO O. PEREZ, District of Columbia Department of Transportation JACK R. STICKEL, Alaska Department of Transportation and Public Facilities (retired) JOSHUA CLAY WHITAKER, Florida Department of Transportation KATHERINE A. PETROS, Federal Highway Administration (Liaison) TIANJIA TANG, Federal Highway Administration (Liaison) LISA LOYO, Transportation Research Board THOMAS PALMERLEE, Transportation Research Board 2

11 NCHRP Synthesis Studies Staff JON M. WILLIAMS, Program Director, IDEA and Synthesis Studies TANYA M. ZWAHLEN, Consultant 3

12 Complexity and Amount of Transportation Data Multiple uses/business functions Inventory Construction Crash Other Multiple databases Multiple forms Increasing volume Rule of thump: Corporate data double every 18 months Multimedia Attribute databases Geospatial databases CAD Other 4

13 Objective & Scope Objective: Discuss a collection of transportation agency data management practices and experiences. Topics: Data Governance Data Warehousing and Cloud Computing Data Integration and Sharing Data Quality 5

14 2-Phase Survey Methods DOTs Responses: 43 (Phase 1) & 34 (Phase 2) MPOs & Municipalities Responses: 19 (Phase 1) & 11 (Phase 2) Interviews: Three DOTs (Alaska, Iowa, Nebraska) One Local Agency (Chicago Metropolitan Agency for Planning) Literature Review 28 publications Websites of majority of state DOTs and FHWA 6

15 Webinar Segments Practices in Data Governance Practices in Data Integration and Sharing Practices in Data Quality Findings 7

16 Practices in Data Governance 8

17 Data Governance: Establishment, execution, and enforcement of authority over the management of data assets Examples: Florida DOT Minnesota DOT US DOT Data Governance Data Life-Cycle Adapted from Ladley (2012) 9

18 Practice at DOTs Does your agency have a designated data governance board/council? 37% (16 agencies) 19% (8 agencies) Yes No In development 44% (19 agencies) 10

19 Practice at DOTs Does your agency have designated data coordinators? 26% (11 agencies) Yes No 14% (6 agencies) 60% (26 agencies) In development 11

20 Practice at DOTs What data in your agency have designated stewards? Pavement inventory & condition data HPMS Roadway inventory Bridge inventory & condition data Crash Data Traffic monitoring data Contracts/procurement data Financial data Inventory and condition data for other assets Project construction data Environmental impact and compliance data Transportation improvement programs data Bridge work history data Real estate data Travel modeling data Pavement work history data Project design and materials data Others % of DOT respondents 12

21 Practice at DOTs To what extent do the following factors limit progress on instituting data governance in your agency? Other mission-related issues are more pressing Hard to justify the cost and effort 6 19* Major Factor Lack of resources Minor Factor Lack of staffing Not an Issue * indicates number of respondents % of Respondents 13

22 Practice at Local Agencies Does your agency have designated data coordinators? While only 1 out of 19 agencies indicated they have a designated data governance board/council, 5 indicated they have designated data coordinators. 2 agencies 5 agencies Yes No In development 12 agencies 14

23 Practice at Local Agencies What data in your agency have designated stewards? Travel modeling data Transportation improvement programs data Roadway inventory Crash Data Financial data Traffic monitoring data Pavement inventory & condition data Contracts/procurement data Bridge inventory & condition data Environmental impact and compliance data Real estate data Others Pavement work history data Bridge work history data Inventory and condition data for other assets Project design and materials data HPMS Project construction data No. of local agency respondents 15

24 Practice at Local Agencies To what extent do the following factors limit progress on instituting data governance in your agency? Other mission-related issues are more pressing Hard to justify the cost and effort Lack of resources Lack of staffing 4 5 7* Major Factor Minor Factor Not an Issue *indicates number of respondents No. of respondents 16

25 Practices in Data Warehousing and Cloud Computing 17

26 Practice at DOTs What data in your agency are maintained in data warehouses or marts? Roadway inventory Bridge inventory & condition data Traffic monitoring data Crash Data HPMS Financial data Pavement inventory & condition data Transportation improvement programs data Project construction data Bridge work history data Contracts/procurement data Inventory and condition data for other assets Pavement work history data Environmental impact and compliance data Travel modeling data Project design and materials data Real estate data Others % of DOT respondents 18

27 Practice at DOTs What data in your agency are archived systematically to retain historical information? Pavement inventory & condition data Traffic monitoring data HPMS Roadway inventory Project construction data Crash Data Bridge inventory & condition data Financial data Project design and materials data Contracts/procurement data Pavement work history data Transportation improvement programs data Bridge work history data Inventory and condition data for other assets Environmental impact and compliance data Real estate data Travel modeling data Others % DOT respondents 19

28 Practice at DOTs Approximately, what percentage of your agency s transportation-related data is currently stored and managed using commercial cloud computing services? 10% (3 agencies) 10% (3 agencies) 6% (2 agencies) 3% (1 agency) 21-50% 11-20% 1-10% None Unknown 71% (22 agencies) 20

29 Practice at Local Agencies What data in your agency are maintained in data warehouses or marts? Roadway inventory Traffic monitoring data Travel modeling data Transportation improvement programs data Crash Data Bridge inventory & condition data Pavement inventory & condition data HPMS Environmental impact and compliance data Contracts/procurement data Financial data Bridge work history data Inventory and condition data for other assets Real estate data Others Pavement work history data Project design and materials data Project construction data No. of local agency respondents 21

30 Practice at Local Agencies What data in your agency are archived systematically to retain historical information? Travel modeling data Crash Data Financial data Pavement inventory & condition data Contracts/procurement data Roadway inventory Real estate data Traffic monitoring data Bridge inventory & condition data Project design and materials data Others HPMS Pavement work history data Bridge work history data Project construction data No. of local agency respondents 22

31 Practice at Local Agencies Approximately, what percentage of your agency s transportation-related data is currently stored and managed using commercial cloud computing services? 1 agency 3 agencies 1-10% None Unknown 5 agencies 23

32 Practices in Data Integration and Sharing 24

33 Practice at DOTs Which of these data sets are integrated in your agency to serve various business needs? A&E A&G A&C A&B A&I A&J B&C % of DOT respondents 25

34 Practice at DOTs What location referencing methods are used in your agency? % of agencies Geographic coordinates Route milepoint Link node Route reference post Route street reference Multilevel LRS Other 26

35 Practice at DOTs How does your agency share data with outside users (public and private entities)? % of agencies Online (open access) Upon request Not shared outside agency Online (preauthorized access) Other 27

36 Practice at DOTs What strategies would improve (or have improved) data sharing and access within your agency? Improved metadata 15* 15 1 Increased use of web-based data storage and access Improved database management systems Major Effect Minor Factor No effect Reduced use of hardware and software that require specialized data format Not applicable * indicates number of respondents % of DOT respondents 28

37 Practice at Local Agencies Which of these data sets are integrated in your agency to serve various business needs? Four out of 8 respondents indicated that their agencies integrate the following data sets: Roadway inventory data and traffic monitoring data Roadway inventory data and travel modeling data Environmental impact and compliance data and project design and materials data 29

38 Practice at Local Agencies What location referencing methods are used in your agency? No. of agencies Geographic coordinates Other Route milepoint Link node 30

39 Practice at Local Agencies How does your agency share data with outside users (public and private entities)? No. of agencies Online (open access) Upon request Not shared outside agency Other Online (preauthorized access) 31

40 Practice at Local Agencies What strategies would improve (or have improved) data sharing and access within your agency? Improved metadata [VALUE]* Major Effect Increased use of web-based data storage and access Minor Factor Improved database management systems No effect Reduced use of hardware and software that require Not applicable No. of respondents 32

41 Practices in Data Quality 33

42 Practice at DOTs To what extent are data quality elements evaluated in your agency? Evaluation Of Data Quality (By % of DOTs) Quality element Accuracy Completeness Timeliness Relevancy Access Consistency Accessibility security Evaluated in all or most areas Evaluated in some areas Evaluated in a few areas

43 Practice at Local Agencies How does your agency share data with outside users (public and private entities)?? Evaluation Of Data Quality (By number of local agencies) To what extent are data quality elements evaluated in your agency? Accuracy Completeness Timeliness Relevancy Consistency Accessibility Access security Evaluated in all or most areas Evaluated in some areas Evaluated in a few areas Not evaluated

44 Findings 36

45 Findings Related to Data Governance A bottom-up approach for data management appears to be taking place. A more top-down data governance approach could: help recognize and leverage the value of data generated and/or stored in various agency silos, and spur increased data integration and sharing. The majority of responding DOTs and local agencies identified the following as major factors in limiting progress towards implementing data governance: a) Lack of staffing b) Other mission-related issues are more pressing c) Lack of resources It is likely that data governance is more mature in DOTs compared to local agencies. 37

46 Findings Related to Data Warehousing and Cloud Computing Data sets that have data stewards tend to reside in data warehouses or marts. Conversely, data sets that do not have designated stewards tend to reside in disparate files and databases. Transportation agencies have a wealth of archived data. For DOTs, a clear majority of respondents (70% or more) routinely archive these data: pavement inventory & condition data, traffic monitoring data, HPMS data, project construction data, crash data, and bridge inventory & condition data. The use of cloud computing services is expected to grow in the future, but most surveyed agencies are uncertain about the magnitude and nature of this growth in their agencies. 38

47 Findings Related to Data Integration and Sharing Transportation agencies use multiple location referencing methods (LRMs); indicating that the use of incompatible LRMs remains an impediment to increased data integration in these agencies. Geographic coordinates and route mile-point are the most commonly used LRMs. An area prime for reducing the duplication of data within DOTs is the creation of digital as-builts from 3D models used in design and construction. Majority of surveyed agencies indicated that the following strategies have major effect on improving data sharing and access: a) Increased use of web-based data storage and access b) Improved database management systems. 39

48 Findings Related to Data Quality Most respondents indicated that all seven data quality elements are evaluated in at least some data areas in their agencies. For DOTs, timeliness, accuracy, and access security are most commonly evaluated. Slightly more than half of the DOT respondents indicated that their agencies have mechanisms in place for incorporating feedback from data users into the data collection process. These feedback mechanisms include: Surveys, steering committees, web forms, and ad hoc communications (e.g. meetings, s). 40

49 Thank you for your attention 41

50 Florida DOT Implementation of Data Governance Jared Causseaux GIS Program Manager Florida Department of Transportation

51 1 Overview and Mission 2 Why ROADS? 3 History 4 Approach 5 What s Next

52 Mission The goal of the ROADS initiative is to improve data reliability, simplify data sharing across FDOT, and have readily available and accurate data to make informed decisions. Reliable, Organized, Accurate Data Sharing

53 Why Roads It s hard to know what data is available Data is hard to access Lack of standardized approach No enterprise-level view of data Teams want a one stop shop Information is organized around applications instead of around perspectives end users can relate to (e.g., business subject areas). Security controls are scattered across many different tools and processes (e.g., RACF, AARF, applications, SharePoint, etc.) throughout 170+ centralized and local systems. Many districts are moving toward the adoption of the GIS system as the entry point for information searches, but the approach has not been standardized across the districts. A majority of the district data efforts appear to focus on singular business issues without the added benefit of looking at the data from an overall business view to improve performance or reduce risk. Teams consistently stated they would like a "one stop shop" to access all information they need in one place, with a "Google Type" Search. Over 40 "Search Criteria" elements have been identified.

54 Why Roads Better: Ensure information is secure, accurate, reliable, and at the appropriate level to empower you to do your job better. Easier: Provide the ability to access relevant business data more quickly and efficiently by knowing where to find it. Faster: Reduce the amount of time to locate the data you need and more time to analyze the data. Sharable: Effectively share information across our organization to enable better and faster decisions. Integrated: Enable a greater capability to link data together from different Districts, Functional Areas, and Systems. Streamlined: Remove the barriers currently in place that prevent the efficient sharing of information.

55 History 1 In the fall of 2014, FDOT began to develop an Enterprise-wide Information Technology Strategy. 2 Office of Information Technology sponsored a critical assessment of the Department s information technology capabilities, personnel, and infrastructure. 3 A recommendation was made to develop a comprehensive Enterprise Information Management structure, once the current information management practices at the Department had been assessed. 4 To accomplish this task, Phase I of the Reliable, Organized, and Accurate Data Sharing (ROADS) Initiative began in March of The ROADS Initiative expects to drive many benefits by streamlining information across the organization to enable better, faster decisions by removing the current barriers that prevent the efficient sharing of information.

56 Approach People: Managing a formal Data Governance Structure to make key decisions related to Data / Information. Process: Training FDOT on the Data Governance Component Model and Implementing Standard Processes & Routines to provide a formal approach to Data Governance. Technology: Providing common standardized BI / DW Tools, Technologies and Frameworks that will be used across FDOT to make data/information more accessible. Data Strategy and Architecture Master Data Management Data Integration Data Quality Security and Privacy Metadata Management Dashboards Scorecards Reporting Analytics

57 Approach Chaired by CIM Officer Champions Data Quality Improvement Represents Organizational Stakeholders Prioritizes Data Issues Sets Data Governance Rules and Procedures Technical Function Expert Supports and Implements Data Governance Activities Assists With Documentation Maintains Knowledge of Data Structure Identifies and Resolves Data Quality Defects Implements Data Confidentiality & Security Requirements Chaired by Data Governance Administrator Leads the Data Governance Working Group Acts as Liaison between ROADS Executive Team & the Data Stewards and Data Custodians Ensures Data Governance Compliance Business Function Expert Collaborates Across Dept. on Data Governance Activities Defines and Maintains Business Metadata Coordinates Adequate Documentation Develops Data Quality Process and Standards Drives Remediation of Data Quality Defects Manages Data Confidentiality & Security Classification

58 Bi/DW Structure Other key roles for ROADS Executive Sponsor (Chief of Transportation Technology) Project Sponsor (CIM Officer) Business Functional Coordinator (CIM Officer) Technical Functional Coordinator (IT Bureau Chief) ROADS Business Liaison BI/DW Project Manager

59 Bi/DW Next Steps Business Capability Iterations Safety #1 Enhance existing Metadata Enhance current Data Quality process Improve Data Integration within Safety Office Improve Data Integration with other Departmental Offices Develop Safety Dashboards Create Enterprise Reports Create Self Service Reports

60 Questions Jared Causseaux Civil Integrated Management GIS Program Manager 605 Suwannee Street, MS 5-L Tallahassee, FL fdot.gov/gis (850)

61 Today s Participants Peggi Knight, Iowa Department of Transportation, peggi.knight@iowadot.us Nasir Gharaibeh, Texas A&M University, ngharaibeh@civil.tamu.edu Johanna Zmud, Texas A&M Transportation Institute, j-zmud@tamu.edu Clay Whitaker, Florida Department of Transportation, clay.whitaker@dot.state.fl.us Jared Casseaux, Florida Department of Transportation, Jared.Causseaux@dot.state.fl.us

62 Get Involved with TRB Getting involved is free! Join a Standing Committee ( Become a Friend of a Committee ( Networking opportunities May provide a path to become a Standing Committee member For more information: Create your account Update your profile

63 Get involved with NCHRP Suggest NCHRP research topics Volunteer to serve on NCHRP panels Lead pilot projects and other implementation efforts at your agency For more information:

64 Receiving PDH credits Must register as an individual to receive credits (no group credits) Credits will be reported two to three business days after the webinar You will be able to retrieve your certificate from RCEP within one week of the webinar