Recent Advances in Structured Data and the Web

Size: px

Start display at page:

Download "Recent Advances in Structured Data and the Web"

Stanley Cox
5 years ago
Views:

1 Recent Advances in Structured Data and the Web Alon Halevy Google April 10, 2013 Joint work with: Jayant Madhavan, Cong Yu, Fei Wu, Hongrae Lee, Nitin Gupta, Warren Shen Anish Das Sarma, Boulos Harb, Zack Ives, Afshin Rostamizadeh, Sree Balakrishnan, Anno Langen, Steven Whang, Heidi Lam, Rod McChesney, Ken Wilder, Karen Jacqmin-Adams, and others

2 Structured Data in Search Results

5 Recognizing Set-Oriented Queries

6 Data in Movies!

9 But Will it Stick?

10 City planning Sustainability: water, coffee, Crisis response Shirts Advancing public discourse (e.g., gun control) Data philanthropy corporations encouraged to contribute data to the good of society.

11 How Can We Help? We need to empower a new generation of factivists : Data enthusiasts who know nothing about DBMS They need to find good data, do meaningful data integration, and tell compelling stories We need to extract the semantics of structured data from text Natural Language + DB is poised for breakthrough.

12 Fusion Tables google.com/fusiontables Goal: an easy-to-use database system that is integrated with the Web. Key: support common workflows Easy upload (CSV, KML, spreadsheets) Sharing (even outside your company) Visualizations front and center Easy publishing Goal 2: Fusion in the data cloud -- discover others data and combine with yours.

14 ICDE Acceptance Rate

15 Coffee Producing Countries

16 Coffee Consumption Per Capita

incidents Colors indicate addresses of Rioting and Rioters Best UK

18 Table Facts: English poverty rates: 32,000 wards with a total of 1.8 million vertices Colors indicate poverty levels 2011 Rioting: 2100 incidents Colors indicate addresses of Rioting and Rioters Best UK Internet Journalist Knight-Batten Award for Innovations in Journalism Big Data for Regular People

22 Crowd Sourcing

23 Public toilets in Australia

24 Other Visualizations

25 From Visualization to Story Telling

Census: 254 counties with 543000 vertices Colored

28 See SIGMOD 2012 paper for details on scaling map visualizations Table Facts: Texas Counties 2010 Census: 254 counties with vertices Colored based on various demographics Big Data Integration

29 Data Integration as Search

31 Join with Population Data: What is a City?

32 Data Integration by the Masses: Challenges Guess as much as possible: suggest visualizations and data sets to join with Tools for telling good stories about the data Find join candidates that make sense Identifying reliable data Distinguishing between the source data and its derivatives

33 Finding Good Data on the Web The possible sources of data What are we trying to accomplish? Main classes of challenges The deeper challenge: semantics in text Also the opportunity: Natural Language is here.

34 HTML Tables

35 HTML Lists See Elmeleegy et al., VLDB 2009

36 The Deep Web Tree Search Amish quilts Parking tickets in India Horses

37 Repeating Structures

38 Other Sources of Data Spreadsheets CSV files Tables embedded in PDF XML, RDF Visualizations Online databases (Fusion Tables, Tableau, ) Each source has its particularities, but most problems are common to all.

39 What Will We Do With the Data? research.google.com/tables (give feedback to Cong Yu, in person)

40 Give Answers from Tables

41 It Better Be Right!

42 Answer with a Visualization

43 Nuclear Facilities in Earthquake Prone Areas

44 Long Term Goal: A Data-Guided Decision Engine Support decision making: Healthcare debate Should I install solar in my house? Which charity should I contribute to? Show relevant data Expose facets of the decision and enable drilldown Show opposing views Manually curated examples of decision engines: Justfacts.com, followthemoney.com, decide.com

45 Finding The Needle in the Haystack Categories of challenges: Non-tabular data displayed in HTML tables Good data, optimized for presentation on a page Good data, optimized for presentation on a site Table quality is often subjective! The ultimate challenge: recovering the table semantics

46 Non-Tabular Data in HTML

47 Data Optimized for Page Layout

49 Tabular Data Optimized for Site Layout See [Ling et al, IJCAI 2013] for stitching tables within a site.

50 Vertical Tables

51 Semantics are in Text

52 Semantics Can Be Brittle

53 Text and Structured Data Structured data in textual context: Spreadsheets PDF files Attribute descriptions in databases Comments field (e.g., customer care data) Everywhere on the Web

54 The Challenge Analyze natural language text as it pertains to structured data. Different from (open) information extraction that builds databases entirely from text. Good news: natural language parsing technology is now scalable.

55 First Step: Annotating Columns [Venetis et al., VLDB 2011]

56 Next Step: Understanding Relationships Between Columns

57 A Dictionary of Attributes Countries have population, coffee production Urban population is a subset of population Synonyms: for mountains, height = elevation Time dependence: population (by year) Units Coffee production: 60Kg bags Mountain elevations: meters, feet GDP: measured per capita Patterns of mentions in text The coffee production of X increased by Y percent in Z.

58 Dependency Parsing is Ready

59 Tower of Babel: Internet Style Coffee production el salvador 2013 Knowledge Graph El Salvador exports coffee 2013 Tables Text In 2013, the coffee production of El Salvador dropped by 20% due to the coffee rust disease.

60 Conclusions This was a talk about Big Data: Millions of people creating data sets Billions of people seeing the data being impacted Get out there and find your favorite application. Dreams do come true: At least as it pertains to structured data on the Web!

61 References Fusion Tables: SIGMOD 2010, 2012 WebTables: VLDB 2008, 2009, 2011

Big Data Integration for Data Enthusiasts. Jayant Madhavan Structured Data Research Google Inc.

Big Data Integration for Data Enthusiasts. Jayant Madhavan Structured Data Research Google Inc. for Data Enthusiasts Jayant Madhavan Structured Data Research Google Inc. Big Data Challenge Running computations over ginormous datasets Petabytes, Exabytes, maybe more! Only one aspect of the challenge!