Exploiting Discourse Analysis for Article-Wide Temporal Classification

Transcription

1 Exploiting Discourse Analysis for Article-Wide Temporal Classification Jun-Ping Ng Min-Yen Kan Ziheng Lin Vanessa Wei Feng Bin Chen Jian Su Chew-Lim Tan National University of Singapore National University of Singapore SAP Asia Pte. Ltd. University of Toronto Institute for Infocomm Research Institute for Infocomm Research National University of Singapore

2 Outline Preliminaries Discourse Analysis Methodology Experiments Discussion 2

3 Temporal Classification At least 19 people were killed and 114 people were wounded in Tuesday s southern Philippines airport blast, officials said, but reports said the death toll could climb to 30. 3

4 Temporal Classification At least 19 people were killed and 114 people were wounded in Tuesday s southern Philippines airport blast, officials said, but reports said the death toll could climb to 30. wounded said killed time 3

5 Temporal Classification At least 19 people were killed and 114 people were wounded in Tuesday s southern Philippines airport blast, officials said, but reports said the death toll could climb to 30. OVERLAP wounded said killed time 3

6 Temporal Classification At least 19 people were killed and 114 people were wounded in Tuesday s southern Philippines airport blast, officials said, but reports said the death toll could climb to 30. BEFORE OVERLAP wounded said killed time 3

7 Temporal Classification At least 19 people were killed and 114 people were wounded in Tuesday s southern Philippines airport blast, officials said, but reports said the death toll could climb to 30. AFTER OVERLAP wounded said killed time 3

8 Article-Wide Event Pairs Decide on temporal relationship between any two events in an article Helps give us a more complete picture of temporal relations between event pairs 4

9 Challenge Distance between event pairs can potentially be significant Current approaches which are heavy on lexical and syntactic features are not useful 5

10 Discourse Analysis Tells us how sentences are composed together Relating sentences together gives us a better idea of temporal ordering 6

11 Discourse Analysis Rhetorical Structure Theory (RST) PDTB-styled Discourse Relations Topical Text Segmentation 7

12 RST Breaks up text into basic textual units called Elementary Discourse Units (EDU) Relates neighbouring EDUs via a set of typed discourse relations 8

13 RST Max switched off the light. The room was pitch dark. RESULT Max switched off the light. The room was pitch dark. 9

14 PDTB-styled Discourse Relations Identifies explicit and implicit relations between text Also relates text fragments together via a set of typed relations 10

15 PDTB-styled Discourse Relations Max switched off the light. The room was pitch dark. CONTINGENCY :: CAUSE arg1 Max switched off the light. arg2 The room was pitch dark. 11

16 Topical Text Segmentation Groups neighbouring sentences about the same topic together Transitioning across groups of sentences represents a shift in the topic being discussed Coarse-grained discourse analysis 12

17 Topical Text Segmentation The Davao Medical Centre, a regional government hospital, recorded 19 deaths with 50 wounded. Medical evacuation workers however said the injured list was around 114, spread out at various hospitals. A powerful bomb tore through a waiting shed at the Davao City international airport at about 5.15 pm (0915 GMT) while another explosion hit a bus terminal at the city. 13

18 Topical Text Segmentation The Davao Medical Centre, a regional government hospital, recorded 19 deaths with 50 wounded. Medical evacuation workers however said the injured list was around 114, spread out at various hospitals. A powerful bomb tore through a waiting shed at the Davao City international airport at about 5.15 pm (0915 GMT) while another explosion hit a bus terminal at the city. 13

19 Topical Text Segmentation The Davao Medical Centre, a regional government hospital, recorded 19 deaths with 50 wounded. Medical evacuation workers however said the injured list was around 114, spread out at various hospitals. A powerful bomb tore through a waiting shed at the Davao City international airport at about 5.15 pm (0915 GMT) while another explosion hit a bus terminal at the city. 13

20 Outline Preliminaries Discourse Analysis Methodology Experiments Discussion 14

21 Classifying Temporal Relations RST Parsing PDTB Parsing Text Segmentation Identify events within article Feature Generation Convolution Kernel w SVM Temporal Relations 15

22 Classifying Temporal Relations 1 RST Parsing PDTB Parsing Text Segmentation Identify events within article Feature Generation Convolution Kernel w SVM Temporal Relations 15

23 Classifying Temporal Relations 1 2 RST Parsing PDTB Parsing Text Segmentation Identify events within article Feature Generation Convolution Kernel w SVM Temporal Relations 15

24 Classifying Temporal Relations 1 2 RST Parsing PDTB Parsing Text Segmentation 3 Identify events within article Feature Generation Convolution Kernel w SVM Temporal Relations 15

25 Classifying Temporal Relations 1 2 RST Parsing PDTB Parsing Text Segmentation 3 Identify events within article Feature Generation 4 Convolution Kernel w SVM Temporal Relations 15

26 Classifying Temporal Relations 1 2 RST Parsing PDTB Parsing Text Segmentation Identify events within article Feature Generation 3 Convolution Kernel w SVM Temporal Relations

27 RST - Feature conjunction elaboration elaboration elaboration at (0915 GMT) while bus terminal at the city. A powerful. shed at the. airport 16

28 RST - Feature conjunction elaboration elaboration elaboration at (0915 GMT) while bus terminal at the city. A powerful. shed at the. airport 16

29 RST - Feature conjunction elaboration elaboration elaboration at (0915 GMT) while bus terminal at the city. A powerful. shed at the. airport conjunction elaboration 16

30 PDTB - Feature synchrony A powerful bomb. at about 5.15 pm (0915 GMT) while another explosion. at the city 17

31 PDTB - Feature synchrony A powerful bomb. at about 5.15 pm (0915 GMT) while another explosion. at the city 17

32 PDTB - Feature synchrony A powerful bomb. at about 5.15 pm (0915 GMT) while another explosion. at the city synchrony 17

33 Text Segmentation - Feature The Davao Medical Centre, a regional government hospital, recorded 19 deaths with 50 wounded. Medical evacuation workers however said the injured list was around 114, spread out at various hospitals. A powerful bomb tore through a waiting shed at the Davao City international airport at about 5.15 pm (0915 GMT) while another explosion hit a bus terminal at the city. 18

34 Text Segmentation - Feature 1 The Davao Medical Centre, a regional government hospital, recorded 19 deaths with 50 wounded. Medical evacuation workers however said the injured list was around 114, spread out at various hospitals. 2 A powerful bomb tore through a waiting shed at the Davao City international airport at about 5.15 pm (0915 GMT) while another explosion hit a bus terminal at the city. 18

35 Text Segmentation - Feature 1 The Davao Medical Centre, a regional government hospital, recorded 19 deaths with 50 wounded. Medical evacuation workers however said the injured list was around 114, spread out at various hospitals. 2 A powerful bomb tore through a waiting shed at the Davao City international airport at about 5.15 pm (0915 GMT) while another explosion hit a bus terminal at the city. 18

36 SVM + Tree Kernels Proposed discourse features are structural Tree kernels with SVM can help us capture structural similarity easily Does away with difficult feature engineering to vectorise discourse structures 19

37 Measuring Similarity with Tree Kernels R1 R1 R2 R3 R2 R4 R4 20

38 Measuring Similarity with Tree Kernels R1 R1 R1 R2 R1 R3 R2 R1 R3 R4 R4 21

39 Measuring Similarity with Tree Kernels R1 R1 R1 R2 R1 R3 R2 R1 R3 R4 R4 21

40 Outline Preliminaries Discourse Analysis Methodology Experiments Discussion 22

41 Data Set Subset of 20 news articles from ACE 2005 corpus Annotated with 375 event pairs Applied temporal transitivity rules to obtain 7994 event pairs 23

42 Temporal Transitivity Max opened the door. He went into the room. He placed the book on the table. 24

43 Temporal Transitivity Max opened the door. He went into the room. He placed the book on the table. opened went placed time 24

44 Temporal Transitivity Max opened the door. He went into the room. He placed the book on the table. opened went placed time 24

45 Temporal Transitivity Max opened the door. He went into the room. He placed the book on the table. opened went placed time 24

46 Data Set Breakdown Class Proportion OVERLAP 10% BEFORE 45% AFTER 45% 25

47 Sentence Gap and Temporal Class 600 AFTER BEFORE OVERLAP 450 # E-E Pairs Sentence Gap 26

48 Results System P R F1 1 Do Base Base + RST + PDTB + TS Base + RST + PDTB + TS + CR Base + ORST + PDTB + OTS + OCR

49 Results System P R F1 1 Do Base Base + RST + PDTB + TS Base + RST + PDTB + TS + CR Base + ORST + PDTB + OTS + OCR

50 Results System P R F1 1 Do Base Base + RST + PDTB + TS Base + RST + PDTB + TS + CR Base + ORST + PDTB + OTS + OCR

51 Results System P R F1 1 Do Base Base + RST + PDTB + TS Base + RST + PDTB + TS + CR Base + ORST + PDTB + OTS + OCR

52 Results System P R F1 1 Do Base Base + RST + PDTB + TS Base + RST + PDTB + TS + CR Base + ORST + PDTB + OTS + OCR

53 Results System P R F1 1 Do Base Base + RST + PDTB + TS Base + RST + PDTB + TS + CR Base + ORST + PDTB + OTS + OCR

54 Results System P R F1 1 Do Base Base + RST + PDTB + TS Base + RST + PDTB + TS + CR Base + ORST + PDTB + OTS + OCR

55 Useful Temporal Relations (Temporal.. (Temporal (Elaboration... (Elaboration (Background... 34

56 RST Before Relation Milosevic and his wife wielded enormous power in Yugoslavia for more than a decade before he was swept out of power after a popular revolt in October temporal Milosevic wielded a decade temporal before.. swept out.. power after a October

57 RST Before Relation Milosevic and his wife wielded enormous power in Yugoslavia for more than a decade before he was swept out of power after a popular revolt in October temporal Milosevic wielded a decade temporal before.. swept out.. power after a October

58 Confusion Matrix Predicted O B A N Actual O B A 14.7% 14.1% 12.8% 58.5% 0.5% 57.9% 15.5% 26.0% 0.5% 15.7% 57.3% 26.5% 36

59 Confusion Matrix Predicted O B A N Actual O B A 14.7% 14.1% 12.8% 58.5% 0.5% 57.9% 15.5% 26.0% 0.5% 15.7% 57.3% 26.5% 37

60 Confusion Matrix Predicted O B A N Actual O B A 14.7% 14.1% 12.8% 58.5% 0.5% 57.9% 15.5% 26.0% 0.5% 15.7% 57.3% 26.5% 38

61 Doing Better Investigate how to exploit other aspects of discourse analysis Integrate features with joint-inference approaches for better overall results 39

62 Conclusion Discourse analysis is important for temporal classification Rich structural information can be robustly captured with tree kernels Improvements to automatic discourse parsing results will help 40