Castanet: Using WordNet to Build Facet Hierarchies. Emilia Stoica and Marti Hearst School of Information, Berkeley

Transcription

1 Castanet: Using WordNet to Build Facet Hierarchies Emilia Stoica and Marti Hearst School of Information, Berkeley

2 Motivation Want to assign labels from multiple hierarchies

3 Motivation Hot and Sweet Chicken: 1 pepper, 2 apricots, 1 pound chicken breast, 1 Tbsp gingerroot Meat Chicken Fruit Apricot Vegetables pepper Flavor gingerroot

4 Castanet Carves out a structure from the hypernym (IS-A) relations within WordNet Produces surprisingly good results for a wide range of subjects e.g., arts, medicine, recipes, math, news, bibliographical records

5 WordNet Challenges A word may have more than one sense - Fine granularity of word sense distinctions e.g., newspaper (#1) - daily publication on folded sheets newspaper (#3) - physical object - Ambiguity for the same sense tuna #1 cactus #2 fish food fish bony fish

6 WordNet Challenges (cont.) The hypernym path may be quite long (e.g., sense #3 of tuna has 14 nodes) Sparse coverage of proper names and noun phrases (not addressed)

7 Algorithm Goals Build a set of facet hierarchies Balance depth and breadth Avoid skinny paths Don t go too deep or too broad Choose understandable labels Disambiguate words Currently a word can take on only one sense

8 Our Approach Documents Select terms Build core tree Augment core tree WordNet Compress Tree Remove top level categories Divide into facets

9 1. Select Terms Select well-distributed terms from the collection Eliminate stopwords Documents Select terms Build core tree Augm. core tree Retain only those terms with a distribution higher than a threshold WordNet Comp. tree Remove top level categ. (default: top 10%)

10 2. Build Core Tree Build a backbone Create paths from unambiguous terms only Bias the structure towards appropriate senses of words Documents Select terms WordNet Build core tree Comp. tree Augm. core tree Remove top level categ. entity substance,matter nutriment dessert frozen dessert ice cream sundae sundae entity substance,matter nutriment dessert frozen dessert sherbet,sorbet sherbet Get hypernym path if term: - has only one sense, or - matches a pre-selected WordNet domain Adding a new term increases a count at each node on its path by # of docs with the term.

11 2. Build Core Tree (cont.) Merge hypernym paths to build a tree entity substance,matter nutriment dessert frozen dessert entity substance,matter nutriment dessert frozen dessert entity substance,matter nutriment dessert frozen dessert ice cream sundae sherbet,sorbet ice cream sundae sherbet,sorbet sundae sherbet sundae sherbet

12 3. Augment Core Tree Attach to Core tree the terms with more than one sense Documents Select terms Build core tree Augm. core tree Favor the more common path over other alternatives WordNet Comp. tree Remove top level categ.

13 Augment Core Tree (cont.) Date (p1) Date (p2) entity abstraction substance,matter measure, quantity food, nutrient fundamental quality nutriment time period food calendar day (18) edible fruit (78) date Choose this path since it has more items assigned date

14 Optional Step: Domains To disambiguate, use Domains Wordnet has 212 Domains medicine, mathematics, biology, chemistry, linguistics, soccer, etc. A better collection has been developed by Magnini 2000 Assigns a domain to every noun synset Automatically scan the collection to see which domains apply The user selects which of the suggested domains to use or may add own Paths for terms that match the selected domains are added to the core tree

15 Using Domains dip glosses: Sense 1: A depression in an otherwise level surface Sense 2: The angle that a magnet needle makes with horizon Sense 3: Tasty mixture into which bite-size foods are dipped dip hypernyms Sense 1 Sense 2 Sense 3 solid shape, form food => concave shape => space => ingredient, fixings => depression => angle => flavorer Given domain food, choose sense 3

16 4. Compress Tree Rule 1: Eliminate a parent with fewer than k children unless it is the root or its distribution is larger than 0.1*max dist Documents Select terms WordNet Build core tree Comp. tree abstraction Augm. core tree Remove top level categ. ice cream sundae sundae dessert frozen dessert parfait sherbet,sorbet sherbet dessert frozen dessert sundae parfait sherbet

17 4. Compress Tree (cont.) Rule 2: Eliminate a child whose name appears within the parent s name Documents Select terms WordNet Build core tree Comp. tree abstraction Augm. core tree Remove top level categ. dessert frozen dessert dessert sundae parfait sherbet sundae parfait sherbet

18 5. Divide into Facets Divide into facets

19 5. Divide into Facets (Remove top levels) entity substance,matter food,nutriment Divide into facets food stuff,food product ingredient,fixings flavorer flavorer herb sweetening herb sweetening parsley oregano sugar syrup parsley oregano sugar syrup Rule 1: Eliminate very general categories (e.g., entity, abstraction). If no paths are longer than threshold t, then done. Else: Rule 2: Undo first step. Then eliminate all top levels until the maximum length of any path in the resulting hierarchy is t.

20 Example: Recipes (3500 docs)

21 Castanet Output (shown in Flamenco)

22 Castanet Output

23 Castanet Output

24 Castanet Output

25 Castanet Output

26

27 Castanet Evaluation This is a tool for information architects, so people of this type did the evaluation We compared output on Recipes Biomedical journal titles We compared to two state-of-the-art algorithms LDA (Blei et al. 04) Subsumption (Sanderson & Croft 99)

28 Subsumption Output

29 Subsumption Output

30 Subsumption Output

31 Subsumption Output

32 LDA Output

33 LDA Output

34 LDA Output

35 Evaluation Method Information architects assessed the category systems For each of 2 systems output: Examined and commented on top-level Examined and commented on two sub-levels Then comment on overall properties Meaningful? Systematic? Likely to use in your work?

36 Evaluation (cont.) Sample questions for top level categories: - Would you add/remove/rename any category? - Did this category match your expectations? Sample questions for a specific category: - Would you add/move/remove any sub-categories? - Would you promote any sub-category to top level? General questions: - Would you use Castanet? - Would you use LDA? - Would you use Subsumption? - Would you use list of most frequent terms?

37 Evaluation Results Results on recipes collection for Would you use this system in your work? # Yes in some cases or yes, definitely : Castanet: 29/34 LDA: 0/18 Subsumption: 6/16 Baseline: 25/34 Average response to questions about quality (4 = strongly agree )

38 Evaluation Results Average responses for top-level categories 4= no changes, 1 = change many Average responses for 2 subcategories

39 Needed Improvements Take spelling variations and morphological variants into account Use verbs and adjectives, not just nouns Normalize noun phrases Allow terms to have more than one sense Improve algorithm for assigning documents to categories.

40 Opportunities for Tagging New opportunity: Tagging, folksonomies (flickr, de.lici.ous) People are created facets in a decentralized manner They are assigning multiple facets to items This is done on a massive scale This leads naturally to meaningful associations

41 Conclusions Flexible application of hierarchical faceted metadata is a proven approach for navigating large information collections. Midway in complexity between simple hierarchies and deep knowledge representation. Currently in use on e-commerce sites; spreading to other domains Systems are needed to help create faceted metadata structures Our WordNet-based algorithm, while not perfect, seems like it will be a useful tool for Information Architects.

42 Conclusions Castanet builds a set of faceted hierarchies by finding IS-A relations between terms using WordNet. The method has been tested on various domains: medicine, recipes, math, news, arts, bibliographical records Usability study shows: Castanet is preferred to other state-of-the art solutions. Information architects want to use the tool in their work.

43 Learn More Funding This work supported in part by NSF (IIS ) For more information: Stoica, E., Hearst, M., and Richardson, M., Automating Creation of Hierarchical Faceted Metadata Structures, NAACL/HLT 2007 See