Generation of Large Hypertext and Document Networks from Collective User Retrieval Patterns

Size: px

Start display at page:

Download "Generation of Large Hypertext and Document Networks from Collective User Retrieval Patterns"

Eustace Hawkins
5 years ago
Views:

1 Generation of Large Hypertext and Document Networks from Collective User Retrieval Patterns Johan Bollen Old Dominion University Department of Computer Science jbollen July 5, 2002 Page 1

2 A Personal Introduction Johan Bollen 1. Principia Cybernetica Project, U. Brussels ( ) (a) Web site on philosophy of cybernetics (b) Designed to represent knowledge network, systems science encyclopedia (c) Collective effort of 4 editors (d) Adaptive Hypertext and models of navigation 2. Active Recommendation Project, Los Alamos National Laboratory ( ) (a) Focus on Distributed Information Systems (b) Generation of document proximities for DL recommendation systems (c) Application of spreading activation systems 3. Old Dominion University, Computer Science Department (a) Systems for scientometric analysis (b) Large scale applications of spreading activation for cross-resource recommendation (c) Temporal analysis of changes in user community preferences July 5, 2002 Page 2

3 Problem Statement 1. Initially: Hypertext (a) Design and use: Collection of documents need to be meaningfully linked and user retrieval needs to be made more efficient (b) Design: i. Manual Design: Uneconomical, and user preferences? ii. Taxonomies: Correspondance to user preferences? iii. IR methods: Dependance on text, and explicit queries (c) Use: i. Lost in Hyperspace ii. Users prefer Boolean searching 2. More generally: Digital Libraries (a) Desire to link collections and provide retrieval facilities beyond keyterm matching (b) Heterogeneous collections (format, language, etc) (c) Scalability issues July 5, 2002 Page 3

4 Basic Approach 1. Hypertext linking: (a) Shift from focus on design to usage (b) Patterns of retrieval determine generation of hyperlinks (c) Relations to Machine Learning: auto-associators and Hebbian learning 2. Hypertext retrieval: (a) Focus on use of network structure rather than document content (b) Recurrent spreading activation (c) Relations to CLEVER and PAGERANKER 3. Digital Libraries (a) Reconstruction of retrieval patterns from server logs (b) Generation of large scale document and journal networks (c) Network analysis for document and journal impact ranking July 5, 2002 Page 4

5 Presentation Structure 1. Background: Hypertext linking (a) The early experiments (b) Results and applications (c) Relevancy to general WWW interfaces 2. Digital Libraries (a) System for document linking from server logs (b) Journal and document impact ranking (as opposed to ISI IF) (c) Spreading Activation retrieval July 5, 2002 Page 5

6 USER USER MM Automated Hypertext Linking: first experiments MM Designer MM USER USER MM Learning Rules 0.03 MM USER USER MM MM Hypertext USER MM 0.4 Adaptive Hypertext USER MM USER MM 1. Replace explicit design or text generated links by user retrieval 2. Frequency of retrieval sequence indicates strength of associative relation 3. Shift from explicit author or designer preferences, to implicit user preferences July 5, 2002 Page 6

7 Experiment Setup 1. Representation (a) Weighted Hyperlinks: weights indicate degree of association (b) Directed Weighted Graph Representation of Network Structure i. Assumption is asymetric associations ii. Corresponds to word association research and WWW hyperlinks 2. Adaptation (a) Reinforcements based on Hyperlink Traversals (b) Learning Rules Operate on Hyperlink Weights 3. Interface i. Small additions to link weights for each traversal ii. Aaimed at gradual adaptation of structure (a) Weighted Ordered Ranking of Hyperlinks (b) Dynamic Re-Ordering (c) Display slices of 10 hyperlinks July 5, 2002 Page 7

8 Graph Representation Hypertext Graph Representation (Furner,1996), (Kleinberg, 1999) G V L V v 0 v 1 v n 1 L V V v i v j L represents hyperlink v i V v j V n n matrix, H, binary values h i j 0 1 hyperlink: v i v j L : h i j 1 absence hyperlink: v i v j L : h i j 0 July 5, 2002 Page 8

9 Graph Representation Assumptions 1. Hypertext Pages Graph Nodes? (a) Homogeneous content (b) Singular Concept 2. Hyperlinks Graph Edges (a) Singular anchor and target (b) Hyperlink Semantics (c) Directed Connections Simplification 1. Feature rather than bug 2. Goal is not to construct best hypertext representation but study larger class of linked information systems July 5, 2002 Page 9

10 Learning Rules G V L add weight function W, G V L W W : V 2 W v i v j User Request Sequence (RS): p RS Sub-path Triplet: p s v 0 v 1 v n 1 n v i v j v k i j k j i 1 k i 2 i n 2 July 5, 2002 Page 10

11 Learning Rules Frequency (F) or F p s W : W v i v j t W v i v j t 1 r f Symmetry (S), or S p s W : W v j v k t W v j v i t W v j v k t 1 r f W v j v i t 1 r s Transitivity (T), or T p s W : W v k v j t W v i v k t W v k v j t 1 r s W v j v k t 1 r t July 5, 2002 Page 11

12 Learning rules 1. Frequency: (a) Operates on link traversal frequency (b) Reinforce existing connections (c) Hebbian Learning 2. Transitivity: (a) Operates on transitive co-retrievals (b) Bridges plausible connections 3. Symmetry: (a) Reinforce connection symmetric to those traversed (b) Reinforce plausible connections July 5, 2002 Page 12

13 freq. freq. freq. house table stool bar Example symm. symm. symm. trans. trans. July 5, 2002 Page 13

14 house door table family stool chair bottle 0.9 board 0.8 bar 0.3 wood Example, cont. 0.6 chair 1.5 table bottle door 0.6 house bar stool 0.9 board 0.3 family 0.3 wood July 5, 2002 Page 14

15 Interface July 5, 2002 Page 15

16 Link Ranking weight rank hyperlink period moment history year change day section age situation thought man end experience week blood development action century evening light mind morning theory part movement life course attention service nature sort society music government way... GROUP 1 GROUP 2 GROUP 3 GROUP 4 Hyperlink Selection "More Items" Hyperlink Selection "More Items" Hyperlink Selection "More Items" Displaying links 1. Display shows groups of 10 links 2. Rank ordered: first slice displays 10 strongest links 3. User can shift from one group to the other July 5, 2002 Page 16

17 Adaptive Hypertext System Construction Representation learning rules Interface link order page generation path Adaptation request Users July 5, 2002 Page 17

18 Experiments 1. Open Participation (a) No Registration of Subjects (b) No Data Gathered on Characteristics Motives of Subjects 2. Instructions to Browse Experimental Hypertext System (a) Associative Browsing: Position based Hyperlink Selection (b) Request not to Apply High-level Strategies 3. Specific Choice of Node Labels (a) English nouns (b) 150 most frequent 4. Two Experiments (a) Frequency Transitivity (r f 1, r t 0 5) (b) Frequency Transitivity + Symmetry (r s 0 3) July 5, 2002 Page 18

19 Nodes act action age amount area art attention authority bed blood board body book boy building car case century change church city committee commonwealth company conference control council country course court day death development door doubt education effect end evening evidence example experience face fact family father field figure film food form friend girl government group hand head health heart history house idea industry influence interest job kind knowledge labor land language law level life light line love man market matter meeting method mind moment money morning mother movement music name nature need night number office order paper part party peace period person place point policy position power problem question rate reason research result road room school section sense service side situation society sort stage state story system table tax theory thing thought time town trade training type use value view voice water way week wife woman word work world year July 5, 2002 Page 19

20 Results 1. Participation (a) 6000 requests (b) 600 participants experiment 2. Resulting Networks: (a) Sensity (b) Weight values: x 3 07, max = 80 (c) Clustered structure (d) Wide range of generated connections July 5, 2002 Page 20

21 Network Matrix number label (1) act (2) action (3) age (4) amount (5) area (6) art (7) attention (8) authority (9) bed (10) blood Table 1: Sample of matrix representing final structure of adaptive hypertext network trained in experiment 1 July 5, 2002 Page 21

22 Centrality Un-normalized Centrality node i: n j 0 c w i j n j 0 w ji i 2 10 most central nodes: (1) label centrality knowledge experience development 261 mind education method thought system 181 body person 168 July 5, 2002 Page 22

23 action movement change influence power authority knowledge control system method experience reason development mind education thought body research Graph structure person life July 5, 2002 Page 23

24 Cluster Analysis Cluster Time Space Movement Control Cognition Intimacy Vitality Society Office Documents Age, time, century, day, evening, moment, period, week, year Place, area, point, stage Action, change, movement, road, car Authority, control, power, influence Knowledge, fact, idea, thought, interest, book, course, development, doubt, education, example, experience, language, mind, name, word, problem, question, reason, research, result, school, side, situation, story, theory, training, use, voice Love, family, house, peace, father, friend, girl, hand, body, face, head, figure, heart, church, kind, mother, woman, music, bed, wife Boy, man, wife, health Society, state, town, commonwealth Building, office, work, room July 5, 2002 Page 24

25 Hyperlink Development CONTROL Rank 1 Country Authority Authority Authority Power 2 Thing Society Rate Power Authority 3 Government Rate Society Effect Effect 4 Policy Theory Theory Knowledge Influence 5 Film Problem Problem Government Government 6 Heart Job Government Society Law 7 Group Service Job Money Knowledge 8 Room Money Money Father Society 9 Paper Knowledge Effect Law Order 10 Order Day Service Rate Money Requests: July 5, 2002 Page 25

26 Conclusion 1. Link generation: (a) Spontaneous proliferation of hyperlinks (b) Fast development of stable collection of links (c) Ranking reflects associative validity 2. Network structure: (a) Clustering of related items (b) In- and out-degree of ndoes reflects level of abstraction (c) Different from word association norms 3. Note: (a) Link generation independent from text or format (b) No requirement for explicit user preference statements (c) Distribution of link weight: few retrievals required July 5, 2002 Page 26

27 Digital Library Applications 1. Article and Journal Linking (a) Citation or IR based (b) Static (c) Non-specific (User Community) 2. Application of Adaptive HT: Off-line DL Setting (a) No intervention in actual library systems (b) Registration of User Requests: server logs (c) Reconstruction of User Paths 3. Large Number of Articles and Journals (a) Heterogeneous: Validity Atomic Concept representation? (b) Scalability? July 5, 2002 Page 27

28 Methodology 1. Reconstruction of co-retrieval events from DL Server logs (a) Use Sequence of Retrieval Requests to Derive Article and Journal Associations (b) Integrate access Patterns for Large Range of Applications, Users, etc. 2. Generate Associative Network of Journals (a) Every Article appears in a journal (b) Journal Relations subsume Article Relations 3. Use Network (a) Generate document links without resorting to text-content based IR (b) Map Preferences of LANL User Community (c) Recommendations Systems that Operate on Network Structure July 5, 2002 Page 28

29 Test Application 1. Los Alamos National Laboratory Research Library: (a) Large digital library with focus on management of own material (b) User community: focus on physics and nuclear science (c) SciSearch database: +20,000,000 articles (d) Downloadable: PDF format after search and metadata retrieval 2. Availability of server logs from 1998 to present (a) Download rates: 10,000/mo. (b) Server logs used: 2001 (c) Log file used: 5 months, 40,847 requests, 1829 journals and 20,720 articles July 5, 2002 Page 29

30 Interface July 5, 2002 Page 30

31 Interface July 5, 2002 Page 31

32 Network Generation Set of co-retrieval events: E e 1 e 2 e k Co-retrieval event e i : e i v i v j t v i v j where t v i v j represent time in seconds elapsed retrieval requests for document v i and v j e i v i v j t v i v j r, r f e i reinforcement value so that: m i j r i jm i j 0 for (i=1; i n+1; i++) e i v i v j t v i v j : m i j f e i July 5, 2002 Page 32

Retrieval Events D6 D5 D1 { Server Log D2 Document Network D3 D4 U100S1 U101S1 U102S1 U102S2 E(1,2) E(2,3) E(4,5) E(5,6) E(2,4) E(6,1) D1 + D2 + D3 W1,2 W2,3 D4 + D5 + D6 W4,5 W5,6 D2 +

33 Retrieval Events D6 D5 D1 { Server Log D2 Document Network D3 D4 U100S1 U101S1 U102S1 U102S2 E(1,2) E(2,3) E(4,5) E(5,6) E(2,4) E(6,1) D1 + D2 + D3 W1,2 W2,3 D4 + D5 + D6 W4,5 W5,6 D2 + W2,4 D4 D6 + W6,1 D1 User100 Session1 User101 Session1 User102 Session1 US: User Session D: Document Retrieval E: Co retrieval Event User102 Session2 W + Link Reinforcement July 5, 2002 Page 33

34 SciSearch Retrieval Log userid data/time Document ID Service /Aug/2001:17:12: ;14;4;69 ddacfrtip SciServer /Aug/2001:17:14: ;25;4;499 prtuauma SciServer /Aug/2001:17:15: ;19;2;281 apiaaaortars SciServer /Jun/2001:12:03: ;38;3;347 otfrim SciServer /Jun/2001:12:04: ;19;3;211 aotdfmfct SciServer /Jun/2001:13:13: ;25;2;113 asrgt3tr SciServer Table 2: An example of the transformed Science Server log used to reconstruct journal and document co-retrieval events. July 5, 2002 Page 34

35 SciSearch Retrieval Log Analysis 100 ( , ) ( , ) 101 ( , ) ( , ) Table 3: Co-retrieval events derived from set of user retrieval requests in table 2. July 5, 2002 Page 35

36 Analysis application July 5, 2002 Page 36

37 Generated Network Journal ADV COLLOID INTERFAC ADVAN MATH ANALYT CHIM ANALYT BIOC ANIMAL BEHAVIOUR ANN PHYS-NEW YORK APPLIED ACOUSTICS APPL PHYS LETT ARCH RATION MECH AN ARCH BIOCHEM BIOPHYS ARTIF INTELL ASTRONOM J PUBL ASTRON SOC JPN ASTRON ASTROPHYS ASTROPHYSICAL JOURNAL July 5, 2002 Page 37

38 Generated Network 1. density 0.176% 2. mean link weight = 1.196, standard deviation=0.821 for all m i j : m i j 0 July 5, 2002 Page 38

39 Network Analysis and Comparison to Impact Factor Structural Features 1. Highest link weights 2. Graph-theoretical network characteristics compared to Impact Factor (a) Impact Factor: based on citation frquency (ratio of citations over publications for last 2 years) (b) Journal Consultation Frequency: F v i n j 1 m i j n j 1 m ji (c) Impact Factor Discrepancy Ratio: r f i JCF i IF i July 5, 2002 Page 39

40 Highest Link Weights Frequency Start node End Node 22.0 PHYSICA C PHYSICA B 18.0 PHYSICA B PHYSICA C 14.0 NUCL INSTRUM METH A IEEE T NUCL SCI 12.0 PHYSICA B J MAGN MAGN MATER 11.0 J MAGN MAGN MATER PHYSICA B Table 4: Five pairs of journals for which highest co-retrieval frequencies have been found. July 5, 2002 Page 40

41 JCF vs IF Frequency IF Journal Title PHYSICA B NUCL INSTRUM METH A APPL CATAL A-GEN MAT SCI ENG A-STRUCT J ALLOY COMPD IEEE T NUCL SCI CATAL TODAY J CATAL SURF SCI J NUCL MATER Table 5: List of 10 most frequent journals in LANL RL Science Server network and 2000 Impact Factors. July 5, 2002 Page 41

42 Scatterplot JCF vs. ISI IF (2000) 35 JCF vs. IF r s = 0.13 (p 0 05) IF(2000) JCF July 5, 2002 Page 42

43 Impact Factor Discrepancy Ratio r f JCF IF Journal Title ASTROPHYS SPACE SCI NUCL ENG DES PHYSICA B J MATER PROCESS TECH ADV SPACE RES ACTA ASTRONAUT J RADIOANAL NUCL CH PROG NUCL ENERG NUCL INSTRUM METH A MATH COMPUT SIMULAT Table 6: List of 10 journals for which highest ratio of JCF and ISI IF has been found. July 5, 2002 Page 43

44 Spreading Activation 1. Model Facilitated, Automatic Retrieval from human memory. 2. Assumes Associative Network of Concepts. 3. Concept Activation spreads through network, modulated by connection weights. July 5, 2002 Page 44

45 Spreading Activation for Information Retrieval 1. Promises (a) Text and Language Independent (b) Cue-based retrieval or retrieval by example (c) Use of Network structure: incomplete structure can still lead to complete retrieval 2. Problems (a) Establishment of Document Networks (b) Questions regarding efficiency (Salton, 1988) (c) Main Application in Concept Networks, Query Expansion July 5, 2002 Page 45

46 Spreading Activation on Journal Titles 1. Features (a) Transformation of initial keyterm query to network initial activation values (b) Recommendation Refinement by Query by example (c) Network interface (Servlet), marked-up output 2. Implementation (a) Java Servlet i. Third-party service for LANL DL ii. jbollen (b) Packed Query (c) Link to LANL RL journal pages i. Where available ii. Added link to LinkSeeker service 3. Prototype: SciS loop July 5, 2002 Page 46

47 Implementation July 5, 2002 Page 47

48 Query July 5, 2002 Page 48

49 Results July 5, 2002 Page 49

50 Query Expansion July 5, 2002 Page 50

51 Dynamic Network representation July 5, 2002 Page 51

52 Conclusion 1. What have we got? (a) Methodology to link documents across formats, content and languages. (b) Method to analyse networks and obtain information on preferences and structure of user community (c) Generation of alternatives for citation based impact factors (d) Spreading Activation Retrieval 2. Issues: (a) Applications to other information resources (b) Scalability (c) Sufficient server logs for network training (d) Hebbian learning assumption (invalid labeling, user relevance) July 5, 2002 Page 52

Lecture 9: I: Web Retrieval II: Webology. Johan Bollen Old Dominion University Department of Computer Science

Lecture 9: I: Web Retrieval II: Webology Johan Bollen Old Dominion University Department of Computer Science jbollen@cs.odu.edu http://www.cs.odu.edu/ jbollen April 10, 2003 Page 1 WWW retrieval Two approaches