The Information Retrieval Series. Series Editor W. Bruce Croft

Transcription

1 The Information Retrieval Series Series Editor W. Bruce Croft

2 Sándor Dominich The Modern Algebra of Information Retrieval 123

3 Sándor Dominich Computer Science Department University of Pannonia Egyetem u Veszprém, Hungary ISBN: e-isbn: Library of Congress Control Number: ACM Computing Classification (1998): H.3, G.1, G.3 c 2008 Springer-Verlag Berlin Heidelberg This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Cover Design: KünkelLopka, Heidelberg Printed on acid-free paper springer.com

4 To my parents Jolán and Sándor To my wife Em ke and our daughter Em ke

5 Acknowledgments Special thanks, first, to my family, the two Em ke, who were very understanding during the entire time I was involved in writing this book. Next, I would like to thank Ferenc Friedler, the head of the Department of Computer Science, University of Pannonia (Veszprém, Hungary), for providing a highly supportive environment. At some stages I derived much benefit from discussions with the following colleagues: Rozália Piglerné Lakner, Tamás Kiezer, Júlia Góth (all from the University of Pannonia, Veszprém, Hungary), Iadh Ounis (Glasgow University, Glasgow, Scotland, U.K.) and I enjoyed their active support. Parts of this book were included in a B.Sc. curriculum on Information Retrieval (Faculty of Information Technology, University of Pannonia, Hungary). The experience thus gained as well as students feedback proved very helpful, especially for elaborating examples and proposed problems, and in presenting methods. May all find here my expression of gratitude. Last but not least, I am indebted to Springer-Verlag Gmbh for making the publication of this book possible. Sándor Dominich

6 Contents 1 Introduction Information Retrieval Brief History of Information Retrieval Definition of Information Retrieval Retrieval Methods Modern Algebra Equations Solving by Radicals Birth of Modern Algebra Modern Algebra Lattice Importance of Lattices Lattices in Information Retrieval Retrieval Systems Boolean Retrieval Vector Space Retrieval Fuzzy-Algebra-Based Retrieval Methods Probabilistic Retrieval Web Retrieval and Ranking Exercises and Problems Mathematics Basics Elements of Mathematical Logic Proposition Negation Conjunction Disjunction Implication Equivalence Elements of Set Theory Set Subset Equality of Sets...34

7 X Contents Set Union Set Intersection Set Difference Cartesian Product Set Complement Powerset Cardinality of Set Properties of Set Operations Elements of Relations Theory Binary Relations Function Predicate Equivalence Relation Ordering Relation Partially Ordered Set Partition Exercises and Problems Bibliography Elements of Lattice Theory Lattice Lattice and Poset Duality Hasse Diagram Complete, Atomic Lattice Modular Lattice Sublattice Distributive Lattice Complemented, Orthomodular Lattice Boolean Algebra Important Lattices Powerset Lattice Lattice of Logical Propositions Lattice of Logical Predicates Lattice of Logical Implications Lattice Types Exercises and Problems Bibliography...64

8 Contents XI 4 Basics of Information Retrieval Technology Documents Power Law Stoplist Stemming Inverted File Structure Term-Document Matrix General Architecture of a Retrieval System Elements of Web Retrieval Technology World Wide Web Major Characteristics of the Web General Architecture of a Web Search Engine General Architecture of a Web Metasearch Engine Measurement of Relevance Effectiveness Relevance Measures Precision-Recall Graph Method Uncertainty of Measurement Measurement of Search Engine Effectiveness M-L-S Method RP Method Exercises and Problems Lattice-Based Retrieval Systems Mooers Model Lattice of Documents Lattice of Unstructured Queries Lattice of Term Hierarchies Lattice of Boolean Queries and Documents The FaIR System Galois (Concept) Lattice-Based Models Galois (Concept) Lattice Term-Document Matrix and Concept Lattice BR-Explorer System Rajapakse-Denham System The FooCA System Query Refinement, Thesaurus Representation Properties of the Lattices Applied Exercises and Problems...123

9 XII Contents 6 Boolean Retrieval Boolean Retrieval Method Technology of Boolean Retrieval Lattice-Based Boolean Retrieval Exercises and Problems Lattices of Subspaces and Projectors Metric Space Complete Metric Space Linear Space Subspace of Linear Space Linear Operator Banach Space Hilbert Space Euclidean Space Projection Theorem Projector Basis of Subspace Lattice of Subspaces Exercises and Problems Bibliography Vector Space Retrieval Introduction Lattices in Vector Space Retrieval Vector Space Retrieval Technology of Vector Space Retrieval Calculation of Meaning Using the Hilbert Lattice Queries with Negation Queries with Disjunction Compatibility of Relevance Assessments Vector Space Retrieval: Lattice-Lattice Mapping Discussion Query Lattice and Free Will Vector Space Retrieval? Vector Space Retrieval and Quantum Mechanics Exercises...177

10 Contents XIII 9 Fuzzy Algebra-Based Retrieval Elements of Tensor Algebra Similarity Measure and Scalar Product Latent Semantic Indexing Retrieval Eigenvalue, Eigenvector Singular Value Decomposition Latent Semantic Indexing Generalized Vector Space Retrieval Principle of Invariance Elements of Fuzzy Set Theory Fuzzy Set Fuzzy Intersection Fuzzy Union Fuzzy Complement Fuzzy Subset Retrieval Using Linear Space Fuzzy Algebra-Based Retrieval Methods Fuzzy Jordan Measure Fuzzy Entropy Retrieval Method Fuzzy Probability Retrieval Method Experimental Results Discussion More on Measures More on Algebra, Entropy, and Probability Information Retrieval and Integration Theory Principle of Invariance and String Theory Exercises and Problems Probabilistic Retrieval Elements of Probability Theory Principles of Probabilistic Retrieval Probabilistic Retrieval Method Language Model Retrieval Method Lattice Theoretical Framework for Probabilistic Retrieval Bayesian Network Retrieval Exercises Web Retrieval and Ranking Web Graph Link Structure Analysis...246

11 XIV Contents 11.3 The PageRank Method The HITS Method Application of the HITS Method in Web Retrieval Latent Semantic Indexing and HITS The SALSA Method The Associative Interaction Method Artificial Neural Networks Associative Interaction Method Application of the Associative Interaction Method in Web Retrieval Combined Methods Similarity Merge Belief Network Inference Network Aggregated Method Lattice-Based View of Web Ranking Web Lattice Chain Ranking Global Ranking Structure-Based Ranking P2P Retrieval P2P Network Information Retrieval Lattice-Based Indexing Exercises and Problems Solutions to Exercises and Problems Reference Index...321