Automatic Text Processing

Transcription

1 Automatic Text Processing The Transformation, Analysis, and Retrieval of Information by Computer Gerard Salton Cornell University Technlsche Univerariat Darmstadt FACHBEREICH1NFORMATJK BIBLIOTHE.K Invented.: Sachgebteta: Standort: TT ADDISON-WESLEY PUBLISHING COMPANY Reading, Massachusetts Menlo Park, California New York Don Mills, Ontario Wokingham, England Amsterdam Bonn Sydney Singapore Tokyo Madrid San Juan

2 Contents 2 Part 1: The Information-Processing Environment 1 The Information Environment Automatic Information Processing Types of Information Text Processing Speech Processing Graphics Processing Semantic and Behavioral Processing 8 The Computer Environment Computer Architecture Large versus Small Machines Sequential versus Parallel Computing Multiprocessor and Multicomputer Configurations Types of Computers Storage Technology Input-Output and Peripheral Equipment 23 { Terminal Equipment Printing Equipment Document Input Computer Networks Integrated Computing Systems 36

3 The Automated Office The Office Environment Analyzing Office Systems File-management Systems System Characteristics Relational Database Systems Relational Data Manipulations Data Security, Integrity, and Recovery 60 3*4 Office Display Systems Office-Information Retrieval 63 Part 2: Word Processing and File Access 71 Text Editing and Formatting Introduction Approaches to Word Processing Text Editing and Formatting Typical Processing Systems Off-line Text-Editing Systems Interactive Graphics-Editing Systems Automatic Typesetting Systems Typesetting Systems Automatic Typefont Design 98 Text Compression Statistical Language Characteristics 105 <$ Frequency Considerations 105 2> Entropy Measurements Rationale for Text Compression Text-Compression Methods 114 ^ Special-purpose Compression Systems Basic Fixed-Length Codes Restricted Variable-length Codes 119 5J.4 Variable-length Codes Word-Fragment Encoding 125 Text Encryption Basic Cryptographic Concepts Conventional Cryptographic Systems Sample Cryptographic Ciphers The Data Encryption Standard (DES) 146 «8» 6.5 Ciphers Based on Computationally Difficult Problems 149

4 7 8 9 Automatic File-Accessing Systems Basic Concepts Single-Key Searching: Sequential Search Single-Key Indexed Searches Tree Searching Balanced Search Trees Multiway Search Trees Hash-Table Access Indexed Searches for Multikey Access Bitmap Encoding for Multikey Access Multidimensional Access Structures 216 Part 3: Information-Retrieval Systems 227 Conventional Text-Retrieval Systems Database Management and Information Retrieval Text Retrieval Using Inverted Indexing Methods Extensions of the Inverted Index Operations Distance Constraints Term Weights Synonym Specifications Term Truncation Typical File Organization Optimization of Inverted-List Procedures Reducing the Number of Index Terms Quorum-level Searches Partial List Searching Text-scanning Systems General Considerations Elementary String Matching Fast String Matching Hardware Aids to Text Searching 266 Indexing Indexing Environment Indexing Aims Single-term Indexing Theories Term-frequency Considerations Term-discrimination Value Probabilistic Term Weighting Term Relationships in Indexing Term-phrase Formation Thesaurus-Group Generation A Blueprint for Automatic Indexing 303

5 10 II Advanced Information-Retrieval Models The Vector Space Model Basic Vector-processing Model Vector Modifications Automatic Document Classification General Considerations Hierarchical Cluster Generation Heuristic Clustering Methods Cluster Searching Probabilistic Retrieval Model Extended Boolean Retrieval Model Fuzzy Set Extensions Extended Boolean System Integrated System for Processing Text and Data Advanced Interface Systems 365 Part 4. Text Analysis and Language Processing 375 Language Analysis and Understanding The Linguistic Approach Dictionary Operations Morphological Decomposition Dictionary Types Syntactic Analysis Typical Syntactic-Analysis Systems Semantic Grammars Knowledge-based Processing Knowledge Structures Prospects for Knowledge-based Processing Specialized Language Processing Robust Parsing Sublanguage Analysis Natural-Language Interface to Information Systems Automatic Text Transformations Text Transformations Automatic Writing Aids Automatic Spelling Checkers Automatic Spelling Correction Syntax and Style Checking 436

6 Automatic Abstracting Systems Automatic Extracting Abstracting Based on Text Understanding Automatic Text Generation 448 { Approaches to Text Generation 448 J* Typical Text-generation Systems Automatic Translation Main Approaches Typical Machine-translation Systems 461 Paperless Information Systems Paperless Processing Processing Complex Documents Graphics Processing Basic Display Systems Object Transformations Picture Recognition Speech Processing Speech Synthesis Speech Recognition Automatic Teleconferencing Systems Electronic Mail and Messages Electronic Information Services Teletext Videotex Electronic Publications and the Electronic Library 507 Author Index 517 Subject Index 523 XIII