The Architecture of a System for the Indexing of Images by. Content

Transcription

1 The Architecture of a System for the Indexing of s by Content S. Kostomanolakis, M. Lourakis, C. Chronaki, Y. Kavaklis, and S. C. Orphanoudakis Computer Vision and Robotics Laboratory Institute of Computer Science, Foundation for Research and Technology, HELLAS 1 and Department of Computer Science, University of Crete P.O. Box 1385, GR-71110, Heraklion, Crete, Greece Summary This paper presents the architecture of an image database system which provides a platform for the accommodation of various algorithms for interactive and automatic indexing, storage, and retrieval of medical images by content. The system maintains a dynamic hierarchy of image classes. The class hierarchy is used to narrow down the search to images of the same modality, anatomical characteristics, etc. Each image is classied into an image class based on information supplied by the user or obtained from the image itself. An important feature of the system is its ability to support multiple image indexing by content methods, in the form of description types. During system installation, one or more description types are selected for each image class based on the inherent characteristics of the class. Then, for each description type, a content description of each image in the class is generated and inserted in the description database. In the case of a query, the system generates one or more descriptions of the query image, automatically or interactively, and searches the logical database for similar descriptions. The images whose descriptions match those of the query image are retrieved for browsing by the user. This system may also be used as a platform for evaluating image content description methods, since new methods can be added to it easily. Introduction An Database (IDB) is an integrated system for the storage and retrieval of large amounts of image data and related information[3]. indexing and retrieval by content addresses the problem of retrieving from an IDB images similar to a query image. We believe that image retrieval by content will prove valuable to the medical imaging eld by providing considerable clinical decision support and increased eciency and effectiveness in diagnostic image interpretations. In this paper, we present the architecture of a system that serves as a platform for the implementation, testing, and evaluation of class-based indexing by content algorithms, since such algorithms of general applicability are not currently available. Our system is built around the concept of image classes to achieve eectiveness and eciency. We achieve eectiveness by employing algorithms tuned to perform well on a particular image class and eciency in retrievals by directing queries to appropriate image classes. 1 This work was funded in part under project EURIPACS of the CEC AIM programme.

2 The remainder of the paper is organized as follows: First, we present a brief overview of the proposed architecture. Then, we focus on the indexing by content module of our system, and conclude with plans for future work. The Architecture Since the main goal of this work is the development of a platform for the implementation, testing, and evaluation of indexing by content methods, this paper does not address image content description, or search and retrieval strategies, directly. In our system, distinct approaches to content description generation, storage, and matching are encapsulated in description type objects. The emphasis is on the development of an objectoriented environment (core) capable of easily integrating dierent description types and image processing algorithms. This core (see Figure 1), acts as a host to various state-ofthe-art algorithms which are incorporated to the system, at regular system maintainance. The main modules of the system core are the image visualization module that hosts image enhancement algorithms, the segmentation module that hosts segmentation algorithms, the image description module that hosts description types, and the storage manager that maintains the image classes known to the system and their attributes in the form of persistent objects (persistent is an object whose lifetime exceeds that of the program that created it.). In the current version of the system, the storage manager is based on EXODUS[1]. Appropriate User Interface tools[2], such as image browser, the contour editor, etc., support user friendly input and access to the results of algorithms. This design approach resulted in a loosely coupled architecture, consisting of a set of autonomous communicating modules. Various modules request services from each other using specialized communication objects. Classes s are assigned to classes using criteria such as imaging modality, anatomical characteristics, orientation, plain of cut, etc. For each image class, the system maintains a class object that encapsulates attributes relevant to the images in it. Class attributes are segmentation algorithm types, description types and general information on the class. A segmentation algorithm type identies a segmentation algorithm object (see section on segmentation) and the class default values for the input parameters of the segmentation algorithm. We expect that images in the same class have the same source and represent similar objects, hence call for similar treatment. Specialized algorithms can be developed and tuned for the images in a specic class. Such algorithms are likely to yield far better performance than general purpose algorithms, since they can exploit knowledge of the content of images in the class. For ecient retrieval by content, we impose a class based partitioning of the image space. Each description type supported by a given class maintains its own database of content descriptions of the images in the class. In this way, a CT image of the head is never compared for similarity to a CT image of the abdomen. The association of segmentation algorithm types and description types to image classes is dynamic. The user may bind (or unbind) an algorithm or description type to a class.

3 Segmentation Algorithms... Enhancement Alogrithms Core Segmentation Module Types. Visualization Module User Interface Module. Storage Manager Figure 1: The system architecture Segmentation Usually, the rst step towards description generation is image segmentation. Segmentation derives a set of disjoint regions or objects from an input image. The segmentation module provides the support for class-based segmentation of images. The introduction of a new segmentation algorithm to the system is an o-line operation that registers the new algorithm with the segmentation module, usually with minimal impact to the rest of the system. For each segmentation algorithm known to the system, the segmentation module maintains a persistent object, namely, asegmentation algorithm object. This object contains the name of the algorithm, the name, type, and default value for each of its input parameters, and a means of identifying the code which implements the algorithm. A segmentation algorithm takes an image object as input and produces a contour object as output. The role of the derived contour object is twofold. First, it is used as an alternative input to the description generation algorithms and, second, it is stored in the

4 database and used in browsing through the contours of images retrieved by a query. Segmentation may proceed either as an interactive or as an automatic process. In interactive segmentation, the user rst enhances the appearance of the image in the image visualization module (see Figure 2). Then, the user experiments with one or more available segmentation algorithms, changing their default values, comparing results, etc. When a contour image that is relatively satisfactory is produced, it can also be edited interactively using the contour editor (see Figure 2). Figure 2: The user interface of the prototype. Since interactive segmentation is a time consuming process, class-based automatic segmentation oers a viable alternative. We view our system as evolving: users may change the default parameters of a segmentation algorithm within an image class, change bindings of algorithms to classes, and introduce new class-specialized segmentation algorithms. The description generation method, the description manager, and the description matching method are the basic components of a description type (see Figure 3). The description generation method produces the content description consisting of a set of persistent objects. The description manager stores, retrieves, and modies content descriptions in the logical database of the description type. Finally, the description matching method compares the content description of a query image to content descriptions of images in the logical database and identies images similar to the query image. The object-oriented approach we followed imposes minimal constraints on the description methods that can be supported by the system. Each description type is viewed as an object capable of generating image content descriptions, storing them and retrieving

5 image contour image query query Generation Method description Matching Method similar images Manager (persistent store) Figure 3: type structure images with similar descriptions from the database. Thus, the system does not concern itself with the schema of the logical database or the search and retrieval strategies associated with a specic indexing and retrieval by content algorithm. Such knowledge is embedded in the description type object of each individual indexing by content algorithm. What the system provides is a set of primitive operations that enable indexing by content methods to create persistent clusters of objects, maintain indices on them, retrieve/insert objects from/to a cluster, etc. Indexing by content methods make use of these primitives to dene their storage and retrieval strategies. The description generation process associated with a description type can be either interactive or automatic. The description generation and description matching processes, are generally automatic. However, when the content description is contour-based, the user may interact with the system for the generation of the contour object. Two basic forms of query by content are currently supported. First, the user displays an image in the image visualization module, and selects one of the description types available to the relevant class to serve the query. The retrieved images are displayed in the image browser as miniatures. Second, the user draws a rough sketch on the contour editor, and again selects an image class and a description type to serve the query. Conclusions This paper presented the architecture of an image database system for the indexing of images by content. It forms the basis for a prototype that is currently under development in our laboratory. Furthermore it is expected to give added value to integrated Picture Archiving and Communication Systems and to support computer aided diagnosis, medical training, and research.

6 References [1] Using the EXODUS Storage Manager V2.2. Technical report, Computer Science Department, University of Wisconsin Madison, November 92. [2] Petros Kofakis and Stelios C. Orphanoudakis. Graphical Tools and Retrieval Strategies for Medical Databases. In Proceedings of the International Symposium on Computer Assisted Radiology 91, pages 519{524. Springer-Verlang, [3] Hideyuki Tamura and Naokazu Yokaya. Database Systems: A survey. Pattern Recognition, 17(1):29{49, 1984.