TENCON 2000 explore2 Page:1/6 11/08/00 EXPLORING ON STEGANOGRAPHY FOR LOW BIT RATE WAVELET BASED CODER IN IMAGE RETRIEVAL SYSTEM S. Areepongsa, N. Kaewkamnerd, Y. F. Syed, and K. R. Rao The University of Texas at Arlington, Box 19016, TX 76019. e-mail: sxa5039@omega.uta.edu, nxk4652@utarlg.uta.edu, yassers@home.com, krrao@exchange.uta.edu Abstract: In this paper, the image retrieval system that provides an efficient retrieval, management and transmission of selected image(s) from the database is proposed. The key point of research is utilizing steganographic technique to achieve the efficient use of resources by embedding attributes into the image contents. To avoid the degradation of image quality, the attributes are invisibly embedded in the edge representations of the compressed domain of the ZTE (Zerotree Entropy)/modified SPIHT (Set Partitioning in Hierarchical Trees) wavelet based coder. The evaluations of the proposed algorithm have shown several significant advantages. For example, (1) fast transmission of the retrieved image to the receiver, (2) allows searching based on the retrieved images, (3) no reprocessing of the attributes for other applications, (4) no extra bits for the conventional thumbnail and (5) no extra bits for attributes. keywords: Retrieval system, Steganography, Wavelet based coder, Embedding information. I. INTRODUCTION According to the forthcoming MPEG-7 standard [1], the activities in image retrieval have been increasing. Strategies proposed for image retrieval systems basically approach content-based algorithms that use the visual content of image to retrieve the similar images from the database. Normally, the contents (attributes) used in content-based are color, shape, texture and spatial properties [2]. By utilizing the combination of this information, several retrieval systems have achieved promising performance [4,5]. In these systems, once the relevant images have been selected, only the image data and not the attribute information is delivered to the user terminal. In terms of resources, it is an ineffective use of information. In this paper, the novel retrieval system in which the attributes are embedded into an image content and transmitted to the user s terminal is proposed. Embedding attributes into the image content provides more advantages for the end user such as, (1) enhancing the retrieval performance of conventional systems and (2) allowing an efficient use of the resources (attributes). First, the system allows further searches using the attributes of the retrieved image without repeating the query process. This differs from the conventional systems in which further search based on the retrieval results is impossible due to the unavailability of the attributes. Second, the attribute information is useful for purposes besides retrieval. As the attributes (shape, texture, color and spatial properties) are valuable information, they can be exploited for other applications i.e., manipulating the individual object in an image such as image analysis, image segmentation and etc. Transmitting the attribute information along with the image to the receiver is practical for the real world implementations. However, it requires more storage space and transmission cost. To overcome this problem, steganographic approach is utilized to transmit the attributes to the user without sacrificing the transmission performance and preserving the image quality. This can be accomplished by embedding the attributes into the insignificant information of the image. Note that the embedded information does not degrade the perceptual image quality and needs no extra bits. To achieve the reliability and controllability of the embedded information, generally steganography is performed in pixel domain. However, in the retrieval systems, embedding information in pixel domain requires enormous extracting time since the information contained in the database is compressed (i.e., using JPEG compression). Though attempts on embedding information in compressed domain have been investigated, both embedding and extracting operations are very complex. To solve this problem, embedding information in the compressed domain of the ZTE/modified SPIHT wavelet coder or HC-RIOT (Homogenous Connected-Region Interested Ordered Transmission) is considered. As it encodes an image into compressed bitstream using bit-plane approach, the bits of an image in the compressed domain are coded in a descending order from the most significant bits (MSB) to the least significant bits (LSB). This provides a flexibility to the steganographic environment. The attributes can be simply selected to invisibly embed in the less significant bit areas thus providing a fast and reliable extractability. II. OVERVIEW OF THE PROPOSED SYSTEM The proposed retrieval system consists of two main phases (Fig. 1), attribute generation phase and query handling phase. In this section, the descriptions of these two phases, including an additional part (user feedback) are provided.
TENCON 2000 explore2 Page:2/6 11/08/00 1) s generation Phase The attribute generation phase serves as a preprocessing step for an image retrieval system. It prepares, arranges, stores and manages information in the database. Generally, this phase consists of segmentation, feature extraction and data compression operations. However, in the proposed system, the embedding process is added to enhance the system performance. Note that the details of feature extraction used in the proposed system are described in section 5. To obtain the attributes, first, an image is converted into HSV format. Utilizing the brightness information (V) and color information (S and H), the image is segmented into groups of regions (objects) of interest (ROI). The proposed system uses the semi-automatic hybrid segmentation technique (region growing method) such that the manual operation can be performed among related ambiguous subregions to achieve the higher quality of object representation. After segmentation, each object is analyzed (attribute extraction process) obtaining content features (color, texture, shape and spatial properties). In the proposed algorithms, instead of storing separately, the attributes are embedded in the corresponding image using steganographic approach. Note that in the proposed system, the images are compressed by using the HC-RIOT algorithm and stored in the database. Using HC-RIOT does not only allow a proper environment for steganographic process, but can provide other benefits to the retrieval system. Generally the retrieval database needs to provide three types of information for different purposes: first, the attribute information for retrieval, second, the thumbnail (lowresolution image version) for fast browsing, and third, the fine image (high-resolution image). Since a thumbnail is a coarse version of the fine image, storing both versions in the database is redundant. The HC- RIOT provides two bitstreams: base layer (high compression) and enhancement layer bitstreams, the base layer provides a recognizable version of image analogous to thumbnail in conventional systems. Therefore, using HC-RIOT can eliminate the need to store the conventional thumbnail. 2) Query handling phase The query handling phase serves as the user interface. It mainly consists of three processes, query generation, matching operation and content extraction. As we introduce the steganographic process in the proposed system, the content extraction is required to decode the embedded attributes. In query generation process, user picks up the query image from the available sample images, which are initially given by the system. The selected query image is segmented into meaningful regions. Then only the interested regions are analyzed and extracted to form the query attributes. These attributes are sent to the matching process to search for the possible outputs in the database. Then, in matching process, the similarity between attributes of the query images and database images is measured. After matching operation, the images with maximum likelihood are retrieved and the thumbnails of all candidate images are delivered to the user's terminal. Once user selects images from a candidate set, the rest of corresponding image contents including embedded attribute information are transmitted. 3) User Feedback User feedback is an optional task in the proposed system. This allows further feedback by modifying the query to the system. If the retrieved images do not satisfy the user's need, he/she can choose the new interested regions that may describe more specific details or interact with the system to narrow the results of the previous retrieval stage. This process is being developed. III. THE HC-RIOT CODER By using a combination ZTE [10] and SPIHT[11] coder, the HC-RIOT coder accommodates several properties such as scalability and progressive transmission. As shown in Fig. 2, the HC-RIOT outputs two bitstreams, the base layer and the enhancement layer. The base layer, a very high compressed bitstream, gives a decodable image with a fixed rate transmission while the enhancement layer improves the image quality. HC - RIOT Encoder in Base Layer Enhancement Layer Network Constraints & Viewer limitations fixed rate transmission progres sive transmission Figure 2. Transmission of images using HC-RIOT coder HC - RIOT Decoder The multi-layer bitstream provides both perceptual optimization for fixed bit rate transmission (proper for video transmission at low bit rates) in base layer and progressive transmission in the enhancement layer. The key development that adds value to this encoder compared to the state-of-art is the use of the Wavelet Block Chain (WBC) in the base layer to identify and label homogenous block based regions of the image easily for transmission to the decoder. It can improve perceptual coding in two ways. First, the WBC aids in classifying image blocks into edge, smooth or detail such that the ZTE and SPIHT coders can factor this information into its rate distortion criteria. Second, a list of insignificant regions (LIR) can be created from the WBC information, which can reduce the amount of insignificant coefficients in the LIP. Spatial correction information similar to the SPECK [12] techniques is applied to indicate insignificant groups, which are then organized into regions. This improves the image quality by reducing the number of bits early in transmission that
TENCON 2000 explore2 Page:3/6 11/08/00 does not contribute to reducing the distortion metric. The WBC method can also aid in spatial segmentation methods by identifying large homogenous regions by a single label. This allows objects to be easily segmented later on from a video transmission in progress and allows for separate enhancement layers to be established for additional object information. For steganography, the HC-RIOT allows bits to be hidden in insignificant wavelet coefficients of regions in the image, which are less sensitive to visual degradation IV. THE STEGANOGRAPHY IN HC-RIOT CODER Embedding attributes into the image content is a new concept in image retrieval system. It provides more advantages for the end user in terms of resource exploitations. In this section, the steganography in HC- RIOT compressed domain is proposed. The encoder and decoder of a steganographic retrieval system are illustrated in Fig. 3 and Fig. 4. The proposed system is designed based on the consideration of three significant requirements: (1) invisibility, (2) capacity and (3) detectability. To improve the invisibility, the location of embedding is determined. In the proposed strategy, the attributes are embedded in only the enhancement layer representing edge and detail region (embedding in base layer is proposed in [3]). Due to the spatial segmentation strategy of the HC-RIOT coder, the WBC (Wavelet Block Chain) identifies and labels an image into homogenous regions such as edge, detail and smooth areas. These homogenous areas provide a crucial information for embedding process. Based on the HVS model, we know that human responds to these areas (edge, detail and smooth) differently. Human eye is more sensitive to noise that appears in smooth areas rather than in edge and detail regions. Therefore, embedding attributes in edge and detail regions provides more perceptual invisibility. HC-RIOT Encoder Base layer Enhancement layer Embedded bits Entropy Coding In the attribute embedding process, the visual mask threshold is used to determine whether the coefficient represents smooth or edge or detail. If the coefficient represents edge or detail region, then one bit of binary attribute is substituted into the coefficient bit. Then the embedded bits are stored in a separate bitstream for the simplicity of detection and retrieval. The base layer and enhancement layer bitstreams may be further coded using the arithmetic coding (providing a little gain especially in enhancement layer). Consequently, after the embedding process, an image is compressed into three bitstreams, base layer, enhancement layer and embedded bitstream. By separating the embedded bitstream from other two bitstreams, the similarity measurement of query attributes and database image of retrieval process is easily determined. V. RESULTS AND DISCUSSION To evaluate the performance of the proposed system, public domain test images of 512 x 512 RGB images with 8 bpp (each component) are used for evaluating the performance of compression and embedding processes. Then, the comparison between the proposed system and the conventional retrieval system is shown. 1) Compression performance To examine the compression performance, the Lena image is compressed by three different algorithms, HC- RIOT without color components (HC-RIOT), SPIHT and JPEG. The results of these three methods are compared in Fig. 5. The performance of HC-RIOT is comparable to SPIHT and both outperform JPEG. Also HC-RIOT provides the multi-layer property that is useful for fast browsing and fast transmission. This property makes HC-RIOT a powerful compression method for the retrieval system. PSNR 45 40 35 30 SPIHT HC-RIOT JPEG 25 information in binary sequence Base layer Figure 3. The steganographic encoder Enhancement layer Embedded bits Entropy Decoding information in binary sequence Figure 4. The steganographic decoder HC-RIOT Decoder 20 0 20 40 60 80 100 120 Compression Ratio Figure 5. The compression performances of SPHIT, HC-RIOT and JPEG (Lena) 2) Embedding process In order to originate the retrieval database, first, a RGB image is converted to HSV format. Then, it is segmented into meaningful objects using both brightness (V) and color information (H and S). Due to the
TENCON 2000 explore2 Page:4/6 11/08/00 ineffectiveness of automatic segmentation algorithms, the semi-automatic process using a graphical interface is exploited to improve the segmentation results [8]. After the image is segmented, the attributes (color, shape etc.) of each object are analyzed. These attributes represent the characteristics of the image needed for retrieval. The attributes are color, shape, texture, and spatial information. Note that in the proposed system, not only using the content-based attributes but also the text-based attribute is exploited. This text-based attribute provides a description of an image such as date, author and specific name. It can support the Query by Text used in current search engines. In our simulation, the text attribute consists of place, date and author of the image. For example, the text may be Place: Forth Worth Zoo Date: 2/25/2000 Photographer: Saowaluck Areepongsa. Even though the proposed steganographic retrieval system is developed to support all kinds of attributes, only significant attributes are used for the system evaluation. These are color, texture, shape, text and spatial information. Note that all attributes are extracted using the efficient methods shown in Fig 6. In the following paragraphs, the extraction of each attribute is discussed. Original image Hybrid Segmentation Texture Color Shape Spatial Text Texture Color Shape Spatial Text Figure 6. The segmentation and feature extraction processes?? Color For the color attribute, the color HSV histogram is used. The histogram is obtained by using the stems of dominant colors, which is known as the color set backprojection method. This algorithm consists of four steps: (1) conversion to HSV color space, (2) quantization of HSV space, (3) color median filtering and (4) color histogram extraction. Details of the color set backprojection are available in [4].?? Texture Due to the translation and rotation invariant properties, texture attributes are extracted from the histogram of the overcomplete steerable pyramid [9]. To perform the extraction process, the image is decomposed into a pyramid structure using steerable filters. Then, the histogram of each subband is computed. Because of the large dimensions of this histogram, the use of this information is ineffective. To reduce the dimensions of the attributes, only two significant parameters (standard deviation and shape information) are used to represent texture characteristics of an image.?? Shape The shape attribute is another crucial information for the retrieval system. It is represented by the object bounding box. This box is the tightest box that fully encompasses the visual object representing the coarse shape description [1]. This shape attribute is independent of the orientation and illumination changes. Moreover, the scaling invariant property can be obtained from the normalized form.?? Spatial Properties The spatial information is the key to render the complex query when the user requests the specific properties in a specific location. Moreover, the spatial information can represent the correlation among objects in an image providing further meaningful information. In this paper, the position of centroid of mass of an individual object (in each image) is used to provide an inter-spatial information.?? Text Even though, text-based search is not efficient when applied to images, it should not be fully replaced by the content-based method. Information like date, author, id etc. have to be described by text. In this proposal, text attribute is another parameter used in implementation. The text attribute includes date, place, specific name and author information. Bit Allocation Requirement The number of bits needed for attribute information is an important issue. To develop the generic system, the capacity of embedding system (without sacrificing the image quality) must be sufficient enough to carry the necessary attribute information. In Table 1, the average bit allocation required for each selected attribute is presented. Note that in general, an image can be segmented into 5-7 meaningful regions. Therefore, the total requirement for all attributes (color, texture, shape and spatial properties) for the entire image (all regions) obtained by the selected attributes is around 5-10 Kbits. The proposed steganographic system is designed to carry the embedded information that needs more than 10 Kbits. This available space is enough for various types of images. Table 1 The number of bits for attributes per region s Number of bits Color 198 Texture 384 Shape 64 Spatial 64 Total per region 710 Details of the bit allocations (Table 3.1) for each attribute are:?? Color attribute: 166 color set (1 bit per color) and 1 energy (32 bits).
TENCON 2000 explore2 Page:5/6 11/08/00?? Texture attribute: 1 standard deviation (16 bits) and 1 shape parameter (16 bits) for 3- level steerable pyramid with 4 orientations.?? Shape attribute: 2 coordinates x and y (64 bits)?? Spatial attribute: 2 coordinates x and y (64 bits).?? Text attribute: additional bits per image, 72 characters (8 bits per character). In the embedding process, the attributes in binary sequence form are embedded in the edge and detail regions of the HC-RIOT enhancement layer. In Fig. 7, the payload of Lena image are shown. Even though the PSNR is reduced when the attributes are embedded but the visual quality of embedded images are comparable to the non-embedded image. Several embedded images (Lena) at different payload are illustrated in Fig. 8. images of (37.06 db) provide good visual quality compared to the image compressed by JPEG (36.86 db) at the same compression ratio (10:1). Note that the thumbnails of the proposed system (using base layer) also provide recognizable images with good visual quality. 38 37 a) non-embedded image b) 1 Kbits 36 35 PSNR 34 33 32 31 Embedded in edge No embedding 30 0 2 4 6 8 10 12 Payload (K bits) c) 3 Kbits d) 4 Kbits Figure 7. The comparison of payload versus PSNR 3) Comparison with JPEG based system Generally, images stored in retrieval databases (compressed by the JPEG standard) consist of three separate types of data, the thumbnail (low-resolution image for fast display), the high resolution one (fine image) and its attributes. To keep all these data together, the proposed algorithm incorporates the steganographic strategy in the compressed domain using multi-layer bitstreams (HC-RIOT). The base layer (coarse image) of HC-RIOT can be used as a thumbnail resulting in reducing the storage space. The full version of image (fine image) can be obtained by combining the enhancement layer with the base layer. According to the steganographic method, the attributes are embedded into the images providing another reduction in data storage. In Fig. 9, the evaluation of bit requirements for the JPEG compressed images (generally used in conventional retrieval systems) and the proposed method is shown. Here, the bit requirements refer to the total number of bits used to store three types of data (thumbnail, fine image and attributes). The size of thumbnail used in this simulation is 32 Kbits (compression ratio = 10:1) with 200x200 image representation. The attributes size is 5Kbits. This number of bits (710 bits per region) reserved for all attributes (color, texture, shape, text and spatial property) is sufficient since the number of meaningful regions for each image are 5-7. In Fig. 10 b, the fine PSNR 40 38 36 34 32 30 28 26 24 22 e) 5 Kbits f) 7 Kbits Figure 8. The embedded images at different payloads HC-RIOT embedding attributes JPEG (with thumbnail and attributes) 0 50 100 150 200 250 300 350 400 Number of bits (Kbits) Figure 9. The bit requirements of the retrieval database (per image) versus PSNR
TENCON 2000 explore2 Page:6/6 11/08/00 a) Fine image and thumbnail (JPEG) b) Fine image and thumbnail Figure 10. The comparison of conventional compression method (JPEG) and the proposed system VI. CONCLUSIONS In this paper, the steganographic retrieval system is proposed. Based on the results, it is not only providing an efficient use of resources but also enhances the performance of the conventional retrieval system such as fast transmission of retrieved image and eliminating the redundancy of the thumbnail and the fine image. For further research, we will emphasize on embedding the information in color components. VII. REFERENCES [1] ISO/IEC JTC1/SC29/WG11, Document N2822, MPEG-7 visual part of experimentation model version 2.0, Vancouver, Canada, July 1999. [2] Y. Rui, T. S. Huang and S. Chang, Image Retrieval: current techniques, promising directions, and open issues, J. Visual Communication and Image Representation, vol. 10, pp. 39-62, March 1999. [3] S. Areepongsa et al., Steganography for low bit rate wavelet based image coder, to be publicized in ICIP 2000 [4] J. R. Smith and S. F. Chang, VisualSEEk: a fully automated content-based image query system, ACM Multimedia, Boston, MA., pp. 87-98, Nov. 1996. [5] W. Ma and B. S. Manjunath, NeTra: A toolbox for navigating large image databases," ACM Multimedia System, vol. 7, pp. 184-198, 1999. [6] L. M. Marvel et al., Spread Spectrum Image Steganography, IEEE Trans. Image Processing, vol. 8, pp. 1075-1083, Aug. 1999. [7] Y. F. Syed and K. R. Rao, Scalable Low Bit Rate Coding Using an HC-RIOT Coder, Proc. Of Asilomar Conf. on Signals, Systems, and Computers, Pacific Grove, CA, pp. 24-27, Oct. 1999. [8] S. Medasani and R. Krishnapuram, A Fuzzy Approach to Content-Based Image Retrieval, Proc. IEEE Int. Fuzzy System, Seoul Korea, vol. 3, pp. 1251-1260, Aug. 1999. [9] P. Blancho, H. Konik and K. Knoblauch, Steerable Pyramid-Based Features for Image Retrieval from a Texture Database, SPIE, Human Vision and Electronic Imaging III, vol. 3299, pp. 552-562, Jan. 1998. [10] S. A. Martucci, et al, A Zerotree Wavelet Video Coder, IEEE Trans. CSVT, vol. 7, pp.109-118, Feb. 1997. [11] A. Said and W. A. Pearlman, A new, fast, and efficient image codec based on set partitioning in hierarchical trees, IEEE Trans. CSVT, vol. 6, pp. 243-250, June 1996. [12] A. Islam, and W. A. Pearlman, An embedded and efficient low-complexity hierarchical image coder, Proc. IS&T/SPIE Conf. Visual Communications and Image Processing, vol. 3653, pp. 294-305, Jan. 1999. Input Image s Generation Phase Segmentation Feature s Embedding in Compressed domain embedded image Query Phase Database in compressed domain Query by Example Query Query Matching Content Retrieved Images Relevance Feedback Figure 1. The steganographic retrieval system