SALIENCY CLASSIFICATION OBJECT AND REGION DETECTION USING HDCT METHOD

Transcription

1 Journal of Recent Research in Engineering and Technology, 3(10), OCT 2016, PP ISSN (Online): , ISSN (Print): Bonfay Publications, 2016 Research Article SALIENCY CLASSIFICATION OBJECT AND REGION DETECTION USING HDCT METHOD M.Lalitha 1, Mrs.S.Subadra 2 1 Research Scholar, Department of Computer Science, Sri Jayendra Saraswathy Maha Vidyalaya College of Arts and Science, Coimbatore. 2 Associate Professor Department of Information Technology, Sri Jayendra Saraswathy Maha Vidyalaya College of Arts and Science, Coimbatore Abstract Received 21 Aug 2016; Accepted 23 Oct 2016 Detecting and segmenting salient objects in natural scenes, also known as salient object detection, has attracted a lot of focused research in computer vision and has resulted in many applications. However, while many such models exist, a deep understanding of achievements and issues is lacking. The Proposed approach is simple, efficient, naturally multi-scale, and produces full-resolution, high-quality saliency maps. These saliency maps are further used to initialize a novel iterative version of GrabCut, namely SaliencyCut, for high quality unsupervised salient object segmentation along with a saliency map of an image by using a linear combination of colors in a high-dimensional color space. This is based on an observation that salient regions often have distinctive colors compared with backgrounds in human perception, however, human perception is complicated and highly nonlinear. By mapping the low-dimensional red, green, and blue color to a feature vector in a high-dimensional color space, we show that we can composite an accurate saliency map by finding the optimal linear combination of color coefficients in the high-dimensional color space. To further improve the performance of our saliency estimation, our second key idea is to utilize relative location and color contrast between super pixels as features and to resolve the saliency estimation from a trimap via a learning-based algorithm. Keywords GrabCut, high-dimensional color space, Salient region detection, trimap, learning-based algorithm. 1. INTRODUCTION Images appearing on websites, mobile devices, as well as TVs and computer screens enrich our daily life. However, processing such large amount of visual information in images in short time is a difficult task. Information in images differs in importance. Some are crucial while others are negligible. An automatic 1

2 M.Lalitha Journal of Recent Research in Engineering and Technology and selective mechanism that answers which information is necessary to pick up from an image for further analysis can be useful. A feasible way is by the selective mechanism of human visual attention according to studies of neurobiology and cognitive psychology human brains are capable of selecting a certain subset of visual information for further processing. Modeling human visual attention on images is referred to as saliency detection, which aims at detecting salient image parts that can easily attract human attention. Although attention processes of human rely on bottom-up influences and top-down influences saliency detection focused in this thesis only considers bottom-up factors (usually the influences from low-level features). This type of saliency detection is stimulusdriven as well as widely studied in the past decade. Saliency detection results indicating potential regions of interest (ROI) provide some guidance to further analysis. This has been used in many applications, e.g. object detection and recognition, image compression, video summarization, content based image editing and image retrieval. In the last decade, saliency detection has become a research field in computer vision attracting much attention. Saliency detection methods can be categorized into either eye fixation modeling or salient region detection. Most early saliency models belong to the former, aiming at predicting where human look in the scene. A. IMAGE PROCESSING Image processing is computer imaging where application involves a human being in the visual loop. In other words the image are to be examined and a acted upon by people. The major topics within the field of image processing include: 1). Image restoration. 2)Image enhancement. 3)Image compression. Image Restoration :Is the process of taking an image with some known, or estimated degradation, and restoring it to its original appearance. Image restoration is often used in the field of events, but the received events are lost during communication called the missing messages. photography or publishing where an image was somehow degraded but needs to be improved before it can be printed. Image Enhancement : It involves taking an image and improving it visually, typically by taking advantages of human Visual Systems responses. One of the simplest enhancement techniques is to simply stretch the contrast of an image. Enhancement methods tend to be problem specific. For example, a method that is used to enhance satellite images may not suitable for enhancing medical images. Image Compression: It involves reducing the typically massive amount of data needed to represent an image. This done by eliminating data that are visually unnecessary and by taking advantage of the redundancy that is inherent in most images. 2

3 Journal of Recent Research in Engineering and Technology M.Lalitha B. SALIENT OBJECT DETECTION IMAGE PROCESSING Salient object detection approaches segment only the salient foreground object from the background, rather than partition an image into regions of coherent properties as in general segmentation algorithms. Salient object detection models also differ from eye fixation prediction models that predict a few fixation points in an image rather than uniformly highlighting the entire salient object region. In practice, salient object detection methods are commonly used as a first step of many graphics/vision applications including object-of interest image segmentation object recognition adaptive compression of images, content-aware image editing, image retrieval, etc. Although extraction of salient objects in a scene is related to accurate image segmentation and object retrieval, interestingly, reliable saliency estimation is often feasible without any actual scene understanding. Saliency, as widely believed, is a bottom-up process that originates from visual distinctness, rarity, or surprise and is often attributed to variations in image attributes such as color, gradient, edges, and boundaries Visual saliency, being closely related to our perception and processing of visual stimuli, is investigated across many disciplines including cognitive psychology neurobiology and computer vision. Based on our observed reaction times and estimated signal transmission times along biological pathways, human attention theories hypothesize that the human vision system processes only parts of an image in detail, while leaving the rest nearly unprocessed 2. RELATED WORK The contributions of various scholars are studied for analyzing the merits and demerits in order to enhance the consequences for making the system work better. BLOCK-BASED MODELS WITH INTRINSIC CUES: In this subsection, we mainly review salient object detection models which utilize intrinsic cues extracted from blocks. Following the seminal work of Itti et al. salient object detection is widely defined as capturing the uniqueness, distinctiveness, or rarity of a scene. uniqueness is widely studied as pixel-wise center-surround contrast. Hu et al. represent the input image in a 2D space using polar transformation of its features so that each region in the images can be mapped into a 1D linear subspace. After that, the Generalized Principal Component Analysis (GPCA) is used to estimate the linear subspaces without actually segmenting the image. Finally, the attentive (salient) regions are selected by measuring feature contrasts as well as geometric properties of regions. Rosin proposes an efficient approach for detecting salient objects. The whole approach is parameter-free and requires only very simple pixel-wise operations such as edge detection, threshold 3

4 M.Lalitha Journal of Recent Research in Engineering and Technology decomposition and moment preserving binarization. Valenti et al propose an isophote based framework where the saliency map is estimated by linearly combining the saliency maps computed in terms of curvedness, color boosting, and isocenters clustering. Achanta et al. adopt a frequency-tuned approach to compute full resolution saliency maps by simply measuring the pixel-wise color difference between the smoothed image pixels and average color of the image. The saliency of pixel x is computed as: s(x) = kiµ Iωhc (x)k 2 where Iµ is the mean pixel value of the image (e.g., RGB/Lab features) and Iωhc is a Gaussian blurred version of the input image (e.g., using a 5 5 kernel). Without any prior knowledge of the sizes of salient objects, multi-scale contrast is frequently adopted for robustness purpose. To that end, a L-layer Gaussian pyramid is first constructed Let I (l) be the image at the lth-level of the pyramid, the saliency score of pixel x is defined as: s(x) = XL l=1 X x0 N(x) I (l) (x) I (l) (x 0 ) 2 Later, such center-surround contrasts are computed in an information-theoretic way by using the Kullback-Leibler Divergence on difference features such as intensity, color and orientation. Li et al. study the center-surround contrast as a cost sensitive max-margin classification problem. In particular, the center patch is thought of as a positive sample while the surrounding patches are all used as negative samples. The saliency of the center patch is determined by its separability from surroundings based on the trained cost-sensitive Support Vector Machine (SVM). Patch uniqueness is also defined as its global contrast with others. Intuitively, a patch is considered to be salient if it is remarkably distinct from its most similar patches, while their spatial distances are taken into account. Margolin et al. propose to define the uniqueness of a patch by measuring its distance to the average patch based on the observation that distinct patches are more scattered than non-distinct ones in the high-dimensional space. To further incorporate the patch distributions on each single image, the uniqueness of a patch is measured by projecting its path to the average patch onto the principal components of the image. To this end, the saliency score of a patch px centred at pixel x is defined as s(px) = kp xk1, REGION-BASED MODELS WITH INTRINSIC CUES Saliency models in the second subgroup adopt intrinsic cues extracted from image regions to estimate their saliency scores. Different from the block-based models, region-based models often segment an input image into regions aligned with 4

5 Journal of Recent Research in Engineering and Technology M.Lalitha intensity edges first and then compute a regional saliency map. As an early attempt,, the regional saliency score is defined as the average saliency scores of its contained pixels, defined in terms of multi-scale contrast. Yu et al. [85] propose a set of rules to determine the background scores of each region based on observations from background and salient regions. Saliency, defined as uniqueness in terms of global regional contrast, is widely studied in existing approaches. In, a region-based saliency algorithm is introduced by measuring the global contrast between the target region with respect to all other regions in the image. Margolin et al. propose to combine the regional uniqueness and patch distinctiveness to form the saliency map. Instead of maintaining a hard region index for each pixel, a soft abstraction is proposed in to generate a set of large scale perceptually homogeneous regions using histogram quantization and a global Gaussian Mixture Model (GMM). By avoiding the hard decision boundaries of superpixels, such soft abstraction provides large spatial support and can more uniformly highlight the salient object Jiang et al. propose a multi-scale local region contrast based approach, which calculates saliency values across multiple segmentations for robustness purpose and combines these regional saliency values to get a pixel-wise saliency map. Specifically, the input image is segmented with the algorithm using Ns different groups of parameters Shen and Wu propose a unified approach to incorporate traditional low-level features with higher-level guidance, e.g., center prior, face prior, and color prior, to detect salient objects based on a learned feature transformation4. Instead, Zou et al. propose to exploit bottom up segmentation as a guidance cue of the low-rank matrix recovery for robustness purpose. high-level priors are also adopted where a tree-structured sparsity-inducing norm regularization is introduced to hierarchically describe the image structure with the aim to more uniformly highlight the entire salient object. Jiang et al. propose to formulate the saliency detection via absorbing Markov Chain where the transient and absorbing nodes are super pixels around the image center and border, respectively. The saliency of each super pixel is computed as the absorbed time for the transient node to the absorbing nodes of the Markov Chain. Beyond these approaches, the generic object ness prior5 is also exploited to facilitate the detection of salient objects by leveraging Chang et al. present a computational framework by fusing the objectness and regional saliency into a graphical model. These two terms are jointly estimated by iteratively minimizing the energy function 5

6 M.Lalitha Journal of Recent Research in Engineering and Technology that encodes their mutual interactions. In regional objectness is defined as the average objectness values of its contained pixels, which will be incorporated for regional saliency computation 3. PROPOSED METHODOLOGIES Based on the investigations from the literature survey of various scholars the solution to the problem is achieved which are classified and presented below. The goal is to find a description of each object class that has tolerance to intra-class variations and to a certain degree of illumination or viewpoint changes. Each class is described at two levels. First, at a very general level, with a set of low-level features associated with very simple classifiers, each based on one single feature. Second, at a more specific level, using more complex features and classifiers (specific descriptors). The first description is used to simplify the search, eliminating many of the candidate nodes, while the second, more costly, is applied only to the remaining nodes. Generic descriptors: They are associated with lowlevel visual attributes which are common to any object. They are simple and relatively easy to measure, and they are pre-computed for all the nodes when creating the tree. To associate each descriptor with a simple threshold classifier which is trained on sample data. In order to cover the four scenarios proposed in this work with the following descriptors. Position: the mass center of the region. Size: the number of pixels that form the region. Orientation: computed using the region central moments. Color mean: the mean value of the pixels within the region in each color component in the YCbCr color space. Aspect Ratio and Oriented Aspect Ratio: of the bounding box and the oriented bounding box of the region, respectively. Compactness: the quotient between the region size and the size of its bounding box. Circularity: the quotient between squared perimeter and region size. Homogeneity: measured as the power of the coefficients within the region in the Haar wavelet transform (LH, HL and HH sub bands, 2 levels). 4. SALIENCYCUT: AUTOMATIC SALIENT REGION EXTRACTION In a highly influential work, GrabCut made critical changes to the graphcut formulation to allow processing of noisy initialization. This enabled users to roughly annotate (e.g., using a rectangle) a region of interest, and then use GrabCut to extract a precise image mask. Using 6

7 Journal of Recent Research in Engineering and Technology M.Lalitha our estimated saliency masks, we remove even the need for user annotated rectangular regions. Saliency Cut, which uses the computed saliency map to assist in automatic salient object segmentation. This immediately enables automatic analysis of large internet image repositories. Specifically, we make two enhancements to GrabCut : iterative refine and adaptive fitting, which together handle considerably more noisy initializations. To automatically initialize the segmentation according to the detected saliency map Algorithm initialization Instead of manually selecting a rectangular region to initialize the process, as in classical GrabCut, we automatically initialize using a segmentation obtained by binarizing the saliency map using a fixed threshold Tb. Similar to GrabCut, we use incomplete trimap for the initialization. For image pixels with saliency value bigger than Tb, the largest connected region is considered as initial candidate region of the most dominate salient object. This candidate region is labeled as unknown part of the trimap, while other regions are labeled as background. Note that we do not initialize any hard foreground labeling. These unknown regions are initially used to train foreground color models thus helps the algorithm to identify the foreground pixels. Since the initial background regions are retained while other regions may be changed during the GrabCut optimization, preference to confident background labels in the trimaps. 5. HIGH-DIMENSIONAL COLOR TRANSFORM FOR SALIENCY DETECTION: Our high-dimensional color transform and describe a step-by-step process to obtain our final saliency map starting from the initial saliency map from the previous section. Trimap Construction The initial saliency map usually does not detect salient objects accurately and may contain many ambiguous regions. This trimap construction step is to identify very salient pixels from the initial saliency map that definitely belong to salient regions and backgrounds, and use our high-dimensional color transform method to resort the ambiguities in the unknown regions. In order to catch the salient pixels more accurately, instead of using a single global threshold to obtain our trimap from the initial saliency map, we propose using a multi-scale analysis with adaptive thresholding. the initial map into 2 2, 3 3, and 4 4 regions and apply thresholding for each region individually. The multilevel adaptive thresholding to control the rate between the foreground, background and unknown regions in each sub region. In this research work use a seven-level threshold in each sub region. After merging the three different scale threshold saliency maps by summation to obtain a locally threshold 21-level map T 0. This new saliency map has better local contrast than the initial saliency map. Therefore, it is able to locally capture very salient regions even 7

8 M.Lalitha Journal of Recent Research in Engineering and Technology though the local region might not be the most salient globally within the whole image HIGH-DIMENSIONAL TRANSFORM COLORS: COLOR The RGB color space does not fully correspond to the space where the human brain processes colors. It is also inconvenient to process colors in the RGB space since illumination and colors are nested here. For these reasons, many different color spaces have been introduced such as YUV, YIQ, CIELab, HSV, etc. Nevertheless, it is still unknown which color space is the best to process images, especially for applications like saliency detection that are tightly correlated to our human perception. Instead of picking a particular color space for processing, we introduce a high-dimensional color transform which unifies the strength of many different color representations. Our goal is to find a linear combination of color coefficients in the high dimensional color transform space such that colors of salient regions and colors of backgrounds can be distinctively separated. To build our highdimensional color transform space, to concatenate different nonlinear RGB transformed color space representations concatenate only the nonlinear RGB transformed color space because the effects of the coefficients of linear transformed color space such as YUV/YIQ, will be cancelled to linearly combine the color coefficient to form our saliency map. The color spaces are concatenated include the CIELab color space, and the hue and saturation channel in the HSV color space. The color gradients in the RGB space since our human perception is more sensitive to relative color differences instead of absolute color values. 6. PROPOSED METHOD In the proposed HDCT based graphcut detection Step 1: Read the input image. Step 2: Convert input image into gray scale image if it is color image. Step 3: Select the control parameter K1 and K2. Step 4: Calculate fmin and fmax. These are weight calculated as follows: fmin= min(min(f)) and fmax= max(max(f)). Step 5: Determine f=f-fmin and also Calculate f=f/fmax. Step 6: Calculate output of Graphcut modified MMR function. Step 7: Output image of modified MMR function is further passes through contrast limited adaptive histogram equalization. Step 8: Repeat steps 6 to 7 for entire image. 8

9 Journal of Recent Research in Engineering and Technology M.Lalitha K1 is control parameter which controls the actual contrast of input image. If the value of K1 is selected 5 then its effect on the input image is little change in the contrast, if the value of K1 is selected 1 then its reduces contrast to about 20% of original and if the value of K1 is selected 10 then its increase contrast about to 2.5 times the input image K2 is another control parameter which represents the normalized gray value about which contrast is increased or decreased. The initial value of K2 is selected 0.5 but different images may require different points of the gray scale to be enhanced. PERFORMANCE ANALYSIS We evaluate and compare the performances of our algorithm against previous algorithms on two representative benchmark datasets: the MSRA-B dataset 50 image,msra-b dataset 100 image. To evaluate the performance of the proposed system quantitatively proposed method compared the detected segments with the ground truth.the metrics used for performance evaluation are three: Recall, Precision and Similarity that are defined by Recall = TP / TP+FN Precision = TP / TP+FP number of correct/incorrect and foreground/background edge pixel detection with respect to the ground truth. These metrics are commonly used in the region based methods where they indicate the percentage of region matched, the same is true for pixel and edge based methods. Recall gives the percentage of true positives detected. Precision gives the percentage of detected items that are true positives. MSRA-B DATASET: The MSRA-B salient object dataset contains, images with the pixel-wise ground truth used by the authors provided by Jiang et al. This dataset mostly contains comparatively obvious salient objects in which the colors are definitely different from the background, and therefore, it is considered a less challenging dataset for salient object detection. We use the same training set including 3,500 images and the test set including 2,000 images used in as the training and test data, respectively. DATASET MERITIC RESU MSRA-B Dataset 50 IMAGE LT PRECISION 0.91 RECALL.076 F-MEASURE 0.85 Similarity= TP / TP+FN+FP where, TP is the total of true positives, FN is the total of false negatives, FP is the total of false positives and FP is the total of true negatives. These metrics indicate the total MSRA-B Dataset 100 IMAGE PRECISION 0.85 RECALL.076 F-MEASURE

10 M.Lalitha Journal of Recent Research in Engineering and Technology CONCLUSION We have presented a salient object detection for HDCT images based on evolution strategy. It fully explores the potential of color cue and depth cue in the whole procedure of salient object detection, including super-pixel generation, initial saliency map generation and saliency propagation. And the two-step saliency evolution strategy ensures the high precision and completeness of the detected salient objects. Where the classic salient object/region detection and segmentation, fixation prediction, and category-independent objectness measurement or object proposals generation. Detecting and segmenting salient objects is very useful for scene understanding. Objects in an image will automatically catch more attention The experimental results show that the proposed method outperforms the state-ofthe-art methods for both HDCT images object detectors. Quantum-Cuts (QCUT) is a recent saliency map generation technique. It automatically detects the most interesting (e.g. the object) region in an image and can highlight where people tend to look at first or at the most. It is inspired from and based on Quantum Mechanics. QCUT provides a parameterfree -hence dataset independent-, unsupervised and fully automatic saliency map generation, which outperforms current state-of-the-art algorithms. Its salient object segmentation results exhibit such a promising accuracy that pushes the frontier in this field to the borders of the input-driven processing only without the 10

11 Journal of Recent Research in Engineering and Technology M.Lalitha use of object knowledge aided by longterm human memory and intelligence. Furthermore, with the near-future technologies for measuring a quantum wave function, QCUT may have a unique potential: Automatic object segmentation in an actual physical setup in nano-scale. Such an unprecedendent property would not only produce segmentation results instantaneously, but may be a unique opportunity to achieve accurate object segmentation in real-time for the massive visual repositories of today s Big Data. REFERENCES [1] C. Rother, V. Kolmogorov, and A. Blake. Grabcut Interactiveforeground extraction using iterated graph cuts. ACM TOG, 23(3): , [2] T. Liu, Z. Yuan, J. Sun, J. Wang, N. Zheng, X. Tang, and H. Shum. Learning to detect a salient object. In CVPR, pages 1 8, [3] J. Harel, C. Koch, and P. Perona, Graphbased visual saliency, in Proc Conf. Adv. Neural Inf. Process. Syst. (NIPS), 2006, pp [4] R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, Frequency-tuned salient region detection, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2009, pp [5] S. Goferman, L. Zelnik-Manor, and A. Tal, Context-aware saliency detection, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2010, pp [6] Y. Zhai and M. Shah, Visual attention detection in video sequences using spatiotemporal cues, in Proc. ACM Multimedia, 2006, pp [7] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk, SLIC superpixels compared to state-of-the-art superpixel methods, IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 11, pp , Nov [8] J. Kim, D. Han, Y.-W. Tai, and J. Kim, Salient region detection via highdimensional color transform, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2014, pp [9] Z. Wang and B. Li. A two-stage approach to saliency detection in images. IEEE Conference on Acoustics, Speech and Signal Processing, [10] A. Borji, M.-M. Cheng, H. Jiang, and J. Li. (2014). Salient object detection: A survey. [Online]. Available: [11] A. Borji, M.-M. Cheng, H. Jiang, and J. Li. (2015). Salient object detection: A benchmark. [Online]. Available: [12] L. Itti, J. Braun, D. K. Lee, and C. Koch, Attentional modulation of human pattern discrimination psychophysics reproduced by a quantitative model, in Proc. Conf. Adv. Neural Inf. Process. Syst. (NIPS), 1998, pp [13] D. A. Klein and S. Frintrop, Centersurround divergence of feature statistics for salient object detection, in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Nov. 2011, pp

12 M.Lalitha Journal of Recent Research in Engineering and Technology [14] V. Navalpakkam and L. Itti, An integrated model of top-down and bottom-up attention for optimizing detection speed, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2006, pp [15] P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, Object detection with discriminatively trained part-based models, IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp , Sep [16] B. Su, S. Lu, and C. L. Tan, Blurred image region detection and classification, in Proc. ACM Int. Conf. Multimedia, 2011, pp [17] L. Marchesotti, C. Cifarelli, and G. Csurka, A framework for visual saliency detection with applications to image thumbnailing, in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Sep./Oct. 2009, pp [18] L. Itti, Automatic foveation for video compression using a neurobiological model of visual attention, IEEE Trans. Image Process., vol. 13, no. 10, pp , Oct [19] W. Hou, X. Gao, D. Tao, and X. Li, Visual saliency detection using information divergence, Pattern Recognit., vol. 46, no. 10, pp , Oct [20] H. Jiang, J. Wang, Z. Yuan, T. Liu, and N. Zheng, Automatic salient object segmentation based on context and shape prior, in Proc. Brit. Mach. Vis. Conf. (BMVC), 2011, pp [21] A. Y. Ng, M. I. Jordan, and Y. Weiss, On spectral clustering: Analysis and an algorithm, in Proc. Conf. Adv. Neural Inf. Process. Syst. (NIPS), 2002, pp [22] Q. Yan, L. Xu, J. Shi, and J. Jia, Hierarchical saliency detection, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2013, pp [23] L. Wang, J. Xue, N. Zheng, and G. Hua, Automatic salient object extraction with contextual cue, in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Nov. 2011, pp [24] A. Borji and L. Itti, Exploiting local and global patch rarities for saliency detection, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2012, pp [25] P. Siva, C. Russell, T. Xiang, and L. Agapito, Looking beyond the image: Unsupervised learning for object saliency and detection, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2013, pp