Multi-Scale Kernel Operators for Reflection and Rotation Symmetry: Further Achievements

Transcription

1 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops Multi-Scale Kernel Operators for Reflection and Rotation Symmetry: Further Achievements Shripad Kondra Mando Softtech India Gurgaon (Delhi), India Alfredo Petrosino, Sara Iodice University of Naples Parthenope, Department of Applied Science Centro Direzionale, Isola C/4, Naples, Italy Abstract Symmetry is a crucial dimension which aids the visual system, human as well as artificial, to organize its environment and to recognize forms and objects. In humans, detection of symmetry, especially bilateral and rotational, is considered to be a primary factor for discovering and interacting with the surrounding environment. We report an enhanced version of the Kondra and Petrosino symmetry detection algorithm, already reported at the Symmetry Detection from Real World Images competition at IEEE CVPR2011[1]. The paper includes experimental results achieved by the reflection and rotation symmetry detection algorithm on the datasets made available for the 2013 Symmetry Detection from Real World Images competition. 1. Introduction From a Gestalt point of view, the law of symmetry stresses that we tend to perceive objects as symmetrical structures around a center, and it assigns a relevant role in the attentive mechanism, both in visual and auditory systems. In particular, by facilitating perceptual grouping, as well as figure/ground organization, symmetry is one of the most important factors allowing perceptual structures to emerge. Indeed, when we perceive disconnected but alike elements, that are symmetrical to each other, we tend to integrate them in a coherent percept. Moreover, in figure/ground segregation process, symmetrical images generally emerge as figure, rather than as ground. Symmetry detection is also highly relevant in shape recognition. Indeed, the description of a shape may be different when it is embedded in a context with horizontal or vertical symmetry [2]. Besides, in tasks requiring the completion of partially occluded visual stimuli, subjects systematically tend to produce symmetrical figures [3]. The concept of symmetry is not univocal: various kinds of properties of an image are defined as symmetry [4, 5]. There are two different approaches in literature to detect symmetries: global region-based methods [6] and local feature-based algorithms [7]. Global region-based methods can characterize all potential symmetries in the image, whilst local feature-based algorithms have the advantage of being more efficient, but less robust. The algorithm we are presenting is a global region-based method because it uses the information deriving from all the pixels in the image. In Section 2.1 we describe the rationale behind our symmetry detection algorithm, as reported in [1], while in Sections 3 and 4 we provide a hands-on analysis of experimental results about the reflection and rotational symmetries over the Symmetry Detection from Real World Images competition datasets respectively. Conclusions are drawn in Section Multi-scale Symmetry Detection The property of being symmetrical finds correspondence in size, shape, and relative position of parts on opposite sides of a dividing line or median plane or about a center or axis. In particular, we deal with bilateral symmetry. We specifically take advantage of a measure obtained by using correlation with the image flipped around a particular axis. As demonstrated in [8], in any direction, the optimal symmetry axis corresponds to the maximal correlation of a pattern with its symmetric version. The symmetry detection algorithm operates in two steps. Firstly, correlation measures along all discrete directions are computed and then symmetrical regions are identified looking for matches in the directions characterized by maximun correlations. The algorithm can detect both reflection and rotation symmetries. We extended the Kondra and Petrosino algorithm reported at the First Symmetry Detection Competition (CVPR 2011) to cope with symmetries at different scale values Estimation of Correlation Measurements This step is common to both reflection and rotation algorithms. It requires a whitened (removing its mean) image I as input, that can either be in grayscale or RGB /13 $ IEEE DOI /CVPRW

2 Measures of correlation are calculated extracting patches from the image, centered at locations sampled at regularly spaced SIFT locations. Let P be the set of n p extracted patches. We set the grid step s =10, while the patch side S assumes different values in the range [a, b]. At each scale in the range [a, b], plane rotations of the image are computed with directions in the set D = {d i : i =1,...,n d }. A correlation measure c j i for each patch pj i, corresponding to a single direction d i, is computed between the original patch p j i and its reflected version rj i, as follows: c j i = ( ( ) p j i (x, y) rj i (x, y) / ) p j i (x, y)2 (1) x,y x,y For color images, the patch is reflected with respect to its three bands before computing the correlation. Only the direction corresponding to the maximum correlation value is kept out for each patch and only the patches with correlation measures greater than a threshold T are considered to be symmetric. For an optimal threshold value T, we experimentally derived an approximate formula by providing a scale value σ: T =0.7 (0.005 σ) (2) For instance, T is in the set {0.6, 0.5} for scale values σ in {20, 40}, respectively. Finally, we obtain a set of symmetric } patches P S = {p j i : i =1,...,n d,j =1,...,n ps characterized by the parameters: ( ) position x j i,yj i of its center, maximum correlation value c j i, direction corresponding to the maximum value of correlation d i, and scale σ j at which the maximum correlation is catched. Fig. 1 shows the flow chart representing the process described in this section, repeated for each value of σ in the scale range R =[a, b] Reflection Symmetry Detection Reflection Symmetries (RfS) are detected for patches with maximum correlation in the same direction d. For each discrete direction in D = {d i : i =1,...,n d }, we selected the number of patches n di such that n di n ps and each patch gets the maximum correlation in the direction d i. Then, for each direction, we built a binary matrix B having the same size of image I. If there is a patch p j P S centered at (x, y) then B (x, y) =1. The connected regions CR = {CR i : i =1,...,n CR }, extracted from the binary matrix B, represent the reflection symmetric areas of the image at a specific direction. Finally, for each reflection symmetric area, we compute: Figure 1. Flow Chart of Estimation of Correlation Measurements centroid, cirection, and endpoints of symmetry axis Rotation Symmetry Detection In the case of Rotation Symmetries (RtS), the binary matrix B is not built for each discrete direction. Rather, we derive only one binary matrix B for all the discrete directions. So, if there is a patch p j P S centered in (x, y), independently from their associated direction then B (x, y) = 1. Applying to B the procedure explained in Section 2.2, rotation symmetric areas are detected. For each rotational symmetric area, we extract some parameters that characterize rotational symmetry: center, major axis, minor axis, and orientation. Specifically, the center of rotational symmetry is the centroid of the rotational symmetic area Symmetry in Textures The algorithm turns out to be efficient even in detecting symmetries in textures [9]. In order to use symmetry to represent texture, it is necessary to validate that the symmetry distribution for an image is rotation invariant. To show this property, the symmetry distribution of some images is compared with their rotated versions, i.e., the symmetry is obtained as the maximum over eight different patch rotations. Figure 2 shows the distribution of symmetry and the corresponding angle at which maximum symmetry was de- 218

3 Figure 5. Symmetry Detection Execution Times Figure 2. Single scale symmetry distributions for some textures tected. The angle distribution is further sorted to make it rotation invariant. First the image was normalized in the range [0, 1] and whitened. Each image pixel can be characterized with various parameters extracted from different patch sizes (e.g., 12 12, 24 24, 36 36): 1. multiscale symmetry for the patch at three scales; 2. maximum direction for the symmetry along each of the three scales; 3. mean intensity of the patch for each scale; 4. entropy of the patch for each scale. Figure 6. Comparison of Precision for different values of h on single training dataset. The above features are shown to be effective in classifying a uniform texture versus a non-uniform texture [9], mainly if used in conjunction with a texton-based feature [10]. 3. Experimental Results - Reflection Symmetry The algorithm requires as input a range of the scale values R and a rank k. It returns n axes of symmetry. We consider, as positive predictions, the k axis with maximum correlation value, while as negative predictions, the n k axes with minimum correlation values. In this section, we experimentally show that the precision and true positive results are related to the value of k and scale range R. Firstly, we tested our algorithm on the training datasets, in order to detect the best scale range R and value of rank k. Lastly, these parameters were used as input to detect the symmetry axes on testing datasets. Fig. 5 shows execution times of our algorithm in different cases achieved on 2 GHz Intel Core i7. The times are Figure 7. Comparison of Precision for different values of h on mutiple training datasets. True Positive 27 False Positive 35 Table 1. Results on the Single Training Dataset for h = 4 and k =3. high because we did not work to reduce the number of clock cycles. 219

4 Figure 3. Results on Single Training Dataset. Figure 4. Results on Multiple Training Dataset. True Positive 25 False Positive 22 True Negative 14 False Negative 2 Table 2. Results on Multiple Training Datasets for h = 6 and k = Single Training Dataset After choosing R as the range [a, b], the algorithm considers the scale values in the subranges R h [a : h : b] :h [c, d]. We can select h>1 to reduce the execution time and improve the accuracy of the results. The subrange R h includes the values in range R with offset h. For instance, if a =10, b =20, c =1, d =10and h =2, the subrange R 2 includes the set of values {10, 12,...,18, 20}. We achieved greater precision increasing h for all values of rank k, as depicted in the Precision Curve shown in Figure 12. Finally, we have chosen those values of rank k for which the true positives are maximal and the precision is greater than a certain threshold s1. We achieved the results in Table 1 with k =3and h =4. The algorithm detected 27 over 35 groundtruth reflection axes on this dataset. Similar results are shown in Figure Multiple Training Dataset When there are more symmetries in each image, the problem becomes more difficult. In this case, it is impor- Figure 8. Comparisons of True-Positive with different values of h on the Single Training Dataset. tant to establish the value of rank k and the subrange R h, in order to decrease the algorithm execution time and to identify a larger number of groundtruth reflection axes. We choose them following the procedure explained in the previous section. Looking at the graphs in Figures 13 and 9, it turns out that the best value of h is 6 and that of k is 4. We achieved the results shown in Table 2 for the Multiple Training dataset with k =4and h =6. The algorithm detected 25 over 39 groundtruth reflection axis. Table 2 shows the results with k =3and h =4. The results are shown in figure

5 Figure 10. Results on the Single Training Dataset. Figure 11. Results on Multiple Training Datasets. 1. Small-scale symmetries are characterized by high correlation values, while large-scale symmetries are characterized by low values. Figure 9. Comparisons of True-Positive with different values of h on the Multiple Training Datasets. 4. Experimental Results - Rotational Symmetry The algorithm requires in input a range R of scale values and detects n rotational symmetries. It was found that: 2. True-Negative results, corresponding to uniform background areas, get higher correlation values. If we consider as true positives the k rotational symmetries having maximum correlation values, the largescale symmetries would be discarded. So, we divided the interval R into m equal parts R i, i =1,...,m, and we selected, for each subinterval R i, the k i rotational symmetries having maximum correlation value. True-Negative results, corresponding to uniform areas of background, can be discarded through a preliminary training phase. The aim is to search in each subinterval R i a threshold for correlation t i. The rotational symmetries detected on subinterval k i having a correlation value exceeding the threshold t i are discarded. We have chosen for each subinterval k i the value of t i that maximizes the number of rotational symmetries that we considered correct. From examining the experiments done on the datasets made available for the competition at the IEEE CVPR2013, we found that the algorithm not only works for exact circle but can also work for finding skewed symmetric objects in real word images. We reported the results on single and multiple training dataset in Tables 4 and 3, respectively. Specifically, in the case of multiple centers, we 221

6 True Positive 8 False Positive 30 Table 3. Rotation Symmetries Results on Single Training Dataset. True Positive 18 False Positive 25 Table 4. Rotation Symmetries Results on Multiple Training Datasets. 5. Conclusions We have reported the validation of our multiscale algorithm to detect reflection and rotation symmetry axes on the datasets made available for the Symmetry Detection from Real World Images competition at the IEEE CVPR2013. As shown, the algorithm is able to efficiently detect reflection symmetries in natural and artificial images; as experimentally tested, the algorithm accuracy is related to the value k of the rank and the range R of scale values. Mainly, we showed the accuracy of the improved results, appropriately selecting the range R and rank k through a preliminary training phase. References Figure 12. Comparison of Precision for different values of h on Single Training Dataset. Figure 13. Comparison of Precision for different values of h on Mutiple Training Datasets. [1] S. Kashyap J. Liu I. Rauschert, K. Brocklehurst and Y. Liu, First symmetry detection competition: Summary and results, CSE Dept Technical Report No. CSE [2] S.R. Palmer, The role of symmetry in shape perception, Acta Psychologica, vol. 59, pp , [3] R. van Lier and J. S. Wagemans, From images to objects: Global and local completion of self-occluded parts, Journal of Experimental Psychology: Human Perception and Performance, vol. 25, pp , [4] H. Zabrodsky, Simmetry - a review, Tech. Rep , CS Dept., The Hebrew University of Jerusalem, vol. 25, [5] P. Wenderoth, The salience of vertical symmetry, Perception, vol. 23, pp , [6] R. T. Collins S. Lee and Y. Liu, Rotation symmetry group detection via frequency analysis of frieze-expansions, Computer Vision and Pattern Recognition, pp. 1 8, [7] L. Cornelius, Detecting rotational symmetry under affine projection, International Conference on Pattern Recognition., pp , [8] V. D. Gesú and B. Zavidovique, The s-kernel: A measure of symmetry of objects, Pattern Recognition, vol. 40, no. 3, pp , [9] S. Kondra and A. Petrosino, Self-similarity and points of interest in textured images, in PerMIn2012, 2012, pp [10] S. Kondra and V. Torre, Texture classification using three circular filters, in Computer Vision, Graphics & Image Processing, ICVGIP 08. Sixth Indian Conference on, 2008, pp obtained a considerable number of groundtruth rotational symmetries 18 over 43, whilst in the case of single-center, the number of true-positive results decreases 8 over 38. Figures 12 and 13 show respectively the comparisons in terms of Precision for different values of h on single and multiple training datasets, indicating the robustness against scale values. 222