CIRCULAR MOIRÉ PATTERNS IN 3D COMPUTER VISION APPLICATIONS Setiawan Hadi Mathematics Department, Universitas Padjadjaran e-mail : shadi@unpad.ac.id Abstract Geometric patterns generated by superimposing two identical random dot images are an interesting phenomenon that has drawn attention of researchers for many [2, 3, 4, 5]. One of those patterns that shows the circular rings when two random dot images are overlaid one on top of another is investigated in this research including its applications in three dimensional computer vision. Two practical applications of the Moiré rings have been explored. These are depthperception application and similarity measurement application. Experiments using artificially generated random dot images and real images taken using an image capturing device such as a camera or a scanner and converted to pseudo-random dot representation has been conducted successfully. Keyword : Moiré, Vision, Depth, 1. Moiré Patterns From Random-Dots The effect of Moiré generated using random dots has been reported widely in the literature [2, 3, 4]. The effect can be demonstrated by superimposing two identical random dot images where one of them is turned by a small angle relative to the original. Figure 1 below shows circular random dot image (c) that are created by overlaying two identical random dot images (a and b) which one of them is rotated in small angle. (a) (b) (c) Figure 1. Formation of circular random dot image One of the requirements for successful application of the Moiré based method is the conversion of gray level images to images formed by (random) dots only. High pass filtering followed by thresholding is a simple way to transform a gray level image to a set of dots, but the distribution of these points normally reflects the layout of image features (edges) and is not uniform. In particular this transformation may generate very few points in some regions in the image. Two transformations of an image to pseudo random dot representation are described in this paper. The first one is based on high pass filtering and using thresholding. The second technique is based on bit slicing. In each method the effect of transformation parameters on the density and distribution of points in the dot image is investigated. In the Figure 2, sample image conversion from bitmap image to random dot image is presented. A-17
A-18 Prooceedings, Komputer dan Sistem Intelijen (KOMMIT 2002) Auditorium Universitas Gunadarma, Jakarta, 21-22 Agustus 2002 Grey Level Image Enhanced Gray Level First Step Conversion After Binarization Figure 2. Gray level image conversion using a simple filter; note non-uniform dot distribution. Steps that are involved in random dot image creation can be explained as follows: 1. Grey Level Image Capturing Picture from the real world is captured using capturing device such as scanner and digital camera. 2. Digital Image Enhancement In this step, image that is captured should be enhanced to obtain clear and sharp image. This process can be performed by averaging pixel distribution and equalization. 3. Conversion Process This step can be performed using one of these methods: (i) High pass filtering and (ii) Bit slicing 4. Binarization Process Transformation of the result above into two-value image which consist only 0 and 1. 2. Disparity Measurement Application Application of Moiré patterns in measuring disparity is given in this section. Disparity is calculated using this equation fd ( f Z) = (1) x x and based upon the location of the fixed point. The fixed point is the centre of circular rings which are generated by superimposing two random dot images. In this case, those two random dot images are considered as stereo images (left and right images). Testing of this application was conducted using simulated and actual images. Simulated image was in the form of random dot image while the actual image is the image acquired using a camera and then converted into pseudo-random dot image by applying the filtering method. For the simulated image, the multiplicative congruential method (MCM) based random dot generation, is used for generating random dot image. The second image (the right image) is then created by introducing shift to the left image. 2.1 Experiment using artificial image Two sets of experiment using simulated images were conducted. Firstly, a whole image which is treated as a single region, and 5 different shifts (disparities) were used. One of the images was shifted and rotated and then superimposed on the other. The centre of the resulting circular pattern was then detected (using the Hough transform). The sample images from this experiment are given in Figure 3 while the result is in Table 1. 1 2
Circular Moiré Patterns In 3D Computer Vision Applications A-19 Figure 3. Superimposed and rotated images with various disparities: 1, 3 and 5 pixels Table 1. Result of experiment using simulated image single region Shift Theoritical centre Detected centre Detected X Y X Y disparity 1 0.50 11.45 0.54 11.37 0.99 1 2 1.00 22.90 1.27 22.07 1.93 2 3 1.50 34.36 1.92 33.20 2.90 3 4 2.00 45.81 1.58 45.86 4.00 4 5 2.50 57.26 2.29 55.13 4.81 5 The second set of experiments using simulated random dot images was conducted by dividing each image into two horizontal regions. The y=64 border was selected and shifts were applied to the both regions. Each of the regions was processed individually. Figure 4 shows the sample images of applying different disparity to the regions. In this case, the disparities 1, 3 and 5 pixels are only applied to the region 1 (top portion). The results of detecting the circular rings are given in Table 2. Figure 4. Sample images with various disparity for region 1 The detected disparity was computed by reversing the second equation and substituting the value y to the value of detected centre. For example: if the y coordinate of detected centre is 54.14, the disparity or dx=4.73 x 2 x tan 5/2 which is equal 4.73, rounded to 5. From the results above we can see that the application of disparity measurement can be performed successfully using the concept of Moiré pattern. The average errors in X and Y direction based on the experimental result are 0.27 and 0.85 pixels for whole image, and 0.13 and 1.78 pixels for two regions. Further experiments were conducted to observe the maximum values in the Hough space after processing two regions simultaneously. The result shows two peaks that indicate two centres of circular rings as shown in Figure 5.
A-20 Prooceedings, Komputer dan Sistem Intelijen (KOMMIT 2002) Auditorium Universitas Gunadarma, Jakarta, 21-22 Agustus 2002 Region 1 2 Table 2. Result of experiment using simulated image, two region Shift Theoritical centre Detected centre Detected X Y X Y disparity 1 0.50 11.45 0.59 13.24 1.16 1 2 1.00 22.90 1.10 24.22 2.11 2 3 1.50 34.36 1.66 34.63 3.02 3 4 2.00 45.81 1.90 44.03 3.84 4 5 2.50 57.26 2.74 53.42 4.66 5 1 0.50 11.45 0.75 9.90 0.86 1 2 1.00 22.90 1.13 24.69 2.16 2 3 1.50 34.36 1.41 34.71 3.03 3 4 2.00 45.81 1.89 43.82 3.83 4 5 2.50 57.26 2.43 54.14 4.73 5 Figure 5. Representation of the Hough space for image with two regions 2.2 Experiment Using Realistic Image Three experiments were performed using real images. The images were acquired using Alhazen image capturing facility. The first and the second experiments used a single real image while the other experiment used two images captured by stereo cameras of the Alhazen. For all experiments, the images were converted into random dot images using the filtering and binarized into 5% dot density level. In the first and the second experiment, the superimposed images were obtained by overlaying the original image with the copy of itself, where the copy was shifted by 2 pixels. The difference between the first and the second experiments is that in the second experiment, noise was added. In the third experiment, the image captured by the left camera was superimposed with the image captured by the right camera. So in this case the disparity was not exactly known. To simplify the experiment, the cameras were setup so that the average difference between the images was around 2 pixels. For this experiment, the view at one of the University of New Brunswick labs was used (Figure 6) Cameras were placed 12.6 cm apart (baseline = 17 cm) and approximately 751.5 cm from the objects in the scene (it is assumed that objects in the centre of the view are approximately at the same distance).
Circular Moiré Patterns In 3D Computer Vision Applications A-21 Figure 6. View at one of the UNB labs The left image, the right image and the noisy image (the left image with noise added) are shown in Figure 7 including their histograms and pseudo-random dot images. As we can see the white noise was not clearly visible in the real image, however it is visible in the histogram. Figure 7. Images used in this experiment
A-22 Prooceedings, Komputer dan Sistem Intelijen (KOMMIT 2002) Auditorium Universitas Gunadarma, Jakarta, 21-22 Agustus 2002 The superimposed images for all three experiments can be seen in Figure 8 Meanwhile the result of the experiments can be seen in Table 4.4. Figure 8. Result of superimposing images Table 3 Result of experiments using real image Shift Theoritical centre Detected centre Detected X Y X Y disparity 1 0.50 11.45 0.54 11.37 0.99 1 2 1.00 22.90 1.27 22.07 1.93 2 3 1.50 34.36 1.92 33.20 2.90 3 4 2.00 45.81 1.58 45.86 4.00 4 5 2.50 57.26 2.29 55.13 4.81 5 The experimental result shows the successful application of finding disparity. However the accurate results were obtained only for the Y coordinate. This error occurs due to the quantization of rotated image and limited resolution of the Hough transform 3. Similarity Measurement Application Computing similarity between images is a basic task, required for image classification and matching. A typical similarity measure for image regions is based on the difference of intensity: m n D = I (,) xy I (,) xy (2) x= 1 y= 1 1 2 where I 1 and I 2 are image intensities for two mxn regions under investigation. The above measure can be applied to image regions and normalized with variances and averages, or expressed in term of correlation between two regions: 2 σ 1,2 C = (3) 2 2 ( σ σ ) 1/ 2 1 where σ 1 and σ 2, are variances and the covariance of the respective regions. Such a similarity measure, when applied to matching problem, is typically thresholded using an appropriately chosen value to render a Boolean: TRUE or FALSE result. 2
Circular Moiré Patterns In 3D Computer Vision Applications A-23 3.1 Matching Based On Moiré Patterns Matching based on Moiré patterns differs from conventional matching in that the matching points are not compared under a search mask but rather a circular pattern resulting from superimposing two images is analyzed. Since the pattern is either present or not the proposed technique for similarity measure based on Moiré patterns by its nature gives a binary result. The origins of the circular Moiré patterns can be best explained by analyzing a mapping between two disks. When two identical random dot fields are superimposed and one of them is rotated by a small angle then the concentric circles can be seen around the axis of rotation [4]. The centre of these circles corresponds to the location of the fixed point predicted by the Brouwer s theorem. The Brouwer s theorem [7] guarantees that in a continuous mapping of a non empty set in the n- dimensional Euclidean space R n into itself there exists at least one fixed point, i.e. a point that is mapped into itself. The simplest example of this transformation is a rotation of a disk about its centre. The corresponding feature points in the vicinity of the fixed point lie on concentric contours. Since the concentric circle of Moiré texture can be observed visually, an attempt has been made to capture the properties of these patterns automatically. The detection of the circles is accomplished by calculating the point of intersection of vectors normal to the circle boundary. These vectors, in the case of concentric circles, have common intersection point located at the fixed point. 3.2 Implementation In order to apply the proposed matching algorithm, gray level images must be transformed to random dot like patterns and then rotated and superimposed. Transformation to a dot pattern is performed using a high pass filtering technique (3x3 Laplacian) and then binarizing the image. Superimposing two dot images, one of which has been rotated by 3 degrees results in appearance of concentric circles only in the case of a match. The detection of this pattern is accomplished using a Hough technique. First the standard Hough transform [8] is applied to detect straight lines locally using a small window (16x16) and then the detected lines are traced in the circular Hough transform [9]. The circular Hough transform accumulator array is smoothed with a 20x20 mask and then investigated. A high count at the central location in the array indicates a large number of lines crossing at this location and thus marks the presence of concentric circles in the superimposed image. In the experiments reported in this paper, the image resolution is 128x128, the angular resolution of the linear Hough transform is 5 degrees and the resolution of the circular Hough transform matches the size of the image (128x128). Circles are detected when the maximum count in the circular HT accumulator array is in the centre and is at least twenty five (intersection point of at least 25 lines detected). 3.3 Application The concept of matching through the detection of concentric Moiré rings has been verified by applying the proposed algorithm to the face recognition problem. Four images were used, and one of them (image 4) was selected and compared to all of them. The results of applying the Moiré based matching algorithm are shown in Figure 9. The original images (a) were processed with a high pass filter to obtain the random dot representations (b). Then the random dot representation of image 4 was rotated and superimposed on each of the images 1 to 4. The resulting patterns are shown on Figure 9(c). In case of the match: (i) the concentric circles are observed visually and (ii) the Hough based detection algorithm accumulated the highest count and thus the matching pair of images was successfully detected. It should be noted that the circular Moiré patterns are only visible for highly correlated images resulting in good (binary) discrimination between match and no match results.
A-24 Prooceedings, Komputer dan Sistem Intelijen (KOMMIT 2002) Auditorium Universitas Gunadarma, Jakarta, 21-22 Agustus 2002 The performance of the matching was also tested in the presence of random noise. In this experiment the unknown image (image to be recognized), image 4, was altered by random noise. For random noise generation the built in option available in Paint Shop Pro [10] image processing package was used. Testing was conducted for random noise levels 1 to 20. The resulting changes, i.e. the decrease, of the maximum count were observed when the noisy image 4 was compared with a non-matching template (Figure 11). The detection method is expected to perform successfully up to the noise level 14, beyond which the maximum counts for the matching and non-matching pairs become the same (compare Figures 10 and 11). (1) (2) (3) (4) (a) (b) (c) (d) Figure 9. Face recognition using Moiré based algorithm: (a) original images, (b) result of transformation to random dots, (c) result of matching algorithm, (d) highest counts in the HS
Circular Moiré Patterns In 3D Computer Vision Applications A-25 Figure 10 Figure 11 4. Conclusion The Moiré based technique presented in this paper is designed to compute a disparity of image and binary similarity measure for two digital images. When two identical images are superimposed, with one of them rotated beforehand, circular Moiré patterns are exhibited. The circles with the centre at the fixed point predicted by the Brouwer s theorem can be observed visually or detected using a Hough based technique. The Moiré based technique can be applied to region matching and disparity measurement in stereo vision applications [5]. Simulation experiments reported in this paper demonstrate a practical application of the above technique to face recognition. 5. References [1] Shirai, Y. Three-dimensional computer vision, Springer-Verlag, 1987. [2] Glass, L. Moiré Effect from Random Dots, Nature, vol. 223, 1969, pp. 578-580. [3] Giger, H. Moirés, Computers & Mathematics with Applications, vol. 12b, 1986, pp. 329-361. [4] Glass, L.; Pérez, R. Perception of Random Dot Interference Patterns, Nature, vol. 246, 1973, pp. 360-362. [5] Pochec, P. Moiré Patterns in Disparity Measurement, in Proceedings IEEE PACRIM '95, Victoria, B.C., 17-19 May 1995, pp. 420-422. [6] Lacourse, P., Pochec, P. Stereo Matching Based on Fixed Point Theorem, Vision Interface'93, York University, Toronto, May 18-21, 1993, pp. 107-109. [6] Flegg, G. H., From Geometry to Topology, Crane, Russak & Company, Inc., N.Y., 1974. [7] Hough P. V. C., Method and means for recognizing complex patterns, U.S. Patent 306954, Dec. 19, 1962. [8] Illingworth, J., Kittler, J., 'The Adaptive Hough Transform', IEEE Transactions on Patterns Analysis and Machine Intelligence, Vol. PAMI-9, No. 5, Sept. 1987, pp. 690-698. [9], Paintshop Pro Image Processing Package, JASC Inc, PO Box 44997 Eden Praire, MN 553444 USA Acknowledge Thank you to Dr. P. Pochec (pochec@unb.ca) from University of New Brunswick for his kindness support, idea, spirit for accomplishing this research