of 3D Models from Specular Motion Using Circular Lights

Transcription

1 Reconstruction of 3D Models from Specular Motion Using Circular Lights Jiang Yu Zheng, Akio Murata, Yoshihiro Fukagawa, and1 Norihiro Abe Faculty of Computer Science and Systems Engineering Kyushu Institute of Technology Iizuka, Fukuoka 820, Japan Abstract This work generalizes a shape recovery method for specular surjaces. We have used linear lights that generate planes of ray to highlight a rotating object so that its specular su$aces m computed from continuous images. By setting lights properly, shape at each rotation plane can be measured and a 30 graphics model can be subsequently reconstructed. In this paper, we extend our lights to circular ones to produce cones of ray. Under such illuminations, all su&ce points with different normals will be highlighted in a rotation period. The computation of shape is even simple compared with linear light case. We give results of shape recovery on real objects. 1. Introduction The current techniques of 3D shape reconstruction in vision can not deal with specular surfaces successfully. Even laser range finders are not able to get correct shape of a surface with strong specular reflectance[ 131. In the real world, however, many objects from industrial products to daily goods contain specular reflection. The objective of this work is to recover 3D shapes of specular surfaces by rotating them so as to generate graphics models for display. The constructed models will be useful in CAD, multimedia database, information network, and virtual reality. It is well known that, on a specular surface, motion of surface texture differs from motion of a reflected pattern of environment as view point shifts. This is a cue used by human in perceiving specularity. Estimation of 3D points on a specular surface has been studied on surface normal aspect in static image[ 1,2], on disparity aspect from two view points[3], on motion aspect with view point move [4,12] or object move[9,11], generating a surface curve or surfaces under a point or extended illumination. This work is an extension of the model recovery methods in [9,11]. We use circular lights to illuminate rotating objects. Every object point, regardless of its surface normal, will be highlighted once during the rotation. This allows us to recover more specular surfaces. Moreover, the generalization yields the simplicity in the geometry relation between normal, light and camera so that effective computation can be achieved. In the following sections, we will summarize the previous related works and derive light shape that can satisfy our needs of extension. Then we will apply our algorithms to different objects. Finally, we summarize the relation between illuminations, objects, and algorithms. 2. Previous Works Using Plane of Ray Under orthogonal projection, the newest results on 3D estimation of specular objects are 3D surface and model recovery by rotating objects [9.1 I], and the computation of a 3D surface curve by controlled view point move For a rotating object, our camera axis is set orthogonal to the rotation axis and continuous images are taken (Fig.1). In order to illuminate specular surface, we have put extended lights (linear, curved, or combination of linear lights around object) in a plane that contains the rotation axis. These lights are far away from the object compared with object size so that rays from a point of light to different surface points have the same direction. The plane is named as Plane of Ray, since the rays arriving at the object are almost in the plane. Fig. 1 Taking images of objects with specular reflection. Two kinds of highlight information help us know local and global surface shape. The first is the observed shape of highlight stripe and the second is its motion on surfaces due to the object rotation. These informations can be parameterized by normal direction of highlight stripe in the image and trajectory of highlight in the image sequence. For each rotation plane at different height, we can determine, in our camera configuration, an image line that is the projection of the plane. An Epipolar-plane image is collected from the line and the motion of highlights is tracked in the EPI. One of such example is given in Fig. 2. As the object rotates, a highlight moves over surfaces gradually. We have computed 3D positions of the passed points on each rotation plane; using moving trajectory of highlight in the FPI and corresponding highlight directions (or gradient direction at highlight stripe) in the images[l4]. With one or two planes of ray setting at various directions from the camera axis, we give a summary of our algorithms (Table 1) for different types of shapes. If a single plane of ray Y /96 $ IEEE Proceedings of ICPR

2 is set, the shape recovery is a 1st order differential equation. Multiple planes of ray located at orientation $t and $2 from the camera axis will make the problem simple as linear equations. If the object is a cylinder or the plane of ray aligns with the camera direction, extraction of normal of highlight in the image is unnecessary. All the surface points are possible to be highlighted only in the case where the plane of ray is with the ny along the rotation axis is constant during the rotation and the component (n,, n,) in the rotation plane faces to all orientations periodically. The extended illuminant we select will make every point be highlighted when its normal component (n,, n,) rotates to a designated angle; regardless what its ny component is. We can thus degenerate 3D reconstruction problem to 2D recovery problem in each rotation plane. We will figure out the shape of illuminant using a Gaussian Sphere. R(Y) Fig. 2 Epipolar Plane Image parallel to rotation plane on u which a trace of highlight can be observed. Single Plane of Ray Plane of ray Multiple (linear lights) lane Of Ray Light f = 0 Fig. 3 Gaussian Sphere showing relations between light, Cylinder by==o) surface normal and viewing direction. linear 1st order differential I equations I equation & fixed points As depicted in Fig. 3, a surface normal N is displayed, without its real position on object, on the sphere at the system C-xyz. The camera direction is V(O,O,l). We want a point being highlighted when its nxz rotates toward a given angle, Table 1. Shape recovery under different conditions such as object shape, number of plane of ray and their locations. ny is the component of surface normal along the rotation axis. 3 Generalization on Light Shape What is an ideal illumination? A point light can only recover a surface curve. Generating a complete model needs as many highlight points as possible in a rotation. We will use circular lights, which not only satisfies the requirement but also omits the computation of highlight shape in the image. The shape recovery is thus done on each EPI separately, which is more stable and efficient. We locate object coordinates system O-XYZ so that its Y axis is on the rotation axis. The rotation angle 0 is known and is clockwise in the rotation plane. Through a simple computation, we can detect the rotation axis in the image frame. The image can be transformed so that its y axis is on the projected rotation axis. As an object rotates, a spatialtemporal volume can be piled from its continuous images. A surface point P(X, Y, Z) in the system 0-XYZ is mapped into the volume continuously. It forms a trajectory p(x(0), y, 0) in the volume, where y in the camera coordinate system C-xyz is equal to Y constantly. Surface normal at the point is denoted as N(n,, ny, n,) in the system 0-XYZ. A ray from a point of light is denoted as L(l,,l,,l,) in the system C-xyz. The component say $I/~E (-n/2, n/2), in the rotation plane. Surface normal N is then in the vertical plane P-RON0 where LVONo=+/2. According to the specular reflection criterion, the ray L highlighting the point is in plane P-VON and has angle of incident LLON equal to angle of reflection LNOV. Changing N from -n to +K in P-RONu, plane P-VON will rotate around V. We can deduce that ray L draws a circle on the Gaussian Sphere symmetry to the horizontal plane and through vector V and LO (LLfjOV=$). The way of proof is given in the following. According to constraint of specular reflection LLON = LNOV (1) we have the product of vectors as LxN=Nx(-V) G9, where N is a unit vector in P-RON0 with changeable n,,. It can be written as (( 1 -n,*) *sin+/2, ny, -( 1 -n,*)* *cosq/2). From Eq. 2, we obtain three sub-equations. Eliminating ny in the sub-equations, we can deduce that the direction of light L(l,,l,,l,) satisfies 1, = 1, ctg(q/2) +l (3)7 which means vector L is in a plane parallel to the rotation axis (y axis). Because L is also a unit vector on the sphere, it must be on the intersection of the sphere and the plane, which draws a circle. The rays from the light compose a cone, named Cone of Ray. Fortunately, we can set a circular light to realize the illumination. The circular light should be set symmetrically to the xz plane in the system C-xyz. The two points on both 870

3 circular light and X2 plane should be located (1) on camera axis and orientation $ from the camera axis respectively, and (2) having the same distance from the rotation axis. A large light compared with object size is preferred, because its rays to object are close to ideal rays centralized at the sphere center. Given a light with known diameter y and selecting an orientation $/2 for catching highlight on objects, the light can be set uniquely with the two points at distance (y/2)scs$/2 from the rotation axis, and at orientations I$ and I[: from the camera axis respectively. stereo, except the focused points are highlights other than edges from texture or corners. The computation is very simple and the result is only related to light setting, rotation angle, and image positions of highlights. l 4 Shape Estimation under Two Cones of Ray The circular light guarantees that a surface point will be highlighted when its normal rotates to a given orientation. From the camera geometry (Fig. 4). we know the viewing direction under orthogonal projection is V(-sine, 0, cos0) in system O-XYZ. The image position of a point viewed at the rotation angle 8 can be written as x(e) = P x = X(e) cost3 + z(e) sine (4), where x is the unit vector of the horizontal image axis. Obviously, the position that reflects the light to the camera will shift on surfaces during rotation (surface points in turn pass the direction $/2). I,highlight traces LvOI,,; SpLial circular lights Temporal orthogonal projection Image frame )a Fig. 4 Geometry of camera and object in the rotation plane. If we set two circular lights passing through V and at the directions $1 and $2 from the camera axis as Fig. 5 shows, an arbitrary point will reflects them to camera when its normal rotates in turn to two vertical planes which have angles 4rt/2 from the camera axis respectively. The point has a delay Ar$=$t/2-$2/2 of being highlighted. Two lines of sight through the projected highlights will cross at the surface point geometrically, which determines the point position as shown in Fig. 6. The equations of lines of sight could be written as xl(e) = X(9) c0se + Z(e) sin0 (5) x@+a$) = X(e) cos(b+a$) + Z(0) sin(fl+a$) (a which yield the position of the surface just as i u ;;)=zk&z$t..:;) ;:::[x2$!$j(, where X(e) and Z(e) are the coordinates of surface points at the rotation plane. The principle used here is similar to motion EPI (b) Fig. 5 Two circular lights set to produce two cones of ray. (a) light and system setting. (b) top view of light setting. In the EPI, a surface point has its moving trace as sinusoidal function of the rotation angle according to Eq. 1, even it is unable to locate visually in the image. To know 3D position of a point needs to match two projections on the trajectory. Two highlighted positions prolvide the information. The sinusoidal trajectory crosses two highlight traces when the xz component of the normal for the point rotates to and $~/2. We thus track a highlight trace and find correspondence of each point at the second highlight trace that has delay A$ (Fig. 7). The detection and tracking of highlight has been in [9,11,14]. Any zero curvature point (appearing on planes) on the rotation plane will yield two horizontal segments in the EPI; having the same delay A@ in reflecting two lights. By measuring the difference of 9 between the two segments, we calibrate the difference in the orientation of two lights described above. 871

4 5. Shape Estimation under a Cone of Ray With one circular light, surface points with any kinds of normals can be computed by solving a 1st order differential equation as what has been used for cylinder (Table. 1). The equations are similar as [ 141 and are written as ~cosp+zsin (,so =$+in(o+~)+~tanqsin(o+~) *+- (8) ii6 2 zcosp /OS!,, 6+- =~cos(6+f)-xcrg6cos(6+~) CM 2- in the domains ti#tn/2, 3x/2, and &O, n, respectively. We can compute their solutions and corresponding X and Z using Eq. 4. Objects can be in general shape, and the light can be set free other than aligning with the camera. The light around an object at the camera orientation given in Table 1 is one of our case here where the circular light is set at the direction $=O. The cone of ray degenerates to plane of ray. Table 2 summarizes the algorithms used under cone of ray illumination. The computation can be carried out on each EPI so that it can achieve a good performance. circular fluorescent lights are commercial products with diameter 32cm. The image projection of the rotation axis is determined by imaging a standing pole standing at the center of the turn table. The accurate light direction A$ is then calibrated using EPI of a vertical planer mirror at the center of turn table. The camera is at about 2.5m away from the object and a lens with long focal length is used. On principle, every point is able to be highlighted once under a circular light in a rotation period. However, if an object has very deep concave shape, the ray might be occluded so that no highlight is observable on such surface. Occlusion in stereo also happens here. Figure 8 gives an example of a specular sphere under two cones of ray and its recovered shape displayed in graphics. We can notice the result is very accurate. Fig. 8 Test of our algorithm on a sphere. (a) Original image. (b) recovered model. Fig. 6 Two lines of sights cross at the surface point. X 64 (b) irfate I point conto tor Fig. 7 Tracking highlight traces in an EPI and matching Table 2. Shape recovery method under different lights for different types of objects. 6. Experimental Results We have done experiments on real objects. The system setting for modeling objects is displayed in Fig. 5. Two (d) Fig. 9 A bottle made of metal and its recovered 3D model. (a) Original image. (b) Recovered whole model. (c) Top part of the object in wire frame. (d) top part model in shading. Figure 9 and 10 show two objects and their established models. One is a bottle made of metal (Fig. 9(a)) that has surface normal at all directions. One of its EPI is shown in Fig. 2. The reconstructed model is shown in Fig. 9(b). The overall shape is recovered well, except the place with high ny. This is because the bottle size is relatively large compared with the circular lights. The rays arriving at such a place are not as good as ideal cone of ray. We take top part of the bottle and put it 872

5 as ideal cone of ray. We take top part of the bottle and put it near the rotation center. The computed model is displayed by shading mode Fig. 9(c,d) and wire frame mode in Fig. 9(e). Those shapes in Fig. 9(b) are improved. Figure 10 shows a plastic toy and its recovered models. Although the specular reflection component is not strong, we can still recover major part of the object displayed in Fig. IO(c)(d). The real acquired data can have almost 512x360 (image size x rotation angle) points. The unevenness on the model is due to our limited resolution in model display. Only 113 of the real measured data are used in the display. The surface will be very smooth if data are all used. The missing parts on the model are because unclearness of highlights on weak specular surfaces, occlusion of ray by other parts, and failure in highlight extraction. The stronger the specular reflection, the better the result is. For those objects with weak specular reflectance, highlight tracking is more difficult than those shining metal objects. Also for a deep valley on a surface, highlight may be occluded by other parts. Moreover, uneven surfaces may cause difficulty in tracking waving highlight traces in EPI because of resolution limitation. 7. Conclusion In this paper, we generalized a method of shape recovery for objects with specular reflectance. By using circular lights to illuminate rotating objects, every point on objects is possible to be. highlighted at a desired orientation regardless of its surface normal. General types of objects are able to be measured and modeled with a simple computation. Light position is free compared with the previous method of using linear lights. We have also done experiments on real objects and obtained good results. References [l] K. Ikeuchi, Determining surface orientations of specular surfaces by using the photometric stereo method, IEEE PAMI, Vol. 3, No. 6, , [2] G. Healey and T. Binford, Local shape from specularity, ICCV, pp.lsl [3] A. Black, G. Brelstaff, Geometry from specularities, 2nd ICCV, pp , [4] A. Zisserman, P. Giblin and A. Blake, The information available to a moving observer from specularities, Image and visual computing, 7, pp , [5] H. Baker, and R. Belles, Generalizing epipolar-plane image analysis on the spatiotemporal surface, CVPR-88, pp.2-9, [6] P. K. Horn, Shape from shading The MIT Press [7] J. Y. Zheng, and F. Kishino, Verifying and combining different visual cues into a complete 3D model, CVPR92, pp , [8] J.Y. Zheng, Acquiring 3D models from sequences of contours, IEEEPAMI, Vol.16, No.2, Feb. pp.l63-178, [9] J. Y. Zheng, Y. Fukagawa, T. Ohtsuka, N. Abe, Acquiring 3D models from rotation and a highlight, 12th ICPR, Vol. 1, pp , [IO] J. Y. Zheng, H. Kakinoki, K. Tanaka, N. Abe, Acquiring 3D models from fixed points during rotation, ICARCV 94, Vol. 1, pp , [11] J. Y. Zheng, Y. Fukagawa, N. Abe, Shape and Model from Specular Motion, 5th ICCV, pp.92-97, [ 121 M. Oren, S. K. Nayar, A theory of specular surface geometry, 5th ICCV, pp , [13] J. Clark, E. Trucco and H. Cheung, Improving laser triangulation sensors using polarization, 5th ICCV, pp , [ 141 J. Y. Zheng, Y. Fukagawa, N. Abe, 3D surface estimation and model construction from specular motion, submitted to IEEE Trans. PAMI, (c) extracted highlight traces. ( I I I am-l (e) estimated shape at the corresponding rotation plane. (d) matching points on highlight traces. (f) recovered 3D surface Matched segments are connected. model. (la (h) (9 - Fig. 10 A plastic toy and. its recovered model under two cones of ray illumination.