Acquiring 3D Models from Rotation and Highlights

Transcription

1 Acquiring 3D Models from Rotation and Highlights Jiang Y U Zheng, Y oshihiro Fukagawa, Tetsuo Ohtsuka and Norihiro Abe Faculty of Computer Science and Systems Engineering Kyushu nstitute of Technology Kawazu, izuka, Fukuoka 820, Japan Abstract Thts paper proposes an approach to acquire 3D models of objects with specular reflectance for graphcs use. Highlight and rotation informations are employed in the model recovery. We control object rotation and extract motion of a hghlight stripe, from which the object shape can be qualitatively inferred and quantitatively reconstructed by solving a first-order linear differential equation. We have experimented on simulated and real objects to obtain their models. 1. ntroduction The objective of ths work is to establish a 3D graphcs model of an object when it is rotated. For an object with rich texture, one can use shape from motion in model recovering [3,4]. For objects with only convex surfaces, shape from contour method is applicable [l]. n this paper, we deal with smooth surfaces with specular reflectance which may yield hghlights. A hghlight has been considered as a noise of surface features and some effort has been made to eliminate it from the object surface. However, in the case where few texture appears on a smooth surface, highlights plays an important rule in perceiving a shape. n order to recover a complete model, we put the object on a turn table and rotate it so that it reveals all of its surfaces to camera. The rotation is readable. The direction of a static illuminant is known by a simple calibration. The issue becomes shape from shadmg and known rotation. To simplify the problem, we use orthogonal projection and parallel illumination. At present, we assume objects have smooth surfaces. The motion information is taken from Epipolar-Plane mages parallel to the rotation plane. Two lunds of visual features fixed features either from comer points or texture points, and hghlights are considered to use. Among them, the hghlight gradually shift on object surface during the rotation; the motion of it can be tracked in EPs at different heights. From obtained trajectories, object surfaces can be calculated. t is not difficult to realize that normal can be determined from hghlights shifting on the surfaces during the rotation. As to the surface itself, we found that a smooth surface can be described by a first-order differential equation which has a unique solution. Combining those fixed features as boundary condition, we can finally reconstruct the model. n the following, we will introduce some qualitative characteristics of highlight trajectory and the principle of shape recovery scheme. Then we describe how to extract those necessary information from continuous images. Finally, experimental results are &scussed. 2 Motion Characteristics of Features 2.1 Motion of Fixed Points The system setting is &splayed in Fig. 1. When an object rotates around a known axis, its continuous images are taken and a spatial-temporal volume is piled. The y axis of the image frame is set parallel to the rotation axis using a simple calibration and the optical axis of camera goes through the rotation axis. An object point Po(, Y, Z) described in the object centered coordinates system is projected as p(x,y,8) in the volume. The surface normal at the point is n=[nx, ny, nd. The rotation angle 8 is known and is clockwise in the analysis. A linear illuminant set in the vertical hrection is long enough to produce highlights for surface normals with different n, components. t locates at distant position so that the horizontal components of incident lights are written as an approximate vector L=[L,,L.J. The components L, and L, can be further denoted by its angle $0 from the camera direction, whch can be written as L=[-sincpo, coscpo] and $oe(-zi2, ~12). Obviously, the highlight stripes move on object surface during the rotation. We therefore denote the hghlight point as H(X(B), Y@), Z(8)) and its image position as h(x(b), Orthogonal projection Axis Light Fig. 1 mage formation geometry of shape from rotation. As an object rotates, its surface points have their traces in the correspondmg EPs as sinusoidal curves of half visible period, even they are not distinct enough to be tracked. f the component of a surface normal in the rotation plane is discontinuous at a point, the shadings on /94 $ EEE 33 1

2 its both sides are different and its trace in EP appears as an edge (typically a segment of sinusoidal curve). f the albedo has a discontinuity at a point, the point also draws a sinusoidal trace of edge in the EP. These two kinds of points are called fixed points since, for such a point, multiple lines of sight through its projections in dfferent images cross at the same 3D position. / Rotational axis illumination direction mage fame / Fig. 2 One cross section of an object parallel to the rotation plane. 2.2 Qualitative Motion of Highlights Let us first qualitatively look at the motion types of hghlights according to shapes such as comer, convex, linear, concave, etc. Based on analysis of highlight's trajectories, we can qualitatively infer the shapes. f the surface normal and surface albedo are continuous at a point, matching its projections in continuous images is no longer possible as what has been done for fixed points [1,2,6]. An alternative way is to look at the shading. As an object rotates, its surface elements, in turn, face to the illuminant. A highlight, determined by the surface normal, shlft on surfaces and its trace is possible to be located with respect to the rotation angle. According to curvatures along object boundary on a horizontal cross section, shapes are categorized as either of convex corner, convex, linear, concave. We hence find an interesting effect that hghlights have some basic types of trajectories over the traces of surface points in the correspondmg Epipolar- Plane mage. Figure 3 shows hghlight moves on the surfaces and correspondlng trajectories over traces of surface points. On a convex shape, for example, the highlight moves relatively in the inverse direction of the rotation, which yields its image velocity lower than that of surface points (see its trace at right). A linear boundary, however, has its points face the light direction at the same time, which generates a horizontal stripe of highlights in the EP. Further, a concave shape has its highlight moving in the same direction of rotation; its image velocity is hgher than that of surface points. At a corner, hghlight does not appear if the corner is strictly a zero curvature. Combining dlfferent surfaces together corresponds to connecting their hghlight trajectories in EP. Figure 4 shows two types of shapes, for instance, whch are convexconcave-convex and comer-linear-convex combinations, respectively. n the first case, hghlight A moves on the first convex surface. n the meanwhile, point B with zero curvature (where shape changes from concave to convex) becomes a highlight point. t splits into two highlight points C and D that move on convex and concave shapes separately. Point D then merges with hghlight point A at another zero curvature point E and disappears. Thls splitting and merging process can be observed from trajectories of hghlights in the correspondmg EP. We can assert that trajectories of concave and convex shapes should have a smooth connection. The tangent of trajectory is horizontal at the connecting points. Ths is because splitting and merging points are zero curvature as a short linear segment which has a horizontal trajectory in EP. Similarly, we can qualitatively know the trajectory of highlight for the comer-linear-convex combination (Fig. 4(b)). t is a sinusoidal curve A followed by a horizontal segment AB, and then a trace of convex shape. At the corner point (an extreme case of convex shape), the hghlight stays constant. shape & highlight trace of highlight ir!p comer C=oO convex c>o plane c=o :oncave C d fixed $J o \ U \ % o \ 0 position of highlight 0 direction i~u"ation direction c) h trace of hiphdight s trace of surface points Fig. 3 Trajectories of highhghts over traces of surface points of Mferent shapes. As a result, a highlight shifts on object surfaces and passes all surface points at least one time if no serious 332

3 occlusion of light occurs. Generated from either fixed or shifting points on object surfaces, a queue of connected trajectories in EP acrosses trajectories of all surface points in one period of rotation. We hence attempt to compute positions of all surface points from this queue. Curvatures of shapes r>n -.. c=o Rotyc<o., Shins of highlights '. i Shapes c>o w- A Camera 0 k illumination a1 * x 3 B A Camera c] bl k illumination / * x equation from the camera geometry. ~(0) = x H = X(0) cos0 + Z(0) sin0 (4) Assuming the shape is not linear, the corresponding highlight trace is then not horizontal and we can take derivative of (4) with respect to 0 to obtain (9 x'(0) = Xe'cosO+Ze'sinO - Xsine +Zcose The first two terms express a possible shift of H on the surface and (X'e, Z'e) gwes the tangent direction of the boundary. n fact, the tangent direction at the hghlight point dlvides half the angle between camera viewing &ration and the drection of illumination in the rotational plane (incident angle equals to reflecting angle). Therefore, the surface normal dlrects to the angle 0+(p0/2-~/2 in the object centered coordinate system and we obtain equation (Xe', Ze') ( -sin(o+cpo/2), cos(0+cp0/2) )=O (6) From Eq. (4-6), two differential equations can be written as az To sin(e+~0/2) --cqs-+z ae 2 cos 0 f n \ ' ae ' a2 Fig 3 Connechon of hghhght traces accordmg to the comlnnahon of shapes (a) convex-concave-convex shape, (b) comer-plane-convex shape (1) hghhght movement, (2) hghhght traces in EF's 3 Shape Recovery Schemes 3.1 Shape Estimation from Fixed Points From the camera geometry (Fig. 2). we know the viewing &rection under orthogonal projection is v=[-sine, coso] in the object centered coordinate system. The image position of a fixed point can be written as x(0) = P x = X(0) cos0 + Z(0) sin0 (1) where x is the unit vector of the horizontal image axis. nfferentiating equation (1) with respect to 0 and using constraint of fixed point (X'e, Z'e) =(O, 0) we obtain the position of P as (2) 7 X = x cos0 - de sin0 Z = x sin0 + xtg cos0 (3) whch means the position can be estimated from x(0) and its tangent direction of trajectory at 0. For a fixed point, tlus computation can be carried out many times along its tracked trajectory. This is an over-constrained issue and fusion of multiple measurements to yield a more accurate result is possible; using either Kalman Filter or least square error method. The estimation not only gives positions at nonsmooth p'arts, but also serves ay boundary conditions for estimation of remaining parts. 3.2 Recovery of Smooth Specular Surfaces For a highlight point, we also have the following ae in the domains O+n/2, 31~12, and O*O, These equations have unique solutions as where and where Q( e) = ae cos ecos 2-; 2 J, respectively. (9) - (Do sin ecos A 2 (12) n the equations, (Xei, Zei) is a boundary condition for the 333

4 solution. We (9) and Q. (11) separately in the domains [-n/4, ~141, [n-n/4, n+n/4], and [n/2-n/4, n/2+n/4], [3n/2+n/4, 3n/2+~/4]. After Z(8) or X(8) is obtained, X(8) or Z(8) is given by equation (4). n order to improve accuracy of the solution, we take images of rotational objects at very small interval of 8. This will make the summation used in numeric calculation close to the integral in the formulae (9) and (11). f, on the other hand, a boundary is linear, multiple points become highlights at the same angle 8 so that taking derivative of x(0) with respect to 8 as done for Eiq. (5) is impossible. There are multiple values of x(8) along a horizontal line in EP. Nevertheless, a linear segment can be determined in an even simpler way. Suppose at least one points (Xi, Z;) on the line is known, any other points (X, Z) can be estimated from their image positions with the known point, i.e. X = COS(&C~()/~) (~(0) - xi(8))+ Xi Z = sin(&qi0/2) (x(8) - xi( )))+ Z; (13) since we know that the dmection of the line should be perpendicular to the drection in the middle of the camera direction and illuminating direction. 3.3 Boundary Conditions for the Solutions The boundary condition for the solutions of surface shapes using equations (9,11,13) are from positions of fixed points which are very accurate, as well as contours if there is no single fixed point available [l]. Trajectory of a comer point (discontinuity at surface normal) connects trajectories of hghlights. Whle the trajectory of a texture point (discontinuity at albedo) crosses hghlight trajectory without brealilng it as Fig. 5 depicts. At end points or crossing points of hghlight trace, 3D positions become known. From these known points, we can start estimation of smooth surface simultaneously by computing integrals along trajectories of hghlights in EPs. This process is done until a singular point with infinite x (8) (where the trace has horizontal tangent) is reached. We need to change the direction of integrals there for further propaganda of shapes. 4. Detectability of Feature Trajectories Let us examine how traces of features in the EPs can be extracted for shape recovery. Assume the diffused components of surface reflectance on both sides of a texture point are R1 and R2, and its component of the surface normal in the rotation plane is np(0). During the rotation period without hghlights, the difference of image intensities at both sides of the point is A@) = (R1-R2) S cos+@, L> = (Rl-Rz) Clcos<n(B), L> +C2 (14) where S is the intensity of the illumination and C1. C2 are constants. This means that a texture point is most ddnct for traclung when it is facing the light. We can use a normal edge detecting operator to locate the trajectory of a texture point. Assuming the horizontal components of two normals at a comer point are n 1(0) and na(8) and the albedo there is R, the difference of image intensities at the corner becomes A = RS(cos<n1(8), L> - costn2(e), L>)+c~ (15) where C3 is a constant. The variable in it is only the angle <nl(8),l>. f the comer is very sharp, A becomes zero at a particular angle of rotation where vector L divides half of the angle between nl(8) and n2(8). Th~s means the comer is undetectable and its trajectory is hard to follow when it faces the light. X trace of a corner Doint 8 from curved surface trace of a texture point, trace of a texture point x( e 3,from linear surface Fig. 5 Estimation of smooth shape startingfrom fixed points. The intensity of a surface point viewed by a camera is determined by normal direction and the illumination with respect to the camera. n order to follow the movement of a highlight, we filter the EP with an 1-D ridge-type filter and pick up peaks. We align the ridge-type filter both horizontally and vertically for detecting hghlight traces from non-linear surfaces and a linear segment. 5. Experimental Results We have done experiment on simulated scenes and real scenes. Figure 6 gives a simulation result whch has three types of shapes, their corresponding EPs and trajectories of surface points and highlights. Figure 7a shows another simulated EP from a cylinder; the light is from the same direction of the camera (rpo=o) and the rotation axis is at the center of the cylinder. The recovered shape using our method is shown in Fig. 7b. Similarly, Fig. 8a shows a hghlight trace generated from a shape which is recovered in Fig. 8b. For real objects, a bottle is put on the rotation table (Fig. 94. One of its EP is in Fig. 9b and a highlight trajectory is visible. We filter the EP to track the trajectory (Fig.9~). Figure 9d displays the shape on that rotation plane and the model of the bottle is recovered by connecting shapes on each rotation planes at different heights (Fig. 9e). For objects containing plane, a box and one of its EP is shown in Fig. loa. The light direction is about 60 degree from that of the camera. We can see that horizontal stripes of ~ghlights at four side planes in the EP; each has an interval of A63 = d2 with their neighbors. The 3D location of the box is computed by equation (3,13) and is shown in Fig. lob. 334

5 Fig. 6 A simulated result of surface point traces and highlight traces (bold curve) when objects are ro&ting. (a) three lands of shapes containg convex, concave and linear shapes (surface point are with * sign). The camera is the same as the Z axis, (b) EPs. Both specular reflectance and diffused reflectance exist. The light direction (p0=60. (c) traces of surface points and hlghhghts. Fig. 7 An ideal cylinder for shape recovery. (a) EP with a trace of fixed point (dark curve), (b) recovered shape at the rotation plane. Fig. Sa. A simulated EP whose shape is in Fig. 8b 335

6 6. Conclusion n ths paper, we described a new method for qualitative identification and quantitative recovery of shapes with specular reflectance by controlling object rotation and detecting hghlights. We are workmg on various shapes and improving accuracy of the method. We will investigate surface without specular reflectance and generalize the method. References [l] J.Y. Zheng, "Acquiring 3D models from sequences of contours", EEE PAM, Vo1.16, No.2, Feb pp , J. Y. Zheng, and F. Kishino, "Verifying and combining different visual cues into a complete 3D model", CVPR92,. pp ,1992. [3] R. Szelisla, "Shape from rotation", CVFW1, pp ,1991. [4] C. Tomasi and T. Kanade, "Shape without depth", 3rd CCV, pp.91-95, [5] A. Black, G. Brelstaff, "Geometry from specularities", 2th CCV, pp [6] P. K. Horn, "Shape from shading'' The MT Press [7] H. Baker, and R. Bolles, "Generabang eppolar-plane image analysis on the spatiotemporal surface", CVFR-88, pp.2-9, (b> Fig. 8. A simulated EP with the recovered shape. Fig. 9 (a) Fig9 (b) \ OOlliDO w"-.mdo" w Y""XCEln* %vnvlsm3m. n (9 a 43 Fig 9 Modehng a real object with smooth surfaces (a) A bottle on the rotatlon plane, a dark he is on the surface for the boundary condhon of the shape eshtnatlon (b) One EP of the bottle (c) Extracted hgbhght trace (d) The shape at the cross sectlon of the bottle (e) 3D model constructed by connechng shapes at all rotatlon plane 10 Fig. Recovering hear shape of a box. (a) Objects, (b) EP at the height of the box, (c) estimated cross section of the box 336