Lens Screw Pad Arm (a) Glasses frames Screw (b) Parts Bridge Nose Pad Fig. 2 Name of parts composing glasses flames Temple Endpiece 2.2 Geometric mode

Transcription

1 3D Fitting Simulation of Glasses Frames Using Individual s Face Model Noriaki TAMURA and Katsuhiro KITAJIMA Toko Universit of Agriculture and Technolog, Toko, Japan @st.tuat.ac.jp, kitajima@cc.tuat.ac.jp Abstract This paper proposes a novel 3D simulation method of wearing glasses frames that fits an individual s face. The method is based on our face modeling technique that can quickl generate an individual s face polgon model with sufficient accurac from several pieces of image of a digital camera. In the optician s shops, 2D simulators based on the image composition of a face and glasses frames are widel used at both real optician s shops and virtual shops. Although 3D simulators have recentl been put on the market, little of them are actuall used. The main reason is that it is difficult to generate 3D individual s face models rapidl and in low cost. We solved both problems b appling our 3D face modeling technique which we call the Generalied Free-Form Deformation (GFFD) method. Moreover, in most of conventional 3D simulators, there is such a problem that glasses frames do not fit one s face because the frames in standard sie are just located on the standard position. So, users can not be satisfied with the visual effect of such simulators. We also solved this problem b newl developing a 3D fitting simulator to make glasses frames fit an individual s face. We achieved this b formulating the fitting (adjustment) procedure that is actuall conducted b skilled operators at real optician s shops and then developed a novel method to automaticall scale and deform the glasses frames in standard shape and sie into the frames that fit the face. Kewords: glasses, glasses frames, individual s face model, fitting simulation, GFFD 1. Introduction Simulators of wearing glasses frames are widel used at both real optician s shops and virtual shops through the Internet. The biggest advantage of using them is that customers can virtuall put various tpes of glasses frames on and can watch their own faces wearing them, even though the are not kept in the real shops. 3D simulators of wearing glasses frames have recentl been put on the market [1]. Most of simulators used now are, however, not 3D, but 2D [2, 3, 4]. The main reason wh the conventional 3D simulators have never become popular is that the need special expensive devices such as laser scanners and grating pattern projection scanners. Moreover, at real shops, it is necessar to quickl generate an individual s face model ever time a new customer comes to the shops. So, the authors solved the former problem b using digital cameras as input devices, and the latter one b appling our 3D face modeling technique which we call the Generalied Free-Form Deformation (GFFD) method [5] to our simulator of wearing glasses frames. The main advantage of 3D simulators compared to 2D ones is that the face wearing frames can be watched from various directions. However, in conventional 3D simulators, it often occurs that the nose pad or the temple of frames usuall intersect or do not touch the face when it is watched from various directions. This reason is that the most of conventional 3D simulators has such a problem that glasses frames do not fit one s face because the frames in standard sie are just located on the standard position and never be adjusted. When we purchase glasses at real optician s shops, the shop s operator adjusts frames temples, nose pads, etc. to the face so that the frames fit the face untightl and unloosel. This adjustment operation is called fitting [6]. B formulating this fitting operation, we have developed a 3D simulator of wearing glasses frames that fits frames to individual s face models. 2. Geometric Model of Frames and Faces 2.1 Tpes of frames in terms of their form First, we mention the form of glasses frames we use in our simulator. Frames are classified into three tpes such as full rim, half rim and rimless in terms of the form of frames rims that hold lenses (Fig. 1). Among them, it is rimless frames that can most easil adjust their sie to the face s one because their lens sie can be freel decided at the lens edging process. Therefore, taking practical use into consideration, we have first started to target rimless frames. Fig. 2 shows the whole glasses frames and their composing parts. (a) Full rim (metal material) (c) Half rim (b) Full rim (plastic material) (d) Rimless Fig. 1 Glasses frames classified b rim tpe 91

2 Lens Screw Pad Arm (a) Glasses frames Screw (b) Parts Bridge Nose Pad Fig. 2 Name of parts composing glasses flames Temple Endpiece 2.2 Geometric model of frames The frames consist of 17 parts. The geometric model for each part is a triangle polgon. The authors made it from standard sie real frames b 3D modeling software. 2.3 Geometric model of faces As mentioned in 1.introduction, we generate individual s face models b using our GFFD method. It first extracts feature points from the face images of three views and computes their 3D coordinates. Then, it transforms a generic face model which we generated as a polgon model in advance to the individual s face model b treating the feature points as the control points of the GFFD and deforming the space. The method makes it possible to generate an individual's face model rapidl with sufficient accurac without an special expensive devices. Fig. 3 shows the coordinate sstem that the face model is defined. We define the frontal plane, the sagittal plane and the horiontal plane that are used in the facial restoration method of forensic medicine as x-, - and -x plane, respectivel. Then the point that three planes intersect is determined as the origin. In the next chapter, we will mention the method of frames fitting simulation using the models of both frames and faces. Tip 3. Method of Frames Fitting Simulation 3.1 Positioning the frames for a face (Preprocessing) In order to transform the frames parts models so that the whole frames fit a face, we first decide the method of setting the frames on the face at the position that the are worn. One of the authors knows how the skilled operators at real optician's shops set the frames. The process is as follows: First, we locate the center of frames on the - plane of the face (Fig. 4(a)). Next, we lean the frames so that the angles between their lenses and the x- plane will be 10 degrees (Fig. 4(b)). Then, we decide the position of direction of the frames so that the center of the lens will be 2mm below the center of the pupil (Fig. 4(c)). The value 2mm, which is called ee point height, is usuall adopted at the most of optician s shops. Finall, we decide the position of direction so that the distance between the lens back and the center of the pupil will be 12mm (Fig. 4(d)). (a) Deciding x position (b) Leaning frames 10 degrees pupil 2mm 1 1 Frontal plane (x- plane) Horiontal plane (-x plane) Origin Sagittal plane (- plane) Fig. 3 Coordinate sstem representing a face model x (c) Deciding position 12mm pupil lens back surface (d) Deciding position Fig. 4 Positioning frames to a face 92

3 3.2 Bending pad arms and leaning nose pads along nose's shape After preprocessing, nose pads do not touch the nose surface but apart from it or might stick into it (Fig. 5). B bending the pad arms and leaning the nose pads, we can make the nose-contacting surfaces of the pads fit the nose. The whole process is divided into the following Steps A, B. Step A: Making the position of the nose-contacting surface coincide with that of the nose surface First, we designate a vertex P on the polgon near the center of the nose-contacting surface. Next, we obtain the point Q which is the cross point between the straight line of direction from P and the surface of the nose that should contact the nose pad (Fig. 6). Then, we bend the pad arm b the angle λ x around the vector of x direction passing the point O so that P will coincide with Q (Fig. 7). The angle λ x is calculated as follows: (a) Not touching (b) Sticking Fig. 5 Examples of nose pad not fitting nose P Distance PQ M N N Q λ x = cos 1 ( P + PQ 1 ) cos ( P + P P 2 P + P 2 ), (1) Fig. 6 Top view for frames and a face where (P x, P, P ) is the coordinate in a local coordinate sstem, the origin of which is the point O. At this point, the position of the nose-contacting surface coincides with that of the nose surface, but their orientations are different each other. Step B: Making the orientation of the nose-contacting surface coincide with that of the nose surface In this step, we rotate the pad to make the orientation of the nose-contacting surface coincide with that of the nose surface. The rotating angle α and the vector A for the axis of rotation (as shown in Fig. 8) are easil calculated using the inner product and outer product of two vectors, as follows: O λ x Bending point (Center of rotation) Direction of rotation ( ) cos 1 M N M ( N ) α = ( ), A =. (2) M Ν M ( N ) Fig. 7 Bending a pad arm where M and N are the average normal vectors of all faces on the nose-contacting surface of the pad and the nose surface, respectivel. After Step B, the position of the points P and Q ma slightl change. So, we judge the distance PQ between P and Q. When PQ becomes within a certain threshold value Td, the whole process is over, otherwise it returns to Step A. Fig. 9 and 10 show the results of the nose pad fitting procedure for five face models. Center of rotation Fig. 8 Rotating a nose pad A : Axis of rotation 93

4 Not touched Fit (a) Not touching (b) Sticking Fig. 11 Tips not riding on ears (a) Before pads fitting (b) After pads fitting Fig. 9 An example of nose pads fitting Center of rotation R T T R TR β RS S S β (a) Side view (b) Top view B Fig. 12 Straightening a tip T H S φx Fig. 10 Other examples of nose pads fitting for four people 3.3 Bending endpieces to make tips ride on the ears and adjusting lens width In general, after the nose pad fitting, the left/right tip does not fit each ear and the width of frames does not fit that of one s face, as shown in Fig. 11. In the real optician s shops, skilled operators conduct the following three procedures to solve the above mentioned state. The first straighten the tip, then bend the endpieces and finall change the width of lenses so that the width of frames fits that of the face. Therefore, we formulated those procedures as the following three Steps C E. Step C: Straightening the tip Usuall, tips are bent at the standard bending point R for a standard face, as shown in Fig. 12. So, we first straighten the tips. To straighten a tip, we bend the point R on it. We can easil obtain the rotation angle β and the direction vector K of rotation axis, b calculating the inner and outer products of RS and TR. B (a) Side view E T H (b) Top view Fig. 13 Bending an endpiece S Step D: Bending the endpiece We bend the endpiece b φ x around the rotation axis that passes the point B and is parallel to the x-axis so that the tip should contact the ear at the point E when viewing from x direction, as shown in Fig. 13. The φ x is calculated as follows: 1 b 1 c cos ( ) cos ( ) (a 0) a + b a + b φ x = (3) 1 c 1 b cos ( ) cos ( ) (a<0), a + b a + b where a = m E + n E, b = n E m E, c = n T m T, m = S T and n = S T. x 94

5 (a) Side view Fig. 14 Tip riding on an ear (b) Top view E1 Ei-1 Fig. 15 Bending a tip Ei T Ei+1 En S Step E: Adjustment of the width of the frame In this step, we adjust the width of the frame b enlarging the sie of the lens b the amount D x from the contact point E on the ear to the cross point H between the tip and the vector of x direction from E. After the endpiece is bent b φ x, T and S move to T' and S', respectivel. Then, D x is calculated as follows: ( S x' T x')( E T ') Dx = Ex T x'. (4) S ' T ' Fig. 14 shows an example of the whole process. 3.4 Bending tips along the ears Finall, after making the width of frames fit that of the face, we bend the tip from the point E 1 to E n (Fig. 15) along an ear. This procedure is as follows: Let U i and V i be E i-1e and i E ie i+ ( U 1 1 = TS ), respectivel, and the angle between U i and V i θ i, then it is obtained as the inner product of U i and V i. Then, we have to obtain each rotation axis for i, because the orientation of each axis ma alter. Let the outer product of U i and V i be G i, then the portion from the point E i to the tip s end is rotated b θ i around the vector that is parallel to G i and passes E i.. In order to rotate the portion, we need to get the vertices that compose it. Let all vertices that compose the tip model be P j (j=1,2,,k), then we can decide the vertices composing the portion that P j satisf the following expression: N i EiPj> Ui Vi 0, N i = +. (5) Ui Vi Fig. 16 An example of tip fitting Fig. 17 Other examples of tip fitting for three tpes of face models 4. Experimental results We conducted some experiments for the whole procedure b using various tpes of face models. Fig. 18 shows the results of the simulation that are viewed from some directions. The results show that our method of simulation works well and their visual effects are satisfactor The process is repeated from 1 to n-1for i. Fig. 16 and 17 show the results of the tip fitting procedure for four face models. In this chapter, we mentioned how to position the frames in preprocessing, how to deform the pad arms and lean the nose pads along the nose s shape, how to deform the endpieces to make tips ride on the ears and adjust the lens width and how to bend tips along the ears. B formulating each method, we can realie the whole frames fitting simulation for individuals faces, which corresponds to what skilled operators at real optician s shops do. (a) Model A 95

6 5. Conclusion In this paper, we proposed a novel method which solves the problems in conventional 3D simulators of wearing glasses. We can conclude as follows: (b) Model B (c) Model C (1) We developed a novel method of 3D fitting simulation of glasses flames. The characteristic feature of the method is that it can cope with an individuals faces with sufficient accurac without an special expensive devices b using the GFFD method. GFFD is one of the FFD (Free-From Deformation) methods, which was developed in our laborator. (2) The method is based on the fitting operation that skilled operators at real optician s shops are conducting. (3) We formulated the fitting operation stepwise. The whole procedure is roughl divided into the following three steps: (a) Bending pad arms and leaning nose pads along the nose s surface (b) Bending endpieces to make tips ride on the ears and adjusting lens width (c) Bending tips along the ears (4) We showed some results of fitting simulation applied to some people. The results are ver satisfactor. We consider that our 3D fitting simulator of glasses frames will be used not onl at virtual shops but also at real optician s shops in the near future. (d) Model D Fig. 18 Results of simulation References [1] Visionix, 3DiView, cians [2] FramesDirect, FrameFinder Virtual Eeglass Tr-On, [3] FramesRx, Virtual Frame Selector, [4] Optical Direct, E Tr-on, [5] K. Kitajima, Y. Akagi, A. Yamauchi, N. Okaawa and Y. Higuchi: A Stud on Facial Modeling Based on the GFFD Method, Journal of the Japan Societ of Precision Engineering, 74(8), pp , (in Japanese) [6] Kauhiro Tsuji: Scientific Manufacturing of Glasses, Gawk Publishing Co., Ltd., (in Japanese) [7] N. Yoshida, K. Kanou and K. Kitajima: Free-Form Deformations Based on Gaussian Functions: Fundamental Theor for Interactive Modeling, Journal of the Japan Societ of Precision Engineering, 65(7), pp , (in Japanese) 96