NICOGRAPH Internatonal 2012, pp. 114-119 3D Vrtual Eyeglass Frames Modelng from Multple Camera Image Data Based on the GFFD Deformaton Method Norak Tamura, Somsangouane Sngthemphone and Katsuhro Ktama Tokyo Unversty of Agrculture and Technology, Tokyo, Japan 50008834305@st.tuat.ac.p, s.sngthemphone@gmal.com, ktama@cc.tuat.ac.p Abstract Ths paper proposes a novel method of generatng a 3D model of eyeglass frames from multple camera mage data based on the GFFD deformaton method. rapdly wth consderably low cost. Models of eyeglass frames can be generated easly and Based on the postons of the feature ponts of eyeglass frames n photos, the 3D generc model of eyeglass frames, whch s prepared beforehand, s deformed nto ndvdual models that correspond to the eyeglass frames n the photos. We confrmed that the shapes of the models are almost same as the frames n photos by comparng the generated models wth the photos from some drectons. Snce we can generate the models easly and rapdly wth the proposed method, our 3D fttng smulators of eyeglass frames wll be put to practcal use n the near future. Keywords: eyeglass frames modelng, GFFD, free form deformaton, camera mage data, operaton ponts, fttng smulaton 1. Introducton Nowadays, t s wdespread to desgn and dsplay the products n three dmensons. Dsplayng them n 3D s the most effectve method as the means to show a whole or a part of the products that desgners and users want to see from varous drectons. In the feld of manufacturng and sellng eyeglass frames, however, 2D CAD systems are stll wdely used [e.g. 1, 2] and they are almost dsplayed wth 2D mages [e.g. 3, 4, 5], such as photos. The man reason s that the prce of eyeglass frames s comparatvely lower than other products and so usng the 3D CAD systems and dsplayng systems does not counterbalance the above-mentoned effectveness. From ths reason, the 3D CAD and dsplayng systems have never been used n the feld of manufacturng and sellng eyeglass frames. We have developed a 3D fttng smulator of eyeglass frames [6] for ndvdual s faces usng our orgnal space deformaton method [7] whch we call GFFD (Generalzed Free Form Deformaton). By usng GFFD, we can generate 3D ndvdual s face models easly and rapdly. Moreover, snce the nput devce we use s an only dgtal camera, the cost of the smulator s very low. Therefore, t s expected that the smulator wll be put to practcal use n the near future. However, one problem to overcome has remaned. Although we got to generate 3D ndvdual s face models easly and rapdly wth low cost, t takes much tme and cost to generate a 3D eyeglass frame model. The reason s that there s no tool of modelng varous and many eyeglass frames easly and rapdly wth low cost. In ths paper, we wll menton a method to apply the GFFD method to eyeglass frames. To apply GFFD to a certan set of obects, we need a generc model whch has characterstc features common to a whole set of the obects. Fortunately, smlarly to an ndvdual s face, most of eyeglass frames have topologcally common structure and shape. So we wll be able to apply GFFD to eyeglass frames. Frst, we wll descrbe about GFFD n detal n the next chapter. 2. GFFD(Generalzed Free Form Deformaton) In ths study, we apply the technque to generate an obectve face model by transformng a generc face model prepared beforehand. We use GFFD, whch s the FFD by the arbtrary functon, for the transformaton of the generc geometry model. GFFD has a characterstc feature whch allows to control the transformaton by changng the base functon. If we use conventonal FFD methods, we have to put control ponts on the vertces of grds, but then we cannot antcpate the shape of deformed obect beforehand. On the contrary, f we use our GFFD method, we can get such a mert that the shape can be qute antcpated, because the operaton ponts after moved do not fal to be on the shape. Snce the postons of all ponts except the operaton ponts are automatcally decded by nterpolatng methods based on the base functon, t s mportant to use the most sutable base functon to rase 114
NICOGRAPH Internatonal 2012, pp. 114-119 the accuracy level around the operaton ponts. Let the pont p(p x, p y, p z ) denote an arbtrary pont n the space, The transformed pont q by GFFD s represented as follows: q, (1) where V s a coordnate of a control pont and G s the base functon. If control ponts do not exst on the obect, ts transformaton by the movements of control ponts s not ntutve. So, we consder that the movements of the ponts (operaton ponts) that exst on the obect transform the space. After the transformaton, the obect must passes through the operaton ponts after moved. For ths requrement, usng operaton pont P whose number s the same as the number of control pont, t s necessary to satsfy the followng expresson: q. (2) Here, although G s the functon of the dfference between operaton ponts and control ponts, control ponts are not obtaned. Therefore, defnng G to be a functon of the dfference between operaton ponts, we obtan as follows: q n 1 n 1 n 1 V G V G V G. (3) We obtan V by solvng (3). Substtutng V nto (1) gves the coordnates of arbtrary ponts after moved. p V p V p p We call G the base functon of GFFD and use Eucld-norm for the functon when we deform a model of eyeglass frames. The functon s as follows: G 2 2 2 ( p ) p V ( x x ) ( y y ) ( z z ). (4) Ths base functon s susceptble to faraway control ponts. Thus, n case of usng many control ponts, we can transform the obect smoothly, keepng orgnal shape [8]. 3. Eyeglass frames modelng wth GFFD We descrbe the method of eyeglass frames modelng wth GFFD. We decde the generc model of eyeglass frames (secton 3.1) and locate the feature ponts (secton 3.2) on t n advance. Next, we take photos of eyeglass frames from sx drectons (front, back, rght sde, left sde, top and bottom). Then, we specfy the feature ponts of the frames n each photo (secton 3.3) and obtan the correspondng 3D postons of the feature ponts usng a stereo computng method. A 3D poston s decded from two photos selected from among the sx photos. Fnally, by usng the feature ponts as the operaton ponts at GFFD converson, we deform the 3D generc model nto the model that correspond to the eyeglass frames n the photos wth GFFD based on the obtaned 3D postons of the feature ponts. Fg. 1 shows the outlne of whole procedures. The sze of the deformed model s correspondng to that of the real obect. The how to decde the sze s mentoned n secton 3.4. Feature ponts Generc model of eyeglass frames Calculate control vectors Transform (GFFD) Photos taken from the sx drectons Obtan the 3D postons Feature ponts 3D model of eyeglass frames Fg. 1 Outlne of whole procedures 115
NICOGRAPH Internatonal 2012, pp. 114-119 3.1 Generc model of eyeglass frames We descrbe a generc model of eyeglass frames. Eyeglass frames are classfed nto three types such as full rm, half rm and rmless n terms of the form of frames rms, whch are parts holdng lenses (Fg. 2). Among them, most popular frames are full rm frames. They vary n ther structure n terms of whether ther materal s metal or plastc. Plastc frames have the structure that s smpler than metal frames. Metal frames consst of the parts called a brdge, rms, nose pads, pad arms (clngs), endpeces, temples and tps [9]. Meanwhle, plastc frames generally consst of only two parts that one s unfed wth a brdge, rms, nose pads, pad arms and endpeces and the other s unfed wth temples and tps. In addton, plastc frames nvolve thck parts compared wth metal frames owng to strength of the materal. These make t easer to specfy the feature ponts of plastc frames than metal frames. Therefore, we start to target plastc frames as a model of eyeglass frames. Next step, we decde the shape of the plastc frames as a generc model. We classfed the shape nto three portons: rm, endpece and temple. In eyeglasses ndustry, the rm s shape s named such as round, oval, square, boston, wellngton, fox, etc [10, 11]. Among them, the shape that s seen most frequently n optcal shops s the type of oval and square. So we decded the rm s shape of the generc model to be the mddle shape between oval and square. The endpece s shape of plastc frames s roughly classfed nto two types. One s the proectng type from a rm such as Fg. 3(a). The other s the embedded type nto a rm such as Fg. 3(b). If the generc model s the embedded type, t s dffcult to produce the shape of the proectng endpece. So we decded the endpece s shape of the generc model to be the proectng type. The temple s shape of plastc frames has varous types such as Fg. 8(c), (e). However, the standard shape s a straght type such as Fg. 8(a). So we decded the temple s shape of the generc model to be the straght type. As a result, comparng several above-mentoned types of real plastc frames, we made a generc model usng modelng software. It s the frames shown n Fg. 2(b). polygons. The geometrc model conssts of trangle P 7 P 8 P 1 P 2 P 3 P 4 P 6 (a) Front vew (b) Back vew (c) Rght sde vew (d) Left sde vew (e) Top vew P 9 P 5 P 10 Tp Endpece: a part connectng a rm and a temple Brdge Temple Rm (a) Full rm (metal materal) (b) Full rm (plastc materal) (c) Half rm (d) Rmless (f) Bottom vew Fg. 2 Eyeglass frames classfed n terms of ther rm type Fg. 4 Frames generc model and ther feature ponts 116
NICOGRAPH Internatonal 2012, pp. 114-119 3.2 Locatng of feature ponts on the generc model We locate the feature ponts on the generc model of eyeglass frames. The feature ponts are determned at the postons where ts shape change greatly, ts desgn s mportant and accuracy s requred. In ths study, we used 152 feature ponts. Fg. 4 shows the feature ponts located. When the frame model s used for smulaton of wearng eyeglass frames, the ponts where the frame model hangs on the nose and ears of a face model play an mportant role. In Fg. 4(a), the reason the lower rm s located more the feature ponts than the upper rm s that lower rms tend to be desgned longer than upper rms (e.g. Fg. 8 (b)). 3.3 Specfyng of feature ponts on the frames n photos We decde the poston of the feature ponts on eyeglass frames n photos. (a) Proectng endpece In our current mplementaton, we manually specfy the feature ponts on eyeglass frames n each photo by clckng and movng the ponts wth a mouse. We are workng on technques for automatc extracton of these ponts. Fg. 5 shows an example of specfyng the feature ponts. Frst a ntal arrangement of the feature ponts s put on the photo, and then each pont s arranged n the outlne of the eyeglass frames. At ths procedure, the feature pont needs to be arranged on the poston where the outlne changes remarkably. If segments of the outlne change smoothly, the feature ponts n the segments are arranged at equal ntervals along the segments. (b) Embedded endpece Fg. 3 Endpece type of plastc frames 3.4 Makng the sze of the model correspond to that of the real frames We wll menton the way to make the sze of the deformed model correspond to that of the real obect. Most of eyeglass frames have some sze nformaton prnted or engraved on the nsde of the frames. Fg. 6 shows a sample of the nformaton. The value 54 means eye sze. Eye sze s the horzontal wdth n mllmeters of one of the frame s lenses. The value 17 means brdge sze. Brdge sze s the dstance n mllmeters between the two lenses. It s measured between the two closest ponts of the two lenses. The value 140 means temple sze. Temple sze s measured along the length of the temple, from one end to the other, ncludng the bend. Fg. 7 shows what these szes mean [12]. In the generc model, eye sze, brdge sze and temple sze correspond to the dstance between the feature P 1 and P 2 (P 3 and P 4 ), the dstance between the feature P 2 and P 3, and the total of the dstance between the feature P 5 and P 6 (P 8 and P 9 ) and the dstance between the feature P 6 and P 7 (P 9 and P 10 ), respectvely. If we use the frame models n the smulaton that makes a face model wth accurate sze wear them accurately, we use the actual values that are measured. Usng these szes, we enlarge or reduce the shapes of eyeglass frames to the sze of real obect when the generc model s deformed. Intal arrangement of the feature ponts Fg. 6 A sample of the sze nformaton prnted Temple sze Brdge sze Eye sze Fg. 7 Three knds of szes Fg. 5 An example of specfyng the feature ponts 117
4. Results We generated several 3D models of eyeglass frames wth the proposed method. Fg. 8 shows the frames photos, the results of ths method and the results of our fttng smulaton for the eyeglass frames. NICOGRAPH Internatonal 2012, pp. 114-119 The comparsons of the generated models and the photos ndcated that the generc model s deformed nto ndvdual models that correspond to the eyeglass frames n the photos. In ths experment, the average of the operatng tme was about 25 mnutes. Ths s rapd compared wth modelng software. We compared the szes of the generated models wth those of the real frames. The szes compared are the length A, B, C, D and E n Fg. 9. We got the lengths of the real frames by measurng them wth calpers. To enlarge or reduce the frame models generated wth GFFD, we used the length A, B, and E measured for eye sze, brdge sze and temple sze, respectvely. Table.1 shows the actual lengths of real frames and the lengths obtaned from models. The average errors between real and model are wthn 1.0 mm, whch s the acceptable error range for shapes of eyeglass frames. 5. Concluson In ths paper, we proposed a novel method of generatng a 3D model of eyeglass frames from multple camera mage data based on the GFFD deformaton method. The outlne of the whole procedures s as follows: (1) Take photos of eyeglass frames from front, back, rght sde, left sde, top and bottom. (2) Specfy the feature ponts of the frames n each photo. (3) Obtan the correspondng 3D postons of the feature ponts usng a stereo computng method. (4) Deform the 3D generc model of eyeglass frames nto the model that correspond to the eyeglass frames n the photos wth GFFD based on the obtaned 3D postons of the feature ponts. We showed some models obtaned usng ths method. The results ndcated that the shapes of the models are almost same as the frames n photos. Usng ths method, we wll try to develop a catalog system of 3D eyeglass frames that has smulaton of wearng eyeglass frames for ndvdual s faces. References [1] Glass-talor, http://www.glass-talor.com/procces/procces.html [2] Takeuch Optcal, http://takeuch-opt.co.p/technology/process/ [3] FramesDrect, http://www.framesdrect.com/ [4] FramesRx, http://www.framesrx.com/ [5] Optcal Drect, http://www.optcaldrect.com.au/ [6] N. Tamura and K. Ktama, 3D Fttng Smulaton of Eyeglass Frames for Indvdual s Faces Usng Inverse Knematcs, Proceedng I of Asan Conference on Desgn and Dgtal Engneerng 2011, pp 227-232, 2011. [7] K. Ktama, Y. Akag, A. Yamauch, N. Okazawa and Y. Hguch, A Study on Facal Modelng Based on the GFFD Method, Journal of the Japan Socety of Precson Engneerng, Vol. 74, No. 8, pp. 883-890, 2008. (n Japanese) [8] N. Yoshda, K. Kanou and K. Ktama, Free-Form Deformatons Based on Gaussan Functons: Fundamental Theory for Interactve Modelng, Journal of the Japan Socety of Precson Engneerng, Vol. 65, No. 7, pp. 971-975, 1999. (n Japanese) [9] Sabaemeganefactory, http://www.cty.sabae.fuku.p/users/monodukur/sabaemegane/pont/01.html [10] AJOC, http://www.aoc.or.p/vc_avenue/frame/ [11] MEGANE PARK Web, http://www.meganepark.bz/kso/kso-shape.htm [12] Om Megane, http://www.om.gr.p/opt/frame/hyou.html [13] H. Yanagsawa and S. Fukuda, Development of Interactve Industral Desgn Support System Consderng Customer s Evaluaton: Shape Desgn of Eyeglass Frame, JSME nternatonal ournal. Seres C, Vol. 47, No. 2, pp 762-769, 2004. D B Fg. 9 Four lengths for comparng Table.1 Actual lengths of real frames and obtaned lengths from models (unt: mm) E A C Length Frame model A Frame model B Frame model C Frame model D Frame model E Average error Actual Model Actual Model Actual Model Actual Model Actual Model A 46.3 46.6 54.1 53.9 52.4 53.0 48.5 48.6 52.7 52.3 0.32 B 15.1 14.5 15.7 16.0 17.2 16.5 17.4 16.8 15.7 14.8 0.63 C 32.8 32.5 45.3 45.7 35.5 36.2 27.5 27.0 30.7 31.2 0.46 D 126.3 127.2 134.8 134.3 134.8 135.2 129.4 129.1 132.4 129.7 0.97 E 138.9 138.6 144.8 144.8 144.4 145.0 136.6 136.0 134.5 134.0 0.39 118
NICOGRAPH Internatonal 2012, pp. 114-119 (a) Eyeglass frame model A Fttng Smulaton of eyeglass frames (b) Eyeglass frame model B Fttng Smulaton of eyeglass frames (c) Eyeglass frame model C Fttng Smulaton of eyeglass frames (d) Eyeglass frame model D Fttng Smulaton of eyeglass frames (e) Eyeglass frame model E Fg. 8 Results of smulaton Fttng Smulaton of eyeglass frames 119