Human Body Shape Modeling Using 3D Range Data for Healthcare Applications and Beauty Services

Transcription

1 Human Body Shape Modeling Using 3D Range Data for Healthcare Applications and Beauty Services Igor Goncharenko 3D Incorporated Yokohama, Japan Abstract The paper describes a system to model variations of body shapes by individual 3D scan data and to provide the modeling results to the users via the Internet. The clients of the system are involved into unloading dieting programs and exercises in sport clubs. After scanning and data processing, they can control modeling of their prognostic body shapes in case of weight gain/loss. The client interface allows the users to browse the processed data in a 3D GUI, to simulate body shapes with new desired body measurements (such as weight, waist, chest circumferences, etc.), and to generate printed reports. Modeling can be controlled by a few body dimensions, or even by one measurement. Keywords: 3D Body Scanners, Digital Human Modeling. 1. INTRODUCTION Recent advances in the development of high-resolution and accurate 3D scanners for whole-body shape capturing make it attractive to utilize the scan data in many application fields, e.g., in individual design of car interiors, clothing, anthropometry, healthcare, social studies, realistic 3D media content creation, aerospace, defense, and national surveys. Formerly body dimensions were widely used for apparel design, ergonomic assessment of working/operational spaces, and reverse engineering. The data are, for instance, weight, height, leg length, waist, hip or chest circumferences, etc. Body dimensions are defined as the distances between some specific points on human body surface (landmarks), or circumferences measured at the level of the landmarks. The landmarks are anatomically defined features, for example, top of the head, tip of the spinous process of the 7 th cervical vertebra, the most lateral point of acromion, etc [8]. With modern 3D scanning technologies, it becomes possible to automatically locate the landmarks and to retrieve body dimensions from accurately scanned range data. And, the main advantage of 3D scanning is that it provides a highly detailed individual s body surface shape, which can be used in further human-oriented modeling. Human body shape modeling (HBSM) utilizing 3D scan data is a rapidly evolving multidisciplinary area [7], applicable also to biomedical applications [11]. The beauty services and healthcare applications include: anthropometric surveys; body deformity, asymmetry and obesity control; rehabilitative and training monitoring; weight control ([6],[11]). Now, even some visual cognition factors can quantitatively be defined with the aid of HBSM, for instance, woman body attractiveness [10], or adolescent body perception [1]. Various methods and tools are used in HBSM depending on application fields and input 3D scan data types, however, controls of HBSM are still traditional body dimensions mentioned above, because ergonomics designers (or, HBSM end users) clearly understand the nature of the measurements and can easily manipulate with them during modeling. It is also important to provide the end users of HBSM systems the ability to control the modeling process by a limited number of variables to avoid ambiguity and complexity of control, especially, if modeling is done though the Internet in interactive way. Moreover, in some cases a complete set of body dimensions is not provided with 3D scan data. Recently, Internet-based applications using 3D scan data appeared mostly for apparel design. In [4] several virtual dressing systems are described (MyShape, Virtual try-on, Lands End, My Virtual Model). These systems use static, or pre-defined 3D models from databases, without on-line body shape modeling. Israeli company OptiTex presented a Web-based garment fitting system with 3D preview of a synthetic avatar created by user s body dimensions retrieved from individual 3D scan data. It would be worthwhile to implement Web-based systems allowing the endusers to manipulate with their personal 3D scan data, to model, for instance, the body shape after weight loss/gain, after physical or rehabilitative training, and to display the virtually corrected postures of the models. The users of such systems would be motivated not only in taking care on their health and physical exercises, but also in participating in 3D scanning process itself and contributing into 3D scan databases. The purpose of this work is to develop a Web-based system to realistically model variations of prognostic body shapes by individual 3D scan measurements and to provide the results of modeling to the users via the Internet through a 3D graphical interface. The paper is organized as follows. First, in Section 2, input data processing methods and human body shape modeling techniques, which are used in the system, are described. In Section 3, we present some statistical results allowing us to reduce the number of body dimensions to control body shape modeling. Section 4 describes the Web system s implementation details such as server-side functionality, client interface with 3D graphics, etc. Finally, conclusions are summarized in Section MODELING METHOD 2.1 Data Nowadays, several active and passive optical 3D whole-body scanning technologies are used, for example, optical laser scanning, photogrammetry based on structured light, modulated light, passive stereoscopy, etc. The success of CAESAR Project (see, e.g., [2]) focused attention of professionals on the industrialtype equipment providing harmless scanning with high quality output data (with about million of 3D points). Such 3D scanning systems are produced, for instance, by Cyberware, Hamamatsu Photonics, Human Solutions/Vitronic, [TC] 2, and other manufacturers. The trend of the 3D body scanners development Russia, Vladivostok, September 16-20,

2 The 23rd International Conference on Computer Graphics and Vision is characterized by decreasing the cost of hardware and time of scanning (to few of seconds), easiness of installation and operation, truly 360 degree scanning with low noise and data missing, nearly millimeter range accuracy. It results in satisfactory quality of the produced data with high level of details of face and body parts of scanned subjects, ability to automatically extract dozens of body dimensions and landmarks. We mostly used data collected with the Bodyline Scanner from Hamamatsu Photonics [5]. The standard manufacturer s preprocessing software, Bodyline Manager, allows 54/55 (male/female) auto-landmarks, and about one hundred body dimensions for one data range scan. Original data are recorded as a polygonal model in Wavefront graphical format (OBJ) with several hundred thousand vertices, accompanied by personal information (age, weight, height, BMI, gender, etc). Even Bodyline Manager has an interactive functionality to correct landmark position by a 3D editor, our processing software (see Section 2.2) was developed with the assumption that location of some landmarks may not be correct. We also considered the case, when landmarks and body measurements are not given at all. The last case was tested with the range data collected by other body scanner [3]. Several thousand subjects participated in 3D body scanning campaigns managed by Nihon Unisys. The data were collected with the system 3D-Navi from Japanese males and females aged from 15 to 96 years old. On the server side of the system, 3D scan data and measurements were sorted into ten gender-age groups: younger than 30 years old, from 30 to 40, between 40 and 50, from 50 to 60, and older than 60 years old for both, male and female subjects. This grouping reduces intra-population variations, because it is known, for instance, that on the average the younger generation is taller, older generation tends to lose muscularity, etc. For operational use, statistical analyses were conducted separately for the sex/age groups, and overall inter-population statistics were investigated experimentally D model creation The data collected with 3D scanners are characterized by significant variations of body shapes, postures, number of vertices, presence of holes and noise in the data. Nowadays, the most proven approach for HBSM is based on template models and reconstruction of body surfaces from eigen-spaces. Pioneered in [2], the method uses Principal Component Analysis (PCA), which captures the body shape variations from an average, or template shape. In the method (utilizing landmarks) all the vertices of a template model are non-rigidly morphed until the best fitting to the point cloud of the input 3D scan data. The resulting fitted model is called homologous. Therefore, there is exact correspondence of vertices for any pair of homologous models, or a homologous model and template. This property makes it possible to apply PCA to model shape variations. The approach also resolves hole (missed data) interpolation, and it has variants for landmark-less fitting. If the template is accompanied by a skeleton model with body mesh skinning weights, the bone-skin technique (widely used in computer graphics animation) can be applied for fitting if the postures of template and scan data are significantly different. We implemented an HBSM method based on male (13379 vertices) and female (16780 vertices) Dhaiba templates [9] in the form of closed polyhedra instrumentally connected with the link-joint skeleton models and landmark sets (Figure 1). Figure 1: Dhaiba template models. The process of model creation is as follows. First, the homologous model is created iteratively for the posture of original 3D scan data, and the joint centers and link lengths are estimated during the iterations. Finally, the subject-specific skeleton is completely defined. To avoid posture variations during modeling, the homologous model is created for the standard template posture, and the main body part is extracted. The overall process is depicted in Figure 2. Figure 2: Original scan data (A) and homologous models in original (B) and standard (C) postures. 2.3 Body shape modeling For Web-based modeling providing the customer his own prognostic body shape view in 3D, the following scenarios were considered: visualize the user prognostic body shape in case of weight loss/gain; visualize the shape in case of increase/decrease of the waist circumference; visualize the body model in correct (standard) posture without body asymmetry and deformations (which can appear, for instance, due to early stage of scoliosis); estimate the required weight loss for the desired waist decrease. In all the above scenarios, it is quite natural to vary weight and/or body dimensions (except height and other measurements, whose variations lead to significant bone deformations). We suppose that these variations mostly result in changes of the main body part shape (Figure 3D). It was also supposed that head, face, hands, and feet variations are small in case of weight loss. The main body part surface is composed from a smaller number of vertices 44 GraphiCon'2013

3 (N=2618 in the case of our male homologous model), and statistical calculations can be done much faster for it. Similarly to [2], for each statistical group (collection of K main body homologous models, where each body shape is composed from N vertices), we describe the body shape of the k-th individual (k=1,,k) by 3N-dimensional vector a ( x, y, z, x, y, z,..., x, y, z ) T. k 1k 1k 1k 2k 2k 2k Nk Nk Nk Calculating average shape a 0 and applying PCA, we can find the eigenvector decomposition (basis of vectors e i, i=1,..,k-1) for variation of any shape a k from the collection: Each principal component (PC) cannot clearly be associated with a single anthropometric body measurement such as height, weight, or waist circumference, it is rather influenced by combination of the measurements and variations of postures. Each body shape a k has L supplementary measurements {d k1, d k2,, d kl }, derived from the range data and measured with scale. Supposing that scores of the i-th PC (i=1,..,k-1) depend on L (L<K) multiple measurements {d 1, d 2,, d L }, we may propose control of body shape modeling through them for the case of simple linear dependency: L i 1 2 L i0 ij j j 1 PC ( d, d,..., d ) b b d Taken into account that PC scores can directly be reconstructed from the homologous model, we can compare them with ones obtained from the above linear model to clarify how accurate the assumption of linearity is. Figure 4 shows the comparison for PC 1 for one of the scanned inter-population female group for 533 subjects aged from 16 to 96 years old. (In the Figure, gray squares depict correlation without skeleton measurements, and black triangles show correlation with skeleton measurements.) On the average, for this population the results of linear modeling are very good for lower PCs. Particularly, correlation is for PC 1 if skeleton measurements are taken into account (Figure 4). Figure 3: Correlation of measured and modeled PC 1. For the case of individual body simulation with available homologous model a k and measurement vector d k = (d k1, d k2,, d kl ) T, it is preferable to use increments of the measurements. Then, denoting new measurement set as d = d k + d, where d = ( d 1, d 2,, d L ) T is the measurement increment, we can use our regression results to simulate new shape in terms of the increments: K 1 a ( d) a ( PC ( d) PC ( d )) e. sim k i i k i i 1 For high order eigenvalues with low significance, variation magnitudes are significantly decreased. Limiting the value K, the last formulae can be used for fast modeling in real-time on the client sites of Web-based systems. 2.4 Reduction of the number of controls As the developed body shape modeling method was supposed to be implemented for interactive control through the Web, it is reasonable to make such control more user-friendly and simple. If the number L of controlling measurements is large enough (e.g., basic software for our 3D scanner can provide up to 72 body dimensions) and the user is allowed to vary them independently, the resulting simulated body shapes would be very unrealistic and the control process itself would be confusing and difficult for the users. Therefore, more simplified and robust method of modeling was proposed. Once the developed modeling system is mostly intended for personal body weight control, it was decided to select 16 measurements: weight, height, BMI, and 3D scanner s measurements obviously dependent from weight gain/loss.. The body dimensions are mostly the circumferences of body parts (neck, torso, arms, and legs) measured in horizontal cross sections. Additional 28 skeleton measurements (bone lengths and distances between joints) calculated during building the homologous model were also used. The skeleton measurements, when being kept constant in individual shape modeling, result in constraining effect and prevent bone deformations, and as a consequence, the modeled shapes look much more realistic, and the regression errors (see, e.g., Figure 4) become smaller. The final composition of body shapes from PCs calculated with the basic measurement set and the complete set appeared very similar for the task of weight variations. Being included in the model, skeleton data are not changed in modeling. After analysis of measurements it was found that correlations of symmetric pairs of arm and leg circumferences are high. Also, BMI measurement is redundant, because it can explicitly be calculated through weight and height. Height should not be varied by the users, when they model their body shapes. Based on the above speculations, a reduced set of control measurements was found. However, change in one measurement requires adjustment of all others simultaneously for realistic body shape view. It was found that horizontal body circumferences are highly correlated with weight. Based on this fact, we can approximately model the changes of the measurements by weight change in accordance with the following scheme. Suppose, all the measurements depend on weight w linearly: di( w) di ci ( w d2), i 1..16, where d 2 is weight measured on the day of 3D body scanning together with all other measurements d i. Thus, it is possible to predict the change of waist circumference by the change of weight, and vice versa. Coefficients c i are defined by linear regression through all the data for each statistical group. The standard deviations estimated from the regression are small enough to reconstruct all the measurements even at high gain or loss of weight. Russia, Vladivostok, September 16-20,

4 The 23rd International Conference on Computer Graphics and Vision 3. WEB-BASED IMPLEMENTATION The methods described in the above Section were implemented in our Web-based system 3D-Navi managed by Nihon Unisys Ltd. Also, the methods were realized in a standalone application, but there are many disadvantages of this approach, because the collected 3D scan data are commercially sensitive and should not arbitrary be accessed or updated by the users. Also, there are serious ethical reasons (e.g., the requirement of non-disclose of almost undressed personal body shapes) restricting a stand-alone implementation. Therefore, it was decided to implement centralized control of data access from Web clients with a secure authorization mechanism. In order to motivate people in participating in 3D scanning, the main scenario of the usage of the 3D-Navi Web-based system was supposed to provide the subjects with ability to observe their current personal body shapes, model body shapes in cases of weight loss or gain, visualize their virtually regal carriage, and, therefore, to stimulate them in physical training, exercises, and dieting. To realize the above scenario, 3D-Navi was designed to provide user-friendly and secure access to personal data, to quickly react on the users requests from the client side, to intuitively convenient 3D browse the whole body shape and selected body parts, to generate reports with 3D views of current and modeled body shapes and body dimensions in textual form. Real-time rendering of 3D contents on the client side (not only homologous models, but also original 3D scans with up to one million vertices) implies high requirements on the client system performance. Also, on-line modeling requires the efficiency of server-side calculations and network transfer. In case of gradual increase of collected 3D scans in the database, the data required for one body shape modeling, namely, eigenvectors and PCs, can be much more bulky than the size of file with modeled 3D body shape. Thus, it was decided to implement modeling as an optimized server-side application without cutting of higher order eigenvectors. PCA analysis is periodically updated on the server by administrator s request. Modeled body shapes are transferred to the client side in a compressed form of the Wavefront graphical format. To support the variety of browsers and platforms, the client side sub-system was implemented as an HTML form with JavaScript control calling Java applet for 3D visualization in a panel inside the main frame of the client window. 3D graphics and navigation in 3D is implemented in Java3D on top of OpenGL, which provide real-time rendering of modeled shapes. The applet also provides a variety of service functions, such as model loading, high quality off-screen image rendering for modeling session report generation, modeling measurement settings, etc. As described in the above Section, modeling can be done in several modes: with measurement masking, independent, and setting by one measurement. In the 3D window the user can visualize up to two models (for example, original homologous and prognostic with over-weight of 20kg is shown in Figure 4), rotate, scale, align them, or start autorotation mode. Different body parts can be colored with specially assigned colors for displaying the body part surface difference. The important feature of the system is overlapping the shapes in a semi-transparent mode, which displays the difference of the overall shapes or body parts. 4. CONCLUSION Figure 4: 3D view of client GUI. Web-based 3D-Navi system s design and implementation are described in this paper. The system is intended for remote access to individual 3D scan data and personalized body shape modeling by interactive changes of limited number of body dimensions (or, even one such measurement). Typical scenarios are, for instance, visualization of the desired user s body shape after weight loss, or prognostic shape after wished waist decrease. The system is supposed to motivate the people in physical exercises and follow to unloading diets. The system is supported by real-time graphical rendering and browsing of the body shapes and/or body parts in a 3D window on the Web. Simple, intuitive and user-friendly 3D navigation allows easy comparison of current and prognostic body (or, body part) shapes. Also, the system allows to generate printed reports of the results of scanning, measurements, and body shape modeling. For the Web-based system, special rapid methods of 3D range data preprocessing, canonical model creation with corrected posture, and statistical analysis were implemented. The methods are based on templates (body meshes with associated skeleton) with high level of details for face, hands, and feet. The individual features of face are well transferred to the modeled shapes. To simplify modeling by user control through a limited set of anthropometric data, an original method of measurement settings was proposed and implemented in the system. The system was used commercially in Japan for a beauty service company. In future works, it would be also worthwhile to collect statistical data separately for persons actively participating in body building in sports clubs, because eigen-spaces for muscular people can be different for subjects with overweight. 5. REFERENCES [1] Aleong, R. and Duchesne, S., Assessment of adolescent body perception: Development and characterization of a novel tool for morphing images of adolescent bodies, Behavior Research Methods, Vol. 39 No.3, 2007, pp [2] Allen, B., Curless, B. and Popovic, Z., The space of human body shapes: reconstruction and parameterization from range 46 GraphiCon'2013

5 scans, ACM Transactions on Graphics (ACM SIGGRAPH 2003), Vol.22 No.3, 2003, pp [3] Cartesia: Body Scanner [online]. Available at: (Accessed 15 June 2013). [4] Duffy, V. (Ed.), Digital Human Modeling: Proc. ICDHM Lecture Notes in Computer Science, Vol. 5620, Springer Publ., 2009, ISBN , 767p. [5] Horiguchi, C., BL (Body Line) scanner: the development of a new 3D measurement and reconstruction system, International Archives of Photogrammetry and Remote Sensing, Vol.32, B5, 1998, pp [6] Kouchi, M. and Mochimaru, M., Simulation of the body shape after weight change for health-care services in AHFE 2010: Proceedings of the 3rd International Conference on Applied Human Factors and Ergonomics. Miami, Florida, USA, 2010, pp [7] Lerch, T., MacGillivray, M. and Domina, T., 3D laser scanning: a model of multidisciplinary research, Journal of Textile and Apparel, Technology and Management, Vol. 5 No. 4, 2007, pp [8] Martin, R. and Knussmann, R. Anthropologie, Band I, Gustav Fischer Verlag Publ., Stuttgart, Germany, 1988, ISBN [9] Mochimaru, M. Dhaiba: functional human models to represent variation of shape, motion and sub subjective assessment, SAE Technical Paper, [10] Smith, K., et al., An analysis of body shape attractiveness based on image statistics: Evidence for a dissociation between expressions of preference and shape discrimination, Visual Cognition, Vol. 15 No.8, 2007, pp [11] Treleaven, P. and Wells, J., 3D body scanning and healthcare applications, Computer, Vol. 40 No.7, 2007, pp About the author Igor Goncharenko is a Chief Researcher at 3D Incorporated, Yokohama, Japan. Russia, Vladivostok, September 16-20,