SAE 06DHM 57 Retreval and Clusterng from a 3D Human Database based on Body and Head Shape Afzal Godl, Sandy Ressler Natonal Insttute of Standards and Technology ABSTRACT In ths paper, we descrbe a framework for smlarty based retreval and clusterng from a 3D human database. Our technque s based on both body and head shape representaton and the retreval s based on smlarty of both of them. The 3D human database used n our study s the CAESAR anthropometrc database whch contans approxmately 5000 bodes. We have developed a web based nterface for specfyng the queres to nteract wth the retreval system. Our approach performs the smlarty based retreval n a reasonable amount of tme and s a practcal approach. INTRODUCTION Wth the wde avalablty of 3D scannng technologes, 3D geometry s becomng an mportant type of meda. Large numbers of 3D models are created every day and many are stored n publcly avalable databases. Understandng the 3D shape and structure of these models s essental to many scentfc actvtes. These 3D scentfc databases requre methods for storage, ndexng, searchng, clusterng, retreval and recognton of the content under study. Searchng a database for 3D obects whch are smlar to a gven 3D obect s an mportant and challengng problem. Ths doman of research s also called query by example (QBE). We have developed technques for searchng a human database and have used the CAESAR anthropometrc database whch conssts of a database of approxmately 5000 human subects. In our study we have mplemented methods for smlarty based retreval from the CAESAR human database based on both human body and head shape. Prevous work on human body retreval based on body shape was performed by [Paquet and Roux 2003]. They performed content based anthropometrc data mnng of three dmensonal scanned human by representng them wth compact support feature vectors. They showed a vrtual envronment to perform vsual data mnng on the clusters and to characterze the populaton by defnng archetypes. [Paquet 2004] ntroduced cluster analyss as a method to explore 3D body scans together wth the relatonal anthropometrc data as contaned n the CAESAR anthropometrc database. [Azouz 2002, 2004] analyzed human shape varablty usng a volumetrc representaton of 3D human bodes and appled a prncpal components analyss (PCA) to the volumetrc data to extract domnant components of shape varablty for a target populaton. Through vsualzaton, they also showed the man modes of human shape varaton. The work of [Allen 2004] demonstrated a system of syntheszng 3D human body shapes, accordng to user specfed parameters; they used 250 CAESAR body scans for tranng. Retreval based on head shape was performed by [Ip and Wong 2002]. Ther smlarty measure was based on Extended Gaussan Images of the polygon normal. They also compared t to an Egenhead approach. The 3D scans of human bodes n the CAESAR human database contan over two hundred ffty thousand grd ponts. To be used effectvely for ndexng, searchng, clusterng and retreval, ths human body data requres a compact representaton. We have developed two such representatons based on human body shape: 1) a descrptor vector d, based on lengths mostly between sngle large bones. Thus, we form a 3D body descrpton vector of ffteen dstances, d, wth wrst to elbow, elbow to shoulder, hp to knee etc.; and 2) three slhouettes of the human body are created by renderng the human body from the front, sde and top. These slhouettes are then encoded as Fourer descrptors of features for later smlarty based retreval. These two methods are explaned n more detals n the Body Shape Descrptor secton. We also have developed two compact representaton based on human head shape: 1) applyng Prncpal Component Analyss (PCA) on the 3D facal surface and creatng PCA based facal descrptors; and 2) n the second method the 3D trangular grd of the head s transformed to a sphercal coordnate system by a least squares approach and expanded n a bass of sphercal harmoncs. More explanaton of these two
representatons of human head shape follow n the Head shape Descrptor secton. We also have used these four descrptors for clusterng of human bodes based on each descrptor. The four descrptors allow the selecton of the best descrptor for the applcaton, such as the use of a head descrptor for Helmet Desgn. CAESAR database The CAESAR (Cvlan Amercan and European Surface Anthropometry Resource) proect has collected 3D Scans, seventy three Anthropometry Landmarks, and Tradtonal Measurements data for each of the 5000 subects. The obectve of ths study was to represent, n three dmensons, the anthropometrc varablty of the cvlan populatons of Europe and North Amerca and t was the frst successful anthropometrc survey to use 3 D scannng technology. The CAESAR proect employs both 3 D scannng and tradtonal tools for body measurements for people ages 18 65. A typcal CAESAR body s shown n Fgure 1. Fgure 2. A Caesar body wth landmark numbers and postons BODY SHAPE DESCRIPTOR We now descrbe two methods for creatng descrptors based on human body shape: Dstance Based The frst method uses a descrptor vector d based on lengths mostly between sngle large bones. For descrptor vector purposes, we requre lengths only between landmark ponts where ther separaton dstance s somewhat pose ndependent. The reason t s not completely pose nvarant s that dstance are between landmark ponts whch are on the surface body compared to the dstance between the center of the ont axs. Ths apples to ponts connected by a sngle large bone as shown n Fgure 3. Thus, we form a descrptor vector of ffteen dstances, d, wth d1 wrst to elbow, d2, elbow to shoulder, d3 hp to knee etc. For whch the Eucldean dstance Fgure 1a. A CAESAR body n standng pose Fgure 1b. A CAESAR body n sttng pose The seventy three anthropometrc landmarks ponts were extracted from the scans as shown n Fgure 2. These landmark ponts are pre marked by pastng small stckers on the body and are automatcally extracted usng landmark software. There are around 250,000 ponts n each surface grd on a body and ponts are dstrbuted unformly. d = P P s somewhat nvarant across dfferent poses. Dstances such as chn knee are avoded. The dstance based descrptor s then used wth the L1 and L2 norm to create a smlarty matrx. The L1 dstance: d ( P, P ) = k = 1 P P The L2 dstance: d ( P, P ) = k = 1 P 2 P 2 More detals and shortcomngs about ths descrptor were descrbed n the paper [Godl 2003] To test how well the dstance based descrptor performs, we studed the dentfcaton rate of a subset of 200
subects of CESAR database where the gallery set contans the standng and the probe set contans the sttng pose of each subect. In ths dscusson, the gallery s the group of enrolled descrptor vector and the probe set refers to the group of unknown test descrptor vectors. The measure of dentfcaton performance s the rank order statstc called the Cumulatve Match Characterstc (CMC). The rank order statstcs ndcates the probablty that the gallery subect wll be among the top r matches to a probe mage of the same subect. Ths probablty depends upon both gallery sze and rank. The CMC at rank 1 for the study s 40%. Subect 0082 s rendered n three vews Each slhouette s then represented as R Front Sde Top R Angle d 6 d 5 d Body shape descrptor consst of of dstances b/w landmark pts d = {d 1,d 2,d 3, d 4 } Dstances: d 1 hp to knee d 2 knee to ankle d 3 wrst to elbow d 4 elbow to shoulder etc Fgure 3. A dstance based body shape descrptor Slhouette Fourer Based d 7 The second method of body shape descrptor that we propose s based on renderng the human body from the front, sde and top drectons and creatng three slhouettes of the human body as shown n Fgure 4. The theory s that 3D models are smlar f they also look smlar from dfferent vewng angles. The slhouette s then represented as R(radus) of the outer contour from the center of orgn of the area of the slhouettes. These three contours are then encoded as Fourer descrptors whch are used later as features for smlarty based retreval. The number of real part of Fourer modes used to descrbe each slhouette s sxteen (16); hence each human body s descrbed by a vector of length forty eght (48). Ths method s pose dependent, so only bodes of the same pose can be compared. The Fourer based descrptor s then used wth the L1 and L2 norm to create a smlarty matrx. d 1 d 2 d 4 d 3 Rgd Connectons (Bones) Dstances are some what Invarant to movement, poston, and pose Fgure 4. Subect 00082 s rendered n three slhouette vews HEAD SHAPE DESCRIPTOR We now descrbe two methods for creatng descrptors based on human head shape: PCA Based In ths method we neglected the effect of facal expresson. By cuttng part of the facal grd from the whole CAESAR body grd usng the landmark ponts 5 and 10 as shown n Fgure 5 and lsted n Table 1. Table 1 lst all the numbers and names of landmark ponts used n our 3D face recognton study. The new generated facal grd for some of the subects wth two dfferent vews s shown n Fgure 6. In the case of people standng the mnmum number of grd ponts s 2445 and the mean number s 5729. For the case of people sttng the mnmum number of grd ponts n the facal surface s 660 and the mean number s 4533. Ths shows that the grd s very coarse for some of the subects n the seated pose.
neghbor method when there are vods n the orgnal facal grd. For some of the subects there are large vods n the facal surface grds. Fgure 7 shows the facal surface and the new rectangular grd. Fgure 5. Landmark ponts 1, 2, 3, 4, 5 and 10. Vertcal and horzontal lnes are the cuttng plane Table 1. Numbers and names of landmark ponts used n our 3D face 1 Sellon 2 Rt Infraobtale 3 Lt Infraobtale 4 Supramenton 5 Rt.Tragon 6 Rt. Gonon 7 Lt. Tragon 8 Lt. Gonon 10 Rt. Clavcale 12 Lt.Clavcale Fgure 7. Shows the new facal rectangular grd for two subects We properly postoned and algned the facal surface and then nterpolated the surface nformaton on a regular rectangular grd whose sze s proportonal to the dstance between the landmark ponts. Next we perform Prncpal Component Analyss (PCA) on the 3D surface and smlarty based descrptors are created. In ths method the head descrptor s only based on the facal regon. The PCA recognton method s a nearest neghbor classfer operatng n the PCA subspace. The smlarty measure n our study s based on L1 dstance and Mahalanobs dstance. To test how well the PCA based descrptor performs, we studed the dentfcaton between 200 standng and sttng subects. The CMC at rank 1 for the study s 85%. More detals about ths descrptor are descrbed n the paper by [Godl 2004] Fgure 6. Facal surfaces after the cut from the CAESAR body n two dfferent vews. Next, we use four anthropometrc landmark ponts (L1, L2, L3, L4) as shown n Fgure 5, located on the facal surface, to properly poston and algn the face surface usng an teratve method. There s some error n algnment and poston because of error n measurements of the poston of these landmark ponts. Ths Max error was 15 mm, obtaned by takng the dfference of dstance between landmark ponts L1 L2 and between L3 L4 for subects standng compared to subects sttng. Then we nterpolate the facal surface nformaton and color map on a regular rectangular grd whose sze s proportonal to the dstance between the landmark ponts L2 and L3 ( d= L3 L2 ) and whose grd sze s 128 n both drectons. We use a cubc nterpolaton and handle mssng values wth the nearest Sphercal Harmoncs Based In the second method the 3D trangular grd of the head s transformed to a sphercal coordnate system by a least square approach and expanded n a sphercal harmonc bass as shown n Fgure 8. Snce the CAESAR head grd has large vods n the top of the head and also because of cuttng the grd at the neck there s crcular hole. Snce these holes are not flled properly, we have a convergence problem wth 10% of the head grds. The man advantage of the Sphercal Harmoncs Based head descrptor s that t s orentaton and poston ndependent. In the near future we plan to fx ths problem usng a method whch flls vods. The sphercal harmoncs based descrptor s then used wth the L1 and L2 norm to create smlarty measure. To test how well the Sphercal Harmoncs Based head descrptor performs, we studed the dentfcaton of the human head between 220 standng and sttng subects. The CMC at rank 1 for the study s 94%.
Fgure 8. 3D head grd s mapped nto a sphere RESULTS Retreval Results The web based nterface enables us to select a partcular body, or a random body or bodes, based on some crtera such as weght, age, heght, etc as shown Fgure 9. Subsequently, we can perform smlarty based retreval based on a sngle descrptor (out of the four descrptors). Usng four descrptors allows users to select the best descrptor for ther applcaton, such as the use of head descrptor for helmet or eyeglasses desgn. The partal results from a body shape based smlarty retreval for subect number 16270 are shown Fgure 10. Fgure 10. Smlarty based retreval for 16270 based on body shape The partal results from a head shape PCA based smlarty retreval for subect number 00068 are shown Fgure 11 and for subect number 00014 are shown n Fgure 12. The ntal results show that the results and amount of tme for retreval are very reasonable. Fgure 11. Smlarty based retreval for 00068 based on PCA facal shape Fgure 9. The web based nterface allows one to select a partcular body, or a random body Fgure 12. Smlarty based retreval for 00014 based on PCA facal shape
Clusterng Results We have used the compact body and head descrptors for clusterng. Clusterng s the process of organzng a set of bodes/heads nto groups n such a way that the bodes/heads wthn the group are more smlar to each other than they are to other bodes belongng to dfferent clusters. Many methods for clusterng are found n varous communtes; we have tred a herarchcal clusterng method. We then use Dendrogram whch s a vsual representaton of herarchcal data to show the clusters. The Dendrogram tree starts at the root, whch s at the top for a vertcal tree (the nodes represent clusters). Fgure 12 shows the Agglomeratve Clusterng of Body Shape Dstances descrptor wth number of clusters = 100 and Fgure 13 shows the same wth number of clusters = 30. Concluson We have developed four methods for searchng the human database usng smlarty of human body and head shape. Based on some of our ntal observatons and from our CMC results for an dentfcaton study between 200 standng subects and 200 sttng subects, t can be sad that the body and head descrptors represent the CAESAR bodes qute accurately. We have seen that our approach performs the smlarty based retreval and clusterng n a reasonable amount of tme and therefore, has potental to be a practcal approach. References CAESAR: Cvlan Amercan and European Surface Anthropometry Resource web ste: http://www.hec.afrl.af.ml/cardlab/caesar/ndex.html Paquet, E., Roux, M. 2003 "Anthropometrc Vsual Data Mnng: A Content Based Approach," Submtted to IEA 2003 Internatonal Ergonomcs Assocaton XVth Trennal Congress. Seoul, Korea. NRC 44977 Paquet, E., "Explorng Anthropometrc Data Through Cluster Analyss," Dgtal Human Modelng for Desgn and Engneerng (DHM). Oakland Unversty, Rochester, Mchgan, USA. June 15 17, 2004. NRC 46564 Ben Azouz, Z., Roux, M., Lepage, R. "3D Descrpton of the Human Body Shape: Applcaton of Karhunen Loève Expanson to the CAESAR Database," Proceedngs of the 16th Internatonal Congress Exhbton of Computer Asssted Radology Surgery. Pars, France. June 26 29, 2002 Fgure 12. Agglomeratve Clusterng of Body Shape Dstances descrptor (number of clusters=100 ) Ben Azouz, Z., Roux, M., Shu, C., Lepage, R., Analyss of Human Shape Varaton usng Volumetrc Technques, The 17th Annual Conference on Computer Anmaton and Socal Agents (CASA2004). Geneva, Swtzerland. July 7 9, 2004. Ip, H. H. S. and Wong W. 2002 3D Head Model Retreval Based on Herarchcal Facal Regon Smlarty, Proc. of 15th Internatonal Conference on Vsual Interface (VI2002), Canada. Afzal Godl, Patrck Grother, Sandy Ressler, Human Identfcaton from Body Shape, proceedngs of 4th IEEE Internatonal Conference on 3D Dgtal Imagng and Modelng, Oct 6 10 2003, Banff, Canada. Fgure 13. Clusterng of Body Shape Dstances descrptor (number of clusters=30 ) Afzal, Godl, Sandy Ressler and Patrck Grother, "Face Recognton usng 3D surface and color map nformaton: Comparson and Combnaton", the SPIE s symposum on Bometrcs Technology for Human Identfcaton, Aprl 12 13, 2004, Orlando, FL
Allen, B., Curless, B., and Popovc, Z. 2004. Explorng the space of human body shapes: data drven synthess under anthropometrc control To appear n Proc. Dgtal Human Modelng for Desgn and Engneerng Conference, Rochester, MI, June 15 17. SAE Internatonal CONTACT Afzal Godl can be contacted at afzal.godl@nst.gov NIST, 100 Bureau Dr, MS 8940, Gathersburg, MD 20899 ACKNOWLEDGMENTS We would lke to thank Dr. Kathleen Robnette of Wrght Patterson Ar Force Base, Dayton, USA for provdng us the CAESAR Anthropometry database.