Attributed Relational Graph Based Feature Extraction of Body Poses In Indian Classical Dance Bharathanatyam

Transcription

1 Athra.Sugathan et al Int. Journal of Engneerng Research and Applcatons ISSN : , Vol. 4, Issue 5( Verson 7), May 2014, pp RESEARCH ARTICLE OPEN ACCESS Attrbuted Relatonal Graph Based Feature Extracton of Body Poses In Indan Classcal Dance Bharathanatyam Athra. Sugathan *, Suganya. R ** * (Department of Computer Scence, Amrta Vshwa Vdhyapeetham, Combatore ) ** (Department of Computer Scence, Amrta Vshwa Vdhyapeetham, Combatore ) ABSTRACT Artculated body pose estmaton n computer vson s an mportant problem because of convoluton of the models. It s useful n real tme applcatons such as survellance camera, computer games, human computer nteracton etc. Feature extracton s the man part n pose estmaton whch helps for a successful classfcaton. In ths paper, we propose a system for extractng the features from the relatonal graph of artculated upper body poses of basc Bharatanatyam steps, each performed by dfferent persons of dfferent experences and sze. Our method has the ablty to extract features from an attrbuted relatonal graph from challengng mages wth background clutters, clothng dversty, llumnaton etc. The system starts wth skeletonzaton process whch determnes the human pose and ncreases the smoothness usng B-Splne approach. Attrbuted relatonal graph s generated and the geometrcal features are extracted for the correct dscrmnaton between shapes that can be useful for classfcaton and annotaton of dance poses. We evaluate our approach expermentally on 2D mages of basc Bharatanatyam poses. Keywords Attrbuted relatonal graph, feature extracton, skeletonzaton I. Introducton The man goal of the work s to extract geometrcal features from the attrbuted relatonal graph of the 2D dance pose n mages that are under uncontrolled magng condtons. Features lke orentaton, length of branches, strength, and vector poston of each node are extractng from the graph for classfcaton. The uncontrolled mages wll have background clutter, dversty n clothng, llumnaton and the object may appear at any orentaton and scale. In Bharatanatyam, each person s appearance s unconstraned, as he or she can wear any knd of bharathanatyam dresses, wth hgh complex desgns and colours/textures. Bharatanatyam s the most popular classcal dance whch has 3 man aspects namely Nrtta, Nrtya, and Natya e; poses, shorthand sgn languages and facal expressons. Ths work nvolves only superor body poses e; upper body poses of basc bharathanatyam poses of unque persons from 2D mages to evaluate the smlarty. The basc step of Bharathanatyam s called ADAVU. An adavu s the combnaton of standng posture, hand symbols and leg postons. Here, we are concentratng only on poses and short hand sgns of basc dance steps. We bult skeletonzaton and attrbuted relatonal graph pror to the feature extracton of dance poses from 2D mages. Skeleton of a human structure s the powerful tool for effcent shape matchng and t helps to understand both the shape and topology of object. Thnnng and prunng methods ncluded n skeletonzaton where thnnng method makes the skeleton nto a sngle pxel structure and prunng method removes the unwanted branches presented n the thnned skeleton. B-splne approach s a set of contnuous cubc splne curves whch s used to get smoothed skeleton. The attrbuted relatonal graph s generated to fnd the geometrcal features by settng the nodes n each endpont, jont ponts and vertex ponts. The vectors of each nodes n the graph e; the (x, y) poston of each nodes, length of each branch, strength and orentaton of the object are taken as the features for extracton. Ths allows us to leads to more correct dscrmnaton between shapes by matchng the geometrcal features between the tranng data and testng data usng a classfer. The experments performed usng 110 mages of dfferent bharatanatyam poses of dfferent persons taken by HTC camera of 8 megapxel and downloaded 83 mages of hand mudhras and poses of bharatanatyam from /photosmages/ exponent/bharatanatyam-dance.html. The work of recognzng dance acton has a lot of practcal applcatons that ncludes the software for the study and understandng of Indan classcal dance Bharathanatyam. 11 P a g e

2 Athra.Sugathan et al Int. Journal of Engneerng Research and Applcatons ISSN : , Vol. 4, Issue 5( Verson 7), May 2014, pp Fg.1.Basc pose The next sectons nclude lterature survey, proposed method and obtaned results. In Secton II we descrbe compettve approaches related to our methods. Secton III presents detals of our method, whle Secton IV presents the expermental results usng nput mages. Also, we dscuss some lmtatons and contrbutons of our work atlast. II. Lterature Survey Apratm Sharma et al. [1] proposed a method to fnd the group of poses and classfy the acton sequences of Bharathanatyam steps n real tme usng knect. Skeletonzaton of each pose s obtaned and jont angles are computed based on the orentaton of every jont n the form of rotaton matrces based on ts orentaton and scale. Cluster all the poses and hstogram of poses s used as a feature vector for classfcaton. The methods used here are good n real-tme knect vdeos but not good n mages. Marcn Echner et al. [2] ntroduced Human Pose Co-estmaton where many people n the mage are n a common pose and estmate ther poses jontly. Ths method demonstrated a weakly supervsed learnng of poses that s from mages but not from manual annotatons. Dctonary of poses can be generated from the annotated colored stckmen n the human body parts and each pose class s represented by ts prototypes. Drect model s used n ths method where t fnds a sngle prototype and all local poses are equal to t. Bangpeng Yao et al. [3], proposed a method of object detecton that provdes better human pose estmaton and human pose estmaton mproves the accuracy of detectng the objects by consderng the mutual context. Poses are obtaned by clusterng the confguratons of human body parts. Frst denote the annotaton of human body parts n an mage and then algn the annotatons. Use herarchcal clusterng wth the maxmum lnkage measure to obtan a set of clusters. Ths method descrbes the human object nteracton and needs to annotate the human body parts and objects n each tranng mage. In ths work, weakly supervsed or unsupervsed approach to understand human-object nteracton actvtes s not used. Lng Shao et al. [4] proposed a method for human acton recognton from slhouettes that combnes the advantages of both local and global representatons. Frst, boundng box s normalzed to reduce the dmenson of each frame and remove the varatons of translaton and scale. The normalzed slhouettes are used as the features n each frame by extendng the set of features model. The 2D slhouette mask s converted to 1D mask from each feature vector by scannng t from top to bottom. So, each frame s represented as the bnary element vector and the length s taken as the multplcaton of both row and column. Ths s not an effectve method to encode correlatons of acton descrptors. M. Andrluka et al. [5], proposed a method to develop a generc model for human detecton and pose estmaton that allows to detect hghly artculated human body parts and to estmate ther poses. A pctoral structure model s created and the parts are obtaned usng the part detector. The mean relatve jont poston s learned usng maxmum lkelhood estmaton. A fully connected graphcal model for representng artculatons whch uses dscrmnatve part detectors s proposed. C. D Ruberto et al. [6], proposed a method to use the skeleton of objects n the mages n computer vson and pattern recognton where the object features are needed for classfcaton. Skeletonzaton method s used here to get a model of the graph that leads to more correct representaton of shapes by an attrbuted graph matchng algorthm. The features lke jont postons, edge postons, length are taken as the features n ths method. Ths work fals to get more accurate graph model and fals to get the better determnaton between shapes whle attrbuted graph matchng process. III. Proposed method The archtecture of the system s shown n the fgure 2. Fg.2.Proposed system archtecture 3.1. Skeletonzaton Skeleton of a human structure s a good representaton of human pose. The reason for obtanng the skeleton of an object s that t s easer 12 P a g e

3 Athra.Sugathan et al Int. Journal of Engneerng Research and Applcatons ISSN : , Vol. 4, Issue 5( Verson 7), May 2014, pp to analyze than the orgnal mage and also mantanng the essental propertes of the orgnal mage. Gven an nput bnary mage of poses and short hand sgns (mudhras), skeletonzaton changes non-skeletal object pxels nto background pxels. Dfferent skeletonzaton technques are used to get a skeleton. Some of them preserve topology whle others preserve geometry of the orgnal object. To accurately preserve both topology and geometry from 2-D mages, thnnng algorthm s used Thnnng Algorthm Thnnng algorthm preserves both topologcal and geometrcal propertes of object from 2D mages. It retans the topology of the orgnal object and forces the skeleton to be n the mddle of the object and also preserves the end ponts of the orgnal mage. A morphologcal thnnng operaton s used here, whch s used to remove the foreground pxels from bnary mages. It results n a sngle pxel lne thck and connected skeleton of the nput mage whch s useful for shape descrpton. The nput mage s thnned wth a seres of structurng elements and the object s reduced to a set of one pxel wdth connected lnes. The skeleton s consdered as a connected graph therefore each vertex s consdered as an endpont or juncton ponts and each edge s consdered as curve ponts. The end ponts, juncton ponts and curve ponts of skeleton are detected whch s mportant for the structural descrpton. Skeleton s obtaned by translatng the orgn of the structurng element to each possble pxel poston n the mage and comparng t wth the underlyng mage pxels. If foreground and background pxels n the structurng element exactly match the pxels n the mage, pxel underneath the orgn of the structurng element s set to the background. Otherwse, there s no change further. Image A s thnned by a structurng element B and t s defned n terms of the ht-or-mss transform: AzB = A (A B) = A (A B) c (1) Thnnng of mage A by a sequence of structurng elements: Az {B} = (... ((AzB1) zb2)...) zbn (2) The process s to thn A by structurng element B1, then thn the result wth structurng element B2 and repeated untl A s thnned wth one pass of Bn. The entre process s repeated untl no further changes occur Prunng Algorthm Prunng algorthm s an essental component for skeletonzaton after thnnng. The skeleton of an object often contans both spurous and rough branches due to the boundary rregulartes. To elmnate spurs on a skeleton a morphologcal prunng transformaton s appled. It starts at the end ponts and recursvely removes a gven number of ponts from each branch proceeds untl stablty s reached. Assume that the length of the spur component s wthn a specfc number of pxels. Proposed algorthm has a sze E for the prunng s equal to 4% of the skeleton length. Frst partton the skeleton nto branch parts by subtractng the juncton ponts from t to elmnate only the branches wth dstance E from an end pont. By usng a morphologcal reconstructon method, remove the branches that contanng end ponts only. Then remove all the branches contanng lesser pxels than E. The remanng branches wth juncton ponts and the branches that were not reconstructed from end ponts are unted and obtaned a pruned skeleton Smoothened Skeleton In skeletal representaton, to mprove the smoothness and to reduce the amount of unwanted data, a set of contnuous splne curves s used. It approxmates the unnecessary branches of the orgnal skeleton. Such a contnuous representaton allows us to compute the graph more easly and accurately. B-splne curves are pecewse polynomals that are defned by knots and ponts (t, y ). In ths work we used cubc splne curves wth degree 3. Gven the nodes t and control ponts V the pecewse cubc B-splne curve s gven by: S t t 3 B V 13 P a g e (3) Here, B 3 s the pecewse cubc B-splne. The control ponts V have both x and y components, and the curve s defned as S x y B t V B t x, B t y t t t S S, (4) The x and y components of the curve are separately multpled by B-splnes, and the resultng set of (x, y) ponts gves a curve. A B-splne curve actually needs more nodes than control ponts and t s not defned over the entre nterval t, t ] where [ 1 m m s the number of knots. Instead, for a cubc B- splne curve, the number of control ponts s m 4, and the curve s defned over [ t 4, 3]. 3.2 Attrbuted Graph Skeleton can be represented as graph wth end ponts and juncton ponts as vertces and branches as edges. An attrbuted relatonal graph s bult and used as a structural object for classfcaton by means of graph matchng. Attrbuted graph has set of nodes whch are unon of end ponts and juncton ponts and set of edges whch are represented by skeleton branches. It s parttoned nto number of skeletal branches S (x),=1,2 N. The graph s represented by adjacency matrces and for each relaton type t have dfferent adjacency matrx. t m

4 Athra.Sugathan et al Int. Journal of Engneerng Research and Applcatons ISSN : , Vol. 4, Issue 5( Verson 7), May 2014, pp Weghts are chosen to assocate wth the graph that lnks both geometrcal and topologcal features. The arrangements of skeleton branches can be obtaned by skeletal branch orentaton. The angle between the horzontal axs and the axs by the least second moment s the angle of object tan (5) Where, µ j s the second moments that are computed over the nodes of the splne curves and formng approxmated skeleton. The rate of change of the tangent vector n terms of arc length along wth the curve s defned as the curvature of a curve. The planar curve c(t) = (x(t), y(t)) s not parameterzed wth respect to arc length, therefore ts curvature s: t ' '' '' ' x y x y ' ' x y k (6) Wth x =dx/dt and y =dy/dt. s (t) s the splne whch we used to represent S (x) that s dfferentable twce and can evaluate k(t). 3.3 Feature Extracton The man part of pose estmaton s the feature extracton whch helps for the correct dscrmnaton between shapes of the objects. In ths work the weghts lke angle, feature vector, length of the branches, strength etc are taken as features and extracted. Feature vector s taken as a feature for extracton. Each node poston s obtaned as (x,y) coordnates and the nodes for every object of same shape wll be n the same (x,y) poston n the graph. The graph wndow s normalzed nto a fxed sze for the fndng the node poston. Angle of an object s calculated by the least second moment of nerta and the horzontal axs. Ths gves the correct orentaton of all the branches from the juncton ponts. Angle s a good feature whch gves the exact poston of each branches and t s easy for classfcaton. Each of the curves s (t) has a length λ has been computed by the classcal formula for the arc length and the ntegral has been numercally calculated. t 1 2 n ds 2 ds dt (7) t0 dt 2 The rato between the length λ of the branch and the eucldean dstance between the extreme ponts gves the strength of the skeleton branch. (8) 2 x 2 m x0 ym y0 Where a () 0 =(x () 0,y () 0 ) and a () m =(x () m,y () m ) are the extreme ponts of S (x). A graph s modeled whose set of lnks s represented through the set of vector of weghts wth dt an object O wth a skeleton X parttoned n N parts, s modeled by a graph: E= {(m, θ, λ,σ )} =1, 2.N (9) IV. Expermental result The experments are performed usng the dataset of 2-D mages of dfferent basc steps of Indan classcal dance Bharathanatyam. It ncludes 110 RGB mages of basc bharathanatyam poses of dfferent people of unque experences and sze that are taken by normal HTC camera of 8 megapxels and downloaded 83 mages of hand mudhras and poses of bharatanatyam from stme.com/photosmages/exponent/bharatanatyamdance.html. The mages used n ths project are of plan background, so there s no background clutter present. Every mage has only one person present n t. Fg.3.Input Image Fg.4. Bnary Image 14 P a g e

5 Athra.Sugathan et al Int. Journal of Engneerng Research and Applcatons ISSN : , Vol. 4, Issue 5( Verson 7), May 2014, pp Fg.5. Thnned Image Fg.8. Skeleton wth Nodes Fg.6. Pruned Image Fg.9. Graphcal Representaton Fg.7. Nodes n the Image Fg.10. Feature Vector 15 P a g e

6 Athra.Sugathan et al Int. Journal of Engneerng Research and Applcatons ISSN : , Vol. 4, Issue 5( Verson 7), May 2014, pp Fg.11. Angle Feature Extracton Fg.12. Length Fg.13. Strength V. Concluson In ths work a method s proposed to ncrease the qualty of skeleton of human object n 2- D mages of Indan classcal dance- Bharathanatyam. A morphologcal skeleton of a bnary mage s created to enhance the skeleton qualty n order to fnd more effcent features for classfcaton. Ths allows fndng a graph model wth more attrbutes. In ths method the skeleton characterstc ponts s represented as an attrbuted relatonal graph to model the skeleton. The future research wll nclude fndng a more accurate graph model, allowng the determnaton of more effcent attrbutes for the nodes and the edges. Next work ncludes classfcaton and annotaton of the poses and hand mudhras where t can leads to an automatc applcaton for studyng Bharathanatyam. In ths work the basc steps of unque person s consdered and evaluated. References [1] Apratm Sharma under the gudance of Dr. Amtabha Mukerjee, Recognsng Bharatanatyam Dance Sequences usng RGB-D Data, A Thess Submtted n Partal Fulfllment of the Requrements for the Degree of Master of Technology from Indan Insttute of Technology,Kanpur,2013. [2] Marcn Echner and Vttoro Ferrar, Human Pose Co-Estmaton and Applcatons, Proceedngs of the IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 34, No. 11, November 2012 pp [3] Bangpeng Yao and L Fe-Fe, Recognzng Human-Object Interactons n Stll Images by Modelng the Mutual Context of Objects and Human Poses, Proceedngs of the IEEE Transacton on Pattern Analyss and machne ntellgence,vol.34,no.9,2012,pp [4] D Wu and Lng Shao, Slhouette Analyss- Based Acton Recognton Va Explotng Human Poses, Proceedngs of the IEEE transactons on crcuts and systems for vdeo technology, vol. 23, no. 2, February [5] M. Andrluka, S. Roth, and B. Schele, Pctoral Structures Revsted, People Detecton and Artculated Pose Estmaton, Proceedngs of the IEEE Conference of Computer Vson and Pattern Recognton, 2009, Vol.pp [6] C. D Ruberto, G.Rodrguez, Recognton of shapes by morphologcal attrbuted relatonal graphs, Proceedngs of the IEEE Conference of Computer Vson and Pattern Recognton,2004,37,1,Vol.pp [7] C. D Ruberto and A.G. Dempster, Attrbuted skeleton graphs usng mathematcal 16 P a g e

7 Athra.Sugathan et al Int. Journal of Engneerng Research and Applcatons ISSN : , Vol. 4, Issue 5( Verson 7), May 2014, pp morphology, Electroncs Letters, vol.37 num.27 (2001) [8] C. D Ruberto, Matchng and Recognton of Shapes by Morphologcal Attrbuted Relatonal Graphs, submtted [9] S. Gold and A. Rangarajan, A graduated assgnment algorthm for graph matchng, Proceedngs of the IEEE Transacton on Pattern Analyss and Machne Intellgence, vol. 18 num. 4 (1996) [10] S. Gold and A. Rangarajan, Graph matchng by graduated assgnment, Proceedngs of the IEEE Computer Socety Conference on Computer Vson and Pattern Recognton 1996, IEEE Computer Socety (1996), [11] Yanshu Zhu, Feng Sun, Y-Kng Cho, Bert, Juettler, WenpngWang, Splne Approxmaton to Medal Axs, arxv: v1[cs.gr]29,june [12] K.J Maccallum and J.M Zhang, Curve- Smoothng Technques Usng B-Splnes, The Computer Journal, Vol.29, No.6, [13] B. Jang and R.T. Chen, Analyss of thnnng algorthms usng mathematcal morphology, IEEE Transacton on Pattern Analyss and Machne Intellgence, vol. 12 num. 6 (1990) [14] Salm Joul and Salvatore Tabbone, Grap Matchng based on Node Sgnatures, publshed n "7th IAPR-TC-15 Workshop on Graph-based Representatons n Pattern Recognton-GbRPR (2009) ",pp ,SprngerVerlag Berln, Hedelberg P a g e