A Case Study of Clustering the Source Code

Transcription

1 A Case Study of Clusterig the Source Code NADIM ASIF, FAISAL SHAHZAD, NAJIA SAHER, WASEEM NAZAR Dept. of Computer Sciece The Islamia Uiversity of Bahawalpur Baghdad campus, Bahawalpur, PAKISTAN Abstract:- The software systems are developed usig the multi-laguages with differet dialects ad scripts. Whe the chages are performed, the source code drifts away from the existig available system documetatio (specificatios, desig, mauals), which represet the fuctioality of the software systems. The systems are required to uderstad ad preset at higher levels of abstractios to perform the chages ad meet the curret requiremets. The source code clusterig is used for the purpose of recoverig the artifacts, uderstadig the system ad idetifyig the relatioships amog the source code to pla, desig ad execute the chages i the software systems. This paper presets a clusterig approach usig the available source code, documetatio, experiece ad kowledge about the domai ad applicatio to cluster the source code. Key Words:- Clusterig, Source Code Clusterig, Source Code Aalysis, Re-Egieerig, Reverse Egieerig, Desig Recovery, Program Uderstadig, Software Maiteace. 1. Itroductio The software egieers perform the code aalysis ad differet maiteace activities by extractig the differet types of artifacts at differet levels of details usig the clusterig. The software artifacts exist at implemetatio, structural, fuctioal ad domai abstractio levels. The chages are performed i the software systems ad the existig documets are drifted away form the implemetatio ad fail to represet the curret implemetatio of the system. The reverse egieerig techiques help to represet the software systems at higher levels of abstractio tha code to recover the desired artifacts, uderstad ad comprehed the source code ad elaborate the fuctioality of the software systems to pla, desig ad execute the differet types of maiteace activities. The software egieers cluster the source codes i differet formats to represet the systems at higher levels of abstractio for uderstadig ad represetig software systems for maiteace activities. The clusters represet the higher level of abstractio of a source code. These clusters help to explore, search the specific features ad relatioships amog the source code, uderstad the code, fuctioality ad behavior of software systems for maiteace tasks at had [1,3,4,5]. The source code clusterig is used for the followig purposes For uderstadig the programs For idetificatio (physically ad coceptually) of codes (specific lie of codes), where chages ca be performed Categories ito physical or coceptual compoets To model ad perform chages. To pla, desig ad execute the chages i the source code ad also predicts the impacts of these chages i the source code. Moitor the effects of chages The available source code exist i may forms; may be writte i multi-laguages or have differet dialects ad scripts, ca ot be compiled or have errors ad complete code is ot available. The software egieers debug the source code to fid the relatioships ad fuctioality amog the source code, associate them with relevat etities to uderstad, which is a time cosumig ad laborious task. This paper presets a approach of clusterig the source code usig the available source code, documets, experiece ad kowledge of applicatio ad domai as required by the task at had. 2. Backgroud A cluster is a collectio of objects that are similar to oe other withi the same cluster ad are dissimilar to objects i other clusters. The process of groupig the physical or abstract objects ito etities of similar objects is called clusterig. The set of techiques ISSN: ISBN:

2 based o clusterig to extract the desig artifacts from the system's artifacts (source code ad available documetatio) are used for the purpose of maiteace tasks. The clusterig approaches helps user to perform the clusterig by adaptig the topdow, bottom-up ad combiatio (hybride) of both strategies required by the maiteace task at had The bottom up strategy start by placig each object i its ow cluster ad the merge these cluster ito larger ad larger cluster, util all of the objects are i a sigle cluster or certai termiatio coditios are satisfied. The top-dow strategy divide the cluster ito smaller ad smaller pieces, util each object forms a cluster o its ow or util satisfy termiatio coditios, such as a desired umbers of clusters are obtaied. The hybride strategy allow the user to form the clusters startig at ay level of available sample data to perform bottom-up ad top-dow clusterig, combiatio of both strategies to develop the desired clusters to certai levels for the task at had. The major clusterig techiques ca be divided ito the followig categories o the bases of the type of method it adapt to cluster the data objects [2, 29, 30, 31]. Hierarchical Methods : The hierarchical methods ca be divided ito two major categories o the bases of the strategy, (top dow or bottom up), it adapt to cluster the objects; Agglomerative ad Divisive Hierarchical Clusterig. Agglomerative hierarchical clusterig place each objects i its ow cluster ad the combies these clusters ito larger ad larger cluster util certai termiatio is satisfied. The Divisive hierarchical clusterig use the top dow strategy ad divide the cluster ito smaller clusters, util each object from a cluster o its ow or satisfy certai termiatio coditio e.g. Agglomerative (AGNES), DIvisive ANAlysis (DIANA), Balaced Iterative Reducig ad Clusterig usig Hierarchies (BIRCH), Clusterig Usig REpresetative (CURE), Chameleo. Partitio Methods : The typical methods icludes k-meas, k-mediods. The partitio methods first create set of k partitios, the use the relocatio techique to improve the partitios by movig objects from oe group to aother group. Grid Methods : The grid methods first quatizes the objects space ito a fiite umber of cells that form a grid structure, ad the clusterig is performed o the grid structure e.g. STatistical Iformatio Grid (STING), WaveCluster, Clusterig I QUEst (CLIQUE). Desity-Based Methods : The objects are clustered based o the otio of desity. The clusters grow o the bases of the desity of eighborhood objects or accordig to some desity fuctio e.g. A Desity- Based Spatial Clusterig of Applicatio with Noise (DBSCAN), Orderig Poits To Idetify the Clusterig Structure (OPTICS), DENsity-based CLUstrig (DENCLUE). Model Based Methods: The model based methods hypothesizes a model for each of the cluster ad the fid the best fit of the data objects to that model. These type of clustig methods use the statistical or eural etwork approach e.g. COBWEB, CLASSIT, Auto-Class. Hybride Clusterig Methods : The methods which itegrate the idea of may clusterig methods ad do ot belog uiquely to particular clusterig method category. The other types of methods are the fuzzy clusterig methods [2]. The clusterig approaches classificatio based o the artifacts recovery ca be divided ito two categories, automatic or semi-automatic clusterig techiques. The automatic techiques [10, 11, 12] use the similarity metric (associatio coefficiet, correlatio coefficiet or probabilistic measure) to partitio the system ito related group etities. The semi-automatic techiques perform user-assisted clusterig process usig domai kowledge ad visualizatio meas [6, 13, 14, 15, 16, 17]. Some popular clusterig techiques use source code compoet similarity (Hutches ad Basili,[12]; Schwake [23]; Choi ad Scacchi [21]; Muller et al [26,27] ). Aother class of techiques use the implemetatio iformatio such as module, directory, ad/or package ames to derive the subsystems [22], ad third type of techiques are based o heuristic search techiques ( Macoridis et al., [24, 25]; Mitchell et al, [11,24]). The Lakhotia [28] also preseted twelve reverse egieerig techiques based o clusterig. The Rigi Tool [26, 27] operates semi-automatically, o a geeric set of source code model relatio. Rigi user extracts the structural iformatio from the system artifacts ad represets that iformatio as a set of relatio. The Rigi tool takes these relatios as iput ad displays them as collectio of overlappig graphs. The user tha maipulate the graph(s) ad idetifies the source code etities (odes) that should be clustered. The user idetifies the set of etities to cluster by applyig graph-theoretic algorithms (i.e. idetify the strogly coected compoets), by quality metrics such as couplig ad cohesio or by defiig scripts such as searchig for odes (etities) coformig to particular amig covetios. The user produces the opaque model by usig the Rigi that does ot show the details of the source model. This model may be appropriate for some tasks, ad while for others, the details may be beeficial. I ISSN: ISBN:

3 Rigi the odes of graph are clustered based o source model iformatio such as the ames of etities, the clusterig is performed by usig the iterface or by usig a procedural script. Murphy et. al Reflexio model techique cluster source code etities through the use of declarative map [32] to produce a high level model of a system. The declarative map is easy for the user to specify. The declarative map is shorter tha procedural map, ad the map is likely simple i format, improvig the likelihood that the user specifies the desired mappig. With Rigi, however, a user must express the desired clusterig of source etities, either maually or programmatically or both based o the etire source model. Eve whe sufficiet clusterig is performed to derive a high-level model with Rigi, the model that results is ot ecessarily a view of iterest to the egieer. The Rigi method is drive from the bottom-up, ad aother way to improve the desired view with Rigi is to apply the domai ad system-specific kowledge durig the bottom-up clusterig process. Macoridis et al. [25] describes usig automatic clusterig to produce high-level system orgaizatios of source code. The approach explais a collectio of algorithms that were developed ad implemeted to facilitate the automatic recovery of the modular structure of a software system from its source code. Automatic modularizatio is treated as a optimizatio problem ad the algorithms described use traditioal hill-climbig ad geetic algorithms. A automatic software modularizatio eviromet is defied ad a case study is show to illustrate the effectiveess of the modularizatio techique. Clusterig is cosidered as a optimizatio problem where the goal is to maximize a objective fuctio based o a formal characterizatio of the trade-off betwee iter ad itra-coectivity. Kamra Sartipi [15] preseted aother user assisted clusterig techique for architecture recovery based o approximate measure, ad compute o the shared properties amog of highly related system etities. Maletic ad Marcus applied the iformatio retrieval techique called Latet Sematic Idexig (LSI) for software reverse egieerig [17, 18]]. They used LSI to aalyze the sematic clusters of the files of Mosaic. The user ca iteract ad avigate the visualizatios of the sematical clusters, aided by complemetary lower level iformatio about the properties ad itercoectios betwee the compoets of the clusters. Aother approach to software reverse egieerig is the use of visualizatio techiques to represet the software etities ad their relatioships [19, 20] at higher levels of abstractio. Tools that use structural exploratio are the Rigi [26], SHriMP [20] ad Buch [24]. SHriMP supports a top-dow approach to software exploratio while employig a estedgraph visualizatio techique. Bruch first determies the resources ad relatios i the source code ad store the resultat iformatio i a database. Available source code aalysis tools of a variety of programmig laguages is used for this step. After the resources ad relatios have bee stored i a database, the database is queried ad a Module Depedecy Graph (MDG) is created. MDG is a directed graph that represets the software modules (e.g., classes, files, packages) as odes, ad the relatios (e.g., fuctio, ivocatio, variable usage, class iheritace) betwee modules as directed edges. The the clusterig algorithms are used to create the partitioed MDG. The clusters i the partitioed MDG represet subsystems that cotai oe or more modules, relatios, ad possibly other subsystems. The fial result ca be visualized ad browsed usig a graph visualizatio tool such as dotty. 3. Source Code Clusterig Process The first step of clusterig process is to idetify the etities usig the available documets, experiece ad kowledge about the domai ad the applicatio. A etity (E i ) defie/comprehed a cocept ad is used to represet higher abstractio level of compoets/modules, data sources ad processes i a domai, which are used i the high level models to represet the software systems [6,7,8,9]. The subetities represet the lower levels of abstractios as compared to a etity. For example accout is a etity i a bakig domai ad persoal accout, corporate accout are examples of sub-etities represet the specific types of accouts. The user specifies the etities ad writes the abstract regular expressios to cluster the source code physically or coceptually. I the secod step, the etities are used to cluster the source codes which represet the coceptual or physical (directories, files) associatio with the etities. The process is repeated util the desired clusters are formed. The source code clusterig must satisfy the followig requiremets. 1. Uit cluster cotais miimum a sigle lie of code. 2. Let Cp represet a sigle cluster of source code formed by usig the physical relatioships, the files or type of files ad directories associatio with source code. ISSN: ISBN:

4 3. Let Cc represet a sigle cluster of source code developed usig the coceptual relatioships, the compoets/sub-compoets, classes/sub-classes ad fuctios associatio with the source code. 4. The cluster C i will be similar to the cluster C j physically, if they have the same lie of code i the same sequece the C i = C j. Clusters dissimilar physically, if they have differet lie of code i differet sequece the C i C j. 5. The cluster C i will be similar to the cluster C j coceptually, if they perform the same fuctio but may differ physically C i = C j. Clusters dissimilar coceptually, if they perform differet fuctios the C i C j OR similar physically C i = C j. 6. The cluster formed usig the etity E i is represeted by. Cp = C i i = 1 Cc = C i i = 1 The clusters (Cc) formed usig the cocepts represeted by etity (E i ), which abstract the cocepts. The compoets, modules, classes, fuctios which represets the etity (cocepts) implemeted i the source code. The source code is orgaized physically i the files or types of files ( *.pp, *.jar, *.exe etc) ad directories. The cluster Cp is formed usig the etity (E i ), which associate the files (code of lies exists i differet files ad directories) to the cluster. The C (i,j) S(E i,e j ) = C (j,i) S(E j,e i ) ad C (i,j) D(E i,e j ) = C (j,i) D(E j,e i ) The mea ad average similarity, ad dissimilarity of clusters are calculated by usig the followig equatios. 8.The cluster C i ad C j will be merged if the differece C (i,j) D(E i,e j ) = 0 or both have equal umber of similar objects the C (i,j) S(E i,e j ) = C (i,j) D(E i,e j ). C S = C (i,j) S(E i,e j ) i = 1 j = i+ 1 C D = C (i,j) D(E i,e j ) i = 1 j = i+ 1 The Similarity ad dissimilarity of etity E i with the other etities (E 1,E 2,.. E ) is represeted by the followig equatios C S = C (i,j) S(E i,e j ) j = 1 C D = C (i,j) D(E i,e j ) j = 1 For example, clusters C i ad C j are formed usig the etities E i ad E j i figure 1. The cluster C i cotais 6 objects (A,B,C,D,E,F) ad cluster C j cotais 4 Objects (A, C, K, M). 7. Each etity (E i ) cotribute i the cluster has weight equal to 1 (W Ei = 1). The Similarity ad dissimilarity of clusters are represeted by S o ad D o. S o = C (i,j) S(E i,e j ) D o = C (i,j) D(E i,e j ) = T o - S o x 2 A F C i E B C E i D M A C E j C j K Where S o is the umber of similar objects i cluster C i ad C j ad the etities E i ad E j are used to form the cluster C (i,j) S(E i,e j ). TheT o represet the total umber of objects i clusters C i ad C j. The C (i,j) D(E i,e j ) represet the dissimilarity betwee the clusters C i ad C j ad the cetre poits of clusters are the etities E i ad E j used to form the cluster. C i Figure 1. The Similarity ad dissimilarity of clusters are calculated below. S o = C (i,j) S(E i,e j ) = 2 D o = C (i,j) D(E i,e j ) = T o - S o x 2 = 10 2 x 2 = 6 C j ISSN: ISBN:

5 4. Case Study The clusters are developed usig the etities CToke, Scaer ad Parse, which are idetified from the existig available Mozilla HTML Parser source code, documetatio ad kowledge about the applicatio ad domai. The clusters depicted i figures 2, 3, 4 & 5 represet the coceptual relatioship with classes ad fuctios. The approach has the followig features required for clusterig the source code User-orieted: The approach ivolves the user ad also allows the user to cluster the source code usig the experiece, domai ad applicatio kowledge. +AllClasses.txt 7 class CRTFCotrolWord : public CToke { ** 8 class CRTFGroup: public CToke { ** 9 class CRTFCotet: public CToke { ** 12 class CTokeFider: public sdequefuctor{ ** 17 class CTokeDeallocator: public sdequefuctor{ ** 18 class CTokeRecycler : public sitokerecycler { ** 30 class CHTMLToke : public CToke { ** 71 class CToke { ** 72 class CTokeHadler : public CITokeHadler { Figure 2 Cluster the classes usig the Etity CToke Figure 3. Clusterig the CToke Class Fuctios ISSN: ISBN:

6 +shtmltokes.cpp 30 CHTMLToke::CHTMLToke(cost sstrig& aname,ehtmltags atag) : CToke(aName) { ** 35 CHTMLToke::CHTMLToke(eHTMLTags atag) : CToke(aTag) { ** 40 void CHTMLToke::SetStrigValue(cost char* ame){ ** 589 CHTMLToke::Reiitialize(aTag,aStrig); Figure 4. Clusterig the CHTMToke Class fuctios Figure 5. Clusterig the sparser Class Fuctios Iterative: The process is iterative ad the clusters are formed to the desire level required by the task at had. Partial: oly the desired source code clusters are formed for the task at had. 5. Coclusio The software source code exist i may forms; may be writte i multi-laguages or have differet dialects ad scripts, ca ot be compiled or have errors ad complete code is ot available. The software egieers debug the source code ad fid the relatioships ad fuctioality ad associate ISSN: ISBN:

7 them with relevat etities to uderstad ad fid the relatioships amog the differet pieces of source code exist i differet types of files ad directories, which is a time cosumig ad laborious task. The approach clusters the source code usig the available documetatio, experiece, kowledge of applicatio ad domai. The source code clusters are formed usig the etities which represet the cocepts implemeted i the software source code. The approach clusters the source code coceptually (usig coceptual relatioships compoets, classes, fuctios, variables) ad physically (directories, types of files where the lies of source code exist). The clusters are formed usig the top-dow, bottom-up ad hybride (combiatio of both) strategy as required by the task at had to the desired level of clusterig. Refereces [1] A.K Jai, M.N Murty ad P.J Fly, Data Clusterig: A survey. ACM Computig Survey. 31, 1999, pp [2] L. Kaufma ad P.J Rousseeuw, Fidig Groups i Data: A Itroductio to Cluster Aalysis. New York: Joh Wiley & Sos, [3] Nadim Asif, M. Dixo, J. Filay ad G. Coxhead, Recover the Desig Artifacts. I proceedigs of Iteratioal Coferece of Iformatio ad Kowledge Egieerig (IKE02), 24 th 27 th Jue, 2002, Las Vegas, Nevada, USA, CSREA Press, pp [4] Nadim Asif, Reverse Egieerig Methodology to Recover the Desig Artifacts: A Case Study. I proceedigs of Iteratioal Coferece of Software Egieerig Research ad Practice (SERP03), 23 rd -26 th Jue, 2003, Las Vegas, USA,CSREA Press, pp [5] Nadim Asif, Muthu Ramachadra, Recover the Use Case Models. I proceedigs of Iteratioal Coferece of Software Egieerig Research ad Practice (SERP05), 27 th -30 th Jue, Las Vegas USA, 2005, CSREA Press. [6] Nadim Asif, Developig High Level Models for Artifacts Recovery ad Uderstadig Usig Statistical Iformatio. I proceedigs of 8 th Islamic Coutries Coferece o Statistic. 19 th - 23 rd Dec, [7] Nadim Asif, Software Reverse Egieerig, SoftResearch Press, (ISBN : ). [8] Nadim Asif, Artifacts Recovery at Differet levels of Abstractio. Iformatio Techology Joural, 7(1), pp. 1-15, [9] Nadim Asif, Artifacts Recovery Techiques, Iteratioal Joural of Software Egieerig, Vol.1, No 1, 2007, pp [10] Kuz, T, Black, JP, Usig automatic clusterig process for desig recovery ad distributed debuggig. IEEE Trasactios o Software Egieerig; 21(6), p ,1995. [11] Mitchell, Bria S., Macoridis, Spiros, O the Automatic Modularizatio of Software Systems Usig the Buch Tool. IEEE Trasactios o Software Egieerig; Vol. 32 Issue 3, p , March [12] D. Hutches ad R. Basili. System Structure Aalysis: Clusterig with Data Bidigs. IEEE Trasactios o Software Egieerig, 11: , Aug [13] Fiiga. P et. al., The Software Bookshelf, IBM Systems Joural, 4, , [14] Muller, H.A et al, A reverse Egieerig Approach to subsystem structure idetificatio. Joural of Software Maiteace: Research ad Practice.5(4), , [15] Sartipi, K.,Kotogiais K. A User-assisted approached to compoet Clusterig. Joural of Software Maiteace: Research ad Practice, 00:1-32, [16] T. Wiggerts. Usig clusterig algorithms i legacy systems remodularizatio. I Proc. Workig Coferece o Reverse Egieerig, [17] J. I. Maletic ad A. Marcus. Supportig program comprehesio usig sematic ad structural iformatio. I Proceedigs of the Iteratioal Coferece o Software Egieerig (ICSE 2001), pages , [18] A. Kuh, S. Ducasse, ad T. Gˆýrba. Erichig reverse egieerig with sematic clusterig. I Proceedigs of Workig Coferece O Reverse Egieerig (WCRE 2005), Nov [19] M. Laza ad S. Ducasse. Polymetric views a lightweight visual approach to reverse egieerig. IEEE Trasactios o Software Egieerig, 29(9): , Sept [20] J. Michaud, M.-A. Storey, ad H. Muller. Itegratig iformatio sources for visualizig Java programs. I Proceedigs of IEEE Iteratioal Coferece o Software Maiteace (ICSM 01), pages IEEE, Nov ISSN: ISBN:

8 [21] S.Choi ad W. Scacchi. Extractig ad restructurig the desig of large systems. I IEEE Software, pages 66 71, [22] N. Aquetil, C. Fourrier, ad T. Lethbridge. Experimets with hierarchical clusterig algorithms as software remodularizatio methods. I Proc. Workig Cof. o Reverse Egieerig, October [23] R. Schwake. A itelliget tool for reegieerig software modularity. I Proc. 13th Itl. Cof. Software Egieerig, May [24] Macoridis, S., B.S. Mitchell, Y. Che, ad E.R. Gaser. Buch: A clusterig tool for the recovery ad maiteace of software system structures. I Proceedigs of Iteratioal Coferece of Software Maiteace, pages 50 59, August [25] Macoridis, S., Mitchell, B.S., Rorres, C., Che, Y. ad Gaser, E. R., Usig Automatic Clusterig to Produce High-Level System Orgaizatios of Source Code. I: Proceedigs of the Sixth Iteratioal Workshop o Program Comprehesio, 24th 26th Jue, IEEE Computer Soc. Press. pp , [26] Muller, H.A et al., Rigi. Available from: < [27] Wog, K., Tilly, S., Muller, H. ad Storey, M., Structural Redocumetatio: A Case Study. IEEE Software, Vol. 12, No. 1: Jauary, pp , [28] Lakhotia, A., A Uified Framework for Expressig Software Subsystem Classificatio Techiques. Joural of Systems ad Software, 36, pp , [29] A.K Jai, M.N Murty ad P.J Fly, Data Clusterig: A survey. ACM Computig Survey. 31: pp , [30] Romero, C., Vetura, S. Educatioal Data Miig: A survey from 1995 to 2005, Expert Systems with Applicatios; Vol. 33 Issue 1, pp , July [31] N. Aquetil ad T.C. Lethbridge, Comparative study of clusterig algorithms ad abstract represetatios for software remodularisatio, IEE Proc.-Software., Vol. 150, No. 3, Jue [32] Murphy, G., Notki, D., ad Sulliva, K., Software Reflexio Models: Bridgig the Gap betwee Desig ad Implemetatio. IEEE Trasactio o Software Egieerig. Vol. 27. No 4: April, pp , ISSN: ISBN: