Resolving Linkage Anomalies in Extracted Software System Models

Transcription

1 Resolving Linkage Anomalies in Extracted Software System Models Jingwei W and Richard C. Holt School of Compter Science University of Waterloo Waterloo, Canada j25w, Abstract Program model linking has been largely overlooked and not constrained properly in the extraction of software system models. This often reslts in inaccrate system models at different levels of abstraction even if programs can be extracted correctly. This paper describes two constrained approaches toward accrate linkage resoltion. The first approach is prely based on a set of linking heristics. The second approach leverages the software bild process and also tilizes linking heristics. We compare these two approaches and discss their benefits and limitations. The empirical reslts from a case stdy of the PostgreSQL database system are also presented. Or stdy shows that inappropriate linkage resoltion leads to a relatively large nmber of dependency anomalies at higher levels of abstraction. These anomalies can be effectively removed sing or proposed approaches. 1 Introdction Modern software systems are extraordinarily large. For example, large database systems have millions of lines of code and are extremely complex. In order to analyze sch large systems, it is often necessary to extract system models that have linkage dependencies at varios levels of abstraction, sch as fnction level, modle level, and sbsystem level. A linkage dependency normally refers to a fnction call, a variable access, or a se dependency between two modles. This kind of system model represents static software strctre and spports downstream software analysis, for example, reflexion modelling [8], software clstering [7, 12], and architectre extraction and repair [6, 11]. To ensre the accracy of extracted system models, a significant amont of effort has been expended in developing high qality program extractors [1, 2, 3, 10]. By contrast, the process of linking separately extracted program models into system models has been largely overlooked. If not properly done, this linking process can either prodce nexpected linkage or case linkage absences. We refer to both phenomena as linkage anomalies. In an extracted system model, linkage anomalies only accont for a very small percentage of all the resolved linkage dependencies bt they propagate to higher levels of abstraction and case ndesirable impacts on downstream software analysis. In a case stdy of the PostgreSQL system [9], we fond that a small nmber of linkage anomalies cased by nconstrained linking prodced a relatively large nmber of dependency anomalies at higher levels of abstraction. We will explain nconstrained linking in Section 2.2. Abot 1.7% of the resolved cross-references in PostgreSQL were anomalies. The high-level dependency anomalies cased by these anomaly cross-references acconted for 3.5% of dependencies at the modle level and 7.2% at the sbsystem level. If we do reflexion modelling analysis on the extracted system model, we will end p with many architectral divergences that are not cased by the system implementation bt by improper linkage resoltion. This paper discsses what cases linkage anomalies and how to resolve them nder the assmption that programs are extracted separately and accrately. The rest of this paper is organized as follows. Section 2 explains program model linking and linkage anomalies. Section 3 discsses a set of heristics and proposes two approaches to resolving linkage anomalies. Section 4 presents the empirical reslts of or case stdy. Section 5 concldes the paper. 2 Program Model Linking We define program model linking as the process of combining mltiple linkable program models (LPM) into one model and replacing symbolic references with direct references. In this definition, a LPM is a semantic representation of an object modle. Therefore, we can see that program model linking plays the role of code linking, in which object modles are replaced with LPMs.

2 LPM ExternSymbol ResolvedByExtern 2.1 LPM Schema GlobalScope RefersToExtern RefersToExtern Fnction NameRef File RefersTo Figre 1. LPM Schema contain Variable RefersTo A large nmber of program extractors exist [1, 2, 3, 10]. They otpt facts in accordance with different schemas. To facilitate or discssion, we adopt a fact schema shown in Figre 1, which is a redced version of the CPPX/DATRIX schema [2, 4]. Many abstract syntax tree (AST) nodes sch as Type and Statement have been removed. The GlobalScope represents the niversal root of all LPMs. A File entity is a sorce file or an object modle. Each ExternSymbol entity has a niqe identifier that is a symbolic name in the global naming space. A NameRef entity is the domain of a symbolic reference. There are for relations in the schema. The contain is a strctral relation, which forms a tree in any LPM. The RefersTo represents the resolved references within a file. The ResolvedByExtern relation represents symbolic references, which need to be resolved by looking for extern definitions. The RefersToExtern means that a global definition is exported as a symbolic name. 2.2 LPM Linking The resoltion of symbolic references among a collection of LPMs can be concisely expressed and calclated as a relational composition of two binary relations: LinksTo = ResolvedByExtern o RefersToExtern We se o to refer to relational composition. The LinksTo relation denotes cross-references from NameRef entities to Fnction and Variable entities. For example, if X is resolved by Y and Y refers to Z, then X links to Z. By contrast, the RefersTo relation denotes resolved references within a LPM. These two relations together denote all direct references within a software system. Program model linking provides a mechanism to derive software system models. A system model is the combination of program models that are progressively generated as the system is being bilt, and it has no dangling symbolic references among those generated program models. In this definition, the prohibition of dangling symbolic references mandates that any symbolic references resolvable within the system mst be resolved. However, it is optional to resolve those symbolic references pointing to the environment, in which the system is embedded. An approach to deriving system models is nconstrained if it solely ses the above eqation to resolve symbolic references withot considering constraints, sch as program bild dependencies and heristics, which we will discss in more detail in Section 3. An nconstrained approach may prodce linkage anomalies de to the mlti-resoltion of symbolic references. 2.3 Linkage Anomalies A linkage anomaly means an nexpected or a missing linkage dependency in a software system model. In the following, we discss possible cases of linkage anomalies. Mlti-Resoltion A symbolic name with more than one definition can lead to mltiple resoltions of a symbolic reference that refers to the name. This can occr when the software system comprises a collection of exectables and dynamic libraries. In this case, it is highly possible that one symbolic name has more than one definition in varios parts of the system. If model linking is not properly controlled, the resoltion of a symbolic reference may lead to mltiple direct references with only one of them representing the tre cross-reference dependency. For example, the PostgreSQL system has 18 exectable programs and 28 dynamic libraries targeted at the Linx platform. There exist 75 symbolic names with mltiple global definitions. One of them is a fnction called EncodeDateTime, which is defined in two different sorce files, datetime.c and dt common.c. The first file is sed to generate the exectable postgres, which is the backend database server, and the second file is sed to bild the dynamic library libecpg.so, which provides spport for embedding SQL in C. If we apply nconstrained linking, every symbolic reference to the name EncodeDateTime will be resolved to two direct references. Incomplete Resoltion To overcome the mlti-resoltion problem, one soltion is to leverage the software bild process to constrain model linking. We refer to this approach as simlation-based linking, which will be described in Section 3. Unfortnately, the simlation-based approach does not garantee complete linkage resoltion. A software system bild often incldes 2

3 dynamic libraries. When generating a dynamic library, a code linker sch as the GNU linker leaves some dangling symbolic references, which will be resolved by the rntime linker as the dynamic library is loaded into memory. However, a system model reqires that any dangling symbolic references mst be resolved if they refer to definitions within the system. We now describe sch a case fond in PostgreSQL. The fnction tplestore begin heap defined in the file tplestore.c is called by the fnction exec init tple store defined in pl exec.c. The file tplestore.c is part of the backend exectable postgres, and the file pl exec.c is compiled and then linked into a shared library called libplpgsql.so, which provides a loadable procedral langage. If we only follow the steps of bilding PostgreSQL to do model linking, we will not be able to resolve any symbolic references to tplestore begin heap in the file pl exec.c. 3 Resolving Linkage Anomalies We now describe two approaches to linkage resoltion. In contrast to nconstrained linking, these approaches apply constraints to control the process of model linking. The first approach prely tilizes five linking heristics. The second approach simlates the software bild process to gide model linking and then ses heristics for postprocessing. 3.1 Linking Heristics In the following, we discss five heristics, which can be applied to improve the accracy of program model linking. f1 n v r (a) Modle-Modle f1 n v mx r1 r2 (b) Modle-Modle Transitive Closre ss1 ss2 f1 n v H1 Uniqe Definition If a symbolic name refers to exactly one definition, all the symbolic references pointing to that symbolic name can be resolved withot ambigity. This heristic is the most sefl since it resolves the majority of symbolic references. H2 Modle-Modle If a cross-reference already exists from modle to modle, we give higher priority when searching for the definition of any nresolved symbolic references in. Figre 2(a) provides an illstration of this heristic. When resolving the symbolic reference in modle to the name n, we have two cross-reference candidates, v and. We se the notation x.n to indicate that x is named n. Since there already exists a resolved cross-reference r between and, v is determined to be the correct reference in this case. (c) Same Sbsystem ss1 S1 S2 ss2 f1 n v (d) Nearest Sper Sbsystem Figre 2. Linking Heristics (H2-H5) 3

4 H3 Modle-Modle Transitive Closre This heristic is an extension of H2. We lift resolved crossreferences to dependencies at the modle level and then compte the transitive closre of these modle dependencies. This closre is then sed to gide H2 to resolve linkage anomalies. Figre 2(b) shows a simple case. There is no direct crossreference between and, bt an indirect modle dependency between them is formed by r1 and r2. We give priority over when searching for the symbolic name n. Ths, is determined to be the anomaly to eliminate. H4 Same Sbsystem In the C programming langage, a static definition in a compilation nit has higher priority of being linked to when a symbolic reference is to be resolved in that nit, even if an extern definition with the same name exists. We extend this rle and apply it as a heristic in the resoltion of symbolic names within the scope of the sbsystem 1. We refer to this heristic as the Same Sbsystem. De to the natre of software organization, files in the same sbsystem are generally more closely related to each other than those distribted in different sbsystems. Figre 2(c) illstrates this heristic. The symbolic name n sed inside fnction f1 is ambigosly linked to fnctions in and. Since the two modles and are in sbsystem ss1, v is determined to be correct linkage while is an anomaly. H5 Nearest Sper Sbsystem Given a nesting hierarchy of sbsystems, a modle s sper sbsystem is the one that indirectly contains that modle. In heristic H5, a nearer sper sbsystem of a modle has a higher priority to be searched than a farther sper sbsystem when a symbolic reference in that modle needs to be resolved withot ambigity. In Figre 2(d), the nearest sper sbsystem of that has the definition of the symbolic name n is S1. So, is determined to be an anomaly. 3.2 Heristics-Based Linking The following describes a heristics-based algorithm. R represents the resolved LinksTo linkage relation, and U stores the nresolved ResolvedByExtern symbolic references. H refers to the set of heristics to apply. The algorithm simply loops throgh all the heristics in H ntil no symbolic reference in U can be resolved. 1 In or stdies, we adopted the sorce directory strctre as the hierarchy of sbsystems. H = Set of Heristics R = Empty Resolved References U = Unresolved Symbolic References loop for h in H r = h(u) R = R + r U = U - (dom r) o U end for while(u is redced) In the beginning of this algorithm, R is initialized to be be an empty relation, and U contains all the symbolic references to resolve. The expression h(u) applies heristic h to U to determine correct linkage. The expression dom r retrns the domain of the binary relation r, and the expression (dom r) o U calclates the resolved symbolic references in U. This algorithm stops if no heristic in H is able to redce U. In or implementation of this algorithm, we specified an order of the five heristics discssed in Section 3.1, from H1 to H5. In addition, we removed the oter loop and applied heristics H2 and H3 after the exection of the inner for loop to change R and U accordingly. That is, we do not loop the five heristics in random bt apply them in the order of H1, H2, H3, H4, H5, H2, H3. This order gives an optimal soltion and redces linkage anomalies to a minimm degree. The order is important for the following reasons: (1) H1, H4 and H5 cannot resolve any symbolic references after their first se; (2) H3 is less accrate than H2; (3) H5 is less accrate than H4; (4) it is sefl to apply H2 and H3 again if H4 or H5 have resolved at least one symbolic reference; and (4) H2 and H3 are not effective after their second se. 3.3 Simlation-Based Linking In contrast to pre heristics-based linking, or second approach simlates the process of bilding programs and ses heristics for postprocessing. The basic idea is to instrment the bild process to collect program bild dependencies, which are then seqentially applied to gide model linking. As described in Section 2.3, dangling symbolic references may appear if dynamic or shared libraries are generated. We apply heristics H1-H5 to eliminate dangling references. We instrmented the GNU linker ld to dmp modle dependencies. For example, as the heap sbsystem in the PostgreSQL is being bilt, the following fact is dmped: SUBSYS.o : hio.o heapam.o tptoaster.o It means that three object modles are linked into a relocatable modle called SUBSYS.o. Or model linker, called ldm, is instrcted to create the linked model SUBSYS.ta based on hio.ta, heapam.ta and tptoaster.ta, 4

5 Approach Resolved Unresolved Anomalies Unconstrained Heristics Simlation Symbolic References = Resolved Cross References = Table 1. Linking Reslts of the PostgreSQL which are files representing the LPMs of the three object modles. The ldm applies the eqation given in Section 2.2. An implicit naming convention is sed to map modles to LPMs in the form of TA [5]. After processing all the bild dependencies, the ldm frther applies linking heristics to eliminate dangling symbolic references resolvable within the system. 4 Case Stdy We stdied the PostgreSQL, which is a large open sorce database system with 570 KLOC [9]. Its sorce code was dated Janary 01, For brevity, two kinds of trivial symbolic references were ignored: references resolvable within files or modles, and references to system-wide libraries sch as libc.so and libcrypt.so. We were only interested in stdying cross-references among files within the system. As shown in Table 1, the PostgreSQL system model had symbolic references and the same nmber of crossreferences. The simlation-based approach resolved references, which we verified to be correct linkage. By contrast, nconstrained linking prodced 706 anomalies and heristic linking left 43 symbolic references nresolved. This stdy indicates that heristic linking is better than nconstrained linking bt itself does not garantee a complete linkage resoltion. We now discss what impacts linkage anomalies have on the derivation of models at higher levels of abstraction. Based on the simlation reslt, we calclated 3763 dependencies among 462 modles (files) and 657 dependencies among 87 sbsystems. In the case of nconstrained linking, the 706 anomalies introdced 138 dependency anomalies at the modle level and 51 at the sbsystem level. Correspondingly, the error rates were 3.5% and 7.2%. In the case of heristics-based linking, the 43 nresolved symbolic references cased 20 missing dependencies at the modle level and 12 at the sbsystem level. The inaccracy rates were approximately 0.5% and 1.8%. 5 Conclsions This paper investigated linkage resoltion in the extraction of large software system models. We showed that nconstrained linking reslted in linkage anomalies. To solve this problem, we proposed two approaches. One is prely based on a set of heristics. The other is gided by program bild simlation and constrained by heristics. We condcted a case stdy to evalate their effectiveness. The empirical reslts showed that heristics-based linking and simlation-based linking redced linkage anomalies effectively. References [1] Acacia. The C++ Information Abstraction System Online References [2] CPPX. The C/C++ Sorce Extractor Online References. cppx, [3] R. Ferenc, A. Beszedes, F. Magyar, and T. Gyimothy. A short introdction to colmbs/can. Technical Report, [4] A. E. Hassan, R. C. Holt, B. Lage, S. Lapierre, and C. Ledc. E/r schema for the datrix c/c++/java exchange format. In Proceedings of the 7th Working Conference on Reverse Engineering, pages , Brisbane, Astralia, November [5] R. C. Holt. An Introdction to TA: the Tple-Attribte Langage. holt/cv/papers.html, [6] R. Kazman and S. J. Carrière. View extraction and view fsion in architectral nderstanding. In Proceedings of the 5th International Conference on Software Rese, Victoria, BC, Canada, Jne [7] S. Mancoridis, B. S. Mitchell, C. Rorres, Y. Chen, and E. R. Gansner. Using atomatic clstering to prodce high-level system organizations of sorce code. In Proceedings of the 6th International Workshop on Program Comprehension, pages 45 52, Ischia, Italy, Jne [8] G. C. Mrphy, D. Notkin, and K. Sllivan. Software reflexion models: Bridging the gap between sorce and high-level models. In ACM SIGSOFT Symposim on the Fondations of Software Engineering, pages 18 28, New York, NY, October [9] PostgreSQL. The PostgreSQL Database System Online References [10] TkSee/SN. The TkSee/SN Sorce Code Extractor Online References. tcl/kbre, [11] J. B. Tran, M. W. Godfrey, E. H. S. Lee, and R. C. Holt. Architectre repair of open sorce software. In Proceedings of the 8th International Workshop on Program Comprehension, pages 48 59, Limerick, Ireland, Jne [12] V. Tzerpos and R. C. Holt. Acdc: An algorithm for comprehension-driven clstering. In Proceedings of the 7th Working Conference on Reverse Engineering, pages , Brisbane, Astralia, November