On Modeling Software Architecture Recovery as Graph Matching. Outline. Motivation for Software Architecture Recovery. Software Architecture

Transcription

1 On Modelng Software Archtecture Recovery as Graph Matchng Kamran Sartp Deptartment of Computng and Software McMaster Unversty Canada September, 003 Outlne Motvaton and defnton for software archtecture and software archtecture recovery Issues to be addressed n a software archtectural recovery envronment Proposed approach to support reflectve and patrernbased archtectural recovery Concluson and future research drectons Motvaton for Software Archtecture Recovery Average lfe-tme of large systems s 0- years. Replacement of these systems s very expensve. Adoptng a new technology such as: object-orentaton, component-based programmng, or network-centrc requres changes n the desgn of system. Mantenance actvtes such as error-correcton and feature enhancement, nvaldate the desgn documents. Mgratng a legacy system to a new platform such as Wndows or Unx requres functonal descrpton of the system s components. 3 Software Archtecture A generally accepted defnton: The structure of the components of a program/system, ther nterrelatonshps, and prncples and gudelnes governng ther desgn and evoluton over tme [SEI ] However, software archtecture s more than components and connectors, or major elements of a system. It s a collecton of vews, patterns, stakeholders, and roles [SEI]. Therefore, Software archtecture provdes the necessary means to formalze and nterpret the propertes of a software system. Software Archtecture Recovery Extractng hgh-level structural nformaton from low-level system representaton such as source-code Major archtecture recovery technques: Clusterng [MQ-parttonng, ACDC] Concept lattce analyss [Reparng, Horzontal] Pattern-based technques [Dal, Recognzers] System vsualzaton and analyss [Pbs, Rg] Issues to be addressed by an archtectural recovery envronment What vew of the system to recover? How to represent the software system? How to model the hgh-level vew of system? What recovery technque to use? How to scale the recovery process? How to nvolve the user n recovery? How to valdate the archtecture?

2 Graph Matchng technques Exact and approxmate graph matchng technques: Comparng prmtves of prototype and nput graph. Decomposng the graphs nto smple trees to match. Generatng an state space usng cost of graph edt operatons and search for mnmum path. Envronment for Pattern-based Software Archtecture Recovery Parsng RSF Software System C / Pascal / Off-lne: pre-process AST On-lne: analyss - System analyss - Doman & Document - Decson makng? Module- Interconnecton pattern Query generaton AQL query Graph n reverse engneerng: Adopted as standard for nformaton exchange among tools. Unform mechansm for representng the software system and performng pattern matchng process. Data mnng Software as graph Archtecture & Evaluaton Graph generaton Pattern graph Graph regons & Smlarty matrx Graph matchng engne (search & evaluaton) Software system representaton Doman model for software system Abstract doman model provdes abstracton of the source-level doman model Entty-types: a subset of entty-types n source-code Relaton-type: an aggregaton of one or more relatontypes n source-code Abstract Doman model L: Fle-abs l: Fle use-r F: Functon-abs Fle-abs Id: L Integer mports: set (Entty-abs) exports: set (Entty-abs) contans: set (Entty-abs) uses: set (Entty-abs) Varable-abs Id: V Integer Type-abs Id: T Integer Functon-abs Id: F Integer usefuncs: set(functon-abs) usetypes: set (Type-abs) usevars: set (Varable-abs) use-r cont-r use-v mp-r exp-r use-t from: Fle-abs to: Entty-abs from: Fle-abs to: Entty-abs from: Fle-abs to: Entty-abs Entty-abs name: Strng fle #: Integer lne #: Integer mplement-d: Char Integer from: Fle-abs to: Entty-abs from: Functon-abs to: Entty-abs from: Functon-abs to: Entty-abs use-f from: Functon-abs to: Entty-abs Source-level Doman model f : Functon call f call f: Functon Relaton-abs fle #: Integer lne #: Integer mplement-d: Char Integer 0 Dvdng the system graph nto regons System representaton: the collecton of source-regons Source graph G s = (N s, R s ) Source-regon Data Source-regon mnng Man-seed Man-seed Query graph Archtecture Query Language (AQL) Module Interconnecton Pattern?F(3..) M Modelng hgh-level vew of system?t(0..) M?F(..) M3?T(..3)?F(..3)?F3(0..)?F(3..) M MODULE: M MAIN-SEED: func search_class IMPORTS: FUNCTIONS: func?if, func?f(3..) M TYPES: type?it, type?t(0..) M3 VARIABLES: var?iv EXPORTS: FUNCTIONS: func?ef, func?f(..) M3 TYPES: type?et VARIABLES: var?ev CONTAINS: FUNCTIONS: func $CF(.. ), func search_class (), func nhert_facts (), Nodes = {,, 0,, 3,,,, } smlarty = [,,,, 3., 3, 3, 3] exports mports module nterconnecton TYPES: type $CT(0.. ) VARIABLES: var $CV(3.. ) END-ENTITY

3 Doman model for AQL (Conceptual Archtecture) Graph Matchng Model of Recovery Conn-entty entty: Entty-abs type: Relaton-abs from: Component to: Component..n Comp-placeholders groupid: $ CL / CF / CT / CV mncont: Integer maxcont: Integer enttes: set(entty-abs) Imports: set(conn-placeholders) Exports: set(conn-placeholders) 3 Entty-abs..n..n Conn-placeholders groupid:? R / F / T / V Integer type: Relaton-abs mnenttes: Integer maxenttes: Integer enttes: set(conn-entty) from: Component to: Component Subsystem Module name: Strng manseeds: set(entty-abs) part: Comp-placeholders name: Strng manseeds: set(entty-abs) part: seq(comp-placeholders) Relaton-abs Component..n AQL-query name: Strng contans: seq(component) use-f use-t use-v mp-r exp-r Software system Fle-abs Functon-abs Type-abs Varable-abs 3 Approxmate graph matchng Dfferent types of graphs f: G à G maps the nodes and edges of G onto G. Dfferent forms of functon f: Homomorphsm: f can map two nodes of G to one node of G. Monomorphsm: f s one-to-one (.e., sub-graph somorphsm). Isomorphsm: f s one-to-one n both drectons. Exact graph matchng: Identfes exact set of nodes and edges of G that matches wth G (n most real applcatons s not feasble). Approxmate graph matchng: An optmal sequence of graph edt operatons, such as: nserton / deleton of nodes and edges of G so that G and G become somorphc. Source-graph: G s Query graph: G q Source-regon: G g() Pattern-regon: G pr Input graph: G I Pattern graph: G p sr Matched graph: G m Modelng software archtecture recovery as graph pattern matchng Gven a query graph G q = (N q, R q ) that s expanded to a pattern graph G p, gven a system graph G s = (N s, R s ), and gven a graph dstance threshold d t, the problem s to fnd a sub-graph of G s.e. G m that approxmately matches wth the pattern graph G p, so that: dst(g p, G m ) < d t & dst(g p, G m ) mn Graph algebrac model of matchng process G m - G m - m sr (R sr = g() + G ) m pr (R pr + G ) G I = G p Match At each phase of the matchng process, G I s approxmately matched aganst G p whch results n G m The graph edt operatons are performed on pattern-regon G pr and ts edge-bundles Rm pr to match them aganst sr selected source-regon G g() and ts connector-edges Rm sr G m 3

4 M M query Example: ncremental graph-pattern matchng ( phase ) M M Query graph use-f: (, ) F: (, ) F: (, 3) already matched n, 0 3 n, G m m pr pr (R G ) + = p G G m m sr sr + (R G ) = g() n,3 Pattern graph G I Match Input graph m G G m = Matched graph + (R 3 m mr mr G ) Internal-edge deleton cost Objectve: generatng hghly cohesve modules Internal-edge deleton cost must relate to: M: maxmal smlarty between two nodes n the regon s: smlarty between correspondng nodes k: number of already matched nodes n the module d: number of deleted edges between two nodes M s 0. d s c = + k k c = M s Expanded-graph c = M 0. s c = M 0. s c = M 0. s Two cases: matchng nodes and Placeholder-node to be matched 0 Imported & exported connector-edge deleton costs IMPORT - r = number of remanng edge-bundles ncludng the current edge-bundle - Keep r edges from the current edge-bundle and delete the rest wth cost zero 3- Match the edges from r edges n edge-bundle - From r edges, each edge that s not matched, s deleted wth cost: EXPORT IF one or more edges matched from edge-bundle THEN delete unmatched edges wth cost zero ELSE delete all edges wth cost: 0. x C ed n r Example: r = 3, and edges matched deleted: cost /3 x 0. Cn ed matched edge matched edge deleted: cost zero deleted: cost zero deleted edge matched edge deleted edge matched edge Current edge-bundle Example: edges matched Cost = zero Current edge-bundle Matched node Matched node Imported edge-bundles Generatng pattern-graph from query-graph mr G u Matched-regon qn u qr k: use-f (, ) Query-graphs wth nodes qn pr G Pattern-regon n, n, n,3 qn u qr k: use-f (, ) Expanded edge Matched edge qn Exported edge-bundles Query-graph wth query-nodes Generated pattern-graph at phase Edge matchng for mported edge-bundles Edge matchng for exported edge-bundle Part of pattern-graph at phase G mr u G pr n, n, = 3 n,3 n, One edge matched. Others deleted wth some cost Part of pattern-graph at phase G mr u n, G pr n, n,3 n, = One edge matched. Cost = 0 Edge bundles Edge bundles Three edges matched. Cost = max Exceeds max edges n, = Duplcate mport n, = 3 n, = Duplcate mport s not counted. Cost = 0 Three edges matched Cost = 0 n, = n, = n, = n, = Edge-bundle redrected wth cost. matched node matched node No edge matched. Redrect wth cost n, = n, = 3 n, = 3 No edge matched. Edge-bundle deleted wth cost. n, = Mn # edges may be n,3 = volated No edge-bundle deleted Cost = max n, = n, = n,3 = n, = n, = n,3 = 0 No edges matched. Edges deleted. Mn # may be volated. 3

5 ?R (0..0) Steps for ncremental pattern generaton Technques to Address Tractablty ) Select man-seed for next module usng tool provded technques. ) Recover next module wth no lnk constrants 3) Based on the nteracton wth other components, and user s objectves defne the constraned lnks for ths module. * Maxmum range s used to encourage hgh nteracton * Mnmum range s used to restrct the number of nteracton?r (0..0) u-elastc u-drag S-S S3 e-scale?r (0..00)?R3 (0..00) e-edt S S f-readtf Archtectural pattern usng AQL query Xfg system Pattern matchng 3 fles funcs R (30) 0 fles 3 funcs S-S R (00) S3 R3 () S S Recovered archtecture R (0) 3 fles funcs 0 fles funcs Incremental recovery by dvdng the search space nto sub-spaces Herarchcal recovery Decomposng system nto subsystem of fles Decomposng a subsystem nto modules of F/T/V Sub-optmal search technques, e.g., bounded path-queue A* (BQ-A*) Implementaton technques A * search wth Bounded path-queue Space Complexty Reducton Sub-optmal soluton to acheve tractable search. Root,, 3, : Sequence of expanson A * produces queue of sorted ncomplete paths. Storng, sortng, duplcate path checkng are bottlenecks. 3 Path deleted from queue Path n queue Soluton In successful search most of paths at the end of queue are not expanded. Max / mn thresholds: multples of the sze of domans. Max Mn Number of paths n queue Determned by score rato Tme Implementaton Related Technques User assstance HERE THE WAYS THAT THE COMPLEXITY REDUCDES: HOW PRESENT THE EDGE-BUNDLES CACHING THE INFORMATION OF THE SOURCE AND SINK NODES OF EDGES EXPONENTIAL COMPLEXITY WHEN SEARCH SPACE IS REDUCED TO SOURCE-REGIONS Statstcal Metrcs Overall assocaton among fles Fan-n fan-out Desgn vews Vsualzaton Smplfyng the graph vews Browsng mechansm through HTML pages Assstance wth pattern generaton Identfyng the locus of nteractons 30

6 Representng the archtecture usng graph vsualzer (Rg) rest-of-sys S-S S3 S S Archtecture of Apache.. Parttonng S S Rest-of-sys Dfferent types of lnks between boxes: Assocaton-lnks Entty-usage lnks Assocaton-lnks wth dfferent strengths to smplfy the vew Vewng the locus of nteracton among enttes to evaluate the recovery process Insght nto the system before startng the recovery Manual recovery Fle-level analyss Functon-level analyss S S3 S 3 3 Representng the archtecture usng Web browser (NetScape) Valdaton of the recovery Hypertext lnks to actual enttes n the source fle. Informaton presented ncludes: Evaluaton metrcs: modularty qualty, average smlarty Statstcal nformaton for lnkconstrant volatons Interactons among components Browsng the query Swtch between fle-level and functon-level analyss Modularty qualty Connectvty based Assocaton based User nvestgaton of the graphs Smplfed graphs Conformance wth documented archtecture Precson and Recall 33 3 Accuracy of the recovered archtecture Concluson Clps expert system 0 KLOC fles 3 functons global vars aggregate types Xfg drawng edtor KLOC fles functons 3 global vars 3 aggregate types Recovered No. of Clps subsystems subsystems fles - Defrule structures S - Inference engne S 0 - Rule manpulaton S3 - Object S - Expresson eval - System functon S 0 - User nterface rest-of-sys Recovered No. of Xfg subsystems subsystems fles edtng & S-S 3 utlty & drawng S 3 X-wndowng edtng & S3 0 utlty S 0 fle manpulaton No. of Precson Recall fles 3 % 0% 0% 3% 3 % 00% % % % % No. of Precson Recall fles % 3% e- % u- % 00% d- % w- 3 % 3% e- 0% 3% u- % f- Presented an nteractve envronment for archtectural recovery and evaluaton, and the supportng toolkt Hghlghts of the approach: Modeled the recovery process as graph pattern matchng Used data mnng technques to defne smlarty metrc Lmted the complexty of recovery process by two technques Developed a query language based on ADL features Represented the recovery result through HTML pages and graphs to be vsualzed rest-of-sys zero sze fles 3 3

7 Behavor recovery: Future drectons Extractng frequently repeated traces of event usng technques such as sequental pattern dscovery Recovery of more archtectural styles Ppe & flter Clent & Server Conformance wth standard nformaton exchange GXL A Pattern-based Envronment for Archtectural Recovery and Evaluaton Kamran Sartp Software Engneerng Group School of Computer Scence Unversty of Waterloo Ksartp@matah.uwaterloo.ca May, Web servces Systems ntegraton requres more than the ablty to conduct smple nteractons by usng standard protocols The full potental of Web Servces as an ntegraton platform wll be acheved only when applcatons and busness processes are able to ntegrate ther complex nteractons by usng a standard process ntegraton model. Models for busness nteractons typcally assume sequences of peer-to-peer message exchanges. Both synchronous and asynchronous, wthn stateful, long runnng nteractons nvolvng two or more partes 3 0 Motvaton Pattern matchng problem Clusterng problem Constrant satsfacton problem Lattce parttonng problem Composton and vsualzaton problem Motvaton Lack of a reflectve and unform model for pattern-based archtectural recovery, whereby the software system, archtectural pattern, and pattern matchng process, are all unformly represented usng a graph formalsm. System representaton: Attrbuted Relatonal Graph (ARG) An ARG s a sx-tuple G = (N, R, A, E, f, g ): N = {n, n,, nn}: attrbuted nodes (enttes) R = {r, r,, rm}: drected attrbuted edges (relatons) A & E: alphabets for node & edge attrbutes/values µ & ε: node & edge labelng functons Example of attrbutes n software system: Label: path-name and dentfer for nodes and edges Type: type of node or edge Locaton: two ntegers for fle# and lne# µ(n) = (type, Functon-abs), (name, /u/ /foo ), (d, F) ε(r) = (from, n), (to, n3), (type, use-f), (lne#, ), (fle#, ))

8 Abstract Doman Model Smlarty between two enttes based on maxmally assocated groups Abstracton of the source-level doman model Entty-types: a subset of entty-types n source-code Relaton-type: an aggregaton of one or more relaton-types n source-code Both functon-level & fle-level L: Fle-abs l: Fle f: Functon cont-r defne F: Functon-abs f: Functon Abstract Doman model Source-level Doman model F: Functon-abs F : Functon-abs use-f f: Functon f : Functon call call f L: Fle-abs F: Functon-abs use-r l: Fle f : Functon f: Functon call call f 3 Maxmally assocated group: A maxmum group of enttes (sources) sharng the same relatons on another maxmum group of enttes (snks) Source regon: Collecton of enttes that are assocated wth a regon s man-seed Smlarty between two enttes: Defned based on source and snk nodes n an assocated group Smlarty between two components: (fles, modules, subsystems) defned based on overlap between graph regon enttes and component enttes Source Snk Maxmally assocated group Man-seed Source regon Archtectural desgn of Alborz Reverse Engneerng Toolkt Software Archtecture Recovery Extractng hgh-level structural nformaton from low-level software representaton (e.g., source code) Conssts of two phases: Extracton phase: an automatc tool generates source-model. Analyss phase: a user-asssted tool constructs archtecture. Consttutes a major part of software mantenace. Should relate wth specfc re-engneerng objectves Employed recovery technques Consderng dfferent levels of abstracton Subsystem contanment herarchy to acheve Concept lattce analyss Data mnng technques Pattern based technques (graph matchng) P Clusterng technques Herarchcal Parttonng P Incremental P At fle-level the software system s decomposed nto a number of subsystems of fles At functon-level a subsystem s decomposed nto a number of modules of functons, datatypes, and varables Subsystems Software system of fles Modules Subsystem of funcs, types, vars Fle Functon, type, or varable

9 Graph dstance Mn-cost of a number of changes that are performed on graph G to transform t nto graph G Specfc characterstcs: No node / edge relabelng No node nserton snce maxmum sze of nodes expanded Node deleton s allowed up to mnmum sze A certan cost s assocated wth each graph change: Connector-edge deleton / nserton cost to comply wth pattern Internal-edge deleton cost to acheve cohesve modules