Intermediate representations of functional programming languages for software quality control

Transcription

1 Intermediate representations of functional programming languages for software quality control Melinda Tóth, Gordana Rakić Eötvös Loránd University Faculty of Informatics University of Novi Sad Chair of Computer Science Aug. 27., 2013 The 13 th Workshop on "Software Engineering Education and Reverse Engineering" T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 1 / 31

2 Static Analyser Tools Motivation RefactorErl Static source code analyser and transformer Erlang SSQSA Set of Software Quality Static Analyzers Language independent T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 2 / 31

3 Static Analyser Tools Motivation RefactorErl Code Comprehension Refactoring Query Language Clone detection Metrics Dependence analysis SSQSA Clone detection Metrics Software network analysis Analyze evolutionary changes T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 3 / 31

4 Static Analyser Tools Motivation RefactorErl Semantic Program Graph, SPG SSQSA Enriched Concrete Syntax Tree, ecst T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 4 / 31

5 Static Analyser Tools Motivation RefactorErl Semantic Program Graph, SPG SSQSA Enriched Concrete Syntax Tree, ecst T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 5 / 31

6 SSQSA SSQSA T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 6 / 31

7 RefactorErl Erlang Functional Highly concurrent Distributed Fault-tolerant Supervisor Soft real-time Hot code swap Erlang VM T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 7 / 31

8 RefactorErl History Started in 2006 Software Technology Lab (4*4 ECTS) since 2008 One of the 8 labs students (BSc, MSc, PhD) ELTE-Ericsson Software Technology Lab since 2011 T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 8 / 31

9 RefactorErl RefactorErl Static source code analyser and transformer tool for Erlang Refactoring less error prone & fast Program comprehension Support in everyday work & debugging & complex tasks, e.g. Rename a function, search definition Find the value of a variable Program comprehension, component relation detection Program restructuring T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 9 / 31

10 RefactorErl (cont.) RefactorErl Platform for source code transformations 24 implemented refactorings Rename/Move definition Expression structure Function interface Structural source code analysis Clustering Support program comprehension Call graph visualisation Dependency analysis Semantic Query Language / Metric Query Language T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 10 / 31

11 User Interfaces RefactorErl T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 11 / 31

12 RefactorErl Semantic Program Graph 1 Lexical level Tokens Preprocessing Comments, whitespace 2 Syntactic level Abstract Syntax Tree Files 3 Semantic level Module, function, record, variable nodes Links to definition and reference points T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 12 / 31

13 RefactorErl -module(my). -define(eol(x), X ++ "\n"). f(s) -> io:put_chars(?eol(s)). my.erl function put_chars my modq var atom atom f io app S ++ var string S ++ "\n" f ( S ) -> io : put_chars (? EOL ( S ) ). T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 13 / 31

14 RefactorErl -module(my). -define(eol(x), X ++ "\n"). f(s) -> io:put_chars(?eol(s)). my.erl function put_chars my modq var atom atom f io app S ++ var string S ++ "\n" f ( S ) -> io : put_chars (? EOL ( S ) ). T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 13 / 31

15 RefactorErl -module(my). -define(eol(x), X ++ "\n"). f(s) -> io:put_chars(?eol(s)). my.erl function put_chars my modq var atom atom f io app S ++ var string S ++ "\n" f ( S ) -> io : put_chars (? EOL ( S ) ). T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 13 / 31

16 RefactorErl -module(my). -define(eol(x), X ++ "\n"). f(s) -> io:put_chars(?eol(s)). my.erl function put_chars my modq var atom atom f io app S ++ var string S ++ "\n" f ( S ) -> io : put_chars (? EOL ( S ) ). T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 13 / 31

17 RefactorErl -module(my). -define(eol(x), X ++ "\n"). f(s) -> io:put_chars(?eol(s)). my.erl function put_chars my modq var atom atom f io app S ++ var string S ++ "\n" f ( S ) -> io : put_chars (? EOL ( S ) ). T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 13 / 31

18 RefactorErl -module(my). -define(eol(x), X ++ "\n"). f(s) -> io:put_chars(?eol(s)). my.erl function put_chars my modq var atom atom f io app S ++ var string S ++ "\n" f ( S ) -> io : put_chars (? EOL ( S ) ). T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 13 / 31

19 RefactorErl -module(my). -define(eol(x), X ++ "\n"). f(s) -> io:put_chars(?eol(s)). my.erl function put_chars my modq var atom atom f io app S ++ var string S ++ "\n" f ( S ) -> io : put_chars (? EOL ( S ) ). T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 13 / 31

20 variable 32 S ) 13 ) llex/3 llex/1 arg ( 11 ( form/2 llex/1 atom 8 define llex/2 flex/5 variable 12 X param orig/1 ( 9 ( flex/2 flex/3 variable 10 EOL flex/4 llex/4 flex/6, 14, mref/1 ( 31 ( flex/7 body variable 15 X vardef/1 llex/3 llex/1 token subst ) 33 ) llex/1 llex/5 flex/8 ) 18 ) RefactorErl varvis/1 llex/1? 29? llex/2 flex/9 stop flex/1 llex/3 visib/1 varintro/1 llex/1 llex/2 variable 30 EOL ) 23 ) varbind/1 varref/1 token string 17 " " clex/2 -> 24 llex/1 -> orig/1 clex/3 variable 22 S name/1 atom 20 f token orig/1 esub/2 flow/1 functx/1scope/1 clex/1 esub/1 flow/1 dep/1 flow/1 atom 27 put_chars ( 21 ( esub/1 funcl/1 dep/1 esub/2 stop 35. pattern/1 : 26 : flex/1 esub/1 ( 28 ( flow/1 body/1 esub/2 esub/1 module 2 module modctx/1 visib/2 elex/2 ) 34 ) atom 25 io dep/1 form/3 flex/2 ( 3 ( fundef/1 flex/3 atom 4 my flex/4 form/1 flex/5 ) 5 ) fpar/1 flow/1 funeref/1 modref/1 fpar/1 incl/1 flex/6 fret/1 func/1 stop 6. fret/1 file/1 flex/1 func/1-1 - funcall/1 moddef/1 module/1 module/2 Example graph ROOT my.erl function form clause 1 fundef expr 1 atom f my f/1 undefined 11 fpar pattern 2 variable S expr 6 variable S S expr 8 infix_expr ++ expr 7 string expr 10 application expr 5 expr 9 infix_expr : arglist undefined 12 fret form expr 4 atom put_chars expr 3 atom io io put_chars/1 undefined 13 undefined 14 fpar fret T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 14 / 31

21 Analysis Details RefactorErl Extended syntax description Defines the representation Source for parser, lexer, and token updater Semantic Analyser framework Extensible, modular structure Works on syntactic subtrees (incremental) Asynchronous parallel execution Side-effect analysis, data-flow analysis, dynamic function call analysis T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 15 / 31

22 Mapping Mapping Attila Páter-Részeg, TDK Thesis, Scientific Student Association Conference, June 2013 convert: SG ECST convert N : SG N ECST M file form... COMPILATION_UNIT FUNCTION_DECL... T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 16 / 31

23 Simple Erlang Module Mapping -module(a). f() -> ok. T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 17 / 31

24 Mapping SPG representation for module a ROOT file/1 a.erl incl/1 form/2 form/1 module/1 function form moddef/1 flex/1 funcl/1 flex/1 flex/2 flex/3 flex/4 flex/5 flex/6 stop 12. clause 1 fundef functx/1scope/1-1 - module 2 module ( 3 ( atom 4 a ) 5 ) stop 6. fundef/1 modctx/1 clex/3 body/1 visib/1 name/1 clex/1 clex/2 a -> 10 -> expr 2 atom ok expr 1 atom f ( 8 ( ) 9 ) func/1 f/0 flow/1 atom 11 ok atom 7 f fret/1 undefined 3 fret T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 18 / 31

25 Mapping XML representation for module a T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 19 / 31

26 Mapping ecst representation for module a T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 20 / 31

27 Mapping Example: convert function (x) = y Erlang source code f( ) -> ok. Imperative source code private any f( ){ ok; } T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 21 / 31

28 Mapping Example: convert function (x) = y Erlang source code -export([f/0]). f( ) -> ok. Imperative source code public any f( ){ ok; } T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 22 / 31

29 Mapping Example: convert function (x) = y Erlang source code -export([f/2]). f(a, B) -> ok. Imperative source code public any f(any Var1, any Var2){ ok; } T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 23 / 31

30 Mapping Example: convert function (x) = y Erlang source code -export([f/2]). f(a, A) -> ok. Imperative source code public any f(any Var1, any Var2){ if(var1 == Var2) ok; } T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 24 / 31

31 Mapping Example: convert function (x) = y Erlang source code -export([f/2]). f(a, A) -> ok; f(_, _) -> nok. Imperative source code public any f(any Var1, any Var2){ if(var1 == Var2) ok; else nok; } T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 25 / 31

32 Mapping The steps of the mapping Exported public/private Return value, type any Formal parameter list new variable Pattern matching, guards branching conditions T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 26 / 31

33 Simple Erlang Module Mapping -module(a). -export([f/2]). f(a, A) -> ok; f(_, _)-> nok. T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 27 / 31

34 atom 23 f ( 24 ( variable 27 A, 26, ) 28 ) -> 29 variable 25 A -> stop 40. atom 30 ok ; 31 ; Mapping _ 34 _ atom 39 nok ( 33 ( _ 36 _, 35, ( 3 ( atom 32 f atom 4 a ) 5 ) ) 37 ) stop 6. -> 38 -> module 2 module export 14 export ( 15 ( ) 21 ) stop 22. atom 17 f ] 20 ] / 18 / integer 19 2 [ 16 [ SPG representation for module a ROOT file/1 a.erl incl/1 form/3 form/1 form/2 module/1 function form form funcl/1 flex/2 flex/1 moddef/1 funcl/2 flex/3 flex/4 flex/5 flex/6 flex/1 flex/2 flex/2 flex/3 flex/4 flex/5 flex/1 eattr/1 clause 2 fundef functx/1scope/1 clause 3 fundef functx/1scope/1 attr 7 funlist name/1 clex/1 clex/2 clex/3 clex/4 modctx/1 visib/3 body/1 modctx/1 fundef/1 visib/3 body/1 clex/1 clex/2 name/1 clex/3 clex/4 elex/2 esub/1 expr 8 atom f a expr 11 atom ok expr 15 atom nok expr 12 atom f attr 6 funref funexp/1 func/1 funlref/1 esub/1 esub/2 visib/1 pattern/1 f/2 flow/1 visib/1 flow/1 pattern/1 visib/2 pattern/2 attr 4 atom f attr 5 integer 2 pattern/2 visib/2 fpar/2 fpar/1 fret/1 varvis/1 vardef/1 undefined 18 fpar undefined 17 fpar undefined 16 fret flow/1 flow/2 flow/2 pattern 9 flow/1 variable A pattern 13 joker pattern 14 joker flow/1 pattern 10 variable A varbind/1 varintro/1 varref/1 A T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 28 / 31

35 Mapping ecst representation for module a T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 29 / 31

36 Results Results Defined Mapping Syntax driven, recursive Prototype implementation Available through RefactorErl T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 30 / 31

37 Results Future Work Refinements of the mapping Direct connection between the two tools Validating the results, example: Compare the value of metrics Evaluate the result T., Melinda, R., Gordana (ELTE-IK, PERUN) RefactorErl & SSQSA DAAD-WS 31 / 31