How We Refactor, and How We Know It

Transcription

1 Emerson Murphy-Hill, Chris Parnin, Andrew P. Black How We Refactor, and How We Know It Urs Fässler Urs Fässler () How We Refactor, and How We Know It / 14

2 Refactoring Definition Refactoring is the process of changing the structure of a program without changing the way that it behaves. Benefit understand software improve code metrics readability complexity maintainability Urs Fässler () How We Refactor, and How We Know It / 14

3 About the study Motivation replicate old studies with bigger data sets confirm other refactoring studies contribute new refactoring detecting strategies Urs Fässler () How We Refactor, and How We Know It / 14

4 Data Source 4 data sets mainly Java developers Data set People Source Toolsmiths 4 Eclipse refactoring tool developers detailed refactoring histories Eclipse CVS Eclipse and JUnit developers extracted from Eclipse CVS Users 41 programmers in the wild captured with Mylin (Eclipse plugin) Everyone over developers Eclipse usage collector Urs Fässler () How We Refactor, and How We Know It / 14

5 Usage Data Collector - Commands yearmonth command bundleid bundleversion executecount usercount org.eclipse.ui.file.save org.eclipse.ui I org.eclipse.ui.file.save org.eclipse.ui M org.eclipse.ui.file.save org.eclipse.ui M org.eclipse.ui.file.save org.eclipse.ui I org.eclipse.ui.file.save org.eclipse.ui M org.eclipse.ui.edit.delete org.eclipse.ui I org.eclipse.ui.edit.delete org.eclipse.ui M org.eclipse.ui.edit.delete org.eclipse.ui M org.eclipse.ui.edit.delete org.eclipse.ui I org.eclipse.ui.edit.delete org.eclipse.ui M org.eclipse.ui.edit.copy org.eclipse.ui I org.eclipse.ui.edit.copy org.eclipse.ui I org.eclipse.ui.edit.copy org.eclipse.ui M org.eclipse.ui.edit.copy org.eclipse.ui M org.eclipse.ui.edit.copy org.eclipse.ui I org.eclipse.ui.edit.copy org.eclipse.ui I org.eclipse.ui.edit.copy org.eclipse.ui M org.eclipse.ui.file.revert org.eclipse.ui I org.eclipse.ui.edit.paste org.eclipse.ui I org.eclipse.ui.edit.paste org.eclipse.ui M org.eclipse.ui.edit.paste org.eclipse.ui M org.eclipse.ui.edit.paste org.eclipse.ui I org.eclipse.ui.edit.paste org.eclipse.ui M Urs Fässler () How We Refactor, and How We Know It / 14

6 Investigated aspects of refactorings and results Tool-Usage behavior differences of Toolsmiths and Users they differ kind of usage 40% in batch mode configuration 90% use default configuration Detecting refactoring commit messages don t predict refactorings changes on signatures often the signature isn t changed Refactoring practice frequency of refactorings they are frequent way of refactoring mostly mixed with other work on the code frequency of tool usage they are underused differences of tool usage wide variety Urs Fässler () How We Refactor, and How We Know It / 14

7 Hypothesis #3 Hypothesis Programmers often don t configure refactoring tools Urs Fässler () How We Refactor, and How We Know It / 14

8 Hypothesis #3 - Programmers don t configure refactoring tools analyzed top 5 refactoring tools Toolsmiths data set overall change frequency < 10% programmers don t use configurations limitations just Toolsmiths data set only 5 most-common tools Urs Fässler () How We Refactor, and How We Know It / 14

9 Hypothesis #1 Hypothesis Toolsmiths and Users differ Refactoring tool usage by groups Toolsmiths Users Everyone Rename Extract Local Variable Move Extract Method Change Method Signature Inline rest Urs Fässler () How We Refactor, and How We Know It / 14

10 Hypothesis #1 - Toolsmiths and Users differ they differ designer of software can t be used as reference limitations gathered data over different periods of time Users included non-java refactorings other aspect of study shows that refactoring tools are underused why is the rename tool so popular? Urs Fässler () How We Refactor, and How We Know It / 14

11 Discussion Tool-Usage Behavior Users underuse refactoring tools improve tools improve documentation refactoring tools are used repeatedly and with default configuration omit configuration implement batch mode Detecting Refactoring commit messages don t contain information about refactoring needs further replication many refactorings are low-level rethink refactoring detecting strategies Urs Fässler () How We Refactor, and How We Know It / 14

12 Discussion Refactoring Practice results support evidence of earlier studies refactoring tools need improvements programmers refactor frequently increase tool support refactoring is mixed with coding allow quick switches between tasks teach refactoring during courses refactoring tools are underused explore alternative interfaces identify common work-flows why are they underused? Urs Fässler () How We Refactor, and How We Know It / 14

13 Conclusions new insights how programmers refactor how refactoring tools are used how refactoring tools can be improved challenge certain assumptions about refactoring Urs Fässler () How We Refactor, and How We Know It / 14

14 Threads to validate Limitations of the Study mainly Java developers with Eclipse Users and Toolsmiths may not represent programmers in general Users and Everyone may overlap with Toolsmiths small data sets Owen thoughts to many hypotheses finding of hypothesis 1 contradicts conclusion of hypothesis 3 reflects agile software development behavior user-story driven development Urs Fässler () How We Refactor, and How We Know It / 14

15 Appendix Urs Fässler () How We Refactor, and How We Know It / 15

16 Data Source 1/2 Toolsmiths from 4 developers who maintain Eclipse s refactoring tools include detailed histories about refactoring tools which tools, when used, what configuration between December 2005 and August 2007 Eclipse CVS developer of the Eclipse and JUnit code extracted from their Concurrent Versioning System (CVS) randomly sampled from 3400 commits and search manually for refactorings many used automatic detection strategies first who compared tool logs with code histories same time period, same projects, same developers as represented in Toolsmiths Urs Fässler () How We Refactor, and How We Know It / 15

17 Data Source 2/2 Users from 41 volunteer programmers in the wild Everyone captured with Mylyn, a Plugin for Eclipse used the data already for other study latter half of % Java on average 66 development hours per programmer captured with the Eclipse Usage Collector from over developers first use of these data in research Java and others between April 2008 and January 2009 Urs Fässler () How We Refactor, and How We Know It / 15

18 Investigated Aspects 1/4 investigated 9 aspects in 3 categories Refactoring-Tool-Usage behavior differences of Toolsmiths and Users compared refactoring tool usage in Toolsmiths against Users and Everyone they differ H Toolsmiths and Users differ kind of usage Toolsmiths and Users same tool used within 1 minute (batch mode) 40% in batch mode H Programmers repeat refactorings configuration analyzed 5 most popular tools from Toolsmiths 90% use default configuration H Programmers often don t configure refactoring tools Urs Fässler () How We Refactor, and How We Know It / 15

19 Investigated Aspects 2/4 Refactoring detection commit messages searched manually in 40 commits for refactorings compared these results with the commit messages don t predict refactorings a H Commit messages don t predict refactoring changes on class and method signatures looked which refactorings don t change signature the signature often isn t changed b H Many refactorings are medium and low-level a Pure refactorings were all labeled, Some refactorings were sometimes labeled, No refactorings were often labeled as refactorings Eclipse CVS Toolsmiths b Low 18% 33% Medium 22% 27% High 60% 40% Urs Fässler () How We Refactor, and How We Know It / 15

20 Investigated Aspects 3/4 Refactoring practice 1/2 frequency of refactorings analyzed data to see how often refactorings occur Users: 41% of programming sessions contained refactoring activity Toolsmith: on average 30 refactorings each week refactorings are frequent H Refactorings are frequent way of refactoring looked for semantic changes when refactorings occurred 30% of commits contains refactorings and other work on code 3% of commits are pure refactorings most time refactoring is mixed with other work on the code H Floss refactoring is common Urs Fässler () How We Refactor, and How We Know It / 15

21 Investigated Aspects 4/4 Refactoring practice 2/2 frequency of tool usage compared manually found refactorings with tool usage data 89% were performed by hand refactoring tools are underused H Refactoring tools are underused differences of tool usage same method as before some tools are used often, some rarely H Different refactorings are performed with and without tools Urs Fässler () How We Refactor, and How We Know It / 15

22 Level of refactoring High Change the signature of classes, methods and fields Examples: rename class, move static field, add parameter Medium Change signature and significantly change blocks of code Examples: extract method, inline constant, convert anonymous type to nested type Low Changes only on blocks of code Examples: extract local variable, rename local variable, add assertion Urs Fässler () How We Refactor, and How We Know It / 15

23 CVS Commit Messages Revision Author Comment 1.83 johna Updated copyrights btripkovic clean up 1.77 btripkovic digest and pack200 tool 1.76 btripkovic icu reverse and import clean-up 1.71 btripkovic digest fix 1.68 btripkovic digest stuff 1.64 kkolosow unnecessary non nls tags 1.55 dbirsan fix support for mirrors 1.53 dbirsan Update copyright using releng tool 1.50 dbirsan Add API disclaimer 1.49 dbirsan cleanup 1.48 dbirsan remove dependency on compatibility layer dbirsan partial refactoring 1.46 dbirsan fix test failures 1.45 dbirsan backup previous change 1.42 wmelhem organized imports 1.41 droberts Copyright fix - automated changes 1.32 celek merged into celek merge core 1.30 celek Improve Search 1.29 celek Improve Search 1.28 celek multistatus.iserror + CPP 1.27 celek Refactoring logging and debugging 1.23 vlad javadoc pass 1.21 celek NL and reconciliation revised 1.2 vlad merge 2.0 changes 1.18 celek new spec (2.0.4) 1.16 celek End API review g 1.14 celek retry site and parsing defensive coding Partial history of org.eclipse.update.core.model.defaultsiteparser.java Urs Fässler () How We Refactor, and How We Know It / 15

24 Refactoring tool usage in Eclipse Name Toolsmiths Users Everyone Rename Extract Local Variable Inline Extract Method Move Change Method Signature Convert Local To Field Introduce Parameter Extract Constant Convert Anonymous To nested Move Member Type to New File Pull Up Encapsulate Field Extract Interface Generalize Declared Type Push Down Infer Generic Type Arguments Use Supertype Where Possible Introduce Factory Extract Superclass Extract Class Introduce Parameter Object Introduce Indirection Urs Fässler () How We Refactor, and How We Know It / 15

25 Refactoring tool configuration in Eclipse from Toolsmiths Refactoring Tool Configuration Option Default Value Change Frequency Extract Local Variable Declare the local variable as final false 0.05 Extract Method New method visibility private 0.06 Declare thrown runtime exceptions false 0.24 Generate method comment false 0.09 Rename Type Update references true 0.03 Update similarly named variables and methods false 0.24 Update textual occurrences in comments and strings false 0.15 Update fully qualified names in non-java text files true 0.07 Rename Method Update references true 0 Keep original method as delegate to renamed method false 0.01 Inline Method Delete method declaration true 0.09 Urs Fässler () How We Refactor, and How We Know It / 15

26 Refactorings over 40 sessions Rename Constant (H) Push Down (H) Generalize Declared Type (H) Extract Method (M) Move Member (H) Remove Parameter (H) Inline Local (L) Extract Local (L) Inline Method (M) Rename Local (L) Rename Field (H) Rename Method (H) Rename Type (H) Add Parameter (H) Decrease Method Visibility (H) Increase Method Visibility (H) Introduce Parameter (M) Reorder Parameter (H) Extract Class (H) Extract Constant (M) Inline Constant (M) Introduce Factory (H) Rename Resource (H) Manual (Labeled) Manual (Unlabeled) Tool (Labeled) Tool (Unlabeled) Urs Fässler () How We Refactor, and How We Know It / 15

27 Normalization Procedure In the paper, we discussed a normalization procedure for some reported data. To explain the procedure we used for calculating, we ll give the intuitive explanation and an example calculation below: In section 3.5, we wish to estimate how many pure-refactoring commits were made to CVS. Recall that previously, we sampled 20 Labeled projects and 20 Unlabeled projects, and we know that 6 Labeled commits were pure-refactoring and 0 Unlabeled commits were pure-refactoring. Naively, might simply do the addition (6+0) and divide over the total commits to get the estimate: 6/40 = 15%. However, this is a good estimate for our sample, but a bad estimate for the population as a whole, because our sample was drawn from two unequal strata. Specifically, in this naive estimate, we are giving too much weight to the 6 pure-refactoring commits, because Labeled commits only account for about 10% of total commits. So what do we do? Instead of the naive approach, we normalize our estimate for the relative proportions of Labeled ( 10%) to Unlabeled commits ( 90%). The following calculation gives the normalized result: ( 1 ) 290 = is the number of Labeled pure-refactoring commits. 0 is the number of Unlabeled pure-refactoring commits. 290 is the number of Labeled commits is the number of Unlabeled commits. Urs Fässler () How We Refactor, and How We Know It / 15

28 The (now more than) 72 different refactorings 1/2 From Add Parameter Change Bidirectional Association to Unidirectional Change Reference to Value Change Unidirectional Association to Bidirectional Change Value to Reference Collapse Hierarchy Consolidate Conditional Expression Consolidate Duplicate Conditional Fragments Convert Dynamic to Static Construction Convert Static to Dynamic Construction Decompose Conditional Duplicate Observed Data Eliminate Inter-Entity Bean Communication Encapsulate Collection Encapsulate Downcast Encapsulate Field Extract Class Extract Interface Extract Method Extract Package Extract Subclass Extract Superclass Form Template Method Hide Delegate Hide Method Hide presentation tier-specific details from the business tier Inline Class Inline Method Inline Temp Introduce A Controller Introduce Assertion Introduce Business Delegate Introduce Explaining Variable Introduce Foreign Method Introduce Local Extension Introduce Null Object Introduce Parameter Object Introduce Synchronizer Token Localize Disparate Logic Merge Session Beans Move Business Logic to Session Move Class Move Field Urs Fässler () How We Refactor, and How We Know It / 15

29 The (now more than) 72 different refactorings 2/2 Move Method Parameterize Method Preserve Whole Object Pull Up Constructor Body Pull Up Field Pull Up Method Push Down Field Push Down Method Reduce Scope of Variable Refactor Architecture by Tiers Remove Assignments to Parameters Remove Control Flag Remove Double Negative Remove Middle Man Remove Parameter Remove Setting Method Rename Method Replace Array with Object Replace Assignment with Initialization Replace Conditional with Polymorphism Replace Conditional with Visitor Replace Constructor with Factory Method Replace Data Value with Object Replace Delegation with Inheritance Replace Error Code with Exception Replace Exception with Test Replace Inheritance with Delegation Replace Iteration with Recursion Replace Magic Number with Symbolic Constant Replace Method with Method Object Replace Nested Conditional with Guard Clauses Replace Parameter with Explicit Methods Replace Parameter with Method Replace Record with Data Class Replace Recursion with Iteration Replace Static Variable with Parameter Replace Subclass with Fields Replace Temp with Query Replace Type Code with Class Replace Type Code with State/Strategy Replace Type Code with Subclasses Reverse Conditional Self Encapsulate Field Separate Data Access Code Separate Query from Modifier Split Loop Split Temporary Variable Substitute Algorithm Use a Connection Pool Wrap entities with session Urs Fässler () How We Refactor, and How We Know It / 15