Presentation of Results Experiment:

Transcription

1 Presentatin f Results Experiment: Metrics & CdePr AnalytiX Grup 6: Brun Cards Felipe César Tarcísi Filó William Marcndes Empirical Sftware Engineering /1

2 Intrductin Sftware Measure is imprtant: Evaluate is Cntrl Plan changes Estimate resurces Sftware and prcess imprvement Plan evlutin test Validated Metric Tells smething useful (crrelatin,causality, etc.) Hw t Measure?

3 Intrductin Mst famus IDEs d nt feature measurement tls! (Why?) Develpers/Stakehlders want a fast and simple way t calculate them Eclipse has great & extensive plugin architecture Free! Eclipse Public License (~GPL) Active Cmmunity

4 Experiment Purpse: Evaluate and cmpare tw Eclipse plugins t cllect sftware metrics Usability Infrmatin Visualizatin Exprt f Results User Experience Metrics & CdePr AnalytiX Metrics

5 Scping Prblem: Gals: Hw specialists sample evaluate tw tls t gather sftware metrics? Are they wrth the usage? What are their strengths and weakness? Evaluate and cmpare tw Eclipse plugins fr Sftware Measurement Hypthesis: One is better than the ther? Plugins are easy t use?

6 Scping Analyze <Metrics, AnalyticX> frm the purpse f <evaluatin and cmparisn f Eclipse measurement plugins> with respect t their <usability, results visualizatin and exprt> frm the pint f view f the <develpers> in the cntext f <sftware measurement>

7 Assumptins Learnability (Nielsen(1994)) Users were nt allwed t use dcumentatin unless in case f failure perfrming the task Is it wrth the time? All cmputers desktps had the same specificatins There were n evident bias during the experiment eg. users lgging in r facebk during it

8 Design Chse plugins Get very familiar with them Explre functinalities Prepare frms t cllect infrmatin Prepare Eclipse IDE with plugins Chse target prject (JHtDraw) Set up cmputers desktps in the lab

9 JHtDraw JHtDraw is a Java GUI framewrk fr technical and structured Graphics It has been develped as a "design exercise" but is already quite pwerful Its design relies heavily n sme well-knwn design patterns. GPL License It is nt a simple and very small prject.

10 Executin - step 1 Run the plugin t extract sme metrics Time t perfrm the task Difficulty t perfrm the task Likert Scale (0 t 5)

11 Results - step 1

12 Executin - step 2 Fill a table with the values extracted fr the requested sftware artifacts Results Visualizatin Time t perfrm the task Difficult t perfrm the task Likert scale (1 t 5)

13 Results - step 2

14 Executin - step 3 Exprt the results t a XML file Acceptable way t share and cmpare results in prductin envirnment Time t perfrm the task Is there an acceptable usage threshld? Difficulty t perfrm the task Will develpers use it again if it was t hard?

15 Results - step 3

16 Executin - step 4 Overall User Experience in the use f the plugin General difficulty Likert Scale Wuld recmmend t ther users? Yes / N Relevance f the cllected metrics Yes / Yes, but miss sme (Which?) / N What did yu like best/least?

17 Results - step 4 80% 100% +Calcula autmaticamente a cada build -Usabilidade ruim +Facilidade de us -Exprtaçã ruim *Dificuldade: de 0 a 5

18 Executin prblems & Incidents Bth plugins were already installed in Eclipse, but ne pair gt cnfused, despite instructins One machine culd nt cmpute metrics (stuck) during the experiment Metrics plugin Was it a bug r a machine prblem? One f the artifacts was rdered with the wrng name in step 2

19 Prir Knwledge Eclipse IDE Never used Smetimes Well Advanced Metrics / CdePr AnalytiX Used / Nt Used Used Once Smetimes Well

20 Prir Knwledge

21 Prir Knwledge

22 Cmments Frm Participants Lack f cncern metrics was mentined in the tw plugins Metrics des nt prvide search tls and efficient navigatin AnalytiX presents great ease f use and practicality AnalytiX has difficulty in exprting the results

23 Distributin f Knwledge Metrics t 3 - Eclipse 0 t 2 - Plugin AnalytiX Sum f the knwledge f each persn in the grup

24 Cmments Frm Participants Lack f cncern metrics was mentined in the tw plugins Metrics des nt prvide search tls and efficient navigatin AnalytiX presents great ease f use and practicality AnalytiX has difficulty in exprting the results

25 Dcumentatin Supprt Step 1 Step 2 Step 3 Metrics: tw peple used AnalytiX: nne Metrics: ne persn used AnalytiX: nne Metrics: ne persn used AnalytiX: tw peple used

26 Cnclusins Results suggest that AnalytiX has imprved usability cmpared t Metrics They als suggest that AnalytiX prvides a better results visualizatin Hwever, Metrics was easier t use, prviding mre insight t exprt the results

27 Cnclusins Bth plugins shuld imprve navigatin n the results with artifacts filters In general, AnalytiX shwed t be a better and mre accepted tl Hwever, bth plugins shwed ease f use. All pairs cmpleted the task in a relatively shrt time, even with limited knwledge in Eclipse

28 Limitatins What is a representative sample in Sftware Engineering? Des using students bias the results? Statistical Tests Hard t perfrm Inferential Statistics Only Descriptive Statistics Culd have used muse click cunter, fr example

29 References Curse Website Ntes frm C. Whlin et al. Experimentatin in Sftware Engineering, Springer Jeff Offutt - Gerge Masn University Jakb Nielsen Website