Automated Program Debugging Research vs. Prac7ce?

Transcription

1 Automated Program Debugging Research vs. Prac7ce? Franz Wotawa Technische Universität Graz Ins2tute for So7ware Technology Inffeldgasse 16b/2, 8010 Graz, Austria

2 Some ques7ons asked for How usefully can these techniques be transferred from laboratory sejngs to real- life industry environments? Are automated debugging techniques actually helping prac22oners? Are the assump2ons adopted by research- oriented debugging techniques valid in prac2ce? Do we have solid data from rigorous case studies to prove the feasibility and advantage of using exi2ng research prototypes? What are the gaps between the best- of- breed prac2ces in industry and the most advanced techniques proposed in academia? What are the limita2ons and challenges of current research in program debugging and what are the most urgent needs in prac2ce?

3 (IN ADDITION) A SHORT PRESENTATION ON EXPERIENCES OBTAIN SO FAR

4 AND HERE COMES MY STORY

5 The beginnings Debugging VHDL programs / designs: Friedrich, Gerhard and Stumptner, Markus and Wotawa, Franz, Model- based diagnosis of hardware designs, Ar2ficial Intelligence 111(1-2):3-39, Franz Wotawa, On the Rela6onship between Model- Based Debugging and Program Slicing, Ar2ficial Intelligence 135(1-2): , Franz Wotawa, Debugging Hardware Designs using a Value- Based Model (ar2cle), Applied Intelligence 16(1):71-92, Kluwer Academic Publishers, 2002 Bernhard Peischl and Franz Wotawa, Automated Source Level Error Localiza6on in Hardware Designs, IEEE Design & Test of Computers, Jan- Feb, 2006.

6 .. the 7me in the middle Java debugging: Cris2nel Mateis and Markus Stumptner and Dominik Wieland and Franz Wotawa, Model- Based Debugging of Java Programs, Proceedings of the 4th Interna2onal Workshop on Automated and Algorithmic Debugging, AADEBUG '00, Cris2nel Mateis and Markus Stumptner and Franz Wotawa, Modeling Java Programs for Diagnosis, Proceedings of the European Conference on Ar2ficial Intelligence (ECAI), IOS Press, Wolfgang Mayer and Markus Stumptner and Dominik Wieland and Franz Wotawa, Can AI help to improve debugging substan6ally? Debugging experiences with value- based models, Proceedings of the European Conference on Ar2ficial Intelligence (ECAI), pp , IOS Press, 2002.

7 ..to some more recent work Switching to constraints: M. Nica and S.A. Nica and F. Wotawa, On the use of muta6ons and tes6ng for debugging, So7ware: Prac2ce & Experience, F. Wotawa and M. Nica and I.- D Moraru, Automated debugging based on a constraint model of the program and a test case, The journal of logic and algebraic programming 81(4), And spreadsheets: Birgit Hofer and André Riboira and Franz Wotawa and Rui Abreu and Elisabeth Getzner, On the Empirical Evalua6on of Fault Localiza6on Techniques for Spreadsheets, Proceedings of the 16th Interna2onal Conference on Fundamental Approaches to So7ware Engineering (FASE 2013), 2013.

8 Characteris7cs of the approach Logic or constraint model of statements and their connec2ons with other statements Based on the seman2cs of the language and at least one failing test case Guarantees to find a sound solu2on (wrt the model) Connec2ons to various other techniques Slicing Spectrum- based fault localiza2on

9 Debugging using constraints 1. begin 2. i = 2 * x; 3. j = 2 * y; 4. o1 = i + j; 5. o2 = i * i; 6. end; x = 1, y = 2, o1 = 8, o2 = 4 Programm execu7on Ab(2) i = 2 * x; Ab(3) j = 2 * y; Ab(4) o1 = i + j; Ab(5) o2 = i * i; x = 1 y = 2 o1 = 8 o2 = 4 Constraint solving / equa7on solving

10 Finding bugs using constraints Ab(2) i = 2 * x; Ab(3) j = 2 * y; Ab(4) o1 = i + j; Ab(5) o2 = i * i; x = 1 y = 2 o1 = 8 o2 = 4 Ab(2) Ab(3) Ab(4) Ab(5) j = 2 * 2 = 4 o1 = i + j = 8 = i + 4 i = 4 o2 = 4 = i * i = 4 * 4 FAIL!!!! Ab(2) Ab(3) Ab(4) Ab(5) i = 2 * 1 = 2 o1 = 8 = 2 + j j = 6 o2 = 4 = i * i = 2 * 2 And so on... finally leading to 2 possible diagnoses statement 3 and statement 4

11 Experimental results

12 Experimental results

13 Grand challenges What dis2nguishes research in (automated) debugging from debugging currently used in prac2ce? What are the difficul2es when trying to bring research into prac2ce?

14 Differences Research Simplifying assump2ons Do not consider programming environments Assume addi2onal knowledge (spec) Do not consider applica2on area (development vs. maintenance) Prac7ce External programs, libs, etc. Programmer knows expected behavior, structure, effects, Needs advice & explana2ons Full automa2on would be fine (incl. correc2on) Par2al automa2on requires addi2onal effort

15 Example There are libraries and sub- libraries etc. o7en used in prac2ce Huge amount of code to consider! General behavior might not be so difficult to handle Modeling of sub- libraries necessary Abstrac2on needed O7en not really difficult to handle (from a theore2cal perspec2ve)

16 Another example Debugging approaches need to be adapted to Programming language Programming environment Both change over 2me! Difficult to adapt for basic research (because nothing really changes) Strong need for applied research and industry engagement!

17 So what now? There is a lot of work to do! Applied research dealing with specific applica2on domains Bringing together different methods Consider the programmer as part of the process!!!

18 THE JOURNEY JUST STARTS IWPD 2014, Naples, Italy