Master thesis: Automatic Extraction of Design Decision Relationships from a Task Management System

Transcription

1 Master thesis: Automatic Extraction of Design Decision Relationships from a Task Management System Matthias Ruppel, 8 th of November 2017, Munich Chair of Software Engineering for Business Information Systems (sebis) Faculty of Informatics Technische Universität München wwwmatthes.in.tum.de

2 Outline I. Introduction and Motivation II. Concepts Architectural Design Decision Quality Attributes III. OSS and NFR Dataset IV. Classification by Keywords V. Methodology VI. Results VII.Conclusion and Outlook Matthias Ruppel, 8 th of November 2017, Master thesis final presentation sebis 2

3 Introduction Motivation Many architectural design decisions are made during development & maintenance Documenting takes a lot of effort, time & costs Architectural design decisions are hard to capture Current design decisions may interfere with previous design decisions Implicitly taken, not explicitly captured & documented Rational / Cause / Concern is not evident in the documentation Matthias Ruppel, 8 th of November 2017, Master thesis final presentation sebis 3

4 Architectural Design Decision Definition A description of the choice and considered alternatives that (partially) realize one or more requirements. Alternatives consist of a set of architectural additions, subtractions and modifications to the software architecture, the rationale, and the design rules, design constraints and additional requirements. Source: Jansen, A. G. J. (2008). Architectural design decisions s.n. Source: Zimmermann et. al. (2009). Managing architectural decision models with dependency relations, integrity constraints, and production rules sebis 4

5 Non-Functional Requirements: ISO Types of Quality ISO classifies software quality in a structured set of characteristics and subcharacteristics. Each quality sub-characteristic is further divided into attributes. Source: SO/IEC :2001 Software engineering - Product quality - Part 1: Quality model. sebis 5

6 OSS and NFR Dataset NFR Dataset - Dataset of requirements of a software project, provided by PROMISE - Used by other scholars within text classification publications - 40% FR and 60% NFR - Potential issue: underrepresentation of certain Quality Attributes Data Extraction Data Curation Manual Labeling OSS Dataset OSS Dataset - Apache Spark and Apache Hadoop OSS - Public available Jira Issues - Complex and extensive open source frameworks provided for Scala, Java & Python - Limited documentation Usability (US) Security (SE) Scalability (SC) Portability (PO) Performance (PE) Operational (O) Maintainability (MN) Look & Feel (LF) Legal (L) Fault Tolerance (FT) Availability (A) Functinal (F) sebis 6

7 Classification by Keywords Quality Attribute Security Performance Keywords Confidentiality, integrity, completeness, accuracy, perturbation, virus, access, authorization, rule, validation, audit, biometrics, card, key, password, alarm, encryption, noise Space, time, memory, storage, response, throughput, peak, mean, index, compress, uncompress, runtime, perform, execute, dynamic, offset, reduce, fixing, early, late Results Source were puplication, which extracted keyword to predict quality attributes Dependent on Context Only for a few NFRs Poor performance on OSS and NFR dataset: Very low precision rate i.e. 1% (Usability with keywords from Slankas et al., (2013) Recall rate on NFR dataset is very high, with mostly 92% - 100% Keyword Matching Quality Attribute Design Decision Keyword Classification Source: Cleland-Huang et. Al. (2007). Automated classification of non-functional requirements sebis 7

8 Methodology: Text Classification Quality Attribute Feature Extraction remov authent test Machine Learning Algorithm Training Design Decision Features Quality Attribute Feature Extraction cooki token on Classifier Model Prediction Documents to Categorize Features Source: Adapted from Witten (2016). Practical Machine Learning Tools and Techniques sebis 8

9 Methodology: Feature Extraction and Selection Quality Attribute Feature Extraction remov authent test Machine Learning Algorithm Training Removing Digits & Punctuation Marks Stemming Removing Stop Words Tokenizing Text Feature Selection with Feature Extraction Information Gain Source: Own Illustration sebis 9

10 Methodology: Tokenization and Machine Learning Algorithms OSS Dataset NFR Dataset Machine Learning Algorithm SVM C4.5 Multinomial Naïve Bayes Tokenizing Text Bag of Words N-gram sebis 10

11 Methodology: Features Features OSS Dataset NFR Dataset Bag of words N-gram Bag of words N-gram remov,authent,test,add,support,upgrad,configur,unsaferow,perform,column,renam,auth,token,credenti,spee, cooki, password remov,add,authent,test,support,upgrad,perform,improv,unsaferow in, column, renam, unsaferow, support unsaferow, in,auth, support unsaferow,token,remove it, credenti, speed, improve perform,is based on, cooki, authentication mechan,the authent,http authent,password second,onli,us,no,access,with,interfac,avail,user,than,oper,minut,time,year,compli,author,easi,more,under,90 %,hour,player,allow,server,support,after,standard,respons,let,includ,updat,0,can,class,per,train,longer,regul, mainten,ensur,environ,successfulli,simultan,expect us,second,onli,user,the,shall,no,product shall b,access,with,interfac,interface with, avail, be avail, to,oper,minut,time,than,of,year,after,comply with, compli,author,updat,interface with th,easi,in und,under,allow,90%,90% of,shall interfac,hour,shall allow,shall be avail,be easi,to us,server,shall interface with,standard,and,the product must,product must,shall be easi,player,users shal,system shall let,5 second,response tim,shall let,let,be available for,available for, includ, by, 0, per, respons, train, longer, regul,class,displai,mainten,ensur,environ,for us,shall ensur,only author,shall ensure that,ensure that,easy to,be easy to,longer than,successfully,available for us,seconds th,using th,simultan,no mor,no more than,expect,expected to,have access to,have access,to successfulli,in under 5,be no mor,under 5,be no sebis 11

12 Results: Performance Evaluation NFR Dataset Bag of Words OSS Dataset Bag of Words 1 1 0,8 0,8 F-Measure 0,6 0,4 F-Measure 0,6 0,4 0,2 0,2 0 F A L LF MN O PE SC SE US FT PO 0 PO F PE MN FT O US SE A J4.8 NaiveBayesMult SVM J4.8 NaiveBayesMult SVM NFR Dataset N-gram OSS Dataset N-gram 1 1 0,8 0,8 F-Measure 0,6 0,4 F-Measure 0,6 0,4 0,2 0,2 0 F A L LF MN O PE SC SE US FT PO 0 PO F PE MN FT O US SE A J4.8 NaiveBayesMult SVM J4.8 NaiveBayesMult SVM sebis 12

13 Conclusion and Outlook - Quality Attributes (QAs) are often considered as the most important decision drivers and have a positive influence on the satisfaction of stakeholders - During the elicitation process, requirements are kept in various documents and different formats, and usually they are not properly categorized. Ø Information should kept on a central place - A framework should be used to capture design decisions Ø How could be this included into the development process of a real project? sebis 13

14 Matthias Ruppel Technische Universität München Faculty of Informatics Chair of Software Engineering for Business Information Systems Boltzmannstraße Garching bei München Tel Fax wwwmatthes.in.tum.de