Automated Planning for Feature Model Configuration based on Functional and Non-Functional Requirements

Transcription

1 Automated Planning or Feature Model Coniguration based on Functional and Non-Functional Requirements Samaneh Soltani 1, Mohsen Asadi 1, Dragan Gašević 2, Marek Hatala 1, Ebrahim Bagheri 2 1 Simon Fraser University, Canada 2 Athabasca University, Canada {ssoltani, masadi, dgasevic, mhatala}@suca, ebagheri@athabascauca ABSTRACT Feature modeling is one o the main techniques used in Sotware Product Line Engineering to manage the variability within the products o a amily Concrete products o the amily can be generated through a coniguration process The coniguration process selects and/or removes eatures rom the eature model according to the stakeholders requirements Selecting the right set o eatures or one product rom amongst all o the available eatures in the eature model is a complex task because: 1) the multiplicity o stakeholders unctional requirements; 2) the positive or negative impact o eatures on non-unctional properties; and 3) the stakeholders preerences wrt the desirable non-unctional properties o the inal product Many conigurations techniques have already been proposed to acilitate automated product derivation However, most o the current proposals are not designed to consider stakeholders preerences and constraints especially with regard to non-unctional properties We address the sotware product line coniguration problem and propose a ramework, which employs an artiicial intelligence planning technique to automatically select suitable eatures that satisy both the stakeholders unctional and non-unctional preerences and constraints We also provide tooling support to acilitate the use o our ramework Our experiments show that despite the complexity involved with the simultaneous consideration o both unctional and non-unctional properties our coniguration technique is scalable Keywords Sotware Product Line Engineering, Feature Model, Coniguration, Artiicial Intelligence 1 Introduction Sotware Product Line Engineering (SPLE) aims at developing a set o sotware systems that share common eatures and satisy the requirements o a speciic domain [1] SPLE decreases development costs and time to market, and improves sotware quality through strategic reuse o assets within a domain o interest A technique adopted in SPLE or managing reusability is commonality and variability modeling through which common assets and their variability are ormalized A sotware product line liecycle encompasses a domain engineering process and an application engineering process In the domain engineering process, a comprehensive ormal representation o the products o the domain is developed This includes a variability model and core assets o the product amily Feature models are among the prevalent variability modeling techniques in SPLE and represent variability in terms o dierences between the eatures o the products that belong to the sotware amily A eature is a logical unit o behavior speciied by a set o unctional and non-unctional requirements [2] On the other hand, the application engineering process is responsible or capturing target application requirements, deriving a concrete product rom the variability model through a coniguration process, and deploying the product into the users environment [3] Using eature models as variability modeling tools, the coniguration process selects a suitable set o eatures to satisy the stakeholders requirements Selecting the best set o eatures based on the stakeholders needs is a complicated process because: 1 Features may have either positive or negative impact on the dierent business concerns o a product, and hence expose dierent quality attributes We reer to business concerns o a product (eg, security and customer satisaction) and its quality attributes (eg, perormance and cost) as non-unctional properties (NFPs), where a non-unctional property is deined as: A property, or quality, that the product must have, such as an appearance, or a speed or accuracy property [9] For example, a eature may have a negative impact on security, but a positive impact on customer satisaction or it could have high perormance but low reliability 2 In addition to unctional requirements, stakeholders may have several constraints and preerences over non-unctional properties in the product derivation For example, one stakeholder may ask or a product with high security, high customer satisaction, and cost less than $1000; and can mention that the customer satisaction is more important than security The need or considering these additional requirements regarding non-unctional properties and preerences over non-unctional properties (point 2 above) leads to the increased complexity o the coniguration process Thereore, selecting a set o desirable eatures, which satisy the stakeholders requests and expectations and optimize their preerences, is a very hard task Various coniguration techniques and tools have been developed to help reduce the complexity o the coniguration task by automating some steps o it A ew existing techniques allow or the coniguration o eature models based on both unctional and non-unctional requirements [20] [25] Some techniques have addressed this problem by transorming the eature model coniguration problem into a Constraints Satisaction Problem (CSP), and have used CSP-solvers to build optimal conigurations [20] [26] The main problem with these techniques is time ineiciency Other techniques solve this problem by applying approximation algorithms, but their inal conigurations are only partially optimal [25] To our knowledge, almost all these works except [18] only support limited types o NFPs (ie, quantitative NFPs such as ootprint and cost) and do not consider qualitative NFPs (eg, security) Moreover, no work has considered the preerences o stakeholders in terms o the relative importance between non-unctional properties in the process o eature model coniguration; and relative importance varies depending on the stakeholders standpoint and application domain [6] Relative importance o non-unctional properties is especially important or the stakeholders and sotware designers who are able to deine the relative importance among the available unctional and non-unctional options but have diiculty in deterministically picking their choice rom those options [6] Thus, a product line coniguration technique should not only be able to operate over deterministic unctional choices, but should also be able to operate given the relative importance between both unctional and non-unctional properties The above issues motivated us to address the ollowing research question: How can a product be automatically derived rom a eature model in such a way that it satisies stakeholders requested

2 unctionality and at the same time optimizes their preerences and requirements over the non-unctional properties? To address this research question, we look into preerence-based planning techniques Various preerence-based planning techniques exist that produce a plan by optimizing a set o given preerences [4] [28] Hierarchical Task Network (HTN) planning is a popular planning technique, which is suited or domains with hierarchical task decomposition [28] [15] The HTN Planning technique generates plans rom a developed hierarchal network o domain tasks and actions [4] That is, having modeled tasks, actions, and their constraints in the HTN ormalism, HTN planners produce a sequence o actions that perorm some given tasks In comparison to other planning techniques, the network structure o HTN can signiicantly reduce the search space or a plan [4] Recently, requirements engineering research has employed planning techniques or preerence-driven goal-oriented requirements engineering Their reported results have shown the suitability o planning techniques or similar structures [29] By way o analogy between the eature model coniguration process and the HTN planning problem, we are motivated to investigate the applicability o HTN planning or the product line coniguration problem Hence, we hypothesize that HTN planning can orm the basis or a coniguration technique that can answer our research question We propose and develop a ramework, similar to [18], which extends eature models with annotations relecting dierent nonunctional properties We then use HTN planning [4] [5] to select a set o eatures which: 1) satisy the stakeholders unctional requirements; and 2) optimize the non-unctional requirements and preerences o the stakeholders In comparison to existing coniguration approaches, our approach has the beneit o not only satisying the structural and syntactic constraints o eature models during the coniguration process, but also taking both qualitative and quantitative NFPs as well as the relative importance over NFPs into account In our proposed approach, the coniguration problem is converted into an HTN planning problem and planning techniques are utilized to solve the problem [5] SHOP2 [5], an HTN-based planning system widely used or planning problems, is employed to identiy an optimal plan To conigure a product line, we select the eatures chosen by the SHOP2 planner In the context o eature model coniguration, the main contributions o this paper are as ollows: We introduce an easy-to-understand ormalism or capturing the stakeholders preerences over non-unctional properties represented in terms o relative importance; We produce an optimal eature model coniguration, by transorming a eature model and stakeholder preerences into a planning problem by considering both unctional and nonunctional requirements This paper is organized as ollows: Section 2 introduces eature modeling, non-unctional properties, and the HTN ormalism Next, in Section 3, the coniguration problem is ormally deined and a utility unction is developed based on non-unctional requirements and preerences Section 4 introduces the transormation rules or converting a eature model annotated with nonunctional properties into the HTN ormalism Implementation aspects and tooling are explained in Section 5, which is ollowed with a comprehensive evaluation and analysis given in Section 6 Section 7 systematically compares our approach with related works and Section 8 concludes the paper and outlines the uture work 2 Foundation 21 Feature Models In SPLE, a eature model is used mainly or representing variability between products A eature model provides a ormal and graphical representation o the variability relations, constraints, and dependencies o the product lines eatures It has a tree-like structure [8] in which eatures are typically classiied as: mandatory a eature must be included in the description o its parent eature in each coniguration o the eature model; optional a eature may or may not be included in its parent description in a coniguration o the eature model; alternative eature group one and only one o the eatures rom a eature group can be included in the parent description in a coniguration o the eature model; or eature group one or more eatures rom a eature group can be included in the description o the parent eature in a coniguration o the eature model Additionally, a number o relations are deined to represent mutual interdependencies (also reerred to as integrity constraints) between eatures The two most widely used integrity constraints are [8]: requires the presence o a given (set o) eature(s) requires the inclusion o another (set o) eature(s); and excludes the presence o a given (set o) eature(s) requires the exclusion o another (set o) eature(s) Feature models address commonality through core eatures mandatory eatures whose parents are mandatory as well A eature model can be ormally deined as ollows: Deinition 1 (Feature model) A eature model is a sextuple FM = (F,F,F M,F IR,F R,F ) where 1) F is a set o eatures; 2) F F F is a set o parent and optional child eature pairs; 3) F M F F is a set o parent and mandatory child eature pairs; 4) F IR F (F) andf R F (F) are sets o pairs o child eatures and their common parent eature grouping the child eatures into optional and alternative groups, respectively; 5) F F F is a set o integrity constraint s (ie, requires or excludes) Figure 1 depicts a Check out eature model a part o the online shopping eature model [7] 22 Non-Functional Requirements and Properties According to a commonly reerenced deinition [12], nonunctional requirements describe the properties, characteristics or constraints that a sotware product must exhibit Non-unctional properties o sotware can then encompass aspects like development constraints, business concerns, or external interaces We assume that non-unctional properties can be speciied in either a qualitative orm or a quantitative orm The qualitative nonunctional properties such as customer satisaction or user riendliness can be described using an ordinal scale consisting o a set o predeined qualitative values, which we are reerred to as qualiier tags For example, High negative, Medium negative, Low negative, Low positive, Medium positive, and High positive orm the qualiier tags deined or the customer satisaction A qualiier tag represents a possible impact o unctionality on a qualitative nonunctional property For example, the Credit card eature (ie, unctionality) has high positive impact on international sale; hence it can be annotated with the High-positive qualiier tag or international sale On the other hand, metric based values are deined or the quantitative non-unctional properties and can be measured or a product For example, perormance can be measured or and assigned to the Credit card eature Ater measuring a non-unctional value or eatures, the non-unctional property o a product is computed by aggregating the non-unctional values o eatures involved in the product Based on the nature o NFPs, dierent aggregation unctions can be applied [10] [11] To compute the non-unctional values o a product, or some NFPs such as cost and response time, the values are summed; while or others like availability and reliability values are multiplied

3 When optimizing NFPs or a product, we cannot use the same optimization technique or both qualitative and quantitative NFPs [18] For example, because we cannot deine a metric or customer satisaction, it is hard to compute which eature selection leads to a product with the best customer satisaction On the other hand, we can ind a selection o eatures, which results in a product with better perormance Figure 1b shows non-unctional properties employed in the on-line shopping product line Response time, cost, availability, and reliability are quantitative non-unctional properties and customer satisaction, international sale, and security are qualitative nonunctional properties 23 Stakeholders Preerences and Constraints Non-unctional properties do not have the same priority or the all stakeholders For some stakeholders, a subset o non-unctional properties may be more important and relevant than other nonunctional properties Moreover, in some situations, a conlict may arise between requirements deined over non-unctional properties that have an opposite behavior, in case o which the stakeholder needs to choose between the competing options One approach or speciying the priority between non-unctional properties is through ormalizing their relative importance relation Relative importance is deined as ollow [19] Deinition 2 (Relative Importance) Relative importance between non-unctional properties a and b is: i non-unctional property a is more important than non-unctional property b with coeicient α Usually, the degree o importance o options (non-unctional properties) is speciied using values 1, 3, 5, 7, and 9 corresponding to equality, slight value, strong value, very strong and extreme value, respectively [22] For example, relation represents that security is slightly more important than perormance We reer to relative importance between non-unctional properties as stakeholders preerences Prioritizing the stakeholders preerences, ormalized in terms o relative importance o non-unctional properties, can have a dramatic impact on the design o the system In addition to preerences, stakeholders may deine constraints over non-unctional properties o a system For example, a stakeholder may deine a constraint that an on-line shopping system has to be at least medium secure and cost less than $1000 Thus, the inal system should not have any unctionality (ie, eature) that provides positive impact on security less than the medium level We assume that a product has some level o qualitative nonunctional property (eg, medium security), i all o its eatures have at least that level o non-unctionality (ie, all the eatures must have at least medium security) Formally, we deine stakeholders preerences and constraints as: Figure 1: A subset o the on-line shopping systems eature model Deinition 3 (Stakeholders Preerence and Constraint Model) A stakeholder preerence and constraint model is a triple = (,,) where 1) NFP is a set o non-unctional properties; 2) RI is a set o relative importance relations between nonunctional properties (Deinition 2); and 3) is a set o constraints over the values o non-unctional properties (the values in V can be either numeric values or qualiier tags based on the type o property) 24 Hierarchy Task Network (HTN) Planning HTN planning its well with domains consisting o low level actions and high level tasks High level tasks are hierarchically reined into lower level tasks and inally into actions HTN planning consists o a planning domain, planning problem, and an output plan [4] Deinition 4 ormalizes an HTN domain [4] Deinition 4 (HTN Planning Domain) An HTN planning domain is a quadruple D = (,,M,) where 1) is a set o operators; 2) is a set o tasks; 3) Mis a set o methods; and 4) is a set o domain predicates An operator (denoted as o) represents a low level action, which can be executed in the domain and is ormally deined as a quintuple o = (name(o), pre(o), e(o), del(o), value(o)) pre(o) deines a pre-condition or every operator, which represents required conditions or perorming the operator The eect o perorming the operator can also be represented by using a post condition e(o) del(o) or operator s delete list shows what becomes alse ater perorming the operator For every operator, an optional value value(o) can be deined, which shows a required cost or the execution o that operator The total value o an output plan is the sum o the values o the operators in the plan The task construct (denoted as t) represents higher level activities in HTN and can recursively be decomposed into lower level tasks, and inally operators In HTN, only operators can be executed and tasks can only be reduced into sub-tasks and operators [15] Reinement o a task into sub-tasks is done using one or more methods (denoted as m t ) corresponding to the task So, every method deines how a task is decomposed into lower level tasks or operators A method is a quadruple M = (name(m), task(m), pre(m), dec(m)) where task(m) is a parent task, pre(m) is a pre-condition, and dec(m) is a list o subtasks into which the parent is decomposed A precondition pre(m) deines a required condition or decomposing the parent task A method is applied only when its precondition is satisied [4] The planning problem describes characteristics o a required plan the objective, initial state, and constraints The HTN planning problem is ormally deined as ollows [4]: Deinition5 (HTN planning problem) An HTN planning problem is a triple PP = (,,) where 1) is a set o logical atoms (initial state); 2) is a set o initial tasks; and 3) is a planning

4 domain description deined in Deinition 4 As a result o applying a planning technique, a plan containing a sequence o actions that satisies the objective and the constraints deined in the planning problem is produced HTN planning ormulates the plan by recursively decomposing the tasks into subtasks until it reaches the primitive tasks, which can be perormed [15] Similar to [4], we deine an HTN plan Deinition 6 (HTN Plan) Let PP = (,,) be a planning problem deined according to Deinition 5 A plan or planning problem PP is a double =(,) where 1) is a set o operators that will achieve tasks rom initial state in domain ; and 2) c is the total value o the plan 3 Problem Statement and Inrastructure In this section, we highlight the challenges o modeling nonunctional properties in product lines and optimizing them or every product derived based on stakeholders preerences 31 Modeling Non-unctional Properties In the standard eature modeling notation [1], eatures mainly represent the unctional aspects o a product line and nonunctional aspects are oten neglected For example, in the online shopping, the Credit card eature reers to the unctionality provided by the eature and no inormation regarding its quality such as perormance is provided Some researchers have extended the eature modeling notation with NFPs [20] [18] [38] Similarly, we extended eature models with the notion o NFPs, which can be either qualitative or quantitative In our approach, we assume that atomic eatures (ie, leave eatures) in a eature model have concrete implementations Non-atomic (ie, non-lea) eatures are used or variability and composition relationships o the atomic eatures Hence, NFPs are deined or lea eatures I an intermediate eature contains implementations and non-unctional properties, we create a mandatory child eature or the intermediate eature and assign the non-unctional properties to the child eature Ater identiying domain eatures, developing a eature model and implementing its atomic eatures, we can then analyze impact o eatures on NFPs For qualitative NFPs, based on existing domain knowledge, the impact o each eature on non-unctional properties can be identiied and proper qualiier tags can be assigned to each eature s non-unctional properties On the other hand, quantitative NFPs or the eatures can be measured using a suitable metric and assigned to the eatures We assume that some techniques, like those proposed in [21], can be employed to measure NFPs or each eature For example, as shown in Figure 1, eature Credit Card is annotated with low negative security, high positive international sale, and medium positive customer satisaction and its estimated cost, response time, and availability are $600, 50ms, and 90%, respectively Extended eature models are ormally deined as ollows: Deinition 7 (Extended Feature Model) An extended eature model is a nonuple EFM= (F,F,F M,F IR,F R,F, F,,) where 1) F,F,F M,F IR,F R,F stand or a eature model (FM) according to Deinition 1; 2) F F is a set o atomic eatures; 3) F ( ) is a set o pairs o atomic eatures and their annotation pairs o non-unctional properties and their value 32 Optimizing Stakeholders Preerences In the application engineering process, a concrete product is generated by coniguring a eature model based on the target application requirements and by instantiating the reerence architecture based on the conigured eature model Stakeholders o each application may have dierent preerences over NFPs that must be considered in the coniguration process Coniguring a eature model based on the stakeholders requirements and preerences usually means selecting eatures such that a eature model coniguration satisies the stakeholders unctional requirements and constraints and optimizes their preerences To optimize the coniguration with respect to preerences, eature ranks must be computed based on their impact on the NFPs which may be o dierent importance or the target stakeholders Additionally, both qualitative and quantitative NFPs must be considered in the computation o eature ranks As mentioned in Sec 2, the stakeholders preerences are in the orm o relative importance over non-unctional properties To calculate the absolute ranks o non-unctional properties based on the preerences, we applied the Stratiied Analytical Hierarchy Process (S-AHP) algorithm proposed in our previous work [14] S- AHP is based on the Analytic Hierarchy Process (AHP) [22], which is a well-known pair-wise comparison method used to calculate the relative ranking o dierent options based on stakeholders judgments We used S-AHP because it enables ranking nonunctional properties based on the deined relative importance between them; according to the study in [14], S-AHP is easy to use and does not need too much eort rom stakeholders; and inally, it signiicantly reduces the number o needed pairwise comparisons A complete description o S-AHP is outside o the scope o this paper and is available in [14] To consider qualitative non-unctional properties in eature ranks, the stakeholders or application engineers should provide a mapping unction rom qualiier tags onto real values For example, or customer satisaction, one can deine the ollowing mapping unction 1 h Μ ()= h The other way or calculating the corresponding real-numbers or the qualiier tags o a qualitative non-unctional property is to use S-AHP In this way, stakeholders speciy the relative importance between the qualiier tags o each non-unctional property and S- AHP calculates the rank o each qualiier tag For example, the stakeholder speciies that or the international sale property h, which means a eature with high positive impact on international sale is slightly more important than the eature with medium positive on international sale In both the methods (ie, both the mapping unction and S-AHP), we assume the values are normalized into the [-1, +1] range Ater deining the ranks o non-unctional properties and the mapping unction or the qualitative non-unctional properties, we describe the ollowing utility unction to calculate the ranks o each eature based on an extension to [23] The rank o eatures is calculated based on their impact on non-unctional properties by considering preerences o the stakeholders ormulated in terms o the weight o non-unctional properties Deinition 8 (Utility unction) Let us assume there are quantitative NFPs to be maximized, quantitative non-unctional properties to be minimized, and qualitative non-unctional properties whose impact needs to be maximized The utility unction or eature is deined as: R()= () + (1 () ) + (()) where w is the weight o each non-unctional property calculated by S-AHP such that 0,, 1 and + +

5 =1 and (()) returns the real number corresponding to quality tags o k-th non-unctional property and are the average values and the standard deviation o the quantitative nonunctional properties or all atomic eatures in the eature model The overall rank o a product is calculated by aggregating the ranks o eatures selected or the product The aggregation unction used or calculating the product rank depends on the aggregation unctions, which exist over non-unctional properties o a eature As discussed in Sec 2, some quantitative non-unctional properties such as response-time are additive and the quality o a composition o eatures is computed by adding up the quality o eatures [10][11] The second type o aggregation unctions deined on quantitative non-unctional properties is multiplication when the quality o composition is calculated by multiplying the quality o eatures involved in the composition The multiplication type can be converted into an additive type by computing the logarithm values o non-unctional values For qualitative non-unctional properties, a qualiier tag assigned to a eature represents a qualitative impact o the eature on the non-unctional property Considering the mapping unction that maps the qualiier tags into real numbers, we can calculate the overall impact o a composition o eatures by adding the impacts o the eatures involved in the composition Hence, the aggregation unction over the utility unctions o eatures is an additive unction and the overall rank o a product is computed as R()= R( ) In order to derive an optimal coniguration (product) P, we need to select the eatures, which maximize R() In addition to preerences, the stakeholders unctional requirements and constraints over non-unctional properties must be considered Hence, the coniguration problem is concerned with selecting eatures that satisy the unctional requirements (ie, the requested unctionality) and constraints and optimize the preerences Formally, the coniguration problem and the coniguration are deined as ollows Deinition 9 (Coniguration Problem) Let = (,, ) and EFM= (F,F,F M,F IR,F R, F,F,,) be a: i) stakeholder s preerence and constraint model; and ii) extended eature model as per Deinitions 3 and 7 A coniguration problem is a triple CP = (,,F ), where EFM is an extended eature model, SPCM is the set o preerences and constraints over NFPs deined by the stakeholders, and F F is a set o required atomic eatures Deinition 10 (Coniguration) Let CP =(,, F )and EFM= (F,F,F M,F IR,F R,F, F,,) be a coniguration problem and extended eature model, respectively A coniguration is a double COF = (F ",RF " ) where 1) F F " F is a set o selected eatures; and 2) RF " is the total rank o the coniguration Thereore, in the coniguration process, an application engineer deals with the selection o a set o eatures containing the stakeholders required unctionality The total rank o the selected eatures is the maximum rank based on the stakeholders preerences To help application engineers in the coniguration process, we propose applying an AI planning technique 4 Automatic Feature Model Coniguration Having annotated a eature model with NFPs, the process or deriving a new product starts by selecting and deselecting eatures based on the stakeholders requirements relected through desired eatures and preerences expressed in terms o relative importance between NFPs To automate the coniguration process, we deine transormation rules to convert a coniguration problem to a planning problem To do so, we develop transormation rules to represent extended eature models in the HTN ormalism The transormation is done in two steps: generating an HTN domain model rom a eature model, and generating a planning problem rom a coniguration problem When transorming a eature model and the corresponding coniguration problem into the HTN domain and HTN problem, we need to convert the maximization problem into a minimization problem, because the SHOP2 planner, used in our implementation, works on minimization only (Negating the ranks o eatures can do this) 41 Generating the HTN Domain Considering the analogy between tasks in HTN and eatures in a eature model, there is a need to deine task decompositions to relect eature relations The HTN method element is used to deine decomposition o tasks into sub-tasks or operators As mentioned in Deinition 4, a method contains the parent task, a list o children, and a pre-condition Here, we have two options: 1) we can deine dierent methods with a common parent task; but, an HTN planner (ie, SHOP2) or decomposing a task into sub-tasks selects just one method or each plan Hence, this option is suitable or mapping the XOR relation to HTN; 2) we can also deine one method or a decomposition in which all children can be considered a list o subtasks in the method In this case, the HTN planner perorms the method i and only i the preconditions o all subtasks or operators are satisied We can then use this option or deining AND-decomposition with mandatory child eatures Also, beore transorming an extended eature model into an HTN domain, a pre-processing step is done to replace optional and OR eatures First, every OR group in the eature model is converted into a set o optional eatures with AND relations between them (Figure 2a) Next, similar to [29] or optional goals, every optional eature is replaced with a new eature p which is decomposed into two alternative eatures and d where d stands or a dummy eature (Figure 2b) Figure 2: The preprocessing step or transorming eature models to HTN domains: (a) OR eature groups (b) optional eatures Ater perorming the above changes in the eature model, we transorm the eature model into an HTN domain using the ollowing three types o transormation rules: domain predicate transormation; operator transormation; method and task transormation (a) eature Model NFP QT1 QT2 QT3 (a) 1 2 (b) a qualitative NFP and its qualiier tags 2 Figure 3: Generating Domain Predicates 1 Generating Domain Predicates We generate domain predicates or each qualiier tag o the qualitative NFPs and atomic eatures in the eature model For example, the and predicates are created or high negative and high positive qualiier tags o the security non-unctional property, respectively Moreover, or eatures Credit Card and Debit Card, we generate domain predicates v cc and v dc, respectively For each non-atomic eature, a proposi o (b) Domain Predicate or Qualiier Tags: v QT1, v QT2, v QT3 1 Domain Predicates or leave eatures: v 3, v 4, v 5, v 2 Attainment Formula or intermediate eatures: ϕ 1 = v 3 v 4 v 5 ϕ = ϕ 1 v 2 o p (c) Domain Predicates in HTN d

6 tional ormula (called attainment ormula [29]) is created according to the eatures that exist in its sub-tree (Figure 3) For instance, the attainment ormula or eature Payment Gateway rom Figure 1 is ϕ PG = v AN v CS v LP v P These ormulae are later used or pre-conditions o the other eatures in the eature model Generating Operators HTN operators are generated rom atomic eatures in the eature model Each atomic eature is translated into an operator o and the rank o the eature calculated by the utility unction is translated into value(o ) A precondition is deined or each operator based on the non-unctional properties and integrity constraints deined over its corresponding eature The preconditions are deined as logical AND expressions o: 1) domain predicates corresponding to qualiier tags o qualitative non-unctional properties with which the eature is annotated; 2) an evaluation expression to check whether the eature is allowable to be selected or not based on quantitative non-unctional properties constraints; and 3) attainment ormula o eatures having requires and excludes relations with the eature (see Figure 4b) Next, the domain predicate, which corresponds to eature (ie, v ) is added as an eect o the introduced operator (ie, e(o ) = v ) Whenever operator o is perormed, its corresponding domain predicate becomes true (ie, v = true) For handling quantitative NFP constraints, a logical atom is created showing the maximum available value o corresponding NFPs during the planning process At the irst, this value is set by the requested value o stakeholders and added to the initial state o the planning problem Then, in the eect o each operator, this value is updated based on the assigned NFP value to the eature For example, or the Fraud Detection eature in Figure 1, its corresponding operator has value(o FD ) = 05 (calculated by the utility unction) and precondition pre(o FD ) = v Sech v ISl v CSh, (Maxcost- Cost Fraud-detection > 0) (MaxResponsetime Response time Frauddetection >0) The operator corresponding to the Fraud Detection eature is perormed only when its precondition is satisied 1 Required 4 QT 3 Value= Exclude QT 1, QT 3 QT 2 Value=05 Value=-15 2Cost = $98 MaxCost(Requested by stakeholder )= $1000 (a) eature Model, annotated with NFPs and eature ranks Operators or atomic eatures: o 2, o 3, o 4 pre(o 2 ) = QT 1 QT 3 v 4 MaxCost - 2Cost>0, e(o 2 ) = v2, (MaxCost = MaxCost - 2Cost) Value(o 2 ) =05 Pre(o 3 )= QT 2 v 4 e(o 3 )=v 3 Value(o 3 )= -15 Pre(o 4 )= QT 3 v 3 e(o 4 ) = v 4, Value(o 4 ) = -075 (b) Operators generated or atomic eatures Figure 4: Translating Atomic eatures into Operators Generating Tasks and Methods For every non-atomic eature, we deine a task t and method(s) (m t ) To create tasks corresponding to eatures, we use the pre-processed eature model rom which OR eature groups and optional eatures have been removed (c Figure 2) Based on the type o a eature group (ie, XOR group or AND-decomposition), we may deine one or more methods Task: t Required Exclude (a) And Decomposition and corresponding task and method Required Exclude (b) Alternative Decomposition and corresponding tasks and methods One method m or eature with Tsk-m = t Pre-m = ϕ 6 ϕ 7, Dec-m = {o 1, t 2 } Task: t Methods or task ( Alternative Decomposition) methods m 1, m 2 or task t with: Tsk-m 1 = t, Pre-m 1 = ϕ 6 ϕ 7, Dec-m 1 = {o 1 } Tsk-m 2 = t, Pre-m 2 = ϕ 6 ϕ 7, Dec-m 2 = {t 2 } Figure 5: Generating tasks and methods rom non-atomic eatures I a eature is AND-decomposed into eatures 1 and 2, we deine one method which connects corresponding task t to tasks or operators corresponding to 1 and 2, (Figure 5a) For alternative eature groups with n sub-eatures (ie, = XOR( 1, 2,, n )), n methods are deined with a common parent task t corresponding to eature Each method connects task t to one operator o i or task t i corresponding to the sub-eature i o the parent eature (see Figure 5b) The precondition or every method is speciied based on the attainment ormula o eatures required by that eature and eatures excluded by that eature Table 1 gives the mapping rules between extended eature models (ater eliminating OR and optional eatures) and the HTN domain Table 1: Mapping between constructs in extended eature models and HTN Domain models OR and Optional eature Groups are replaced in the pre-procession step Extended Feature Model HTN Domain Formal Rep Semantic Formal Rep Semantic (, ) Qualitative NFP NFP and Qualiier tags Domain predicates F Atomic eature Operator Rank o atomic Property value o ( ) value( eature ) operator Annotated eature with qualitative (,(, )) (,(,)) F F = = NFP and qualiier tag Annotated eature with quantitative NFP and its value Integrity constraints* Integrity constraints* Alternative eature group A eature with set o mandatory child >0 (= ) Domain predicate as precondition o operator Evaluation as precondition and eect o operator Pre-condition or operator Pre-condition or operator One task and n, method one or each alternative child,, One task, and one method with a set o = t sub-tasks *For simplicity in the representation, we considered the includes and excludes relations with atomic eatures, but eatures can also be non-atomic 42 Generating the HTN Planning Problem Ater deining the planning domain, the eature model coniguration problem is transormed into the HTN planning problem (Table 2 summarizes the mappings) The transormation is done by considering the constraints over the NFPs () and setting their corresponding domain predicates as true to orm initial state () For example, i a stakeholder asks or at least medium security, and a speciic cost (eg, $1000), the domain predicates = and = and the rest o qualiier tags are set to alse; the logical atom (cost 1000) is also added as an initial state Next, the set o required atomic eatures (F ) and the root o the eature model are translated into initial tasks o the planning problem () 5 Tooling and Methodology Support As already indicated, SPLE methodologies include two processes [1] [3] [8], namely, domain engineering and application engineering Assuming that a eature model has been annotated with non-unctional properties, a target application can be developed by coniguring the eature model relecting the stakeholders requirements and preerences

7 In the coniguration process, the application engineer captures the stakeholders unctional requirements including the required eature set (F ), the preerences (ie, the relative importance o non- unctional properties ), and the constraints over NFPs () By applying S-AHP and the utility unction (Deinition 8), which are implemented in our developed tool Visual eature model plugin (called Vis-mp) 1 an extension o eature model plug-in (mp) [16] the ranks o NFPs ( ) and eatures R are computed, respectively We employ an eicient planner (ie SHOP2), which uses a search-control strategy called ordered task decomposition to perorm reasoning on the HTN planning domain [15] Table 2: Mappings between the coniguration problem and the HTN planning problem Coniguration Problem HTN Planning problem Formal Rep Semantic Formal Rep Semantic, (,) F Extended eature model Selected NFPs and qualiier tag Constraint over quantitative NFPs Requested Feature Root o extended eature model ( =true) ( =) Domain description Initial state Initial state Intial tasks Initial tasks The SHOP2 planner a domain independent HTN planner is automatically executed with its required inputs (ie, HTN planning domain and HTN planning problem) The output o the planner is a set o planes (ie, sequences o operators or the given tasks), which satisy the initial conditions and have optimal value The optimal coniguration is achieved by selecting the eatures corresponding to the operators in the optimal plan The result is shown to the stakeholders in a visual view that highlights the selected eatures and shows the corresponding NFPs along with eatures Using the visual view, the stakeholders can navigate the coniguration and perorm changes in the coniguration Figure 6 shows a snapshot o a coniguration in the visual view Figure 6: The result o planner in the visual view 6 Evaluation To assess our technique and the corresponding tool Vis-mp, we ormulated the ollowing research questions: RQ1 (Scalability): Can the approach conigure eature mod- 1 els, in a reasonable time, based on unctional and nonunctional requirements and preerences? RQ2 (Eectiveness): How eective is the approach in producing a eature model coniguration? RQ2-1: Does the approach generate reliable results or application engineers? RQ2-2: What is the automation level o the approach? 61 RQ1 (Scalability) The purpose o this research question is to evaluate whether a eature model coniguration can be perormed in a reasonable amount o time Hence, we conducted several experiments to investigate the research question 611 Objects o Study To evaluate the coniguration approach, we adapt Betty FM Generator [34], which enables the random generation o highlycustomized eature models, [34] to generate eature models with dierent characteristics (eg, number o eatures, probability o mandatory and optional eatures, probability o OR and XOR groups, and percentage o integrity constraints) We set the characteristics o generated eature models as: 50%, 25%, and 25% or probability o being eatures in AND, OR, and XOR groups, respectively Moreover, 50% o eatures in AND groups are optional eatures The branching actor is also set to 10 These characteristics are backed up by most o surveyed eature models [32] [33] to relect the characteristics o real eature models Generating optimal conigurations based on the stakeholders preerences and constraints is NP-hard Hence, HTN planners similar to CSP solvers have problems in inding optimal solution or largescale problems Although the SHOP2 planner applies some heuristics or improving search time or inding an optimal plan (See re [5]), due to explicit representations o states in the memory, SHOP2 runs into memory problems or large domains However, our experiments showed or eature models with size o 200, SHOP2 returns an optimal plan in easible time According to the results o investigation on non-unctional properties done by [12], the number o relevant non-unctional properties or dierent application domains is at most 11 Moreover, Sommerville and Sawyer [13] highlighted that the eective number o non-unctional properties is around six Considering these two studies, we deined 10 non-unctional properties; six quantitative; and our qualitative non-unctional properties Five qualiier tags were deined or each non-unctional property 612 Experimental Setup For each eature model, we applied our tool to conigure the eature model based on preerences and constraints The evaluation was perormed on a computer with an Intel Core DUO 22 GHZ CPU, 4GB o RAM, Windows Vista, Java Runtime Environment v50, SHOP2 v28, and SBCL (Steel Bank Common Lisp) v1055 To conigure the eature models, we considered three independent variables including number o eatures, number o constraints, and integrity constraints; and time as a dependent variable For each eature model in the study, eatures were annotated with quantitative non-unctional properties and their values were produced by a random unction with normal distribution From a practical point o view, only some o eatures (not all) may have impact on a qualitative NFP like security Hence, to relect this point, eatures were randomly annotated with 0 to 4 qualitative non-unctional properties with normal distribution, that is, most o the eatures had one or two non-unctional properties Based on our analysis on SPLOT repository [7], which shows average o 18% o integrity constraints or the real eature models [31], we considered two distributions o integrity constraints: 10% and 20% Finally, or the constraints over NFPs, our cases were considered: no constraint and constraints over 2,4, and 6 NFPs to relect the eect o various

8 constraints over NFPs at run time 613 Experimental Results Figure 7 illustrates the average time or coniguring eatures models with dierent number o eatures and percentage o integrity constraints Our experiments revealed that, or eature models with less than 200 eatures, the planner returns results in a easible time (around 16 second) We also investigated the eect o constraints over NFPs on the running time o coniguration technique proposed in the paper To this end, we run the tool with eature models containing 100 eatures and 20% integrity constraints The results are shown in Figure 8 In all o these situations, the process o generating optimal conigurations was successul According to the experiment results, all three independent variables (ie, number o eatures, number o constraints, and integrity constraints) have an impact on running time As shown in Figure 7, or the ixed size o integrity constraints, increasing the number o eatures raises the time or inding optimal coniguration, as increasing the number o eatures expands the search space or inding optimal plan Also, within the same number o eatures, the increase o integrity constraints rom 10% to 20% causes increase the time or inding a plan, as the planner needs to check more pair combinations o tasks and operators or optimizing a inal plan Figure 7: Running time o the coniguration technique or eature models with dierent numbers o eatures and integrity constraints Finally, as illustrated in Figure 8, constraints over NFPs have a signiicant impact on increasing the time or coniguration, even i the number o eatures and integrity constraints are ixed According to the results o the experiment, the impact o constraints over NFPs is higher than the impact o integrity constraints Time(Second) Time(Second) eatures 100 eatures 150 eatures 200 eatures 10% IC % IC No constraint Constraint over 2NFPs Constraint over Constraint over 4NFPs 6NFPs (Feature models with 100 eatures and 20% integrity constraints) Figure 8: Running time o coniguration technique based on dierent number o constraints over NFPs 62 RQ 2(Eectiveness) This research question aims to investigate i application engineers can trust the results returned by the planner and i the approach is beneicial or stakeholders to acilitate their tasks Regarding the reliability o the results, our approach is based on transorming eature models into the HTN ormalisms and applying SHOP2 to ind an optimal coniguration According to the ramework proposed in [27], we can investigate i these representations are surrogate or representing eature models and preerences As shown, we can represent eature model relations and constraints using existing constructs in HTN such as method, tasks, and operator Additionally, operators provide properties to deine a eature rank The constraints over non-unctional properties and integrity constraints can be deined as precondition o operators and method Hence, we can consider HTN to be a surrogate or eature models With respect to the ontological commitment, HTN views the world as a set o actions, tasks, constraints between them; this makes it suitable or representing both eature model and constraints With respect to the planner, SHOP2 have been extensively applied in many projects [30] in government laboratories (eg, evacuation planning), industry (eg, evaluating o enemy threats), and academia (eg, automated composition o Web services), which shows the useulness o the returned plans and their corresponding conigurations One way to acilitate the application engineers tasks during the coniguration process is through automating their task With respect to the automation level, our approach requires only very ew manual interventions The main tasks o the application engineers are: 1) speciying the relative importance o non-unctional properties; 2) creating the mapping unction or qualitative nonunctional properties; and (3) speciying the atomic eatures that must be included based on the unctional requirements Computing ranks o non-unctional properties based on preerences and inding optimal solution are ully automated in our approach 7 Related work To systematically compare the approach, we devise a number o criteria that need to be supported by coniguration techniques in order to be eective or application engineering To deine the criteria or systematic comparisons, similar to [37], we applied bottom-up and top-down approaches Following the bottom-up approach, we identiied various important aspects o eature model coniguration in description o existing related works and added them to the criteria set Following the top-down approach, we used an existing survey on coniguration o sotware product lines We do not claim that this criteria set is complete, but it provides appropriate aspects to compare our work with related works These criteria include: 1) Managing unctional and non-unctional requirements; 2) Modeling stakeholders preerences; 3) Optimization; 4) Considering stakeholder constraints; 5) Providing tooling support; 6) Automating coniguration process; 7) Ensuring the eature model constraints; 8) Eective representation o results to stakeholders; 9) Time eiciency In the ollowing subsections, we review existing coniguration techniques and compare them wrt the above criteria 71 Feature Model Coniguration Approaches The irst attempt, important or our research, is by Czarnecki et al [17] who introduced a stage coniguration process In that work, a number o coniguration steps are introduced to remove variability rom eature models A inal product is developed by a consecutive specialization o the eature model through dierent specialization steps In each specialization step, some part o the eature model variability is resolved Benavides et al [20] developed an automated reasoning technique over extended eature models (ie, eature models with extraunctional eatures) Using their extension, they were able to assign extra-unctionality such as price range or time range to eatures The purpose o their technique is to ind a product o a model based on given constraints Their technique is based on mapping eature models to CSPs and use o CSP solvers [20] [35] Siegmund et al [18] have developed a technique called SPL conqueror which