Event Driven Architecture in software development projects

Transcription

1 Master Thesis Computing Science Event Driven Architecture in software development projects Author: Maxime Klusman Supervisor Radboud University: Prof.dr.ir. Rinus Plasmeijer Supervisor Sogyo: Drs. Ralf Wolter Radboud University, Nijmegen April 8, 2016

2 Table of Contents 1. Introduction Problem statement Relevance Research goal Research method Scope Thesis outline Introduction to software architecture History and definition Architectural representation Architectural views Introduction to Event Driven Architecture 9 4. Selection of quality criteria Long-list of quality criteria Exclusion of criteria Functionality Theoretical assessment of quality criteria Relevance exclusion Short-list of quality criteria Methods of measurement Efficiency Structuredness and Understandability Audit trail Maintainability Case study: Set-up Way of working Case description Functional requirements Case study: Original architecture Process view Development view Problems and Hypotheses Timeliness and Scalability

3 Maintainability Case study: New architecture Process view Development view Extent of implementation Module descriptions EventStore module SAPImport module Gateway module ViewModel modules Design notes Top-level design change Versioning system Entity-Relationship differences Case study: Measurement methods and results Change-scenarios Scenario Scenario Code metrics Lines of Code Cyclomatic Complexity Class Coupling Depth of Inheritance Maintainability Index Time measurements Resource measurements Case study: Interpretation and comparison Efficiency Time behaviour Resource behaviour Structuredness and Understandability Conciseness Consistency Modularity Simplicity Self-descriptiveness Analysability and Diagnosability

4 9.3. Audit trail Maintainability Repairability Modifiability and Extensibility Scalability Testability Conclusions Generalization from used Messageframework Differences attributed to use of messages Differences attributed to asynchronous communication Differences attributed to event-characteristics Recommendations Future work Bibliography 81 Appendices 84 A. Definitions of quality criteria 84 B. Full measurement data Introduction This section gives the motivation for doing this research project and shows its relevance with respect to earlier conducted research. It then presents the research goals and chosen methods to achieve these goals. Subsequently, the scope of the project is outlined by describing some decisions that were made about what is and what is not included within this project. Finally, the organisation of the rest of this thesis is explained Problem statement In computer systems, the standard communication method is request-response, where one part of the system instructs another part to perform some action or retrieve certain data by calling one of its methods [1]. The traditional way to design an information systems is to use this communication style exclusively throughout the program (for example using an Object-Oriented design) and combine it with a relational database to store the data. Alternatives to this approach are lesser known and may therefore 4

5 be overlooked, even if they would fit better. One of the possible alternatives is Event Driven Architecture (EDA) [2]. EDA is an architectural software pattern concerning the communication methods in a system. It differs from a request-response approach, like a completely Object-Oriented approach, in that it separates the system into units that communicate through events instead of method calls. This research project was performed at software company Sogyo [3]. Sogyo has been doing software project since Since 2002, events have played a role in their development, but complete EDA was only experimented with once for an internal application. From this experiment, clear benefits of EDA were observed, which lead to the hypothesis that using EDA in the small to medium sized information systems could have substantial benefits, especially in the area of post-developmental changes and extensions to the system. However, more research was needed for Sogyo to gain enough confidence and experience to use this approach in external projects. EDA has already been shown to work well for inherently event-driven applications. For example, applications where constant input from sensors is used to make decisions and invoke actions, like in [4] and the experiment at Sogyo. This project investigates whether the noticed benefits of EDA carry over to standard information systems, where EDA would not immediately be an obvious choice. Based on the results, members of future development projects can better assess which architecture suits their requirements the best Relevance Event-driven applications have already been around for a longer time, but have recently gained more popularity [5], especially in combination with Service Oriented Architecture (SOA) for enterprise-wide applications [5, 6]. In this project, EDA is studied in a different setting. Instead of enterprise-wide applications, we look at the development of custom software for small to medium size information systems. Also, EDA is studied on its own instead of in combination with SOA. Empirical research on EDA in this setting has not been found in current literature Research goal The goal of this research project is to determine the advantages and disadvantages of using EDA for a development project as opposed to a more traditional approach, and to produce some general recommendations about the use of EDA based on these findings. 5

6 Specifically, the development projects targeted in this research are those for small to medium sized information systems. By the traditional approach, we mean an architecture where communication is done synchronously and through commands/requests and replies. Since the currently most used method is an Object-Oriented design, this is taken as the more specific traditional alternative to compare against EDA. The differences between these approaches have multiple aspects, which is distinguished in the conclusions Research method As the main research method, we perform a case-study. This case-study involves the comparison between systems that are equivalent in terms of functional requirements, but use different software architectures. One has the standard, request-response (Object- Oriented), approach and the other has an EDA approach. For the request-response system, a pre-existing system made by Sogyo is used. The EDA system is designed and partially implemented specifically for this research project. The two systems are compared on a selected set of quality attributes. The selection process involves the collection of a broad set of possible criteria and then narrowing them down by relevance until a manageable set is obtained. The selection process is described in detail in section 4. For the comparison of the two systems on these quality attributes, quantitative measurements are combined with qualitative observations. To get conclusions that fulfil the research goals, the results are generalized where possible Scope To keep an appropriate scope for this research project, choices were made about what to include and and what to leave out. The first scoping choice is to investigate a single case study in this project. More cases would give a better foundation for generalized conclusions, but including more than one was not feasible due to time constraints for this project. Also, for the comparison done in the case study, a selection of the most relevant quality attributes was made (see section 4), instead of considering all of them. Another scope-limiting choice was one regarding the extend of the implementation (see section 7.3). Making an implementation that covers more of the architecture would 6

7 mean more data for a comparison and less extrapolation. On the other hand, a larger implementation would take up more time, which then cannot be spent on other aspects of the research project, so a trade-off needed to be made. Therefore, it was decided to build the implementation incrementally; the size was increased bit by bit until the point where the added value was expected to be low compared to the extra time and effort required. This way, results from the partial implementation could be generalized with high confidence, without needing to spend an excessive amount of time on the implementation. When selecting the change-scenario s to use in the comparison (see section 8.1), some scoping decisions also had to be made. More change-scenario s would provide more comparison data. Initially, two scenario s were selected. They proved to be time-consuming, so due to time constraints, no more were added. Finally, the theoretical research consisted of a literature study about quality attributes, background research about software architecture and most importantly, an extensive study on the topic of EDA Thesis outline Since the subject of this project is software architecture, an introduction on this subject is given in section 2. Section 3 introduces EDA. Subsequently, in section 4, the selection of a suitable set of quality criteria is described. The case study is described in sections 5-9. First, section 5 gives the general overview of how the case study is conducted and which system is used for it. Next, sections 6 and 7 give a description of the original architecture and the newly designed event-driven architecture respectively. Section 8 presents the results of the measurements made on both systems and section 9 gives an interpretation of these results and compares the two systems. In section 10, the conclusions of the case study are generalized to give overall conclusions and make recommendations about the use of EDA in development projects. Finally, appendix A gives definitions of quality criteria used in section 4 and appendix B presents the complete measurement data. 2. Introduction to software architecture In this section, the concept of software architecture is introduced. Additional focus is placed on how architecture can be described, since this is needed in this research 7

8 project History and definition In the late 1980s, the term software architecture was introduced to denote large-scale structures of software systems. This was the result of software intensive systems becoming increasingly larger and therefore their internal structure becoming increasingly important [7]. Software architecture extended other concepts like information hiding and modularization [8]. Initially, the area of software architecture largely existed of qualitative descriptions of structural principles that had proven useful in practice. Since then, the field has grown to envelop a broad set of notations, tools, and analysis techniques [8]. Nowadays, there is also a variety of different structures that are referred to as software architecture by different people. It is a generally acknowledged fact that there is no standard definition of Software Architecture [9]. The often cited Bass et al. [10] define it as follows: The software architecture of a program or computing system is the structure or structures of the system, which comprise software elements, the externally visible properties of those elements, and the relationships among them. [...] An architecture is foremost an abstraction of a system that suppresses details of elements that do not affect how they use, are used by, relate to, or interact with other elements. According to this definition, every software system has an architecture, because a system always consists of elements and interactions. This even holds in a trivial way for a system consisting of a single monolithic unit. The existence of an architecture is independent of whether it is described or documented in any way, or given any consideration in the development process. A description of the architecture and explicit consideration of its design can be of great value [10], for example when maintaining the system after its initial development. This importance of architecture increases for more complex systems Architectural representation An architectural representation provides a way to describe and analyse the high-level properties of a complex system [7]. This gives developers a way to reason and communicate about these high-level properties. It is also a basis for communication with stakeholders and can serve to get a better grasp on the requirements of the system. Furthermore, the architecture often forms the foundation for the realisation of desired 8

9 quality attributes [10, 11]. When it is considered explicitly, this provides opportunities to evaluate these quality attributes to some extent early in the development process and make changes if needed. The architecture consists of properties of the system that are hardest to change later on, so explicit consideration of architecture early in the development process is very important Architectural views What was originally often contained in the concept of an architectural description, were different types of information that became mixed in the architectural representation. For example, the division of the system into smaller parts can be done in terms of the development modules (e.g. division into projects or, finer grained, division into classes or functions) or in terms of runtime units (e.g. division into applications or, finer grained, division into runtime processes) [10]. When these different types of information are mixed together into a single architectural description, it easily gets confusing. To solve this, the concept of views was introduced; each view represents one type of information in a separate description. A well-known example of this separation is the 4+1 view model from [12]. The 4+1 model describes software architecture using five concurrent views. In it, the descriptions of the architectural decisions are organized around four views: logical, process, physical and development, which are illustrated with a few selected scenarios (the fifth view). In an architectural view model, the views are never completely independent. They have certain relationships and connections, which should also be included in the overall architectural description. The architectural descriptions in sections 6 and 7 make use of a suitable set of views. A complete 4+1 view description would be excessive here, so only the process and development views from the 4+1 view model are used. The architectures are described in terms of the individual views, as well as the connections between them. 3. Introduction to Event Driven Architecture In this section, the architectural pattern Event Driven Architecture (EDA) is described. The description includes theoretical strengths and weaknesses found in literature. These are used in section to assess quality criteria on theoretical grounds, and in section 6.3 to make concrete hypotheses about the results of the use of EDA in the casestudy. 9

10 EDA is a software pattern that can be used when designing the architecture of a software system, it deals with communication within the system. In EDA, the system is divided into a set of so-called modules. This division is primarily on an executional level, dividing the system into separate runtime units. Between these modules, the communication is achieved by event notifications ( events ). These events often take the form of messages. An event is a record of something notable that happened within the scope of the system [2, 6]. It is generated by an event source and broadcast into the system [1]. Each module can define which events should be acted upon. In the interested modules, the event triggers some action, while other modules simply ignore it. The nature of events is descriptive; they convey what happened, but do not in any way specify what actions should be taken as a result, or by which or how many modules [1]. Modules do not have any information about each other. The principles of EDA dictate that a sending module should have no knowledge of the way its events are processed. The knowledge that is shared between modules, is about the events themselves: the types of events that are sent in the system, what kind of information they contain and in what form this information is present. This has some consequences. First of all, it makes it possible for the modules to be extremely loosely coupled [2, 6], which is often a desirable quality. Also, it makes EDA work very well with asynchronous flows of work and information [2]; if receiving module(s) of an event are busy, there is no inherent need for the sending module to wait. The advantages of this are that modules have to spend less time waiting and that there is potential for horizontal scaling; work could be divided between different copies of the same module. On the other hand, it decreases the amount of information and assumptions that can be made within modules. Additionally, some amount of synchronization may be required in a system, which then needs to be handled explicitly in an EDA system. The EDA set-up also means that new information is pushed around the system rather than provided on demand. Consequently, updates are made in reaction to changes, which is useful in many systems. It means there is no need to poll for any changes elsewhere in the system, which can improve timing properties. On the other hand, it also means modules that need information on demand, need to keep a copy of it themselves. This increases the overall storage load. Finally, in an EDA system, actions are less sequential. This means it is suitable for situations where a change has multiple consequences, which can be processed more or less independently of one another. It also means the total behaviour of the system is likely to be more difficult to analyse, since the order in which things happen is less defined and can differ between equivalent runs of the system. 10

11 4. Selection of quality criteria This section describes the process of selecting a suitable set of quality criteria to use when comparing the two case study systems. The following steps are used in this process: (a) collecting a broad set of possible quality criteria from literature, (b) narrowing down this list to a feasible set of relevant criteria, (c) defining the way each remaining criterion is measured. Section 4.1 defines the result from step (a) in the form of a long-list of quality criteria. Section 4.2 follows up with an explanation of the process of step (b). The result of this step is shown in the form of a short-list in section 4.3. Finally, section 4.4 elaborates on step (c) Long-list of quality criteria The starting point is a list of possible criteria that can be used in general to determine the quality of software and software architecture. To make this list, information about quality attributes that can be used to evaluate software systems and architectures is collected from the following sources: [13, 14, 10, 15, 16, 17, 18, 19, 20, 21]. The collected information is then combined and aggregated to produce a custom grouping, shown in table 1. Only the entries marked ( ) are included in the short-list. The rest are excluded for various (numbered and color-coded) reasons, which are further explained in section 4.2. Table 18 in appendix A shows the definition of each quality attribute, as well as the direct source it was adapted from. The quality criteria, their grouping, and their definitions are already somewhat modified towards convenient use in subsequent steps, but are still included in a way that is relatively close to the source materials. As in the source materials, a group of sub-criteria does not always form a complete description of its main criterion. Also, some criteria have overlapping definitions. For example, Maintainability contains a sub-criterion of Evolvability, which overlaps with Modifiability and Extensibility. Also, the notion of Scalability, the ability to improve a system s performance by adding more resources, is part of Maintainability but missing from the list of sub-criteria. For the short-list (see section 4.3), the remaining criteria are further modified to make them more suitable for measurement on real systems. 11

12 (1) Functionality (1) Traceability (1) Suitability (1) Appropriateness (1) Correctness / Conformance (1) Completeness (5) Security (5) Accountability (4) Authenticity (4) Integrity (5) Non-repudiation (4) Confidentiality / Privacy (5) Compatibility (4) Co-existence (5) Interoperability ( ) Efficiency (5) Capacity ( ) Performance / Timeliness ( ) Resource Behaviour (2) Usability (2) Operability (2) Appropriateness Recognizability (2) User Error Protection (2) Attractiveness (2) Communicativeness (2) Learnability (5) Reliability (4) Robustness (3) Maturity (3) Accuracy / Precision (5) Fault Tolerance (5) Recoverability (5) Availability (5) Reusability (5) Generality (5) Self-containedness ( ) Structuredness ( ) Consistency / Uniformity ( ) Conciseness ( ) Modularity (4) Portability (4) Machine Independence (4) Replaceability (4) Installability (4) Adaptability (4) Software System Independence ( ) Maintainability ( ) Testability ( ) Modifiability ( ) Extensibility / Augmentability ( ) Repairability ( ) Evolvability ( ) Audit trail ( ) Understandability (3) Legibility ( ) Analyzability ( ) Simplicity ( ) Self-descriptiveness Table 1: Long-list of quality criteria 12

13 4.2. Exclusion of criteria A portion of the quality criteria are excluded for the comparison performed in this research project. The different reasons for exclusion are numbered and color-coded in table 1. First of all, the functional criteria marked (1) are excluded. This is explained in section Additionally a number of non-functional criteria are also excluded. They are first filtered on a theoretical basis; for each criterion it is determined whether or not it is affected by a change from a request-response architecture to EDA. This is done based on the following three arguments: (2) Criteria that involve Usability focus on the user experience. This only involves the external behaviour of the system and is therefore independent from the architectural structure [22]. Hence these criteria are irrelevant for the comparison that we want to make in this project. (3) A few criteria consist only of properties of the implementation (choices made on the programming level) and are therefore independent of the architecture. (4) Criteria may be dependent on whether the architecture has specified certain features, but if these features are independent of the internal structure, they are not relevant for the current comparison; they could be added to or removed from a system with either architectural structure. The exclusion of criteria based on (2) and (3) is fairly straightforward and is not further explained. The assessment of the remaining criteria regarding (4) is explained in section Finally, some of the remaining criteria are of less interest to Sogyo with respect to this research topic. These are excluded to keep a manageable scope for this project, and are marked (5) in the table. The considerations that lead to these choices are clarified in section Functionality In this project, functionality plays a slightly different role than the other, non-functional, quality attributes. Bass et al. [10] describe the relationship between functionality and architecture as follows: Functionality may be achieved through the use of any of a number of possible structures. In fact, if functionality were the only requirement, the system could exist as a single monolithic module with no internal structure at all. 13

14 Instead, it is decomposed into modules to make it understandable and to support a variety of other purposes. In this way, functionality is largely independent of structure. Similarly, the functionality that can be provided by the use of the principles of either EDA or request-response, can be shown to be the same. The most fundamental difference between the two is the semantics, which is mainly reflected by how things are named. However, these choices only influence the way people think about and understand the concepts used in a system and do not limit the possible functionality in any way. Another, more concrete difference that often (but not necessarily) occurs between the two principles, is synchronous versus asynchronous communication. EDA combines well with asynchronous communication, because the sender is not responsible for how, where and when the event is received and can usually continue execution as soon as it has given off its event notification to the underlying framework. These two communication methods can also be shown to be equivalent. As explained by [23], asynchronous communication can be modelled by synchronous communication by introducing a buffer. This buffer means the message can be sent at any moment by the sender, and is stored in the buffer. The receiver can collect it whenever it is ready to, without the sender having to wait. The other way around, as mentioned by [24], synchronous communication can be modelled by hand-shaking of asynchronous processes. This means that in terms of functionality, the two principles are interchangeable. A formal method of conversion from request-response to EDA is unlikely to provide major benefits in terms of non-functional quality attributes. To achieve real benefits, the architecture needs to be reconstructed with an event-driven mindset. Along the same lines, Bass et al. [10] add: Software architecture constrains [the allocation of functionality] to structure when other quality attributes are important. [... ] The interest of functionality is how it interacts with, and constrains, those other qualities. This view corresponds to the importance of functionality in this project. It does not lie in how well functionality requirements can be achieved through different architectures, but rather it lies in the way non-functional quality attributes are influenced by the architectural method chosen, when constrained by the functional requirements. For the purpose of this project, this means the functional attributes from table 1 are not appropriate as quality criteria. Instead, functional requirements being satisfied is a prerequisite to start assessment of non-functional criteria for the architecture. 14

15 Theoretical assessment of quality criteria The following paragraphs explain for each criteria group the reasons why the contained criteria are or are not excluded on theoretical grounds. Portability and Compatibility All criteria in the Portability group have to do with the system s independence from its environment. The same holds more or less for the Compatibility criterion Co-existence; this criterion restricts the way the system influences parts of its environment and resources that it shares with other systems. The environment that is meant here is the environment of the system as a whole. This environment is for example an operating system or a web browser. The mentioned criteria view the behaviour of the whole system with respect to factors that originate from outside of the system. On the architectural level, satisfying the above mentioned criteria requires mechanisms, that act as a layer between the core of the system and some part of its environment, to be specified by the architecture. Examples of such a mechanism for the criterion Adaptability are genericity and parametrization [22]. The mechanisms depend on technical details of the environments and the system itself. They are not dependent on the principle behind the communication method (events versus request-response). This is because the communication method between the layer around a part of the environment and the part of the environment itself does not qualify as communication between modules of the system. Therefore it is not restricted and can be the same in both architectural methods (EDA and request-response). The communication method used on the boundary between the layer around the environmental component and the rest of the system is restricted, but here the only thing that is important is the content of the information that is exchanged and not the semantic form. Since both communication method are able to convey the same information (see section 4.2.1), they are equivalent in this regard. Therefore the criteria in this group do not provide a good basis for comparison of these architectural communication principles. The same argument does not hold for the Compatibility criterion Interoperability. Here, domain information needs to be exchanged between systems. The technical details of the information exchange are in principle independent of EDA. However, the exchange of information between systems may be linked to the internal exchange of information of the system. If a pair of interoperating systems do not share a communication principle (either event- or request-response-driven), a semantic translation may be needed before the systems can communicate effectively. On the other hand, if both systems adopt EDA, the communication could be achieved with relatively little effort by listening to one another s events. This criterion therefore is theoretically relevant in the comparison 15

16 between communication methods in this project. Security Some features of Security (Non-repudiation, Accountability) rely on the ease of keeping an audit trail, discussed above. Authenticity and Integrity, on the architectural level, require a mechanism for authentication to be present which verifies that entities are who they say they are. This mechanism can be completely encapsulated within a single module and is therefore not dependent on communication method between modules. This makes the quality criterion Authenticity irrelevant for this project. Integrity additionally requires the system to keep track of the verified identity of an entity and provide or deny access to certain features based on it. This relies on implementation details, but can also be dependent on communication between modules. For example a module may be required to know which user is logged in to provide or deny access to some data. However, this information can be conveyed with either communication strategy, so this criterion is also not relevant for the comparison. Confidentiality / Privacy can be seen as a sub-criterion of Integrity, so the same reasoning applies. Reliability The Reliability criterion Robustness is concerned with the behaviour of the system under conditions that are not explicitly specified by the specification. This means, for example, how the system handles inappropriate user input. Even if these conditions are not explicitly specified by the specification, the way they are handled in the architecture is the same as for specified conditions. Therefore, where architecture is concerned, Robustness can be treated as a sort of extension of functionality and is independent of communication strategy. As a result, this criterion is excluded from the list. The Reliability criteria that have not been excluded up to this point, are dependent on communication method. This is because in a system with EDA, redundancy can be achieved by having multiple run-time modules that provide the same functionality (implemented either in the same or a different way). None of the other modules need to be changed because of this. If one of these redundant copies fails, the other(s) can provide the functionality to keep the system running correctly. Also, if all events in the system are recorded, the system state can be recovered after a failure by reconstructing it from these events [1]. 16

17 Efficiency The Efficiency criteria are all dependent on the use of EDA. A system with EDA is well suited for asynchronous communication and therefore also for parallelization [2]. Both can greatly enhance performance. Additionally, with EDA, state keeping is usually handled in a significantly different way.[1] Instead of requesting data from one central source each time it is needed, all modules that need access to certain data keep a local copy. Whenever the source experiences a change, it generates a corresponding event. All modules that keep a copy of the affected data, respond to the event by updating their copy. Whenever these modules need access to a piece a data, they can access it locally. This form of redundancy means that the resource usage for an EDA system is higher. The performance and capacity are higher, because modules never have to wait for response from the source. Therefore the data source never becomes a bottleneck even when many other modules need its data at the same time.[1] Audit trail For a system with request-response, setting up an audit trail during run-time of a system requires extra logging effort. In a system with EDA, it is easy to track and analyse all interaction between modules by capturing all events that are created in the system [1]. This information can be used on its own or in combination with other logging information. This means this criterion is dependent on the communication method and therefore theoretically relevant in the comparison. Structuredness EDA demands a certain degree of Structuredness. The division of the system into modules that only communicate through events, makes the modules very loosely coupled [2, 1], which provides a high Modularity. It also provides top-level architectural Consistency. Conciseness could potentially also be heavily influenced by the choice in communication strategy, because certain functionality may be expressed more easily and with fewer code in one strategy compared to the other. These criteria are therefore all relevant in the comparison. Understandability All main architectural choices naturally influence the Understandability of the whole system. For EDA specifically, the control flow may be harder to distinguish, since in the source of an event, it is not known what responses it triggers [2, 7]. On the other hand, in 17

18 an EDA system it is easier to track and therefore analyse all interactions that have taken place between modules (see Audit trail, page 17). Also, events tend to be descriptive, so they inherently convey information about their purpose. Understandability criteria (aside from Legibility, which is defined on the implementation) are therefore relevant in the comparison. Maintainability Maintainability concerns the ease for a system to be changed with respect to different aspects. The reparation of faults (Repairability) and changes in functional features (contained in Modifiability, Extensibility, Evolvability) are mainly dependent on the criteria described in Structuredness, page 17 and Understandability, page 17. The ease of verification of changes in the system (Testability) is also dependent on Structuredness because highly independent modules can be tested separately. Additionally, it is dependent on the ease of keeping an audit trail (Audit trail, page 17). This is because test cases can easily be constructed from recorded sequences of events. It is further dependent on the choice in communication method, because EDA allows for less static validation. As a result less errors can be found at build-time, so they have to be found through run-time testing.[1] An increase in desired capacity or throughput of the system (contained in Extensibility, Evolvability) is dependent on parallelization and possibility for distribution, which are both influenced by the chosen communication strategy [2]. Together this means all of the Maintainability criteria are relevant in the comparison. Reusability Reusability is influenced by aspects of Structuredness (page 17), because loose coupling improves reuse [1, 14]. Part of this is because any module can be introduced in an event-driven system simply by registering it for the system s events [7]. Reusability is also dependent on the way the functionality is distributed over several modules, which is influenced by the communication strategy. Therefore, these criteria are theoretically relevant in the comparison Relevance exclusion Of the remaining criteria, some are more relevant to this research project than others. For Sogyo, the main importance for this research topic lies in the possible improvement in the criteria in the Maintainability group. Nowadays there are often rapid changes in user 18

19 requirements as well as technology and environment. This means that a software system is continuously being changed throughout its lifetime. It is therefore very important that a system can be relatively easily modified to accommodate these changes, while maintaining architectural integrity. If the integrity is not maintained, the cost of further modification increases. This makes maintainability aspects an important concern for virtually every software system. Additionally, there is an interest in Efficiency aspects. EDA has a very different approach to information distribution and storage compared to traditional request-response systems that often rely on a central relational database. It is difficult to predict how much this difference in approach influences the time and resource behaviour. Specifically for the case-study project, there are issues with reaction times, which may be solved by an EDA approach. Since these mentioned criteria are together heavily dependent on the criteria in groups Structuredness, Understandability and Audit Trail, the criteria in these groups are also kept. The other criteria are not explicitly evaluated in order to keep a manageable scope for this research project Short-list of quality criteria When all excluded criteria are removed from the long-list, a short-list is obtained. These remaining criteria are used to compare both architectural approaches in the case study. Some criteria are redivided in a more complete and less overlapping set of sub-criteria. Since several criteria are dependent on other criteria, a graph-like structure is appropriate for the representation of this list. This representation is shown in figure 1. In this figure, A B means that A influences B, i.e. B depends on A. The dashed arrows mean a conditional form of influence; time and resource behaviour depends on scalability only when the system actually needs to be scaled up. The definitions of all the (possibly modified) remaining sub-criteria are shown in table 2. In the reformed structure, the set of criteria is better suited for the next step of the research process: the definition of measurement methods for the different criteria. 19

20 Figure 1: The short-list of criteria, in graph from to show dependencies Time Behaviour Resource Behaviour Repairability Modifiability Extensibility Scalability Testability degree to which the response and processing times and throughput rates of a system, when performing its functions, meet requirements degree to which the amounts and types of resources used by a system, when performing its functions, meet requirements ease with which errors can be located and repaired in an operational program while maintaining architectural integrity ease with which existing features can be changed to suit changes in requirements in an operational program while maintaining architectural integrity ease with which new features can be added to meet added requirements in an operational program while maintaining architectural integrity ease with which the system can be changed to accommodate and increase in scale: more data / users / nodes, while maintaining architectural integrity ease with which test criteria can be established for a system or component and tests can be performed to determine whether those criteria have been met 20

21 Diagnosability Analyzability Self-descriptiveness Simplicity Audit Trail Modularity Consistency Conciseness ease with which deficiencies can be diagnosed and causes of failure can be located ease with which the parts to be modified for an intended change can be identified extend to which objectives, assumptions, constraints and in-/outputs can be inferred from the code alone (without comments) ease with which the overall structure (elements and interactions) and the control-flow of the system can be understood degree to which actions performed during the system runtime can be reproduced degree to which changes in one module have the least impact on other modules degree to which uniform design and implementation techniques and notation are used degree to which the implementation of functionality is provided with a minimum amount of code Table 2: Custom definitions of short-list quality criteria 4.4. Methods of measurement In order to compare different architectural approaches based on the selected criteria, these criteria need to be measured. To this day, there is no evaluation method that is completely objective, extremely precise and also requires very little effort to use [11]. Therefore, a set of appropriate techniques that give a reasonable balance between accuracy and effort, are chosen. This section briefly describes the different chosen techniques. Some methods include metrics that are measured on the implementation, either on the code base or on the running system. Since the alternative solution using EDA is only partly implemented, a comparable set-up for both systems needs to be created. This is done by selecting part of the functionality and creating set-ups for both versions of the system that include only the parts that are necessary to provide the selected functionality. 21

22 Efficiency The most reliable way to measure both time and resource behaviour, is by measuring these quantitatively during runtime of both versions of the system. The time behaviour is assessed by specifically measuring the completion time of a set of user actions. Since all user actions for the selected system are server requests, the completion times are measured in terms of loading time of the request. These loading times are measured with a browsed-based tool called HttpWatch (Basic Edition ) [25]. The resource behaviour is assessed by manually calculating the total amount of used RAM and disk space used by both methods at different characteristic times during runtime of the system Structuredness and Understandability The below mentioned metrics on the source code is calculated using Visual Studio Code Metrics Powertool. The results are aggregated and viewed in Code Metrics Viewer. Conciseness is measured by the metric Lines of Code. Modularity is measured by the metric Class Coupling. The Maintainability Index gives an indication of Diagnosability and Analyzability. Cyclomatic Complexity and Depth of Inheritance is used to give an indication of Simplicity Audit trail For the ease of creating a recollection of notable things that occur at runtime, no real quantitative measures were found. Instead, a qualitative comparison is made, which describes the points of difference between the two approaches Maintainability For the assessment of Modifiability and Extensibility, a scenario-based approach is used. This involves the construction of several change-scenarios and evaluation of the amount of effort it would take to apply these changes to the system. The approach is based on the methods described in [26], [27] and [28]. The actual scenarios that are used are based on actual changes that have happened in the original system, so that they are as 22

23 realistic as possible. To determine these scenarios and the amount of work they took in the original system, the source control repository is used. Scalability is determined by analysing how the system s behaviour changes depending on size characteristics. The time and resource behaviour measurements are used for part of this analysis. In accordance with figure 1, the results for some of the other quality criteria are also used and combined to make statements about Maintainability aspects. 5. Case study: Set-up In this section, the case study is introduced. Section 5.1 describes the way the case study is conducted. In section 5.2, a brief description of the background of the used case is given, along with the general functional requirements Way of working This case study provides a comparison of two systems that fulfil the same functional requirements, but have different underlying architectures; one has a traditional architecture and the other is event-driven. This comparison is then used to make more general statements about the advantages and disadvantages of EDA. The starting point of the case study is an existing solution that was previously designed and built by Sogyo. We call this existing solution the original system. The original system takes a more traditional approach and is not event-driven. Section 6 describes the architecture of this original system. With the same functional requirements as the original system, an alternative solution is designed using the EDA pattern. This alternative solution is called the EDA-system. Part of the designed EDA-system is also built as an application, using an existing eventframework that was previously made by Sogyo for the purpose of experimenting with EDA. By actually implementing part of the EDA application, the practical lower-level implications of the top-level design can be seen. Additionally, the implementation can be used for (quantitative) measurement. Section 7 describes the architectural design of the EDA-system. Finally, the original system and the EDA-system are compared using the quality criteria selected in section 4. The results of this comparison are presented in section 8. 23

24 As introduced in section 2.2.1, the structure of a system s architecture can be described in different views. To make comparison between the two systems easier, their architectures are described using the same set of views. The used views are the following: The process view shows the division into runtime execution units that run in parallel, like applications, processes and/or threads. The development view shows the division of the system into development units like libraries, projects, namespaces and/or classes Case description The chosen case is a system requested by a company (the client) that sells products to stores, which in turn sell them to consumers. These stores are called the company s customers. Contact with a customer is handled by a sales person. The customers are divided into areas and each area is represented by one sales person. This person maintains the relationship with the customer and establishes which products they want to buy. This is captured in so-called budget proposals. Budget proposals are made once a year; each is a signed agreement with a customer, which describes what products the customer will buy in the upcoming year. Additionally, a sales person tries to enlist new customers. One of the main purposes of the system is to assist each sales person in their sales process. A sales person wants to be able to view comparisons between the number of sales estimated in budget proposals and the actual turnovers for the same customer and time periods. This helps them make more accurate budget proposals for upcoming years. The system is also meant to provide the director and area managers with data to support decisions and changes in policies Functional requirements The system takes the form of a SalesPortal, which provides a set of features to users according to their role. There are four available roles: sales person, area manager, director and administrator. The global actions they can each perform using the SalesPortal, are: A sales person can (1) enter new customers into the system, (2) view different kinds of data about their customers (contact details, characteristics, budget proposals, actual turnovers, etc.), and (3) make budget proposals. An area manager can view different summaries about the status of business in his/her areas. The director is able to see summaries about the entire set of areas. 24

25 The actions for the administrator are (a) management of SalesPortal users, (b) correcting the imported data, and (c) checking whether the system functions correctly. The system should additionally be able to process input data from an external system (in this case SAP), which the client also uses to support their business processes. 6. Case study: Original architecture This section gives a description of the original system s architecture. As described in section 5.1, the architectural description is divided into a process view (section 6.1) and a development view (section 6.2). Additionally, section 6.3 presents the weak points of the original system and hypothesize about how EDA could improve upon them Process view The process view shows the division into runtime execution units. A schematic representation of this view for the original system is shown in figure 2. In this figure, the green entries are separate runtime execution processes and the orange entries are forms of data storage and exchange. In this view, the system is divided into an database server (Microsoft SQL Server) and three separate applications that connect to it. The way this connection is handled, is described in the development view. Two of these applications, the batchimport and the batchpdfcreator, are processes that are run overnight at predefined times. The third is the web application that provides the interaction with the users of the system. The web application is hosted on an ISS Express server and an arbitrary number of users can connect to it to use the system. The batchimport process handles the import of data from SAP. This data is presented in the form of XML files, which are read in by the batchimport application and stored in the database. The batchpdfcreator uses data from the database to generate a set of PDF and HTML tables. These are stored on disk and used by the web application to include in certain web pages that are sent to the user. This means some tables in the web application are pre-generated by the batchpdfcreator. However, there are also tables in the web application that are not generated by the batchpdfcreator, but instead are generated on demand (pull-based). 25

26 Figure 2: Original system schematic representation of the process view The web application is constructed using ASP.NET MVC, further described in the development view. The client-server interaction is shown in more detail in the (*) labelled dashed box in the figure. It is handled in the following way: A user specifies the URL of the page they want to view, either by clicking a hyperlink or typing it in manually. The URL specifies which action on which controller on the server should be executed. This action results in the creation of an HTML page that is sent back to the client and shown to the user Development view The development view shows the division of the system into development units. A schematic representation of this view for the same system is shown in figure 3. In this figure, the dashed boxes are separate projects / libraries. Dark blue represents the developed software, while light blue represents imported third-party libraries. The red entries denote relevant C# classes to show connections between the projects. First of all, the batchimport and batchpdfcreator projects in this view directly correspond to the applications of the same name in the process view. 26

27 Figure 3: Original system schematic representation of the development view Additionally, this view shows how the ASP.NET MVC library is incorporated into the SalesPortal project to construct the web application. The MVC library provides base classes for the Controllers and the Views. Each controller implementing this class, can define a set of actions that are initiated by the client by navigating to the corresponding URL. These actions each connect to a view, which defines the HTML page that is constructed and sent back to the client. The connection between controller and view is provided by the MVC library. The ViewModels can be used as an easy way to transfer data from the controller to one of its views. They are called ViewModels instead of just Models (as they are called in MVC) because they do not correspond to the actual domain entities. Instead, they correspond with how the data is organized as it is shown to the user. Finally, the three mentioned projects make use of a set of layers that handle the connection to the SQL database. The top layer consists of the sales domain. It contains classes for the entities that exist in the system domain, defines a repository interface for each of these entities and also defines a service for some of the more important entities. The entity classes are directly linked to the database by using the class-persistence functionality of the NHibernate library [29]. This functionality provides get and set operations on specified attributes to be automatically translated to database queries. 27

28 The repository interfaces define a set of queries to be performed on the collection of records of their corresponding entity type. These repository interfaces are realized by a set of actual repositories in the Repository.NHibernate layer. They use the functionality from the NHibernate library to perform these queries on the database. The services use multiple repositories to provide more complicated queries on the data. Finally, the NHibernate library also contains functionality to define sessions and transaction with respect to the database. This functionality is directly accessed by the other projects for initialization purposes Problems and Hypotheses The original system functions adequately, but there are a few issues that could be improved upon. This section highlights these issues and speculate as to how an event-driven solution would improve them Timeliness and Scalability One of the main problems in the original project, is that some user actions take a long time to complete, in the order of seconds. For users, the tolerable waiting time for information retrieval is approximately 2 seconds [30], and this is exceeded in some places. The user actions that take a long time are actions where a set of data needs to be displayed (usually in a report/table) and this data needs to be collected at the time of the request. Since it involves large amounts of data that are distributed over multiple database tables, this takes a while to complete. While this happens, the user is waiting for their report to load. Additionally, the batchpdfcreator process takes over two hours time to complete every night (in previous versions even longer). The batchpdfcreator creates a set of pdf and html reports overnight. These reports are built from scratch every time and the old set of files is overwritten. Again, a large amount of data needs to be collected and combined. Here, the user is not waiting while these actions are completed, but another timing issue arises. The batchpdfcreator process can only be started once the batchimport process has been finished, and the import in turn can only start once the XML files from SAP are available. The web application is offline during these processes, so the longer the combined task takes, the higher the risk that it has not finished yet at the time the users want to start using the web application again in the morning. Both these issues have recently been improved upon, but the performance is still not optimal. They both get increasingly worse as the amount of data grows over time. 28

29 An event-driven approach could solve both issues by saving a copy of the data necessary for each report in separate modules. The main data storage managed by a main module. This module generates events once some part of the data changes, and all affected reports are updated by their respective modules in response. This first of all means that the data does not need to be collected over and over. Consequently, the time it takes to generate the reports is not affected by the total amount of data in the system. Secondly, the processing of data, and updating of reports, is done as soon as new data is available rather than at the time a report is requested by the user Maintainability Another thing that could be improved is the ease with which certain changes in the requirements can be implemented. The client makes these changes relatively frequently and they usually involve the change or addition of some columns in a set of reports shown by the system. Sometimes these changes can be made by simply making some changes in the ViewModels and Views of the web application. However, sometimes the changes are made by changing the domain and the structure of the database, which involves at lot more work. Also, some reports are created overnight and loaded from a file and others are created by the web application itself. If a report needs to be changed, it is not immediately clear where the change should be made. This could be improved by using the same EDA set-up mentioned for the previous issues. The main data storage should be kept as general as possible and save as much information as is available. The way the information is displayed, can then be changed by changing only the module corresponding to the involved reports, letting them respond to different events if necessary. This means the domain would only need to be changed if the types of data provided to the system change, and not when the data is used in a different way. This would also mean that all reports are made in the same way, so changes would require similar actions for all reports. Since modules are separate parts of the system that only communicate via events, the addition or removal of report -modules would not affect the data-storage module, input modules or other report -modules in any way. 7. Case study: New architecture This section describes the design of the alternative architecture. The architectural description is again divided into a process view (section 7.1) and a development view (section 7.2). Section 7.3 describes the extent of the built implementation. The different modules in the architecture are elaborated upon in section 7.4. Finally, some noteworthy 29

30 things encountered during the design process are explained in section 7.5. The alternative architecture uses the EDA pattern as a basis. As motivated in section 6.3, the division into modules is as follows. one central data storage and management module (the EventStore module) one input module for the data from SAP (the SAPImport module) a ViewModel module for each type of report one front end module (the Gateway module) 7.1. Process view A schematic representation of this view for the EDA-system is shown in figure 4. Just as in figure 2, the green entries are separate runtime execution processes and the orange entries are forms of data storage and exchange. The four different (types of) modules in the system are highlighted by four dotted boxes with different colors. These boxes in the same colors are also used in the schematic representation of the development view (figure 6) to show the connection between the two views. Figure 4: EDA-system schematic representation of the process view The core of the system, the modules that communicate via events, are shown in two different representations. The representation in the dashed box shows how all modules send their events into the system and all other modules can receive them. The representation outside the box shows which modules actually exchange information. For example, there 30

31 is an arrow from the EventStore module to the ViewModel modules. This means there are events of certain types (in this case the types EntityChanged ) that are sent by the EventStore module and received and reacted to by one or more of the ViewModel modules Development view A schematic representation of this view for the EDA-system is shown in figures 5 and 6. Just as in figure 3, the dashed boxes are separate projects / libraries. Dark blue represents the developed software, while light blue represents imported third-party libraries. The red and orange entries denote respectively relevant C# classes and JavaScript scripts, to show connections between the projects. The used Messageframework was previously developed by Sogyo. It contains base classes with which message sending and receiving can be implemented. Figure 5: EDA-system schematic representation of the connection between the modules and the Messageframework in the development view 31

32 Figure 6: EDA-system schematic representation of the modules in the development view In figure 5, the connection between the modules and the Messageframework is shown. Each module contains a Payload class that inherits from the abstract Payload class in the Messageframework. The framework also provides a Message class that represents the events that are sent between modules. Additionally, the MessageCreator class is used to create and send events and the MessageFilterScript can be used in module Payloads to receive events. The knowledge that is shared between all modules is contained in a separate Helper library. It contains for instance representations of the Entities the way they are contained in events. It also contains the different event-types that exist in the system and methods to add and retrieve specific information from events. Figure 6 shows the four modules and their dependence on other libraries. As mentioned in the process view paragraph, a set of coloured dotted boxes is used to highlight the different modules. The EventStore module uses the EventStore.CientAPI library to access an EventStore and store and retrieve data. For the Gateway module, a separate FrontEnd project was developed that uses the SignalR library [31] to establish a connection between a Microsoft Owin [32] server and JavaScript code on the client. This FrontEnd project, as well as the Microsoft.Owin.Hosting library, are used by the Gateway module to run an Owin web server. 32

33 7.3. Extent of implementation Some details of the architecture described in the previous section, are derived from the implementation process. For example, the choices of event-framework, server etc., were all made for practical reasons. These choices are reflected in the libraries used and shown in the process view. This also means that the parts that were not implemented are less detailed in the architectural description given in this thesis. The implementation covers first of all the core of the system, which consists of all functionality for sending and receiving events and the storage, front-end and SAP import modules. Additionally, it contains two ViewModel modules: one for an area overview and one for a customer overview per area. All other ViewModels modules are omitted, because their implementation was expected to be analogous to the already implemented ViewModel modules. Another thing that was not implemented, is all functionality regarding accounts, logins and user roles. This functionality required substantial additional research and is relatively independent of the event-driven characteristics of the system. In the just mentioned implemented functionality, some of the domain model entities do not play any role and are therefore also not included in the system (see section 7.4.1) Module descriptions The different modules are explained in more detail in the next few sections EventStore module The central data storage and management module receives any external changes to the EDA-system as events and stores the information in a persistent way. As storage mechanism, an event store is chosen. This has the advantage that instead of just the current state, the entire evolution of the state is saved and can be used for various purposes. In addition, the information from incoming events can be stored in a semantically similar way, so no complicated transformations are needed. The used event store implementation is called EventStore [33]. It contains a client API for C# so that it can be interacted with from within the program. The incoming events for this module denote the creation, change or deletion of a set of entities that are present within the domain of the EDA-system. These entities, with the respective actions that can be performed on them, are shown in table 3. Some additional entities are processed by the original system, but left out in the implementation of the EDA-system because it was scaled down to a smaller version in which these entities did 33

34 not play a part. These left out entities can be seen by comparing figures 9 and 11 in section The concept of User, which is similar to an entity, was also left out in the EDA-system implementation. For BudgetProposals and Turnovers, the create actions are not defined. These entities have a periodic aspect and have to do with monetary values. To the outside, an entry for each valid period is presented as if it exist, with a default value of zero if it has never been edited. Internally, they are only stored once they have been edited. For UploadBatches, only create actions are defined, because they represent a record of an upload happening, which is not something that can later be changed. Entity Customer Create Change Delete BudgetProposal Change Turnover Change SalesPerson Create Change Delete Area Create Change Delete ProposalYear Change UploadBatch Create Table 3: Entities and the actions that can be performed on them Along with the actual event store, this module contains a mechanism to construct an object-representation of (parts of) the current state. This is used to build the complete new state as result of an incoming change in order to send it out as a new event again. Additionally, where possible, the state-changes are specified in terms of delta s rather than overrides. For example for monetary values, this means that the difference in value is included in the event, instead of the new total value. On the other hand, if the name of a Customer changes, this cannot be expressed as a delta and the new name is simply sent as an override of the previous value. The output of this model consists of events containing the complete state-change as result of each of the received inputs. However, if the state is not changed, no outgoing event is sent, because nothing noteworthy has happened. Each type of event that is received by the EventStore module has a corresponding type of event that is sent out by the EventStore module as a result of it. For example, when a user changes a customer s data, a CustomerChangeInputted event is sent. This event is processed by the EventStore module and if the incoming event really results in a state change, a CustomerChanged event is sent by it. These two events seem similar, but their meaning differs. The -Inputted events carry the meaning of an external action; a user or external system (e.g. the SAP system) inputs some information about an entity 34

35 in the system. On the other hand, the events that are generated by the EventStore module mean that the state of the system has actually undergone a change SAPImport module This module reads in XML files that are produced by the SAP system and translates them to events. These XML files contain new data that serves as input for the EDAsystem. Currently, the data contained in the XML files that needs to be entered into the EDAsystem, is a set of turnover values. To keep a record of this process, a set of UploadBatches are used. These are sent in separate events at the beginning of the import process and are received and stored in a straight forward way by the EventStore module. The main structure of the XML files is as follows. The entries are on top level separated by date and category. Below that, there is a set of salespersons (number + name), which each contain a set of customers (number + name). For each customer, there is a single turnover value. This first of all means that the data needs to be turned inside-out. In the EDA-system, turnovers are first sorted by customer and within a certain customer, they are separated by date and category. Secondly, SAP does not know the identifiers that are used within the EDA-system, so these need to be inferred by using the available data. To accomplish this, data in the XML files representing incomplete EDA-system customers and salespersons, are first put into separate incomplete entities. These are denoted by the prefix Unknown, since the EDA-system does not know yet which specific entity is meant. The subsequent import of a SalesPerson, one of its Customers and a Turnover for this customer is shown in figure 7. When the XML file references a specific EDA-system Sales- Person, the SAPImport module creates an UnknownSalesPerson with a newly generated identifier, and put its number and name into it. This UnknownSalesPerson is included in an event and sent. The EventStore module then receives this and matches it to an actual SalesPerson that already exists within the system. When the SAPImport subsequently sends an UnknownCustomer that was contained in that (Unknown)SalesPerson, it includes the identifier that it gave to that UnknownSalesPerson. The EventStore then deduces the actual SalesPerson for the Customer from that identifier it previously matched. The UnknownCustomer is matched to an actual existing Customer by using the SalesPerson it belongs to and its own number and name. When the SAPImport sends the turnover data belonging to that Customer, it similarly uses the identifier of 35

36 Figure 7: EDA-system schematic representation of part of the SAP import process the UnknownCustomer to link it to the actual Customer and enter the turnover data as a Turnover object belonging to that Customer Gateway module The Gateway module contains the user interface and handles all user interaction. In the original system, this was done using an MVC library. However, for the EDA-system, this is not possible. The MVC library runs the web application on an ASP.NET server. This server is not compatible with the used Messageframework, so the library could not be used. As an additional requirement, to accomplish the event-driven characteristic of the new system, the user interface should be able to react and change upon changes in the internal state of the system. To accomplish this, the SignalR library was used and the application is hosted locally using Owin. The SignalR library enables method calls between the C# code on the server and JavaScript code on the client. This way, if a user is viewing a page, it can be updated in response to an event. The part of the Gateway module that actually uses the SignalR library to deal with the connection between client and server, is contained in a separate.dll. This.dll is 36