Software modular bauen. Ulf Fildebrandt

Transcription

1 Software modular bauen Ulf Fildebrandt

2 Some words before This presentation should give some insights in big software products and how the structure of these products is managed

3 About the presenter Ulf Fildebrandt Studied computer science Works for SAP since 1998 Starting with C++, once SAP decided to use Java part of the Java stack Product architect of SAP Netweaver Composition Environment (Java stack) Program architect of SAP Netweaver 7.30 Program architect of Integration on Java on demand stack

4 Agenda Problem statement Goals Means to explain Basics of modularity with OSGi Principles and patterns Design Patterns (GoF) SOLID System architecture (for decoupling) Comparison of frameworks

5 In every phenomenon the beginning remains always the most notable moment. Thomas Carlyle Findbugs Analysis by Structure 101 SAP Netweaver Structure Problem statement

6 Findbugs Analysis Famous analysis done by Ian Sutton Can be found at are-erosion-findbugs/ Uses Structure 101 (see also JAX 2012 innovation award for Restructure 101)

7 Findbugs 0.7.2

8 Findbugs 0.8.6

9 Findbugs 0.8.7

10 Findbugs 1.0.0

11 Findbugs 1.3.0

12 Findbugs 1.3.5

13 Server Analysis Own analysis tool of SAP Java server Structure of current Netweaver server is shown Not only Java dependencies (packages) difference to Structure 101 Usage Types Software Components Packages

14 Structure of Complex System I Overview of product en/dbg/java/packaged/full/_doc_pkg/content/index.html

15 Structure of a Complex System II Basic Scenario

16 Structure of a Complex System III Detailed dependencies

17 ...it is almost always incorrect to begin the decomposition of a system into modules on the basis of a flowchart. We propose instead that one begins with a list of difficult design decisions or design decisions which are likely to change. Each module is then designed to hide such a decision from the others. David Parnas Substitutability Extensibility Goals

18 Extensibility Mo dule A Mo dule E Mo dule B Mo dule D Mo dule F Mo dule C A module is a self-contained component of a system, often interchangeable, which has a well-defined interface to the other components.

19 Substitutability Mo dule A Mo dule E Mo dulemo D dule B Mo dule D Mo dule C A module is a self-contained component of a system, often interchangeable, which has a well-defined interface to the other components.

20 Goal: Sustainable Software Mo dule A Mo dule E Mo dule B Mo dule D Mo dule F Mo dule C System can evolve over time and modules can be replaced without impacting the other modules.

21 Programming without an overall architecture or design in mind is like exploring a cave with only a flashlight: You don't know where you've been, you don't know where you're going, and you don't know quite where you are. Danny Thorpe Levels of Architecture Web Application as an example Means to explain

22 Levels of Architecture Same principles can be applied on all levels of architecture Systemarchitek tur Komponentenarchitektur CodingArchitektur Facade Factory Dependency Injection Liskov

23 Web Application o m e D

24 Good fences make good neighbors. Robert Frost Packages Services Lifecycle Modularity Maturity Model Basic modularity with OSGi

25 Package Structure of a Bundle Bundle exports only defined packages Exported Package Packages can be imported from other archives (bundles) Exported Package (Internal Packages) Bundle Imported Package Imported Package All other packages are only visible inside the bundle

26 Package Dependencies of a Bundle Bundle Imported Package Imported Package Exported Package Exported Package Bundle

27 Service Structure of a Bundle Exposed Service Exposed Service Bundle Used Service Used Service A service is an implementation of a Java interface Bundle exposes services to service registry Used services are taken from service registry

28 Service Dependencies of a Bundle Exposed Service Exposed Service Bundle Used Service Used Service Exposed Service Exposed Service Bundle Service implementations are taken from service registry (not shown here) Bundle

29 Lifecycle of a Bundle install installed start starting resolve uninstall resolved uninstall uninstalled active stop stopping

30 How to achieve modularity... OSGi is a framework on top of Java Complements Java very good...but modularity does not come for free

31 Modularity Maturity Model I Level 1: Ad Hoc At the Ad-hoc level, there isn't any formal modularity. A flat class path issued with a bunch of classes with no, or limited, structure.they may use 'library JARs' to access functionality but typically results in a monolithic application. Benefits The main benefit of this is the low cost and low barrier to entry. Level 2: Modules Modules are explicitly versioned and have formal module identities, instead of merely classes (or JARs of classes). In particular, and a key point of this, is that dependencies are done against the module identity (including version) rather than the units themselves. Maven, Ivy, RPM and OSGi are all examples of where dependencies are managed at the versioned identity level instead of at the JAR level. Benefits Decouple module from artefact Clearer view of module assembly Enables version awareness through build, development and operations Enables module categories

32 Modularity Maturity Model II Level 3: Modularity Modules are declared via module contracts, not via artefacts. The private parts of the modules are an implementation detail. In this level, dependency resolution comes first and module identity is of lesser importance. Benefits Fine-grained impact awareness (for bug fixes, implementation or client breaking changes) System structure awareness Client/provider independence Requirement-based dependency checking Level 4: Loose coupling There is a separation of interface from implementation; they are not acquired via factories or use constructors to access the implementations. This provides a services-based module collaboration (seen in OSGi, but also present in some other frameworks like Spring or JNDI to hide the construction of the services from the user of those services).in addition, the dependencies must be semantically versioned. Benefits Implements a client/provider independence

33 Modularity Maturity Model III Level 5: Devolution At the devolved level, artefact ownerships are devolved to modularity-aware repositories. They may support collaboration or governance for accessing the assets by relation to the services and capabilities required. Benefits Greater awareness of existing modules Reduced duplication and increased quality Collaboration and empowerment Quality and operational control Level 6: Dynamism Provides a dynamic module life-cycle, which allows modules to participate in the life cycle events (or initiate them). Will have operational support for module addition/removal/replacement. Benefits No brittle ordering dependencies Ability to dynamically update Can allow fixes to be hot-deployed and to extend capabilities without needing to restart the system

34 Each pattern describes a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice. Christopher Alexander Facade Factory Principles and patterns - Design Pattern (GoF)

35 Factory I A factory is an object for creating other objects. It is an abstraction of a constructor. t) A factory externalizes the creation of objects Creation and usage of objects is decoupled Pattern definition works for coding Without factory: DataAggregator da = new DataAggregator(); With factory: IDataAggregator aggregator = DataAggregatorFactory.getInstance();

36 Factory II Apply on component level Externalization of creation of objects is a general pattern How can OSGi help: Service registry provides an abstraction between provider and consumer of a service (object instance) Creation of instances of a service can be handled by frameworks: Declarative services Blueprint

37 Facade I A facade is an object that provides a simplified interface to a larger body of code, such as a class library. Facade: rn public interface IDataAggregator { public List<IDataItem> get(); } Implementation of facade: final class IdentityDataAggregator extends DataAggregator public ArrayList<IDataItem> get() { List<IDataItem> itemlist = new ArrayList<IDataItem>();... return itemlist; } }

38 Facade II Apply on component level Access to a component as a general pattern How can OSGi help: 1. Services are Java interfaces 1. Service implementations are accessed via interface = facade 2. Exported packages are the only external visible entities of a bundle 1. Not exported packages are not accessible 2. Clear definition of a facade of the bundle

39 Each pattern describes a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice. Christopher Alexander Dependency Injection (SOLID) Liskov (SOLID) Principles and patterns - SOLID

40 Dependency Injection I Dependency injection involves at least three elements: a dependent consumer, a declaration of a component's dependencies, defined as interface contracts, an injector (sometimes referred to as a provider or container) that creates instances of classes that implement a given dependency interface on request. The dependent object describes what software component it depends on to do its work. The injector decides what concrete classes satisfy the requirements of the dependent object, and provides them to the dependent. n

41 property method method property method method method property method Class 1 method property method Dependency Injection II Class 2

42 Dependency Injection III Apply on component level Reverse the dependency is a general pattern How can OSGi help: Service registry provides an abstraction between provider and consumer of a service (object instance) Injecting dependencies can be handled by additional frameworks Declarative services Blueprint

43 Web Application e v i t a r a l c e s D e c i v o r m e S De

44 Liskov Let q(x) be a property provable about objects x of type T. Then q(y) should be provable for objects y of type S where S is a subtype of T. Facade = T: ple public interface IDataAggregator { public List<IDataItem> get(); } Implementation of facade = S: final class IdentityDataAggregator extends DataAggregator public ArrayList<IDataItem> get() { List<IDataItem> itemlist = new ArrayList<IDataItem>();... return itemlist; } }

45 Example to apply coding patterns

46 Definition of a Composite Application Wikipedia: A Composite Application is an application built by combining multiple existing functions into a new application. Extended Definition: Composite Applications are user centric applications supporting highly collaborative and dynamic business processes which span beyond functional, system, and organizational boundaries by using data and functions provided as services by platforms and applications. SAP AG All rights reserved. / Page 46

47 COMPOSITE APPLICATION Anatomy of Composite Applications Role 2 Role 1 Step 1 Workcenter Step 2 UI Remote Services Step 3 Step 4 UI UI Business Entities Composite Process User Interfaces Local Services Business Entities, Services Remote Local Enterprise Service Bus BACKEND (optional) SAP AG All rights reserved. / Page 47 Services CRM Services Services BW ERP Service Enablement Systems

48 DEMO Composite Application Structure

49 Simple approach: build a framework on top of the tools, but Installation problem All or nothing -approach, because all tools have to be installed Integration Framework R Domain Tool Domain R Domain Tool Domain R Domain Tool Domain New tools The integration framework does not have a reference to the new tool Domain Tool Domain Note: same problem appears by integrating all domains with a common layer at the bottom, e.g. modeling framework/repository SAP 2007 / Page 49

50 Adapter Basic principle: Adapter mediates between tool and capability Target is exposing interfaces and extension points (Eclipse) Source is implementing extensions and using public APIs Adapter connects tool and capabilities Tool does not reference the capability functionality integration is completely done by the adapter Adapter is completely separated from tool tool is built on minimal requirements tool is able to run without integration SAP 2007 / Page 50 Source Target

51 Consistent visualization of a Composite Application Graphical overview and relations Display consistency or inconsistencies between parts of a Composite Application Integration Framework (Composite Designer) Pluggable infrastructure to integrate separated tools Optional extensions to allow step-wise integration Tools can start with a basic integration and extend it later Tool provides extension points SAP 2007 / Page 51 Domain Domain Domain Tool R Domain Tool R Adapter R Domain Tool R Adapter R Capability Integration Framework Facade Domain Adapter R Integration framework provides extension points

52 SAP 2008 / Page 52

53 Main element factory provider registration <extension point="com.sap.ide.comp.pers.core.modeler.graphicalelementfactories " id="wd" name="wd Content in Overview"> <factory dctype="web Dynpro" class = "com.sap.ide.comp.pers.adapter.wd.wdgraphicalelementfactory" layer="user Interface"/> </extension> Signature of: Interface IGraphicalElementFactory java.util.list<iresolvedelement> getmainelements(igraphicscontext context) Public interface IElement Factory Base protocol for all entities that Composite Designer can deal with. Elements can appear in two basic forms: Resolved: such elements point to an entity from a concrete domain (e.g. J2EE, Web Dynpro). They usually appear in the form of an graphical element. Unresolved: such elements cannot be resolved in the adapter which created them, but usually in a different adapter. For example the Process Composer adapter could create an unresolved element for a Web Dynpro Component by just specifying its key, and only the Web Dynpro adapter is able to resolve it into a complete (resolved) element. SAP 2008 / Page 53

54 SAP 2008 / Page 54

55 Dependency contributor registration <extension point="com.sap.ide.comp.pers.api.dependencycontributors"> <contributor class="com.sap.ide.comp.pers.adapter.wd.wddependencycontributor"/> </extension> Signature of: Interface IDependencyContributor java.util.list<iresolvedelement> getincomingelements(iresolvedelement element) java.util.list<ielement> getoutgoingelements(iresolvedelement element) IResolvedElement resolveelement(iresolutioncontext context) Separated interface Public interface IElement Base protocol for all entities that Composite Designer can deal with. Elements can appear in two basic forms: Resolved: such elements point to an entity from a concrete domain (e.g. J2EE, Web Dynpro). They usually appear in the form of an graphical element. Unresolved: such elements cannot be resolved in the adapter which created them, but usually in a different adapter. For example the Process Composer adapter could create an unresolved element for a Web Dynpro Component by just specifying its key, and only the Web Dynpro adapter is able to resolve it into a complete (resolved) element. SAP 2008 / Page 55

56 Each pattern describes a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice. Christopher Alexander Architecture Layer SEDA System Architecture - for decoupling

57 Architecture Layer I Package Module Sub system System Layer System Platform A platform (in terms of runtime framework) hosts a system and is a versioned artifact A system consists of a set of sub-systems and is a versioned artifact running on a platform A system layer groups sub-systems providing structure and direction in terms of dependencies and is a layer which has no associated versioned artifact A system layer incarnation is the bill of materials or assembled sub-systems of a system layer relevant for a specific use case A sub-system consists of a set of modules and is a versioned artifact A module contains packages and is a versioned artifact A package contains components A component is a File A file is of type resource or source

58 Architecture Layer (as is) II Package Module = Subsystem = Bundle Package analysis of web application Uses Structure 101 (see also JAX 2012 innovation award for Restructure 101) System Layer

59 Architecture Layer (to be) III Data Display (Servlet) Data Aggregator 1 Aggregator Data Source 1 Data Source 2 Data Aggregator 2 Data Provider Data Aggregator Interface Data Aggregator Interface Data Source Interface UI Data Source Interface

60 Architecture Layer IV ConQAT (to-be diagram) ConQAT ( Compared to Structure 101: extensible because of open source Used to implement modularity metrics Easy integration in build process (Maven) automatic check of to-be and as-is

61 Modularity Metrics I Coupling: determines the coupling of this component to other components and is computed based on instability (Robert C. Martin). I = Ce / (Ca+Ce) 1..0 (Ce (efferent) = outgoing, Ca (afferent) = ingoing) Consequence: the more components THIS component depends upon, the more instable it is (0= stable, 1=instable) Ca = usages of exported packages Exported Package Exported Package (Internal Packages) Bundle Imported Package Imported Package Ce = imported classes in packages

62 Modularity Metrics II Relational Cohesion: a component should consist of cohesive elements, otherwise, it should be splitted. Average number of internal relationships per type. rc = Td / T (Td.. type relationships that are internal to this component; T.. number of types within the component) As classes inside an assembly should be strongly related, the cohesion should be high. On the other hand, too high values may indicate overcoupling. A good range for RelationalCohesion is 1.5 to 4.0. Assemblies where RelationalCohesion < 1.5 or RelationalCohesion > 4.0 might be problematic. Therefore rci is the normalized RC, having value 1 for 1.5 to 4, decreasing to 0 outside this range based on gaussian bell curve.

63 Modularity Metrics III Encapsulation: Components should encapsulate knowledge and offer a slim interface. ep = pt / T (pt = private Types, T = all types) pt = types in internal packages, T = types in internal packages + types in exported packages Exported Package Exported Package (Internal Packages) Bundle Imported Package Imported Package The Overall Modularity Score is now defined as: M = ((1-I) + rci + ep) / 3.0

64 Demo ConQAT I b e w n o o ti m a De pplic a

65 Demo ConQAT II r e v r e s AP o S m De m o fr

66 Don ts in Layering I Class 1 verwendet Parameter Class Schicht ruft auf Class 2 Schicht ruft per Reflection auf

67 Don ts in Layering II Class 1 Eclass 1 ruft auf Class 2 referenziert Strin g Strin g Eclass 2 Schicht referenziert Schicht

68 SEDA Functional modularization Queue Thread Thread Thread Thread Thread Thread Thread... Stage Queue Thread... Stage Queue Thread... Stage

69 Old protected void doget(httpservletrequest req, HttpServletResponse resp) throws ServletException, IOException { StringBuilder buffer = new StringBuilder(200); PrintWriter writer = resp.getwriter(); renderheader(buffer); List<List<String>> list1 = dataaggregateservice.getdata(req.getparameter("type")); TableObject analyzedtable = new TableObject(list1, DisplayServlet.ANALYZED_DATA, buffer); rendertable(analyzedtable); List<List<String>> list2 = dataaggregateservice.getdata("identity"); TableObject originaltable = new TableObject(list2, ORIGINAL_DATA, buffer); rendertable(originaltable); renderfooter(buffer); writer.write(buffer.tostring()); }

70 Old protected void doget(httpservletrequest req, HttpServletResponse resp) throws ServletException, IOException { StringBuilder buffer = new StringBuilder(200); PrintWriter writer = resp.getwriter(); renderheader(buffer); List<List<String>> list1 = dataaggregateservice.getdata(req.getparameter("type")); TableObject analyzedtable = new TableObject(list1, DisplayServlet.ANALYZED_DATA, buffer); rendertable(analyzedtable); List<List<String>> list2 = dataaggregateservice.getdata("identity"); TableObject originaltable = new TableObject(list2, ORIGINAL_DATA, buffer); rendertable(originaltable); renderfooter(buffer); writer.write(buffer.tostring()); }

71 Rendering protected void doget(httpservletrequest req, HttpServletResponse resp) throws ServletException, IOException { StringBuilder buffer = new StringBuilder(200); PrintWriter writer = resp.getwriter(); renderheader(buffer); t.setdataaggregateservice(dataaggregateservice); triggerrendering(req.getparameter("type"), DisplayServlet.ANALYZED_DATA); triggerrendering("identity", DisplayServlet.ORIGINAL_DATA); String message1 = getresult(); StringTokenizer tokenizer1 = new StringTokenizer(message1, "\n"); String type1 = tokenizer1.nexttoken(); String body1 = tokenizer1.nexttoken(); String message2 = getresult();

72 Trigger Rendering private void triggerrendering(string type, String header) { Connection connection = null; // Create the connection. try { ActiveMQConnectionFactory connectionfactory = new ActiveMQConnectionFactory(ActiveMQConnection.DEFAULT_USER, ActiveMQConnection.DEFAULT_PASSWORD, ActiveMQConnection.DEFAULT_BROKER_URL); connection = connectionfactory.createconnection(); connection.start(); // Create the session Session session = connection.createsession(false, Session.AUTO_ACKNOWLEDGE); destination = session.createqueue(in); // Create the producer. MessageProducer producer = session.createproducer(destination); producer.setdeliverymode(deliverymode.non_persistent); TextMessage message = session.createtextmessage(type + "\n" + header); producer.send(message);

73 Get Result ActiveMQConnectionFactory connectionfactory = new ActiveMQConnectionFactory(ActiveMQConnection.DEFAULT_USER, ActiveMQConnection.DEFAULT_PASSWORD, ActiveMQConnection.DEFAULT_BROKER_URL); connection = connectionfactory.createconnection(); connection.start(); // Create the session Session session = connection.createsession(false, Session.AUTO_ACKNOWLEDGE); Queue destination = session.createqueue(out); // Create the consumer MessageConsumer consumer = session.createconsumer(destination); Message message = consumer.receive(); if ( message instanceof TextMessage ) { TextMessage textmessage = (TextMessage)message; text = textmessage.gettext(); } return text;

74 Dispatcher Thread while(true) { Queue destination = session.createqueue(in); MessageConsumer consumer = session.createconsumer(destination); Message message = consumer.receive(); Queue destination = session.createqueue(out); MessageProducer producer = session.createproducer(destination); TextMessage message = session.createtextmessage(type + "\n" + body); producer.send(message); }

75 Example: OSGi vs. Maven + Spring Comparison of frameworks

76 Criteria for Comparison Modules: software is structured in clearly separated units. Decoupling: software modules have clearly defined interfaces and frameworks support development based on interfaces. Repository: software modules are decoupled entities that can be stored in a repository. Versioning: software modules are versioned and framework can handle multiple versions.

77 OSGi vs. Maven + CDI Versioning Repository Versioning Decoupling Module OSGi Repository 0 Module Decoupling OSGi is not the only technology, other technologies serve the same purpose Maven CDI

78 Summary

79 Key take-aways Big software systems have to be structured, but more high-level than packages Additional frameworks support modularity (e.g. OSGi, Maven, Spring, CDI implementations ) Modularity can be measured by metrics