Applicability of MDD to. Software Factories. The FAST Experience. ISP Industrialisation of Software Production

Transcription

1 Applicability of MDD to Software Factories The FAST Experience ISP Industrialisation of Software Production

2 1- Industrialisation Strategies

3 Industrialisation of Software Production ISP as key expertise area for Tecnalia/ESI with more than 15 years of experience on ISP related proyects (14 FP7-EUREKA and 30 market references) Objective: To support an organisation to increase applications production capacity through a software development faster, cheaper and with a better quality through the industrialisation process of software production There are several ISP strategies, and there is no one-size-fits-all technology nor method that supports the wide range of potential scenarios. We group all strategies in 4 groups.

4 Industrialisation of Software Production Product Lines for productcentric reuse strategies Software Factories for multi domain-centric reuse Architecture Driven for reference architecture constrained development Community Driven for fuzzy and less structured organisations

5 We select strategies based on characteristics such as reuse opportunities and risks analysis, current and desired reuse practices, technical context, productivity control and measurement and important issues of the organizational context such as company size, customer base, metrics of projects and products, etc.

6 ISP Community Driven Community Driven: GMC by Tecnalia GMC Platform GMCCrawler GMCClones GMC4Migration GMC IDE GMCLicenceAnalyst GMCDeveloperProfile GMCPeople Enterprise EcoSystem GMCDojo

7 ISP Architecture Driven Architecture Driven, examples: GEMDE MDE toolbox for different architectures MDE for RT Embedded systems with MARTE/SysML MDE for legacy modernisation with X2U and custom made DSLs

8 ISP Product Line Generic tool for the design, implementation and management of Software Product Lines following a MDSD (Model Driven Software Development) approach. Version 3.0 available. Selected for the European CESAR RTP. CVL support. Ready for verticalisations and joint delivery with ICT companies. FAST powered.

9 IIndustrialisation Control & Monitor IDE Productivity tools Process Organisation SOA OSGi Eclipse ISP Software Factory Next Generation

10 2 MDA, MDD, MDE, MDSD

11 Model Driven X Let s not focus on the conceptual differences between MDA-MDD, MDE, MDSD.. Let s focus quickly on basic ideas and value Models Abstraction of an object/phenomena in the real world Simplified view of the complex reality Conforms to metamodels! Metamodels Defines what a model looks like A kind of model grammar (-meta) A collection of concepts of a certain domain Formal description ensuring information consistency

12 End-user Ease of integration Functionality Process Functionality Technology Sales and field support Ease of use Ease of customization Qualities Complexity Development manager Reliability Price Development costs, skills Skills Business Concerns Organization and culture On time delivery Architect Developer Performance and throughput Stability and maintainability System administrator Ease of diagnosing problems Ease of introducing modifications Testability and tractability Structure and dependencies between parts Ease of installation FOR COMMUNICATION

13 CASE Tool Model Checking Code Generation Requirements Manag. Analyze Requirem. Design Validate & Verify Build Code Common Repository Reliability Analysis Schedulability Analysis WCET Analysis Performance Simulation FOR INTEGRATION

14 Software Electronics Mechanics Control Abstraction Level Business Process Systems Engineering Legend: Development Artifacts: Design artifacts Product artifacts Artifact Dependencies: Relations among domain-specific artifacts and system-level artifacts Artifacts of varying abstractions and engineering domains Domain in Embedded Systems Development Original slide from Peak (GIT), SysML Presentation 2005 FOR MULTIDISCIPLINARY APPROACHES

15 MD VALUE: DESIGN-BUILD-VALIDATE-VERIFY Evaluate architectural design alternatives Multiple points of view Best software engineering practices Configurations, dependencies,redundancy reliability, performance, cost, refinement/abstraction, multiperspective, layering

16 MD VALUE: DESIGN-BUILD-VALIDATE-VERIFY

17 3 FAST

18 Software Factory: An organisation for the disciplined development of software with engineering methods The Software Factory leads to innovation in: The organisational structure Technical Office The methods and techniques of work Domain Engineering / Application Engineering The production tools FAST, among others.

19 What s the Technical Office in this context? Objective Strategy Functions Control Differentiate the production means from the actual software production Creation and development of a specific organisation: Technical Office Domain Engineering Create, develop and manage development means Reuse Productivity Performance

20 The Paradigm Normalisation and standardisation are the principles, the techniques and organisation are the instruments and productivity and quality are the results. Objectives Reach high levels (>60%) of code reuse Produce software almost independent of its author Increase productivity and quality in the production of software Strategies Solve the dilemmas of model based software production Models with models, or without models Adapt the technology to the factory domain

21 What difficulties can we come across? The maturity of the organisation The development culture The initial workload of the Technical Office Technology stacks

22 New Generation In 2009 one of the major Core Banking organisations in Spain requests a new generation of Software Factory technologies to Tecnalia. FAST.

23 What s FAST? FAST is an Eclipse based tool to build Software Factories and not a software factory itself. Our approach is to build an extremely configurable framework so you can use it to generate industrialised software production platforms This basically means that you should have or plan to build a family of similar products or developments.

24 What s FAST? FAST uses models, but models are invisible to the developer. The code and artifacts generated have no dependencies with the factory that generated them. You can use our software to build your own domain specific software factories using the tools and guidelines provided with it.

25 Why use Eclipse at all? The customer answered this for us by making it a non-functional requirement. We are already using tools integrated in eclipse and we want to be able to keep integrating these tools and the new software factory, and anything else we might come up in the future. So we needed to build an And what better way to do it by using eclipse to make it And since we are using Eclipse, why not make use of every technology available to us? And many more

26 Model Driven = EMF? Safe to think so if you are working in Eclipse But... What if the end user does not use EMF models? What if the end user does not even know what EMFlike models are? What if the end user does not want to use graphical CASE tools? There are people who actually do not like them! What if the end user has its data/models in many different formats like text files and excel sheets? EMF and models are good but not all the users like EMF or models at all!

27 Model driven without models? The fact that the user does not perceive the existence or interact with anything that they consider models does not prevent that: What they are using are actually models, they are just not aware of this (nor do they need/should to be). The internal representation of data and processes within the tool use models. Technical Office engineers should be able to transform whatever data format to an EMF representation.

28 FAST capabilities Generates/Builds things, mostly code but not only. And put traces everywhere! Allows end users to decide the representation in which that are going to give instructions and parameters to the factory. Allows easy development of transformations/scripts without relying in any specific transformation/scripting technology. Allows development of transformations/scripts for nonmodel savvy people (using plain normal code and algorithms and data)

29 FAST capabilities Allows concurrent non-blocking generations Allows distributed generations and deployments of their factory Any kind of distributed, not client-server only. Integrates with distributed repository systems Allows each company to use the factory as they want The factory must adapt to the company, not the other way around Allows data/control/gui integration with other Eclipse and if possible non-eclipse tools

30 Which are some of FAST s self imposed requirements? Modularity So we can reuse features developed for this project in other developments. Standards compliant OSGi and Java standards are to be followed and never broken. Reduce third party API level dependencies The APIs are always presented as Java interfaces and dependencies to libraries outside OSGi and Java SE are avoided.

31 The experience It s possible to use MD for the Technical Office, not exposing models to the developers Domain Engineering assisted by models and transformations Application Engineering working on native coding Providers developed for different languages and sources In the Core banking particular case: COBOL developers working 10% to 20% faster than before 60% to 80% of sustainable reuse Integrated with several Eclipse based tools (e.g. lifecycle)

32 Thank you very much. Joseba Laka Market Director ICT/European Software Institute Tecnalia Jason Mansell Productivity technologies lead ICT/European Software Institute Tecnalia

33 4 Annex

34 What is this about OSLC? Open Services for Lifecycle Collaboration (OSLC) OSLC is a new approach to solving the problem of integrating lifecycle tools Standardize the way that software lifecycle tools can share data with one another: Requirements Defects Test cases Plans Code See: Members:

35 Is OSLC just another middleware? No, OSLC is not middleware It does not really define kind of API, instead it reuses already existing standards. Communication is done via HTTP1.1/REST Basic semantics are inherited from RDF/DCMI specifications OSLC defines extended semantics to allow semantic data transfer between tools. OSLC sometimes looks more like a convention rather than a full fledged specification. Proposes how data should be accessible How to create catalogs of the exposed resources How to navigate these catalogs For an application to be OSLC compliant it does not need to adhere to any middleware or architecture.

36 What are OSLC domain specifications? OSLC Core Specification How to define OSLC resources. How to offer services. How to inform clients of resource shapes. How to offer delegate UIs. How to offer query capabilities. How to offer resource creation. What authentication is allowed. How specification versioning works. How to represent OSLC defined resources. OSLC Domain Specification Defines OSLC resources. Offers services. May offer resource shapes. May offer delegate UIs. May offer query capabilities. May offer resource creation. Provides examples of representations. Definitions of how tools working for a particular domain should share their resources. Some finalized OSLC domains: Change Management Quality management Requirements Management Architecture Management Software Configuration Management

37 What is the relationship between IBM Jazz and OSLC? Jazz based/powered tools implement OSLC specifications Jazz is a middleware Which implements OSLC services Facilitates exposing other tools resources as OSLC Jazz Integration Architecture (JIA) extends OSLC further JIA defines Jazz Foundation Services Jazz team Server provides an implementation of those. Both are IBM initiatives. Jazz is more IBM centric Latest versions of IBM tools moving towards Jazz -> OSLC. OSLC is more community centric. More information at: oslcjazz

38 Why would someone like to integrate an IDE with FAST? Whatever you can do with the tools that are already known you want to keep doing it with the same tools. Diminishes resistance to change. If the GUI is consistent between both tools the learning curve diminishes a lot. The HMI between the tools uses the same channels and conventions. If both tools are integrated into a single IDE the end user only sees one tool. Keeps thing simpler Reduces end user annoyance and frustration. No need to switch between tools If you are generating code, you would like to see it right away in your IDE. Same for any other thing you are generating and is managed by the IDE. Why not? Integration is generally a good thing if done correctly.

39 How can I integrate an Eclipse IDE with FAST? Several ways to integrate: Install FAST modules together with RAD in the same Eclipse environment IDE integration GUI unification Can create plug-ins based on OSGi services or Java APIs to add data and control integration to the applications. Use OSLC Example: RAD integrates with IBM Rational Team Concert which is built upon Jazz Enables to create RAD plug-ins which can use provided FAST OSLC services and vice versa. No need to be installed in the same Eclipse (or even the same machine) Control integration Data integration IDE and GUI integration still an option but not achieved freely Do both or something else FAST is extremely flexible and modular

40 What integration approach is better between an Eclipse IDE and FAST? Deploy in the same Eclipse The IDE RAD for example FAST s Generation Client Services Host APIs for generation management FAST s Generation Client GUI Eclipse based Desired FAST s Request Grammar Editors Might be Eclipse Editors Might be editors based on the installed IDE Might be external tools FAST s Provisioning services Resources access and cache FAST s Networking services Seamless network interaction with at least one communication supported (OSLC) Deploy in the same or other Eclipse s within the organization s network FAST s Generation Server Services Generator implementation FAST s Provisioning services FAST s Networking services FAST s Persistence services Automated distributed storage abstraction services (for generation results and requests) The IDE and FAST client do not communicate but share the same environment FAST clients can be networked via OSLC to other remote FAST servers

41 Can FAST be used as replacement or complement of cheat sheets for design pattern enforcement? Yes! For both questions. Replacement If all the user information for the pattern can be obtained at the beginning of the guidance FAST can be used to create a batch generation Complement If the information cannot be compiled at the beginning of the process Cheat sheet are still a better approach The automation of each cheat sheet can be implemented via a FAST generation. One of the advantages of software factories is: Standardization (relative to the organization) of all the generated code. For architectural patterns the benefit is even greater.

42 What does FAST offer in regards of team collaboration? Depends on the definition of Team Collaboration User management Authentication (via JAAS and OSGi User Admin) Authorization (via OSGi Conditional Permission Admin) Process management FAST offers APIs and services that allow it to be automated by process management and automation tools. Deep integration with Mylyn is planned but not critical at the moment. OSGi User Admin integration with Mylyn Brisges to FAST s editors to Mylyn. Traceability FAST automatically creates traces of the generations, with complete information about the inputs/outputs and requesters. Exposes traces as OSLC links. See: Versioning FAST Provisioning features and modules are being designed to offer seamless management of versioned resources. It relies on third party backends The integration via Eclipse s Team Provision is granted. FAST is tool designed to allow the implementation of a wide selection of services that a software factory based development process would need. The tool does not impose any methodology of its own. Although it comes with off the shelf support to implement Tecnalia s approach

43 What does FAST offer in regards of process enforcement? FAST is designed to help automate development processes. Mainly code generation Documentation Configuration generation It does this by allowing each organization to code their own rules, processes and languages In their selected coding languages In their selected representation models Finally it provides a distributed batch generation environment. All the data is gathered before a generation from one more heterogeneous sources The generation is requested, then processed, the output stored and indexed and the requester notified. This allows organizations to ensure the quality of their products by taking developers through semi-automated processes Generate as much as possible If the generators are well built the results will be homogeneous, predictable and easily maintainable. Only code where allowed Furthermore, the special code each program might need can be asked in advance of the generation and allow it to place it where appropriate.

44 What integration can be achieved with process management tools/features? Defining Users and Roles FAST uses the OSGi UserAdmin Retrieve/Set the users/roles Manage Users (create/update/query) Manage Roles (create/update/query) Each project area in Process Management tools can usually define the set of roles that are available within that project. Fast provides APIs to automatically synchronize the OSGi repository with User and Role information Providers. Event and notification based Automation FAST provides several control integration interfaces to enable the automation of its usage. REST services. OSGi services. Open Java API. OSGi Events.

45 What integration can be achieved with team awareness tools/features? Tracing each element to the user that Generated it. Contributed to it. Created a GenBot. Created a work order. Traceability is handled internally by FAST Traceability is stored within the log of each generation OSLC external link representation is available All traces can be represented via links These traces can eventually be stored in other tools.

46 What integration can be achieved with source control tools/features? FAST Provisioning Services Handle versioned resources Manage local caches for remote resources Notification management Automatic synchronization of resources Querying via URLs But it need to work against a backend Eclipse Team Provider APIs ease backend implementation/integration Not restricted to it. Open enough to adapt to any kind of backed and version control model.

47 What integration can be achieved with administration tools/features? FAST provides administration features: User/Role management Authorization for Roles Network management Remote FAST configuration. Most administrative tools in the market provide web frontends for these things User management and LDAP integration FAST authorization rule APIs use OSGi Conditional Permission for their enforcement Provides Adaptors and APIs to integrate rule sources and providers.

48 What is the benefit of integrating FAST with Software Development Process control tools? FAST provides services and tools to IMPLEMENT Software Factory related processes FAST does not define those processes nor does it manage them Software Development Process control tools Define software processes Allow automation of those processes Integration allows: Defining and implementing software factory based development processes. Intagrating automated generation features to your existing development processes.

49 What are the benefits of integrating FAST with testing automation tools? Depends on what is tested and how it is tested There might be little benefit or there might be a lot. FAST creates software product lines / factories Product lines create similar products One test for everything (little benefit) Product line usage ensures product compliance to the test definitions» Ensures relevancy of test across all products. Customized tests per customized product (lots of benefit) Benefit comes from product lining the tests» For each generated product the tests for that specific product should be generated. Need to create TWO product lines» One for the products» One for the tests

50 What does FAST s Functional Architecture look like?

51 Is FAST Service oriented? It follows one so technically yes, but architecturally not really OSGi services and compliance is used. But direct calls to the methods of the services are avoided as much as possible between service implementations. Event/Command/Message oriented! OSGi EventAdmin service is used as the main control mechanism. Events and their respective handlers define the commands/functionality exposed and required by each module Application is reactive and context aware Easily extensible thanks to the whiteboard pattern employed by the EventAdmin service

52 Why the difference between client and server? Peer 2 Peer (P2P) is the reason. Do NOT confuse with with Point 2 Point. In a P2P network/architecture each node can act as: a client/consumer of services/functionality a server/provider of services/functionality Both simultaneously. None and act as simple relay In order to fully decouple the design of the client from the server we use a messaging infrastructure. The interfaces of the messages need to be know by both the client and the server But the client does not know that the server even exists and vice versa

53 If it is event based why use services at all? OSGi standards compatibility The fact that we recommend and only communicate via messages does not mean that there might not be an scenario in the future where the services need to be accessed directly Division of responsibility The classes and modules implementing OSGi services focus on providing the functionality of the domain. Data Calculus The EventHandlers are the responsible of calling the appropriate services and their operations to perform a certain command Security, communications and other horizontal issues are handled at event/message level or above. Integration with third party tools There are more tools providing services for control than tools relying on event or messages.

54 How does FAST know what providers are available? FAST AUTOMATED FACTORY OSGi service This is service that we have implemented taking what we liked from the technologies defined in the previous slide. It basically allows a user of the AutomatedFactory service to request the creation of an instance of an object just knowing the interfaces it needs to implement and the parameters for its construction/initialization. Also allows querying for how many implementations are available for a certain interface and notifications.

55 How does FAST handle any/all data types? FAST should be able to handle all Java Types Should because the only way to prove it doesn't is to find a data type it cannot handle which has not happened yet. Internally it makes heavy use of reflection so basically, as long as reflection is available for a type and the object can be created into memory there should be no problem to pass it as a parameter to the transformations/bots and use it. Runtime expression languages are the system that FAST uses to transform a representation of an object into an actual Java Object Thus FAST can handle virtually any data type which has a Java implementation to handle it.

56 What are runtime expression languages? Basically they are plain text or strings which represent an expression or an object. They need to be evaluated and upon evaluation they return a Java Object OCL XPATH FAST String Substitution Language (VAR) FAST Java Constructor Language Any other... These languages need to be evaluated in runtime and for that to happen there must be plugins providing a FAST EvaluationContext.

57 What is a FAST Evaluation Context? It is the interface that classes need to implement in order to register a new language provider. Registration takes place automatically using the Automated Factory. This interface is not simple, and the number of methods that need an actual implementation instead of an empty one will vary depending solely on the nature of the language.

58 How do you use the expression languages? Via FAST Expression service With an ID of the language you are trying to interpret it will allow you to get an Evaluation context for it. Once you have the evaluation context you can populate it with variables and other contextual information (if relevant). Then you just need to call the method evaluate with the string you want to evaluate, and optionally the return type you expect and a placeholder for errors for feedback. If you can specify in you DSLs, GenRequestProviders, or anywhere a method for extracting the ID of the expression language and the expression content as a string the you can use this system. Which means that if a new expression language gets supported, it will be usable automatically by any user of the FAST Expression service.

59