G l a r I m y Presentation on OSGi with Apache Karaf Krishna Mohan Koyya Proprietor & Principle Consultant Glarimy Technology Services Benguluru Bharat http://www.glarimy.com krishna@glarimy.com
. The Classroom Protocol Few Administrative Formalities Sign in the attendance sheet, if provided Fill-in the feedback at the end of sessions, if provided Derive Maximum Value Ask questions, any time Participate in discussions and lab exercises Avoid brining personal or office work to the class room Stay away from internet while class in the session Lets Learn Together Switch-off or mute personal phones Leave/enter the class room without disturbing the class Take phone calls outside the class room Keep the class room clean and professional No cross discussions Reference Material Download from http://www.glarimy.com No rint/cd. Protect Environment!
Glarimy Technology Services About Glarimy Established in 2008 and Registered in 2010 Based out of Benguluru, Bharat (Bangalore, India) Business Interests Technology Consulting Architectural Appreciation Architectural Review Prototyping Corporate Training Software Design, Architecture and Processes Web 2.0 Technologies Standard and Enterprise Technologies Weekend Public Workshops One-day Executive Workshops on Saturdays Business Reach Through 3 rd Party Vendor Organizations and Direct Relationships Geographies: Banguluru, Chennai, Pune, Hyderabad, Kochi and etc., Client Base: HSBC, Robert Bosch, Oracle, Alcatel-Lucent and etc.,
Krishna Mohan Koyya Career spanning across 17 years 9 Years into Software Engineering with Cisco Systems, Wipro/Lucent and HP 6 Years as Principle Consultant and Founder of Glarimy Technology Services 1 Year as CEO at Sudhari IT Services, Bangalore 1 Year as HoD at Sasi Institute of Technology and Engineering, Tadepalligudem Proven Delivery Web 2.0: ExtJS, Dojo, Jquery Frameworks, HTML5, CSS3, XML Enterprise Java: Spring, Hibernate, EJB, JSF, OSGi, Web Services Design Patterns, Architectural Patterns, TDD Technology Expertise Distributed Systems and Network Management Systems Object Oriented Systems Development Infrastructure, Middleware and Web Layers with Java and Javascript Academics M.Tech (Computer Science & Tech) from Andhra University, Visakhapatnam BE (Electronics & Comm. Engg) from SRKR Engg. College, Bhimavaram
OSGi with Karaf Getting Started The what and why of OSGi An architectural overview of OSGi First Look at Bundles Introducing bundles Configuring bundles Defining bundles with metadata OSGi bundle lifecycle Lifecycle and modularity Working with Karaf Setting up Karaf Karaf Features: Console and Logging Deploying bundles Working with Bundles Splitting an application into bundles Versioning packages and bundles OSGi dependency resolution Ordering bundle activation Starting bundles lazily
OSGi with Karaf Advanced Bundle Management Handling exports and Imports Requiring bundles Dividing bundles into fragments Turning JARs into bundles Cyclic Dependencies OSGi services in action Extender Pattern Compendium Services Event Services Configuration Management Services Service Locater Service Declarative Services Working with BluePrint OSGi and Spring Framework Enterprise OSGi Migrating tests to OSGi Debugging bundles Solving class-loading issues Tracking down memory leaks 2014, Glarimy. All rights reserved. http://www.glarimy.com
OSGi Bundle Basics An OSGi bundle as the unit of modularization It must be a cohesive and self-contained unit It must explicitly define its dependencies to other modules and services. It must also explicitly define its external API. A bundle is a.jar files With additional meta information in META- INF/MANIFEST.MF file. The MANIFEST.MF file is part of the standard Java specification. OSGi adds additional metadata. Any non-osgi runtime will ignore the OSGi metadata. Therefore OSGi bundles can be used without restrictions in non-osgi Java environments.
OSGi Bundle Basics A bundle is uniquely identified by its name and version Bundle-SymbolicName Starts with the reverse domain name of the bundle author Bundle-Version property (major.minor.patch) patch is increased if all changes are backwards compatible. minor is increased if public API has changed but all changes are backwards compatible. major is increased if changes are not backwards compatible Both are defined in MANIFEST.MF file
OSGi Bundle Basics An Example Manifest File Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: Bank-Api Plug-in Bundle-SymbolicName: com.glarimy.osgi; Bundle-Version: 1.0.0 Bundle-Activator: com.glarimy.osgi.activator Require-Bundle: org.eclipse.ui, org.eclipse.core.runtime Bundle-ActivationPolicy: lazy Bundle-RequiredExecutionEnvironment: JavaSE-1.7 Export-Package: com.glarimy.osgi.api
OSGi Bundle Basics Example Bundle Manifest Headers Bundle-Name: Defines short description of the bundle Bundle-SymbolicName: Defines the unique identifier of the bundle Bundle-Version: Defines the bundle version Bundle-Activator: Defines an optional activator class Implements the BundleActivator interface. Bundle-RequiredExecutionEnvironment (BREE) Defines the Java version to run the bundle Bundle-ActivationPolicy: Defines the policy to decide when to activate a bundle Bundle-ClassPath: Location of the bundle classes. Defaults to root of the bundle
OSGi Bundle Basics A bundle define its dependency to other bundles or packages. Based on a certain or range of versions of other bundles Classes in the non-dependent bundles can not be found Except the packages/classes from JVM. A bundle defines its API via the Export-Package Identifier All packages which are not explicitly exported are not visible to other bundles. Each bundle has its own OSGi class-loader. It enforces restrictions on access to various classes according to the defined dependencies.
OSGi Bundle Lifecycle
OSGi Bundle Lifecycle OSGi manages dependency among the bundles. Installation The bundle is persisted into local cache Assigns a unique ID for the bundle (INSTALLED) Resolution Reads the MANIFEST.MF file Ensures that all dependent bundles are resolved (RESOLVED) Recursive process Chooses a bundle with highest version Chooses a bundle with lowest ID If the dependencies are not met, then the bundle is not resolved Activation Activates the bundle as per activation policy Ensures that all dependent bundles are activated
Manifest Headers Bundle-Classpath An ordered, comma-separated list Of relative bundle JAR file locations To be searched for class and resource requests. The default value DOT (.) if and only if there is no specified value Example Bundle-ClassPath:.,other-classes/,embedded.jar
Manifest Headers Export-Package A comma-separated list Of internal bundle packages To expose for sharing with other bundles. Unless exported, no package is visible outside the bundle Exception: Java Packages are always visible Example Export-Package: com.glarimy.api,com.glarimy.service Export-Package: com.glarimy.api; vendor="manning", com.glarimy.service; vendor= Glarimy Export-Package: com.glarimy.api; com.glarimy.service;vendor= Glarimy Export-Package: com.glarimy.api; com.glarimy.service; version="2.0.0"
Manifest Headers Import-Package A comma-separated list of packages As needed by internal bundle code From other bundles Can not import bundles, unless exported by the other bundles Example Import-Package: com.glarimy.api Import-Package: com.glarimy.api,com.glarimy.util Import-Package: org.osgi.framework; version="[1.3.0,2.0.0)
Manifest Headers Version Range Specification Square bracket for inclusive range Parenthesis for exclusive range Default 0.0.0: Any version Examples [min,max) : min <= x < max [min,max] : min <= x <= max (min,max) : min < x < max (min,max] : min < x <= max min : min <= x
Bundle Resolution Bundle Resolution Process The class is from a package starting with java The parent class loader is asked for the class. If the class is found, it s used. If there is no such class, the search ends with an exception. The class is from a package imported by the bundle The framework asks the exporting bundle for the class. If the class is found, it s used. If there is no such class, the search ends with an exception. The bundle class path is searched for the class If it s found, it s used. If there is no such class, the search ends with an exception Resolves a bundle automatically when another bundle attempts to use it.
OSGi Bundle Resolution Different versions of the same package in the same running JVM Highest priority to already-resolved candidates Multiple matches of resolved candidates are sorted according to version and then installation order Next priority to unresolved candidates Multiple matches of unresolved candidates are sorted according to version and then installation order Inconsistent import-export or dynamic deployments may cause class-cast exception
OSGi Bundle Resolution Uses Directive Attached to exported packages A comma-delimited list of packages Exposed by the associated exported package Example If package org.osgi.service.http uses package javax.servlet, importers of org.osgi.service.http are constrained to the same javax.servlet Export-Package: org.osgi.service.http; uses:="javax.servlet"; version="1.0.0" Import-Package: javax.servlet; version="2.3.0 Uses constraints are transitive
Updating Bundles Updates The new bundle version is put in place, but the old version is still kept around So bundles depending on it can continue loading classes from it. Each time an update without a refresh is performed, an another version of the bundle is introduced. Uninstalls The bundle is removed from the installed list of bundles, but it isn t removed from memory. Again, the framework keeps it around so dependent bundles can continue to load classes from it. Update followed by Refresh
Managing Exports Importing own packages It s generally a good idea to import and export interface packages Scenario: P is an interface package. A is a client bundle importing package P Both bundles B and C exporting P along with some implementations. OSGi allows either B or C only to export P as part of resolution. By importing as well as exporting P, either A or B can use/provide object of classes in P. Less number of versions of classes causes less number of conflicts. Manifest Snippet in bundles B and C Export-Package: com.glarimy.api; Version= 1.0.0. Import-Pacakge: com.glarimy.api; Version= 1.0.0.
Managing Exports Mandatory Export Attributes Bundle resolution is largely controlled by import declaration Only attributes mentioned in the Import are considered Scenario: Import matches the Export Declaration Export-Package: com.glarimy.dir.api;version= 1.0.0 ;license= gts Import-Package: com.glarimy.dir.api;version=[1.0.0,2.0.0) Scenario: Import fails to match the Export Declaration Export-Package: com.glarimy.dir.api;version= 1.0.0 ;license= gts ;mandatory:= license Import-Package: com.glarimy.dir.api;version=[1.0.0,2.0.0) Scenario: Import matches the Export Declaration Export-Package: com.glarimy.dir.api;version= 1.0.0 ;license= gts ;mandatory:= license Import-Package: com.glarimy.dir.api;version=[1.0.0,2.0.0);license= gts
Managing Exports Include and Exclude Directives To specify a comma-delimited list of class names to exclude or include from the exported package, respectively. A class is only visible if it is matched by an entry in the include directive and not matched by any entry in the exclude directive. Wildcards are allowed Default value for Include is * (All) Default value of Exclude is empty string (None) Scenario: To export everything except Factory classes Export-Packages: com.glarimy.api;exclude:= *Factory Scenario: To export all Factory classes Export-Packages: com.glarimy.api;include:= *Factory
Managing Exports Duplicate Exports A given bundle can only see one version of a class Importing a package more than once is not allowed However, duplicate exports of the same package is allowed Scenario: Resolving for strict imports on 1.0.0 and 1.2.0 Export-Package: com.glarimy.api:version= 1.0.0,com.glarimy.api:Version= 1.2.0 Scenario: Masking some of the packages for different imports Export-Package: com.glarimy.api; version="1.0.0"; exclude:="*factory", com.glarimy.api; version="1.0.0"; license= gts"; mandatory:= license"
Managing Imports Resolution Directive Can be mandatory for successfull resolution Can be optional for successful resolution Scenario: The logger is an optional library Import-Package: org.log4j.logger;version:2.3.0;resoltion:= Optional Dynamic Import Can be used when the package name is not known in advance Used as the last resort in resolution process Code must handle exceptions Scenario: Load the mysql driver classes dynamically DynamicImport-Package=com.mysql.jdbc.*;Version= 5.0.0
Activation Policy Bundle-ActivationPolicy: lazy Activates the bundle only when a class is loaded from it It remains in the started state until then Bundle-ActivationPolicy: lazy; include:="com.acme.service"
Requiring Bundles Require-Bundle Imports all packages exported from the other bundle Require-Bundle: A; bundle-version="[1.0.0,2.0.0)"; resolution:="optional" These packages are used only if Import-Package fails Enables splitting of bundles Splitting packages into more than one bundle PackageAdmin look for all Require-Bundles for imports Directive: reexport When bundle B requires bundle A with reexport visibility, any packages exported from A become visible to any bundles requiring B. Require-Bundle: A; bundle-version="[1.0.0,2.0.0)"; visibility:="reexport"
Working with Split Packages Avoid, if possible Always export split packages with a mandatory attribute to avoid unsuspecting bundles from using them. Use either Require-Bundle or Import-Package plus the mandatory attribute to access parts of the split packages. Bundle-SymbolicName: A Bundle-Version: 2.0.0 Export-Package: org.foo.bar; version="2.0.0"; split="part1"; mandatory:="split To provide access to the whole package, create a facade bundle that requires the bundles containing the package parts and exports the package in question. Bundle-SymbolicName: C Bundle-Version: 1.0.0 Require-Bundle: A; version="[2.0.0,2.1.0)", B; version="[2.0.0,2.1.0)" Export-Package: org.foo.bar; version="2.0.0"
Bundle Fragments Fragment-Host Bundle can be split into host & fragments This header specifies the single symbolic name of the host bundle on which the fragment depends, along with an optional bundle version range A host bundle is technically usable without fragments, but the fragments aren t usable without a host. Fragments are treated as optional host-bundle dependencies by the OSGi framework. But the host bundle isn t aware of its fragments Fragment-Host: org.foo.hostbundle; bundleversion="[1.0.0,1.1.0)"
Bundle Fragments Resolution The content and metadata from the fragments are conceptually merged with the host s content and metadata. Rather than giving each fragment its own class loader, the framework attaches the fragment content to the host s class loader. Two classes in the same package, but loaded by two different class loaders, can t access each others packageprivate members. By loading fragment classes with the host s class loader, the fragment classes are properly recombined to avoid this issue. This isn t true for split packages accessed through Require- Bundle.
OSGi with Karaf Bundle-RequiredExecutionEnvironment This header specifies a comma delimited list of supported execution environments. Bundle-RequiredExecutionEnvironment: J2SE- 1.4,J2SE-1.5,JavaSE-1.6 It s possible to install a bundle on a given execution environment even if it s not compatible with it, but it wouldnt be resolved unless its required execution environment matches the current one.
Manifest Headers Bundle-ActivationPolicy A marker to tell the OSGi runtime whether this bundle should be activated Bundle-Activator A class implementing the BundleActivator which will be called at bundle activation time and deactivation time. Bundle-Category A comma-separated list of category names. Bundle-ClassPath Where to load classes from from the bundle. Bundle-ContactAddress header is used to store a human-readable physical address of where to find out more information about the bundle.
Manifest Headers Bundle-Copyright May be used to store some text indicating the copyright owner of the bundle. Bundle-Description A header allows you to provide a human-readable textual description of the bundle. Bundle-DocURL Is a textual header, which can contain a URL (typically a website) that the user can find more information about the bundle. DynamicImport-Package Is not widely used. Its purpose is to allow a bundle to be wired up to packages that may not be known about in advance.
Manifest Headers Export-Package consumable by other bundles. A comma-separated list of fullyqualified packages, to be consumed by other bundles, often with a version attribute. If not specified, the version defaults so 0.0.0. Bundle-Host Declares this bundle to be a Fragment, and specifies which parent bundle to attach to: Import-Package A header is used to declare dependencies at a package level from the bundle. The Bundle-ManifestVersion Is not mandatory, but if missing defaults to version 1. OSGi R4 introduced Bundle-ManifestVersion: 2, and so in practice, almost 2014, Glarimy. All rights reserved. http://www.glarimy.com
Manifest Headers Bundle-Name is a textual identifier for the bundle. Bundle-NativeCode contains information about native libraries that should be loaded by this bundle Require-Bundle header is used to express a dependency on a bundle's exports by reference to its symbolic name instead of via specific packages. Bundle-RequiredExceptionEnviornment presents a list of Execution Environments that must be present for this bundle to install.
Manifest Headers Service-Component Is a header used in Declarative Services to define many individual XML files (or patterns) which can be used to declare components. Bundle-SymbolicName Is used together with Bundle-Version to uniquely identify a bundle in an OSGi runtime. Bundle-Vendor Contains a human-readable textual description string which identifies the vendor of the bundle Bundle-Version Specifies the version of this bundle, in major.minor.micro.qualifier format. The default, if not specified, is assumed to be 0.0.0. 2014, Glarimy. All rights reserved. http://www.glarimy.com
The Final Resolution Requests for classes in java. packages are delegated to the parent class loader; search stops Requests for classes in an imported package are delegated to the exporting bundle; search stops Requests for classes in a package from a required bundle are delegated to the exporting bundle; searching continues with a failure The host bundle class path is searched for the class; searching continues with a failure Fragment bundle class paths are searched for the class in the order in which the fragments were installed. Searching stops if found but continues through all fragment class paths and then to the next step with a failure. If the package in question isn t exported or required, requests matching any dynamically import package are delegated to an exporting bundle if one is found. Searching stops with either a success or a failure
The Extender Pattern Some bundle in the application acts as the extender: it listens for bundles being started and/or stopped. When a bundle is started, the extender probes it to see if it s an extension bundle. The extender looks in the bundle s manifest (using Bundle.getHeaders()) or the bundle s content (using Bundle.getEntry()) for specific metadata it recognizes. If the bundle does contain an extension, the extension is described by the metadata. The extender reads the metadata and performs the necessary tasks on behalf of the extension bundle to integrate it into the application. The extender also listens for extension bundles to be stopped, in which case it removes the associated extensions from the application.
Services They are not proxies They are real objects Nullify the references for proper garbage collection Hold them only for using not for storing Only services of same version Services might not be visible though active Use Service Factory It creates an instance on demand Same instance to the same bundle Destroys once the bundle is done with it
Services objectclass Class name the service was registered under. Can t be changed after registration. service.id Unique registration sequence number, assigned by the framework when registering the service. service.pid Persistent (unique) service identifier service.ranking Ranking used when discovering services. Defaults to 0. Services are sorted by their ranking (highest first) and then by their ID (lowest first) service.description Description of the service service.vendor Name of the vendor providing the service 2014, Glarimy. All rights reserved. http://www.glarimy.com
Services Perform attribute matching: (name=john Smith) (age>=20) (age<=65) Perform fuzzy matching: (name~=johnsmith) Perform wildcard matching: (name=jo*n*smith*) Determine if an attribute exists: (name=*) Match all the contained clauses: (&(name=john Smith)(occupation=doctor)) Match at least one of the contained clauses: ( (name~=john Smith)(name~=Smith John)) Negate the contained clause: (!(name=john Smith))
OSGi Event Admin Provides a publish-subscribe model for handling events. Based on OSGi Event Admin Service Specification. Dispatches events between Event Publishers and Event Subscribers By interposing an event channel. Publishers and Subscribers have no direct knowledge of each other Communication can be synchronous and asynchronous. Mechanism org.osgi.service.event.event : topic and properties org.osgi.service.event.eventadmin: sendevent, postevent org.osgi.service.event.eventhandler: handleevent org.osgi.service.event.eventconstants: EventConstants.EVENT_TOPIC
OSGi Config Admin Makes it possible to write dynamically configurable services Configuration updates without restart Programmatic configuration Based on OSGi Config Admin Service Specification. Manages configuration of managed services By persisting configuration By updating the interested parties about changes in configuration Mechanism org.osgi.service.cm.managedservice updated(dicitionary) org.osgi.service.cm.configurationadmin update(dictionary) service.pid 2014, Glarimy. All rights reserved. http://www.glarimy.com
OSGi Blueprint Services Bean Provides components with the same basic semantics as Spring beans: Construction via reflective construction or static factory methods Support for singletons or prototype instances Injection of properties or constructor arguments Lifecycle callbacks for activation and deactivation Service Allows components to provide OSGi services Environment Provides components access to the OSGi framework and the Blueprint container, including the bundle context of the component
OSGi Blueprint Services Reference Gets a single service from the OSGi service registry for the component based on the service interface and an optional filter over the service properties Reference list Gets one or more services from the OSGi service registry for the component based on the service interface and an optional filter over the service properties
OSGi Blueprint Service
OSGi Blueprint Services Consuming Services <blueprint <bean id= directory" class= com.glarimy.glarimydirectory init-method="activate destroy-method="deactivate"/> <reference-list id="shape interface="org.foo.shape.simpleshape availability="optional"> <reference-listener bindmethod="addshape unbindmethod="removeshape ref="paintframe"/> </reference-list> <service id="window 2014, Glarimy. All rights reserved. http://www.glarimy.com
Testing OSGi Bundles Junit Runner Mocking with Mockito and etc., PAX-Exam Pax Exam is a testing toolkit that addresses the need for bundlelevel testing. When a Pax Exam test is run, it starts an OSGi framework, installs and starts a selection of bundles, and then makes an OSGi BundleContext available. @Inject Injects the BundleContext ServiceTracker helps in getting the service for assertions
Common Issues The identity of a class at execution time is not only associated with its fully qualified class name, it s also scoped by the class loader that loads it. The exact same class loaded by two different class loaders is loaded twice by the Java VM And is treated as two different and incompatible types. Casting an instance of one to the other, results in ClassCastException. When embedding and exporting common open source packages in a bundle, import them as well!
Class Loading Issues ClassNotFoundException Package-Private classes only visible to classes from the same class loader Often found in split-packages ClassCastException itemtracker.open(true) discovers all the services Results in incompatible services Missing/Mismatched Uses Class.forName()
Debugging in Karaf Use remote debugging. > bin\karaf.bat debug set KARAF_DEBUG=true Then, you can launch Karaf using the usual way: Last, inside your IDE, connect to the remote application (the default port to connect to is 5005). This option works fine when it is needed to debug a project deployed top of Apache Karaf.
Thank You Glarimy Technology Services Consulting & Corporate Training: http://www.glarimy.com Weekend Executive Workshops: http://workshops.glarimy.com Queries: krishna@glarimy.com Twitter: @Glarimy