Karaf Cellar 4.1.1 deserialization issue

Due to a import limitation on the Cellar project, it is impossible to deserialize objects outside the cellar or hazelcast namespace:

[KARAF-5636] Enable dynamic import on cellar-hazelcast and dynamic import in Cellar DOSGi in order to support user defined class transport

The issue is solved in the next version, but there is a simple workaround. Make sure your bundles containing the classes for deserialization have the following namespace added to the manifest import-package statement:

org.apache.karaf.cellar

Forcing this into the manifest with the maven bundle plugin is not that straightforward when the bundle does not contain a real dependency on Cellar. But it is possible by declaring the namespace optional:

<Import-Package>

    org.apache.karaf.cellar*;resolution:=optional,*

</Import-Package>

On Karaf startup or on bundle install, Cellar will scan the import statements and detect and register your API bundles (OSGi extender pattern). Now it is able to find the required classes for deserialization and do a tccl switch to load it from the API bundle classpath.

Happy clustering...

JSR 349 Hibernate Bean Validation 1.1 on OSGi

Validation of objects and object graphs is a cross cutting concern and used within several architectural layers, and therefore should be separated from the business logic. The JSR349 specification describes a mechanism using annotations. At the time of writing, the latest reference implementation of the JSR 349 spec is the Hibernate Validator version 5.4.2.Final. The metadata API can be configured by XML, and the constraints can be extended by creating custom validation annotations.

Hibernate Validator requires special handling on OSGi, a proof of concept is located under net.betzel.javax.osgi.beanvalidation. Build the project and drop the produced Karaf archive file into the Karaf deploy folder to see the mechanism in action.

Readables:

http://beanvalidation.org
http://www.baeldung.com/javax-validation

JavaCPP libpostal preset

Parsing and normalizing street addresses around the world

Street addresses are among the most difficult artifacts of human language for any piece of parsing software. If you have ever tried it yourself, you know what i am talking about. Having to parse addresses from around the world where each country’s addressing system has its own set of conventions and peculiarities makes this even more difficult.

Searching the net for solutions to this problem i stumbled upon libpostal: a multilingual street address parsing/normalization library, written in C, that can handle addresses all over the world.

Libpostal uses machine learning and is informed by tens of millions of real-world addresses from OpenStreetMap. It currently supports normalization's in 60 languages and can parse addresses in more than 100 countries.

Since my veins are filled with caffeine i needed a Java JNI bridge to span my platform gap. There are bindings available under jpostal,  but having to hassle with all the compiler stuff and the native library handling in your project afterwards makes this not an attractive option.
To ease Java integration there is a great JNI library called JavaCPP. A tool that can not only generate JNI code but also build native wrapper library files from an appropriate interface file written entirely in Java. It can also parse automatically C/C++ header files to produce the required Java interface files. As a topping it generates a Maven artifact containing the whole setup including the specific platform native lib and loading capabilities, ready for use in your Maven projects.
I have used some presets of this project several times before and i believe it is a perfect fit for libpostal, giving the opportunity to add a complete new preset to this great project.

A big thanks to Samuel Audet for the support he provided throughout the process, providing the needed native platform knowledge i was lacking.

The 1.4 release also incorporates the new Systems preset, providing low level API access for Linux, MacOS an Windows.

Resources:

libpostal
bytedeco blog

JavaCPP

JavaCPP Presets

Dynamic Vaadin on OSGi

Vaadin 8 - Java 8 - Karaf 4 - OSGi 6 : The easy way to real µservice web apps.

There have been numerous articles on describing how to avoid or split up a monolithic architecture into a micro-services setup. A YouTube presentation of Christian Schneider springs to mind (audio gets better after about a minute). I especially like this slide that says it all:

OSGi has done a great job enabling µservice architectures and enforcing clean modular design on the middleware and backend layers but i have not had the chance of putting it into action on the frontend. Finding a modular web UI technology was no great deal, thanks to the people at Vaadin and the basic OSGi integration they have mastered on there latest major version. Getting the two frameworks to play nice is still no trivial task, and resources on the net for Vaadin 8 on OSGi are rare. So, time to get my hands dirty.

My requirements for a modular UI are:

• Split the UI in vertical modules based on separation of business concerns
• Runtime addition, removal and updating UI modules and configuration
• Communicate between these modules using events and/or messaging
• Discover remote UI modules using DOSGi
• And maybe we can reach the ultimate goal of SCS, but this could be too much ivory

I cooked up a small setup that demonstrates a possible way of dynamic updating the UI using OSGi declarative services. To get it up and running do the following:

First check out and maven install ops4j-pax-web version 6.0.7.1 from my github account as this contains several corrections to make Vaadin-push work on Karaf. Details can be found here.

Download Karaf 4.1.2 extract and execute.  On the Karaf console add the pax web feature:

feature:repo-add mvn:org.ops4j.pax.web/pax-web-features/6.0.7.1/xml/features

and install the pax-http-whitboad extender:

feature:install pax-http-whiteboard/6.0.7.1

Maven install this project producing a Karaf Archive file (osgi-karaf-vaadin-kar-2.0.0.kar)

Drop this file into the deploy folder found under the Karaf main folder to install all needed bundles at once.

Open up a web browser and open http://localhost:8181/service to display the Vaadin SPA.

Start the dynamic bundle on the Karaf console to see the dynamic parts working.

Any new Ideas regarding this project are always welcome!

Background information:

Christian Posta: Prefer verticals not layers and The Hardest Part About Microservices: Your Data
innoQ: Self contained systems
Arun Gupta: Monolithic to Microservices Refactoring for Java EE Applications

Vaadin 8 on OSGi

Java 8 - OSGi 6 - Karaf 4 - Vaadin 8

 

The new Vaadin 8 release comes with improved OSGi integration and promises true dynamic and modular UI development. There is not much information on the net concerning this topic so i created this simple example to get up and running quickly. The service is packaged as a JAR file with a valid OSGi bundle manifest created by the bnd-maven-plugin. All the dependencies of the application are OSGi bundles and get installed on deploy.

For more information about Vaadin OSGi support please visit  https://vaadin.com/docs/framework/v8/advanced/advanced-osgi.html.

The project is published on GitHub: Vaadin on OSGi.
Be aware of issue 10220 on version 8.1.x .

Have fun analyzing.

Lightning Memory-mapped Database on OSGi

Now and then i get what i call a niche requirement, something that does not hold to any standard or good practice. These kind of requirements are usually very development intensive, time consuming and great fun, or a real PITA when having to deal with a deadline.

I am in need of a fast, local, in process atomic counter for daily file name creation, which is thread and crash save, runs within a Java/OSGi environment and can participate in a XA transactions. The files being published are getting consumed and modified by other applications as soon as they are marked with a done file, making a stateless file counting solution impossible. A RDBMS springs to mind thinking of state, but the overhead in production is unproportional as the data generated is only valid for a short period of time and in limited scope.

After turning my toolbox upside down, finding nothing even remotely fitting this job description, I heated up the search engines ready to burn time. My data structure is simple and the amount of data generated is small so a simple key/value store should be just right. This results in a long hit list, but only few libraries support file persistence, and only one made a near perfect match: lmdb. Let me emphase, near perfect match, since it is no Java, no OSGi and has no transactional integration.

The Symas page lists wrappers, and sure enough, java is listed with a JNI implementation called lmdbjni. Strange thing is that this binary has no more Maven support for Windows, MacOS and Android from 0.4.7 onward, bummer. While browsing the github repo i stumbled (by accident via RxLMDB) upon lmdbjava supporting the latest and greatest lmdb version.

So, Java is tackled with a big thanks to the people of lmdbjava. OSGi brings its own mechanism for handeling runtime files and since version 0.0.6 of  lmdbjava the path to the lmdb binary, extracted by OSGi, can be set manually to allow the JNI part to link to it. A example bundle is on github and i leave the JCA integration as exercise.

Happy coding.

Twelvemonkeys on OSGi

Using Java for working with images has never been first choice, especially if the requirement for advanced computer vision functionalities needs to be met. For these cases please study OpenCV and JavaCV.
For the more daily use cases there is of course the standard Java ImageIO, providing limited image format support. My project required extensive MTIFF with CCITT compression capabilities and the good old JAI-ImageIO seemed to be a good candidate. But MTIFF support requires some extra steps to be taken, and the 10 jear old code base is missing active development. I wanted something fresh and after some research i found myself staring at the github page of twelvemonkeys ImageIO.
TwelveMonkeys is a pure Java based collection of plugins and extensions for Java's ImageIO with few optional dependencies, and under active development. It provides all i need, with a flick of the service provider switch. But lets not make this too easy now. The library is not bundled like JAI, and since i am a heavy OSGi user i want to put it on Karaf.
The first thing to do is get the jars in a bundle, ready for Karaf deployment. The Apache-felix-maven-bundle-plugin tool brings the solution in the form of embedding dependencies, putting the twelvemonkeys jars inside of a single bundle. The registration of these plugins with ImageIO gets handled by the classloader through the javax.imageio.spi.ServiceRegistry. This is a type of Java service loader responseble for finding ImageIO extensions in the classpath under the META-INF/services folder, bringing us straight to the core of our problem. Every OSGi bundle has its own classloader, meaning any image plugin loaded in one bundle is not visible to ImageIO in others. This is also true for every other library on OSGi using the java service loader mechanism.
The OSGi Enterprise R5 ServiceLoader Mediator specs discribe a standard solution for this kind of integration problem, and the reference implementation is Apache Aries Spi Fly. Under the hood it uses some fancy runtime bytecode weaving, resulting in a short classloader switch as soon as the ImageIO Serviceregistry in a bundle starts to look for plugins on the classpath, loading the plugins from the bundle containing the twelvemonkeys embedded dependencies.
Getting the MTIFF utilities in the contrib package functional under OSGi is as simple as adding a  blueprint or declarative service to the bundle containing the twelvemonkeys jars.

Sample project on github.

Recommended reading:

OSGi Alliance javautilserviceloader in OSGi