A quick look at Apache Karaf

Today, I decided to take a brief look at Apache Karaf. Apache Karaf is an OSGI runtime that embeds Apache Felix as OSGI container. It ships with couple of OSGI bundles and provides a nice shell interface.

I am not really covering some technical details in this post. I haven't deployed an application under Karaf yet, as I would want to do a bit more than the typical "Hello World".

My previous OSGI experience in context
Few years ago, I was trying to understand OSGI and I created an OSGI based IRC bot, called Jerkbot. Prior to that experience, OSGI looked inaccessible, unpopular with a low return value. Even after reading couple of OSGI articles, I still couldn't figure out 100% what to do with it.

Building Jerkbot
Within a day, I was able to deal with most of the bot features. I went quickly over Apache Felix tutorials and I was ready to write some code.

• As soon I introduced a persistence layer, I hit minor issues with JPA(classloading). After struggling with OpenJPA and Hibernate, I settled with EclipseLink.
• I used mainly Declarative Services with annotations to save time, instead of writing couple of BundleActivators and/or XML descriptors.
• SpringSource had a Maven repository with OSGIfied jars, so I used it instead of wrapping every single non OSGI jar

My build process involved the following :

• Run the Maven install goal at the root of the multi-module project
• Copy all the files with rsync from  my desktop to a laptop running FreeBSD (/)
• Setup the Apache Felix configuration to specify the startup order of each bundle depending on what I was testing
• Start Apache Felix from the command line with my configuration

Alternatively, I could update the bundles within the Apache Felix console without restarting the program sometimes.

Packaging
When the time came to actually shipping the application, I was annoyed. I had to write a Maven assembly configuration to package the application as a zip file. After playing with couple of deployment options, I had generated few temporary folders and it became difficult to identify what was still needed.

I ended up only shipping source code, as the PAX Maven plugin could run the application from the project tree.

If I recall correctly Karaf already existed, as mentioned in a blog post about Apache Karaf history. I believe that I did look at it, but not in details. I probably couldn't figure out quickly whether or not it would be worth it (read 2 paragraphs and give a yes/no/maybe answer).

Application Frond-end
As the bot itself had a JMX interface, I could just use JConsole to hook into the bot admin interface. I wasn't really planning on hooking up some Apache Felix commands.
With Karaf, the shell extension process seems similar to the one provided by Apache Felix.


Discovering Karaf

I was really impressed by the product's features and usability. Below are some of the topics that got my interest.

Installation
Installing Karaf on my iMac was pretty straightforward. The only problem that I run into was launching the webconsole as soon as I installed it. I got a NullPointerException and I just restarted Karaf. I guess that some initialization logic is probably missing in the bundle activation or something related.

Documentation
I usually don't read much technical documentation. The Karaf folks did a decent job with the user guide. I found myself scrolling and jumping only to topics of interest.

Console
The console is really neat with tab completion. The contextual help for each command is brief and easy to grasp.

Deployment
In additional to standard OSGI bundles, Karaf also includes support for wrapping non OSGI jar files. I guess that they achieved that with some integration with the BND tool. What really surprised me was the Maven integration in addition to standard deployment options : http://stackoverflow.com/questions/9414375/karaf-development

It is also possible to configure the available Maven repositories: https://cwiki.apache.org/KARAF/66-installing-additional-features.html

Remoting
Apache Karaf exposes couple of JMX MBeans, it has a nice Web Console and SSH access is also supported.

Security
Karaf developers really made a decent effort here. The JAAS integration seems rather smooth in addition to basic encryption services. I found a blog with a JAAS module example and I didn't expect it to be that easy http://iocanel.blogspot.ca/2010/09/karafs-jaas-modules-in-action.html

Opened questions (running foreground)
If I recall correctly, it is possible to start Apache Felix in the foreground by specifying a config property (embedded or something). I didn't figure out how to do it with Karaf, but I didn't search a lot.


Conclusion
I easily see myself using Apache Karaf on side projects in the near future. It provides many  features out of the box that would require the following :

• Adding couple of bundles to the OSGI container manually
• Writing few modules to support administration services with additional options. Especially with you can grab few OSGI bundles here and there that I only provide basic functionalities(Felix config/admin bundles, PAX bundles, etc.).

I still don't seem much OSGI adoption even though there are more articles and tutorials available. I believe that Apache Karaf makes OSGI somehow more "accessible". However, the OSGI adoption in the corporate world is very slow from what I see. OSGI needs more simplicity, more mediatization, more real life examples to reach many companies' doors.


References

• http://www.anova.be/files/OSGi%20with%20Apache%20Felix%20Karaf.pdf
• https://karaf.apache.org/manual/2.2.5/users-guide/index.html
• http://www.liquid-reality.de/pages/viewpage.action?pageId=131134
• https://cwiki.apache.org/KARAF
• http://felix.apache.org/site/index.html

The making of an OSGI based IRC Bot

I've released the source code of JerkBot, an IRC bot based on Jerklib and OSGI. JerkBot source code is a multi-module maven project. It's not some OSGI/Java blueprints as I'm no OSGI expert, plus JerkBot is basically a 24 hours effort without bug fixes.

The library itself is distributed under BSD license, but the bot provides plugins such as SVN for example using SVNKit , which is subject to other licensing terms.

The bot uses Declarative Services for OSGI and Java technologies such as :

• Apache Lucene for Javadoc Search
• Quartz for job scheduling(session tracking, pending registrations, etc.)
• Javamail to send emails for user registration
• EclipseLink for persistence
• A subset of JAAS for security
• JMX for administrative commands (accessible through JConsole or through the bot jmx command)
• The usual Jakarta libraries and couple of other libraries

I didn't provide scripting languages to the bot for security reasons, there's only JMX. I would prefer to offer scripting languages in a secure way with a custom SecurityManager to prevent drama from happening :-).
Let's say scripting is enabled, and someone accidently tries one or all the following instructions in a scripting language.

• File("/somepath").delete()
• System.exit(0)
• Download("http://website/hugefile.iso").saveToDisk();

JerkBot was roughly a one day effort with 4 rewrites(1 full day each), and of course bug fixes time to time:

• The first version was using traditional OSGI, well not so traditional :-), with BundleActivators service trackers and listeners, etc.
• The second rewrite used Declarative Services with manually written XML descriptors
• The third rewrite was based on Felix IPOJO. I would have preferred to use IPOJO, but found some annoyances(abstract based classes, JMX flexibility, etc.).
• The last rewrite is using Felix SCR but with annotations to generate XML descriptors for components. Simplify code, remove unnecessary abstractions, consistent logic, etc.

I'll be providing the binary distribution soon. The binary distribution contains everything necessary to run the bot : OSGI configuration files, Bot configuration, jars, a user/developer guide, etc.

What the bot doesn't provide is a logging mechanism, for IRC channel logs. To do it right, I think it would be better to create a new project. In my opinion, when a bot logs channel it should have, if possible, the following features:

• Configurable tasks to schedule flexible timed delivery of existing logs(local filesystem, ftp, samba share, ssh, http put, etc.)
• Configurable log format(HTML, CSV, TXT) with optional generation of html logs or text logs.
• Database storage or any other persistence mechanism(Apache Lucene Index vs flat text files, database logs, etc.)
• Ability to stop/start logging for every channel
• Optional Web front-end to publish logs(Could be a static HTML pages with Javascript search, JSF, Wicket, GWT, Rest interface + Apache Lucene or a DB for search).
  

Providing a quick and dirty plugin for channel logs would be trivial but not flexible :-). I started implementing it, but I decided to stop there.

The first draft of the bot manual can be found in the svn repository. Please bear with me for grammar and spelling mistakes, it was a written very quickly at early AM :-)

In JerkBot plugins are provided by OSGI bundles. I learned a lot from my previous experience with JPF when writing XPontus.

For now the bot is sitting on irc.freenode.net in the ##swing channel, running on an old laptop(FreeBSD-CURRENT).

Playing with OSGI and JPA

I've been writing an IRC bot for fun, but it's far from being done. My goal is to experiment with JPA in an OSGI environment.

At work, I am using Hibernate, XDoclet to generate the XML and JPA isn't coming soon :-). My last JPA application was a pastebin application, with Wicket, hibernate search, Lucene, etc.

I'm using a friend's library Jerklib with Dynamic JPA. I'm still having some minor issues when deploying but I'll probably find a solution soon.

About the application

Environment

I have a multi-project with Maven and I'm using the maven bundle plugin and I'm developing in Eclipse 3.5 on Debian testing.

I'm using openjpa , dynamic jpa and couple of other dependencies

Design overview

a) Commands implemented as plugins : The bot has a set of factoids(learn, forget, etc...). Each command gets created using a factory.

// message listener //String operation = getOperation();

CommandFactory factory = ServiceFromOSGI.getCommandFactory(operation);

// if the factory is not null, redundancy for the operation parameter

// as a factory can have multiple commands

Command command = factory.createCommand(operation);

ircChannel.say(command.render(ircMessageContext));

b) The command service listens for removal or installation of commands and updates itself.

After writing couple of "users' commands", I would need to implement some administration commands(load/unload plugins, irc specific tasks, etc.).

Problems

The dynamic discovery is failing for now, the persistence provider class is not resolved, it might just be a bundling problem for openjpa. I wrapped it myself.

Conclusion

It's strange that many open source projects still don't provide an OSGI manifest. The maven-bundle plugin is very trivial to use for maven enabled project and there's still bnd.

While OSGI is an interesting technology, I personally don't know anybody using it in the enterprise unless they are an Eclipse shop. I think that it's mostly due to

• the lack of "OSGI enabled jars"
• the fear that OSGI might introduce unnecessary complexity
• the lack of step by step complete examples(if possible with screencasts)

There's plenty of documentation about OSGI and lots of successful applications(Web, Desktop) using OSGI. Hopefully my bot will be one those applications :-)