About Java Software Installers and Launchers

1 Introduction

In the early stages of your software projects, it's a good practice to think about the distribution aspect. It doesn't matter how good a product is, if no one can perform the installation or run the application.

Generating software installers is not always easy, regardless of the installer product used.

If you're lucky enough to have a good commercial software installer, you can generate quickly packages without too much pain:

• Easy generation of application launchers.
• Good default settings with flexibility for splash screens, installer icons, pre-installation/post-installation actions, etc.
• Generation of software packages for a variety of platforms (OSX, Unix, Linux, Windows, etc.).

2 Application launchers

Now that your application development phase is complete, you may be worried about writing couple of launchers.

Writing a Java application launcher can become fairly complicated depending on :

• The amount of libraries dependencies used by the program.
• The configurations and properties to resolve or create in order to run the application properly.
• Any tasks that need to be executed prior to launching the application.

For Java based applications, below are the common steps performed by launchers scripts:

• Find the Java executable (JAVA_HOME detection if needed, common locations depending on the Operating System).
• Validate the Java version requirements as well as potential optional settings.
• Setup any environment variables or system properties needed by the application.
• Construct the Java classpath from the application dependencies.
• Perform any actions need prior to launching the program.
• Invoke the Java command with the classpath and JVM arguments to start the application.

ClassWorlds is a simple Java application launcher framework that has been around for a while. It superseeded the forehead framework launcher. Forehead was used by many tools such as Maven (now using ClassWorlds).

What makes ClassWorlds compelling is that it's really easy to bootstrap an application launcher without much effort. All that Classworlds needs is a simple java command that references a boot jar file and your application configuration file(to load libraries).

3 Software Installers and packaging

Depending on the application's type, target audience and operating systems, many options are available. Commercial Software package generators usually provide good results without too much work. Decent to really good products for Java software packaging include InstallAnywhere and Install4j.

Some free Java-oriented installers generators can be found on java-source.net. A popular choice is IzPack.

Generating software packages by hand still gives you an overall better user's installation experience at the expense of time and potential bugs (typos, logic errors, etc.).

When your installation packages are ready, you then need to test them on all supported platforms, just to be safe.

Zip distributions are very convenient for many users:

• No administration rights needed most of the time, which is useful when you don't have administrative rights on a machine.
• One step installation, just a matter of extracting a software archive.
• Easy uninstallation which is a simple folder deletion.

When generating software packages, don't forget the simple way of distributing files via zip archives.

A quick look at Apache Karaf

Today, I decided to take a brief look at Apache Karaf. Apache Karaf is an OSGI runtime that embeds Apache Felix as OSGI container. It ships with couple of OSGI bundles and provides a nice shell interface.

I am not really covering some technical details in this post. I haven't deployed an application under Karaf yet, as I would want to do a bit more than the typical "Hello World".

My previous OSGI experience in context
Few years ago, I was trying to understand OSGI and I created an OSGI based IRC bot, called Jerkbot. Prior to that experience, OSGI looked inaccessible, unpopular with a low return value. Even after reading couple of OSGI articles, I still couldn't figure out 100% what to do with it.

Building Jerkbot
Within a day, I was able to deal with most of the bot features. I went quickly over Apache Felix tutorials and I was ready to write some code.

• As soon I introduced a persistence layer, I hit minor issues with JPA(classloading). After struggling with OpenJPA and Hibernate, I settled with EclipseLink.
• I used mainly Declarative Services with annotations to save time, instead of writing couple of BundleActivators and/or XML descriptors.
• SpringSource had a Maven repository with OSGIfied jars, so I used it instead of wrapping every single non OSGI jar

My build process involved the following :

• Run the Maven install goal at the root of the multi-module project
• Copy all the files with rsync from  my desktop to a laptop running FreeBSD (/)
• Setup the Apache Felix configuration to specify the startup order of each bundle depending on what I was testing
• Start Apache Felix from the command line with my configuration

Alternatively, I could update the bundles within the Apache Felix console without restarting the program sometimes.

Packaging
When the time came to actually shipping the application, I was annoyed. I had to write a Maven assembly configuration to package the application as a zip file. After playing with couple of deployment options, I had generated few temporary folders and it became difficult to identify what was still needed.

I ended up only shipping source code, as the PAX Maven plugin could run the application from the project tree.

If I recall correctly Karaf already existed, as mentioned in a blog post about Apache Karaf history. I believe that I did look at it, but not in details. I probably couldn't figure out quickly whether or not it would be worth it (read 2 paragraphs and give a yes/no/maybe answer).

Application Frond-end
As the bot itself had a JMX interface, I could just use JConsole to hook into the bot admin interface. I wasn't really planning on hooking up some Apache Felix commands.
With Karaf, the shell extension process seems similar to the one provided by Apache Felix.


Discovering Karaf

I was really impressed by the product's features and usability. Below are some of the topics that got my interest.

Installation
Installing Karaf on my iMac was pretty straightforward. The only problem that I run into was launching the webconsole as soon as I installed it. I got a NullPointerException and I just restarted Karaf. I guess that some initialization logic is probably missing in the bundle activation or something related.

Documentation
I usually don't read much technical documentation. The Karaf folks did a decent job with the user guide. I found myself scrolling and jumping only to topics of interest.

Console
The console is really neat with tab completion. The contextual help for each command is brief and easy to grasp.

Deployment
In additional to standard OSGI bundles, Karaf also includes support for wrapping non OSGI jar files. I guess that they achieved that with some integration with the BND tool. What really surprised me was the Maven integration in addition to standard deployment options : http://stackoverflow.com/questions/9414375/karaf-development

It is also possible to configure the available Maven repositories: https://cwiki.apache.org/KARAF/66-installing-additional-features.html

Remoting
Apache Karaf exposes couple of JMX MBeans, it has a nice Web Console and SSH access is also supported.

Security
Karaf developers really made a decent effort here. The JAAS integration seems rather smooth in addition to basic encryption services. I found a blog with a JAAS module example and I didn't expect it to be that easy http://iocanel.blogspot.ca/2010/09/karafs-jaas-modules-in-action.html

Opened questions (running foreground)
If I recall correctly, it is possible to start Apache Felix in the foreground by specifying a config property (embedded or something). I didn't figure out how to do it with Karaf, but I didn't search a lot.


Conclusion
I easily see myself using Apache Karaf on side projects in the near future. It provides many  features out of the box that would require the following :

• Adding couple of bundles to the OSGI container manually
• Writing few modules to support administration services with additional options. Especially with you can grab few OSGI bundles here and there that I only provide basic functionalities(Felix config/admin bundles, PAX bundles, etc.).

I still don't seem much OSGI adoption even though there are more articles and tutorials available. I believe that Apache Karaf makes OSGI somehow more "accessible". However, the OSGI adoption in the corporate world is very slow from what I see. OSGI needs more simplicity, more mediatization, more real life examples to reach many companies' doors.


References

• http://www.anova.be/files/OSGi%20with%20Apache%20Felix%20Karaf.pdf
• https://karaf.apache.org/manual/2.2.5/users-guide/index.html
• http://www.liquid-reality.de/pages/viewpage.action?pageId=131134
• https://cwiki.apache.org/KARAF
• http://felix.apache.org/site/index.html

GWT in general and the CellTree widget

    I've been toying a bit with GWT and it's quite nice.

     In the past, I've used many frameworks to some extent, depending on what was needed : raw JSP with or without JSTL, Struts, VRaptor 1.x,  Cocoon, JSF, Velocity, Wicket, etc.
     I had more fun using Wicket than something like Struts 1.x for example...
     I never liked Struts but it seemed good enough compared to other frameworks at the time(Early ASP.NET versions were already more usable for medium size apps). If I were compelled to use something similar to Struts today, with the luxury of choice, I would likely select Stripes or another technology close to it (simple but flexible).
     Overall, I've always preferred component based Web Frameworks to the Actions based ones.

A) GWT and its UI frameworks (more than UI usually)
   GXT and others such as SmartGWT seem to be much slower than raw GWT for rendering but it is expected as the default look and feel looks good without any effort. The default look and feel is cool.

   I stumbled upon GXT buttons rendering. While the buttons are VERY beautiful, the amount of nested elements required for rendering is almost incredible.


1) Normal HTML

<input type="button" value="OK" class='dummyClass' />

2) GXT (dummy example to illustrate the problem)

<table cellspacing="0" role="presentation"
class="x-btn x-component x-btn-noicon" id="x-auto-175"
style="margin-right: 5px;">
  <tbody class="x-btn-small x-btn-icon-small-left">
    <tr>
      <td class="x-btn-tl">
        <i> </i>
      </td>
      <td class="x-btn-tc"></td>
      <td class="x-btn-tr">
        <i> </i>
      </td>
    </tr>
    <tr>
      <td class="x-btn-ml">
        <i> </i>
      </td>
      <td class="x-btn-mc">
        <em unselectable="on" class="">
          <button style="position: relative; width: 69px;"
          type="button" class="x-btn-text" tabindex="0">
          Save</button>
        </em>
      </td>
      <td class="x-btn-mr">
        <i> </i>
      </td>
    </tr>
    <tr>
      <td class="x-btn-bl">
        <i> </i>
      </td>
      <td class="x-btn-bc"></td>
      <td class="x-btn-br">
        <i> </i>
      </td>
    </tr>
  </tbody>
</table>

The above example was taken from http://my.opera.com/lienngoc/blog/2010/08/10/how-to-improve-performance-in-a-gwt-gxt-application and is realistic enough.

B) GWT CellTree
The GWT CellTree is a very nice widget. I managed to use it with RPC calls within a small amount of time and minimal reading.

1) Rendering
I was expecting to be able to provide easily a custom renderer and that was the case. In the GWT showcase, for each level of the tree, the same kind of node is rendered which means a single renderer at each level of the tree. I was like well, it is just an 'example'. In real life, you usually have objects of different kinds at the same level of a tree and they need to be rendered differently.

I ended up using a single renderer and calling instanceof to select the appropriate styling for each node of the tree. I couldn't find another solution within a short amount of time.

public void render(Context context, MyObject value, SafeHtmlBuilder sb) {
 if (value != null) {
  sb.appendHtmlConstant((value instanceof MyBaseObject) ? baseImgHTML : advancedImgHTML).appendEscaped(" ");
  sb.appendHtmlConstant("");
  sb.appendEscaped(value.toString());
  sb.appendHtmlConstant("");
 }
}

2) Can I haz vertical scrollbars???
After testing a simple application, I noticed that Google Chrome was displaying a vertical scrollbar while I was expanding the CellTree nodes. Firefox didn't render any scrollbars.... I came accross a stackoverflow link, and for a second I thought 'this is it!', I guess it would've been too easy...
http://stackoverflow.com/questions/7854086/horizontal-scrollpanel-not-displaying-with-celltree

After 'googling' a bit and Firebug's help, I found a solution.
1. Add few CSS style attributes to the CellTree and its parent container
The celltree has the following CSS style : width:100%;height:100%;overflow:auto. It is enclosed within an HTMLPanel with the same CSS style. There's no need to combine a ScrollPanel with let's say a VerticalPanel. Same problem and additional widgets used.


2. Ensure that the first element has a visible overflow and a dummy size in pixels
I didn't test it with a small tree, but I figure that as long as you supply a dummy size that will always be lower than the height of a non empty tree, it'll be fine. I used 100px.

Style firstCellTreeChildStyle = cellTree.getElement().getFirstChildElement().getStyle();
firstCellTreeChildStyle.setOverflow(Overflow.VISIBLE);
firstCellTreeChildStyle.setHeight(100, Unit.PX);   

C) Overall opinion about pure GWT vs GWT frameworks

It looks like building a professional looking pure GWT application requires a serious commitment in terms of styling widgets. For medium-small applications, I think that using a framework will give good looking results with less time without many performance issues.

Playing with Apache Cocoon 3.0

This morning, I needed to generate quickly a simple website without learning some template engine or recalling it.

I was thinking about couple of years ago when I was working a lot with Apache Cocoon and XML processing in general.

I downloaded the latest build of Cocoon 3.0 alpha 2 and gave it a shot. Below are my settings for the site generation :

• Input folder containg XML files (root element and html content)
• Toc file for site navigation (xml + html markup)
• Resources to include 
• An XSL stylesheet for the website generation (dynamic include of the TOC)

The site was ready after few minutes and minor adjustments. It looks like there's a bug in the XIncludeTransformer so I couldn't use it with the alpha 2 version.

Below is some quick and dirty code(all in one class, without other files used) written after downloading it. There are some errors(generics and invalid content) in the code below because of the code formatter that I'm using on this blog.

import java.io.File;
import java.io.FileFilter;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.net.MalformedURLException;
import java.nio.channels.FileChannel;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.EnumMap;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
import java.util.logging.Level;
import java.util.logging.Logger;

import javax.xml.transform.OutputKeys;

import org.apache.cocoon.pipeline.NonCachingPipeline;
import org.apache.cocoon.pipeline.Pipeline;
import org.apache.cocoon.sax.SAXPipelineComponent;
import org.apache.cocoon.sax.component.XMLGenerator;
import org.apache.cocoon.sax.component.XMLSerializer;
import org.apache.cocoon.sax.component.XSLTTransformer;

// Simple website generation demo with Apache Cocoon 3.0 alpha2
public final class StaticSiteGenerator {

    // Generator configuration parameters
    private static enum SiteConfiguration {
        INPUT_FOLDER_LOCATION,
        OUTPUT_FOLDER_LOCATION,
        WEBSITE_XSL_STYLESHEET_LOCATION,
        RESOURCES_FOLDER_LOCATION,
        TOC_FILE_LOCATION,
        INPUT_FILES_EXTENSION
    }
    
    // Error thrown when the configuration appears to be invalid
    private static class InvalidConfiguration extends RuntimeException {
        private static final long serialVersionUID = -8661543651471092797L;

        public InvalidConfiguration(String message) {
            super(message);
        }
    }

    // Parameter passed to the XSL stylesheet to include a navigation bar
    private static final String TOC_FILE_XSLT_PARAMETER_NAME = "tocURL";

    // Current link in the TOC
    private static final String TOC_FILE_CURRENT_LINK = "currentLink";    

    // Copy a file to a folder
    static void copyToFolder(File in, File outputFolder) throws IOException {
        File out = new File(outputFolder, in.getName());
        
        FileChannel inChannel = new FileInputStream(in).getChannel();
        FileChannel outChannel = new FileOutputStream(out).getChannel();
        
        try {
            inChannel.transferTo(0, inChannel.size(), outChannel);            
        } 
        catch (IOException e) {
            throw e;
        }
        finally {
            if (inChannel != null)  inChannel.close();
            if (outChannel != null) outChannel.close();
        }
    }

    // Generator configuration parameters
    private EnumMap configuration;
    
    
    // Creates a new StaticSiteGenerator with a given configuration
    StaticSiteGenerator(EnumMap configuration) {
        validateConfiguration(configuration);
        this.configuration = configuration;
    }
    
    // Simple configuration validation
    private void validateConfiguration(EnumMap configuration) throws InvalidConfiguration {
        if (configuration == null) {
            throw new IllegalArgumentException("You must provide a valid configuration");
        }
        
        Collection allConfigurations = new ArrayList(Arrays.asList(SiteConfiguration.values()));
        Collection providedConfigurations = new ArrayList(configuration.keySet());
        allConfigurations.removeAll(providedConfigurations);
        
        if (!allConfigurations.isEmpty()) {
            StringBuilder sb = new StringBuilder(allConfigurations.size() * 3);
            sb.append("Some configuration parameters were not found:");
            SiteConfiguration[] missingConfigurations = allConfigurations.toArray(new SiteConfiguration[allConfigurations.size()]);
            sb.append(missingConfigurations[0]);
            for (int i = 1; i < missingConfigurations.length; i++) {
                sb.append(", ").append(missingConfigurations[i]);
            }
            throw new InvalidConfiguration(sb.toString());
        }
    }

    private void clean(File outputFolder) {
        for (File outputFile : outputFolder.listFiles()) {
            outputFile.delete();
        }
        
    }
    // Generates the static web site
    private void execute() throws Exception {        
        final File inputFilesFolder = new File(configuration.get(SiteConfiguration.INPUT_FOLDER_LOCATION).toString());
        final File outputFolder = new File(configuration.get(SiteConfiguration.OUTPUT_FOLDER_LOCATION).toString());
        final File resourceFolder = new File(configuration.get(SiteConfiguration.RESOURCES_FOLDER_LOCATION).toString());
        final File xslStylesheet = new File(configuration.get(SiteConfiguration.WEBSITE_XSL_STYLESHEET_LOCATION).toString());
        final String inputFilesExtension = configuration.get(SiteConfiguration.INPUT_FILES_EXTENSION).toString();
        final File tocFile = new File(configuration.get(SiteConfiguration.TOC_FILE_LOCATION).toString()); 
        
        clean(outputFolder);
        
        final String tocURL = tocFile.getAbsolutePath();
        
        // Get the list of XML files for each page of the website
        File[] inputFiles = inputFilesFolder.listFiles(new FileFilter() {
            public boolean accept(File pathname) {
                return pathname.getName().endsWith(inputFilesExtension) && pathname.isFile();
            }
        });
        
        File[] resources = resourceFolder.listFiles();

        // Copy resources to the output folder
        for (File resource : resources) {
            copyToFolder(resource, outputFolder);
        }

        // Loop through the input files
        for (File inputFile : inputFiles) {
            OutputStream outputFileStream = null;

            String msg = "Transforming '%s' to '%s'";

            try {
                String outputFilename = inputFile.getName().replaceAll(inputFilesExtension, ".html");

                File outputFile = new File(outputFolder, outputFilename);

                Logger.getAnonymousLogger().log(Level.INFO, String.format(msg, inputFile.getAbsolutePath(), outputFile.getAbsolutePath()));
                
                outputFileStream = new FileOutputStream(outputFile);
                
                Pipeline pipeline = new NonCachingPipeline();
                
                pipeline.addComponent(new XMLGenerator(inputFile.toURI().toURL()));
                pipeline.addComponent(newXSLTransformer(xslStylesheet, tocURL, outputFilename));
                pipeline.addComponent(newXMLSerializer());

                pipeline.setup(outputFileStream);
                pipeline.execute();
            }
            finally {
                if (outputFileStream != null)  outputFileStream.close();
            }           
        }        

    }
    
    private XMLSerializer newXMLSerializer() {
        Properties format = new Properties();
        
        format.put(OutputKeys.METHOD, "xml");
        format.put(OutputKeys.INDENT, "yes");
        
        return new XMLSerializer(format);
        
    }
    
    private XSLTTransformer newXSLTransformer(File stylesheet, String tocFileURL, String currentLink) throws MalformedURLException {
        Map parameters = new HashMap(1);
        parameters.put(StaticSiteGenerator.TOC_FILE_XSLT_PARAMETER_NAME, tocFileURL);
        parameters.put(StaticSiteGenerator.TOC_FILE_CURRENT_LINK, currentLink);

        XSLTTransformer transformer = new XSLTTransformer(stylesheet.toURI().toURL());
        transformer.setParameters(parameters);

        return transformer;
    }

   public static void main(String[] args) throws Exception {
        String wwwRoot = "/Users/yves/Desktop/www"; 
        
        EnumMap configuration;
        configuration = new EnumMap(StaticSiteGenerator.SiteConfiguration.class);

        configuration.put(SiteConfiguration.INPUT_FOLDER_LOCATION, wwwRoot + "/in");
        configuration.put(SiteConfiguration.INPUT_FILES_EXTENSION, ".xml");
        configuration.put(SiteConfiguration.TOC_FILE_LOCATION, wwwRoot + "/toc/toc.xml");
        configuration.put(SiteConfiguration.OUTPUT_FOLDER_LOCATION, wwwRoot + "/out");
        configuration.put(SiteConfiguration.WEBSITE_XSL_STYLESHEET_LOCATION, wwwRoot + "/resources/xsl/site.xsl");
        configuration.put(SiteConfiguration.RESOURCES_FOLDER_LOCATION, wwwRoot + "/resources/files");
        
        StaticSiteGenerator generator = new StaticSiteGenerator(configuration);
        generator.execute();
    }

}

Playing with OSGI and JPA

I've been writing an IRC bot for fun, but it's far from being done. My goal is to experiment with JPA in an OSGI environment.

At work, I am using Hibernate, XDoclet to generate the XML and JPA isn't coming soon :-). My last JPA application was a pastebin application, with Wicket, hibernate search, Lucene, etc.

I'm using a friend's library Jerklib with Dynamic JPA. I'm still having some minor issues when deploying but I'll probably find a solution soon.

About the application

Environment

I have a multi-project with Maven and I'm using the maven bundle plugin and I'm developing in Eclipse 3.5 on Debian testing.

I'm using openjpa , dynamic jpa and couple of other dependencies

Design overview

a) Commands implemented as plugins : The bot has a set of factoids(learn, forget, etc...). Each command gets created using a factory.

// message listener //String operation = getOperation();

CommandFactory factory = ServiceFromOSGI.getCommandFactory(operation);

// if the factory is not null, redundancy for the operation parameter

// as a factory can have multiple commands

Command command = factory.createCommand(operation);

ircChannel.say(command.render(ircMessageContext));

b) The command service listens for removal or installation of commands and updates itself.

After writing couple of "users' commands", I would need to implement some administration commands(load/unload plugins, irc specific tasks, etc.).

Problems

The dynamic discovery is failing for now, the persistence provider class is not resolved, it might just be a bundling problem for openjpa. I wrapped it myself.

Conclusion

It's strange that many open source projects still don't provide an OSGI manifest. The maven-bundle plugin is very trivial to use for maven enabled project and there's still bnd.

While OSGI is an interesting technology, I personally don't know anybody using it in the enterprise unless they are an Eclipse shop. I think that it's mostly due to

• the lack of "OSGI enabled jars"
• the fear that OSGI might introduce unnecessary complexity
• the lack of step by step complete examples(if possible with screencasts)

There's plenty of documentation about OSGI and lots of successful applications(Web, Desktop) using OSGI. Hopefully my bot will be one those applications :-)

Handling database changes without complete migration

The ORM market
ORM tools are great. Products like Hibernate, JDO, JPA, IBATIS, Torque, and others made life easier for developing database enabled applications.
Using JDBC when your application is database intensive with lots of table can be lots of work especially if lots of your existing code base doesn't provide some DAO classes.
Usually in ORM tools, you map a set of fields to some columns, using XML or annotations, and you're done.

Most J2EE and core Java developers have faced database changes and migration issues at least once. The problem is pretty crucial when your database model is shared by other applications which can't be upgraded(for many reasons).

The concern

• How to handle database changes which keep happening?
• Should/Could you stop providing backward compatibility?
• Is upgrading the database model your only solution?
  

The application history

• You have an existing application with a model which has been designed carefully and everything is going well.
• You have a server side application with a database model and client applications with the same database model as the "main server".
  
• A month or a year later, you need to make lots of changes in couple of tables, replace some primary keys, introduce some non null foreign keys, etc.
• You were using raw JDBC mostly and plain SQL. Now, you would like to use that brand new bleeding edge technology(Hibernate, JPA, Ibatis, name it).
  
• Here and there, you might have been using a very old ORM tool which was convenient at the time and is still getting the job done.
  

Constraints

• You need to be able to support simultaneously clients(applications) running older and newer versions of the database schema.
• You cannot force the customer to upgrade for many reasons(hardware dependencies, partner application compatibility, the customer doesn't want to, etc.)
• You need to keep adding new features which might involve altering again the existing schema
• Your table contents are now messed up, invalid or irrelevant values here and there because the database column has a "NOT NULL" property.
  

Possible solutions

• One might be tempted to maintain different versions of the same database, but let's say I have 100 versions since 1994.
• Ok, let's use JCR to provide another abstraction level, maybe checking a node property before deciding which class to map, overkill in most cases?
  
• "Dear customer, please, upgrade and buy the new pack to be able to use that version which also provide bug fixes and new features"
  
• Hum... last resort "Dear customer, you should upgrade because that X, Y, W feature fixes lots of serious security holes which will affect your network"
• Ok, from now on, every table will be like a key-pair, probably not very wise most of the time, especially if it will involve rewriting most parts of a huge application.
  

The problem is here, a fix must be delivered!
What I would probably do is :

• Stick with raw SQL and migrate ORM mappings to JDBC, as needed(if the ORM tools cannot ommit fields), to ignore some properties depending on the client database version. I can insert some dummy values when I have no choice when dealing with "old software clients".
  
• Use native SQL queries and JDBC only
• Use named SQL queries with binding and anything tool that supports it Hibernate, Ibatis, a resultset handler from JdbcTemplate or DbUtils, etc. Some dummy data will need to be inserted when not available(new non null columns).
  
• If the problem gets out of hand, way too many changes, I'll probably want to use a non relational database and handle relationships myself(An object or XML database might do, but might not scale)
  
• Another solution, would be JCR. I messed with JackRabbit once, and the pain was brought. Performance, concurrency and the API probably improved since then.
  

I would definintely try to avoid running multiple database versions at the same time. You can easily go from 1, 2 versions and then reach 100.

What would you developers do in such a situation?