Tuesday, December 23, 2008

Merry Christmas

XPontus and VFSJFileChooser were released this evening. I wanted to release before Christmas! Still the same stress and anxiety before and after publishing a new version. Now, I need to start advertising in forums, etc.

The last few days, well more nights than days, have been mostly about testing XPontus installers, fixing bugs, reviewing as much code as I could.

Merry Christmas everybody and happy new year!

Tuesday, November 25, 2008

Handling database changes without complete migration

The ORM market
ORM tools are great. Products like Hibernate, JDO, JPA, IBATIS, Torque, and others made life easier for developing database enabled applications.
Using JDBC when your application is database intensive with lots of table can be lots of work especially if lots of your existing code base doesn't provide some DAO classes.
Usually in ORM tools, you map a set of fields to some columns, using XML or annotations, and you're done.

Most J2EE and core Java developers have faced database changes and migration issues at least once. The problem is pretty crucial when your database model is shared by other applications which can't be upgraded(for many reasons).

The concern
  • How to handle database changes which keep happening?
  • Should/Could you stop providing backward compatibility?
  • Is upgrading the database model your only solution?
The application history
  • You have an existing application with a model which has been designed carefully and everything is going well.
  • You have a server side application with a database model and client applications with the same database model as the "main server".
  • A month or a year later, you need to make lots of changes in couple of tables, replace some primary keys, introduce some non null foreign keys, etc.
  • You were using raw JDBC mostly and plain SQL. Now, you would like to use that brand new bleeding edge technology(Hibernate, JPA, Ibatis, name it).
  • Here and there, you might have been using a very old ORM tool which was convenient at the time and is still getting the job done.
Constraints
  • You need to be able to support simultaneously clients(applications) running older and newer versions of the database schema.
  • You cannot force the customer to upgrade for many reasons(hardware dependencies, partner application compatibility, the customer doesn't want to, etc.)
  • You need to keep adding new features which might involve altering again the existing schema
  • Your table contents are now messed up, invalid or irrelevant values here and there because the database column has a "NOT NULL" property.
Possible solutions
  • One might be tempted to maintain different versions of the same database, but let's say I have 100 versions since 1994.
  • Ok, let's use JCR to provide another abstraction level, maybe checking a node property before deciding which class to map, overkill in most cases?
  • "Dear customer, please, upgrade and buy the new pack to be able to use that version which also provide bug fixes and new features"
  • Hum... last resort "Dear customer, you should upgrade because that X, Y, W feature fixes lots of serious security holes which will affect your network"
  • Ok, from now on, every table will be like a key-pair, probably not very wise most of the time, especially if it will involve rewriting most parts of a huge application.
The problem is here, a fix must be delivered!
What I would probably do is :
  • Stick with raw SQL and migrate ORM mappings to JDBC, as needed(if the ORM tools cannot ommit fields), to ignore some properties depending on the client database version. I can insert some dummy values when I have no choice when dealing with "old software clients".
  • Use native SQL queries and JDBC only
  • Use named SQL queries with binding and anything tool that supports it Hibernate, Ibatis, a resultset handler from JdbcTemplate or DbUtils, etc. Some dummy data will need to be inserted when not available(new non null columns).
  • If the problem gets out of hand, way too many changes, I'll probably want to use a non relational database and handle relationships myself(An object or XML database might do, but might not scale)
  • Another solution, would be JCR. I messed with JackRabbit once, and the pain was brought. Performance, concurrency and the API probably improved since then.
I would definintely try to avoid running multiple database versions at the same time. You can easily go from 1, 2 versions and then reach 100.

What would you developers do in such a situation?

Sunday, November 16, 2008

JDK7 changes

I had a surprise this morning while playing with System.getProperties. I am using JDK7 and displaying the system properties in a Swing JTable.
When I don't specify the number of columns I get a null pointer exception. It happens when populating the array contents, not at initialization.
This will throw a NPE:

data = new Object[NB_ROWS][];

This will not

data = new Object[NB_ROWS][NB_COLUMNS];

Code excerpt

public JavaEnvironmentModel()
{
Properties envProperties = System.getProperties();
NB_ROWS = envProperties.size();
data = new Object[NB_ROWS][NB_COLUMNS];

// for google blogger parser, no generics(Entry)
Iterator it = envProperties.entrySet().iterator();

for (int i = 0; it.hasNext(); i++)
{
Entry entry = (Entry)it.next();
data[i][PROPERTY_COLUMN] = entry.getKey();
data[i][VALUE_COLUMN] = entry.getValue();
}
}

Monday, September 01, 2008

Drinking a huge cup of Java

I am working again on XPontus XML Editor. I believe that I'm ready to go for another round.

I am happy to see that there are so many people using the software. Since the intial XPontus release, there are about 10 000 official downloads from Sourceforge and probably 12 000 from other sites grabbing the files directly by HTTP. I didn't expect such a thing at all and it gives me energy to work again on the application.

Here are the main things that I learned since the XPontus inception:
  • Provide a project roadmap even if it's ambiguous or you won't do anything listed :-)
  • Don't use build tools such as Maven if you expect lots of people to contribute to a project, use a simpler but powerful tool such as Ant.
  • If the component is has a public API, finish writing all the documentation before releasing: 20 bugs+good documentation is better than 5 bugs + no documentation, there will be bugs anyway.
  • Advertise a project enough but not too much, people will expect the application to be at the best commercial level even if there are only 2 or 3 people working on the project.
  • Don't assume you know what you're writing because you do code similar things often, read books or articles about the subject whenever you can.
  • Don't try to make a big application without taking time to do it well, or as well as possible
  • Try to think more like a user or a client of an API, rather than a programmer. The application is for users at the moment you distribute it.
  • Make it look good and then make it work : you love what you see and then you love what's inside.
  • Don't necessarily provide many features, few working features are better than a zillion of unstable features.
My goals are simple for the next release:
  • The application will probably be lighter and faster (java reflection abuses, bad programming, race conditions, etc.)
  • Partial rewrite and more use of design patterns(without trying to recognize them everywhere though! ).
  • Fix the current bugs, usability/stability issues. No new features except maybe XPath2, XQuery. VFSJFileChooser will be introduced in the release as an optional plugin.
  • New, simple but powerful API : If you explain the components relationships and the logic of the application, it should almost sounds like common sense, not magic or absurd.
  • One should be able to reassemble, disassemble or extend XPontus without much effort. It looks difficult though, how can someone integrate very easily an application that uses a plugin system?? Last time I integrated parts of JEdit into an application I won't say it was too tough but it wasn't that simple even if I believe that JEdit modules are well written most of the time.

Friday, August 15, 2008

VFSJFileChooser 0.0.3 released

VFSJFileChooser was released today after some additional tests. Nothing much to say, I guess I'll wait for comments.

Tuesday, August 12, 2008

About VFSJFileChooser 0.0.3

I guess I'm ready for the third release of VFSJFileChooser thanks to Stephan Schuster. He helped a lot for that release(bug reports, patches, suggestions). I'll probably release tomorrow after few tests.

Here is the changelog:
  • One noticeable feature is speed :-). VFSJFileChooser was too slow I.M.H.O.
  • Navigation icons are "always" visible now. They didn't show when the java look and feel set, didn't derive from MetalLookAndFeel. I borrowed some icons from Tango and famfamfam which are now the default icons used.
  • Bug fixes for directory selection among other things
  • The VFSUtils class supports the methods setFileSystemManager and setFileSystemOptions. You can set those values at anytime. when the VFSJFileChooser class is instanciated, it checks if VFSUtils has a filesystemmanager set, if not it creates one. Files are always resolved with the FileSystemOptions object in VFSUtils.
  • Upgrade to webdavclient4j (http://webdavclient4j.sf.net) as Jakarta Slide is dead.
  • Cleaner but incompatible API (Enums instead of int fields) : I started to refractor code here and there. I am making full use of JDK5 as VFSJFileChooser is not compatible with jdk14 and older releases. Enums are introduced for few classes which breaks the API. The method setFileselectionMode of VFSJFileChooser now accepts an Enum as parameter. The methods "showOpenDialog" and "showSaveDialog" return an Enum too.
  • Sorting support : The details table has now sorting support again in the jdk5 branch. The jdk5 branch is the most up to date(patches, general improvements, etc.). The jdk5 will become the default branch. I now develop on jdk5 to ensure code compatibility.
The Windows look is still not supported. I'll look at it and see what can be done. For now, I only have my old laptop running Linux so I can't really work on it. Any help in that regard would be appreciated.

Thursday, July 31, 2008

About XPontus and VFSJFileChooser

Lately, I keep receiving questions about VFSJFileChooser and XPontus. Even if I didn't answer, I did read all the emails. I thank you all for your interest and your comments.

I have started to relocate to Toronto and I am still looking for a new job which is my primary focus right now. I am still looking at XPontus and VFSJFileChooser APIs time to time(the good, the bad and the ugly).

I intend to work again on those projects as soon as I'll become professionaly stable again.

Thank you for understanding that.

Thursday, June 05, 2008

Upcoming VFSJFileChooser release

I've been trying to make VFSJFileChooser compatible with jdk 1.5 lately. Know bugs have been fixed. There's no sorting for now in the details view, but the rest is working well.

SwingUtilities.getWindowAncestor seems buggy in jdk5 SwingUtilities.getWindowAncestor(Component ComponentThatCouldBeAFrame) seems have issues. Trying to open the file dialog was throwing null pointer exceptions in jdk5. I needed to make another check to see if the component is a frame before creating and displaying the file chooser dialog with jdk5.


Window window = SwingUtilities.getWindowAncestor(parent);

if (window == null)
{
if (parent instanceof Window)
{
window = (Window) parent;
}
else
{
window = new Frame();
}
dialog = new JDialog((Frame) window, title, true);
}

else if (window instanceof Frame)
{
dialog = new JDialog((Frame) window, title, true);
}
else
{
dialog = new JDialog((Dialog) window, title, true);
}

I would like to avoid maintaining 2 branches if possible. There are lots of things that are easily done with jdk6 whereas in jdk5 additional classes are necessary.

Monday, May 19, 2008

OSGI bundles packaging and deployment

I created my first non trivial swing application based on OSGI. I deployed it on Apache Felix and Knopflerfish for testing purposes.

The swing application has about 130 classes, not huge classes. I tried to make a good design using couple of interfaces with some abstractions(docking framework abstraction, file system abstraction, gui components abstraction, plugin manager abstraction, etc.). After few days, I was satisfied with the design and started coding few concrete classes. I made sure that most of the code could be reused quickly in a non-OSGI environment.

In 4 days, I finished programming the core of the application. The only thing left was the gui main window. I wanted to use a docking framework(LGPL, BSD or Apache License).

Using docking frameworks such as MyDoggy or Flexdock in an OSGI environment looks complicated. Most of those libs make some static calls while trying to resolve images or configuration files. Sometimes some properties or other resources are stored in the META-INF folder of the jar files.

MyDoggy
I was surprised to see that, as the library wasn't finding its default configuration file(from the classloader), it was looking for it in my home folder. There's no way to set the default properties using a static method without messing with the source code of MyDoggy (or maybe I didn't see a way).
With Apache Felix I couldn't import sun.awt package used in MyDoggy but I was able to do it with knopflerfish.

Flexdock
With Flexdock, I had an infinite loop, an exception which was thrown all the time, preventing the program to run. It was complaining about not being able to create an instance of the persistencemanager or something like that. Must have been some class loading issues using reflection calls.

I decided to use few JSplitPane at last and I could see the main window of the application, after few hours spent looking into those docking frameworks source code.

To test quickly I was using Apache Felix(embedded) in the application. I had my activators written but no OSGI manifests yet. Later I saw the bundles issues. I didn't want to embed some dependencies directly in my bundles. So I started browsing the web for "osgi bundles" and chatting on IRC. Some guys told me to look at Eclipse Orbit.

Eclipse Orbit is a community effort to create OSGI bundles for common third party librairies. It's a lot of work. I packaged few libraries and it took a bit to import and export the right packages.

The last thing left before going on with that swing application, is to add some listeners so that when a bundle is removed/updated, the GUI and some non visual parts are notified to update themselves.

Wednesday, May 14, 2008

What makes an API great?

After few years of Java programming, I improved slower than I expected but still... When designing complex software which needs to be reused, I still find it difficult to make quick and "good" choices without adding too much complexity.

In the last branch of XPontus XML Editor(major refractoring), I had a dilemma:
  • Release soon : couple of patches and dirty classes which do the job
  • Wait 4 or 5 months to be ready : clean up the code, perform thorough testing, remove unused classes, add or redesign interfaces, etc.
I choose to release early and fix what I could see or had time to fix. One month after the release, I was like "why did I do it that way? what is that useless piece of code doing over here, etc."

IMHO, an API is great when it's
  • simple
  • useful
  • flexible(without adding too much complexity when it's unnecessary).

I tend to prefer APIs which expose few interfaces, abstract classes and some concrete classes. Complex applications APIs without/with few interfaces, are somehow difficult to maintain, refractor, make evolve.

Monday, May 05, 2008

Playing with Apache Felix

I'm trying to learn more about OSGI and become proefficient with it quickly. Neil Bartlett is writing a book about OSGI. It's available for free but it's not completed yet.

I looked at IPOJO from Apache Felix and managed to create a simple application with it. I think I'll start with plain OSGI programming first. Once I get it right, it will be easier to know what IPOJO or other tools can do for me and when it's better not to use them.

For example to build a simple GUI with plugins support using OSGI, the only thing I need to do if I understand, is :
  • Create few services, one can think about them as extension points
  • Create BundleActivators for each plugin
  • Register couple of services and use some ServiceTrackers
  • Layout and display your GUI application when all the bundles are activated
  • Launch background services if their startup needed to be delayed
I downloaded the SIP communicator's code. It's an audio/video Internet phone and instant messenger using Apache Felix. The graphical interface is nice and the code is "clean enough", quite good IMHO. I found a little bit strange/unconventional the way they package bundles. It seems that they build bundles from some packages/classes using an Ant build script, in a single project. I believe in an IDE it would look like a project with multiple source folders and a jar target for each source folder.

In XPontus XML Editor I have about 18 plugins which could become bundles. I started cleaning up some code. One shall never hurry too much to release, patch code here and there, leave too many unused packages, because the pain comes soon enough when you need to do some refractorings... .

In XPontus, I have 1 master project and about 30 sub-projects. Well it can be difficult to manage too, but I don't have to create many ant targets/tasks to build specific jars.

Project structure overview
  • xpontus_core
  • etc.
  • indentation_plugin(dummy maven pom project)
-> xml_indentation_plugin(sub-project, maven jar project)
-> html_indentation_plugin (sub-project, maven jar project)

The annoyance with a plugin architecture is about the deployment, but mostly the packaging. Most of the time the plugin framework or OSGI framework you'll find has a console or a main class from where you can launch the bundles/plugins. Usually in a big application, you want total control and a customized behaviour which means a custom launcher to embed the plugin framework or the OSGI framework.

Let's say I have a bundle called bundle0. I created it in Eclipse, I have few jars as dependencies, etc. How do you auto-package all that with minimal effort(zip file with the bundle0.jar, a lib directory holding the dependencies)?

In XPontus XML Editor the plugins have an "Eclipse like" folder structure
- com.mycompany.plugin
* plugin.xml
* lib(folder containing the jars)

Every time I want to deploy a plugin:
  • I create a plugin folder with a unique id
  • I add the plugin descriptor
  • Create a lib folder with all the jars needed by the plugin
  • I zip the folder and it could be ready to be deployed.

Thursday, May 01, 2008

VFSJFileChooser is out

I uploaded the new release of VFSJFileChooser this evening. It will be available on all Sourceforge mirrors probably tomorrow.

Tuesday, April 29, 2008

Preparing the new VFSJFileChooser release

VFSJFileChooser will be out soon.

Here is the changelog :
  • Sort file by names : Some patches were submitted to sort file by names.
  • The "home" button will not bring you back anymore to your local home folder if you're browsing a remote directory.
  • A details view is being added as a complement to the existing list view
There are still some issues when you choose the native look and feel with : UImanager.setlookandfeel

Saturday, April 26, 2008

A closer look at OSGI

I was reading about OSGI and I found out that it's not as complicated as it looks like. As I am not an Eclipse fan, I tried Knopflerfish and Apache Felix. I might use OSGI in my next projects if they're big enough.

After half an hour, I was able to get a simple bundle running with Apache Felix. I wrote a simple program embedding Apache Felix and created another project which provided a bundle.

My main interest in OSGI is dependencies management handling :
  • Service A is started
  • Service B depends on Service A
    • if service A is not available -> do not start service B
    • if service A is available -> start the service B and register it

Sunday, April 20, 2008

Java Enterprise Edition

Most of small companies don't use Java Enterprise Edition, even if few thousands guys need to access an application at the same time, even if the application needs to support transactions, to be scalable, etc. They already have deployed "medium to big" successful applications using Tomcat and J2SE, so why bother?

It's difficult to find very young people(less than 30) which have a J2EE knowledge. You don't learn J2EE in a technical school, you don't learn it at University and you probably won't learn it by working in a small company. IMHO, the general opinion about J2EE is that "It's interesting, looks complicated too... and I probably don't need it, (solo talking it's probably worth not my time and my salary learning it.)". When in a small company a guy has a good J2EE working knowledge, there is probably a big chance, that he's the only one who will create and deploy enterprise applications and hopefully transfer all his knowledge to others someday.

I looked only once at Java Enterprise Edition few years ago. Going by the book, and doing few examples, I was able to get started using notepad and standard Java tools. The experience was painful with Corba, RMI, etc. Few days ago, I had a look at JavaEE 5. It looks nice and quite simple. However it seems that most businesses are still using EJB 2.1 and not 3.0.

Tuesday, April 08, 2008

XPontus - Schema completion

Schema completion support has been improved... It was kind of not working in XPontus.
A schema location declaration contains most of the time one or multiple uri references(namespaces and schema locations) with some whitespaces, carriage returns, ...
Now XPontus is able to resolve each schema url locations when they end with .xsd.

Latest screenshot

Sunday, April 06, 2008

Working on XPontus 1.0.0.2

XPontus 1.0.0.2 is coming up. I've fixed few issues this week-end.

General interface problems
Sometimes the file chooser dialog can take a while to load(I only noticed that problem under Windows). That issue has been adressed lots of time to Sun. I don't think it's gonna be fixed definitely any time soon. It seems that the file chooser freezes when you have some zip files on your desktop and it can take up to 10 seconds to show on a Quad Core... Can you tell a user who reports the bug "Damn, you've got lots of zip files on your desktop, clean it up!". That would be funny...

I am not sure that I'll replace the file chooser by VFSJFileChooser. If I don't do it well, it will bloat the program(5 to 6 additional libraries). Letting the user chooser between the default file chooser(better OS integration) and VFSJFileChooser(remote files able) would involve rewriting partially many classes.


Windows vista GUI issues
The tab names are now visible under Windows Vista. The gray rectangle is still visible in the menubar. I could create a special layout for the menubar or make it a java.awt.GridLayout(ugly but no gray rectangle) to solve the issue. The menubar problem is a swing related issue, not XPontus' fault.


The code completion is more usable

Here is the new use case :

Let's say you trigger an element completion by typing "<" in a xsl stylesheet document
  • Suppose you were getting typing xsl:apply-te.
  • That would select xsl:apply-templates in the list
  • Then you change your mind and select xsl:apply-import and press enter
  • The text inserted in the document is xsl:apply-te and it will now be converted to xsl:apply-import instead of xsl:apply-teport

XPath improvements

  • The xpath plugin couldn't display a boolean value or a number value. For example count(//node) would throw an exception and now it returns the number of node found as text.
  • XPath 2.0 support has been added

Architecture

I didn't fix the architecture problems. I will address that matter when the version 1.0.0 of XPontus will be considered as stable. Rewrite things here and there, without breaking too much code.

What's next

I will think about the plugins manager next week and give myself sometimes to do it well. I don't it to be complicated, I just want it to work as well as possible for the next release.

Friday, April 04, 2008

Random talk[SPECIAL]

Nowadays, I tend to think a lot about that Albert Einstein quote
You have to learn the rules of the game. And then you have to play better than anyone else.

It seems to very true in most circumstances...(work, girls, normal life stuff...) Programmer with real skills or programmer which seems to have great skills...?

It's all about that they think you are and what they think you know... That's what a guy told me during my first year at university. The guy didn't know anything but he truely believed that he deserved a great income....

XPontus 1.0.0.2 will be available soon

I'll release a 1.0.0.2 version of XPontus XML Editor sometime in May. I got a copy of Windows Vista installed and I've been doing some bug fixes for Windows Vista.

I'll try to dedicate 3 days to complete and test the plugins manager(Install/Uninstall). I believe it's possible. If I come up with a good strategy, the rest will be easy.

The roadmap for 1.0.02 :
  • Install/Uninstall features in the plugins manager
  • Tab names visibility in Windows Vista
  • Printing isssues on windows vista
  • Fix indentation which is sometimes crashing the editor when the XML is not well-formed
  • VFSJFileChooser integration
  • A minor rewrite of the XSL transformation to specify a transformation type(XQuery for example)
  • file extensions registration to mime types
  • XPath 2.0 plugin
  • file encoding issues
  • Update the user guide and the developer guide

Friday, March 28, 2008

Fixing bugs and design errors in XPontus

I made lots of design errors in the last release of XPontus XML Editor. I didn't have time to rewrite many parts of the application, it would have delayed the 1.0.0.1 release. I'll try to fix all reported bugs for the 1.0.0.2 version. In the next release, I'll switch from Maven to Apache Ivy for dependencies management. Ivy is simple and anyone will be able to open XPontus if the ivyIDE plugin is installed in Eclipse.

I've been thinking about the plugin manager lately. I might finish it sooner than expected.

Lots of people have been asking me if I wrote the plugin framework that XPontus is using. I didn't. Yes, I could have wrote one, but there are more and more plugin frameworks available now. Lots of projects have developed their own plugin framework, made it evolve over the years, and they have been happy with it. As OSGI is more popular among common Java developers, guys are looking at Eclipse equinox implementation or Apache Felix.

Wednesday, March 26, 2008

Multi-platform deployment

Java is multi-platform but Java applications don't behave the same on different OS. I was disappointed to see that XPontus is ugly on Vista. The tab names are in white color, while Windows Vista use translucent windows so... the tab names are almost invisible...

Is it reasonable to build installers for platforms I can't even test on??When a user is telling me, on windows vista I see that... On Vista, multiple pages printing doesn't work... what can I do?? Most of the time I run Linux, I do have Windows installed to perform XPontus tests but I don't really use Windows that often...

Sometimes I have an idea about what a problem is related to, on Vista or Mac but still... I don't have a Mac, I don't have Vista(which seems to be used a lot now)... I can't test on it

So basically I'm running Debian Linux all the time, building installers for OS which can run Java without testing it on those platforms.

I should probably add a notice on the XPontus homepage NEVER BEEN TESTED ON WINDOWS VISTA AND MAC

Tuesday, March 25, 2008

What build tool to use for java development?

People who checkout the XPontus XML Editor's code in the subversion repository expect to open it as a simple Java project in their Java IDE. Well, it's not a simple Java project, it's a Java maven based project.

I use Netbeans and I don't have any problems with Maven. Eclipse and IDEA plugins for Maven are "ALMOST USELESS" compared to what Netbeans provides in terms of usability and flexibility...

I've been more than happy with Maven( manage multiple projects and their dependencies, compile, run , test the applications, generate some websites, etc.)

I do believe that simple Java projects or simple multi-projects ("with few projects as dependencies") don't require an advanced build tool : use your IDE to manage the java classpath or create a simple Ant build file and you're done.

Lately, I've been looking at Apache Ivy as a replacement for Maven to build XPontus. Maven is preventing new members to get started quickly(learn maven, understand its usage in XPontus, etc.). I don't want to have to say all the time something like "Use Netbeans to ease the pain if you never used Maven..."

I don't mind using IVY or Apache Ant when starting a Java project. However if the project is already entirely built using Maven, it sucks to switch to another build system.

Personally, I think about switching toIvy as a regression at the moment:
  • Most IDEs don't have a "fantastic IVY support". It does work but that's it.
  • If most Java developers don't use Maven, do they use Ivy? I don't think so... The learning curve is shorter than Maven but still...
  • Ivy won't do the Maven black magic :-( ... (build some website pages, etc.)
  • Multiproject support?? Will I have to go back to the standard way of creating multi-projects in popular Java IDES?? I'll need to read about that.
The creator of IVY made some intersting comments on IVY vs Maven :
http://xhab.blogspot.com/2006/09/antivy-vs-maven-my-biased-opinion.html

New members of XPontus team will probably be able to use Apache Ivy easily. I guess I'll need to add in capital letters "YOU NEED THE IVY IDE PLUGIN TO OPEN THE PROJECT". I don't really expect lots of people to read it or to read instructions on how to open a XPontus as a Java project. Most developers know how to open a standard Java Project in an IDE, so they'll probably think "yeah must be some newbie stuff..."

Thursday, March 20, 2008

What happen when new users join a team??

As some users are joining XPontus XML Editor project, I have to write quickly a readable and simple documentation.
Who will bother read 500 pages of text before programming? Not me at least!! I have used Hibernate but never read the whole manual. Same goes with other technologies I have used(Apache Cocoon, Wicket, JSF, etc...)

I assumed that most people in the industry use Maven which is not true. I work for a non lucrative company and we use maven a lot. Most people don't use Maven at all, they know about it, they know what some guys have been doing with it, but that's all. Well, I can understand them, why bother with a tool if you feel you don't need it?

I have been using Maven for about 4 years. It has its pros and cons. I think that a simple Java project don't require Maven. It's just overkill.

With XPontus new members, I got to fill the gap between what I know and what others don't:
  • The learning curve of newcomers shouldn't be high.
  • The unfinished developer guide should be done and should almost make it look easy

Basically, I use Maven at 10% of its capabilities. With Apache Maven I do the following steps :
  • Compile and run the program(manage dependencies too)
  • Generate the website
  • Generate the java web start installer and sign the jar files
I could also commit to subversion, deploy the website using SFTP, all that with maven, but I don't.

I noticed that Eclipse has a poor maven support. I use Eclipe time to time but that's it. I switched to Netbeans 2 years ago. Netbeans has great maven support compared to others IDEs.
So when a guy, is telling me "what do I do in Eclipse to do...", in Netbeans it would be a simple click... I might need to switch to Eclipse to come up with good answers as most people are using Eclipse.


I am very excited that some guys are showing interest in XPontus. So what I though was clear in my mind, I have to explain to others and be patient. They have to be patient with themselves too!

XPontus is not like Eclipse, but I don't think average guys could code it in a week. I made some choices which are sometimes good, but sometimes they have limitations. I will rewrite some parts of XPontus when I'll feel that the team has a good understanding of XPontus architecture.

Wednesday, March 19, 2008

XPontus XML Editor 1.0.0.1 is out

I released XPontus XML Editor yersterday night. I didn't release it on monday as planned. I was very tired. I was like don't test like a blind man, don't write bullshit to announce the release, anything can happen if I'm too worn out.

Now is the time to complete the documentation(user guide, developer guide), keep on testing, etc. Can I write a comprehensive developer guide?

I'll take a short break(1 week or less) to look at XPontus from a distance. What did I miss? Is it as stable as I thought it was when releasing? Are there too many bugs? Is it usable? What can be done to improve the architecture....?

Sunday, March 16, 2008

Too many bugs in XPontus

Will I release, should I release now? I've been delaying it forever...

I found so many bugs in XPontus today.... Lots of people have been helping me test the software. There are lots of usability issues. I know that XPontus is not at the same level than XML Spy for example, but still... There are some obvious problems I didn't see. Instead of testing the application all the time by myself, I should ask lots of people to help me test it... Couple of guys have been trying the Java Web Start snapshots, but they don't always fit with the "normal user profile" because :
- They understand or can evaluate the amount of work behind XPontus features and architecture. They may have already encountered the issues that I face.
- They are used to XML technologies

Here's what was done today:
  • I tested few installers. I had a silly mistake in the user settings , preventing the application to start even when the installation was successful.
  • I've been fixing bugs in the schema generator/converter plugin. That plugin takes XML or a Schema/DTD as input and generate a schema(relax ng, dtd or xml schema). Because it can perform generation or conversion between DTD and other schemas, I used the name "schema generator/converter plugin". It can be unintuitive for lots of people.
  • The find/replace dialog was too buggy. I took it from Groovy's code. I replaced it today by some SOAPGui code. I adapted it for usability a little bit. It's not perfect, but it does work!
  • I added a keyboard shortcut to switch between opened documents. "Alt W" will switch between windows(select next document).
  • The XPATH textbox has history support. It remembers up to 10 expressions you type in the textbox. The history is cleared when you close the application. Now you can execute an XPATH expression by pressing the "ENTER" key instead of having to click on the "Evaluate button" all the time.
  • When you click on the "insert comment" button or "insert cdata section" button, the editor get the focus back, no need to click again on the document to be able to insert some text.
I'll release tomorrow and the version will be called 1.0.0pre1-alpha. I added "alpha" to let the users know it's not a mature version. Things don't always work as expected. There are more features, but with features come bugs.

I'll keep on fixing bugs, updating the website and the developer guide until tomorrow.

Friday, March 14, 2008

Under pressure for the new XPontus release

I am getting ready to release XPontus XML Editor next monday : looking here and there, for simple bugs I can fix or forgot to fix, looking at the plugins manager, etc.

Until next monday, I'll be busy to make sure the software is enough usable and stable so I'll be :
  • Creating and testing installers for windows, linux, mac os, bsd, unix
  • Updating the website
  • Coding utility scripts to automate the release announcement in some forums and mailing lists.
  • Fixing bugs time to time
I'll ship all the plugins that I have written and tested enough. So basically, you'll get a big fat and "features-full" version of XPontus. I cannot make a light installer of XPontus as the plugins manager is not enough stable yet. I won't ask all users to open a hidden folder, to copy some zip files and to handle the plugins dependencies logic by themselves.

I'll also have a java webstart version of XPontus but you'll have to install plugins by hand like in the snapshot version. I hope to come with ideas for the plugins manager soon.

I feel happy to release, I've been saying "next week, next month" for too long. I would have wanted to do more for that release but it would have taken few others months to achieve some significant improvements.

Stay tuned!

Tuesday, March 11, 2008

Them "viagra boys", Them "here's the money man"

I didn't know that I would get that much spam on my blog or my sourceforge email account... To those doing that, if you're reading this post :
  • I don't need viagra, I'm in a great shape, still young, still full of energy, no issues...
  • I won some money...??? Well, I could use that cash only if it's real. Trying to lure me is kind of playing with yourself. It's 2008 baby... I guess I haven't seen it all but I think I saw most of it... But why not try? Short answer : it's a waste of your time and mine.

Sunday, March 09, 2008

Releasing next week

Next week, I'll start intensive testing of XPontus XML Editor. I will try to release at the end of the week if possible.

Some issues persist but I think that the snapshot version is better than the stable one. If I keep on coding and testing, writing documentation, creating new plugins, I guess I'll never be able publish a new version of XPontus. I've been planning the upcoming release for at least 8 months. I guess now is the time.

I expect plenty of bug reports, constructive comments, nasty critics, but it's all good, it's all fair. It'll help the development of XPontus XML Editor. If you want to contribute, leave me a message at yveszoundi at users dot sf dot net. I try to take my time to write good software, but I could use some help.

I'll try to find a way to get a hand on a Mac OSX pc to build a dmg installer.

There will be 5 installers : Windows(standard exe), Linux(rpm, tgz, deb), MacOSX(dmg), Unix(sh) and a crosss-platform installer.

Saturday, March 08, 2008

Code completion might be in next release of XPontus!

Code completion seems to be working well enough even though there are few bugs. Here is the complete status of the XML code completion plugin.

Code completion creation
I need to find a way to check if the schema(dtd/xml schema) location of the document is relative or absolute. For the moment, the code completion database is only created if the schema location is absolute.

Intelligent completion
I am now able to get an "almost reliable" suggestion list for code completion based on element insertion location. The suggestion list is sorted by name.

The completion window
It is very buggy. I get"nasty stuff" inserted when I try to select a suggested element. Sometimes, text is inserted twice or more. The completion doesn't have a scroller to view elements if their name are too long.

Completion database caching
I don't know how to do it correctly. I could serialize the completion database for now and store it in a folder.

Thursday, March 06, 2008

Bug fixes day

I will get the list of bugs which were mentionned in earlier in previous versions of XPontus XML Editor and fix them one by one.

In the new version of XPontus the following features are not well working yet :
  • XML validation
  • Code completion(should I remove it?)
  • Grammar caching is not reused by plugins
  • Maybe the save actions

Wednesday, March 05, 2008

XPontus platform developer guide

I started writing the developer guide of XPontus XML Editor today. I'll update it on a daily basis. The developer guide shows how XPontus XML Editor is written and provides a tutorial on how to build and devploy plugins.

I uploaded the initial version of the developer guide on the website. You can download it at the following address : http://xpontus.sourceforge.net/dev_guide.pdf

Tuesday, March 04, 2008

New release at the end of march

The first alpha of XPontus XML Editor 1.0.0 pre1 (End of March)

I've been working hard on XPontus for the last 6 months. It is very different than the previous versions in terms of usability and architecture. Now the software has more features and is more extensible. XPontus has plugins, XPath, an "immature" code completion, etc. Thank you everybody for the bugs reports, the tests and the suggestions. I'll release a new release candidate of 1.0.0 every month until I think that the software is enough stable. The stable version will be called 1.0.0(probably june).

I need to release, so I have to remove incomplete features :
  • The plugins manager(the core of the application) : I was planning to have a complete plugins manager with auto-updates, dependency checking/autoselection, etc. I cannot achieve that goal in a small amount of time. To install a plugin the user must download it for now. I can add a version checking feature to notify the user that a new version is out, but that's it at least for now.
  • Perspectives(Tree Editing, Text editing). Sometimes you want to edit an XML document in a simple window. But when your document is big, you might want to have some drag and drop(elements) features, cut and paste node quickly. I started implement perspectives in xpontus but it involved rewriting lots of code. The big issue with perspectives is how(or how often) you synchronize the text of the source view and other kind of views(Tree view, diagram view for example). I associated persectives to file content types(text/xml, text/xquery, etc.) and file modes(docbook, ant, etc.). When a document needs to be created, the perspective matching the document content type creates the appropriate view to display the document.
So what's the plan?
  • Finish the plugins manager. Find a strategy to uninstall completely a plugin. I cannot do it when the application is running : I can unregister the plugin, but I am not able to delete the plugin installation directory.
  • Ensure that the core features are working well : save, save as , print, validation, etc.
  • Fix previous versions bugs
  • Update code comments and write a developer guide
  • Write the software's architecture manual
  • Update the user's guide
  • Release

Sunday, March 02, 2008

XPontus plugins manager behaviour

I'm removing the components which are not ready yet in the plugins manager. I find it painfull to write a software' dependencies manager. The plugin framework jpf can handle some things from the software perspective(plugins integrity, dependencies not met, etc.). From the user's perspective, I need to add some features like auto-selecting required dependencies of a plugin or if I can't do that popup a dialog telling "This plugin required that other plugin to run".

I was testing the plugins search yesterday and the results are almost good. I have some other issues not tied to the search engine itself : incomplete plugins metadata indexed, etc. Choosing Apache Lucene for plugins full text search was an interesting move. At first sight, I was telling myself "Apache Lucene will be usefull but might be overkill for a simple gui application, ... but for good search results... I need a good search engine instead of using regular expressions to match the results".

I will start writing a developer guide in few days.

Tuesday, February 26, 2008

The plugin manager

I am starting to make things work in the plugins manager of XPontus XML Editor.

The plugins metadata have the following attributes for now:
  • author
  • contributors
  • date
  • version
  • homepage
  • license
  • description, etc.
This will allow full text search of the plugins using Apache Lucene.

When a plugin is added, the plugin is extracted in the XPontus plugins directory. The plugin descriptor is later indexed in a thread.

Here is a screenshot of the plugins manager.

Sunday, February 24, 2008

Everything is going smooth

I am working on XPontus' user settings. Most of them are working now. It is taking me forever to code it. Dealing with lots of user settings is a huge pain.

I choose not to use my fatal combo(JGoodies Bindings + XStream) all the time. Using both a good binding framework and a good serialization framework makes the hard work easy. But when you change your model classes then your serialization data is useless. You have to delete them all and to store them again. Therefore the user lose all his saved settings. However if you use simple text files(xml, properties file), you can easily discard deprecated data, restore it, etc. The counterpart is that it will take you time to code it all.

Here is my schedule :
- Finish the user settings Monday
- Have a functional plugins browser(Friday)
- Fix as many bugs as possible in the code completion plugins (saturday)
- Test all the software features (Sunday)
- Make an official release date of the 1.0.0-pre1_alpha1 version

There will be 3 releases 1.0.0_pre1_alpha branch I guess. I plan to release a new alpha every month. After 3 months, xpontus 1.0.0-pre1 shall become the stable release.

Wednesday, February 20, 2008

The hard work is done!

I have been trying to release XPontus XML Editor for the last few months. There are now less features missing. Most bugs have been fixed, new bugs arrived too :-(.

I'll be working on the plugins installer and the user settings. I hope I'll be done with that at the end of the week.

The only thing left after that will be working on an intelligent and decent code completion with caching support.

I plan to move to OSGI architecture for plugins when the final 1.0.0 release is ready. This will involve some code rewriting. But it shouldn't be major if I get a good understanding on OSGI. The only thing that is bugging me is Java Web Start deployment with OSGI and the java security manager. I expect to see lots of errors like "File read permission denied..."

Friday, February 08, 2008

Getting ready to release

Some friends are helping me test the upcoming snapshot release of XPontus. I had some nice comments and I am fixing bugs. Some critical issues affecting the current release are being resolved. Some features present in the stable release are still missing in the snapshot release.

To those who reported bugs on the sourceforge tracker "I am working on that..."

Sunday, February 03, 2008

XPontus has a new look

I often heard : "is it Eclipse? It sure look like Eclipse..."

I borrowed some icons here and there without bothering about license stuff. The new icons are from gartoon gnome icon theme mostly.

Working on User's settings

The plugins settings dialog of XPontus is almost done. I wrote simple interfaces to allow plugins to register some configuration options.


Wednesday, January 30, 2008

Still holding my cup of Java

I am still working on some basic user settings for XPontus XML Editor. The "Plugins advanced" buttons will show a tree with the installed plugins ordered by category. I think it is a more intuitive approach instead of having a single bloated dialog.

Basic settings



Advanced settings

Some releases

I released the first beta of VFSJFileChooser yersterday. It will be integrated as a plugin in XPontus XML Editor.

XPontus snapshot release might hit sourceforge mirrors this friday. I will provide a link to the java web start download. I have a couple of bugs to fix :
  • The transformations profile editor isn't stable.
  • The preferences panel and the plugins manager are not ready yet
  • Code completion database doesn't support caching.
  • The XML schema completion is not ready(sometimes it works well, sometimes not)
  • I miss some informations in the plugins descriptors (author, homepage, etc.). This will be useful for the plugins manager and the preferences panel.
  • The xpath plugin is not stable.
I'll do my best to resolve the above issues. As it will be a java web start release, I'll be able to provide fixes on a daily basis according to the bug complexity. I'll attempt to fix the bugs reported in the previous versions of XPontus too!(If it is a new release, there must be some improvements!).

Tuesday, January 29, 2008

VFSJFileChooser to be released soon

TODO
Before releasing VFSJFileChooser, here are the features I would like to add :
  • A bookmarks manager : I found a bookmark dialog in some old jedit code, I might improve it and use it, at leastl for now.
  • A connection manager : You have to type the complete address with the username/password in the filename textfield to open a remote connection. I will be a simple dialog with few textboxes.
What have been fixed :
  • I patched commons-vfs to support http directs. For example : Trying to open http://vfsjfilechooser.sf.net/index.html would not work, I had to use http://vfsjfilechooser.sourceforge.net/index.html.
  • I removed the user credentials from the directory combobox

I need to 2 days to perform enough tests.

Monday, January 28, 2008

Settings are back

I didn't know how I was going to add a dialog to customize XPontus settings. Here is my first try.



Second attempt

Wednesday, January 23, 2008

You've been waiting for it, it is almost here

I am testing the snapshot release of XPontus. I was struggling yersterday with Java Web Start and the securityManager. The plugins weren't picked because of some java permissions issues. I was able to fix that.
I'm confident I will publish a snapshot release in a week or two. In the meantime, I am working on the plugins installer. I will try to get it ready for the snapshot release. If not, I will add a link where you can download the plugins and the instructions to install them.

Tuesday, January 22, 2008

VFSJFileChooser is now on Sourceforge.net

The VFSJFileChooser project has been accepted at Sourceforge. I has started its integration into XPontus as a plugin. XPontus will use an extended version of it to provide plugin support.

Validating an XML document or checking if it was well-formed was taking forever with XPontus 1.0.0-pre1 compared to the previous versions so I fixed it today.

Sunday, January 20, 2008

Yes I did it

I rewrote the virtual file system explorer from scratch. It is fast enough and is behaving well. I guess I can add it right now to XPontus XML Editor. I didn't want to copy too much existing interfaces and to borrow code here and there.

Features
  • Browse FTP, SAMBA, SFTP and Webdav file systems. You can select http files but you can't browse directories even if direct listing is allowed.
  • It behaves almost like JFileChooser

What's missing
  • There is no connection manager to add specific settings like proxy
  • The username and password are shown is the directory list of files
  • There is no details view
  • There is no real caching of files
  • Drag and drop is not supported
  • I need to patch commons vfs to support redirects

Friday, January 18, 2008

The virtual filesystem explorer is ok

The new file explorer will make it in the snapshot release. I've been testing it a lot. Bookmarks support is still missing but it doesn't matter for the snapshot distribution.


Thursday, January 17, 2008

Not quite why I want but...

I am not satified with the virtual file system explorer. When allowing multiple file selections, I have lots of issues. Conditions here and there, bad code make it more complex than it is. When I have a look at Sun JFileChooser implementation, I see about 1000 lines of code. I should have classes with less than 500 lines and simple interfaces.

I will leave the code as it is for now and rewrite it later. I will have a better look at jfilechooser's code to see how they handle multiple file selections vs single file selections.

Wednesday, January 16, 2008

XPontus File Explorer improvements

I am now able to hide the user credentials in the location bar. The problem is that I still have duplicated entries in the location bar history list. For the snapshot release I guess it is ok as long as the browser behaves well enough.



The file explorer supports sftp, ftp, cifs, webdav protocols. It can browse local files too!.

Tuesday, January 15, 2008

Virtual FileSystem Browser almost done

I am almost done with XPontus virtual filesystem browser. I am missing the following parts :
  • Bookmarks supports
  • Path history
  • User notifications(when the control takes time to display the directories or files)

Monday, January 14, 2008

What's new

I am coding a virtual file system browser based on Apache Commons VFS. I found existing components, but I didn't get the permission to use them yet.

Friday, January 11, 2008

Almost there

Done today
  • I fixed some bugs in the transformation profiles
  • I completed the transformation profiles support for saxon6.x , saxon 8.x and stx and fop.
  • The outline plugin is behaving well
  • I took the time to test most parts of the program
  • I started programming the XML signature module
Remaining issues
  • Code completion
    • The code completion doesn't work well with nested schemas
    • The code completion doesn't store a cache on the user's hard drive
    • I need to add built-in completion for XSL, Apache ant files, xml schemas, HTML
  • Virtual file system
    • The implementation is not complete(ssh, ftp, smb)
    • Sometimes I hardcoded the files path. I should always use a filesystem abstraction so that remote files can be found.
  • Users settings
    • In the previous versions of xpontus there was no plugins so I had a big panel with a tree to browse and set the preferences
    • Now most parts of the software are plugins and some of them can/should be customized by the user(font color, encoding, etc.)
    • I should write a settings importer for the old versions of XPontus
    • Using a relational database to store user settings might not be a good idea. If I upgrade XPontus and there are some major changes in the API, then all the users settings are gone unless I keep lots of old code in the program. I might revert to XStream which serialize objects in XML. If XStream API changes, I can still parse the data.
  • Plugin management
    • Because there are plugins the plugins browser must be at least able to install, list, remove plugins. I will think about the upgrade later.
    • The themes and icons modules are not ready yet. I need a settings initializer because the theme and icon set must be known before loading the program
  • Expression evaluator
    • The expression evaluator is not working well when evaluating xpath expressions. If the result is a boolean value, then no navigation link should be provided to look at the result. Sometimes it seems that the xpath expression is correct but no result is returned...
    • The xquery expression evaluator should have it's own output window to display the result. Sometimes the result is a node, but sometimes it will be plain text or whatever else.

Thursday, January 10, 2008

Catching up with the previous development version fo XPontus(without plugins)

I am adding features quickly, maybe too quickly. Well I'm going to clean up the code later.

The following components have been added today :
  • a simple quicktoolbar plugin(Comment XML selection and INSERT CDATA section)
  • a code structure view of XML documents