The Holy Java

Building the right thing, building it right, fast

Archive for February, 2010

Eclipse: Open Type/Resource working again under Linux!

Posted by Jakub Holý on February 26, 2010

Some weeks ago the extremely useful features of Eclipse Open Type and Open Resource stopped working, throwing an uninformative  Error instead, no matter which version or JRE vendor. I was desperate. Until finally I found a bug regarding this issue which pointed me to the solution – switching off Gnome’s assistive technologies (System – Preferences – Assistive Technologies – uncheck Enable assistive technologies under my Ubuntu 9.04 Jaunty). Thanks, allmighty powers of digitalized universe!

Posted in eclipse | Comments Off

Released DbUnit Test Skeleton 1.1.0 – also in Maven Central

Posted by Jakub Holý on February 23, 2010

Released DbUnit Test Skeleton 1.1.0 – also in Maven Central

I’ve released version 1.1.0 of my DbUnit extension, which is now also available in the Maven central repository under the group id net.jakubholy.testing and artifact id dbunit-embeddedderby-parenttest. It brings some API changes, new utillities and removes the need to extend the base TestCase class.

What is it?

Creating a a DbUnit test case may be time consuming. With the dbunit-embeddedderby-parenttest you can have your DbUnit test including an embedded database with structures and data ready in a few minutes. (At least if you are fast enough in writing SQL and preparing your data :-) ).

The added value of this project is that provides classes pre-configured to interact with an embedded Derby database and the only thing you need to do is to provide a DDL file, execute a utillity to create the database, and enter data into the predefined data set XML file. In addition to that it also provides some nice utility methods and classes such as RowComparator and IEnhancedDatabaseTester and it overrides the default DbUnit setting to use fully qualified table names (in the form schema.table), which gives you more freedom.

New and noteworthy

As you can guess from the version number change, the API has changed and may be not compatible with some code written against a previous version. The main changes are:

  • All functionality moved to EmbeddedDbTester so that extending the AbstractEmbeddedDbTestCase isn’t necessary anymore, which is useful e.g. in JUnit 4 or when extending another JUnit derivation. It’s also a standard DbUnit’s IDatabaseTester.
  • It’s now possible to change completely the DB used by defining some JDBC properties in dbunit-embedded.properties
  • Added utility methods getDataSource, getSqlConnection and a convenience method createCheckerForSelect, some of those implemented in the EnhancedDatabaseTesterDecorator so that they can be added to any IDatabaseTester.
  • Switched from Java util logging to commons-logging for better configurability.

    You can download the release also from the Maven Central Repository at http://repo1.maven.org/maven2/net/jakubholy/testing/dbunit-embeddedderby-parenttest/1.1.0/

Posted in Languages | Tagged: | Comments Off

The Art of Logging (review)

Posted by Jakub Holý on February 10, 2010

Colin Eberhardt, the co-author of the Simple Logging Facade, has written a very good article The Art of Logging, which should be a compulsory reading for every developer especially in the server side development domain. It’s good both as an introduction as it covers all the important aspects and also for experienced developers because it has some very valuable insights.

Few ideas and sentences that really should be stressed:

  • "Logging is the process of recording application actions and state to a secondary interface." Developers should really realize that a log is indeed an alternative interface to their application and may be the only one they’ll have access to in a production environment.
  • "Considering the importance of log levels, when embarking on a project, the development team should agree on exactly how these levels are used in order to ensure consistency between developers." I saw some projects where consistency was crouched and crying in the corner :-) With consistency it’s easier to understand a log, to know what to look for and how to search for it.
  • Regarding exceptions, you should be aware that not all exceptions are actually errors and you should always consider what level is appropriate for reporting a particular exception.
  • What to log:
    • The level of detail needed in a log depends on the presence or absence of other monitoring tools for the application.
    • On the importance of context: "In summary, a log message without context information is often as useful as no log message at all!." This sentence shall be in gold and also burnt to the skin of each developer who fails to apply it. How many times have I been desperate troubleshooting a problem for hours and days that wouldn’t take more than 5 minutes to diagnose and solve if proper context information was provided in the exception and/or log!
  • "… it is worth thinking about applying a standard format to your logged messages. This can prove very valuable if you need to analyse large log files (where tools like grep are indispensible)." I can sign this, trust the guy, he knows what he’s talking about!
  • "good messages are brief and provide the most important context information"

Logging to little is terrible if not outright evil but logging too much isn’t good either and brings numerous problems, for example it unnecessarily complicates the code, too detailed logs are hard to analyze, and, from my experience, large logs in production environments are cleared more often for space reasons and thus you’re more likely to have no log at all when you need it.

There is also a good point about unit testing your logging code to verify that important exceptions are indeed logged at the appropriate level and including the exception itself (not actually testing the message as that would be unnecessarily complicated to do and to maintain). I actually got recently the idea that while running unit tests you should always also review the log produced and verify that it communicates everything necessary for troubleshooting the tested cases and this at both the usually encountered log levels, i.e. info and debug.

Summary: Make sure to read the article and apply it in your day to day coding! And if you know about other sources regarding logging best practices, I’d be glad to learn about them too!

Posted in General | 1 Comment »

Releasing a project to Maven Central repository via Sonatype

Posted by Jakub Holý on February 7, 2010

If you have an open-source project and want it published into the Maven Central repository – even if it itself isn’t build with maven – to make it visible to all Maven users without any special effort (at least once you set everything up), read on.

This is excatly what I wanted to do with my dbunit-embeddedderby-parenttest, an extension of DbUnit that extremly simplifies setup of unit tests that need a database by using an embedded Derby DB and providing suitable defaults and utility methods.

The essential step is to deploy your artifacts to the Sonatype’s OSS hosting repository, which is free for open-source project, and which will take care of hourly automatic synchronization with Maven Central once you release your artifacts to Sonatype, satisfy some requirements and let them know about it. There is a good guide for setting up hosting with Sonatype but there a few tricky parts and that’s why I’ve written down this blog.

The procedure is as follows:

  1. Mavenize your project. You don’t actually need to build it with Maven, you only need to provide a maven metadata file (pom.xml). You may want to check my project’s pom.xml v1.0.2 (or the latest) for inspiration and to see how I’ve configured the other things mentioned here.
  2. Follow the steps described in the guide for setting up hosting with Sonatype:
    1. Sign up on the Sonatype repository manager Nexus at http://oss.sonatype.org/
    2. Sign up on the Sonatype ticketing system at https://issues.sonatype.org/browse/OSSRH
    3. Create a new ticket asking the creation of a maven repository for your project, see my OSSRH-141 as an example
    4. Make sure that your pom.xml satisfies all the requirements for publishing to the Maven Central Repo, listed in the guide (SCM url, site url, no <repositories> etc.)
    5. Wait for creation of the repositories, once finished, a comment will be added to the ticket, which should also result in send an email notification to you
  3. Build and sign your artifacts (see below)
  4. Deploy your artifacts to the new repository at Sonatype (see below)
  5. (optional?) Promote the published artifacts from the staging to the release repository (only applicable if you do not deploy directly to the release repo – but it seems obligatory to use staging now.)
  6. When the artifacts are in the release repository, add a comment to the original ticket asking for them to be published to Maven Central. For the first time they need to do it manually but since that on it will be synchronized automatically every hour

Build and sign your artifacts

Build your project’s artifacts as you are used to, for example with mvn package if you’re a Maven guy or with ant.

Next you need to sign them with your private key because valid signatures are one of the requirements for publishing to Maven Central. This is already well described in the article How to Generate PGP Signatures with Maven so I’ll only summarize it and add few tips.

  1. Create a new GPG or PGP key if you haven’t one already. You can use the command-line application gpg or a GUI tool such as seahorse (under Ubuntu: Applications > Accessories > Passwords and Encryption Keys).
  2. Publish the key to the required PGP server pgp.mit.edu:
    gpg –keyserver pgp.mit.edu –send-key <ID of your key; for me it was 66AE163A>
  3. Sign your files using either gpg directly or the maven-gpg-plugin (also described in the article mentioned above) – see below.

Signing with maven-gpg-plugin and automatic deployment

Using the maven-gpg-plugin has the advantage that running

mvn deploy -DperformRelease=true -Dgpg.passphrase=**** -Dgpg.keyname=<your key ID>

will automatically sign your artifacts and deploy them together with the signatures. On the other hand, as with any plugin, its configuration and use may have some issues. (Setting the property performRelease=true is necessary in my case because I have configured the plugin to be actived only if this is set.)

Tips

  • If the key you want to use for signing isn’t your defaul key then you need to tell Maven which key to use by storing its id in the property gpg.keyname either by passing -Dgpg.keyname=… on the command line or setting it in the pom.xml as you can is in the mine.

Signing and deploying signatures manually

You may instead decide to sign your artifacts manually using directly gpg (or whatever tool you have), for example like this:

gpg -u <your key ID> –sign –detach-sign -a target/dbunit-embeddedderby-parenttest.jar

This command would create the signature file target/dbunit-embeddedderby-parenttest.jar.asc.

Next you will need to deploy the signature to the repository along with the original artifact:

mvn deploy:deploy-file -Durl=http://oss.sonatype.org/service/local/staging/deploy/maven2 -DrepositoryId=nexus-releases -Dfile=target/dbunit-embeddedderby-parenttest.jar.asc -DpomFile=pom.xml -Dpackaging=asc

  • The url will be the same for everybody who wants to publish artifacts via a staging repository and not directly to to the release one otherwise you need to provide the URL of the release repository created for you by Sonatype.
  • The repositoryId must correspond to a <server> record in your <user home>/.m2/.settings.xml with the same id and your login name and password  for oss.sonatype.org, it tells maven what credentials to use to authenticate with the target repository
  • Beware that the name of the uploaded file isn’t the same as of the local file being uploaded but is constructed by Maven from the artifactId, version, and optionally packaging and classifier provided in the pom.xml or on the command line

And perhaps also deploy the artifact itself if not done otherwise:

mvn deploy:deploy-file
-Durl=http://oss.sonatype.org/service/local/staging/deploy/maven2
-DrepositoryId=nexus-releases
-Dfile=target/dbunit-embeddedderby-parenttest.jar -DpomFile=pom.xml

Examples of manual deployment

The following examples show you how to deploy files if different names to the Sonatype’s staging repository. They all use a common pom.xml (you’ll perhaps want change the groupId and artifactId) and you will need to change the -DrepositoryId to correspond to the server id you’ve assigned to your Sonatype repository credentials in your settings.xml (discussed below).

The common pom.xml:

 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <groupId>eu.ibacz.pokusy.jakubholy</groupId>
        <artifactId>smazMe</artifactId>
        <name>dummy pokusny projekt</name>
        <version>1.0-SNAPSHOT</version>       
</project>
$ mvn deploy:deploy-file -Durl=http://oss.sonatype.org/service/local/staging/deploy/maven2 -DrepositoryId=nexus-releases -Dfile=smazMe.jar -DpomFile=pom.xml -Dpackaging=jar

=> Uploading: http://oss.sonatype.org/service/local/staging/deploy/maven2/eu/ibacz/pokusy/jakubholy/smazMe/1.0-SNAPSHOT/smazMe-1.0-20100210.143500-1.jar

$ mvn deploy:deploy-file-Durl=http://oss.sonatype.org/service/local/staging/deploy/maven2 -DrepositoryId=nexus-releases -Dfile=smazMe.jar.asc -DpomFile=pom.xml

=> Uploading: http://oss.sonatype.org/service/local/staging/deploy/maven2/eu/ibacz/pokusy/jakubholy/smazMe/1.0-SNAPSHOT/smazMe-1.0-20100210.143508-2.jar.asc

 $  mvn deploy:deploy-file -Durl=http://oss.sonatype.org/service/local/staging/deploy/maven2 -DrepositoryId=nexus-releases -Dfile=smazMe-all.zip -DpomFile=pom.xml -Dpackaging=zip -Dclassifier=all

=> Uploading:

http://oss.sonatype.org/service/local/staging/deploy/maven2/eu/ibacz/pokusy/jakubholy/smazMe/1.0-SNAPSHOT/smazMe-1.0-20100210.143513-3-all.zip

$  mvn deploy:deploy-file -Durl=http://oss.sonatype.org/service/local/staging/deploy/maven2 -DrepositoryId=nexus-releases -Dfile=smazMe-all.zip.asc -DpomFile=pom.xml -Dpackaging=zip.asc -Dclassifier=all

=> Uploading:

http://oss.sonatype.org/service/local/staging/deploy/maven2/eu/ibacz/pokusy/jakubholy/smazMe/1.0-SNAPSHOT/smazMe-1.0-20100210.143524-4-all.zip.asc

Note: You may also supply the option -DuniqueVersion=false to avoid the addition of a generated number to file names.

Deploy your artifacts

Since we are going to deploy our artifacts to a Maven repository, we need to use Maven for that. We have a couple of ways of doing that: we can let Maven to deploy all that is necessary (which will also build and sign the artifact, if necessary), alternatively we can deploy already prepared artifacts manually, one by one, or finally we can do any of these using Maven Ant integration.

Notice that it’s now required to publish artifacts via a staging repository instead
of directly to the release one so that you can review and have to approve them before they are actually release (details in the following section).

A. Deploying directly with Maven

Unless you decide to use the release plugin (which I do use), which does some neat and useful things such as creating an SVN tag, increasing the project’s version etc., the deployment is as easy as executing

mvn deploy -DperformRelease=true -Dgpg.passphrase=**** -Dgpg.keyname=<your key ID>

(see the section about maven-gpg-plugin to understand why I need these three properties).

There are two important configurations that make this possible. First of all, you must configure the target deployment repository in your pom.xml:

<project ...>
   ...
    <distributionManagement>
        <repository>
            <id>nexus-releases</id>
            <name>Nexus Release Repository</name>
            <url>http://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
        </repository>
    </distributionManagement>
</project>

The <id> is arbitrary and the <url> is same for all, it’s actually a kind of a virtual repository and the artifacts will be in reality deployed to a temporary repository based on your username.

Next you need to store your username and password for oss.sonatype.org under the same <id> as above in the file <user home>/.m2/settings.xml so that maven can authenticate with the repository:

<settings>
   ...
   <servers>
      <server>
         <id>nexus-releases</id>
         <username>malyvelky</username>
         <password>******</password>
      </server>
   </servers>
</settings>

Troubleshooting deployment failures

Error 401 (Unauthorized)

If the deployement fails with an error like this:

Error deploying artifact: Failed to transfer file: http://oss.sonatype.org/service/local/staging/deploy/maven2//net/jakubholy/testing/dbunit-embeddedderby-parenttest/1.0.2/dbunit-embeddedderby-parenttest-1.0.2.jar. Return code is: 401

(notice the error code 401) it most likely means that you haven’t configured your username/password correctly or that the two IDs (repository id in pom.xml and server id in settings.xml) do not match.

Error 400 (Bad Request)

This error indicates that the repository manager has rejected your artifact for some reason. This may happen for example if you try to deploy a snapshot version into the staging repository, which is not allowed by its policy. (For snapshots you should reference the snapshot repository created for you as distributionManagement/snapshotRepository in your pom.xml.)

B. Deploying manually

If you have already built and signed your artifacts, you can deploy them manually with the mvn deploy:deploy-file goal as in the example below:

$ mvn deploy:deploy-file-Durl=http://oss.sonatype.org/service/local/staging/deploy/maven2 -DrepositoryId=nexus-releases -Dfile=smazMe.jar.asc -DpomFile=pom.xml

See the section Signing and deploying signatures manually above for explanation and examples of deployment of different types of files (.jar, .zip, signatures).

In this case you do not need the distributionManagement section in your pom.xml (because you’re providing all the information on the command line) but you still need to provide your login and password for the target repository (option -DrepositoryId=…) in your settings.xml as described above.

Of course you can run the deployment commands manually or for instance from Ant.

C. Deploying with Ant nad the Maven Ant Tasks

Update 10/2010: Sonatype user guide now contains the section 7c. Stage Artifacts with Ant.

You may also use the Maven Ant Tasks to invoke Maven operations from an Ant build including a task for file deployment. It does basically the same as invoking mvn deploy:deploy-file from the command line but you can provide the repository credentials here without a need for settings.xml and you may also find this way of Maven invocation nicer than calling a command-line application.

Promote the published artifacts from the staging to the release repository

This step is only necessary if you deploy via the staging repository and not directly to the release repository. But according to a recent email by Juven Xu it seems to be now obligatory to deploy via staging:

Staging is one of the most outstanding features in Nexus Pro. We’ve
been provided you this service for a long time, but maybe you were not
aware of this. So I changed your release repository configuration and
you will no longer be able to deploy release artifacts directly into
the release repository. Instead, you should deploy your release
artifacts into the staging repository: http://oss.sonatype.org/service/local/staging/deploy/maven2/,
then log in from Nexus UI, do things like closing staging repository,
and promoting the staged artifacts into your release repository.

If you are not familiar with Staging, it’s well documented at http://www.sonatype.com/books/nexus-book/reference/staging.html.

New instructions (6/2011)

  1. Log in to oss.sonatype.com
  2. Click on “Staging Repositories” under Build Promotion
  3. Verify the content of the repository (in the bottom pane), check it, click Close, confirm
  4. Check the repo again, click “Release”
  5. You shall now see your artifacts in the release repository created for you (mine is http://oss.sonatype.org/content/repositories/jakubholy-releases/).
  6. In some hours (?) it should also appear in Maven Central

Old instructions

You must do the following to promote the deployed artifacts from the staging to the release repository:

  1. Log in to oss.sonatype.com
  2. In the right menu, click on “Staging” in the box called “Enterprise” (#1 in the screenshot) => you should see the “staging profile” named <your project name> (for me it was “net.jakubholy”; see #2)
  3. Click on your staging repository target (“net.jakubholy”, #2) – you should see the individual repositories (some generated name such as net.jakubholy-028) within that including their status, which is “open” right after a deployment (which means that files may be still deployed and redeployed to this temporary repository)
  4. Verify that the project artifacts are as expected (see below)
  5. “Close” and “Promote” the staging repository to promote its content to the release repository: right-click on its status and you should see a little “context menu” with options Close and Drop, select Close (#3). You’ll be asked for a description, which will be associated with that release. After some time you should also receive a confirmation email from the “Nexus Repository Manager” with the subject “Nexus: Staging Completed.”
  6. “Promote” the closed staging repository to the release one in a way similar to closing it, only now seeing the current status as “closed” and selecting “Promote” from the context menu (beware: there may be some delay before this becomes possible), select your release repository as the target (Jakubholy Releases for me).
  7. You shall now see your artifacts in the release repository created for you (mine is http://oss.sonatype.org/content/repositories/jakubholy-releases/).

Sonatype Repository Manager - closing a stagin repo.

Required artifacts

[Since 2010-04-19] It is no required that you include -javadoc.jar along with -sources.jar with your artifacts otherwise you will not be allowed to promote. You can easily create them either with Maven or manually.

Verifying project artifacts before promoting from the staging repository

Before you promote the project’s artifacts from the staging to the release repository you may want to check few things to be sure that the release is OK:

  • Are all artifacts included?
  • pom.xml:
    • Is the version all right?
    • Are the SCM URLs correct? (Should be set by the release plugin to the tag for that release.)

When will the artifacts be available in Maven Central?

In ideal case within one hour, in reality it seems to be few hours, even 12h sometimes.

Summary

We’ve learned how to create a maven repository at oss.sonatype.org, how to sign our artifacts with a private key, how to use the staging repository and finally how to get the released artifacts pushed to the Maven Central repository, where they will be available to all maven users out of the box.

Other useful links

Unrelated but interesting:

Posted in Tools | Tagged: , , , | 1 Comment »