Author: Ceki Gülcü
Copyright © 2009-2013, QOS.ch

 Creative Commons License

This document is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License

CAL10N Manual

The goal of CAL10N project is to enhance the existing internationalization functionality offered by the Java platform with consistency and verification primitives. It is assumed that you are already somewhat familiar with using ResourceBundles.

Acknowledgment

The original idea behind CAL10N is attributed to Takeshi Kondo. It consolidated into what it is today subsequent to a discussion involving Ralph Goers, Ceki Gülcü, Takeshi Kondo and Pete Muir on the slf4j-dev mailing list.

Core idea

Instead of using String-typed keys for each message, CAL10N uses enums.

For example, let us assume that you wanted to internationalize color names. We are using a very small set of colors just as an example. In CAL10N you could start by writing an enum type, named Colors. You can choose any name for the enum type. Colors is just the name this particular author picked.

package com.foo.somePackage;


import ch.qos.cal10n.LocaleData;
import ch.qos.cal10n.Locale;
import ch.qos.cal10n.BaseName;

@BaseName("colors")
@LocaleData( { @Locale("en_UK"), @Locale("fr") })
public enum Colors  {
  BLUE,
  RED,
  YELLOW;
}

Once you define a few color keys, you can create a regular resource bundle named after the value of the @BaseName annotation in Colors and the appropriate locale. For example, for the UK locale, you would name your resource bundle as colors_en_UK.properties. It should also be placed in the appropriate folder on your class path or in the root folder of an appropriate jar file. (It is assumed that you know how to work with resource bundles.)

Here is a sample a colors_en_UK.properties file:

BLUE=violets are blue
RED=roses are red
GREEN={0} are green

For the french locale, the resource bundle would be named colors_fr.properties. Here are sample contents.

BLUE=les violettes sont bleues
RED=les roses sont rouges
GREEN=les {0} sont verts

Strictly speaking, the @LocaleData annotation is optional. It is used by verification tools discussed below.

Retrieving internationalized messages

In your application, you would retrieve the localized message via an IMessageConveyor instance. 

// obtain a message conveyor for France
IMessageConveyor mc = new MessageConveyor(Locale.FRANCE);

// use it to retrieve internationalized messages
String red = mc.getMessage(Colors.RED);
String blue = mc.getMessage(Colors.BLUE);  
String green = mc.getMessage(Colors.GREEN, "pommes");  // note the second argument

CAL10N leverages the existing resource bundle infrastructure you have been accustomed to, but adds compiler verification on top. The default IMessageConveyor implementation, namely MessageConveyor, uses the standard Java convention for parameter substitution as defined by the java.text.MessageFormat class. In particular, for messages with parameters, the rules for using quotes defined by the MessageFormat class apply as is.

An astute reader will comment that even if the messages keys are now verified by the compiler, it is still possible to have mismatching message keys in the resources bundles. For this reason, CAL10N comes with additional tools, including support for JSR-269, writing Junit tests as well as a maven plugin.

Requirements / Installation

CAL10N requires JDK version 1.5. JSR-269 requries JDK 1.6 or later.

In order to use CAL10N in a project, all you need is to add cal10n-api-0.8.1.jar onto your project's class path.

For Maven users, this is done by adding the following dependency in a project's pom.xml file:

<dependency>
  <groupId>ch.qos.cal10n</groupId>
  <artifactId>cal10n-api</artifactId>
  <version>0.8.1</version>
</dependency>

Simplified bundle look-up procedure

The ResourceBundle class defines a rather involved look-up procedure for finding resource bundles. While formally this procedure is deterministic, it is also error-prone. More importantly, it clashes with CAL10N's goal of verifiability. We thus took the bold initiative to define a simplified bundle look up procedure, described below.

Given a locale, the simplified look up procedure only takes into account that locale, ignoring the default locale and the resource bundle corresponding to the naked base name, i.e. the default resource bundle. However, if the locale has both a language and a country, and corresponding bundle files exist, then CAL10N will take into account both bundles, establishing the same parent child relationship as the JDK ResourceBundle class.

For example, for base name "colors" if the following bundles exist on the class path: 

colors.properties
colors_en_US.properties
colors_en.properties
colors_fr_FR.properties

and the system's default locale is "fr_FR", when CAL10N is asked to find resource bundles corresponding to the "en_US" locale, it will systematically ignore the colors.properties (i.e. the default bundle), and combine the colors_en_US.properties and colors_en.properties bundles in the usual parent-child relationship. Since CAL10N is asked to lookup resource bundles corresponding to the "en_US" locale, the bundle corresponding to the default locale, i.e. colors_fr_FR.properties will also be ignored. Thus, the CAL10N bundle lookup procedure differs from the standard by ignoring the default bundle, the one with the naked base name, e.g. colors.properties.

We hope that the simplified look-up procedure, while deviating from the conventions of the Java platform as defined in the ResourceBundle class, will still cause less surprise.

Pick your charset, per locale (no native2ascii)

CAL10N allows you to encode the resource bundle for a given locale in the charset of your choice. The set of supported charsets depends on your Java platform. See this java encoding and charset list for more details.

Assume you have four resource bundles, for the English, French, Turkish and Greek languages. You decide to encode all of them in UTF-8. To tell CAL10N that UTF-8 is the default encoding for all locales, one would write:

@BaseName("colors")
@LocaleData(
  defaultCharset="UTF8",
  value = { @Locale("en_UK"),
            @Locale("fr_FR"),
            @Locale("tr_TR"),
            @Locale("el_GR")
           }
   )
public enum Colors {
 BLUE,
 RED,
 GREEN;
}

If for some reason the Turkish bundle was encoded in ISO8859_3, but the others locales in UTF8, you would write:

@BaseName("colors")
@LocaleData(
  defaultCharset="UTF8",
  value = { @Locale("en_UK"),
            @Locale("fr_FR"),
            @Locale(value="tr_TR", charset="ISO8859_3"),
            @Locale("el_GR")
           }
   )
public enum Colors {
 BLUE,
 RED,
 GREEN;
}

The defaultCharset directive specified in the @LocaleData annotation applies to all nested @Locale annotations, unless the value is overriden by a charset directive (as in the "tr_TR" locale in the example above). If not specified, the default value for defaultCharset is the empty string. In the absence of a defaultCharset directive, the default value for the charset directive is also the empty String.

If both charset and defaultCharset are empty, CAL10N will use the default encoding for your Java platform to read resource bundles.

Automatic reloading of resource bundles upon change

When a resource bundle for a given locale is changed on the file system, MessageConveyor will automatically reload the resource bundle. It may take up to 10 minutes for the change to be detected.

Automatic reloading applies if the resource bundle is a regular file but not if nested within a jar file.

Deferred localization

Under certain circumstances, the appropriate locale is unknown at the time or place where the message key and message args are emitted. For example, a low level library might wish to emit a localized message but it is only at the UI (user interface) layer that the locale is known. As another example, imagine that the host where the localized messages are presented to the user is in a different locale, e.g. Japan, than the locale of the source host. e.g. US.

The MessageParameterObj class is intended to support this particular use case. It allows you to bundle the message key (an enum) plus any message arguments. At a later time, when the locale is known, you would invoke the getMessage(MessageParameterObj) method in IMessageConveyor to obtain the localized message.

Verification as a test case

A convenient and low hassle method for checking for mismatches between a given enum type and the corresponding resource bundles is through Junit test cases.

Here is a sample Junit test for the Colors enum discussed above.

package foo.aPackage;

import static org.junit.Assert.assertEquals;

import java.util.List;
import java.util.Locale;

import org.junit.Test;

import ch.qos.cal10n.verifier.Cal10nError;
import ch.qos.cal10n.verifier.IMessageKeyVerifier;
import ch.qos.cal10n.verifier.MessageKeyVerifier;

import ch.qos.cal10n.sample.Colors;

public class MyColorVerificationTest {

  @Test
  public void en_UK() {
    IMessageKeyVerifier mkv = new MessageKeyVerifier(Colors.class);
    List<Cal10nError> errorList = mkv.verify(Locale.UK);
    for(Cal10nError error: errorList) {
      System.out.println(error);
    }
    assertEquals(0, errorList.size());
  }

  @Test
  public void fr() {
    IMessageKeyVerifier mkv = new MessageKeyVerifier(Colors.class);
    List<Cal10nError> errorList = mkv.verify(Locale.FRANCE);
    for(Cal10nError error: errorList) {
      System.out.println(error);
    }
    assertEquals(0, errorList.size());
  }
}

The above unit tests start by creating a MessageKeyVerifier instance associated with an enum type, Colors in this case. The test proceeds to invoke the verify() method passing a locale as an argument. The verify() method returns the list of errors, that is the list of discrepancies between the keys listed in the enum type and the corresponding resource bundle. An empty list will be returned if there are no errors.

The unit test verifies that no errors have occurred by asserting that the size of the error list is zero.

Suppose the key "BLUE" was misspelled as "BLEU" in the colors_fr.properties resource bundle. The unit test would print the following list of errors and throw an AssertionError. 

Key [BLUE] present in enum type [ch.qos.cal10n.sample.Colors] but absent in resource bundle \
   named [colors] for locale [fr_FR]
Key [BLEU] present in resource bundle named [colors] for locale [fr_FR] but absent \ 
   in enum type [ch.qos.cal10n.sample.Colors]

One test to rule them all

Instead of a separate unit test case for each locale, assuming you declared the locales in the enum type via the @LocaleData and nested @Locale annotations, you can verify all locales in one fell swoop.

package foo.aPackage;

import static org.junit.Assert.assertEquals;

import java.util.List;
import org.junit.Test;
import ch.qos.cal10n.verifier.Cal10nError;
import ch.qos.cal10n.verifier.IMessageKeyVerifier;
import ch.qos.cal10n.verifier.MessageKeyVerifier;
import ch.qos.cal10n.sample.Colors;

public class MyAllInOneColorVerificationTest {

  // verify all locales in one step
  @Test
  public void all() {
    IMessageKeyVerifier mkv = new MessageKeyVerifier(Colors.class);
    List<Cal10nError> errorList = mkv.verifyAllLocales();
    for(Cal10nError error: errorList) {
      System.out.println(error);
    }
    assertEquals(0, errorList.size());
  }
}

JSR-269 support, i.e. verification at compile time

Verify your bundles for every compilation of an enum class annotated with @BaseName.

As of version 0.8, CAL10N ships with an JSR-269 compliant annotation processor named CAL10NAnnotationProcessor. By enabling this processor you can have javac, i.e. the java compiler, report discrepancies in your bundles at compile-time without leaving your favorite IDE. The verification is triggered at every compilation of an enum class annotated with @BaseName.

All majors IDEs, including Eclipse, IntelliJ IDEA and Netbeans support annotation processors.

The fully qualified name of CAL10NAnnotationProcessor is ch.qos.cal10n.verifier.processor.CAL10NAnnotationProcessor.

As all annotation processors CAL10NAnnotationProcessor can be integrated with javac by creating a file named META-INF/services/javax.annotation.processing.Processor containing the fully qualified name of the processor, i.e. ch.qos.cal10n.verifier.processor.CAL10NAnnotationProcessor. The artifact cal10-api-0.8.1.jar ships precisely with such a file.

Activating CAL10NAnnotationProcessor in Eclipse

The internet abounds with documentaton on installing an annotation processor for Eclipse. The author found the blog entry entitled Using Java 6 processors in Eclipse quite helpful. In the author's experience, the project's Compiler comliance level in Eclipse should be 1.6 or higher. Otherwise, the annotation processor is not activated.

As an additional caveat, it appears that Eclipse (Indigo and Juno) hides resources located udner "src/test/resources" while resources under "src/main/resources" are visible to the annotation processor. Thus, if resource bundles are placed under "src/test/resources" the compiler will complain about missing resource bundles. As soon as the bundles are moved under "src/main/resources" the errors will disappear (assuming the bundles are complete).

Activating CAL10NAnnotationProcessor in IntelliJ IDEA

IntelliJ IDEA's support for annotation processors is superb. Searching for "annotation processor intellij idea" on Google should yeild the appopriate documentation. Just remember to define a "Anotation Processor" profile using CAL10NAnnotationProcessor fully qualified name and add the modules in your project which depend on cal10n to that profile. The integration should proceed rather smoothly.

Maven Plugin

JSR-269 obsoletes maven plugin The maven-compiler-plugin automatically integrates CAL10NAnnotationProcessor for every dependecy of cal10-api rendering the Maven Plugin obsolete.

The CAL10N project ships with a maven plugin designed to verify that the keys specified in a given enum type match those found in the corresponding resource bundles and for each locale. Our plugin is unsurprisingly called mvn-cal10n-plugin.

Using maven-cal10n-plugin is pretty easy. To verify enums in a given project, just declare the maven-cal10n-plugin in the <build> section, enumerating all the enum types you would like to see checked.

Here is a sample pom.xml snippet.

<build>
  <plugins>
   ... other plugins
    <plugin>
      <groupId>ch.qos.cal10n.plugins</groupId>
      <artifactId>maven-cal10n-plugin</artifactId>
      <executions>
        <execution>
          <id>aNameOfYourChoice</id>
          <phase>verify</phase>
          <goals>
            <goal>verify</goal>
          </goals>
          <configuration>            
            <enumTypes>
               <!-- list every enum type you would like to see checked -->
               <enumType>some.enumTpe.Colors</enumType>
               <enumType>another.enumTpe.Countries</enumType>
            </enumTypes>
          </configuration>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

After you add the above snippet to the pom.xml file of your project, maven-cal10n-plugin will make sure that your resource bundles and your enum type are in synchronized.

The plugin will iterate through every resource bundle for every locale listed in the enum type via the @LocaleData and @Locale annotations.

Eclipse plug-in

We are looking for volunteers willing to implement IDE support for CAL10N. Below is a list of possible features of this IDE which we think could have added value.

If interested please contact the cal10n-dev mailing list.

Ant task

We are looking for volunteers to implement an Ant task to verify resource bundles. The Ant task could be modeled after the maven-cal10n-plugin although the Ant plugin is likely to be simpler. If interested please contact the cal10n-dev mailing list.