out.println

Simplified varargs for dummies

2011-01-04T21:34:00.015+01:00

I've migrated this post to my new blog.

Breakpoint dependencies in Eclipse

2010-12-16T19:33:00.015+01:00

This post describes a hack to achieve the following behavior in the Eclipse debugger: stop at a given breakpoint only if another breakpoint has been reached before.
I'm not aware of an existing solution to do that (although IDEA seems to have that functionality) — and if I'm wrong, I've just lost a couple of hours of my life :-)

The problem

Suppose I'm debugging a unit test that runs the same assertion over and over with different parameters (I know unit tests should have few assertions, but sometimes this pattern is just more practical):

  assertEquals(43, SomeClass.someMethod(1));
  assertEquals(44, SomeClass.someMethod(2));
  assertEquals(45, SomeClass.someMethod(3));
  assertEquals(50, SomeClass.someMethod(4));

Suppose the last assertion is failing. I want to debug it, and I'm particularly interested in an internal method called indirectly by someMethod, deep down the call stack:

private static int internalMethod(int p) {
  return p + 42;
}

Basically, I have three options:

set a breakpoint on internalMethod. But the debugger will suspend every time the method is reached, including in the first N-1 calls to someMethod that I'm not interested in. That sucks.
set the breakpoint on the assertion that fails. But when the debugger suspends, I still have to go down the call stack to reach internalMethod. This involves using Go into repeatedly, which may go into a lot of auxiliary code I'm not interested in (any method invoked to build parameters, class loads, etc.). That also sucks.
set the breakpoint on the outer method, and when it suspends, set the breakpoint on the inner method manually, and hit Continue to reach it directly. That sucks less, but still involves manual work (to be undone and redone everytime I rerun the test).

Wouldn't it be nice to set both breakpoints, and configure the second one to "activate" only after the first one has been reached? Well that's possible: breakpoints can be conditioned by Java code, and that code can have side effects.

The solution

I drop a class named Breaks in my test folder (the source code of the class is available here). On the outer breakpoint, I add the following condition:

This sets a "marker" that is accessible from other breakpoints. The set method returns false, so the debugger will never suspend on that breakpoint (if I want it to, there is an equivalent setAndBreak method).

On the second breakpoint, the condition invokes a method that returns true only if the marker is set (alternatively, testAndClear also deletes the marker):

Voilà! As long as the outer breakpoint hasn't been reached, the inner breakpoint will not suspend.

This is portable to any debugger that supports conditions, and more elaborate strategies are possible (for instance, associating counters to the markers, instead of simple on / off state).

Update 2010/12/18: you will run into compile errors if Breaks and the debugged classes reside in different packages. To avoid this, put Breaks in a package with a short name, and use the fully qualified name in the breakpoint conditions (putting it in the default package doesn't seem to work, and import statements are not allowed in breakpoint conditions).

Remove boilerplate with Guava

2010-06-20T17:32:00.008+02:00

Guava describes itself as "Google's core Libraries for Java 1.5+". It is a superset of the Google Collections framework, along with other utilities, released as an open-source project under the Apache 2.0 license.

What I want to cover in this post are not Guava's major features, but rather some simple "shortcut" methods (each factoring a common Java idiom) that can make a developer's life easier.

To illustrate this, let's consider this simple method:

public void save(final String info, final File file) throws IOException {
  if (file == null) throw new NullPointerException("file should not be null");
  Writer writer = null;
  boolean threw = true;
  try {
    writer = new BufferedWriter(new OutputStreamWriter(
        new FileOutputStream(file), Charset.forName("UTF-8")));
    String data = info == null ? "Nothing" : info;
    writer.write(data);
    threw = false;
  } finally {
    if (writer != null)
      try {
        writer.close();
      } catch (IOException e) {
        // Do not swallow original exception
        if (!threw) throw e;
      }
  }
}

Here is a version with the corresponding Guava methods:

public void save(final String info, final File file) throws IOException {
  Preconditions.checkNotNull(file, "file should not be null");
  Writer writer = null;
  boolean threw = true;
  try {
    writer = Files.newWriter(file, Charsets.UTF_8);
    writer.write(Objects.firstNonNull(info, "Nothing"));
    threw = false;
  } finally {
    Closeables.close(writer, threw);
  }
}

Actually, there is even simpler, since we are only writing one string to the file:

public void save(final String info, final File file) throws IOException {
  Preconditions.checkNotNull(file, "file should not be null");
  Files.write(Objects.firstNonNull(info, "Nothing"), file, Charsets.UTF_8);
}

Optionally, you can also static import everything (which increases readability but makes the origin of the methods less obvious — the choice is a matter of personal taste):

public void save(final String info, final File file) throws IOException {
  checkNotNull(file, "file should not be null");
  write(firstNonNull(info, "Nothing"), file, UTF_8);
}

That's it. Nothing groundbreaking really, but I like how this makes the code more expressive, focusing on what you do rather than how you do it. Guava has a myriad of these utilities (look for static methods in the classes that have a plural name: Lists, Strings, Resources...).

Effectively immutable

2010-06-13T19:42:00.007+02:00

Following on my previous post, another situation where shared state can be accessed without synchronization is when dealing with effectively immutable objects.

Here is a class to gather simple statistics during a multi-threaded batch process:

public final class Stats {
  public enum Counter { INPUTS, PARSE_ERRORS, PROCESSING_ERRORS, SUCCESSES, };
  
  private final EnumMap<Counter, AtomicInteger> values;
  
  public Stats() {
    values = new EnumMap<Counter, AtomicInteger>(Counter.class);
    // Pre-fill the map:
    for (Counter counter: Counter.values())
      values.put(counter, new AtomicInteger());
  }
  
  public void increment(Counter counter) { values.get(counter).incrementAndGet(); }
  
  @Override
  public String toString() {
    StringBuilder sb = new StringBuilder();
    for (Entry<Counter, AtomicInteger> entry: values.entrySet())
      sb.append(entry.getKey() + " = " + entry.getValue() + "\n");
    return sb.toString();
  }
}

A single instance is shared among all worker threads. Each worker increments the appropriate counters as it progresses:

stats.increment(Stats.Counter.INPUTS);

The statistics are output when all work is done:

System.out.println(stats);

The important point here is that access to the values field is not synchronized, yet Stats is still thread-safe. This is because values is effectively immutable: after it has been pre-filled in the constructor and safely published through a final field, it is never structurally modified. Therefore it is safe to assume that all invocations of increment will see it in a consistent state.

On the other hand, here is an example where the modification of an EnumMap is not safely published:

public class UnsafeEnumMapMutation { // DOESN'T WORK
  static enum Command { STOP }
  private static final EnumMap<Command, Integer> MAP =
    new EnumMap<Command, Integer>(Command.class);
  
  public static void main(final String[] args) throws InterruptedException {
    final ExecutorService executor = Executors.newFixedThreadPool(2);
    final Runnable looping = new Runnable() {
      public void run() {
        int i = 0;
        while (!MAP.containsKey(Command.STOP)) i++;
      }
    };
    executor.submit(looping);
    final Runnable stopper = new Runnable() {
      public void run() { MAP.put(Command.STOP, 1); }
    };
    executor.submit(stopper);
    executor.shutdown();
  }
}

As in my previous post, task stopper is trying to tell task looping to terminate. This time, the signal is the presence of a given key the map. However, this change is not protected by synchronization, and the map is not effectively immutable, since it is mutated after its initial publication as a final field. And again, this example loops when I run it with the -server option.

Piggybacking on synchronization

2010-06-09T14:02:00.006+02:00

Sometimes you can take advantage of an existing synchronization to guarantee the visibility of some changes. In Java Concurrency In Practice, Brian Goetz calls this "piggybacking on synchronization".

A typical example is the publication of read-only resources that will be shared by all tasks in a thread pool: one of my projects at work parses and processes a few thousands XML files in parallel. I use XStream for the parsing. The javadoc of XStream — the base class that lets you configure the framework and [de]serialize objects — states that it is safe to use by multiple threads once it has been configured. So it makes sense to use a single instance all along:

public class GoodPiggy {
  private ExecutorService executor;
  private XStream xstream;
  
  public void init() {
    executor = Executors.newFixedThreadPool(someSize);    
    final XStream xstream = new XStream();
    // configure xstream...
  }
  private class Task implements Runnable {
    public void run() {
      // do something with xstream... (threadsafe)
    }
  }
  public void submit() {
    executor.submit(new Task());
  }
}

The important question here is: are the changes to xstream safely published, i.e. do the worker threads running the tasks "see" the configuration performed in init()? This is where piggybacking comes into play: submitting a task to an executor happens-before the task begins execution, so the code using xstream in run() is guaranteed to see it in a consistent state (assuming of course that init() and submit() are run by the same thread).
JCIP warns that piggybacking on synchronization is a fragile construct, but I think this case is a reasonable use.

Note: ~~an even better solution is to make xstream a final field initialized in the class's constructor~~. EDIT 2010/06/13 — that was actually wrong: a final field would only guarantee the safe publication of the reference, but not of the changes performed afterwards on the (mutable) object designated by that reference. Piggybacking would come into play even with a final field.

Of course, things don't work so well if the tasks need to modify the shared state. Here is a counter-example (adapted from item 66 of Effective Java, 2nd edition):

public class BadPiggy { // DOESN'T WORK
  private static boolean stop;
  
  public static void main(final String[] args) throws InterruptedException {
    final ExecutorService executor = Executors.newFixedThreadPool(2);
    Runnable looping = new Runnable() {
      public void run() {
        int i = 0;
        while (!stop) i++;
      }
    };
    executor.submit(looping);
    Runnable stopper = new Runnable() {
      public void run() { stop = true; }
    };
    executor.submit(stopper);
    executor.shutdown();
  }
}

Here the modification of stop is not safely published, because there is no happens-before relationship between the moment stopper sets it, and the moment looping is expected to see the change. I can get this example to loop forever on my machine (windows, Java 6u16 VM with -server option).

The solution here is to make the field volatile, which guarantees visibility of the changes. If we needed mutual exclusion as well (for instance, if the shared state was an incrementing counter), we would need synchronization, either explicitly, or through a higher-level construct (like an AtomicInteger).

Using Scala implicit conversions for backward compatibility

2010-05-26T09:33:00.012+02:00

This post explains how to use implicit conversions to work around API incompatibilities between Scala 2.7 and 2.8.
I learnt about this technique from reading the source code of SXR. What follows is a step-by-step explanation.

Implicit conversion basics

Implicit conversions can make an object behave like it has another type. In particular, it can be used to add new methods to existing types. When a method is invoked on an object and no match is found, Scala will try to apply any implicit conversion currently in scope. Try typing this in the Scala REPL:

implicit def intToHello(i: Int) = {
  println("intToHello was applied")
  new { def message = "hello from " + i }
}

In this contrived example, we define an implicit conversion to add a message method to integers. The conversion returns an anonymous instance that provides the method. I've added a print statement for illustrative purposes. We can now invoke message on any integer:

scala> println(42.message)
intToHello was applied
hello from 42

A case of API incompatibility: `Symbol.fullNameString`

SXR is a Scala compiler plugin that generates an HTML cross-reference of the sources of a Scala project. In order to support partial recompilation and cross-project links, it needs to generate "stable" IDs for public compiler symbols; in particular, these IDs contain the full name of the symbol, which, in Scala 2.7.7, is retrieved with the method Symbol.fullNameString :

def nameString(s: Symbol) = s.fullNameString

(full source)

Unfortunately (for this particular situation!), symbols have been refactored in Scala 2.8. Class Symbol now extends scala.reflect.generic.Symbols.AbsSymbol, and the method we need is now called fullName. The code above no longer compiles:

<console>:12: error: value fullNameString is not a member of Demo.this.global.Symbol
   def nameString(s: Symbol) = s.fullNameString

Implicits to the rescue

To compile the code with Scala 2.8, Symbol must behave like it has a fullNameString method. This is done exactly as in our first intToHello example:

def nameString(s: Symbol) = s.fullNameString
private implicit def symCompat(sym: Symbol): SymCompat = new SymCompat(sym)
private final class SymCompat(s: Symbol) {
    def fullNameString = s.fullName;
}

(full source)

An implicit is used to add the missing fullNameString method, which delegates to the original fullName. The code now compiles with Scala 2.8. However, it no longer compiles with 2.7:

<console>:12: error: value fullName is not a member of Demo.this.global.Symbol
   def fullNameString = s.fullName;

The 2.7 version of Symbol has no fullName method. Although the symCompat implicit will never be invoked at runtime in Scala 2.7 (since Symbol does defines fullNameString), this is still a compile error. To solve this, the missing method is added to the same implicit conversion:

def nameString(s: Symbol) = s.fullNameString
private implicit def symCompat(sym: Symbol): SymCompat = new SymCompat(sym)
private final class SymCompat(s: Symbol) {
    def fullNameString = s.fullName; def fullName = sourceCompatibilityOnly
}
private def sourceCompatibilityOnly = assert(false, "should not get here")

(full source)

To stress the fact that the method will never actually be invoked at runtime, it delegates to a method that throws an assertion error.

Conclusion

The final version of the code compiles with both versions of Scala. The implicit conversion and its supporting type provide runtime compatibility for Scala 2.8, and compile-time compatibility for Scala 2.7. Note that this can also be done the other way around; if the code is intended primarily for one version, it is probably a good idea to have the implicit applied on the other one. Finally, this technique should be properly encapsulated, to avoid leaking the implicits to the rest of your code.

How NOT to microbenchmark

2010-01-16T09:53:00.004+01:00

A terrible article on developer.com spawned a number of reactions in various forums: Java vs. C++: The Performance Showdown. I won't go over the flaws in the author's testing "procedure", since this is already done in the article's comments (in particular this one).

But even bad articles can have positive effects: following some of the links in the comments, I eventually came across this excellent article by Brian Goetz. Here is his advice on micro-benchmarking:

So, how do you write a perfect microbenchmark? First, write a good optimizing JIT. Meet the people who have written other good, optimizing JITs (they're easy to find, because not too many good, optimizing JITs exist!). Have them over to dinner, and swap stories of performance tricks on how to run Java bytecode as fast as possible. Read the hundreds of papers on optimizing the execution of Java code, and write a few. You will then have the skills you need to write a good microbenchmark for something like the cost of synchronization, object pooling, or virtual method invocation.

Wow! It seems Mr Goetz is telling us we should not try to write microbenchmarks. So what should we do instead? Go read the article!

Hibernate: disabling all lazy associations programmatically

2010-01-07T13:50:00.004+01:00

In an ideal world, you would define a global fetch plan for your application, carefully weighing the pros and cons of making each association lazy.

In real life, you might find yourself forced to reuse existing mappings that use (and abuse of) lazy associations. This is annoying in a performance-critical use case, since you probably don't want 10 requests to go to the database for each loaded entity.

I found an old post about this issue on the Hibernate forums. "christian" of the Hibernate team (whom I assume is Christian Bauer) had this suggestion:

Usually, you override fetching strategies with HQL and Criteria queries. If you like to disable/enable some setting globally, you might rebuild the SessionFactory after modifying the metadata from the Configuration programatically. This is easier than it sounds, get all PersistentClass objects from Configuration (the API), loop over all collections and setLazy(true). Then, build a new SessionFactory from this Configuration.

This is acceptable for once-a-day batch operations, I think.

This is exactly what I needed. Here is an implementation:


public static Configuration setAllToLazy(final Configuration cfg) {
 @SuppressWarnings("unchecked")
 Iterator<PersistentClass> classes = cfg.getClassMappings();
 while (classes.hasNext()) {
  PersistentClass clazz = classes.next();
  setPropertiesToLazy(clazz.getPropertyIterator());

  Property key = clazz.getIdentifierProperty();
  // Don't forget composite keys, which can contain associations
  if (key.isComposite()) {
   Component c = (Component) key.getValue();
   setPropertiesToLazy(c.getPropertyIterator());
  }
 }

 @SuppressWarnings("unchecked")
 Iterator<Collection> collections = cfg.getCollectionMappings();
 while (collections.hasNext()) {
  Collection collection = collections.next();
  collection.setLazy(true);
 }
 return cfg;
}

// Handle "simple" (*-to-one) properties
@SuppressWarnings("unchecked")
private static void setPropertiesToLazy(final Iterator iterator) {
 Iterator<Property> properties = iterator;
 while (properties.hasNext()) {
  Value value = properties.next().getValue();
  if (value instanceof ToOne) {
   ((ToOne) value).setLazy(true);
  }
 }
}

Follow-up on "Maven modules with independent versions"

2009-03-17T00:05:00.004+01:00

Frank provided insightful feedback on my previous post Maven modules with independent versions:

First of all, he ran into the same issues as me, which proves that my post was not completely pointless :-)

More importantly, he found a weakness in my solution: some repository managers (such as Nexus) can be configured to prevent redeploys of release versions -- which, on a sidenote, is probably a good idea.
However, in my example from the previous post, this prevents global deployments to be run from the root directory, since some of the modules are still in a previous version that is already in the repository.

for release deployments, this is not really a problem: my solution assumed individual module releases anyway, and the release plugin takes care of deploying.
for snapshot deployments, this is more annoying: they are typically performed nightly by the continuous integration system. But the modules that are not impacted by the current development iteration have not been incremented to a snapshot version, therefore they break the nightly deploy. And you know how grumpy our developers get when they don't have their fresh snapshots in the morning.

To address these issues, I've added two system properties that will condition the deployment of each module:

maven.delivery.version: if set, only the modules with this version will be deployed. This takes care of release deployments. For instance, in the case where webapp is in version 1.0.1 and the business modules are still in 1.0.0, you run this command from the global POM to ensure 1.0.0 artifacts are not redeployed:
```
mvn deploy -Dmaven.delivery.version=1.0.1
```
maven.delivery.deploySnapshotsOnly: if set to true, only snapshot modules are deployed. For instance, during the 1.0.1 iteration, webapp is in version 1.0.1-SNAPSHOT, and the business modules are still in 1.0.0. You configure this command in your CI system:
```
mvn deploy -Dmaven.delivery.deploySnapshotsOnly=true
```

These properties are handled by a new mojo in our custom plugin. When it decides that a module must not be redeployed, it simply sets maven.deploy.skip (a built-in property of the deploy plugin) to true on the project.
This mojo must be executed before the deploy phase, so we bind its execution to the install phase of parent:

<plugin>
  <groupId>com.acme</groupId>
  <artifactId>delivery-maven-plugin</artifactId>
  <executions>
    <execution>
      <id>check-deploy-needed</id>
      <phase>install</phase>
      <goals><goal>check-deploy-needed</goal></goals>
    </execution>
  </executions>
</plugin>

I also added the execution on the delivery module, although in practice it's not essential: if you deploy any module, you necessarily need to deploy delivery.

Here are the sources of the example (ZIP, 16.9Kb).

HTML forms the way God intented

2009-03-07T13:37:00.002+01:00

I've been busy with a small web application prototype for the last few days. In my opinion, user interface is very important, even in the early stages of development. After all, users care about what they see, not about the soundness of the internal architecture (although I do). So I tried to make my prototype look very much like the final application, which has, among other things, multiple-column HTML forms.

As we all know, using tables for layout is stupid, so my tabular forms would clearly require some CSS trickery.

I had heard about the 960 grid system a while ago, and decided to give it a try. But I quickly realized that it was not the right tool for the job: 960gs was designed to handle the global layout of the page (header, menu, main body, etc.), not the low-level details of the page's content. (Note that it is not a criticism of 960gs, I was the one who misused it).

After some more research, I came across this excellent online tutorial: Fancy Form Design Using CSS. It is a step-by-step explanation of how to properly design table-free, accessible forms. The key points are: use <fieldset> and <label> tags, and put your fields in an ordered list formatted with CSS styles.

My multiple-column forms build on this solution, separating the clear:left; property in a specific class, that I use to mark only the first field of each line.

This tutorial is also part of a book called "The Art and Science of CSS". However, according to this amazon.com review, a lot its information was already covered in other books.

See for yourself. As for me, the online tutorial was an interesting find.

An application of typesafe heterogeneous containers

2009-02-27T23:09:00.006+01:00

One of our customers is a bit reluctant to use open source frameworks. This means we often find ourselves reinventing the wheel (making sure it's not of the square variety).

Today I was back to implementing an abstract factory to provide DAOs. This was a nice opportunity to put into practice the typesafe heterogeneous container pattern described in item 29 of Effective Java (2nd Edition).

Here is the signature of the factory method (where Dao is a base interface implemented by all DAOs):

public <T extends Dao> T getDao(Class<T> daoInterface);

To request a DAO implementation, the client code passes the interface it must implement. Assuming UserDao defines the contract for user-related data access:

UserDao dao = factory.getDao(UserDao.class);

The beauty of this pattern lies in the type safety provided by the generic type system: if the DAO is stored under the key UserDao.class, the returned type has to be an instance of this type. On the contrary, the program will fail at compile time.

Here is how you would back the factory with a map:

private final Map<Class<? extends Dao>, Dao> daos =
    new HashMap<Class<? extends Dao>, Dao>();

@Override
public <T extends Dao> T getDao(Class<T> daoInterface) {
    return daoInterface.cast(daos.get(daoInterface));
}

private <T extends Dao> void addDao(
    Class<T> daoInterface,
    T implementation)
{
    this.daos.put(daoInterface, implementation);
}

In this case, I don't need to use a thread-safe Map implementation, because the factory is effectively immutable: the map is filled at instantiation time in the constructor, and can never be modified after that (note that addDao is private).

For more details about how the above implementation works and ensures type safety, refer to the book (I'm not going to plagiarize the whole item after all).

Update 16/01/2010: this eventually turned into a more general factory —not just for DAOs—, very similar to Google Guice (I would have used Guice if not for my customer's restrictions).

Wicket / Swarm / Spring Security How-to

2009-02-26T22:22:00.024+01:00

This post explains how to integrate Wicket and Spring Security. It is based on an older article in the Wicket Stuff wiki (not written by me), which covered the now deprecated Acegi. I've rewritten my own prototype, using recent versions of the frameworks. The complete source is available here.

About the frameworks

Apache Wicket is an open-source web application framework for the Java platform. This example uses version 1.4-rc2.

Wicket-Security is an extension of Wicket. It consists in: WASP (Wicket Abstract Security Platform), an API that allows individual Wicket components (pages, links, form widgets, etc.) to do their own authentication and/or authorization checks; and SWARM, the default implementation of WASP. This example uses version 1.4-SNAPSHOT.

Spring Security (formerly Acegi) is a framework to secure enterprise applications developed using the Spring framework. This example uses version 2.0.4, which depends on Spring 2.0.8.

Why use Spring Security AND Swarm?

Assuming your web application uses both Wicket and Spring, their respective security frameworks complement each other nicely:

Swarm provides fine-grained authorizations for your Wicket pages and components; for example, if you want a link to be active only when the user has certain privileges. But if you need to secure non-Wicket web resources (for instance, web services), you can use Spring Security's URL-based filtering.

Swarm has an API to perform authentication, but does not provide authentication modules out of the box. On the other hand, Spring Security has a number of Authentication Providers for common authentication sources: DAO-based, LDAP directory, OpenId... (see chapter III of the online manual).

Spring Security has powerful mechanisms to secure the business side: annotation-based method autorizations and domain object security.

About the example

The example application aims at illustrating various aspects of security, while staying as simple as possible:

it defines two roles (ROLE_ADMIN and ROLE_USER), and two users (admin, who has both roles, and user, who only has ROLE_USER).

authentication is performed by a Spring Security Authentication Provider.

a successful login redirects to a page displaying the user's name and roles. This page also has an administration panel only displayed for, er... administrators.

finally, we use a business service. As it returns a very strategical piece of information, its use is reserved to administrators. The administration panel displays the result of a call to the service.

Getting started: configuring Spring Security

We bootstrap the Wicket application with the standard Maven 2 Archetype, as described in QuickStart.

Our Spring Security configuration is directly inspired by the framework's tutorial sample. Apart from declaring the dependencies in the POM, we add a few elements to WEB-INF/web.xml:

the listener to load the Spring Security context:

<context-param>
  <param-name>contextConfigLocation</param-name>
  <param-value>/WEB-INF/applicationContext-security.xml</param-value>
</context-param>
...
<listener>
  <listener-class>
    org.springframework.web.context.ContextLoaderListener
  </listener-class>
</listener>

and the Spring security filter (the mapping must appear before the Wicket filter's):

<filter>
  <filter-name>springSecurityFilterChain</filter-name>
  <filter-class>
    org.springframework.web.filter.DelegatingFilterProxy
  </filter-class>
</filter>
...
<filter-mapping>
  <filter-name>springSecurityFilterChain</filter-name>
  <url-pattern>/*</url-pattern>
</filter-mapping>

In WEB-INF/applicationContext-security.xml, we configure the Spring context used for security. This is done in a very concise and expressive manner thanks to Spring's namespace configuration feature:

the snippet below will create the beans necessary for the security filter. Note that we tell Spring Security not to secure any URL, since this will be taken care of by Swarm. Of course, if we had to protect web resources other than those served by Wicket, we would need a way to distinguish those URLs (different prefix / extension...) and a finer-grained configuration here.
```
<http auto-config="true">
  <intercept-url pattern="/**"
    access="IS_AUTHENTICATED_ANONYMOUSLY" />
</http>
```

for simplicity's sake, we use the in-memory authentication provider, which lets us define users and roles directly in the configuration:

  <authentication-provider>
    <user-service>
      <user name="admin" password="admin"
        authorities="ROLE_ADMIN, ROLE_USER" />
      <user name="user" password="user" authorities="ROLE_USER" />
    </user-service>
  </authentication-provider>

finally, the following line enables @Secured annotation support:
```
<global-method-security secured-annotations="enabled" />
```

We can immediately take advantage of that last point in our business service (which, by the way, is a dummy implementation):

public class AdminService {
 @Secured("ROLE_ADMIN")
 public int getResult() {
  return 42;
 }
}

Making our Wicket Application a Swarm application

As explained in Getting started with Swarm, the easiest way to do this is by extending SwarmWebApplication:

public class WicketSpringSecApplication extends SwarmWebApplication

This requires implementing three new methods. getHiveKey()and setupHive() are detailed in the page linked above, so I'll let you refer to it. The third one simply specifies our login page:

public Class<LoginPage> getLoginPage() {
  return LoginPage.class;
}

The login page contains a form, which onSubmit() method will call the Wasp API for authentication (more on this later), and redirect to the home page if it is successful.

Connecting Swarm authentication to the Spring Security provider

Time to bridge the gap between the two security frameworks. First of all, a bit of terminology:

with Swarm, you pass a throw-away instance of a subclass ofLoginContext to WaspSession.login(), and get a Subject in return.

with Spring Security, you pass an AbstractAuthenticationToken to an AuthenticationManager, and get an Authentication object in return.

We need a couple of adapter classes to make all this work together. They can be found in the package com.example.wicketspringsec.security.spring:

SpringSecuritySubject is a Subject implementation that can be constructed from an Authentication. In other words, it translates Spring Security's response to something Wasp understands. I won't detail the code here, it's just a matter of converting the list of roles (GrantedAuthoritys to Subjects). If you look at the source right now, you'll see that I introduced a custom NamedSubject interface; that's just an implementation detail, to which I'll come back later.

SpringSecurityLoginContext is a Swarm LoginContext that delegates authentication to Spring Security. The core logic is resumed in these 3 lines of the login() method (I'll explain how we get the authenticationManager instance in just a minute):
```
Authentication authResult = authenticationManager.authenticate(token);
[...]
NamedSubject wicketSubject = new SpringSecuritySubject(authResult);
[...]
return wicketSubject;
```

to be completely generic, we designed SpringSecurityLoginContext to be constructed from an AbstractAuthenticationToken. As our sample application uses form authentication, we define a simple subclass that takes just a login and a password: SpringSecurityUsernamePasswordLoginContext.

So how do we get hold of the AuthenticationManager reference needed to perform authentication? Since we used namespace configuration in WEB-INF/applicationContext-security.xml, the Spring bean's ID not directly visible. Fortunately, the namespace provides us an element to create an alias to the bean:

<authentication-manager alias="authenticationManager" />

To make this bean easily accessible from our custom security classes, we're going to inject it in the Wicket Application object. We start by making this object a Spring bean. To keep this separate from the security configuration, we do this in a new file WEB-INF/applicationContext.xml (which needs to be added to the contextConfigLocation parameter in WEB-INF/web.xml):

<bean id="wicketApplication"
   class="com.example.wicketspringsec.ui.WicketSpringSecApplication"/>

For Wicket to find this bean, we need to go once again to WEB-INF/web.xml
and change the declaration of the Wicket filter to use SpringWebApplicationFactory:

<filter>
  <filter-name>wicket-springsec</filter-name>
  <filter-class>org.apache.wicket.protocol.http.WicketFilter</filter-class>
  <init-param>
    <param-name>applicationFactoryClassName</param-name>
    <param-value>org.apache.wicket.spring.SpringWebApplicationFactory</param-value>
  </init-param>
</filter>

Finally, we inject authenticationManager in wicketApplication(after defining the appropriate getter and setter in the Java class):

<bean id="wicketApplication"
  class="com.example.wicketspringsec.ui.WicketSpringSecApplication">
  <property name="authenticationManager" ref="authenticationManager" />
</bean>

Back to our custom SpringSecurityLoginContext; we can now get the reference of the authentication manager through the application:

AuthenticationManager authenticationManager = WicketSpringSecApplication
    .get().getAuthenticationManager();

One last detail: since we will also use Spring Security for certain authorizations, it must be notified of user login and logoff. We do this in the method SpringSecurityLoginContext.setAuthentication(Authentication).

That's it! There were a lot of details to handle, but our custom Swarm login context is finally ready. It is invoked from the login page (in LoginPage.LoginForm.onSubmit()):

LoginContext context = new SpringSecurityUsernamePasswordLoginContext(
    username.getModelObject(), password.getModelObject());
try {
  ((WaspSession) getSession()).login(context);
  ...

The user interface

As I said earlier, the main page displays information about the user's name and roles. This information is stored in the typical Wicket way: as a property in a custom WebSession implementation. We call this class WicketSpringSecSession, and in our case it must extend WaspSession (don't forget this, or security won't work anymore). In WicketSpringSecApplication, we override newSession to return the new implementation. In SpringSecurityLoginContext, we update this information after a successful login.

Instead of creating a new object to hold the information, I chose to reuse the Subject, adding username support to it (hence the NamedSubject interface). This is probably a bad practice, as it gives programmatic access to the user's roles from the webapp components. A convenient shortcut in the context of this prototype, but a real application would probably need to do better.

The last part to implement is the administration panel, aptly named AdminPanel. The Java class extends SecurePanel, an authorization-aware version of Panel. It just contains a Label that displays the admin service's result (we inject the service in the Wicket application object like we did for authenticationManager). You may be tempted to call the service directly from the label's constructor:

this.add(new Label("adminServiceResult", String.valueOf(WicketSpringSecApplication.get()
    .getAdminService().getResult()));

But that wouldn't work: every time the page is constructed, the panel is also constructed, even if it is not actually rendered. That would mean calling the admin service when a normal users accesses the page, which will throw an exception since the service is secured on the business side. The workaround is to build the label with a separate Model instance, which calls the service in its getObject() method. This method will not be invoked unless the label is actually rendered.

The hive configuration file

The Swarm authorizations are defined in the file WEB-INF/wicket-springsec.hive. We give access to the home page to all users, but filter the administration panel (I had to add line breaks for blogger to display this correctly, refer to the source for the exact syntax):

grant principal org.apache.wicket.security.hive.authorization.SimplePrincipal
"ROLE_USER"
{
  permission ${ComponentPermission} "com.example.wicketspringsec.ui.HomePage", "render";
  permission ${ComponentPermission} "com.example.wicketspringsec.ui.HomePage", "enable";
};

grant principal org.apache.wicket.security.hive.authorization.SimplePrincipal
"ROLE_ADMIN"
{
  permission ${ComponentPermission}
 "com.example.wicketspringsec.ui.HomePage:adminPanel","render";
};

On a sidenote, I used the ${ComponentPermission} alias in the example for conciseness, but it doesn't work for me :-( I have to use the full name org.apache.wicket.security.hive.authorization.permissions.ComponentPermission.

Update 2009/03/12: those interested in securing Wicket applications should also be aware that there is an alternative to Wicket-Security, called wicket-auth-roles. This thread will give you a good overview of the status of the two frameworks. Integrating wicket-auth-roles with Spring Security is covered here.
One compelling feature of wicket-auth-roles is the ability to configure authorizations with Java annotations. I find it somehow more elegant than a centralized configuration file. There is an example here.

Maven modules with independent versions

2008-10-21T14:14:00.008+02:00

Examples of Maven multi-module projects generally assume that all modules share the same version. However, there are scenarios where a set of modules are too independent to have a common version, yet not independent enough to be completely separate projects. This post presents such a scenario, and proposes a mechanism to handle it.

Let's consider an application with a basic structure: a webapp module and a business service module. However, these two modules are not packaged together: the webapp module will be deployed on machine A, and the service module on machine B (the webapp will access the services remotely).

The Maven projects are structured like this (red arrows denote a "depends-on" relationship):
We followed the best practice of decoupling the service API (interfaces, value objects...) from its implementation. We also created a parent POM to hold common information, for instance a dependencyManagement section defining the versions of external dependencies.

In a classical multi-module structure, webapp, service-api and service-impl are modules of parent. A common version is defined in parent's POM and inherited downwards. Internal dependencies are typically expressed with ${project.version}. Everytime we need a new version, the release plugin lets us update all the modules simultaneously.
The problem is that our modules can evolve independently: suppose our next version only consists in graphical changes. webapp is the only impacted module, but we have to perform a global release to version 1.0.1. We deliver our 1.0.1 artifacts to the production guys, but they don't agree: if the business module wasn't impacted, why go through the costly process of a full redeploy? So we decide to only redeploy webapp:1.0.1 in production, and leave service-impl:1.0.0 in place.
But something doesn't feel right: webapp:1.0.1 depends on version 1.0.1 of service-api, while the version service-impl:1.0.0 really implements is 1.0.0. Of course, we know that the API hasn't changed between the two versions, but this is purely based on our functional knowledge of the project. Multiply this by many versions and many modules, and tracking what must be redeployed and what can stay in a previous version becomes complex. A mistake can result in marshalling errors at runtime (hopefully detected during testing).

The next logical solution is to manage each module as an independent project with its own version. Now webapp:1.0.1 can be released independently, but that doesn't solve all of our issues: we still have to make sure it depends on the right version of service-api (the one the service in production will implement). We can track these target versions ourselves, but once again this is a manual process. Nothing in the build safeguards us from accidentally releasing webapp:1.0.1 with a wrong version of service-api.

What we need is a way to express the cohesive set of versions our system will be composed of, and enforce these versions in all internal dependencies; a way to say: "the system currently consists in webapp:1.0.1, service-api:1.0.0 and service-impl:1.0.0. Therefore webapp:1.0.1 must depend on service-api:1.0.0".

A first idea is to declare all the module versions as properties in the parent POM, and use these properties throughout the project. But that doesn't work either: everytime we change one of these properties, we would need to release a new version of the parent POM; to take this new version into account, we'd have to change all the modules. Therefore they wouldn't be independent.

The trick is to let the modules define their versions freely, but to add a new module to check consistency. We make a few structural changes to our multi-module project (blue arrows denote "has-module" relationships):

Don't panic :-) We only added two POMs:

global is just a convenience to make building the whole project easier. It is an aggregate POM that declares all the other projects as modules, but they don't inherit from it. Therefore, it doesn't impose any version or dependency to itself (in fact, the modules don't know anything about global). On the filesystem, we put global's POM in the root directory and use a flat structure for the other modules:

pom.xml (global)
[-] delivery
    \- pom.xml
[-] parent
    \- pom.xml
[+] service-api
[+] service-impl
[+] webapp

delivery is our way of describing the system as a whole. In this POM, we add a dependency to every other module, in the version this module is expected to have in the target environment. To check the consistency of the system, we use a custom mojo, binded to the initialize phase. This mojo analyzes the dependencies between the modules, and checks that they conform to the base versions defined in delivery's POM.
For instance, to describe the previous example, we put the following dependencies in delivery's POM: webapp:1.0.1, service-api:1.0.0, service-impl:1.0.0, and parent:1.0.0. The mojo checks the declared dependencies of webapp:1.0.1 and finds a dependency to service-api; if the version of this dependency is not 1.0.0, the build fails, because our webapp depends on the wrong implementation of the service.

Additionally, we can use the following convention: since delivery's version will evolve for each release, we consider this version to be the "global" version of the system. When any other module is impacted, it will "jump" to the global version, whatever its previous version was. This simplifies identifying the contents of a partial delivery.

Here is an illustrative scenario:

in the first version, all the projects have version 1.0.0. Everything is delivered.
the next version (delivery:1.0.1) consists in graphical changes. webapp needs to be modified, so it is raised to 1.0.1 (we also update the corresponding dependency in delivery's POM). We run the global build and don't get any errors. We do a partial redeploy of all deliverables in version 1.0.1, in that case only webapp.
the next version (delivery:1.0.2) is a bugfix in the service implementation, without any impact to the API. service-impl needs to be modified, so it "jumps" from 1.0.0 to 1.0.2. The global build is successful, therefore we do a partial redeploy of service-impl:1.0.2.
the next version (delivery:1.0.3) is a change to the service, which, this time, impacts the API. The business modules are impacted: service-impl is raised to 1.0.3, and service-api jumps from 1.0.0 to 1.0.3. But we forgot to reflect that change in the webapp. When we run the global build, the mojo fails:

[INFO] [delivery:check-version-consistency {execution: check-version-consistency }] [ERROR] Inconsistency in versions: com.acme:webapp:jar uses com.acme:service-api :jar in version 1.0.0, but in delivery the reference version is defined as 1.0.3
We realize that webapp is also impacted, and therefore must be raised to 1.0.3. This time we'll need a full redeploy of both the business and the presentation modules.

This project structure fulfills our requirements:

modules can be released independently. A module that hasn't been impacted can stay as-is in production.
we have a way to describe our system as a cohesive set of modules in definite versions. A look at delivery's dependencies gives us a clear view of the current situation.
we have a way to enforce these versions throughout the project. The mojo will detect dependencies that don't honor this global contract. This also serves as a dependency analysis to detect the impacts of a change to a module.

Here is a ZIP archive containing the mojo and the example.

Update 2009/03/17: my discussion with Frank gave rise to a new blog post.

Scripting Ant tasks with BSF

2008-10-07T17:17:00.004+02:00

Today I wrote a couple of custom Ant tasks in Groovy, using the Scriptdef optional task. I had never used scripting languages within Ant (nor was I familiar with task development for that matter), but it proved to be really quick and straightforward.

Here is a very simple example: I wanted to configure the load order for a Weblogic deployment, but it is not feasible with BEA's wldeploy task. The workaround is to do it programmatically through the JMX API:

<!-- Sets the load order on a Weblogic deployment -->
<scriptdef name="wlSetLoadOrder" language="groovy" classpath="${weblogic.home}/server/lib/weblogic.jar">
  <!-- The name of the deployment -->
  <attribute name="name" />
  <!-- The load order -->
  <attribute name="loadOrder" />
  <!-- The admin URL of the domain -->
  <attribute name="adminUrl" />
  <!-- A weblogic user with administration rights -->
  <attribute name="user" />
  <!-- The user's password -->
  <attribute name="password" />
  <![CDATA[
    import java.util.Hashtable
    import javax.naming.Context
    import javax.naming.InitialContext
    import weblogic.management.MBeanHome
    import weblogic.management.deploy.DeployerRuntime
    import weblogic.management.deploy.DeploymentData
    import org.apache.tools.ant.BuildException

    try {
      int loadOrder = Integer.parseInt(attributes["loadorder"])
      self.log("Configuring load order " + loadOrder + " for " + attributes["name"])

      // Connect to the MBean server
      def ctx = new InitialContext(new Hashtable(
        (Context.PROVIDER_URL) : attributes["adminurl"],
        (Context.INITIAL_CONTEXT_FACTORY) : "weblogic.jndi.WLInitialContextFactory",
        (Context.SECURITY_PRINCIPAL) : attributes["user"],
        (Context.SECURITY_CREDENTIALS) : attributes["password"]))
      def adminHome = ctx.lookup(MBeanHome.ADMIN_JNDI_NAME)

      // Lookup the deployment's ApplicationMBean
      def appMBean = adminHome.getMBean(attributes["name"], "Application")

      // Set the JMX attribute
      appMBean.setLoadOrder(loadOrder)
    } catch (NumberFormatException e) {
      throw new BuildException("Invalid value for loadOrder: " + attributes["loadorder"])
    }
  ]]>
</scriptdef>

This new task is unsurprisingly invoked like any regular Ant task:

<wlSetLoadOrder name="myKillerApp" loadorder="200"
  adminurl="t3://localhost:7001" user="admin" password="weblogic"
/>

Notes:

this works for Weblogic 8.1. I have not tried with later versions, but it can probably be easily adapted.
you will need to put the appropriate dependencies in Ant's classpath, namely: Groovy, Commons Logging and BSH. See this page.

ZIP gotcha: you could be a few milliseconds from build failure

2008-08-06T17:19:00.002+02:00

In Apache Ant's documentation, you will find the following notice in the description of the Zip task: Please note that ZIP files store file modification times with a granularity of two seconds. If a file is less than two seconds newer than the entry in the archive, Ant will not consider it newer.

I had never really paid attention to this, until I actually ran into a situation where it creates a problem: in a build script, I need to extract a deployment descriptor from a generated EAR, modify it, and rezip it in the EAR (ugly, but I really don't have any control over the generation of the EAR).

So what had to happen happened: the interval between the generation of the EAR and the time I make my modification is less than two seconds. But, in the true spirit of Murphy's law, it is not always less: on my machine and during the nightly build, it takes slighty longer, which explains why I hadn't seen the problem in the first place. But when the build is run manually on the integration server, the interval suddendly becomes slightly shorter than two seconds (probably due to a higher process priority), and the descriptor is not updated.

I added an artificial two-second pause to make sure that the modified file will always be "newer enough".

By the way, this is not specific to Ant. From Wikipedia: The FAT filesystem of DOS only has a timestamp resolution of two seconds; ZIP file records mimic this. As a result, the built-in timestamp resolution of files in a ZIP archive is only two seconds, though extra fields can be used to store more accurate timestamps.

Maven 2, Xalan, and the evil 2.2D11 / 2.4.1 version

2008-03-07T18:43:00.006+01:00

A few days ago, I ran into one of those frustrating situations with Maven 2: a multi-module build where each sub-project builds perfectly individually, but building from the top-level POM fails.

In that particular case, Maven was trying to download the Spring JARs in a mysterious "2.2D11" version, which was not at all the one declared in my POM:

Missing: ---------- 1) org.springframework:spring-beans:jar:2.2.D11
Try downloading the file manually from the project website.

Fortunately, this was not the first time I ran into phantom dependency versions: there used to be a similar bug with the m2eclipse plugin, where "2.4.1" versions would sometimes inexplicably pop up in Eclipse. But, what is even more fortunate, someone pintpointed the problem:

- if Java 1.4.2 is the first entry in the %PATH% on the machine, this java version will be used to run eclipse - Java 1.4.2 contains some Apache XML parser that defines, among others, a silly environment variable "version" with value "2.4.1" - m2plugin tells Maven (if I understand correctly) via Maven Embedder to evaluate dependencies - Maven Embedder has a bug (or "feature") by which variables like {project.version} are overridden by corresponding system variables (i.e. "version"). - as a result, Maven is looking for version "2.4.1" of such dependencies instead of the desired value of {project.version}, and fails to build

"Some Apache XML parser" is actually Xalan-J. And it turns out that my project uses a custom mojo that outputs an XML file using the javax.xml.transform.Transformer interface, which is precisely what Xalan implements:

   TransformerFactory transformerFactory = TransformerFactory.newInstance();
   Transformer transformer = transformerFactory.newTransformer();
   DOMSource source = new DOMSource(document);

   StreamResult result = new StreamResult(dest);
   transformer.transform(source, result);

I figured that if a system property called version created the problem, removing it could be the solution. Therefore, I added the following line just after the preceding code:

   System.getProperties().remove("version");

And that did the trick :-)

So the bottom line is: Maven + Xalan = beware of the evil version property...

Environment:

Apache Maven 2.0.8
JDK 1.4.1_05 from BEA Weblogic 8.1 SP2 (I guess that explains the different Xalan version)
Windows XP

Writing Maven POMs in Groovy

2008-02-24T18:11:00.010+01:00

2010/06/04 — a word of caution as I see people are still finding this old post: this is an experiment I did two years ago, it has nothing to do with the new 'polyglot' features of Maven 3.

I recently came across Brett Porter's post about condensing Maven's POMs using XML attributes. Simplifying Maven's syntax sounds like a great idea, since like a lot of people I'm a bit tired of typing all those XML tags ;-)

It also turns out that I want to learn Groovy better these days, so I keep a Groovy console open on my computer for my little experiments.

Although unrelated, these two thoughts somehow collided in my brain, resulting in the idea of writing Maven POMs using Groovy rather than XML. I started by putting the maven-model JAR in the Groovy console's classpath and fiddling with its classes. I quickly realized that I liked this syntax:

project.dependencies =
   [ new Dependency(groupId: 'junit', artifactId: 'junit',
                    version: '3.8.1', scope: 'test') ]

A lot more than this one:

  <dependencies>
   <dependency>
     <groupId>junit</groupId>
     <artifactId>junit</artifactId>
     <version>3.8.1</version>
     <scope>test</scope>
   </dependency>
 </dependencies>

Therefore, I decided to go a bit further and integrate this to the Maven runtime. My goal was to be able to run commands like mvn package with a pom.groovy file instead of a pom.xml at the root of my project. I dived into the Maven sources and came up with the following changes (NB: the links below are to the original source files in Subversion, for the modified files see the bottom of this post):

in DefaultMaven.java, I modified the methods that collect the pom.xml and release-pom.xml files, to support the new pom.groovy file (which name I also declared as a constant in the interface Maven.java). This new file has the least priority: if any of the XML POMs exist in the same directory, it takes precedence over the Groovy POM.

in DefaultMavenProjectBuilder.java, I added the evaluation of the POM with a GroovyShell if the file name ends with .groovy. This also required adding a dependency to Groovy in maven-project's pom.xml.

then I needed to reflect these changes to my local Maven installation. Here I must confess that I lamely ran mvn assembly:assembly -DdescriptorId=jar-with-dependencies in the two projects I had modified, and manually merged the resulting JARs with maven-2.0.8-uber.jar. This results in a 5 Mb maven-2.0.8-uber.jar containing Groovy and its transitive dependencies. Probably not The Right Way^TM to do this, but sufficient for a first test.

Speaking of testing, I started with a very simple POM generated from the standard project archetype:

<project>
 <modelVersion>4.0.0</modelVersion>
 <groupId>test</groupId>
 <artifactId>test</artifactId>
 <packaging>jar</packaging>
 <version>1.0-SNAPSHOT</version>
 <name>test</name>
 <dependencies>
   <dependency>
     <groupId>junit</groupId>
     <artifactId>junit</artifactId>
     <version>3.8.1</version>
     <scope>test</scope>
   </dependency>
 </dependencies>
</project>

And here is the Groovy equivalent in pom.groovy (I decided that the contract for the Groovy script was to expose a variable named project):

import org.apache.maven.model.*

project = new Model(
 groupId: 'test',
 artifactId: 'test',
 packaging: 'jar',
 version: '1.0-SNAPSHOT',
 name: 'test',
 modelVersion: '4.0.0')

project.dependencies =
   [ new Dependency(groupId: 'junit', artifactId: 'junit',
                    version: '3.8.1', scope: 'test') ]

The moment of truth had arrived... I ran mvn clean package from the command line, held my breath and then shouted "Tadaa!!" as the build executed and the BUILD SUCCESSFUL message appeared.

I plan more tests to see how this adapts to more complex POMs. Note that while I chose Groovy, you could as well try this at home with your favorite overhyped, buzzword-compliant dynamic language ;-)

Here are links to Subversion diffs of the two modified projects: maven-core-2.0.8.diff, maven-project-2.0.8.diff. Or if you prefer, here is a ZIP containing all the modified files.

Note: I later found out that Sam Pullara had already had a similar idea about 2 months ago. However, my approach being slightly different (he has a Groovy script generate the pom.xml and invoke Maven, while I have the Maven core execute the script), I've decided to keep this posted anyway.

What protected really means

2007-09-18T21:22:00.001+02:00

Section 3.5 of The Java™ Programming Language, 4th Edition defines protected as:
"beyond being accessible within the class itself and to code within the same package [...], a protected member can also be accessed from a class through object references that are of at least the same type as the class."

The rationale is given a bit further:
"Suppose that one subclass, as part of its expanded contract, places constraints on the values of protected members of the superclass. If a different subclass could access the protected members of objects of the first subclass then it could manipulate them in a way that would break the first subclass's contract - and this should not be permissible."

The text gives examples, but does not explain precisely how a subclass could try to break the contract of another. Here is a very simple example I put together to understand this.

But first, let's remember that in Java, visibility is per-class, not per-object. For example, an object can access the private members of other instances of the same class. In other words, the following code compiles and works perfectly:


public class TestPrivate {
  
  static class MyInt {
    private int value;

    public MyInt(int value) {this.value = value;}
    
    public int getValue() {return value;}
    
    public void resetValue(MyInt other) {other.value = 0;}
  }

  public static void main(String[] args) {
    MyInt resetter = new MyInt(0);
    MyInt other = new MyInt(42);

    resetter.resetValue(other);
    System.out.println(other.getValue());
  }
}

It may be obvious for you, but I've found that a number of people will tell you intuitively that the code doesn't compile (and, to be honest, I'm not sure of the answer I would have given a few weeks ago). Try to run a survey among your fellow programmers, I'd be interested to hear about the results.

Going back to the protected example, here is the parent class, which encapsulates an int (I told you it would be a very simple example):


package test.protected1;

public class MyInt {
  protected int value;
  
  public MyInt(int value) {
    this.value = value;
  }
  
  public String toString() {
    return String.valueOf(value);
  }
}

And here is the first subclass, which expands the contract by enforcing positivity:


package test.protected1;

public final class MyPositiveInt extends MyInt {
  public MyPositiveInt(int value) {
    super(value);
    if (value < 0) {
      throw new IllegalArgumentException("value must be positive");
    }
  }
}

MyPositiveInt enforces its contract strictly: there is no way to modify the int once the instance has been constructed, and furthermore the class is final, preventing attacks by subclasses.

And now here is the would-be attacker subclass. Note that it is defined in a separate package (see below):


package test.protected2;

import test.protected1.MyInt;
import test.protected1.MyPositiveInt;

public class MyPositiveIntBreaker extends MyInt {

  public MyPositiveIntBreaker(int value) {
    super(value);
  }

  public static void breakPositiveInt(MyPositiveInt toBreak) {
    MyInt broken = (MyInt) toBreak;
    broken.value = -1;
  }
}

The idea is: 1) casting the MyPositiveInt to a MyInt is perfectly valid; 2) MyPositiveBreaker extends MyInt, so the protected field value is accessible; 3) visibility is per-class, to there should be no problem accessing value of another MyInt instance.

If this worked, we would be able to construct a MyPositiveInt that breaks its contract, without the original class being able to do anything about it. That's where the rule cited above comes into play: the third affirmation is actually false, and accessing broken.value gives a compile-time error. This is because we access the protected field from class MyPositiveIntBreaker through a reference of type MyInt, and MyInt is not "of at least the same type as" MyPositiveIntBreaker, it's the other way around.

Finally, protected also means package-private, so if you moved MyPositiveIntBreaker into package test.protected1 the "attack" would work. However, it is generally considered that classes in the same package are trustworthy; if you don't fully control the class path, you can use security permissions (old link but still seems valid) to prevent new classes from being added to your package.

As a conclusion, this example is of course very simplistic. For instance, testing value after calling super(value) in MyPositiveInt(int) feels awkward: in real life it would probably be better to make the variable private and define protected accessors. But the objective was to illustrate the reasons that motivate the "real" meaning of protected; a real-life example would probably be a bit more complex.

A development environment installation tool?

2007-09-10T20:11:00.001+02:00

It had always struck me that the Tomcat documentation didn't explain how to use CATALINA_BASE to run multiple instances from a single installation. I recently realized that this is detailed in the RUNNING.txt file that comes in the base directory of the distribution :-o

I like to keep a clean installation of a product and have separate directories for each project that uses it. I'd like to generalize this idea to create a development environment installation tool: installing everything from scratch is always time-consuming for new developers; what we did for some projects at my company was to create a giant archive containing everything you need to build and run a specific project (JDK, application server, build tool, IDE, workspace, etc.). All the developer has to do is get the archive, unzip it, and adapt the configuration. With a few assumptions (for instance, a standardized installation path), you can keep the personalization phase to the strict minimum (changing the SCM user and updating the sources to the current version).

The downside to this approach is that each project brings its own installation of potentially similar products. In the end, you would end up with 4 installations of JDK version X or Eclipse version Y on your hard drive. That's where I see a parallel with Maven's dependency mechanism - instead of handling your third-party JARs yourself, you declare them, and Maven handles all the necessary downloading and copying.

I believe a similar repository mechanism could be used for development tools:

The central repository would be on a server somewhere in your company and host zipped product installations.
The local repository on your machine would contain a clean unzipped version of the products. No project would be allowed to modify these files.
Each project would independently provide (for example from an SCM repository):

an "environment descriptor" listing dependencies to the required products.
project-specific product configuration, for instance a Tomcat base, a Weblogic domain, an Eclipse workspace...
launch scripts (which would rely on a standardized product root and project directory).
and the project sources of course.

The environment installation process would read the descriptor and update the local repository, by unzipping the products that are not already installed.

I think I already know how to handle a couple of tools could in this way: the Java SDK, Maven, ANT, Tomcat, Weblogic...

Eclipse is a bit trickier though, because of the plugins: different projects may need the same version of the IDE, but with different and potentially conflicting plugins. It would be nice to say "I need Eclipse version x, and plugin A, and plugin B", but to keep everything in separate directories. I think this is feasible as well: thanks to the Eclipse manual and a post from Colin Sampaleanu's blog, I've set up an example with an Eclipse 3.2 installation in a directory, the Subclipse plugin in another, and a third directory that stores external configuration and workspace locations. I can launch a workbench with Subclipse installed, without affecting the first two directories.

So how about a development environment installation tool? Some may argue that an IDE should already provide everything, but I think that in practice you often need a few additional tools on the side. I'm not aware of any tool providing the features I described, but I might as well be reinventing the wheel (or worse, the square wheel)... comments are welcome.

SLF4J and Logback

2007-09-03T17:31:00.001+02:00

SLF4J and Logback are alternatives to the popular Jakarta Commons-Logging (JCL) and Log4J logging frameworks. Both projects are led by Log4J founder Ceki Gülcü.

SLF4J's main goal is to avoid the class loader problems JCL suffered from.While JCL uses a dynamic mechanism to discover which logging frameworks are available at runtime, SLF4J is bound to its underlying implementation statically, at compile-time. To achieve this, you place two JARs in the classpath: the SLF4J API itself, and one "binding" JAR corresponding to the chosen implementation.

SLF4J is being adopted by a growing number of open-source projects (see a non-exhaustive list on the project's main page). For situations where it has to coexist with JCL, migration solutions are available. If you use Maven, you may also have problems with dependencies that drag along JCL transitively ; to solve this, see this hack.

Logback is the official successor to Log4J (Log4J currently has 3 branches : 1.2 - stable with limited evolution, 1.3 - discontinued, and 2.0 - experimental). Here is a list of evolutions and enhancements I noted while reading the Logback manual:

the basic syntax, as illustrated in the Hello World example, should look quite familiar to JCL users. Note that this example actually uses SLF4J classes - the Logback manual promotes using SLF4J from the start.
parameterized logging: logging messages now accept parameters. For instance, we can use logger.debug("The entry is {}", entry) instead of logger.debug("The entry is " + entry). The advantage is that the first version will not try to convert entry to a String if logging is disabled (this is equivalent to using the isDebugEnabled method with Log4J), thus improving performance (by a factor of at least 30 according to the manual). The framework provides similar methods for two and a variable number of parameters (the latter using Object[], probably to preserve 1.4 compatibility).
performance has apparently been a strong focus. Apart from the preceding item, the manual also mentions improvements in the way the framework determines whether to log or not (Log4J walks the logger hierarchy - see the getEffectiveLevel method in the Category class - while Logback propagates the level to child loggers if it changes), and in formatting and writing the messages.
configuration is now performed with an integrated framework called Joran. On a basic user level, this doesn't change much, except that configuration is now stored in an XML file, which supports substitution properties and file inclusion. Joran can also be used independently.
filters provide a way to, well, filter logging messages. They can be used to refine log selection at the appender level, or to control logging event creation at the more general context level (via "TurboFilters"). The manual provides the following examples:

refining an appender for a specific message level (LevelFilter) or a level threshold (ThresholdFilter).
refining an appender by checking messages against java expressions specified in the configuration file (these are compiled on-the-fly with Janino, an optional dependency).
defining a TurboFilter to refine logging events for a specific Marker. Markers are objects used to enrich log statements. All the logging methods (logger.debug, logger.info, etc.) have a version taking a Marker as a first argument.

the MDC (short for Mapped Diagnostic Context) is a mechanism to deal with multi-threaded applications. It is basically a ThreadLocal map-like object, where each thread can put specific key-value pairs. These properties can then be referenced in the layout pattern with the %X{...} notation, or used by a filter.
the Context Selector is a JNDI-based mechanism to allow several web applications all running on the same server to each have their own configuration file.
logback-access is a specialized module that provides HTTP-access log functionality with Tomcat and Jetty.

SLF4J requires JDK 1.3 or above.
Logback requires JDK 5.0. The Maven POM is configured to generate retrotranslated JARs for JDK 1.4, however I haven't found them in the central repository.

Whether or not to switch to Logback and/or SLF4J is a personal choice. However, Logback seems to be the designated successor to Log4J, which apparently won't evolve a lot from now on. There is also a lot of debate about whether to use yet another abstraction layer like SLF4J or JCL (why not stick to plain 1.4 logging after all?). If you are developing a library, it is probably a good idea not to lock your clients into a specific implementation. For an application, it boils down to the question: do I want to keep the possibility of changing my logging implementation?

Remember the following migration options:

if you are using SLF4J but depend on libraries that use JCL: jcl104-over-slf4j.jar (drop-in replacement of JCL, redirects JCL calls to SLF4J), and commons-logging version 99-0-does-not-exist to avoid dragging the transitive Maven dependencies.
if you are using SLF4J but depend on libraries that use Log4J directly: log4j-over-slf4j.jar (drop-in replacement of Log4J, redirects Log4J calls to SLF4J).
if you are using SLF4J but realize your environment mandates JCL: slf4j-jcl.jar (JCL binding for SLF4J, redirects SLF4J calls to JCL).

Maven repository browsing

2007-08-30T12:17:00.001+02:00

mvnrepository.com is a web interface to Maven 2's central repository.

It will allow you to quickly find the ultimate version of an artifact and view interesting details (dependencies, JAR / POM download, copy/paste ready declaration, etc.). There is also an RSS feed of all newly deployed artifacts, which number of posts seems to average a reasonable 20 per day.

I think I had seen similar sites in the past but never settled for one (instead using m2eclipse or worse, browsing ibiblio.org directly!). This one seems simple and straightforward enough.

Compile-time checks with AspectJ - part 2

2007-08-24T18:28:00.001+02:00

As announced in my previous post, here is the configuration to integrate AspectJ compile-time checks to a Maven build. It is really straightforward, thanks to the AspectJ Maven plugin. The following code is added in the pom.xml file of the aop-test project:


<project xmlns=[...]>
   [...]
   <profiles>
     <profile>
       <id>dev-env</id>
       <activation>
         <activeByDefault>true</activeByDefault>
       </activation>
       <dependencies>
         <dependency>
           <groupId>aspectj</groupId>
           <artifactId>aspectjrt</artifactId>
           <version>1.5.2a</version>
         </dependency>
       </dependencies>
       <build>
         <plugins>
           <plugin>
             <groupId>org.codehaus.mojo</groupId>
             <artifactId>aspectj-maven-plugin</artifactId>
             <executions>
               <execution>
                 <goals>
                   <goal>compile</goal>
                 </goals>
               </execution>
             </executions>
           </plugin>
         </plugins>
       </build>
     </profile>
     <profile>
       <id>dev-prod</id>
       <activation>
         <property>
           <name>env</name>
           <value>prod</value>
         </property>
       </activation>
     </profile>
   </profiles>
 </project>

You'll notice the use of Maven profiles to distinguish between development and production environments: the focus of this post being on contract enforcement at development time, I wanted to handle the case where AspectJ would only be used for that purpose; we might want to weave the compile-time aspects only during development, and not leak the aspectjrt dependency to the production environment (for instance, this might be an already existing project and our customer might not want to add a new dependency).
Of course, if AspectJ is used for anything else than these compile-time checks, profiles are not necessary: the <dependency> and <build> tags can be added directly under the <project> element, and the aspects will be compiled and weaved in any case.

Going back to project aop-test, we can now try the two profiles:

launching mvn clean compile -Denv=prod triggers the prod-env profile, which bypasses the compile-time checks;

launching mvn clean compile triggers the default dev-env profile and compile-time checks are activated. The error is raised and the message even points us to the corresponding aspect:


[INFO] [aspectj:compile {execution: default}]
[ERROR] substring can only be called from package com.acme.util
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Compiler errors :
error at myPostalCode.substring(0, 2);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
D:\projects\tests\src\aop-test\src\main\java\com\acme\business\ExampleBusiness.j
ava:19:0::0 substring can only be called from package com.acme.util
        see also: D:\projects\tests\src\aop-test\src\main\aspect\com\acme\aspect
s\CompileChecks.aj:21::0

However, I ran into a strange bug: running mvn compile a second time does not invoke the AspectJ compiler, and the project compiles without any error!

Running Maven with the -X option, I noticed the following message: [DEBUG] No modifications found skipping aspectJ compile. I localized it in the online sources of the Maven plugin: at line 459 of AbstractAjcCompiler.java, we can see in the isBuildNeeded() method that the AspectJ compiler is only invoked when one of the following conditions is met:

we are running a new build from a clean project;
the AspectJ configuration has changed since the last build (it is stored in target/classes/builddef.lst);
the project sources have changed.

Running the compile phase two times in a row meets neither of these conditions, which explains the bug. We can suppose that this would not happen without any source modification in the meantime; however, leaving open a way to bypass the compile-time checks still bugs me. Possible solutions include:

running mvn clean compile each and every time;
adding a snippet of ANT code in the POM to manually delete builddef.lst;
filing a bug with the aspectj-maven-plugin team, to request a switch to bypass the isBuildNeeded() method.

Compile-time checks with AspectJ

2007-08-14T15:22:00.001+02:00

While reading a post on The Server Labs Blog the other day, this paragraph immediately caught my attention : using the AspectJ DSL to specify domain pattern rules that should be checked at compile time. [...] This technique can be used whenever you have large projects and there are developers with less experience, highlighting using compile-time warnings that they are violating the established architectural patterns.

This reminded me of a discussion I had with a colleague a few months ago : he complained that string manipulation methods were spread all over the business code. A single problem (for instance, extracting part of a string, converting to another type, etc.) would be handled in 30 different ways in 30 different methods. He was looking for a way to force developers to use utility classes, instead of using a different home-grown solution in each business method.

I had never worked with AOP before (shame on me!), so I decided to give it a shot.
Disclaimer : there is nothing particularly innovative in this post (the contract enforcement example is explained in the first chapter of the AspectJ Programming Guide); rather, my objective was to provide a complete running example.

We start by creating a simple project with Maven:
mvn archetype:create -DgroupId=com.acme -DartifactId=aop-test

Then we import it in Eclipse and delete the default generated classes (App.java and TestApp.java) as they won't be used.

We add a utility class com.acme.util.PostalCodeUtils, which handles the arduous task of extracting the province code from a spanish postal code:


/**
 * Utility class to manipulate postal codes.
 */
public class PostalCodeUtils {
 /**
  * Extract the province code from a full postal code.
  * 
  * @param postalCode
  *            the full postal code.
  * @return the province code.
  */
 public static String getProvinceCode(String postalCode) {
  return postalCode.substring(0, 2);
 }
}

Then we add a bogus business component called com.acme.business.ExampleBusiness. Our objective will be to prevent the business layer from calling the substring method directly (if you don't like this example, think of any other situation where you want to prevent an application layer to call a given method).


/**
 * Example business component.
 */
public class ExampleBusiness {
 /**
  * Do something.
  */
 public void doSomething() {
  String myPostalCode = "28015";
  
  // The "right" way: using utility method
  PostalCodeUtils.getProvinceCode(myPostalCode);
  
  // The "wrong" way: direct call to substring
  myPostalCode.substring(0, 2);
 }
}

At this point we need to install the AspectJ Development Tools for Eclipse. This is done with Eclipse's software update, you can find the update site URL corresponding to your IDE version at http://www.eclipse.org/ajdt/downloads.

After installing and restarting the workbench, we need to convert the Eclipse project to be able to use AJDT : right-click on the project / AspectJ Tools / Convert to AspectJ Project.

We can now create a new source directory src/main/aspect and add an aspect : File / New / Other / AspectJ / Aspect. We choose the name com.acme.aspects.CompileChecks:


/**
 * Enforces project-specific development contracts.
 */
public aspect CompileChecks {
 /**
  * Matches calls to the String.substring method.
  */
 pointcut callSubstring(): call(String String.substring(int,int));

 /**
  * Matches code from the utility package.
  */
 pointcut substringAllowed(): withincode(* com.acme.util..*.*(..));

 /**
  * Raises a compile-time error if substring is called outside the utility
  * package.
  */
 declare error: callSubstring() && !substringAllowed(): "substring can only be called
from package com.acme.util";
}

The code is pretty much self-explanatory (check the AspectJ documentation if you're not familiar with pointcut definitions). The declare error instruction tells AspectJ to raise an error at compile time.

As soon as Eclipse recompiles the project, the error pops out in the editor:

That's it! Although the example itself is simplistic, I think it gives a pretty good overview of how development contracts can be enforced at compile-time. I'll try to cover Maven integration in a next post.

EJB Dependency Graphing

2007-08-13T09:35:00.002+02:00

One of the goals of SOA is to promote modularity of services. One of the projects I work on contains a bunch of EJBs that have been so far deployed as part of a monolithic EAR. Due to the lack of modularity in the development environment (all classes are configured as a single Eclipse project - ouch), the dependencies between these EJBs were not really clear. In order to partition the business logic into "silos", the first task was to assess the dependencies between the different components. Being a lazy person, I put together a set of tools and scripts to automatize this.

The first step is an analysis of the Java dependencies between all the classes of the project, performed by the excellent Dependency Finder tool. The result of the analysis is stored in an XML file.

Now, the expected granularity is EJB-level, not class-level, so two additional steps are required:

"merge" the EJB classes and interfaces into a single unit. For instance, we want to transform the proposition "the implementation class of EJB Xxx has a dependency on the home interface of EJB Yyy" into : "EJB Xxx has a dependency on EJB Yyy". To do this, I merge all the ejb-jar.xml files (which contain information about which classes correspond to which EJB) and parse the result with an AWK script. This script creates a SED replacement file, which is in turn applied to the output of dependency finder to replace the java classes with the EJB name (a special naming convention is used to make sure there is no collision with existing classes). These replacements may generate duplicate dependencies in the file, but thankfully Dependency Finder doesn't complain about this in the next steps.
filter the results to keep only EJBs and DAOs. Once again, Dependency Finder does the job, however, there is a subtility : I want to keep regular classes that might be on a dependency path between two components. For instance, if EJB Xxx depends on class Aaa that depends on EJB Yyy, I want to keep class Aaa in the results. This is achieved by having Dependency Finder compute closures, which are essentially all types that are reachable by following the dependencies from a given set of types. So I compute all classes that are accessible by following outbound dependencies from the EJB and DAOs, then the same with inbound dependencies. The intersection of the two sets (performed with the comm UNIX command) corresponds to the set of classes that must be retained. These are included in the filtering process.

This pretty much sums up the extraction process. The information is then loaded into a MySQL database. The schema is straightforward: one table for components and one for dependencies. We also added tables to express membership to a silo (this information is hand-loaded and used in the reports).

Finally, a set of scripts extract information from this database and generate comprehensive reports. In particular, I used GraphViz to generate dependency graphs. This tool has a subgraph feature that allows me to put my silos in nice little boxes. The generated graphs can be navigated with ZGRViewer, or printed to decorate office walls (I found a nice utility named poster to scale PostScript images on any number of pages).

Here is an example of output (the names have been blurred to preserve the anonimity of the protagonists):

The blue houses are facade SLSBs, the red ellipses ellipses Entity Beans, the orange hexagons DAOs, the grey parallelograms regular classes, and the green boxes database tables.

These reports help us refine silo definitions and identify dependencies that should be refactored.

Change POST data in Firefox

2007-08-06T19:14:00.001+02:00

Tamper Data is a Firefox extension to view and modify HTTP headers and POST data, and track and time requests. When you turn it on, each time the browser issues a POST, a window pops-up and allows you to change the data before the request is actually sent.

Didn't solve the problem I had today (apparently a very badly handled exception on the server side), but it could be quite handy.

BEA Workshop touchy about service packs

2007-08-01T10:03:00.001+02:00

I had to deploy a web application developped with workshop yesterday. Upon accessing it, I kept getting the same JSP error for the netui-tag-* tag library declarations:No tag library could be found with this URI.

After checking and re-checking my configuration and the spec versions, I finally found a lead: the web application had been built with Weblogic Platform 8.1 SP4, while I was trying to deploy to 8.1 SP5. I installed SP5 and updated my webapp's libraries by copying them from a newly created project. Then I rebuilt with SP5 and redeployed. That solved the problem.

That's it, end of story. Not very fascinating stuff I admit, but since the initial error message wasn't very explicit, I think it was worth mentionning.