Let's consider an application with a basic structure: a webapp module and a business service module. However, these two modules are not packaged together: the webapp module will be deployed on machine A, and the service module on machine B (the webapp will access the services remotely).
The Maven projects are structured like this (red arrows denote a "depends-on" relationship):
We followed the best practice of decoupling the service API (interfaces, value objects...) from its implementation. We also created a parent POM to hold common information, for instance a dependencyManagement section defining the versions of external dependencies.In a classical multi-module structure, webapp, service-api and service-impl are modules of parent. A common version is defined in parent's POM and inherited downwards. Internal dependencies are typically expressed with ${project.version}. Everytime we need a new version, the release plugin lets us update all the modules simultaneously.
The problem is that our modules can evolve independently: suppose our next version only consists in graphical changes. webapp is the only impacted module, but we have to perform a global release to version 1.0.1. We deliver our 1.0.1 artifacts to the production guys, but they don't agree: if the business module wasn't impacted, why go through the costly process of a full redeploy? So we decide to only redeploy webapp:1.0.1 in production, and leave service-impl:1.0.0 in place.
But something doesn't feel right: webapp:1.0.1 depends on version 1.0.1 of service-api, while the version service-impl:1.0.0 really implements is 1.0.0. Of course, we know that the API hasn't changed between the two versions, but this is purely based on our functional knowledge of the project. Multiply this by many versions and many modules, and tracking what must be redeployed and what can stay in a previous version becomes complex. A mistake can result in marshalling errors at runtime (hopefully detected during testing).
The next logical solution is to manage each module as an independent project with its own version. Now webapp:1.0.1 can be released independently, but that doesn't solve all of our issues: we still have to make sure it depends on the right version of service-api (the one the service in production will implement). We can track these target versions ourselves, but once again this is a manual process. Nothing in the build safeguards us from accidentally releasing webapp:1.0.1 with a wrong version of service-api.
What we need is a way to express the cohesive set of versions our system will be composed of, and enforce these versions in all internal dependencies; a way to say: "the system currently consists in webapp:1.0.1, service-api:1.0.0 and service-impl:1.0.0. Therefore webapp:1.0.1 must depend on service-api:1.0.0".
A first idea is to declare all the module versions as properties in the parent POM, and use these properties throughout the project. But that doesn't work either: everytime we change one of these properties, we would need to release a new version of the parent POM; to take this new version into account, we'd have to change all the modules. Therefore they wouldn't be independent.
The trick is to let the modules define their versions freely, but to add a new module to check consistency. We make a few structural changes to our multi-module project (blue arrows denote "has-module" relationships):
Don't panic :-) We only added two POMs:
global is just a convenience to make building the whole project easier. It is an aggregate POM that declares all the other projects as modules, but they don't inherit from it. Therefore, it doesn't impose any version or dependency to itself (in fact, the modules don't know anything about global). On the filesystem, we put global's POM in the root directory and use a flat structure for the other modules:
pom.xml (global)delivery is our way of describing the system as a whole. In this POM, we add a dependency to every other module, in the version this module is expected to have in the target environment. To check the consistency of the system, we use a custom mojo, binded to the initialize phase. This mojo analyzes the dependencies between the modules, and checks that they conform to the base versions defined in delivery's POM.
[-] delivery
\- pom.xml
[-] parent
\- pom.xml
[+] service-api
[+] service-impl
[+] webapp
For instance, to describe the previous example, we put the following dependencies in delivery's POM: webapp:1.0.1, service-api:1.0.0, service-impl:1.0.0, and parent:1.0.0. The mojo checks the declared dependencies of webapp:1.0.1 and finds a dependency to service-api; if the version of this dependency is not 1.0.0, the build fails, because our webapp depends on the wrong implementation of the service.
Additionally, we can use the following convention: since delivery's version will evolve for each release, we consider this version to be the "global" version of the system. When any other module is impacted, it will "jump" to the global version, whatever its previous version was. This simplifies identifying the contents of a partial delivery.
Here is an illustrative scenario:
- in the first version, all the projects have version 1.0.0. Everything is delivered.
- the next version (delivery:1.0.1) consists in graphical changes. webapp needs to be modified, so it is raised to 1.0.1 (we also update the corresponding dependency in delivery's POM). We run the global build and don't get any errors. We do a partial redeploy of all deliverables in version 1.0.1, in that case only webapp.
- the next version (delivery:1.0.2) is a bugfix in the service implementation, without any impact to the API. service-impl needs to be modified, so it "jumps" from 1.0.0 to 1.0.2. The global build is successful, therefore we do a partial redeploy of service-impl:1.0.2.
- the next version (delivery:1.0.3) is a change to the service, which, this time, impacts the API. The business modules are impacted: service-impl is raised to 1.0.3, and service-api jumps from 1.0.0 to 1.0.3. But we forgot to reflect that change in the webapp. When we run the global build, the mojo fails:
}]
[ERROR] Inconsistency in versions: com.acme:webapp:jar uses com.acme:service-api
:jar in version 1.0.0, but in delivery the reference version is defined as 1.0.3
We realize that webapp is also impacted, and therefore must be raised to 1.0.3. This time we'll need a full redeploy of both the business and the presentation modules.
This project structure fulfills our requirements:
- modules can be released independently. A module that hasn't been impacted can stay as-is in production.
- we have a way to describe our system as a cohesive set of modules in definite versions. A look at delivery's dependencies gives us a clear view of the current situation.
- we have a way to enforce these versions throughout the project. The mojo will detect dependencies that don't honor this global contract. This also serves as a dependency analysis to detect the impacts of a change to a module.
Update 2009/03/17: my discussion with Frank gave rise to a new blog post.
Posts
8 comments:
We are in the exact situation as you (needing to release child modules independently). Your post was extremely helpful on providing a scheme that might also work for us.
One question: when you do your multi-module build from global, do you do the deploy phase? Or, do you do you just build locally (i.e. package or install)? If you do the deploy phase don't you get build errors because an un-revved module will already be in the Maven repository.
For example, in your second bullet under Here is an illustrative scenario, webapp changes but not the service-api or service-impl. If you did a 'mvn deploy' from global, wouldn't the build fail as it would try to re-deploy service-api-1.0.0.jar. That version of the JAR would already exist in the repository, so you'd get an error (assuming you have a repository manager that doesn't allow re-deploys of the same version).
Or, if only one of the modules changes, do you just build that module on its own instead of doing the multi-module build from global?
Hi Frank,
Yes, you would need to deploy the child modules independently (I didn't think of the situation where the repo manager prevents redeploys, which one are you using?).
You'd still do the global build locally to ensure consistency. Then, if you follow the versioning convention I suggest in the post,
all you have to do is go to delivery's POM, and take note of all the module that have the same version as delivery. These are the ones you need to deploy.
There might also be a clever way to do this automatically: the deploy plugin has a maven.deploy.skip property, that we could try to set by comparing the versions. But that's tricky: remember that the key is that the other modules don't depend on delivery, so how would they get its version? Maybe delivery could create a system property on the initialize phase...
I'll try to look into it. In the meantime, let me know about your progress.
Frank,
Forget what I said about delivery creating a system property: that won't work, as it is built after the other modules in a reactor build.
At best, the main version could be passed manually with a system property, and condition the deploy as I explained before (if that's unclear, I can post an example). This means you would have to look it up in delivery's POM before doing the global deploy.
That takes care of release deployments, but I just realized that the problem can also arise during continuous integration: as unimpacted modules stay at an existing release version, you want to skip them from your nightly redeploy. That can also be handled by adding another system property (e.g maven.deploy.skipReleases) that will be taken into account when calculating maven.deploy.skip.
Thanks for all of the details. Regarding the repository manager, we use Nexus. Redeployment can prevented by configuring Nexus per these instructions.
> if that's unclear, I can post an example
I think I get it, but if you could post an example, that would be awesome.
I'm working on it. Stay tuned.
I've posted a new entry. Tell me what you think.
I have posted an article where I discuss an alternative approach to solving this same version numbering issue.
http://blog.orfjackal.net/2009/05/version-number-management-for-multi.html
I have just read you post I found yesterday because I wonder me the same questions.
So I am not wrong :-)
But to my mind (I have not work deeply on this subject, I give you a quick feedback)
1 - It is difficult for my to set up a such solution for my company because it does not relies on a official way of doing things.
2 - I have the same problem than you however so my solutions :
- Limit the scope of module with only very couply project for example EAR, WAR, AAR. Each time I make a release for one I need a release for all.
- The benefits of module is to launch a global and reccursive compilation, Maven add some services as incremental build, ordering etc... with a continuous integration server like Hudson I do not need to create to much module, when a project dependency is build, the dependent project is build too, Hudson is a layer above module that can make you work easier. So do not create any module in the same project for your API, WEBAPP and IMPL, I think you can deal it easy with hudson if you not need to have the same versions.
Post a Comment