Difference between revisions of "Creating gCube Maven components: How-To"
(→Dependencies) |
(→Profile) |
||
Line 240: | Line 240: | ||
<Version>1.0.0</Version> | <Version>1.0.0</Version> | ||
<MavenCoordinates> | <MavenCoordinates> | ||
− | <groupId>org.gcube</groupId> | + | <groupId>org.gcube.data.access</groupId> |
<artifactId>tree-manager-service</artifactId> | <artifactId>tree-manager-service</artifactId> | ||
<version>1.0.0-SNAPSHOT</version> | <version>1.0.0-SNAPSHOT</version> | ||
Line 259: | Line 259: | ||
<Version>1.0.0</Version> | <Version>1.0.0</Version> | ||
<MavenCoordinates> | <MavenCoordinates> | ||
− | <groupId>org.gcube</groupId> | + | <groupId>org.gcube.data.access</groupId> |
<artifactId>tree-manager-stubs</artifactId> | <artifactId>tree-manager-stubs</artifactId> | ||
<version>1.0.0-SNAPSHOT</version> | <version>1.0.0-SNAPSHOT</version> |
Revision as of 17:19, 3 May 2012
Developing a gCube Maven Component
We assume familiarity with Maven concepts and terminology and notice that Maven components for gCube must satisfy the following requirements:
- their
groupId
must be of the following form:
org.gcube.<class>
- where class identifies a functionally correlated set of components and helps a range of users (developers, administrator) to discover components in various UIs/reports used within the system and in tools. Logically, the class corresponds to the gCube class of the component's, as specified in its gCube Software Profile. However, more specific, or else cross-class groups may be agreed upon among developers. In all cases, the group should align with official Maven guidelines. By agreement, for example, components in gCube class
DataAccess
declare agroupId
oforg.gcube.data.access
and those in gCube classDataTransfer
declare agroupId
oforg.gcube.data.transfer
. On the other hand, also by agreement, components in gCube classInformationSystem
declare agroupId
oforg.gcube.informationsystem
.
- their
groupId
must follow Maven conventions (e.g. avoid capital letters or punctuation other than hyphens).
- they must inherit from maven-parent. This ensure and promotes compliance with project-wide requirements, from the enforcement of minimal Java and Maven versions to generation and packaging of Javadoc documentation and component sources;
- for each deployable artefact they produce (primary or secondary), they must include the Maven coordinates of the arteact in the corresponding package section of the gCube Software Profile. Specifying package dependencies in the Software Profile, on the other hand, is no longer required;
- whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary
tar.gz
artefact with classifierservicearchive
. Note that the SA of Maven components does not need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);
We illustrate how these requirements can be satisfied by example, distinguishing the case of library components and service components:
Libraries
The streams
library is a Maven component in gCube class DataAccess
. The Maven coordinates of one of its development versions are:
<groupId>org.gcube.data.access</groupId> <artifactId>streams</artifactId> <version>1.0.0-SNAPSHOT</version>
Its POM specifies the following Maven parent:
<parent> <artifactId>maven-parent</artifactId> <groupId>org.gcube.tools</groupId> <version>1.0.0</version> <relativePath /> </parent>
Profile
The gCube Software profile of the library is as follows:
<?xml version="1.0" encoding="UTF-8"?> <Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <ID /> <Type>Service</Type> <Profile> <Description>Embedded Domain-Specific Language for Stream Transformations</Description> <Class>DataAccess</Class> <Name>streams</Name> <Version>1.0.0</Version> <Packages> <Software> <Name>streams</Name> <Version>1.0.0-SNAPSHOT</Version> <MavenCoordinates> <groupId>org.gcube.data.access</groupId> <artifactId>streams</artifactId> <version>1.0.0-SNAPSHOT</version> </MavenCoordinates> <Files> <File>streams-1.0.0-SNAPSHOT.jar</File> </Files> </Software> </Packages> </Profile> </Resource>
Notice that:
- the profile includes a single package and this package corresponds to the main build artefact of the component;
- the whole profile and the package have usual gCube coordinates;
- the package name is aligned with the Maven
artifactId
of the component; - the package version is aligned with the Maven
version
of the component; - the package includes
MavenCoordinates
that can be directly copied and pasted from the POM; - the package does not specify dependencies;
- the package points to the main artefact of the Maven build;
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.
In particular, streams
uses POM variables top define its profile:
<?xml version="1.0" encoding="UTF-8"?> <Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <ID /> <Type>Service</Type> <Profile> <Description>${description}</Description> <Class>DataAccess</Class> <Name>${artifactId}</Name> <Version>1.0.0</Version> <Packages> <Software> <Name>${artifactId}</Name> <Version>${version}</Version> <MavenCoordinates> <groupId>${groupId}</groupId> <artifactId>${artifactId}</artifactId> <version>${version}</version> </MavenCoordinates> <Files> <File>${build.finalName}.jar</File> </Files> </Software> </Packages> </Profile> </Resource>
Note that the variables must be resolved (i.e. the profile is interpolated) before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.
Service Archive
streams
is responsible for generating its own SA as part of its Maven build. This requires dedicated logic in the POM but has the advantage that:
- SA validity can be verified locally;
- no SA configurations are required in ETICS;
- the required build logic can be easily reused across components;
streams
uses the Maven Assembly Plugin to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as README
, svnpath.txt
, MAINTAINERS
, changelog.xml
, etc. In particular, the Assembly
plugin takes care of variable interpolation in the profile and other static files. The approach is best illustrated with a reference to the [sources] of the streams
library.
As we discuss below, streams
has the final requirement to produce an interpolated profile during ETICS builds, outside the context of the SA.
streams
meets this requirement with the Maven Resources Plugin. Again, consult the [sources] of the streams
library to reuse this approach.
Services
The tree-manager
service is a Maven component in gCube class DataAccess
.
Project Structure
Differently from a library, the service is associated with a number of distinguished build artefacts:
- the service implementation (a Jar);
- the stubs automatically generated from the WSDL interfaces (a Jar);
- the archive required for deployment in gCore (a Gar);
- the archive required for registration in the system (a SA);
Producing these artefacts requires sharing a number of static and dynamic files (from profiles to WSDLs to pre-processed WSDLs). The usual Maven approach of mapping the artefacts onto different Maven projects is then genuinely difficult. It is instead more practical to use a single multi-module Maven project. The tree-manager
uses this module structure:
tree-manager (parent module)------- (produces) --- (POM only) | |----- tree-manager-service (service impl module)------- (produces) --- jar, gar, SA | | | | (depends on) | | | \/ |----- tree-manager-stubs (service stubs module)------- (produces) --- jar
As customary, the parent does not produce any artefact (is a POM-only project). It declares these coordinates in its POM:
<groupId>org.gcube.data.access</groupId> <artifactId>tree-manager</artifactId> <version>1.0.0-SNAPSHOT</version> <packaging>pom</packaging>
The parent physically includes the sub-modules;
<modules> <module>tree-manager-stubs</module> <module>tree-manager-service</module> </modules>
Finally, the parent inherits from maven-parent:
<parent> <artifactId>maven-parent</artifactId> <groupId>org.gcube.tools</groupId> <version>1.0.0</version> <relativePath /> </parent>
The submodules inherit from the parent:
<parent> <artifactId>tree-manager</artifactId> <groupId>org.gcube.data.access</groupId> <version>1.0.0-SNAPSHOT</version> <relativePath>..</relativePath> </parent>
and share its version
and groupId
. Their coordinates are then minimal:
<artifactId>tree-manager-service</artifactId>
<artifactId>tree-manager-stubs</artifactId>
The tree-manager-service
module produces a Jar of its own implementation, as well the GAR and the SA of the whole services as secondary artefacts with classifiers of, respectively, gar
and service archive
.
The tree-manager-stubs
module produces a Jar of its own implementation.
Finally, the parent module coordinates the build of its sub-modules and centralises the declaration of common dependencies and configuration. Refer to the [project sources] to reuse this approach.
Generating Stubs and Gars
The tree-manager-stubs
uses the gCube Service Plugin to generate its classes from WSDL interfaces.
Similarly, the tree-manager-service
uses the plugin to generate its secondary GAR artefact.
The configuration of the plugin is conveniently shared in the parent module.
Refer to the [project sources] of the streams
for details.
Profile
The gCube Software profile (profile.xml
) of a development version of the service is as follows:
<?xml version="1.0" encoding="UTF-8"?> <Resource> <ID></ID> <Type>Service</Type> <Profile> <Class>DataAccess</Class> <Name>tree-manager-service</Name> <Version>1.0.0</Version> <Packages> <Main> <Name>tree-manager-service</Name> <Version>1.0.0</Version> <MavenCoordinates> <groupId>org.gcube.data.access</groupId> <artifactId>tree-manager-service</artifactId> <version>1.0.0-SNAPSHOT</version> </MavenCoordinates> <GARArchive>tree-manager-service-1.0.0-SNAPSHOT.gar</GARArchive> <PortType> <Name>gcube/data/tm/binder</Name> </PortType> <PortType> <Name>gcube/data/tm/reader</Name> </PortType> <PortType> <Name>gcube/data/tm/writer</Name> </PortType> </Main> <Software> <Name>tree-manager-stubs</Name> <Version>1.0.0</Version> <MavenCoordinates> <groupId>org.gcube.data.access</groupId> <artifactId>tree-manager-stubs</artifactId> <version>1.0.0-SNAPSHOT</version> </MavenCoordinates> <Type>library</Type> <Files> <File>tree-manager-stubs-1.0.0-SNAPSHOT.jar</File> </Files> </Software> </Packages> </Profile> </Resource>
Notice that:
- the profile includes two packages and this package corresponds to the main build artefacts of the submodules;
- the whole profile and the packages have usual gCube coordinates;
- package names are aligned with the Maven
artifactId
s of the submodules; - package versions are aligned with the Maven
version
s of the submodules; - package include
MavenCoordinates
s that can be directly copied and pasted from the POM of the submodules; - packages do not specify dependencies;
- packages points to the main artefact of the Maven build of the submodules;
Notice that, unlike libraries, services cannot easily use Maven variables in their profiles (as these artefacts are used for different purposes by different modules). Accordingly, developers needs to maintain versions used in profiles across releases. In particular, they need to:
- replace SNAPSHOT versions with release versions in release branches;
- advance SNAPSHOT versions in HEAD;
Service Archive
tree-manager-service
generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the module and its parent.
As shown for libraries, the service uses the Maven Assembly Plugin to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as README
, svnpath.txt
, MAINTAINERS
, changelog.xml
, etc. In particular, the Assembly
plugin takes care of variable interpolation for these files. Refer to the [project sources] for details.
Dependencies
Service and libraries that depend on other gCube components (Maven-based or Ant-based) can find them in the gCube Maven Repositories: gcube-snapshots
and gcube-releases
.
Repositories
As a one-off task, developers must configure these repositories in the ~/.m2/settings.xml
file of their local Maven installation, as follows:
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"> <servers> <server> <id>gcube-snapshots</id> <username>gcube-user</username> <password>maven</password> </server> <server> <id>gcube-releases</id> <username>gcube-user</username> <password>maven</password> </server> </servers> <profiles> <profile> <id>gcube</id> <repositories> <repository> <id>gcube-snapshots</id> <name>gCube Snapshots</name> <url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url> <releases> <enabled>false</enabled> </releases> <snapshots> <enabled>true</enabled> </snapshots> </repository> <repository> <id>gcube-releases</id> <name>gCube Releases</name> <url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-releases</url> <snapshots> <enabled>false</enabled> </snapshots> <releases> <enabled>true</enabled> </releases> </repository> </repositories> <pluginRepositories> <pluginRepository> <id>gcube-snapshots</id> <name>gCube Snapshots</name> <url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url> <releases> <enabled>false</enabled> </releases> <snapshots> <enabled>true</enabled> </snapshots> </pluginRepository> <pluginRepository> <id>gcube-releases</id> <name>gCube Releases</name> <url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-releases</url> <snapshots> <enabled>false</enabled> </snapshots> <releases> <enabled>true</enabled> </releases> </pluginRepository> </pluginRepositories> </profile> </profiles> <activeProfiles> <activeProfile>gcube</activeProfile> </activeProfiles> </settings>
Note:Developers must not configure these repositories in their POMs, only in settings.xml
.
This is to:
- keep their projects independent from future changes in repository locations;
- avoid interference with ETICS builds, where access to these repositories may comprise correct system integration;
On the other hand, repositories for third-party dependencies which are not in Maven Central must be declared in POMs, as these need to be available during integration builds.
As usual, third-party dependencies available in Maven Central require no explicit configuration.
Dependency Versions
ETICS builds ensure correct integration of gCube components, regardless of the versions their dependencies. It also ensures that components are released with fixed versions for their dependencies, i.e. ensures build reproducibility.
Accordingly, developers can be liberal in the choice of dependency versions. i.e. rely on version ranges to:
- minimise the requirements for version management;
- be readily informed of any mis-alignment with their dependencies;
As we start adopting Maven, the lower-bound of version ranges will be a SNAPSHOT version. For example, a dependency on the streams
library can be specified as follows:
<dependency> <groupId>org.gcube.data.access</groupId> <artifactId>streams</artifactId> <version>[1.0.0-SNAPSHOT,2.0.0)</version> </dependency>
With this dependency, a component will be soon notified of a non-retro compatible change that, contrary to conventions, streams
may have introduced in, say, version 1.3.0
(typically the day after at the latest). The notification will be delivered in the IDE, assuming the availability of Maven integration (e.g. m2e
in Eclipse).
The component is in fact free to open the range further if it wants to be aligned with its dependencies across major versions.
Common Dependencies
Many gCube components depend on gcf
, which is now available as a Maven component since version 1.4
. At the time of writing, the component has not been released yet. A dependency on it is then specified as follows:
<dependency> <groupId>org.gcube.core</groupId> <artifactId>gcf</artifactId> <version>[1.4.0-SNAPSHOT,2.0.0)</version> <scope>provided</scope> </dependency>
The scope provided
indicates that these dependency will be satisfied in the development environment (the gHN) and should not be included in GAR and SA archives.
Some components depend on the libraries available in all gHNs, though they do not depend on gcf
itself. The subset of these libraries that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called ghn-core-runtime
. A dependency on this Jar can be specified as follows:
<dependency> <groupId>org.gcube.distribution</groupId> <artifactId>ghn-core-runtime</artifactId> <version>1.0.0</version> <scope>provided</scope> </dependency>
Note: a component that depends on gcf
indirectly depends on gun-core-runtime
.
Another common dependency is on the gCube gRS2, an Ant component that the build infrastructure make available in Maven repositories. A dependency on this component can be specified as follows:
<dependency> <groupId>org.gcube.execution</groupId> <artifactId>grs2library</artifactId> <version>[1.2.0-SNAPSHOT,2.0.0)</version> </dependency>
Creating the ETICS Configurations
This sections is meant to provide operative instructions about most relevant aspects of creation of ETICS configurations for Maven-based components.
Versioning
Version field of the ETICS configuration must match the version specified for the Maven project in the pom.xml, excepting for the "-SNAPSHOT" postfix. The match will be checked during the invocation of mvn to guarantee that ETICS versions are always in sync with Maven versions.
Build Commands
- the compile target of their Build Commands must invoke Maven to build the components up to the deploy phase (e.g.
mvn deploy
). This allows the deployment of components into Maven repositories directly from ETICS builds
- the install target of their Build Commands must copy the gCube profile of the component in the
${prefix}
directory. This allows the registration of the profile with the Software Gateway when the component is released. It remains a good practice to copy also the outcome of the compilation (usually jar files) in${prefix}
in order to include them in packages generated by ETICS
Sample build commands for a maven-based component should look like the one in the picture below:
Dependencies
The configuration must directly or indirectly depend on the ETICS configurations of all gCube components that are specified in the POM, including maven-parent. This guarantees that at build-time all dependencies requested by the pom.xml have been already compiled and installed on the Maven's local reposiotry.
Dependencies on third-party components available in Maven Central (or other Maven repository specified in the POM) do not need to be configured in ETICS (i.e. Maven repositories replace ETICS externals).
All dependencies must be declared of type dynamic and the actual version will be resolved at project level.
Example of dependencies for a Maven component:
Environment
Unlike Ant-based components, Maven components do not use the CLASSPATH
variable for compile-time dependency resolution, but rely on the Maven local repository. This means that the Environment section of Maven component's configurations will be empty (unless other environment variables has to be set for other purposes).
There exist an exception, though. If the component's artifact is meant to be used as compile-time dependency by others Ant-based components, the artifact produced by the Maven build needs to be exported in the CLASSPATH to allows the Ant-based component to find it. Only in this case the Environment section will contain an entry like:
CLASSPATH: ${moduleDir}/target/artifactFilename.jar