ETICS

Overview

ETICS stands for e-Infrastructure for Testing, Integration and Configuration of Software. ETICS is an on-line collaborative service for managing software projects by managing their configuration, enforcing quality standards, building packages and testing them in environments as close as possible to real-world infrastructures. The ETICS tools can be used to run remote builds and tests on multiple platforms at the same time.

The main architectural components of ETICS (depicted in the figure above) are:

• ETICS Web Portal is the main access point to ETICS. It allows to access (view and edit) the configuration of gCube components, submit builds and tests and check the reports;
• ETICS Worker Nodes are the nodes were builds and tests take place. Users do not need to access to these nodes;
• ETICS Command-Line Client is a set of command-line tools to access ETICS services. The Command-Line Client provides similar functionalities as the web application. In addition, it allows to run local builds on the developer's machine and is expecially suitable for scripting and batch executions;

How to create ETICS configurations for gCube components

This sections is meant to provide operative instructions about most relevant aspects of creation of ETICS configurations for Maven-based components.

Versioning

Version field of the ETICS configuration must match the version specified for the Maven project in the pom.xml, excepting for the "-SNAPSHOT" postfix. The match will be checked during the invocation of mvn to guarantee that ETICS versions are always in sync with Maven versions.

Build Commands

• the compile target of their Build Commands must invoke Maven to build the components up to the deploy phase (e.g. mvn deploy). This allows the deployment of components into Maven repositories directly from ETICS builds

• the install target of their Build Commands must copy the gCube profile of the component in the ${prefix} directory. This allows the registration of the profile with the Software Gateway when the component is released. It remains a good practice to copy also the outcome of the compilation (usually jar files) in ${prefix} in order to include them in packages generated by ETICS

Sample build commands for a maven-based component should look like the one in the picture below:

Note: please note that configuration's commands (mainly install target) could change slightly depending on the actual project structure.

Build Commands for gCube Services

gCube services components are usually multi-modules maven projects (see gCyve Service development guide for more information) containing a module for the stubs, one for the service and optionally one for test-suites. In ETICS a service must be modelled with one ETICS component for each Maven project's module.

For each ETICS component, build commands should contains commands to deploy only the correspondent Maven module, and, if needed, compile the other modules needed by the first.

Picture below shows the typical build commands for the stubs component. First the mvn deploy -N is invoked to deploy the project's parent pom. If not executed, stubs' compilation will fail because parent's pom will not be resolved. Then the mvn deploy -pl exampleservice-stubs is invoked to build and compile only the stubs module.

Finally, picture below shows the typical build commands for the service component. Since the service module of a gCube service needs the stubs to be compiled locally in order to generate the gar archive, before execute the mvn deploy -pl exampleservice-service it is needed to also build the stubs mvn install -pl exampleservice-stubs. Note that it is not needed to invoke target deploy for stubs, since they are deployed already by the stubs ETICS' component. The parent's pom will already exist in the local repository (deployed by the stubs' component) and, thereofore, it is not needed to compile it again.

Dependencies

The configuration must directly or indirectly depend on the ETICS configurations of all gCube components that are specified in the POM, including maven-builder. This guarantees that at build-time all dependencies requested by the pom.xml have been already compiled and installed on the Maven's local reposiotry.

Dependencies on third-party components available in Maven Central (or other Maven repository specified in the POM) do not need to be configured in ETICS (i.e. Maven repositories replace ETICS externals).

All dependencies must be declared of type dynamic and the actual version will be resolved at project level.

Example of dependencies for a Maven component:

Environment

Unlike Ant-based components, Maven components do not use the CLASSPATH variable for compile-time dependency resolution, but rely on the Maven local repository. This means that the Environment section of Maven component's configurations will be empty (unless other environment variables has to be set for other purposes).

There exist an exception, though. If the component's artifact is meant to be used as compile-time dependency by others Ant-based components, the artifact produced by the Maven build needs to be exported in the CLASSPATH to allows the Ant-based component to find it. Only in this case the Environment section will contain an entry to add to CLASSPATH the artifact produced. It is suggested to use following convention:

CLASSPATH: ${moduleDir}/target/<artifactid>-${version}-[SNAPSHOT|${maven.system-version}].jar

where:

• <artifactId> must be replaced with the name of the artifact produced by Maven compilation. By default it is the same of <artifactId> value in the pom
• SNAPSHOT must be added only if you are setting up a configuration for org.gcube.HEAD otherwise
• you should use the ${maven.system-version} property
• ${moduleDir} and ${version} are ETICS properties and will be resolved by ETICS itself

Valid examples of CLASSPATH:

CLASSPATH: ${moduleDir}/target/home-library-${version}-${maven.system-version}.jar
CLASSPATH: ${moduleDir}/target/home-library-${version}-SNAPSHOT.jar
CLASSPATH: ${moduleDir}/SoftwareGateway-stubs/target/softwaregateway-stubs-${version}-${maven.system-version}.jar
CLASSPATH: ${moduleDir}/SoftwareGateway-stubs/target/softwaregateway-stubs-${version}-SNAPSHOT.jar		
...

Tip: you can easily find exactly which artifact is created during an ETICS build by searching in the build report of the component for the line "Installing /home/gbuilder/catalogueoflife-spd-plugin/target/catalogue-of-life-spd-plugin-1.0.0-2.12.0.jar to..."

How to submit remote builds of gcube subsystems/components

A pre-requisite to submit a build is to have right to do that. Developers have rights to build their components, Subsystem Managers have rights to build their subsystem and the Release Manager have the rights to build the entire project.

In ETICS Portal, to access the Remote Build submitting form (see picture below) just right-click on the configuration you wish to build and then select "Remote Build..." from the contextual menu that will appear. The submission form is shown below:

The most common options to set for gCube builds are listed in the table below.

option	description	usage
verbose	useful to debug the build process	recommended
Ignore Locking	A locked configuration in ETICS has a "snapshot" of the gCube configuration in which it has been locked. When a locked configuration is built in a different version of gCube of the one in which has been locked, it will try to bring in the build all the configurations belonging to the "snapshot"; most likely this will result in multiple configurations for the same component that will break the build. The usage of "Ignore Locking" option will prevent configurations from the snapshots of locked configuration to be brought in the build.	mandatory
Propagate environment and ...	essential. It will instruct ETICS about the gCube version in which context (gCube's version) build the configuration.	mandatory
Checkout from Volatile area	every ETICS build stores the binary artifacts in a remote area. By setting this option the build can use, if they exist, binary artifacts from past builds instead of re-compiling it from source (it only applies to dependencies, not on the build target component).	recommended
Do not stop on errors	ETICS will try to complete the build also in case the build of one or more components fails.	recommended
Host selection	The platform against which the build will be performed. gCube is completely in java and, thus, it is completely cross-platform and, theoretically, any of available platforms can be chosen. However, the current ETICS worker nodes all runs on CentOS 7 and, therefore, that is the only possible platform.	mandatory

Once options have been set, just click on the submit icon (top-left of the submission form) and the build will start. You can keep trace of your builds in the 'Submissions tab of ETICS Portal.

ETICS Issues

The ETICS Issues section in the ETICS Portal lists some issues found in components during the build. Issues are related not only to the result of the builds but also to quality guidelines defined by projects. Some issues specific for gCube have been defined (e.g. compliance of README files) and are reported in this list.

As depicted in the figure below, the user can group (in the top-right of the page) the issue list by different attributes: module, priority, type and owner. A summary for each group is displayed at the top of the page reporting the type and subtype of the issue, the total number of issues in the group and the sum of the technical debt computed for the issues.

Under the groups summary, there is the list of all issues ordered by group. By clicking on the issue, the details are displayed (see figure below).

Issue Owners

For each issue, ETICS computes a list of owners. It is the list of users that have permissions on the component associated to the issue order by proximity: users that have permissions at component-level are listed first, then the users that have permission on the component's subystem and finally users that have permission at level of project.

Issue Lifecycle

ETICS runs a number of checkers during the remote builds and creates the discovered issues at the end of each build. An issue is automatically closed by ETICS when it runs again the build of the affected component and does not find the issue anymore. Closed issues appears with in Straight-through style for few days and then disappear completely.

Some checks run only during BTRT builds and, therefore, are updated only during BTRT builds.

External Links

Etics Web Portal

Etics Build Report

Etics Client Installation HowTo