Continuous Integration of gCube System: Overview

From Gcube Wiki
Jump to: navigation, search

Workflow

CI pipeline V6.png

How many Builds

One of biggest risks of a non-properly configured Continuous Integration pipeline is to trigger too many builds and transform the pipeline into something we can call Continuous Building. Multiple builds of the same project simultaneously on the same slave can interfere with each other and inconsistent situations are very common. For instance, this is the case when each commit is immediately to the master branch or when the Git repository is wrongly used as backup system.

In this guide, we set up a few procedures to prevent this misbehavior. Some of them assume a proper usage of Git from the user.

Here's a summary of these practices:

  • the Jenkins project builds always the master branch
  • if a Jenkins project is configured to build a second branch, this branch must generate a software artifact with a different version than master branch
  • the development of new features and bug fixing are done in dedicated branches (task branching)
  • merges into master branch are performed when the feature/fix is stable
  • commits are not always pushed to the remote repository, but only when they add a significant, self-contained and consistent piece of working code
  • the master branch MUST be always in a releasable state

By following them, builds are triggered only when a stable feature is merged into master, while commits in the other branches do not involve Jenkins. If two branches are built at the same time in a Jenkins project (which should be temporary), their different versions guarantee that there are no conflict in the published artifacts.

More suggestions for a proper exploitation of the pipeline are in the Best Practices section.

Maven Isolation

Golden Rule: Jenkins builds MUST depend only on the output of other Jenkins builds (exceptions are Releases and third-party artifacts).

Another mistake to avoid with the pipeline is to use the public SNAPSHOT Maven Repository. We must guarantee that the SNAPSHOT dependencies resolved during the builds on the servers are ALWAYS coming from other Jenkins builds. If we let the builds use the SNAPSHOT versions manually deployed by the developers, the entire pipeline is INVALID.

One long-standing solution to this is to use the “Use private Maven repository” option in the “Advanced” section of the Maven build. This creates an isolated local Maven repository for the job (in $WORKSPACE/.repository) which prevents these problems. Jenkins releases since 1.448 let you specify a Maven repository per executor, which is a more efficient way to solve the same problem. Read more in the Jenkins section.

Regarding the dependencies of released artifacts:

  • for gCube components, they are resolved against the RELEASE Maven Repository
  • for third-party components, they are resolved against Maven Central or other external repositories.

Back to the CI guide.