FHNManager

The integration of the gCube system with FedCloud and, in general, with research/commercial cloud providers, aims to add elasticity to the D4Science infrastructure by allowing programmatic and dynamic creation, configuration and decommissioning of cloud resources on external infrastructures. The benefits of this integration facility for the gCube system are threefold:

• to exploit infrastructural resources in a pay-per-use model
• to scale in/out quickly in response to specific needs
• to reduce manual interventions of Infrastructure Managers.

Scenarios

• the VRE Manager, according to the demand of data analysis executions in the VRE, decides the number of SmartExecutor nodes needed in the VRE. Through the VRE administration portal, the VRE Manager chooses (from a list of templates) to create the required number of SmartExecutors. She/he chooses on which infrastructure the resources should be created and their characteristics in terms of computation and/or storage capacity. The gCube system, via the integration facility here described, will automatically create the new nodes, register them to the D4Science infrastructure and make them available in the VRE.
• the gCube system intelligently adapt the number of SmartExecutor nodes to the actual demand. In the VRE administration portal, the VRE Manager decides the minimum and maximum number of SmartExecutor expected for the VRE. Then, the gCube system via the integration facility here described, automatically scales-in/out the pool of SmartExecutors (by creating/destroying nodes) predicting the demand for SmartExecutors by analysing VRE accounting and monitoring data.

Functional requirements

It is possible to recognize the following functionalities that should be available in the integration facilities:

• Registration and management of cloud providers, access credentials and cloud resources templates on a per-VRE basis. Credentials must be securely stored in the infrastructure;
• Cloud resource lifecycle management. Creation, configuration and decommissioning of cloud resources must be supported. The integration facility allows to use templates for the configuration of cloud resources (e.g. SmartExecutor Template, cpu and memory characteristics);
• Accountability and monitorability. Accounting and monitoring data on usage and workload of resources created on the cloud must be collected and made available in the infrastructure. This is both a requisite to comply with D4Science infrastructure policies and for the automatic scale-in/out;
• User interface. The user interface is integrated in the VRE administration portal since it already contains all the other VRE administration tools

Non-Functional requiriments

The solution should have modular architecture in order to support the integration of new cloud providers in a simple and pluggable way.

Architecture

The architecture highlights a number of integration-specific components and their interaction among them as well as with existing components, either belonging to the D4Science infrastructure or to the EGI infrastructure.

Deployment

Components related to the D4Science-FedCloud integration are essentially assembled in two packages that are deployed independently:

• the Federated Hosting Node Manager, embedding cloud libraries and connectors, is deployed in a SmartGears container
• the FHNManagerPortlet is deployed in the gCube portal.

Data Model