Difference between revisions of "Resource Management Specification"
Manuele.simi (Talk | contribs) (→Small deployment) |
Manuele.simi (Talk | contribs) (→Small deployment) |
||
| Line 96: | Line 96: | ||
:If a VO does not want to support dynamic deployment, several services may be excluded from the deployment scheme: | :If a VO does not want to support dynamic deployment, several services may be excluded from the deployment scheme: | ||
| − | :* Deployers do not need to be deployed on the nodes | + | :* Deployers do not need to be deployed on the nodes, since nothing will be never deployed there by the system (manual deployments are always possible) |
| − | :* VRE-Wizard, VRE-Modeler, Software Gateway and Resource Broker loose their roles | + | :* VRE-Wizard, VRE-Modeler, Software Gateway and Resource Broker loose their roles |
''limited or zero Web Applications available'' | ''limited or zero Web Applications available'' | ||
Revision as of 23:14, 28 February 2012
Overview
Resource Management focuses on the optimal exploitation of software and hardware resources. Software resources include a range of entities from packages ready to be deployed to already activated services, while hardware resources model platforms hosting a service suitable to be exploited by the e-infrastructure.
Resource Management covers the entire lifecycle of such resources through a set of services and libraries. This document outlines their design rationale, key features, and high-level architecture as well as the opportunities they collectively offer.
Key features
- Dynamic Deployment
- First-class features for deploying gCube software and external software packaged following the gCube policies, support for deployment of software hosted on recognized Maven repositories
- Optimal Resource Allocation
- Optimal selection and usage of resources during the deployment phase
- Monitoring
- Tools for pro-actively monitoring the current state of nodes and active services. Local information are collected and published in the Information System as well as returned on demand
- Easy Remote Administration
- Remotely reconfiguration of nodes and active services upon the e-infrastructure/manager's needs
- Failover capabilities
- Self-recovering from erroneous states
- Extensible Bridging over Virtual Platforms
- A flexible model for transparently interfacing a potentially unlimited number of hosting environments
- Maven Integration
- Any software available in a Maven repository is eligible to become part of the Data e-Infrastructure
- Coordination and Elastic Management
- New services can be fired up on demand and through self-government
- Cloud-orientation
- Gateway to Cloud platforms with Java support, looking forward for the proposed Topology and Orchestration Specification for Cloud Applications (or TOSCA) for short, standards specification
- Scalability
- e-Infrastructures composed by thousands of nodes and software can be concurrently managed
Design
Philosophy
The Data e-Infrastructure requires dynamic allocation of services and nodes and a strong capability to adapt itself to changes occurring during its operation. Components of Resource Management subsystem address these requirements by providing a robust ground for the overall infrastructure lifetime.
Software has different externalizations at different level, ranging from a tarball of binaries to an activated instance hosted on a platform. Resource management has to be able to deal with such different forms and the designed architecture reflect this role.
Different components have different perceptions of the resources to manage. There are components operating at a logical level (VO and VRE) that only matter about how to optimize the VO/VRE delivered functionality by replicating/duplicating services when they see this need. These components do not obviously care of the physical form of the software and will never change if it will. Other components work at service level (the constituents of the VO/VRE) and know how to optimize their allocation. Finally, there exist components that operates at local level (node) and manage only local deployed software.
Other principles on which the design of the presented architecture is piled are:
- components must be generic with respect to the resource managed
- fault recovery is a crucial feature: nodes and services must be guaranteed to have in a consistent state under any condition
- external software has to be as transparently as possible managed in the same way of the gCube software
Architecture
- VRE-Creation-Wizard: this graphical wizard guides a VRE-manager in the creation of a VRE in a step-by-step fashion
- VRE-Modeler: this service receives the high level requirements and criteria produced by the wizards and translates them in a concrete set of services to deploy or join
- Resource-Management-Portlet: integrated in the portal and accessible to authorized users, this portlet exposes a graphical visualization of the current state of the resources belonging the e-infrastructure and provides instruments for corrective and management actions (such as restarting a node, adding/removing node and service to/from a VO/VRE, etc.)
- Resource-Manager: this service stays on top of many other services and coordinates them to satisfy the VRE-Modeler requests.More specifically, the service manages resources belonging the Scope and creates new instances of services within it. It interacts with the Software gateway (for dependencies resolution), the Resource Broker (for allocating services on gHNs), the Deployer (for deploying services), the gHN Manager (for managing node's scopes) and the Information System (for publishing and retrieving resource's profiles
- Resource-Broker: this service promotes and supports the optimal selection and usage of resources during the deployment phase. In particular, it is invoked to select the most appropriate pool of gHNs to be used, among those available in the context of the deployment operation
- Resource-Broker-Serialization: a collection of structures and definitions needed to (de-)serialize data managed by the Resource-Broker
- Software-Gateway: a gateway over a cluster of Maven Repositories granting access to the stored information and software for deployment purposes
- gHN-Manager: a service providing operations for the remote management of a node
- Deployer: a service providing facilities for deploying, managing and un-deploying software components on a node
- Virtual-Platform: a library with classes and interfaces to be extended/implemented to bridge and manage the lifecycle of software running on a different platform than gCore
- Virtual-Platform for Tomcat6: an implementation of the Virtual-Platform model to manage Web Applications on Apache Tomcat 6.x
- RainyCloud
Deployment
Resource Management components have almost a fixed deployment scheme (with a few variations) because of their nature and responsibilities.
Large deployment
VRE creations have their natural place inside a VO. Therefore, VRE-Wizard, VRE-Modeler and Resource Broker, which play their respective role at VRE creation, are mandatory to be deployed in each VO aiming at supporting dynamic VREs.
Resource-Management-Portlet is also suggested to be deployed at VO level. VO views are more restricted that the whole infrastructure and can be better administrated by VO managers.
Each organized subset of resources (VO or VRE) requires an instance (and only one) of the Resource Manager that is charge of moving resource inside and outside such a subset.
The Software Gateway is an entry point for Maven Repositories, hence one instance at infrastructure level is enough.
Finally, the class of components (gHN-Manager, Deployer, Virtual Platforms) working at Package/Node level, are optimally deployed in each gHN supporting their respective platform (gCore, Tomcat, Venus-C or future bridged platforms). This way, the functionality they expose are available on every and each node by maximizing the possible targets of deployments, replica creation and balancing decided by managers or by the self-government of the system itself.
Small deployment
There exist few variations of the optimal scheme presented above. They can be adopted in some specific situations where a subset of the management functions is not requested or supported.
no support for dynamic deployment
- If a VO does not want to support dynamic deployment, several services may be excluded from the deployment scheme:
- Deployers do not need to be deployed on the nodes, since nothing will be never deployed there by the system (manual deployments are always possible)
- VRE-Wizard, VRE-Modeler, Software Gateway and Resource Broker loose their roles
limited or zero Web Applications available
- If a VO does not have any Web Application to deploy, Virtual Platforms for Tomcat do not need to be deployed. If the VO has Web Applications that do not support replication or distribution, the same cardinality of nodes supporting the Tomcat platform should be made available.
VENUS-C interoperability
- RainyCloud provides an interface to VENUS-C platform. If this is not exploited by the current configuration of a VO, there is no point of having it deployed.
Use Cases
The subsystem has been conceived to support a number of use cases moreover it will be used to serve a number of scenarios. This area will collect these "success stories".
