Difference between revisions of "Resource Manager"

Revision as of 00:25, 29 May 2010

The Resource Manager service is in charge of managing gCube Scope contexts. An instance (and only one) of the service must be deployed for each scope (at any level, i.e. infrastructure, VO, VRE) in order to handle gCube Resources within the assigned scope. It manages requests of adding and removing gCube Resource from the assigned Scope by keeping, at the same time, a consistent state in the whole infrastructure. A Scope Resource is published in the Information System with constantly updated information about the current composition of the Scope.

Design

Service architecture and code design

The service implements a singleton WSRF pattern. This means that there is a single stateful resource holding the state of the managed Scope.

The following class diagram shows a simplified view of the organization and structure of the Resource Manager's source code.

Figure 1. Resource Manager Class Diagram

[TBD]

Concept, role and design of ScopedResource

A Resource Manager instance handles the Scope it is managing as a set of Scoped Resources. A Scoped Resource models a gCube Resource in a specific Scope. At the heart of the Scope management activities there is the management of the state of the Scoped Resources belonging the Scope.

The following diagram depicts the possible Scope Resource's statuses and the legal transitions among them. The starting point of the life of a Scope Resource is always the request to join an existing gCube Resource to the Scope.

Figure 2. Scoped Resource - life cycle

The crucial statuses are ADDED and REMOVED, where the concrete (and resource specific) actions to join the gCube Resource to the Scope are performed. For instance, in case of a Service Resource, a new instance of the service is deployed when managing the ADDED status, while undeployment is performed in the REMOVED status.

[TBC]

Scope state

Scope state management versus old VRE state management
Raw state separated from the state behavior to avoid mis-serializations due to any future change in the service implementation
Local XStream serialization
Synchronization between local and remote (i.e. stored in the IS) state and vice versa
...

[TBC]

Observer pattern strategy

The service adopts an observer pattern-based internal strategy for managing the scope state. Each time the state of any Scoped Resource changes, registered observers are properly notified in order to manage the new state.

[TBC]

Interface

Resource Report

All the resource management requests work in an asynchronous manner. They return an ID that can be used to query the getReport operation in order to retrieve a detailed report of the activities performed.

The basic structure of the report is:

<ResourceReport>
  <ID><!--report ID --></ID>
  <LastUpdate><!--last update time --></LastUpdate>
  <TargetScope><!-- target scope of the operations reported in the following section --></TargetScope>
  <Resources>
     <!-- list of gCube resources involved and their actual status-->  
     <Resource>
      <ID>..</ID>
      <Type>...</Type>
      <Status>ADDED/REMOVED/FAILED</Status>
    </Resource>
     <Resource>
      <ID>..</ID>
      <Type>...</Type>
      <Status>ADDED/REMOVED/FAILED</Status>
    </Resource>
  </Resources>
  <DeploymentPlanCreation>
      <!-- outcomes of the deployment plan creation -->
      <Status>SUCCESS / FAILED</Status>
      <Message></Message>
   </DeploymentPlanCreation>
  <Services>
   <!-- per each service involved in the request, a section like the following-->
    <Service>
 
     <!-- service data --> 
 
     <DependenciesResolution>
 
     <!-- information about service's dependencies (missed, resolved))-->
 
     </DependenciesResolution>
     <DeploymentActivity> or <UndeploymentActivity>
 
     <!-- information about (un)deployment activities on the target gHN-->
 
     </DeploymentActivity> or </UndeploymentActivity>
    <Service>
  </Services>
</ResourceReport>

The Services' section is reported in case that the activities performed involve the deployment or undeployment of services within the scope. For each service, information about the service's dependencies resolution (if any) and package deployment/undeployment are reported.

Possible dependencies statuses are:

SUCCESS: all the dependencies have been successfully resolved
FAILED: some dependencies are missing

Periodically, the Deployer service sends a report on the deployment activity (performed asynchronously with respect to the VREManager request). Such report has a status:

OPEN: the activity is still ongoing, other reports will follow
CLOSED : the activity is closed, no further report will be sent

Possible package statuses are:

WAITING: request is still pending
ALREADYDEPLOYED: the package was already available on the target gHN
SKIPPED: the package was not deployed due to an error in another service from which it depends on
FAILED: the deployment failed
DEPLOYED: the package has been deployed
NOTVERIFIED: the package has been deployed, but the correctness of the operation cannot be checked (and therefore guaranteed)
ACTIVATED: the package has been activated
RUNNING: the related instance is running

Possible resource statuses:

ADDED: the resource was successfully added to the scope
REMOVED: the resource was successfully removed to the scope
FAILED: an error occurred and the resource has not been managed

This is a sample report showing the results of a request to add 2 services, a generic resource, a collection, a metadata collection and an existing running instance to the managed scope.

Sample Usage

All the operation described below are exposed by the ResourceManager portType .

Creating a new scope

To create a new scope, the CreateScope operation can be invoked. In reality, with the current version of the Resource Manager, this call will not create a new scope from scratch, but it will initialize the scope options and will establish the initial set of resources forming the scope. To be successful in the creation, the following preconditions must be valid:

in case of VO scope, the gHN hosting the Resource Manager instance must already have locally the Service Map of the VO;
in case of VRE scope, the gHN hosting the Resource Manager instance must already have locally the Service Map of the parent's VO;
the Resource Manager instance must already join the scope (that's odd)

import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.AddResourcesParameters;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.CreateScopeParameters;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.OptionsParameters;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ResourceItem;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ResourceList;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ResourceManagerPortType;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ScopeOption;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ServiceItem;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ServiceList;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.service.ResourceManagerServiceAddressingLocator;
 
 
EndpointReferenceType endpoint = new EndpointReferenceType();
 
try {
	endpoint.setAddress(new Address("http://"+ args[0]+":"+ args[1] +"/wsrf/services/gcube/vremanagement/ResourceManager"));	
 
	GCUBESecurityManagerImpl managerSec = new GCUBESecurityManagerImpl() {  
	public boolean isSecurityEnabled() {return false;}};
	ResourceManagerPortType pt = GCUBERemotePortTypeContext.getProxy(new ResourceManagerServiceAddressingLocator().getResourceManagerPortTypePort(endpoint), 
				GCUBEScope.getScope("SCOPE"),managerSec);
 
	logger.info("Setting the Scope parameters...");
	OptionsParameters options = new OptionsParameters();
	ScopeOption[] scopeOptionList = new ScopeOption[I];
	for (int i=0; i < optionNames.length; i++) 
		scopeOptionList[i] = new ScopeOption("name", "value");	
	options.setScopeOptionList(scopeOptionList);
 
	AddResourcesParameters add = new AddResourcesParameters();
 
	// prepare the list of services to be deployed
	ServiceItem[] servicelist = new ServiceItem[Y];
	for (int i = 1 ; i < (servicelist.length +1); i++) {
		servicelist[i-1] = new ServiceItem();
		servicelist[i-1].setServiceClass("CLASS");
		servicelist[i-1].setServiceName("NAME");
		servicelist[i-1].setServiceVersion("VERSION");
                //if we want to have the service deployed on this specific GHN
		servicelist[i-1].setGHN("GHNID"); 
	}
	ServiceList l = new ServiceList();
	l.setService(servicelist);
	//set the suggested set of GHNs to use for deployment, if any
	String[] ghns = new String[] {"GHNID1", "GHNID2", "GHNID3"};
	l.setGHN(ghns);
	add.setServices(l);
 
	//prepare the list of resources to join to the new VRE
	ResourceItem[] resourcelist = new ResourceItem[N];
	for (int i = 1 ; i < (resourcelist.length +1); i++) {
		resourcelist[i-1] = new ResourceItem();
		resourcelist[i-1].setID("ID");
		resourcelist[i-1].setType("TYPE");
	}
	ResourceList r = new ResourceList();
	r.setResource(resourcelist);
	add.setResources(r);
 
	add.setTargetScope(resources.getProperty("targetScope"));
 
	//ready to start...
	logger.info("Sending the creation request....");			
	CreateScopeParameters params = new CreateScopeParameters();
	params.setName("SCOPE");
	params.setOptionsParameters(options);
	params.setAddResourcesParameters(add);
	String reportID = pt.createScope(params);
	logger.info("Returned report ID: " + reportID);
 
} catch (Exception e) {
	logger.fatal("Failed to create the scope",e);
}

Set/change the scope options

A number of options can be set on a scope by invoking the ChangeScopeOptions operation. Actually, the following properties are accepted:

Designer: the designer of the scope
Creator: the creator of the scope
StartTime: the activation time of the scope (accepted format is yyyy-MM-dd'T'HH:mm:ssZ, example: 2008-11-11T16:58:12+01:00)
EndTime: the last time of the scope (accepted format is yyyy-MM-dd'T'HH:mm:ssZ, example: 2008-11-11T16:58:12+01:00)
Description: a scope description
DisplayName: a name to display in any GUI
SecurityEnabled: a boolean flag to set up a secure or non-secure Scope

All of them are optionals and they can be set/updated any time, even if for some of them (like changing the start time once the scope is activated) such an action do not have any effect.

The ChangeScopeOptions operation can be invoked as follows:

import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.OptionsParameters;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ScopeOption;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ResourceManagerPortType;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.service.ResourceManagerServiceAddressingLocator;
 
//...
 
protected static String[] optionNames = new String[] {"creator", "designer", "endTime", "startTime", 
			"description", "displayName", "securityenabled"};
//...
EndpointReferenceType endpoint = new EndpointReferenceType();
ResourceManagerPortType pt;	
try {
	endpoint.setAddress(new Address("http://"+ <ResourceManager host>+":"+ <ResourceManager port> +"/wsrf/services/gcube/vremanagement/ResourceManager"));
 
	GCUBESecurityManagerImpl managerSec = new GCUBESecurityManagerImpl() {  
					public boolean isSecurityEnabled() {return false;}};
	pt = GCUBERemotePortTypeContext.getProxy(new ResourceManagerServiceAddressingLocator().getResourceManagerPortTypePort(endpoint), 
							GCUBEScope.getScope(<caller scope>),managerSec);	
	OptionsParameters options = new OptionsParameters();
	ScopeOption[] scopeOptionList = new ScopeOption[optionNames.length];
	for (int i=0; i < optionNames.length; i++) 
		scopeOptionList[i] = new ScopeOption(optionNames[i], <PROPERTY VALUE>);
 
	options.setScopeOptionList(scopeOptionList);
	pt.changeScopeOptions(options);
} catch (Exception e) {
	//manage the error
}

Request to join a resource to the scope

A request to join a GCUBEResource to the VREManager scope is performed through the AddResources operation. For each resource there are two main information to provide:

the resource identifier
the resource type (must be one of: GHN, RunningInstance, Collection, MetadataCollection, GenericResource)

The AddResources operation can be invoked as follows:

import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.AddResourcesParameters;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ResourceItem;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ResourceList;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ServiceItem;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ServiceList;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ResourceManagerPortType;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.service.ResourceManagerServiceAddressingLocator;
 
//...
 
EndpointReferenceType endpoint = new EndpointReferenceType();
 
try {
	endpoint.setAddress(new Address("http://"+ <ResourceManager host>+":"+ <ResourceManager port> +"/wsrf/services/gcube/vremanagement/ResourceManager"));
 
	GCUBESecurityManagerImpl managerSec = new GCUBESecurityManagerImpl() {  
					public boolean isSecurityEnabled() {return false;}
	};
	ResourceManagerPortType pt = GCUBERemotePortTypeContext.getProxy(new ResourceManagerServiceAddressingLocator().getResourceManagerPortTypePort(endpoint), 
		GCUBEScope.getScope("callerScope"),managerSec);
 
	AddResourcesParameters add = new AddResourcesParameters();
 
	ResourceItem[] resourcelist = new ResourceItem[N];
	for (int i = 1 ; i < (resourcelist.length +1); i++) {
		resourcelist[i-1] = new ResourceItem();
		resourcelist[i-1].setID("ID");
		resourcelist[i-1].setType("resource type");
	}
	ResourceList r = new ResourceList();
	r.setResource(resourcelist);
	add.setResources(r);
	add.setTargetScope(resources.getProperty("targetScope"));
	logger.info("Sending the Resource request....");
	String reportID = pt.addResources(add);
	logger.info("Returned report ID: " + reportID);
 
} catch (Exception e) {
	//manage the error
}

Request to remove a resource from the scope

Request to deploy a new service in the scope

To deploy a new service in the scope, the addResource operation has to be invoked passing a list of ServiceItem (one per each service one wants to deploy). Beside joining the service's resource itself, this action has two side effects on the scope's realm:

the target gHN is automatically added the scope, if the scope is of VRE type
the newly generated instances of the service hosted on the target gHN automatically joins the scope at start up time

import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.AddResourcesParameters;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ServiceItem;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ServiceList;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ResourceManagerPortType;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.service.ResourceManagerServiceAddressingLocator;
 
 
EndpointReferenceType endpoint = new EndpointReferenceType();
 
try {
	endpoint.setAddress(new Address("http://"+ args[0]+":"+ args[1] +"/wsrf/services/gcube/vremanagement/ResourceManager"));
	ResourceManagerPortType pt;
 
	GCUBESecurityManagerImpl managerSec = new GCUBESecurityManagerImpl() {  
			public boolean isSecurityEnabled() {return false;}};
	pt = GCUBERemotePortTypeContext.getProxy(new ResourceManagerServiceAddressingLocator().getResourceManagerPortTypePort(endpoint), 
					GCUBEScope.getScope("callerScope"),managerSec);
 
	AddResourcesParameters add = new AddResourcesParameters();
 
	// prepare the list of service to ask to deploy
	ServiceItem[] servicelist = new ServiceItem[N];
	for (int i = 1 ; i < (servicelist.length +1); i++) {
		servicelist[i-1] = new ServiceItem();
		servicelist[i-1].setServiceClass("SERVICECLASS");
		servicelist[i-1].setServiceName("SERVICENAME");
		servicelist[i-1].setServiceVersion("SERVICEVERSION");
		servicelist[i-1].setGHN("GHNID"));//suggested gHN
	}
 
	ServiceList servicesToDeploy = new ServiceList();
	servicesToDeploy.setService(servicelist);
	//set the set of GHN, if any
	String[] ghns = new String[] {"GHNID1","GHNID2","GHNID3","GHNID4","GHNID5"};
	servicesToDeploy.setGHN(ghns);
 
	add.setServices(servicesToDeploy);
 
        add.setTargetScope(resources.getProperty("targetScope"));
	System.out.println ("Sending the Resource request....");
	String reportID = pt.addResources(add);
	System.out.println ("Returned report ID: " + reportID);
 
} catch (Exception e) {
	e.printStackTrace();
}

Request to remove a Running Instance from the scope

There are two ways to obtain the removal of a Running Instance from a scope:

request the removal of th RI from the scope, in this case the RI is removed from the scope and, IF this was the only scope to which it was joined to, the RI is also undeployed from the target gHN.
request the undeployment of the related service from the scope, this will force all the RIs of the service to be undeployed from the scope, regardless any other condition (e.g. sharing with other scopes);

Whether way is used, the undeployment works only if the Running Instance was originally deployed within this scope (and not joined from another one), i.e. by the Resource Manager instance that is contacted.

Removing a Running Instance from the scope

This is the recommended way to proceed in the light of maintaining consistencies among the scopes. The RI is undeployed only if it is not shared across multiple scopes.

import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.RemoveResourcesParameters;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ResourceItem;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ResourceList;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ResourceManagerPortType;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.service.ResourceManagerServiceAddressingLocator;
import org.gcube.common.core.resources.GCUBERunningInstance;
 
 
EndpointReferenceType endpoint = new EndpointReferenceType();
 
try {
	//create the port-type
	endpoint.setAddress(new Address("http://"+  <ResourceManager host>+":"+ <ResourceManager port> +"/wsrf/services/gcube/vremanagement/ResourceManager"));
	GCUBESecurityManagerImpl managerSec = new GCUBESecurityManagerImpl() {  
	public boolean isSecurityEnabled() {return false;}};
	ResourceManagerPortType pt = GCUBERemotePortTypeContext.getProxy(new ResourceManagerServiceAddressingLocator().getResourceManagerPortTypePort(endpoint), 
		GCUBEScope.getScope(<caller Scope>),managerSec);
 
	//prepare the parameters
	RemoveResourcesParameters params = new RemoveResourcesParameters();
	ResourceItem[] resourcelist = new ResourceItem[1];
	resourcelist[0] = new ResourceItem();
	resourcelist[0].setID(<ID>);
	resourcelist[0].setType(GCUBERunningInstance.TYPE);
	ResourceList r = new ResourceList();
	r.setResource(resourcelist);
	params.setResources(r);
	params.setTargetScope("<target Scope>");
 
	//sending the request
	logger.info ("Sending the Remove Resource request....");
	String reportID = pt.removeResources(params);
	logger.info ("Returned report ID: " + reportID);
 
} catch (Exception e) {
        logger.error ("Failed to remove the RI ", e);
}

This is a sample report returned by invoking the RemoveResources with one or more RIs that was then undeployed.

Undeploy a Service from the scope

By requesting the undeployment of a service, all the instances of that service belonging the current scope will be undeployed. If they belong to multiple scopes, they will result as LOST resources there.

import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.RemoveResourcesParameters;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ResourceManagerPortType;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ServiceItem;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ServiceList;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.service.ResourceManagerServiceAddressingLocator;
 
EndpointReferenceType endpoint = new EndpointReferenceType();
 
try {
	//create the port-type
	endpoint.setAddress(new Address("http://"+  <ResourceManager host>+":"+ <ResourceManager port> +"/wsrf/services/gcube/vremanagement/ResourceManager"));
	GCUBESecurityManagerImpl managerSec = new GCUBESecurityManagerImpl() {  
	public boolean isSecurityEnabled() {return false;}};
	ResourceManagerPortType pt = GCUBERemotePortTypeContext.getProxy(new ResourceManagerServiceAddressingLocator().getResourceManagerPortTypePort(endpoint), 
					GCUBEScope.getScope(<caller scope>),managerSec);
	//prepare the parameters
	RemoveResourcesParameters params = new RemoveResourcesParameters();
	ServiceItem[] servicelist = new ServiceItem[1];
	servicelist[0] = new ServiceItem();
	servicelist[0].setServiceClass(<Service Class>);
	servicelist[0].setServiceName(<Service Name>);
	servicelist[0].setServiceVersion(<Service Version>);
	ServiceList l = new ServiceList();
	l.setService(servicelist);
	params.setServices(l);	
	params.setTargetScope(<target Scope>);
 
	//sending the request
	logger.info ("Sending the Remove Resource request....");
	String reportID = pt.removeResources(params);
	logger.info ("Returned report ID: " + reportID);
 
 
} catch (Exception e) {
	logger.error("Failed to undeploy the Service",e);
}

Obtaining the report of the activities performed

To retrieve a Resource Report of the activities performed by any operation, the getReport operation has to be invoked by passing as parameter the report identifier returned by the related operation.

import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.ResourceManagerPortType;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.service.ResourceManagerServiceAddressingLocator;
 
EndpointReferenceType endpoint = new EndpointReferenceType();
 
try {
	endpoint.setAddress(new Address("http://"+  <ResourceManager host>+":"+ <ResourceManager port> +"/wsrf/services/gcube/vremanagement/ResourceManager"));
 
	GCUBESecurityManagerImpl managerSec = new GCUBESecurityManagerImpl() {  
		public boolean isSecurityEnabled() {return false;}};
 
	ResourceManagerPortType pt = GCUBERemotePortTypeContext.getProxy(new ResourceManagerServiceAddressingLocator().getResourceManagerPortTypePort(endpoint), 
					GCUBEScope.getScope(<SCOPE>), 90000, managerSec);
 
	String report = pt.getReport(<REPORT ID>);
	logger.debug("REPORT");
	logger.debug("************");
	logger.debug(report);
	logger.debug("************");
} catch (Exception e) {
	e.printStackTrace();
}

Scope Resource

The Scope Resource is a view on the current composition of the Scope context published in the IS. Each time the Scope is modified, the view is updated.

[Describe the structure here]

Here is an example of Scope Resource published in the IS:

On Resource Brokering

When a new service is requested to be added to the scope an appropriate node must be selected in order to host a new instance of the service. This work is performed by a Resource Broker in charge of matching available resources with caller requests and service's requirement and considering infrastructure information statuses during matchmaking.

The Resource Manager service defines a simple interface to which any broker must comply to:

 
package org.gcube.vremanagement.resourcemanager.impl.brokerage;
 
import java.util.Set;
import org.gcube.common.core.scope.GCUBEScope;
import org.gcube.vremanagement.resourcemanager.impl.resources.ScopedDeployedService;
 
/**
 * 
 * Models the expected behavior of a Broker. 
 * A Broker is an entity that can create Deployment Plans for a set of {@link ScopedDeployedService}s to be deployed
 *
 * @author Manuele Simi (ISTI-CNR)
 *
 */
public interface Broker {
 
	/**
	 * Initializes the Broker instance
	 * @param scope the scope where the broker will act
	 * @throws Exception if the initialization fails (depends on the broker's nature)
	 */
	public void initialize(GCUBEScope scope) throws Exception;
 
	/**
	 * Creates a deployment plan
	 * @param services the services to deploy
	 * @param suggestedGHNs the GHNs suggested for the deployment, if specified, only these nodes will be used by the planner
	 * @throws Exception if the preparation of the plan fails
	 */
	public void makePlan(Set<ScopedDeployedService> services, String[] suggestedGHNs) throws Exception;
 
	/**
	 * Sends feedback to the broker about the execution of the plan
	 * @param services the services belonging the plan
	 * @throws Exception if the broker is not able to manage the feedback
	 */
	public void sendFeedback(Set<ScopedDeployedService> services) throws Exception;
 
}

Then, the RM instance can be configured to dynamically load and exploit any class implementing this interface. This configuration setting is done via the resourceBrokerClass parameter in the service's JNDI:

 
<service name="gcube/vremanagement/ResourceManager/service">
 
	 <environment 
		name="resourceBrokerClass" 
	 	value="..."
	 	type="java.lang.String"
	 	override="false" />
 
</service>

The current version of the RM is shipped with 2 Broker implementation descibed in the following subsections.

The Internal Broker

To enable this broker, the value of the resourceBrokerClass parameter has to be set to org.gcube.vremanagement.resourcemanager.impl.brokerage.InternalBroker. This broker is a very simple one and it has to be used when no other broker is available. In fact, the internal broker performs a minimal work and it's not capable to matchmake nodes and services. It can only manage requests where there is a list of suggested gHN per each service or per request. In this case, it checks the availability of those gHNs and then prepare the plan according to the caller suggestion (when a gHN is suggested for a specific service) or a simple round robin algorithm (when a list of gHN is suggested for the entire request).

The Service Broker

To enable this broker, the value of the resourceBrokerClass parameter has to be set to org.gcube.vremanagement.resourcemanager.impl.brokerage.ServiceBroker. This broker is in reality an interface over the Resource Broker service and can be used when an instance of such a service is available in the VO.

By default, the Resource Manager is configured to use this broker.

Broker's Report

If the deployment operations reach the broker's stage, a dedicated section of the Resource Report shows the result of the query to the selected broker.

The following report shows the case where two services were requested to be added, they did, but no running instance was activated for them, since the broker didn't match the user's request with the actual state of the infrastructure. In particular, here the caller asked to deploy on a specific gHN which turned out to do not exist.

<?xml version="1.0" encoding="UTF-8"?>
<ResourceReport>
  <ID>97d510b0-674d-11df-94fa-c1efdde5fc2d</ID>
  <Operation>AddResources</Operation>
  <LastUpdate>2010-05-24T17:01:26+01:00</LastUpdate>
  <TargetScope>/gcube/devsec/CNR-VRE</TargetScope>
  <Resources>
    <Resource>
      <ID>25bfb640-04e0-11de-b58c-828df53a36e6</ID>
      <Type>Service</Type>
      <Status>ADDED</Status>
    </Resource>
    <Resource>
      <ID>26774670-04e0-11de-b58c-828df53a36e6</ID>
      <Type>Service</Type>
      <Status>ADDED</Status>
    </Resource>
  </Resources>
  <DeploymentPlanCreation>
    <Status>FAILED</Status>
    <Message>The creation of the Deployment Plan failed. Broker says: The GHNs required in the plan request are not available. Suggested GHN: f9f5dfa0-674c-11df-b094-a90b5bbcf085</Message>
  </DeploymentPlanCreation>
  <Services>
    <Service>
      <ID>25bfb640-04e0-11de-b58c-828df53a36e6</ID>
      <Class>MetadataManagement</Class>
      <Name>MetadataManager</Name>
      <Version>1.00.00</Version>
      <DeploymentActivity>The report is still not available for this service</DeploymentActivity>
    </Service>
    <Service>
      <ID>26774670-04e0-11de-b58c-828df53a36e6</ID>
      <Class>MetadataManagement</Class>
      <Name>XMLIndexer</Name>
      <Version>1.00.00</Version>
      <DeploymentActivity>The report is still not available for this service</DeploymentActivity>
    </Service>
  </Services>
</ResourceReport>

Here it is the report for the same services when everything goes OK:

<?xml version="1.0" encoding="UTF-8"?>
<ResourceReport>
  <ID>a771bad0-674f-11df-94fa-c1efdde5fc2d</ID>
  <Operation>AddResources</Operation>
  <LastUpdate>2010-05-24T17:16:10+01:00</LastUpdate>
  <TargetScope>/gcube/devsec/CNR-VRE</TargetScope>
  <Resources>
    <Resource>
      <ID>25bfb640-04e0-11de-b58c-828df53a36e6</ID>
      <Type>Service</Type>
      <Status>ADDED</Status>
    </Resource>
    <Resource>
      <ID>26774670-04e0-11de-b58c-828df53a36e6</ID>
      <Type>Service</Type>
      <Status>ADDED</Status>
    </Resource>   
  </Resources>
  <DeploymentPlanCreation>
    <Status>SUCCESS</Status>
    <Message>The Deployment Plan was successfully created</Message>
  </DeploymentPlanCreation>
  <Services>
  <!-- services' section here-->
 
  </Services>
</ResourceReport>

Test-suite

The Resource Manager comes with a test-suite package allowing to test its functionalities.The test-suite is completely independent and does not require any other gCube package, except than a local gCore installation. The package is composed by a set of classes, sample configuration files and scripts ready to be executed.

|-lib
|--org.gcube.vremanagement.resourcemanager.testsuite.jar
|
|-samples
|---...
|
|-addresource.sh
|-getreport.sh
|-changescopeoptions.sh
|-...

Each script allows to test a different service's operation or group of operations logically related. In the following, an explanation of each script and its usage is provided.

Add Resource(s)

The AddResources script invokes the AddResources operation to add new GCUBEResources on a scope. It must be executed as follows

./addresource.sh <ResourceManager host> <ResourceManager port> <configuration file>

Clearly, the Resource Manager host and port are the information needed to contact the Resource Manager instance, while the configuration file reports the information about the resources to add or create and the scope in which they have to be managed.

This is an example of configuration file requesting to add 4 gCube Resources to the Scope /gcube/devsec/CNR-VRE:

#global properties
 
numOfServicesToAdd=0
numOfResourcesToAdd=4
targetScope=/gcube/devsec/CNR-VRE
callerScope=/gcube/devsec/CNR-VRE
 
# resources' list
resource.1.id=9ad3ce60-27aa-11df-9df7-e0bf8afcd20f
resource.1.type=Collection
resource.2.id=6e6e2ce0-1ca5-11df-a0fe-c8f6680864ab
resource.2.type=RunningInstance
resource.3.id=a5db71f0-27aa-11df-8ee2-c1a4489ea34b
resource.3.type=MetadataCollection
resource.4.id=df899630-82ae-11de-a6cc-f15c9cf886be
resource.4.type=GenericResource

If everything works, the following report is produced (it can be retrieved via the getReport script):

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ResourceReport>
  <ID>bdffb830-33a4-11df-97f6-8d3129726fea</ID>
  <LastUpdate>2010-03-19T18:14:10-04:00</LastUpdate>
  <TargetScope>/gcube/devsec/CNR-VRE</TargetScope>
  <Resources>
    <Resource>
      <ID>a5db71f0-27aa-11df-8ee2-c1a4489ea34b</ID>
      <Type>MetadataCollection</Type>
      <Status>ADDED</Status>
      <ErrorDescription>-</ErrorDescription>
    </Resource>
    <Resource>
      <ID>6e6e2ce0-1ca5-11df-a0fe-c8f6680864ab</ID>
      <Type>RunningInstance</Type>
      <Status>ADDED</Status>
      <ErrorDescription>-</ErrorDescription>
    </Resource>
    <Resource>
      <ID>df899630-82ae-11de-a6cc-f15c9cf886be</ID>
      <Type>GenericResource</Type>
      <Status>ADDED</Status>
      <ErrorDescription>-</ErrorDescription>
    </Resource>
    <Resource>
      <ID>9ad3ce60-27aa-11df-9df7-e0bf8afcd20f</ID>
      <Type>Collection</Type>
      <Status>ADDED</Status>
      <ErrorDescription>-</ErrorDescription>
    </Resource>
  </Resources>
  <Services/>
</ResourceReport>

Remove Resource(s)

The RemoveResources script invokes the RemoveResources operation to remove GCUBEResources from a scope. It must be executed as follows

./removeresource.sh <ResourceManager host> <ResourceManager port> <configuration file>

This is an example of configuration file requesting to remove a Collection from the Scope /gcube/devsec/CNR-VRE:

#global properties
numOfServicesToRemove=0
numOfResourcesToRemove=1
targetScope=/gcube/devsec/CNR-VRE
callerScope=/gcube/devsec/CNR-VRE
 
# resources' list
resource.1.id=9ad3ce60-27aa-11df-9df7-e0bf8afcd20f
resource.1.type=Collection

If everything works, the following report is produced (it can be retrieved via the getReport script):

<?xml version="1.0" encoding="UTF-8"?>
<ResourceReport>
  <ID>cfb41e90-3863-11df-9bfb-cfdb92619067</ID>
  <LastUpdate>2010-03-26T00:11:58+01:00</LastUpdate>
  <TargetScope>/gcube/devsec/CNR-VRE</TargetScope>
  <Resources>
    <Resource>
      <ID>9ad3ce60-27aa-11df-9df7-e0bf8afcd20f</ID>
      <Type>Collection</Type>
      <Status>REMOVED</Status>
    </Resource>
  </Resources>
  <Services/>
</ResourceReport>

Create a new Scope

The CreateScope script invokes the CreateScope operation in order to initialise a new Scope and assign to it the first set of gCube resources. It must be executed as follows

./createScope.sh <ResourceManager host> <ResourceManager port> <configuration file>

This is an example of configuration file requesting to

create a new VRE (named /gcube/devsec/CNR-VRE)with the indicated options
add to it 4 existing gCube Resources
deploy 2 new Services in the new Scope

######################################
# Sample VRE creation - properties file
######################################
 
targetScope=/gcube/devsec/CNR-VRE
callerScope=/gcube/devsec/CNR-VRE
 
#Scope properties
creator=CN=Manuele Simi, L=NMIS-ISTI, OU=Personal Certificate, O=INFN, C=IT
designer=CN=Manuele Simi, L=NMIS-ISTI, OU=Personal Certificate, O=INFN, C=IT
securityenabled=false
displayName=CNR-VRE
description=Sample VRE for testing purposes
 
#Scope resources
numOfServicesToAdd=2
numOfResourcesToAdd=4
 
# resources' list
resource.1.id=9ad3ce60-27aa-11df-9df7-e0bf8afcd20f
resource.1.type=Collection
resource.2.id=6e6e2ce0-1ca5-11df-a0fe-c8f6680864ab
resource.2.type=RunningInstance
resource.3.id=a5db71f0-27aa-11df-8ee2-c1a4489ea34b
resource.3.type=MetadataCollection
resource.4.id=df899630-82ae-11de-a6cc-f15c9cf886be
resource.4.type=GenericResource
 
# services' list
service.1.name=MetadataManager
service.1.class=MetadataManagement
service.1.version=1.00.00
service.2.name=XMLIndexer
service.2.class=MetadataManagement
service.2.version=1.00.00
 
#GHN IDs on which the service will be deployed
GHNSet=84c3aa10-3857-11df-b245-cbf0ea5a1843,05921df0-382d-11df-8c8f-f5ea2a36aefa

Dispose an existing Scope

[TDP]

Get Report

The GetReport script invokes the GetReport operation to retrieve a Resource Report. It must be executed as follows

./getreport.sh <ResourceManager host> <ResourceManager port> <caller scope> <report ID>

This request will print out the resource report related to the given report ID.

Difference between revisions of "Resource Manager"

Revision as of 00:25, 29 May 2010

Contents

Design

Service architecture and code design

Concept, role and design of ScopedResource

Scope state

Observer pattern strategy

Interface

Resource Report

Sample Usage

Creating a new scope

Set/change the scope options

Request to join a resource to the scope

Request to remove a resource from the scope

Request to deploy a new service in the scope

Request to remove a Running Instance from the scope

Obtaining the report of the activities performed

Scope Resource

On Resource Brokering

The Internal Broker

The Service Broker

Broker's Report

Test-suite

Add Resource(s)

Remove Resource(s)

Create a new Scope

Dispose an existing Scope

Get Report

Navigation menu

Views

Personal tools

gCube Wiki

gCube features

gCube documentation

Integration and Distribution

Search

Tools