Difference between revisions of "Resource Manager"

From Gcube Wiki
Jump to: navigation, search
(→‎Concept, role and design of ScopedResource)
 
(158 intermediate revisions by one other user not shown)
Line 1: Line 1:
−
The ResourceManager service is in charge of managing gCube Scope contexts. An instance (and only one) of the service must be deployed for each scope (at any level, i.e. infrastructure, VO, VRE) in order to handle ''gCube Resources'' within the assigned scope.
+
The Resource Manager service is in charge of managing gCube Scope contexts. An instance (and only one) of the service must be deployed for each scope (at any level, i.e. infrastructure, VO, VRE) in order to handle ''gCube Resources'' within the assigned scope. It manages requests of adding and removing gCube Resource from the assigned Scope by keeping, at the same time, a consistent state in the whole infrastructure.
−
 
+
A [[Resource_Manager#Scope_Resource|Scope Resource]] is published in the [[gCore Based Information System]] with constantly updated information about the current composition of the Scope.
−
...
+
  
 
==== Design ====
 
==== Design ====
  
−
===== Code design =====
+
===== Service architecture and code design =====
 +
 
 +
The service adopts the singleton WSRF pattern with a single stateful resource holding the state of the instance. All the code is designed around the concept of ''Scope'' and its related state. The operations exposed by the service allow to change the state a Scope managed by the instance by adding and removing gCube resources and internal triggers take subsequent actions to manage these changes.
 +
 
 +
The public interface of the service is organized as follows:
 +
 
 +
[[File:ResourceManagerPT.png|frame|center|Figure 1. Resource Manager - PortTypes]]
 +
 
 +
* <code>ScopeController</code> portType offers create, read, update and delete (CRUD) operations for managing scopes
 +
* <code>ResourceBinder</code> portType allows to add/remove resources to a managed scope
 +
* <code>Reporting</code> porType receives and returns reports on the operations performed on managed resources
 +
 
 +
The following class diagram shows a simplified view of the organization and structure of the Resource Manager's source code.
  
−
[[File:ResourceManagerArchitecture.png|frame|center|Resource Manager Class Diagram]]
+
[[File:ResourceManagerArchitecture.png|frame|center|Figure 2. Resource Manager - Class Diagram]]
  
−
[TBD]
+
==== Internal State ====
  
 
===== Concept, role and design of ''ScopedResource'' =====
 
===== Concept, role and design of ''ScopedResource'' =====
  
−
A Resource Manager instance handles the Scope it is managing as a set of ''Scoped Resources''. A Scoped Resource models a gCube Resource in a specific Scope. At the heart of the Scope management activities there is the management of the state of the Scoped Resources belonging the Scope.
+
A Resource Manager instance handles a Scope as a set of ''Scoped Resources''. A Scoped Resource models a gCube Resource in a specific Scope. At the heart of the Scope management activities there is the management of the state of the Scoped Resources belonging the Scope.
  
−
The following diagram depicts the possible Scope Resource's statuses and the legal transitions among them. The starting point of the life of a Scope Resource is always the request to join an existing gCube Resource to the Scope.
+
The following diagram depicts the Scope Resource's statuses and the legal transitions among them. The starting point of the life of a Scope Resource is always the request to join an existing gCube Resource to the Scope.
  
  
−
[[File:ScopedResourceLifecycle.png|frame|center|Scoped Resource life cycle]]
+
[[File:ScopedResourceLifecycle.png|frame|center|Figure 3. Scoped Resource - life cycle]]
  
−
The crucial statuses are ''ADDED'' and ''REMOVED'', where the concrete (and resource specific) actions to join the gCube Resource to the Scope are performed. For instance, in case of a Service Resource, a new instance of the service is deployed when managing the ''ADDED'' status, while undeployment is performed in the ''REMOVED'' status.
+
The most important statuses are ''ADDED'' and ''REMOVED'', where the concrete (and resource specific) actions to join/dis-join the gCube Resource to the Scope are performed. For instance, in case of a Service Resource, a new instance of the service is deployed when managing the ''ADDED'' status, while undeployment is performed in the ''REMOVED'' status.
  
−
''[TBC]''
+
===== Instance State =====
  
−
===== Other points to expand =====
+
[[File:RM2-state.png|frame|center|Figure 4. Resource Manager - Instance State]]
  
−
* Scope state management versus old VRE state management
+
===== Scope State =====
−
* Raw state separated from the state behavior to avoid mis-serializations due to any future change in the service implementation
+
−
* Observer pattern-based internal strategy for managing the scope state
+
−
* Local [http://xstream.codehaus.org/ XStream] serialization
+
−
* Synchronization between local and remote (i.e. stored in the IS) state and vice versa
+
−
* ...
+
  
−
==== Interface  ====
+
[[File:RM2-Scope-state.png|frame|center|Figure 5. Resource Manager - Scope State]]
−
The service exposes a single <code>ResourceManager</code> portType ...
+
  
 +
===== Scope state serialization =====
  
 +
The Scope state is formed by a set of Scoped Resources. Such a state is persisted in two places:
 +
* locally, via an [http://xstream.codehaus.org/ XStream] serialization of the objects handling the state
 +
* remotely, on the IS as ScopedResource
  
 +
There is a continuous synchronization between the two. To be more precise , at start up time, the Resource Manager instance looks for its serialized state locally, if it does not found the information, it tries to harvest its state from the IS. In case even the IS does not return a valid state, a new one is created. Then, each time the local state changes, the remote one is updated too.
  
−
===== Resource Report =====
+
The remote serialization contains less information with respect to the local one for internal and logical reasons. Nevertheless, this doubled state allows to partially recovers the state of the Scope whenever the RM instance is moved from one gHN to another and if the state of the gHN is wrongly sweep out.  
−
After each resource management request, a report is produced and disseminated (via the <code>getReport</code> operation).
+
  
−
For each service, information about the service's dependencies resolution and package deployment are reported.  
+
Lastly, it's important to notice that the current implementation is not capable of "recreating" from scratch the Scope state in case both of its serialization are lost. Therefore, if an instance of a service is undeployed by a human or by other means, it is not removed from the Scope state. The same is for the manipulation of any other gCube resources within the Scope.
  
−
Possible dependencies states are:
+
===== Observer pattern strategy =====
 +
 
 +
The service adopts an observer pattern-based internal strategy for managing the scope state. Each time the state of a Scoped Resource changes, registered ''Scope Observers'' are properly notified in order to manage the new state.
 +
 
 +
An observer is a class that extends the abstract class <code>org.gcube.vremanagement.resourcemanager.impl.state.observers.ScopeObserver</code>. The current version of the Resource Manager includes the following 3 Scope Observers:
 +
 
 +
* <code>Executor</code> (<code>org.gcube.vremanagement.resourcemanager.impl.state.observers.Executor</code>): each time a resource is in a ADDREQUEST or REMOVEREQUEST status, the Executor observer adds or removes the resource to/from the state;
 +
 
 +
* <code>Publisher</code> (<code>org.gcube.vremanagement.resourcemanager.impl.state.observers.Publisher</code>): each time a resource is in added or removed to/from the state, the Publisher observer synchronizes the local state with the remote state published in the IS;
 +
 
 +
* <code>Serializer</code> (<code>org.gcube.vremanagement.resourcemanager.impl.state.observers.Serializer</code>): each time a change occurs in a resource, the Serializer observer performs a serialization of the state on the local file system.
 +
 
 +
==== Interface  ====
 +
===== Resource Report =====
 +
All the operations performed by the Resource Manager work in an asynchronous manner with respect to their invocation. They return an ID that can be used to query the [[Resource_Manager#Obtaining_the_report_of_the_activities_performed | getReport]] operation in order to retrieve a detailed report of the ongoing or completed activities.
 +
 
 +
The basic structure of the report is:
 +
 
 +
<source lang="xml">
 +
<ResourceReport>
 +
  <ID><!--report ID --></ID>
 +
  <Status>OPEN/CLOSED</Status>
 +
  <Operation>Create/AddResources/UpdateResources/RemoveResources/Dispose</Operation>
 +
  <LastUpdate><!--last update time --></LastUpdate>
 +
  <TargetScope><!-- target scope of the operations reported in the following section --></TargetScope>
 +
  <Resources>
 +
    <!-- list of gCube resources involved and their actual status--> 
 +
    <Resource>
 +
      <ID>..</ID>
 +
      <Type>...</Type>
 +
      <Status>ADDED/REMOVED/FAILED</Status>
 +
      <ErrorDescription>..</ErrorDescription> <!--only if status=FAILED -->
 +
    </Resource>
 +
    <Resource>
 +
      <ID>..</ID>
 +
      <Type>...</Type>
 +
      <Status>ADDED/REMOVED/FAILED</Status>
 +
      <ErrorDescription>..</ErrorDescription> <!--only if status=FAILED -->
 +
    </Resource>
 +
  </Resources>
 +
  <DeploymentPlanCreation>
 +
      <!-- outcomes of the deployment plan creation -->
 +
      <Status>SUCCESS/FAILED</Status>
 +
      <Message></Message>
 +
  </DeploymentPlanCreation>
 +
  <Software>
 +
  <!-- per each software involved in the request, a section like the following-->
 +
    <Service>
 +
      <ID>...</ID>
 +
      <Class>...</Class>
 +
      <Name>...</Name>
 +
      <Version>...</Version>
 +
      <PackageName>...</PackageName>
 +
      <PackageVersion>...</PackageVersion>
 +
      <DependenciesResolutionStatus>SUCCESS/FAILED</DependenciesResolutionStatus>
 +
      <DeployedOn>...</DeployedOn> <!- GHN ID-->
 +
      <ErrorDescription>-</ErrorDescription>
 +
      <DependenciesResolution>
 +
        <!-- information about service's dependencies (missed, resolved))-->
 +
        <ResolvedDependencies/>
 +
        <MissingDependencies/>
 +
      </DependenciesResolution
 +
    <DependenciesResolution>       
 +
    <DeploymentActivity> or <UndeploymentActivity>
 +
        <!-- information about (un)deployment activities on the target gHN-->
 +
    </DeploymentActivity> or </UndeploymentActivity>
 +
    <Service>
 +
  </Software>
 +
</ResourceReport>
 +
</source>
 +
 
 +
The first elements contain general information about the report itself. The most important one is the '''Status''' element which states whether the activities have been concluded or no. The two possible values here are ''OPEN'' or ''CLOSED''.
 +
 
 +
The '''Software'''' section is reported when that the activities performed involve the deployment or undeployment of software. For each software, information about dependencies resolution (if any) and package deployment/undeployment are reported.
 +
 
 +
Possible dependencies' statuses are:
  
 
* ''SUCCESS'': all the dependencies have been successfully resolved
 
* ''SUCCESS'': all the dependencies have been successfully resolved
 
* ''FAILED'': some dependencies are missing
 
* ''FAILED'': some dependencies are missing
  
−
Periodically, the Deployer service sends a report on the deployment activity (performed asynchronously with respect to the VREManager request). Such report has a state:
+
Periodically, the Deployer service sends a report on the deployment activity (performed asynchronously with respect to the Resource Manager request). Such report has a status:
  
 
* ''OPEN'': the activity is still ongoing, other reports will follow
 
* ''OPEN'': the activity is still ongoing, other reports will follow
 
* ''CLOSED'' : the activity is closed, no further report will be sent
 
* ''CLOSED'' : the activity is closed, no further report will be sent
  
−
Possible package states are:
+
Possible package statuses are:
  
 
* ''WAITING'': request is still pending
 
* ''WAITING'': request is still pending
Line 65: Line 149:
 
* ''RUNNING'': the related instance is running
 
* ''RUNNING'': the related instance is running
  
−
Possible resource states:
+
Possible resource statuses:
  
 
* ''ADDED'': the resource was successfully added to the scope
 
* ''ADDED'': the resource was successfully added to the scope
Line 71: Line 155:
 
* ''FAILED'': an error occurred and the resource has not been managed
 
* ''FAILED'': an error occurred and the resource has not been managed
  
−
This is a sample report showing the results of a request including the add of a service, a generic resource, a collection and an existing running instance to the managed scope:
+
This is a [[ResourceReport.xml|sample report]] showing the results of a request to add a bunch of software to a managed scope.
 +
 
 +
===== Interacting with the Resource Manager =====
 +
 
 +
All the operations described below are exposed by the <code>ResourceManager</code> portType. Most of them make a distinction between the services and all the rest of the gCube resources: this is because managing services often requires different kind of parameters and produces a most complex feedback.
 +
 
 +
======  Creating a new scope ======
 +
 
 +
To create a new scope, the <code>CreateScope</code> operation of the ScopeController portType has to be invoked.
 +
To be successful in the creation, the following preconditions must be valid:
 +
 
 +
* in case of VO scope, the gHN hosting the Resource Manager instance must already have locally the Service Map of the VO;
 +
* in case of VRE scope,  the gHN hosting the Resource Manager instance must already have locally the Service Map of the parent's VO;
 +
* the Resource Manager instance must already join at least the parent scope.
 +
 
 +
Optionally, a list of scope options can be passed (explained in the next section).
 +
 
 +
Here it is a sample usage of the operation:
 +
 
 +
<source lang="java">
 +
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.CreateScopeParameters;
 +
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.OptionsParameters;
 +
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.ScopeControllerPortType;
 +
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.ScopeOption;
 +
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.service.ScopeControllerServiceAddressingLocator;
 +
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.service.ResourceManagerServiceAddressingLocator;
 +
 
 +
protected static String[] optionNames = new String[] {"creator", "designer", "endTime", "startTime",
 +
"description", "displayName", "securityenabled"};
 +
 +
 
 +
EndpointReferenceType endpoint = new EndpointReferenceType();
 +
 +
try {
 +
endpoint.setAddress(new Address("http://" + args[0] + "/wsrf/services/gcube/vremanagement/resourcemanager/scopecontroller"));
 +
ScopeControllerPortType pt = new ScopeControllerServiceAddressingLocator().getScopeControllerPortTypePort(endpoint);
 +
pt = GCUBERemotePortTypeContext.getProxy(pt, GCUBEScope.getScope(resources.getProperty("callerScope").trim()), 90000);
 +
 +
logger.info("Setting the Scope parameters...");
 +
OptionsParameters options = new OptionsParameters();
 +
ScopeOption[] scopeOptionList = new ScopeOption[optionNames.length];
 +
for (int i=0; i < optionNames.length; i++)
 +
if (resources.getProperty(optionNames[i]) != null) {
 +
logger.info("Setting prop " +  optionNames[i]);
 +
scopeOptionList[i] = new ScopeOption(optionNames[i], resources.getProperty(optionNames[i]));
 +
}
 +
options.setScopeOptionList(scopeOptionList);
 +
 +
//ready to start...
 +
logger.info("Sending the creation request....");
 +
CreateScopeParameters params = new CreateScopeParameters();
 +
params.setTargetScope("my scope name");
 +
params.setOptionsParameters(options);
 +
pt.createScope(params);
 +
 
 +
} catch (Exception e) {
 +
logger.fatal("Failed to create the scope",e);
 +
}
 +
 
 +
 
 +
</source>
 +
 
 +
======  Set/change the scope options ======
 +
 
 +
A number of information related to the scope can be set by invoking the <code>ChangeScopeOptions</code> operation of the ScopeController portType. Actually, the following properties are accepted:
 +
* ''Designer'': the designer of the scope
 +
* ''Creator'': the creator of the scope
 +
* ''StartTime'': the activation time of the scope (accepted format is yyyy-MM-dd'T'HH:mm:ssZ, example: 2008-11-11T16:58:12+01:00)
 +
* ''EndTime'': the last time of the scope (accepted format is yyyy-MM-dd'T'HH:mm:ssZ, example: 2008-11-11T16:58:12+01:00)
 +
* ''Description'': a scope description
 +
* ''DisplayName'': a name to display in any GUI
 +
* ''SecurityEnabled'': a boolean flag to set up a secure or non-secure Scope
 +
 
 +
All of them are optionals and they can be set/updated any time, even if in some cases (like changing the start time once the scope is activated) this does not have any effect.
 +
 
 +
 
 +
The <code>ChangeScopeOptions</code> operation can be invoked as follows:
 +
 
 +
<source lang="java">
 +
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.OptionsParameters;
 +
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.ScopeControllerPortType;
 +
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.ScopeOption;
 +
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.service.ScopeControllerServiceAddressingLocator;
 +
 
 +
//...
 +
 
 +
String[] optionNames = new String[] {"creator", "designer", "endTime", "startTime",
 +
"description", "displayName", "securityenabled"};
 +
 
 +
//...
 +
EndpointReferenceType endpoint = new EndpointReferenceType();
 +
 +
try {
 +
endpoint.setAddress(new Address(args[0]));
 +
ScopeControllerPortType pt = new ScopeControllerServiceAddressingLocator().getScopeControllerPortTypePort(endpoint);
 +
pt = GCUBERemotePortTypeContext.getProxy(pt, GCUBEScope.getScope(resources.getProperty("callerScope").trim()), 90000);
 +
 +
OptionsParameters options = new OptionsParameters();
 +
ScopeOption[] scopeOptionList = new ScopeOption[optionNames.length];
 +
for (int i=0; i < optionNames.length; i++)
 +
scopeOptionList[i] = new ScopeOption(optionNames[i], resources.getProperty(optionNames[i]));
 +
 +
options.setScopeOptionList(scopeOptionList);
 +
pt.changeScopeOptions(options);
 +
 +
} catch (Exception e) {
 +
        // manage the error
 +
}
 +
</source>
 +
 
 +
======  Request to join a resource to the scope ======
 +
 
 +
A request to join a GCUBEResource to a scope is performed through the <code>AddResources</code> operation of the ResourceBinder portType. For each resource there are two main information to provide:
 +
* the resource identifier
 +
* the resource type 
 +
 
 +
plus the target scope of the operation.
 +
 
 +
The <code>AddResources</code> operation can be invoked as follows:
 +
 
 +
<source lang="java">
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.AddResourcesParameters;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.ResourceBinderPortType;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.ResourceItem;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.ResourceList;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.PackageItem;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.SoftwareList;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.service.ResourceBinderServiceAddressingLocator;
 +
 
 +
//...
 +
 
 +
EndpointReferenceType endpoint = new EndpointReferenceType();
 +
 +
try {
 +
endpoint.setAddress(new Address("http://" + args[0] + "/wsrf/services/gcube/vremanagement/resourcemanager/binder"));
 +
ResourceBinderPortType pt = new ResourceBinderServiceAddressingLocator().getResourceBinderPortTypePort(endpoint);
 +
pt = GCUBERemotePortTypeContext.getProxy(pt,  
 +
GCUBEScope.getScope(resources.getProperty("callerScope").trim()));
 +
 +
AddResourcesParameters add = new AddResourcesParameters();
 +
ResourceItem[] resourcelist = new ResourceItem[<#Resources>)];
 +
for (int i = 1 ; i < (resourcelist.length +1); i++) {
 +
resourcelist[i-1] = new ResourceItem();
 +
resourcelist[i-1].setID("<resource ID>");
 +
resourcelist[i-1].setType("<resource type>");
 +
}
 +
ResourceList r = new ResourceList();
 +
r.setResource(resourcelist);
 +
add.setResources(r);
 +
 +
add.setTargetScope("my scope");
 +
System.out.println ("Sending the Resource request....");
 +
String reportID = pt.addResources(add);
 +
System.out.println ("Returned report ID: " + reportID);
 +
 +
} catch (Exception e) {
 +
e.printStackTrace();
 +
}
 +
 
 +
</source>
 +
 
 +
======  Request to remove a resource from a scope ======
 +
 
 +
A request to remove a GCUBEResource from a scope is performed through the <code>RemoveResources</code> operation of the ResourceBinder portType. For each resource there are two main information to provide:
 +
* the resource identifier
 +
* the resource type 
 +
 
 +
plus the target scope of the operation.
 +
 
 +
The <code>RemoveResources</code> operation can be invoked as follows:
 +
<source lang="java">
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.RemoveResourcesParameters;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.ResourceBinderPortType;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.ResourceItem;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.ResourceList;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.service.ResourceBinderServiceAddressingLocator;
 +
 
 +
//...
 +
EndpointReferenceType endpoint = new EndpointReferenceType();
 +
try {
 +
endpoint.setAddress(new Address("http://" + args[0] + "/wsrf/services/gcube/vremanagement/resourcemanager/binder"));
 +
ResourceBinderPortType pt = new ResourceBinderServiceAddressingLocator().getResourceBinderPortTypePort(endpoint);
 +
pt = GCUBERemotePortTypeContext.getProxy(pt,
 +
    GCUBEScope.getScope("<Caller Scope>"));
 +
 +
RemoveResourcesParameters params = new RemoveResourcesParameters();
 +
ResourceItem[] resourcelist = new ResourceItem[#resource to remove];
 +
for (int i = 1 ; i < (resourcelist.length +1); i++) {
 +
resourcelist[i-1] = new ResourceItem();
 +
resourcelist[i-1].setID("<ID>");
 +
resourcelist[i-1].setType("<type>");
 +
}
 +
ResourceList r = new ResourceList();
 +
r.setResource(resourcelist);
 +
params.setResources(r);
 +
params.setTargetScope("<target Scope>"));
 +
System.out.println ("Sending the Remove Resource request....");
 +
String reportID = pt.removeResources(params);
 +
System.out.println ("Returned report ID: " + reportID);
 +
} catch (Exception e) {
 +
e.printStackTrace();
 +
}
 +
 
 +
</source>
 +
 
 +
======  Deploying a new software in a scope ======
 +
 
 +
To deploy a new software in the scope, the <code>addResource</code> operation of the Resource Binder portType has to be invoked passing a list of <code>PackageItem</code> (one per each service to deploy). Beside joining the service's resource itself, this action has two side effects on the scope's realm:
 +
* the target gHN is automatically added the scope, if the scope is of type VRE
 +
* the newly generated instances of the service hosted on the target gHN automatically joins the scope at start up time
 +
 
 +
<source lang="java">
 +
 
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.AddResourcesParameters;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.ResourceBinderPortType;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.PackageItem;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.SoftwareList;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.service.ResourceBinderServiceAddressingLocator;
 +
 
 +
//...
 +
 
 +
EndpointReferenceType endpoint = new EndpointReferenceType();
 +
 +
try {
 +
endpoint.setAddress(new Address("http://" + args[0] + "/wsrf/services/gcube/vremanagement/resourcemanager/binder"));
 +
ResourceBinderPortType pt = new ResourceBinderServiceAddressingLocator().getResourceBinderPortTypePort(endpoint);
 +
pt = GCUBERemotePortTypeContext.getProxy(pt,
 +
GCUBEScope.getScope("<caller scope>"));
 +
 +
AddResourcesParameters add = new AddResourcesParameters();
 +
 +
// prepare the list of service to ask to deploy
 +
PackageItem[] packagelist = new PackageItem[#numOfSoftwareToAdd];
 +
for (int i = 1 ; i < (packagelist.length +1); i++) {
 +
packagelist[i-1] = new PackageItem();
 +
packagelist[i-1].setServiceClass("<serviceclass>");
 +
packagelist[i-1].setServiceName("<servicename>");
 +
packagelist[i-1].setServiceVersion("<serviceversion>");
 +
packagelist[i-1].setPackageName("<packagename>");
 +
packagelist[i-1].setPackageVersion("<packageversion>");
 +
//suggest a ghn for the deployment
 +
if (resources.getProperty("software." + i + ".GHN") != null)
 +
packagelist[i-1].setTargetGHNName(resources.getProperty("software." + i + ".GHN"));
 +
}
 +
 +
SoftwareList l = new SoftwareList();
 +
l.setSoftware(packagelist);
 +
//set the suggested set of GHNs to use, if any
 +
String[] ghns = ...;
 +
l.setSuggestedTargetGHNNames(ghns);
 +
 +
add.setSoftware(l);
 +
add.setTargetScope(resources.getProperty("targetScope"));
 +
System.out.println ("Sending the Resource request....");
 +
String reportID = pt.addResources(add);
 +
System.out.println ("Returned report ID: " + reportID);
 +
 +
} catch (Exception e) {
 +
e.printStackTrace();
 +
}
 +
</source>
 +
 
 +
======  Removing a software from a scope ======
 +
The removal of a software from a scope is requested by invoking the <code>removeResources</code> operation of the ResourceBinder portType.
 +
By requesting the removal of a software, the software is:
 +
* undeployed from the specified gHN if it is running only in the target scope
 +
* removed from the target scope, if the software is joined to other scopes
 +
 
 +
Do note that the implicit rule of undeploying a software from a node when the related Running Instance is requested to be removed from a single scope does not apply anymore in RM2.x. The only way to undeploy a software is to list it in the SoftwareList parameter.
 +
 
 +
<source lang="java">
 +
 
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.RemoveResourcesParameters;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.ResourceBinderPortType;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.PackageItem;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.SoftwareList;
 +
import org.gcube.vremanagement.resourcemanager.stubs.binder.service.ResourceBinderServiceAddressingLocator;
 +
 
 +
//...
 +
EndpointReferenceType endpoint = new EndpointReferenceType();
 +
try {
 +
endpoint.setAddress(new Address("http://" + args[0] + "/wsrf/services/gcube/vremanagement/resourcemanager/binder"));
 +
ResourceBinderPortType pt = new ResourceBinderServiceAddressingLocator().getResourceBinderPortTypePort(endpoint);
 +
 +
pt = GCUBERemotePortTypeContext.getProxy(pt,
 +
GCUBEScope.getScope(resources.getProperty("callerScope").trim() ));
 +
 +
RemoveResourcesParameters params = new RemoveResourcesParameters();
 +
PackageItem[] packagelist = new PackageItem[#numOfSoftwareToRemove))];
 +
for (int i = 1 ; i < (packagelist.length +1); i++) {
 +
packagelist[i-1] = new PackageItem();
 +
packagelist[i-1].setServiceClass("<service  class>");
 +
packagelist[i-1].setServiceName("<service name>"));
 +
packagelist[i-1].setServiceVersion("<service version>");
 +
packagelist[i-1].setPackageName("<service packagename>");
 +
packagelist[i-1].setPackageVersion("<service packageversion>");
 +
packagelist[i-1].setTargetGHNName("<target ghnname>"));
 +
 
 +
}
 +
 +
SoftwareList l = new SoftwareList();
 +
l.setSoftware(packagelist);
 +
params.setSoftware(l);
 +
 +
params.setTargetScope(resources.getProperty("targetScope").trim());
 +
System.out.println ("Sending the Remove Resource request....");
 +
String reportID = pt.removeResources(params);
 +
System.out.println ("Returned report ID: " + reportID);
 +
} catch (Exception e) {
 +
e.printStackTrace();
 +
}
 +
 
 +
</source>
 +
 
 +
======  Disposing a scope ======
 +
A scope managed by an instance of the service can be disposed by invoking the <code>DisposeScope</code> operation of the ScopeController portType. This call will:
 +
 
 +
* remove all the resources from the scope
 +
* undeploy the services belonging only this scope
 +
* unpublish the Scope Resource
 +
* remove the RM instance from the scope
 +
 
 +
The following code shows how to invoke the <code>DisposeScope</code> operation:
 +
<source lang="java">
 +
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.ScopeControllerPortType;
 +
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.service.ScopeControllerServiceAddressingLocator;
 +
 
 +
//...
 +
 +
EndpointReferenceType endpoint = new EndpointReferenceType();
 +
try {
 +
endpoint.setAddress(new Address("http://" + args[0] + "/wsrf/services/gcube/vremanagement/resourcemanager/scopecontroller"));
 +
ScopeControllerPortType pt = new ScopeControllerServiceAddressingLocator().getScopeControllerPortTypePort(endpoint);
 +
pt = GCUBERemotePortTypeContext.getProxy(pt, GCUBEScope.getScope(args[1].trim()));
 +
logger.info("Sending the destroy request....");
 +
String reportID = pt.disposeScope(" scope to dispose ");
 +
logger.info("Returned report ID: " + reportID);
 +
} catch (Exception e) {
 +
logger.fatal("Failed to dispose the scope",e);
 +
}
 +
</source>
 +
 
 +
======  Obtaining the report of the activities performed ======
 +
To retrieve a [[Resource_Manager#Resource_Report|Resource Report]] of the activities performed by any operation, the <code>getReport</code> operation of the Reporting portType has to be invoked. The single paramenter of the operation is the report identifier returned by the related operation.
 +
 
 +
This is an example about how to invoke the operation:
 +
 
 +
<source lang="java">
 +
import org.gcube.vremanagement.resourcemanager.stubs.reporting.ReportingPortType;
 +
import org.gcube.vremanagement.resourcemanager.stubs.reporting.service.ReportingServiceAddressingLocator;
 +
 
 +
//...
 +
EndpointReferenceType endpoint = new EndpointReferenceType();
 +
 +
try {
 +
endpoint.setAddress(new Address("http://" + args[0] + "/wsrf/services/gcube/vremanagement/resourcemanager/reporting"));
 +
ReportingPortType pt = new ReportingServiceAddressingLocator().getReportingPortTypePort(endpoint);
 +
pt = GCUBERemotePortTypeContext.getProxy(pt, GCUBEScope.getScope(args[1].trim()));
 +
String report = pt.getReport(args[2]);
 +
System.out.println(report);
 +
 
 +
} catch (Exception e) {
 +
e.printStackTrace();
 +
}
 +
</source>
 +
 
 +
===== Published Scope State Resource =====
 +
The Published Scope State Resource is a view on the current composition of the Scope context published in the IS. Each time the Scope is modified, the view is updated by a Scope Observer and republished. As for publishing, the resource is modeled as a Generic Resource with the following structure:
  
 
<source lang="xml">
 
<source lang="xml">
−
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
+
<Resource version="0.4.x">
 +
  <ID>8ff2a1f0-674d-11df-94fa-c1efdde5fc2d</ID>
 +
  <Type>GenericResource</Type>
 +
  <Scopes>
 +
    <Scope>/gcube/devsec/CNR-VRE</Scope>
 +
  </Scopes>
 +
  <Profile>
 +
    <SecondaryType>VRE/VO/Infrastructure</SecondaryType>
 +
    <Name> display name </Name>
 +
    <Description>... a description...</Description>
 +
    <Body>
 +
      <Scope>managed scope</Scope>
 +
      <!--- URI of the RM instance managing the scope-->
 +
      <Service>http://host:port/wsrf/services/gcube/vremanagement/ResourceManager</Service>
 +
      <Creator>...</Creator>
 +
      <Designer>...</Designer>
 +
      <StartTime> creation time </StartTime>
 +
      <EndTime>automatic disposal time (future, not considered now)</EndTime>
 +
      <SecurityEnabled>true/false</SecurityEnabled>
 +
      <!-- list of scoped resources joining the scope-->
 +
      <ScopedRescources>
 +
        <ScopedRescource>
 +
          <ResourceID> resource identifier </ResourceID>
 +
          <ResourceType> resource type </ResourceType>
 +
          <JointTime> when the resource joined the scope (date and time)</JointTime>
 +
        </ScopedRescource>
 +
        <ScopedRescource>
 +
        <!-- -->
 +
        </ScopedRescource>
 +
        <ScopedRescource>
 +
        <!-- -->
 +
        </ScopedRescource>
 +
      </ScopedRescources>
 +
    </Body>
 +
  </Profile>
 +
</Resource>
 +
</source>
 +
 
 +
Do note that the first part of the information reported is set by invoking the <code>ChangeOptions</code> operation.
 +
 
 +
[[ScopeResource.xml|Here]] is an example of Scope State Resource published in the IS:
 +
 
 +
==== On Resource Brokering ====
 +
When a new service is requested to be added to the scope an appropriate node must be selected in order to host a new instance of the service. This work is performed by a Resource Broker in charge of matching available resources with caller requests and service's requirement and considering infrastructure information statuses during matchmaking.
 +
 
 +
The Resource Manager service defines a simple interface to which any broker must comply to:
 +
<source lang="java">
 +
package org.gcube.vremanagement.resourcemanager.impl.brokerage;
 +
 
 +
import java.util.Set;
 +
import org.gcube.common.core.scope.GCUBEScope;
 +
import org.gcube.vremanagement.resourcemanager.impl.resources.ScopedDeployedService;
 +
 
 +
/**
 +
*
 +
* Models the expected behavior of a Broker.
 +
* A Broker is an entity that can create Deployment Plans for a set of {@link ScopedDeployedService}s to be deployed
 +
*
 +
* @author Manuele Simi (ISTI-CNR)
 +
*
 +
*/
 +
public interface Broker {
 +
 
 +
/**
 +
* Initializes the Broker instance
 +
* @param scope the scope where the broker will act
 +
* @throws Exception if the initialization fails (depends on the broker's nature)
 +
*/
 +
public void initialize(GCUBEScope scope) throws Exception;
 +
 +
/**
 +
* Creates a deployment plan
 +
* @param services the services to deploy
 +
* @param suggestedGHNs the GHNs suggested for the deployment, if specified, only these nodes will be used by the planner
 +
* @throws Exception if the preparation of the plan fails
 +
*/
 +
public void makePlan(Set<ScopedDeployedService> services, String[] suggestedGHNs) throws Exception;
 +
 +
/**
 +
* Sends feedback to the broker about the execution of the plan
 +
* @param services the services belonging the plan
 +
* @throws Exception if the broker is not able to manage the feedback
 +
*/
 +
public void sendFeedback(Set<ScopedDeployedService> services) throws Exception;
 +
 +
}
 +
 
 +
</source>
 +
 
 +
Then, the RM instance can be configured to dynamically load and exploit any class implementing this interface. This configuration setting is done via the <code>resourceBrokerClass</code> parameter in the service's JNDI:
 +
 
 +
<source lang="xml">
 +
<service name="gcube/vremanagement/ResourceManager/service">
 +
 +
<environment
 +
name="resourceBrokerClass"
 +
value="..."
 +
type="java.lang.String"
 +
override="false" />
 +
 +
</service>
 +
</source>
 +
 
 +
The current version of the RM is shipped with 2 Broker implementation descibed in the following subsections.
 +
 
 +
===== The Internal Broker =====
 +
 
 +
To enable this broker, the value of the <code>resourceBrokerClass</code> parameter has to be set to '''org.gcube.vremanagement.resourcemanager.impl.brokerage.InternalBroker'''. This broker is a very simple one and it has to be used when no other broker is available.
 +
In fact, the internal broker performs a minimal work and it's not capable to matchmake nodes and services. It can only manage requests where there is a list of suggested gHN per each service or per request. In this case, it checks the availability of those gHNs and then prepare the plan according to the caller suggestion (when a gHN is suggested for a specific service) or a simple round robin algorithm (when a list of gHN is suggested for the entire request).
 +
 
 +
===== The Service Broker =====
 +
 
 +
To enable this broker, the value of the <code>resourceBrokerClass</code> parameter has to be set to '''org.gcube.vremanagement.resourcemanager.impl.brokerage.ServiceBroker'''. This broker is in reality an interface over the [[Resource_Broker|Resource Broker]] service and can be used when an instance of such a service is available in the VO.
 +
 
 +
By default, the Resource Manager is configured to use this broker.
 +
 
 +
===== Broker's Report =====
 +
 
 +
If the deployment operations reach the broker's stage, a dedicated section of the [[Resource_Manager#Resource_Report |Resource Report]] shows the result of the query to the selected broker.
 +
 
 +
The following report shows the case where two services were requested to be added, they did, but no running instance was activated for them, since the broker didn't match the user's request with the actual state of the infrastructure. In particular, here the caller asked to deploy on a specific gHN which turned out to do not exist.
 +
 
 +
<source lang="xml">
 +
<?xml version="1.0" encoding="UTF-8"?>
 
<ResourceReport>
 
<ResourceReport>
−
   <ID>4522f7c0-a54c-11dd-8525-e7e836b0fdde</ID>
+
   <ID>97d510b0-674d-11df-94fa-c1efdde5fc2d</ID>
−
   <LastUpdate>2008-10-29T00:58:17+01:00</LastUpdate>
+
  <Operation>AddResources</Operation>
−
   <TargetScope>/gcube/devsec</TargetScope>
+
   <LastUpdate>2010-05-24T17:01:26+01:00</LastUpdate>
 +
   <TargetScope>/gcube/devsec/CNR-VRE</TargetScope>
 +
  <Resources>
 +
    <Resource>
 +
      <ID>25bfb640-04e0-11de-b58c-828df53a36e6</ID>
 +
      <Type>Service</Type>
 +
      <Status>ADDED</Status>
 +
    </Resource>
 +
    <Resource>
 +
      <ID>26774670-04e0-11de-b58c-828df53a36e6</ID>
 +
      <Type>Service</Type>
 +
      <Status>ADDED</Status>
 +
    </Resource>
 +
  </Resources>
 +
  <DeploymentPlanCreation>
 +
    <Status>FAILED</Status>
 +
    <Message>The creation of the Deployment Plan failed. Broker says: The GHNs required in the plan request are not available. Suggested GHN: f9f5dfa0-674c-11df-b094-a90b5bbcf085</Message>
 +
  </DeploymentPlanCreation>
 
   <Services>
 
   <Services>
 
     <Service>
 
     <Service>
−
       <Class>InformationSystem</Class>
+
      <ID>25bfb640-04e0-11de-b58c-828df53a36e6</ID>
−
       <Name>IS-Notifier</Name>
+
       <Class>MetadataManagement</Class>
 +
       <Name>MetadataManager</Name>
 
       <Version>1.00.00</Version>
 
       <Version>1.00.00</Version>
−
       <DependenciesResolutionStatus>SUCCESS</DependenciesResolutionStatus>
+
       <DeploymentActivity>The report is still not available for this service</DeploymentActivity>
−
      <DeployedOn>9bf172d0-a30c-11dd-88bf-83f7edc13611</DeployedOn>
+
    </Service>
−
      <ErrorDescription>-</ErrorDescription>
+
    <Service>
−
       <DependenciesResolution>
+
       <ID>26774670-04e0-11de-b58c-828df53a36e6</ID>
−
        <ResolvedDependencies>
+
      <Class>MetadataManagement</Class>
−
          <Dependency>
+
      <Name>XMLIndexer</Name>
−
            <ServiceClass>InformationSystem</ServiceClass>
+
      <Version>1.00.00</Version>
−
            <ServiceName>IS-Notifier</ServiceName>
+
      <DeploymentActivity>The report is still not available for this service</DeploymentActivity>
−
            <ServiceVersion>1.00.00</ServiceVersion>
+
−
            <PackageName>Notifier-service</PackageName>
+
−
            <PackageVersion>1.01.00</PackageVersion>
+
−
          </Dependency>
+
−
          <Dependency>
+
−
            <ServiceClass>InformationSystem</ServiceClass>
+
−
            <ServiceName>IS-Notifier</ServiceName>
+
−
            <ServiceVersion>1.00.00</ServiceVersion>
+
−
            <PackageName>Notifier-stubs</PackageName>
+
−
            <PackageVersion>1.00.01</PackageVersion>
+
−
          </Dependency>
+
−
        </ResolvedDependencies>
+
−
        <MissingDependencies/>
+
−
      </DependenciesResolution>
+
 
     </Service>
 
     </Service>
 
   </Services>
 
   </Services>
−
  <DeploymentActivity>
+
</ResourceReport>
−
    <GHN>
+
</source>
−
      <Host>dlib33.isti.cnr.it:8001</Host>
+
 
−
      <ID>9bf172d0-a30c-11dd-88bf-83f7edc13611</ID>
+
Here it is the report for the same services when everything goes OK:
−
      <LastReportReceivedOn>2008-10-29T00:58:16+01:00</LastReportReceivedOn>
+
<source lang="xml">
−
      <LastReportReceived>
+
<?xml version="1.0" encoding="UTF-8"?>
−
        <Status>OPEN</Status>
+
<ResourceReport>
−
        <Packages>
+
  <ID>a771bad0-674f-11df-94fa-c1efdde5fc2d</ID>
−
          <Package>
+
  <Operation>AddResources</Operation>
−
            <ServiceClass>InformationSystem</ServiceClass>
+
  <LastUpdate>2010-05-24T17:16:10+01:00</LastUpdate>
−
            <ServiceName>IS-Notifier</ServiceName>
+
  <TargetScope>/gcube/devsec/CNR-VRE</TargetScope>
−
            <ServiceVersion>1.00.00</ServiceVersion>
+
−
            <PackageName>Notifier-service</PackageName>
+
−
            <PackageVersion>1.01.00</PackageVersion>
+
−
            <Status>RUNNING</Status>
+
−
            <Message>The package has NOT been successfully deployed. Possible cause: unable to deploy because of a RI of the service is already deployed on this node</Message>
+
−
          </Package>
+
−
          <Package>
+
−
            <ServiceClass>InformationSystem</ServiceClass>
+
−
            <ServiceName>IS-Notifier</ServiceName>
+
−
            <ServiceVersion>1.00.00</ServiceVersion>
+
−
            <PackageName>Notifier-stubs</PackageName>
+
−
            <PackageVersion>1.00.01</PackageVersion>
+
−
            <Status>ACTIVATED</Status>
+
−
            <Message>The package could not be deployed because it depends on a package which has an error</Message>
+
−
          </Package>
+
−
        </Packages>
+
−
      </LastReportReceived>
+
−
    </GHN>
+
−
  </DeploymentActivity>
+
 
   <Resources>
 
   <Resources>
 
     <Resource>
 
     <Resource>
−
       <ID>83080450-a235-11dd-8a29-b025e02401d5</ID>
+
       <ID>25bfb640-04e0-11de-b58c-828df53a36e6</ID>
 +
      <Type>Service</Type>
 +
      <Status>ADDED</Status>
 +
    </Resource>
 +
    <Resource>
 +
      <ID>26774670-04e0-11de-b58c-828df53a36e6</ID>
 +
      <Type>Service</Type>
 +
      <Status>ADDED</Status>
 +
    </Resource> 
 +
  </Resources>
 +
  <DeploymentPlanCreation>
 +
    <Status>SUCCESS</Status>
 +
    <Message>The Deployment Plan was successfully created</Message>
 +
  </DeploymentPlanCreation>
 +
  <Services>
 +
  <!-- services' section here-->
 +
   
 +
  </Services>
 +
</ResourceReport>
 +
</source>
 +
 
 +
==== Test-suite ====
 +
The Resource Manager comes with a test-suite package allowing to test its functionalities.The test-suite is completely independent and does not require any other gCube package, except than a local gCore installation.
 +
The package is composed by a set of classes, sample configuration files and scripts ready to be executed.
 +
 
 +
 
 +
<pre>
 +
|-lib
 +
|--org.gcube.vremanagement.resourcemanager.testsuite.jar
 +
|
 +
|-samples
 +
|---...
 +
|
 +
|-addresource.sh
 +
|-getreport.sh
 +
|-changescopeoptions.sh
 +
|-...
 +
 
 +
</pre>
 +
Each script allows to test a different service's operation or group of operations logically related. In the following, an explanation of each script and its usage is provided.
 +
 
 +
===== Add Resource(s) =====
 +
----
 +
The AddResources script invokes the <code>AddResources</code> operation to add new GCUBEResources on a scope. It must be executed as follows
 +
 
 +
<pre>
 +
./addresource.sh <ResourceManager host> <ResourceManager port> <configuration file>
 +
</pre>
 +
 
 +
Clearly, the Resource Manager host and port are the information needed to contact the Resource Manager instance, while the configuration file reports the information about the resources to add or create and the scope in which they have to be managed.
 +
 
 +
This is an example of configuration file requesting to add 4 gCube Resources to the Scope <code>/gcube/devsec/CNR-VRE</code>:
 +
 
 +
<source lang="text">
 +
#global properties
 +
 
 +
numOfServicesToAdd=0
 +
numOfResourcesToAdd=4
 +
targetScope=/gcube/devsec/CNR-VRE
 +
callerScope=/gcube/devsec/CNR-VRE
 +
 
 +
# resources' list
 +
resource.1.id=9ad3ce60-27aa-11df-9df7-e0bf8afcd20f
 +
resource.1.type=Collection
 +
resource.2.id=6e6e2ce0-1ca5-11df-a0fe-c8f6680864ab
 +
resource.2.type=RunningInstance
 +
resource.3.id=a5db71f0-27aa-11df-8ee2-c1a4489ea34b
 +
resource.3.type=MetadataCollection
 +
resource.4.id=df899630-82ae-11de-a6cc-f15c9cf886be
 +
resource.4.type=GenericResource
 +
</source>
 +
 
 +
If everything works, the following report is produced (it can be retrieved via the [[Resource_Manager#Get_Report|getReport]] script):
 +
 
 +
<source lang="xml">
 +
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
 +
<ResourceReport>
 +
  <ID>bdffb830-33a4-11df-97f6-8d3129726fea</ID>
 +
  <LastUpdate>2010-03-19T18:14:10-04:00</LastUpdate>
 +
  <TargetScope>/gcube/devsec/CNR-VRE</TargetScope>
 +
  <Resources>
 +
    <Resource>
 +
      <ID>a5db71f0-27aa-11df-8ee2-c1a4489ea34b</ID>
 +
      <Type>MetadataCollection</Type>
 +
      <Status>ADDED</Status>
 +
      <ErrorDescription>-</ErrorDescription>
 +
    </Resource>
 +
    <Resource>
 +
      <ID>6e6e2ce0-1ca5-11df-a0fe-c8f6680864ab</ID>
 
       <Type>RunningInstance</Type>
 
       <Type>RunningInstance</Type>
 
       <Status>ADDED</Status>
 
       <Status>ADDED</Status>
Line 146: Line 793:
 
     </Resource>
 
     </Resource>
 
     <Resource>
 
     <Resource>
−
        <ID>61927680-a996-11dd-961c-8eba14a244c3</ID>
+
      <ID>df899630-82ae-11de-a6cc-f15c9cf886be</ID>
−
        <Type>MetadataCollection</Type>
+
      <Type>GenericResource</Type>
−
        <Status>ADDED</Status>
+
      <Status>ADDED</Status>
−
        <ErrorDescription>-</ErrorDescription>
+
      <ErrorDescription>-</ErrorDescription>
 
     </Resource>
 
     </Resource>
 
     <Resource>
 
     <Resource>
−
        <ID>f2c04460-b446-11dd-8aab-e7a1e3fdae89</ID>
+
      <ID>9ad3ce60-27aa-11df-9df7-e0bf8afcd20f</ID>
−
        <Type>GenericResource</Type>
+
      <Type>Collection</Type>
−
        <Status>ADDED</Status>
+
      <Status>ADDED</Status>
−
        <ErrorDescription>-</ErrorDescription>
+
      <ErrorDescription>-</ErrorDescription>
 
     </Resource>
 
     </Resource>
 
   </Resources>
 
   </Resources>
 +
  <Services/>
 
</ResourceReport>
 
</ResourceReport>
 
</source>
 
</source>
  
−
==== gHNManager integration ====
+
===== Remove Resource(s) =====
−
...
+
----
 +
The RemoveResources script invokes the <code>RemoveResources</code> operation to remove GCUBEResources from a scope. It must be executed as follows
 +
 
 +
<pre>
 +
./removeresource.sh <ResourceManager host> <ResourceManager port> <configuration file>
 +
</pre>
 +
 
 +
This is an example of configuration file requesting to remove a Collection from the Scope <code>/gcube/devsec/CNR-VRE</code>:
 +
 
 +
<source lang="text">
 +
#global properties
 +
numOfServicesToRemove=0
 +
numOfResourcesToRemove=1
 +
targetScope=/gcube/devsec/CNR-VRE
 +
callerScope=/gcube/devsec/CNR-VRE
 +
 
 +
# resources' list
 +
resource.1.id=9ad3ce60-27aa-11df-9df7-e0bf8afcd20f
 +
resource.1.type=Collection
 +
</source>
 +
 
 +
If everything works, the following report is produced (it can be retrieved via the [[Resource_Manager#Get_Report|getReport]] script):
 +
 
 +
<source lang="xml">
 +
<?xml version="1.0" encoding="UTF-8"?>
 +
<ResourceReport>
 +
  <ID>cfb41e90-3863-11df-9bfb-cfdb92619067</ID>
 +
  <LastUpdate>2010-03-26T00:11:58+01:00</LastUpdate>
 +
  <TargetScope>/gcube/devsec/CNR-VRE</TargetScope>
 +
  <Resources>
 +
    <Resource>
 +
      <ID>9ad3ce60-27aa-11df-9df7-e0bf8afcd20f</ID>
 +
      <Type>Collection</Type>
 +
      <Status>REMOVED</Status>
 +
    </Resource>
 +
  </Resources>
 +
  <Services/>
 +
</ResourceReport></source>
 +
 
 +
===== Create a new Scope =====
 +
----
 +
The CreateScope script invokes the <code>CreateScope</code> operation in order to initialise a new Scope and assign to it the first set of gCube resources. It must be executed as follows
 +
 
 +
<pre>
 +
./createScope.sh <ResourceManager host> <ResourceManager port> <configuration file>
 +
</pre>
 +
 
 +
This is an example of configuration file requesting to
 +
* create a new VRE (named ''/gcube/devsec/CNR-VRE'')with the indicated options
 +
* add to it 4 existing gCube Resources
 +
* deploy 2 new Services in the new Scope
 +
 
 +
<source lang="text">
 +
######################################
 +
# Sample VRE creation - properties file
 +
######################################
 +
 
 +
targetScope=/gcube/devsec/CNR-VRE
 +
callerScope=/gcube/devsec/CNR-VRE
 +
 
 +
#Scope properties
 +
creator=CN=Manuele Simi, L=NMIS-ISTI, OU=Personal Certificate, O=INFN, C=IT
 +
designer=CN=Manuele Simi, L=NMIS-ISTI, OU=Personal Certificate, O=INFN, C=IT
 +
securityenabled=false
 +
displayName=CNR-VRE
 +
description=Sample VRE for testing purposes
 +
 
 +
#Scope resources
 +
numOfServicesToAdd=2
 +
numOfResourcesToAdd=4
 +
 
 +
# resources' list
 +
resource.1.id=9ad3ce60-27aa-11df-9df7-e0bf8afcd20f
 +
resource.1.type=Collection
 +
resource.2.id=6e6e2ce0-1ca5-11df-a0fe-c8f6680864ab
 +
resource.2.type=RunningInstance
 +
resource.3.id=a5db71f0-27aa-11df-8ee2-c1a4489ea34b
 +
resource.3.type=MetadataCollection
 +
resource.4.id=df899630-82ae-11de-a6cc-f15c9cf886be
 +
resource.4.type=GenericResource
 +
 
 +
# services' list
 +
service.1.name=MetadataManager
 +
service.1.class=MetadataManagement
 +
service.1.version=1.00.00
 +
service.2.name=XMLIndexer
 +
service.2.class=MetadataManagement
 +
service.2.version=1.00.00
 +
 
 +
#GHN IDs on which the service will be deployed
 +
GHNSet=84c3aa10-3857-11df-b245-cbf0ea5a1843,05921df0-382d-11df-8c8f-f5ea2a36aefa
 +
</source>
 +
 
 +
===== Dispose an existing Scope =====
 +
----
 +
The DisposeScope script invokes the <code>DisposeScope</code> operation in order to throw away an entire scope. It must be executed as follows
 +
 
 +
<pre>
 +
./disposeScope.sh <ResourceManager host> <ResourceManager port> <Scope>
 +
</pre>
 +
 
 +
===== Get Report =====
  
−
==== Deployer integration ====
+
----
−
...
+
The GetReport script invokes the GetReport operation to retrieve a Resource Report. It must be executed as follows
 +
<pre>
 +
./getreport.sh <ResourceManager host> <ResourceManager port> <caller scope> <report ID>
 +
</pre>
  
−
==== Resource Broker integration ====
+
This request will print out the [[Resource_Manager#Resource_Report|resource report]] related to the given report ID.
−
...
+

Latest revision as of 14:07, 19 October 2016

The Resource Manager service is in charge of managing gCube Scope contexts. An instance (and only one) of the service must be deployed for each scope (at any level, i.e. infrastructure, VO, VRE) in order to handle gCube Resources within the assigned scope. It manages requests of adding and removing gCube Resource from the assigned Scope by keeping, at the same time, a consistent state in the whole infrastructure. A Scope Resource is published in the gCore Based Information System with constantly updated information about the current composition of the Scope.

Design

Service architecture and code design

The service adopts the singleton WSRF pattern with a single stateful resource holding the state of the instance. All the code is designed around the concept of Scope and its related state. The operations exposed by the service allow to change the state a Scope managed by the instance by adding and removing gCube resources and internal triggers take subsequent actions to manage these changes.

The public interface of the service is organized as follows:

Figure 1. Resource Manager - PortTypes
  • ScopeController portType offers create, read, update and delete (CRUD) operations for managing scopes
  • ResourceBinder portType allows to add/remove resources to a managed scope
  • Reporting porType receives and returns reports on the operations performed on managed resources

The following class diagram shows a simplified view of the organization and structure of the Resource Manager's source code.

Figure 2. Resource Manager - Class Diagram

Internal State

Concept, role and design of ScopedResource

A Resource Manager instance handles a Scope as a set of Scoped Resources. A Scoped Resource models a gCube Resource in a specific Scope. At the heart of the Scope management activities there is the management of the state of the Scoped Resources belonging the Scope.

The following diagram depicts the Scope Resource's statuses and the legal transitions among them. The starting point of the life of a Scope Resource is always the request to join an existing gCube Resource to the Scope.


Figure 3. Scoped Resource - life cycle

The most important statuses are ADDED and REMOVED, where the concrete (and resource specific) actions to join/dis-join the gCube Resource to the Scope are performed. For instance, in case of a Service Resource, a new instance of the service is deployed when managing the ADDED status, while undeployment is performed in the REMOVED status.

Instance State
Figure 4. Resource Manager - Instance State
Scope State
Figure 5. Resource Manager - Scope State
Scope state serialization

The Scope state is formed by a set of Scoped Resources. Such a state is persisted in two places:

  • locally, via an XStream serialization of the objects handling the state
  • remotely, on the IS as ScopedResource

There is a continuous synchronization between the two. To be more precise , at start up time, the Resource Manager instance looks for its serialized state locally, if it does not found the information, it tries to harvest its state from the IS. In case even the IS does not return a valid state, a new one is created. Then, each time the local state changes, the remote one is updated too.

The remote serialization contains less information with respect to the local one for internal and logical reasons. Nevertheless, this doubled state allows to partially recovers the state of the Scope whenever the RM instance is moved from one gHN to another and if the state of the gHN is wrongly sweep out.

Lastly, it's important to notice that the current implementation is not capable of "recreating" from scratch the Scope state in case both of its serialization are lost. Therefore, if an instance of a service is undeployed by a human or by other means, it is not removed from the Scope state. The same is for the manipulation of any other gCube resources within the Scope.

Observer pattern strategy

The service adopts an observer pattern-based internal strategy for managing the scope state. Each time the state of a Scoped Resource changes, registered Scope Observers are properly notified in order to manage the new state.

An observer is a class that extends the abstract class org.gcube.vremanagement.resourcemanager.impl.state.observers.ScopeObserver. The current version of the Resource Manager includes the following 3 Scope Observers:

  • Executor (org.gcube.vremanagement.resourcemanager.impl.state.observers.Executor): each time a resource is in a ADDREQUEST or REMOVEREQUEST status, the Executor observer adds or removes the resource to/from the state;
  • Publisher (org.gcube.vremanagement.resourcemanager.impl.state.observers.Publisher): each time a resource is in added or removed to/from the state, the Publisher observer synchronizes the local state with the remote state published in the IS;
  • Serializer (org.gcube.vremanagement.resourcemanager.impl.state.observers.Serializer): each time a change occurs in a resource, the Serializer observer performs a serialization of the state on the local file system.

Interface

Resource Report

All the operations performed by the Resource Manager work in an asynchronous manner with respect to their invocation. They return an ID that can be used to query the getReport operation in order to retrieve a detailed report of the ongoing or completed activities.

The basic structure of the report is: 

<ResourceReport>
  <ID><!--report ID --></ID>
  <Status>OPEN/CLOSED</Status>
  <Operation>Create/AddResources/UpdateResources/RemoveResources/Dispose</Operation>
  <LastUpdate><!--last update time --></LastUpdate>
  <TargetScope><!-- target scope of the operations reported in the following section --></TargetScope>
  <Resources>
     <!-- list of gCube resources involved and their actual status-->  
     <Resource>
      <ID>..</ID>
      <Type>...</Type>
      <Status>ADDED/REMOVED/FAILED</Status>
      <ErrorDescription>..</ErrorDescription> <!--only if status=FAILED --> 
    </Resource>
     <Resource>
      <ID>..</ID>
      <Type>...</Type>
      <Status>ADDED/REMOVED/FAILED</Status>
      <ErrorDescription>..</ErrorDescription> <!--only if status=FAILED --> 
    </Resource>
  </Resources>
  <DeploymentPlanCreation>
      <!-- outcomes of the deployment plan creation -->
      <Status>SUCCESS/FAILED</Status>
      <Message></Message>
   </DeploymentPlanCreation>
  <Software>
   <!-- per each software involved in the request, a section like the following-->
    <Service>
      <ID>...</ID>
      <Class>...</Class>
      <Name>...</Name>
      <Version>...</Version>
      <PackageName>...</PackageName>
      <PackageVersion>...</PackageVersion>
      <DependenciesResolutionStatus>SUCCESS/FAILED</DependenciesResolutionStatus>
      <DeployedOn>...</DeployedOn> <!- GHN ID-->
      <ErrorDescription>-</ErrorDescription>
      <DependenciesResolution> 
         <!-- information about service's dependencies (missed, resolved))-->
        <ResolvedDependencies/>
        <MissingDependencies/>
      </DependenciesResolution
     <DependenciesResolution>        
     <DeploymentActivity> or <UndeploymentActivity>
        <!-- information about (un)deployment activities on the target gHN-->
     </DeploymentActivity> or </UndeploymentActivity>
    <Service>
  </Software>
</ResourceReport>

The first elements contain general information about the report itself. The most important one is the Status element which states whether the activities have been concluded or no. The two possible values here are OPEN or CLOSED.

The Software' section is reported when that the activities performed involve the deployment or undeployment of software. For each software, information about dependencies resolution (if any) and package deployment/undeployment are reported.

Possible dependencies' statuses are:

  • SUCCESS: all the dependencies have been successfully resolved
  • FAILED: some dependencies are missing

Periodically, the Deployer service sends a report on the deployment activity (performed asynchronously with respect to the Resource Manager request). Such report has a status:

  • OPEN: the activity is still ongoing, other reports will follow
  • CLOSED : the activity is closed, no further report will be sent

Possible package statuses are:

  • WAITING: request is still pending
  • ALREADYDEPLOYED: the package was already available on the target gHN
  • SKIPPED: the package was not deployed due to an error in another service from which it depends on
  • FAILED: the deployment failed
  • DEPLOYED: the package has been deployed
  • NOTVERIFIED: the package has been deployed, but the correctness of the operation cannot be checked (and therefore guaranteed)
  • ACTIVATED: the package has been activated
  • RUNNING: the related instance is running

Possible resource statuses:

  • ADDED: the resource was successfully added to the scope
  • REMOVED: the resource was successfully removed to the scope
  • FAILED: an error occurred and the resource has not been managed

This is a sample report showing the results of a request to add a bunch of software to a managed scope.

Interacting with the Resource Manager

All the operations described below are exposed by the ResourceManager portType. Most of them make a distinction between the services and all the rest of the gCube resources: this is because managing services often requires different kind of parameters and produces a most complex feedback.

Creating a new scope

To create a new scope, the CreateScope operation of the ScopeController portType has to be invoked. To be successful in the creation, the following preconditions must be valid:

  • in case of VO scope, the gHN hosting the Resource Manager instance must already have locally the Service Map of the VO;
  • in case of VRE scope, the gHN hosting the Resource Manager instance must already have locally the Service Map of the parent's VO;
  • the Resource Manager instance must already join at least the parent scope.

Optionally, a list of scope options can be passed (explained in the next section).

Here it is a sample usage of the operation: 

import org.gcube.vremanagement.resourcemanager.stubs.scontroller.CreateScopeParameters;
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.OptionsParameters;
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.ScopeControllerPortType;
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.ScopeOption;
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.service.ScopeControllerServiceAddressingLocator;
import org.gcube.vremanagement.resourcemanager.stubs.resourcemanager.service.ResourceManagerServiceAddressingLocator;
 
protected static String[] optionNames = new String[] {"creator", "designer", "endTime", "startTime", 
		"description", "displayName", "securityenabled"};
 
 
EndpointReferenceType endpoint = new EndpointReferenceType();
 
try {
	endpoint.setAddress(new Address("http://" + args[0] + "/wsrf/services/gcube/vremanagement/resourcemanager/scopecontroller"));
	ScopeControllerPortType pt = new ScopeControllerServiceAddressingLocator().getScopeControllerPortTypePort(endpoint);
	pt = GCUBERemotePortTypeContext.getProxy(pt, GCUBEScope.getScope(resources.getProperty("callerScope").trim()), 90000);
 
	logger.info("Setting the Scope parameters...");
	OptionsParameters options = new OptionsParameters();
	ScopeOption[] scopeOptionList = new ScopeOption[optionNames.length];
	for (int i=0; i < optionNames.length; i++) 
		if (resources.getProperty(optionNames[i]) != null) {
			logger.info("Setting prop " +  optionNames[i]);
			scopeOptionList[i] = new ScopeOption(optionNames[i], resources.getProperty(optionNames[i]));
		}
	options.setScopeOptionList(scopeOptionList);
 
	//ready to start...
	logger.info("Sending the creation request....");			
	CreateScopeParameters params = new CreateScopeParameters();
	params.setTargetScope("my scope name");
	params.setOptionsParameters(options);
	pt.createScope(params);
 
} catch (Exception e) {
	logger.fatal("Failed to create the scope",e);
}
Set/change the scope options

A number of information related to the scope can be set by invoking the ChangeScopeOptions operation of the ScopeController portType. Actually, the following properties are accepted:

  • Designer: the designer of the scope
  • Creator: the creator of the scope
  • StartTime: the activation time of the scope (accepted format is yyyy-MM-dd'T'HH:mm:ssZ, example: 2008-11-11T16:58:12+01:00)
  • EndTime: the last time of the scope (accepted format is yyyy-MM-dd'T'HH:mm:ssZ, example: 2008-11-11T16:58:12+01:00)
  • Description: a scope description
  • DisplayName: a name to display in any GUI
  • SecurityEnabled: a boolean flag to set up a secure or non-secure Scope

All of them are optionals and they can be set/updated any time, even if in some cases (like changing the start time once the scope is activated) this does not have any effect.


The ChangeScopeOptions operation can be invoked as follows: 

import org.gcube.vremanagement.resourcemanager.stubs.scontroller.OptionsParameters;
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.ScopeControllerPortType;
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.ScopeOption;
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.service.ScopeControllerServiceAddressingLocator;
 
//...
 
String[] optionNames = new String[] {"creator", "designer", "endTime", "startTime", 
						"description", "displayName", "securityenabled"};
 
//...
EndpointReferenceType endpoint = new EndpointReferenceType();
 
try {
	endpoint.setAddress(new Address(args[0]));
	ScopeControllerPortType pt = new ScopeControllerServiceAddressingLocator().getScopeControllerPortTypePort(endpoint);
	pt = GCUBERemotePortTypeContext.getProxy(pt, GCUBEScope.getScope(resources.getProperty("callerScope").trim()), 90000);
 
	OptionsParameters options = new OptionsParameters();
	ScopeOption[] scopeOptionList = new ScopeOption[optionNames.length];
	for (int i=0; i < optionNames.length; i++) 
		scopeOptionList[i] = new ScopeOption(optionNames[i], resources.getProperty(optionNames[i]));
 
	options.setScopeOptionList(scopeOptionList);
	pt.changeScopeOptions(options);
 
} catch (Exception e) {
        // manage the error
}
Request to join a resource to the scope

A request to join a GCUBEResource to a scope is performed through the AddResources operation of the ResourceBinder portType. For each resource there are two main information to provide:

  • the resource identifier
  • the resource type

plus the target scope of the operation.

The AddResources operation can be invoked as follows: 

import org.gcube.vremanagement.resourcemanager.stubs.binder.AddResourcesParameters;
import org.gcube.vremanagement.resourcemanager.stubs.binder.ResourceBinderPortType;
import org.gcube.vremanagement.resourcemanager.stubs.binder.ResourceItem;
import org.gcube.vremanagement.resourcemanager.stubs.binder.ResourceList;
import org.gcube.vremanagement.resourcemanager.stubs.binder.PackageItem;
import org.gcube.vremanagement.resourcemanager.stubs.binder.SoftwareList;
import org.gcube.vremanagement.resourcemanager.stubs.binder.service.ResourceBinderServiceAddressingLocator;
 
//...
 
EndpointReferenceType endpoint = new EndpointReferenceType();
 
try {
	endpoint.setAddress(new Address("http://" + args[0] + "/wsrf/services/gcube/vremanagement/resourcemanager/binder"));
	ResourceBinderPortType pt = new ResourceBinderServiceAddressingLocator().getResourceBinderPortTypePort(endpoint);
	pt = GCUBERemotePortTypeContext.getProxy(pt, 
	GCUBEScope.getScope(resources.getProperty("callerScope").trim()));
 
	AddResourcesParameters add = new AddResourcesParameters();
	ResourceItem[] resourcelist = new ResourceItem[<#Resources>)];
	for (int i = 1 ; i < (resourcelist.length +1); i++) {
		resourcelist[i-1] = new ResourceItem();
		resourcelist[i-1].setID("<resource ID>");
		resourcelist[i-1].setType("<resource type>");
	}
	ResourceList r = new ResourceList();
	r.setResource(resourcelist);
	add.setResources(r);
 
	add.setTargetScope("my scope");
	System.out.println ("Sending the Resource request....");
	String reportID = pt.addResources(add);
	System.out.println ("Returned report ID: " + reportID);
 
} catch (Exception e) {
	e.printStackTrace();
}
Request to remove a resource from a scope

A request to remove a GCUBEResource from a scope is performed through the RemoveResources operation of the ResourceBinder portType. For each resource there are two main information to provide:

  • the resource identifier
  • the resource type

plus the target scope of the operation.

The RemoveResources operation can be invoked as follows: 

import org.gcube.vremanagement.resourcemanager.stubs.binder.RemoveResourcesParameters;
import org.gcube.vremanagement.resourcemanager.stubs.binder.ResourceBinderPortType;
import org.gcube.vremanagement.resourcemanager.stubs.binder.ResourceItem;
import org.gcube.vremanagement.resourcemanager.stubs.binder.ResourceList;
import org.gcube.vremanagement.resourcemanager.stubs.binder.service.ResourceBinderServiceAddressingLocator;
 
//...
EndpointReferenceType endpoint = new EndpointReferenceType();
try {
	endpoint.setAddress(new Address("http://" + args[0] + "/wsrf/services/gcube/vremanagement/resourcemanager/binder"));
	ResourceBinderPortType pt = new ResourceBinderServiceAddressingLocator().getResourceBinderPortTypePort(endpoint);
	pt = GCUBERemotePortTypeContext.getProxy(pt, 
	    GCUBEScope.getScope("<Caller Scope>"));
 
	RemoveResourcesParameters params = new RemoveResourcesParameters();
	ResourceItem[] resourcelist = new ResourceItem[#resource to remove];
	for (int i = 1 ; i < (resourcelist.length +1); i++) {
		resourcelist[i-1] = new ResourceItem();
		resourcelist[i-1].setID("<ID>");
		resourcelist[i-1].setType("<type>");
	}
	ResourceList r = new ResourceList();
	r.setResource(resourcelist);
	params.setResources(r);
	params.setTargetScope("<target Scope>"));
	System.out.println ("Sending the Remove Resource request....");
	String reportID = pt.removeResources(params);
	System.out.println ("Returned report ID: " + reportID);
} catch (Exception e) {
	e.printStackTrace();
}
Deploying a new software in a scope

To deploy a new software in the scope, the addResource operation of the Resource Binder portType has to be invoked passing a list of PackageItem (one per each service to deploy). Beside joining the service's resource itself, this action has two side effects on the scope's realm:

  • the target gHN is automatically added the scope, if the scope is of type VRE
  • the newly generated instances of the service hosted on the target gHN automatically joins the scope at start up time
import org.gcube.vremanagement.resourcemanager.stubs.binder.AddResourcesParameters;
import org.gcube.vremanagement.resourcemanager.stubs.binder.ResourceBinderPortType;
import org.gcube.vremanagement.resourcemanager.stubs.binder.PackageItem;
import org.gcube.vremanagement.resourcemanager.stubs.binder.SoftwareList;
import org.gcube.vremanagement.resourcemanager.stubs.binder.service.ResourceBinderServiceAddressingLocator;
 
//...
 
EndpointReferenceType endpoint = new EndpointReferenceType();
 
try {
	endpoint.setAddress(new Address("http://" + args[0] + "/wsrf/services/gcube/vremanagement/resourcemanager/binder"));
	ResourceBinderPortType pt = new ResourceBinderServiceAddressingLocator().getResourceBinderPortTypePort(endpoint);		
	pt = GCUBERemotePortTypeContext.getProxy(pt, 
		GCUBEScope.getScope("<caller scope>"));
 
	AddResourcesParameters add = new AddResourcesParameters();
 
	// prepare the list of service to ask to deploy
	PackageItem[] packagelist = new PackageItem[#numOfSoftwareToAdd];
	for (int i = 1 ; i < (packagelist.length +1); i++) {
		packagelist[i-1] = new PackageItem();
		packagelist[i-1].setServiceClass("<serviceclass>");
		packagelist[i-1].setServiceName("<servicename>");
		packagelist[i-1].setServiceVersion("<serviceversion>");
		packagelist[i-1].setPackageName("<packagename>");
		packagelist[i-1].setPackageVersion("<packageversion>");
		//suggest a ghn for the deployment
		if (resources.getProperty("software." + i + ".GHN") != null)
			packagelist[i-1].setTargetGHNName(resources.getProperty("software." + i + ".GHN"));
	}
 
	SoftwareList l = new SoftwareList();
	l.setSoftware(packagelist);
	//set the suggested set of GHNs to use, if any	
	String[] ghns = ...;
	l.setSuggestedTargetGHNNames(ghns);
 
	add.setSoftware(l);	
	add.setTargetScope(resources.getProperty("targetScope"));
	System.out.println ("Sending the Resource request....");
	String reportID = pt.addResources(add);
	System.out.println ("Returned report ID: " + reportID);
 
} catch (Exception e) {
	e.printStackTrace();
}
Removing a software from a scope

The removal of a software from a scope is requested by invoking the removeResources operation of the ResourceBinder portType. By requesting the removal of a software, the software is:

  • undeployed from the specified gHN if it is running only in the target scope
  • removed from the target scope, if the software is joined to other scopes

Do note that the implicit rule of undeploying a software from a node when the related Running Instance is requested to be removed from a single scope does not apply anymore in RM2.x. The only way to undeploy a software is to list it in the SoftwareList parameter. 

import org.gcube.vremanagement.resourcemanager.stubs.binder.RemoveResourcesParameters;
import org.gcube.vremanagement.resourcemanager.stubs.binder.ResourceBinderPortType;
import org.gcube.vremanagement.resourcemanager.stubs.binder.PackageItem;
import org.gcube.vremanagement.resourcemanager.stubs.binder.SoftwareList;
import org.gcube.vremanagement.resourcemanager.stubs.binder.service.ResourceBinderServiceAddressingLocator;
 
//...
EndpointReferenceType endpoint = new EndpointReferenceType();
try {
	endpoint.setAddress(new Address("http://" + args[0] + "/wsrf/services/gcube/vremanagement/resourcemanager/binder"));
	ResourceBinderPortType pt = new ResourceBinderServiceAddressingLocator().getResourceBinderPortTypePort(endpoint);
 
	pt = GCUBERemotePortTypeContext.getProxy(pt, 
		GCUBEScope.getScope(resources.getProperty("callerScope").trim()	));
 
	RemoveResourcesParameters params = new RemoveResourcesParameters();
	PackageItem[] packagelist = new PackageItem[#numOfSoftwareToRemove))];
	for (int i = 1 ; i < (packagelist.length +1); i++) {
		packagelist[i-1] = new PackageItem();
		packagelist[i-1].setServiceClass("<service  class>");
		packagelist[i-1].setServiceName("<service name>"));
		packagelist[i-1].setServiceVersion("<service version>");					
		packagelist[i-1].setPackageName("<service packagename>");
		packagelist[i-1].setPackageVersion("<service packageversion>");
		packagelist[i-1].setTargetGHNName("<target ghnname>"));
 
	}
 
	SoftwareList l = new SoftwareList();
	l.setSoftware(packagelist);			
	params.setSoftware(l);
 
	params.setTargetScope(resources.getProperty("targetScope").trim());
	System.out.println ("Sending the Remove Resource request....");
	String reportID = pt.removeResources(params);
	System.out.println ("Returned report ID: " + reportID);
} catch (Exception e) {
	e.printStackTrace();
}
Disposing a scope

A scope managed by an instance of the service can be disposed by invoking the DisposeScope operation of the ScopeController portType. This call will:

  • remove all the resources from the scope
  • undeploy the services belonging only this scope
  • unpublish the Scope Resource
  • remove the RM instance from the scope

The following code shows how to invoke the DisposeScope operation: 

import org.gcube.vremanagement.resourcemanager.stubs.scontroller.ScopeControllerPortType;
import org.gcube.vremanagement.resourcemanager.stubs.scontroller.service.ScopeControllerServiceAddressingLocator;
 
//...
 
EndpointReferenceType endpoint = new EndpointReferenceType();		
try {
	endpoint.setAddress(new Address("http://" + args[0] + "/wsrf/services/gcube/vremanagement/resourcemanager/scopecontroller"));
	ScopeControllerPortType pt = new ScopeControllerServiceAddressingLocator().getScopeControllerPortTypePort(endpoint);
	pt = GCUBERemotePortTypeContext.getProxy(pt, GCUBEScope.getScope(args[1].trim()));
	logger.info("Sending the destroy request....");			
	String reportID = pt.disposeScope(" scope to dispose ");
	logger.info("Returned report ID: " + reportID);			
} catch (Exception e) {
	logger.fatal("Failed to dispose the scope",e);
}
Obtaining the report of the activities performed

To retrieve a Resource Report of the activities performed by any operation, the getReport operation of the Reporting portType has to be invoked. The single paramenter of the operation is the report identifier returned by the related operation.

This is an example about how to invoke the operation: 

import org.gcube.vremanagement.resourcemanager.stubs.reporting.ReportingPortType;
import org.gcube.vremanagement.resourcemanager.stubs.reporting.service.ReportingServiceAddressingLocator;
 
//...
EndpointReferenceType endpoint = new EndpointReferenceType();
 
try {
	endpoint.setAddress(new Address("http://" + args[0] + "/wsrf/services/gcube/vremanagement/resourcemanager/reporting"));
	ReportingPortType pt = new ReportingServiceAddressingLocator().getReportingPortTypePort(endpoint);
	pt = GCUBERemotePortTypeContext.getProxy(pt, GCUBEScope.getScope(args[1].trim()));
	String report = pt.getReport(args[2]);
	System.out.println(report);
 
} catch (Exception e) {
	e.printStackTrace();
}
Published Scope State Resource

The Published Scope State Resource is a view on the current composition of the Scope context published in the IS. Each time the Scope is modified, the view is updated by a Scope Observer and republished. As for publishing, the resource is modeled as a Generic Resource with the following structure: 

<Resource version="0.4.x">
  <ID>8ff2a1f0-674d-11df-94fa-c1efdde5fc2d</ID>
  <Type>GenericResource</Type>
  <Scopes>
    <Scope>/gcube/devsec/CNR-VRE</Scope>
  </Scopes>
  <Profile>
    <SecondaryType>VRE/VO/Infrastructure</SecondaryType>
    <Name> display name </Name>
    <Description>... a description...</Description>
    <Body>
      <Scope>managed scope</Scope>
      <!--- URI of the RM instance managing the scope-->
      <Service>http://host:port/wsrf/services/gcube/vremanagement/ResourceManager</Service>
      <Creator>...</Creator>
      <Designer>...</Designer>
      <StartTime> creation time </StartTime>
      <EndTime>automatic disposal time (future, not considered now)</EndTime>
      <SecurityEnabled>true/false</SecurityEnabled>
      <!-- list of scoped resources joining the scope-->
      <ScopedRescources>
        <ScopedRescource>
          <ResourceID> resource identifier </ResourceID>
          <ResourceType> resource type </ResourceType>
          <JointTime> when the resource joined the scope (date and time)</JointTime>
        </ScopedRescource>
        <ScopedRescource>
         <!-- -->
        </ScopedRescource>
        <ScopedRescource>
         <!-- -->
        </ScopedRescource>
      </ScopedRescources>
    </Body>
  </Profile>
</Resource>

Do note that the first part of the information reported is set by invoking the ChangeOptions operation.

Here is an example of Scope State Resource published in the IS:

On Resource Brokering

When a new service is requested to be added to the scope an appropriate node must be selected in order to host a new instance of the service. This work is performed by a Resource Broker in charge of matching available resources with caller requests and service's requirement and considering infrastructure information statuses during matchmaking.

The Resource Manager service defines a simple interface to which any broker must comply to: 

 
package org.gcube.vremanagement.resourcemanager.impl.brokerage;
 
import java.util.Set;
import org.gcube.common.core.scope.GCUBEScope;
import org.gcube.vremanagement.resourcemanager.impl.resources.ScopedDeployedService;
 
/**
 * 
 * Models the expected behavior of a Broker. 
 * A Broker is an entity that can create Deployment Plans for a set of {@link ScopedDeployedService}s to be deployed
 *
 * @author Manuele Simi (ISTI-CNR)
 *
 */
public interface Broker {
 
	/**
	 * Initializes the Broker instance
	 * @param scope the scope where the broker will act
	 * @throws Exception if the initialization fails (depends on the broker's nature)
	 */
	public void initialize(GCUBEScope scope) throws Exception;
 
	/**
	 * Creates a deployment plan
	 * @param services the services to deploy
	 * @param suggestedGHNs the GHNs suggested for the deployment, if specified, only these nodes will be used by the planner
	 * @throws Exception if the preparation of the plan fails
	 */
	public void makePlan(Set<ScopedDeployedService> services, String[] suggestedGHNs) throws Exception;
 
	/**
	 * Sends feedback to the broker about the execution of the plan
	 * @param services the services belonging the plan
	 * @throws Exception if the broker is not able to manage the feedback
	 */
	public void sendFeedback(Set<ScopedDeployedService> services) throws Exception;
 
}

Then, the RM instance can be configured to dynamically load and exploit any class implementing this interface. This configuration setting is done via the resourceBrokerClass parameter in the service's JNDI: 

 
<service name="gcube/vremanagement/ResourceManager/service">
 
	 <environment 
		name="resourceBrokerClass" 
	 	value="..."
	 	type="java.lang.String"
	 	override="false" />
 
</service>

The current version of the RM is shipped with 2 Broker implementation descibed in the following subsections.

The Internal Broker

To enable this broker, the value of the resourceBrokerClass parameter has to be set to org.gcube.vremanagement.resourcemanager.impl.brokerage.InternalBroker. This broker is a very simple one and it has to be used when no other broker is available. In fact, the internal broker performs a minimal work and it's not capable to matchmake nodes and services. It can only manage requests where there is a list of suggested gHN per each service or per request. In this case, it checks the availability of those gHNs and then prepare the plan according to the caller suggestion (when a gHN is suggested for a specific service) or a simple round robin algorithm (when a list of gHN is suggested for the entire request).

The Service Broker

To enable this broker, the value of the resourceBrokerClass parameter has to be set to org.gcube.vremanagement.resourcemanager.impl.brokerage.ServiceBroker. This broker is in reality an interface over the Resource Broker service and can be used when an instance of such a service is available in the VO.

By default, the Resource Manager is configured to use this broker.

Broker's Report

If the deployment operations reach the broker's stage, a dedicated section of the Resource Report shows the result of the query to the selected broker.

The following report shows the case where two services were requested to be added, they did, but no running instance was activated for them, since the broker didn't match the user's request with the actual state of the infrastructure. In particular, here the caller asked to deploy on a specific gHN which turned out to do not exist. 

<?xml version="1.0" encoding="UTF-8"?>
<ResourceReport>
  <ID>97d510b0-674d-11df-94fa-c1efdde5fc2d</ID>
  <Operation>AddResources</Operation>
  <LastUpdate>2010-05-24T17:01:26+01:00</LastUpdate>
  <TargetScope>/gcube/devsec/CNR-VRE</TargetScope>
  <Resources>
    <Resource>
      <ID>25bfb640-04e0-11de-b58c-828df53a36e6</ID>
      <Type>Service</Type>
      <Status>ADDED</Status>
    </Resource>
    <Resource>
      <ID>26774670-04e0-11de-b58c-828df53a36e6</ID>
      <Type>Service</Type>
      <Status>ADDED</Status>
    </Resource>
  </Resources>
  <DeploymentPlanCreation>
    <Status>FAILED</Status>
    <Message>The creation of the Deployment Plan failed. Broker says: The GHNs required in the plan request are not available. Suggested GHN: f9f5dfa0-674c-11df-b094-a90b5bbcf085</Message>
  </DeploymentPlanCreation>
  <Services>
    <Service>
      <ID>25bfb640-04e0-11de-b58c-828df53a36e6</ID>
      <Class>MetadataManagement</Class>
      <Name>MetadataManager</Name>
      <Version>1.00.00</Version>
      <DeploymentActivity>The report is still not available for this service</DeploymentActivity>
    </Service>
    <Service>
      <ID>26774670-04e0-11de-b58c-828df53a36e6</ID>
      <Class>MetadataManagement</Class>
      <Name>XMLIndexer</Name>
      <Version>1.00.00</Version>
      <DeploymentActivity>The report is still not available for this service</DeploymentActivity>
    </Service>
  </Services>
</ResourceReport>

Here it is the report for the same services when everything goes OK: 

<?xml version="1.0" encoding="UTF-8"?>
<ResourceReport>
  <ID>a771bad0-674f-11df-94fa-c1efdde5fc2d</ID>
  <Operation>AddResources</Operation>
  <LastUpdate>2010-05-24T17:16:10+01:00</LastUpdate>
  <TargetScope>/gcube/devsec/CNR-VRE</TargetScope>
  <Resources>
    <Resource>
      <ID>25bfb640-04e0-11de-b58c-828df53a36e6</ID>
      <Type>Service</Type>
      <Status>ADDED</Status>
    </Resource>
    <Resource>
      <ID>26774670-04e0-11de-b58c-828df53a36e6</ID>
      <Type>Service</Type>
      <Status>ADDED</Status>
    </Resource>   
  </Resources>
  <DeploymentPlanCreation>
    <Status>SUCCESS</Status>
    <Message>The Deployment Plan was successfully created</Message>
  </DeploymentPlanCreation>
  <Services>
  <!-- services' section here-->
 
  </Services>
</ResourceReport>

Test-suite

The Resource Manager comes with a test-suite package allowing to test its functionalities.The test-suite is completely independent and does not require any other gCube package, except than a local gCore installation. The package is composed by a set of classes, sample configuration files and scripts ready to be executed. 


|-lib
|--org.gcube.vremanagement.resourcemanager.testsuite.jar
|
|-samples
|---...
|
|-addresource.sh
|-getreport.sh
|-changescopeoptions.sh
|-...

Each script allows to test a different service's operation or group of operations logically related. In the following, an explanation of each script and its usage is provided.

Add Resource(s)

The AddResources script invokes the AddResources operation to add new GCUBEResources on a scope. It must be executed as follows 

./addresource.sh <ResourceManager host> <ResourceManager port> <configuration file>

Clearly, the Resource Manager host and port are the information needed to contact the Resource Manager instance, while the configuration file reports the information about the resources to add or create and the scope in which they have to be managed.

This is an example of configuration file requesting to add 4 gCube Resources to the Scope /gcube/devsec/CNR-VRE: 

#global properties
 
numOfServicesToAdd=0
numOfResourcesToAdd=4
targetScope=/gcube/devsec/CNR-VRE
callerScope=/gcube/devsec/CNR-VRE
 
# resources' list
resource.1.id=9ad3ce60-27aa-11df-9df7-e0bf8afcd20f
resource.1.type=Collection
resource.2.id=6e6e2ce0-1ca5-11df-a0fe-c8f6680864ab
resource.2.type=RunningInstance
resource.3.id=a5db71f0-27aa-11df-8ee2-c1a4489ea34b
resource.3.type=MetadataCollection
resource.4.id=df899630-82ae-11de-a6cc-f15c9cf886be
resource.4.type=GenericResource

If everything works, the following report is produced (it can be retrieved via the getReport script): 

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ResourceReport>
  <ID>bdffb830-33a4-11df-97f6-8d3129726fea</ID>
  <LastUpdate>2010-03-19T18:14:10-04:00</LastUpdate>
  <TargetScope>/gcube/devsec/CNR-VRE</TargetScope>
  <Resources>
    <Resource>
      <ID>a5db71f0-27aa-11df-8ee2-c1a4489ea34b</ID>
      <Type>MetadataCollection</Type>
      <Status>ADDED</Status>
      <ErrorDescription>-</ErrorDescription>
    </Resource>
    <Resource>
      <ID>6e6e2ce0-1ca5-11df-a0fe-c8f6680864ab</ID>
      <Type>RunningInstance</Type>
      <Status>ADDED</Status>
      <ErrorDescription>-</ErrorDescription>
    </Resource>
    <Resource>
      <ID>df899630-82ae-11de-a6cc-f15c9cf886be</ID>
      <Type>GenericResource</Type>
      <Status>ADDED</Status>
      <ErrorDescription>-</ErrorDescription>
    </Resource>
    <Resource>
      <ID>9ad3ce60-27aa-11df-9df7-e0bf8afcd20f</ID>
      <Type>Collection</Type>
      <Status>ADDED</Status>
      <ErrorDescription>-</ErrorDescription>
    </Resource>
  </Resources>
  <Services/>
</ResourceReport>
Remove Resource(s)

The RemoveResources script invokes the RemoveResources operation to remove GCUBEResources from a scope. It must be executed as follows 

./removeresource.sh <ResourceManager host> <ResourceManager port> <configuration file>

This is an example of configuration file requesting to remove a Collection from the Scope /gcube/devsec/CNR-VRE: 

#global properties
numOfServicesToRemove=0
numOfResourcesToRemove=1
targetScope=/gcube/devsec/CNR-VRE
callerScope=/gcube/devsec/CNR-VRE
 
# resources' list
resource.1.id=9ad3ce60-27aa-11df-9df7-e0bf8afcd20f
resource.1.type=Collection

If everything works, the following report is produced (it can be retrieved via the getReport script): 

<?xml version="1.0" encoding="UTF-8"?>
<ResourceReport>
  <ID>cfb41e90-3863-11df-9bfb-cfdb92619067</ID>
  <LastUpdate>2010-03-26T00:11:58+01:00</LastUpdate>
  <TargetScope>/gcube/devsec/CNR-VRE</TargetScope>
  <Resources>
    <Resource>
      <ID>9ad3ce60-27aa-11df-9df7-e0bf8afcd20f</ID>
      <Type>Collection</Type>
      <Status>REMOVED</Status>
    </Resource>
  </Resources>
  <Services/>
</ResourceReport>
Create a new Scope

The CreateScope script invokes the CreateScope operation in order to initialise a new Scope and assign to it the first set of gCube resources. It must be executed as follows 

./createScope.sh <ResourceManager host> <ResourceManager port> <configuration file>

This is an example of configuration file requesting to

  • create a new VRE (named /gcube/devsec/CNR-VRE)with the indicated options
  • add to it 4 existing gCube Resources
  • deploy 2 new Services in the new Scope
######################################
# Sample VRE creation - properties file
######################################
 
targetScope=/gcube/devsec/CNR-VRE
callerScope=/gcube/devsec/CNR-VRE
 
#Scope properties
creator=CN=Manuele Simi, L=NMIS-ISTI, OU=Personal Certificate, O=INFN, C=IT
designer=CN=Manuele Simi, L=NMIS-ISTI, OU=Personal Certificate, O=INFN, C=IT
securityenabled=false
displayName=CNR-VRE
description=Sample VRE for testing purposes
 
#Scope resources
numOfServicesToAdd=2
numOfResourcesToAdd=4
 
# resources' list
resource.1.id=9ad3ce60-27aa-11df-9df7-e0bf8afcd20f
resource.1.type=Collection
resource.2.id=6e6e2ce0-1ca5-11df-a0fe-c8f6680864ab
resource.2.type=RunningInstance
resource.3.id=a5db71f0-27aa-11df-8ee2-c1a4489ea34b
resource.3.type=MetadataCollection
resource.4.id=df899630-82ae-11de-a6cc-f15c9cf886be
resource.4.type=GenericResource
 
# services' list
service.1.name=MetadataManager
service.1.class=MetadataManagement
service.1.version=1.00.00
service.2.name=XMLIndexer
service.2.class=MetadataManagement
service.2.version=1.00.00
 
#GHN IDs on which the service will be deployed
GHNSet=84c3aa10-3857-11df-b245-cbf0ea5a1843,05921df0-382d-11df-8c8f-f5ea2a36aefa
Dispose an existing Scope

The DisposeScope script invokes the DisposeScope operation in order to throw away an entire scope. It must be executed as follows 

./disposeScope.sh <ResourceManager host> <ResourceManager port> <Scope>
Get Report

The GetReport script invokes the GetReport operation to retrieve a Resource Report. It must be executed as follows 

./getreport.sh <ResourceManager host> <ResourceManager port> <caller scope> <report ID>

This request will print out the resource report related to the given report ID.

Retrieved from "https://wiki.gcube-system.org/index.php?title=Resource_Manager&oldid=27341"