Difference between revisions of "Metadata Manager"

From Gcube Wiki
Jump to: navigation, search
(MetadataManagerService)
(Getting all the MOIDs of a Metadata Collection)
 
(82 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
=== Introduction ===
 
=== Introduction ===
  
The Metadata Manager service manages the modelling of arbitrary metadata relationships (IDB-relationships). The only assumption it does is that the metadata objects are serialized as well-formed XML documents.
+
The Metadata Manager manages the modelling of arbitrary metadata relationships (IDB-relationships) in accordance with the [[Metadata_Management#gCube_Metadata_Model|gCube Metadata Model]]. The only assumption it does is that the Metadata Objects are serialized as well-formed XML documents.
 
The service has a two-fold role:
 
The service has a two-fold role:
 
*to manage Metadata Objects and Metadata Collections
 
*to manage Metadata Objects and Metadata Collections
 
*to establish secondary role-typed links. Such relationships can be in place between any type of Information Object and in the scope of a Collection or not
 
*to establish secondary role-typed links. Such relationships can be in place between any type of Information Object and in the scope of a Collection or not
  
Clients interacts also with the Metadata Manager library for a higher level of abstraction which simplifies clients interaction with the service.
+
The service relies on the [[Storage_Management|Storage Management]] Service to persist the information handled.
  
=== Metadata Object ===
+
Clients interact also with the [[Metadata_Manager#MMLibrary|Metadata Manager library]] for a higher level of abstraction which simplifies clients interaction with the service.
  
A Metadata Object (MO) is a source Information Object for an IDB-relationship. An MO can be associated to one and only one target Information Object (even if in the context of different Metadata Collections) which is described by the MO. An MO here is always intended as an XML document wrapped in a well defined envelope.
+
=== Metadata Manager Service ===
As to content, the Metadata Manager Service adopts an exchange model to add a number of system-level properties needed for the appropriate management of the Metadata Objects within the context of the Metadata Manager Service.
+
The main functionality of the Metadata Manager Service is the management of Metadata Objects, Metadata Collection and their relationships.
  
=== The Metadata Manager Components ===
+
From an architectural point of view, the service adopts the Factory pattern and  Collection Managers are implemented as stateful WS-Resources. Therefore, the service is composed by:
  
The main functionality of the Metadata Manager components is the management of Metadata Objects, Metadata Collection and their relationships.
+
* '''MetadataManagerFactory''', a portType that creates new Collection Managers and offers some cross-Collection operations
To operate over Metadata Collections, the Metadata Catalog instantiates Collection Managers (or shortly, Managers) for each collection. A Collection Manager is the access point to all the possible operations over a specific Metadata Collection.
+
* '''MetadataManager''', a portType that operates over Metadata Collections (MCs) and on Metadata Objects as Elements, i.e. members of a specific Metadata Collection
From an architectural point of view, the Metadata Manager adopts the Factory pattern and  Collection Managers are implemented as a stateful WS-Resource. Physically, the service is composed by:
+
  
*the '''MetadataManagerFactory''',  a factory service that creates new Collection Managers and offers some cross-Collection operations
+
To operate over Metadata Collections, the service instantiates a Collection Manager (or shortly, Manager) for each collection. A Collection Manager is the access point to all the possible operations over a specific Metadata Collection.
*the '''MetadataManagerService''', a service that operates over Metadata Collections (MCs) and on Metadata Objects as Elements, i.e. members of a specific Metadata Collection
+
  
 +
===== MetadataManagerFactory portType =====
  
==== MetadataManagerFactory ====
+
The <em>MetadataManagerFactory</em> creates new Collection Managers and offers some cross-Collection operations. Moreover, it operates on Metadata Objects as Information Objects related to other Information Objects and not as Members of Metadata Collections.
  
The MetadataManagerFactory Service creates new Collection Managers and offers some cross-Collection operations. Moreover, it operates on Metadata Objects as Information Objects related to other Information Objects and not as Members of Metadata Collections.
+
====== Creating a new Metadata Collection ======
  
*''EndPointReferenceType createManager(CollectionID, params)'':  This operation takes a set of creation parameters and creates a new Manager in order to manage a Metadata Collection bound to such a Collection. If a Metadata Collection with the specified Metadata characteristics does not exist, the Manager creates the Metadata Collection, binds it with the Document Collection with the given secondary role relationship and publishes its profile in the IS. The Creation parameters are a set of key-value; the following keys are defined in the MMLibrary and accepted by the operation:
+
----
  
These ones are mandatory:
+
<code>EndPointReferenceType '''createManager'''(CollectionID, params)</code>:  This operation takes as input a Content Collection ID and a set of creation parameters. Then, it creates a new Manager in order to manage a Metadata Collection bound to the given Content Collection. If a Metadata Collection with the specified Metadata characteristics does not exist, the Manager creates the Metadata Collection, binds it with the Content Collection and publishes its profile in the IS.
** COLLECTIONNAME -> name of the collection
+
** METADATANAME -> metadata name (e.g. “dc”)
+
** METADATALANG -> metadata language (e.g. “English”), as specified in the ISO 639-2
+
** METADATAURI -> metadata URI: the XML Schema that defines the MO payloads
+
** ROLE -> secondary role
+
  
The optional ones are:
+
:The Creation parameters are a set of key-value pairs; the following keys are defined in the [[Metadata_Manager#The_MMLibrary|MMLibrary]] (<code>org.gcube.metadatamanagement.mmlibrary.descriptions.CollectionDescription.CreationParameters</code>) and accepted as parameters:
** DESCRIPTION -> description
+
** INDEXABLE -> if the collection is indexable or not (“True”/”False”)
+
** USER -> if the collection is a user collection or not (“True”/”False”)
+
** RELATEDBY_COLLID -> the id of the source content collection
+
** RELATEDBY_ROLE -> the source XML Schema from which the current one has been generated
+
** GENERATEDBY_COLLID -> the source Metadata Collection from which the current one has been generated (by the Metadata Broker), if any
+
** GENERATEDBY_URI -> the URI of the source schema of the collection from which it has been generated
+
  
**Among the others, two parameters can modify the way in which the Metadata Collection is managed:
+
:Mandatory parameters:
***Indexable: if the new Metadata Collection is indexable, the Manager creates also a new MetadataXMLIndexer (see Section 4.2.2.3) for such a Collection using the XMLIndexer Service;
+
:* COLLECTIONNAME -> name of the collection
***User: a user Collection is shown in the Portals and an end-user can operate on it; a non-user Collection is intended for internal puporses (like to collect parts of persisted Indexes).
+
:* METADATANAME -> metadata name (e.g. “dc”)
 +
:* METADATALANG -> metadata language (e.g. “English”), as specified by the [http://www.loc.gov/standards/iso639-2/ ISO 639-2]
 +
:* METADATAURI -> metadata URI: the XML Schema that defines the MO payloads
 +
:* ROLE -> secondary role
 +
:* RELATEDBY_COLLID -> the ID of the Content Collection (it must report the same value of the first parameter of the invocation, i.e. ''CollectionID'')
  
* '''ResultSetService'''
+
: Optional parameters:
 +
:* DESCRIPTION -> description
 +
:* INDEXABLE -> if the collection is indexable or not (“True”/”False”), default is true
 +
:* USER -> if the collection is a user collection or not (“True”/”False”), default is true
 +
:* RELATEDBY_ROLE -> the source XML Schema from which the current one has been generated, if any
 +
:* GENERATEDBY_COLLID -> the source Metadata Collection from which the current one has been generated (by the Metadata Broker), if any
 +
:* GENERATEDBY_URI -> the URI of the source schema of the collection from which it has been generated, if any
  
*''EndPointReferenceType  createManagerFromCollection (MetadataCollectionID)'': This operation takes a Metadata Collection ID. It returns:
+
:Among the others, two parameters can modify the way in which the Metadata Collection is managed:
**the related CollectionManager, if it already exists
+
::*Indexable: if the new Metadata Collection is indexable, the Manager creates also a new XML Index for such a Collection using the [[XML_Indexer|XMLIndexer]] Service;
**creates a new CollectionManager and returns its EPR, if the Metadata Collection exists
+
::*User: a user Collection is shown in the Portals and an end-user can operate on it; a non-user Collection is intended for internal purposes (like to collect parts of persisted Indexes).
**an error, if the Collection ID is not valid
+
  
*''MOID addMetadata(ObjectID, MO, SecondaryRole)'':  This operation takes a new non-collectable Metadata Object and  
+
This example shows how to invoke the createManager operation (it reads the input from a properties file):
 +
 
 +
<source lang="java">
 +
import java.io.IOException;
 +
import java.util.Properties;
 +
 
 +
import org.gcube.common.core.contexts.GCUBERemotePortTypeContext;
 +
import org.gcube.common.core.scope.GCUBEScope;
 +
import org.gcube.common.core.utils.logging.GCUBEClientLog;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.CollectionParameter;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.CreateCollectionParameters;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.CreateManagerRequest;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.CreateManagerResponse;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.MetadataManagerFactoryPortType;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.service.MetadataManagerFactoryServiceAddressingLocator;
 +
import org.gcube.metadatamanagement.mmlibrary.descriptions.CollectionDescription.CreationParameters;
 +
import org.apache.axis.message.addressing.Address;
 +
import org.apache.axis.message.addressing.EndpointReferenceType;
 +
 
 +
/**
 +
* Tester for <em>createManager</em> operation of the Metadata Manager service
 +
*
 +
* @author Manuele Simi (ISTI-CNR)
 +
*
 +
*/
 +
public final class CreateManagerTest {
 +
 
 +
private static Properties resources = new Properties();
 +
 +
private static GCUBEClientLog logger = new GCUBEClientLog(CreateManagerTest.class);
 +
 +
/**
 +
* @param args
 +
* <OL>
 +
* <LI> MM host
 +
* <LI> MM port
 +
* <LI> Scope
 +
* <LI> Properties file
 +
* </OL>
 +
* @throws Exception
 +
*/
 +
public static void main(String[] args) throws Exception {
 +
 +
if (args.length != 4) {
 +
logger.error("Usage: CreateManagerTest <MM host> <MM port> <Scope> <properties file>");
 +
return;
 +
}
 +
logger.debug("Loading props file");
 +
try {
 +
resources.load(CreateManagerTest.class.getResourceAsStream("/" + args[3]));
 +
} catch (IOException e1) {
 +
logger.error("Unable to load the properties file", e1);
 +
Runtime.getRuntime().exit(1);
 +
 +
}
 +
String factoryURI = "http://" + args[0]+ ":" + args[1] + "/wsrf/services/gcube/metadatamanagement/metadatamanager/MetadataManagerFactory";
 +
String scope = args[2];
 +
 +
MetadataManagerFactoryServiceAddressingLocator factoryLocator = new MetadataManagerFactoryServiceAddressingLocator();
 +
MetadataManagerFactoryPortType mFactory;
 +
try {
 +
EndpointReferenceType factoryEPR = new EndpointReferenceType();
 +
factoryEPR.setAddress(new Address(factoryURI));
 +
logger.info("Creating the new Factory portType..");
 +
 
 +
mFactory= factoryLocator.getMetadataManagerFactoryPortTypePort(factoryEPR);
 +
 +
CreateManagerRequest request = new CreateManagerRequest();
 +
request.setCollectionID(resources.getProperty("RelatedContentCollectionID"));
 +
 +
// initialise all the params
 +
CollectionParameter [] parameters = new CollectionParameter[11];
 +
for ( int i =0; i<parameters.length; i++ )
 +
parameters[i] = new CollectionParameter();
 +
 +
parameters[0].setName(CreationParameters.COLLECTIONNAME.toString());
 +
parameters[0].setValue(resources.getProperty("Name"));
 +
if (parameters[0].getValue() == null)
 +
parameters[0].setValue("test-collection");
 +
 +
parameters[1].setName(CreationParameters.METADATALANG.toString());
 +
parameters[1].setValue(resources.getProperty("MetadataLang"));
 +
 +
parameters[2].setName(CreationParameters.METADATANAME.toString());
 +
parameters[2].setValue(resources.getProperty("MetadataName"));
 +
 +
parameters[3].setName(CreationParameters.METADATAURI.toString());
 +
parameters[3].setValue(resources.getProperty("MetadataURI"));
 +
 +
parameters[4].setName(CreationParameters.ROLE.toString());
 +
parameters[4].setValue(resources.getProperty("Role"));
 +
 +
parameters[5].setName(CreationParameters.INDEXABLE.toString());
 +
parameters[5].setValue(resources.getProperty("IsIndexable"));
 +
 +
parameters[6].setName(CreationParameters.EDITABLE.toString());
 +
parameters[6].setValue(resources.getProperty("IsEditable"));
 +
 +
parameters[7].setName(CreationParameters.RELATEDBY_COLLID.toString());
 +
parameters[7].setValue(resources.getProperty("Related-by-collID"));
 +
 +
parameters[8].setName(CreationParameters.RELATEDBY_ROLE.toString());
 +
parameters[8].setValue(resources.getProperty("Related-by-role"));
 +
 +
parameters[9].setName(CreationParameters.DESCRIPTION.toString());
 +
parameters[9].setValue(resources.getProperty("Description"));
 +
 +
parameters[10].setName(CreationParameters.VALIDATING.toString());
 +
parameters[10].setValue(resources.getProperty("IsValidated"));
 +
 +
CreateCollectionParameters ccp = new CreateCollectionParameters();
 +
ccp.setParams(parameters);
 +
request.setCollectionParameters(ccp);
 +
CreateManagerResponse response = new CreateManagerResponse();
 +
try{
 +
mFactory = GCUBERemotePortTypeContext.getProxy(mFactory,GCUBEScope.getScope(scope), 120000);              
 +
response = mFactory.createManager(request);
 +
}catch(Exception e){
 +
logger.error("", e);
 +
return;
 +
}
 +
logger.info("Metadata Collection successfully created, at "+response.getEndpointReference().toString());
 +
} catch (Exception e) {
 +
logger.error("", e);
 +
}
 +
}
 +
}
 +
 
 +
</source>
 +
 
 +
====== Creating a new Manager from an existing Metadata Collection ======
 +
 
 +
----
 +
 
 +
<code>EndPointReferenceType  '''createManagerFromCollection''' (MetadataCollectionID)</code>:  This operation takes a Metadata Collection ID as input and returns:
 +
* the related CollectionManager, if it already exists
 +
* an EPR of a newly created CollectionManager, if the Metadata Collection exists
 +
* an error, if the Collection ID is not valid
 +
 
 +
The following example shows how to invoke the operation:
 +
 
 +
<source lang="java">
 +
import org.apache.axis.message.addressing.Address;
 +
import org.apache.axis.message.addressing.EndpointReferenceType;
 +
import org.gcube.common.core.contexts.GCUBERemotePortTypeContext;
 +
import org.gcube.common.core.scope.GCUBEScope;
 +
import org.gcube.common.core.utils.logging.GCUBEClientLog;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.CreateManagerResponse;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.MetadataManagerFactoryPortType;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.MetadataManagerPortType;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.service.MetadataManagerFactoryServiceAddressingLocator;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.service.MetadataManagerServiceAddressingLocator;
 +
 
 +
/**
 +
* Tester for <em>CreateManagerFromCollection</em> operation of the Metadata Manager service
 +
*
 +
*  @author Manuele Simi (ISTI-CNR)
 +
*/
 +
public class CreateManagerFromCollection {
 +
 
 +
private static GCUBEClientLog logger = new GCUBEClientLog(CreateManagerFromCollection.class);
 +
 +
/**
 +
* @param args
 +
* <OL>
 +
* <LI> MM host
 +
* <LI> MM port
 +
* <LI> Scope
 +
* <LI> Metadata Collection ID
 +
* </OL>
 +
*/
 +
public static void main(String[] args) {
 +
 +
if (args.length != 4) {
 +
System.err.println("Usage: CreateManagerFromCollection <MM host> <MM port> <Scope> <collection ID>");
 +
return;
 +
}
 +
String factoryURI = "http://" + args[0]+ ":" + args[1] + "/wsrf/services/gcube/metadatamanagement/metadatamanager/MetadataManagerFactory";
 +
String scope = args[2];
 +
String collectionID = args[3];     
 +
 +
MetadataManagerPortType manager = CreateManagerFromCollection.createManager(factoryURI, scope, collectionID);
 +
if (manager != null)
 +
logger.info("Manager successfully created");
 +
}
 +
 +
public static MetadataManagerPortType createManager(String factoryURI, String scope, String collectionID) {
 +
 +
MetadataManagerPortType manager = null;
 +
 +
try {
 +
EndpointReferenceType factoryEPR = new EndpointReferenceType();
 +
factoryEPR.setAddress(new Address(factoryURI));
 +
logger.info("Creating the new Factory portType");
 +
MetadataManagerFactoryServiceAddressingLocator factoryLocator = new MetadataManagerFactoryServiceAddressingLocator();
 +
MetadataManagerFactoryPortType mFactory= factoryLocator.getMetadataManagerFactoryPortTypePort(factoryEPR);
 +
mFactory = GCUBERemotePortTypeContext.getProxy(mFactory,GCUBEScope.getScope(scope));              
 +
CreateManagerResponse response = mFactory.createManagerFromCollection(collectionID);
 +
EndpointReferenceType instanceEPR = response.getEndpointReference();
 +
logger.info("Creating manager at " + instanceEPR.toString());
 +
MetadataManagerServiceAddressingLocator  instanceLocator = new MetadataManagerServiceAddressingLocator();
 +
manager = instanceLocator.getMetadataManagerPortTypePort(instanceEPR);
 +
 +
} catch (Exception e) {
 +
logger.error("Unable to create the Manager", e);
 +
Runtime.getRuntime().exit(1);
 +
}
 +
return manager;
 +
}
 +
 +
}
 +
</source>
 +
 
 +
* <code>ArrayOfMetadata[] '''getMetadata'''(MetadataSelector[])</code>
 +
:...
 +
 
 +
*<code>MOID '''addMetadata'''(ObjectID, MO, SecondaryRole)</code>:  This operation takes a new non-collectable Metadata Object and  
 
**completes the metadata header information (e.g. the MOID, if it is not specified)
 
**completes the metadata header information (e.g. the MOID, if it is not specified)
**stores (or updates if the MOID is already included in the MO header) the object on the Storage Management Service as Information Object
+
**stores (or updates if the MOID is already included in the MO header) the object on the [[Storage_Management|Storage Management]] Service as Information Object
 
**creates a <is-described-by, <SecondaryRole>>  binding in the Storage Management Service between the Metadata Object and the Information Object identified by the given Object ID
 
**creates a <is-described-by, <SecondaryRole>>  binding in the Storage Management Service between the Metadata Object and the Information Object identified by the given Object ID
 
**returns the assigned MOID
 
**returns the assigned MOID
  
*''void deleteMetadata(MOID)'':  This operation deletes from the Storage Management Service the Metadata Object identified by the given ID.
+
*<code>void '''deleteMetadata'''(MOID)</code>:  This operation deletes from the Storage Management Service the Metadata Object identified by the given ID.
  
*''(ObjectID, (MO, SecondaryRole)[])[] getMetadata ((ObjectID,  SecondaryRole, CollectionID, Rank)[])'':  For each given ObjectID, this operation returns the Metadata Objets. They are:  
+
*<code>(ObjectID, (MO, SecondaryRole)[])[] '''getMetadata''' ((ObjectID,  SecondaryRole, CollectionID, Rank)[])</code>:  For each given ObjectID, this operation returns the Metadata Objets. They are:  
 
**bound with the specified secondary role (the primary role is, of course, is-described-by) to the Information Object identified by that ObjectID  
 
**bound with the specified secondary role (the primary role is, of course, is-described-by) to the Information Object identified by that ObjectID  
**members of the specified Metadata Collection. The operation relies on the String[] retrieveReferred(String targetObjectID, String role, String secondaryrole) operation of the Storage Management Service.
+
**members of the specified Metadata Collection
  
 +
==== MetadataManager portType ====
 +
The MetadataManager allows user to manipulate a Metadata Collection. It provides the following operations:
  
==== MetadataManagerService ====
+
*<code>AddElementsResponse '''addElements'''(addElements)</code>: Adds the given elements to the MetadataCollection
The MetadataManagerService allows user to manipulate a MetadataCollection, the operation provided are:
+
**addElements is a list, where each item of the list is a valid ID of an IO available in the Storage and a list of new MOs to bind to such IO
 +
**AddElementsResponse is a list, which is a map with the ID of the IO and the list of MOs bound to it in the operation
  
*''String getMetadataCollectionID()'': Gets the Metadata Collection identifier
+
*<code>String '''getMetadataCollectionID'''()</code>: Gets the Metadata Collection identifier
*''void deleteCollection()'': Deletes the Metadata Collection and destroys the related Collection Manager.  
+
*<code>void deleteCollection()</code>: Deletes the Metadata Collection and destroys the related Collection Manager.  
 
**The elements are also removed from the storage service, if they do not belong to other Metadata Collections  
 
**The elements are also removed from the storage service, if they do not belong to other Metadata Collections  
**If the Collection is indexable, it is also destroyed from the XMLIndexer service
+
**If the MetadataCollection is indexable, it is also destroyed from the XMLIndexer service
*''ListOfUpdateElementsResults updateElements(UpdateElements)'': Updates the given MOs either in the Storage Management and in the XMLIndexer (if the Collection is Indexable)
+
 
**UpdateElements is a list of XML String representation of the MOs to update
+
*<code>ListOfUpdateElementsResults '''updateElements'''(UpdateElements)</code>: Updates the given MOs either in the Storage Management and in the XMLIndexer (if the Collection is Indexable)
 +
**UpdateElements is a list of XML String representation of the MetadataObjects(MOs) to update
 
**ListOfUpdateElementsResults is an array of the same cardinality of the input string. Each item is a boolean value reporting true if the MO in the same position in the input array was successfully updates, false otherwise.
 
**ListOfUpdateElementsResults is an array of the same cardinality of the input string. Each item is a boolean value reporting true if the MO in the same position in the input array was successfully updates, false otherwise.
*''AddElementsResponse addElements(addElements)'': Adds the given elements to the MetadataCollection
+
 
**addElements is a list, where each item of the list is a valid ID of an IO available in the Storage and a list of new MOs to bind to such IO
+
*<code>GetElementResponse '''getElements'''(InformationObjectIDList)</code>: Gets the MOs bound to the  by the given MOIDs
**AddElementsResponse is a list, which is a map with the ID of the IO and the list of MOs bound to it in the operation
+
 
*''String getElements(String)'': Adds the given elements to the MetadataCollection using the ResultSetService  
+
===== Getting all the MOIDs of a Metadata Collection =====
**the input string is the reference to an RSLocator with the MOs to store
+
 
 +
----
 +
 
 +
<code>MOIDList '''getMOIDs()'''</code>: Gets all the Metadata Object IDs of the Metadata Collection.
 +
 
 +
A sample usage of the operation is reported below:
 +
 
 +
<source lang="java">
 +
 
 +
import org.gcube.common.core.contexts.GCUBERemotePortTypeContext;
 +
import org.gcube.common.core.scope.GCUBEScope;
 +
import org.gcube.common.core.utils.logging.GCUBEClientLog;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.MOIDList;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.MetadataManagerPortType;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.VoidType;
 +
 
 +
/**
 +
* Tester for <em>GetMOIDs</em> operation of the Metadata Manager service
 +
*
 +
*  @author Manuele Simi (ISTI-CNR)
 +
*/
 +
public class GetMOIDsTest {
 +
 
 +
private static GCUBEClientLog logger = new GCUBEClientLog(GetMOIDsTest.class);
 +
 +
/**
 +
* @param args
 +
* <OL>
 +
* <LI> MM host
 +
* <LI> MM port
 +
* <LI> Scope
 +
* <LI> Metadata Collection ID
 +
* </OL>
 +
*/
 +
public static void main(String[] args) {
 +
String factoryURI = "http://" + args[0]+ ":" + args[1] + "/wsrf/services/gcube/metadatamanagement/metadatamanager/MetadataManagerFactory";
 +
String scope = args[2];
 +
String collectionID = args[3];     
 +
 +
MetadataManagerPortType manager = CreateManagerFromCollection.createManager(factoryURI, scope, collectionID);
 +
try {
 +
manager = GCUBERemotePortTypeContext.getProxy(manager,GCUBEScope.getScope(scope));
 +
MOIDList moids = manager.getMOIDs(new VoidType());
 +
if (moids == null) {
 +
logger.warn("The returned list of MOIDs is null");
 +
return;
 +
}
 +
if (moids.getOID() == null) {
 +
logger.warn("The returned list of MOIDs is empty");
 +
return;
 +
}
 +
logger.info("The returned list of MOIDS includes " + moids.getOID().length + " element(s)");
 +
 +
for (String id : moids.getOID())
 +
logger.info("MOID="+ id);
 +
 +
} catch (Exception e) {
 +
logger.error("Something went wrong", e);
 +
Runtime.getRuntime().exit(1);
 +
}
 +
 
 +
}
 +
 
 +
}
 +
</source>
 +
 
 +
*<code>GetElementsFromMOIDResponse '''getElementsFromMOID'''(MOIDList)</code>: Gets the MOs identified by the given MOIDs, where:
 +
**MOIDList is a list of Metadata Object IDs
 +
**GetElementsFromMOIDResponse contains the MetadataObjects related to the given IDs
 +
 
 +
*<code>MOIDList '''deleteElements'''(MOIDList)</code>: Deletes from storage service the Metadata Object identified by the given ID
 +
**If the MetadataCollection is indexable it deletes the Metadata Object from the related MetadataXMLIndexer using the XMLindexer service
 +
**MOIDList is a list with the IDs of MetadataObjects(MOs), in input the IDs to delete, in output it returns the ID of the elements not deleted
 +
 
 +
*<code>GetAllElementsResponse '''getAllElements'''(void)</code>: returns all the MOs of the current Metadata Collection
 +
 
 +
*<code>String '''addElementsRS'''(String)</code>: Adds the given elements to the MetadataCollection using the ResultSetService  
 +
**the input string is the reference to a RSLocator with the MOs to store, each element of the RS must have the entire MO in its payload
 
**the output string is  the reference to an RSLocator with a list of the stored MOs' IDs  
 
**the output string is  the reference to an RSLocator with a list of the stored MOs' IDs  
*''getElementsFromMOID''
 
*''deleteElements''
 
*''getAllElements''
 
*''addElementsRS''
 
*''getAllElementsRS''
 
*''getElementsRS''
 
  
==== The MMLibrary ====
+
*<code>String '''getAllElementsRS'''()</code>: Returns all the MOs of the current Metadata Collection using the ResultSetService
The library offers a number of facilities for interacting with the creation and manipulation of Metadata Object.
+
**the output String is a reference to an RSLocator with the retrieved elements
  
First of all,the library operates at client-side by moving some of the logic on that side avoiding unnecessary interactions between clients and the Metadata Manager service.
+
*<code>String '''getAllElementsRSWithTimestamp'''(TimestampInterval)</code>: Returns all the MOs of the current Metadata Collection filtered by Date  using the ResultSetService
 +
**the output String is a reference to an RSLocator with the retrieved elements
  
The library offers a a set of method to easily wrap an XML documents in the model exchanged format as required by Metadata Manager service.
+
<code> An example of the operation is reported below:
In addition, the library offers support for  wrap the Object with the appropriate metadata envelope, to create a valid Metadata Object that can be stored on the Metadata Manager Service.
+
<source lang="java">
 +
...
 +
Calendar fromDate= Calendar.getInstance();
 +
fromDate.set(2001,10,1);
 +
Calendar toDate= Calendar.getInstance();
 +
toDate.set(2010,1,1);
 +
String locator = manager.getAllElementsRSWithTimestamp(new TimestampInterval(fromDate.getTimeInMillis(),toDate.getTimeInMillis()));
 +
...
 +
</source>
 +
</code>
  
In addition, the library permits to extract the basic MO information from a string representing a Metadata Object.
+
*<code>String '''getElementsRS'''(String)</code>:Gets the list of MOs describing the Information Object included in the input list using the ResultSetService. The returned MOs belong the current Metadata Collection
 +
**the input string is the reference to an RSLocator with the OIDs
 +
**the output string is the reference to an RSLocator with the retrieved MOs
  
The new feature of the Metadata Manager: "generated-by" relationship between metadata objects
+
===== Getting all the Target Objects of a Metadata Collection =====
The Metadata Manager handle this new relationship between MO
+
To be able to use this feature it is needed to create a Metadata Collection specifying the following CollectionParameter:
+
**the GENERATEDBY_COLLID: the ID of the Metadata Collection from which it has been generated
+
  
Then,the Metadata envelope need to have a new field <generatedByMOID> to specify the Metadata Object from which it has been generated
+
----
The Metadata Manager will take care of this new field and will store the information in the Storage as secondary role, to be able to retrieve it later.
+
  
=== Dependencies ===
+
<code><IO,MOID>[]'''getTargets'''(MOID[])</code>: This operation allows to retrieve per each input MO the described Content Object.
 +
A sample usage of the operation is reported below:
  
These are the dependencies of the Service :
+
<source lang="java">
  
 +
import org.gcube.common.core.contexts.GCUBERemotePortTypeContext;
 +
import org.gcube.common.core.scope.GCUBEScope;
 +
import org.gcube.common.core.utils.logging.GCUBEClientLog;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.GetTargetsObjectResponse;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.MOIDList;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.MetadataManagerPortType;
 +
import org.gcube.metadatamanagement.metadatamanager.stubs.VoidType;
 +
 +
/**
 +
* Tester for <em>GetTargets</em> operation of the Metadata Manager service
 +
*
 +
*  @author Manuele Simi (ISTI-CNR)
 +
*/
 +
public final class GetTargets {
 +
 +
private static GCUBEClientLog logger = new GCUBEClientLog(GetTargets.class);
 +
 +
/**
 +
* @param args
 +
* <OL>
 +
* <LI> MM host
 +
* <LI> MM port
 +
* <LI> Scope
 +
* <LI> Metadata Collection ID
 +
* </OL>
 +
*/
 +
public static void main(String[] args)  {
 +
String factoryURI = "http://" + args[0]+ ":" + args[1] + "/wsrf/services/gcube/metadatamanagement/metadatamanager/MetadataManagerFactory";
 +
String scope = args[2];
 +
String collectionID = args[3];     
 +
 +
MetadataManagerPortType manager = CreateManagerFromCollection.createManager(factoryURI, scope, collectionID);
 +
try {
 +
manager = GCUBERemotePortTypeContext.getProxy(manager,GCUBEScope.getScope(scope));
 +
 +
MOIDList moids = manager.getMOIDs(new VoidType());
 +
if (moids == null) {
 +
logger.warn("The returned list of MOIDs is null");
 +
return;
 +
}
 +
 +
if (moids.getOID() == null) {
 +
logger.warn("The returned list of MOIDs is empty");
 +
return;
 +
}
 +
 +
logger.info("The returned list of MOIDS includes " + moids.getOID().length + " element(s)");
 +
 +
GetTargetsObjectResponse[] mccontents = manager.getTargets(moids).getGetTargetObjectResponse();
 +
if ( (mccontents == null) || (mccontents.length == 0)) {
 +
logger.info("The target Metadata Collection is empty");
 +
return;
 +
}
 +
 +
for (GetTargetsObjectResponse pair : mccontents)
 +
logger.info("IO=" + pair.getInformationObjectID()+ ", MOID="+ pair.getMetadataObjectID());
 +
 +
} catch (Exception e) {
 +
logger.error("Something went wrong", e);
 +
Runtime.getRuntime().exit(1);
 +
}
 +
}
 +
}
 +
 +
</source>
 +
 +
==== Dependencies ====
 +
 +
The MetadataManager Service depends by the following gCube components:
  
 
* '''MMLibrary'''
 
* '''MMLibrary'''
Line 124: Line 489:
 
* '''ContentManagementLibrary'''
 
* '''ContentManagementLibrary'''
  
=== Usage Examples ===
+
=== MMLibrary ===
 +
 
 +
The MMLibrary offers a number of facilities for creating and manipulating Metadata Objects. The library operates at client-side by moving some of the logic on that side avoiding unnecessary interactions between clients and the Metadata Manager service.
 +
 
 +
Basically, it offers a a set of method to easily wrap an XML documents in the model exchanged format as required by Metadata Manager service. i.e. support for wrapping the Objects with the appropriate metadata envelope, in order to create a valid Metadata Object that can be stored on the Metadata Manager Service. 
 +
 
 +
In addition, the library provides some utility classes to extract the MO information from a string representing a serialized Metadata Object.
 +
 
 +
=== The test-suite ===
 +
The Metadata Manager comes with a test-suite package allowing to test its functionalities. The test-suite is completely independent and does not require any other gCube package, except than a local gCore installation.
 +
The package is composed by a set of classes, sample configuration files and scripts ready to be executed.
  
This example shows how to create a MetadataManager:
 
  
 
<pre>
 
<pre>
 +
|-lib
 +
|--org.gcube.metadatamanagement.metadatamanager.testsuite.jar
 +
|
 +
|-samples
 +
|---...
 +
|
 +
|-createCollection.sh
 +
|-createManagerFromCollection.sh
 +
|-getTargets.sh
 +
|-getMOIDs.sh
  
...
+
</pre>
 +
Each script allows to test a different service's operation or group of operations logically related. In the following, an explanation of each script and its usage is provided.
  
                String factoryURI ="http://node2.d.d4science.research-infrastructures.eu:8080/wsrf/services/gcube/metadatamanagement/metadatamanager/MetadataManagerFactory";
+
'''createCollection'''
+
String id = args[0]; // collection name
+
+
MetadataManagerFactoryServiceAddressingLocator factoryLocator = new MetadataManagerFactoryServiceAddressingLocator();
+
//MetadataManagerServiceAddressingLocator  instanceLocator = new MetadataManagerServiceAddressingLocator();
+
MetadataManagerFactoryPortType mFactory;
+
  
try {
+
----
EndpointReferenceType factoryEPR = new EndpointReferenceType();
+
The createCollection script invokes the [[Metadata_Manager#Creating_a_new_Metadata_Collection|CreateCollection]] operation to create a new Metadata Collection. It must be executed as follows:
factoryEPR.setAddress(new Address(factoryURI));
+
+
+
System.out.println("Creating the new Factory portType");
+
  
mFactory= factoryLocator.getMetadataManagerFactoryPortTypePort(factoryEPR);
+
<pre>
 +
./createCollection.sh <MM host> <MM port> <scope> <configuration file>
 +
</pre>
  
System.out.println("Created instance.\n");
+
This is an example of configuration file that can be provided:
+
<source lang="text">
CreateManagerRequest request = new CreateManagerRequest();
+
Name=test-coll
request.setCollectionID(id);
+
Description=collection from MM test-suite
CollectionParameter [] ss = new CollectionParameter[6];
+
Role=is-described-by
CreateCollectionParameters ccp = new CreateCollectionParameters();
+
MetadataName=dc
+
MetadataURI=http://mail.google.com/mail
for ( int i =0; i<ss.length; i++ )
+
MetadataLang=it
ss[i] = new CollectionParameter();
+
IsIndexable=true
+
IsEditable=false
ss[0].setName(CreationParameters.COLLECTIONNAME.toString());
+
Related-by-collID=c4e9d8f0-fd00-11dd-a0ab-fac8f9dc8a80
ss[0].setValue(id);
+
Related-by-role=is-annotated-by
+
IsValidated=false
+
</source>
// metadatalang, metadataname, metadataformat are all used for MetadataFormat
+
// they are mandatory
+
ss[1].setName(CreationParameters.METADATALANG.toString());
+
ss[1].setValue("en");
+
+
ss[2].setName(CreationParameters.METADATANAME.toString());
+
ss[2].setValue("dc");
+
+
ss[3].setName(CreationParameters.METADATAURI.toString());
+
ss[3].setValue("http://mail.google.com/mail");
+
+
ss[4].setName(CreationParameters.ROLE.toString());
+
ss[4].setValue(SecondaryRoleDescription.ROLE_GENERATION);
+
+
String factoryURI = //"http://dlib31.isti.cnr.it:8080/wsrf/services/gcube/metadatamanagement/metadatamanager/MetadataManagerFactory";
+
"http://node2.d.d4science.research-infrastructures.eu:8080/wsrf/services/gcube/metadatamanagement/metadatamanager/MetadataManagerFactory";
+
+
String id = "CMProve";   
+
if (args.length ==1)
+
        id = args[0];
+
+
MetadataManagerFactoryServiceAddressingLocator factoryLocator = new MetadataManagerFactoryServiceAddressingLocator();
+
//MetadataManagerServiceAddressingLocator  instanceLocator = new MetadataManagerServiceAddressingLocator();
+
MetadataManagerFactoryPortType mFactory;
+
  
try {
+
'''createManagerFromCollection'''
EndpointReferenceType factoryEPR = new EndpointReferenceType();
+
factoryEPR.setAddress(new Address(factoryURI));
+
+
+
System.out.println("Creating the new Factory portType");
+
  
mFactory= factoryLocator.getMetadataManagerFactoryPortTypePort(factoryEPR);
+
----
 +
The createCollection script invokes the [[Metadata_Manager#Creating_a_new_Manager_from_an_existing_Metadata_Collection | CreateManagerFromCollection]] operation to create a new Manager for an existing Metadata Collection. It must be executed as follows:
 +
<pre>
 +
./createManagerFromCollection.sh <MM host> <MM port> <scope> <metadata collection ID>
 +
</pre>
  
System.out.println("Created instance.\n");
+
'''getMOIDs'''
+
 
CreateManagerRequest request = new CreateManagerRequest();
+
----
request.setCollectionID(id);
+
The getMOIDs script invokes the [[Metadata_Manager#Getting_all_the_MOIDs_of_a_Metadata_Collection | GetMOIDs]] operation to retrieve all the MOIDs belonging to an existing Metadata Collection. It must be executed as follows:  
CollectionParameter [] ss = new CollectionParameter[8];
+
<pre>
CreateCollectionParameters ccp = new CreateCollectionParameters();
+
./getMOIDs.sh <MM host> <MM port> <scope> <metadata collection ID>
+
</pre>
for ( int i =0; i<ss.length; i++ )
+
 
ss[i] = new CollectionParameter();
+
'''getTargets'''
+
 
ss[0].setName(CreationParameters.COLLECTIONNAME.toString());
+
----
ss[0].setValue(id);
+
The getTargets script invokes the [[Metadata_Manager#Getting_all_the_Target_Objects_of_a_Metadata_Collection | GetTargets]] operation to retrieve per each MO belonging the given Metadata Collection, the ID of the Content Object it describes. It must be executed as follows:  
System.out.println("COLLECTIONNAME:"+id);
+
<pre>
/*ss[1].setName(CreationParameters.DESCRIPTION.toString());
+
./getTargets.sh <MM host> <MM port> <scope> <metadata collection ID>
ss[1].setValue("Description Metadata Collection");
+
String editable = "True";
+
ss[2].setName(CreationParameters.EDITABLE.toString());
+
ss[2].setValue(editable);*/
+
+
/*System.out.println(CreationParameters.DESCRIPTION.toString()+": Description Metadata Collection");
+
System.out.println(CreationParameters.EDITABLE.toString()+ ": "+editable);
+
+
// Indexable mandatory
+
String indexable = "True";
+
+
ss[3].setName(CreationParameters.INDEXABLE.toString());
+
ss[3].setValue(indexable);
+
+
System.out.println(CreationParameters.INDEXABLE.toString() +": "+indexable);
+
+
String user = "True";
+
ss[4].setName(CreationParameters.USER.toString());
+
ss[4].setValue(user);
+
System.out.println(CreationParameters.USER.toString()+": " +user);*/
+
+
// metadatalang, metadataname, metadataformat are all used for MetadataFormat
+
// they are mandatory
+
ss[1].setName(CreationParameters.METADATALANG.toString());
+
ss[1].setValue("en");
+
+
ss[2].setName(CreationParameters.METADATANAME.toString());
+
ss[2].setValue("dc");
+
+
ss[3].setName(CreationParameters.METADATAURI.toString());
+
ss[3].setValue("http://mail.google.com/mail");
+
+
ss[4].setName(CreationParameters.ROLE.toString());
+
ss[4].setValue(SecondaryRoleDescription.ROLE_GENERATION);
+
+
ss[5].setName(CreationParameters.INDEXABLE.toString());
+
ss[5].setValue("true");
+
//String relatedbyCollID = "3971d1a0-4771-11dd-8639-ae5f2f1f5aef";
+
+
+
String generatedbyCollID = "1222d1a0-4771-11dd-8639-ae5f2f1f5ae";
+
ss[6].setName(CreationParameters.GENERATEDBY_COLLID.toString());
+
ss[6].setValue(generatedbyCollID);
+
System.out.println(CreationParameters.GENERATEDBY_COLLID.toString() + ": GeneratedID");
+
ss[7].setName(CreationParameters.GENERATEDBY_URI.toString());
+
ss[7].setValue("http://mail.google.com/mail");
+
+
ccp.setParams(ss);
+
request.setCollectionParameters(ccp);
+
CreateManagerResponse response  = new CreateManagerResponse();
+
try{
+
mFactory = GCUBERemotePortTypeContext.getProxy(mFactory,GCUBEScope.getScope("/gcube/devsec"));              
+
response = mFactory.createManager(request);
+
}catch(Exception e ){
+
e.printStackTrace();
+
return;
+
}
+
+
System.out.println("Created MC:"+response.getEndpointReference().toString());
+
} catch (Exception e) {
+
e.getMessage();
+
e.printStackTrace();
+
}
+
}
+
....
+
 
</pre>
 
</pre>
  
  
 
[[Category:Metadata Management]]
 
[[Category:Metadata Management]]

Latest revision as of 15:51, 5 July 2010

Introduction

The Metadata Manager manages the modelling of arbitrary metadata relationships (IDB-relationships) in accordance with the gCube Metadata Model. The only assumption it does is that the Metadata Objects are serialized as well-formed XML documents. The service has a two-fold role:

  • to manage Metadata Objects and Metadata Collections
  • to establish secondary role-typed links. Such relationships can be in place between any type of Information Object and in the scope of a Collection or not

The service relies on the Storage Management Service to persist the information handled.

Clients interact also with the Metadata Manager library for a higher level of abstraction which simplifies clients interaction with the service.

Metadata Manager Service

The main functionality of the Metadata Manager Service is the management of Metadata Objects, Metadata Collection and their relationships.

From an architectural point of view, the service adopts the Factory pattern and Collection Managers are implemented as stateful WS-Resources. Therefore, the service is composed by:

  • MetadataManagerFactory, a portType that creates new Collection Managers and offers some cross-Collection operations
  • MetadataManager, a portType that operates over Metadata Collections (MCs) and on Metadata Objects as Elements, i.e. members of a specific Metadata Collection

To operate over Metadata Collections, the service instantiates a Collection Manager (or shortly, Manager) for each collection. A Collection Manager is the access point to all the possible operations over a specific Metadata Collection.

MetadataManagerFactory portType

The MetadataManagerFactory creates new Collection Managers and offers some cross-Collection operations. Moreover, it operates on Metadata Objects as Information Objects related to other Information Objects and not as Members of Metadata Collections.

Creating a new Metadata Collection

EndPointReferenceType createManager(CollectionID, params): This operation takes as input a Content Collection ID and a set of creation parameters. Then, it creates a new Manager in order to manage a Metadata Collection bound to the given Content Collection. If a Metadata Collection with the specified Metadata characteristics does not exist, the Manager creates the Metadata Collection, binds it with the Content Collection and publishes its profile in the IS.

The Creation parameters are a set of key-value pairs; the following keys are defined in the MMLibrary (org.gcube.metadatamanagement.mmlibrary.descriptions.CollectionDescription.CreationParameters) and accepted as parameters:
Mandatory parameters:
  • COLLECTIONNAME -> name of the collection
  • METADATANAME -> metadata name (e.g. “dc”)
  • METADATALANG -> metadata language (e.g. “English”), as specified by the ISO 639-2
  • METADATAURI -> metadata URI: the XML Schema that defines the MO payloads
  • ROLE -> secondary role
  • RELATEDBY_COLLID -> the ID of the Content Collection (it must report the same value of the first parameter of the invocation, i.e. CollectionID)
Optional parameters:
  • DESCRIPTION -> description
  • INDEXABLE -> if the collection is indexable or not (“True”/”False”), default is true
  • USER -> if the collection is a user collection or not (“True”/”False”), default is true
  • RELATEDBY_ROLE -> the source XML Schema from which the current one has been generated, if any
  • GENERATEDBY_COLLID -> the source Metadata Collection from which the current one has been generated (by the Metadata Broker), if any
  • GENERATEDBY_URI -> the URI of the source schema of the collection from which it has been generated, if any
Among the others, two parameters can modify the way in which the Metadata Collection is managed:
  • Indexable: if the new Metadata Collection is indexable, the Manager creates also a new XML Index for such a Collection using the XMLIndexer Service;
  • User: a user Collection is shown in the Portals and an end-user can operate on it; a non-user Collection is intended for internal purposes (like to collect parts of persisted Indexes).

This example shows how to invoke the createManager operation (it reads the input from a properties file):

import java.io.IOException;
import java.util.Properties;
 
import org.gcube.common.core.contexts.GCUBERemotePortTypeContext;
import org.gcube.common.core.scope.GCUBEScope;
import org.gcube.common.core.utils.logging.GCUBEClientLog;
import org.gcube.metadatamanagement.metadatamanager.stubs.CollectionParameter;
import org.gcube.metadatamanagement.metadatamanager.stubs.CreateCollectionParameters;
import org.gcube.metadatamanagement.metadatamanager.stubs.CreateManagerRequest;
import org.gcube.metadatamanagement.metadatamanager.stubs.CreateManagerResponse;
import org.gcube.metadatamanagement.metadatamanager.stubs.MetadataManagerFactoryPortType;
import org.gcube.metadatamanagement.metadatamanager.stubs.service.MetadataManagerFactoryServiceAddressingLocator;
import org.gcube.metadatamanagement.mmlibrary.descriptions.CollectionDescription.CreationParameters;
import org.apache.axis.message.addressing.Address;
import org.apache.axis.message.addressing.EndpointReferenceType;
 
/**
 * Tester for <em>createManager</em> operation of the Metadata Manager service
 * 
 * @author Manuele Simi (ISTI-CNR)
 *
 */
public final class CreateManagerTest {
 
	private static Properties resources = new Properties();
 
	private static GCUBEClientLog logger = new GCUBEClientLog(CreateManagerTest.class); 
 
	/**
	 * @param args 
	 * <OL>
	 * <LI> MM host
	 * <LI> MM port
	 * <LI> Scope
	 * <LI> Properties file
	 * </OL>
	 * @throws Exception 
	 */
	public static void main(String[] args) throws Exception {		
 
		if (args.length != 4) {
			logger.error("Usage: CreateManagerTest <MM host> <MM port> <Scope> <properties file>");
			return;
		}		
		logger.debug("Loading props file");
		try {
			resources.load(CreateManagerTest.class.getResourceAsStream("/" + args[3]));
		} catch (IOException e1) {
			logger.error("Unable to load the properties file", e1);
			Runtime.getRuntime().exit(1);
 
		}
		String factoryURI = "http://" + args[0]+ ":" + args[1] + "/wsrf/services/gcube/metadatamanagement/metadatamanager/MetadataManagerFactory";
		String scope = args[2];						
 
		MetadataManagerFactoryServiceAddressingLocator factoryLocator = new MetadataManagerFactoryServiceAddressingLocator();
		MetadataManagerFactoryPortType mFactory;
		try {			
			EndpointReferenceType factoryEPR = new EndpointReferenceType();
			factoryEPR.setAddress(new Address(factoryURI));			
			logger.info("Creating the new Factory portType..");
 
			mFactory= factoryLocator.getMetadataManagerFactoryPortTypePort(factoryEPR);			
 
			CreateManagerRequest request = new CreateManagerRequest();
			request.setCollectionID(resources.getProperty("RelatedContentCollectionID"));			
 
			// initialise all the params
			CollectionParameter [] parameters = new CollectionParameter[11];
			for ( int i =0; i<parameters.length; i++ )
				parameters[i] = new CollectionParameter();
 
			parameters[0].setName(CreationParameters.COLLECTIONNAME.toString());
			parameters[0].setValue(resources.getProperty("Name"));
			if (parameters[0].getValue() == null)
				parameters[0].setValue("test-collection");
 
			parameters[1].setName(CreationParameters.METADATALANG.toString());
			parameters[1].setValue(resources.getProperty("MetadataLang"));
 
			parameters[2].setName(CreationParameters.METADATANAME.toString());
			parameters[2].setValue(resources.getProperty("MetadataName"));
 
			parameters[3].setName(CreationParameters.METADATAURI.toString());
			parameters[3].setValue(resources.getProperty("MetadataURI"));
 
			parameters[4].setName(CreationParameters.ROLE.toString());
			parameters[4].setValue(resources.getProperty("Role"));			
 
			parameters[5].setName(CreationParameters.INDEXABLE.toString());
			parameters[5].setValue(resources.getProperty("IsIndexable"));
 
			parameters[6].setName(CreationParameters.EDITABLE.toString());
			parameters[6].setValue(resources.getProperty("IsEditable"));
 
			parameters[7].setName(CreationParameters.RELATEDBY_COLLID.toString());
			parameters[7].setValue(resources.getProperty("Related-by-collID"));
 
			parameters[8].setName(CreationParameters.RELATEDBY_ROLE.toString());
			parameters[8].setValue(resources.getProperty("Related-by-role"));
 
			parameters[9].setName(CreationParameters.DESCRIPTION.toString());
			parameters[9].setValue(resources.getProperty("Description"));
 
			parameters[10].setName(CreationParameters.VALIDATING.toString());
			parameters[10].setValue(resources.getProperty("IsValidated"));
 
			CreateCollectionParameters ccp = new CreateCollectionParameters();
			ccp.setParams(parameters);
			request.setCollectionParameters(ccp);
			CreateManagerResponse response = new CreateManagerResponse();
			try{
				mFactory = GCUBERemotePortTypeContext.getProxy(mFactory,GCUBEScope.getScope(scope), 120000);	 	               
				response = mFactory.createManager(request);
			}catch(Exception e){
				logger.error("", e);
				return;
			}			
			logger.info("Metadata Collection successfully created, at "+response.getEndpointReference().toString());
		} catch (Exception e) {
			logger.error("", e);	
		}
	}
}
Creating a new Manager from an existing Metadata Collection

EndPointReferenceType createManagerFromCollection (MetadataCollectionID): This operation takes a Metadata Collection ID as input and returns:

  • the related CollectionManager, if it already exists
  • an EPR of a newly created CollectionManager, if the Metadata Collection exists
  • an error, if the Collection ID is not valid

The following example shows how to invoke the operation:

import org.apache.axis.message.addressing.Address;
import org.apache.axis.message.addressing.EndpointReferenceType;
import org.gcube.common.core.contexts.GCUBERemotePortTypeContext;
import org.gcube.common.core.scope.GCUBEScope;
import org.gcube.common.core.utils.logging.GCUBEClientLog;
import org.gcube.metadatamanagement.metadatamanager.stubs.CreateManagerResponse;
import org.gcube.metadatamanagement.metadatamanager.stubs.MetadataManagerFactoryPortType;
import org.gcube.metadatamanagement.metadatamanager.stubs.MetadataManagerPortType;
import org.gcube.metadatamanagement.metadatamanager.stubs.service.MetadataManagerFactoryServiceAddressingLocator;
import org.gcube.metadatamanagement.metadatamanager.stubs.service.MetadataManagerServiceAddressingLocator;
 
/**
 * Tester for <em>CreateManagerFromCollection</em> operation of the Metadata Manager service
 *
 *  @author Manuele Simi (ISTI-CNR)
 */
public class CreateManagerFromCollection {
 
	private static GCUBEClientLog logger = new GCUBEClientLog(CreateManagerFromCollection.class);
 
	/**
	 * @param args 
	 * <OL>
	 * <LI> MM host
	 * <LI> MM port
	 * <LI> Scope
	 * <LI> Metadata Collection ID
	 * </OL>
	 */
	public static void main(String[] args) {
 
		if (args.length != 4) {
			System.err.println("Usage: CreateManagerFromCollection <MM host> <MM port> <Scope> <collection ID>");
			return;
		}
		String factoryURI = "http://" + args[0]+ ":" + args[1] + "/wsrf/services/gcube/metadatamanagement/metadatamanager/MetadataManagerFactory";
		String scope = args[2];						
		String collectionID = args[3];       	
 
		MetadataManagerPortType manager = CreateManagerFromCollection.createManager(factoryURI, scope, collectionID);
		if (manager != null)
			logger.info("Manager successfully created");
	}
 
	public static MetadataManagerPortType createManager(String factoryURI, String scope, String collectionID) {
 
		MetadataManagerPortType manager = null;
 
		try {
			EndpointReferenceType factoryEPR = new EndpointReferenceType();		
			factoryEPR.setAddress(new Address(factoryURI));
			logger.info("Creating the new Factory portType");
			MetadataManagerFactoryServiceAddressingLocator factoryLocator = new MetadataManagerFactoryServiceAddressingLocator();
			MetadataManagerFactoryPortType mFactory= factoryLocator.getMetadataManagerFactoryPortTypePort(factoryEPR);						
			mFactory = GCUBERemotePortTypeContext.getProxy(mFactory,GCUBEScope.getScope(scope));	 	               
			CreateManagerResponse response = mFactory.createManagerFromCollection(collectionID);										
			EndpointReferenceType instanceEPR = response.getEndpointReference();
			logger.info("Creating manager at " + instanceEPR.toString());
			MetadataManagerServiceAddressingLocator  instanceLocator = new MetadataManagerServiceAddressingLocator();				
			manager = instanceLocator.getMetadataManagerPortTypePort(instanceEPR);
 
		} catch (Exception e) {
			logger.error("Unable to create the Manager", e);
			Runtime.getRuntime().exit(1);
		}
		return manager;
	}
 
}
  • ArrayOfMetadata[] getMetadata(MetadataSelector[])
...
  • MOID addMetadata(ObjectID, MO, SecondaryRole): This operation takes a new non-collectable Metadata Object and
    • completes the metadata header information (e.g. the MOID, if it is not specified)
    • stores (or updates if the MOID is already included in the MO header) the object on the Storage Management Service as Information Object
    • creates a <is-described-by, <SecondaryRole>> binding in the Storage Management Service between the Metadata Object and the Information Object identified by the given Object ID
    • returns the assigned MOID
  • void deleteMetadata(MOID): This operation deletes from the Storage Management Service the Metadata Object identified by the given ID.
  • (ObjectID, (MO, SecondaryRole)[])[] getMetadata ((ObjectID, SecondaryRole, CollectionID, Rank)[]): For each given ObjectID, this operation returns the Metadata Objets. They are:
    • bound with the specified secondary role (the primary role is, of course, is-described-by) to the Information Object identified by that ObjectID
    • members of the specified Metadata Collection

MetadataManager portType

The MetadataManager allows user to manipulate a Metadata Collection. It provides the following operations:

  • AddElementsResponse addElements(addElements): Adds the given elements to the MetadataCollection
    • addElements is a list, where each item of the list is a valid ID of an IO available in the Storage and a list of new MOs to bind to such IO
    • AddElementsResponse is a list, which is a map with the ID of the IO and the list of MOs bound to it in the operation
  • String getMetadataCollectionID(): Gets the Metadata Collection identifier
  • void deleteCollection(): Deletes the Metadata Collection and destroys the related Collection Manager.
    • The elements are also removed from the storage service, if they do not belong to other Metadata Collections
    • If the MetadataCollection is indexable, it is also destroyed from the XMLIndexer service
  • ListOfUpdateElementsResults updateElements(UpdateElements): Updates the given MOs either in the Storage Management and in the XMLIndexer (if the Collection is Indexable)
    • UpdateElements is a list of XML String representation of the MetadataObjects(MOs) to update
    • ListOfUpdateElementsResults is an array of the same cardinality of the input string. Each item is a boolean value reporting true if the MO in the same position in the input array was successfully updates, false otherwise.
  • GetElementResponse getElements(InformationObjectIDList): Gets the MOs bound to the by the given MOIDs
Getting all the MOIDs of a Metadata Collection

MOIDList getMOIDs(): Gets all the Metadata Object IDs of the Metadata Collection.

A sample usage of the operation is reported below:

import org.gcube.common.core.contexts.GCUBERemotePortTypeContext;
import org.gcube.common.core.scope.GCUBEScope;
import org.gcube.common.core.utils.logging.GCUBEClientLog;
import org.gcube.metadatamanagement.metadatamanager.stubs.MOIDList;
import org.gcube.metadatamanagement.metadatamanager.stubs.MetadataManagerPortType;
import org.gcube.metadatamanagement.metadatamanager.stubs.VoidType;
 
/**
 * Tester for <em>GetMOIDs</em> operation of the Metadata Manager service
 *
 *  @author Manuele Simi (ISTI-CNR)
 */
public class GetMOIDsTest {
 
	private static GCUBEClientLog logger = new GCUBEClientLog(GetMOIDsTest.class);
 
	/**
	 * @param args 
	 * <OL>
	 * <LI> MM host
	 * <LI> MM port
	 * <LI> Scope
	 * <LI> Metadata Collection ID
	 * </OL>
	 */
	public static void main(String[] args) {
		String factoryURI = "http://" + args[0]+ ":" + args[1] + "/wsrf/services/gcube/metadatamanagement/metadatamanager/MetadataManagerFactory";
		String scope = args[2];						
		String collectionID = args[3];       	
 
		MetadataManagerPortType manager = CreateManagerFromCollection.createManager(factoryURI, scope, collectionID);		
		try {
			manager = GCUBERemotePortTypeContext.getProxy(manager,GCUBEScope.getScope(scope));			
			MOIDList moids = manager.getMOIDs(new VoidType());
			if (moids == null) {
				logger.warn("The returned list of MOIDs is null");
				return;
			}			
			if (moids.getOID() == null) {
				logger.warn("The returned list of MOIDs is empty");
				return;
			}			
			logger.info("The returned list of MOIDS includes " + moids.getOID().length + " element(s)");
 
			for (String id : moids.getOID()) 
				logger.info("MOID="+ id);
 
		} catch (Exception e) {
			logger.error("Something went wrong", e);
			Runtime.getRuntime().exit(1);
		} 
 
	}
 
}
  • GetElementsFromMOIDResponse getElementsFromMOID(MOIDList): Gets the MOs identified by the given MOIDs, where:
    • MOIDList is a list of Metadata Object IDs
    • GetElementsFromMOIDResponse contains the MetadataObjects related to the given IDs
  • MOIDList deleteElements(MOIDList): Deletes from storage service the Metadata Object identified by the given ID
    • If the MetadataCollection is indexable it deletes the Metadata Object from the related MetadataXMLIndexer using the XMLindexer service
    • MOIDList is a list with the IDs of MetadataObjects(MOs), in input the IDs to delete, in output it returns the ID of the elements not deleted
  • GetAllElementsResponse getAllElements(void): returns all the MOs of the current Metadata Collection
  • String addElementsRS(String): Adds the given elements to the MetadataCollection using the ResultSetService
    • the input string is the reference to a RSLocator with the MOs to store, each element of the RS must have the entire MO in its payload
    • the output string is the reference to an RSLocator with a list of the stored MOs' IDs
  • String getAllElementsRS(): Returns all the MOs of the current Metadata Collection using the ResultSetService
    • the output String is a reference to an RSLocator with the retrieved elements
  • String getAllElementsRSWithTimestamp(TimestampInterval): Returns all the MOs of the current Metadata Collection filtered by Date using the ResultSetService
    • the output String is a reference to an RSLocator with the retrieved elements

An example of the operation is reported below:

...
Calendar fromDate= Calendar.getInstance();
fromDate.set(2001,10,1);
Calendar toDate= Calendar.getInstance();
toDate.set(2010,1,1);
String locator = manager.getAllElementsRSWithTimestamp(new TimestampInterval(fromDate.getTimeInMillis(),toDate.getTimeInMillis()));
...

  • String getElementsRS(String):Gets the list of MOs describing the Information Object included in the input list using the ResultSetService. The returned MOs belong the current Metadata Collection
    • the input string is the reference to an RSLocator with the OIDs
    • the output string is the reference to an RSLocator with the retrieved MOs
Getting all the Target Objects of a Metadata Collection

<IO,MOID>[]getTargets(MOID[]): This operation allows to retrieve per each input MO the described Content Object. A sample usage of the operation is reported below:

import org.gcube.common.core.contexts.GCUBERemotePortTypeContext;
import org.gcube.common.core.scope.GCUBEScope;
import org.gcube.common.core.utils.logging.GCUBEClientLog;
import org.gcube.metadatamanagement.metadatamanager.stubs.GetTargetsObjectResponse;
import org.gcube.metadatamanagement.metadatamanager.stubs.MOIDList;
import org.gcube.metadatamanagement.metadatamanager.stubs.MetadataManagerPortType;
import org.gcube.metadatamanagement.metadatamanager.stubs.VoidType;
 
/**
 * Tester for <em>GetTargets</em> operation of the Metadata Manager service
 *
 *  @author Manuele Simi (ISTI-CNR)
 */
public final class GetTargets {
 
	private static GCUBEClientLog logger = new GCUBEClientLog(GetTargets.class);
 
	/**
	 * @param args 
	 * <OL>
	 * <LI> MM host
	 * <LI> MM port
	 * <LI> Scope
	 * <LI> Metadata Collection ID
	 * </OL>
	 */
	public static void main(String[] args)  {
		String factoryURI = "http://" + args[0]+ ":" + args[1] + "/wsrf/services/gcube/metadatamanagement/metadatamanager/MetadataManagerFactory";
		String scope = args[2];						
		String collectionID = args[3];       	
 
		MetadataManagerPortType manager = CreateManagerFromCollection.createManager(factoryURI, scope, collectionID);		
		try {
			manager = GCUBERemotePortTypeContext.getProxy(manager,GCUBEScope.getScope(scope));
 
			MOIDList moids = manager.getMOIDs(new VoidType());
			if (moids == null) {
				logger.warn("The returned list of MOIDs is null");
				return;
			}
 
			if (moids.getOID() == null) {
				logger.warn("The returned list of MOIDs is empty");
				return;
			}
 
			logger.info("The returned list of MOIDS includes " + moids.getOID().length + " element(s)");
 
			GetTargetsObjectResponse[] mccontents = manager.getTargets(moids).getGetTargetObjectResponse();
			if ( (mccontents == null) || (mccontents.length == 0)) {
				logger.info("The target Metadata Collection is empty");
				return;
			}
 
			for (GetTargetsObjectResponse pair : mccontents) 
				logger.info("IO=" + pair.getInformationObjectID()+ ", MOID="+ pair.getMetadataObjectID());
 
		} catch (Exception e) {
			logger.error("Something went wrong", e);
			Runtime.getRuntime().exit(1);
		} 
	}
}

Dependencies

The MetadataManager Service depends by the following gCube components:

  • MMLibrary
  • XMLIndexer
  • ResultSetService
  • ResultSetLibrary
  • ResultSetClientLibrary
  • ContentManagement
  • ContentManagementService
  • ContentManagementLibrary

MMLibrary

The MMLibrary offers a number of facilities for creating and manipulating Metadata Objects. The library operates at client-side by moving some of the logic on that side avoiding unnecessary interactions between clients and the Metadata Manager service.

Basically, it offers a a set of method to easily wrap an XML documents in the model exchanged format as required by Metadata Manager service. i.e. support for wrapping the Objects with the appropriate metadata envelope, in order to create a valid Metadata Object that can be stored on the Metadata Manager Service.

In addition, the library provides some utility classes to extract the MO information from a string representing a serialized Metadata Object.

The test-suite

The Metadata Manager comes with a test-suite package allowing to test its functionalities. The test-suite is completely independent and does not require any other gCube package, except than a local gCore installation. The package is composed by a set of classes, sample configuration files and scripts ready to be executed.


|-lib
|--org.gcube.metadatamanagement.metadatamanager.testsuite.jar
|
|-samples
|---...
|
|-createCollection.sh
|-createManagerFromCollection.sh
|-getTargets.sh
|-getMOIDs.sh

Each script allows to test a different service's operation or group of operations logically related. In the following, an explanation of each script and its usage is provided.

createCollection


The createCollection script invokes the CreateCollection operation to create a new Metadata Collection. It must be executed as follows:

./createCollection.sh <MM host> <MM port> <scope> <configuration file>

This is an example of configuration file that can be provided:

Name=test-coll
Description=collection from MM test-suite
Role=is-described-by
MetadataName=dc
MetadataURI=http://mail.google.com/mail
MetadataLang=it
IsIndexable=true
IsEditable=false
Related-by-collID=c4e9d8f0-fd00-11dd-a0ab-fac8f9dc8a80
Related-by-role=is-annotated-by
IsValidated=false

createManagerFromCollection


The createCollection script invokes the CreateManagerFromCollection operation to create a new Manager for an existing Metadata Collection. It must be executed as follows:

./createManagerFromCollection.sh <MM host> <MM port> <scope> <metadata collection ID>

getMOIDs


The getMOIDs script invokes the GetMOIDs operation to retrieve all the MOIDs belonging to an existing Metadata Collection. It must be executed as follows:

./getMOIDs.sh <MM host> <MM port> <scope> <metadata collection ID>

getTargets


The getTargets script invokes the GetTargets operation to retrieve per each MO belonging the given Metadata Collection, the ID of the Content Object it describes. It must be executed as follows:

./getTargets.sh <MM host> <MM port> <scope> <metadata collection ID>