Revision as of 16:01, 18 October 2010

Design Overview

The gCube Document Library (gDL) is a library that works on top of the Content Manager. This library offers a simpler way for client to access documents in the d4science system. The major change in the ContentManager is the document model (more generic, more extensible), the gDL reduces the development costs for using this new feature.

Document Model

Implementation Overview

The gDL is composed by tree types of access interfaces:

the CollectionManager is used to manage the Collections in the system.
the CMReader is used to read document from a collection. it offers methods to get a document or part of it (Metadata, Annotations, Parts, Alternatives);
the CMWriter is used to add document or update it or a part of it;
the CMView is used to get documents from a specific view of a collection.

Collection Manager

The CollectionManager class offers a way to manager a collection it is created passing to the constructor the collection id, the Scope and a SecurityManager:

public CollectionManager(String collectionId, GCUBEScope scope, GCUBESecurityManager ... securityManagers) throws Exception

it exposes a lot of operations for retrieving of Readers, Writers and Views related to that collection

public CMReader getReader() throws Exception

returns the CMReader related to the collection

public CMWriter getWriter() throws Exception

returns the CMWriter related to the collection

public <E extends GCubeElement> void createView(CMView<E> v) throws GCUBEException, Exception

stores a specific view (MetadataView or AnnotationView) in the system

public List<CMView<?>> findViews() throws Exception

retrieves all the views for the current collection

public <E extends GCubeElement,V extends CMView<E>> List<V> findViews(V v) throws Exception

retrieves all the views that reflect the properties defined in v

CMReader

CMWriter

CMView

Configuring the environment

Supporting the SMS protocol

The Storage Management Service introduces a new protocol, called sms, with an handler implementation.

To let know the JVM about the existence of a new protocol handler you can use one of this solutions:

Specify a new java property at JVM startup: -Djava.protocol.handler.pkgs=org.gcube.contentmanagement.storagelayer.storagemanagementservice.stubs.protocol
- if you are developing a service and are you using the last distribution you don't have to change anything because the new GHN distribution already do it.
- if you are developing a portlet or a portal library check if your portal instance have the option specified in the environment variable CATALINA_OPTS.
- if you are running your client locally on Eclipse you can specify the options on run configuration, Arguments tab, VM arguments section.
Use this sentence in your client testing program:

System.setProperty("java.protocol.handler.pkgs", "org.gcube.contentmanagement.storagelayer.storagemanagementservice.stubs.protocol");

Examples

Here some examples on gDL use. More examples are on examples folder in gDL project on SVN.

Managing Collections examples

We shows some examples on managing Collections.

List all collections

This example shows how to list all collection on a scope.

 GCUBEScope scope = GCUBEScope.getScope("/gcube");
 
 System.out.println("Collections in "+scope+" :");
 for (CMCollection collection:Collections.list(scope)){
   System.out.println(collection);
 }

Find collection by name

This example shows how to find a collection using his name.

 GCUBEScope scope = GCUBEScope.getScope("/gcube");
 
 String collectionName = "My Test Collection";
 
 System.out.println("Collections in "+scope+" with name \""+collectionName+"\":");
 for (CMCollection collection:Collections.findByName(scope, collectionName)){
   System.out.println(collection);
 }

Create a collection using SMS-Plugin

This code create a new collection with the given name and description.

The create method returns a list of collection references because a create command can produce more than one collection (for example OAI-Plugin).

To run this example the SMS-Plugin jar is required.

 GCUBEScope scope = GCUBEScope.getScope("/gcube");
 
 //collection informations
 String collectionName = "MyTestCollection";
 String collectionDescription = "This is my test collection";
 
 //we don't want to propagate the request to others CM
 boolean propagateRequest = false;
 
 //we want the collection to be readable and writable
 boolean readable = true;
 boolean writable = true;
 
 //finally we create the collection
 List<CollectionReference> collectionReferences =SMSUtil.createCollection(scope, propagateRequest, collectionName, collectionDescription, readable, writable);
 
 System.out.println("Created collections:");
 for (CollectionReference collectionReference:collectionReferences){
   System.out.println("Collection id: "+collectionReference.getCollectionID());
   if (collectionReference.getReader()!=null) System.out.println("Reader EPR: "+collectionReference.getReader());
   if (collectionReference.getWriter()!=null) System.out.println("Writer EPR: "+collectionReference.getWriter());
   System.out.println();
 }

Wrap a collection using SMS-Plugin

This example shows how to wrap an existing collection on SMS. To run this example the SMS-Plugin jar is required.

 GCUBEScope scope = GCUBEScope.getScope("/gcube");
 
 //the collection id on SMS (the id of collection IO on the SMS)
 String collectionId = "74fe4120-8f5b-11df-8083-c574cd534edf";
 
 //we don't want to propagate the request to others CM
 boolean propagateRequest = false;
 
 //we want the collection to be readable and writable
 boolean readable = true;
 boolean writable = true;
 
 //finally we wrap the collection
 List<CollectionReference> collectionReferences = SMSUtil.wrapCollection(scope, propagateRequest, collectionId, readable, writable);
 
 System.out.println("Collections wrapped:");
 for (CollectionReference collectionReference:collectionReferences){
   System.out.println("Collection id: "+collectionReference.getCollectionID());
   if (collectionReference.getReader()!=null) System.out.println("Reader EPR: "+collectionReference.getReader());
   if (collectionReference.getWriter()!=null) System.out.println("Writer EPR: "+collectionReference.getWriter());
   System.out.println();
 }

Reading examples

We shows some examples on reading documents.

Get a single document

This example shows how to get a single document using his id.

 //we instantiate the CMReader
 CMReader cmReader = new CMReader(collectionId, scope);
 
 //then we retrieve the document, we are not specifying projections, therefore the entire document is returned.
 GCubeDocument document = cmReader.getDocument(documentId);
 
 System.out.println("Retrieved document:");
 System.out.println("id: " + document.getId());
 System.out.println("name: " + document.getName());
 System.out.println("mimeType: " + document.getMimeType());

Get documents by name

This example shows how to get documents from a collection using a name constraint.

 String collectionId = "74fe4120-8f5b-11df-8083-c574cd534edf";
 System.out.println("collectionId: "+collectionId);
 System.out.println();
 
 String documentName = "Column dedication by Eumachos and Amias";
 
 //we instantiate the CMReader
 CMReader cmReader = new CMReader(collectionId, scope);
 
 //we define a gcube main document projection
 GCubeMainDocumentProjection projection = ProjectionsFactory.MAIN_DOCUMENT();
 //we set a constraint over the name field
 projection.constrainName(is(documentName));
 
 //we are asking for all documents having the specified name
 Iterator<GCubeDocument> documentIterator = cmReader.getDocuments(projection);
 
 int i = 0;
 System.out.println("Retrieved documents:");
 
 //finally we iterate over the collection
 while(documentIterator.hasNext()){
   GCubeDocument document = documentIterator.next();
   System.out.println(i+") "+document);
   i++;
 }
 
 System.out.println();
 System.out.println("Found "+i+" documents");

To use the is function remember to add this import:

 import static org.gcube.contentmanagement.contentmanager.stubs.model.constraints.Constraints.*;

Get all documents in a collection

This example shows how to get all documents from a collection.

 //we instantiate the CMReader
 CMReader cmReader = new CMReader(collectionId, scope);
 
 //then we retrieve all collection documents
 //we are not specifying projections, therefore the entire document is returned.
 Iterator<GCubeDocument> documentIterator = cmReader.getDocuments();
 
 int i = 0;
 System.out.println("Collection documents:");
 
 //finally we iterate over the collection
 while(documentIterator.hasNext()){
    GCubeDocument document = documentIterator.next();
 
    String id = document.getId();
    int parts = document.getParts().size();
    int alternatives = document.getAlternatives().size();
    int metadata = document.getMetadata().size();
    int annotations = document.getAnnotations().size();
 
    System.out.printf("%d id: %s parts: %d alternatives: %d metadata: %d annotations: %d %n", i, id, parts, alternatives, metadata, annotations);
    i++;
 }

Get a document using projections

This example shows how to use projections during document retrieving.

 //we instantiate the CMReader
 CMReader cmReader = new CMReader(collectionId, scope);
 
 //then we retrieve the document, 
 //we are specifying the MAIN_DOCUMENT and the METADATA projections, the document returned will have main document data and all related metadata	
 GCubeDocument document = cmReader.getDocument(documentId, ProjectionsFactory.MAIN_DOCUMENT(), ProjectionsFactory.METADATA());
 
 System.out.println("Retrieved document:");
 System.out.println("id: " + document.getId());
 System.out.println("name: " + document.getName());
 System.out.println("mimeType: " + document.getMimeType());
 System.out.println("length: " + document.getLength());
 System.out.println("contentURL: " + document.getContentURL());
 System.out.println();
 
 for (GCubeMetadata metadata:document.getMetadata()){
    System.out.println("Metadata:");
    System.out.println(" id: " + metadata.getId());
    System.out.println(" mimeType: " + metadata.getMimeType());
    System.out.println(" length: " + metadata.getLength());
    System.out.println(" Metadata info:");
    System.out.println("  name: " + metadata.getMetadataName());
    System.out.println("  schema: " + metadata.getMetadataSchema());
    System.out.println("  language: " + metadata.getMetadataLanguage());
    System.out.println();
 }

Get a document content

This example shows how to get a document content.

 //we instantiate the CMReader
 CMReader cmReader = new CMReader(collectionId, scope);
 
 //then we retrieve the document, we are requesting only the main document
 GCubeDocument document = cmReader.getDocument(documentId, Projections.MAIN_DOCUMENT);
 
 System.out.println("Retrieved document:");
 System.out.println("id: " + document.getId());
 System.out.println("name: " + document.getName());
 System.out.println("contentURL: " + document.getContentURL());
 System.out.println();
 
 System.out.println("Retrieving content...");
 //the content is retrieved by the CML library through the content URL
 InputStream is = document.getContent();
 
 //finally we copy the content to a tmp file
 File tmpFile = File.createTempFile("mycontent", ".tmp");
 IOUtils.copy(is, new FileOutputStream(tmpFile));
 
 System.out.println("Content saved in tmp file: " + tmpFile.getAbsolutePath());

Get Collection dynamic properties

This example shows how to get dynamic properties for a given collection.

 String collectionId = "18611ed0-f162-11dd-96f7-b87cc0f0b075";
 System.out.println("collectionId: "+collectionId);
 
 CollectionManager collectionManager = new CollectionManager(collectionId, scope);
 System.out.println("Cardinality: "+collectionManager.getCardinality());
 SimpleDateFormat sdf = new SimpleDateFormat();
 System.out.println("Last Update: "+sdf.format(collectionManager.getLatUpdate().getTime()));
 System.out.println();

Writing examples

We shows some examples on manipulating collections.

Add a new document to a collection

Add a new document to a collection.

 //we instantiate the CMWriter
 CMWriter cmWriter = new CMWriter(collectionId, scope);
 
 //we create an instance of gcube document
 GCubeDocument myNewDocument = new GCubeDocument("My Test Document", Calendar.getInstance(), Calendar.getInstance(), "application/octet-stream", 100l);
 
 //then we ask for adding the new document
 String myNewDocumentId = cmWriter.add(myNewDocument);
 System.out.println("document created, id: " + myNewDocumentId);
 
 
 //finally we retrieve the created document
 CMReader cmReader = new CMReader(collectionId, scope);
 GCubeDocument document = cmReader.getDocument(myNewDocumentId);
 
 System.out.println("Created document:");
 System.out.println("id: " + document.getId());
 System.out.println("name: " + document.getName());
 System.out.println("mimeType: " + document.getMimeType());
 System.out.println("length: " + document.getLength());
 System.out.println("contentURL: " + document.getContentURL());
 
 System.out.println("parts: " + document.getParts().size());
 System.out.println("alternatives:" + document.getAlternatives().size());
 System.out.println("metadata: " + document.getMetadata().size());
 System.out.println("annotations: " + document.getAnnotations().size());

Remove a document from a collection

Remove a document from a collection.

 //we instantiate the CMWriter
 CMWriter cmWriter = new CMWriter(collectionId, scope);
 
 writer.remove(documentId);
 
 System.out.println("document removed.");

Views examples

We shows some examples on handling views.

Create a Metadata View

Shows how to create a metadata view.

 String collectionId = "1c583800-d5d8-11df-9cd6-b72b661eb83e";
 System.out.println("collectionId: "+collectionId);
 System.out.println();
 
 MetadataView myView = new MetadataView();
 myView.setName("MyView");
 myView.setDescription("This is my test view");
 myView.setEditable(false);
 myView.setIndexable(false);
 myView.setUserCollection(false);
 myView.setMetadaName("aquamaps");
 myView.setMetadataLanguage("en");
 myView.setMetadataSchema("http://aquamaps.xsd");
 
 CollectionManager manager = new CollectionManager(collectionId, scope);
 manager.createView(myView);
 
 System.out.println("View created: "+myView);
 
 MetadataViewReader reader = new MetadataViewReader(myView, scope);
 
 System.out.println("Metadata:");
 Iterator<GCubeMetadata> iterator = reader.getAll();
 while(iterator.hasNext()) System.out.println(iterator.next());

List collection views

List all the views for a certain collection.

 //we instantiate a collection manager.
 CollectionManager collectionManager = new CollectionManager(collectionId, scope);
 
 //then we ask for all the available views
 List<CMView<?>> views = collectionManager.findViews();
 
 System.out.println("found "+views.size()+" views:");
 for (CMView<?> view:views) System.out.println(view);

Get Metadata elements

Get a metadata view for a collection and the all the elements of the view.

 //first we instantiate a collection manager
 CollectionManager collectionManager = new CollectionManager(collectionId, scope);
 
 //then we create a view kind, in this case a MetadataView without specifying any property
 MetadataView myMedataView = new MetadataView();
 
 //finally we ask to the collection manager to list all views of the kind specified.
 List<MetadataView> views = collectionManager.findViews(myMedataView);
 System.out.println("Found views: "+views.size());
 
 //in case we not found views
 if (views.size()==0) throw new Exception("No views found");
 
 //we get the first of the list
 MetadataView view = views.get(0);
 
 //we instantiate the view reader
 MetadataViewReader metadataReader = new MetadataViewReader(view, scope);
 
 //we request all the elements
 Iterator<GCubeMetadata> metadata = metadataReader.getAll();
 
 //and print it
 while (metadata.hasNext()) System.out.println(" >"+metadata.next());

Get Metadata elements using only the metadata view ID

Get a metadata view for a collection and the all the elements of the view. Without knowing the collection ID.

 //Use the metadata view ID to find the view
 List<MetadataView> views = CMViews.findViews(scope, new MetadataView(metadataViewID));
 if (views.isEmpty()){
    System.out.println("Metadata view not found. View ID: " + metadataViewID);
 }
 
 //Just get the first view
 MetadataView myMedataView = views.get(0);	
 MetadataViewReader metadataReader = new MetadataViewReader(myMedataView, scope);

F.A.Q.

Running the examples I get the following exception:

[main] FATAL contexts.GHNClientContext  - [0.0s] GHNClientContext: gHN could not complete initialisation
java.io.FileNotFoundException: null/config/GHNConfig.xml (No such file or directory)
	at java.io.FileInputStream.open(Native Method)
	at java.io.FileInputStream.<init>(FileInputStream.java:106)
	at org.gcube.common.core.contexts.GHNContext.configureGHN(GHNContext.java:314)
	at org.gcube.common.core.contexts.GHNClientContext.configureGHN(GHNClientContext.java:35)
	at org.gcube.common.core.contexts.GHNClientContext.initialise(GHNClientContext.java:27)
	at org.gcube.common.core.contexts.GHNContext.<clinit>(GHNContext.java:252)
	at org.gcube.common.core.utils.calls.WSCall.getInitQuery(WSCall.java:55)
	at org.gcube.common.core.utils.calls.WSCall.getInitQuery(WSCall.java:25)
	at org.gcube.common.core.utils.calls.GCUBECall.<init>(GCUBECall.java:72)
	at org.gcube.common.core.utils.calls.GCUBECall.<init>(GCUBECall.java:84)
	at org.gcube.common.core.utils.calls.WSCall.<init>(WSCall.java:43)
	at org.gcube.contentmanagement.contentmanager.stubs.calls.BaseCall.<init>(BaseCall.java:25)
	at org.gcube.contentmanagement.contentmanager.stubs.calls.ManagerCall.<init>(ManagerCall.java:24)
	at org.gcube.contentmanagement.contentmanager.stubs.calls.ManagerCall.<init>(ManagerCall.java:35)
	at org.gcube.contentmanagement.contentmanager.stubs.calls.ReadManagerCall.<init>(ReadManagerCall.java:97)
	at org.gcube.contentmanagement.contentmanagerlibrary.CMReader.<init>(CMReader.java:62)
	at org.gcube.contentmanager.contentmanagerlibrary.GetDocumentsWithProjections.main(GetDocumentsWithProjections.java:34)
Exception in thread "main" java.lang.NullPointerException
	at org.gcube.common.core.utils.calls.WSCall.getInitQuery(WSCall.java:55)
	at org.gcube.common.core.utils.calls.WSCall.getInitQuery(WSCall.java:25)
	at org.gcube.common.core.utils.calls.GCUBECall.<init>(GCUBECall.java:72)
	at org.gcube.common.core.utils.calls.GCUBECall.<init>(GCUBECall.java:84)
	at org.gcube.common.core.utils.calls.WSCall.<init>(WSCall.java:43)
	at org.gcube.contentmanagement.contentmanager.stubs.calls.BaseCall.<init>(BaseCall.java:25)
	at org.gcube.contentmanagement.contentmanager.stubs.calls.ManagerCall.<init>(ManagerCall.java:24)
	at org.gcube.contentmanagement.contentmanager.stubs.calls.ManagerCall.<init>(ManagerCall.java:35)
	at org.gcube.contentmanagement.contentmanager.stubs.calls.ReadManagerCall.<init>(ReadManagerCall.java:97)
	at org.gcube.contentmanagement.contentmanagerlibrary.CMReader.<init>(CMReader.java:62)
	...

In this case you have to configure your eclipse running configuration properly [1].

Trying to get the document content I get this exception:

Exception in thread "main" java.net.MalformedURLException: unknown protocol: sms
	at java.net.URL.<init>(URL.java:574)
	at java.net.URL.<init>(URL.java:464)
	at java.net.URL.<init>(URL.java:413)
	at org.gcube.contentmanagement.contentmanagerlibrary.model.GCubeDocument.getContent(GCubeDocument.java:227)
        ...

Check your configuration about SMS protocol

Eclipse don't find some functions in the example code (like is, matches, all, either, not, more, less, after, before).

Those are helper function from the CM and are defined on Constraints class. You can statically import it with this statement:

 import static org.gcube.contentmanagement.contentmanager.stubs.model.constraints.Constraints.*;

Trying to interact with the CM I get this exception:

org.apache.commons.discovery.DiscoveryException: org.gcube.common.core.utils.handlers.GCUBEServiceHandler$NoQueryResultException
*****
  org.gcube.common.core.utils.handlers.GCUBEServiceHandler$NoQueryResultException
	at org.gcube.common.core.utils.handlers.GCUBEStatefulServiceHandler.findInstances(GCUBEStatefulServiceHandler.java:38)
	at org.gcube.common.core.utils.handlers.GCUBEServiceHandler.getInstances(GCUBEServiceHandler.java:140)
	at org.gcube.common.core.utils.handlers.GCUBEServiceHandler.run(GCUBEServiceHandler.java:110)
	at org.gcube.contentmanagement.contentmanager.stubs.calls.BaseCall$CallHandler.run(BaseCall.java:93)
	at org.gcube.contentmanagement.contentmanager.stubs.calls.WriteManagerCall.add(WriteManagerCall.java:98)
	at org.gcube.contentmanagement.contentmanager.stubs.calls.WriteManagerCall.add(WriteManagerCall.java:75)
	at org.gcube.contentmanagement.gcubedocumentlibrary.CMWriter.add(CMWriter.java:84)
	at org.gcube.personalization.userprofileaccess.impl.CMSUtils.createDocument(CMSUtils.java:67)
	at org.gcube.personalization.userprofileaccess.impl.UserProfileAccessFactoryService.createResource(UserProfileAccessFactoryService.java:331)

The exception means that there is not plugin instantiated for the specified collection. Check if the collection id is correct. Otherwise contact the support team.

Difference between revisions of "GCube Document Library"