Difference between revisions of "Annotation Management"
(→Annotation Collection Types) |
|||
(15 intermediate revisions by one other user not shown) | |||
Line 1: | Line 1: | ||
+ | [[Category: TO BE REMOVED]] | ||
+ | |||
The '''Annotation Back-End (ABE)''' service manages the entire life-cycle of annotation relationships between information objects, from their creation and collation to their retrieval, update, and deletion, independently from application-specific models of annotation content. | The '''Annotation Back-End (ABE)''' service manages the entire life-cycle of annotation relationships between information objects, from their creation and collation to their retrieval, update, and deletion, independently from application-specific models of annotation content. | ||
Line 69: | Line 71: | ||
=Design= | =Design= | ||
− | The design of the service is distributed across three port-types: the <code>Manager</code>, the <code>Broker</code>, and the < | + | The design of the service is distributed across three port-types: the <code>Manager</code>, the <code>Broker</code>, and the <code>Factory</code>. |
[[Image:ABEdesign1.png]] | [[Image:ABEdesign1.png]] | ||
Line 90: | Line 92: | ||
[[Image:ABEdesign3.png]] | [[Image:ABEdesign3.png]] | ||
− | Managers are created in response to client requests to the <code>Factory</code> [[#The Factory|port-type]. At the time and within the scope of their creation, a selection of the state of Managers is published in the Information System. In WSRF terminology, these are the Resource Properties that identify the target collection and their annotation collections and by which Managers can be discovered by clients. | + | Managers are created in response to client requests to the <code>Factory</code> [[#The Factory|port-type]]. At the time and within the scope of their creation, a selection of the state of Managers is published in the Information System. In WSRF terminology, these are the Resource Properties that identify the target collection and their annotation collections and by which Managers can be discovered by clients. |
Following publication, Managers are kept up-to-date with respect to the creation and deletion of annotation collections which do not occur through interaction with the Managers themselves. This form of maintenance is achieved by polling the Information System at regular intervals. During their lifetime, Managers satisfy client requests by interacting with WS-Resources and Running Instances of the Metadata Manager service. The interactions are based on best-effort and caching strategies. | Following publication, Managers are kept up-to-date with respect to the creation and deletion of annotation collections which do not occur through interaction with the Managers themselves. This form of maintenance is achieved by polling the Information System at regular intervals. During their lifetime, Managers satisfy client requests by interacting with WS-Resources and Running Instances of the Metadata Manager service. The interactions are based on best-effort and caching strategies. | ||
Line 102: | Line 104: | ||
[[Image:ABEdesign4.png]] | [[Image:ABEdesign4.png]] | ||
− | Like for ''Managers'', the operations of the | + | Like for ''Managers'', the operations of the <code>Broker</code> operate in bulk, either by-value or by-reference, and report fine-grained failures within return values. Similarly, the <code>Broker</code> is stateful and its state is an aggregation of the state of the <code>Manager</code>. In particular the local ‘collection proxies’ of Managers are grouped in ''collection sets'', and then collections sets are bound to the <code>Broker</code> into WS-Resources informally called ''Brokers''. |
[[Image:ABEdesign5.png]] | [[Image:ABEdesign5.png]] | ||
− | Like Managers, Brokers are created by the <code>Factory</code> [[#The Factory|port-type] persisted and publish the identifiers of the target collections in the bound set as a Resource Property. During their lifetime, Brokers also interact with WS-Resources and Running Instances of the Metadata Manager service. The interactions, however, occur indirectly, in that they are delegated to Managers dedicated to the collections in the collection set. | + | Like Managers, Brokers are created by the <code>Factory</code> [[#The Factory|port-type]] persisted and publish the identifiers of the target collections in the bound set as a Resource Property. During their lifetime, Brokers also interact with WS-Resources and Running Instances of the Metadata Manager service. The interactions, however, occur indirectly, in that they are delegated to Managers dedicated to the collections in the collection set. |
Line 200: | Line 202: | ||
* <code>AnnotationCollection</code>s are parametric in the type of annotations they handled. Here the class <code>Annotation</code> is used to indicate that no specific knowledge of annotations is required for the task. | * <code>AnnotationCollection</code>s are parametric in the type of annotations they handled. Here the class <code>Annotation</code> is used to indicate that no specific knowledge of annotations is required for the task. | ||
+ | |||
+ | * the list of <code>AnnotationCollection</code>s returned by the <code>AnnotationManager</code> are for read-only access. Any attempt to change them will generate an <code>IllegalAccessError</code> at run-time. | ||
== Reading Annotations == | == Reading Annotations == | ||
Line 230: | Line 234: | ||
Note that: | Note that: | ||
− | * <code>AnnotationCollection</code>s return the serialisations of all the annotations which could not be parsed into objects; | + | * <code>AnnotationCollection</code>s return the serialisations of all the annotations which could not be parsed into objects. Clients may simply report or otherwise act upon these failures; |
− | * | + | * <code>localiseAnnotations()</code> may be invoked multiple times to incrementally localise more and more annotations. Do note that localisation will apply only to objects whose annotations either have not been previously localised, or have been already localised but have not changed, or have changed but their changes have been committed (i.e. are synchronized with their remote counterparts). Changes to localised annotations are discussed [[#Editing Annotations | later]]. |
* <code>getAnnotations()</code> returns all the annotations currently localised for the <code>AnnotationCollection</code>, indexed first by the identifiers of the annotated objects and then by their own identifiers. | * <code>getAnnotations()</code> returns all the annotations currently localised for the <code>AnnotationCollection</code>, indexed first by the identifiers of the annotated objects and then by their own identifiers. | ||
Line 238: | Line 242: | ||
* <code>localiseAnnotations()</code> and <code>getAnnotations()</code> may also be invoked on <code>AnnotationManager</code>s in order to localise and retrieve all the annotations of the annotated objects - regardless of the annotation collection in which they are persistently stored - in a single interaction with the ABE service. In this case, the localised annotations are distributed across all the corresponding <code>AnnotationCollection</code>s of the <code>AnnotationManager</code>, and will be returned later if <code>getAnnotations()</code> is invoked on such collections. | * <code>localiseAnnotations()</code> and <code>getAnnotations()</code> may also be invoked on <code>AnnotationManager</code>s in order to localise and retrieve all the annotations of the annotated objects - regardless of the annotation collection in which they are persistently stored - in a single interaction with the ABE service. In this case, the localised annotations are distributed across all the corresponding <code>AnnotationCollection</code>s of the <code>AnnotationManager</code>, and will be returned later if <code>getAnnotations()</code> is invoked on such collections. | ||
− | + | Note also that incremental localisations may soon increase memory consumptions. Clients that wish to discard localised annotations for one or more objects may do so by invoking <code>discardAnnotations()</code> on <code>AnnotationCollection</code>s with the identifiers of the annotated objects, as follows: | |
− | + | <pre> | |
− | + | collection.discardAnnotations(someArraysOfObjectIds); | |
+ | </pre> | ||
− | + | Please note that: | |
+ | |||
+ | * annotated objects that have no local annotations, or whose annotations have been locally changed but have not been committed yet (see [[#Editing Annotations|below]], will be ignored. To force annotation removal for objects with uncommitted changes, clients may specify an optional boolean parameter: | ||
+ | |||
+ | <pre> | ||
+ | collection.discardAnnotations(someArraysOfObjectIds,true);//forces removal in spite of changes. | ||
+ | </pre> | ||
+ | |||
+ | * when no annotated objects are specified (e.g. via a <code>null</code> input), <code>discardAnnotations</code> operates on all the objects that have local annotations. | ||
+ | |||
+ | * <code>discardAnnotations()</code> is also available on <code>AnnotationManager</code>s, where it operates on all the <code>AnnotationCollection</code>s but with unvaried semantics. | ||
+ | |||
+ | == Working with Specific Types == | ||
+ | |||
+ | The <code>Annotation</code>s used so far reflect properties shared by all types of annotations and, more generically, all types of metadata (e.g. their identifier and the date of their last modification, as above). Similarly, annotations have been collected in equally generic <code>AnnotationCollection</code>s fully defined within the ABE library. | ||
+ | |||
+ | This generality is necessary to work uniformly across collections of different annotation types, but it limits what clients can do with annotations once they have been localised. Upon parsing into objects, any type-specific information is discarded and thus cannot be serialised again. In particular, <code>Annotation</code>s cannot be remotely persisted, nor can their collections be remotely created, as clients may often wish to do. Similarly, clients may wish to add capabilities to annotation collections which reflect the specificity of the annotations they manage. | ||
+ | |||
+ | The ABE library can be instructed to parse and expose the properties of specific types of annotations, and even to create collections of annotations of such types. This requires clients to: | ||
− | * define an object model | + | * define an object model for the required annotation type, say <code>TestAnnotation</code>; |
+ | |||
+ | * define an object model for collections of <code>TestAnnotation</code>s, say <code>TestCollection</code>; | ||
+ | |||
+ | * bind <code>TestCollection</code> to the namespace of <code>TestAnnotation</code>s, say <code>http://example.org</code>. | ||
+ | |||
+ | After the binding, the ABE library will: | ||
+ | |||
+ | * create <code>TestCollection</code>s for annotation collections defined in the namespace <code>http://example.org</code>; | ||
+ | |||
+ | * create <code>TestAnnotation</code>s when parsing serialisations of annotations defined in the namespace <code>http://example.org</code>. | ||
− | + | === Annotation Types === | |
− | To create | + | To create <code>TestAnnotation</code>, clients subclass the generic <code>Annotation</code> class as follows: |
<pre> | <pre> | ||
Line 335: | Line 368: | ||
* serialises its own state in <code>toXML()</code>, propagates it to its superclass for embedding in the superclass' envelope, and finally returns the result of the embedding. As a concrete class at the bottom of the annotation hierarchy, it does not need to embed the serialisation of subclasses in its own serialisation in turn. For this reason, it ignores the input it would have received otherwise from its subclasses. | * serialises its own state in <code>toXML()</code>, propagates it to its superclass for embedding in the superclass' envelope, and finally returns the result of the embedding. As a concrete class at the bottom of the annotation hierarchy, it does not need to embed the serialisation of subclasses in its own serialisation in turn. For this reason, it ignores the input it would have received otherwise from its subclasses. | ||
− | + | === Collection Types === | |
− | + | To create <code>TestCollection</code>, clients subclass the generic <code>AnnotationCollection</code> class as follows: | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
<pre> | <pre> | ||
− | |||
** Collection of test annotations.*/ | ** Collection of test annotations.*/ | ||
class TestCollection extends AnnotationCollection<TestAnnotation> { | class TestCollection extends AnnotationCollection<TestAnnotation> { | ||
− | + | /**{@inheritDoc}*/ | |
− | public | + | public Class<TestAnnotation> getAnnotationClass() {return TestAnnotation.class;} |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
} | } | ||
</pre> | </pre> | ||
− | + | Note that <code>TestCollection</code>: | |
− | * | + | * satisfies the type parameterisation of <code>AnnotationCollection</code> which is declared as <code>AnnotationCollection<ANNOTATIONTYPE extends Annotation></code>. |
− | * | + | * overrides <code>getAnnotationClass()</code> in <code>AnnotationCollection</code> to return the type of annotations associated with the collection, here <code>TestAnnotation</code> of course. |
+ | |||
+ | The final step is to bind <code>TestCollection</code> to <code>http://example.org</code>, typically during initialisation of the library: | ||
+ | |||
+ | <pre> | ||
+ | |||
+ | //defines a binding between the schema of the test annotations and the class which model them. | ||
+ | //this lets the library serialise/de-serialise test annotations correctly. | ||
+ | AnnotationContext.bindNamespace(TestAnnotation.NS, TestCollection.class); | ||
+ | |||
+ | </pre> | ||
− | The following test method | + | Note now that, although the ABE library will create <code>TestCollection</code>s whenever appropriate, <code>AnnotationManager</code>s will continue to continue to deal with all possible <code>AnnotationCollection</code>s. |
+ | The following test method illustrates how clients could dynamically assert type-specific information on the collections of an <code>AnnotationManager</code>: | ||
<pre> | <pre> | ||
Line 404: | Line 427: | ||
static public TestCollection createTestCollection(AnnotationManager manager) throws Exception { | static public TestCollection createTestCollection(AnnotationManager manager) throws Exception { | ||
− | TestCollection collection = new TestCollection(); | + | TestCollection collection = new TestCollection() { |
+ | /** {@inheritDoc} */ | ||
+ | public Map<CreationParameters, String> getCreationParameters() {//minimal set of parameters | ||
+ | Map<CollectionDescription.CreationParameters,String> map = new HashMap<CollectionDescription.CreationParameters,String>(); | ||
+ | map.put(CollectionDescription.CreationParameters.COLLECTIONNAME,"TestAnnotationCollection"); | ||
+ | map.put(CollectionDescription.CreationParameters.METADATAURI,TestAnnoation.NS); | ||
+ | map.put(CollectionDescription.CreationParameters.METADATANAME,"someformat"); | ||
+ | map.put(CollectionDescription.CreationParameters.METADATALANG,"somelanguage"); | ||
+ | return map; | ||
+ | } | ||
+ | }; | ||
manager.createAnnotationCollection(collection); | manager.createAnnotationCollection(collection); | ||
logger.info("Created Annotation Collection "+collection.getID()); | logger.info("Created Annotation Collection "+collection.getID()); | ||
} | } | ||
+ | </pre> | ||
+ | |||
+ | Note the anonymous subclass of <code>TestCollection</code> which is created here to overrides <code>getCreationParameters()</code> in <code>AnnotationCollection</code> and return the parameters which are necessary for its creation. Anonymous subclassing is here a convenient pattern of specialisation of <code>TestCollection</code>s for the purposes of collection creation, but not a mandatory one of course. | ||
+ | |||
+ | === Validation === | ||
+ | |||
+ | By default, the ABE library validates annotations just before they are committed (discussed [[#Editing Annotation|below]]). However, validation is limited to generic properties of annotations until clients provide a schema for the properties of specific annotation types. To do so, they may override the method <code>getSchema()</code> of <code>Annotation</code> when they define classes of specific annotations. In the case of <code>TestAnnotation</code>: | ||
+ | |||
+ | <pre> | ||
+ | /**{@inheritDoc*/ | ||
+ | protected Reader getSchema() throws Exception { | ||
+ | return new InputStreamReader(TestAnnotation.class.getResourceAsStream("....../schema.xsd")); | ||
+ | } | ||
+ | </pre> | ||
+ | |||
+ | Note that the schema is here loaded as a classpath resource, but any source which be streamed via a <code>Reader</code> will equally do. | ||
+ | |||
+ | As to the schema, clients are free to define it as they wish as long as they import explicitly the ABE namespace to enable validation across the inheritance hierarchy. For <code>TestAnnotation</code>s: | ||
+ | |||
+ | <pre> | ||
+ | <?xml version="1.0" encoding="UTF-8"?> | ||
+ | <xsd:schema | ||
+ | xmlns:xsd="http://www.w3.org/2001/XMLSchema" | ||
+ | xmlns:tns="http://myannotation" | ||
+ | targetNamespace="http://myannotation" | ||
+ | elementFormDefault="qualified" attributeFormDefault="unqualified"> | ||
+ | |||
+ | <xsd:import namespace="http://gcube-system.org/namespaces/annotationmanagement/abe/model"/> | ||
+ | |||
+ | <xsd:element name="foo" type="xsd:string"/> | ||
+ | |||
+ | </xsd:schema> | ||
+ | </pre> | ||
+ | |||
+ | Finally, note that if and when required, validation can be enabled or disabled at runtime through the <code>AnnotationContext</code>: | ||
+ | |||
+ | <pre> | ||
+ | ... | ||
+ | AnnotationContext.setValidation(false); | ||
+ | ... | ||
+ | |||
+ | if (AnnotationContext.isValidating()) ... | ||
+ | ... | ||
</pre> | </pre> | ||
Line 504: | Line 580: | ||
} | } | ||
</pre> | </pre> | ||
+ | |||
+ | Finally, note that <code>discardAnnotations()</code> is also available on <code>AnnotationBroker</code>s, where it applies to all <code>AnnotationManager</code>s and, in turn, <code>AnnotationCollection</code>s. |
Latest revision as of 17:15, 6 July 2016
The Annotation Back-End (ABE) service manages the entire life-cycle of annotation relationships between information objects, from their creation and collation to their retrieval, update, and deletion, independently from application-specific models of annotation content.
Clients may interact with the ABE service through a standard library of stubs generated automatically from the public service interfaces. They may also do so at a higher level of abstraction through the Annotation Back-end Library, an extension of the stub library which simplifies clients interaction with the service.
Contents
Annotation Relationships
Annotation relationships are specialisations of metadata relationships that give target objects the broad semantic of subjective and contextual assertions about source objects. More formally, annotation relationships are binary relationships with primary role is-described-by and secondary role is-annotated-by (IAB).
As specializations of metadata relationships, annotation relationships inherit all their properties:
- they are exclusive on their targets but repeatable on their sources: an annotation describes one and only one object even though the number of annotations for any given object may be unbounded;
- they preserve membership: an annotation belongs to a collection if and only if the annotated object does.
Annotation relationships induce a related specialization of the notion of collection: a collection is an annotation collection if it is a metadata collection of type IAB
. More formally, A
is an annotation collection for C
if:
-
A
is anIAB
-collection in the scope ofC
; - all the members of
A
are annotations of members ofC
, and -
A
is an annotation ofC
.
The definition of annotation relationships and annotation collections – as well as the relationships between these definitions and more generic notions in the gCube Information Model can be graphically illustrated as follows:
Annotation Model
The ABE service and library adopt a model of annotation content that is suitable for the exchange of potentially very heterogeneous annotations. In particular, the ABE exchange model complements arbitrary, application-specific notions of annotations with a small number of system-level and application-independent properties.
The model is defined in terms of XML serialisations of annotations and is in itself an extension of the exchange model adopted by Metadata Management services. In particular, it follows the same design pattern of grouping system-level properties into a header element and application-defined properties into a body element. The model is formally defined by the following schema:
<?xml version="1.0" encoding="UTF-8"?> <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:tns="http://gcube-system.org/namespaces/annotationmanagement/abe/model" targetNamespace="http://gcube-system.org/namespaces/annotationmanagement/abe/model" elementFormDefault="qualified" attributeFormDefault="unqualified"> <xsd:import namespace="http://gcube-system.org/namespaces/metadatamanagement/mm/model"/> <xsd:element name="annotation" type="tns:annotationType"/> <xsd:complexType name="annotationType"> <xsd:sequence> <xsd:element name="header" type="tns:headerType" /> <xsd:element name="body" type="tns:bodyType" /> </xsd:sequence> </xsd:complexType> <xsd:complexType name="headerType"> <xsd:sequence> <xsd:element name="previous" type="xsd:string" minOccurs="0"/> </xsd:sequence> </xsd:complexType> <xsd:complexType name="bodyType"> <xsd:sequence> <xsd:any namespace="##other" processContents="lax"/> </xsd:sequence> </xsd:complexType> </xsd:schema>
Architecture
Within the gCube architecture, the ABE service is placed at the top of a stack of Information Organisation services, from Storage Management services to Content Management services and Metadata Management services. Its main role within the stack is one of mediation between the users of Information Presentation services and Metadata Management services one step below. In this role, the value it offers to its clients is one of specialisation and transparent aggregation of functionality available more generically for the management of metadata objects.
Design
The design of the service is distributed across three port-types: the Manager
, the Broker
, and the Factory
.
Managers
The Manager
port-type manages annotations of objects which belong to a target collection. It is the preferred point of contact for clients that:
- wish to work with such annotations regardless of the annotation collections to which individual annotations belong;
- wish to work with annotations which belong to specific annotation collections.
Clients may create annotation collections for the target collection, and then add, update, retrieve or delete annotation from them. These operations map directly onto equivalent operations of the Metadata Management service and are offered to present a single point of contact for annotations. Clients may also retrieve annotations across multiple annotation collections, though of course no write operation is offered at this level of abstraction.
All operations process and produce bulk data to avoid the latency of finer-grained interactions. Similarly, all operations are conceptually overloaded to work either by-value – inputs and outputs are entirely included in message payloads – or else by-reference – input and outputs are throttled in ‘pages’ using the resultset abstraction. Finally, all operations are tolerant to fine-grained failures, through they record them and then report them as return values.
The Manager
is stateful, in that it maintains (in memory and on the local file system) summary information about the annotation collections of target collections. The information is grouped under local ‘proxies’ of the target collections, and the proxies are bound to the port-type interface on a per-request basis, in line with the implied resource pattern of WSRF. In particular, the pairing of the Manager
interface and collection proxies identifies WS-Resources informally referred to as Managers.
Managers are created in response to client requests to the Factory
port-type. At the time and within the scope of their creation, a selection of the state of Managers is published in the Information System. In WSRF terminology, these are the Resource Properties that identify the target collection and their annotation collections and by which Managers can be discovered by clients.
Following publication, Managers are kept up-to-date with respect to the creation and deletion of annotation collections which do not occur through interaction with the Managers themselves. This form of maintenance is achieved by polling the Information System at regular intervals. During their lifetime, Managers satisfy client requests by interacting with WS-Resources and Running Instances of the Metadata Manager service. The interactions are based on best-effort and caching strategies.
The public interface of the Manager
port-type can be found here.
Brokers
The Broker
port-type manages annotations of objects which belong to either one of a number of target collections. Clients may retrieve annotations for these objects regardless of the collections to which the objects or their annotations belong. For example, the Broker
is the recommended point of contact for clients that wish to browse the annotations of the results of a distributed search. Clients may also narrow their interaction to single target collections by obtaining a reference to dedicated Managers.
Like for Managers, the operations of the Broker
operate in bulk, either by-value or by-reference, and report fine-grained failures within return values. Similarly, the Broker
is stateful and its state is an aggregation of the state of the Manager
. In particular the local ‘collection proxies’ of Managers are grouped in collection sets, and then collections sets are bound to the Broker
into WS-Resources informally called Brokers.
Like Managers, Brokers are created by the Factory
port-type persisted and publish the identifiers of the target collections in the bound set as a Resource Property. During their lifetime, Brokers also interact with WS-Resources and Running Instances of the Metadata Manager service. The interactions, however, occur indirectly, in that they are delegated to Managers dedicated to the collections in the collection set.
The public interface of the Broker
port-type can be found here.
The Factory
The Factory
is the point of contact to the ABE for clients that wish to create Managers and Brokers for one or more target collections, starting from their public identifiers. In this role, it is stateless.
The public interface of the Factory
port-type can be found here.
The ABE Library
The library offers a number of facilities for interacting with the port-types and WS-Resources of the ABE service.
First of all, it encapsulates stub-based interaction behind a local object-oriented interface. The interface is distributed across a set of classes which model remote port-types (AnnotationManager, AnnotationBroker) and related state abstractions (Collection
, AnnotationCollection
), or else manage configuration aspects of the interaction with the ABE, such as scope and security(AnnotationContext
).
Throughout, the library makes use of language features which are not found in the stubs automatically generated from the remote interfaces, including high-level models of inputs and outputs, method overloading, and parametric type-checking. Behind these abstractions, the library engages in optimised and best-effort interactions with factory and WS-Resources of the ABE service. In particular, it hides from clients the discovery or creation of WS-Resources, but not the existence of remote interactions and the possibility of their failure.
In addition, the library offers convenient object bindings for the XML representations of annotations that are required by the remote interface in input or produced by it in output. Clients that wish to do so can model annotations as objects with the advantages of expressiveness and early type checking which normally characterise data binding solutions. Bindings occurs within an extensible framework defined by a root class (BaseAnnotation
) that can be extended by classes which model particular types of annotations. In particular, the root class defines the protocol that subclasses implement to serialise their ‘body’ to and from the exchange model. The framework is built atop analogous frameworks offered by Metadata and ResultSet libraries, so that annotations are transparently suitable for use as generic metadata objects and as records of a resultset.
When it comes to actually manipulate annotations, the library provides annotation buffering so as to mediate between the fine-grained nature of operations in interactive sessions and the coarse-grained nature of interaction with the remote port-types. This may enable responsive user interfaces by concentrating bulk network interactions at distinguished 'commit' points which are entirely under client control. Specifically, clients that retrieve annotations from managers or brokers transparently fill buffers specific to individual annotation collections. They can then modify the buffered annotations locally for maximum responsiveness and commit the changes remotely before filling the buffer with the next retrieval operation.
Finally, the library offers built-in support for linking annotations of the same object into 'threads' similar to those typically formed by postings in mailing lists and discussion forums, or those associated with chains of versions.
Sample Usage
Interaction with the ABE service is illustrated with the examples below. The examples are based on the ABE library, partly because it is the recommended way to interact with the service and partly because the use of plain stubs can be inferred from the public interfaces of the service
The examples are modelled as static methods of a hypothetical test class, so as to explicitate their inputs. We assume that the following variables are defined in the closure of the test methods:
//some logger GCUBELog logger = ... //some scope GCUBEScope scope = ....; //some credentials GSSCredentials credentials = ....;
Preliminaries
First, we configure the library's behaviour through its AnnotationContext
. In this sense, the AnnotationContext
acts as a GCUBEScopeManager
and a GCUBESecurityManager
for the remote interactions with the ABE service which the library undertakes on behalf of its clients. As all the test methods are assumed to execute within the same thread, the simplest of scope and security configurations will suffice:
//configures the library to call the ABE service with specific scope and credentials in this thread AnnotationContext.setScope(scope) AnnotationContext.setCredentials(credentials)
In practice, the client would use scopes and credentials as appropriate to its semantics, ensuring that they are correctly chosen and set in any execution thread that may rely on library code. In this sense, note that the AnnotationContext
:
- exposes the full
GCUBEScopeManager
andGCUBESecurityManager
interfaces for multi-threaded executions; - it can be configured to use any
GCUBEScopeManager
orGCUBESecurityManager
implementations which may already exist in the client environment (e.g. aGCUBEServiceContext
in clients that operate as services in their own right):
//configures the library to use an existing scope manager //AnnotationContext.setScopeManager(someExistingScopeManager); //AnnotationContext.setSecurityManager(someExistingSecurityManager);
Managers and Collections
The first test method uses the AnnotationContext
to create and return an AnnotationManager
for some annotated collection, i.e. a collection whose elements (the annotated objects) may be associated with annotations in one or more annnotation collections:
/** * Creates an annotation manager for an annotated collection and display the creation properties of all the related annotation collections. * @param collectionID the collection identifier. */ public static void getAnnotationManager(String collectionID) throws Exception { AnnotationManager manager = AnnotationContext.createManager(collectionID); for (AnnotationCollection<Annotation> collections : manager.getAnnotationCollections().values()) { //collections are indexed by their identifiers for easy lookup, don't need it here logger.info("***********************"); logger.info("ID:"+collection.getID()); for (Map.Entry<CollectionDescription.CreationParameters,String> prop : collection.getCreationParameters().entrySet()) logger.info(prop.getKey().name()+":"+prop.getValue()); logger.info("***********************"); } }
Note the following:
-
getAnnotationManager()
triggers a remote interaction with the ABE services and may return aGCUBEException
(we don't specifically deal with it here). This behaviour characterises all the other remote operations of the ABE library.
- The creation properties form a subset of those defined by Metadata Management Library, which the ABE library extends (indeed, it is its only dependency).
-
AnnotationCollection
s are parametric in the type of annotations they handled. Here the classAnnotation
is used to indicate that no specific knowledge of annotations is required for the task.
- the list of
AnnotationCollection
s returned by theAnnotationManager
are for read-only access. Any attempt to change them will generate anIllegalAccessError
at run-time.
Reading Annotations
The following test methods show how remotely persisted annotations can be localised into AnnotationCollection
s:
/** * Localises all the annotations in a given annotation collection for one of more annotated objects. * @param collection the annotation collection. * @param oids the object identifiers. */ public static void getAnnotationsByCollection(AnnotationCollection collection, List<String> oids) { Map<String,List<String>> unparsedAnnotations = collection.localiseAnnotations(oids); //remote operation logger.info("Unparsed Annotations:"); for (String oid : unparsedAnnotations.keySet()) { logger.info("Object "+oid); for (String annotationString : unparsedAnnotations.get(oid)) logger.info("Annotations\n"+annotationString); } logger.info("Localised Annotations:"); //all the localised annotations, indexed by annotated object first and by their own identifier next. Map<String,Map<String,Annotation> annotations = collection.getAnnotations(); for (String oid : annotations) { logger.info("Object "+oid); for (Annotation annotation : annotations.get(oid).values()) logger.info("Annotations\n"+annotation); } }
Note that:
-
AnnotationCollection
s return the serialisations of all the annotations which could not be parsed into objects. Clients may simply report or otherwise act upon these failures;
-
localiseAnnotations()
may be invoked multiple times to incrementally localise more and more annotations. Do note that localisation will apply only to objects whose annotations either have not been previously localised, or have been already localised but have not changed, or have changed but their changes have been committed (i.e. are synchronized with their remote counterparts). Changes to localised annotations are discussed later.
-
getAnnotations()
returns all the annotations currently localised for theAnnotationCollection
, indexed first by the identifiers of the annotated objects and then by their own identifiers.
-
localiseAnnotations()
andgetAnnotations()
may also be invoked onAnnotationManager
s in order to localise and retrieve all the annotations of the annotated objects - regardless of the annotation collection in which they are persistently stored - in a single interaction with the ABE service. In this case, the localised annotations are distributed across all the correspondingAnnotationCollection
s of theAnnotationManager
, and will be returned later ifgetAnnotations()
is invoked on such collections.
Note also that incremental localisations may soon increase memory consumptions. Clients that wish to discard localised annotations for one or more objects may do so by invoking discardAnnotations()
on AnnotationCollection
s with the identifiers of the annotated objects, as follows:
collection.discardAnnotations(someArraysOfObjectIds);
Please note that:
- annotated objects that have no local annotations, or whose annotations have been locally changed but have not been committed yet (see below, will be ignored. To force annotation removal for objects with uncommitted changes, clients may specify an optional boolean parameter:
collection.discardAnnotations(someArraysOfObjectIds,true);//forces removal in spite of changes.
- when no annotated objects are specified (e.g. via a
null
input),discardAnnotations
operates on all the objects that have local annotations.
-
discardAnnotations()
is also available onAnnotationManager
s, where it operates on all theAnnotationCollection
s but with unvaried semantics.
Working with Specific Types
The Annotation
s used so far reflect properties shared by all types of annotations and, more generically, all types of metadata (e.g. their identifier and the date of their last modification, as above). Similarly, annotations have been collected in equally generic AnnotationCollection
s fully defined within the ABE library.
This generality is necessary to work uniformly across collections of different annotation types, but it limits what clients can do with annotations once they have been localised. Upon parsing into objects, any type-specific information is discarded and thus cannot be serialised again. In particular, Annotation
s cannot be remotely persisted, nor can their collections be remotely created, as clients may often wish to do. Similarly, clients may wish to add capabilities to annotation collections which reflect the specificity of the annotations they manage.
The ABE library can be instructed to parse and expose the properties of specific types of annotations, and even to create collections of annotations of such types. This requires clients to:
- define an object model for the required annotation type, say
TestAnnotation
;
- define an object model for collections of
TestAnnotation
s, sayTestCollection
;
- bind
TestCollection
to the namespace ofTestAnnotation
s, sayhttp://example.org
.
After the binding, the ABE library will:
- create
TestCollection
s for annotation collections defined in the namespacehttp://example.org
;
- create
TestAnnotation
s when parsing serialisations of annotations defined in the namespacehttp://example.org
.
Annotation Types
To create TestAnnotation
, clients subclass the generic Annotation
class as follows:
** Sample annotation type.*/ public class TestAnnotation extends Annotation { //Using XML Pull technology here for parsing and serialising /** XML Parser. */ private KXmlParser parser = new KXmlParser(); /** XML Serializer. */ private KXmlSerializer serializer = new KXmlSerializer(); /**Sample*/ String foo; /** Elements name of sample annotation-specific property */ private static final String FOO_ELEMENT_NAME = "foo"; /** Changes the property of the annotation * @return the property. */ public String getFoo() {return this.foo;} /** * Returns the property of the annotation * @param foo the property */ public void setFoo(String foo) {this.foo = foo;} /** Annotation serialisation namespace. */ public static final String NS="http://example.org"; /**{@inheritDoc}*/ public void fromXML(String xml) throws Exception { //parses annotation-specific properties super.fromXML(xml); //trigger parsing of generic annotation properties try { parser.setInput(new StringReader(xml)); parser.setFeature(KXmlParser.FEATURE_PROCESS_NAMESPACES, true); loop: while (true) { int tokenType = parser.next(); switch (tokenType){ case KXmlParser.START_TAG : // remember position and name of tag String tag = parser.getName(); if (tag.equals(FOO_ELEMENT_NAME) && parser.getNamespace().equals(NS)) this.setFoo(parser.nextText()); break; case KXmlParser.END_DOCUMENT : break loop; } } } catch (Exception e){ logger.error("Could not parse annotation:"+e.toString()+":"+e.getStackTrace()[0].toString()); throw new Exception("Could not parse annotation:", e); } } /**{@inheritDoc}*/ public String toXML(String body) throws Exception {//serialises annotation-specific properties StringWriter output = new StringWriter(); serializer.setOutput(output); parser.setFeature(KXmlParser.FEATURE_PROCESS_NAMESPACES, true); serializer.setPrefix("test", NS); serializer.startTag(NS,FOO_ELEMENT_NAME).text(this.getFoo()==null?"":this.getFoo()).endTag(NS,FOO_ELEMENT_NAME); return super.toXML(output.toString()); //passes serialisation up for embedding in annotation envelope //ignore body parameters, this is a 'leaf annotation' does not define an envelope to fill by subclasses //if it did, it would have had to embed the body of subclasses in its own envelope in turn. }}
Essentially, TestAnnotation
:
- guarantees high-level access to type-specific annotation state (such as the
foo
property);
- overrides
fromXML()
andtoXML()
of theAnnotation
class to, respectively, parse and serialise type-specific state.
In particular, note that TestAnnotation
:
- uses a pull-approach to parsing and serialise its own state but the choice is irrelevant to the ABE library. Others approaches and technologies could have been equally chosen against the
String
inputs.
- parses its own state in
fromXML()
simply by extracting type-specific state from the full serialisation of the annotation which receives in input and by propagating parsing of generic state to its superclasse (here as a pre-condition to its own parsing).
- serialises its own state in
toXML()
, propagates it to its superclass for embedding in the superclass' envelope, and finally returns the result of the embedding. As a concrete class at the bottom of the annotation hierarchy, it does not need to embed the serialisation of subclasses in its own serialisation in turn. For this reason, it ignores the input it would have received otherwise from its subclasses.
Collection Types
To create TestCollection
, clients subclass the generic AnnotationCollection
class as follows:
** Collection of test annotations.*/ class TestCollection extends AnnotationCollection<TestAnnotation> { /**{@inheritDoc}*/ public Class<TestAnnotation> getAnnotationClass() {return TestAnnotation.class;} }
Note that TestCollection
:
- satisfies the type parameterisation of
AnnotationCollection
which is declared asAnnotationCollection<ANNOTATIONTYPE extends Annotation>
.
- overrides
getAnnotationClass()
inAnnotationCollection
to return the type of annotations associated with the collection, hereTestAnnotation
of course.
The final step is to bind TestCollection
to http://example.org
, typically during initialisation of the library:
//defines a binding between the schema of the test annotations and the class which model them. //this lets the library serialise/de-serialise test annotations correctly. AnnotationContext.bindNamespace(TestAnnotation.NS, TestCollection.class);
Note now that, although the ABE library will create TestCollection
s whenever appropriate, AnnotationManager
s will continue to continue to deal with all possible AnnotationCollection
s.
The following test method illustrates how clients could dynamically assert type-specific information on the collections of an AnnotationManager
:
/** * Iterates over all the collections of an AnnotationManager and returns the first TestCollection it finds, if any. * @param manager the AnnotationManager * @return the TestCollection or nul */ static public TestCollection findTestCollection(AnnotationManager manager) throws Exception { logger.info("Looking for a collection with test annotations"); for (AnnotationCollection<Annotation> coll : manager.getAnnotationCollections().values()) if (TestAnnotation.class.isAssignableFrom(coll.getAnnotationClass())) return (TestCollection) coll; //cast to re-assert static type-checking when working with this collection return null; }
The following test methods shows instead how to create a TestCollection
:
/** * Creates a TestCollection with an AnnotationManager. * @param manager the AnnotationManager * @return the TestCollection or nul */ static public TestCollection createTestCollection(AnnotationManager manager) throws Exception { TestCollection collection = new TestCollection() { /** {@inheritDoc} */ public Map<CreationParameters, String> getCreationParameters() {//minimal set of parameters Map<CollectionDescription.CreationParameters,String> map = new HashMap<CollectionDescription.CreationParameters,String>(); map.put(CollectionDescription.CreationParameters.COLLECTIONNAME,"TestAnnotationCollection"); map.put(CollectionDescription.CreationParameters.METADATAURI,TestAnnoation.NS); map.put(CollectionDescription.CreationParameters.METADATANAME,"someformat"); map.put(CollectionDescription.CreationParameters.METADATALANG,"somelanguage"); return map; } }; manager.createAnnotationCollection(collection); logger.info("Created Annotation Collection "+collection.getID()); }
Note the anonymous subclass of TestCollection
which is created here to overrides getCreationParameters()
in AnnotationCollection
and return the parameters which are necessary for its creation. Anonymous subclassing is here a convenient pattern of specialisation of TestCollection
s for the purposes of collection creation, but not a mandatory one of course.
Validation
By default, the ABE library validates annotations just before they are committed (discussed below). However, validation is limited to generic properties of annotations until clients provide a schema for the properties of specific annotation types. To do so, they may override the method getSchema()
of Annotation
when they define classes of specific annotations. In the case of TestAnnotation
:
/**{@inheritDoc*/ protected Reader getSchema() throws Exception { return new InputStreamReader(TestAnnotation.class.getResourceAsStream("....../schema.xsd")); }
Note that the schema is here loaded as a classpath resource, but any source which be streamed via a Reader
will equally do.
As to the schema, clients are free to define it as they wish as long as they import explicitly the ABE namespace to enable validation across the inheritance hierarchy. For TestAnnotation
s:
<?xml version="1.0" encoding="UTF-8"?> <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:tns="http://myannotation" targetNamespace="http://myannotation" elementFormDefault="qualified" attributeFormDefault="unqualified"> <xsd:import namespace="http://gcube-system.org/namespaces/annotationmanagement/abe/model"/> <xsd:element name="foo" type="xsd:string"/> </xsd:schema>
Finally, note that if and when required, validation can be enabled or disabled at runtime through the AnnotationContext
:
... AnnotationContext.setValidation(false); ... if (AnnotationContext.isValidating()) ... ...
Editing Annotations
Localised annotations in AnnotationCollection
s can be updated or deleted, while new ones can be created and added to the collections. Clients may do so with AnnotationWriter
s, i.e. managers of state-changing operations for annotations of given annotated objects in given AnnotationCollection
s.
The following test method shows how to obtain an AnnotatonWriter
for an annotated object from an AnnotationCollection
, and how to use it to create or change the state of its annotations in the collection:
/** * Randomly edits the annotations of an object in an AnnotationCollection. * @param collection the collection * @param objID the object identifier * @throws Exception if the annotations could not be edited. */ public static void editAnnotations(TestCollection collection, String objID) throws Exception { AnnotationWriter<TestAnnotation> writer = collection.getWriter(objID); for (TestAnnotation annotation : writer.getAnnotations().values()) { switch ((int) (Math.random()*2+1)) {//randomly choosing editing operation case 1 : annotation.setFoo(annotation.getFoo()+"_updated");writer.updateAnnotation(annotation);break; case 2 : writer.deleteAnnotations(annotation); } writer.addAnnotations(new TestAnnotation(),new TestAnnotation()); } }
Editing operations change or add localised annotations, but the changes need to be remotely committed to persist the lifetime of the editing session with the ABE library (except for deletions of newly added annotations, of course):
... List<TestAnnotation> uncommitted = writer.commitAnnotations(); logger.info("Uncommitted Annotations:"); for (TestAnnotation annotation: uncommitted) logger.info("Annotation("+annotation.getStatus()+")\n"+annotation.serialize()); ...
Please note that:
- like for
localiseAnnotations()
,commitAnnotations()
returns its own failures, i.e. the lists of annotation objects which could not be committed;
- suitable commit points must be identified by the client in relation to their semantics, e.g. they may be explicitly requested by end-users or else implicitly triggered by interactive or non-interactive events.
- if between commit points annotations have changed for multiple annotated objects, then it is efficient to commit all the changes at once, in a single transaction with the ABE service. For this reason,
commitAnnotation()
operations are also available onAnnotationCollection
s, either over one or moreAnnotationWriter
s or over all the annotations currently localised, as the next code samples show:
... Map<String,List<TestAnnotation>> uncommitted = collection.commitAnnotations(writers); for (String oid :uncommitted.keySet()) { logger.info("Object "+oid); for (TestAnnotation anno : uncommitted.get(oid)) logger.info("Annotation("+anno.getStatus()+")\n"+anno.serialize()); } ... ... Map<String,List<TestAnnotation>> uncommitted = collection.commitAnnotations(); for (String oid :uncommitted.keySet()) { logger.info("Object "+oid); for (TestAnnotation anno : uncommitted.get(oid)) logger.info("Annotation("+anno.getStatus()+")\n"+anno.serialize()); } ...
Working with Brokers
While AnnotationManager
s aggregate annotations that belong to many AnnotationCollection
s, AnnotationBrokers
s go a step further and aggregate annotations for objects in different annotated collections. Using patterns similar to those associated with AnnotationManager
s, the following test method shows how to obtain an AnnotationBroker
from the AnnotationContext
and how to use it to localise annotations for one or more objects across one or more annotated collections:
/** * Creates an AnnotationBroker for one or more annotated collection and uses it to localise annotations for one or * more objects across one or more annotated collections. * @param collectionIDs the identifiers of the annotated collections. * @param ids the identifiers of the annotated object. * @throws Exception if the test could not be completed. */ public static void testBroker(List<String> collectionIDs, Map<String, List<String>> ids) throws Exception { AnnotationBroker broker = AnnotationContext.createBroker(collectionIDs); Map<String, List<String>> unparsedAnnotations = broker.localiseAnnotations(ids); logger.info("Unparsed Annotations:"); for (String oid : unparsedAnnotations.keySet()) { logger.info("Object "+oid); for (String annotationString : unparsedAnnotations.get(oid)) logger.info("Annotations\n"+annotationString); } logger.info("Localised Annotations:"); //all the localised annotations, indexed by annotated object first and by their own identifier next. Map<String,Map<String,Map<String, Annotation>>> annotations = broker.getAnnotations(); for (String id : annotations.keySet()) { logger.info("Collection "+id); for (String oid : annotations.keySet()) { logger.info("Object "+oid); for (Annotation annotation : annotations.get(oid).get(id).values()) logger.info("Annotations\n"+annotation); } } }
Finally, note that discardAnnotations()
is also available on AnnotationBroker
s, where it applies to all AnnotationManager
s and, in turn, AnnotationCollection
s.