Revision as of 09:49, 12 March 2013

gCube provides client libraries for many of its services and defines a general model for their design. The model requires that all libraries offer a common set of capabilities and adopt uniform patterns for the design of their APIs, regardless of the semantics of target services or the technology stacks available to call them. The model, however, does not indicate how capabilities and patterns should be implemented, nor does it mandate low-level API details.

The client library framework supports the implementation of client libraries which comply with the model. Through code sharing, the framework reduces development costs for client libraries and ensures the consistency and correctness of their implementations.

In this document, we assume familiarity with the design model and illustrate how the framework can be used to develop a model-compliant client library for a hypothetical foo service.

Distribution and Dependencies

The framework is layered across as a set of components, all of which are available in our Maven repositories as artifacts in the org.gcube.core group.

common-clients is the top layer of the framework and comprises classes and interfaces that do not depend on particular client stacks. In this sense, common-clients is as general as the design model.

Lower layers of the framework adapt common-clients to specific client stacks. At the time of writing, two such layers are available:

• common-gcore-clients: adapts common-clients to the gCore stack, the very same JAX-RPC stack used by the dominant class of gCube services. We refer to client libraries developed against common-gcore-clients as gCore client libraries.
• common-fw-clients: adapts common-clients to the Featherweight Stack, i.e. a newer and leaner stack based on JAX-WS specifications. We refer to client libraries developed against common-fw-clients as featherweight client libraries.

Given the advantages of the FeatherWeight Stack, common-gcore-clients should be seen as a bridge to a legacy client stack. Developing client libraries against common-fw-clients is thus the recommended choice. To facilitate migration of existing gCore client libraries, however, in what follows we show how to develop a client library against both stacks, i.e. as a gCore client library or as a featherweight client library. The two layers of the framework diverge in fact only in a few points, besides which they expose the same APIs (under different packages).

In both cases, we assume that the client library for foo is developed as a Maven project, in line with system recommendations. To use the framework, a gCore client library declares a compile-time dependency on common-gcore-clients in its POM, as follows:

<dependency>
 <groupId>org.gcube.core</groupId>
 <artifactId>common-gcore-clients</artifactId>
 <version>...</version>
</dependency>

Similarly, a featherweight library declares a compile-time dependency on common-fw-clients, as follows:

<dependency>
 <groupId>org.gcube.core</groupId>
 <artifactId>common-fw-clients</artifactId>
 <version>...</version>
</dependency>

Other distinguished dependencies of the client library are of course to the target stack. For a gCore client library, the key dependency is on JAX-RPC service stubs, which are typically provided in stand-alone libraries used also by service implementations, e.g.:

<dependency>
 <groupId>org.gcube.samples</groupId>
 <artifactId>foo-stubs</artifactId>
 <version>...</version>
</dependency>

A featherweight client library also relies on a layer of service stubs, though the relevant standard is now JAX-WS. JAX-WS stubs include annotated Service Endpoint Interfaces (SEIs) and data transfer objects, and we discuss elsewhere how such artefacts can be produced and used to call the service. We notice here that JAX-WS stubs are typically included in the client library itself. Since services based on JAX-RPC have no use for JAX-WS stubs, there is less incentive within the system to distribute such stubs in standalone libraries.

Overview

We consider first the requirements that the model raises against client libraries. This illustrates the challenges faced by client libraries to achieve compliance with the model, hence the likelihood of variations in style and quality across their implementations. We then overview the support offered by the framework towards meeting those challenges in a consistent and cost-effective manner.

Implementation Requirements

The design model for client libraries mandates the use of service proxies. The library represents foo with an interface Foo and its default implementation DefaultFoo. Foo defines methods that eventually map on (often directly correspond to) the remote operations of foo endpoints, and DefaultFoo implements the methods against the lower-level API of the target stack.

For gCore client libraries and featherweight client libraries this lower-level API is defined by, respectively, JAX-RPC and JAX-WS service stubs. For example, if FooPortType is the type of foo stubs and bar() one of their String-valued methods, the proxy pattern maps onto code of this form:

public interface Foo { 
  String bar() throws ...; 
}
 
public class DefaultFoo implements Foo { 
  public String bar() throws ... { 
   ...FooPortType endpoint...
 
   try {
 
     return endpoint.bar();
 
   }
   catch(...) { //fault handling
 
   }
 
}

In itself, the pattern is straightforward. Some complexity may arises from the design requirements of particular Foo methods, including particular types inputs or outputs (e.g. e.g. streams)), faults with diverse semantics (e.g. outages vs. contingencies), and particular invocation semantics (e.g. asynchronous)). In most cases, the design directives provided by the model to address these requirements do not require dedicated support, or else point clearly to dedicated gCube libraries that provide it (e.g. the streams library, the scope library, or the security library). The framework includes the classes and interfaces upon which its directives are based, and complements the support offered by dedicated libraries whenever requires. We discuss this kind of support here.

Arguably, the strongest demand that the model makes on the client library concerns how Foo proxies bind to service endpoints. The requirement is for two binding modes:

• in direct mode, the proxies obtain the address of given service endpoints from clients and execute all their methods against those endpoints;

• in discovery mode, the proxies identify service endpoints from queries to gCube discovery services;

Implementing direct mode is fairly simple, as clients provide all the binding information. They model addresses as W3CEndpointReferences or - depending on wether foo is a stateless or stateful service - as (host, port) pairs or (host, port, key) triples. When necessary (e.g. for gCore client libraries that use proprietary models of addresses and references, cf. EndpointReferenceType), the library is required to implement address conversion and address validation. Though conceptually simple, the task is error-prone and sufficiently boilerplate to call for reuse through the framework.

Implementing discovery mode is significantly more complicated, as the proxies are responsible for using query results in a fault-tolerant and optimised manner. The model requires that the library implements binding and caching strategies which depend on correct handling of a variety of different fault types. Queries must be value objects that hide the lower-level idioms of query formulation and submission required by the gCube discovery services.

Since the two modes are markedly different, combining them in a single proxy implementation presents its own challenges. In particular, it becomes difficult to implement Foo’s methods uniformly, regardless of the binding mode of proxy instances. Lack of homogeneity extends to proxy configuration and threatens the overall testability of the code. Solutions to these problems are likely to vary in style and quality across client libraries.

Framework Support

We now give a tour of the support offered by the framework towards meeting the implementation challenges discussed above. We expand on the role and use of individual framework components in later sections.

The key contribution of the framework comes in the form of ProxyDelegates, i.e. components that know how to make calls in a given mode on behalf of proxies. The idea is that the library defines explicit Call objects and its proxies pass them to the delegates for execution.

With this pattern, Foo proxies can be implemented as follows:

public class DefaultFoo implements Foo {
 
  private ProxyDelegate<FooPortType> delegate; 
  public void bar() throws ... {
 
   Call<FooPortType,String> barCall = ... 
   try {
 
     return delegate.make(barCall); 
   }
   catch(...) { //fault handling
 
   }  
   ...
}

Calls are anonymous implementations of a simple callback interface:

Call<FooPortType,String> barCall = new Call<FooPortType,String>() {
 @Override
 public String call(FooPortType endpoint) {  return endpoint.bar(); }}

We discuss Calls in more detail here.

Delegates make the callback above, providing the required stub instance fully configured with the address of the target service endpoint, the scope associated with the current thread, and any other call-specific information. Foo proxies need not concern themselves with how the delegate discovers and/or binds to endpoints. They can implement their methods uniformly against the delegate. We discuss ProxyDelegates in more detail here.

Of course, delegates need to be configured to act on behalf of Foo proxies. The main piece of configuration is a Plugin object that implements an interface of callback methods. The delegates will consult the plugin to obtain information and services which are specific to theclient library. They will use this information to adapt their binding strategies to foo endpoints.

public class FooPlugin implements Plugin<FooPortType,Foo> {
  ...
}

Plugins are thus the main point of interface between the framework and the client library. We discuss their callbacks in detail here.

Besides Plugins, delegates need configuration specific to the mode in which they are to operate. Delegates that operate in direct mode needs given endpoint addresses, and delegates that operate in discovery mode need queries. Some of the required configuration is provided by Foo clients, other is be provided by the client library.

Building and configuring delegates does not need to fall upon the client library either. The library can use builders provided by the framework instead, a StatelessBuilder if foo is stateless and a StatefulBuilder if foo is stateful. The library needs only to create these builders on behalf of its clients, ideally from a static factory method that can be conveniently imported by clients. Assuming a a stateless foo for example, the library can expose builders as follows:

public class FooProxies {
 
  private static final FooPlugin plugin = new FooPlugin(); 
  public static StatelessBuilder<Foo> foo() {    return new StatelessBuilderImpl<FooPortType,Foo>(plugin);  }}

Here, the library creates builders with its Plugin. The builders will then:

• gather the required configuration from the clients, using a fluent and statically typed API;

• create and configure a delegate with the Plugin and the configuration provided by clients. Different forms of configurations result in delegates that work in direct or in discovery mode;

• collaborate with the Plugin to give back to clients Foo proxies configured with the delegate;

For example, library clients may use the StatelessBuilder above as follows:

import static ...FooProxies.*;
...
Foo proxy = foo().at(“acme.org”,8080).build();

Since the client is providing the address of a given endpoint here, the builder creates a Foo proxy that uses a delegate which makes calls in direct mode to that endpoint. On the other hand, if the client uses the DSL as follows:

Foo proxy = foo().build();

the builder creates a Foo proxy that uses a delegate which makes calls in discovery mode. Thus clients are fluently driven towards the proxies they need, and the library can implement its proxies ignoring configuration and binding mode issues.

The builders can also gather additional configuration required by the model (e.g. timeouts), as well as configuration which is specific to the client library. We discuss these possibilities in detail here.

If foo is stateful, the DSL of StatefulBuilders makes room for the configuration of instance queries. Again, the framework provides StatefulQuerys for foo instances and the client library needs only to customise the queries and return them to clients. For example:

public class FooProxies {
  private static final FooPlugin plugin = new FooPlugin();
  ...
  public static StatefulQuery name(String name) {    StatefulQuery query = new StatefulQuery(plugin);    query.addCondition(“//Name”,name);    return query;  }}

Here the library exposes queries for service instances that verify a given condition, using XPath to reach within the instance descriptions published within the system. Clients can then embed queries in the DSL of builders. The following example illustrates the approach for a gCore client library:

import static ...FooProxies.*;
...
Foo proxy = foo().matching(name(“..”)).build();

We discuss queries in more detail here.

Calls

A call to a foo endpoint is represented in the framework as an object of type Call, where Call is an interface defined in common-clients as follows:

package org.gcube.common.clients;
 
public interface Call<S, R> {
 
	R call(S endpoint) throws Exception;
}

The type parameters describe, respectively, the service stubs and the values returned by the call. Thus a Call to foo endpoints that returns String values is typed as Call<FooPortType,String>.

For its simplicity, Call lends itself to anonymous implementations within proxy classes, e.g.:

public class DefaultFoo implements Foo {
  ...
  @Override
  public void bar() throws ... {
 
   Call<FooPortType,String> barCall = new Call<FooPortType,String>() {      @Override      public String call(FooPortType endpoint) {        return endpoint.bar();    };       ...
 }
}

Alternatively, Calls can be returned from factory methods of a dedicated class (e.g. FooCalls), though the approach is verbose and should be pursued only when implementations are substantial. This should rarely be the case, however, since Calls are expected to do little more than delegate to stubs. In particular, Calls should not:

• include code that converts inputs and outputs between the stub API and proxy API, if such code is needed. The task may fall within the scope of Calls but it is a better practice to factor conversion code outside Call classes, and in fact proxy classes, where it can also be more easily unit tested. Wether the conversion is performed by static methods of some utility class (e.g. SomeStubType Utils.convert(SomeProxyType)) or in more object-oriented fashion (e.g. SomeStubType SomeProxyType.toStub()), Calls should limit themselves to invoke conversion code before and after delegation;

• engage in failure handling and let error propagate outside the scope of the call()<code> method, which is intentionally designed to throw any <code>Exception. Failures should be handled instead in proxy classes, as Calls are passed to ProxyDelegates for execution, as we discuss next;

• set timeouts on stubs instances, or else proxy them in order to set a scope on the outgoing calls. As we shall see, these services are offered transparently by the framework.

Delegates

ProxyDelegates implement a strategy for making Calls to foo endpoints on behalf of proxies. Strategies are encapsulated within delegates and proxies need not concern with their details. The ProxyDelegate interface is thus defined as follows:

package org.gcube.common.clients.delegates;
import ...
 
public interface ProxyDelegate<S> {
	<V> V make(Call<S, V> call) throws Exception;
	ProxyConfig<?,S> config();
}

The type parameter describes the service stubs used in Calls. Thus a ProxyDelegate for Foo proxies is typed as ProxyDelegate<FooPortType>.

The config() method of the interface exposes the configuration of the delegate, an object of type ProxyConfig defined as follows:

package org.gcube.common.clients.config;
import ....;
 
public interface ProxyConfig<A,S> {
 ProxyPlugin<A,S,?> plugin();
 int timeout();
 void addProperty(String name, Object value);
 void addProperty(Property property);
 boolean hasProperty(String property);
 <T> T property(String property, Class<T> clazz)
 throws IllegalStateException, IllegalArgumentException;
}

Thus delegates expose three pieces of configuration:

• the Plugin of the client library, here under the more generic interface ProxyPlugin under which it is known in common-clients. As we discuss in more detail later, plugins provide delegates with the information they require to adapter their strategy to the target services. This piece of configuration is typically of less relevance to proxies, which normally do not need to access library-specific information and, when they do, can always obtain them through direct means (e.g. exposing singleton plugins as constants, or returning non-singleton plugins from factory methods in utility classes);

• the timeout of calls. As we will see later, timeouts may be either defined by defaults in the framework or the client library, or they may be explicitly configured by clients at proxy creation time. Like Plugins, timeouts are set directly by ProxyDelegates and remain transparent to proxies;

• zero or more Propertys, i.e. arbitrary named objects that capture the custom configuration of proxies. When we discuss builders, we show how client libraries can define defaults for such properties as well as accept clients overrides. Accordingly, this is the only piece of configuration that relates directly to proxies rather than ProxyDelegates. The delegates will ignore its meaning and will not use it, but they will make it conveniently available to proxies as part of their configuration. The ProxyConfig interface allows clients to add properties in a couple of different forms, inspect the configuration for the existence of given properties, and access given properties under their specific type.

Notice that, since ProxyDelegates carry the custom configuration of proxies, proxies are fully configured with delegates. This allows proxy classes to resolve initialisation in a minimal manner:

public class DefaultFoo implements Foo {
 
  private final ProxyDelegate<FooPortType> delegate; 
  public DefaultFoo(ProxyDelegate<FooPortType> delegate) {    this.delegate=delegate;  }}

The key method of ProxyDelegate is make(), which proxies invoke to delegate the execution of their Calls. The method is parametric in the output type of Calls, i.e. can be passed any Call that expects a stub instance of the right type.

The method is executed differently by different implementations of the interface. Unsurprisingly, the implementations mirror the binding model required by the design model for client libraries. DirectDelegate makes Calls in direct mode, and DiscoveryDelegate makes Calls in discovery mode. Different delegates require different configurations too. DirectDelegate expects an EndpointConfig with a given endpoint address, while DiscoveryDelegate expects a DiscoveryConfig with a Query and an EndpointCache of endpoint addresses. Caches are used internally by DiscoveryDelegates and client libraries need not be aware of their existence (though we will see later on that they can provide specific cache implementations, if they wish). We discuss queries in detail in a later section.

 All delegates, however, perform make the following callbacks on the Plugin found in their configuration:

• given an address of a foo endpoint they ask the plugin to resolve that address in a foo stub instance, which they then pass to Calls;

• given a failure thrown by Calls, the ask the plugin to convert that fault into an equivalent fault to rethrow to proxies from make().

We show in later sections how Plugins implement these callbacks, and the implications of fault conversion for the overall fault handling strategy of client libraries.

Builders

ProxyDelegates and proxies can be created and configured with builders provided by the framework. StatelessBuilders are suitable for proxies that target stateless services, while StatefulBuilders serve proxies that target stateful services. Both interfaces are defined in common-gcore-clients and common-fw-clients, where they adapt more general facilities defined in common-clients.

Client libraries construct builders with their Plugins and then pass them to their clients, typically from factory methods that can be statically imported, e.g.:

public class FooProxies {
  private static final FooPlugin plugin = new FooPlugin();
  public static StatelessBuilder<Foo> foo() {
    return new StatelessBuilderImpl<FooPortType,Foo>(plugin);
  }
}

Clients use builders to construct sentences of an embedded Domain Specific Language (DSL) suitable for proxy creation and configuration. Sentence construction progresses through a number of method invocations, each of which returns the same builder implementation but under different interfaces that represent the different stages of a partially built sentence. In particular, the interface exposed by the builder at each step allows only the invocation of methods that take clients to the next possible steps. Accordingly, clients are guided by the completion function of their IDEs (as well as the guard of the typechecker) towards well-formed sentences. For example:

import static ...FooProxies.*;
import static java.util.concurrent.TimeUnit.*;
...
Foo proxy = foo().at(“acme.org”,8080).withTimeout(10,SECONDS).build();

Here, foo() returns a StatelessBuilderImpl under an interface which is suitable for the start of the sentence, i.e. StatelessBuilder. One of the available methods at this stage is at(String,int), and invoking it returns the builder under a second interface that allows clients to invoke the method withTimeout(int, TimeUnit). Calling this latter method, leaves the client with less options to continue the sentence (e.g. it cannot invoke at() again). One option allows the client to end the sentence by invoking the method build(), and the client chooses it. The option is available because the builder has gathered all the information required to create a ProxyDelegate which can work with the configuration provided so far by the client. The builder can then consult the Plugin to obtain a DefaultFoo instance configured with the ProxyDelegate, which the builder can then relay back to the clients.

Under this fluent interface, clients can use StatelessBuilders to produce a number of sentences, including:

at(String,int).build();
at(URI).build();
at(URL).build();
at(String,int).withTimeout(int,TimeUnit).build();
at(URL).withTimeout(int,TimeUnit).build();
at(URI).withTimeout(int,TimeUnit).build();
at(URL).with(Property).build();
withTimeout(int,TimeUnit)..build();
with(Property).with(“...”,Object).withTimeout(int,TimeUnit).build();
build();
...

and many others. The grammar of the DSL supported by StatelessBuilders is in fact the following:

<clause> = <first>.<final> | <final> 
<first> = <address>.<second> | <property>.<first> | <second>
<second> = <timeout>.<final> | <final>
<address> = at(String,int)|at(URL)|at(URI)
<final> = build()
<timeout> = withTimeout(int,TimeUnit)
<property> = with(Property) | with(String, Object)

Thus Propertys may be optionally specified at the beginning of sentences, optionally followed by one endpoint address and/or a timeout. If an address is specified, the builder creates a ProxyDelegate that operates in direct mode, otherwise it creates a ProxyDelegate that operates in discovery mode.

As discussed above, Propertys are used to capture library-specific configuration. Client libraries may expose them as constants, return them from factory methods, and even provide their own DSL extensions to create complex Propertys that clients can then inject in the core DSL defined above. Default Propertys may also be passed to builders by client libraries, at builder creation time. In what follows, for example, the library defines factory methods for two properties and sets a default on the first. Clients may set both in their configuration sentences:

public class FooProxies {
 
  private static final FooPlugin plugin = new FooPlugin();
 
  public static final String P1 = “...”   public static final String P2 = “...”  private static final Property P1(boolean val) = new Property(p1,val);  private static final Property P2(String val) = new Property(p2,val);    private static final Property[] defaults = new Property[] {p1(false}; 
  public static StatelessBuilder<Foo> foo() {
    return new StatelessBuilderImpl<FooPortType,Foo>(plugin,defaults);  }
}
 
...
 
Foo foo = foo().with(p1(false)).with(p2(“...”).build();

Similar considerations can be repeated for StatefulBuilders, which support a DSL with the following grammar:

<clause> = <first>.<final> | <final> 
<first> = <address>.<second> | <query>.<second> | <second>
<second> = <timeout>.<final> | <property>.<second> | <final>
<final> = build()
<query> = matching(StatefulQuery)
<address> = at(String,String,int)| at(String,URL) | at(String,URI) | 
at(W3CEndpointReferenceType)
<timeout> = withTimeout(int,TimeUnit)
<property> = with(Property) | with(String, Object)

Besides the different options to build instance addresses, here the choice between address (hence direct mode) or queries (hence discovery mode) is clearer. Arbitrarily many Propertys may specified after this choice. The basic implementation of StatefulBuilders is StatefulBuilderImpl.

Plugins

Plugins expose the information and services that allow framework components to act on behalf of specific client libraries. Delegates, builders, and queries all solicit information, often in overlapping subsets, from Plugins.

Plugin is an interface defined in common-gcore-clients and common-fw-clients as an extension of a more generic ProxyPlugin interface defined in common-clients. The generic interface is defined as follows:

package org.gcube.common.clients.delegates;
import ...
 
public interface ProxyPlugin<A,S,P> {
  String name();
  String namespace();
  Exception convert(Exception fault, ProxyConfig<?,?> config);
  S resolve(A address, ProxyConfig<?,?> config) throws Exception;
  P newProxy(ProxyDelegate<S> delegate);
}

while its extensions in common-gcore-clients and common-fw-clients are defined as follows:

package org.gcube.common.clients.gcore.plugins;
import ...;
 
public interface Plugin<S,P> extends 
ProxyPlugin<EndpointReferenceType,S,P> {
 String serviceClass();
 String serviceName();
}
 
 
package org.gcube.common.clients.fw.plugins;
import ...;
 
public interface Plugin<S,P> extends 
ProxyPlugin<EndpointReference,S,P> {
 String serviceClass();
 String serviceName();
}

Note that the only difference between the two extensions of ProxyPlugin is in the type with which endpoint address are modelled in the corresponding stacks, as proprietary EndpointReferenceTypes in the gCore stack and as Java-standard EndpointReference in the Featherweight Stack.

The client library for foo implements directly Plugin. With the callbacks name(), serviceName(), serviceClass(), and namespace() the library describes foo to the framework. The Implementations are straightforward once it is noted that gCube services can encompass multiple port-types, and that port-types correspond to services in the terminology adopted by the design model for client libraries. In particular, serviceName() and serviceClass() should return standard gCube coordinates for foo, while name() should return the name of the port-type that implements foo. Equally, namespace() should returns the namespace declared in the WSDL of that port-type.

The resolve() callback returns instances of foo stubs configured with the address parameter. This is where gCore client libraries and featherweight libraries differ most explicitly. The former implement resolve() as follows:

public FooPortType resolve(EndpointReferenceType address, ProxyConfig<?,?> config) throws Exception {
  return new FooServiceAddressingLocator().getFooPortTypePort(address);
}

the second instead as follows:

final GCoreService<FooPortType> stateful = ...

public FooPortType resolve(EndpointReference reference, ProxyConfig<?,?> config) throws Exception {
  return StubFactory.stubFor(Constants.stateful).at(reference);
}

where GCoreService and StubFactory are elements provided by common-gcore-stubs, a component of the Featherweight Stack with which assume familiarity here.

Note that the ProxyConfig parameter may be used by the library to translate configuration parameters into stub parameters, though this will be rarely needed. Timeout is one such property, but it is set transparently by the framework. Similarly, the library should not set call scopes on stub instances, as again the framework takes care of it.

The newProxy() callback returns Foo instances configured with ProxyDelegates, and is easily implemented with the approach to proxy configuration discussed above:

public Foo newProxy(ProxyDelegate<FooPortType> delegate) {
 return new DefaultFoo(delegate);
}

The implementation of the convert() callback can be slightly less immediate, and we leave it for a later discussion.

Queries

Queries are used by ProxyDelegates that operate in discovery mode to retrieve addresses of service endpoints to which Calls should be directed. A query returns multiple results if the service is replicated within the system, i.e. if there are multiple endpoints that can process the Calls. As described in the design model, this multiplicity is key to the fault-tolerant strategy of ProxyDelegates that operate in discovery mode.

Stateless and Stateful Queries

The generic interface of queries is defined in common-clients as follows:

package org.gcube.common.clients.queries;
import ...;
 
public interface Query<A> {
 
        List<A> fire() throws DiscoveryException;
 
	@Override 
        public boolean equals(Object query);
 
        @Override
  	public int hashCode();
 
        @Override 
        public String toString();
}

The type parameter described the addresses of service endpoints, which are the results expected from executing the query. For gCore client libraries, the parameter is instantiated to EndpointReferenceType by the implementations of the interface which are provided in common-gcore-clients. For featherweight libraries, the parameter is instantiated to EndpointReference by the implementations of the interface which are provided in common-fw-clients

In line with the design model for client libraries, the interface emphasises that queries are expected to be value objects, i.e. implement hashcode() and equals() towards state equivalence. This is because the framework uses queries as keys into caches of service endpoint addresses, hence their equivalence is important to the correct use of the caches. The interface also emphasises that queries should have a textual description, as the framework will invoke toString() to log them on behalf of client libraries.

The key method in the interface is fire(), which returns the results of the query. Notice that the framework is not concerned with how queries are executed, only in their results. Encapsulating the details of query execution makes the framework resilient to the evolution of gCube discovery services.

On the other hand, client libraries are unconcerned with the Query interface, which is entirely framework-facing. Rather, libraries work with specific implementations of the interface provided by common-gcore-clients and common-fw-clients: StatelessQuerys for stateless services and StatefulQuerys for stateful services. Both classes are instantiated with a Plugin, where queries find enough information to identify endpoints of the target service, e.g.:

FooPlugin plugin = new FooPlugin():
StatefulQuery query = new StatefulQuery(plugin);

Client libraries may further customise the queries to restrict the target service endpoints. However, the APIs offered by StatelessQuerys and StatefulQuerys changes across common-gcore-clients and common-fw-clients.

Customising Queries in gCore Client Libraries

In common-gcore-clients, StatelessQuery and StatefulQuery support query customisation through the notions of conditions and result matchers, which they inherit from AbstractQuery, a partial implementation of the Query interface included in common-clients.

Conditions are simple pairs of Strings which can be added to queries to further characterise the target service endpoints (cf. AbstractQuery.addProperty(String,String)) The first String identifies an abstract property of service endpoints, and the second String specifies a value for this property. The framework does not mandate the syntax of conditions, i.e. push towards client libraries any dependency on the low-level capabilities of the discovery mechanisms used within the system. At the time of writing, the system requires that properties are XPath expressions into XML descriptions of service endpoints that are published within the system, e.g:

StatefulQuery query = ...
query.addCondition(“//Name”,”...”);

The framework, on the other hand, considers the state of queries to be entirely defined by their conditions, and implements the requirement for query equivalence accordingly. This means that client libraries need only to create queries and add conditions to them, without worrying about internal requirement for state equivalence of queries.

Besides adding conditions, client libraries can also configure ResultMatchers on queries (cf. AbstractQuery.setMatcher(ResultMatcher)). ResultMatchers are used to filter out query results before these are used by the framework. Client libraries can use them whenever conditions alone are not sufficient to characterise the service endpoints. ResultMatcher is an interface defined in common-clients as follows:

 
package org.gcube.common.clients.queries;
 
public interface ResultMatcher<R> {
 boolean match(R result);
}

The type parameter describes the service endpoints and is instantiated to GCUBERunningInstance for StatelessQuerys and to RPDocument for StatefulQuerys. The match() method is invoked by the framework and ResultMatchers can inspect the service endpoints and return false whenever they do not match the required conditions, e.g:

StatefulQuery query = ...
 
ResultMatcher<RPDocument> matcher = new ResultMatcher<RPDocument>() {
  @Override
  boolean match(RPDocument result) {     boolean match = ...result...;     return match;  }};
 query.setMatcher(matcher);


Customising Queries in Featherweight Libraries

In common-fw-clients, the API of StatelessQuery and StatefulQuery offers a view over the API of SimpleQuerys in |ic-client, i.e the library of the Featherweight Stack that for query submission to the Information Collector service.

In particular, StatelessQuery and StatefulQuery let clients:

• specify free-form conditions (cf. addCondition(String))
• declare auxiliary query variables (cf. addVariable(String,String))
• add namespaces to use in conditions and variable declarations (cf. addNamespace(String,String)).

Consult the documentation of ic-client for details on these facilities.

Given the added flexibility in customising queries through generic conditions, namespaces and auxiliary variables, StatelessQuery and StatefulQuery in common-fw-clients do not need to provide support for post-query filtering, such as through the ResultMatchers available to gCore Client libraries.

Queries and Builders

Typically, StatelessQuerys and StatefulQuerys are treated different by client libraries. Since the endpoints of stateless are largely indistinguishable, libraries will rarely need to add conditions or set matchers on StatelessQuerys. StatelessBuilders will thus generate them on behalf of libraries, by default. If needed, however, client libraries can provide their own StatelessQuerys to builders, e.g.:

StatefulQuery query = ...
...
StatelessBuilder<Foo> builder=	new StatelessBuilderImpl<FooPortType,Foo>(plugin,query);

In contrast, clients will need to distinguish the instances of stateful services based on some characterise property of their state. The client libraries will then need to generate custom StatefulQuerys and make them available to clients for embedding in the DSL of StatefulBuilders, typically through static methods that clients can statically import. For example:

public static StatefulQuery name(String name) {
 StatefulQuery query = new StatefulQuery(plugin);
 ...customise query...
 return query;
}
 
...
 
Foo proxy = foo().matching(name(“..”)).build();

Failures

The design model specifies how client libraries ought to represent interaction failures, and how their proxies should present them to clients. In particular, the model:

• separates between errors, outages and contingencies, and requires that client libraries represent and reports errors and outages as unchecked ServiceExceptions and contingencies as checked Exceptions.

• identifies three common outages, DiscoveryException, NoSuchEndpointException, and IllegalScopeException, and requires that they are modelled as subclasses of ServiceExceptions;

In addition, the model requires that client libraries:

• attempt to recognise which outages and contingencies have an ‘‘unrecoverable'’ semantics for proxies that operate in discovery mode, i.e. induce the proxies to avoid binding to other service endpoints that their queries may have discovered;

• whenever appropriate, present clients with their checked Exceptions for contingencies, rather than exposing them to the Exceptions with which the same contingencies are modelled by the underlying stack;

If, for example, foo raises a FooFault contingency in its bar operation, the client library for foo is required to document and/or report the following exception in the interface of its proxies.:

/**
*  ...
*
* @throws ServiceException ...*/
public interface Foo {
 ...
 String bar() throws FooException;
 ...
}

where FooException is a local mapping of the FooFault thrown by the implementation stack used by the library.

Client libraries find in the org.gcube.common.clients.exceptions package of common-clients the ServiceExceptions classes identified by the model. They also find in the framework support towards meeting the implementation requirements raised by the model.

To begin with, in common-gcore-clients and common-fw-clients, the framework recognises the standard outages identified by the model from their counterparts in gCore stack. When ProxyDelegates encounter these faults they automatically convert them in the corresponding ServiceExceptions and propagate them to proxies if they are known to be unrecoverable (e.g. DiscoveryExceptions) or if the prove to be in practice.

The framework also allows allows client libraries to concentrate in their Plugins the mapping between the Exceptions that model contingencies locally to the libraries and their counterparts in the gCore stack. As we have anticipated above, libraries can implement the convert() callback of Plugins for this purpose, e.g.:

public Exception convert(Exception failure) {
  if (failure instanceof FooFault)
    return new FooException();
  return failure;
}

ProxyDelegates will make the callback and report the converted Exceptions to proxies. This is useful for two reasons. The first is that contingencies that are thrown from multiple methods do not need to be converted within each and every methods. Rather, the fault conversion strategy can be conveniently centralised in a singe place.

Note: Featherweight libraries that manually generate their stubs can avoid exception translations altogether, i.e. define the same exceptions on stub and proxy APIs. In this case, convert() can simply be implemented by returning the same exception received in input.

The second reason relates to a further form of support that the framework provides for fault handling. The framework defines an @Unrecoverable annotation in common-clients that client libraries can use on their Exception classes to mark them as unrecoverable contingencies, e.g.:

@Unrecoverablepublic class FooException {...}

ProxyDelegates recognise Unrecoverable annotations and act accordingly upon them when working in discovery mode. Thus converting contingencies in the convert() callback of Plugins, rather than within proxy methods, allows client libraries to easily and effectively cooperate with ProxyDelegates.

Collectively, the facilities provided by the framework shape the fault handling strategy of client libraries as follows:

• define Exceptions for service contingencies, marking them with Unrecoverable where appropriate;

• avoid handling failures in Call implementations;

• iimplement Plugin.convert() to perform contingencies conversion;

• catch and re-throw failures within proxy methods as follows:

public class DefaultFoo implements Foo {
 
  private ProxyDelegate<FooPortType> delegate;
 
  ...
 
  @Override
  public String bar() throws FooException {
 
   Call<FooPortType,String> barCall = new Call<FooPortType,String>() {
       @Override
       public String call(FooPortType endpoint) {
        return endpoint.bar();
       }
   };
 
   try {
 
     return delegate.make(barCall);
 
   }   catch(FooException e) {      throw e;   }   catch(ServiceException e) {      throw e;   }   catch(Exception e) {      throw new ServiceException(e);   }
 
   ...
}

Converted contingencies and ServiceExceptions generated by the framework for standard outages need to be explicitly caught and re-thrown. The generic Exception thrown by ProxyDelegate.make() needs also to be caught and wrapped in a ServiceException before being thrown to clients. Notice that it is not possible to perform this catch-all conversions in Plugin.convert(), as the compiler demands anyway that proxies deal explicitly with the generic Exception declared in ProxyDelegate.make().

This fault handling pattern is admittedly verbose, and in fact conceptually unnecessary as it serves only to satisfy the compiler. We can simplify it significantly with the static methods of the FaultDSL class, which implement a simple DSL of rethrow sentences. The following example illustrates usage:

import static org.gcube.common.clients.exceptions.FaultDSL.*; 
public class DefaultFoo implements Foo {
 
  private ProxyDelegate<FooPortType> delegate;
 
  ...
 
  @Override
  public String bar() throws FooException {
 
   Call<FooPortType,String> barCall = new Call<FooPortType,String>() {
       @Override
       public String call(FooPortType endpoint) {
        return endpoint.bar();
       }
   };
 
   try {
 
     return delegate.make(barCall);
 
   }
   catch(Exception e) {
      throw again(e).as(FooException.class);   }
 
   ...
}

Here we catch only the generic Exception and use the DSL to rethrow it based on its runtime type. If it is a FooException or a ServiceException we rethrow it untouched, effectively using the DSL to narrow the generic Exception we caught. If it has any different type we rethrow it wrapped in a ServiceException. Thus we achieve exactly the same effect of thee scatch clauses using a single one.

This facility generalises to multiple contingencies, e.g.:

   ...
 
 @Override
  public String bar() throws FooException, AnotherFooException, YetAnotherFooException { 
    ...
 
    catch(Exception e) {
      throw again(e).as(FooException.class, AnotherFooException.class, YetAnotherFooException.class);    }
 
   ...
}

The DSL is most useful when proxies methods throw many contingencies, but it reduces boilerplate code even when bar throws no contingency, e.g.:

   ...
 
 @Override
  public String bar() {
 
    ...
 
    catch(Exception e) {
      throw again(e).asServiceException();    }
 
   ...
}

Additional Support

Asynchronous Methods

The model recognises that services may deliver the outcomes of remote operations asynchronously (service-side asynchrony), or that client libraries may do so even when services do not (client-side asynchrony). It then gives design directives for proxy interfaces that allow clients to poll or be notified of call outcomes. Polling occurs through standard Java Futures and notifications are fed to client-provided Callbacks.

At the time of writing, service-side asynchrony is not common within the system, partly because the dominant gCore stack does not define any standard mechanisms for it. Without a standard, the framework can offer no help towards implementing the model's directives. On the other hand, the framework makes it easy to implement client-side asynchrony through its AsyncProxyDelegates.

AsyncProxyDelegates are ProxyDelegates with an additional suite of makeAsync()s methods:

• makeAsync(Call) executes a normal Call and returns a Future of the value it will produce. Libraries can directly return this Future to clients, perhaps as a wildcard Future if the operation is conceptually fire-and-forget;
• makeAsync(Call,Callback) takes a Call and a Callback, executes the Call and delivers its outcomes to the Callback;

In alignment with the model, both operations throw RejectedExecutionExceptions. They in fact make direct use of an ExecutorService provided by the framework, though the operations are overloaded to accept external ExecutorServices if required.

AsyncProxyDelegates add these facilities to existing ProxyDelegates, which they wrap and delegate to for synchronous Call execution (e.g. make(Call)).

package org.gcube.common.clients.delegates;
import ...
 
public class AsyncProxyDelegate<S> {
 
   ....
 
   private final ProxyDelegate<S> inner;
 
   public AsyncProxyDelegate(ProxyDelegate<S> delegate) {       this.inner=delegate;    }    ....
}

Accordingly, proxies that expose asynchronous methods simply replace the ProxyDelegates with which they are created with AsyncProxyDelegates, e.g. as follows:

public class DefaultFoo implements Foo {
 
  private final AsyncProxyDelegate<FooPortType> delegate; 
  public DefaultFoo(ProxyDelegate<FooPortType> delegate) {    this.delegate=new AsyncProxyDelegate<FooPortType>(delegate);  }}

They can then use the delegates as usual, e.g. for a method based on polling:

public class DefaultFoo implements Foo {
 
  private final AsyncProxyDelegate<FooPortType> delegate;
 
  ...
 
  @Override
  public Future<String> barAsync() {    Call<FooPortType,String> barCall = new Call<FooPortType,String>() {       @Override       public String call(FooPortType endpoint) {        return endpoint.bar();       }   };    return delegate.makeAsync(barCall); }

Methods based on callbacks follow the same pattern:

public class DefaultFoo implements Foo {
 
  private final AsyncProxyDelegate<FooPortType> delegate;
 
  ...
 
  @Override
  public Future<?> barAsync(Callback callback) {    Call<FooPortType,String> barCall = new Call<FooPortType,String>() {       @Override       public String call(FooPortType endpoint) {        return endpoint.bar();       }   };    return delegate.makeAsync(barCall,callback);  }}

Given these facilities, supporting both delivery mechanisms is just matter of sharing Calls objects across method implementations, e.g.:

public class DefaultFoo implements Foo {
 
  private final AsyncProxyDelegate<FooPortType> delegate;
 
  ...
 
  @Override
  public Future<String> barAsync() {
 
   return delegate.makeAsync(barCall()); 
  }
 
  @Override
  public Future<?> barAsync(Callback callback) {
 
   return delegate.makeAsync(barCall(),callback); 
  }
 
  //shared
  private Call<FooPortType,String> barCall() {     return new Call<FooPortType,String>() {       @Override       public String call(FooPortType endpoint) {        return endpoint.bar();       }     };  } 
}

The last requirement on libraries is to separate synchronous and asynchronous operations in different interfaces. The models explains how this separation avoids mis-interactions between the timeouts clients provide in Future.get() and Callback.timeout(), and the call timeouts set globally on the proxies by builders. In particular, the model requires a Foo interface with synchronous methods, and an asynchronous interface with asynchronous methods, e.g.:

public interface Foo  {  ...
 
  /**
  *
  * ....
  * 
  * @throws FooException if ...
  * @throws ServiceException if ...
  */
  public String bar() throws FooException; 
  ...
 
} 
 
public interface FooAsync  {  ...
 
  /**
  *
  * ....
  * 
  * @throws RejectedExecutionException if ...
  */
  public Future<String> barAsync(); 
  /**
  *
  * ....
  * 
  * @throws RejectedExecutionException if ...
  */
  public Future<?> barAsync(Callback callback);  ...
 
}

Proxies created under the Foo interface have a standard default timeout, either defined by the framework or overridden by the library. Proxies created under the FooAsync interface have an infinite timeout by default, which clients will override for individual calls,either in Future.get() or in Callback.timeout().

The two interfaces above do not necessarily require two different implementations. Rather, a single implementation can implement them both and share Call definitions (among other code) across both synchronous and asynchronous methods:

public class DefaultFoo implements Foo, FooAsync  {  ...
 
}

The separation can instead be reflected in the factory methods that present builders to clients, as follows:

public class FooProxies { 
  private static final FooPlugin plugin = new FooPlugin();
   public static StatelessBuilder<? extends Foo> foo() {    return new StatelessBuilderImpl<FooPortType,DefaultFoo>(plugin);  }
   public static StatelessBuilder<? extends FooAsync> asyncFoo() {   	Property infinite_timeout = Property.timeout(0);	return new StatelessBuilderImpl<FooPortType,DefaultFoo>(plugin,infinite_timeout);  }
}

Here foo() method returns a StatelessBuilder that produces proxy instances under the Foo interface while asyncFoo() returns a StatelessBuilder that produces proxy instances under the FooAsync interface and with a default infinite timeout. Clients can use these methods as follows:

Foo proxy = foo().....build();
...
FooAsync asyncProxy = asyncFoo().....build();

Note that the typing used above for foo() and fooAsync() requires a change to the definition of FooPlugin we've considered previously. Now that there are two interfaces for the same proxies, FooPlugin must be typed with respect the single implementation rather either interface, i.e.:

public class FooPlugin implements Plugin<FooPortType, DefaultFoo> { 
  ...
 
  public DefaultFoo newProxy(ProxyDelegate<FooPortType> delegate) {   return new DefaultFoo(delegate);  }}

Client Libraries for Third-party services

The CL framework can play an important role also in the integration of Third party services which have not been implemented using the gCube framework. Falls into this category mainly