Revision as of 12:25, 26 November 2012

The ic-client is a client library for the Information Collector service. It helps clients formulating queries for gCube resource descriptions, submitting them to the service, and processing their results.

Similar facilities for resource discovery are traditionally provided by the gCube Application Framework (gCF) and the IS Client library. The ic-client improves over the latter in a number of ways, most noticeably:

• it is completely independent from the gCore stack.

the ic-client can be easily embedded in a variety of client runtimes without out-of-band installation or configuration requirements. Clients may be external to gCube, or they may be 2nd-generation gCube services developed and running on stacks other than gCore stack. In this sense, the ic-client is a key part of the Featherweight Stack for gCube clients.

• it helps formulating a wider range of queries based only on knowledge of resource schemas.

simple queries can be configured with custom namespaces and custom result expressions. As a result, clients can retrieve only parts of resource descriptions or arbitrary combinations of parts. Fine-grained results are more easily processed and improve the performance of both clients and service.

• it offers increased flexibility in how query results are processed.

clients can configure how results ought to be parsed, or else take direct responsibility for parsing them. For example, clients may configure their own JAXB object bindings, while preprepared bindings for whole resource descriptions or specific properties thereof are readily available.

The ic-client is available in our Maven repositories with the following coordinates:

<groupId>org.gcube.resources.discovery</groupId>
<artifactId>ic-client</artifactId>
<version>...</version>

The library depends on a small set of components of the Featherweight Stack. Among these, the following are visible to library clients:

• common-gcore-resources: the object-based implementation of the gCube resource model.

clients may and normally will use the classes in common-gcore-resources to parse and process query results.

• discovery-client: a layer of interfaces and abstract implementations for queries and query submission API.

ic- client customises this layer for queries to the Information Collector.

Note: in what follows, we blur the distinction between the ic- client and the discovery-client. The distinction reflects modular choices for the design of the library but is otherwise of little consequence for its clients. The visibility of the discovery-client is limited only to the package of certain components that we discuss below, which starts with org.gcube.resource.discovery.client. The components of the ic-client are instead in packages that start with org.gcube.discovery.icclient.

Quick Tour

We introduce the API of the ic-client through a set of examples.

SubmittingPredefined Queries

Let us submit a query for, say, service endpoints:

import static org.gcube.resources.discovery.icclient.ICFactory.*;
 
...
 
Query query = queryFor(ServiceEndpoint.class);
 
DiscoveryClient<ServiceEndpoint> client = clientFor(ServiceEndpoint.class);
 
List<ServiceEndpoint> resources = client.submit(query);

queryFor() and clientFor() are static factory methods of the ICFactory class, which we import for improved legibility of the code (cf import static ...*).

queryFor() gives us a predefined Query for descriptions of service endpoints, which we lookup with the corresponding class in the object model of common-gcore-resources. Had we wanted to query the Information Collector for, say, hosting nodes then we would have used the HostingNode class of the object model.

clientFor() gives us a DiscoveryClient which can submit the query and parse its results as instances of the ServiceEndpoint class.

Finally, we ask the client to submit the query to the remote service and collect the parsed results. Once the results are back we can navigate through the ServiceEndpoint instances to get to the information we need. Again, we need to acquire familiarity with the resource model, nothing that some documentation and a good IDE cannot help with. In what follows, we assume such familiarity.

Thus interaction with the ic-client is two-phased:

• in the first phase we define the query we want, here simply pick a predefined one for a given resource type.
• in the second phase we submit the query with a DiscoveryClient that returns the results in the form we want.

Note that we use use the ServiceEndpoint class twice: a first time to lookup predefined queries and a second time to indicate how we want the results parsed. This appears redundant until we realise that the ic-client allows us to separate what we want to query from how we want the results to be returned. In this case, ServiceEndpoint identifies both query and parsing strategy, but this is not always the case, as we show next.

Conditions and Result Expressions

We now customise a predefined query for service endpoints so that it returns only the addresses of endpoints of database services. Since the results are now plain strings our choice of parser is to have none at all.

import static org.gcube.resources.discovery.icclient.ICFactory.*;
 
...
 
SimpleQuery query = queryFor(ServiceEndpoint.class);
 
query.addCondition("$resource/Profile/Category/text() eq 'Database'")
   .setResult("$resource/Profile/AccessPoint/Interface/Endpoint/text()");
 
DiscoveryClient<String> client = client();
 
List<String> addresses = client.submit(query);

We lookup the predefined query as we did earlier, but this time we type it under a more specific interface, SimpleQuery, which enables customisations. We first add an XQuery condition, which is the query language of the Information Collector. The condition filters out endpoints that do not give access to databases.

We then customise the result expression to get back only their addresses, chaining setResult() to addCondition. For both tasks, we rely on the schema of service endpoint descriptions to formulate expressions, and adhere to the (documented) convention to use $resource as a variable ranging over target resources.

Finally, we replace the method ICFactory#clientFor(Class) with ICFactory#client(), which returns a client that does not attempt to parse the results using classes from the resource model.

Note that if we needed to chain multiple conditions, we would invoke addCondition() multiple times. The chain would then be based on the AND operator. To chain conditions with other operators we would instead need to do the chaining ourself and add the result as a single condition.

Retrieving Resource Parts

In the next example, we move to middle ground: rather than looking for whole resource descriptions or individual strings we focus on selected parts of descriptions, e.g. we retrieve all the available access information. We thus return to needing result parsing, but this time pass the class of the resource model that describes the resource properties that we want retrieved, rather than the top-level class:

import static org.gcube.resources.discovery.icclient.ICFactory.*;
 
...
 
SimpleQuery query = queryFor(ServiceEndpoint.class);
 
query.addCondition("$resource/Profile/Category/text() eq 'Database'")
         .setResult("$resource/Profile/AccessPoint");
 
DiscoveryClient<AccessPoint> client = client(AccessPoint.class);
 
List<AccessPoint> accesspoints = client.submit(query);
 
for (AccessPoint point : accesspoints) {
    ...point.name()....point.address()....
}

It's now clear that the resource model classes that we pass to the DiscoveryClient drive a generic parser embedded in the client. The classes of the model are in fact decorated with JAXB annotations, and these annotations define the binding of classes from XML.

Custom Results

In our next example, we show how different parts of service endpoint descriptions, say access points and identifiers, can be composed together to form the desired results. We then show how these ad-hoc combination can be conveniently parsed:

import static org.gcube.resources.discovery.icclient.ICFactory.*;
 
...
 
SimpleQuery query = queryFor(ServiceEndpoint.class);
 
query.addCondition("$resource/Profile/Category/text() eq 'Database'")
         .setResult("<perfect>" +
                            "<id>{$resource/ID/text()}</id>" +
                            "{$resource/Profile/AccessPoint}" +
                            "</perfect>");
 
DiscoveryClient<PerfectResult> client = clientFor(PerfectResult.class);
 
List<PerfectResult> results = client.submit(query);
 
for (PerfectResult result : results) {
	...result.id...result.ap);
}

where PerfectResult is the simple bean defined as:

@XmlRootElement(name="perfect")
class PerfectResult {
 
   @XmlElement(name="id")
   String id;
 
    @XmlElementRef
    AccessPoint ap;
}

Here we give results a custom shape, hence we cannot cherry-pick from the resource model the class to be used for parsing. Rather, we define it as our PerfectResult, a very simple bean which we decorate with annotations to drive the underlying JAXB parser. Note that the bean reuses AccessPoint from the resource model, we do not need to reinvent the wheel. We then pass the bean to the discovery client and collect results as its instances.

Namespaces

Oftentimes, we need to specify conditions and result expressions that refer to resource properties with qualified names. We can then add namespace prefix declarations to the query and use those prefixes in conditions and result expressions. The following example illustrates the case for a resource property of an instance of the tree-manager service:

import static org.gcube.resources.discovery.icclient.ICFactory.*;
 
...
 
SimpleQuery query = queryFor(ServiceInstance.class);
 
query.addNamespace("tm",URI.create("http://gcube-system.org/namespaces/data/tm"))
	 .addCondition("$resource/Data/tm:Plugin/name/text() eq 'species-tree-plugin'");
 
DiscoveryClient<ServiceInstance> client = clientFor(ServiceInstance.class);
 
List<ServiceInstance> resources = client.submit(query);

Note that predefined queries for service instances already include a namespace declaration for the instance properties common to all such instances (e.g. ServiceClass or ServiceName). In particular, their namespace is automatically bound to the prefix gcube. Accordingly, a query for all service instances of the tree-manager could be defined as follows:

import static org.gcube.resources.discovery.icclient.ICFactory.*;
 
...
 
SimpleQuery query = queryFor(ServiceInstance.class);
 
query.addCondition("$resource/Data/gcube:ServiceClass/text() eq 'DataAccess'")
        .addCondition("$resource/Data/gcube:ServiceName/text() eq 'tree-manager-service'");
 
DiscoveryClient<ServiceInstance> client = clientFor(ServiceInstance.class);
 
List<ServiceInstance> resources = client.submit(query);

Auxiliary Variables

In our final example, we show how to introduce variables in the query in addition to $resource. Typically this allows us to characterise complex resource properties that can repeat within resources. For example, we need an auxiliary variable to characterise the service endpoints that have a property with a given name and value:

import static org.gcube.resources.discovery.icclient.ICFactory.*;
 
...
 
SimpleQuery query = queryFor(ServiceEndpoint.class);
 
query.addVariable("$prop", "$resource/Profile/AccessPoint/Properties/Property")
       .addCondition("$prop/Name/text() eq 'dbname'")
       .addCondition("$prop/Value/text() eq 'timeseries'");
 
DiscoveryClient<ServiceEndpoint> client = clientFor(ServiceEndpoint.class);
 
List<ServiceEndpoint> resources = client.submit(query);

Here we declare the auxiliary variable $prop to range over the resources Propertys, and then use it to characterise on such property. Had we not declared the variables, we could have retrieved undesired results, i.e. resource descriptions that have two properties, one with the required name and another with the required value.

Queries

In the examples above, we have introduced two interfaces for queries, Query and SimpleQuery. We discuss the interfaces and their implementations below, starting from the most generic of the two.

Freeform Queries

Query defines a read-only API, exposing only the textual expression of a query.

public interface Query {
 String expression();
}

The interface is thus only suitable to clients that consume queries. The DiscoveryClient which submits queries to the Information Collector is one such client. More generally, this is a good type to move queries from the place of building to the place of consumption.

The simplest Query implementation is no more than a box for the textual expression of the query:

Query q = new QueryBox("….a query…");

We resort to boxing a query whenever we cannot build it with higher-level facilities, such as those discussed below. Normally, this happens when the query is too complex to be conveniently composed in object-oriented terms. Then we can always write it as a string, accepting full exposure to the details of the target database. For the ic-client this means to know the details of the database in which the Information Collector stores resource descriptions.

Templated Queries

QueryTemplate is the first implementation of Query which introduces the notion of building a query. The idea is to derive a query from a template, by filling named holes with equally named parameters. Who instantiates a QueryTemplate provides the template:

QueryTemplate template = new QueryTemplate("….template…");

and who uses the instance fills the holes, ignoring other details of the query:

template.addParameter("name","value");

Then, whenever a consumer invokes expression() on the QueryTemplate, the parameters are interpolated and the complete query expression is returned.

The approach makes sense whenever there is a natural separation between query creators and query builders. Creators define the template and 'see' the details of the remote database, builders work instead with a simpler view of the query, typically one that hides the database. The main use case here is for library code built on top of the ic-client. The ic-client itself predefines all its queries for resource types as QueryTemplates.

If we are not writing library code, however, QueryTemplates are of little use and we can move on to the facilities discussed next. Otherwise, let us look at templates in more details.

Templates are strings with empty XML elements, optionally with a def attribute. Here's a template for an unlikely query language:

all results that satisfy <cond1/> or <cond2 def='that'/> </extra>

Whenever expression() is invoked, the empty elements in the template are replaced according to the first rule that applies among the following:

• by the value of an equally named parameter, if one exists
• by the value of the def attribute, if one exists
• by the empty string

For example, after adding the single parameter cond1="this" to the template above, expression() returns the string:

all results that satisfy this or that

QueryTemplates are intended for subclassing rather than directly for clients. Subclasses are expected to present a more typed interface to their clients, where the parameters are added behind dedicated setters, e.g.:

public class MyQuery extends QueryTemplate {
 
	public MyQuery() {
		super("..mytemplate");
	}
        ….
	setCond(String cond1) { …this.addParameter("cond1",cond1)…..}
	setCond2(String cond2) { …this.addParameter("cond2",cond2)…..}
	setExtra(String value) {…this.addParameter("extra",extra)…..}
        …...
}

We discuss below one such subclass and the API that it presents to clients.

Note that, for added flexibility, QueryTemplate defines a constructor that allows a subclass to provide an initial set of parameters:

QueryTemplate(String template, Map<String,String> parameters);

Thus a family of subclasses can share the same generic template, partially interpolate it at construction time, and offer many different specialisations to their clients. We can see this approach in action in the next section.

Simple Queries

As shown in our firsts examples, a SimpleQuery allows clients to customise a query in terms of namespaces, conditions, and result expressions, concepts that tend to recur across query languages:

public interface SimpleQuery extends Query {
 
 void addCondition(String condition);
 void addNamespace(String prefix, URI uri);
 void setResult(String expression);
 void clearConditions();
}

Naturally, the ic-client includes an implementation of SimpleQuery for the XQuery language, the language of the Information Collector. XQuery implements the SimpleQuery over a simple XQuery template, extending the QueryTemplate discussed above for the purpose:

public class XQuery extends QueryTemplate implements SimpleQuery {…}

The template declares a number of initial namesapaces for gCube resources, as well as a single variable, $result. The ic-client then predefines a number of XQuery instances where the template is specialised in such a way that $result ranges over different resource types. When we invoke the method queryFor() of the ICFactory we get back the predefined XQuery for the resource type that we indicate in input. We have shown above how the $result variable is used in specifying conditions and result expressions.

Recap

In conclusion, we have two main options for building queries:

• we can use predefined queries and customise them.
• we can box more complex queries inside QueryBoxes.

Furthermore, if we are writing libraries:

• we can define our own QueryTemplates and present them to our clients, typically under ad-hoc APIs like SimpleQuery's;

Query Submission