Difference between revisions of "Resource Model (2nd generation)"

Revision as of 16:57, 18 June 2012

Introduction

At its core, gCube provides a set of management functions over broad classes of computational resources. Functions include lifetime management, monitoring, sharing, reporting, deployment, security, and process execution. Resources include software components, the hardware on which those components run, and the data sources they produce and consume.

Each management function targets one or more resource classes. For example, deployment and process execution are concerned with software and hardware, security is concerned with software and data sources, and reporting applies to all resources. The implementation of the function is distributed across one or more system components, which are themselves primary examples of managed software resources. A single component may contribute to the implementation of multiple functions.

System components consume or produce information about resources in the target classes. gCube defines a descriptive model that regulates the representation and exchange of this information. Based on this resource model, it includes directory services in its Information System which allow components to publish and discover resource descriptions (cf. Figure 1). Resource publication and discovery are thus the primitive system functions which enable the implementation of all the other management functions.

Figure 1: A model governs the description of resource descriptions produced and consumed by components that implements management functions defined over classes of related resources. The components rely on directory services to publish and discover resource descriptions.

First Generation

At its inception, the system took a fairly static view over its management functions and equated them with their latest implementations. Resource classes were thus pre-defined and the description of resources within each class was centrally and fully prescribed. Quite simply, it collected the information required or produced by all the functions defined over that class.

Semantic variations within classes were modelled with hierarchies, insofar as the variations could be anticipated or added over time as new specialisations of existing subclasses. Classes and subclasses had their own XML bindings, where the latter added or specialised the elements defined by the former. Such definitions were governed by XML Schemas, and the schemas were used to validate the publication and discovery of resource descriptions. Validation was performed by discovery services as well as their clients. Classes, subclasses and their XML bindings were mirrored in an object-based implementation of the model, which was distributed along with the schemas in a single component deployed ubiquitously within the system.

Since resource descriptions were centrally defined and implemented, changing the model had an endemic impact within the system. Change was accommodated pragmatically, with optional extensions to the resource definitions and the associated schemas. Optionality was also used to encompass new class specialisations under existing definitions, whenever extending the existing hierarchy was conceptually or technologically problematic. Conceptually, problematic extensions marked lack of sufficient generality in the existing hierarchy, i.e. proved that the original viewpoint on resource classification was too narrow to accommodate the new requirements. Technically, problematic extensions marked limitations or excessive complexity in the use of XML Schemas constructs. Human error in the use of both modelling primitives and specification tools had its part in hampering or altogether preventing change. A distinguished class of “generic” resources was introduced to accommodate resource descriptions that could not be retrofitted within existing hierarchies using the techniques above.

The system evolved within this scheme, occasionally acquiring new management functions and, far more commonly, refining the implementation of existing functions. The catch-all class of generic resources grew beyond initial expectations.

Second Generation

With its new version of the resource model, gCube takes a different approach to resource description. The approach is motivated by two evolutionary forces in the development of the system:

• the system is evolving towards managing a wider and more varied set of computational resources;

While broad resource classes remain fairly stable, there is an increasing demand for accommodating greater variation within those classes. This is particularly the case for software resources, where management requirements span across multiple technologies. SOAP Web Services, Rest Web Services and conventional Web Applications, remotely accessible Database Management Systems, custom networked software, and even support libraries and service plugins are all examples of resources that fall within the management scope intended for the system.

This heterogeneity indicates that any attempt to capture resource semantics within extensible hierarchies is likely to renew the evolution challenges encountered so far, only on a larger scale. The success of any such attempt cannot be excluded in principle, but previous experience suggests that it is bound to fail in practice.

• the system is evolving towards a more modular and specification-driven offering of its management functions;

As more and more varied resources fall within its scope, the extent and the uniformity with which they can be managed varies accordingly. Not all functions will be available for all resources, and not all available functions will be delivered with a single implementation. As a case in point, not all services will be dynamically deployable and those that are will be packaged differently to address different service containers. Functions will then be described solely in terms of the APIs, and different implementations of the same functions will exist within the system, over time or even concurrently.

A modular and specification-driven offering of management functionality is inherently at odds with centralised and fully prescribed definitions of resource descriptions. Different implementations will require different information about resources in the same class, and the same information may be shared across resources of different classes.

These forces indicate that gCube needs a resource model that matches the openness and modularity required by its evolution. Accordingly, the new resource model foregoes hierarchical structures and caters for a more dynamic approach to resources descriptions. Equally, the new model avoids centralised definitions and allows for a more fine-grained and distributed approach to resource description. As a result, many changes that were non retro-compatible in the old model become extensions in the new model, and those that remain non-retro compatible are either isolated or contained in scope.

Facets

In the new model, function implementations are responsible for defining the resource properties that they require or produce, and for binding that information to XML.

The semantics of a resource is no longer found within class hierarchies, but it is carried implicitly by the properties that, at any point in time, implementation components have independently attached to the resource in order to manage it (cf. Figure 2). Each such piece of information is a resource facet and, at ay point in time, the semantics of a resource is the collection of its current facets. The system remains responsible for defining the framework in which arbitrary facets may be collated into completed resource descriptions, and this framework fully defines the new resource model.

Figure 2: Producers and consumers in the implementations of given management functions add and use only relevant facets in resource descriptions.

The grain and modularity associated with facets transfer to schemas and object implementations. Facet schemas remain relevant for specification and validation purposes, but their scope is now local within resource descriptions. Similarly, object bindings of facets remain available within the system, but they can be distributed across the system components that are responsible for their definition, rather than in a single, ubiquitously deployed components. The addition of new facets, their schemas, and their bindings does not propagate upgrades throughout the system. The non-retrocompatible change of existing facets limits the scope of upgrades to the subset of system components that are effectively impacted by it.

Interfaces

Consumers often raise overlapping requirements on the information that describes resources. Requirements may converge around resources of a given class, or specialisations thereof. In many cases, they may converge across resource classes and along functions. Some requirements may apply indiscriminately to all resources managed by the system.

It is undesirable to replicate information required by unrelated consumers within different, independently produced facets. Equally, it is undesirable to create unnecessary dependencies between unrelated publishers and consumers, i.e. expect the latter to access facets defined by the former. Converging modelling requirements are best addressed by sharing information within standard facets. These facets serve as information-oriented interfaces for resources, whereby the resources may be discovered and their models consumed for management purposes. The information that comprises interfaces is public within the system, while the rest remains private to a set of related publishers and consumers (cf. Figure 3).

The new resource model does not address standardisation issues beyond the definition of a small set of common properties shared by all resource descriptions. Rather, it leaves the standardisation of facets to multi-party agreements of varying scope. System-wide agreements define interfaces common to all resources of a given resource class (class interfaces). Function-wide agreements define interfaces common to all the implementations of a given function (function interfaces).

Collectively, the interfaces identify a modelling layer built on top of the resources model, the interface layer. The interface layer is concerned with concrete applications of the modelling primitives and is, by definition, in constant evolution. The interfaces that comprise it at any point in time are documented in a separate Interface Registry.

Figure 3: Producers and consumers in the implementations of different functions synchronise on interfaces of shared information.

Resource Properties

All resources managed by the system are described by the following properties:

• a unique identifier;

the identifier of a resource allows system components to unambiguously separate it from other resources and to resolve references to the resource included in the description of other resources. Accordingly, the model requires its uniqueness and immutability but does not otherwise mandate a concrete syntax for it. Implementations will ensure that resource identifiers can be embedded in URIs (through escaping of characters forbidden by RFC 2396) and will offer guarantees for their immutability and global uniqueness.

• a name;

the name of a resource serves solely for reporting purposes. The model does not mandate a syntax for it and allows it to change over time.

• a description;

the description of a resource serves solely for reporting purposes. The model does not mandate a syntax for it and allows it to change over time.

• one or more scopes;

the scopes of a resource limit access to the resource and visibility of its description to clients that operate in one of the scopes. The syntax of scopes is defined outside the model, as it is the responsibility for enforcing scopes. Scopes may be added to or removed from resources over time.

• one or more facets;

facets are uniquely named but the model does not constrain a-priori their structure and content. Constraints on particular facets are introduced in the interface layer of the model. Facets may be added to or removed from resources over time.

XML Binding

Resources serialize as XML documents rooted in a resource element with at least an id attribute for the resource identifier, name, description, and a list of one or more scope elements nest inside the document root in that order and before facet serializations. All have simple, non-empty content and the content of name can be qualified with an arbitrary namespace. All the elements and attributes are defined in the namespace http://gcube-system.org/namespaces/resource, except for facets which may be named in any namespace, as long as one such namespace exists. Facets may be simple or complex elements, though they cannot mix elements and text nor include XML structures other than elements.

Formally, XML serializations of resources must validate against the following XML Schema:

<schema xmlns="http://www.w3.org/2001/XMLSchema" targetNamespace="http://gcube-system.org/namespaces/resource"
 xmlns:tns="http://gcube-system.org/namespaces/resource" elementFormDefault="qualified">
 
<element name="resource">
 <complexType>
 
   <sequence>
      <element name="name" type="tns:nonemptyname" minOccurs="0" />
      <element name="description" type="tns:nonempty" minOccurs="0" />
      <element name="scope" type="tns:nonempty" maxOccurs="unbounded" />
      <any processContents="lax" namespace="##other"  maxOccurs="unbounded" />
   </sequence>
 
    <attribute name="id" type="tns:nonempty" />
    <anyAttribute processContents="lax" namespace="##other"  />
 
</complexType>
</element>
 
<simpleType name="nonempty">
 <restriction base="string">
   <minLength value="1" />
   <pattern value=".*[^\s].*" />
 </restriction>
</simpleType>
 
<simpleType name="nonemptyname">
 <restriction base="QName">
   <minLength value="1" />
   <pattern value=".*[^\s].*" />
 </restriction>
</simpleType>
</schema>

Note that the schema lacks the expressiveness to capture all the constraints defined by the model (e.g. unique naming enforcement for facets). Note also that a lax validation model for facets caters for modular validation, as long as facet schemas are defined in different namespaces and are registered programmatically with the schema validator.

Implementation Requirements

The resource model is fully defined by its primitives and does not describe their implementation in a programming language. It also does not address the design of local or remote APIs that allow producers and consumers to publish or discover resource descriptions (cf. Figure 4).

Figure 4: The resource model is implemented in an object model and the object model is a dependency of publication and discovery APIs that producers and consumers use to interact with remote directory services.

Equally, the model is purely descriptive and does not address lifetime issues. From the perspective of the model, the lifetime of resources begins when their descriptions are first published within the system and it terminates when those descriptions are removed from the system. An arbitrary number of independently developed and functionally unrelated components may be involved in the creation, manipulation, and retrieval of resource descriptions during their lifetime.

However, the model raises implicit requirements against those implementations and APIs, as well as against the architecture of the components that manage the lifetime of resource descriptions. Namely, the model requires an overall alignment with the modularity that characterizes its primitives.

Thus object models should be cast in terms of arbitrary facets, publication APIs will allow arbitrary modification of arbitrary facets, and discovery APIs will retrieve resources based on characterizations of arbitrary facets. The standard models defined in the interface layer, as well as the definitions of private facets, should be both introduced separately from those object models and APIs. Typically, they will take the form of object bindings included in the components that are responsible for the definition of the interfaces and private facets (cf. Figure 5).

Figure 5: Object model and publication/discovery APIs align with the facet layer, i.e. deal with arbitrary facets. Object bindings of interfaces and private facets are modularly introduced by separate components.

Components that manage the lifetime of resource descriptions should support an equally modular approach. A subset of producers, in particular, will be responsible for creating resource descriptions as these are first published in the system. Some of these resource builders will present interactive interfaces to administrative users, who will provide the actual descriptions. Others will operate non-interactively, synthesizing resource descriptions from configuration files. These latter builders are known as resource keepers. Keepers should not centralize the task of description building, but distribute it modularity across a set of co-deployed delegates, each of which is independently developed to create given facets of the descriptions. Keepers should be responsible for dynamically discovering the delegates which are available on the hosting node, for coordinating them through the task, and for publishing the resulting resource descriptions. Individual delegates may produce interfaces or private facets which are private to specific implementation of system functions (cf. Figure 6).

Figure 6: While some producers update resource descriptions, builders are producers that create resource descriptions. Some builders do so interactively, but keepers are builders that create resource configuration automatically based on configuration files. Keepers act modularly, delegating the building task to co-deployed and dynamically discovered components.

Glossary

→ builder: producer that creates and publishes resource description;

→ common property: property shared by all resources and defined independently from facets;

→ consumer: component that discovers and consumes resource descriptions

→ delegate: component invoked by a resource keeper to create given facets of resource descriptions

→ directory services: system services that store and give access to resource descriptions

→ discovery API: the local interface of discovery services whereby consumers can retrieve resource descriptions

→ facet: uniquely named subset of a resource description

→ function interface: interface that describes resources in in support of given management functions

→ interface: facet with a standard definition within the system

→ interface layer: set of interfaces documented in the Interface Registry

→ keeper: non-interactive builder that synthesizes resource descriptions from configuration files

→ object model: an implementation of the facet layer of the model in an object-oriented language

→ producer: component that creates/updates, and publishes resource descriptions

→ prototype interface: interface that describes the prototypical resource in a given class, independently from management functions;

→ publisher API: the local interface of discovery services whereby producers can publish resource descriptions

→ resource: computational resource described within the system for management purposes

→ resource description: information about managed resources available within the system;