Resource Model (2nd generation)
Introduction
At its core, gCube provides a set of management functions over broad classes of computational resources. Functions include lifetime management, monitoring, sharing, reporting, deployment, security, and process execution. Resources include software components, the hardware on which those components run, and the data sources they produce and consume.
Each management function targets one or more resource classes. For example, deployment and process execution are concerned with software and hardware, security is concerned with software and data sources, and reporting applies to all resources. The implementation of the function is distributed across one or more system components, which are themselves primary examples of managed software resources. A single component may contribute to the implementation of multiple functions.
System components consume or produce information about resources in the target classes. gCube defines a descriptive model that regulates the representation and exchange of this information. Based on this resource model, it includes directory services that allow components to publish and discover resource descriptions (cf. Figure 1).
First Generation
At its inception, the system took a fairly static view over its management functions and equated them with their latest implementations. Resource classes were thus pre-defined and the description of resources within each class was centrally and fully prescribed. Quite simply, it collected the information required or produced by all the functions defined over that class.
Semantic variations within classes were modelled with hierarchies, insofar as the variations could be anticipated or added over time as new specialisations of existing subclasses. Classes and subclasses had their own XML bindings, where the latter added or specialised the elements defined by the former. Such definitions were governed by XML Schemas, and the schemas were used to validate the publication and discovery of resource descriptions. Validation was performed by discovery services as well as their clients. Classes, subclasses and their XML bindings were mirrored in an object-based implementation of the model, which was distributed along with the XML Schemas in a single component deployed ubiquitously within the system.
Since resource descriptions were centrally defined and implemented, changing the model had an endemic impact within the system. Change was accommodated pragmatically, with optional extensions to the definitions and their schemas. Optionality was also used to encompass new class specialisations under existing definitions and schemas whenever extending the existing hierarchy was conceptually or technologically problematic. Conceptually, problematic extensions marked lack of sufficient generality in the classification embodied by the existing hierarchy, i.e. it showed that the original viewpoint on resource classification was too narrow to accommodate the new requirements. Technically, problematic extensions marked limitations or excessive complexity in the use of XML Schemas constructs. Human error in the use of both modelling primitives and specification tools had its part in hampering or altogether preventing change. A distinguished class of “generic” resources was introduced to accommodate resource descriptions that could not be retrofitted within existing hierarchies using the techniques above.
The system evolved within this scheme, occasionally acquiring new management functions and, far more commonly, refining the implementation of existing functions. The catch-all class of generic resources grew beyond initial expectations.
Second Generation
With its new version of the resource model, gCube takes a different approach to resource description. The approach is motivated by two evolutionary forces in the development of the system:
- the system is evolving towards managing a wider and more varied set of computational resources;
- While broad resource classes remain fairly stable, there is an increasing demand for accommodating greater variation within those classes. This is particularly the case for software resources, where management requirements span across multiple technologies. SOAP Web Services, Rest Web Services and conventional Web Applications, remotely accessible Database Management Systems, custom networked software, and even support libraries and service plugins are all examples of resources that fall within the management scope intended for the system.
- This heterogeneity indicates that any attempt to capture resource semantics within extensible hierarchies is likely to renew the evolution challenges encountered so far, only on a larger scale. The success of any such attempt cannot be excluded in principle, but our experience suggests that it is bound to fail in practice.
- the system is evolving towards a more modular and specification-driven offering of its management functions;
- As more and more varied resources fall within its scope, the extent and the uniformity with which they can be managed varies accordingly. Not all functions will be available for all resources, and not all available functions will be served by a single implementation. As a case in point, not all resources will be dynamically deployable and those which are will be packaged differently to address different containers. In particular, functions will be described solely in theirs of the APIs, and different implementations of the same functions may exist within the system, over time or even concurrently.
- A modular and specification-driven offering of management functionality is inherently at odds with centralised and fully prescribed definitions of resource descriptions. Different implementations will require different information about resources in the same class, and the same kind of information may be shared across resources of different classes.
These forces indicate that gCube needs a resource model that matches the openness and modularity required by its evolution. Accordingly, the new resource model foregoes hierarchical structures and caters for a more dynamic approach to resources descriptions. Equally, the new model avoids centralised definitions and allows for a more fine-grained and distributed approach to resource description. As a result, many changes that were non retro-compatibile in the old model become extensions in the new model, and those that remain non-retro compatible are either isolated or contained in scope.
Facets
In the new model, in particular, function implementations are responsible for defining the resource properties that they require or produce, and for binding that information to XML. The semantics of a resource is no longer found within class hierarchies, but it is carried implicitly by the information that, at any point in time, implementation components have independently attached to the resource in order to manage it (cf. Figure 2). Each such piece of information is a resource facet and, at ay point in time, the semantics of a resource is the collection of its current facets.
The system remains responsible for defining the framework in which arbitrary facets may be collated into completed resource descriptions. This framework constitutes the facet layer of the new resource model.
Interfaces
Consumers often raise overlapping requirements on the information that describes resources. Requirements may converge around resources of a given class, or specialisations thereof. In many cases, they may converge across resource classes and along functions. Some requirements may apply indiscriminately to all resources managed by the system.
It is undesirable to replicate information required by unrelated consumers within different, independently produced facets. Equally, it is undesirable to create unnecessary dependencies between unrelated publishers and consumers, i.e. expect the latter to access facets defined by the former. Converging modelling requirements are best addressed by sharing information within standard facets. These facets serve as information-oriented interfaces for resources, whereby the resources may be discovered and their models consumed for management purposes. The information that comprises interfaces is public within the system, while the rest remains private to a set of related publishers and consumers (cf. Figure 3).
The facet layer of the new resource model does not address standardisation issues beyond the definition of a small set of common properties shared by all resource descriptions. Rather, it leaves the standardisation of facets to multi-party agreements of varying scope. System-wide agreements define interfaces common to all resources of a given resource class (class interfaces). Function-wide agreements define interfaces common to all the implementations of a given function (function interfaces).
Collectively, the interfaces identify a higher and evolving layer of the model, the interface layer. The flexibility of the facet layer underneath allows resources to have multiple interfaces, as well as to acquire interfaces over time.
Resource Properties
All resources managed by the system are described by the following properties:
- a unique identifier;
- the identifier of a resource allows system components to unambiguously separate it from other resources and to resolve references to the resource included in the description of other resources. Accordingly, the model requires its uniqueness and immutability but does not otherwise mandate a concrete syntax for it. Implementations will ensure that resource identifiers can be embedded in URIs (through escaping of characters forbidden by RFC 2396) and will offer guarantees for their immutability and global uniqueness.
- an optional name;
- the name of a resource serves solely for reporting purposes. The model does not mandate a syntax for it and allows it to change over time.
- an optional description;
- the description of a resource serves solely for reporting purposes. The model does not mandate a syntax for it and allows it to change over time.
- one or more scopes;
- the scopes of a resource limit access to the resource and visibility of its description to clients that operate in one of the scopes. The syntax of scopes is defined outside the model, as it is the responsibility for enforcing scopes. Scopes may added to or removed from resources over time
- one or more facets;
- facets are uniquely named but the model does not constrain a-priori their structure and content. Constraints on particular facets are introduced in the interface layer of the model. Facets may be added to or removed from resources over time.
XML Binding
Resources serialize as XML documents rooted in a resource element with at least an id attribute for
the resource identifier, name, description, and a list of one or more scope elements nest inside the document root in that order and before facet serializations. All have simple, non-empty content
and the content of name can be qualified with an arbitrary namespace. All the elements and attributes are defined in the namespace http://gcube-system.org/namespaces/resource, except for facets which may be named in any namespace, as long as one such namespace exists.
Facets may be simple or complex elements, though they cannot mix elements and text nor include XML structures other than elements.
Formally, XML serializations of resources must validate against the following XML Schema:
<schema xmlns="http://www.w3.org/2001/XMLSchema" targetNamespace="http://gcube-system.org/namespaces/resource" xmlns:tns="http://gcube-system.org/namespaces/resource" elementFormDefault="qualified"> <element name="resource"> <complexType> <sequence> <element name="name" type="tns:nonemptyname" minOccurs="0" /> <element name="description" type="tns:nonempty" minOccurs="0" /> <element name="scope" type="tns:nonempty" maxOccurs="unbounded" /> <any processContents="lax" namespace="##other" maxOccurs="unbounded" /> </sequence> <attribute name="id" type="tns:nonempty" /> <anyAttribute processContents="lax" namespace="##other" /> </complexType> </element> <simpleType name="nonempty"> <restriction base="string"> <minLength value="1" /> <pattern value=".*[^\s].*" /> </restriction> </simpleType> <simpleType name="nonemptyname"> <restriction base="QName"> <minLength value="1" /> <pattern value=".*[^\s].*" /> </restriction> </simpleType> </schema>
Note that the schema lacks the expressiveness to capture all the constraints defined by the model (unique naming and content model enforcement for facets).
Interfaces
The model defines a number of interfaces that standardize properties common to broad classes of managed resources.
Interfaces may be classified according to their kind:
- class interfaces: describe the prototypical resource of a given class, independently of how it is managed within the system;
- function interfaces: describe resources in support of one or more management functions;
For convenience of reference, the definitions of prototype and function interfaces are grouped below per resource class. Function interfaces that describes resources in two more classes are grouped separately, as common interfaces.
Common Interfaces
<lifetime>
- kind:function
- name:
{http://gcube-system.org/namespaces/resource/common/lifetime}lifetime - role: describe the lifetime of the resource
The interface defines the following properties:
- @TODO
- @TODO
The interface is bound to the XML representation defined by the following XML Schema:
<schema targetNamespace="http://gcube-system.org/namespaces/resource/function/common/lifetime" elementFormDefault="qualified" xmlns="http://www.w3.org/2001/XMLSchema"> <element name="lifetime"> <complexType> <sequence> <element name="status" type="string"/> <element name="activationTime" type="dateTime"/> <element name="deactivationTime" type="dateTime" use="optional"/> <element name="lastUpdate" type="dateTime"/> <element name="nextUpdate" type="dateTime"/> </sequence> </complexType> </element> </schema>
A sample serialisation is the following:
<l:lifetime xmlns:l="http://gcube-system.org/namespaces/resource/function/common/lifetime" > <l:status>certified</l:status> <l:activationTime>2012-06-10T21:29:05+02:00</l:activationTime> <l:deactivationTime>2012-07-23T21:29:05+02:00</l:deactivationTime> <l:lastUpdate>2012-06-29T06:36:50+01:00</l:lastUpdate> <l:nextUpdate>2012-06-29T06:37:50+01:00</l:nextUpdate> </l:lifetime>
<certificate>
- kind:function
- name:
{http://gcube-system.org/namespaces/resource/common/certificate}certificate - role: report information on the credentials associated to the resource through an X.509 certificate
The interface defines the following properties:
- ca: the Certification Authority that issued the certificate
- dn: the Distinguished Name of the certificate
- expireOn: the expiration date and time of the certificate
The interface is bound to the XML representation defined by the following XML Schema:
<schema targetNamespace="http://gcube-system.org/namespaces/resource/function/common/certificate" elementFormDefault="qualified" xmlns="http://www.w3.org/2001/XMLSchema"> <element name="certificate"> <complexType> <sequence> <element name="ca" type="string"/> <element name="dn" type="string"/> <element name="expireOn" type="dateTime"/> </sequence> </complexType> </element> </schema>
A sample serialisation is the following:
<s:certificate xmlns:s="http://gcube-system.org/namespaces/resource/function/common/certificate"> <s:ca>/C=IT/O=INFN/CN=INFN CA</s:ca> <s:dn>/C=IT/O=INFN/OU=Host/L=NMIS-ISTI/CN=node2.tsec.d4science.research-infrastructures.eu</s:dn> <s:expireOn>2012-06-06T14:55:49+02:00</s:expireOn> </s:certificate>
<labels>
- kind:function
- name:
{http://gcube-system.org/namespaces/resource/common/label}labels - role: list of labels characterizing the resource
The interface defines the following properties:
- @TODO
The interface is bound to the XML representation defined by the following XML Schema:
<schema targetNamespace="http://gcube-system.org/namespaces/resource/function/common/label" elementFormDefault="qualified" xmlns="http://www.w3.org/2001/XMLSchema"> <element name="labels"> <complexType> <sequence> <element name="label" maxOccurs="unbounded"> <complexType> <simpleContent> <extension base="string"> <attribute name="name" type="string" use="required"/> </extension> </simpleContent> </complexType> </element> </sequence> </complexType> </element> </schema>
A sample serialisation is the following:
<l:labels xmlns:l="http://gcube-system.org/namespaces/resource/function/common/label"> <l:label name="java">...any...</l:label> <l:label name="gcore">...any...</l:label> <l:label name="ghn">...any...</l:label> </l:labels>
Hardware
<hardware>
- kind: class
- name:
{http://gcube-system.org/namespaces/resource/class}hardware - role: describe any hardware exploited in the infrastructure
The interface defines the following properties:
- @TODO
- @TODO
The interface is bound to the XML representation defined by the following XML Schema:
<schema targetNamespace="http://gcube-system.org/namespaces/resource/class" elementFormDefault="qualified" xmlns="http://www.w3.org/2001/XMLSchema"> <element name="hardware"> <complexType> <sequence> <element name="architecture"> <complexType> <simpleContent> <extension base="string"> <attribute name="platformType" type="string"/> <attribute name="SMPSize" type="string" use="optional"/> <attribute name="SMTSize" type="string" use="optional"/> </extension> </simpleContent> </complexType> </element> <element name="operatingSystem"> <complexType> <simpleContent> <extension base="string"> <attribute name="name" type="string"/> <attribute name="release" type="string" use="optional"/> <attribute name="version" type="string" use="optional"/> </extension> </simpleContent> </complexType> </element> <element name="processor" maxOccurs="unbounded"> <complexType> <simpleContent> <extension base="string"> <attribute name="vendor" type="string"/> <attribute name="model" type="string"/> <attribute name="family" type="string"/> <attribute name="modelName" type="string"/> <attribute name="ClockSpeedMhz" type="string"/> <attribute name="Bogomips" type="string"/> <attribute name="CacheL1" type="string"/> <attribute name="CacheL2" type="string"/> </extension> </simpleContent> </complexType> </element> </sequence> </complexType> </element> </schema>
A sample serialisation is the following:
<h:hardware xmlns:h="http://gcube-system.org/namespaces/resource/class"> <h:architecture platformType="amd64" SMPSize="0" SMTSize="0"/><!-- 1 --> <h:operatingSystem name="Linux" release="" version="2.6.32-5-xen-amd64"/> <h:processor vendor="GenuineIntel" model="44" modelName="Intel(R) Xeon(R) CPU E5630 @ 2.53GHz" family="6" ClockSpeedMhz="2533.468" Bogomips="5066.93" CacheL1="12288" CacheL2="0"/> <h:processor vendor="GenuineIntel" model="44" modelName="Intel(R) Xeon(R) CPU E5630 @ 2.53GHz" family="6" ClockSpeedMhz="2533.468" Bogomips="5066.93" CacheL1="12288" CacheL2="0"/> </h:hardware>
<coordinates>
- kind: function
- name:
{http://gcube-system.org/namespaces/resource/function/hardware/coordinate}coordinates - role: give information on the geographical location of the hardware
The interface defines the following properties:
- @TODO
- @TODO
The interface is bound to the XML representation defined by the following XML Schema:
<schema targetNamespace="http://gcube-system.org/namespaces/resource/function/hardware/coordinate" elementFormDefault="qualified" xmlns:co="http://gcube-system.org/namespaces/resource/function/hardware/coordinate" xmlns="http://www.w3.org/2001/XMLSchema"> <element name="coordinates"> <complexType> <sequence> <element name="location" type="string"/> <element name="country" type="co:countryCode"/> <element name="latitude" type="double"/> <element name="longitude" type="double"/> <element name="domain" type="string"/> </sequence> </complexType> </element> <simpleType name="countryCode"> <restriction base="string"> <length value="2"/> </restriction> </simpleType> </schema>
A sample serialisation is the following:
<c:coordinates xmlns:c="http://gcube-system.org/namespaces/resource/function/hardware/coordinate"> <c:location>Pisa</c:location> <c:country>it</c:country> <c:latitude>43.719627</c:latitude> <c:longitude>10.421626</c:longitude> <c:domain>research-infrastructures.eu</c:domain> </c:coordinates>
<runtime>
- kind: function
- name:
{http://gcube-system.org/namespaces/resource/function/hardware/runtime}runtime - role: report runtime information on the hardware status
The interface defines the following properties:
- @TODO
- @TODO
The interface is bound to the XML representation defined by the following XML Schema:
<schema targetNamespace="http://gcube-system.org/namespaces/resource/function/hardware/runtime" elementFormDefault="qualified" xmlns="http://www.w3.org/2001/XMLSchema"> <element name="runtime"> <complexType> <sequence> <element name="load"> <complexType> <simpleContent> <extension base="string"> <attribute name="last1Min" type="decimal" use="required"/> <attribute name="last5Min" type="decimal" use="required"/> <attribute name="last15Min" type="decimal" use="required"/> </extension> </simpleContent> </complexType> </element> <element name="mainMemory"> <complexType> <simpleContent> <extension base="string"> <attribute name="RAMSize" type="integer" use="required"/> <attribute name="RAMAvailable" type="integer" use="required"/> <attribute name="virtualSize" type="integer" use="required"/> <attribute name="virtualAvailable" type="integer" use="required"/> </extension> </simpleContent> </complexType> </element> <element name="localAvailableSpaceInBytes" type="nonNegativeInteger"/> <element name="uptimeInDays" type="integer"/> </sequence> </complexType> </element> </schema>
A sample serialisation is the following:
<r:runtime xmlns:r="http://gcube-system.org/namespaces/resource/function/hardware/runtime"> <r:load last1Min="0.04" last5Min="0.05" last15Min="0.0"/> <r:mainMemory RAMSize="1995" virtualSize="1994" RAMAvailable="207" virtualAvailable="73"/> <r:localAvailableSpaceInBytes>21628452</r:localAvailableSpaceInBytes> <r:uptimeInDays>205</r:uptimeInDays> </r:runtime>
<addressing>
- kind: function
- name:
{http://gcube-system.org/namespaces/resource/function/hardware/addressing}addressing - role: report access-related information on the hardware
The interface defines the following properties:
- @TODO
- @TODO
The interface is bound to the XML representation defined by the following XML Schema:
<schema targetNamespace="http://gcube-system.org/namespaces/resource/function/hardware/addressing" xmlns:a="http://gcube-system.org/namespaces/resource/function/hardware/addressing" elementFormDefault="qualified" xmlns="http://www.w3.org/2001/XMLSchema"> <element name="addressing"> <complexType> <sequence> <element name="domainName" type="string" maxOccurs="unbounded"/> <element name="networkAdapter"> <complexType> <simpleContent> <extension base="string"> <attribute name="name" type="string" use="optional"/> <attribute name="ipAddress" type="a:ip" use="required"/> <attribute name="inboundIP" type="a:ip" use="optional"/> <attribute name="outboundIP" type="a:ip" use="optional"/> </extension> </simpleContent> </complexType> </element> </sequence> </complexType> </element> <simpleType name="ip"> <restriction base="string"> <pattern value="(([1-9]?[0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\.){3}([1-9]?[0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])"/> </restriction> </simpleType> </schema>
A sample serialisation is the following:
<a:addressing xmlns:a="http://gcube-system.org/namespaces/resource/function/hardware/addressing"> <a:domainName>node4.tsec.d4science.research-infrastructures.eu</a:domainName> <a:networkAdapter inboundIP="146.48.122.108" outboundIP="146.48.122.108" name="local-adapter" ipAddress="146.48.122.108" /> </a:addressing>
Software
<software>
- kind: class
- name:
{http://gcube-system.org/namespaces/resource/class}software
The interface defines the following properties:
- @TODO
- @TODO
The interface is bound to the XML representation defined by the following XML Schema:
<schema xmlns="http://www.w3.org/2001/XMLSchema" targetNamespace="http://gcube-system.org/namespaces/resource/class" xmlns:tns="http://gcube-system.org/namespaces/resource/class" elementFormDefault="qualified"> <element name="software"> <complexType> <!-- TODO --> </complexType> </schema>
A sample serialisation is the following:
<!-- TODO -->Software Deployment
@TODO definition and binding
A deployment of any software resource identified above.
Service Endpoints
@TODO definition and binding
- endpoint of system services (current and future);
- endpoint of external services (web-app, RDBMS, apps);
Service Instances
@TODO definition and binding
- mostly instances of stateful system services (current and future);
Data Sources
@TODO definition and binding
Implementation Requirements
The model is fully defined by its facet and interface layers and does not address issues related to the implementation of its primitives, or to the design of local or remote APIs that allow producers and consumers to publish or discover resource descriptions (cf. Figure 4).
Equally, the model is purely descriptive and does not address lifetime issues. From the perspective of the model, the lifetime of resources begins when their descriptions are first published within the
system and it terminates when those descriptions are removed from the system. An arbitrary number of independently developed and functionally unrelated components may be involved in the creation, manipulation, and retrieval of resource descriptions during their lifetime.
However, the model raises implicit requirements against those implementations and APIs, as well as against the architecture of the components that manage the lifetime of resource descriptions. Namely, the model requires an overall alignment with the modularity that characterizes its facet layer.
Thus object models will be cast in terms of arbitrary facets, publication APIs will allow arbitrary modification of arbitrary facets, and discovery APIs will retrieve resources based on characterizations of arbitrary facets. The standardisations defined in the interface layer, and the definitions of private facets which are outside the model, will be both introduced separately from those object models and APIs. Typically, they will take the form of typed object bindings included in the components that are responsible for the definition of the interfaces and private facets (cf. Figure 5).
Components that manage the lifetime of resource descriptions will support an equally modular approach. A subset of producers, in particular, will be responsible for creating resource descriptions as these are first published in the system. Some of these resource builders will present interactive interfaces to privileged users, who will provide the models. Others will operate non-interactively, synthesizing resource descriptions from configuration files. These latter builders are known as resource keepers. Keepers will not centralize the task of description building, but distribute it modularity across a set of co-deployed delegates, each of which is independently developed to create given facets of the descriptions. Keepers will be responsible for dynamically discovering the delegates which are available on the hosting node, for coordinating them through the task, and for publishing the resulting resource descriptions. Individual delegates may produce interfaces or private facets which are private to specific implementation of system functions (cf. Figure 6).
Glossary
→ builder: producer that creates and publishes resource description;
→ common property: property shared by all resources and defined independently from facets;
→ consumer: component that discovers and consumes resource descriptions
→ delegate: component invoked by a resource keeper to create given facets of resource descriptions
→ directory services: system services that store and give access to resource descriptions
→ discovery API: the local interface of discovery services whereby consumers can retrieve resource descriptions
→ facet: uniquely named subset of a resource description
→ facet layer: set of primitives defined by the model, including common properties and facets
→ function interface: interface that describes resources in in support of given management functions
→ interface: facet defined by the model
→ interface layer: set of interfaces defined by the model
→ keeper: non-interactive builder that synthesizes resource descriptions from configuration files
→ object model: an implementation of the facet layer of the model in an object-oriented language
→ producer: component that creates/updates, and publishes resource descriptions
→ prototype interface: interface that describes the prototypical resource in a given class, independently from management functions;
→ publisher API: the local interface of discovery services whereby producers can publish resource descriptions
→ resource: computational resource described within the system for management purposes
→ resource description: information about managed resources available within the system;
