Difference between revisions of "Resource Model (2nd generation)"

Revision as of 04:52, 11 June 2012

Introduction

At its core, gCube provides a set of management functions over broad classes of computational resources. Functions include lifetime management, monitoring, sharing, reporting, deployment, security, and process execution. Resources include software components, the hardware on which those components run, and the data sources they produce and consume.

Each management function targets one or more resource classes. For example, deployment and process execution are concerned with software and hardware, security is concerned with software and data sources, and reporting applies to all resources. The implementation of the function is distributed across one or more system components, which are themselves primary examples of managed software resources. A single component may contribute to the implementation of multiple functions.

System components consume or produce information about resources in the target classes. gCube defines a descriptive model that regulates the representation and exchange of this information. Based on this resource model, it includes directory services that allow components to publish and discover resource descriptions (cf. Figure 1).

Figure 1: A model governs the description of resource descriptions produced and consumed by components that implements management functions defined over classes of related resources. The components rely on directory services to publish and discover resource descriptions.

First Generation

At its inception, the system took a fairly static view over its management functions and equated them with their latest implementations. Resource classes were thus pre-defined and the model of resources within each class was centrally and fully prescribed. Quite simply, it collected the information required or produced by all the functions defined over that class.

Semantic variations within classes were modelled with hierarchies, insofar as the variations could be anticipated. Classes and subclasses had their own XML bindings and the bindings were defined by schemas. The schemas were used to validate the publication and discovery of resource descriptions.

Since changing the models had an endemic impact within the system, the schemas evolved to become less strict and a distinguished class of “generic” resources was introduced to accommodate resource descriptions that could not be retrofitted within existing hierarchies.

The system evolved within this scheme, occasionally acquiring new management functions and, far more commonly, refining the implementation of existing functions. The catch-all class of generic resources grew beyond initial expectations.

Second Generation

Starting from this version of the resource model, the system takes a more modular and abstract view over its management functions. Functions are described solely by their APIs, and different implementations of the same functions may exist within the system, over time or even concurrently.

Facet Layer

As different implementations may require different information about resources, the system no longer prescribes concrete resource models for them. Rather, implementations are responsible for defining the resource descriptions that they require or produce, and for binding that information to XML.

With this approach, the semantics of a resource is not found within class hierarchies. It is instead carried implicitly by the information that, at any point in time, implementation components have independently attached to the resource in order to manage it (cf. Figure 2). Each such piece of information is a resource facet and, at ay point in time, the semantics of a resource is the collection of its current facets.

The system remains responsible for defining the framework in which arbitrary facets may be collated into completed resource descriptions. This framework constitutes the facet layer of the new resource model.

Figure 2: Producers and consumers in the implementations of given management functions add and use only relevant facets in resource descriptions.

Interface Layer

Consumers often raise overlapping requirements on the information that describes resources. Requirements may converge around resources of a given class, or specialisations thereof. In some cases, they may converge across resource classes and along functions. Some requirements may apply indiscriminately to all resources managed by the system.

It is undesirable to replicate information required by unrelated consumers within different, independently produced facets. Equally, it is undesirable to create unnecessary dependencies between unrelated publishers and consumers, i.e. expect the latter to access facets defined by the former. Converging modelling requirements are best addressed by sharing information within standard facets. These facets serve as information-oriented interfaces for resources, whereby the resources may be discovered and their models consumed for management purposes. The information that comprises interfaces is public within the system, while the rest remains private to a set of related publishers and consumers (cf. Figure 3).

The facet layer of the new resource model does not address standardisation issues beyond the definition of a small set of common properties shared by all resource descriptions. Rather, it leaves the standardisation of facets to multi-party agreements of varying scope. System-wide agreements define interfaces common to all resources of a given resource class (prototype interfaces). Function-wide agreements define interfaces common to all the implementations of a given function (function interfaces).

Collectively, the interfaces identify a higher and evolving layer of the model, the interface layer. The flexibility of the facet layer underneath allows resources to have multiple interfaces, as well as to acquire interfaces over time.

Figure 3: Producers and consumers in the implementations of different functions synchronise on interfaces of shared information.

Resource Descriptions

All resources managed by the system are described by the following properties:

• a unique identifier;

the identifier of a resource allows system components to unambiguously separate it from other resources and to resolve references to the resource included in the description of other resources. Accordingly, the model requires its uniqueness and immutability but does not otherwise mandate a concrete syntax for it. Implementations will ensure that resource identifiers can be embedded in URIs (through escaping of characters forbidden by RFC 2396) and will offer guarantees for their immutability and global uniqueness.

• an optional name;

the name of a resource serves solely for reporting purposes. The model does not mandate a syntax for it and allows it to change over time.

• an optional description;

the description of a resource serves solely for reporting purposes. The model does not mandate a syntax for it and allows it to change over time.

• one or more scopes;

the scopes of a resource limit access to the resource and visibility of its description to clients that operate in one of the scopes. The syntax of scopes is defined outside the model, as it is the responsibility for enforcing scopes. Scopes may added to or removed from resources over time

• one or more facets;

facets are uniquely named but the model does not constrain a-priori their structure and content. Constraints on particular facets are introduced in the interface layer of the model. Facets may be added to or removed from resources over time.

XML Binding

Resources serialize as XML documents rooted in a resource element with at least an id attribute for the resource identifier, name, description, and a list of one or more scope elements nest inside the document root in that order and before facet serializations. All have simple, non-empty content and the content of name can be qualified with an arbitrary namespace. All the elements and attributes are defined in the namespace http://gcube-system.org/namespaces/resource, except for facets which may be named in any namespace, as long as one such namespace exists. Facets may be simple or complex elements, though they cannot mix elements and text nor include XML structures other than elements.

Formally, XML serializations of resources must validate against the following XML Schema:

<schema xmlns="http://www.w3.org/2001/XMLSchema" targetNamespace="http://gcube-system.org/namespaces/resource"
 xmlns:tns="http://gcube-system.org/namespaces/resource" elementFormDefault="qualified">
 
<element name="resource">
 <complexType>
 
   <sequence>
      <element name="name" type="tns:nonemptyname" minOccurs="0" />
      <element name="description" type="tns:nonempty" minOccurs="0" />
      <element name="scope" type="tns:nonempty" maxOccurs="unbounded" />
      <any processContents="lax" namespace="##other"  maxOccurs="unbounded" />
   </sequence>
 
    <attribute name="id" type="tns:nonempty" />
    <anyAttribute processContents="lax" namespace="##other"  />
 
</complexType>
</element>
 
<simpleType name="nonempty">
 <restriction base="string">
   <minLength value="1" />
   <pattern value=".*[^\s].*" />
 </restriction>
</simpleType>
 
<simpleType name="nonemptyname">
 <restriction base="QName">
   <minLength value="1" />
   <pattern value=".*[^\s].*" />
 </restriction>
</simpleType>
</schema>

Note that the schema lacks the expressiveness to capture all the constraints defined by the model (unique naming and content model enforcement for facets).

Interfaces

The model defines a number of interfaces that standardize properties common to broad classes of managed resources.

Interfaces may be classified according to their kind:

• class interfaces: describe the prototypical resource of a given class, independently of how it is managed within the system;
• function interfaces: describe resources in support of one or more management functions;

For convenience of reference, the definitions of prototype and function interfaces are grouped below per resource class. Function interfaces that describes resources in two more classes are grouped separately, as common interfaces.

Common Interfaces

<lifetime>

kind:function

name: {http://gcube-system.org/namespaces/resource/common/lifetime}lifetime

role: describe the lifetime of the resource

Properties

• @TODO
• @TODO

Binding

<schema targetNamespace="http://gcube-system.org/namespaces/resource/function/common/lifetime" 
elementFormDefault="qualified" xmlns="http://www.w3.org/2001/XMLSchema">
 
<element name="lifetime">
   <complexType>
      <sequence>
         <element name="status" type="string"/>
         <element name="activationTime" type="dateTime"/>
         <element name="deactivationTime" type="dateTime" use="optional"/>
         <element name="lastUpdate" type="dateTime"/>
         <element name="nextUpdate" type="dateTime"/>
      </sequence>
   </complexType>
</element>
 
</schema>

Sample

<l:lifetime xmlns:l="http://gcube-system.org/namespaces/resource/function/common/lifetime" >
   <l:status>certified</l:status>
   <l:activationTime>2012-06-10T21:29:05+02:00</l:activationTime>
   <l:deactivationTime>2012-07-23T21:29:05+02:00</l:deactivationTime>
   <l:lastUpdate>2012-06-29T06:36:50+01:00</l:lastUpdate>
   <l:nextUpdate>2012-06-29T06:37:50+01:00</l:nextUpdate>
</l:lifetime>

<certificate>

kind:function

name: {http://gcube-system.org/namespaces/resource/common/certificate}certificate

role: report information on the credentials associated to the resource through an X.509 certificate

Properties

• ca: the Certification Authority that issued the certificate
• dn: the Distinguished Name of the certificate
• expireOn: the expiration date and time of the certificate

Binding

<schema targetNamespace="http://gcube-system.org/namespaces/resource/function/common/certificate" 
elementFormDefault="qualified" xmlns="http://www.w3.org/2001/XMLSchema">
 
<element name="certificate">
   <complexType>
      <sequence>
         <element name="ca" type="string"/>
         <element name="dn" type="string"/>
         <element name="expireOn" type="dateTime"/>
      </sequence>
   </complexType>
</element>
 
</schema>

Sample

<s:certificate xmlns:s="http://gcube-system.org/namespaces/resource/function/common/certificate">
   <s:ca>/C=IT/O=INFN/CN=INFN CA</s:ca>
   <s:dn>/C=IT/O=INFN/OU=Host/L=NMIS-ISTI/CN=node2.tsec.d4science.research-infrastructures.eu</s:dn>
   <s:expireOn>2012-06-06T14:55:49+02:00</s:expireOn>
</s:certificate>

<labels>

kind:function

name: {http://gcube-system.org/namespaces/resource/common/label}labels

role: list of labels characterizing the resource

Properties

• @TODO

Binding

<schema targetNamespace="http://gcube-system.org/namespaces/resource/function/common/label" 
elementFormDefault="qualified" xmlns="http://www.w3.org/2001/XMLSchema">
 
<element name="labels">
   <complexType>
      <sequence>
         <element name="label" maxOccurs="unbounded">
            <complexType>
               <simpleContent>
                  <extension base="string">
                     <attribute name="name" type="string" use="required"/>
                  </extension>
               </simpleContent>
            </complexType>
         </element>
      </sequence>
   </complexType>
</element>
 
</schema>

Sample

<l:labels xmlns:l="http://gcube-system.org/namespaces/resource/function/common/label">   
   <l:label name="java">...any...</l:label>
   <l:label name="gcore">...any...</l:label>
   <l:label name="ghn">...any...</l:label>
</l:labels>

Hardware

<hardware>

kind: class

name: {http://gcube-system.org/namespaces/resource/class}hardware

role: describe any hardware exploited in the infrastructure

Properties

• @TODO
• @TODO

Binding

<schema targetNamespace="http://gcube-system.org/namespaces/resource/class" elementFormDefault="qualified" 
 xmlns="http://www.w3.org/2001/XMLSchema">
 
<element name="hardware">
   <complexType>
      <sequence>
         <element name="architecture">
            <complexType>
               <simpleContent>
                  <extension base="string">
                     <attribute name="platformType" type="string"/>
                     <attribute name="SMPSize" type="string" use="optional"/>
                     <attribute name="SMTSize" type="string" use="optional"/>
                  </extension>
               </simpleContent>
            </complexType>
         </element>
         <element name="operatingSystem">
            <complexType>
               <simpleContent>
                  <extension base="string">
                     <attribute name="name" type="string"/>
                     <attribute name="release" type="string" use="optional"/>
                     <attribute name="version" type="string" use="optional"/>
                  </extension>
               </simpleContent>
            </complexType>
         </element>
         <element name="processor" maxOccurs="unbounded">
            <complexType>
               <simpleContent>
                  <extension base="string">
                     <attribute name="vendor" type="string"/>
                     <attribute name="model" type="string"/>
                     <attribute name="family" type="string"/>
                     <attribute name="modelName" type="string"/>
                     <attribute name="ClockSpeedMhz" type="string"/>
                     <attribute name="Bogomips" type="string"/>
                     <attribute name="CacheL1" type="string"/>
                     <attribute name="CacheL2" type="string"/>
                  </extension>
               </simpleContent>
            </complexType>
         </element>
      </sequence>
   </complexType>
</element>
 
</schema>

Sample

<h:hardware xmlns:h="http://gcube-system.org/namespaces/resource/class">
   <h:architecture platformType="amd64" SMPSize="0" SMTSize="0"/><!-- 1 -->
   <h:operatingSystem name="Linux" release="" version="2.6.32-5-xen-amd64"/>
   <h:processor vendor="GenuineIntel" model="44" modelName="Intel(R) Xeon(R) CPU E5630  @ 2.53GHz" family="6"   
ClockSpeedMhz="2533.468" Bogomips="5066.93" CacheL1="12288" CacheL2="0"/>
   <h:processor vendor="GenuineIntel" model="44" modelName="Intel(R) Xeon(R) CPU E5630  @ 2.53GHz" family="6" 
ClockSpeedMhz="2533.468" Bogomips="5066.93" CacheL1="12288" CacheL2="0"/>
</h:hardware>

<coordinates>

kind: function

name: {http://gcube-system.org/namespaces/resource/function/hardware/coordinate}coordinates

role: give information on the geographical location of the hardware

Properties

• @TODO
• @TODO

Binding

<schema targetNamespace="http://gcube-system.org/namespaces/resource/function/hardware/coordinate" 
elementFormDefault="qualified" xmlns:co="http://gcube-system.org/namespaces/resource/function/hardware/coordinate"
xmlns="http://www.w3.org/2001/XMLSchema">
 
<element name="coordinates">
   <complexType>
      <sequence>
         <element name="location" type="string"/>
         <element name="country" type="co:countryCode"/>
         <element name="latitude" type="double"/>
         <element name="longitude" type="double"/>
         <element name="domain" type="string"/>
      </sequence>
   </complexType>
</element>
 
<simpleType name="countryCode">
   <restriction base="string">
      <length value="2"/>
   </restriction>
</simpleType>
 
</schema>

Sample

<c:coordinates xmlns:c="http://gcube-system.org/namespaces/resource/function/hardware/coordinate">
   <c:location>Pisa</c:location>
   <c:country>it</c:country>
   <c:latitude>43.719627</c:latitude>
   <c:longitude>10.421626</c:longitude>
   <c:domain>research-infrastructures.eu</c:domain>
</c:coordinates>

<runtime>

kind: function

name: {http://gcube-system.org/namespaces/resource/function/hardware/runtime}runtime

role: report runtime information on the hardware status

Properties

• @TODO
• @TODO

Binding

<schema targetNamespace="http://gcube-system.org/namespaces/resource/function/hardware/runtime" 
elementFormDefault="qualified" xmlns="http://www.w3.org/2001/XMLSchema">
 
<element name="runtime">
   <complexType>
      <sequence>
         <element name="load">
            <complexType>
               <simpleContent>
                  <extension base="string">
                     <attribute name="last1Min" type="decimal" use="required"/>
                     <attribute name="last5Min" type="decimal" use="required"/>
                     <attribute name="last15Min" type="decimal" use="required"/>
                  </extension>
               </simpleContent>
            </complexType>
         </element>
         <element name="mainMemory">
            <complexType>
               <simpleContent>
                  <extension base="string">
                     <attribute name="RAMSize" type="integer" use="required"/>
                     <attribute name="RAMAvailable" type="integer" use="required"/>
                     <attribute name="virtualSize" type="integer" use="required"/>
                     <attribute name="virtualAvailable" type="integer" use="required"/>
                  </extension>
               </simpleContent>
            </complexType>
         </element>
         <element name="localAvailableSpaceInBytes" type="nonNegativeInteger"/>
         <element name="uptimeInDays" type="integer"/>
      </sequence>
   </complexType>
</element>
 
</schema>

Sample

<r:runtime xmlns:r="http://gcube-system.org/namespaces/resource/function/hardware/runtime">
   <r:load last1Min="0.04" last5Min="0.05" last15Min="0.0"/>
   <r:mainMemory RAMSize="1995" virtualSize="1994" RAMAvailable="207" virtualAvailable="73"/>
   <r:localAvailableSpaceInBytes>21628452</r:localAvailableSpaceInBytes>
   <r:uptimeInDays>205</r:uptimeInDays>
</r:runtime>

<addressing>

kind: function

name: {http://gcube-system.org/namespaces/resource/function/hardware/addressing}addressing

role: report access-related information on the hardware

Properties

• @TODO
• @TODO

Binding

<schema targetNamespace="http://gcube-system.org/namespaces/resource/function/hardware/addressing" 
xmlns:a="http://gcube-system.org/namespaces/resource/function/hardware/addressing" 
elementFormDefault="qualified" xmlns="http://www.w3.org/2001/XMLSchema">
 
<element name="addressing">
   <complexType>
      <sequence>
         <element name="domainName" type="string" maxOccurs="unbounded"/>
         <element name="networkAdapter">
            <complexType>
               <simpleContent>
                  <extension base="string">
                     <attribute name="name" type="string" use="optional"/>
                     <attribute name="ipAddress" type="a:ip" use="required"/>
                     <attribute name="inboundIP" type="a:ip" use="optional"/>
                     <attribute name="outboundIP" type="a:ip" use="optional"/>
                  </extension>
               </simpleContent>
            </complexType>
         </element> 
      </sequence>
   </complexType>
</element>
 
<simpleType name="ip">
   <restriction base="string">
      <pattern value="(([1-9]?[0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\.){3}([1-9]?[0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])"/>
   </restriction>
</simpleType>
 
</schema>

Sample

<a:addressing xmlns:a="http://gcube-system.org/namespaces/resource/function/hardware/addressing">
   <a:domainName>node4.tsec.d4science.research-infrastructures.eu</a:domainName>
   <a:networkAdapter inboundIP="146.48.122.108" outboundIP="146.48.122.108" name="local-adapter" ipAddress="146.48.122.108" />
</a:addressing>

Software

<software>

kind: class

name: {http://gcube-system.org/namespaces/resource/class}software

Properties

• @TODO
• @TODO

Binding

<schema xmlns="http://www.w3.org/2001/XMLSchema" targetNamespace="http://gcube-system.org/namespaces/resource/class"
 xmlns:tns="http://gcube-system.org/namespaces/resource/class" elementFormDefault="qualified">
 
<element name="software">
 <complexType>
 
   <!-- TODO -->
 
 </complexType>
</schema>

Sample

   <!-- TODO -->

Software Deployment

@TODO definition and binding

A deployment of any software resource identified above.

Service Endpoints

@TODO definition and binding

• endpoint of system services (current and future);
• endpoint of external services (web-app, RDBMS, apps);

Service Instances

@TODO definition and binding

• mostly instances of stateful system services (current and future);

Data Sources

@TODO definition and binding

Implementation Requirements

The model is fully defined by its facet and interface layers and does not address issues related to the implementation of its primitives, or to the design of local or remote APIs that allow producers and consumers to publish or discover resource descriptions (cf. Figure 4).

Figure 4: The resource model is implemented in an object model and the object model is a dependency of publication and discovery APIs that producers and consumers use to interact with remote discovery services.

Equally, the model is purely descriptive and does not address lifetime issues. From the perspective of the model, the lifetime of resources begins when their descriptions are first published within the system and it terminates when those descriptions are removed from the system. An arbitrary number of independently developed and functionally unrelated components may be involved in the creation, manipulation, and retrieval of resource descriptions during their lifetime.

However, the model raises implicit requirements against those implementations and APIs, as well as against the architecture of the components that manage the lifetime of resource descriptions. Namely, the model requires an overall alignment with the modularity that characterizes its facet layer.

Thus object models will be cast in terms of arbitrary facets, publication APIs will allow arbitrary modification of arbitrary facets, and discovery APIs will retrieve resources based on characterizations of arbitrary facets. The standardisations defined in the interface layer, and the definitions of private facets which are outside the model, will be both introduced separately from those object models and APIs. Typically, they will take the form of typed object bindings included in the components that are responsible for the definition of the interfaces and private facets (cf. Figure 5).

Figure 5: Object model and publication/discovery APIs align with the facet layer, i.e. deal with arbitrary facets. Object bindings of interfaces and private facets are modularly introduced by separate components.

Components that manage the lifetime of resource descriptions will support an equally modular approach. A subset of producers, in particular, will be responsible for creating resource descriptions as these are first published in the system. Some of these resource builders will present interactive interfaces to privileged users, who will provide the models. Others will operate non-interactively, synthesizing resource descriptions from configuration files. These latter builders are known as resource keepers. Keepers will not centralize the task of description building, but distribute it modularity across a set of co-deployed delegates, each of which is independently developed to create given facets of the descriptions. Keepers will be responsible for dynamically discovering the delegates which are available on the hosting node, for coordinating them through the task, and for publishing the resulting resource descriptions. Individual delegates may produce interfaces or private facets which are private to specific implementation of system functions (cf. Figure 6).

Figure 6: While some producers update resource descriptions, builders are producers that create resource descriptions. Some builders do so interactively, but keepers are builders that create resource configuration automatically based on configuration files. Keepers act modularly, delegating the building task to co-deployed and dynamically discovered components.

Glossary

→ builder: producer that creates and publishes resource description;

→ common property: property shared by all resources and defined independently from facets;

→ consumer: component that discovers and consumes resource descriptions

→ delegate: component invoked by a resource keeper to create given facets of resource descriptions

→ directory services: system services that store and give access to resource descriptions

→ discovery API: the local interface of discovery services whereby consumers can retrieve resource descriptions

→ facet: uniquely named subset of a resource description

→ facet layer: set of primitives defined by the model, including common properties and facets

→ function interface: interface that describes resources in in support of given management functions

→ interface: facet defined by the model

→ interface layer: set of interfaces defined by the model

→ keeper: non-interactive builder that synthesizes resource descriptions from configuration files

→ object model: an implementation of the facet layer of the model in an object-oriented language

→ producer: component that creates/updates, and publishes resource descriptions

→ prototype interface: interface that describes the prototypical resource in a given class, independently from management functions;

→ publisher API: the local interface of discovery services whereby producers can publish resource descriptions

→ resource: computational resource described within the system for management purposes

→ resource description: information about managed resources available within the system;