Difference between revisions of "Common-accounting ABANDONED"

From Gcube Wiki
Jump to: navigation, search
(Created page with '= Scope = This library offers an easy way to produce and consume resource accounting data. = Design and implementation notes = The "common-accounting" library contains the resou…')
 
 
(14 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
[[Category:TO BE REMOVED]]
 +
 
= Scope =
 
= Scope =
This library offers an easy way to produce and consume resource accounting data.
+
This library abstracts the underlying messaging infrastructure in order to provide some facilities both for the services and for the accounting system (i.e. Usage Tracker).
 +
The common-accounting-lib allows the services to send the accounting records on the broker and allows the accounting system to receive these messages.
 +
In our enviroment, the broker is represented by ActiveMQ, which is already replicated with some failover endpoints, in order to increase fault tolerance and scalability. Each infrastructure instance (such as dev, testing and production) has a running instance,  that is related to the scope of the infrastructure. This behavior needs to be mirrored even on the accounting library.
 +
 
 +
Find <code>common-accounting-lib</code> on [http://maven.research-infrastructures.eu/nexus/index.html#nexus-search;quick~common-accounting-lib Nexus Repository Browser] for an artifact with the following coordinates:
 +
 
 +
<source lang="xml">
 +
<dependency>
 +
<groupId>org.gcube.accounting</groupId>
 +
<artifactId>common-accounting-lib</artifactId>
 +
</dependency>
 +
</source>
  
 
= Design and implementation notes =
 
= Design and implementation notes =
The "common-accounting" library contains the resource accounting data-model, a producer to publish accounting data on the messaging broker and a consumer to retrieve them.
+
The "common-accounting" library contains:
 +
* the resource accounting data-model ([https://gcube.wiki.gcube-system.org/gcube/index.php?title=Common-accounting-model common-accounting-model]);
 +
* a producer to publish raw accounting data on the messaging broker  
 +
* a consumer to retrieve raw accounting data
 +
 
 +
At the producer-side, the library uses the [https://gcube.wiki.gcube-system.org/gcube/index.php/Common_Messaging_endpoints Messaging Endpoints library] to harvest the Message Broker endpoints that are published on the IS (the scope is the only mandatory information to retrieve the endpoints and it is already available in each Usage Record).
 +
At the consumer-side, the library makes no assumptions on the underlying infrastructure. The service that acts as a consumer needs to provide broker endpoint(s) and scope(s) where the accounting information are available and ready to be consumed. The library abstracts the complexity of the Message Broker and hides completely how the system is implemented. This behaviour stems from the fact that the consumer is the Usage Tracker which is agnostic from the gCube infrastructure.
 +
 
 +
= Usage =
 +
In order to use the common-accounting-lib to publish accounting records some steps are required:
 +
 
 +
* add the Maven dependency of the library in your pom
 +
* decide where is more appropriate to implement the accounting logic.
 +
 
 +
* get the ResourceAccounting instance through the available factory.
 +
<source lang="java">
 +
  ResourceAccounting raFactory = null;
 +
  try {
 +
      raFactory = ResourceAccountingFactory.getResourceAccountingInstance();
 +
  }
 +
  catch (IOException e) {
 +
      e.printStackTrace();
 +
  }
 +
</source>
 +
* instantiate a new RawUsageRecord object, which represents the Java implementation of the Resource Accounting model  , and fill all the basic fields of the record and, optionally, fill even the resource specific fields (following record is an example of Storage-Usage Usage Record):
 +
<source lang="java">
 +
  RawUsageRecord ur = new RawUsageRecord();
 +
 
 +
  //generic properties
 +
  ur.setResourceType("storage-usage");
 +
  ur.setConsumerId("ermanno.travaglino");
 +
  ur.setResourceOwner("paolo.fabriani");
 +
  ur.setResourceScope("/gcube/devsec");
 +
 
 +
  Calendar createTime = new GregorianCalendar();
 +
  Calendar startTime = new GregorianCalendar();
 +
  Calendar endTime = new GregorianCalendar();
  
== Data-model ==
+
  ur.setCreateTime(createTime.getTime());
The structure of a generic accounting record (Usage Record, UR) will be composed of a set of common fields for all resource types, in particular:
+
  
* id : an unique identifier for the UR
+
  try {
* consumerId : the user actually consuming the resource (optional, for future purposes)
+
  ur.setStartTime(startTime.getTime());
* createTime : when the UR was created
+
  ur.setEndTime(endTime.getTime());
* startTime, endTime : the time window the UR refers to
+
  }
* resourceType : the type of resource the UR tracks
+
  catch (InvalidValueException e) {
* scope : the scope of the resource
+
  e.printStackTrace();
* resourceOwner : who owns the resource and/or who creates the UR
+
  }
  
Furthermore, for each UR there will be a section to be filled with the specific properties per resource type (key-value pairs).
+
  //specific properties
 +
  ur.setResourceSpecificProperty("providerId","cloud.eng.it");
 +
  ur.setResourceSpecificProperty("objectURI", "http://example.org/absolute/URI/with/absolute/path/to/object.bulk");
 +
  ur.setResourceSpecificProperty("operationType","PUT");
 +
  ur.setResourceSpecificProperty("qualifier","text/xml");
 +
  ur.setResourceSpecificProperty("dataType","STORAGE");
 +
  ur.setResourceSpecificProperty("dataVolume", "10231089");
 +
  ur.setResourceSpecificProperty("dataCount", "1000");
 +
  ur.setResourceSpecificProperty("callerIP", "etics.eng.it");
 +
</source>
  
== Resource Types ==
+
* and, finally, send a message to the accounting system with the sendAccountingMessage method, passing as a parameter the RawUsageRecord object.
The resource types we've identified are: Execution, Service, Data-access and Storage.
+
<source lang="java">
 +
  raFactory.sendAccountingMessage(ur);
 +
</source>

Latest revision as of 13:37, 19 October 2016


Scope

This library abstracts the underlying messaging infrastructure in order to provide some facilities both for the services and for the accounting system (i.e. Usage Tracker). The common-accounting-lib allows the services to send the accounting records on the broker and allows the accounting system to receive these messages. In our enviroment, the broker is represented by ActiveMQ, which is already replicated with some failover endpoints, in order to increase fault tolerance and scalability. Each infrastructure instance (such as dev, testing and production) has a running instance, that is related to the scope of the infrastructure. This behavior needs to be mirrored even on the accounting library.

Find common-accounting-lib on Nexus Repository Browser for an artifact with the following coordinates:

<dependency>
	<groupId>org.gcube.accounting</groupId>
	<artifactId>common-accounting-lib</artifactId>
</dependency>

Design and implementation notes

The "common-accounting" library contains:

  • the resource accounting data-model (common-accounting-model);
  • a producer to publish raw accounting data on the messaging broker
  • a consumer to retrieve raw accounting data

At the producer-side, the library uses the Messaging Endpoints library to harvest the Message Broker endpoints that are published on the IS (the scope is the only mandatory information to retrieve the endpoints and it is already available in each Usage Record). At the consumer-side, the library makes no assumptions on the underlying infrastructure. The service that acts as a consumer needs to provide broker endpoint(s) and scope(s) where the accounting information are available and ready to be consumed. The library abstracts the complexity of the Message Broker and hides completely how the system is implemented. This behaviour stems from the fact that the consumer is the Usage Tracker which is agnostic from the gCube infrastructure.

Usage

In order to use the common-accounting-lib to publish accounting records some steps are required:

  • add the Maven dependency of the library in your pom
  • decide where is more appropriate to implement the accounting logic.
  • get the ResourceAccounting instance through the available factory.
 
   ResourceAccounting raFactory = null;
   try {
      raFactory = ResourceAccountingFactory.getResourceAccountingInstance();
   } 
   catch (IOException e) {
      e.printStackTrace();
   }
  • instantiate a new RawUsageRecord object, which represents the Java implementation of the Resource Accounting model , and fill all the basic fields of the record and, optionally, fill even the resource specific fields (following record is an example of Storage-Usage Usage Record):
   RawUsageRecord ur = new RawUsageRecord();
 
   //generic properties
   ur.setResourceType("storage-usage");	
   ur.setConsumerId("ermanno.travaglino");
   ur.setResourceOwner("paolo.fabriani");
   ur.setResourceScope("/gcube/devsec");
 
   Calendar createTime = new GregorianCalendar();
   Calendar startTime = new GregorianCalendar();
   Calendar endTime = new GregorianCalendar();
 
   ur.setCreateTime(createTime.getTime());
 
   try {
	   ur.setStartTime(startTime.getTime());
	   ur.setEndTime(endTime.getTime());
   }
   catch (InvalidValueException e) {
   	e.printStackTrace();
   }
 
   //specific properties
   ur.setResourceSpecificProperty("providerId","cloud.eng.it");
   ur.setResourceSpecificProperty("objectURI", "http://example.org/absolute/URI/with/absolute/path/to/object.bulk");
   ur.setResourceSpecificProperty("operationType","PUT");
   ur.setResourceSpecificProperty("qualifier","text/xml");
   ur.setResourceSpecificProperty("dataType","STORAGE");
   ur.setResourceSpecificProperty("dataVolume", "10231089");
   ur.setResourceSpecificProperty("dataCount", "1000");
   ur.setResourceSpecificProperty("callerIP", "etics.eng.it");
  • and, finally, send a message to the accounting system with the sendAccountingMessage method, passing as a parameter the RawUsageRecord object.
   raFactory.sendAccountingMessage(ur);