Difference between revisions of "Accounting Aggregator"
Luca.frosini (Talk | contribs) (→Provide regular expression) |
Luca.frosini (Talk | contribs) (→Provide regular expression) |
||
| Line 88: | Line 88: | ||
For example to properly manage the following '''ServiceUsageRecord''' | For example to properly manage the following '''ServiceUsageRecord''' | ||
| − | < | + | <pre> |
{ | { | ||
"operationCount": 1, | "operationCount": 1, | ||
| Line 110: | Line 110: | ||
"operationResult": "SUCCESS" | "operationResult": "SUCCESS" | ||
} | } | ||
| − | </ | + | </pre> |
an example of json to provide is | an example of json to provide is | ||
| − | < | + | <pre> |
{ | { | ||
serviceClass : "Index" | serviceClass : "Index" | ||
| Line 121: | Line 121: | ||
replace : "/delete/COLLECTION_ID/ITEM_ID" | replace : "/delete/COLLECTION_ID/ITEM_ID" | ||
} | } | ||
| − | </ | + | </pre> |
Please note that RESTFull services like the Resource Registry cannot properly managed in this way so it is important that the developer fixes the service as soon as possible. This is due to misisng operation type in '''calledMethod''' which is not part of the URL instead of the HTTP method. | Please note that RESTFull services like the Resource Registry cannot properly managed in this way so it is important that the developer fixes the service as soon as possible. This is due to misisng operation type in '''calledMethod''' which is not part of the URL instead of the HTTP method. | ||
Revision as of 12:00, 2 January 2018
This component is responsible of aggregating the collected Accounting data.
Accounting Aggregator in the Accounting Architecture
The following image evidences the accounting-aggregator components as part of Accounting Architecture:
Accounting Aggregator Execution
This component is a Smart Executor plugin and is launched as Global Scheduled Task (see Smart Executor for further information)
It aggregates (lossless) accounted Usage Record. The scheduling plan is to run on different smart-executor equipped with this plugin with the following parameters:
- Hourly => accounting records per hour are available for the past week;
- Daily => accounting records per day are available for the past 3 months;
- Monthly => accounting records per month are available for the past 3 years;
- Yearly => accounting records per year are available for the past 10 years;
How to manage calledMethod lead to ServiceUsageRecord cannot be aggregated
RESTFull (and REST-like) services contain the ID of the managed resource inside the path of the URL. Smartgears auto calculate calledMethod by using the relative path of the URL. This lead to have ServiceUsageRecord cannot be aggregated.
To solve such an issue some actions are need from developers:
- Rewrite calledMethod in the service code
- Provide regular expression to match old accounted records to allow to rewrite them from accoutnign-aggrator
=== Rewrite calledMethod ===calledMethod
The developer have to insert as first line of the entry method the following line of code to instruct Smartgears to set the desired calledMethod
CalledMethodProvider.instance.set("rewrittenCalledMethod");
Resource Registry is an example of how to properly use such a feature for RESTFull services.
For example Resource Registry use the following URLs to read/update/delete a Facet having UUID 4023d5b2-8601-47a5-83ef-49ffcbfc7d86
GET /resource-registry/er/facet/4023d5b2-8601-47a5-83ef-49ffcbfc7d86
PUT /resource-registry/er/facet/4023d5b2-8601-47a5-83ef-49ffcbfc7d86
DELETE /resource-registry/er/facet/4023d5b2-8601-47a5-83ef-49ffcbfc7d86
If the service does not rewrite the calledMethod we have two effect:
- the three operation cannot be distinguished from accounting and lead to aggregate the records operating to the same facet.
- the uuid does not allow to aggregate the operation across facets.
Resource Registry instruct smartgears to rewrite the calledMethod as following:
CalledMethodProvider.instance.set("GET /resource-registry/er/facet/ID");
CalledMethodProvider.instance.set("PUT /resource-registry/er/facet/ID");
CalledMethodProvider.instance.set("DELETE /resource-registry/er/facet/ID");
I such a way the different methods are distinguishable and the ServiceUsageRecords can be aggregated across facets.
Provide regular expression
Accounting provide a way to rewrite the the old calledMethod to be compliant with the one declared in the service by using CalledMethodProvider.
The developer has to provide the following a json declaring the service class and name to match to check if the provided regular expression match the old calledMethod so that is rewritten with the provided replacement.
For example to properly manage the following ServiceUsageRecord
{
"operationCount": 1,
"creationTime": 1507334430724,
"serviceClass": "Index",
"callerHost": "88.197.53.47",
"callerQualifier": "UNKNOWN",
"recordType": "ServiceUsageRecord",
"consumerId": "UNKNOWN",
"aggregated": true,
"serviceName": "FullTextIndexNode",
"duration": 3,
"maxInvocationTime": 58,
"scope": "/d4science.research-infrastructures.eu/gCubeApps/iSearch",
"host": "dewn10.madgik.di.uoa.gr:8080",
"startTime": 1507334430481,
"id": "12a631da-1541-4771-8961-8d7e4a694bc0",
"calledMethod": "/delete/3121eccdcbf9c6e982dcc5a2164b8b4029f51d8b1c448ddaf46316178d755c6f/oai:oai.datacite.org:352019",
"endTime": 1507334430481,
"minInvocationTime": 1,
"operationResult": "SUCCESS"
}
an example of json to provide is
{
serviceClass : "Index"
serviceName : "FullTextIndexNode"
regex : "/delete/[^\\t\\n\\r\\f\\v]+/[^\\t\\n\\r\\f\\v]+"
replace : "/delete/COLLECTION_ID/ITEM_ID"
}
Please note that RESTFull services like the Resource Registry cannot properly managed in this way so it is important that the developer fixes the service as soon as possible. This is due to misisng operation type in calledMethod which is not part of the URL instead of the HTTP method.
