Difference between revisions of "GFeed-Service"

From Gcube Wiki
Jump to: navigation, search
(Get available collectors)
(Submission)
(One intermediate revision by the same user not shown)
Line 19: Line 19:
 
==Capabilities==
 
==Capabilities==
 
===Get available collectors===
 
===Get available collectors===
In order to get information on available collectors clients can perform a '''GET''' HTTP method on ''<BASE_URL>/capabilities/controllers''. The response is a JSON representation of available collectors.
+
In order to get information on available collectors clients can perform a '''GET''' HTTP method on ''<BASE_URL>/capabilities/harvesters''. The response is a JSON representation of available collectors.
  
 
===Get available controllers===
 
===Get available controllers===
Line 28: Line 28:
 
Following parameters are expected to be declared in the query string (multiple values can be specified):  
 
Following parameters are expected to be declared in the query string (multiple values can be specified):  
  
* Parameter '''collector'''  
+
* Parameter '''harvester'''  
 
** expected value : to invoke collector ID
 
** expected value : to invoke collector ID
 
** default value ''ALL''  
 
** default value ''ALL''  
Line 36: Line 36:
 
** default value ''ALL''  
 
** default value ''ALL''  
  
The resulting execution will be the combination of all requested ''collectors'' publishing their data to all requested ''controllers'' (only implemented transformation will be performed).
+
The resulting execution will be the combination of all requested ''harvester'' publishing their data to all requested ''controllers'' (only implemented transformation will be performed).
  
 
Available plugins ID can be retrieved by invoking related [[#Capabilities]] methods.
 
Available plugins ID can be retrieved by invoking related [[#Capabilities]] methods.
Line 46: Line 46:
  
 
* Perform all available combinations : <BASE_URL>/execution
 
* Perform all available combinations : <BASE_URL>/execution
* Collect '''DataMiner''' Algorihtms information and push them in '''gCat''' service : <BASE_URL>/execution?collector=DATAMINER_ALGORITHMS_COLLECTOR&controller=GCAT
+
* Collect '''DataMiner''' Algorihtms information and push them in '''gCat''' service : <BASE_URL>/execution?harvester=DATAMINER_ALGORITHMS_COLLECTOR&controller=GCAT
  
 
===Get submission history===
 
===Get submission history===

Revision as of 13:30, 19 May 2020

Aim of this service is to describe the implementation of gFeed-Service (for more information refer to GFeed).

Architecture

Deployment

Plugins

Plugins are expected to be found in the service classpath. They are typically distributed as uber-jar and their deployment depends on the hosting container.

To see a complete list of available plugins implemetations please refer to gFeed-plugins-list

IS Requirements

IS Requirements are listed here.

HTTP Interface

Following is a list of methods exposed by gFeed HTTP interface. All methods require authentication so keep in mind that a gcube-token is expected. In this section <BASE_URL> stands for http(s)://<HOSTNODE>/gCat-Feeder/gcube/service/ where <HOSTNODE> should be determined by querying the gCube Information System.

Capabilities

Get available collectors

In order to get information on available collectors clients can perform a GET HTTP method on <BASE_URL>/capabilities/harvesters. The response is a JSON representation of available collectors.

Get available controllers

In order to get information on available controllers clients can perform a GET HTTP method on <BASE_URL>/capabilities/controllers. The response is a JSON representation of available controllers.

Executions

Submission

In order to submit an execution clients can perform a POST HTTP method on <BASE_URL>/execution. Following parameters are expected to be declared in the query string (multiple values can be specified):

  • Parameter harvester
    • expected value : to invoke collector ID
    • default value ALL
  • Parameter controller
    • expected value : to invoke controller ID
    • default value ALL

The resulting execution will be the combination of all requested harvester publishing their data to all requested controllers (only implemented transformation will be performed).

Available plugins ID can be retrieved by invoking related #Capabilities methods.

The resulting response is the ID of the submitted execution.

Examples

Following is a list of typical usages :

  • Perform all available combinations : <BASE_URL>/execution
  • Collect DataMiner Algorihtms information and push them in gCat service : <BASE_URL>/execution?harvester=DATAMINER_ALGORITHMS_COLLECTOR&controller=GCAT

Get submission history

In order to get the history of submitted executions clients can perform a GET HTTP method on <BASE_URL>/execution. The response is a JSON array of reports referring to submitted executions.

Get report

In order to get a report for a specific execution clients can perform a GET HTTP method on <BASE_URL>/execution/<EXECUTION_ID>, where <EXECUTUION_ID> is the id returned from submission method. Aim of this method is to monitor the outcome of a submitted execution (Asynch logic).

Please keep in mind that detailed reports are provided as a text file, accessible at reportUrl. The following is a report example :

 {
    "id": 4,
    "collectors": [
      "DATAMINER_ALGORITHMS_COLLECTOR"
    ],
    "catalogues": [
      "GCAT"
    ],
    "callerEncryptedToken": ...,
    "callerIdentity": ...,
    "callerContext": ...,
    "status": "SUCCESS",
    "reportUrl": ...,
    "startTime": ...,
    "endTime": ...
  }