Difference between revisions of "SmartExecutor"
Luca.frosini (Talk | contribs) (→Launching Tasks) |
Luca.frosini (Talk | contribs) (→Monitor Task Execution) |
||
Line 64: | Line 64: | ||
=== Monitor Task Execution === | === Monitor Task Execution === | ||
+ | |||
+ | |||
+ | the are two API to monitor the task execution. | ||
== Plugin Development == | == Plugin Development == |
Revision as of 16:20, 20 October 2015
The SmartExecutor service allows to execute "gCube Tasks" and monitor their execution status. Each instance of the SmartExecutor service can run the "gCube Tasks" related to the plugins available on such an instance.
Each instance of the SmartExecutor service publishes descriptive information about the co-deployed plugins.
Clients may interact with the SmartExecutor service through a library (SmartExecutor Client) of high-level facilities to simplify the discovery of available plugins in those instances. Each client can request to execute a "gCube Tasks" or getting informations about the state of their execution.
Contents
The Service
SmartExecutor service allows tasks execution through the use of co-deployed plugins. The service allows to pass inputs parameter to the plugin requested to run.
SmartExecutor service can run the tasks in two mode:
- Run and Die
- Scheduled
In the Run and Die mode the requested plugin has been run just one time using the input parameters contained in the invocation request. When the execution terminate because is finished, is stopped or stop due to exception a new execution can be run only if is explicitly requested.
In Scheduled mode, the way to schedule the plugin execution is indicated in the invocation parameters. The execution is invoked every time it matches the scheduling parameters.
Plugins
At startup time the SmartExecutor discover the available plugins (using java.util.ServiceLoader). Once discovered the SmartExecutor gather some required informations of the plugin (e.g. version, name) and ask to the plugin to discover some capabilities of the machine it needs to publish for client which make us of it (e.g. available python version, available R version, a certain hardware capability). Is responsibility of the plugin to collects this information. The SmartExecutor publish the required information and the discovered capabilities on IS using a ServiceEndpoint with Category: VREManagement and Name: SmartExecutor.
The SmartExecutor Client search for this service endpoints to retrieve available SmartExecutor instances running a certain plugin.
Client
The SmartExecutor service provides a client library to simplify the following procedures:
- discover service instances that can execute the target task. This requires interaction with the Information System.
- launch the execution of the task with one the discovered instances.
- monitor the execution of the running task.
Instance Discovery
SmartExecutor Client provide some facilities to get the SmartExecutor service instances.
It allows to retrieve one of the the running SmartExecutor services or filter them in the following way:
- By plugin name only
- By plugin name and one or more of the following requirements:
- providing KEY-VALUE list. Each KEY-VALUE must be found on the Service Endpoints.
- providing an SimpleQuery expression to filter the Service Endpoints. The expression is added to the standard query performed.
- providing a list of endpoints to be selected
Using SmartExecutor Client a proxy for remote invocation can be obtained. Using this proxy is very easy to call a remote method as the method was local.
Launching Tasks
One of the API exposed by the SmartExecutor service is used to launch a plugin execution instance.
Launch Inputs
The invocation parameters are:
- name of the plugin to be instantiated and run (MANDATORY)
- inputs to pass to plugin instance execution (MANDATORY)
- capabilities the plugin has to satisfy to correctly run an execution (e.g. the machine has a certain version of python interpreter) (OPTIONAL)
- scheduling strategy to launch the execution (e.g. every day at 22:00) (OPTIONAL)
- persist if the scheduling has to be persisted. If the SmartExecutor instance terminate another SmartExecutor must take in charge that scheduling. Due to a scheduling can survive over the time is strongly recommend to declare the needed capabilities.
Please note that the capabilities matches are checked before launching the plugin execution. This is a service check and is conceptually different from filtering in instance discovery using the client.
Monitor Task Execution
the are two API to monitor the task execution.
Plugin Development
SmartExecutor plugins may have arbitrary size and dependencies but must include the following components:
Tasks
Note: Plugins are instantiated by the Executor and thus must have have a zero-argument constructor.
Development Environment and Testing
This [Eclipse project] may be used to kick-off plugin development and testing.