Difference between revisions of "ExecutionPlan Elements"
(→Boundary) |
(→Know issues) |
||
(31 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
=Overview= | =Overview= | ||
− | All plan elements are automatically assigned with an unique '''identifier''' through which they are retrievable and identified. Additionally, all elements can be assigned with a '''name''' that they use when emitting progress events. This way the execution monitoring can be made more user friendly. | + | All plan elements are automatically assigned with an unique '''identifier''', through which they are retrievable and identified. Additionally, all elements can be assigned with a '''name''' that they use when emitting progress events. This way the execution monitoring can be made more user friendly. |
− | The elements listed below can be roughly separated in two large categories based on the position they may have in the execution tree. | + | The elements listed below can be roughly separated in two large categories, based on the position they may have in the execution tree. |
*Nodes | *Nodes | ||
**[[ExecutionPlan_Elements#Sequence | Sequence]] | **[[ExecutionPlan_Elements#Sequence | Sequence]] | ||
Line 21: | Line 21: | ||
**[[ExecutionPlan_Elements#Shell_Script | Shell Script]] | **[[ExecutionPlan_Elements#Shell_Script | Shell Script]] | ||
All the ''Nodes'' elements have placeholders where other elements, either ''Nodes'' themselves or ''Leaves'' can be plugged in. The ''Leaves'' on the other hand can be plugged in other ''Nodes'' but the execution plan path ends to them. | All the ''Nodes'' elements have placeholders where other elements, either ''Nodes'' themselves or ''Leaves'' can be plugged in. The ''Leaves'' on the other hand can be plugged in other ''Nodes'' but the execution plan path ends to them. | ||
+ | |||
+ | All elements, while executing report on their progress, use the [[ExecutionEngine#Execution_Events | Events]] mechanism to inform the client on its state. For the elements that perform operations internal to the engine, this keeps a constant event flow reporting back to the client. The elements though that pass the execution to external elements, such as the [[ExecutionPlan_Elements#Web_Service | Web Service]], [[ExecutionPlan_Elements#Java_Object_.28POJO.29 | Java Object (POJO)]] and [[ExecutionPlan_Elements#Shell_Script | Shell Script]], when the execution is forwarded to these elements, this flow of information is interrupted. To cover this gap, the [[ExecutionEngine#Execution_Context | Execution Context]] construct is provided. | ||
=Plan Elements= | =Plan Elements= | ||
==Sequence== | ==Sequence== | ||
− | The ''Sequence'' construct offers a simple ordering utility for a set of execution steps. This element is a simple container maintaining the order of execution of its sub elements. When executed, the ordered list of sub elements is executed one after the other and the ''Sequence'' element is completed when its last sub-element is completed. | + | The ''Sequence'' construct offers a simple ordering utility for a set of execution steps. This element is a simple container maintaining the order of execution of its sub elements. When executed, the ordered list of sub elements is executed one after the other, and the ''Sequence'' element is completed when its last sub-element is completed. |
− | + | ||
+ | '''''XML definition''''' | ||
+ | |||
<source lang=xml> | <source lang=xml> | ||
<planElement id="element id" name="element name" type="Sequence"> | <planElement id="element id" name="element name" type="Sequence"> | ||
Line 38: | Line 42: | ||
==Flow== | ==Flow== | ||
− | The ''Flow'' construct offers a simple parallelization utility for a set of execution steps. This element is a simple container maintaining a set of sub elements that can be executed in parallel. When executed, the set of sub elements are executed all at once in parallel and the ''Flow'' | + | The ''Flow'' construct offers a simple parallelization utility for a set of execution steps. This element is a simple container, maintaining a set of sub elements that can be executed in parallel. When executed, the set of sub elements are executed all at once in parallel, and the ''Flow'' is completed when all the sub elements that have started are terminated. If one of the elements is terminated with an error, after all the elements have terminated, the exception indicating the error is re-thrown. If more than one elements finished with an error, only the first error found iterating through the set of sub elements is thrown. |
− | + | ||
+ | '''''XML definition''''' | ||
+ | |||
<source lang=xml> | <source lang=xml> | ||
<planElement id="element id" name="element name" type="Flow"> | <planElement id="element id" name="element name" type="Flow"> | ||
Line 52: | Line 58: | ||
==Bag== | ==Bag== | ||
− | The ''Bag'' construct acts as a container of other sub elements. It orchestrates their execution based on their execution preconditions. These elements as well as their conditions are evaluated every time one of the previously running elements have completed their execution. While this element is | + | The ''Bag'' construct acts as a container of other sub elements. It orchestrates their execution, based on their execution preconditions. These elements, as well as their conditions are evaluated every time one of the previously running elements have completed their execution. While this element is executed, every time an element previously running has terminated, the set of not yet run elements is scanned. If no element can be run while no more elements are still running, the process is terminated, if such a behavior is requested. Even if more elements are still running, if termination condition is set and is evaluated to true, the execution of the element is terminated. This condition follows the syntax defined by [[ExecutionPlan_Elements#Execution_Conditions | Execution Conditions]]. The execution preconditions of each element also follow the syntax defined by [[ExecutionPlan_Elements#Execution_Conditions | Execution Conditions]]. |
− | + | ||
+ | '''''XML definition''''' | ||
+ | |||
<source lang=xml> | <source lang=xml> | ||
<planElement id="element id" name="element name" type="Bag"> | <planElement id="element id" name="element name" type="Bag"> | ||
Line 72: | Line 80: | ||
</planElement> | </planElement> | ||
</source> | </source> | ||
+ | |||
===Know issues=== | ===Know issues=== | ||
− | It is possible that if it has not been requested for the ''Bag'' to terminate when no progress can be made, and the termination condition, if set, does not cover this case, that the ''Bag'' element will never terminate its execution. | + | It is possible that, if it has not been requested for the ''Bag'' to terminate when no progress can be made, and the termination condition, if set, does not cover this case, that the ''Bag'' element will never terminate its execution. |
− | Furthermore, currently the execution conditions of the bag's elements are only evaluated once at the beginning of the execution and then every time a previously running element terminates. This means that even if some execution condition is not dependent of some other element being terminated, it will not be discovered before that. | + | Furthermore, currently the execution conditions of the bag's elements are only evaluated once at the beginning of the execution and then every time a previously running element terminates. This means that, even if some execution condition is not dependent of some other element being terminated, it will not be discovered before that. |
==File Transfer== | ==File Transfer== | ||
Line 82: | Line 91: | ||
When the ''File Transfer'' direction is ''Retrieve'' it is also possible to define a ''MoveTo'' parameter that can dictate a new filename to rename the retrieved file to. Additionally, the permission attributes of the retrieved file can also be set. A simplified ''IsExecutable'' flag can be used to make the file executable, but the more flexible ''Permissions'' field can be used to set the full permissions of the file. For these permissions the common four digit UNIX format is used. | When the ''File Transfer'' direction is ''Retrieve'' it is also possible to define a ''MoveTo'' parameter that can dictate a new filename to rename the retrieved file to. Additionally, the permission attributes of the retrieved file can also be set. A simplified ''IsExecutable'' flag can be used to make the file executable, but the more flexible ''Permissions'' field can be used to set the full permissions of the file. For these permissions the common four digit UNIX format is used. | ||
− | + | '''''XML definition''''' | |
+ | |||
<source lang=xml> | <source lang=xml> | ||
<planElement id="element id" name="element name" type="FileTransfer"> | <planElement id="element id" name="element name" type="FileTransfer"> | ||
Line 100: | Line 110: | ||
==Conditional== | ==Conditional== | ||
− | The ''Conditional'' construct represents a set of flows that are only executed in case their respective conditions are met. The element follows the known pattern of '''if/else if/else''' construct. The if and else if flows are only executed in case their respective conditions are met. The else flow is only executed , if defined, if none of the if and else if flows have been executed. When the execution is started, the conditions of the flows are initialized and evaluated in turn until the first condition that is evaluated to true is found. The respective flow is executed. If none of the conditions are met, the else flow is executed if available. The conditions defined follow the syntax defined by [[ | + | The ''Conditional'' construct represents a set of flows that are only executed in case their respective conditions are met. The element follows the known pattern of '''if/else if/else''' construct. The if and else if flows are only executed in case their respective conditions are met. The else flow is only executed , if defined, if none of the if and else if flows have been executed. When the execution is started, the conditions of the flows are initialized and evaluated in turn until the first condition that is evaluated to true is found. The respective flow is executed. If none of the conditions are met, the else flow is executed if available. The conditions defined follow the syntax defined by [[ExecutionPlan_Elements#Execution_Conditions | Execution Conditions]]. |
+ | |||
+ | '''''XML definition''''' | ||
− | |||
<source lang=xml> | <source lang=xml> | ||
<planElement id="element id" name="element name" type="Conditional"> | <planElement id="element id" name="element name" type="Conditional"> | ||
Line 125: | Line 136: | ||
==Loop== | ==Loop== | ||
− | The ''Loop'' construct acts as a container of a sub plan which is executed iteratively until the loop condition defined is evaluated to false. If during the execution of the sub plan a ''break'' is caught, the loop is terminated without bubbling up the exception. The loop condition defined follows the syntax defined by [[ | + | The ''Loop'' construct acts as a container of a sub plan which is executed iteratively until the loop condition defined is evaluated to false. If during the execution of the sub plan a ''break'' is caught, the loop is terminated without bubbling up the exception. The loop condition defined follows the syntax defined by [[ExecutionPlan_Elements#Execution_Conditions | Execution Conditions]]. |
+ | |||
+ | '''''XML definition''''' | ||
− | |||
<source lang=xml> | <source lang=xml> | ||
<planElement id="element id" name="element name" type="Loop"> | <planElement id="element id" name="element name" type="Loop"> | ||
Line 138: | Line 150: | ||
The ''Break'' construct is simply used to be able to declare the termination of the execution before all its steps are completed or to break from a loop present somewhere higher in the hierarchy. | The ''Break'' construct is simply used to be able to declare the termination of the execution before all its steps are completed or to break from a loop present somewhere higher in the hierarchy. | ||
− | + | '''''XML definition''''' | |
+ | |||
<source lang=xml> | <source lang=xml> | ||
<planElement id="element id" name="element name" type="Break"> | <planElement id="element id" name="element name" type="Break"> | ||
Line 146: | Line 159: | ||
==Wait== | ==Wait== | ||
− | The ''Wait'' construct is simply used to pause the thread of execution that contains this element for a specific period of time. Although used by it self, the element can have some usability, it is expected to be used in conjunction with one of the condition enabled elements such as the [[ExecutionPlan_Elements#Loop]] element in such a way as to be able to define execution pausing until some condition is met. | + | The ''Wait'' construct is simply used to pause the thread of execution that contains this element for a specific period of time. Although used by it self, the element can have some usability, it is expected to be used in conjunction with one of the condition enabled elements such as the [[ExecutionPlan_Elements#Loop | Loop]] element in such a way as to be able to define execution pausing until some condition is met. |
+ | |||
+ | '''''XML definition''''' | ||
− | |||
<source lang=xml> | <source lang=xml> | ||
<planElement id="element id" name="element name" type="Wait"> | <planElement id="element id" name="element name" type="Wait"> | ||
Line 160: | Line 174: | ||
The matching of the error caught with the catch blocks is not performed as in typical programming languages such as for example Java. Error may be originating from some remote host where the original exception bubbled up to this construct was of some application specific type which is not available in the host evaluating this element. Therefore the [[ExecutionEngine]] cannot rely on strongly typed classes or types. To circumvent this problem, the matching is performed using the exception name. Either the fully qualified name or the simple name to have a more relaxed approach to error matching. A ''catch'' block may also specify an empty error name to act as a catch all. | The matching of the error caught with the catch blocks is not performed as in typical programming languages such as for example Java. Error may be originating from some remote host where the original exception bubbled up to this construct was of some application specific type which is not available in the host evaluating this element. Therefore the [[ExecutionEngine]] cannot rely on strongly typed classes or types. To circumvent this problem, the matching is performed using the exception name. Either the fully qualified name or the simple name to have a more relaxed approach to error matching. A ''catch'' block may also specify an empty error name to act as a catch all. | ||
− | + | '''''XML definition''''' | |
+ | |||
<source lang=xml> | <source lang=xml> | ||
<planElement id="element id" name="element name" type="TryCatchFinally"> | <planElement id="element id" name="element name" type="TryCatchFinally"> | ||
Line 183: | Line 198: | ||
The ''Checkpoint'' construct is meant to be used in order to create a restoration point for the data used in the context of an execution plan. These currently include only a set of [[gRS2_Locator | Proxy Locators]] that are expected as inputs by the construct. The [[gRS2_Store | Record Store]] is used to create a persistent copy of the set of inputs. Additional configuration parameters needed for the [[gRS2_Store | Record Store]] initialization are also part of the element's definition. | The ''Checkpoint'' construct is meant to be used in order to create a restoration point for the data used in the context of an execution plan. These currently include only a set of [[gRS2_Locator | Proxy Locators]] that are expected as inputs by the construct. The [[gRS2_Store | Record Store]] is used to create a persistent copy of the set of inputs. Additional configuration parameters needed for the [[gRS2_Store | Record Store]] initialization are also part of the element's definition. | ||
− | This construct also offers the possibility to react to errors that may happen during its execution using contingency reactions as these are described in [[ | + | This construct also offers the possibility to react to errors that may happen during its execution using contingency reactions as these are described in [[ExecutionEngine#Contingency_Triggers | Contingency Triggers]]. |
+ | |||
+ | '''''XML definition''''' | ||
− | |||
<source lang=xml> | <source lang=xml> | ||
<planElement id="element id" name="element name" type="Checkpoint"> | <planElement id="element id" name="element name" type="Checkpoint"> | ||
Line 215: | Line 231: | ||
==Processing== | ==Processing== | ||
− | The ''Processing'' construct acts as a processing step not directly relevant to any execution step. Its purpose is to enable "offline" processing of [[ExecutionPlan_Variables | Variables]] outside any other plan element as they would be normally declared. The [[ | + | The ''Processing'' construct acts as a processing step not directly relevant to any execution step. Its purpose is to enable "offline" processing of [[ExecutionPlan_Variables#Variables | Variables]] outside any other plan element as they would be normally declared. The [[ExecutionPlan_Variables#Parameters | Parameters]] that are used to retrieve inputs or store outputs in other plan elements are usually used for on the fly processing of the inputs and outputs. Some values however may be needed more than once for multiple elements. Some parameters may also have complex definitions and there may be an interest in simplifying the definition of a construct. |
− | The ''Processing'' construct is simply a placeholder for a number of [[ | + | The ''Processing'' construct is simply a placeholder for a number of [[ExecutionPlan_Variables#Parameters | Parameters]]. During execution, the list of [[ExecutionPlan_Variables#Parameters | Parameters]] are iterated and for each one the processing they define is invoked. The return values are not used, so to be of value, each parameter registered with this element should be of filtering type with an underlying filter that can store its output. |
+ | |||
+ | '''''XML definition''''' | ||
− | |||
<source lang=xml> | <source lang=xml> | ||
<planElement id="element id" name="element name" type="Filter"> | <planElement id="element id" name="element name" type="Filter"> | ||
Line 232: | Line 249: | ||
==Boundary== | ==Boundary== | ||
− | The ''Boundary'' construct represents a requirement that the sub plan that is defined under it is to be executed in a remote execution container. The access information for this remote container is specifically provided at definition time. During execution, the tree of execution under this element is moved to the remote execution container. Additionally to the sub plan moved to the remote host, the subset of the available [[ExecutionPlan_Variables | Variables]] that are needed by this sub plan are also moved. Additionally, files locally available to the hosting node that executes the initiating end of the transfer, can be moved as attachments to the newly instantiated node using the same connection. The execution in the remote container can be isolated so as any byproducts of the execution do not overlap with other executions that take place in parallel. Files that should be cleaned up after execution can also be declared. | + | The ''Boundary'' construct represents a requirement that the sub plan that is defined under it is to be executed in a remote execution container. The access information for this remote container is specifically provided at definition time. During execution, the tree of execution under this element is moved to the remote execution container. Additionally to the sub plan moved to the remote host, the subset of the available [[ExecutionPlan_Variables#Variables | Variables]] that are needed by this sub plan are also moved. Additionally, files locally available to the hosting node that executes the initiating end of the transfer, can be moved as attachments to the newly instantiated node using the same connection. The execution in the remote container can be isolated so as any byproducts of the execution do not overlap with other executions that take place in parallel. The isolation directory is automatically generated if requested Files that should be cleaned up after execution can also be declared. |
− | This construct also offers the possibility to react to errors that may happen during its execution using contingency reactions as these are described in [[ | + | This construct also offers the possibility to react to errors that may happen during its execution using contingency reactions as these are described in [[ExecutionEngine#Contingency_Triggers | Contingency Triggers]]. This means that in case of error a different execution container can be picked and the execution restarted there. |
− | After the execution in the remote execution container is finished, the [[ExecutionPlan_Variables | Variables]] that are modified by the sub plan are send back to the caller and are used to update the local copy. If the execution terminated with an error, the error is retrieved and re-thrown. | + | After the execution in the remote execution container is finished, the [[ExecutionPlan_Variables#Variables | Variables]] that are modified by the sub plan are send back to the caller and are used to update the local copy. If the execution terminated with an error, the error is retrieved and re-thrown. |
+ | |||
+ | '''''XML definition''''' | ||
+ | |||
+ | <source lang=xml> | ||
+ | <planElement id="element id" name="element name" type="Boundary"> | ||
+ | <triggers> | ||
+ | <contingency .../> | ||
+ | ... | ||
+ | </triggers> | ||
+ | <boundaryConfig hostname="..." port="..."> | ||
+ | <nozzleConfig broadcast="..." restrict="..." type="..."/> | ||
+ | </boundaryConfig> | ||
+ | <isolation cleanUp="..." isolate="..."> | ||
+ | <baseDir>...</baseDir> | ||
+ | </isolation> | ||
+ | <cleanup> | ||
+ | <file name="..."/> | ||
+ | ... | ||
+ | </cleanup> | ||
+ | <attachments> | ||
+ | <attachment type="..." cleanup="..."> | ||
+ | <value>...</value> | ||
+ | <restore>...</restore> | ||
+ | <permissions value="..."/> | ||
+ | </attachment> | ||
+ | ... | ||
+ | </attachments> | ||
+ | </planElement> | ||
+ | </source> | ||
− | |||
− | |||
==Shell Script== | ==Shell Script== | ||
+ | The ''Shel Script'' construct acts as an invocation wrapper of a shell script, or in general an external executable, enabling it to be directly included in the execution of a plan. The executable is defined using its absolute or relative path located in the execution container that is hosting the execution of the plan element. Any parameters it may require to be provided are specified in the arguments collection of the element. Additionally, any environmental variables that need to be present for the execution of the command can also be defined and set prior to the execution. The execution of the external executable is done using a ProcessBuilder. | ||
+ | |||
+ | It is also possible to define the data that will be written to the process's standard input. The standard output and standard error of the process can be retrieved by setting the respective parameters. These inputs and outputs can be retrieved / dumped from /to files by setting respective flags. | ||
+ | |||
+ | The exit code of the executable can be used to identify successful completion. A mapping of the possible exit values, or a negation of them can be used to identify specific errors or successful completion. Depending on the exit code, the element can be defined to throw specific exception to signal different types of errors thereby binding non Java error to the internal error handling facilities. | ||
+ | |||
+ | This construct also offers the possibility to react to errors that may happen during its execution using contingency reactions as these are described in [[ExecutionEngine#Contingency_Triggers | Contingency Triggers]]. | ||
+ | |||
+ | '''''XML definition''''' | ||
+ | |||
+ | <source lang=xml> | ||
+ | <planElement id="element id" name="element name" type="Checkpoint"> | ||
+ | <triggers> | ||
+ | <contingency .../> | ||
+ | ... | ||
+ | </triggers> | ||
+ | <command>...</command> | ||
+ | <arguments> | ||
+ | <attrParam isFile="..."> | ||
+ | <param ... /> | ||
+ | </attrParam> | ||
+ | ... | ||
+ | </arguments> | ||
+ | <stdIn isFile="..."> | ||
+ | <param ... /> | ||
+ | </stdIn> | ||
+ | <stdOut isFile="..."> | ||
+ | <param ... /> | ||
+ | </stdOut> | ||
+ | <stdErr isFile="..."> | ||
+ | <param ... /> | ||
+ | </stdErr> | ||
+ | <stdExit> | ||
+ | <param ... /> | ||
+ | </stdExit> | ||
+ | <exitCodeErrors> | ||
+ | <exitCodeError> | ||
+ | <exitCode value="..." type="..."/> | ||
+ | <errorFullName value="java.lang.Exception"/> | ||
+ | <errorSimpleName value="Exception"/> | ||
+ | <message value="..."/> | ||
+ | </exitCodeError> | ||
+ | ... | ||
+ | </exitCodeErrors> | ||
+ | <env> | ||
+ | <item key="...">...</item> | ||
+ | ... | ||
+ | </env> | ||
+ | </planElement> | ||
+ | </source> | ||
+ | |||
+ | ==Java Object (POJO)== | ||
+ | The ''Java Object'' construct acts as an invocation wrapper of a Java object enabling it to be directly included in the execution of a plan. The Java class that is to be used is specified using its full name and is accessed using reflection. The list of actual invocations on this class are specified declaratively. | ||
+ | |||
+ | The list of invocations is organized as an ordered list, and the invocations may be either constructor calls or method invocations. The method invocations can be made to both member functions as well as static methods. The values provided as arguments to the method invocations are retrieved from [[ExecutionPlan_Variables#Parameters | Parameters]] defined at the level of plan. Any return value of method invocation can also be stored as a [[ExecutionPlan_Variables#Variables | Variable]] using a [[ExecutionPlan_Variables#Parameters | Parameter]]. | ||
+ | |||
+ | Every time an instance of the class is created because the invocation type of call is a constructor, if it is so requested and the instantiated object supports it, an [[ExecutionEngine#Execution_Context | Execution Context]] is created and provided to the instance. | ||
+ | |||
+ | This construct also offers the possibility to react to errors that may happen during its execution using contingency reactions as these are described in [[ExecutionEngine#Contingency_Triggers | Contingency Triggers]]. | ||
+ | |||
+ | '''''XML definition''''' | ||
+ | |||
+ | <source lang=xml> | ||
+ | <planElement id="element id" name="element name" type="POJO"> | ||
+ | <triggers> | ||
+ | <contingency .../> | ||
+ | ... | ||
+ | </triggers> | ||
+ | <className value="..."/> | ||
+ | <calls> | ||
+ | <call type="..." order="..."> | ||
+ | <methodName value="..."/> | ||
+ | <arguments> | ||
+ | <argument type="..." order="..." name="..."> | ||
+ | <param ... /> | ||
+ | </argument> | ||
+ | ... | ||
+ | </arguments> | ||
+ | <output> | ||
+ | <param ... /> | ||
+ | </output> | ||
+ | </call> | ||
+ | ... | ||
+ | </calls> | ||
+ | <context supported="..."> | ||
+ | <contextConfig type="..." proxytype="..." keepAlive="..."> | ||
+ | <config random="..." authenticate="..." encrypt="..." protocol="..."> | ||
+ | <ports> | ||
+ | <range start="..." end="..."/> | ||
+ | ... | ||
+ | <ports> | ||
+ | </config> | ||
+ | </contextConfig> | ||
+ | </context> | ||
+ | </planElement> | ||
+ | </source> | ||
+ | ==Web Service== | ||
+ | The ''Web Service'' construct acts as an invocation wrapper of a web service enabling it to be directly included in the execution of a plan. The web service that is to be used is specified using its end point. The list of actual invocations on this class are specified declaratively. | ||
+ | |||
+ | The list of invocations is organized as an ordered list. Every method invocation is performed based on the method name, and the action urn. A single argument is expected for each method which is the full SOAP envelop of the execution. This envelop is provided as a [[ExecutionPlan_Variables#Parameters | Parameter]] which means that for its production a number of processing steps can be made in order for it to be constructed. For each invocation, a number of post envelop creation [[ExecutionPlan_Variables#Filters | Filters]] can also be defined. After the SOAP envelop is created, they are are used to further process the envelop before the invocation. Any return value of a method invocation can be stored as a [[ExecutionPlan_Variables#Variables | Variable]] using a [[ExecutionPlan_Variables#Parameters | Parameter]]. | ||
+ | |||
+ | Every time an invocation to the web service is made, if it is so requested, an [[ExecutionEngine#Execution_Context | Execution Context]] is created and provided. This context is supplied as a member of the SOAP header and injected using the post envelop creation filters. | ||
+ | |||
+ | This construct also offers the possibility to react to errors that may happen during its execution using contingency reactions as these are described in [[ExecutionEngine#Contingency_Triggers | Contingency Triggers]]. This means that in case of error a different end point can be picked and the execution restarted there. | ||
+ | |||
+ | '''''XML definition''''' | ||
+ | |||
+ | <source lang=xml> | ||
+ | <planElement id="element id" name="element name" type="WS"> | ||
+ | <triggers> | ||
+ | <contingency .../> | ||
+ | ... | ||
+ | </triggers> | ||
+ | <endpoint>...</endpoint> | ||
+ | <calls> | ||
+ | <call type="..." order="..."> | ||
+ | <actionURN>...</actionURN> | ||
+ | <executionContextToken>...</executionContextToken> | ||
+ | <methodName value="..."/> | ||
+ | <arguments> | ||
+ | <argument type="..." order="..." name="..."> | ||
+ | <param ... /> | ||
+ | </argument> | ||
+ | </arguments> | ||
+ | <output> | ||
+ | <param ... /> | ||
+ | </output> | ||
+ | <postFilters> | ||
+ | <filter ... /> | ||
+ | ... | ||
+ | </postFilters> | ||
+ | </call> | ||
+ | ... | ||
+ | </calls> | ||
+ | <context supported="..."> | ||
+ | <contextConfig type="..." proxytype="..." keepAlive="..."> | ||
+ | <tcpProxyConfig random="..." authenticate="..." encrypt="..." protocol="..."> | ||
+ | <ports> | ||
+ | <range start="..." end="..."/> | ||
+ | ... | ||
+ | <ports> | ||
+ | </tcpProxyConfig> | ||
+ | <nozzleConfig type="..." broadcast="..." restrict="..."/> | ||
+ | </contextConfig> | ||
+ | </context> | ||
+ | </planElement> | ||
+ | </source> | ||
+ | |||
+ | =Execution Conditions= | ||
+ | One of the constructs that makes the [[ExecutionEngine#Execution_Plan | Execution Plan]] versatile is the capability of defining run time conditions which are evaluated against run time values. A condition is a boolean decision tree. This tree supports two types of nodes | ||
+ | *Nodes | ||
+ | *:A node can have a number of children which can be either other Nodes or Leaves. These children are combined with a verb which can be either a boolean AND operator or a boolean OR operator. There is also the possibility of post processing the result of the cumulative boolean operator by adding a negation over the result. | ||
+ | *Leaves | ||
+ | *:A leaf contains one of the supported conditions that are evaluated against a number of defined [[ExecutionPlan_Variables#Parameters | Parameters]]. The currently supported conditions are the following: | ||
+ | **Boolean Value | ||
+ | **:This condition simply checks the value retrieved by an input [[ExecutionPlan_Variables#Parameters | parameter]] which it expects to be of boolean value and returns this value. | ||
+ | **Counter | ||
+ | **:This condition once initialized starts a counter like behavior, and every time it is called to evaluate its condition, will increment this counter by some defined value (increasing or decreasing it). While this counter is within some defined boundaries, inclusive or not, it returns true. Once the counter has moved beyond these boundaries, the condition evaluates to false. | ||
+ | **Range | ||
+ | **:This condition checks if some [[ExecutionPlan_Variables#Parameters | parameter]]'s value is within a defined range of values. If so, it returns true. Otherwise the condition evaluates to false | ||
+ | **Timeout | ||
+ | **:This condition once initialized marks the current time. Every time it is requested to evaluate its condition it will check if the time past since its initialization is within some defined period. If it is, true is returned. Otherwise false. | ||
+ | **Value Available | ||
+ | **:This condition checks if the defined [[ExecutionPlan_Variables#Variables | Variable]] is marked as available, and has its value initialized or not. if it is, true is returned. Otherwise, false. | ||
+ | |||
+ | Most of the values that take part in the above conditions are accessed through [[ExecutionPlan_Variables#Variables | Variables]] and [[ExecutionPlan_Variables#Parameters | Parameters]]. This way, the behavior and parametrization of these conditions are subject to dynamic configuration and they may in fact be instantiated from some previous step of the execution. | ||
+ | |||
+ | '''''XML definition''''' | ||
+ | |||
+ | <source lang=xml> | ||
+ | <conditionTree> | ||
+ | <treeElement type="Node" verb="AND" post="AsIs"> | ||
+ | <treeElement type="Node" verb="OR" post="Negate"> | ||
+ | <treeElement type="Leaf"> | ||
+ | <condition type="Timeout"> | ||
+ | <threshold value="some timeout in milliseconds"/> | ||
+ | </condition> | ||
+ | </treeElement> | ||
+ | <treeElement type="Leaf"> | ||
+ | <condition type="IsAvailable"> | ||
+ | <var name="variable name"/> | ||
+ | </condition> | ||
+ | </treeElement> | ||
+ | <treeElement type="Leaf"> | ||
+ | <condition type="DecimalRange"> | ||
+ | <start> | ||
+ | <param ... /> | ||
+ | </start> | ||
+ | <end> | ||
+ | <param ... /> | ||
+ | </end> | ||
+ | <current> | ||
+ | <param ... /> | ||
+ | </current> | ||
+ | <leftinclusive value="true"/> | ||
+ | <rightinclusive value="false"/> | ||
+ | </condition> | ||
+ | </treeElement> | ||
+ | </treeElement> | ||
+ | <treeElement type="Leaf"> | ||
+ | <condition type="BooleanVariable"> | ||
+ | <flag> | ||
+ | <param .../> | ||
+ | </flag> | ||
+ | </condition> | ||
+ | </treeElement> | ||
+ | <treeElement type="Leaf"> | ||
+ | <condition type="Counter"> | ||
+ | <start> | ||
+ | <param .../> | ||
+ | </start> | ||
+ | <end> | ||
+ | <param .../> | ||
+ | </end> | ||
+ | <current> | ||
+ | <param .../> | ||
+ | </current> | ||
+ | <increment> | ||
+ | <param .../> | ||
+ | </increment> | ||
+ | <rightinclusive value="true"/> | ||
+ | </condition> | ||
+ | </treeElement> | ||
+ | </treeElement> | ||
+ | </conditionTree> | ||
+ | </source> |
Latest revision as of 13:52, 28 September 2011
Contents
Overview
All plan elements are automatically assigned with an unique identifier, through which they are retrievable and identified. Additionally, all elements can be assigned with a name that they use when emitting progress events. This way the execution monitoring can be made more user friendly.
The elements listed below can be roughly separated in two large categories, based on the position they may have in the execution tree.
- Nodes
- Leaves
All the Nodes elements have placeholders where other elements, either Nodes themselves or Leaves can be plugged in. The Leaves on the other hand can be plugged in other Nodes but the execution plan path ends to them.
All elements, while executing report on their progress, use the Events mechanism to inform the client on its state. For the elements that perform operations internal to the engine, this keeps a constant event flow reporting back to the client. The elements though that pass the execution to external elements, such as the Web Service, Java Object (POJO) and Shell Script, when the execution is forwarded to these elements, this flow of information is interrupted. To cover this gap, the Execution Context construct is provided.
Plan Elements
Sequence
The Sequence construct offers a simple ordering utility for a set of execution steps. This element is a simple container maintaining the order of execution of its sub elements. When executed, the ordered list of sub elements is executed one after the other, and the Sequence element is completed when its last sub-element is completed.
XML definition
<planElement id="element id" name="element name" type="Sequence"> <list> <planElement .../> <planElement .../> <planElement .../> ... </list> </planElement>
Flow
The Flow construct offers a simple parallelization utility for a set of execution steps. This element is a simple container, maintaining a set of sub elements that can be executed in parallel. When executed, the set of sub elements are executed all at once in parallel, and the Flow is completed when all the sub elements that have started are terminated. If one of the elements is terminated with an error, after all the elements have terminated, the exception indicating the error is re-thrown. If more than one elements finished with an error, only the first error found iterating through the set of sub elements is thrown.
XML definition
<planElement id="element id" name="element name" type="Flow"> <list> <planElement .../> <planElement .../> <planElement .../> ... </list> </planElement>
Bag
The Bag construct acts as a container of other sub elements. It orchestrates their execution, based on their execution preconditions. These elements, as well as their conditions are evaluated every time one of the previously running elements have completed their execution. While this element is executed, every time an element previously running has terminated, the set of not yet run elements is scanned. If no element can be run while no more elements are still running, the process is terminated, if such a behavior is requested. Even if more elements are still running, if termination condition is set and is evaluated to true, the execution of the element is terminated. This condition follows the syntax defined by Execution Conditions. The execution preconditions of each element also follow the syntax defined by Execution Conditions.
XML definition
<planElement id="element id" name="element name" type="Bag"> <list> <bagElement> <planElement ... /> <conditionTree .../> </bagElement> <bagElement> <planElement ... /> <conditionTree .../> </bagElement> ... </list> <termination onNoProgress="true"> <conditionTree .../> </termination> </planElement>
Know issues
It is possible that, if it has not been requested for the Bag to terminate when no progress can be made, and the termination condition, if set, does not cover this case, that the Bag element will never terminate its execution.
Furthermore, currently the execution conditions of the bag's elements are only evaluated once at the beginning of the execution and then every time a previously running element terminates. This means that, even if some execution condition is not dependent of some other element being terminated, it will not be discovered before that.
File Transfer
The File Tranfer construct can be used to mediate at the execution level the staging of files to and from the StorageSystem. The transfer can have a direction of either Store or Retrieve. In case the direction is Store, the File Tranfer utility expects as input the file path that is to be stored in the StorageSystem and is located in the same hosting machine as the one executing the element. After the operation is completed, the output of the File Transfer will be the identifier assigned to the stored file by the StorageSystem. In case the direction is Retrieve, the input is expected to be the identifier of the document that is stored in the StorageSystem and after the execution, the output of Filte Transfer will have the local filename where the document is stored.
When the File Transfer direction is Retrieve it is also possible to define a MoveTo parameter that can dictate a new filename to rename the retrieved file to. Additionally, the permission attributes of the retrieved file can also be set. A simplified IsExecutable flag can be used to make the file executable, but the more flexible Permissions field can be used to set the full permissions of the file. For these permissions the common four digit UNIX format is used.
XML definition
<planElement id="element id" name="element name" type="FileTransfer"> <input ... /> <output ... /> <direction value="Store"/> </planElement> <planElement id="element id" name="element name" isexec="true" type="FileTransfer"> <input ... /> <output ... /> <direction value="Retrieve"/> <mvto ... /> <perms value="0700"/> </planElement>
Conditional
The Conditional construct represents a set of flows that are only executed in case their respective conditions are met. The element follows the known pattern of if/else if/else construct. The if and else if flows are only executed in case their respective conditions are met. The else flow is only executed , if defined, if none of the if and else if flows have been executed. When the execution is started, the conditions of the flows are initialized and evaluated in turn until the first condition that is evaluated to true is found. The respective flow is executed. If none of the conditions are met, the else flow is executed if available. The conditions defined follow the syntax defined by Execution Conditions.
XML definition
<planElement id="element id" name="element name" type="Conditional"> <if> <conditionalFlow> <conditionTree ... /> <planElement ... /> </conditionalFlow> </if> <elif> <conditionalFlow> <conditionTree ... /> <planElement ... /> </conditionalFlow> ... </elif> <else> <planElement ... /> </else> </planElement>
Loop
The Loop construct acts as a container of a sub plan which is executed iteratively until the loop condition defined is evaluated to false. If during the execution of the sub plan a break is caught, the loop is terminated without bubbling up the exception. The loop condition defined follows the syntax defined by Execution Conditions.
XML definition
<planElement id="element id" name="element name" type="Loop"> <conditionTree ... /> <planElement ... /> </planElement>
Break
The Break construct is simply used to be able to declare the termination of the execution before all its steps are completed or to break from a loop present somewhere higher in the hierarchy.
XML definition
<planElement id="element id" name="element name" type="Break"> <message>...</message> </planElement>
Wait
The Wait construct is simply used to pause the thread of execution that contains this element for a specific period of time. Although used by it self, the element can have some usability, it is expected to be used in conjunction with one of the condition enabled elements such as the Loop element in such a way as to be able to define execution pausing until some condition is met.
XML definition
<planElement id="element id" name="element name" type="Wait"> <wait value="..."/> </planElement>
Try Catch Finally
The Try Catch Finally construct acts as the common programmatic try/catch/finally block of execution. During execution the sub element that is declared in the try block is executed. In case an exception is caught, the catch blocks are iterated to match the exception caught with one of the handled ones. If some catch block can handle the exception, the sub element defined in that block is executed. If the selected catch block does not define that the exception is re-thrown after the execution of the compensation code, the execution is moved to the finally block sub element. If however the exception is to be re-thrown or a new exception is thrown from the compensation code, the exception is bubbled up after the finally block is executed.
The matching of the error caught with the catch blocks is not performed as in typical programming languages such as for example Java. Error may be originating from some remote host where the original exception bubbled up to this construct was of some application specific type which is not available in the host evaluating this element. Therefore the ExecutionEngine cannot rely on strongly typed classes or types. To circumvent this problem, the matching is performed using the exception name. Either the fully qualified name or the simple name to have a more relaxed approach to error matching. A catch block may also specify an empty error name to act as a catch all.
XML definition
<planElement id="element id" name="element name" type="TryCatchFinally"> <try> <planElement ... /> </try> <catch> <catch name="java.lang.SecurityException" isFullName="true" rethrow="true"> <planElement ... /> </catch> <catch rethrow="false"> <planElement ... /> </catch> </catch> <finally> <planElement ... /> </finally> </planElement>
Checkpoint
The Checkpoint construct is meant to be used in order to create a restoration point for the data used in the context of an execution plan. These currently include only a set of Proxy Locators that are expected as inputs by the construct. The Record Store is used to create a persistent copy of the set of inputs. Additional configuration parameters needed for the Record Store initialization are also part of the element's definition.
This construct also offers the possibility to react to errors that may happen during its execution using contingency reactions as these are described in Contingency Triggers.
XML definition
<planElement id="element id" name="element name" type="Checkpoint"> <triggers> <contingency .../> ... </triggers> <output ... /> <inputs> ... </inputs> <checkpointConfig bufferedEntries="..." depthType="OnRequest" emptyBufferThresholdFactor="..." inactiveGracePeriodInMilliseconds="..." leasingAccessCount="..." millisToWaitBeforeTimeout="..." multiplexType="..." typeOfLocator="..."> <proxyConfig authenticate="..." encrypt="..." protocol="..." random="..."> <ports> <range start="..." end="..."/> ... </ports> </proxyConfig> <storeConfig random="true"> <ports> <range start="..." end="..."/> ... <ports> </storeConfig> </checkpointConfig> </planElement>
Processing
The Processing construct acts as a processing step not directly relevant to any execution step. Its purpose is to enable "offline" processing of Variables outside any other plan element as they would be normally declared. The Parameters that are used to retrieve inputs or store outputs in other plan elements are usually used for on the fly processing of the inputs and outputs. Some values however may be needed more than once for multiple elements. Some parameters may also have complex definitions and there may be an interest in simplifying the definition of a construct.
The Processing construct is simply a placeholder for a number of Parameters. During execution, the list of Parameters are iterated and for each one the processing they define is invoked. The return values are not used, so to be of value, each parameter registered with this element should be of filtering type with an underlying filter that can store its output.
XML definition
<planElement id="element id" name="element name" type="Filter"> <list> <param .../> <param .../> <param .../> ... </list> </planElement>
Boundary
The Boundary construct represents a requirement that the sub plan that is defined under it is to be executed in a remote execution container. The access information for this remote container is specifically provided at definition time. During execution, the tree of execution under this element is moved to the remote execution container. Additionally to the sub plan moved to the remote host, the subset of the available Variables that are needed by this sub plan are also moved. Additionally, files locally available to the hosting node that executes the initiating end of the transfer, can be moved as attachments to the newly instantiated node using the same connection. The execution in the remote container can be isolated so as any byproducts of the execution do not overlap with other executions that take place in parallel. The isolation directory is automatically generated if requested Files that should be cleaned up after execution can also be declared.
This construct also offers the possibility to react to errors that may happen during its execution using contingency reactions as these are described in Contingency Triggers. This means that in case of error a different execution container can be picked and the execution restarted there.
After the execution in the remote execution container is finished, the Variables that are modified by the sub plan are send back to the caller and are used to update the local copy. If the execution terminated with an error, the error is retrieved and re-thrown.
XML definition
<planElement id="element id" name="element name" type="Boundary"> <triggers> <contingency .../> ... </triggers> <boundaryConfig hostname="..." port="..."> <nozzleConfig broadcast="..." restrict="..." type="..."/> </boundaryConfig> <isolation cleanUp="..." isolate="..."> <baseDir>...</baseDir> </isolation> <cleanup> <file name="..."/> ... </cleanup> <attachments> <attachment type="..." cleanup="..."> <value>...</value> <restore>...</restore> <permissions value="..."/> </attachment> ... </attachments> </planElement>
Shell Script
The Shel Script construct acts as an invocation wrapper of a shell script, or in general an external executable, enabling it to be directly included in the execution of a plan. The executable is defined using its absolute or relative path located in the execution container that is hosting the execution of the plan element. Any parameters it may require to be provided are specified in the arguments collection of the element. Additionally, any environmental variables that need to be present for the execution of the command can also be defined and set prior to the execution. The execution of the external executable is done using a ProcessBuilder.
It is also possible to define the data that will be written to the process's standard input. The standard output and standard error of the process can be retrieved by setting the respective parameters. These inputs and outputs can be retrieved / dumped from /to files by setting respective flags.
The exit code of the executable can be used to identify successful completion. A mapping of the possible exit values, or a negation of them can be used to identify specific errors or successful completion. Depending on the exit code, the element can be defined to throw specific exception to signal different types of errors thereby binding non Java error to the internal error handling facilities.
This construct also offers the possibility to react to errors that may happen during its execution using contingency reactions as these are described in Contingency Triggers.
XML definition
<planElement id="element id" name="element name" type="Checkpoint"> <triggers> <contingency .../> ... </triggers> <command>...</command> <arguments> <attrParam isFile="..."> <param ... /> </attrParam> ... </arguments> <stdIn isFile="..."> <param ... /> </stdIn> <stdOut isFile="..."> <param ... /> </stdOut> <stdErr isFile="..."> <param ... /> </stdErr> <stdExit> <param ... /> </stdExit> <exitCodeErrors> <exitCodeError> <exitCode value="..." type="..."/> <errorFullName value="java.lang.Exception"/> <errorSimpleName value="Exception"/> <message value="..."/> </exitCodeError> ... </exitCodeErrors> <env> <item key="...">...</item> ... </env> </planElement>
Java Object (POJO)
The Java Object construct acts as an invocation wrapper of a Java object enabling it to be directly included in the execution of a plan. The Java class that is to be used is specified using its full name and is accessed using reflection. The list of actual invocations on this class are specified declaratively.
The list of invocations is organized as an ordered list, and the invocations may be either constructor calls or method invocations. The method invocations can be made to both member functions as well as static methods. The values provided as arguments to the method invocations are retrieved from Parameters defined at the level of plan. Any return value of method invocation can also be stored as a Variable using a Parameter.
Every time an instance of the class is created because the invocation type of call is a constructor, if it is so requested and the instantiated object supports it, an Execution Context is created and provided to the instance.
This construct also offers the possibility to react to errors that may happen during its execution using contingency reactions as these are described in Contingency Triggers.
XML definition
<planElement id="element id" name="element name" type="POJO"> <triggers> <contingency .../> ... </triggers> <className value="..."/> <calls> <call type="..." order="..."> <methodName value="..."/> <arguments> <argument type="..." order="..." name="..."> <param ... /> </argument> ... </arguments> <output> <param ... /> </output> </call> ... </calls> <context supported="..."> <contextConfig type="..." proxytype="..." keepAlive="..."> <config random="..." authenticate="..." encrypt="..." protocol="..."> <ports> <range start="..." end="..."/> ... <ports> </config> </contextConfig> </context> </planElement>
Web Service
The Web Service construct acts as an invocation wrapper of a web service enabling it to be directly included in the execution of a plan. The web service that is to be used is specified using its end point. The list of actual invocations on this class are specified declaratively.
The list of invocations is organized as an ordered list. Every method invocation is performed based on the method name, and the action urn. A single argument is expected for each method which is the full SOAP envelop of the execution. This envelop is provided as a Parameter which means that for its production a number of processing steps can be made in order for it to be constructed. For each invocation, a number of post envelop creation Filters can also be defined. After the SOAP envelop is created, they are are used to further process the envelop before the invocation. Any return value of a method invocation can be stored as a Variable using a Parameter.
Every time an invocation to the web service is made, if it is so requested, an Execution Context is created and provided. This context is supplied as a member of the SOAP header and injected using the post envelop creation filters.
This construct also offers the possibility to react to errors that may happen during its execution using contingency reactions as these are described in Contingency Triggers. This means that in case of error a different end point can be picked and the execution restarted there.
XML definition
<planElement id="element id" name="element name" type="WS"> <triggers> <contingency .../> ... </triggers> <endpoint>...</endpoint> <calls> <call type="..." order="..."> <actionURN>...</actionURN> <executionContextToken>...</executionContextToken> <methodName value="..."/> <arguments> <argument type="..." order="..." name="..."> <param ... /> </argument> </arguments> <output> <param ... /> </output> <postFilters> <filter ... /> ... </postFilters> </call> ... </calls> <context supported="..."> <contextConfig type="..." proxytype="..." keepAlive="..."> <tcpProxyConfig random="..." authenticate="..." encrypt="..." protocol="..."> <ports> <range start="..." end="..."/> ... <ports> </tcpProxyConfig> <nozzleConfig type="..." broadcast="..." restrict="..."/> </contextConfig> </context> </planElement>
Execution Conditions
One of the constructs that makes the Execution Plan versatile is the capability of defining run time conditions which are evaluated against run time values. A condition is a boolean decision tree. This tree supports two types of nodes
- Nodes
- A node can have a number of children which can be either other Nodes or Leaves. These children are combined with a verb which can be either a boolean AND operator or a boolean OR operator. There is also the possibility of post processing the result of the cumulative boolean operator by adding a negation over the result.
- Leaves
- A leaf contains one of the supported conditions that are evaluated against a number of defined Parameters. The currently supported conditions are the following:
- Boolean Value
- This condition simply checks the value retrieved by an input parameter which it expects to be of boolean value and returns this value.
- Counter
- This condition once initialized starts a counter like behavior, and every time it is called to evaluate its condition, will increment this counter by some defined value (increasing or decreasing it). While this counter is within some defined boundaries, inclusive or not, it returns true. Once the counter has moved beyond these boundaries, the condition evaluates to false.
- Range
- This condition checks if some parameter's value is within a defined range of values. If so, it returns true. Otherwise the condition evaluates to false
- Timeout
- This condition once initialized marks the current time. Every time it is requested to evaluate its condition it will check if the time past since its initialization is within some defined period. If it is, true is returned. Otherwise false.
- Value Available
- This condition checks if the defined Variable is marked as available, and has its value initialized or not. if it is, true is returned. Otherwise, false.
Most of the values that take part in the above conditions are accessed through Variables and Parameters. This way, the behavior and parametrization of these conditions are subject to dynamic configuration and they may in fact be instantiated from some previous step of the execution.
XML definition
<conditionTree> <treeElement type="Node" verb="AND" post="AsIs"> <treeElement type="Node" verb="OR" post="Negate"> <treeElement type="Leaf"> <condition type="Timeout"> <threshold value="some timeout in milliseconds"/> </condition> </treeElement> <treeElement type="Leaf"> <condition type="IsAvailable"> <var name="variable name"/> </condition> </treeElement> <treeElement type="Leaf"> <condition type="DecimalRange"> <start> <param ... /> </start> <end> <param ... /> </end> <current> <param ... /> </current> <leftinclusive value="true"/> <rightinclusive value="false"/> </condition> </treeElement> </treeElement> <treeElement type="Leaf"> <condition type="BooleanVariable"> <flag> <param .../> </flag> </condition> </treeElement> <treeElement type="Leaf"> <condition type="Counter"> <start> <param .../> </start> <end> <param .../> </end> <current> <param .../> </current> <increment> <param .../> </increment> <rightinclusive value="true"/> </condition> </treeElement> </treeElement> </conditionTree>