Tabular Data Flow Manager
A service for tabular data flow management. The aim is to provide a tool for support and management of tabular data flow between infrastructure services.
This document outlines the design rationale, key features, and high-level architecture, as well as the options deployment.
Overview
The goal of this service is to offer a central service for tabular data flow creation, management and monitoring. The data flow can touch different services in order to produce the desired output. Planned flow can be scheduled for deferred execution and the user notified about the flow progress.
Design
Philosophy
This represents an endpoint for users who want to process tabular data using the services offered in the infrastructure.
Architecture
The subsystem comprises the following components:
- Tabular Data Flow Service: the central system for the flow creation, management and monitoring;
- Tabular Data Flow UI: the user interface of the service where the user can create, execute and monitor the data flow;
- Tabular Data Agent: an helper component for the service that want to expose tabular data functionality to the data flow service.
A diagram of the relationships between these components is reported in the following figure:
Deployment
The Service should be deployed in a single node, while the agents should be deployed with the service that want to offer his functionality to the flow service. The User Interface can be deployed in the infrastructure portal.
Use Cases
Well suited Use Cases
This component well fit all the cases where is necessary to manage a flow of tabular data between the infrastructure services. An example can be the enhancement of catch statistics offered by the Time Series Service and elaborated using both the Statistical Service and the Occurrence Service.
