Data Transfer 2

Overview

The implementation of a reliable data transfer mechanisms between the nodes of a gCube-based Hybrid Data Infrastructure is one of the main objectives when dealing with large set of multi-type datasets distributed across different repositories.

To promote an efficient and optimized consumption of these data resources, a number of components have been designed to meet the data transfer requirements.

This document outlines the design rationale, key features, and high-level architecture, the options for their deployment and as well some use cases.

Key features

The components belonging to this class are responsible for:

Point to Point transfer

direct transfer invocation to a gCube Node

reliable data transfer between Infrastructure Data Sources and Data Storages

by exploiting the uniform access interfaces provided by gCube and standard transfer protocols

automatic transfer optimization

by exploiting best available transfer options between invoker and target nodes

advanced and extensible post transfer processing

plugin - oriented implementation to serve advanced use case