Difference between revisions of "StorageHub REST API"
(→DataMiner and SAI Interactions) |
|||
(35 intermediate revisions by 3 users not shown) | |||
Line 28: | Line 28: | ||
* '''Folder Listing''': to list the content of a folder; | * '''Folder Listing''': to list the content of a folder; | ||
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token; | * '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token; | ||
+ | * '''Find''': to find a file or a folder by name or pattern; | ||
* '''Delete''': to remove a file or a folder (including subfolders); | * '''Delete''': to remove a file or a folder (including subfolders); | ||
* '''Download''': to download a file or a folder in ZIP format; | * '''Download''': to download a file or a folder in ZIP format; | ||
Line 34: | Line 35: | ||
* '''Unzip''': to upload a zip file in a specific folder; | * '''Unzip''': to upload a zip file in a specific folder; | ||
* '''Upload file''': to upload a file in a folder. | * '''Upload file''': to upload a file in a folder. | ||
+ | * '''Versions''': to get a specific version of a file; | ||
== API == | == API == | ||
Line 43: | Line 45: | ||
=== Java === | === Java === | ||
<source lang="java"> | <source lang="java"> | ||
+ | import org.gcube.common.storagehub.client.dsl.StorageHubClient; | ||
+ | import org.gcube.common.storagehub.client.dsl.FileContainer; | ||
+ | import org.gcube.common.storagehub.model.items.Item; | ||
+ | |||
+ | |||
StorageHubClient shc = new StorageHubClient(); | StorageHubClient shc = new StorageHubClient(); | ||
− | FolderContainer | + | FolderContainer rootContainer = shc.getWSRoot() |
</source> | </source> | ||
Line 55: | Line 62: | ||
200 The workspace root item (in json format) is returned. | 200 The workspace root item (in json format) is returned. | ||
− | + | 500 The error is specified in the body of the response message | |
+ | |||
+ | == Get Item ById == | ||
+ | |||
+ | === Java === | ||
+ | <source lang="java"> | ||
+ | import org.gcube.common.storagehub.client.dsl.FileContainer; | ||
+ | import org.gcube.common.storagehub.client.dsl.StorageHubClient; | ||
+ | import org.gcube.common.storagehub.model.items.Item; | ||
+ | |||
+ | ... | ||
+ | |||
+ | String theId = {itemIdentifier} // the identifier of the item | ||
+ | StorageHubClient shc = new StorageHubClient(); | ||
+ | FileContainer fileContainer = shc.open(theId).asFile(); | ||
+ | </source> | ||
+ | |||
+ | == List Item Versions, Get Item Version == | ||
+ | |||
+ | === Java === | ||
+ | <source lang="java"> | ||
+ | import org.gcube.common.storagehub.client.dsl.FileContainer; | ||
+ | import org.gcube.common.storagehub.client.dsl.StorageHubClient; | ||
+ | import org.gcube.common.storagehub.model.items.Item; | ||
+ | import org.gcube.common.storagehub.model.service.Version; | ||
+ | |||
+ | ... | ||
+ | |||
+ | String theId = {itemIdentifier} // the identifier of the item | ||
+ | StorageHubClient shc = new StorageHubClient(); | ||
+ | FileContainer fileContainer = shc.open(theId).asFile(); | ||
+ | List<Version> fileVersions = fileContainer.getVersions(); | ||
+ | |||
+ | //to download a version | ||
+ | fileContainer.downloadSpecificVersion(versionName) | ||
+ | </source> | ||
== Folder Listing == | == Folder Listing == | ||
Line 71: | Line 113: | ||
=== REST API === | === REST API === | ||
− | GET /workspace/items/{ | + | GET /workspace/items/{folder-identifier}/children?gcube-token={userToken} |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
==== responses ==== | ==== responses ==== | ||
Line 81: | Line 119: | ||
200 the list of the Items (in JSON format) | 200 the list of the Items (in JSON format) | ||
− | + | 500 The error is specified in the body of the response message | |
== Retrieve VRE Folder == | == Retrieve VRE Folder == | ||
Line 101: | Line 139: | ||
200 The VRE folder root item (in json format) is returned. | 200 The VRE folder root item (in json format) is returned. | ||
− | + | 500 The error is specified in the body of the response message | |
+ | == Create Folder == | ||
− | == DataMiner and SAI Interactions | + | Creates a new folder under another folder (specified by its id) |
− | Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by DataMiner can interact with StorageHub through the StorageHub REST APIs. | + | |
+ | === Java === | ||
+ | <source lang="java"> | ||
+ | StorageHubClient shc = new StorageHubClient(); | ||
+ | FolderContainer root = shc.getWSRoot(); | ||
+ | //Creating the folder on the root workspace folder | ||
+ | root.newFolder("name", "description"); | ||
+ | </source> | ||
+ | |||
+ | === REST API === | ||
+ | |||
+ | POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} | ||
+ | Content-Type: application/x-www-form-urlencoded | ||
+ | {String name, String description, boolean hidden} | ||
+ | |||
+ | |||
+ | ==== responses ==== | ||
+ | |||
+ | 200 The Folder item identifier is returned. | ||
+ | |||
+ | 500 The error is specified in the body of the response message | ||
+ | |||
+ | == Upload File == | ||
+ | |||
+ | Creates a new file under a folder (specified by its id) | ||
+ | |||
+ | === Java === | ||
+ | <source lang="java"> | ||
+ | StorageHubClient shc = new StorageHubClient(); | ||
+ | FolderContainer root = shc.getWSRoot(); | ||
+ | //Creating the folder on the root workspace folder | ||
+ | FileContainer file = null; | ||
+ | try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){ | ||
+ | file = root.uploadFile(is, "name", "description"); | ||
+ | } catch (Exception e) { | ||
+ | //print the error | ||
+ | } | ||
+ | </source> | ||
+ | |||
+ | == Upload Archive == | ||
+ | |||
+ | Extract the content of the specified archive in a destination Folder (specified by its id) | ||
+ | |||
+ | === Java === | ||
+ | <source lang="java"> | ||
+ | StorageHubClient shc = new StorageHubClient(); | ||
+ | FolderContainer root = shc.getWSRoot(); | ||
+ | //Creating the folder on the root workspace folder | ||
+ | FileContainer file = null; | ||
+ | try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){ | ||
+ | file = root.uploadArchive(is, "parentFolderName"); | ||
+ | } catch (Exception e) { | ||
+ | //print the error | ||
+ | } | ||
+ | </source> | ||
+ | |||
+ | |||
+ | === REST API === | ||
+ | |||
+ | POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token} | ||
+ | Content-Type: multipart/form-data | ||
+ | {String parentFolderName, File file} | ||
+ | |||
+ | ==== CURL Example ==== | ||
+ | |||
+ | curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token} | ||
+ | |||
+ | == Get Public Url of a File == | ||
+ | |||
+ | Returns the public URL of a file | ||
+ | |||
+ | === Java === | ||
+ | <source lang="java"> | ||
+ | StorageHubClient shc = new StorageHubClient(); | ||
+ | shc.open({item-id}).asFile().getPublicLink(); | ||
+ | |||
+ | //returns the public link for a specific version of the file | ||
+ | shc.open({item-id}).asFile().getPublicLink({file-version}); | ||
+ | |||
+ | </source> | ||
+ | |||
+ | === REST API === | ||
+ | |||
+ | GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional | ||
+ | |||
+ | == Find By Name == | ||
+ | |||
+ | Returns all the items with a name that matches a pattern on the first level of a given folder. | ||
+ | The result will be empty in case of none of the item in the folder matches the pattern. | ||
+ | Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) | ||
+ | |||
+ | === Java === | ||
+ | <source lang="java"> | ||
+ | StorageHubClient shc = new StorageHubClient(); | ||
+ | //getting my root workspace folder | ||
+ | FolderContainer myRoot = shc.getWSRoot(); | ||
+ | myRoot.findByName("{pattern}"); | ||
+ | </source> | ||
+ | |||
+ | === REST API === | ||
+ | |||
+ | GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} | ||
+ | |||
+ | ==== responses ==== | ||
+ | |||
+ | 200 A JSON with the retrieved items or an empty itemList. | ||
+ | |||
+ | 500 The error is specified in the body of the response message | ||
+ | |||
+ | = DataMiner and SAI Interactions = | ||
+ | Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. | ||
Here are some examples: | Here are some examples: | ||
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]] | * [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]] | ||
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]] | * [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]] | ||
+ | |||
+ | = R Client = | ||
+ | An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here] |
Revision as of 15:15, 1 October 2019
Contents
|
Overview
The StorageHub APIs components provide a simple access to StorageHub service.
It is conceived to support both Java and REST-based calls.
Dependencies
Maven coordinates
<dependency> <groupId>org.gcube.common</groupId> <artifactId>storagehub-client-library</artifactId> <version>[1.0.0,2.0.0)</version> </dependency>
Key features
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.
StorageHub REST interface supports the following operations:
- Retrieve WS: to retrieve the user Workspace;
- Folder Listing: to list the content of a folder;
- Retrieve VRE Folder: to retrieve the VREFolder related to the token;
- Find: to find a file or a folder by name or pattern;
- Delete: to remove a file or a folder (including subfolders);
- Download: to download a file or a folder in ZIP format;
- Get Public Link: to get a public link of a file;
- Create Folder: to create a folder in the given parent folder;
- Unzip: to upload a zip file in a specific folder;
- Upload file: to upload a file in a folder.
- Versions: to get a specific version of a file;
API
Retrieve Workspace
Returns the Item representing the workspace of the user retrieved by the token used for the call.
Java
import org.gcube.common.storagehub.client.dsl.StorageHubClient; import org.gcube.common.storagehub.client.dsl.FileContainer; import org.gcube.common.storagehub.model.items.Item; StorageHubClient shc = new StorageHubClient(); FolderContainer rootContainer = shc.getWSRoot()
REST API
GET /workspace/?gcube-token={user-token}
responses
200 The workspace root item (in json format) is returned.
500 The error is specified in the body of the response message
Get Item ById
Java
import org.gcube.common.storagehub.client.dsl.FileContainer; import org.gcube.common.storagehub.client.dsl.StorageHubClient; import org.gcube.common.storagehub.model.items.Item; ... String theId = {itemIdentifier} // the identifier of the item StorageHubClient shc = new StorageHubClient(); FileContainer fileContainer = shc.open(theId).asFile();
List Item Versions, Get Item Version
Java
import org.gcube.common.storagehub.client.dsl.FileContainer; import org.gcube.common.storagehub.client.dsl.StorageHubClient; import org.gcube.common.storagehub.model.items.Item; import org.gcube.common.storagehub.model.service.Version; ... String theId = {itemIdentifier} // the identifier of the item StorageHubClient shc = new StorageHubClient(); FileContainer fileContainer = shc.open(theId).asFile(); List<Version> fileVersions = fileContainer.getVersions(); //to download a version fileContainer.downloadSpecificVersion(versionName)
Folder Listing
Returns the content of a Folder
Java
StorageHubClient shc = new StorageHubClient(); FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder(); List<? extends Item> items = folderContainer.list().getItems();
REST API
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}
responses
200 the list of the Items (in JSON format)
500 The error is specified in the body of the response message
Retrieve VRE Folder
Return the Item representing the root of the VRE folder related to the user token.
Java
StorageHubClient shc = new StorageHubClient(); FolderContainer rootContainer = shc.openVREFolder();
REST API
GET /workspace/vrefolder?gcube-token={user-token}
responses
200 The VRE folder root item (in json format) is returned.
500 The error is specified in the body of the response message
Create Folder
Creates a new folder under another folder (specified by its id)
Java
StorageHubClient shc = new StorageHubClient(); FolderContainer root = shc.getWSRoot(); //Creating the folder on the root workspace folder root.newFolder("name", "description");
REST API
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} Content-Type: application/x-www-form-urlencoded {String name, String description, boolean hidden}
responses
200 The Folder item identifier is returned.
500 The error is specified in the body of the response message
Upload File
Creates a new file under a folder (specified by its id)
Java
StorageHubClient shc = new StorageHubClient(); FolderContainer root = shc.getWSRoot(); //Creating the folder on the root workspace folder FileContainer file = null; try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){ file = root.uploadFile(is, "name", "description"); } catch (Exception e) { //print the error }
Upload Archive
Extract the content of the specified archive in a destination Folder (specified by its id)
Java
StorageHubClient shc = new StorageHubClient(); FolderContainer root = shc.getWSRoot(); //Creating the folder on the root workspace folder FileContainer file = null; try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){ file = root.uploadArchive(is, "parentFolderName"); } catch (Exception e) { //print the error }
REST API
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token} Content-Type: multipart/form-data {String parentFolderName, File file}
CURL Example
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}
Get Public Url of a File
Returns the public URL of a file
Java
StorageHubClient shc = new StorageHubClient(); shc.open({item-id}).asFile().getPublicLink(); //returns the public link for a specific version of the file shc.open({item-id}).asFile().getPublicLink({file-version});
REST API
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional
Find By Name
Returns all the items with a name that matches a pattern on the first level of a given folder. The result will be empty in case of none of the item in the folder matches the pattern. Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*)
Java
StorageHubClient shc = new StorageHubClient(); //getting my root workspace folder FolderContainer myRoot = shc.getWSRoot(); myRoot.findByName("{pattern}");
REST API
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token}
responses
200 A JSON with the retrieved items or an empty itemList.
500 The error is specified in the body of the response message
DataMiner and SAI Interactions
Algorithms created in SAI and executed by DataMiner can interact with StorageHub through the StorageHub REST APIs. Here are some examples:
R Client
An R client implementing the above function is available here