Difference between revisions of "StorageHub REST API"

From Gcube Wiki
Jump to: navigation, search
(Java)
(20 intermediate revisions by 3 users not shown)
Line 45: Line 45:
 
=== Java ===
 
=== Java ===
 
<source lang="java">
 
<source lang="java">
 +
import org.gcube.common.storagehub.client.dsl.StorageHubClient;
 +
import org.gcube.common.storagehub.client.dsl.FileContainer;
 +
import org.gcube.common.storagehub.model.items.Item;
 +
 +
 
  StorageHubClient shc = new StorageHubClient();
 
  StorageHubClient shc = new StorageHubClient();
  FolderContainer rootContianer = shc.getWSRoot()
+
  FolderContainer rootContainer = shc.getWSRoot()
 
</source>
 
</source>
  
Line 63: Line 68:
 
=== Java ===
 
=== Java ===
 
<source lang="java">
 
<source lang="java">
 +
import org.gcube.common.storagehub.client.dsl.FileContainer;
 +
import org.gcube.common.storagehub.client.dsl.StorageHubClient;
 +
import org.gcube.common.storagehub.model.items.Item;
 +
 +
...
 +
 
String theId = {itemIdentifier} // the identifier of the item  
 
String theId = {itemIdentifier} // the identifier of the item  
ItemManagerClient client = AbstractPlugin.item().build();
+
StorageHubClient shc = new StorageHubClient();
Item theItem = client.get(theId, "hl:accounting");
+
FileContainer fileContainer = shc.open(theId).asFile();
 
</source>
 
</source>
  
 
== List Item Versions, Get Item Version ==
 
== List Item Versions, Get Item Version ==
 
 
  
 
=== Java ===
 
=== Java ===
Line 104: Line 113:
 
=== REST API ===
 
=== REST API ===
  
GET /workspace/items/{itemIdentifier}/?gcube-token={userToken}
+
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}
 
+
==== parameters ====
+
 
+
{itemIdentifier} the identifier of the item to list the content
+
  
 
==== responses ====
 
==== responses ====
Line 115: Line 120:
  
 
500 The error is specified in the body of the response message
 
500 The error is specified in the body of the response message
 
== Find ==
 
 
Find a file or a folder by name or pattern;
 
=== Java ===
 
 
<source lang="java">
 
StorageHubClient shc = new StorageHubClient();
 
FolderContainer fc = shc.getWSRoot();
 
 
//"namePattern" can be a simple test to look for (e.g. "foo" will either return 1  Item if foo exists or null if nothing is named foo)
 
//Or  you can use the * operator (e.g. "foo*") which will return anything starting with foo.
 
List<? extends Item> theHits = fc.findByName("namePattern").getItems();
 
</source>
 
  
 
== Retrieve VRE Folder ==
 
== Retrieve VRE Folder ==
Line 164: Line 155:
 
=== REST API ===
 
=== REST API ===
  
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} - FormParameters: {String name, String description, boolean hidden}
+
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token}  
 +
Content-Type: application/x-www-form-urlencoded
 +
{String name, String description, boolean hidden}
  
  
Line 189: Line 182:
 
   }
 
   }
 
</source>
 
</source>
 +
 +
== Upload Archive ==
 +
 +
Extract the content of the specified archive in a destination Folder (specified by its id)
 +
 +
=== Java ===
 +
<source lang="java">
 +
  StorageHubClient shc = new StorageHubClient();
 +
  FolderContainer root = shc.getWSRoot();
 +
  //Creating the folder on the root workspace folder
 +
  FileContainer file = null;
 +
  try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){
 +
file = root.uploadArchive(is, "parentFolderName");
 +
  } catch (Exception e) {
 +
//print the error
 +
  }
 +
</source>
 +
  
 
=== REST API ===
 
=== REST API ===
  
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token} - MultipartFormData: {String name, String description, File file}
+
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}
 +
Content-Type: multipart/form-data
 +
{String parentFolderName, File file}
 +
 +
==== CURL Example ====
 +
 
 +
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}
  
== GEt Public Url of a File ==
+
== Get Public Url of a File ==
  
 
Returns the public URL of a file
 
Returns the public URL of a file
Line 212: Line 229:
 
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional
 
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional
  
 +
== Find By Name ==
 +
 +
Returns all the items with a name that matches a pattern on the first level of a given folder.
 +
The result will be empty in case of none of the item in the folder matches the pattern.
 +
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with  *{name}, ends with = {name}*)
 +
 +
=== Java ===
 +
<source lang="java">
 +
  StorageHubClient shc = new StorageHubClient();
 +
  //getting my root workspace folder
 +
  FolderContainer myRoot = shc.getWSRoot();
 +
  myRoot.findByName("{pattern}");
 +
</source>
 +
 +
=== REST API ===
 +
 +
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token}
  
 
==== responses ====
 
==== responses ====
  
200 The public URL is returned.
+
200 A JSON with the retrieved items or an empty itemList.
  
 
500 The error is specified in the body of the response message
 
500 The error is specified in the body of the response message
Line 224: Line 258:
 
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]
 
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]
 
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]
 
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]
 +
 +
= R Client =
 +
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]

Revision as of 14:15, 1 October 2019

Overview

The StorageHub APIs components provide a simple access to StorageHub service.

It is conceived to support both Java and REST-based calls.

Dependencies

Maven coordinates

<dependency>
<groupId>org.gcube.common</groupId>
<artifactId>storagehub-client-library</artifactId>
<version>[1.0.0,2.0.0)</version>
</dependency>

Key features

Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.

StorageHub REST interface supports the following operations:

  • Retrieve WS: to retrieve the user Workspace;
  • Folder Listing: to list the content of a folder;
  • Retrieve VRE Folder: to retrieve the VREFolder related to the token;
  • Find: to find a file or a folder by name or pattern;
  • Delete: to remove a file or a folder (including subfolders);
  • Download: to download a file or a folder in ZIP format;
  • Get Public Link: to get a public link of a file;
  • Create Folder: to create a folder in the given parent folder;
  • Unzip: to upload a zip file in a specific folder;
  • Upload file: to upload a file in a folder.
  • Versions: to get a specific version of a file;

API

Retrieve Workspace

Returns the Item representing the workspace of the user retrieved by the token used for the call.

Java

import org.gcube.common.storagehub.client.dsl.StorageHubClient;
import org.gcube.common.storagehub.client.dsl.FileContainer;
import org.gcube.common.storagehub.model.items.Item;
 
 
 StorageHubClient shc = new StorageHubClient();
 FolderContainer rootContainer = shc.getWSRoot()

REST API

GET /workspace/?gcube-token={user-token}

responses

200 The workspace root item (in json format) is returned.

500 The error is specified in the body of the response message

Get Item ById

Java

import org.gcube.common.storagehub.client.dsl.FileContainer;
import org.gcube.common.storagehub.client.dsl.StorageHubClient;
import org.gcube.common.storagehub.model.items.Item;
 
...
 
String theId = {itemIdentifier} // the identifier of the item 
StorageHubClient shc = new StorageHubClient();
FileContainer fileContainer = shc.open(theId).asFile();

List Item Versions, Get Item Version

Java

import org.gcube.common.storagehub.client.dsl.FileContainer;
import org.gcube.common.storagehub.client.dsl.StorageHubClient;
import org.gcube.common.storagehub.model.items.Item;
import org.gcube.common.storagehub.model.service.Version;
 
...
 
String theId = {itemIdentifier} // the identifier of the item 
StorageHubClient shc = new StorageHubClient();
FileContainer fileContainer = shc.open(theId).asFile();
List<Version> fileVersions = fileContainer.getVersions();
 
//to download a version
fileContainer.downloadSpecificVersion(versionName)

Folder Listing

Returns the content of a Folder

Java

StorageHubClient shc = new StorageHubClient();
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();
List<? extends Item> items = folderContainer.list().getItems();

REST API

GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}

responses

200 the list of the Items (in JSON format)

500 The error is specified in the body of the response message

Retrieve VRE Folder

Return the Item representing the root of the VRE folder related to the user token.

Java

 StorageHubClient shc = new StorageHubClient();
 FolderContainer rootContainer = shc.openVREFolder();

REST API

GET /workspace/vrefolder?gcube-token={user-token}

responses

200 The VRE folder root item (in json format) is returned.

500 The error is specified in the body of the response message

Create Folder

Creates a new folder under another folder (specified by its id)

Java

   StorageHubClient shc = new StorageHubClient();
   FolderContainer root = shc.getWSRoot();
   //Creating the folder on the root workspace folder
   root.newFolder("name", "description");

REST API

POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} Content-Type: application/x-www-form-urlencoded {String name, String description, boolean hidden}


responses

200 The Folder item identifier is returned.

500 The error is specified in the body of the response message

Upload File

Creates a new file under a folder (specified by its id)

Java

   StorageHubClient shc = new StorageHubClient();
   FolderContainer root = shc.getWSRoot();
   //Creating the folder on the root workspace folder
   FileContainer file = null;
   try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){
	file = root.uploadFile(is, "name", "description");
   } catch (Exception e) {
	//print the error
   }

Upload Archive

Extract the content of the specified archive in a destination Folder (specified by its id)

Java

   StorageHubClient shc = new StorageHubClient();
   FolderContainer root = shc.getWSRoot();
   //Creating the folder on the root workspace folder
   FileContainer file = null;
   try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){
	file = root.uploadArchive(is, "parentFolderName");
   } catch (Exception e) {
	//print the error
   }


REST API

POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token} Content-Type: multipart/form-data {String parentFolderName, File file}

CURL Example

curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}

Get Public Url of a File

Returns the public URL of a file

Java

   StorageHubClient shc = new StorageHubClient();
   shc.open({item-id}).asFile().getPublicLink();
 
   //returns the public link for a specific version of the file  
   shc.open({item-id}).asFile().getPublicLink({file-version});

REST API

GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional

Find By Name

Returns all the items with a name that matches a pattern on the first level of a given folder. The result will be empty in case of none of the item in the folder matches the pattern. Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*)

Java

   StorageHubClient shc = new StorageHubClient();
   //getting my root workspace folder
   FolderContainer myRoot = shc.getWSRoot();
   myRoot.findByName("{pattern}");

REST API

GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token}

responses

200 A JSON with the retrieved items or an empty itemList.

500 The error is specified in the body of the response message

DataMiner and SAI Interactions

Algorithms created in SAI and executed by DataMiner can interact with StorageHub through the StorageHub REST APIs. Here are some examples:

R Client

An R client implementing the above function is available here