https://wiki.gcube-system.org/api.php?action=feedcontributions&user=Massimiliano.assante&feedformat=atomGcube Wiki - User contributions [en]2024-03-29T14:18:33ZUser contributionsMediaWiki 1.25.1https://wiki.gcube-system.org/index.php?title=GCat_Service&diff=34644GCat Service2021-10-08T08:20:20Z<p>Massimiliano.assante: /* Profile Collection */</p>
<hr />
<div>gCat Service is a RESTful service enabling any client to programmatically interact with the gCube Catalogue.<br />
gCat Service uses the principles defined in [[GCat_Background | gCat Background]]<br />
<br />
== Request ==<br />
<br />
<br />
=== URL ===<br />
<br />
The URL used to interact with gCat is composed of two parts:<br />
<br />
* '''Base Service URL''': '''https://api.d4science.org/catalogue'''<br />
* Specific API (e.g. /organizations)<br />
<br />
D4Science infrastructure uses cloud facilities allowing to replicate a service to achieve failover and load balancing.<br />
<br />
gCat instances can be deployed and undeployed dynamically. For such a reason the Base Service URL MUST NOT be hard-cabled in the code because it can change over time.<br />
<br />
To dynamically discover the Base Service URL you can use the [https://dev.d4science.org/swagger/registry/ Registry RESTful Service]<br />
<br />
You need to discover gCore Resource having:<br />
* '''class''' : ''DataPublishing''<br />
* '''name''' : ''gcat''<br />
<br />
=== HTTP Headers ===<br />
<br />
==== gCube Authorization Token ====<br />
<br />
Any request performed to gCat MUST contains the gCube Authorization Token.<br />
<br />
This is done using the HTTP Header ''gcube-token''<br />
<br />
<pre>gcube-token: YOUR-TOKEN</pre><br />
<br />
The ''gcube-token'' HTTP header acts as the standard ''Authorization'' HTTP header. Plese note that ''Authorization'' HTTP header cannot be used in place of ''gcube-token'' HTTP header.<br />
<br />
===== Retrieve your gCube Authorization Token =====<br />
<br />
the gCube Authorization Token is a UUID bound to yourself and a given Infrastructure context. To retrieve it, you just need to go to a VRE for which you are interested and use the Authorisation Options portlet (see below)<br />
<br />
[[File:Authorisation_option.png]]<br />
<br />
Click on '''Show''' button and select the token.<br />
<br />
==== Content Type ====<br />
<br />
Any request must contain the indication of the interested content type.<br />
<br />
For any operation returning a result, the client must specify the ''Accept'' HTTP Header. <br />
<br />
<pre>Accept: application/json</pre><br />
<br />
For any operation sending content to the service, it is necessary to specify the ''Content-Type'' HTTP Header.<br />
<br />
<pre>Content-Type: application/json</pre><br />
<br />
Actually, the service accepts and returns only JSON objects.<br />
<br />
Except for a [[#Profile_Collection | profile]] which can be also requested in XML.<br />
<br />
=== HTTP Statuses ===<br />
<br />
Any successful operation returns '''''200 OK''''' status code except for create operation which return '''''201 Created''''' or for the operations which do not provide any content and returns '''''204 No Content'''''.<br />
<br />
The most common error status a client can obtain are <br />
<br />
* '''''400 Bad Request''''' used to indicate a clients error [https://tools.ietf.org/html/rfc7231#section-6.5.1 https://tools.ietf.org/html/rfc7231#section-6.5.1];<br />
* '''''401 Unauthorized''''' used to indicate that the client does not provided the gcube-token HTTP Header or the clinet has not enough right to perform such request [https://tools.ietf.org/html/rfc7235#section-3.1 https://tools.ietf.org/html/rfc7235#section-3.1];<br />
* '''''404 Not Found''''' used to indicate that the requested instance does not exists [https://tools.ietf.org/html/rfc7231#section-6.5.4 https://tools.ietf.org/html/rfc7231#section-6.5.4];<br />
* '''''405 Method Not Allowed''''' the used HTTP method is not supported for the requested URL [https://tools.ietf.org/html/rfc7231#section-6.5.5 https://tools.ietf.org/html/rfc7231#section-6.5.5]. <br />The response contains the ''Allow'' HTTP Header indicating the supported HTTP method for such URL [https://tools.ietf.org/html/rfc7231#section-7.4.1 https://tools.ietf.org/html/rfc7231#section-7.4.1];<br />
* '''''409 Conflict''''' the request could not be completed due to a conflict with the current state of the target resource (e.g. the name of the resource already exists). [https://tools.ietf.org/html/rfc7231#section-6.5.8 https://tools.ietf.org/html/rfc7231#section-6.5.8].];<br />
* '''''500 Internal Server Error''''' indicate a server failure. [https://tools.ietf.org/html/rfc7231#section-6.6.1 https://tools.ietf.org/html/rfc7231#section-6.6.1].<br />
<br />
A complete list of HTTP Status can be found here:<br />
[https://httpstatuses.com/ https://httpstatuses.com/]<br />
<br />
You can report a '''''500 Internal Server Error''''' the ticketing system. Please use this checklist before reporting an error:<br />
<br />
* replicate the request<br />
* the failure could be temporal due to network error, server issue so please retry the request after a certain amount of time<br />
* indicate how to replicate the error<br />
* indicate the time when the error occurred (this simplify the identification of the issue)<br />
<br />
=== HTTP Methods ===<br />
<br />
To be RESTful compliant gCat uses standard HTTP Methods to perform a listing of collections and CRUD (Create Read Update Delete) operations on instances.<br />
<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL || Success HTTP Status || Safe || Idempotent<br />
|-<br />
| Supported<br />HTTP Methods || OPTIONS || /{COLLECTION} || 204 No Content<br />(Supported HTTP Methods in ''Allow'' HTTP Header) || Y || Y<br />
|-<br />
| List || GET || /{COLLECTION} || 200 OK || Y || Y<br />
|-<br />
| Count || GET || /{COLLECTION}?count=true || 200 OK || Y || Y<br />
|-<br />
| Exists || HEAD || /{COLLECTION} || 204 No Content || Y || Y<br />
|-<br />
| Create || POST || /{COLLECTION} || 201 Created || N || N<br />
|-<br />
| Supported<br />HTTP Methods || OPTIONS || /{COLLECTION}/{INSTANCE_ID} || 204 No Content<br />(Supported HTTP Methods in ''Allow'' HTTP Header) || Y || Y<br />
|-<br />
| Exist || HEAD || /{COLLECTION}/{INSTANCE_ID} || 204 No Content || Y || Y<br />
|-<br />
| Read || GET || /{COLLECTION}/{INSTANCE_ID} || 200 OK || Y || Y<br />
|-<br />
| Update || PUT || /{COLLECTION}/{INSTANCE_ID} || 200 OK || N || Y<br />
|-<br />
| Patch || PATCH || /{COLLECTION}/{INSTANCE_ID} || 200 OK || N || Y<br />
|-<br />
| <span id="delete">Delete</span> || DELETE || /{COLLECTION}/{INSTANCE_ID} || 204 No Content || N || N [[#non_safe_delete| ‡]]<br />
|-<br />
| Purge || PURGE || /{COLLECTION}/{INSTANCE_ID} || 204 No Content || N || N[[#non_safe_delete| ‡]]<br />
|-<br />
| Purge || DELETE || /{COLLECTION}/{INSTANCE_ID}?purge=true || 204 No Content || N || N[[#non_safe_delete| ‡]]<br />
|}<br />
<br />
<br />
About URL:<br />
* {COLLECTION} is the plural name of the entity type;<br />
* {INSTANCE_ID} is an identification which enables to univocally identify the instance in the collection.<br />
<br />
About Safety and Idempotency properties:<br />
* A method is '''''Safe''''' if it does not produce any side effects. "''This does not prevent an implementation from including behaviour that is potentially harmful, that is not entirely read-only, or that causes side effects while invoking a safe method''" [https://tools.ietf.org/html/rfc7231#section-4.2.1 https://tools.ietf.org/html/rfc7231#section-4.2.1];<br />
* A method is '''''Idempotent''''' if the same operation repeated multiple times has the same side effect than using it one time. "''repeating the request will have the same intended effect, even if the original request succeeded, though the response might differ''" [https://tools.ietf.org/html/rfc7231#section-4.2.2 https://tools.ietf.org/html/rfc7231#section-4.2.2].<br />
<br />
[[#delete|‡]] DELETE has been defined as idempotent. <span id="non_safe_delete">Allamaraju</span><ref>Allamaraju S. RESTful Web Services Cookbook: Solutions for Improving Scalability and Simplicity . O’Reilly. first ed. 2010</ref> argues that DELETE idempotency should be accomplished client-side. The server should inform the client if a delete succeeded because the resource was really deleted or it was not found i.e., '''''404 Not Found''''' error is suggested instead of '''''204 No Content'''''. The latter situation should be treated as idempotent by the client. For this reason, gCat does not provide server-side idempotency for DELETE and PURGE operations.<br />
<br />
You can find more information about HTTP Methods at [https://restfulapi.net/http-methods/ https://restfulapi.net/http-methods/]<br />
<br />
Uncommon HTTP Methods:<br />
* PATCH method allows to perform a differential update (i.e. an update which provide only the differences and not the whole new representation) [];<br />
* PURGE method is not a standard but is a widely used in service which requires this action (e.g. [https://varnish-cache.org/docs/3.0/tutorial/purging.html Varnish], [https://wiki.squid-cache.org/SquidFaq/OperatingSquid#How_can_I_purge_an_object_from_my_cache.3F Squid]). gCat provide support for this method but to support a wider range of clients it also provides the Purge action via DELETE with the additional get parameter <pre>purge=true</pre><br />
<br />
== Collections ==<br />
<br />
Collections available to any users. Non-safe methods can only be invoked by [[GCat_Background#Users.2C_Roles_and_Groups | Catalogue Editor]]<br />
<br />
* [[#Item_Collection | Item]]<br />
* [[#Resource_Collection | Resource]]<br />
* [[#Profile_Collection | Profile]]<br />
* [[#Namespace_Collection | Namespace]]<br />
* [[#License_Collection | License]]<br />
<br />
Collections available only for [[GCat_Background#Users.2C_Roles_and_Groups | Catalogue Admin]]<br />
<br />
* [[#Group_Collection | Group]]<br />
* [[#Organization_Collection | Organization]]<br />
* [[#User_Collection | User]]<br />
<br />
<br />
For each collection HEAD and OPTIONS operations are not shown in tables even there are always available.<br />
<br />
To keep this documentation simple, for each collection is presented only the relevant attributes of an instance. <br />
This allows getting the picture of the content of a request or the content of a response.<br />
<br />
=== Item Collection ===<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL<br />
|-<br />
| List || GET || <code>/items?limit=10&offset=0</code><br />
|-<br />
| Count || GET || <code>/items?count=true</code><br />
|-<br />
| Create || POST || <code>/items?social_post=true</code><br />
|-<br />
| Read || GET || <code>/items/{NAME}</code><br />
|-<br />
| Update || PUT || <code>/items/{NAME}</code><br />
|-<br />
| Delete || DELETE || <code>/items/{NAME}</code><br />
|-<br />
| Purge || PURGE || <code>/items/{NAME}</code><br />
|-<br />
| Purge || DELETE || <code>/items/{NAME}?purge=true</code><br />
|}<br />
<br />
==== List Items ====<br />
<br />
The listing API provides paginated results by using the query parameters '''limit''' and '''offset'''. The defaults values are 10 for '''limit''' and 0 for '''offset'''.<br />
To get unlimited results the '''limit''' query parameters must be set to '''-1'''. If the results are too much the operation could fail. It is recommended to request no more than 1000 results.<br />
The '''offset''' parameter indicates the page number taking in account pages containing a number of results equivalent to '''limit'''.<br />
<br />
===== Filtering options =====<br />
<br />
The listing method offers options to filter the results, thus enacts to search for items including spatial search (see ext_bbox below). <br />
<br />
It accepts the following query parameters (a subset of Solr search query parameters, see [https://doc.lucidworks.com/fusion/3.1/Search/Query-Language-Cheat-Sheet.html Solr Query Syntax]):<br />
* '''q''' (string) – the solr query. Optional. Default: "*:*" See [http://www.solrtutorial.com/solr-query-syntax.html Solr Query Syntax]<br />
** E.g. <code>/items?q=title:foo</code> returns the items with word "foo" in the title; <br />
* '''fq''' (string) – Filter query. A query string that limits the query results without influencing their scores. Note: +site_id:{ckan_site_id} is added to this string prior to the query being executed.<br />
** E.g. <code>/items?q=title:foo&fq=notes:bar</code> returns with word "foo" in the 'title' and the word "bar" in the 'notes'; <br />
* '''fq_list''' (list of strings) – additional filter queries to apply.<br />
** E.g. <code>/items?q=title:foo&fq_list=...</code> returns the items with word "foo" in the 'title'<br />
* '''sort''' (string) – sorting of the search results. Optional. Default: 'relevance asc, metadata_modified desc'. As per the solr documentation, this is a comma-separated string of field names and sort-orderings.<br />
** E.g. <code>/items?q=title:foo&sort=name+asc</code> returns the items with word "foo" in the 'title' sorting the results by name ascending;<br />
<!--<br />
* '''facet''' (string) – whether to enable faceted results. Default: True.<br />
* '''facet.mincount''' (int) – the minimum counts for facet fields should be included in the results.<br />
* '''facet.limit''' (int) – the maximum number of values the facet fields return. A negative value means unlimited. This can be set instance-wide with the search.facets.limit config option. Default is 50.<br />
* '''facet.field''' (list of strings) – the fields to facet upon. Default empty. If empty, then the returned facet information is empty.<br />
--><br />
<!--<br />
* '''include_drafts''' (bool) – if True, draft datasets will be included in the results. A user will only be returned their own draft datasets, and a sysadmin will be returned all draft datasets. Optional, the default is False.<br />
** ... this is a new concept ... I do ignore what is a draft dataset now;<br />
** E.g. <code>/items?q=...&include_drafts</code> returns ... <br />
--><br />
* '''include_private''' (bool) – if True, private datasets will be included in the results. Only private datasets from the user’s organizations will be returned and sysadmins will be returned all private datasets. Optional, the default is False.<br />
** E.g. <code>/items?include_private=true</code><br />
<!--<br />
* '''use_default_schema''' (bool) – use default package schema instead of a custom schema defined with an IDatasetForm plugin (default: False)<br />
* '''rows''' (int) – the maximum number of matching rows (datasets) to return. (optional, default: 10, upper limit: 1000)<br />
* '''start''' (int) – the offset in the complete result for where the set of returned datasets should begin.<br />
--><br />
<!-- ===== Spatial search can be done using ===== --><br />
* '''ext_bbox''': The coordinates of the upper-right and bottom-left angle of a rectangular to query for. The form is '''Lat,Long,Lat,Long'''<br />
** E.g. <code>/items?limit=10&offset=0&q=Pollution&ext_bbox=-7.535093,49.208494,3.890688,57.372349</code> returns the first 10 items with 'Pollution' having a spatial coverage in the specified bounding box<br />
<br />
==== Create Items ====<br />
<br />
The create API allows to create social posts by indicating '''social_post=true''' in the URL. The '''social_post''' query parameter is optional and the default value is '''false'''.<br />
<br />
An Item is mainly described by the following attributes (* indicate mandatory attributes): <br />
<br />
* '''name*''' (string): the name of the new item, must be between 2 and 100 characters long and contain only lowercase alphanumeric characters, '-' and '_';<br />
* '''title''' (string, defaut="same as name"): the title of the item;<br />
* '''private*''' (bool): If True creates a private item;<br />
<br />
* '''maintainer''' (string): the name of the item’s maintainer;<br />
* '''maintainer_email''' (string): the email address of the item’s maintainer;<br />
* '''license_id*''' (license id string): the id of the item’s license, see license_list() for available values;<br />
* '''notes''' (string): a description of the item;<br />
* '''url''' (string): a URL for the item’s source;<br />
* '''version''' (string, no longer than 100 characters): ...;<br />
* '''state''' (string, default='active'): the current state of the item, e.g. 'active' or 'deleted', only active items show up in search results and other lists of items, this parameter will be ignored if you are not authorized to change the state of the item;<br />
* '''groups''' (list of dictionaries): the groups to which the item belongs, each group dictionary should have one or more of the following keys which identify an existing group: 'id' (the id of the group, string), or 'name' (the name of the group, string). To see which groups exist use list method in [[#Group_Collection | Group Collection]]<br />
* '''tags''' (list of tag dictionaries): the item’s tags. The tag is a dictionary in the format:<br />
** ''name'' : the name for the tag, a string between 2 and 100 characters long containing only alphanumeric characters and '-, '_' and '.'.<br />
* '''resources''' (list of resource dictionaries): the item’s resources, see [[#Resource_Collection | Resource Collection]] for the format of resource dictionaries;<br />
* '''extras''' (list of item extra dictionaries): the item’s extras, extras are arbitrary (key: value) metadata items that can be added to items, each extra dictionary should have keys 'key' (a string), 'value' (a string).<br />
<br />
Parameter automatically managed:<br />
* '''author''' (string): the name of the item’s author (the owner of the gcube-token);<br />
* '''author_email''' (string): the email address of the item’s author (the email of the owner of gcube-token);<br />
* '''owner_org''' (string): the id of the item’s owning organization, see organization_list() or organization_list_for_user() for available values (the VRE corrsponding to the gcube-token).<br />
<br />
<!--<br />
* '''type''' (string): the type of the item, IitemForm plugins associate themselves with different item types and provide custom item handling behaviour for these types;<br />
* '''relationships_as_object''' (list of relationship dictionaries): see package_relationship_create() for the format of relationship dictionaries;<br />
* '''relationships_as_subject''' (list of relationship dictionaries): see package_relationship_create() for the format of relationship dictionaries;<br />
--><br />
<br />
<br />
===== Geo-Indexing your datasets =====<br />
<br />
In order to make an Item searchable by location, it must have a special extra, with its key named ‘spatial’. The value must be a valid GeoJSON geometry, for example:<br />
<br />
<pre><br />
{<br />
"type":"Polygon",<br />
"coordinates":[[[2.05827, 49.8625],[2.05827, 55.7447], [-6.41736, 55.7447], [-6.41736, 49.8625], [2.05827, 49.8625]]]<br />
}<br />
</pre><br />
or <br />
<br />
<pre><br />
{<br />
"type": "Point",<br />
"coordinates": [-3.145,53.078]<br />
}<br />
</pre><br />
<br />
If at least one profile has been defined within this context, then you need to specify the profile's type when creating the item. You need to insert, among the extras of the JSON object describing the item, a '''''system:type''''' property with the proper value (i.e. its value must be equal to the type property contained in the profile). The validation of the submitted request will be performed against the profile whose type has been specified. The other profile's properties need to be specified within the extras field as well.<br />
<br />
If no profile has been defined, then no validation will be performed. Thus you do not need to set any '''''system:type''''' property.<br />
<br />
Suppose you discover that in the context you are willing to publish are defined the following profiles: 'A', 'B' and 'C'.<br />
<br />
You are interested in publishing an item with profile 'A', thus you download the XML of profile 'A' which looks like the following<br />
<br />
<source lang="xml"><br />
<metadataformat type="A"><br />
<metadatafield><br />
<fieldName>Input Parameter</fieldName><br />
<mandatory>false</mandatory><br />
<dataType>String</dataType><br />
<note>Input parameter expected for the execution of the process</note><br />
</metadatafield><br />
<metadatafield><br />
<fieldName>Output Parameter</fieldName><br />
<mandatory>false</mandatory><br />
<dataType>String</dataType><br />
<note>Output parameter expected from the execution of the process </note><br />
</metadatafield><br />
<metadatafield><br />
<fieldName>Process Author</fieldName><br />
<mandatory>false</mandatory><br />
<dataType>String</dataType><br />
<maxOccurs>*</maxOccurs><br />
<defaultValue/><br />
<note>The name of the author, with email and ORCID. The format should be: family, given[, email][, ORCID]. Example: Smith, John, js@acme.org, orcid.org/0000-0002-1825-0097</note><br />
<validator><br />
<regularExpression>^[a-zA-ZÀ-ÿ .'-]+, [a-zA-ZÀ-ÿ .'-]+[,]*([a-zA-Z0-9_!#$%’*+=?`{|}~^.-]+@[a-zA-Z0-9À-ÿ.-]+)?[,]*(orcid.org\/0000-000(1-[5-9]|2-[0-9]|3-[0-4])\d\d\d-\d\d\d[\dX])?$</regularExpression><br />
</validator><br />
</metadatafield><br />
</metadataformat><br />
</source><br />
<br />
The previous information can be easily discovered with the [[#Profile_Collection | Profile Collection]].<br />
Then, the JSON object of the create request will look like<br />
<br />
<pre><br />
{<br />
"metadataformat": {<br />
"metadatafield": [<br />
{<br />
"note": "Input parameter expected for the execution of the process",<br />
"fieldName": "Input Parameter",<br />
"dataType": "String",<br />
"mandatory": false<br />
},<br />
{<br />
"note": "Output parameter expected from the execution of the process",<br />
"fieldName": "Output Parameter",<br />
"dataType": "String",<br />
"mandatory": false<br />
},<br />
{<br />
"note": "The name of the author, with email and ORCID. The format should\t\t\tbe: family, given[, email][, ORCID]. Example: Smith, John,\t\t\tjs@acme.org, orcid.org/0000-0002-1825-0097",<br />
"fieldName": "Process Author",<br />
"defaultValue": "",<br />
"dataType": "String",<br />
"validator": {<br />
"regularExpression": "^[a-zA-ZÀ-ÿ .'-]+, [a-zA-ZÀ-ÿ .'-]+[,\t\t\t\t]*([a-zA-Z0-9_!#$%’*+=?`{|}~^.-]+@[a-zA-Z0-9À-ÿ.-]+)?[,\t\t\t\t]*(orcid.org\\/0000-000(1-[5-9]|2-[0-9]|3-[0-4])\\d\\d\\d-\\d\\d\\d[\\dX])?$"<br />
},<br />
"maxOccurs": "*",<br />
"mandatory": false<br />
}<br />
],<br />
"type": "DataMiner Process"<br />
}<br />
}<br />
</pre><br />
<br />
=== Resource Collection ===<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL<br />
|-<br />
| List || GET || <code>/items/{ITEM_ID}/resources</code><br />
|-<br />
| Create || POST || <code>/items/{ITEM_ID}/resources</code><br />
|-<br />
| Read || GET || <code>/items/{ITEM_ID}/resources/{ID}</code><br />
|-<br />
| Update || PUT || <code>/items/{ITEM_ID}/resources/{ID}</code><br />
|-<br />
| Delete || DELETE || <code>/items/{ITEM_ID}/resources/{ID}</code><br />
|}<br />
<br />
<br />
An Resource is mainly described by the following attributes (* indicate mandatory attributes): <br />
<br />
* '''name*''' (string);<br />
* '''url*''' (string): url of resource;<br />
* <del>'''package_id*''' (string): id of the item that the resource should be added to;</del><br />
* '''revision_id''' (string);<br />
* '''description''' (string);<br />
* '''format''' (string);<br />
* '''mimetype''' (string);<br />
* '''created''' (iso date string);<br />
* '''last_modified''' (iso date string);<br />
<br />
<br />
<!--<br />
* '''size''' (int);<br />
* '''mimetype_inner''' (string) – (optional)<br />
* '''cache_url''' (string) – (optional)<br />
* '''resource_type''' (string) – (optional)<br />
* '''hash''' (string)<br />
* '''cache_last_updated''' (iso date string) – (optional)<br />
* '''upload''' (FieldStorage (optional) needs multipart/form-data) – (optional)<br />
--><br />
<br />
=== Profile (or Type) Collection ===<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL<br />
|-<br />
| List || GET || /profiles<br />
|-<br />
| Count || GET || /profiles?count=true<br />
|-<br />
| Create || PUT || /profiles/{NAME}<br />
|-<br />
| Read || GET || /profiles/{NAME}<br />
|-<br />
| Update || PUT || /profiles/{NAME}<br />
|-<br />
| Delete || DELETE || /profiles/{NAME}<br />
<br />
|}<br />
<br />
Only a [[GCat_Background#Users.2C_Roles_and_Groups | Catalogue Editor]] can manipulate profiles (Create, Update, Delete). These operation are available starting from gCat 1.1.0.<br />
<br />
At creation and update time the provided profile is validated against the defined XSD schema (see [[GCat_Background#Metadata_Profile_v.3 | Item Profile Defintion v3]])<br />
<br />
<br />
A profile is defined using XML. In place of ''application/json'' Use <br />
<br />
<pre>Accept : application/xml</pre><br />
<br />
and <br />
<br />
<pre>Content-Type : application/xml</pre><br />
<br />
<br />
Listing return a JSON Array. It is also possible to read the representation of a profile as JSON which is an automatic mapping to of the original XML definition to JSON.<br />
<br />
An example of Profiles listing is :<br />
<br />
<pre><br />
[<br />
"EOSCService",<br />
"SoBigData.eu: Dataset Metadata",<br />
"SoBigData.eu: Application Metadata",<br />
"SoBigData.eu: Method Metadata"<br />
]<br />
</pre><br />
<br />
An example of Profile is :<br />
<br />
<source lang="xml"><br />
<metadataformat type="SoBigData Method"><br />
<metadatafield><br />
<fieldName>External Identifier</fieldName><br />
<dataType>String</dataType><br />
<mandatory>false</mandatory><br />
<note>This applies only to methods that have been already published. Insert here a DOI, an handle, and any other Identifier assigned when publishing the dataset alsewhere.</note><br />
</metadatafield><br />
<metadatafield><br />
<fieldName>Creator</fieldName><br />
<mandatory>true</mandatory><br />
<note>The name of the creator, with email and ORCID. The format should be: family, given[, email][, ORCID]. Example: Smith, John, js@acme.org, orcid.org//0000-0002-1825-0097</note><br />
<validator><br />
<regularExpression>^[a-zA-Z .'-]+, [a-zA-Z .'-]+[, ]*([a-zA-Z0-9_!#$%’*+=?`{|}~^.-]+@[a-zA-Z0-9.-]+)?[, ]*(orcid.org\/\/0000-000(1-[5-9]|2-[0-9]|3-[0-4])\d\d\d-\d\d\d[\dX])?$</regularExpression><br />
</validator><br />
</metadatafield><br />
<metadatafield><br />
<fieldName>CreationDate</fieldName><br />
<mandatory>true</mandatory><br />
<dataType>Time</dataType><br />
<note>The date of creation of the dataset (different from the date of registration of the dataset automatically added by the system). Use ISO 8601 Date Format: YYYY-MM-DD[ HH:MM] Ex. 1998-11-10 or 2015-05-29 11:55</note><br />
<validator><br />
<regularExpression>^(\d{4}\-(0?[1-9]|1[012])\-(0?[1-9]|[12][0-9]|3[01]))+([ ]+(\d{2}(:?\d{2})?)?)?$</regularExpression><br />
</validator><br />
</metadatafield><br />
<metadatafield><br />
<fieldName>Owner</fieldName><br />
<mandatory>true</mandatory><br />
<note>The name of the owner, with email and ORCID. The format should be: family, given[, email][, ORCID]. Example: Smith, John, js@acme.org, orcid.org//0000-0002-1825-0097</note><br />
<validator><br />
<regularExpression>^[a-zA-Z .'-]+, [a-zA-Z .'-]+[, ]*([a-zA-Z0-9_!#$%’*+=?`{|}~^.-]+@[a-zA-Z0-9.-]+)?[, ]*(orcid.org\/\/0000-000(1-[5-9]|2-[0-9]|3-[0-4])\d\d\d-\d\d\d[\dX])?$</regularExpression><br />
</validator><br />
</metadatafield><br />
....<br />
<metadatafield><br />
<fieldName>License term</fieldName><br />
<mandatory>false</mandatory><br />
<dataType>Time_Interval</dataType><br />
<note>Period of time during which the dataset may be used. Use ISO 8601 Date Format: YYYY-MM-DD[ HH:MM] Ex. 2016-07-31 or 2015-05-10 12:00</note><br />
<vocabulary/><br />
<validator><br />
<regularExpression>^(\d{4}\-(0?[1-9]|1[012])\-(0?[1-9]|[12][0-9]|3[01]))+([ ]+(\d{2}(:?\d{2})?)?)?$</regularExpression><br />
</validator><br />
</metadatafield><br />
<metadatafield><br />
<fieldName>Requirement of non-disclosure (confidentiality mark)</fieldName><br />
<mandatory>false</mandatory><br />
<note>Requirement of non-disclosure (confidentiality mark). Whether the dataset bears confidentiality mark/may be used and shared subject to the obligation of non-disclosure</note><br />
</metadatafield><br />
</metadataformat><br />
</source><br />
<br />
<br />
You can find all details about profiles at [[GCat_Background#Metadata | Item Profile]]<br />
<br />
=== Namespace Collection ===<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL<br />
|-<br />
| List || GET || /namespaces<br />
|}<br />
<br />
An example of given result is:<br />
<br />
<pre><br />
[<br />
{<br />
"id": "contact",<br />
"title": "Contact Title",<br />
"name": "contact",<br />
"description": "This section is about Contact(s)"<br />
},<br />
{<br />
"id": "developer_information",<br />
"title": "Developer Information",<br />
"name": "developer_information",<br />
"description": "This section is about Developer(s)"<br />
},<br />
{<br />
"id": "extra_information",<br />
"title": "Extras",<br />
"name": "extra_information",<br />
"description": "This section is about Extra(s)"<br />
},<br />
{<br />
"id": "ReportDetails",<br />
"title": "Report Details",<br />
"name": "ReportDetails",<br />
"description": "Report Details"<br />
}<br />
]<br />
</pre><br />
<br />
=== License Collection ===<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL<br />
|-<br />
| List || GET || /licenses<br />
|}<br />
<br />
<br />
An example of given result is:<br />
<br />
<pre><br />
[<br />
{<br />
"status": "active",<br />
"maintainer": "Creative Commons",<br />
"od_conformance": "rejected",<br />
"family": "Creative Commons",<br />
"osd_conformance": "not reviewed",<br />
"domain_data": true,<br />
"title": "Creative Commons Attribution-NonCommercial-ShareAlike 4.0",<br />
"url": "https://creativecommons.org/licenses/by-nc-sa/4.0/",<br />
"domain_content": true,<br />
"is_okd_compliant": false,<br />
"is_osi_compliant": false,<br />
"domain_software": false,<br />
"id": "CC-BY-NC-SA-4.0"<br />
},<br />
{<br />
"status": "active",<br />
"maintainer": "Creative Commons",<br />
"od_conformance": "rejected",<br />
"family": "Creative Commons",<br />
"osd_conformance": "not reviewed",<br />
"domain_data": true,<br />
"title": "Creative Commons Attribution-NonCommercial-NoDerivatives 4.0",<br />
"url": "https://creativecommons.org/licenses/by-nc-nd/4.0/",<br />
"domain_content": true,<br />
"is_okd_compliant": false,<br />
"is_osi_compliant": false,<br />
"domain_software": false,<br />
"id": "CC-BY-NC-ND-4.0"<br />
},<br />
...<br />
{<br />
"status": "active",<br />
"maintainer": "Free Software Foundation",<br />
"od_conformance": "not reviewed",<br />
"family": "",<br />
"osd_conformance": "approved",<br />
"domain_data": false,<br />
"title": "GNU Affero General Public License v3",<br />
"url": "http://www.opensource.org/licenses/AGPL-3.0",<br />
"domain_content": false,<br />
"is_okd_compliant": false,<br />
"is_osi_compliant": true,<br />
"domain_software": true,<br />
"id": "AGPL-3.0"<br />
}<br />
]<br />
</pre><br />
<br />
=== Group Collection ===<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL<br />
|-<br />
| List || GET || /groups<br />
|-<br />
| Count || GET || /groups?count=true<br />
|-<br />
| Create || POST || /groups<br />
|-<br />
| Read || GET || /groups/{NAME}<br />
|-<br />
| Update || PUT || /groups/{NAME}<br />
|-<br />
| Patch || PATCH || /groups/{NAME}<br />
|-<br />
| Delete || DELETE || /groups/{NAME}<br />
|-<br />
| Purge || PURGE || /groups/{NAME}<br />
|-<br />
| Purge || DELETE || /groups/{NAME}?purge=true<br />
|}<br />
<br />
Only [[GCat_Background#Users.2C_Roles_and_Groups | Catalogue Admin]] are able to invoke non-safe methods.<br />
<br />
<br />
A group is mainly described by the following attributes ('''*''' indicate mandatory attributes):<br />
* '''name*''' (string): the name of the group, a string between 2 and 100 characters long, containing only lowercase alphanumeric characters, '-' and '_' ;<br />
* '''id''' (string): the id of the group;<br />
* '''title''' (string): the title of the group;<br />
* '''description''' (string): the description of the group;<br />
* '''image_url''' (string): the URL to an image to be displayed on the group’s page;<br />
* '''state''' (string, default: 'active'): the current state of the group, e.g. 'active' or 'deleted', only active groups show up in search results and other lists of groups, this parameter will be ignored if you are not authorized to change the state of the group;<br />
* '''extras''' (list of dataset extra dictionaries): the group’s extras, extras are arbitrary (key: value) metadata items that can be added to groups, each extra dictionary should have keys 'key' (a string), 'value' (a string), and optionally 'deleted'.<br />
<br />
<!-- <br />
Other attributes used internally:<br />
* ''type'' (string): the type of the group. It cannot be 'organization';<br />
* ''approval_status'' (string): indicate if the group has been approved;<br />
* ''packages'' (list of dictionaries): the items that belong to the group, a list of dictionaries each with keys 'name' (string, the id or name of the items) and optionally 'title' (string, the title of the item);<br />
* ''groups'' (list of dictionaries): the groups that belong to the group, a list of dictionaries each with key 'name' (string, the id or name of the group) and optionally 'capacity' (string, the capacity in which the group is a member of the group);<br />
* ''users'' (list of dictionaries): the users that belong to the group, a list of dictionaries each with key 'name' (string, the id or name of the user) and optionally 'capacity' (string, the capacity in which the user is a member of the group).<br />
--><br />
<br />
==== Representation ====<br />
<br />
An example of Group representation is:<br />
<br />
<pre><br />
{<br />
"name": "group_name",<br />
"id": "fd72502e-ddfa-4121-a12a-bb42e572d649",<br />
"title": "Group Title",<br />
"description": "",<br />
"image_url": "",<br />
"state": "active",<br />
"extras": [],<br />
"created": "2018-03-22T11:48:42.612063",<br />
"revision_id": "0b5dda21-9bb0-4993-896b-d865e0d2f1f3"<br />
}<br />
</pre><br />
<br />
=== Organization Collection ===<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL<br />
|-<br />
| List || GET || /organizations<br />
|-<br />
| Count || GET || /organizations?count=true<br />
|-<br />
| Create || POST || /organizations<br />
|-<br />
| Read || GET || /organizations/{NAME}<br />
|-<br />
| Update || PUT || /organizations/{NAME}<br />
|-<br />
| Patch || PATCH || /organizations/{NAME}<br />
|-<br />
| Delete || DELETE || /organizations/{NAME}<br />
|-<br />
| Purge || PURGE || /organizations/{NAME}<br />
|-<br />
| Purge || DELETE || /organizations/{NAME}?purge=true<br />
|}<br />
<br />
Only [[GCat_Background#Users.2C_Roles_and_Groups | Catalogue Admin]] are able to invoke non-safe methods.<br />
<br />
An Organization is mainly described by the following attributes (* indicate mandatory attributes): <br />
<br />
<br />
* '''name*''' (string): the name of the organization, a string between 2 and 100 characters long, containing only lowercase alphanumeric characters, '-' and '_';<br />
* '''id''' (string): the id of the organization;<br />
* '''title''' (string): the title of the organization;<br />
* '''description''' (string): the description of the organization;<br />
* '''image_url''' (string): the URL to an image to be displayed on the organization’s page;<br />
* '''state''' (string, default: 'active'): the current state of the organization, e.g. 'active' or 'deleted', only active organizations show up in search results and other lists of organizations, this parameter will be ignored if you are not authorized to change the state of the organization;<br />
* '''approval_status''' (string);<br />
* '''extras''' (list of dataset extra dictionaries): the organization’s extras (optional), extras are arbitrary (key: value) metadata items that can be added to organizations, each extra dictionary should have keys 'key' (a string), 'value' (a string), and optionally 'deleted'.<br />
<br />
<!--<br />
* '''packages (list of dictionaries): the items that belong to the organization, a list of dictionaries each with keys 'name' (string, the id or name of the dataset) and optionally 'title' (string, the title of the dataset)<br />
* '''users (list of dictionaries): the users that belong to the organization, a list of dictionaries each with key 'name' (string, the id or name of the user) and optionally 'capacity' (string, the capacity in which the user is a member of the organization)<br />
--><br />
<br />
=== User Collection ===<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL<br />
|-<br />
| List || GET || /users<br />
|-<br />
| Create || POST || /users<br />
|-<br />
| Read || GET || /users/{NAME}<br />
|-<br />
| Update || PUT || /users/{NAME}<br />
|-<br />
| Delete || DELETE || /users/{NAME}<br />
|}<br />
<br />
Only [[GCat_Background#Users.2C_Roles_and_Groups | Catalogue Admin]] are able to invoke non-safe methods.<br />
<br />
A User is mainly described by the following attributes (* indicate mandatory attributes):<br />
* '''name*''' (string): the name of the user, a string between 2 and 100 characters in length, containing only lowercase alphanumeric characters, '-' and '_';<br />
* '''id''' (string): the id of the user;<br />
* '''fullname''' (string): the full name of the user;<br />
* '''email''' (string): the email address for the user;<br />
* '''password''' (string): the password of the user, a string of at least 4 characters (parameter only used to create a new user);<br />
* '''about''' (string): a description of the user.<br />
<br />
<br />
<pre><br />
{<br />
"name": "luca_frosini",<br />
"id": " 5f5c2bc0-bcd4-460a-9548-52cd4d33b263",<br />
"fullname": "Luca Frosini"<br />
}<br />
</pre></div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=StorageHub_REST_API&diff=33998StorageHub REST API2021-05-14T10:18:18Z<p>Massimiliano.assante: /* Folder Listing */</p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
= Overview =<br />
<br />
The StorageHub APIs components provide a simple access to StorageHub service. <br />
<br />
It is conceived to support both Java and REST-based calls. <br />
<br />
== Dependencies ==<br />
<br />
=== Maven coordinates ===<br />
<br />
<source lang="java"><br />
<dependency><br />
<groupId>org.gcube.common</groupId><br />
<artifactId>storagehub-client-library</artifactId><br />
<version>[1.0.0,2.0.0)</version><br />
</dependency><br />
</source><br />
<br />
= Key features =<br />
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.<br />
<br />
'''StorageHub REST interface''' supports the following operations:<br />
* '''Retrieve WS''': to retrieve the user Workspace;<br />
* '''Folder Listing''': to list the content of a folder;<br />
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;<br />
* '''Find''': to find a file or a folder by name or pattern;<br />
* '''Delete''': to remove a file or a folder (including subfolders);<br />
* '''Download''': to download a file or a folder in ZIP format;<br />
* '''Get Public Link''': to get a public link of a file;<br />
* '''Create Folder''': to create a folder in the given parent folder;<br />
* '''Unzip''': to upload a zip file in a specific folder;<br />
* '''Upload file''': to upload a file in a folder.<br />
* '''Versions''': to get a specific version of a file;<br />
<br />
== API ==<br />
<br />
The base-url of this service on D4Science production is the following:<br />
<br />
https://api.d4science.org/workspace<br />
<br />
== Retrieve Workspace ==<br />
<br />
Returns the Item representing the workspace of the user retrieved by the token used for the call.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.getWSRoot()<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The workspace root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Get Item ById ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
</source><br />
<br />
<br />
== List Item Versions, Get Item Version ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
import org.gcube.common.storagehub.model.service.Version;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
List<Version> fileVersions = fileContainer.getVersions();<br />
<br />
//to download a version<br />
fileContainer.downloadSpecificVersion(versionName)<br />
</source><br />
<br />
== Folder Listing ==<br />
<br />
Returns the content of a Folder<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
import org.gcube.common.storagehub.model.items.*;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
List<? extends Item> items = folderContainer.list().getItems();<br />
<br />
for (Item item : items) {<br />
System.out.println("name:" + item.getName() + " is a File?: " + (item instanceof AbstractFileItem));<br />
System.out.println("name:" + item.getName() + " is a folder?: " + (item instanceof FolderItem));<br />
System.out.println("name:" + item.getName() + " is a shared folder?: " + (item instanceof SharedFolder));<br />
System.out.println("name:" + item.getName() + " is a VRE folder?: " + (item instanceof VreFolder));<br />
}<br />
<br />
</source><br />
<br />
...<br />
//hide or show folder<br />
<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
folderContainer.setHidden(); //will not appear in the workspace GUI<br />
folderContainer.setVisible();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}<br />
<br />
==== responses ====<br />
<br />
200 the list of the Items (in JSON format)<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Retrieve VRE Folder ==<br />
<br />
Return the Item representing the root of the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.openVREFolder();<br />
</source><br />
<br />
<br />
Check that a folder with name "FolderNameToCheck" exists in the VRE folder related to the user token.<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.*;<br />
import org.gcube.common.storagehub.model.exceptions.StorageHubException;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer vreFolder = shc.openVREFolder();<br />
FolderContainer folderToCheck = null;<br />
try { <br />
OpenResolver oRes = vreFolder.openByRelativePath("FolderNameToCheck");<br />
folderToCheck = oRes.asFolder();<br />
} catch (StorageHubException e) {<br />
System.out.println("FolderNameToCheck does not exists in VRE Folder, creating it");<br />
folderToCheck = vreFolder.newFolder("FolderNameToCheck description goes here");<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/vrefolder?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The VRE folder root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Create Folder ==<br />
<br />
Creates a new folder under another folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
root.newFolder("name", "description");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} <br />
Content-Type: application/x-www-form-urlencoded <br />
{String name, String description, boolean hidden}<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder item identifier is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Share Folder ==<br />
<br />
Shares a folder between users setting the access type (WRITE_ALL, ADMINISTRATOR, READ_ONLY, WRITE_OWNER)<br />
<br />
=== JAVA ===<br />
<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open("{folderID}").asFolder().share(Arrays.asList("user1",.."userN"), AccessType.WRITE_ALL);<br />
</source><br />
<br />
== Move Item ==<br />
<br />
Moves a file or a folder (including subfolders) on another folder.<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
ItemContainer<Item> item = shc.open("itemToMoveId").asItem();<br />
FolderContainer destinationFolder = shc.open("destination").asFolder();<br />
file.move(destinationFolder);<br />
</source><br />
<br />
=== REST API ===<br />
<br />
PUT /workspace/items/{item-to-move-id}/move/?gcube-token={user-token} Content-Type: application/x-www-form-urlencoded {String destinationId}<br />
<br />
== Delete Item ==<br />
<br />
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;<br />
<br />
=== Java ===<br />
<source lang="java"><br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
fileContainer.delete();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)<br />
<br />
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder or item is deleted<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Empty Trash ==<br />
<br />
Removes all the items from the trash<br />
<br />
=== JAVA ===<br />
<br />
<source lang="java"> <br />
StorageHubClient shc = new StorageHubClient();<br />
shc.emptyTrash();<br />
</source><br />
<br />
== Download File or Folder ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
StreamDescriptor streamDescr = shc.open(theId).asFile().download();<br />
<br />
File output = {output file};<br />
try (BufferedInputStream bi = new BufferedInputStream(streamDescr.getStream()); FileOutputStream fo = new FileOutputStream(output)){<br />
byte[] buf = new byte[2046]; <br />
int read = -1;<br />
while ((read=bi.read(buf))!=-1) {<br />
fo.write(buf, 0, read);<br />
}<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{itemId}/download?gcube-token={user-token}<br />
<br />
'''Please note:''' in case the itemId corresponds to a folder you will get a compressed file (zip) of the folder with all its content.<br />
<br />
== Upload File ==<br />
<br />
Creates a new file under a folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadFile(is, "name", "description");<br />
System.out.println("Uploaded " + file.get().getName() + " - Returned Workspace id=" + file.getId()); <br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String name, String description, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}<br />
<br />
== Upload Archive ==<br />
<br />
Extract the content of the specified archive in a destination Folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadArchive(is, "parentFolderName");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String parentFolderName, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}<br />
<br />
== Get Public Url of a File ==<br />
<br />
Returns the public URL of a file<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open({item-id}).asFile().getPublicLink();<br />
<br />
//returns the public link for a specific version of the file <br />
shc.open({item-id}).asFile().getPublicLink({file-version});<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional<br />
<br />
== Find By Name ==<br />
<br />
Returns all the items with a name that matches a pattern on the first level of a given folder. <br />
The result will be empty in case of none of the item in the folder matches the pattern.<br />
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) <br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
//getting my root workspace folder<br />
FolderContainer myRoot = shc.getWSRoot();<br />
myRoot.findByName("{pattern}");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} <br />
<br />
==== responses ====<br />
<br />
200 A JSON with the retrieved items or an empty itemList. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
= DataMiner and SAI Interactions =<br />
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. <br />
Here are some examples:<br />
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]<br />
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]<br />
<br />
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].<br />
<br />
= R Client =<br />
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=StorageHub_REST_API&diff=33997StorageHub REST API2021-05-14T10:16:51Z<p>Massimiliano.assante: /* Java */</p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
= Overview =<br />
<br />
The StorageHub APIs components provide a simple access to StorageHub service. <br />
<br />
It is conceived to support both Java and REST-based calls. <br />
<br />
== Dependencies ==<br />
<br />
=== Maven coordinates ===<br />
<br />
<source lang="java"><br />
<dependency><br />
<groupId>org.gcube.common</groupId><br />
<artifactId>storagehub-client-library</artifactId><br />
<version>[1.0.0,2.0.0)</version><br />
</dependency><br />
</source><br />
<br />
= Key features =<br />
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.<br />
<br />
'''StorageHub REST interface''' supports the following operations:<br />
* '''Retrieve WS''': to retrieve the user Workspace;<br />
* '''Folder Listing''': to list the content of a folder;<br />
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;<br />
* '''Find''': to find a file or a folder by name or pattern;<br />
* '''Delete''': to remove a file or a folder (including subfolders);<br />
* '''Download''': to download a file or a folder in ZIP format;<br />
* '''Get Public Link''': to get a public link of a file;<br />
* '''Create Folder''': to create a folder in the given parent folder;<br />
* '''Unzip''': to upload a zip file in a specific folder;<br />
* '''Upload file''': to upload a file in a folder.<br />
* '''Versions''': to get a specific version of a file;<br />
<br />
== API ==<br />
<br />
The base-url of this service on D4Science production is the following:<br />
<br />
https://api.d4science.org/workspace<br />
<br />
== Retrieve Workspace ==<br />
<br />
Returns the Item representing the workspace of the user retrieved by the token used for the call.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.getWSRoot()<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The workspace root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Get Item ById ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
</source><br />
<br />
<br />
== List Item Versions, Get Item Version ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
import org.gcube.common.storagehub.model.service.Version;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
List<Version> fileVersions = fileContainer.getVersions();<br />
<br />
//to download a version<br />
fileContainer.downloadSpecificVersion(versionName)<br />
</source><br />
<br />
== Folder Listing ==<br />
<br />
Returns the content of a Folder<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
import org.gcube.common.storagehub.model.items.*;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
List<? extends Item> items = folderContainer.list().getItems();<br />
<br />
for (Item item : items) {<br />
System.out.println("name:" + item.getName() + " is a File?: " + (item instanceof AbstractFileItem));<br />
System.out.println("name:" + item.getName() + " is a folder?: " + (item instanceof FolderItem));<br />
System.out.println("name:" + item.getName() + " is a shared folder?: " + (item instanceof SharedFolder));<br />
System.out.println("name:" + item.getName() + " is a VRE folder?: " + (item instanceof VreFolder));<br />
}<br />
<br />
</source><br />
<br />
...<br />
//hide folder<br />
<br />
<source><br />
folderContainer.setHidden(); <br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}<br />
<br />
==== responses ====<br />
<br />
200 the list of the Items (in JSON format)<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Retrieve VRE Folder ==<br />
<br />
Return the Item representing the root of the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.openVREFolder();<br />
</source><br />
<br />
<br />
Check that a folder with name "FolderNameToCheck" exists in the VRE folder related to the user token.<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.*;<br />
import org.gcube.common.storagehub.model.exceptions.StorageHubException;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer vreFolder = shc.openVREFolder();<br />
FolderContainer folderToCheck = null;<br />
try { <br />
OpenResolver oRes = vreFolder.openByRelativePath("FolderNameToCheck");<br />
folderToCheck = oRes.asFolder();<br />
} catch (StorageHubException e) {<br />
System.out.println("FolderNameToCheck does not exists in VRE Folder, creating it");<br />
folderToCheck = vreFolder.newFolder("FolderNameToCheck description goes here");<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/vrefolder?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The VRE folder root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Create Folder ==<br />
<br />
Creates a new folder under another folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
root.newFolder("name", "description");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} <br />
Content-Type: application/x-www-form-urlencoded <br />
{String name, String description, boolean hidden}<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder item identifier is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Share Folder ==<br />
<br />
Shares a folder between users setting the access type (WRITE_ALL, ADMINISTRATOR, READ_ONLY, WRITE_OWNER)<br />
<br />
=== JAVA ===<br />
<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open("{folderID}").asFolder().share(Arrays.asList("user1",.."userN"), AccessType.WRITE_ALL);<br />
</source><br />
<br />
== Move Item ==<br />
<br />
Moves a file or a folder (including subfolders) on another folder.<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
ItemContainer<Item> item = shc.open("itemToMoveId").asItem();<br />
FolderContainer destinationFolder = shc.open("destination").asFolder();<br />
file.move(destinationFolder);<br />
</source><br />
<br />
=== REST API ===<br />
<br />
PUT /workspace/items/{item-to-move-id}/move/?gcube-token={user-token} Content-Type: application/x-www-form-urlencoded {String destinationId}<br />
<br />
== Delete Item ==<br />
<br />
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;<br />
<br />
=== Java ===<br />
<source lang="java"><br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
fileContainer.delete();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)<br />
<br />
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder or item is deleted<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Empty Trash ==<br />
<br />
Removes all the items from the trash<br />
<br />
=== JAVA ===<br />
<br />
<source lang="java"> <br />
StorageHubClient shc = new StorageHubClient();<br />
shc.emptyTrash();<br />
</source><br />
<br />
== Download File or Folder ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
StreamDescriptor streamDescr = shc.open(theId).asFile().download();<br />
<br />
File output = {output file};<br />
try (BufferedInputStream bi = new BufferedInputStream(streamDescr.getStream()); FileOutputStream fo = new FileOutputStream(output)){<br />
byte[] buf = new byte[2046]; <br />
int read = -1;<br />
while ((read=bi.read(buf))!=-1) {<br />
fo.write(buf, 0, read);<br />
}<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{itemId}/download?gcube-token={user-token}<br />
<br />
'''Please note:''' in case the itemId corresponds to a folder you will get a compressed file (zip) of the folder with all its content.<br />
<br />
== Upload File ==<br />
<br />
Creates a new file under a folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadFile(is, "name", "description");<br />
System.out.println("Uploaded " + file.get().getName() + " - Returned Workspace id=" + file.getId()); <br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String name, String description, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}<br />
<br />
== Upload Archive ==<br />
<br />
Extract the content of the specified archive in a destination Folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadArchive(is, "parentFolderName");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String parentFolderName, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}<br />
<br />
== Get Public Url of a File ==<br />
<br />
Returns the public URL of a file<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open({item-id}).asFile().getPublicLink();<br />
<br />
//returns the public link for a specific version of the file <br />
shc.open({item-id}).asFile().getPublicLink({file-version});<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional<br />
<br />
== Find By Name ==<br />
<br />
Returns all the items with a name that matches a pattern on the first level of a given folder. <br />
The result will be empty in case of none of the item in the folder matches the pattern.<br />
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) <br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
//getting my root workspace folder<br />
FolderContainer myRoot = shc.getWSRoot();<br />
myRoot.findByName("{pattern}");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} <br />
<br />
==== responses ====<br />
<br />
200 A JSON with the retrieved items or an empty itemList. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
= DataMiner and SAI Interactions =<br />
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. <br />
Here are some examples:<br />
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]<br />
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]<br />
<br />
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].<br />
<br />
= R Client =<br />
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=Social_Networking_Service&diff=33990Social Networking Service2021-04-28T15:43:33Z<p>Massimiliano.assante: </p>
<hr />
<div>{| align="right"<br />
| __TOC__<br />
|}<br />
<br />
The Social Networking Service is a RESTful application which exposes operations coming from both the [https://wiki.gcube-system.org/gcube/Social_Networking_Library gCube Social-Networking-Library] and the [https://wiki.gcube-system.org/gcube/UserManagement_Core User Management] components. The former is the bridge between the Cassandra Cluster (on which social data are stored) and the social applications, whereas the latter is the abstraction layer over the enabling technology of the Infrastructure portals, [https://www.liferay.com/ Liferay], in terms of: roles, users and organizations and how they are mapped to gCube concepts (VREs, Teams, and so on).<br />
<br />
The web services exposes a subset of such functionalities over '''https''' in a standard, reliable and secure way. <br />
<br />
See the vailable REST-API at a [https://dev.d4science.org/swagger/social-networking/ glance here.]<br />
= Discover the Web Service =<br />
For gCube releases older than 4.3, a Service Endpoint that has<br />
<br />
<source lang="java"><br />
Name = SocialNetworking<br />
Category = Portal<br />
</source><br />
<br />
mantains the base path of the service. Starting from gCube 4.3 version, the service runs on SmartGears, thus a gCore Endpoint needs to be searched having<br />
<br />
<source lang="java"><br />
Service Name = SocialNetworking<br />
Service Class = Portal<br />
</source><br />
<br />
The base path is the entry point of the web service. In the '''development''' environment, its current value is<br />
<br />
https://socialnetworking-d-d4s.d4science.org/social-networking-library-ws/rest or https://api.dev.d4science.org/rest<br />
<br />
<br />
In the '''production''' environment, its current value is<br />
<source lang="xml"><br />
https://api.d4science.org/rest<br />
</source><br />
<br />
This url is referred as ''BASE_URL''.<br />
<br />
= Service versions =<br />
Starting from gCube 4.3, the service exposes its methods using a more standard naming approach. Moreover, they accept (in case of http POST/PUT methods) JSON objects. The result of all methods is always a JSON object of this kind<br />
<br />
<source lang="java"><br />
{"success": false/true,<br />
"message": ...,<br />
"result": ...,<br />
}<br />
</source><br />
<br />
Where<br />
* success reports if the request succeeded or failed;<br />
* message is a status/error message that can be checked in case of errors (success equals false);<br />
* result is the current result object (it can be a list, a single object and so on depending on the invoked method). <br />
<br />
Inputs are automatically validated before the request is served.<br />
<br />
The older version (Version 1) instead has some limitation thus you are kindly suggested to no longer use it. However, in the following, its methods are going to be reported as well. The older version will be removed starting from the next gCube releases.<br />
<br />
= Authorization =<br />
No matter the version number of the service, every call to one of its method require a gcube-token. See [https://wiki.gcube-system.org/gcube/Authorization_Framework here] for more information about this token. This secret token needs to be valid in the current context. It can be put as query parameter of the http request or within the header. Both approaches are valid. If the token is missing, the operation will be rejected and an http forbidden code will be returned.<br />
<br />
= Usage =<br />
The methods of the Web Service can be called by writing your own REST client application or by using (a lot of) already existent rest client plugins. In the following, we will use the RESTClient plugin for Firefox for POST methods. Your browser can be used for GET methods. The return value of each method is an HTTP Response object that has an HTTP CODE (200, 201, 400, 500, ...) and, optionally, a JSON or text object that represents the created/retrieved resource or better specifies the occurred error. In case of errors, please refer to the [[#Troubleshooting|Troubleshooting]] section for more details.<br />
<br />
= Service methods (Version 2) =<br />
The list of sub-services available in the new version is going to be reported next. The name convention is different. For instance, to read user's posts, the method to be called is<br />
<br />
<source lang="java"><br />
BASE_PATH/2/posts/get-posts-user/<br />
</source><br />
<br />
The real result of the method, if the request succeeded, is contained into the 'result' field of the json object<br />
<br />
<source lang="java"><br />
{"success": ..., "message": ..., "result": "[{first post}, {second post}, ...}<br />
</source><br />
<br />
In the following, the list of available sub-services available under the newest version, its methods and the parameters they need in order to be called are reported. Again, you can use your preferred REST client so examples won't be reported.<br />
<br />
== List of Available Sub-Services and methods==<br />
=== Tokens ===<br />
====Generate application token====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/tokens/generate-application-token<br />
</source><br />
<br />
First of all, you need to have a ''user gcube-token'' for authentication purposes. You can get it using the Token Generator portlet available on the VRE you are interesting the app will post in.<br />
Then, to get an ''application gcube-token'' you can invoke the method. It is an HTTP/POST method that accepts as parameter:<br />
<br />
* a json object of this kind {"app_id": "your-app-id-goes-here"}.<br />
<br />
=== User profile and roles ===<br />
<br />
To retrieve user's profile user profile info (and roles) the subservice to call, once the social networking service endpoint has been retrieved, is this one:<br />
<br />
<code><br />
/2/people/profile<br />
</code><br />
<br />
further details: [https://wiki.gcube-system.org/gcube/OAuth2.0#Step_4_-_Get_member_data get member information]<br />
<br />
=== Full text search ===<br />
==== Search by query ====<br />
Social data are indexed into an ElasticSearch engine and are querable using a full-text approach. You can retrieve more information about this index [https://wiki.gcube-system.org/index.php?title=Social_Networking_Data_Discovery|here].<br />
The link to append to '''Base_URL''' is the following<br />
<source lang="java"><br />
2/full-text-search/search-by-query<br />
</source><br />
<br />
The query can be submitted also to the service, by using the following parameters<br />
<br />
* query: the query to perform against the index [mandatory];<br />
* from : the result set should be returned by starting from the n-th result [default is zero, optional];<br />
* quantity: how many enhanced posts need to be returned [default is 10, optional].<br />
<br />
=== Hashtags ===<br />
==== Get hashtags and occurrences in a context====<br />
The link to append to '''Base_URL''' is the following<br />
<source lang="java"><br />
2/hashtags/get-hashtags-and-occurrences<br />
</source><br />
<br />
The result is of the kind<br />
{"success": true,<br />
"message": null,<br />
"result":{{"hashtag1" : x}, {"hashtag2" : y}, ...}<br />
}<br />
=== Posts ===<br />
==== Write application post====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/write-post-app<br />
</source><br />
<br />
By means of the application token requested with the above method, the app can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters in a JSON object<br />
<br />
* text: text of the post; <br />
* params: additional parameters if your application supports the direct opening of of this update's object e.g. id=12345&type=foo [optional];<br />
* preview_title: a title for the preview of the post [optional];<br />
* preview_host: a host for the preview of the post [optional];<br />
* preview_description: a description for the preview of the post [optional];<br />
* preview_url: the preview url [optional];<br />
* image_url: if you want to put an image, pass its url [optional];<br />
* enable_notification: a boolean value to enable notifications [optional, default is false].<br />
<br />
====Write user post====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/write-post-user<br />
</source><br />
<br />
By means of a valid user token, he/she can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters in a JSON object:<br />
<br />
* text: text of the post; <br />
* preview_title: a title for the preview of the post [optional];<br />
* preview_description: a description for the preview of the post [optional];<br />
* preview_host: a host for the preview of the post [optional];<br />
* preview_url: the preview url [optional];<br />
* image_url: if you want to put an image, pass its url [optional];<br />
* enable_notification: a boolean value to enable notifications [optional].<br />
<br />
====Retrieve all application posts==== <br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-app<br />
</source><br />
<br />
In order to retrieve all the posts the application wrote (in every VRE it is allowed to write in) you can invoke this HTTP/GET method.<br />
====Retrieve all user posts====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user<br />
</source><br />
<br />
====Retrieve all user posts since a given date====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user-since<br />
</source><br />
<br />
If you are interested in retrieving user's posts whose timestamps are greater than a given date you can invoke this method.<br />
This HTTP/GET method accepts as parameter:<br />
<br />
* time: the time of the oldest post to retrieve converted in milliseconds.<br />
<br />
====Retrieve all user posts up to a given quantity====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user-quantity<br />
</source><br />
<br />
If you are interested in retrieving user's posts, starting from the newest up to a given quantity you can invoke this method.<br />
This HTTP/GET method accepts as parameter:<br />
<br />
* quantity: a value greater than zero (the default value is 10).<br />
<br />
====Retrieve the identifiers of the posts a user liked====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-id-liked-posts<br />
</source><br />
<br />
By means of this method a user can retrieve the identifiers of the posts he/she liked.<br />
====Retrieve the posts a user liked====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-liked-posts<br />
</source><br />
<br />
By means of this method a user can retrieve the posts he/she liked.<br />
====Retrieve all vre posts====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-vre<br />
</source><br />
<br />
By means of this method a user that has a token valid for a certain VRE, can retrieve all the posts of that VRE.<br />
====Get posts by hashtag====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-by-hashtag<br />
</source><br />
The method requires as parameter:<br />
* hashtag: the hashtag to be contained in the returned posts[mandatory].<br />
<br />
=== Comments ===<br />
====Retrieve comments by user====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/comments/get-comments-user<br />
</source><br />
It is an http GET method.<br />
====Retrieve comments by user and time====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/comments/get-comments-user-by-time<br />
</source><br />
It is an http GET method. It requires:<br />
* time: the time (in milliseconds) since when the comments need to be retrieved [default is zero, optional];<br />
=== Notifications ===<br />
====Retrieve user's notifications in a given range====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/notifications/get-range-notifications<br />
</source><br />
<br />
By means of this method a user can retrieve his/her last notifications in the range [a, b]. The method requires:<br />
<br />
* from: an integer greater than zero;<br />
* quantity: an integer equal or greater than zero.<br />
<br />
The final range is evaluated as [from, from + quantity].<br />
====Notify job status====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/notifications/notify-job-status<br />
</source><br />
<br />
By means of this method a user can be notified about the status of a job in the infrastructure. Jobs are, for instance, SmartExecutor's plugins or DataMiner's algorithms. The method requires the following parameters in a JSON object via HTTP POST<br />
<br />
* recipient: the username of the user to be alerted [mandatory];<br />
* job_id: the identifier of the job (e.g, a uuid) [mandatory];<br />
* job_name: the name of the running job (e.g., Social-data-indexer-plugin)[mandatory];<br />
* service_name: the name of the service that launched the job (e.g., SmartExecutor) [mandatory];<br />
* status: the status reached by the job, that is any among CANCELLED, CANCELLING, DELETED, DELETING, EXECUTING, FAILED, SUCCEEDED, SUBMITTED, TIMED_OUT, WAITING [mandatory];<br />
* status_message: a detailed message about the status [optional].<br />
<br />
=== Messages ===<br />
====Send a message====<br />
It is an HTTP POST method.<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/write-message<br />
</source><br />
<br />
By default, the sender of the message is the owner of the gcube-token. According to the recipients' notifications preferences, they could receive a portal notification, an email notification and so on.<br />
<br />
Parameters, to be put in a JSON object are:<br />
<br />
* subject: the subject of the message to send;<br />
* body: the body of the message;<br />
* recipients: a list of valid comma separated users' ids (e.g., andrea.rossi, mario.rossi) or users' emails (e.g., andrea.rossi@isti.cnr.it, mario.rossi@isti.cnr.it);<br />
<br />
An example of the body of the request is as follows<br />
<source lang="xml"><br />
{"subject": "....", "body": ".....", "recipients":[{"id":"userid1"},{"id":"userid2"}]}<br />
</source><br />
<br />
or, if you want to use valid mails (i.e. recognized in the infrastructure)<br />
<source lang="xml"><br />
{"subject": "....", "body": ".....", "recipients":[{"id":"mail1"},{"id":"mail2"}]}<br />
</source><br />
<br />
====Get received messages====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/get-received-messages<br />
</source><br />
It is an http GET method.<br />
<br />
====Get sent messages====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/get-sent-messages<br />
</source><br />
It is an http GET method.<br />
=== Users ===<br />
====User exists====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/user-exists<br />
</source><br />
<br />
The method requires<br />
<br />
* username: the username of the user which has to be checked.<br />
<br />
'''Note''' The method can be only invoked with a root-context application token.<br />
====Read user's custom fields====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-custom-attribute<br />
</source><br />
<br />
The method requires<br />
<br />
* attribute: that is the name of a valid attribute to be read.<br />
====Get user's email====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-email<br />
</source><br />
<br />
The method returns the email associated to the owner of the security token used.<br />
====Get user's profile====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-profile<br />
</source><br />
<br />
The method returns the profile of the owner of the security token used.<br />
====Get user's fullname====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-fullname<br />
</source><br />
<br />
The method returns the fullname associated to the owner of the security token used.<br />
====Get all users' usernames in VRE====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames<br />
</source><br />
<br />
The method returns the usernames of the context associated to the security token.<br />
<br />
====Get usernames by global role====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames-by-global-role<br />
</source><br />
<br />
The method requires<br />
<br />
* role-name: that is the name of the role to consider.<br />
<br />
'''''This method can be only invoked with an Infrastructure Token.'''''<br />
<br />
<br />
====Get usernames by role====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames-by-role<br />
</source><br />
<br />
The method requires<br />
<br />
* role-name: that is the name of the role to consider.<br />
<br />
=== VREs ===<br />
====Get User VREs====<br />
This method allows a user to retrieve the list of VREs she belongs to<br />
<br />
<source lang="java"><br />
2/vres/get-my-vres<br />
</source><br />
<br />
The method accepts an option parameter:<br />
* getManagers : if set to true, reports also the VRE Managers of the VRE<br />
<br />
== Troubleshooting ==<br />
<br />
There could be many reasons for a failed request. Please always check the HTTP CODE and the message within the response. <br />
<br />
=== HTTP Error Code ===<br />
This table tries to help you:<br />
<br />
{| border="1" cellpadding="20" cellspacing="0"<br />
!HTTP CODE<br />
!Meaning<br />
|-<br />
|400 - BAD_REQUEST<br />
|Please check the parameters you are passing<br />
|-<br />
|403 - FORBIDDEN<br />
|Is the token valid? / Does it belong to an application? [In case of application request]<br />
|-<br />
|404 - NOT_FOUND<br />
|The object(s) you are asking for is(are) not available<br />
|-<br />
|500 - INTERNAL_SERVER_ERROR<br />
|There was an error while serving your request (retry later)<br />
|-<br />
|}<br />
<br />
<br />
=== Always manage HTTP Redirects on POST requests (i.e. 30X codes) ===<br />
The service accepts both incoming http and https client requests, which are automatically redirect to https protocol by the reverse proxy nginx. This redirect mechanism must be taken into account when POST calls are performed, because it is not (always) supported and transparent by clients. In order to be sure, please check if you get a 30X error and extract the redirection urls within the response header. The following is a working java code that extract such information. It uses the apache-client library to perform http calls.<br />
<br />
<source lang="java"><br />
<br />
// build the client and the request<br />
String path = ... ; <br />
HttpClientBuilder clientBuilder = HttpClientBuilder.create();<br />
CloseableHttpClient httpClient = clientBuilder.build();<br />
HttpPost request = new HttpPost(path);<br />
JSONObject obj = new JSONObject();<br />
<br />
... // build the json object<br />
<br />
// set header properties<br />
request.addHeader("gcube-token", ...);<br />
request.addHeader("Content-type", ContentType.APPLICATION_JSON.toString());<br />
StringEntity params = new StringEntity(obj.toJSONString(), ContentType.APPLICATION_JSON);<br />
request.setEntity(params);<br />
HttpResponse response = httpClient.execute(request);<br />
logger.info(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
<br />
int status = response.getStatusLine().getStatusCode();<br />
<br />
// check the response status and look if it was a redirect problem<br />
if (status != HttpURLConnection.HTTP_OK && (status == HttpURLConnection.HTTP_MOVED_TEMP ||<br />
status == HttpURLConnection.HTTP_MOVED_PERM ||<br />
status == HttpURLConnection.HTTP_SEE_OTHER)) {<br />
<br />
// redirect -> fetch new location<br />
Header[] locations = response.getHeaders("Location");<br />
Header lastLocation = locations[locations.length - 1];<br />
String realLocation = lastLocation.getValue();<br />
logger.debug("New location is " + realLocation);<br />
<br />
// perform again the same request<br />
request = new HttpPost(realLocation);<br />
request.addHeader("gcube-token", ...);<br />
request.addHeader("Content-type", ContentType.APPLICATION_JSON.toString());<br />
params = new StringEntity(obj.toJSONString(), ContentType.APPLICATION_JSON);<br />
request.setEntity(params);<br />
response = httpClient.execute(request);<br />
logger.info(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
} else<br />
logger.debug(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
</source><br />
<br />
= Service methods (Version 1) - DEPRECATED =<br />
In the following, the list of available sub-services available under the first version, its methods and the parameters they need in order to be called are reported. The gcube-token is always required so it won't be reported as parameter. <br />
<br />
'''This version has been replaced by version 2, please do not use this version and update your code asap.'''<br />
<br />
== List of available Sub-Services and methods ==<br />
=== Tokens ===<br />
==== Generate application token ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
tokens/generateApplicationToken<br />
</source><br />
<br />
First of all, you need to have a ''user gcube-token'' for authentication purposes. You can get it using the Token Generator portlet available on the VRE you are interesting the app will post in.<br />
Now, to get an ''application gcube-token'' you can invoke the method. It is an HTTP/POST method that accepts as parameters:<br />
<br />
* appid: the identifier of the application as reported in the Application Profile.<br />
<br />
For example, using the RESTClient plugin for Firefox you will have something like this for requiring a new application token:<br />
<br />
[[File:GenerateApplicationToken.png]]<br />
<br />
Push on the SEND button and if all it's ok, your application gcube-token will be generated and returned as ''plain text'' in the response within the HTTP response having code 201.<br />
=== Posts ['''DEPRECATED'''] ===<br />
==== Write application post ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/writePostApplication<br />
</source><br />
<br />
By means of the application token requested with the above method, the app can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters:<br />
<br />
* text: text of the post; <br />
* params: additional parameters if your application supports the direct opening of of this update's object e.g. id=12345&type=foo [optional];<br />
* previewtitle: a title for the preview of the post [optional];<br />
* previewdescription: a description for the preview of the post [optional];<br />
* httpimageurl: if you want to put an image, pass its url [optional];<br />
* enablenotification: a boolean value to enable notifications [optional].<br />
<br />
For example:<br />
<br />
[[File:WriteApplicationPost.png]]<br />
<br />
Push on SEND and check the code associated to the returned response (201 in case of success).<br />
<br />
==== Retrieve all application posts ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByApp<br />
</source><br />
<br />
In order to retrieve all the posts the application wrote (in every VRE it is allowed to write in) you can invoke this HTTP/GET method.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByApp?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByUser<br />
</source><br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts since a given date ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getRecentPostsByUserAndDate<br />
</source><br />
<br />
If you are interested in retrieving user's posts whose timestamps are greater than a given date you can invoke this method.<br />
This HTTP/GET method accepts this parameter:<br />
<br />
* time: the time of the oldest post to retrieve converted in milliseconds.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getRecentPostsByUserAndDate?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&time=xxxxx<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts up to a given quantity ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getRecentPostsByUser<br />
</source><br />
<br />
If you are interested in retrieving user's posts, starting from the newest up to a given quantity you can invoke this method.<br />
This HTTP/GET method accepts two parameter:<br />
<br />
* quantity: a value greater than zero (the default value is 10).<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getRecentPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&quantity=20<br />
</source><br />
<br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Write user post ['''DEPRECATED'''] ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/writePostUser<br />
</source><br />
<br />
By means of a valid user token, he/she can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters:<br />
<br />
* text: text of the post; <br />
* previewtitle: a title for the preview of the post [optional];<br />
* previewdescription: a description for the preview of the post [optional];<br />
* previewHost: [optional];<br />
* previewUrl: [optional];<br />
* httpimageurl: if you want to put an image, pass its url [optional];<br />
* enablenotification: a boolean value to enable notifications [optional];<br />
<br />
For example:<br />
<br />
[[File:Userpost.png]]<br />
<br />
Push on SEND and check the code associated to the returned response (201 in case of success).<br />
<br />
==== Retrieve the identifiers of the posts a user liked ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllLikedPostIdsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve the identifiers of the posts he/she liked.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllLikedPostIdsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the identifiers of the posts are returned in JSON format.<br />
<br />
==== Retrieve the posts a user liked ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllLikedPostsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve the posts he/she liked. <br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllLikedPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
====Retrieve all vre posts====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByVRE<br />
</source><br />
<br />
By means of this method a user that has a token valid for a certain VRE, can retrieve all the posts of that VRE.<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByVRE?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
=== Notifications ===<br />
====Retrieve user's notifications in a given range====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
notifications/getRangeNotificationsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve his/her last notifications in the range [a, b]. The method requires:<br />
<br />
* from: an integer greater than zero;<br />
* quantity: an integer equal or greater than zero.<br />
<br />
The final range is evaluated as [from, from + quantity]<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/notifications/getRangeNotificationsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&from=from&quantity=quantity<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
=== Messages ['''DEPRECATED'''] ===<br />
<br />
====Send a message====<br />
<br />
['''DEPRECATED'''] See version 2 above<br />
<br />
You can send a message via the web service by using the following POST method:<br />
<br />
* writeMessageToUsers<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
messages/writeMessageToUsers<br />
</source><br />
<br />
By default, the sender of the message is the owner of the gcube-token. If you need to specify a different "sender", you can do that by specifying the parameter sender. Furthermore, according to the recipients' notifications preferences, they could receive a portal notification, a mail notification and so on.<br />
<br />
Parameters are:<br />
<br />
* subject: the subject of the message to send;<br />
* body: the body of the message;<br />
* recipients: a list of valid comma separated users' ids (e.g., andrea.rossi, mario.rossi).<br />
* sender: a valid sender's id [optional]<br />
<br />
An example usage of the method is reported in the picture below (without specifying the sender):<br />
<br />
[[File:Send_message_example.png]]<br />
<br />
=== Users ===<br />
====Read user's custom fields====<br />
You can read user's custom attributes using the following method<br />
<br />
*readCustomAttr<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
users/readCustomAttr<br />
</source><br />
<br />
The ''read'' method requires<br />
<br />
* attribute: that is the name of a valid attribute to be read.<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/users/readCustomAttr?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&attribute=attributeX<br />
</source><br />
<br />
= See Also =<br />
<br />
== D4Science Developers website ==<br />
[https://dev.d4science.org D4Science Developers website]<br />
<br />
== SWAGGER documentation==<br />
[https://dev.d4science.org/swagger/social-networking/ SWAGGER documentation of the Social-Networking RESTful interface]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=Social_Networking_Service&diff=33989Social Networking Service2021-04-28T15:42:05Z<p>Massimiliano.assante: /* Discover the Web Service */</p>
<hr />
<div>{| align="right"<br />
| __TOC__<br />
|}<br />
<br />
The Social Networking Service is a RESTful application which exposes operations coming from both the [https://wiki.gcube-system.org/gcube/Social_Networking_Library gCube Social-Networking-Library] and the [https://wiki.gcube-system.org/gcube/UserManagement_Core User Management] components. The former is the bridge between the Cassandra Cluster (on which social data are stored) and the social applications, whereas the latter is the abstraction layer over the enabling technology of the Infrastructure portals, [https://www.liferay.com/ Liferay], in terms of: roles, users and organizations and how they are mapped to gCube concepts (VREs, Teams, and so on).<br />
<br />
The web services exposes a subset of such functionalities over '''https''' in a standard, reliable and secure way. <br />
<br />
See the vailable REST-API at a [https://dev.d4science.org/swagger/social-networking/ glance here.]<br />
= Discover the Web Service =<br />
For gCube releases older than 4.3, a Service Endpoint that has<br />
<br />
<source lang="java"><br />
Name = SocialNetworking<br />
Category = Portal<br />
</source><br />
<br />
mantains the base path of the service. Starting from gCube 4.3 version, the service runs on SmartGears, thus a gCore Endpoint needs to be searched having<br />
<br />
<source lang="java"><br />
Service Name = SocialNetworking<br />
Service Class = Portal<br />
</source><br />
<br />
The base path is the entry point of the web service. In the '''development''' environment, its current value is<br />
<source lang="xml"><br />
https://socialnetworking-d-d4s.d4science.org/social-networking-library-ws/rest or https://api.dev.d4science.org/rest<br />
</source><br />
<br />
In the '''production''' environment, its current value is<br />
<source lang="xml"><br />
https://api.d4science.org/rest<br />
</source><br />
<br />
This url is referred as ''BASE_URL''.<br />
<br />
= Service versions =<br />
Starting from gCube 4.3, the service exposes its methods using a more standard naming approach. Moreover, they accept (in case of http POST/PUT methods) JSON objects. The result of all methods is always a JSON object of this kind<br />
<br />
<source lang="java"><br />
{"success": false/true,<br />
"message": ...,<br />
"result": ...,<br />
}<br />
</source><br />
<br />
Where<br />
* success reports if the request succeeded or failed;<br />
* message is a status/error message that can be checked in case of errors (success equals false);<br />
* result is the current result object (it can be a list, a single object and so on depending on the invoked method). <br />
<br />
Inputs are automatically validated before the request is served.<br />
<br />
The older version (Version 1) instead has some limitation thus you are kindly suggested to no longer use it. However, in the following, its methods are going to be reported as well. The older version will be removed starting from the next gCube releases.<br />
<br />
= Authorization =<br />
No matter the version number of the service, every call to one of its method require a gcube-token. See [https://wiki.gcube-system.org/gcube/Authorization_Framework here] for more information about this token. This secret token needs to be valid in the current context. It can be put as query parameter of the http request or within the header. Both approaches are valid. If the token is missing, the operation will be rejected and an http forbidden code will be returned.<br />
<br />
= Usage =<br />
The methods of the Web Service can be called by writing your own REST client application or by using (a lot of) already existent rest client plugins. In the following, we will use the RESTClient plugin for Firefox for POST methods. Your browser can be used for GET methods. The return value of each method is an HTTP Response object that has an HTTP CODE (200, 201, 400, 500, ...) and, optionally, a JSON or text object that represents the created/retrieved resource or better specifies the occurred error. In case of errors, please refer to the [[#Troubleshooting|Troubleshooting]] section for more details.<br />
<br />
= Service methods (Version 2) =<br />
The list of sub-services available in the new version is going to be reported next. The name convention is different. For instance, to read user's posts, the method to be called is<br />
<br />
<source lang="java"><br />
BASE_PATH/2/posts/get-posts-user/<br />
</source><br />
<br />
The real result of the method, if the request succeeded, is contained into the 'result' field of the json object<br />
<br />
<source lang="java"><br />
{"success": ..., "message": ..., "result": "[{first post}, {second post}, ...}<br />
</source><br />
<br />
In the following, the list of available sub-services available under the newest version, its methods and the parameters they need in order to be called are reported. Again, you can use your preferred REST client so examples won't be reported.<br />
<br />
== List of Available Sub-Services and methods==<br />
=== Tokens ===<br />
====Generate application token====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/tokens/generate-application-token<br />
</source><br />
<br />
First of all, you need to have a ''user gcube-token'' for authentication purposes. You can get it using the Token Generator portlet available on the VRE you are interesting the app will post in.<br />
Then, to get an ''application gcube-token'' you can invoke the method. It is an HTTP/POST method that accepts as parameter:<br />
<br />
* a json object of this kind {"app_id": "your-app-id-goes-here"}.<br />
<br />
=== User profile and roles ===<br />
<br />
To retrieve user's profile user profile info (and roles) the subservice to call, once the social networking service endpoint has been retrieved, is this one:<br />
<br />
<code><br />
/2/people/profile<br />
</code><br />
<br />
further details: [https://wiki.gcube-system.org/gcube/OAuth2.0#Step_4_-_Get_member_data get member information]<br />
<br />
=== Full text search ===<br />
==== Search by query ====<br />
Social data are indexed into an ElasticSearch engine and are querable using a full-text approach. You can retrieve more information about this index [https://wiki.gcube-system.org/index.php?title=Social_Networking_Data_Discovery|here].<br />
The link to append to '''Base_URL''' is the following<br />
<source lang="java"><br />
2/full-text-search/search-by-query<br />
</source><br />
<br />
The query can be submitted also to the service, by using the following parameters<br />
<br />
* query: the query to perform against the index [mandatory];<br />
* from : the result set should be returned by starting from the n-th result [default is zero, optional];<br />
* quantity: how many enhanced posts need to be returned [default is 10, optional].<br />
<br />
=== Hashtags ===<br />
==== Get hashtags and occurrences in a context====<br />
The link to append to '''Base_URL''' is the following<br />
<source lang="java"><br />
2/hashtags/get-hashtags-and-occurrences<br />
</source><br />
<br />
The result is of the kind<br />
{"success": true,<br />
"message": null,<br />
"result":{{"hashtag1" : x}, {"hashtag2" : y}, ...}<br />
}<br />
=== Posts ===<br />
==== Write application post====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/write-post-app<br />
</source><br />
<br />
By means of the application token requested with the above method, the app can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters in a JSON object<br />
<br />
* text: text of the post; <br />
* params: additional parameters if your application supports the direct opening of of this update's object e.g. id=12345&type=foo [optional];<br />
* preview_title: a title for the preview of the post [optional];<br />
* preview_host: a host for the preview of the post [optional];<br />
* preview_description: a description for the preview of the post [optional];<br />
* preview_url: the preview url [optional];<br />
* image_url: if you want to put an image, pass its url [optional];<br />
* enable_notification: a boolean value to enable notifications [optional, default is false].<br />
<br />
====Write user post====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/write-post-user<br />
</source><br />
<br />
By means of a valid user token, he/she can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters in a JSON object:<br />
<br />
* text: text of the post; <br />
* preview_title: a title for the preview of the post [optional];<br />
* preview_description: a description for the preview of the post [optional];<br />
* preview_host: a host for the preview of the post [optional];<br />
* preview_url: the preview url [optional];<br />
* image_url: if you want to put an image, pass its url [optional];<br />
* enable_notification: a boolean value to enable notifications [optional].<br />
<br />
====Retrieve all application posts==== <br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-app<br />
</source><br />
<br />
In order to retrieve all the posts the application wrote (in every VRE it is allowed to write in) you can invoke this HTTP/GET method.<br />
====Retrieve all user posts====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user<br />
</source><br />
<br />
====Retrieve all user posts since a given date====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user-since<br />
</source><br />
<br />
If you are interested in retrieving user's posts whose timestamps are greater than a given date you can invoke this method.<br />
This HTTP/GET method accepts as parameter:<br />
<br />
* time: the time of the oldest post to retrieve converted in milliseconds.<br />
<br />
====Retrieve all user posts up to a given quantity====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user-quantity<br />
</source><br />
<br />
If you are interested in retrieving user's posts, starting from the newest up to a given quantity you can invoke this method.<br />
This HTTP/GET method accepts as parameter:<br />
<br />
* quantity: a value greater than zero (the default value is 10).<br />
<br />
====Retrieve the identifiers of the posts a user liked====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-id-liked-posts<br />
</source><br />
<br />
By means of this method a user can retrieve the identifiers of the posts he/she liked.<br />
====Retrieve the posts a user liked====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-liked-posts<br />
</source><br />
<br />
By means of this method a user can retrieve the posts he/she liked.<br />
====Retrieve all vre posts====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-vre<br />
</source><br />
<br />
By means of this method a user that has a token valid for a certain VRE, can retrieve all the posts of that VRE.<br />
====Get posts by hashtag====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-by-hashtag<br />
</source><br />
The method requires as parameter:<br />
* hashtag: the hashtag to be contained in the returned posts[mandatory].<br />
<br />
=== Comments ===<br />
====Retrieve comments by user====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/comments/get-comments-user<br />
</source><br />
It is an http GET method.<br />
====Retrieve comments by user and time====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/comments/get-comments-user-by-time<br />
</source><br />
It is an http GET method. It requires:<br />
* time: the time (in milliseconds) since when the comments need to be retrieved [default is zero, optional];<br />
=== Notifications ===<br />
====Retrieve user's notifications in a given range====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/notifications/get-range-notifications<br />
</source><br />
<br />
By means of this method a user can retrieve his/her last notifications in the range [a, b]. The method requires:<br />
<br />
* from: an integer greater than zero;<br />
* quantity: an integer equal or greater than zero.<br />
<br />
The final range is evaluated as [from, from + quantity].<br />
====Notify job status====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/notifications/notify-job-status<br />
</source><br />
<br />
By means of this method a user can be notified about the status of a job in the infrastructure. Jobs are, for instance, SmartExecutor's plugins or DataMiner's algorithms. The method requires the following parameters in a JSON object via HTTP POST<br />
<br />
* recipient: the username of the user to be alerted [mandatory];<br />
* job_id: the identifier of the job (e.g, a uuid) [mandatory];<br />
* job_name: the name of the running job (e.g., Social-data-indexer-plugin)[mandatory];<br />
* service_name: the name of the service that launched the job (e.g., SmartExecutor) [mandatory];<br />
* status: the status reached by the job, that is any among CANCELLED, CANCELLING, DELETED, DELETING, EXECUTING, FAILED, SUCCEEDED, SUBMITTED, TIMED_OUT, WAITING [mandatory];<br />
* status_message: a detailed message about the status [optional].<br />
<br />
=== Messages ===<br />
====Send a message====<br />
It is an HTTP POST method.<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/write-message<br />
</source><br />
<br />
By default, the sender of the message is the owner of the gcube-token. According to the recipients' notifications preferences, they could receive a portal notification, an email notification and so on.<br />
<br />
Parameters, to be put in a JSON object are:<br />
<br />
* subject: the subject of the message to send;<br />
* body: the body of the message;<br />
* recipients: a list of valid comma separated users' ids (e.g., andrea.rossi, mario.rossi) or users' emails (e.g., andrea.rossi@isti.cnr.it, mario.rossi@isti.cnr.it);<br />
<br />
An example of the body of the request is as follows<br />
<source lang="xml"><br />
{"subject": "....", "body": ".....", "recipients":[{"id":"userid1"},{"id":"userid2"}]}<br />
</source><br />
<br />
or, if you want to use valid mails (i.e. recognized in the infrastructure)<br />
<source lang="xml"><br />
{"subject": "....", "body": ".....", "recipients":[{"id":"mail1"},{"id":"mail2"}]}<br />
</source><br />
<br />
====Get received messages====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/get-received-messages<br />
</source><br />
It is an http GET method.<br />
<br />
====Get sent messages====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/get-sent-messages<br />
</source><br />
It is an http GET method.<br />
=== Users ===<br />
====User exists====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/user-exists<br />
</source><br />
<br />
The method requires<br />
<br />
* username: the username of the user which has to be checked.<br />
<br />
'''Note''' The method can be only invoked with a root-context application token.<br />
====Read user's custom fields====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-custom-attribute<br />
</source><br />
<br />
The method requires<br />
<br />
* attribute: that is the name of a valid attribute to be read.<br />
====Get user's email====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-email<br />
</source><br />
<br />
The method returns the email associated to the owner of the security token used.<br />
====Get user's profile====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-profile<br />
</source><br />
<br />
The method returns the profile of the owner of the security token used.<br />
====Get user's fullname====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-fullname<br />
</source><br />
<br />
The method returns the fullname associated to the owner of the security token used.<br />
====Get all users' usernames in VRE====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames<br />
</source><br />
<br />
The method returns the usernames of the context associated to the security token.<br />
<br />
====Get usernames by global role====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames-by-global-role<br />
</source><br />
<br />
The method requires<br />
<br />
* role-name: that is the name of the role to consider.<br />
<br />
'''''This method can be only invoked with an Infrastructure Token.'''''<br />
<br />
<br />
====Get usernames by role====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames-by-role<br />
</source><br />
<br />
The method requires<br />
<br />
* role-name: that is the name of the role to consider.<br />
<br />
=== VREs ===<br />
====Get User VREs====<br />
This method allows a user to retrieve the list of VREs she belongs to<br />
<br />
<source lang="java"><br />
2/vres/get-my-vres<br />
</source><br />
<br />
The method accepts an option parameter:<br />
* getManagers : if set to true, reports also the VRE Managers of the VRE<br />
<br />
== Troubleshooting ==<br />
<br />
There could be many reasons for a failed request. Please always check the HTTP CODE and the message within the response. <br />
<br />
=== HTTP Error Code ===<br />
This table tries to help you:<br />
<br />
{| border="1" cellpadding="20" cellspacing="0"<br />
!HTTP CODE<br />
!Meaning<br />
|-<br />
|400 - BAD_REQUEST<br />
|Please check the parameters you are passing<br />
|-<br />
|403 - FORBIDDEN<br />
|Is the token valid? / Does it belong to an application? [In case of application request]<br />
|-<br />
|404 - NOT_FOUND<br />
|The object(s) you are asking for is(are) not available<br />
|-<br />
|500 - INTERNAL_SERVER_ERROR<br />
|There was an error while serving your request (retry later)<br />
|-<br />
|}<br />
<br />
<br />
=== Always manage HTTP Redirects on POST requests (i.e. 30X codes) ===<br />
The service accepts both incoming http and https client requests, which are automatically redirect to https protocol by the reverse proxy nginx. This redirect mechanism must be taken into account when POST calls are performed, because it is not (always) supported and transparent by clients. In order to be sure, please check if you get a 30X error and extract the redirection urls within the response header. The following is a working java code that extract such information. It uses the apache-client library to perform http calls.<br />
<br />
<source lang="java"><br />
<br />
// build the client and the request<br />
String path = ... ; <br />
HttpClientBuilder clientBuilder = HttpClientBuilder.create();<br />
CloseableHttpClient httpClient = clientBuilder.build();<br />
HttpPost request = new HttpPost(path);<br />
JSONObject obj = new JSONObject();<br />
<br />
... // build the json object<br />
<br />
// set header properties<br />
request.addHeader("gcube-token", ...);<br />
request.addHeader("Content-type", ContentType.APPLICATION_JSON.toString());<br />
StringEntity params = new StringEntity(obj.toJSONString(), ContentType.APPLICATION_JSON);<br />
request.setEntity(params);<br />
HttpResponse response = httpClient.execute(request);<br />
logger.info(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
<br />
int status = response.getStatusLine().getStatusCode();<br />
<br />
// check the response status and look if it was a redirect problem<br />
if (status != HttpURLConnection.HTTP_OK && (status == HttpURLConnection.HTTP_MOVED_TEMP ||<br />
status == HttpURLConnection.HTTP_MOVED_PERM ||<br />
status == HttpURLConnection.HTTP_SEE_OTHER)) {<br />
<br />
// redirect -> fetch new location<br />
Header[] locations = response.getHeaders("Location");<br />
Header lastLocation = locations[locations.length - 1];<br />
String realLocation = lastLocation.getValue();<br />
logger.debug("New location is " + realLocation);<br />
<br />
// perform again the same request<br />
request = new HttpPost(realLocation);<br />
request.addHeader("gcube-token", ...);<br />
request.addHeader("Content-type", ContentType.APPLICATION_JSON.toString());<br />
params = new StringEntity(obj.toJSONString(), ContentType.APPLICATION_JSON);<br />
request.setEntity(params);<br />
response = httpClient.execute(request);<br />
logger.info(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
} else<br />
logger.debug(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
</source><br />
<br />
= Service methods (Version 1) - DEPRECATED =<br />
In the following, the list of available sub-services available under the first version, its methods and the parameters they need in order to be called are reported. The gcube-token is always required so it won't be reported as parameter. <br />
<br />
'''This version has been replaced by version 2, please do not use this version and update your code asap.'''<br />
<br />
== List of available Sub-Services and methods ==<br />
=== Tokens ===<br />
==== Generate application token ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
tokens/generateApplicationToken<br />
</source><br />
<br />
First of all, you need to have a ''user gcube-token'' for authentication purposes. You can get it using the Token Generator portlet available on the VRE you are interesting the app will post in.<br />
Now, to get an ''application gcube-token'' you can invoke the method. It is an HTTP/POST method that accepts as parameters:<br />
<br />
* appid: the identifier of the application as reported in the Application Profile.<br />
<br />
For example, using the RESTClient plugin for Firefox you will have something like this for requiring a new application token:<br />
<br />
[[File:GenerateApplicationToken.png]]<br />
<br />
Push on the SEND button and if all it's ok, your application gcube-token will be generated and returned as ''plain text'' in the response within the HTTP response having code 201.<br />
=== Posts ['''DEPRECATED'''] ===<br />
==== Write application post ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/writePostApplication<br />
</source><br />
<br />
By means of the application token requested with the above method, the app can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters:<br />
<br />
* text: text of the post; <br />
* params: additional parameters if your application supports the direct opening of of this update's object e.g. id=12345&type=foo [optional];<br />
* previewtitle: a title for the preview of the post [optional];<br />
* previewdescription: a description for the preview of the post [optional];<br />
* httpimageurl: if you want to put an image, pass its url [optional];<br />
* enablenotification: a boolean value to enable notifications [optional].<br />
<br />
For example:<br />
<br />
[[File:WriteApplicationPost.png]]<br />
<br />
Push on SEND and check the code associated to the returned response (201 in case of success).<br />
<br />
==== Retrieve all application posts ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByApp<br />
</source><br />
<br />
In order to retrieve all the posts the application wrote (in every VRE it is allowed to write in) you can invoke this HTTP/GET method.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByApp?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByUser<br />
</source><br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts since a given date ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getRecentPostsByUserAndDate<br />
</source><br />
<br />
If you are interested in retrieving user's posts whose timestamps are greater than a given date you can invoke this method.<br />
This HTTP/GET method accepts this parameter:<br />
<br />
* time: the time of the oldest post to retrieve converted in milliseconds.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getRecentPostsByUserAndDate?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&time=xxxxx<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts up to a given quantity ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getRecentPostsByUser<br />
</source><br />
<br />
If you are interested in retrieving user's posts, starting from the newest up to a given quantity you can invoke this method.<br />
This HTTP/GET method accepts two parameter:<br />
<br />
* quantity: a value greater than zero (the default value is 10).<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getRecentPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&quantity=20<br />
</source><br />
<br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Write user post ['''DEPRECATED'''] ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/writePostUser<br />
</source><br />
<br />
By means of a valid user token, he/she can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters:<br />
<br />
* text: text of the post; <br />
* previewtitle: a title for the preview of the post [optional];<br />
* previewdescription: a description for the preview of the post [optional];<br />
* previewHost: [optional];<br />
* previewUrl: [optional];<br />
* httpimageurl: if you want to put an image, pass its url [optional];<br />
* enablenotification: a boolean value to enable notifications [optional];<br />
<br />
For example:<br />
<br />
[[File:Userpost.png]]<br />
<br />
Push on SEND and check the code associated to the returned response (201 in case of success).<br />
<br />
==== Retrieve the identifiers of the posts a user liked ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllLikedPostIdsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve the identifiers of the posts he/she liked.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllLikedPostIdsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the identifiers of the posts are returned in JSON format.<br />
<br />
==== Retrieve the posts a user liked ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllLikedPostsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve the posts he/she liked. <br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllLikedPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
====Retrieve all vre posts====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByVRE<br />
</source><br />
<br />
By means of this method a user that has a token valid for a certain VRE, can retrieve all the posts of that VRE.<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByVRE?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
=== Notifications ===<br />
====Retrieve user's notifications in a given range====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
notifications/getRangeNotificationsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve his/her last notifications in the range [a, b]. The method requires:<br />
<br />
* from: an integer greater than zero;<br />
* quantity: an integer equal or greater than zero.<br />
<br />
The final range is evaluated as [from, from + quantity]<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/notifications/getRangeNotificationsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&from=from&quantity=quantity<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
=== Messages ['''DEPRECATED'''] ===<br />
<br />
====Send a message====<br />
<br />
['''DEPRECATED'''] See version 2 above<br />
<br />
You can send a message via the web service by using the following POST method:<br />
<br />
* writeMessageToUsers<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
messages/writeMessageToUsers<br />
</source><br />
<br />
By default, the sender of the message is the owner of the gcube-token. If you need to specify a different "sender", you can do that by specifying the parameter sender. Furthermore, according to the recipients' notifications preferences, they could receive a portal notification, a mail notification and so on.<br />
<br />
Parameters are:<br />
<br />
* subject: the subject of the message to send;<br />
* body: the body of the message;<br />
* recipients: a list of valid comma separated users' ids (e.g., andrea.rossi, mario.rossi).<br />
* sender: a valid sender's id [optional]<br />
<br />
An example usage of the method is reported in the picture below (without specifying the sender):<br />
<br />
[[File:Send_message_example.png]]<br />
<br />
=== Users ===<br />
====Read user's custom fields====<br />
You can read user's custom attributes using the following method<br />
<br />
*readCustomAttr<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
users/readCustomAttr<br />
</source><br />
<br />
The ''read'' method requires<br />
<br />
* attribute: that is the name of a valid attribute to be read.<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/users/readCustomAttr?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&attribute=attributeX<br />
</source><br />
<br />
= See Also =<br />
<br />
== D4Science Developers website ==<br />
[https://dev.d4science.org D4Science Developers website]<br />
<br />
== SWAGGER documentation==<br />
[https://dev.d4science.org/swagger/social-networking/ SWAGGER documentation of the Social-Networking RESTful interface]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=StorageHub_REST_API&diff=33977StorageHub REST API2021-04-14T13:24:45Z<p>Massimiliano.assante: /* REST API */</p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
= Overview =<br />
<br />
The StorageHub APIs components provide a simple access to StorageHub service. <br />
<br />
It is conceived to support both Java and REST-based calls. <br />
<br />
== Dependencies ==<br />
<br />
=== Maven coordinates ===<br />
<br />
<source lang="java"><br />
<dependency><br />
<groupId>org.gcube.common</groupId><br />
<artifactId>storagehub-client-library</artifactId><br />
<version>[1.0.0,2.0.0)</version><br />
</dependency><br />
</source><br />
<br />
= Key features =<br />
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.<br />
<br />
'''StorageHub REST interface''' supports the following operations:<br />
* '''Retrieve WS''': to retrieve the user Workspace;<br />
* '''Folder Listing''': to list the content of a folder;<br />
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;<br />
* '''Find''': to find a file or a folder by name or pattern;<br />
* '''Delete''': to remove a file or a folder (including subfolders);<br />
* '''Download''': to download a file or a folder in ZIP format;<br />
* '''Get Public Link''': to get a public link of a file;<br />
* '''Create Folder''': to create a folder in the given parent folder;<br />
* '''Unzip''': to upload a zip file in a specific folder;<br />
* '''Upload file''': to upload a file in a folder.<br />
* '''Versions''': to get a specific version of a file;<br />
<br />
== API ==<br />
<br />
The base-url of this service on D4Science production is the following:<br />
<br />
https://api.d4science.org/workspace<br />
<br />
== Retrieve Workspace ==<br />
<br />
Returns the Item representing the workspace of the user retrieved by the token used for the call.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.getWSRoot()<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The workspace root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Get Item ById ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
</source><br />
<br />
<br />
== List Item Versions, Get Item Version ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
import org.gcube.common.storagehub.model.service.Version;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
List<Version> fileVersions = fileContainer.getVersions();<br />
<br />
//to download a version<br />
fileContainer.downloadSpecificVersion(versionName)<br />
</source><br />
<br />
== Folder Listing ==<br />
<br />
Returns the content of a Folder<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
import org.gcube.common.storagehub.model.items.*;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
List<? extends Item> items = folderContainer.list().getItems();<br />
<br />
for (Item item : items) {<br />
System.out.println("name:" + item.getName() + " is a File?: " + (item instanceof AbstractFileItem));<br />
System.out.println("name:" + item.getName() + " is a folder?: " + (item instanceof FolderItem));<br />
System.out.println("name:" + item.getName() + " is a shared folder?: " + (item instanceof SharedFolder));<br />
System.out.println("name:" + item.getName() + " is a VRE folder?: " + (item instanceof VreFolder));<br />
}<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}<br />
<br />
==== responses ====<br />
<br />
200 the list of the Items (in JSON format)<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Retrieve VRE Folder ==<br />
<br />
Return the Item representing the root of the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.openVREFolder();<br />
</source><br />
<br />
<br />
Check that a folder with name "FolderNameToCheck" exists in the VRE folder related to the user token.<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.*;<br />
import org.gcube.common.storagehub.model.exceptions.StorageHubException;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer vreFolder = shc.openVREFolder();<br />
FolderContainer folderToCheck = null;<br />
try { <br />
OpenResolver oRes = vreFolder.openByRelativePath("FolderNameToCheck");<br />
folderToCheck = oRes.asFolder();<br />
} catch (StorageHubException e) {<br />
System.out.println("FolderNameToCheck does not exists in VRE Folder, creating it");<br />
folderToCheck = vreFolder.newFolder("FolderNameToCheck description goes here");<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/vrefolder?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The VRE folder root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Create Folder ==<br />
<br />
Creates a new folder under another folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
root.newFolder("name", "description");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} <br />
Content-Type: application/x-www-form-urlencoded <br />
{String name, String description, boolean hidden}<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder item identifier is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Delete File or Folder ==<br />
<br />
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;<br />
<br />
=== Java ===<br />
<source lang="java"><br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
fileContainer.delete();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)<br />
<br />
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder or item is deleted<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Download File or Folder ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
StreamDescriptor streamDescr = shc.open(theId).asFile().download();<br />
<br />
File output = {output file};<br />
try (BufferedInputStream bi = new BufferedInputStream(streamDescr.getStream()); FileOutputStream fo = new FileOutputStream(output)){<br />
byte[] buf = new byte[2046]; <br />
int read = -1;<br />
while ((read=bi.read(buf))!=-1) {<br />
fo.write(buf, 0, read);<br />
}<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{itemId}/download?gcube-token={user-token}<br />
<br />
'''Please note:''' in case the itemId corresponds to a folder you will get a compressed file (zip) of the folder with all its content.<br />
<br />
== Upload File ==<br />
<br />
Creates a new file under a folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadFile(is, "name", "description");<br />
System.out.println("Uploaded " + file.get().getName() + " - Returned Workspace id=" + file.getId()); <br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String name, String description, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}<br />
<br />
== Upload Archive ==<br />
<br />
Extract the content of the specified archive in a destination Folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadArchive(is, "parentFolderName");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String parentFolderName, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}<br />
<br />
== Get Public Url of a File ==<br />
<br />
Returns the public URL of a file<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open({item-id}).asFile().getPublicLink();<br />
<br />
//returns the public link for a specific version of the file <br />
shc.open({item-id}).asFile().getPublicLink({file-version});<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional<br />
<br />
== Find By Name ==<br />
<br />
Returns all the items with a name that matches a pattern on the first level of a given folder. <br />
The result will be empty in case of none of the item in the folder matches the pattern.<br />
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) <br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
//getting my root workspace folder<br />
FolderContainer myRoot = shc.getWSRoot();<br />
myRoot.findByName("{pattern}");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} <br />
<br />
==== responses ====<br />
<br />
200 A JSON with the retrieved items or an empty itemList. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
= DataMiner and SAI Interactions =<br />
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. <br />
Here are some examples:<br />
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]<br />
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]<br />
<br />
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].<br />
<br />
= R Client =<br />
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=StorageHub_REST_API&diff=33976StorageHub REST API2021-04-14T13:24:15Z<p>Massimiliano.assante: /* REST API */</p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
= Overview =<br />
<br />
The StorageHub APIs components provide a simple access to StorageHub service. <br />
<br />
It is conceived to support both Java and REST-based calls. <br />
<br />
== Dependencies ==<br />
<br />
=== Maven coordinates ===<br />
<br />
<source lang="java"><br />
<dependency><br />
<groupId>org.gcube.common</groupId><br />
<artifactId>storagehub-client-library</artifactId><br />
<version>[1.0.0,2.0.0)</version><br />
</dependency><br />
</source><br />
<br />
= Key features =<br />
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.<br />
<br />
'''StorageHub REST interface''' supports the following operations:<br />
* '''Retrieve WS''': to retrieve the user Workspace;<br />
* '''Folder Listing''': to list the content of a folder;<br />
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;<br />
* '''Find''': to find a file or a folder by name or pattern;<br />
* '''Delete''': to remove a file or a folder (including subfolders);<br />
* '''Download''': to download a file or a folder in ZIP format;<br />
* '''Get Public Link''': to get a public link of a file;<br />
* '''Create Folder''': to create a folder in the given parent folder;<br />
* '''Unzip''': to upload a zip file in a specific folder;<br />
* '''Upload file''': to upload a file in a folder.<br />
* '''Versions''': to get a specific version of a file;<br />
<br />
== API ==<br />
<br />
The base-url of this service on D4Science production is the following:<br />
<br />
https://api.d4science.org/workspace<br />
<br />
== Retrieve Workspace ==<br />
<br />
Returns the Item representing the workspace of the user retrieved by the token used for the call.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.getWSRoot()<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The workspace root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Get Item ById ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
</source><br />
<br />
<br />
== List Item Versions, Get Item Version ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
import org.gcube.common.storagehub.model.service.Version;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
List<Version> fileVersions = fileContainer.getVersions();<br />
<br />
//to download a version<br />
fileContainer.downloadSpecificVersion(versionName)<br />
</source><br />
<br />
== Folder Listing ==<br />
<br />
Returns the content of a Folder<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
import org.gcube.common.storagehub.model.items.*;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
List<? extends Item> items = folderContainer.list().getItems();<br />
<br />
for (Item item : items) {<br />
System.out.println("name:" + item.getName() + " is a File?: " + (item instanceof AbstractFileItem));<br />
System.out.println("name:" + item.getName() + " is a folder?: " + (item instanceof FolderItem));<br />
System.out.println("name:" + item.getName() + " is a shared folder?: " + (item instanceof SharedFolder));<br />
System.out.println("name:" + item.getName() + " is a VRE folder?: " + (item instanceof VreFolder));<br />
}<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}<br />
<br />
==== responses ====<br />
<br />
200 the list of the Items (in JSON format)<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Retrieve VRE Folder ==<br />
<br />
Return the Item representing the root of the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.openVREFolder();<br />
</source><br />
<br />
<br />
Check that a folder with name "FolderNameToCheck" exists in the VRE folder related to the user token.<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.*;<br />
import org.gcube.common.storagehub.model.exceptions.StorageHubException;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer vreFolder = shc.openVREFolder();<br />
FolderContainer folderToCheck = null;<br />
try { <br />
OpenResolver oRes = vreFolder.openByRelativePath("FolderNameToCheck");<br />
folderToCheck = oRes.asFolder();<br />
} catch (StorageHubException e) {<br />
System.out.println("FolderNameToCheck does not exists in VRE Folder, creating it");<br />
folderToCheck = vreFolder.newFolder("FolderNameToCheck description goes here");<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/vrefolder?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The VRE folder root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Create Folder ==<br />
<br />
Creates a new folder under another folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
root.newFolder("name", "description");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} <br />
Content-Type: application/x-www-form-urlencoded <br />
{String name, String description, boolean hidden}<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder item identifier is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Delete File or Folder ==<br />
<br />
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;<br />
<br />
=== Java ===<br />
<source lang="java"><br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
fileContainer.delete();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)<br />
<br />
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder or item is deleted<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Download File or Folder ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
StreamDescriptor streamDescr = shc.open(theId).asFile().download();<br />
<br />
File output = {output file};<br />
try (BufferedInputStream bi = new BufferedInputStream(streamDescr.getStream()); FileOutputStream fo = new FileOutputStream(output)){<br />
byte[] buf = new byte[2046]; <br />
int read = -1;<br />
while ((read=bi.read(buf))!=-1) {<br />
fo.write(buf, 0, read);<br />
}<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{itemId}/download?gcube-token={user-token}<br />
<br />
'''Please note:''' in case the itemId corresponds to a folder you will get the a compressed file (zip) of the folder with all its content.<br />
<br />
== Upload File ==<br />
<br />
Creates a new file under a folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadFile(is, "name", "description");<br />
System.out.println("Uploaded " + file.get().getName() + " - Returned Workspace id=" + file.getId()); <br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String name, String description, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}<br />
<br />
== Upload Archive ==<br />
<br />
Extract the content of the specified archive in a destination Folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadArchive(is, "parentFolderName");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String parentFolderName, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}<br />
<br />
== Get Public Url of a File ==<br />
<br />
Returns the public URL of a file<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open({item-id}).asFile().getPublicLink();<br />
<br />
//returns the public link for a specific version of the file <br />
shc.open({item-id}).asFile().getPublicLink({file-version});<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional<br />
<br />
== Find By Name ==<br />
<br />
Returns all the items with a name that matches a pattern on the first level of a given folder. <br />
The result will be empty in case of none of the item in the folder matches the pattern.<br />
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) <br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
//getting my root workspace folder<br />
FolderContainer myRoot = shc.getWSRoot();<br />
myRoot.findByName("{pattern}");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} <br />
<br />
==== responses ====<br />
<br />
200 A JSON with the retrieved items or an empty itemList. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
= DataMiner and SAI Interactions =<br />
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. <br />
Here are some examples:<br />
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]<br />
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]<br />
<br />
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].<br />
<br />
= R Client =<br />
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=StorageHub_REST_API&diff=33975StorageHub REST API2021-04-14T13:23:33Z<p>Massimiliano.assante: /* REST API */</p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
= Overview =<br />
<br />
The StorageHub APIs components provide a simple access to StorageHub service. <br />
<br />
It is conceived to support both Java and REST-based calls. <br />
<br />
== Dependencies ==<br />
<br />
=== Maven coordinates ===<br />
<br />
<source lang="java"><br />
<dependency><br />
<groupId>org.gcube.common</groupId><br />
<artifactId>storagehub-client-library</artifactId><br />
<version>[1.0.0,2.0.0)</version><br />
</dependency><br />
</source><br />
<br />
= Key features =<br />
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.<br />
<br />
'''StorageHub REST interface''' supports the following operations:<br />
* '''Retrieve WS''': to retrieve the user Workspace;<br />
* '''Folder Listing''': to list the content of a folder;<br />
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;<br />
* '''Find''': to find a file or a folder by name or pattern;<br />
* '''Delete''': to remove a file or a folder (including subfolders);<br />
* '''Download''': to download a file or a folder in ZIP format;<br />
* '''Get Public Link''': to get a public link of a file;<br />
* '''Create Folder''': to create a folder in the given parent folder;<br />
* '''Unzip''': to upload a zip file in a specific folder;<br />
* '''Upload file''': to upload a file in a folder.<br />
* '''Versions''': to get a specific version of a file;<br />
<br />
== API ==<br />
<br />
The base-url of this service on D4Science production is the following:<br />
<br />
https://api.d4science.org/workspace<br />
<br />
== Retrieve Workspace ==<br />
<br />
Returns the Item representing the workspace of the user retrieved by the token used for the call.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.getWSRoot()<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The workspace root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Get Item ById ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
</source><br />
<br />
<br />
== List Item Versions, Get Item Version ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
import org.gcube.common.storagehub.model.service.Version;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
List<Version> fileVersions = fileContainer.getVersions();<br />
<br />
//to download a version<br />
fileContainer.downloadSpecificVersion(versionName)<br />
</source><br />
<br />
== Folder Listing ==<br />
<br />
Returns the content of a Folder<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
import org.gcube.common.storagehub.model.items.*;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
List<? extends Item> items = folderContainer.list().getItems();<br />
<br />
for (Item item : items) {<br />
System.out.println("name:" + item.getName() + " is a File?: " + (item instanceof AbstractFileItem));<br />
System.out.println("name:" + item.getName() + " is a folder?: " + (item instanceof FolderItem));<br />
System.out.println("name:" + item.getName() + " is a shared folder?: " + (item instanceof SharedFolder));<br />
System.out.println("name:" + item.getName() + " is a VRE folder?: " + (item instanceof VreFolder));<br />
}<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}<br />
<br />
==== responses ====<br />
<br />
200 the list of the Items (in JSON format)<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Retrieve VRE Folder ==<br />
<br />
Return the Item representing the root of the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.openVREFolder();<br />
</source><br />
<br />
<br />
Check that a folder with name "FolderNameToCheck" exists in the VRE folder related to the user token.<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.*;<br />
import org.gcube.common.storagehub.model.exceptions.StorageHubException;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer vreFolder = shc.openVREFolder();<br />
FolderContainer folderToCheck = null;<br />
try { <br />
OpenResolver oRes = vreFolder.openByRelativePath("FolderNameToCheck");<br />
folderToCheck = oRes.asFolder();<br />
} catch (StorageHubException e) {<br />
System.out.println("FolderNameToCheck does not exists in VRE Folder, creating it");<br />
folderToCheck = vreFolder.newFolder("FolderNameToCheck description goes here");<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/vrefolder?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The VRE folder root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Create Folder ==<br />
<br />
Creates a new folder under another folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
root.newFolder("name", "description");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} <br />
Content-Type: application/x-www-form-urlencoded <br />
{String name, String description, boolean hidden}<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder item identifier is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Delete File or Folder ==<br />
<br />
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;<br />
<br />
=== Java ===<br />
<source lang="java"><br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
fileContainer.delete();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)<br />
<br />
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder or item is deleted<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Download File or Folder ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
StreamDescriptor streamDescr = shc.open(theId).asFile().download();<br />
<br />
File output = {output file};<br />
try (BufferedInputStream bi = new BufferedInputStream(streamDescr.getStream()); FileOutputStream fo = new FileOutputStream(output)){<br />
byte[] buf = new byte[2046]; <br />
int read = -1;<br />
while ((read=bi.read(buf))!=-1) {<br />
fo.write(buf, 0, read);<br />
}<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{itemId}/download?gcube-token={user-token}<br />
<br />
'''Please note: if the '''itemId corresponds to a folder you will get the a compressed file (zip) of the folder with all its content.<br />
<br />
== Upload File ==<br />
<br />
Creates a new file under a folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadFile(is, "name", "description");<br />
System.out.println("Uploaded " + file.get().getName() + " - Returned Workspace id=" + file.getId()); <br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String name, String description, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}<br />
<br />
== Upload Archive ==<br />
<br />
Extract the content of the specified archive in a destination Folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadArchive(is, "parentFolderName");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String parentFolderName, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}<br />
<br />
== Get Public Url of a File ==<br />
<br />
Returns the public URL of a file<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open({item-id}).asFile().getPublicLink();<br />
<br />
//returns the public link for a specific version of the file <br />
shc.open({item-id}).asFile().getPublicLink({file-version});<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional<br />
<br />
== Find By Name ==<br />
<br />
Returns all the items with a name that matches a pattern on the first level of a given folder. <br />
The result will be empty in case of none of the item in the folder matches the pattern.<br />
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) <br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
//getting my root workspace folder<br />
FolderContainer myRoot = shc.getWSRoot();<br />
myRoot.findByName("{pattern}");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} <br />
<br />
==== responses ====<br />
<br />
200 A JSON with the retrieved items or an empty itemList. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
= DataMiner and SAI Interactions =<br />
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. <br />
Here are some examples:<br />
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]<br />
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]<br />
<br />
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].<br />
<br />
= R Client =<br />
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=StorageHub_REST_API&diff=33974StorageHub REST API2021-04-14T13:22:26Z<p>Massimiliano.assante: </p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
= Overview =<br />
<br />
The StorageHub APIs components provide a simple access to StorageHub service. <br />
<br />
It is conceived to support both Java and REST-based calls. <br />
<br />
== Dependencies ==<br />
<br />
=== Maven coordinates ===<br />
<br />
<source lang="java"><br />
<dependency><br />
<groupId>org.gcube.common</groupId><br />
<artifactId>storagehub-client-library</artifactId><br />
<version>[1.0.0,2.0.0)</version><br />
</dependency><br />
</source><br />
<br />
= Key features =<br />
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.<br />
<br />
'''StorageHub REST interface''' supports the following operations:<br />
* '''Retrieve WS''': to retrieve the user Workspace;<br />
* '''Folder Listing''': to list the content of a folder;<br />
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;<br />
* '''Find''': to find a file or a folder by name or pattern;<br />
* '''Delete''': to remove a file or a folder (including subfolders);<br />
* '''Download''': to download a file or a folder in ZIP format;<br />
* '''Get Public Link''': to get a public link of a file;<br />
* '''Create Folder''': to create a folder in the given parent folder;<br />
* '''Unzip''': to upload a zip file in a specific folder;<br />
* '''Upload file''': to upload a file in a folder.<br />
* '''Versions''': to get a specific version of a file;<br />
<br />
== API ==<br />
<br />
The base-url of this service on D4Science production is the following:<br />
<br />
https://api.d4science.org/workspace<br />
<br />
== Retrieve Workspace ==<br />
<br />
Returns the Item representing the workspace of the user retrieved by the token used for the call.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.getWSRoot()<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The workspace root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Get Item ById ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
</source><br />
<br />
<br />
== List Item Versions, Get Item Version ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
import org.gcube.common.storagehub.model.service.Version;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
List<Version> fileVersions = fileContainer.getVersions();<br />
<br />
//to download a version<br />
fileContainer.downloadSpecificVersion(versionName)<br />
</source><br />
<br />
== Folder Listing ==<br />
<br />
Returns the content of a Folder<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
import org.gcube.common.storagehub.model.items.*;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
List<? extends Item> items = folderContainer.list().getItems();<br />
<br />
for (Item item : items) {<br />
System.out.println("name:" + item.getName() + " is a File?: " + (item instanceof AbstractFileItem));<br />
System.out.println("name:" + item.getName() + " is a folder?: " + (item instanceof FolderItem));<br />
System.out.println("name:" + item.getName() + " is a shared folder?: " + (item instanceof SharedFolder));<br />
System.out.println("name:" + item.getName() + " is a VRE folder?: " + (item instanceof VreFolder));<br />
}<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}<br />
<br />
==== responses ====<br />
<br />
200 the list of the Items (in JSON format)<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Retrieve VRE Folder ==<br />
<br />
Return the Item representing the root of the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.openVREFolder();<br />
</source><br />
<br />
<br />
Check that a folder with name "FolderNameToCheck" exists in the VRE folder related to the user token.<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.*;<br />
import org.gcube.common.storagehub.model.exceptions.StorageHubException;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer vreFolder = shc.openVREFolder();<br />
FolderContainer folderToCheck = null;<br />
try { <br />
OpenResolver oRes = vreFolder.openByRelativePath("FolderNameToCheck");<br />
folderToCheck = oRes.asFolder();<br />
} catch (StorageHubException e) {<br />
System.out.println("FolderNameToCheck does not exists in VRE Folder, creating it");<br />
folderToCheck = vreFolder.newFolder("FolderNameToCheck description goes here");<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/vrefolder?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The VRE folder root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Create Folder ==<br />
<br />
Creates a new folder under another folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
root.newFolder("name", "description");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} <br />
Content-Type: application/x-www-form-urlencoded <br />
{String name, String description, boolean hidden}<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder item identifier is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Delete File or Folder ==<br />
<br />
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;<br />
<br />
=== Java ===<br />
<source lang="java"><br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
fileContainer.delete();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)<br />
<br />
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder or item is deleted<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Download File or Folder ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
StreamDescriptor streamDescr = shc.open(theId).asFile().download();<br />
<br />
File output = {output file};<br />
try (BufferedInputStream bi = new BufferedInputStream(streamDescr.getStream()); FileOutputStream fo = new FileOutputStream(output)){<br />
byte[] buf = new byte[2046]; <br />
int read = -1;<br />
while ((read=bi.read(buf))!=-1) {<br />
fo.write(buf, 0, read);<br />
}<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{itemId}/download?gcube-token={user-token}<br />
<br />
'''Please note:'''<br />
<br />
== Upload File ==<br />
<br />
Creates a new file under a folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadFile(is, "name", "description");<br />
System.out.println("Uploaded " + file.get().getName() + " - Returned Workspace id=" + file.getId()); <br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String name, String description, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}<br />
<br />
== Upload Archive ==<br />
<br />
Extract the content of the specified archive in a destination Folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadArchive(is, "parentFolderName");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String parentFolderName, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}<br />
<br />
== Get Public Url of a File ==<br />
<br />
Returns the public URL of a file<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open({item-id}).asFile().getPublicLink();<br />
<br />
//returns the public link for a specific version of the file <br />
shc.open({item-id}).asFile().getPublicLink({file-version});<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional<br />
<br />
== Find By Name ==<br />
<br />
Returns all the items with a name that matches a pattern on the first level of a given folder. <br />
The result will be empty in case of none of the item in the folder matches the pattern.<br />
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) <br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
//getting my root workspace folder<br />
FolderContainer myRoot = shc.getWSRoot();<br />
myRoot.findByName("{pattern}");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} <br />
<br />
==== responses ====<br />
<br />
200 A JSON with the retrieved items or an empty itemList. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
= DataMiner and SAI Interactions =<br />
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. <br />
Here are some examples:<br />
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]<br />
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]<br />
<br />
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].<br />
<br />
= R Client =<br />
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=StorageHub_REST_API&diff=33963StorageHub REST API2021-04-02T10:25:01Z<p>Massimiliano.assante: /* Java */</p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
= Overview =<br />
<br />
The StorageHub APIs components provide a simple access to StorageHub service. <br />
<br />
It is conceived to support both Java and REST-based calls. <br />
<br />
== Dependencies ==<br />
<br />
=== Maven coordinates ===<br />
<br />
<source lang="java"><br />
<dependency><br />
<groupId>org.gcube.common</groupId><br />
<artifactId>storagehub-client-library</artifactId><br />
<version>[1.0.0,2.0.0)</version><br />
</dependency><br />
</source><br />
<br />
= Key features =<br />
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.<br />
<br />
'''StorageHub REST interface''' supports the following operations:<br />
* '''Retrieve WS''': to retrieve the user Workspace;<br />
* '''Folder Listing''': to list the content of a folder;<br />
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;<br />
* '''Find''': to find a file or a folder by name or pattern;<br />
* '''Delete''': to remove a file or a folder (including subfolders);<br />
* '''Download''': to download a file or a folder in ZIP format;<br />
* '''Get Public Link''': to get a public link of a file;<br />
* '''Create Folder''': to create a folder in the given parent folder;<br />
* '''Unzip''': to upload a zip file in a specific folder;<br />
* '''Upload file''': to upload a file in a folder.<br />
* '''Versions''': to get a specific version of a file;<br />
<br />
== API ==<br />
<br />
The base-url of this service on D4Science production is the following:<br />
<br />
https://api.d4science.org/workspace<br />
<br />
== Retrieve Workspace ==<br />
<br />
Returns the Item representing the workspace of the user retrieved by the token used for the call.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.getWSRoot()<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The workspace root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Get Item ById ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
</source><br />
<br />
== List Item Versions, Get Item Version ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
import org.gcube.common.storagehub.model.service.Version;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
List<Version> fileVersions = fileContainer.getVersions();<br />
<br />
//to download a version<br />
fileContainer.downloadSpecificVersion(versionName)<br />
</source><br />
<br />
== Folder Listing ==<br />
<br />
Returns the content of a Folder<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
import org.gcube.common.storagehub.model.items.*;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
List<? extends Item> items = folderContainer.list().getItems();<br />
<br />
for (Item item : items) {<br />
System.out.println("name:" + item.getName() + " is a File?: " + (item instanceof AbstractFileItem));<br />
System.out.println("name:" + item.getName() + " is a folder?: " + (item instanceof FolderItem));<br />
System.out.println("name:" + item.getName() + " is a shared folder?: " + (item instanceof SharedFolder));<br />
System.out.println("name:" + item.getName() + " is a VRE folder?: " + (item instanceof VreFolder));<br />
}<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}<br />
<br />
==== responses ====<br />
<br />
200 the list of the Items (in JSON format)<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Retrieve VRE Folder ==<br />
<br />
Return the Item representing the root of the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.openVREFolder();<br />
</source><br />
<br />
<br />
Check that a folder with name "FolderNameToCheck" exists in the VRE folder related to the user token.<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.*;<br />
import org.gcube.common.storagehub.model.exceptions.StorageHubException;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer vreFolder = shc.openVREFolder();<br />
FolderContainer folderToCheck = null;<br />
try { <br />
OpenResolver oRes = vreFolder.openByRelativePath("FolderNameToCheck");<br />
folderToCheck = oRes.asFolder();<br />
} catch (StorageHubException e) {<br />
System.out.println("FolderNameToCheck does not exists in VRE Folder, creating it");<br />
folderToCheck = vreFolder.newFolder("FolderNameToCheck description goes here");<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/vrefolder?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The VRE folder root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Create Folder ==<br />
<br />
Creates a new folder under another folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
root.newFolder("name", "description");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} <br />
Content-Type: application/x-www-form-urlencoded <br />
{String name, String description, boolean hidden}<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder item identifier is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Delete File or Folder ==<br />
<br />
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;<br />
<br />
=== Java ===<br />
<source lang="java"><br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
fileContainer.delete();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)<br />
<br />
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder or item is deleted<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Upload File ==<br />
<br />
Creates a new file under a folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadFile(is, "name", "description");<br />
System.out.println("Uploaded " + file.get().getName() + " - Returned Workspace id=" + file.getId()); <br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String name, String description, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}<br />
<br />
== Upload Archive ==<br />
<br />
Extract the content of the specified archive in a destination Folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadArchive(is, "parentFolderName");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String parentFolderName, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}<br />
<br />
== Get Public Url of a File ==<br />
<br />
Returns the public URL of a file<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open({item-id}).asFile().getPublicLink();<br />
<br />
//returns the public link for a specific version of the file <br />
shc.open({item-id}).asFile().getPublicLink({file-version});<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional<br />
<br />
== Find By Name ==<br />
<br />
Returns all the items with a name that matches a pattern on the first level of a given folder. <br />
The result will be empty in case of none of the item in the folder matches the pattern.<br />
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) <br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
//getting my root workspace folder<br />
FolderContainer myRoot = shc.getWSRoot();<br />
myRoot.findByName("{pattern}");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} <br />
<br />
==== responses ====<br />
<br />
200 A JSON with the retrieved items or an empty itemList. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
= DataMiner and SAI Interactions =<br />
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. <br />
Here are some examples:<br />
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]<br />
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]<br />
<br />
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].<br />
<br />
= R Client =<br />
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=StorageHub_REST_API&diff=33962StorageHub REST API2021-04-02T10:23:49Z<p>Massimiliano.assante: /* Retrieve VRE Folder */</p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
= Overview =<br />
<br />
The StorageHub APIs components provide a simple access to StorageHub service. <br />
<br />
It is conceived to support both Java and REST-based calls. <br />
<br />
== Dependencies ==<br />
<br />
=== Maven coordinates ===<br />
<br />
<source lang="java"><br />
<dependency><br />
<groupId>org.gcube.common</groupId><br />
<artifactId>storagehub-client-library</artifactId><br />
<version>[1.0.0,2.0.0)</version><br />
</dependency><br />
</source><br />
<br />
= Key features =<br />
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.<br />
<br />
'''StorageHub REST interface''' supports the following operations:<br />
* '''Retrieve WS''': to retrieve the user Workspace;<br />
* '''Folder Listing''': to list the content of a folder;<br />
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;<br />
* '''Find''': to find a file or a folder by name or pattern;<br />
* '''Delete''': to remove a file or a folder (including subfolders);<br />
* '''Download''': to download a file or a folder in ZIP format;<br />
* '''Get Public Link''': to get a public link of a file;<br />
* '''Create Folder''': to create a folder in the given parent folder;<br />
* '''Unzip''': to upload a zip file in a specific folder;<br />
* '''Upload file''': to upload a file in a folder.<br />
* '''Versions''': to get a specific version of a file;<br />
<br />
== API ==<br />
<br />
The base-url of this service on D4Science production is the following:<br />
<br />
https://api.d4science.org/workspace<br />
<br />
== Retrieve Workspace ==<br />
<br />
Returns the Item representing the workspace of the user retrieved by the token used for the call.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.getWSRoot()<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The workspace root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Get Item ById ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
</source><br />
<br />
== List Item Versions, Get Item Version ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
import org.gcube.common.storagehub.model.service.Version;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
List<Version> fileVersions = fileContainer.getVersions();<br />
<br />
//to download a version<br />
fileContainer.downloadSpecificVersion(versionName)<br />
</source><br />
<br />
== Folder Listing ==<br />
<br />
Returns the content of a Folder<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
import org.gcube.common.storagehub.model.items.*;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
List<? extends Item> items = folderContainer.list().getItems();<br />
<br />
for (Item item : items) {<br />
System.out.println("name:" + item.getName() + " is a File?: " + (item instanceof AbstractFileItem));<br />
System.out.println("name:" + item.getName() + " is a folder?: " + (item instanceof FolderItem));<br />
System.out.println("name:" + item.getName() + " is a shared folder?: " + (item instanceof SharedFolder));<br />
System.out.println("name:" + item.getName() + " is a VRE folder?: " + (item instanceof VreFolder));<br />
}<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}<br />
<br />
==== responses ====<br />
<br />
200 the list of the Items (in JSON format)<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Retrieve VRE Folder ==<br />
<br />
Return the Item representing the root of the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.openVREFolder();<br />
</source><br />
<br />
<br />
Check that a folder with name "FolderNameToCheck" exists in the VRE folder related to the user token.<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer vreFolder = shc.openVREFolder();<br />
FolderContainer folderToCheck = null;<br />
try { <br />
OpenResolver oRes = vreFolder.openByRelativePath("FolderNameToCheck");<br />
folderToCheck = oRes.asFolder();<br />
} catch (StorageHubException e) {<br />
System.out.println("FolderNameToCheck does not exists in VRE Folder, creating it");<br />
folderToCheck = vreFolder.newFolder("FolderNameToCheck description goes here");<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/vrefolder?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The VRE folder root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Create Folder ==<br />
<br />
Creates a new folder under another folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
root.newFolder("name", "description");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} <br />
Content-Type: application/x-www-form-urlencoded <br />
{String name, String description, boolean hidden}<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder item identifier is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Delete File or Folder ==<br />
<br />
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;<br />
<br />
=== Java ===<br />
<source lang="java"><br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
fileContainer.delete();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)<br />
<br />
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder or item is deleted<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Upload File ==<br />
<br />
Creates a new file under a folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadFile(is, "name", "description");<br />
System.out.println("Uploaded " + file.get().getName() + " - Returned Workspace id=" + file.getId()); <br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String name, String description, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}<br />
<br />
== Upload Archive ==<br />
<br />
Extract the content of the specified archive in a destination Folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadArchive(is, "parentFolderName");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String parentFolderName, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}<br />
<br />
== Get Public Url of a File ==<br />
<br />
Returns the public URL of a file<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open({item-id}).asFile().getPublicLink();<br />
<br />
//returns the public link for a specific version of the file <br />
shc.open({item-id}).asFile().getPublicLink({file-version});<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional<br />
<br />
== Find By Name ==<br />
<br />
Returns all the items with a name that matches a pattern on the first level of a given folder. <br />
The result will be empty in case of none of the item in the folder matches the pattern.<br />
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) <br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
//getting my root workspace folder<br />
FolderContainer myRoot = shc.getWSRoot();<br />
myRoot.findByName("{pattern}");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} <br />
<br />
==== responses ====<br />
<br />
200 A JSON with the retrieved items or an empty itemList. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
= DataMiner and SAI Interactions =<br />
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. <br />
Here are some examples:<br />
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]<br />
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]<br />
<br />
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].<br />
<br />
= R Client =<br />
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=StorageHub_REST_API&diff=33961StorageHub REST API2021-04-02T10:23:35Z<p>Massimiliano.assante: /* Retrieve VRE Folder */</p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
= Overview =<br />
<br />
The StorageHub APIs components provide a simple access to StorageHub service. <br />
<br />
It is conceived to support both Java and REST-based calls. <br />
<br />
== Dependencies ==<br />
<br />
=== Maven coordinates ===<br />
<br />
<source lang="java"><br />
<dependency><br />
<groupId>org.gcube.common</groupId><br />
<artifactId>storagehub-client-library</artifactId><br />
<version>[1.0.0,2.0.0)</version><br />
</dependency><br />
</source><br />
<br />
= Key features =<br />
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.<br />
<br />
'''StorageHub REST interface''' supports the following operations:<br />
* '''Retrieve WS''': to retrieve the user Workspace;<br />
* '''Folder Listing''': to list the content of a folder;<br />
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;<br />
* '''Find''': to find a file or a folder by name or pattern;<br />
* '''Delete''': to remove a file or a folder (including subfolders);<br />
* '''Download''': to download a file or a folder in ZIP format;<br />
* '''Get Public Link''': to get a public link of a file;<br />
* '''Create Folder''': to create a folder in the given parent folder;<br />
* '''Unzip''': to upload a zip file in a specific folder;<br />
* '''Upload file''': to upload a file in a folder.<br />
* '''Versions''': to get a specific version of a file;<br />
<br />
== API ==<br />
<br />
The base-url of this service on D4Science production is the following:<br />
<br />
https://api.d4science.org/workspace<br />
<br />
== Retrieve Workspace ==<br />
<br />
Returns the Item representing the workspace of the user retrieved by the token used for the call.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.getWSRoot()<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The workspace root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Get Item ById ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
</source><br />
<br />
== List Item Versions, Get Item Version ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
import org.gcube.common.storagehub.model.service.Version;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
List<Version> fileVersions = fileContainer.getVersions();<br />
<br />
//to download a version<br />
fileContainer.downloadSpecificVersion(versionName)<br />
</source><br />
<br />
== Folder Listing ==<br />
<br />
Returns the content of a Folder<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
import org.gcube.common.storagehub.model.items.*;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
List<? extends Item> items = folderContainer.list().getItems();<br />
<br />
for (Item item : items) {<br />
System.out.println("name:" + item.getName() + " is a File?: " + (item instanceof AbstractFileItem));<br />
System.out.println("name:" + item.getName() + " is a folder?: " + (item instanceof FolderItem));<br />
System.out.println("name:" + item.getName() + " is a shared folder?: " + (item instanceof SharedFolder));<br />
System.out.println("name:" + item.getName() + " is a VRE folder?: " + (item instanceof VreFolder));<br />
}<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}<br />
<br />
==== responses ====<br />
<br />
200 the list of the Items (in JSON format)<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Retrieve VRE Folder ==<br />
<br />
Return the Item representing the root of the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.openVREFolder();<br />
</source><br />
<br />
Check that a folder with name "FolderNameToCheck" exists in the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer vreFolder = shc.openVREFolder();<br />
FolderContainer folderToCheck = null;<br />
try { <br />
OpenResolver oRes = vreFolder.openByRelativePath("FolderNameToCheck");<br />
folderToCheck = oRes.asFolder();<br />
} catch (StorageHubException e) {<br />
System.out.println("FolderNameToCheck does not exists in VRE Folder, creating it");<br />
folderToCheck = vreFolder.newFolder("FolderNameToCheck description goes here");<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/vrefolder?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The VRE folder root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Create Folder ==<br />
<br />
Creates a new folder under another folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
root.newFolder("name", "description");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} <br />
Content-Type: application/x-www-form-urlencoded <br />
{String name, String description, boolean hidden}<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder item identifier is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Delete File or Folder ==<br />
<br />
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;<br />
<br />
=== Java ===<br />
<source lang="java"><br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
fileContainer.delete();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)<br />
<br />
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder or item is deleted<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Upload File ==<br />
<br />
Creates a new file under a folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadFile(is, "name", "description");<br />
System.out.println("Uploaded " + file.get().getName() + " - Returned Workspace id=" + file.getId()); <br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String name, String description, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}<br />
<br />
== Upload Archive ==<br />
<br />
Extract the content of the specified archive in a destination Folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadArchive(is, "parentFolderName");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String parentFolderName, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}<br />
<br />
== Get Public Url of a File ==<br />
<br />
Returns the public URL of a file<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open({item-id}).asFile().getPublicLink();<br />
<br />
//returns the public link for a specific version of the file <br />
shc.open({item-id}).asFile().getPublicLink({file-version});<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional<br />
<br />
== Find By Name ==<br />
<br />
Returns all the items with a name that matches a pattern on the first level of a given folder. <br />
The result will be empty in case of none of the item in the folder matches the pattern.<br />
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) <br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
//getting my root workspace folder<br />
FolderContainer myRoot = shc.getWSRoot();<br />
myRoot.findByName("{pattern}");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} <br />
<br />
==== responses ====<br />
<br />
200 A JSON with the retrieved items or an empty itemList. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
= DataMiner and SAI Interactions =<br />
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. <br />
Here are some examples:<br />
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]<br />
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]<br />
<br />
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].<br />
<br />
= R Client =<br />
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=StorageHub_REST_API&diff=33960StorageHub REST API2021-03-25T11:22:55Z<p>Massimiliano.assante: /* Java */</p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
= Overview =<br />
<br />
The StorageHub APIs components provide a simple access to StorageHub service. <br />
<br />
It is conceived to support both Java and REST-based calls. <br />
<br />
== Dependencies ==<br />
<br />
=== Maven coordinates ===<br />
<br />
<source lang="java"><br />
<dependency><br />
<groupId>org.gcube.common</groupId><br />
<artifactId>storagehub-client-library</artifactId><br />
<version>[1.0.0,2.0.0)</version><br />
</dependency><br />
</source><br />
<br />
= Key features =<br />
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.<br />
<br />
'''StorageHub REST interface''' supports the following operations:<br />
* '''Retrieve WS''': to retrieve the user Workspace;<br />
* '''Folder Listing''': to list the content of a folder;<br />
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;<br />
* '''Find''': to find a file or a folder by name or pattern;<br />
* '''Delete''': to remove a file or a folder (including subfolders);<br />
* '''Download''': to download a file or a folder in ZIP format;<br />
* '''Get Public Link''': to get a public link of a file;<br />
* '''Create Folder''': to create a folder in the given parent folder;<br />
* '''Unzip''': to upload a zip file in a specific folder;<br />
* '''Upload file''': to upload a file in a folder.<br />
* '''Versions''': to get a specific version of a file;<br />
<br />
== API ==<br />
<br />
The base-url of this service on D4Science production is the following:<br />
<br />
https://api.d4science.org/workspace<br />
<br />
== Retrieve Workspace ==<br />
<br />
Returns the Item representing the workspace of the user retrieved by the token used for the call.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.getWSRoot()<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The workspace root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Get Item ById ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
</source><br />
<br />
== List Item Versions, Get Item Version ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
import org.gcube.common.storagehub.model.service.Version;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
List<Version> fileVersions = fileContainer.getVersions();<br />
<br />
//to download a version<br />
fileContainer.downloadSpecificVersion(versionName)<br />
</source><br />
<br />
== Folder Listing ==<br />
<br />
Returns the content of a Folder<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
import org.gcube.common.storagehub.model.items.*;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
List<? extends Item> items = folderContainer.list().getItems();<br />
<br />
for (Item item : items) {<br />
System.out.println("name:" + item.getName() + " is a File?: " + (item instanceof AbstractFileItem));<br />
System.out.println("name:" + item.getName() + " is a folder?: " + (item instanceof FolderItem));<br />
System.out.println("name:" + item.getName() + " is a shared folder?: " + (item instanceof SharedFolder));<br />
System.out.println("name:" + item.getName() + " is a VRE folder?: " + (item instanceof VreFolder));<br />
}<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}<br />
<br />
==== responses ====<br />
<br />
200 the list of the Items (in JSON format)<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Retrieve VRE Folder ==<br />
<br />
Return the Item representing the root of the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.openVREFolder();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/vrefolder?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The VRE folder root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Create Folder ==<br />
<br />
Creates a new folder under another folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
root.newFolder("name", "description");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} <br />
Content-Type: application/x-www-form-urlencoded <br />
{String name, String description, boolean hidden}<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder item identifier is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Delete File or Folder ==<br />
<br />
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;<br />
<br />
=== Java ===<br />
<source lang="java"><br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
fileContainer.delete();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)<br />
<br />
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder or item is deleted<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Upload File ==<br />
<br />
Creates a new file under a folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadFile(is, "name", "description");<br />
System.out.println("Uploaded " + file.get().getName() + " - Returned Workspace id=" + file.getId()); <br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String name, String description, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}<br />
<br />
== Upload Archive ==<br />
<br />
Extract the content of the specified archive in a destination Folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadArchive(is, "parentFolderName");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String parentFolderName, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}<br />
<br />
== Get Public Url of a File ==<br />
<br />
Returns the public URL of a file<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open({item-id}).asFile().getPublicLink();<br />
<br />
//returns the public link for a specific version of the file <br />
shc.open({item-id}).asFile().getPublicLink({file-version});<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional<br />
<br />
== Find By Name ==<br />
<br />
Returns all the items with a name that matches a pattern on the first level of a given folder. <br />
The result will be empty in case of none of the item in the folder matches the pattern.<br />
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) <br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
//getting my root workspace folder<br />
FolderContainer myRoot = shc.getWSRoot();<br />
myRoot.findByName("{pattern}");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} <br />
<br />
==== responses ====<br />
<br />
200 A JSON with the retrieved items or an empty itemList. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
= DataMiner and SAI Interactions =<br />
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. <br />
Here are some examples:<br />
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]<br />
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]<br />
<br />
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].<br />
<br />
= R Client =<br />
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=StorageHub_REST_API&diff=33959StorageHub REST API2021-03-25T11:21:22Z<p>Massimiliano.assante: /* Java */</p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
= Overview =<br />
<br />
The StorageHub APIs components provide a simple access to StorageHub service. <br />
<br />
It is conceived to support both Java and REST-based calls. <br />
<br />
== Dependencies ==<br />
<br />
=== Maven coordinates ===<br />
<br />
<source lang="java"><br />
<dependency><br />
<groupId>org.gcube.common</groupId><br />
<artifactId>storagehub-client-library</artifactId><br />
<version>[1.0.0,2.0.0)</version><br />
</dependency><br />
</source><br />
<br />
= Key features =<br />
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.<br />
<br />
'''StorageHub REST interface''' supports the following operations:<br />
* '''Retrieve WS''': to retrieve the user Workspace;<br />
* '''Folder Listing''': to list the content of a folder;<br />
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;<br />
* '''Find''': to find a file or a folder by name or pattern;<br />
* '''Delete''': to remove a file or a folder (including subfolders);<br />
* '''Download''': to download a file or a folder in ZIP format;<br />
* '''Get Public Link''': to get a public link of a file;<br />
* '''Create Folder''': to create a folder in the given parent folder;<br />
* '''Unzip''': to upload a zip file in a specific folder;<br />
* '''Upload file''': to upload a file in a folder.<br />
* '''Versions''': to get a specific version of a file;<br />
<br />
== API ==<br />
<br />
The base-url of this service on D4Science production is the following:<br />
<br />
https://api.d4science.org/workspace<br />
<br />
== Retrieve Workspace ==<br />
<br />
Returns the Item representing the workspace of the user retrieved by the token used for the call.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.getWSRoot()<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The workspace root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Get Item ById ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
</source><br />
<br />
== List Item Versions, Get Item Version ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
import org.gcube.common.storagehub.model.service.Version;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
List<Version> fileVersions = fileContainer.getVersions();<br />
<br />
//to download a version<br />
fileContainer.downloadSpecificVersion(versionName)<br />
</source><br />
<br />
== Folder Listing ==<br />
<br />
Returns the content of a Folder<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
import org.gcube.common.storagehub.model.items.*;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
List<? extends Item> items = folderContainer.list().getItems();<br />
<br />
for (Item item : items) {<br />
System.out.println("name:" + item.getName() + " is a File?: " + (item instanceof AbstractFileItem));<br />
System.out.println("name:" + item.getName() + " is a folder?: " + (item instanceof FolderItem));<br />
System.out.println("name:" + item.getName() + " is a shared folder?: " + (item instanceof SharedFolder));<br />
System.out.println("name:" + item.getName() + " is a VRE folder?: " + (item instanceof VreFolder));<br />
}<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}<br />
<br />
==== responses ====<br />
<br />
200 the list of the Items (in JSON format)<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Retrieve VRE Folder ==<br />
<br />
Return the Item representing the root of the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.openVREFolder();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/vrefolder?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The VRE folder root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Create Folder ==<br />
<br />
Creates a new folder under another folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
root.newFolder("name", "description");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} <br />
Content-Type: application/x-www-form-urlencoded <br />
{String name, String description, boolean hidden}<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder item identifier is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Delete File or Folder ==<br />
<br />
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;<br />
<br />
=== Java ===<br />
<source lang="java"><br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
fileContainer.delete();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)<br />
<br />
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder or item is deleted<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Upload File ==<br />
<br />
Creates a new file under a folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadFile(is, "name", "description");<br />
log.debug("Uploaded " + file.get().getName() + " - Returned Workspace id=" + file.getId()); <br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String name, String description, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}<br />
<br />
== Upload Archive ==<br />
<br />
Extract the content of the specified archive in a destination Folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadArchive(is, "parentFolderName");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String parentFolderName, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}<br />
<br />
== Get Public Url of a File ==<br />
<br />
Returns the public URL of a file<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open({item-id}).asFile().getPublicLink();<br />
<br />
//returns the public link for a specific version of the file <br />
shc.open({item-id}).asFile().getPublicLink({file-version});<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional<br />
<br />
== Find By Name ==<br />
<br />
Returns all the items with a name that matches a pattern on the first level of a given folder. <br />
The result will be empty in case of none of the item in the folder matches the pattern.<br />
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) <br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
//getting my root workspace folder<br />
FolderContainer myRoot = shc.getWSRoot();<br />
myRoot.findByName("{pattern}");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} <br />
<br />
==== responses ====<br />
<br />
200 A JSON with the retrieved items or an empty itemList. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
= DataMiner and SAI Interactions =<br />
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. <br />
Here are some examples:<br />
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]<br />
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]<br />
<br />
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].<br />
<br />
= R Client =<br />
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=OAuth2.0&diff=33900OAuth2.02021-02-25T10:54:44Z<p>Massimiliano.assante: </p>
<hr />
<div>__TOC__<br />
<br />
= OAuth 2.0 gCube Support (DEPRECATED)=<br />
<br />
'''<br />
THIS PAGE IS DEPRECATED; PLEASE REFER TO [https://support.d4science.org D4Science Support page] or contact D4Science to obtain information on how to authorise third party applications over the D4Science infrastructure. <br />
<br />
'''<br />
<br />
By means of the OAuth 2.0 protocol (authorised) third party applications can operate on user's behalf over the D4Science infrastructure (while protecting the member's credentials).<br />
For more information about the OAuth authorization framework please visit the official [https://oauth.net/2/ OAuth site]. For technical details you could also check the [https://tools.ietf.org/html/rfc6749| OAuth 2.0 RFC]. In the following, the needed steps to authorize third party applications to operate on user's behalf and the D4Science infrastructure are explained. <br />
<br />
== Step 1 - Configure your application ==<br />
Authorized third party applications must be registered on the Infrastructure. Up to now the application registration is not automatic but you need to open a ticket at https://support.d4science.org and select 3rd Party App Registration.<br />
<br />
The ticket needs to have '''3rd Party Application Support''' as tracker. The application name as well as the redirect uri need to be provided (https is strongly suggested). This url must be absolute, and without arguments. Once the application is registered, it will have associated:<br />
<br />
* a unique client identifier (client_id);<br />
* a client secret which must be kept secret, as the name itself says (client_secret).<br />
<br />
<source lang="xml"><br />
client_id: ab9d32c9-9425-42f1-99d7-50d68e651e9c<br />
client_secret: y9r93zDv4D<br />
</source><br />
<br />
Both are needed to proceed with the OAuth 2.0 workflow.<br />
<br />
== Step 2 - Require an authorization code ==<br />
Once the application is registered and configured, an authorization code can be requested. This code is not the final one needed to perform operations on user behalf, but is the one that will be exchanged for that in the next step. To require this code that has a very short time to live, the application must invoke the authorization service, which is gateway dependant.<br />
<br />
=== Retrieving the authorization endpoint location===<br />
<br />
The previous authorization endpoint location is portal's gateway dependant and '''can be found in Table 1''' or retrieved dinamically by using the registry service REST-API hosted at <br />
<br />
''https://registry.d4science.org/icproxy/gcube/service/'' <br />
<br />
'''Get authorization endpoint location in the following Table 1'''<br />
<br />
{| class="wikitable"<br />
|+Table 1: authorization endpoints location depending on the D4Science Gateway<br />
! style="font-weight: bold;" | D4Science Gateway name<br />
! style="font-weight: bold;" | Authorization endpoint<br />
|-<br />
| ARIADNEplus Gateway<br />
| https://ariadne.d4science.org/group/ariadneplus-gateway/authorization<br />
|-<br />
| iMarine Gateway<br />
| https://i-marine.d4science.org/group/imarine-gateway/authorization<br />
|-<br />
| AGINFRA+ Gateway<br />
| https://aginfra.d4science.org/group/aginfra-gateway/authorization<br />
|-<br />
| Blue-Cloud Gateway<br />
| https://blue-cloud.d4science.org/group/bluecloud-gateway/authorization<br />
|-<br />
| D4Science.org Gateway<br />
| https://services.d4science.org/group/d4science-services-gateway/authorization<br />
|-<br />
| DESCRAMBLE Gateway<br />
| https://descramble.d4science.org/group/descramble-gateway/authorization<br />
|-<br />
| GEMex Gateway<br />
| https://gemex.d4science.org/group/gemex-gateway/authorization<br />
|-<br />
| OpenAIRE-Connect Gateway<br />
| https://openaire-connect.d4science.org/group/openaire-connect-gateway/authorization<br />
|-<br />
| PARTHENOS Gateway<br />
| https://parthenos.d4science.org/group/parthenos-gateway/authorization<br />
|-<br />
| SoBigData Gateway<br />
| https://sobigdata.d4science.org/group/sobigdata-gateway/authorization<br />
|}<br />
<br />
'''Retrieve the authorization endpoint location dynamically (ADVANCED MODE)'''<br />
<br />
Registry Swagger REST-API avaiable at: https://dev.d4science.org/swagger/registry/<br />
<br />
Which requires an [https://dev.d4science.org/authorization/ authorization token]. The returned profile is something like<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<serviceEndpoints><br />
<Resource version="0.4.x"><br />
<ID>b1c5851f-1d47-4c60-8906-941ab859a334</ID><br />
<Type>RuntimeResource</Type><br />
<Scopes><br />
<Scope>/gcube</Scope><br />
</Scopes><br />
...<br />
<AccessPoint><br />
<Description>AuthorisationServer where to authenticate external SmartGears services oAuth like</Description><br />
<Interface><br />
<Endpoint EntryName="AuthorisationServer">https://next.d4science.org/group/next/authorization</Endpoint><br />
</Interface><br />
</AccessPoint><br />
</Profile><br />
</Resource><br />
</serviceEndpoints><br />
</source><br />
<br />
=== Invoking the authorization service===<br />
<br />
Once you got the AuthorisationServer endpoint and the call is executed, two things can happen:<br />
* the user is requested to perform the login, thus accepting the application operation and granting it;<br />
* if there is a valid existing permission grant for the application, the login operation is by-passed and the redirect_uri is followed.<br />
<br />
The request is an '''HTTP GET''' call, that needs the following parameters to be provided<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| client_id<br />
| the client identifier associated to the application<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the redirect uri provided at registration phase<br />
| yes<br />
|-<br />
| state<br />
| a unique string value hard to guess, needed to prevent CRSF attacks (e.g. AAedBEeF545sdfKef897)<br />
| yes<br />
|-<br />
| scope<br />
| this field is optional. It can be specified to request (with the next step) an access token valid for a given<br />
infrastructure scope. If missing, the scope will be choose by the user on credentials insertion.<br />
| no<br />
|}<br />
<br />
This is an example in development environment<br />
<pre><br />
https://next.d4science.org/group/next/authorization?client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&scope=%2Fgcube&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&state=7d12bf13-111c-4f46-ab06-9e9e08ad377b<br />
</pre><br />
<br />
==== End user Experience ====<br />
<br />
Once redirected, the user will be presented with D4Science's authentication dialog box. This identifies your application as well as outlines the particular member permissions that your application requests. The logo and application name can be configurable, in the example the application name is '''Application X'''<br />
<br />
[[File:Oauth-ux.jpg]]<br />
<br />
==== Application is Accepted ====<br />
<br />
If the user grants permissions to the application, a redirection to the redirect_uri is performed. Attached to the redirect_uri, two important url arguments are provided<br />
<br />
* code: which is the authorization code. It expires in 10 seconds;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
==== Application is Rejected ====<br />
<br />
If, for any reason, the request fails, the redirect_uri is still followed but the url has attached the following parameters<br />
* error: a code that states the type of error;<br />
* error_description: a textual description of the actual error;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
== Step 3 - Exchange Authorization Code for an Access Token ==<br />
This is the step in which the previously obtained code can be exchanged to obtain a real access token. <br />
This time, an '''HTTP POST''' method needs to be performed, with a '''Content-Type''' of type '''application/x-www-form-urlencoded'''.<br />
<br />
The client_id and secret must be passed as the basic auth header Base64 encoded, see the example in Java below:<br />
<br />
e.g.<br />
<source lang="java"><br />
...<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
..<br />
</source><br />
<br />
The request-body must contain the following parameters:<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| grant_type<br />
| The value of this field should always be authorization_code<br />
| yes<br />
|-<br />
| code<br />
| the authorization code you received from the previous step.<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the same 'redirect_uri' value that you<br />
passed in the previous step.<br />
| yes<br />
|}<br />
<br />
For instance, in development environment<br />
<br />
<source lang="xml"><br />
POST /gcube-oauth/v2/access-token HTTP/1.1<br />
Host: socialnetworking-d-d4s.d4science.org<br />
Content-Type: application/x-www-form-urlencoded<br />
<br />
grant_type=authorization_code&code=...&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&client_secret=...<br />
</source><br />
<br />
In case of success, the response will contain a JSON object, having these properties:<br />
<br />
* access_token: the obtained access token to use to perform operations on user's behalf;<br />
* expires_in: the time to live of the token expressed in seconds;<br />
* scope: in which scope the token is valid;<br />
* token_type: that is "Bearer".<br />
<br />
In case of failures, instead, the returned object will contain:<br />
<br />
* error : an error string according the oauth 2.0 RFC.<br />
* error_description : a text description for the error.<br />
<br />
=== D4Science oAuth2 endpoint location === <br />
In the '''production''' environment, its value is<br />
<source lang="xml"><br />
https://api.d4science.org/oauth/v2/access-token<br />
</source><br />
<br />
'''HTTP POST Example in JAVA'''<br />
<br />
<source lang="java"><br />
String ACCESS_TOKEN_ATTR = "access_token";<br />
String SCOPE_ATTR = "scope";<br />
String fullPath2oAuthService = "https://api.d4science.org/oauth/v2/access-token";<br />
try {<br />
CloseableHttpClient httpClient = HttpClientBuilder.create().setRedirectStrategy(new LaxRedirectStrategy()).build();<br />
HttpPost httpPostRequest = new HttpPost(fullPath2oAuthService);<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
httpPostRequest.addHeader("Content-Type", "application/x-www-form-urlencoded");<br />
List<NameValuePair> params = new ArrayList<NameValuePair>();<br />
params.add(new BasicNameValuePair("grant_type", "authorization_code"));<br />
params.add(new BasicNameValuePair("code", tempCode));<br />
params.add(new BasicNameValuePair("redirect_uri", "https://www.gcube-system.org/oauth-authentication-example/example-app"));<br />
httpPostRequest.setEntity(new UrlEncodedFormEntity(params));<br />
<br />
CloseableHttpResponse httpresponse = httpClient.execute(httpPostRequest);<br />
int responseCode = httpresponse.getStatusLine().getStatusCode();<br />
if (responseCode < 200 || responseCode >= 300) {<br />
System.out.println("Error while getting authorization code, response status line from "<br />
+ fullPath2oAuthService + " was: " + responseCode); <br />
}<br />
JSONParser parser = new JSONParser();<br />
JSONObject json = (JSONObject) parser.parse(EntityUtils.toString(httpresponse.getEntity()));<br />
accessToken = json.get(ACCESS_TOKEN_ATTR).toString();<br />
scope = json.get(SCOPE_ATTR).toString();<br />
} catch(Exception e){<br />
System.out.println("Failed to perform request");<br />
e.printStackTrace();<br />
<br />
}<br />
</source><br />
<br />
== Step 4 - Get member data ==<br />
Having an access token, your application can perform some operations on behalf of the user. For instance, it can retrieve user's information profile by exploiting the [https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface social networking web service].<br />
<br />
Its current production endpoint is: https://api.d4science.org/rest/<br />
<br />
The subservice to call, once the social networking service endpoint has been retrieved, is this one<br />
<br />
<source lang="xml"><br />
2/users/get-oauth-profile<br />
</source><br />
<br />
It requires a ''gcube-token'' as query parameter, whose value is exactly the access token retrieved in the previous step. The result is something like this<br />
<br />
<source lang="ruby"><br />
{<br />
id: ".....",<br />
roles: [ ], // roles in the given-token context, e.g. VRE-Manager<br />
picture: "....",<br />
name: "....",<br />
middle_name: "....",<br />
male: as boolean ,<br />
location_industry: "....",<br />
given_name: "....",<br />
email: "....",<br />
job_title: "....",<br />
family_name: "....",<br />
verified_email: as boolean <br />
}<br />
</source><br />
<br />
== See Also ==<br />
<br />
[https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface Social Networking Service]<br />
<br />
[https://dev.d4science.org/swagger/social-networking/ Social Networking Service Docs SWAGGER]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=GCat_Service&diff=33827GCat Service2021-02-03T14:40:35Z<p>Massimiliano.assante: /* Request */</p>
<hr />
<div>gCat Service is a RESTful service enabling any client to programmatically interact with the gCube Catalogue.<br />
gCat Service uses the principles defined in [[GCat_Background | gCat Background]]<br />
<br />
== Request ==<br />
<br />
<br />
=== URL ===<br />
<br />
The URL used to interact with gCat is composed of two parts:<br />
<br />
* '''Base Service URL''': '''https://api.d4science.org/catalogue'''<br />
* Specific API (e.g. /organizations)<br />
<br />
D4Science infrastructure uses cloud facilities allowing to replicate a service to achieve failover and load balancing.<br />
<br />
gCat instances can be deployed and undeployed dynamically. For such a reason the Base Service URL MUST NOT be hard-cabled in the code because it can change over time.<br />
<br />
To dynamically discover the Base Service URL you can use the [https://dev.d4science.org/swagger/registry/ Registry RESTful Service]<br />
<br />
You need to discover gCore Resource having:<br />
* '''class''' : ''DataPublishing''<br />
* '''name''' : ''gcat''<br />
<br />
=== HTTP Headers ===<br />
<br />
==== gCube Authorization Token ====<br />
<br />
Any request performed to gCat MUST contains the gCube Authorization Token.<br />
<br />
This is done using the HTTP Header ''gcube-token''<br />
<br />
<pre>gcube-token: YOUR-TOKEN</pre><br />
<br />
The ''gcube-token'' HTTP header acts as the standard ''Authorization'' HTTP header. Plese note that ''Authorization'' HTTP header cannot be used in place of ''gcube-token'' HTTP header.<br />
<br />
===== Retrieve your gCube Authorization Token =====<br />
<br />
the gCube Authorization Token is a UUID bound to yourself and a given Infrastructure context. To retrieve it, you just need to go to a VRE for which you are interested and use the Authorisation Options portlet (see below)<br />
<br />
[[File:Authorisation_option.png]]<br />
<br />
Click on '''Show''' button and select the token.<br />
<br />
==== Content Type ====<br />
<br />
Any request must contain the indication of the interested content type.<br />
<br />
For any operation returning a result, the client must specify the ''Accept'' HTTP Header. <br />
<br />
<pre>Accept: application/json</pre><br />
<br />
For any operation sending content to the service, it is necessary to specify the ''Content-Type'' HTTP Header.<br />
<br />
<pre>Content-Type: application/json</pre><br />
<br />
Actually, the service accepts and returns only JSON objects.<br />
<br />
Except for a [[#Profile_Collection | profile]] which can be also requested in XML.<br />
<br />
=== HTTP Statuses ===<br />
<br />
Any successful operation returns '''''200 OK''''' status code except for create operation which return '''''201 Created''''' or for the operations which do not provide any content and returns '''''204 No Content'''''.<br />
<br />
The most common error status a client can obtain are <br />
<br />
* '''''400 Bad Request''''' used to indicate a clients error [https://tools.ietf.org/html/rfc7231#section-6.5.1 https://tools.ietf.org/html/rfc7231#section-6.5.1];<br />
* '''''401 Unauthorized''''' used to indicate that the client does not provided the gcube-token HTTP Header or the clinet has not enough right to perform such request [https://tools.ietf.org/html/rfc7235#section-3.1 https://tools.ietf.org/html/rfc7235#section-3.1];<br />
* '''''404 Not Found''''' used to indicate that the requested instance does not exists [https://tools.ietf.org/html/rfc7231#section-6.5.4 https://tools.ietf.org/html/rfc7231#section-6.5.4];<br />
* '''''405 Method Not Allowed''''' the used HTTP method is not supported for the requested URL [https://tools.ietf.org/html/rfc7231#section-6.5.5 https://tools.ietf.org/html/rfc7231#section-6.5.5]. <br />The response contains the ''Allow'' HTTP Header indicating the supported HTTP method for such URL [https://tools.ietf.org/html/rfc7231#section-7.4.1 https://tools.ietf.org/html/rfc7231#section-7.4.1];<br />
* '''''409 Conflict''''' the request could not be completed due to a conflict with the current state of the target resource (e.g. the name of the resource already exists). [https://tools.ietf.org/html/rfc7231#section-6.5.8 https://tools.ietf.org/html/rfc7231#section-6.5.8].];<br />
* '''''500 Internal Server Error''''' indicate a server failure. [https://tools.ietf.org/html/rfc7231#section-6.6.1 https://tools.ietf.org/html/rfc7231#section-6.6.1].<br />
<br />
A complete list of HTTP Status can be found here:<br />
[https://httpstatuses.com/ https://httpstatuses.com/]<br />
<br />
You can report a '''''500 Internal Server Error''''' the ticketing system. Please use this checklist before reporting an error:<br />
<br />
* replicate the request<br />
* the failure could be temporal due to network error, server issue so please retry the request after a certain amount of time<br />
* indicate how to replicate the error<br />
* indicate the time when the error occurred (this simplify the identification of the issue)<br />
<br />
=== HTTP Methods ===<br />
<br />
To be RESTful compliant gCat uses standard HTTP Methods to perform a listing of collections and CRUD (Create Read Update Delete) operations on instances.<br />
<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL || Success HTTP Status || Safe || Idempotent<br />
|-<br />
| Supported<br />HTTP Methods || OPTIONS || /{COLLECTION} || 204 No Content<br />(Supported HTTP Methods in ''Allow'' HTTP Header) || Y || Y<br />
|-<br />
| List || GET || /{COLLECTION} || 200 OK || Y || Y<br />
|-<br />
| Exists || HEAD || /{COLLECTION} || 204 No Content || Y || Y<br />
|-<br />
| Create || POST || /{COLLECTION} || 201 Created || N || N<br />
|-<br />
| Supported<br />HTTP Methods || OPTIONS || /{COLLECTION}/{INSTANCE_ID} || 204 No Content<br />(Supported HTTP Methods in ''Allow'' HTTP Header) || Y || Y<br />
|-<br />
| Exist || HEAD || /{COLLECTION}/{INSTANCE_ID} || 204 No Content || Y || Y<br />
|-<br />
| Read || GET || /{COLLECTION}/{INSTANCE_ID} || 200 OK || Y || Y<br />
|-<br />
| Update || PUT || /{COLLECTION}/{INSTANCE_ID} || 200 OK || N || Y<br />
|-<br />
| Patch || PATCH || /{COLLECTION}/{INSTANCE_ID} || 200 OK || N || Y<br />
|-<br />
| <span id="delete">Delete</span> || DELETE || /{COLLECTION}/{INSTANCE_ID} || 204 No Content || N || N [[#non_safe_delete| ‡]]<br />
|-<br />
| Purge || PURGE || /{COLLECTION}/{INSTANCE_ID} || 204 No Content || N || N[[#non_safe_delete| ‡]]<br />
|-<br />
| Purge || DELETE || /{COLLECTION}/{INSTANCE_ID}?purge=true || 204 No Content || N || N[[#non_safe_delete| ‡]]<br />
|}<br />
<br />
<br />
About URL:<br />
* {COLLECTION} is the plural name of the entity type;<br />
* {INSTANCE_ID} is an identification which enables to univocally identify the instance in the collection.<br />
<br />
About Safety and Idempotency properties:<br />
* A method is '''''Safe''''' if it does not produce any side effects. "''This does not prevent an implementation from including behaviour that is potentially harmful, that is not entirely read-only, or that causes side effects while invoking a safe method''" [https://tools.ietf.org/html/rfc7231#section-4.2.1 https://tools.ietf.org/html/rfc7231#section-4.2.1];<br />
* A method is '''''Idempotent''''' if the same operation repeated multiple times has the same side effect than using it one time. "''repeating the request will have the same intended effect, even if the original request succeeded, though the response might differ''" [https://tools.ietf.org/html/rfc7231#section-4.2.2 https://tools.ietf.org/html/rfc7231#section-4.2.2].<br />
<br />
[[#delete|‡]] DELETE has been defined as idempotent. <span id="non_safe_delete">Allamaraju</span><ref>Allamaraju S. RESTful Web Services Cookbook: Solutions for Improving Scalability and Simplicity . O’Reilly. first ed. 2010</ref> argues that DELETE idempotency should be accomplished client-side. The server should inform the client if a delete succeeded because the resource was really deleted or it was not found i.e., '''''404 Not Found''''' error is suggested instead of '''''204 No Content'''''. The latter situation should be treated as idempotent by the client. For this reason, gCat does not provide server-side idempotency for DELETE and PURGE operations.<br />
<br />
You can find more information about HTTP Methods at [https://restfulapi.net/http-methods/ https://restfulapi.net/http-methods/]<br />
<br />
Uncommon HTTP Methods:<br />
* PATCH method allows to perform a differential update (i.e. an update which provide only the differences and not the whole new representation) [];<br />
* PURGE method is not a standard but is a widely used in service which requires this action (e.g. [https://varnish-cache.org/docs/3.0/tutorial/purging.html Varnish], [https://wiki.squid-cache.org/SquidFaq/OperatingSquid#How_can_I_purge_an_object_from_my_cache.3F Squid]). gCat provide support for this method but to support a wider range of clients it also provides the Purge action via DELETE with the additional get parameter <pre>purge=true</pre><br />
<br />
== Collections ==<br />
<br />
Collections available to any users. Non-safe methods can only be invoked by [[GCat_Background#Users.2C_Roles_and_Groups | Catalogue Editor]]<br />
<br />
* [[#Item_Collection | Item]]<br />
* [[#Resource_Collection | Resource]]<br />
* [[#Profile_Collection | Profile]]<br />
* [[#Namespace_Collection | Namespace]]<br />
* [[#License_Collection | License]]<br />
<br />
Collections available only for [[GCat_Background#Users.2C_Roles_and_Groups | Catalogue Admin]]<br />
<br />
* [[#Group_Collection | Group]]<br />
* [[#Organization_Collection | Organization]]<br />
* [[#User_Collection | User]]<br />
<br />
<br />
For each collection HEAD and OPTIONS operations are not shown in tables even there are always available.<br />
<br />
To keep this documentation simple, for each collection is presented only the relevant attributes of an instance. <br />
This allows getting the picture of the content of a request or the content of a response.<br />
<br />
=== Item Collection ===<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL<br />
|-<br />
| List || GET || <code>/items?limit=10&offset=0</code><br />
|-<br />
| Create || POST || <code>/items?social_post=true</code><br />
|-<br />
| Read || GET || <code>/items/{NAME}</code><br />
|-<br />
| Update || PUT || <code>/items/{NAME}</code><br />
|-<br />
| Delete || DELETE || <code>/items/{NAME}</code><br />
|-<br />
| Purge || PURGE || <code>/items/{NAME}</code><br />
|-<br />
| Purge || DELETE || <code>/items/{NAME}?purge=true</code><br />
|}<br />
<br />
==== List Items ====<br />
<br />
The listing API provides paginated results by using the query parameters '''limit''' and '''offset'''. The defaults values are 10 for '''limit''' and 0 for '''offset'''.<br />
To get unlimited results the '''limit''' query parameters must be set to '''-1'''. If the results are too much the operation could fail. It is recommended to request no more than 1000 results.<br />
The '''offset''' parameter indicates the page number taking in account pages containing a number of results equivalent to '''limit'''.<br />
<br />
===== Filtering options =====<br />
<br />
The listing method offers options to filter the results, thus enacts to search for items including spatial search (see ext_bbox below). <br />
<br />
It accepts the following query parameters (a subset of Solr search query parameters, see [https://doc.lucidworks.com/fusion/3.1/Search/Query-Language-Cheat-Sheet.html Solr Query Syntax]):<br />
* '''q''' (string) – the solr query. Optional. Default: "*:*" See [http://www.solrtutorial.com/solr-query-syntax.html Solr Query Syntax]<br />
** E.g. <code>/items?q=title:foo</code> returns the items with word "foo" in the title; <br />
* '''fq''' (string) – Filter query. A query string that limits the query results without influencing their scores. Note: +site_id:{ckan_site_id} is added to this string prior to the query being executed.<br />
** E.g. <code>/items?q=title:foo&fq=notes:bar</code> returns with word "foo" in the 'title' and the word "bar" in the 'notes'; <br />
* '''fq_list''' (list of strings) – additional filter queries to apply.<br />
** E.g. <code>/items?q=title:foo&fq_list=...</code> returns the items with word "foo" in the 'title'<br />
* '''sort''' (string) – sorting of the search results. Optional. Default: 'relevance asc, metadata_modified desc'. As per the solr documentation, this is a comma-separated string of field names and sort-orderings.<br />
** E.g. <code>/items?q=title:foo&sort=name+asc</code> returns the items with word "foo" in the 'title' sorting the results by name ascending;<br />
<!--<br />
* '''facet''' (string) – whether to enable faceted results. Default: True.<br />
* '''facet.mincount''' (int) – the minimum counts for facet fields should be included in the results.<br />
* '''facet.limit''' (int) – the maximum number of values the facet fields return. A negative value means unlimited. This can be set instance-wide with the search.facets.limit config option. Default is 50.<br />
* '''facet.field''' (list of strings) – the fields to facet upon. Default empty. If empty, then the returned facet information is empty.<br />
--><br />
<!--<br />
* '''include_drafts''' (bool) – if True, draft datasets will be included in the results. A user will only be returned their own draft datasets, and a sysadmin will be returned all draft datasets. Optional, the default is False.<br />
** ... this is a new concept ... I do ignore what is a draft dataset now;<br />
** E.g. <code>/items?q=...&include_drafts</code> returns ... <br />
--><br />
* '''include_private''' (bool) – if True, private datasets will be included in the results. Only private datasets from the user’s organizations will be returned and sysadmins will be returned all private datasets. Optional, the default is False.<br />
** E.g. <code>/items?include_private=true</code><br />
<!--<br />
* '''use_default_schema''' (bool) – use default package schema instead of a custom schema defined with an IDatasetForm plugin (default: False)<br />
* '''rows''' (int) – the maximum number of matching rows (datasets) to return. (optional, default: 10, upper limit: 1000)<br />
* '''start''' (int) – the offset in the complete result for where the set of returned datasets should begin.<br />
--><br />
<!-- ===== Spatial search can be done using ===== --><br />
* '''ext_bbox''': The coordinates of the upper-right and bottom-left angle of a rectangular to query for. The form is '''Lat,Long,Lat,Long'''<br />
** E.g. <code>/items?limit=10&offset=0&q=Pollution&ext_bbox=-7.535093,49.208494,3.890688,57.372349</code> returns the first 10 items with 'Pollution' having a spatial coverage in the specified bounding box<br />
<br />
==== Create Items ====<br />
<br />
The create API allows to create social posts by indicating '''social_post=true''' in the URL. The '''social_post''' query parameter is optional and the default value is '''false'''.<br />
<br />
An Item is mainly described by the following attributes (* indicate mandatory attributes): <br />
<br />
* '''name*''' (string): the name of the new item, must be between 2 and 100 characters long and contain only lowercase alphanumeric characters, '-' and '_';<br />
* '''title''' (string, defaut="same as name"): the title of the item;<br />
* '''private*''' (bool): If True creates a private item;<br />
<br />
* '''maintainer''' (string): the name of the item’s maintainer;<br />
* '''maintainer_email''' (string): the email address of the item’s maintainer;<br />
* '''license_id*''' (license id string): the id of the item’s license, see license_list() for available values;<br />
* '''notes''' (string): a description of the item;<br />
* '''url''' (string): a URL for the item’s source;<br />
* '''version''' (string, no longer than 100 characters): ...;<br />
* '''state''' (string, default='active'): the current state of the item, e.g. 'active' or 'deleted', only active items show up in search results and other lists of items, this parameter will be ignored if you are not authorized to change the state of the item;<br />
* '''groups''' (list of dictionaries): the groups to which the item belongs, each group dictionary should have one or more of the following keys which identify an existing group: 'id' (the id of the group, string), or 'name' (the name of the group, string). To see which groups exist use list method in [[#Group_Collection | Group Collection]]<br />
* '''tags''' (list of tag dictionaries): the item’s tags. The tag is a dictionary in the format:<br />
** ''name'' : the name for the tag, a string between 2 and 100 characters long containing only alphanumeric characters and '-, '_' and '.'.<br />
* '''resources''' (list of resource dictionaries): the item’s resources, see [[#Resource_Collection | Resource Collection]] for the format of resource dictionaries;<br />
* '''extras''' (list of item extra dictionaries): the item’s extras, extras are arbitrary (key: value) metadata items that can be added to items, each extra dictionary should have keys 'key' (a string), 'value' (a string).<br />
<br />
Parameter automatically managed:<br />
* '''author''' (string): the name of the item’s author (the owner of the gcube-token);<br />
* '''author_email''' (string): the email address of the item’s author (the email of the owner of gcube-token);<br />
* '''owner_org''' (string): the id of the item’s owning organization, see organization_list() or organization_list_for_user() for available values (the VRE corrsponding to the gcube-token).<br />
<br />
<!--<br />
* '''type''' (string): the type of the item, IitemForm plugins associate themselves with different item types and provide custom item handling behaviour for these types;<br />
* '''relationships_as_object''' (list of relationship dictionaries): see package_relationship_create() for the format of relationship dictionaries;<br />
* '''relationships_as_subject''' (list of relationship dictionaries): see package_relationship_create() for the format of relationship dictionaries;<br />
--><br />
<br />
<br />
===== Geo-Indexing your datasets =====<br />
<br />
In order to make an Item searchable by location, it must have a special extra, with its key named ‘spatial’. The value must be a valid GeoJSON geometry, for example:<br />
<br />
<pre><br />
{<br />
"type":"Polygon",<br />
"coordinates":[[[2.05827, 49.8625],[2.05827, 55.7447], [-6.41736, 55.7447], [-6.41736, 49.8625], [2.05827, 49.8625]]]<br />
}<br />
</pre><br />
or <br />
<br />
<pre><br />
{<br />
"type": "Point",<br />
"coordinates": [-3.145,53.078]<br />
}<br />
</pre><br />
<br />
If at least one profile has been defined within this context, then you need to specify the profile's type when creating the item. You need to insert, among the extras of the JSON object describing the item, a '''''system:type''''' property with the proper value (i.e. its value must be equal to the type property contained in the profile). The validation of the submitted request will be performed against the profile whose type has been specified. The other profile's properties need to be specified within the extras field as well.<br />
<br />
If no profile has been defined, then no validation will be performed. Thus you do not need to set any '''''system:type''''' property.<br />
<br />
Suppose you discover that in the context you are willing to publish are defined the following profiles: 'A', 'B' and 'C'.<br />
<br />
You are interested in publishing an item with profile 'A', thus you download the XML of profile 'A' which looks like the following<br />
<br />
<source lang="xml"><br />
<metadataformat type="A"><br />
<metadatafield><br />
<fieldName>Input Parameter</fieldName><br />
<mandatory>false</mandatory><br />
<dataType>String</dataType><br />
<note>Input parameter expected for the execution of the process</note><br />
</metadatafield><br />
<metadatafield><br />
<fieldName>Output Parameter</fieldName><br />
<mandatory>false</mandatory><br />
<dataType>String</dataType><br />
<note>Output parameter expected from the execution of the process </note><br />
</metadatafield><br />
<metadatafield><br />
<fieldName>Process Author</fieldName><br />
<mandatory>false</mandatory><br />
<dataType>String</dataType><br />
<maxOccurs>*</maxOccurs><br />
<defaultValue/><br />
<note>The name of the author, with email and ORCID. The format should be: family, given[, email][, ORCID]. Example: Smith, John, js@acme.org, orcid.org/0000-0002-1825-0097</note><br />
<validator><br />
<regularExpression>^[a-zA-ZÀ-ÿ .'-]+, [a-zA-ZÀ-ÿ .'-]+[,]*([a-zA-Z0-9_!#$%’*+=?`{|}~^.-]+@[a-zA-Z0-9À-ÿ.-]+)?[,]*(orcid.org\/0000-000(1-[5-9]|2-[0-9]|3-[0-4])\d\d\d-\d\d\d[\dX])?$</regularExpression><br />
</validator><br />
</metadatafield><br />
</metadataformat><br />
</source><br />
<br />
The previous information can be easily discovered with the [[#Profile_Collection | Profile Collection]].<br />
Then, the JSON object of the create request will look like<br />
<br />
<pre><br />
{<br />
"metadataformat": {<br />
"metadatafield": [<br />
{<br />
"note": "Input parameter expected for the execution of the process",<br />
"fieldName": "Input Parameter",<br />
"dataType": "String",<br />
"mandatory": false<br />
},<br />
{<br />
"note": "Output parameter expected from the execution of the process",<br />
"fieldName": "Output Parameter",<br />
"dataType": "String",<br />
"mandatory": false<br />
},<br />
{<br />
"note": "The name of the author, with email and ORCID. The format should\t\t\tbe: family, given[, email][, ORCID]. Example: Smith, John,\t\t\tjs@acme.org, orcid.org/0000-0002-1825-0097",<br />
"fieldName": "Process Author",<br />
"defaultValue": "",<br />
"dataType": "String",<br />
"validator": {<br />
"regularExpression": "^[a-zA-ZÀ-ÿ .'-]+, [a-zA-ZÀ-ÿ .'-]+[,\t\t\t\t]*([a-zA-Z0-9_!#$%’*+=?`{|}~^.-]+@[a-zA-Z0-9À-ÿ.-]+)?[,\t\t\t\t]*(orcid.org\\/0000-000(1-[5-9]|2-[0-9]|3-[0-4])\\d\\d\\d-\\d\\d\\d[\\dX])?$"<br />
},<br />
"maxOccurs": "*",<br />
"mandatory": false<br />
}<br />
],<br />
"type": "DataMiner Process"<br />
}<br />
}<br />
</pre><br />
<br />
=== Resource Collection ===<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL<br />
|-<br />
| List || GET || <code>/items/{ITEM_ID}/resources</code><br />
|-<br />
| Create || POST || <code>/items/{ITEM_ID}/resources</code><br />
|-<br />
| Read || GET || <code>/items/{ITEM_ID}/resources/{ID}</code><br />
|-<br />
| Update || PUT || <code>/items/{ITEM_ID}/resources/{ID}</code><br />
|-<br />
| Delete || DELETE || <code>/items/{ITEM_ID}/resources/{ID}</code><br />
|}<br />
<br />
<br />
An Resource is mainly described by the following attributes (* indicate mandatory attributes): <br />
<br />
* '''name*''' (string);<br />
* '''url*''' (string): url of resource;<br />
* <del>'''package_id*''' (string): id of the item that the resource should be added to;</del><br />
* '''revision_id''' (string);<br />
* '''description''' (string);<br />
* '''format''' (string);<br />
* '''mimetype''' (string);<br />
* '''created''' (iso date string);<br />
* '''last_modified''' (iso date string);<br />
<br />
<br />
<!--<br />
* '''size''' (int);<br />
* '''mimetype_inner''' (string) – (optional)<br />
* '''cache_url''' (string) – (optional)<br />
* '''resource_type''' (string) – (optional)<br />
* '''hash''' (string)<br />
* '''cache_last_updated''' (iso date string) – (optional)<br />
* '''upload''' (FieldStorage (optional) needs multipart/form-data) – (optional)<br />
--><br />
<br />
=== Profile Collection ===<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL<br />
|-<br />
| List || GET || /profiles<br />
|-<br />
| Create || PUT || /profiles/{NAME}<br />
|-<br />
| Read || GET || /profiles/{NAME}<br />
|-<br />
| Update || PUT || /profiles/{NAME}<br />
|-<br />
| Delete || DELETE || /profiles/{NAME}<br />
<br />
|}<br />
<br />
Only a [[GCat_Background#Users.2C_Roles_and_Groups | Catalogue Editor]] can manipulate profiles (Create, Update, Delete). These operation are available starting from gCat 1.1.0.<br />
<br />
At creation and update time the provided profile is validated against the defined XSD schema (see [[GCat_Background#Metadata_Profile_v.3 | Item Profile Defintion v3]])<br />
<br />
<br />
A profile is defined using XML. In place of ''application/json'' Use <br />
<br />
<pre>Accept : application/xml</pre><br />
<br />
and <br />
<br />
<pre>Content-Type : application/xml</pre><br />
<br />
<br />
Listing return a JSON Array. It is also possible to read the representation of a profile as JSON which is an automatic mapping to of the original XML definition to JSON.<br />
<br />
An example of Profiles listing is :<br />
<br />
<pre><br />
[<br />
"EOSCService",<br />
"SoBigData.eu: Dataset Metadata",<br />
"SoBigData.eu: Application Metadata",<br />
"SoBigData.eu: Method Metadata"<br />
]<br />
</pre><br />
<br />
An example of Profile is :<br />
<br />
<source lang="xml"><br />
<metadataformat type="SoBigData Method"><br />
<metadatafield><br />
<fieldName>External Identifier</fieldName><br />
<dataType>String</dataType><br />
<mandatory>false</mandatory><br />
<note>This applies only to methods that have been already published. Insert here a DOI, an handle, and any other Identifier assigned when publishing the dataset alsewhere.</note><br />
</metadatafield><br />
<metadatafield><br />
<fieldName>Creator</fieldName><br />
<mandatory>true</mandatory><br />
<note>The name of the creator, with email and ORCID. The format should be: family, given[, email][, ORCID]. Example: Smith, John, js@acme.org, orcid.org//0000-0002-1825-0097</note><br />
<validator><br />
<regularExpression>^[a-zA-Z .'-]+, [a-zA-Z .'-]+[, ]*([a-zA-Z0-9_!#$%’*+=?`{|}~^.-]+@[a-zA-Z0-9.-]+)?[, ]*(orcid.org\/\/0000-000(1-[5-9]|2-[0-9]|3-[0-4])\d\d\d-\d\d\d[\dX])?$</regularExpression><br />
</validator><br />
</metadatafield><br />
<metadatafield><br />
<fieldName>CreationDate</fieldName><br />
<mandatory>true</mandatory><br />
<dataType>Time</dataType><br />
<note>The date of creation of the dataset (different from the date of registration of the dataset automatically added by the system). Use ISO 8601 Date Format: YYYY-MM-DD[ HH:MM] Ex. 1998-11-10 or 2015-05-29 11:55</note><br />
<validator><br />
<regularExpression>^(\d{4}\-(0?[1-9]|1[012])\-(0?[1-9]|[12][0-9]|3[01]))+([ ]+(\d{2}(:?\d{2})?)?)?$</regularExpression><br />
</validator><br />
</metadatafield><br />
<metadatafield><br />
<fieldName>Owner</fieldName><br />
<mandatory>true</mandatory><br />
<note>The name of the owner, with email and ORCID. The format should be: family, given[, email][, ORCID]. Example: Smith, John, js@acme.org, orcid.org//0000-0002-1825-0097</note><br />
<validator><br />
<regularExpression>^[a-zA-Z .'-]+, [a-zA-Z .'-]+[, ]*([a-zA-Z0-9_!#$%’*+=?`{|}~^.-]+@[a-zA-Z0-9.-]+)?[, ]*(orcid.org\/\/0000-000(1-[5-9]|2-[0-9]|3-[0-4])\d\d\d-\d\d\d[\dX])?$</regularExpression><br />
</validator><br />
</metadatafield><br />
....<br />
<metadatafield><br />
<fieldName>License term</fieldName><br />
<mandatory>false</mandatory><br />
<dataType>Time_Interval</dataType><br />
<note>Period of time during which the dataset may be used. Use ISO 8601 Date Format: YYYY-MM-DD[ HH:MM] Ex. 2016-07-31 or 2015-05-10 12:00</note><br />
<vocabulary/><br />
<validator><br />
<regularExpression>^(\d{4}\-(0?[1-9]|1[012])\-(0?[1-9]|[12][0-9]|3[01]))+([ ]+(\d{2}(:?\d{2})?)?)?$</regularExpression><br />
</validator><br />
</metadatafield><br />
<metadatafield><br />
<fieldName>Requirement of non-disclosure (confidentiality mark)</fieldName><br />
<mandatory>false</mandatory><br />
<note>Requirement of non-disclosure (confidentiality mark). Whether the dataset bears confidentiality mark/may be used and shared subject to the obligation of non-disclosure</note><br />
</metadatafield><br />
</metadataformat><br />
</source><br />
<br />
<br />
You can find all details about profiles at [[GCat_Background#Metadata | Item Profile]]<br />
<br />
=== Namespace Collection ===<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL<br />
|-<br />
| List || GET || /namespaces<br />
|}<br />
<br />
An example of given result is:<br />
<br />
<pre><br />
[<br />
{<br />
"id": "contact",<br />
"title": "Contact Title",<br />
"name": "contact",<br />
"description": "This section is about Contact(s)"<br />
},<br />
{<br />
"id": "developer_information",<br />
"title": "Developer Information",<br />
"name": "developer_information",<br />
"description": "This section is about Developer(s)"<br />
},<br />
{<br />
"id": "extra_information",<br />
"title": "Extras",<br />
"name": "extra_information",<br />
"description": "This section is about Extra(s)"<br />
},<br />
{<br />
"id": "ReportDetails",<br />
"title": "Report Details",<br />
"name": "ReportDetails",<br />
"description": "Report Details"<br />
}<br />
]<br />
</pre><br />
<br />
=== License Collection ===<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL<br />
|-<br />
| List || GET || /licenses<br />
|}<br />
<br />
<br />
An example of given result is:<br />
<br />
<pre><br />
[<br />
{<br />
"status": "active",<br />
"maintainer": "Creative Commons",<br />
"od_conformance": "rejected",<br />
"family": "Creative Commons",<br />
"osd_conformance": "not reviewed",<br />
"domain_data": true,<br />
"title": "Creative Commons Attribution-NonCommercial-ShareAlike 4.0",<br />
"url": "https://creativecommons.org/licenses/by-nc-sa/4.0/",<br />
"domain_content": true,<br />
"is_okd_compliant": false,<br />
"is_osi_compliant": false,<br />
"domain_software": false,<br />
"id": "CC-BY-NC-SA-4.0"<br />
},<br />
{<br />
"status": "active",<br />
"maintainer": "Creative Commons",<br />
"od_conformance": "rejected",<br />
"family": "Creative Commons",<br />
"osd_conformance": "not reviewed",<br />
"domain_data": true,<br />
"title": "Creative Commons Attribution-NonCommercial-NoDerivatives 4.0",<br />
"url": "https://creativecommons.org/licenses/by-nc-nd/4.0/",<br />
"domain_content": true,<br />
"is_okd_compliant": false,<br />
"is_osi_compliant": false,<br />
"domain_software": false,<br />
"id": "CC-BY-NC-ND-4.0"<br />
},<br />
...<br />
{<br />
"status": "active",<br />
"maintainer": "Free Software Foundation",<br />
"od_conformance": "not reviewed",<br />
"family": "",<br />
"osd_conformance": "approved",<br />
"domain_data": false,<br />
"title": "GNU Affero General Public License v3",<br />
"url": "http://www.opensource.org/licenses/AGPL-3.0",<br />
"domain_content": false,<br />
"is_okd_compliant": false,<br />
"is_osi_compliant": true,<br />
"domain_software": true,<br />
"id": "AGPL-3.0"<br />
}<br />
]<br />
</pre><br />
<br />
=== Group Collection ===<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL<br />
|-<br />
| List || GET || /groups<br />
|-<br />
| Create || POST || /groups<br />
|-<br />
| Read || GET || /groups/{NAME}<br />
|-<br />
| Update || PUT || /groups/{NAME}<br />
|-<br />
| Patch || PATCH || /groups/{NAME}<br />
|-<br />
| Delete || DELETE || /groups/{NAME}<br />
|-<br />
| Purge || PURGE || /groups/{NAME}<br />
|-<br />
| Purge || DELETE || /groups/{NAME}?purge=true<br />
|}<br />
<br />
Only [[GCat_Background#Users.2C_Roles_and_Groups | Catalogue Admin]] are able to invoke non-safe methods.<br />
<br />
<br />
A group is mainly described by the following attributes ('''*''' indicate mandatory attributes):<br />
* '''name*''' (string): the name of the group, a string between 2 and 100 characters long, containing only lowercase alphanumeric characters, '-' and '_' ;<br />
* '''id''' (string): the id of the group;<br />
* '''title''' (string): the title of the group;<br />
* '''description''' (string): the description of the group;<br />
* '''image_url''' (string): the URL to an image to be displayed on the group’s page;<br />
* '''state''' (string, default: 'active'): the current state of the group, e.g. 'active' or 'deleted', only active groups show up in search results and other lists of groups, this parameter will be ignored if you are not authorized to change the state of the group;<br />
* '''extras''' (list of dataset extra dictionaries): the group’s extras, extras are arbitrary (key: value) metadata items that can be added to groups, each extra dictionary should have keys 'key' (a string), 'value' (a string), and optionally 'deleted'.<br />
<br />
<!-- <br />
Other attributes used internally:<br />
* ''type'' (string): the type of the group. It cannot be 'organization';<br />
* ''approval_status'' (string): indicate if the group has been approved;<br />
* ''packages'' (list of dictionaries): the items that belong to the group, a list of dictionaries each with keys 'name' (string, the id or name of the items) and optionally 'title' (string, the title of the item);<br />
* ''groups'' (list of dictionaries): the groups that belong to the group, a list of dictionaries each with key 'name' (string, the id or name of the group) and optionally 'capacity' (string, the capacity in which the group is a member of the group);<br />
* ''users'' (list of dictionaries): the users that belong to the group, a list of dictionaries each with key 'name' (string, the id or name of the user) and optionally 'capacity' (string, the capacity in which the user is a member of the group).<br />
--><br />
<br />
==== Representation ====<br />
<br />
An example of Group representation is:<br />
<br />
<pre><br />
{<br />
"name": "group_name",<br />
"id": "fd72502e-ddfa-4121-a12a-bb42e572d649",<br />
"title": "Group Title",<br />
"description": "",<br />
"image_url": "",<br />
"state": "active",<br />
"extras": [],<br />
"created": "2018-03-22T11:48:42.612063",<br />
"revision_id": "0b5dda21-9bb0-4993-896b-d865e0d2f1f3"<br />
}<br />
</pre><br />
<br />
=== Organization Collection ===<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL<br />
|-<br />
| List || GET || /organizations<br />
|-<br />
| Create || POST || /organizations<br />
|-<br />
| Read || GET || /organizations/{NAME}<br />
|-<br />
| Update || PUT || /organizations/{NAME}<br />
|-<br />
| Patch || PATCH || /organizations/{NAME}<br />
|-<br />
| Delete || DELETE || /organizations/{NAME}<br />
|-<br />
| Purge || PURGE || /organizations/{NAME}<br />
|-<br />
| Purge || DELETE || /organizations/{NAME}?purge=true<br />
|}<br />
<br />
Only [[GCat_Background#Users.2C_Roles_and_Groups | Catalogue Admin]] are able to invoke non-safe methods.<br />
<br />
An Organization is mainly described by the following attributes (* indicate mandatory attributes): <br />
<br />
<br />
* '''name*''' (string): the name of the organization, a string between 2 and 100 characters long, containing only lowercase alphanumeric characters, '-' and '_';<br />
* '''id''' (string): the id of the organization;<br />
* '''title''' (string): the title of the organization;<br />
* '''description''' (string): the description of the organization;<br />
* '''image_url''' (string): the URL to an image to be displayed on the organization’s page;<br />
* '''state''' (string, default: 'active'): the current state of the organization, e.g. 'active' or 'deleted', only active organizations show up in search results and other lists of organizations, this parameter will be ignored if you are not authorized to change the state of the organization;<br />
* '''approval_status''' (string);<br />
* '''extras''' (list of dataset extra dictionaries): the organization’s extras (optional), extras are arbitrary (key: value) metadata items that can be added to organizations, each extra dictionary should have keys 'key' (a string), 'value' (a string), and optionally 'deleted'.<br />
<br />
<!--<br />
* '''packages (list of dictionaries): the items that belong to the organization, a list of dictionaries each with keys 'name' (string, the id or name of the dataset) and optionally 'title' (string, the title of the dataset)<br />
* '''users (list of dictionaries): the users that belong to the organization, a list of dictionaries each with key 'name' (string, the id or name of the user) and optionally 'capacity' (string, the capacity in which the user is a member of the organization)<br />
--><br />
<br />
=== User Collection ===<br />
<br />
{| class="wikitable"<br />
! Operation || HTTP Method || URL<br />
|-<br />
| List || GET || /users<br />
|-<br />
| Create || POST || /users<br />
|-<br />
| Read || GET || /users/{NAME}<br />
|-<br />
| Update || PUT || /users/{NAME}<br />
|-<br />
| Delete || DELETE || /users/{NAME}<br />
|}<br />
<br />
Only [[GCat_Background#Users.2C_Roles_and_Groups | Catalogue Admin]] are able to invoke non-safe methods.<br />
<br />
A User is mainly described by the following attributes (* indicate mandatory attributes):<br />
* '''name*''' (string): the name of the user, a string between 2 and 100 characters in length, containing only lowercase alphanumeric characters, '-' and '_';<br />
* '''id''' (string): the id of the user;<br />
* '''fullname''' (string): the full name of the user;<br />
* '''email''' (string): the email address for the user;<br />
* '''password''' (string): the password of the user, a string of at least 4 characters (parameter only used to create a new user);<br />
* '''about''' (string): a description of the user.<br />
<br />
<br />
<pre><br />
{<br />
"name": "luca_frosini",<br />
"id": " 5f5c2bc0-bcd4-460a-9548-52cd4d33b263",<br />
"fullname": "Luca Frosini"<br />
}<br />
</pre></div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=StorageHub_REST_API&diff=33783StorageHub REST API2021-01-22T16:05:57Z<p>Massimiliano.assante: /* Java */</p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
= Overview =<br />
<br />
The StorageHub APIs components provide a simple access to StorageHub service. <br />
<br />
It is conceived to support both Java and REST-based calls. <br />
<br />
== Dependencies ==<br />
<br />
=== Maven coordinates ===<br />
<br />
<source lang="java"><br />
<dependency><br />
<groupId>org.gcube.common</groupId><br />
<artifactId>storagehub-client-library</artifactId><br />
<version>[1.0.0,2.0.0)</version><br />
</dependency><br />
</source><br />
<br />
= Key features =<br />
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.<br />
<br />
'''StorageHub REST interface''' supports the following operations:<br />
* '''Retrieve WS''': to retrieve the user Workspace;<br />
* '''Folder Listing''': to list the content of a folder;<br />
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;<br />
* '''Find''': to find a file or a folder by name or pattern;<br />
* '''Delete''': to remove a file or a folder (including subfolders);<br />
* '''Download''': to download a file or a folder in ZIP format;<br />
* '''Get Public Link''': to get a public link of a file;<br />
* '''Create Folder''': to create a folder in the given parent folder;<br />
* '''Unzip''': to upload a zip file in a specific folder;<br />
* '''Upload file''': to upload a file in a folder.<br />
* '''Versions''': to get a specific version of a file;<br />
<br />
== API ==<br />
<br />
The base-url of this service on D4Science production is the following:<br />
<br />
https://api.d4science.org/workspace<br />
<br />
== Retrieve Workspace ==<br />
<br />
Returns the Item representing the workspace of the user retrieved by the token used for the call.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.getWSRoot()<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The workspace root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Get Item ById ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
</source><br />
<br />
== List Item Versions, Get Item Version ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
import org.gcube.common.storagehub.model.service.Version;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
List<Version> fileVersions = fileContainer.getVersions();<br />
<br />
//to download a version<br />
fileContainer.downloadSpecificVersion(versionName)<br />
</source><br />
<br />
== Folder Listing ==<br />
<br />
Returns the content of a Folder<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
import org.gcube.common.storagehub.model.items.*;<br />
<br />
...<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
List<? extends Item> items = folderContainer.list().getItems();<br />
<br />
for (Item item : items) {<br />
System.out.println("name:" + item.getName() + " is a File?: " + (item instanceof AbstractFileItem));<br />
System.out.println("name:" + item.getName() + " is a folder?: " + (item instanceof FolderItem));<br />
System.out.println("name:" + item.getName() + " is a shared folder?: " + (item instanceof SharedFolder));<br />
System.out.println("name:" + item.getName() + " is a VRE folder?: " + (item instanceof VreFolder));<br />
}<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}<br />
<br />
==== responses ====<br />
<br />
200 the list of the Items (in JSON format)<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Retrieve VRE Folder ==<br />
<br />
Return the Item representing the root of the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.openVREFolder();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/vrefolder?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The VRE folder root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Create Folder ==<br />
<br />
Creates a new folder under another folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
root.newFolder("name", "description");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} <br />
Content-Type: application/x-www-form-urlencoded <br />
{String name, String description, boolean hidden}<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder item identifier is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Delete File or Folder ==<br />
<br />
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;<br />
<br />
=== Java ===<br />
<source lang="java"><br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
fileContainer.delete();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)<br />
<br />
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder or item is deleted<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Upload File ==<br />
<br />
Creates a new file under a folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadFile(is, "name", "description");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String name, String description, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}<br />
<br />
== Upload Archive ==<br />
<br />
Extract the content of the specified archive in a destination Folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadArchive(is, "parentFolderName");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String parentFolderName, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}<br />
<br />
== Get Public Url of a File ==<br />
<br />
Returns the public URL of a file<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open({item-id}).asFile().getPublicLink();<br />
<br />
//returns the public link for a specific version of the file <br />
shc.open({item-id}).asFile().getPublicLink({file-version});<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional<br />
<br />
== Find By Name ==<br />
<br />
Returns all the items with a name that matches a pattern on the first level of a given folder. <br />
The result will be empty in case of none of the item in the folder matches the pattern.<br />
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) <br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
//getting my root workspace folder<br />
FolderContainer myRoot = shc.getWSRoot();<br />
myRoot.findByName("{pattern}");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} <br />
<br />
==== responses ====<br />
<br />
200 A JSON with the retrieved items or an empty itemList. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
= DataMiner and SAI Interactions =<br />
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. <br />
Here are some examples:<br />
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]<br />
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]<br />
<br />
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].<br />
<br />
= R Client =<br />
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=StorageHub_REST_API&diff=33782StorageHub REST API2021-01-22T16:04:11Z<p>Massimiliano.assante: /* Java */</p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
= Overview =<br />
<br />
The StorageHub APIs components provide a simple access to StorageHub service. <br />
<br />
It is conceived to support both Java and REST-based calls. <br />
<br />
== Dependencies ==<br />
<br />
=== Maven coordinates ===<br />
<br />
<source lang="java"><br />
<dependency><br />
<groupId>org.gcube.common</groupId><br />
<artifactId>storagehub-client-library</artifactId><br />
<version>[1.0.0,2.0.0)</version><br />
</dependency><br />
</source><br />
<br />
= Key features =<br />
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.<br />
<br />
'''StorageHub REST interface''' supports the following operations:<br />
* '''Retrieve WS''': to retrieve the user Workspace;<br />
* '''Folder Listing''': to list the content of a folder;<br />
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;<br />
* '''Find''': to find a file or a folder by name or pattern;<br />
* '''Delete''': to remove a file or a folder (including subfolders);<br />
* '''Download''': to download a file or a folder in ZIP format;<br />
* '''Get Public Link''': to get a public link of a file;<br />
* '''Create Folder''': to create a folder in the given parent folder;<br />
* '''Unzip''': to upload a zip file in a specific folder;<br />
* '''Upload file''': to upload a file in a folder.<br />
* '''Versions''': to get a specific version of a file;<br />
<br />
== API ==<br />
<br />
The base-url of this service on D4Science production is the following:<br />
<br />
https://api.d4science.org/workspace<br />
<br />
== Retrieve Workspace ==<br />
<br />
Returns the Item representing the workspace of the user retrieved by the token used for the call.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.getWSRoot()<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The workspace root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Get Item ById ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
</source><br />
<br />
== List Item Versions, Get Item Version ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
import org.gcube.common.storagehub.model.service.Version;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
List<Version> fileVersions = fileContainer.getVersions();<br />
<br />
//to download a version<br />
fileContainer.downloadSpecificVersion(versionName)<br />
</source><br />
<br />
== Folder Listing ==<br />
<br />
Returns the content of a Folder<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
List<? extends Item> items = folderContainer.list().getItems();<br />
<br />
for (Item item : items) {<br />
System.out.println("item name:" + item.getName() + " is a File?: " + (item instanceof org.gcube.common.storagehub.model.items.AbstractFileItem));<br />
System.out.println("item name:" + item.getName() + " is a private folder?: " + (item instanceof org.gcube.common.storagehub.model.items.FolderItem));<br />
System.out.println("item name:" + item.getName() + " is a shared folder?: " + (item instanceof org.gcube.common.storagehub.model.items.SharedFolder));<br />
System.out.println("item name:" + item.getName() + " is a VRE folder?: " + (item instanceof org.gcube.common.storagehub.model.items.VreFolder));<br />
}<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}<br />
<br />
==== responses ====<br />
<br />
200 the list of the Items (in JSON format)<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Retrieve VRE Folder ==<br />
<br />
Return the Item representing the root of the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.openVREFolder();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/vrefolder?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The VRE folder root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Create Folder ==<br />
<br />
Creates a new folder under another folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
root.newFolder("name", "description");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} <br />
Content-Type: application/x-www-form-urlencoded <br />
{String name, String description, boolean hidden}<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder item identifier is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Delete File or Folder ==<br />
<br />
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;<br />
<br />
=== Java ===<br />
<source lang="java"><br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
fileContainer.delete();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)<br />
<br />
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder or item is deleted<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Upload File ==<br />
<br />
Creates a new file under a folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadFile(is, "name", "description");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String name, String description, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}<br />
<br />
== Upload Archive ==<br />
<br />
Extract the content of the specified archive in a destination Folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadArchive(is, "parentFolderName");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String parentFolderName, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}<br />
<br />
== Get Public Url of a File ==<br />
<br />
Returns the public URL of a file<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open({item-id}).asFile().getPublicLink();<br />
<br />
//returns the public link for a specific version of the file <br />
shc.open({item-id}).asFile().getPublicLink({file-version});<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional<br />
<br />
== Find By Name ==<br />
<br />
Returns all the items with a name that matches a pattern on the first level of a given folder. <br />
The result will be empty in case of none of the item in the folder matches the pattern.<br />
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) <br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
//getting my root workspace folder<br />
FolderContainer myRoot = shc.getWSRoot();<br />
myRoot.findByName("{pattern}");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} <br />
<br />
==== responses ====<br />
<br />
200 A JSON with the retrieved items or an empty itemList. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
= DataMiner and SAI Interactions =<br />
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. <br />
Here are some examples:<br />
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]<br />
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]<br />
<br />
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].<br />
<br />
= R Client =<br />
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=StorageHub_REST_API&diff=33775StorageHub REST API2021-01-12T08:01:50Z<p>Massimiliano.assante: /* API */</p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
= Overview =<br />
<br />
The StorageHub APIs components provide a simple access to StorageHub service. <br />
<br />
It is conceived to support both Java and REST-based calls. <br />
<br />
== Dependencies ==<br />
<br />
=== Maven coordinates ===<br />
<br />
<source lang="java"><br />
<dependency><br />
<groupId>org.gcube.common</groupId><br />
<artifactId>storagehub-client-library</artifactId><br />
<version>[1.0.0,2.0.0)</version><br />
</dependency><br />
</source><br />
<br />
= Key features =<br />
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.<br />
<br />
'''StorageHub REST interface''' supports the following operations:<br />
* '''Retrieve WS''': to retrieve the user Workspace;<br />
* '''Folder Listing''': to list the content of a folder;<br />
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;<br />
* '''Find''': to find a file or a folder by name or pattern;<br />
* '''Delete''': to remove a file or a folder (including subfolders);<br />
* '''Download''': to download a file or a folder in ZIP format;<br />
* '''Get Public Link''': to get a public link of a file;<br />
* '''Create Folder''': to create a folder in the given parent folder;<br />
* '''Unzip''': to upload a zip file in a specific folder;<br />
* '''Upload file''': to upload a file in a folder.<br />
* '''Versions''': to get a specific version of a file;<br />
<br />
== API ==<br />
<br />
The base-url of this service on D4Science production is the following:<br />
<br />
https://api.d4science.org/workspace<br />
<br />
== Retrieve Workspace ==<br />
<br />
Returns the Item representing the workspace of the user retrieved by the token used for the call.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.getWSRoot()<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The workspace root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Get Item ById ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
</source><br />
<br />
== List Item Versions, Get Item Version ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
import org.gcube.common.storagehub.model.service.Version;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
List<Version> fileVersions = fileContainer.getVersions();<br />
<br />
//to download a version<br />
fileContainer.downloadSpecificVersion(versionName)<br />
</source><br />
<br />
== Folder Listing ==<br />
<br />
Returns the content of a Folder<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
List<? extends Item> items = folderContainer.list().getItems();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}<br />
<br />
==== responses ====<br />
<br />
200 the list of the Items (in JSON format)<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Retrieve VRE Folder ==<br />
<br />
Return the Item representing the root of the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.openVREFolder();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/vrefolder?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The VRE folder root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Create Folder ==<br />
<br />
Creates a new folder under another folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
root.newFolder("name", "description");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} <br />
Content-Type: application/x-www-form-urlencoded <br />
{String name, String description, boolean hidden}<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder item identifier is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Delete File or Folder ==<br />
<br />
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;<br />
<br />
=== Java ===<br />
<source lang="java"><br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
fileContainer.delete();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)<br />
<br />
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder or item is deleted<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Upload File ==<br />
<br />
Creates a new file under a folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadFile(is, "name", "description");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String name, String description, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}<br />
<br />
== Upload Archive ==<br />
<br />
Extract the content of the specified archive in a destination Folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadArchive(is, "parentFolderName");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String parentFolderName, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}<br />
<br />
== Get Public Url of a File ==<br />
<br />
Returns the public URL of a file<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open({item-id}).asFile().getPublicLink();<br />
<br />
//returns the public link for a specific version of the file <br />
shc.open({item-id}).asFile().getPublicLink({file-version});<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional<br />
<br />
== Find By Name ==<br />
<br />
Returns all the items with a name that matches a pattern on the first level of a given folder. <br />
The result will be empty in case of none of the item in the folder matches the pattern.<br />
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) <br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
//getting my root workspace folder<br />
FolderContainer myRoot = shc.getWSRoot();<br />
myRoot.findByName("{pattern}");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} <br />
<br />
==== responses ====<br />
<br />
200 A JSON with the retrieved items or an empty itemList. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
= DataMiner and SAI Interactions =<br />
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. <br />
Here are some examples:<br />
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]<br />
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]<br />
<br />
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].<br />
<br />
= R Client =<br />
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=StorageHub_REST_API&diff=33346StorageHub REST API2020-05-20T15:45:11Z<p>Massimiliano.assante: /* Delete File or Folder */</p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
= Overview =<br />
<br />
The StorageHub APIs components provide a simple access to StorageHub service. <br />
<br />
It is conceived to support both Java and REST-based calls. <br />
<br />
== Dependencies ==<br />
<br />
=== Maven coordinates ===<br />
<br />
<source lang="java"><br />
<dependency><br />
<groupId>org.gcube.common</groupId><br />
<artifactId>storagehub-client-library</artifactId><br />
<version>[1.0.0,2.0.0)</version><br />
</dependency><br />
</source><br />
<br />
= Key features =<br />
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.<br />
<br />
'''StorageHub REST interface''' supports the following operations:<br />
* '''Retrieve WS''': to retrieve the user Workspace;<br />
* '''Folder Listing''': to list the content of a folder;<br />
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;<br />
* '''Find''': to find a file or a folder by name or pattern;<br />
* '''Delete''': to remove a file or a folder (including subfolders);<br />
* '''Download''': to download a file or a folder in ZIP format;<br />
* '''Get Public Link''': to get a public link of a file;<br />
* '''Create Folder''': to create a folder in the given parent folder;<br />
* '''Unzip''': to upload a zip file in a specific folder;<br />
* '''Upload file''': to upload a file in a folder.<br />
* '''Versions''': to get a specific version of a file;<br />
<br />
== API ==<br />
<br />
== Retrieve Workspace ==<br />
<br />
Returns the Item representing the workspace of the user retrieved by the token used for the call.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.getWSRoot()<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The workspace root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Get Item ById ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
</source><br />
<br />
== List Item Versions, Get Item Version ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
import org.gcube.common.storagehub.model.service.Version;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
List<Version> fileVersions = fileContainer.getVersions();<br />
<br />
//to download a version<br />
fileContainer.downloadSpecificVersion(versionName)<br />
</source><br />
<br />
== Folder Listing ==<br />
<br />
Returns the content of a Folder<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
List<? extends Item> items = folderContainer.list().getItems();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}<br />
<br />
==== responses ====<br />
<br />
200 the list of the Items (in JSON format)<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Retrieve VRE Folder ==<br />
<br />
Return the Item representing the root of the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.openVREFolder();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/vrefolder?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The VRE folder root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Create Folder ==<br />
<br />
Creates a new folder under another folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
root.newFolder("name", "description");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} <br />
Content-Type: application/x-www-form-urlencoded <br />
{String name, String description, boolean hidden}<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder item identifier is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Delete File or Folder ==<br />
<br />
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;<br />
<br />
=== Java ===<br />
<source lang="java"><br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
fileContainer.delete();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)<br />
<br />
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder or item is deleted<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Upload File ==<br />
<br />
Creates a new file under a folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadFile(is, "name", "description");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String name, String description, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}<br />
<br />
== Upload Archive ==<br />
<br />
Extract the content of the specified archive in a destination Folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadArchive(is, "parentFolderName");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String parentFolderName, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}<br />
<br />
== Get Public Url of a File ==<br />
<br />
Returns the public URL of a file<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open({item-id}).asFile().getPublicLink();<br />
<br />
//returns the public link for a specific version of the file <br />
shc.open({item-id}).asFile().getPublicLink({file-version});<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional<br />
<br />
== Find By Name ==<br />
<br />
Returns all the items with a name that matches a pattern on the first level of a given folder. <br />
The result will be empty in case of none of the item in the folder matches the pattern.<br />
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) <br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
//getting my root workspace folder<br />
FolderContainer myRoot = shc.getWSRoot();<br />
myRoot.findByName("{pattern}");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} <br />
<br />
==== responses ====<br />
<br />
200 A JSON with the retrieved items or an empty itemList. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
= DataMiner and SAI Interactions =<br />
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. <br />
Here are some examples:<br />
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]<br />
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]<br />
<br />
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].<br />
<br />
= R Client =<br />
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=StorageHub_REST_API&diff=33345StorageHub REST API2020-05-20T15:43:42Z<p>Massimiliano.assante: </p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
= Overview =<br />
<br />
The StorageHub APIs components provide a simple access to StorageHub service. <br />
<br />
It is conceived to support both Java and REST-based calls. <br />
<br />
== Dependencies ==<br />
<br />
=== Maven coordinates ===<br />
<br />
<source lang="java"><br />
<dependency><br />
<groupId>org.gcube.common</groupId><br />
<artifactId>storagehub-client-library</artifactId><br />
<version>[1.0.0,2.0.0)</version><br />
</dependency><br />
</source><br />
<br />
= Key features =<br />
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.<br />
<br />
'''StorageHub REST interface''' supports the following operations:<br />
* '''Retrieve WS''': to retrieve the user Workspace;<br />
* '''Folder Listing''': to list the content of a folder;<br />
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;<br />
* '''Find''': to find a file or a folder by name or pattern;<br />
* '''Delete''': to remove a file or a folder (including subfolders);<br />
* '''Download''': to download a file or a folder in ZIP format;<br />
* '''Get Public Link''': to get a public link of a file;<br />
* '''Create Folder''': to create a folder in the given parent folder;<br />
* '''Unzip''': to upload a zip file in a specific folder;<br />
* '''Upload file''': to upload a file in a folder.<br />
* '''Versions''': to get a specific version of a file;<br />
<br />
== API ==<br />
<br />
== Retrieve Workspace ==<br />
<br />
Returns the Item representing the workspace of the user retrieved by the token used for the call.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
<br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.getWSRoot()<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The workspace root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Get Item ById ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
</source><br />
<br />
== List Item Versions, Get Item Version ==<br />
<br />
=== Java ===<br />
<source lang="java"><br />
import org.gcube.common.storagehub.client.dsl.FileContainer;<br />
import org.gcube.common.storagehub.client.dsl.StorageHubClient;<br />
import org.gcube.common.storagehub.model.items.Item;<br />
import org.gcube.common.storagehub.model.service.Version;<br />
<br />
...<br />
<br />
String theId = {itemIdentifier} // the identifier of the item <br />
StorageHubClient shc = new StorageHubClient();<br />
FileContainer fileContainer = shc.open(theId).asFile();<br />
List<Version> fileVersions = fileContainer.getVersions();<br />
<br />
//to download a version<br />
fileContainer.downloadSpecificVersion(versionName)<br />
</source><br />
<br />
== Folder Listing ==<br />
<br />
Returns the content of a Folder<br />
<br />
=== Java ===<br />
<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();<br />
List<? extends Item> items = folderContainer.list().getItems();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}<br />
<br />
==== responses ====<br />
<br />
200 the list of the Items (in JSON format)<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Retrieve VRE Folder ==<br />
<br />
Return the Item representing the root of the VRE folder related to the user token.<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer rootContainer = shc.openVREFolder();<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/vrefolder?gcube-token={user-token}<br />
<br />
==== responses ====<br />
<br />
200 The VRE folder root item (in json format) is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Create Folder ==<br />
<br />
Creates a new folder under another folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
root.newFolder("name", "description");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} <br />
Content-Type: application/x-www-form-urlencoded <br />
{String name, String description, boolean hidden}<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder item identifier is returned. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Delete File or Folder ==<br />
<br />
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)<br />
<br />
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)<br />
<br />
<br />
==== responses ====<br />
<br />
200 The Folder or item is deleted<br />
<br />
500 The error is specified in the body of the response message<br />
<br />
== Upload File ==<br />
<br />
Creates a new file under a folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadFile(is, "name", "description");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String name, String description, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}<br />
<br />
== Upload Archive ==<br />
<br />
Extract the content of the specified archive in a destination Folder (specified by its id)<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
FolderContainer root = shc.getWSRoot();<br />
//Creating the folder on the root workspace folder<br />
FileContainer file = null;<br />
try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){<br />
file = root.uploadArchive(is, "parentFolderName");<br />
} catch (Exception e) {<br />
//print the error<br />
}<br />
</source><br />
<br />
<br />
=== REST API ===<br />
<br />
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}<br />
Content-Type: multipart/form-data<br />
{String parentFolderName, File file}<br />
<br />
==== CURL Example ====<br />
<br />
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}<br />
<br />
== Get Public Url of a File ==<br />
<br />
Returns the public URL of a file<br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
shc.open({item-id}).asFile().getPublicLink();<br />
<br />
//returns the public link for a specific version of the file <br />
shc.open({item-id}).asFile().getPublicLink({file-version});<br />
<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional<br />
<br />
== Find By Name ==<br />
<br />
Returns all the items with a name that matches a pattern on the first level of a given folder. <br />
The result will be empty in case of none of the item in the folder matches the pattern.<br />
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*) <br />
<br />
=== Java ===<br />
<source lang="java"><br />
StorageHubClient shc = new StorageHubClient();<br />
//getting my root workspace folder<br />
FolderContainer myRoot = shc.getWSRoot();<br />
myRoot.findByName("{pattern}");<br />
</source><br />
<br />
=== REST API ===<br />
<br />
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token} <br />
<br />
==== responses ====<br />
<br />
200 A JSON with the retrieved items or an empty itemList. <br />
<br />
500 The error is specified in the body of the response message<br />
<br />
= DataMiner and SAI Interactions =<br />
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs. <br />
Here are some examples:<br />
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]<br />
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]<br />
<br />
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].<br />
<br />
= R Client =<br />
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=Social_Networking_Service&diff=33326Social Networking Service2020-04-15T16:24:30Z<p>Massimiliano.assante: /* Messages */</p>
<hr />
<div>{| align="right"<br />
| __TOC__<br />
|}<br />
<br />
The Social Networking Service is a RESTful application which exposes operations coming from both the [https://wiki.gcube-system.org/gcube/Social_Networking_Library gCube Social-Networking-Library] and the [https://wiki.gcube-system.org/gcube/UserManagement_Core User Management] components. The former is the bridge between the Cassandra Cluster (on which social data are stored) and the social applications, whereas the latter is the abstraction layer over the enabling technology of the Infrastructure portals, [https://www.liferay.com/ Liferay], in terms of: roles, users and organizations and how they are mapped to gCube concepts (VREs, Teams, and so on).<br />
<br />
The web services exposes a subset of such functionalities over '''https''' in a standard, reliable and secure way. <br />
<br />
See the vailable REST-API at a [https://dev.d4science.org/swagger/social-networking/ glance here.]<br />
= Discover the Web Service =<br />
For gCube releases older than 4.3, a Service Endpoint that has<br />
<br />
<source lang="java"><br />
Name = SocialNetworking<br />
Category = Portal<br />
</source><br />
<br />
mantains the base path of the service. Starting from gCube 4.3 version, the service runs on SmartGears, thus a gCore Endpoint needs to be searched having<br />
<br />
<source lang="java"><br />
Service Name = SocialNetworking<br />
Service Class = Portal<br />
</source><br />
<br />
The base path is the entry point of the web service. In the '''development''' environment, its current value is<br />
<source lang="xml"><br />
https://socialnetworking-d-d4s.d4science.org/social-networking-library-ws/rest<br />
</source><br />
<br />
In the '''production''' environment, its current value is<br />
<source lang="xml"><br />
https://api.d4sciende.org/rest<br />
</source><br />
<br />
This url is referred as ''BASE_URL''.<br />
<br />
= Service versions =<br />
Starting from gCube 4.3, the service exposes its methods using a more standard naming approach. Moreover, they accept (in case of http POST/PUT methods) JSON objects. The result of all methods is always a JSON object of this kind<br />
<br />
<source lang="java"><br />
{"success": false/true,<br />
"message": ...,<br />
"result": ...,<br />
}<br />
</source><br />
<br />
Where<br />
* success reports if the request succeeded or failed;<br />
* message is a status/error message that can be checked in case of errors (success equals false);<br />
* result is the current result object (it can be a list, a single object and so on depending on the invoked method). <br />
<br />
Inputs are automatically validated before the request is served.<br />
<br />
The older version (Version 1) instead has some limitation thus you are kindly suggested to no longer use it. However, in the following, its methods are going to be reported as well. The older version will be removed starting from the next gCube releases.<br />
<br />
= Authorization =<br />
No matter the version number of the service, every call to one of its method require a gcube-token. See [https://wiki.gcube-system.org/gcube/Authorization_Framework here] for more information about this token. This secret token needs to be valid in the current context. It can be put as query parameter of the http request or within the header. Both approaches are valid. If the token is missing, the operation will be rejected and an http forbidden code will be returned.<br />
<br />
= Usage =<br />
The methods of the Web Service can be called by writing your own REST client application or by using (a lot of) already existent rest client plugins. In the following, we will use the RESTClient plugin for Firefox for POST methods. Your browser can be used for GET methods. The return value of each method is an HTTP Response object that has an HTTP CODE (200, 201, 400, 500, ...) and, optionally, a JSON or text object that represents the created/retrieved resource or better specifies the occurred error. In case of errors, please refer to the [[#Troubleshooting|Troubleshooting]] section for more details.<br />
<br />
= Service methods (Version 2) =<br />
The list of sub-services available in the new version is going to be reported next. The name convention is different. For instance, to read user's posts, the method to be called is<br />
<br />
<source lang="java"><br />
BASE_PATH/2/posts/get-posts-user/<br />
</source><br />
<br />
The real result of the method, if the request succeeded, is contained into the 'result' field of the json object<br />
<br />
<source lang="java"><br />
{"success": ..., "message": ..., "result": "[{first post}, {second post}, ...}<br />
</source><br />
<br />
In the following, the list of available sub-services available under the newest version, its methods and the parameters they need in order to be called are reported. Again, you can use your preferred REST client so examples won't be reported.<br />
<br />
== List of Available Sub-Services and methods==<br />
=== Tokens ===<br />
====Generate application token====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/tokens/generate-application-token<br />
</source><br />
<br />
First of all, you need to have a ''user gcube-token'' for authentication purposes. You can get it using the Token Generator portlet available on the VRE you are interesting the app will post in.<br />
Then, to get an ''application gcube-token'' you can invoke the method. It is an HTTP/POST method that accepts as parameter:<br />
<br />
* a json object of this kind {"app_id": "your-app-id-goes-here"}.<br />
<br />
=== User profile and roles ===<br />
<br />
To retrieve user's profile user profile info (and roles) the subservice to call, once the social networking service endpoint has been retrieved, is this one:<br />
<br />
<code><br />
/2/people/profile<br />
</code><br />
<br />
further details: [https://wiki.gcube-system.org/gcube/OAuth2.0#Step_4_-_Get_member_data get member information]<br />
<br />
=== Full text search ===<br />
==== Search by query ====<br />
Social data are indexed into an ElasticSearch engine and are querable using a full-text approach. You can retrieve more information about this index [https://wiki.gcube-system.org/index.php?title=Social_Networking_Data_Discovery|here].<br />
The link to append to '''Base_URL''' is the following<br />
<source lang="java"><br />
2/full-text-search/search-by-query<br />
</source><br />
<br />
The query can be submitted also to the service, by using the following parameters<br />
<br />
* query: the query to perform against the index [mandatory];<br />
* from : the result set should be returned by starting from the n-th result [default is zero, optional];<br />
* quantity: how many enhanced posts need to be returned [default is 10, optional].<br />
<br />
=== Hashtags ===<br />
==== Get hashtags and occurrences in a context====<br />
The link to append to '''Base_URL''' is the following<br />
<source lang="java"><br />
2/hashtags/get-hashtags-and-occurrences<br />
</source><br />
<br />
The result is of the kind<br />
{"success": true,<br />
"message": null,<br />
"result":{{"hashtag1" : x}, {"hashtag2" : y}, ...}<br />
}<br />
=== Posts ===<br />
==== Write application post====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/write-post-app<br />
</source><br />
<br />
By means of the application token requested with the above method, the app can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters in a JSON object<br />
<br />
* text: text of the post; <br />
* params: additional parameters if your application supports the direct opening of of this update's object e.g. id=12345&type=foo [optional];<br />
* preview_title: a title for the preview of the post [optional];<br />
* preview_host: a host for the preview of the post [optional];<br />
* preview_description: a description for the preview of the post [optional];<br />
* preview_url: the preview url [optional];<br />
* image_url: if you want to put an image, pass its url [optional];<br />
* enable_notification: a boolean value to enable notifications [optional, default is false].<br />
<br />
====Write user post====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/write-post-user<br />
</source><br />
<br />
By means of a valid user token, he/she can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters in a JSON object:<br />
<br />
* text: text of the post; <br />
* preview_title: a title for the preview of the post [optional];<br />
* preview_description: a description for the preview of the post [optional];<br />
* preview_host: a host for the preview of the post [optional];<br />
* preview_url: the preview url [optional];<br />
* image_url: if you want to put an image, pass its url [optional];<br />
* enable_notification: a boolean value to enable notifications [optional].<br />
<br />
====Retrieve all application posts==== <br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-app<br />
</source><br />
<br />
In order to retrieve all the posts the application wrote (in every VRE it is allowed to write in) you can invoke this HTTP/GET method.<br />
====Retrieve all user posts====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user<br />
</source><br />
<br />
====Retrieve all user posts since a given date====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user-since<br />
</source><br />
<br />
If you are interested in retrieving user's posts whose timestamps are greater than a given date you can invoke this method.<br />
This HTTP/GET method accepts as parameter:<br />
<br />
* time: the time of the oldest post to retrieve converted in milliseconds.<br />
<br />
====Retrieve all user posts up to a given quantity====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user-quantity<br />
</source><br />
<br />
If you are interested in retrieving user's posts, starting from the newest up to a given quantity you can invoke this method.<br />
This HTTP/GET method accepts as parameter:<br />
<br />
* quantity: a value greater than zero (the default value is 10).<br />
<br />
====Retrieve the identifiers of the posts a user liked====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-id-liked-posts<br />
</source><br />
<br />
By means of this method a user can retrieve the identifiers of the posts he/she liked.<br />
====Retrieve the posts a user liked====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-liked-posts<br />
</source><br />
<br />
By means of this method a user can retrieve the posts he/she liked.<br />
====Retrieve all vre posts====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-vre<br />
</source><br />
<br />
By means of this method a user that has a token valid for a certain VRE, can retrieve all the posts of that VRE.<br />
====Get posts by hashtag====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-by-hashtag<br />
</source><br />
The method requires as parameter:<br />
* hashtag: the hashtag to be contained in the returned posts[mandatory].<br />
<br />
=== Comments ===<br />
====Retrieve comments by user====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/comments/get-comments-user<br />
</source><br />
It is an http GET method.<br />
====Retrieve comments by user and time====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/comments/get-comments-user-by-time<br />
</source><br />
It is an http GET method. It requires:<br />
* time: the time (in milliseconds) since when the comments need to be retrieved [default is zero, optional];<br />
=== Notifications ===<br />
====Retrieve user's notifications in a given range====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/notifications/get-range-notifications<br />
</source><br />
<br />
By means of this method a user can retrieve his/her last notifications in the range [a, b]. The method requires:<br />
<br />
* from: an integer greater than zero;<br />
* quantity: an integer equal or greater than zero.<br />
<br />
The final range is evaluated as [from, from + quantity].<br />
====Notify job status====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/notifications/notify-job-status<br />
</source><br />
<br />
By means of this method a user can be notified about the status of a job in the infrastructure. Jobs are, for instance, SmartExecutor's plugins or DataMiner's algorithms. The method requires the following parameters in a JSON object via HTTP POST<br />
<br />
* recipient: the username of the user to be alerted [mandatory];<br />
* job_id: the identifier of the job (e.g, a uuid) [mandatory];<br />
* job_name: the name of the running job (e.g., Social-data-indexer-plugin)[mandatory];<br />
* service_name: the name of the service that launched the job (e.g., SmartExecutor) [mandatory];<br />
* status: the status reached by the job, that is any among CANCELLED, CANCELLING, DELETED, DELETING, EXECUTING, FAILED, SUCCEEDED, SUBMITTED, TIMED_OUT, WAITING [mandatory];<br />
* status_message: a detailed message about the status [optional].<br />
<br />
=== Messages ===<br />
====Send a message====<br />
It is an HTTP POST method.<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/write-message<br />
</source><br />
<br />
By default, the sender of the message is the owner of the gcube-token. According to the recipients' notifications preferences, they could receive a portal notification, an email notification and so on.<br />
<br />
Parameters, to be put in a JSON object are:<br />
<br />
* subject: the subject of the message to send;<br />
* body: the body of the message;<br />
* recipients: a list of valid comma separated users' ids (e.g., andrea.rossi, mario.rossi) or users' emails (e.g., andrea.rossi@isti.cnr.it, mario.rossi@isti.cnr.it);<br />
<br />
An example of the body of the request is as follows<br />
<source lang="xml"><br />
{"subject": "....", "body": ".....", "recipients":[{"id":"userid1"},{"id":"userid2"}]}<br />
</source><br />
<br />
or, if you want to use valid mails (i.e. recognized in the infrastructure)<br />
<source lang="xml"><br />
{"subject": "....", "body": ".....", "recipients":[{"id":"mail1"},{"id":"mail2"}]}<br />
</source><br />
<br />
====Get received messages====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/get-received-messages<br />
</source><br />
It is an http GET method.<br />
<br />
====Get sent messages====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/get-sent-messages<br />
</source><br />
It is an http GET method.<br />
=== Users ===<br />
====User exists====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/user-exists<br />
</source><br />
<br />
The method requires<br />
<br />
* username: the username of the user which has to be checked.<br />
<br />
'''Note''' The method can be only invoked with a root-context application token.<br />
====Read user's custom fields====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-custom-attribute<br />
</source><br />
<br />
The method requires<br />
<br />
* attribute: that is the name of a valid attribute to be read.<br />
====Get user's email====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-email<br />
</source><br />
<br />
The method returns the email associated to the owner of the security token used.<br />
====Get user's profile====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-profile<br />
</source><br />
<br />
The method returns the profile of the owner of the security token used.<br />
====Get user's fullname====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-fullname<br />
</source><br />
<br />
The method returns the fullname associated to the owner of the security token used.<br />
====Get all users' usernames in VRE====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames<br />
</source><br />
<br />
The method returns the usernames of the context associated to the security token.<br />
<br />
====Get usernames by global role====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames-by-global-role<br />
</source><br />
<br />
The method requires<br />
<br />
* role-name: that is the name of the role to consider.<br />
<br />
'''''This method can be only invoked with an Infrastructure Token.'''''<br />
<br />
<br />
====Get usernames by role====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames-by-role<br />
</source><br />
<br />
The method requires<br />
<br />
* role-name: that is the name of the role to consider.<br />
<br />
=== VREs ===<br />
====Get User VREs====<br />
This method allows a user to retrieve the list of VREs she belongs to<br />
<br />
<source lang="java"><br />
2/vres/get-my-vres<br />
</source><br />
<br />
The method accepts an option parameter:<br />
* getManagers : if set to true, reports also the VRE Managers of the VRE<br />
<br />
== Troubleshooting ==<br />
<br />
There could be many reasons for a failed request. Please always check the HTTP CODE and the message within the response. <br />
<br />
=== HTTP Error Code ===<br />
This table tries to help you:<br />
<br />
{| border="1" cellpadding="20" cellspacing="0"<br />
!HTTP CODE<br />
!Meaning<br />
|-<br />
|400 - BAD_REQUEST<br />
|Please check the parameters you are passing<br />
|-<br />
|403 - FORBIDDEN<br />
|Is the token valid? / Does it belong to an application? [In case of application request]<br />
|-<br />
|404 - NOT_FOUND<br />
|The object(s) you are asking for is(are) not available<br />
|-<br />
|500 - INTERNAL_SERVER_ERROR<br />
|There was an error while serving your request (retry later)<br />
|-<br />
|}<br />
<br />
<br />
=== Always manage HTTP Redirects on POST requests (i.e. 30X codes) ===<br />
The service accepts both incoming http and https client requests, which are automatically redirect to https protocol by the reverse proxy nginx. This redirect mechanism must be taken into account when POST calls are performed, because it is not (always) supported and transparent by clients. In order to be sure, please check if you get a 30X error and extract the redirection urls within the response header. The following is a working java code that extract such information. It uses the apache-client library to perform http calls.<br />
<br />
<source lang="java"><br />
<br />
// build the client and the request<br />
String path = ... ; <br />
HttpClientBuilder clientBuilder = HttpClientBuilder.create();<br />
CloseableHttpClient httpClient = clientBuilder.build();<br />
HttpPost request = new HttpPost(path);<br />
JSONObject obj = new JSONObject();<br />
<br />
... // build the json object<br />
<br />
// set header properties<br />
request.addHeader("gcube-token", ...);<br />
request.addHeader("Content-type", ContentType.APPLICATION_JSON.toString());<br />
StringEntity params = new StringEntity(obj.toJSONString(), ContentType.APPLICATION_JSON);<br />
request.setEntity(params);<br />
HttpResponse response = httpClient.execute(request);<br />
logger.info(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
<br />
int status = response.getStatusLine().getStatusCode();<br />
<br />
// check the response status and look if it was a redirect problem<br />
if (status != HttpURLConnection.HTTP_OK && (status == HttpURLConnection.HTTP_MOVED_TEMP ||<br />
status == HttpURLConnection.HTTP_MOVED_PERM ||<br />
status == HttpURLConnection.HTTP_SEE_OTHER)) {<br />
<br />
// redirect -> fetch new location<br />
Header[] locations = response.getHeaders("Location");<br />
Header lastLocation = locations[locations.length - 1];<br />
String realLocation = lastLocation.getValue();<br />
logger.debug("New location is " + realLocation);<br />
<br />
// perform again the same request<br />
request = new HttpPost(realLocation);<br />
request.addHeader("gcube-token", ...);<br />
request.addHeader("Content-type", ContentType.APPLICATION_JSON.toString());<br />
params = new StringEntity(obj.toJSONString(), ContentType.APPLICATION_JSON);<br />
request.setEntity(params);<br />
response = httpClient.execute(request);<br />
logger.info(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
} else<br />
logger.debug(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
</source><br />
<br />
= Service methods (Version 1) - DEPRECATED =<br />
In the following, the list of available sub-services available under the first version, its methods and the parameters they need in order to be called are reported. The gcube-token is always required so it won't be reported as parameter. <br />
<br />
'''This version has been replaced by version 2, please do not use this version and update your code asap.'''<br />
<br />
== List of available Sub-Services and methods ==<br />
=== Tokens ===<br />
==== Generate application token ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
tokens/generateApplicationToken<br />
</source><br />
<br />
First of all, you need to have a ''user gcube-token'' for authentication purposes. You can get it using the Token Generator portlet available on the VRE you are interesting the app will post in.<br />
Now, to get an ''application gcube-token'' you can invoke the method. It is an HTTP/POST method that accepts as parameters:<br />
<br />
* appid: the identifier of the application as reported in the Application Profile.<br />
<br />
For example, using the RESTClient plugin for Firefox you will have something like this for requiring a new application token:<br />
<br />
[[File:GenerateApplicationToken.png]]<br />
<br />
Push on the SEND button and if all it's ok, your application gcube-token will be generated and returned as ''plain text'' in the response within the HTTP response having code 201.<br />
=== Posts ['''DEPRECATED'''] ===<br />
==== Write application post ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/writePostApplication<br />
</source><br />
<br />
By means of the application token requested with the above method, the app can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters:<br />
<br />
* text: text of the post; <br />
* params: additional parameters if your application supports the direct opening of of this update's object e.g. id=12345&type=foo [optional];<br />
* previewtitle: a title for the preview of the post [optional];<br />
* previewdescription: a description for the preview of the post [optional];<br />
* httpimageurl: if you want to put an image, pass its url [optional];<br />
* enablenotification: a boolean value to enable notifications [optional].<br />
<br />
For example:<br />
<br />
[[File:WriteApplicationPost.png]]<br />
<br />
Push on SEND and check the code associated to the returned response (201 in case of success).<br />
<br />
==== Retrieve all application posts ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByApp<br />
</source><br />
<br />
In order to retrieve all the posts the application wrote (in every VRE it is allowed to write in) you can invoke this HTTP/GET method.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByApp?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByUser<br />
</source><br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts since a given date ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getRecentPostsByUserAndDate<br />
</source><br />
<br />
If you are interested in retrieving user's posts whose timestamps are greater than a given date you can invoke this method.<br />
This HTTP/GET method accepts this parameter:<br />
<br />
* time: the time of the oldest post to retrieve converted in milliseconds.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getRecentPostsByUserAndDate?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&time=xxxxx<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts up to a given quantity ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getRecentPostsByUser<br />
</source><br />
<br />
If you are interested in retrieving user's posts, starting from the newest up to a given quantity you can invoke this method.<br />
This HTTP/GET method accepts two parameter:<br />
<br />
* quantity: a value greater than zero (the default value is 10).<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getRecentPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&quantity=20<br />
</source><br />
<br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Write user post ['''DEPRECATED'''] ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/writePostUser<br />
</source><br />
<br />
By means of a valid user token, he/she can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters:<br />
<br />
* text: text of the post; <br />
* previewtitle: a title for the preview of the post [optional];<br />
* previewdescription: a description for the preview of the post [optional];<br />
* previewHost: [optional];<br />
* previewUrl: [optional];<br />
* httpimageurl: if you want to put an image, pass its url [optional];<br />
* enablenotification: a boolean value to enable notifications [optional];<br />
<br />
For example:<br />
<br />
[[File:Userpost.png]]<br />
<br />
Push on SEND and check the code associated to the returned response (201 in case of success).<br />
<br />
==== Retrieve the identifiers of the posts a user liked ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllLikedPostIdsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve the identifiers of the posts he/she liked.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllLikedPostIdsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the identifiers of the posts are returned in JSON format.<br />
<br />
==== Retrieve the posts a user liked ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllLikedPostsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve the posts he/she liked. <br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllLikedPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
====Retrieve all vre posts====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByVRE<br />
</source><br />
<br />
By means of this method a user that has a token valid for a certain VRE, can retrieve all the posts of that VRE.<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByVRE?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
=== Notifications ===<br />
====Retrieve user's notifications in a given range====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
notifications/getRangeNotificationsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve his/her last notifications in the range [a, b]. The method requires:<br />
<br />
* from: an integer greater than zero;<br />
* quantity: an integer equal or greater than zero.<br />
<br />
The final range is evaluated as [from, from + quantity]<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/notifications/getRangeNotificationsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&from=from&quantity=quantity<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
=== Messages ['''DEPRECATED'''] ===<br />
<br />
====Send a message====<br />
<br />
['''DEPRECATED'''] See version 2 above<br />
<br />
You can send a message via the web service by using the following POST method:<br />
<br />
* writeMessageToUsers<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
messages/writeMessageToUsers<br />
</source><br />
<br />
By default, the sender of the message is the owner of the gcube-token. If you need to specify a different "sender", you can do that by specifying the parameter sender. Furthermore, according to the recipients' notifications preferences, they could receive a portal notification, a mail notification and so on.<br />
<br />
Parameters are:<br />
<br />
* subject: the subject of the message to send;<br />
* body: the body of the message;<br />
* recipients: a list of valid comma separated users' ids (e.g., andrea.rossi, mario.rossi).<br />
* sender: a valid sender's id [optional]<br />
<br />
An example usage of the method is reported in the picture below (without specifying the sender):<br />
<br />
[[File:Send_message_example.png]]<br />
<br />
=== Users ===<br />
====Read user's custom fields====<br />
You can read user's custom attributes using the following method<br />
<br />
*readCustomAttr<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
users/readCustomAttr<br />
</source><br />
<br />
The ''read'' method requires<br />
<br />
* attribute: that is the name of a valid attribute to be read.<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/users/readCustomAttr?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&attribute=attributeX<br />
</source><br />
<br />
= See Also =<br />
<br />
== D4Science Developers website ==<br />
[https://dev.d4science.org D4Science Developers website]<br />
<br />
== SWAGGER documentation==<br />
[https://dev.d4science.org/swagger/social-networking/ SWAGGER documentation of the Social-Networking RESTful interface]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=Social_Networking_Service&diff=33325Social Networking Service2020-04-15T16:23:46Z<p>Massimiliano.assante: /* Write user post */</p>
<hr />
<div>{| align="right"<br />
| __TOC__<br />
|}<br />
<br />
The Social Networking Service is a RESTful application which exposes operations coming from both the [https://wiki.gcube-system.org/gcube/Social_Networking_Library gCube Social-Networking-Library] and the [https://wiki.gcube-system.org/gcube/UserManagement_Core User Management] components. The former is the bridge between the Cassandra Cluster (on which social data are stored) and the social applications, whereas the latter is the abstraction layer over the enabling technology of the Infrastructure portals, [https://www.liferay.com/ Liferay], in terms of: roles, users and organizations and how they are mapped to gCube concepts (VREs, Teams, and so on).<br />
<br />
The web services exposes a subset of such functionalities over '''https''' in a standard, reliable and secure way. <br />
<br />
See the vailable REST-API at a [https://dev.d4science.org/swagger/social-networking/ glance here.]<br />
= Discover the Web Service =<br />
For gCube releases older than 4.3, a Service Endpoint that has<br />
<br />
<source lang="java"><br />
Name = SocialNetworking<br />
Category = Portal<br />
</source><br />
<br />
mantains the base path of the service. Starting from gCube 4.3 version, the service runs on SmartGears, thus a gCore Endpoint needs to be searched having<br />
<br />
<source lang="java"><br />
Service Name = SocialNetworking<br />
Service Class = Portal<br />
</source><br />
<br />
The base path is the entry point of the web service. In the '''development''' environment, its current value is<br />
<source lang="xml"><br />
https://socialnetworking-d-d4s.d4science.org/social-networking-library-ws/rest<br />
</source><br />
<br />
In the '''production''' environment, its current value is<br />
<source lang="xml"><br />
https://api.d4sciende.org/rest<br />
</source><br />
<br />
This url is referred as ''BASE_URL''.<br />
<br />
= Service versions =<br />
Starting from gCube 4.3, the service exposes its methods using a more standard naming approach. Moreover, they accept (in case of http POST/PUT methods) JSON objects. The result of all methods is always a JSON object of this kind<br />
<br />
<source lang="java"><br />
{"success": false/true,<br />
"message": ...,<br />
"result": ...,<br />
}<br />
</source><br />
<br />
Where<br />
* success reports if the request succeeded or failed;<br />
* message is a status/error message that can be checked in case of errors (success equals false);<br />
* result is the current result object (it can be a list, a single object and so on depending on the invoked method). <br />
<br />
Inputs are automatically validated before the request is served.<br />
<br />
The older version (Version 1) instead has some limitation thus you are kindly suggested to no longer use it. However, in the following, its methods are going to be reported as well. The older version will be removed starting from the next gCube releases.<br />
<br />
= Authorization =<br />
No matter the version number of the service, every call to one of its method require a gcube-token. See [https://wiki.gcube-system.org/gcube/Authorization_Framework here] for more information about this token. This secret token needs to be valid in the current context. It can be put as query parameter of the http request or within the header. Both approaches are valid. If the token is missing, the operation will be rejected and an http forbidden code will be returned.<br />
<br />
= Usage =<br />
The methods of the Web Service can be called by writing your own REST client application or by using (a lot of) already existent rest client plugins. In the following, we will use the RESTClient plugin for Firefox for POST methods. Your browser can be used for GET methods. The return value of each method is an HTTP Response object that has an HTTP CODE (200, 201, 400, 500, ...) and, optionally, a JSON or text object that represents the created/retrieved resource or better specifies the occurred error. In case of errors, please refer to the [[#Troubleshooting|Troubleshooting]] section for more details.<br />
<br />
= Service methods (Version 2) =<br />
The list of sub-services available in the new version is going to be reported next. The name convention is different. For instance, to read user's posts, the method to be called is<br />
<br />
<source lang="java"><br />
BASE_PATH/2/posts/get-posts-user/<br />
</source><br />
<br />
The real result of the method, if the request succeeded, is contained into the 'result' field of the json object<br />
<br />
<source lang="java"><br />
{"success": ..., "message": ..., "result": "[{first post}, {second post}, ...}<br />
</source><br />
<br />
In the following, the list of available sub-services available under the newest version, its methods and the parameters they need in order to be called are reported. Again, you can use your preferred REST client so examples won't be reported.<br />
<br />
== List of Available Sub-Services and methods==<br />
=== Tokens ===<br />
====Generate application token====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/tokens/generate-application-token<br />
</source><br />
<br />
First of all, you need to have a ''user gcube-token'' for authentication purposes. You can get it using the Token Generator portlet available on the VRE you are interesting the app will post in.<br />
Then, to get an ''application gcube-token'' you can invoke the method. It is an HTTP/POST method that accepts as parameter:<br />
<br />
* a json object of this kind {"app_id": "your-app-id-goes-here"}.<br />
<br />
=== User profile and roles ===<br />
<br />
To retrieve user's profile user profile info (and roles) the subservice to call, once the social networking service endpoint has been retrieved, is this one:<br />
<br />
<code><br />
/2/people/profile<br />
</code><br />
<br />
further details: [https://wiki.gcube-system.org/gcube/OAuth2.0#Step_4_-_Get_member_data get member information]<br />
<br />
=== Full text search ===<br />
==== Search by query ====<br />
Social data are indexed into an ElasticSearch engine and are querable using a full-text approach. You can retrieve more information about this index [https://wiki.gcube-system.org/index.php?title=Social_Networking_Data_Discovery|here].<br />
The link to append to '''Base_URL''' is the following<br />
<source lang="java"><br />
2/full-text-search/search-by-query<br />
</source><br />
<br />
The query can be submitted also to the service, by using the following parameters<br />
<br />
* query: the query to perform against the index [mandatory];<br />
* from : the result set should be returned by starting from the n-th result [default is zero, optional];<br />
* quantity: how many enhanced posts need to be returned [default is 10, optional].<br />
<br />
=== Hashtags ===<br />
==== Get hashtags and occurrences in a context====<br />
The link to append to '''Base_URL''' is the following<br />
<source lang="java"><br />
2/hashtags/get-hashtags-and-occurrences<br />
</source><br />
<br />
The result is of the kind<br />
{"success": true,<br />
"message": null,<br />
"result":{{"hashtag1" : x}, {"hashtag2" : y}, ...}<br />
}<br />
=== Posts ===<br />
==== Write application post====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/write-post-app<br />
</source><br />
<br />
By means of the application token requested with the above method, the app can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters in a JSON object<br />
<br />
* text: text of the post; <br />
* params: additional parameters if your application supports the direct opening of of this update's object e.g. id=12345&type=foo [optional];<br />
* preview_title: a title for the preview of the post [optional];<br />
* preview_host: a host for the preview of the post [optional];<br />
* preview_description: a description for the preview of the post [optional];<br />
* preview_url: the preview url [optional];<br />
* image_url: if you want to put an image, pass its url [optional];<br />
* enable_notification: a boolean value to enable notifications [optional, default is false].<br />
<br />
====Write user post====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/write-post-user<br />
</source><br />
<br />
By means of a valid user token, he/she can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters in a JSON object:<br />
<br />
* text: text of the post; <br />
* preview_title: a title for the preview of the post [optional];<br />
* preview_description: a description for the preview of the post [optional];<br />
* preview_host: a host for the preview of the post [optional];<br />
* preview_url: the preview url [optional];<br />
* image_url: if you want to put an image, pass its url [optional];<br />
* enable_notification: a boolean value to enable notifications [optional].<br />
<br />
====Retrieve all application posts==== <br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-app<br />
</source><br />
<br />
In order to retrieve all the posts the application wrote (in every VRE it is allowed to write in) you can invoke this HTTP/GET method.<br />
====Retrieve all user posts====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user<br />
</source><br />
<br />
====Retrieve all user posts since a given date====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user-since<br />
</source><br />
<br />
If you are interested in retrieving user's posts whose timestamps are greater than a given date you can invoke this method.<br />
This HTTP/GET method accepts as parameter:<br />
<br />
* time: the time of the oldest post to retrieve converted in milliseconds.<br />
<br />
====Retrieve all user posts up to a given quantity====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user-quantity<br />
</source><br />
<br />
If you are interested in retrieving user's posts, starting from the newest up to a given quantity you can invoke this method.<br />
This HTTP/GET method accepts as parameter:<br />
<br />
* quantity: a value greater than zero (the default value is 10).<br />
<br />
====Retrieve the identifiers of the posts a user liked====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-id-liked-posts<br />
</source><br />
<br />
By means of this method a user can retrieve the identifiers of the posts he/she liked.<br />
====Retrieve the posts a user liked====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-liked-posts<br />
</source><br />
<br />
By means of this method a user can retrieve the posts he/she liked.<br />
====Retrieve all vre posts====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-vre<br />
</source><br />
<br />
By means of this method a user that has a token valid for a certain VRE, can retrieve all the posts of that VRE.<br />
====Get posts by hashtag====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-by-hashtag<br />
</source><br />
The method requires as parameter:<br />
* hashtag: the hashtag to be contained in the returned posts[mandatory].<br />
<br />
=== Comments ===<br />
====Retrieve comments by user====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/comments/get-comments-user<br />
</source><br />
It is an http GET method.<br />
====Retrieve comments by user and time====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/comments/get-comments-user-by-time<br />
</source><br />
It is an http GET method. It requires:<br />
* time: the time (in milliseconds) since when the comments need to be retrieved [default is zero, optional];<br />
=== Notifications ===<br />
====Retrieve user's notifications in a given range====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/notifications/get-range-notifications<br />
</source><br />
<br />
By means of this method a user can retrieve his/her last notifications in the range [a, b]. The method requires:<br />
<br />
* from: an integer greater than zero;<br />
* quantity: an integer equal or greater than zero.<br />
<br />
The final range is evaluated as [from, from + quantity].<br />
====Notify job status====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/notifications/notify-job-status<br />
</source><br />
<br />
By means of this method a user can be notified about the status of a job in the infrastructure. Jobs are, for instance, SmartExecutor's plugins or DataMiner's algorithms. The method requires the following parameters in a JSON object via HTTP POST<br />
<br />
* recipient: the username of the user to be alerted [mandatory];<br />
* job_id: the identifier of the job (e.g, a uuid) [mandatory];<br />
* job_name: the name of the running job (e.g., Social-data-indexer-plugin)[mandatory];<br />
* service_name: the name of the service that launched the job (e.g., SmartExecutor) [mandatory];<br />
* status: the status reached by the job, that is any among CANCELLED, CANCELLING, DELETED, DELETING, EXECUTING, FAILED, SUCCEEDED, SUBMITTED, TIMED_OUT, WAITING [mandatory];<br />
* status_message: a detailed message about the status [optional].<br />
<br />
=== Messages ===<br />
====Send a message====<br />
It is an HTTP POST method.<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/write-message<br />
</source><br />
<br />
By default, the sender of the message is the owner of the gcube-token. According to the recipients' notifications preferences, they could receive a portal notification, an email notification and so on.<br />
<br />
Parameters, to be put in a JSON object are:<br />
<br />
* subject: the subject of the message to send;<br />
* body: the body of the message;<br />
* recipients: a list of valid comma separated users' ids (e.g., andrea.rossi, mario.rossi) or users' emails (e.g., andrea.rossi@isti.cnr.it, mario.rossi@isti.cnr.it);<br />
<br />
An example of the body of the request is as follows<br />
<source lang="xml"><br />
{"subject": "....", "body": ".....", "recipients":[{"id":"userid1"},{"id":"userid2"}]}<br />
</source><br />
<br />
or, if you want to use valid mails (i.e. recognized in the infrastructure)<br />
<source lang="xml"><br />
{"subject": "....", "body": ".....", "recipients":[{"id":"mail1"},{"id":"mail2"}]}<br />
</source><br />
<br />
====Get received messages====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/get-received-messages<br />
</source><br />
It is an http GET method.<br />
<br />
====Get sent messages====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/get-sent-messages<br />
</source><br />
It is an http GET method.<br />
=== Users ===<br />
====User exists====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/user-exists<br />
</source><br />
<br />
The method requires<br />
<br />
* username: the username of the user which has to be checked.<br />
<br />
'''Note''' The method can be only invoked with a root-context application token.<br />
====Read user's custom fields====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-custom-attribute<br />
</source><br />
<br />
The method requires<br />
<br />
* attribute: that is the name of a valid attribute to be read.<br />
====Get user's email====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-email<br />
</source><br />
<br />
The method returns the email associated to the owner of the security token used.<br />
====Get user's profile====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-profile<br />
</source><br />
<br />
The method returns the profile of the owner of the security token used.<br />
====Get user's fullname====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-fullname<br />
</source><br />
<br />
The method returns the fullname associated to the owner of the security token used.<br />
====Get all users' usernames in VRE====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames<br />
</source><br />
<br />
The method returns the usernames of the context associated to the security token.<br />
<br />
====Get usernames by global role====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames-by-global-role<br />
</source><br />
<br />
The method requires<br />
<br />
* role-name: that is the name of the role to consider.<br />
<br />
'''''This method can be only invoked with an Infrastructure Token.'''''<br />
<br />
<br />
====Get usernames by role====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames-by-role<br />
</source><br />
<br />
The method requires<br />
<br />
* role-name: that is the name of the role to consider.<br />
<br />
=== VREs ===<br />
====Get User VREs====<br />
This method allows a user to retrieve the list of VREs she belongs to<br />
<br />
<source lang="java"><br />
2/vres/get-my-vres<br />
</source><br />
<br />
The method accepts an option parameter:<br />
* getManagers : if set to true, reports also the VRE Managers of the VRE<br />
<br />
== Troubleshooting ==<br />
<br />
There could be many reasons for a failed request. Please always check the HTTP CODE and the message within the response. <br />
<br />
=== HTTP Error Code ===<br />
This table tries to help you:<br />
<br />
{| border="1" cellpadding="20" cellspacing="0"<br />
!HTTP CODE<br />
!Meaning<br />
|-<br />
|400 - BAD_REQUEST<br />
|Please check the parameters you are passing<br />
|-<br />
|403 - FORBIDDEN<br />
|Is the token valid? / Does it belong to an application? [In case of application request]<br />
|-<br />
|404 - NOT_FOUND<br />
|The object(s) you are asking for is(are) not available<br />
|-<br />
|500 - INTERNAL_SERVER_ERROR<br />
|There was an error while serving your request (retry later)<br />
|-<br />
|}<br />
<br />
<br />
=== Always manage HTTP Redirects on POST requests (i.e. 30X codes) ===<br />
The service accepts both incoming http and https client requests, which are automatically redirect to https protocol by the reverse proxy nginx. This redirect mechanism must be taken into account when POST calls are performed, because it is not (always) supported and transparent by clients. In order to be sure, please check if you get a 30X error and extract the redirection urls within the response header. The following is a working java code that extract such information. It uses the apache-client library to perform http calls.<br />
<br />
<source lang="java"><br />
<br />
// build the client and the request<br />
String path = ... ; <br />
HttpClientBuilder clientBuilder = HttpClientBuilder.create();<br />
CloseableHttpClient httpClient = clientBuilder.build();<br />
HttpPost request = new HttpPost(path);<br />
JSONObject obj = new JSONObject();<br />
<br />
... // build the json object<br />
<br />
// set header properties<br />
request.addHeader("gcube-token", ...);<br />
request.addHeader("Content-type", ContentType.APPLICATION_JSON.toString());<br />
StringEntity params = new StringEntity(obj.toJSONString(), ContentType.APPLICATION_JSON);<br />
request.setEntity(params);<br />
HttpResponse response = httpClient.execute(request);<br />
logger.info(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
<br />
int status = response.getStatusLine().getStatusCode();<br />
<br />
// check the response status and look if it was a redirect problem<br />
if (status != HttpURLConnection.HTTP_OK && (status == HttpURLConnection.HTTP_MOVED_TEMP ||<br />
status == HttpURLConnection.HTTP_MOVED_PERM ||<br />
status == HttpURLConnection.HTTP_SEE_OTHER)) {<br />
<br />
// redirect -> fetch new location<br />
Header[] locations = response.getHeaders("Location");<br />
Header lastLocation = locations[locations.length - 1];<br />
String realLocation = lastLocation.getValue();<br />
logger.debug("New location is " + realLocation);<br />
<br />
// perform again the same request<br />
request = new HttpPost(realLocation);<br />
request.addHeader("gcube-token", ...);<br />
request.addHeader("Content-type", ContentType.APPLICATION_JSON.toString());<br />
params = new StringEntity(obj.toJSONString(), ContentType.APPLICATION_JSON);<br />
request.setEntity(params);<br />
response = httpClient.execute(request);<br />
logger.info(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
} else<br />
logger.debug(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
</source><br />
<br />
= Service methods (Version 1) - DEPRECATED =<br />
In the following, the list of available sub-services available under the first version, its methods and the parameters they need in order to be called are reported. The gcube-token is always required so it won't be reported as parameter. <br />
<br />
'''This version has been replaced by version 2, please do not use this version and update your code asap.'''<br />
<br />
== List of available Sub-Services and methods ==<br />
=== Tokens ===<br />
==== Generate application token ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
tokens/generateApplicationToken<br />
</source><br />
<br />
First of all, you need to have a ''user gcube-token'' for authentication purposes. You can get it using the Token Generator portlet available on the VRE you are interesting the app will post in.<br />
Now, to get an ''application gcube-token'' you can invoke the method. It is an HTTP/POST method that accepts as parameters:<br />
<br />
* appid: the identifier of the application as reported in the Application Profile.<br />
<br />
For example, using the RESTClient plugin for Firefox you will have something like this for requiring a new application token:<br />
<br />
[[File:GenerateApplicationToken.png]]<br />
<br />
Push on the SEND button and if all it's ok, your application gcube-token will be generated and returned as ''plain text'' in the response within the HTTP response having code 201.<br />
=== Posts ['''DEPRECATED'''] ===<br />
==== Write application post ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/writePostApplication<br />
</source><br />
<br />
By means of the application token requested with the above method, the app can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters:<br />
<br />
* text: text of the post; <br />
* params: additional parameters if your application supports the direct opening of of this update's object e.g. id=12345&type=foo [optional];<br />
* previewtitle: a title for the preview of the post [optional];<br />
* previewdescription: a description for the preview of the post [optional];<br />
* httpimageurl: if you want to put an image, pass its url [optional];<br />
* enablenotification: a boolean value to enable notifications [optional].<br />
<br />
For example:<br />
<br />
[[File:WriteApplicationPost.png]]<br />
<br />
Push on SEND and check the code associated to the returned response (201 in case of success).<br />
<br />
==== Retrieve all application posts ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByApp<br />
</source><br />
<br />
In order to retrieve all the posts the application wrote (in every VRE it is allowed to write in) you can invoke this HTTP/GET method.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByApp?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByUser<br />
</source><br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts since a given date ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getRecentPostsByUserAndDate<br />
</source><br />
<br />
If you are interested in retrieving user's posts whose timestamps are greater than a given date you can invoke this method.<br />
This HTTP/GET method accepts this parameter:<br />
<br />
* time: the time of the oldest post to retrieve converted in milliseconds.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getRecentPostsByUserAndDate?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&time=xxxxx<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts up to a given quantity ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getRecentPostsByUser<br />
</source><br />
<br />
If you are interested in retrieving user's posts, starting from the newest up to a given quantity you can invoke this method.<br />
This HTTP/GET method accepts two parameter:<br />
<br />
* quantity: a value greater than zero (the default value is 10).<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getRecentPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&quantity=20<br />
</source><br />
<br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Write user post ['''DEPRECATED'''] ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/writePostUser<br />
</source><br />
<br />
By means of a valid user token, he/she can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters:<br />
<br />
* text: text of the post; <br />
* previewtitle: a title for the preview of the post [optional];<br />
* previewdescription: a description for the preview of the post [optional];<br />
* previewHost: [optional];<br />
* previewUrl: [optional];<br />
* httpimageurl: if you want to put an image, pass its url [optional];<br />
* enablenotification: a boolean value to enable notifications [optional];<br />
<br />
For example:<br />
<br />
[[File:Userpost.png]]<br />
<br />
Push on SEND and check the code associated to the returned response (201 in case of success).<br />
<br />
==== Retrieve the identifiers of the posts a user liked ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllLikedPostIdsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve the identifiers of the posts he/she liked.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllLikedPostIdsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the identifiers of the posts are returned in JSON format.<br />
<br />
==== Retrieve the posts a user liked ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllLikedPostsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve the posts he/she liked. <br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllLikedPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
====Retrieve all vre posts====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByVRE<br />
</source><br />
<br />
By means of this method a user that has a token valid for a certain VRE, can retrieve all the posts of that VRE.<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByVRE?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
=== Notifications ===<br />
====Retrieve user's notifications in a given range====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
notifications/getRangeNotificationsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve his/her last notifications in the range [a, b]. The method requires:<br />
<br />
* from: an integer greater than zero;<br />
* quantity: an integer equal or greater than zero.<br />
<br />
The final range is evaluated as [from, from + quantity]<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/notifications/getRangeNotificationsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&from=from&quantity=quantity<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
=== Messages ===<br />
<br />
====Send a message====<br />
You can send a message via the web service by using the following POST method:<br />
<br />
* writeMessageToUsers<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
messages/writeMessageToUsers<br />
</source><br />
<br />
By default, the sender of the message is the owner of the gcube-token. If you need to specify a different "sender", you can do that by specifying the parameter sender. Furthermore, according to the recipients' notifications preferences, they could receive a portal notification, a mail notification and so on.<br />
<br />
Parameters are:<br />
<br />
* subject: the subject of the message to send;<br />
* body: the body of the message;<br />
* recipients: a list of valid comma separated users' ids (e.g., andrea.rossi, mario.rossi).<br />
* sender: a valid sender's id [optional]<br />
<br />
An example usage of the method is reported in the picture below (without specifying the sender):<br />
<br />
[[File:Send_message_example.png]]<br />
=== Users ===<br />
====Read user's custom fields====<br />
You can read user's custom attributes using the following method<br />
<br />
*readCustomAttr<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
users/readCustomAttr<br />
</source><br />
<br />
The ''read'' method requires<br />
<br />
* attribute: that is the name of a valid attribute to be read.<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/users/readCustomAttr?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&attribute=attributeX<br />
</source><br />
<br />
= See Also =<br />
<br />
== D4Science Developers website ==<br />
[https://dev.d4science.org D4Science Developers website]<br />
<br />
== SWAGGER documentation==<br />
[https://dev.d4science.org/swagger/social-networking/ SWAGGER documentation of the Social-Networking RESTful interface]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=Social_Networking_Service&diff=33324Social Networking Service2020-04-15T16:23:20Z<p>Massimiliano.assante: /* Posts DEPRECATED */</p>
<hr />
<div>{| align="right"<br />
| __TOC__<br />
|}<br />
<br />
The Social Networking Service is a RESTful application which exposes operations coming from both the [https://wiki.gcube-system.org/gcube/Social_Networking_Library gCube Social-Networking-Library] and the [https://wiki.gcube-system.org/gcube/UserManagement_Core User Management] components. The former is the bridge between the Cassandra Cluster (on which social data are stored) and the social applications, whereas the latter is the abstraction layer over the enabling technology of the Infrastructure portals, [https://www.liferay.com/ Liferay], in terms of: roles, users and organizations and how they are mapped to gCube concepts (VREs, Teams, and so on).<br />
<br />
The web services exposes a subset of such functionalities over '''https''' in a standard, reliable and secure way. <br />
<br />
See the vailable REST-API at a [https://dev.d4science.org/swagger/social-networking/ glance here.]<br />
= Discover the Web Service =<br />
For gCube releases older than 4.3, a Service Endpoint that has<br />
<br />
<source lang="java"><br />
Name = SocialNetworking<br />
Category = Portal<br />
</source><br />
<br />
mantains the base path of the service. Starting from gCube 4.3 version, the service runs on SmartGears, thus a gCore Endpoint needs to be searched having<br />
<br />
<source lang="java"><br />
Service Name = SocialNetworking<br />
Service Class = Portal<br />
</source><br />
<br />
The base path is the entry point of the web service. In the '''development''' environment, its current value is<br />
<source lang="xml"><br />
https://socialnetworking-d-d4s.d4science.org/social-networking-library-ws/rest<br />
</source><br />
<br />
In the '''production''' environment, its current value is<br />
<source lang="xml"><br />
https://api.d4sciende.org/rest<br />
</source><br />
<br />
This url is referred as ''BASE_URL''.<br />
<br />
= Service versions =<br />
Starting from gCube 4.3, the service exposes its methods using a more standard naming approach. Moreover, they accept (in case of http POST/PUT methods) JSON objects. The result of all methods is always a JSON object of this kind<br />
<br />
<source lang="java"><br />
{"success": false/true,<br />
"message": ...,<br />
"result": ...,<br />
}<br />
</source><br />
<br />
Where<br />
* success reports if the request succeeded or failed;<br />
* message is a status/error message that can be checked in case of errors (success equals false);<br />
* result is the current result object (it can be a list, a single object and so on depending on the invoked method). <br />
<br />
Inputs are automatically validated before the request is served.<br />
<br />
The older version (Version 1) instead has some limitation thus you are kindly suggested to no longer use it. However, in the following, its methods are going to be reported as well. The older version will be removed starting from the next gCube releases.<br />
<br />
= Authorization =<br />
No matter the version number of the service, every call to one of its method require a gcube-token. See [https://wiki.gcube-system.org/gcube/Authorization_Framework here] for more information about this token. This secret token needs to be valid in the current context. It can be put as query parameter of the http request or within the header. Both approaches are valid. If the token is missing, the operation will be rejected and an http forbidden code will be returned.<br />
<br />
= Usage =<br />
The methods of the Web Service can be called by writing your own REST client application or by using (a lot of) already existent rest client plugins. In the following, we will use the RESTClient plugin for Firefox for POST methods. Your browser can be used for GET methods. The return value of each method is an HTTP Response object that has an HTTP CODE (200, 201, 400, 500, ...) and, optionally, a JSON or text object that represents the created/retrieved resource or better specifies the occurred error. In case of errors, please refer to the [[#Troubleshooting|Troubleshooting]] section for more details.<br />
<br />
= Service methods (Version 2) =<br />
The list of sub-services available in the new version is going to be reported next. The name convention is different. For instance, to read user's posts, the method to be called is<br />
<br />
<source lang="java"><br />
BASE_PATH/2/posts/get-posts-user/<br />
</source><br />
<br />
The real result of the method, if the request succeeded, is contained into the 'result' field of the json object<br />
<br />
<source lang="java"><br />
{"success": ..., "message": ..., "result": "[{first post}, {second post}, ...}<br />
</source><br />
<br />
In the following, the list of available sub-services available under the newest version, its methods and the parameters they need in order to be called are reported. Again, you can use your preferred REST client so examples won't be reported.<br />
<br />
== List of Available Sub-Services and methods==<br />
=== Tokens ===<br />
====Generate application token====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/tokens/generate-application-token<br />
</source><br />
<br />
First of all, you need to have a ''user gcube-token'' for authentication purposes. You can get it using the Token Generator portlet available on the VRE you are interesting the app will post in.<br />
Then, to get an ''application gcube-token'' you can invoke the method. It is an HTTP/POST method that accepts as parameter:<br />
<br />
* a json object of this kind {"app_id": "your-app-id-goes-here"}.<br />
<br />
=== User profile and roles ===<br />
<br />
To retrieve user's profile user profile info (and roles) the subservice to call, once the social networking service endpoint has been retrieved, is this one:<br />
<br />
<code><br />
/2/people/profile<br />
</code><br />
<br />
further details: [https://wiki.gcube-system.org/gcube/OAuth2.0#Step_4_-_Get_member_data get member information]<br />
<br />
=== Full text search ===<br />
==== Search by query ====<br />
Social data are indexed into an ElasticSearch engine and are querable using a full-text approach. You can retrieve more information about this index [https://wiki.gcube-system.org/index.php?title=Social_Networking_Data_Discovery|here].<br />
The link to append to '''Base_URL''' is the following<br />
<source lang="java"><br />
2/full-text-search/search-by-query<br />
</source><br />
<br />
The query can be submitted also to the service, by using the following parameters<br />
<br />
* query: the query to perform against the index [mandatory];<br />
* from : the result set should be returned by starting from the n-th result [default is zero, optional];<br />
* quantity: how many enhanced posts need to be returned [default is 10, optional].<br />
<br />
=== Hashtags ===<br />
==== Get hashtags and occurrences in a context====<br />
The link to append to '''Base_URL''' is the following<br />
<source lang="java"><br />
2/hashtags/get-hashtags-and-occurrences<br />
</source><br />
<br />
The result is of the kind<br />
{"success": true,<br />
"message": null,<br />
"result":{{"hashtag1" : x}, {"hashtag2" : y}, ...}<br />
}<br />
=== Posts ===<br />
==== Write application post====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/write-post-app<br />
</source><br />
<br />
By means of the application token requested with the above method, the app can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters in a JSON object<br />
<br />
* text: text of the post; <br />
* params: additional parameters if your application supports the direct opening of of this update's object e.g. id=12345&type=foo [optional];<br />
* preview_title: a title for the preview of the post [optional];<br />
* preview_host: a host for the preview of the post [optional];<br />
* preview_description: a description for the preview of the post [optional];<br />
* preview_url: the preview url [optional];<br />
* image_url: if you want to put an image, pass its url [optional];<br />
* enable_notification: a boolean value to enable notifications [optional, default is false].<br />
<br />
====Write user post====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/write-post-user<br />
</source><br />
<br />
By means of a valid user token, he/she can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters in a JSON object:<br />
<br />
* text: text of the post; <br />
* preview_title: a title for the preview of the post [optional];<br />
* preview_description: a description for the preview of the post [optional];<br />
* preview_host: a host for the preview of the post [optional];<br />
* preview_url: the preview url [optional];<br />
* image_url: if you want to put an image, pass its url [optional];<br />
* enable_notification: a boolean value to enable notifications [optional].<br />
<br />
====Retrieve all application posts==== <br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-app<br />
</source><br />
<br />
In order to retrieve all the posts the application wrote (in every VRE it is allowed to write in) you can invoke this HTTP/GET method.<br />
====Retrieve all user posts====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user<br />
</source><br />
<br />
====Retrieve all user posts since a given date====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user-since<br />
</source><br />
<br />
If you are interested in retrieving user's posts whose timestamps are greater than a given date you can invoke this method.<br />
This HTTP/GET method accepts as parameter:<br />
<br />
* time: the time of the oldest post to retrieve converted in milliseconds.<br />
<br />
====Retrieve all user posts up to a given quantity====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user-quantity<br />
</source><br />
<br />
If you are interested in retrieving user's posts, starting from the newest up to a given quantity you can invoke this method.<br />
This HTTP/GET method accepts as parameter:<br />
<br />
* quantity: a value greater than zero (the default value is 10).<br />
<br />
====Retrieve the identifiers of the posts a user liked====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-id-liked-posts<br />
</source><br />
<br />
By means of this method a user can retrieve the identifiers of the posts he/she liked.<br />
====Retrieve the posts a user liked====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-liked-posts<br />
</source><br />
<br />
By means of this method a user can retrieve the posts he/she liked.<br />
====Retrieve all vre posts====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-vre<br />
</source><br />
<br />
By means of this method a user that has a token valid for a certain VRE, can retrieve all the posts of that VRE.<br />
====Get posts by hashtag====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-by-hashtag<br />
</source><br />
The method requires as parameter:<br />
* hashtag: the hashtag to be contained in the returned posts[mandatory].<br />
<br />
=== Comments ===<br />
====Retrieve comments by user====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/comments/get-comments-user<br />
</source><br />
It is an http GET method.<br />
====Retrieve comments by user and time====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/comments/get-comments-user-by-time<br />
</source><br />
It is an http GET method. It requires:<br />
* time: the time (in milliseconds) since when the comments need to be retrieved [default is zero, optional];<br />
=== Notifications ===<br />
====Retrieve user's notifications in a given range====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/notifications/get-range-notifications<br />
</source><br />
<br />
By means of this method a user can retrieve his/her last notifications in the range [a, b]. The method requires:<br />
<br />
* from: an integer greater than zero;<br />
* quantity: an integer equal or greater than zero.<br />
<br />
The final range is evaluated as [from, from + quantity].<br />
====Notify job status====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/notifications/notify-job-status<br />
</source><br />
<br />
By means of this method a user can be notified about the status of a job in the infrastructure. Jobs are, for instance, SmartExecutor's plugins or DataMiner's algorithms. The method requires the following parameters in a JSON object via HTTP POST<br />
<br />
* recipient: the username of the user to be alerted [mandatory];<br />
* job_id: the identifier of the job (e.g, a uuid) [mandatory];<br />
* job_name: the name of the running job (e.g., Social-data-indexer-plugin)[mandatory];<br />
* service_name: the name of the service that launched the job (e.g., SmartExecutor) [mandatory];<br />
* status: the status reached by the job, that is any among CANCELLED, CANCELLING, DELETED, DELETING, EXECUTING, FAILED, SUCCEEDED, SUBMITTED, TIMED_OUT, WAITING [mandatory];<br />
* status_message: a detailed message about the status [optional].<br />
<br />
=== Messages ===<br />
====Send a message====<br />
It is an HTTP POST method.<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/write-message<br />
</source><br />
<br />
By default, the sender of the message is the owner of the gcube-token. According to the recipients' notifications preferences, they could receive a portal notification, an email notification and so on.<br />
<br />
Parameters, to be put in a JSON object are:<br />
<br />
* subject: the subject of the message to send;<br />
* body: the body of the message;<br />
* recipients: a list of valid comma separated users' ids (e.g., andrea.rossi, mario.rossi) or users' emails (e.g., andrea.rossi@isti.cnr.it, mario.rossi@isti.cnr.it);<br />
<br />
An example of the body of the request is as follows<br />
<source lang="xml"><br />
{"subject": "....", "body": ".....", "recipients":[{"id":"userid1"},{"id":"userid2"}]}<br />
</source><br />
<br />
or, if you want to use valid mails (i.e. recognized in the infrastructure)<br />
<source lang="xml"><br />
{"subject": "....", "body": ".....", "recipients":[{"id":"mail1"},{"id":"mail2"}]}<br />
</source><br />
<br />
====Get received messages====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/get-received-messages<br />
</source><br />
It is an http GET method.<br />
<br />
====Get sent messages====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/get-sent-messages<br />
</source><br />
It is an http GET method.<br />
=== Users ===<br />
====User exists====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/user-exists<br />
</source><br />
<br />
The method requires<br />
<br />
* username: the username of the user which has to be checked.<br />
<br />
'''Note''' The method can be only invoked with a root-context application token.<br />
====Read user's custom fields====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-custom-attribute<br />
</source><br />
<br />
The method requires<br />
<br />
* attribute: that is the name of a valid attribute to be read.<br />
====Get user's email====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-email<br />
</source><br />
<br />
The method returns the email associated to the owner of the security token used.<br />
====Get user's profile====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-profile<br />
</source><br />
<br />
The method returns the profile of the owner of the security token used.<br />
====Get user's fullname====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-fullname<br />
</source><br />
<br />
The method returns the fullname associated to the owner of the security token used.<br />
====Get all users' usernames in VRE====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames<br />
</source><br />
<br />
The method returns the usernames of the context associated to the security token.<br />
<br />
====Get usernames by global role====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames-by-global-role<br />
</source><br />
<br />
The method requires<br />
<br />
* role-name: that is the name of the role to consider.<br />
<br />
'''''This method can be only invoked with an Infrastructure Token.'''''<br />
<br />
<br />
====Get usernames by role====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames-by-role<br />
</source><br />
<br />
The method requires<br />
<br />
* role-name: that is the name of the role to consider.<br />
<br />
=== VREs ===<br />
====Get User VREs====<br />
This method allows a user to retrieve the list of VREs she belongs to<br />
<br />
<source lang="java"><br />
2/vres/get-my-vres<br />
</source><br />
<br />
The method accepts an option parameter:<br />
* getManagers : if set to true, reports also the VRE Managers of the VRE<br />
<br />
== Troubleshooting ==<br />
<br />
There could be many reasons for a failed request. Please always check the HTTP CODE and the message within the response. <br />
<br />
=== HTTP Error Code ===<br />
This table tries to help you:<br />
<br />
{| border="1" cellpadding="20" cellspacing="0"<br />
!HTTP CODE<br />
!Meaning<br />
|-<br />
|400 - BAD_REQUEST<br />
|Please check the parameters you are passing<br />
|-<br />
|403 - FORBIDDEN<br />
|Is the token valid? / Does it belong to an application? [In case of application request]<br />
|-<br />
|404 - NOT_FOUND<br />
|The object(s) you are asking for is(are) not available<br />
|-<br />
|500 - INTERNAL_SERVER_ERROR<br />
|There was an error while serving your request (retry later)<br />
|-<br />
|}<br />
<br />
<br />
=== Always manage HTTP Redirects on POST requests (i.e. 30X codes) ===<br />
The service accepts both incoming http and https client requests, which are automatically redirect to https protocol by the reverse proxy nginx. This redirect mechanism must be taken into account when POST calls are performed, because it is not (always) supported and transparent by clients. In order to be sure, please check if you get a 30X error and extract the redirection urls within the response header. The following is a working java code that extract such information. It uses the apache-client library to perform http calls.<br />
<br />
<source lang="java"><br />
<br />
// build the client and the request<br />
String path = ... ; <br />
HttpClientBuilder clientBuilder = HttpClientBuilder.create();<br />
CloseableHttpClient httpClient = clientBuilder.build();<br />
HttpPost request = new HttpPost(path);<br />
JSONObject obj = new JSONObject();<br />
<br />
... // build the json object<br />
<br />
// set header properties<br />
request.addHeader("gcube-token", ...);<br />
request.addHeader("Content-type", ContentType.APPLICATION_JSON.toString());<br />
StringEntity params = new StringEntity(obj.toJSONString(), ContentType.APPLICATION_JSON);<br />
request.setEntity(params);<br />
HttpResponse response = httpClient.execute(request);<br />
logger.info(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
<br />
int status = response.getStatusLine().getStatusCode();<br />
<br />
// check the response status and look if it was a redirect problem<br />
if (status != HttpURLConnection.HTTP_OK && (status == HttpURLConnection.HTTP_MOVED_TEMP ||<br />
status == HttpURLConnection.HTTP_MOVED_PERM ||<br />
status == HttpURLConnection.HTTP_SEE_OTHER)) {<br />
<br />
// redirect -> fetch new location<br />
Header[] locations = response.getHeaders("Location");<br />
Header lastLocation = locations[locations.length - 1];<br />
String realLocation = lastLocation.getValue();<br />
logger.debug("New location is " + realLocation);<br />
<br />
// perform again the same request<br />
request = new HttpPost(realLocation);<br />
request.addHeader("gcube-token", ...);<br />
request.addHeader("Content-type", ContentType.APPLICATION_JSON.toString());<br />
params = new StringEntity(obj.toJSONString(), ContentType.APPLICATION_JSON);<br />
request.setEntity(params);<br />
response = httpClient.execute(request);<br />
logger.info(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
} else<br />
logger.debug(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
</source><br />
<br />
= Service methods (Version 1) - DEPRECATED =<br />
In the following, the list of available sub-services available under the first version, its methods and the parameters they need in order to be called are reported. The gcube-token is always required so it won't be reported as parameter. <br />
<br />
'''This version has been replaced by version 2, please do not use this version and update your code asap.'''<br />
<br />
== List of available Sub-Services and methods ==<br />
=== Tokens ===<br />
==== Generate application token ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
tokens/generateApplicationToken<br />
</source><br />
<br />
First of all, you need to have a ''user gcube-token'' for authentication purposes. You can get it using the Token Generator portlet available on the VRE you are interesting the app will post in.<br />
Now, to get an ''application gcube-token'' you can invoke the method. It is an HTTP/POST method that accepts as parameters:<br />
<br />
* appid: the identifier of the application as reported in the Application Profile.<br />
<br />
For example, using the RESTClient plugin for Firefox you will have something like this for requiring a new application token:<br />
<br />
[[File:GenerateApplicationToken.png]]<br />
<br />
Push on the SEND button and if all it's ok, your application gcube-token will be generated and returned as ''plain text'' in the response within the HTTP response having code 201.<br />
=== Posts ['''DEPRECATED'''] ===<br />
==== Write application post ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/writePostApplication<br />
</source><br />
<br />
By means of the application token requested with the above method, the app can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters:<br />
<br />
* text: text of the post; <br />
* params: additional parameters if your application supports the direct opening of of this update's object e.g. id=12345&type=foo [optional];<br />
* previewtitle: a title for the preview of the post [optional];<br />
* previewdescription: a description for the preview of the post [optional];<br />
* httpimageurl: if you want to put an image, pass its url [optional];<br />
* enablenotification: a boolean value to enable notifications [optional].<br />
<br />
For example:<br />
<br />
[[File:WriteApplicationPost.png]]<br />
<br />
Push on SEND and check the code associated to the returned response (201 in case of success).<br />
<br />
==== Retrieve all application posts ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByApp<br />
</source><br />
<br />
In order to retrieve all the posts the application wrote (in every VRE it is allowed to write in) you can invoke this HTTP/GET method.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByApp?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByUser<br />
</source><br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts since a given date ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getRecentPostsByUserAndDate<br />
</source><br />
<br />
If you are interested in retrieving user's posts whose timestamps are greater than a given date you can invoke this method.<br />
This HTTP/GET method accepts this parameter:<br />
<br />
* time: the time of the oldest post to retrieve converted in milliseconds.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getRecentPostsByUserAndDate?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&time=xxxxx<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts up to a given quantity ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getRecentPostsByUser<br />
</source><br />
<br />
If you are interested in retrieving user's posts, starting from the newest up to a given quantity you can invoke this method.<br />
This HTTP/GET method accepts two parameter:<br />
<br />
* quantity: a value greater than zero (the default value is 10).<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getRecentPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&quantity=20<br />
</source><br />
<br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Write user post ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/writePostUser<br />
</source><br />
<br />
By means of a valid user token, he/she can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters:<br />
<br />
* text: text of the post; <br />
* previewtitle: a title for the preview of the post [optional];<br />
* previewdescription: a description for the preview of the post [optional];<br />
* previewHost: [optional];<br />
* previewUrl: [optional];<br />
* httpimageurl: if you want to put an image, pass its url [optional];<br />
* enablenotification: a boolean value to enable notifications [optional];<br />
<br />
For example:<br />
<br />
[[File:Userpost.png]]<br />
<br />
Push on SEND and check the code associated to the returned response (201 in case of success).<br />
<br />
==== Retrieve the identifiers of the posts a user liked ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllLikedPostIdsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve the identifiers of the posts he/she liked.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllLikedPostIdsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the identifiers of the posts are returned in JSON format.<br />
<br />
==== Retrieve the posts a user liked ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllLikedPostsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve the posts he/she liked. <br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllLikedPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
====Retrieve all vre posts====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByVRE<br />
</source><br />
<br />
By means of this method a user that has a token valid for a certain VRE, can retrieve all the posts of that VRE.<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByVRE?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
=== Notifications ===<br />
====Retrieve user's notifications in a given range====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
notifications/getRangeNotificationsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve his/her last notifications in the range [a, b]. The method requires:<br />
<br />
* from: an integer greater than zero;<br />
* quantity: an integer equal or greater than zero.<br />
<br />
The final range is evaluated as [from, from + quantity]<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/notifications/getRangeNotificationsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&from=from&quantity=quantity<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
=== Messages ===<br />
<br />
====Send a message====<br />
You can send a message via the web service by using the following POST method:<br />
<br />
* writeMessageToUsers<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
messages/writeMessageToUsers<br />
</source><br />
<br />
By default, the sender of the message is the owner of the gcube-token. If you need to specify a different "sender", you can do that by specifying the parameter sender. Furthermore, according to the recipients' notifications preferences, they could receive a portal notification, a mail notification and so on.<br />
<br />
Parameters are:<br />
<br />
* subject: the subject of the message to send;<br />
* body: the body of the message;<br />
* recipients: a list of valid comma separated users' ids (e.g., andrea.rossi, mario.rossi).<br />
* sender: a valid sender's id [optional]<br />
<br />
An example usage of the method is reported in the picture below (without specifying the sender):<br />
<br />
[[File:Send_message_example.png]]<br />
=== Users ===<br />
====Read user's custom fields====<br />
You can read user's custom attributes using the following method<br />
<br />
*readCustomAttr<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
users/readCustomAttr<br />
</source><br />
<br />
The ''read'' method requires<br />
<br />
* attribute: that is the name of a valid attribute to be read.<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/users/readCustomAttr?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&attribute=attributeX<br />
</source><br />
<br />
= See Also =<br />
<br />
== D4Science Developers website ==<br />
[https://dev.d4science.org D4Science Developers website]<br />
<br />
== SWAGGER documentation==<br />
[https://dev.d4science.org/swagger/social-networking/ SWAGGER documentation of the Social-Networking RESTful interface]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=Social_Networking_Service&diff=33323Social Networking Service2020-04-15T16:22:53Z<p>Massimiliano.assante: /* Posts */</p>
<hr />
<div>{| align="right"<br />
| __TOC__<br />
|}<br />
<br />
The Social Networking Service is a RESTful application which exposes operations coming from both the [https://wiki.gcube-system.org/gcube/Social_Networking_Library gCube Social-Networking-Library] and the [https://wiki.gcube-system.org/gcube/UserManagement_Core User Management] components. The former is the bridge between the Cassandra Cluster (on which social data are stored) and the social applications, whereas the latter is the abstraction layer over the enabling technology of the Infrastructure portals, [https://www.liferay.com/ Liferay], in terms of: roles, users and organizations and how they are mapped to gCube concepts (VREs, Teams, and so on).<br />
<br />
The web services exposes a subset of such functionalities over '''https''' in a standard, reliable and secure way. <br />
<br />
See the vailable REST-API at a [https://dev.d4science.org/swagger/social-networking/ glance here.]<br />
= Discover the Web Service =<br />
For gCube releases older than 4.3, a Service Endpoint that has<br />
<br />
<source lang="java"><br />
Name = SocialNetworking<br />
Category = Portal<br />
</source><br />
<br />
mantains the base path of the service. Starting from gCube 4.3 version, the service runs on SmartGears, thus a gCore Endpoint needs to be searched having<br />
<br />
<source lang="java"><br />
Service Name = SocialNetworking<br />
Service Class = Portal<br />
</source><br />
<br />
The base path is the entry point of the web service. In the '''development''' environment, its current value is<br />
<source lang="xml"><br />
https://socialnetworking-d-d4s.d4science.org/social-networking-library-ws/rest<br />
</source><br />
<br />
In the '''production''' environment, its current value is<br />
<source lang="xml"><br />
https://api.d4sciende.org/rest<br />
</source><br />
<br />
This url is referred as ''BASE_URL''.<br />
<br />
= Service versions =<br />
Starting from gCube 4.3, the service exposes its methods using a more standard naming approach. Moreover, they accept (in case of http POST/PUT methods) JSON objects. The result of all methods is always a JSON object of this kind<br />
<br />
<source lang="java"><br />
{"success": false/true,<br />
"message": ...,<br />
"result": ...,<br />
}<br />
</source><br />
<br />
Where<br />
* success reports if the request succeeded or failed;<br />
* message is a status/error message that can be checked in case of errors (success equals false);<br />
* result is the current result object (it can be a list, a single object and so on depending on the invoked method). <br />
<br />
Inputs are automatically validated before the request is served.<br />
<br />
The older version (Version 1) instead has some limitation thus you are kindly suggested to no longer use it. However, in the following, its methods are going to be reported as well. The older version will be removed starting from the next gCube releases.<br />
<br />
= Authorization =<br />
No matter the version number of the service, every call to one of its method require a gcube-token. See [https://wiki.gcube-system.org/gcube/Authorization_Framework here] for more information about this token. This secret token needs to be valid in the current context. It can be put as query parameter of the http request or within the header. Both approaches are valid. If the token is missing, the operation will be rejected and an http forbidden code will be returned.<br />
<br />
= Usage =<br />
The methods of the Web Service can be called by writing your own REST client application or by using (a lot of) already existent rest client plugins. In the following, we will use the RESTClient plugin for Firefox for POST methods. Your browser can be used for GET methods. The return value of each method is an HTTP Response object that has an HTTP CODE (200, 201, 400, 500, ...) and, optionally, a JSON or text object that represents the created/retrieved resource or better specifies the occurred error. In case of errors, please refer to the [[#Troubleshooting|Troubleshooting]] section for more details.<br />
<br />
= Service methods (Version 2) =<br />
The list of sub-services available in the new version is going to be reported next. The name convention is different. For instance, to read user's posts, the method to be called is<br />
<br />
<source lang="java"><br />
BASE_PATH/2/posts/get-posts-user/<br />
</source><br />
<br />
The real result of the method, if the request succeeded, is contained into the 'result' field of the json object<br />
<br />
<source lang="java"><br />
{"success": ..., "message": ..., "result": "[{first post}, {second post}, ...}<br />
</source><br />
<br />
In the following, the list of available sub-services available under the newest version, its methods and the parameters they need in order to be called are reported. Again, you can use your preferred REST client so examples won't be reported.<br />
<br />
== List of Available Sub-Services and methods==<br />
=== Tokens ===<br />
====Generate application token====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/tokens/generate-application-token<br />
</source><br />
<br />
First of all, you need to have a ''user gcube-token'' for authentication purposes. You can get it using the Token Generator portlet available on the VRE you are interesting the app will post in.<br />
Then, to get an ''application gcube-token'' you can invoke the method. It is an HTTP/POST method that accepts as parameter:<br />
<br />
* a json object of this kind {"app_id": "your-app-id-goes-here"}.<br />
<br />
=== User profile and roles ===<br />
<br />
To retrieve user's profile user profile info (and roles) the subservice to call, once the social networking service endpoint has been retrieved, is this one:<br />
<br />
<code><br />
/2/people/profile<br />
</code><br />
<br />
further details: [https://wiki.gcube-system.org/gcube/OAuth2.0#Step_4_-_Get_member_data get member information]<br />
<br />
=== Full text search ===<br />
==== Search by query ====<br />
Social data are indexed into an ElasticSearch engine and are querable using a full-text approach. You can retrieve more information about this index [https://wiki.gcube-system.org/index.php?title=Social_Networking_Data_Discovery|here].<br />
The link to append to '''Base_URL''' is the following<br />
<source lang="java"><br />
2/full-text-search/search-by-query<br />
</source><br />
<br />
The query can be submitted also to the service, by using the following parameters<br />
<br />
* query: the query to perform against the index [mandatory];<br />
* from : the result set should be returned by starting from the n-th result [default is zero, optional];<br />
* quantity: how many enhanced posts need to be returned [default is 10, optional].<br />
<br />
=== Hashtags ===<br />
==== Get hashtags and occurrences in a context====<br />
The link to append to '''Base_URL''' is the following<br />
<source lang="java"><br />
2/hashtags/get-hashtags-and-occurrences<br />
</source><br />
<br />
The result is of the kind<br />
{"success": true,<br />
"message": null,<br />
"result":{{"hashtag1" : x}, {"hashtag2" : y}, ...}<br />
}<br />
=== Posts ===<br />
==== Write application post====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/write-post-app<br />
</source><br />
<br />
By means of the application token requested with the above method, the app can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters in a JSON object<br />
<br />
* text: text of the post; <br />
* params: additional parameters if your application supports the direct opening of of this update's object e.g. id=12345&type=foo [optional];<br />
* preview_title: a title for the preview of the post [optional];<br />
* preview_host: a host for the preview of the post [optional];<br />
* preview_description: a description for the preview of the post [optional];<br />
* preview_url: the preview url [optional];<br />
* image_url: if you want to put an image, pass its url [optional];<br />
* enable_notification: a boolean value to enable notifications [optional, default is false].<br />
<br />
====Write user post====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/write-post-user<br />
</source><br />
<br />
By means of a valid user token, he/she can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters in a JSON object:<br />
<br />
* text: text of the post; <br />
* preview_title: a title for the preview of the post [optional];<br />
* preview_description: a description for the preview of the post [optional];<br />
* preview_host: a host for the preview of the post [optional];<br />
* preview_url: the preview url [optional];<br />
* image_url: if you want to put an image, pass its url [optional];<br />
* enable_notification: a boolean value to enable notifications [optional].<br />
<br />
====Retrieve all application posts==== <br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-app<br />
</source><br />
<br />
In order to retrieve all the posts the application wrote (in every VRE it is allowed to write in) you can invoke this HTTP/GET method.<br />
====Retrieve all user posts====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user<br />
</source><br />
<br />
====Retrieve all user posts since a given date====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user-since<br />
</source><br />
<br />
If you are interested in retrieving user's posts whose timestamps are greater than a given date you can invoke this method.<br />
This HTTP/GET method accepts as parameter:<br />
<br />
* time: the time of the oldest post to retrieve converted in milliseconds.<br />
<br />
====Retrieve all user posts up to a given quantity====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user-quantity<br />
</source><br />
<br />
If you are interested in retrieving user's posts, starting from the newest up to a given quantity you can invoke this method.<br />
This HTTP/GET method accepts as parameter:<br />
<br />
* quantity: a value greater than zero (the default value is 10).<br />
<br />
====Retrieve the identifiers of the posts a user liked====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-id-liked-posts<br />
</source><br />
<br />
By means of this method a user can retrieve the identifiers of the posts he/she liked.<br />
====Retrieve the posts a user liked====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-liked-posts<br />
</source><br />
<br />
By means of this method a user can retrieve the posts he/she liked.<br />
====Retrieve all vre posts====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-vre<br />
</source><br />
<br />
By means of this method a user that has a token valid for a certain VRE, can retrieve all the posts of that VRE.<br />
====Get posts by hashtag====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-by-hashtag<br />
</source><br />
The method requires as parameter:<br />
* hashtag: the hashtag to be contained in the returned posts[mandatory].<br />
<br />
=== Comments ===<br />
====Retrieve comments by user====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/comments/get-comments-user<br />
</source><br />
It is an http GET method.<br />
====Retrieve comments by user and time====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/comments/get-comments-user-by-time<br />
</source><br />
It is an http GET method. It requires:<br />
* time: the time (in milliseconds) since when the comments need to be retrieved [default is zero, optional];<br />
=== Notifications ===<br />
====Retrieve user's notifications in a given range====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/notifications/get-range-notifications<br />
</source><br />
<br />
By means of this method a user can retrieve his/her last notifications in the range [a, b]. The method requires:<br />
<br />
* from: an integer greater than zero;<br />
* quantity: an integer equal or greater than zero.<br />
<br />
The final range is evaluated as [from, from + quantity].<br />
====Notify job status====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/notifications/notify-job-status<br />
</source><br />
<br />
By means of this method a user can be notified about the status of a job in the infrastructure. Jobs are, for instance, SmartExecutor's plugins or DataMiner's algorithms. The method requires the following parameters in a JSON object via HTTP POST<br />
<br />
* recipient: the username of the user to be alerted [mandatory];<br />
* job_id: the identifier of the job (e.g, a uuid) [mandatory];<br />
* job_name: the name of the running job (e.g., Social-data-indexer-plugin)[mandatory];<br />
* service_name: the name of the service that launched the job (e.g., SmartExecutor) [mandatory];<br />
* status: the status reached by the job, that is any among CANCELLED, CANCELLING, DELETED, DELETING, EXECUTING, FAILED, SUCCEEDED, SUBMITTED, TIMED_OUT, WAITING [mandatory];<br />
* status_message: a detailed message about the status [optional].<br />
<br />
=== Messages ===<br />
====Send a message====<br />
It is an HTTP POST method.<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/write-message<br />
</source><br />
<br />
By default, the sender of the message is the owner of the gcube-token. According to the recipients' notifications preferences, they could receive a portal notification, an email notification and so on.<br />
<br />
Parameters, to be put in a JSON object are:<br />
<br />
* subject: the subject of the message to send;<br />
* body: the body of the message;<br />
* recipients: a list of valid comma separated users' ids (e.g., andrea.rossi, mario.rossi) or users' emails (e.g., andrea.rossi@isti.cnr.it, mario.rossi@isti.cnr.it);<br />
<br />
An example of the body of the request is as follows<br />
<source lang="xml"><br />
{"subject": "....", "body": ".....", "recipients":[{"id":"userid1"},{"id":"userid2"}]}<br />
</source><br />
<br />
or, if you want to use valid mails (i.e. recognized in the infrastructure)<br />
<source lang="xml"><br />
{"subject": "....", "body": ".....", "recipients":[{"id":"mail1"},{"id":"mail2"}]}<br />
</source><br />
<br />
====Get received messages====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/get-received-messages<br />
</source><br />
It is an http GET method.<br />
<br />
====Get sent messages====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/get-sent-messages<br />
</source><br />
It is an http GET method.<br />
=== Users ===<br />
====User exists====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/user-exists<br />
</source><br />
<br />
The method requires<br />
<br />
* username: the username of the user which has to be checked.<br />
<br />
'''Note''' The method can be only invoked with a root-context application token.<br />
====Read user's custom fields====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-custom-attribute<br />
</source><br />
<br />
The method requires<br />
<br />
* attribute: that is the name of a valid attribute to be read.<br />
====Get user's email====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-email<br />
</source><br />
<br />
The method returns the email associated to the owner of the security token used.<br />
====Get user's profile====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-profile<br />
</source><br />
<br />
The method returns the profile of the owner of the security token used.<br />
====Get user's fullname====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-fullname<br />
</source><br />
<br />
The method returns the fullname associated to the owner of the security token used.<br />
====Get all users' usernames in VRE====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames<br />
</source><br />
<br />
The method returns the usernames of the context associated to the security token.<br />
<br />
====Get usernames by global role====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames-by-global-role<br />
</source><br />
<br />
The method requires<br />
<br />
* role-name: that is the name of the role to consider.<br />
<br />
'''''This method can be only invoked with an Infrastructure Token.'''''<br />
<br />
<br />
====Get usernames by role====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames-by-role<br />
</source><br />
<br />
The method requires<br />
<br />
* role-name: that is the name of the role to consider.<br />
<br />
=== VREs ===<br />
====Get User VREs====<br />
This method allows a user to retrieve the list of VREs she belongs to<br />
<br />
<source lang="java"><br />
2/vres/get-my-vres<br />
</source><br />
<br />
The method accepts an option parameter:<br />
* getManagers : if set to true, reports also the VRE Managers of the VRE<br />
<br />
== Troubleshooting ==<br />
<br />
There could be many reasons for a failed request. Please always check the HTTP CODE and the message within the response. <br />
<br />
=== HTTP Error Code ===<br />
This table tries to help you:<br />
<br />
{| border="1" cellpadding="20" cellspacing="0"<br />
!HTTP CODE<br />
!Meaning<br />
|-<br />
|400 - BAD_REQUEST<br />
|Please check the parameters you are passing<br />
|-<br />
|403 - FORBIDDEN<br />
|Is the token valid? / Does it belong to an application? [In case of application request]<br />
|-<br />
|404 - NOT_FOUND<br />
|The object(s) you are asking for is(are) not available<br />
|-<br />
|500 - INTERNAL_SERVER_ERROR<br />
|There was an error while serving your request (retry later)<br />
|-<br />
|}<br />
<br />
<br />
=== Always manage HTTP Redirects on POST requests (i.e. 30X codes) ===<br />
The service accepts both incoming http and https client requests, which are automatically redirect to https protocol by the reverse proxy nginx. This redirect mechanism must be taken into account when POST calls are performed, because it is not (always) supported and transparent by clients. In order to be sure, please check if you get a 30X error and extract the redirection urls within the response header. The following is a working java code that extract such information. It uses the apache-client library to perform http calls.<br />
<br />
<source lang="java"><br />
<br />
// build the client and the request<br />
String path = ... ; <br />
HttpClientBuilder clientBuilder = HttpClientBuilder.create();<br />
CloseableHttpClient httpClient = clientBuilder.build();<br />
HttpPost request = new HttpPost(path);<br />
JSONObject obj = new JSONObject();<br />
<br />
... // build the json object<br />
<br />
// set header properties<br />
request.addHeader("gcube-token", ...);<br />
request.addHeader("Content-type", ContentType.APPLICATION_JSON.toString());<br />
StringEntity params = new StringEntity(obj.toJSONString(), ContentType.APPLICATION_JSON);<br />
request.setEntity(params);<br />
HttpResponse response = httpClient.execute(request);<br />
logger.info(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
<br />
int status = response.getStatusLine().getStatusCode();<br />
<br />
// check the response status and look if it was a redirect problem<br />
if (status != HttpURLConnection.HTTP_OK && (status == HttpURLConnection.HTTP_MOVED_TEMP ||<br />
status == HttpURLConnection.HTTP_MOVED_PERM ||<br />
status == HttpURLConnection.HTTP_SEE_OTHER)) {<br />
<br />
// redirect -> fetch new location<br />
Header[] locations = response.getHeaders("Location");<br />
Header lastLocation = locations[locations.length - 1];<br />
String realLocation = lastLocation.getValue();<br />
logger.debug("New location is " + realLocation);<br />
<br />
// perform again the same request<br />
request = new HttpPost(realLocation);<br />
request.addHeader("gcube-token", ...);<br />
request.addHeader("Content-type", ContentType.APPLICATION_JSON.toString());<br />
params = new StringEntity(obj.toJSONString(), ContentType.APPLICATION_JSON);<br />
request.setEntity(params);<br />
response = httpClient.execute(request);<br />
logger.info(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
} else<br />
logger.debug(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
</source><br />
<br />
= Service methods (Version 1) - DEPRECATED =<br />
In the following, the list of available sub-services available under the first version, its methods and the parameters they need in order to be called are reported. The gcube-token is always required so it won't be reported as parameter. <br />
<br />
'''This version has been replaced by version 2, please do not use this version and update your code asap.'''<br />
<br />
== List of available Sub-Services and methods ==<br />
=== Tokens ===<br />
==== Generate application token ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
tokens/generateApplicationToken<br />
</source><br />
<br />
First of all, you need to have a ''user gcube-token'' for authentication purposes. You can get it using the Token Generator portlet available on the VRE you are interesting the app will post in.<br />
Now, to get an ''application gcube-token'' you can invoke the method. It is an HTTP/POST method that accepts as parameters:<br />
<br />
* appid: the identifier of the application as reported in the Application Profile.<br />
<br />
For example, using the RESTClient plugin for Firefox you will have something like this for requiring a new application token:<br />
<br />
[[File:GenerateApplicationToken.png]]<br />
<br />
Push on the SEND button and if all it's ok, your application gcube-token will be generated and returned as ''plain text'' in the response within the HTTP response having code 201.<br />
=== Posts '''DEPRECATED''' ===<br />
==== Write application post ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/writePostApplication<br />
</source><br />
<br />
By means of the application token requested with the above method, the app can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters:<br />
<br />
* text: text of the post; <br />
* params: additional parameters if your application supports the direct opening of of this update's object e.g. id=12345&type=foo [optional];<br />
* previewtitle: a title for the preview of the post [optional];<br />
* previewdescription: a description for the preview of the post [optional];<br />
* httpimageurl: if you want to put an image, pass its url [optional];<br />
* enablenotification: a boolean value to enable notifications [optional].<br />
<br />
For example:<br />
<br />
[[File:WriteApplicationPost.png]]<br />
<br />
Push on SEND and check the code associated to the returned response (201 in case of success).<br />
<br />
==== Retrieve all application posts ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByApp<br />
</source><br />
<br />
In order to retrieve all the posts the application wrote (in every VRE it is allowed to write in) you can invoke this HTTP/GET method.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByApp?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByUser<br />
</source><br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts since a given date ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getRecentPostsByUserAndDate<br />
</source><br />
<br />
If you are interested in retrieving user's posts whose timestamps are greater than a given date you can invoke this method.<br />
This HTTP/GET method accepts this parameter:<br />
<br />
* time: the time of the oldest post to retrieve converted in milliseconds.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getRecentPostsByUserAndDate?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&time=xxxxx<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts up to a given quantity ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getRecentPostsByUser<br />
</source><br />
<br />
If you are interested in retrieving user's posts, starting from the newest up to a given quantity you can invoke this method.<br />
This HTTP/GET method accepts two parameter:<br />
<br />
* quantity: a value greater than zero (the default value is 10).<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getRecentPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&quantity=20<br />
</source><br />
<br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Write user post ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/writePostUser<br />
</source><br />
<br />
By means of a valid user token, he/she can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters:<br />
<br />
* text: text of the post; <br />
* previewtitle: a title for the preview of the post [optional];<br />
* previewdescription: a description for the preview of the post [optional];<br />
* previewHost: [optional];<br />
* previewUrl: [optional];<br />
* httpimageurl: if you want to put an image, pass its url [optional];<br />
* enablenotification: a boolean value to enable notifications [optional];<br />
<br />
For example:<br />
<br />
[[File:Userpost.png]]<br />
<br />
Push on SEND and check the code associated to the returned response (201 in case of success).<br />
<br />
==== Retrieve the identifiers of the posts a user liked ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllLikedPostIdsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve the identifiers of the posts he/she liked.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllLikedPostIdsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the identifiers of the posts are returned in JSON format.<br />
<br />
==== Retrieve the posts a user liked ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllLikedPostsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve the posts he/she liked. <br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllLikedPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
====Retrieve all vre posts====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByVRE<br />
</source><br />
<br />
By means of this method a user that has a token valid for a certain VRE, can retrieve all the posts of that VRE.<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByVRE?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
=== Notifications ===<br />
====Retrieve user's notifications in a given range====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
notifications/getRangeNotificationsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve his/her last notifications in the range [a, b]. The method requires:<br />
<br />
* from: an integer greater than zero;<br />
* quantity: an integer equal or greater than zero.<br />
<br />
The final range is evaluated as [from, from + quantity]<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/notifications/getRangeNotificationsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&from=from&quantity=quantity<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
=== Messages ===<br />
<br />
====Send a message====<br />
You can send a message via the web service by using the following POST method:<br />
<br />
* writeMessageToUsers<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
messages/writeMessageToUsers<br />
</source><br />
<br />
By default, the sender of the message is the owner of the gcube-token. If you need to specify a different "sender", you can do that by specifying the parameter sender. Furthermore, according to the recipients' notifications preferences, they could receive a portal notification, a mail notification and so on.<br />
<br />
Parameters are:<br />
<br />
* subject: the subject of the message to send;<br />
* body: the body of the message;<br />
* recipients: a list of valid comma separated users' ids (e.g., andrea.rossi, mario.rossi).<br />
* sender: a valid sender's id [optional]<br />
<br />
An example usage of the method is reported in the picture below (without specifying the sender):<br />
<br />
[[File:Send_message_example.png]]<br />
=== Users ===<br />
====Read user's custom fields====<br />
You can read user's custom attributes using the following method<br />
<br />
*readCustomAttr<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
users/readCustomAttr<br />
</source><br />
<br />
The ''read'' method requires<br />
<br />
* attribute: that is the name of a valid attribute to be read.<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/users/readCustomAttr?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&attribute=attributeX<br />
</source><br />
<br />
= See Also =<br />
<br />
== D4Science Developers website ==<br />
[https://dev.d4science.org D4Science Developers website]<br />
<br />
== SWAGGER documentation==<br />
[https://dev.d4science.org/swagger/social-networking/ SWAGGER documentation of the Social-Networking RESTful interface]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=Statistical_Algorithms_Importer&diff=33322Statistical Algorithms Importer2020-04-15T08:05:41Z<p>Massimiliano.assante: </p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
In this guide we describe the Statistical Algorithms Importer (also known as '''Methods Importer''' or '''Software Algorithms Importer''') web interface.<br />
<br />
== Overview ==<br />
This Methods Importer (SAI) is a tool to import algorithms in the D4Science e-Infrastructure. Currently, it supports different types of software integration. SAI separates software development from its deployment in the infrastructure in a very flexible way. After the first deployment, made in collaboration with the e-Infrastructure team, script developers can modify and update their scripts by themselves, without the intervention of the e-Infrastructure team.<br />
<br />
In order to transform an algorithm, three main passages are required: <br />
<br />
1 - Indicate Input, Output and types of a main script orchestrating the process<br />
<br />
2 - Create the Software: this operation creates the interface from the e-Infrastructure service to the script and should be used each time either the interface (I/O) or the name of the algorithm or the required additional packages change<br />
<br />
3 - Publish the Software: this operation communicates to the infrastructure that a newly create software should be put online<br />
<br />
Additionally, the Repackage function can be used when only the internal code of the orchestrating script changes and the algorithms has already been published.<br />
The pages in this Wiki explain the details of these operations. <br />
<br />
[[Image:StatisticalAlgorithmsImporter1.png|thumb|center|800px|Statistical Algorithms Importer (SAI), portlet. Main interface.]]<br />
<br />
== F.A.Q. ==<br />
Please, read our best practices first: [[Statistical Algorithms Importer: FAQ|F.A.Q.]]<br />
<br />
== Demonstration ==<br />
A demonstration video is available [http://data.d4science.org/SkhVR3AyTUNLaStCV2tNdUhsL2VIQ3AwMG1tTjVka3dHbWJQNStIS0N6Yz0 here].<br />
<br />
== Main Steps==<br />
<br />
# [[Statistical Algorithms Importer: Create Project|Creating a new Project]]<br />
# [[Statistical Algorithms Importer: Publish Algorithms|Publishing Algorithms for deployment]]<br />
# [[Statistical Algorithms Importer: Repackage| Repackaging a script]]<br />
# [[Advanced Input| Advanced Input ]]<br />
# [[Statistical Algorithms Importer: StatusUpdate| Updating the status of a computation]]<br />
# [[Statistical Algorithms Importer: FAQ|F.A.Q.]]<br />
# [[Statistical_Algorithms_Importer:_R_Project#Import_Resources_From_GitHub | Import projects from GitHub ]]<br />
<br />
==Related Links==<br />
* [[DataMiner_Manager|DataMiner Manager]]<br />
<br />
<!--<br />
==References==<br />
{{Reflist}} <br />
[[Template:Statistical Algorithms Importer]] <br />
--><br />
<br />
[[Category:Statistical Algorithms Importer]]<br />
[[Category:DataMiner]]<br />
[[Category:User's Guide]]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=GCube_Portal_Installation&diff=33316GCube Portal Installation2020-03-30T12:30:58Z<p>Massimiliano.assante: /* gCube Infrastructure Gateway 4.7.0-SNAPSHOT (February 2020) */</p>
<hr />
<div>[[Category:Administrator's Guide]]<br />
gCube adopts Liferay Portal as enabling portal technology for its Infrastructure Gateway. gCube portal is built over [https://sourceforge.net/projects/lportal/files/Liferay%20Portal/6.2.5%20GA6/ Liferay Portal 6.2 CE GA6 (Newton / Build 6205 / January 6, 2016)]<br />
<br />
=gCube Infrastructure Gateway 4.7.0-SNAPSHOT (February 2020)=<br />
<br />
Starting from 3.0 release, the gCube Infrastructure Gateway has been mavenized, this make installing it '''a very simple process''' and takes '''less than five minutes''' to complete. We've been '''working hard''' to make this process as '''easiest and quicker as possible'''. <br />
<br />
Supported Operating Systems are Linux and MacOSX. With Java8 installed.<br />
<br />
[[File:Gcube4.0-gateway.png]]<br />
<br />
<br />
==Install the gCube Infrastructure Gateway Bundle==<br />
'''[https://code-repo.d4science.org/gCubeSystem/gcube-portal-bundle gCube Infrastructure Gateway]''' is bundled with [https://tomcat.apache.org/tomcat-7.0-doc/index.html Apache Tomcat 7] and the latest stable relase of [https://gcube.wiki.gcube-system.org/gcube/index.php/Featherweight_Stack gCube Featherweight Stack Distribution], [http://gcube.wiki.gcube-system.org/gcube/index.php/Social_Networking_Library gCube Social Networking Library] and [http://gcube.wiki.gcube-system.org/gcube/index.php/Home_Library gCube Home Library]<br />
<br />
All you need to do is the following:<br />
* Download the latest version of the Portal-Bundle from maven, make sure you get the latest version from our repo: [http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-staging-jenkins/org/gcube/distribution/gcube-portal-bundle/ gcube-portal-bundle]<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>gcube-portal-bundle</artifactId><br />
<version>LATEST</version><br />
<type>tar.gz</type><br />
</dependency><br />
</source><br />
* Extract the gCube Infrastructure Gateway bundle tar.gz into a folder on your machine<br />
* Add to the ~/.bashrc file the following:<br />
** clean CLASSPATH: "export CLASSPATH="<br />
** add CATALINA_HOME env. variable (the home directory of tomcat)<br />
** add JAVA_HOME env. variable (the home directory of java)<br />
** add JRE_HOME env. variable (JAVA_HOME/jre)<br />
** add CATALINA_OPTS env. variable "export CATALINA_OPTS="-Dexternal-properties=portal-developer.properties -Xmx1000m -Xms1000m -Dlog4j.ignoreTCL=true -Dlog4j.configuration=file://$CATALINA_HOME/portal-log4j.properties -Dlogback.configurationFile=file://$CATALINA_HOME/portal-logback.xml"<br />
** add CATALINA_PID env. variable "export CATALINA_PID=~/pid.txt"<br />
<br />
''' IMPORTANT '''<br />
The CATALINA_OPTS env variable: '''-Dexternal-properties=portal-developer.properties''' optimizes Liferay Portal for development by removing all caches and thus making sure that any change made by the developer is visible in the website as soon as technically possible. One of the final goals is to avoid rebooting the server as much as possible. Further Information at: [https://web.liferay.com/community/wiki/-/wiki/Main/Liferay+Developer+Mode LiferayDeveloper Mode Doc]<br />
<br />
After that, say you created a "Portal-Bundle" folder on your user's home(~) and extracted the org.gcube.portal.portal-bundle-2+ in there, the final part of your ~/.bashrc file may look like the following:<br />
<source lang="xml"><br />
export JAVA_HOME=/usr/lib/jvm/java-7-oracle/<br />
export JRE_HOME=$JAVA_HOME/jre<br />
export CATALINA_HOME=~/Portal-Bundle/tomcat-7.0.62<br />
export CATALINA_BASE=$CATALINA_HOME<br />
export CATALINA_TMPDIR=$CATALINA_HOME/temp<br />
<br />
export CLASSPATH=<br />
export CATALINA_OPTS="-Dexternal-properties=portal-developer.properties -Xmx1000m -Xms1000m -Djava.protocol.handler.pkgs=com.sun.net.ssl.internal.www.protocol -Dlog4j.ignoreTCL=true -Dlog4j.configuration=file://$CATALINA_HOME/portal-log4j.properties"<br />
<br />
export CATALINA_PID=~/pid.txt<br />
</source><br />
<br />
==First startup of gCube Infrastructure Gateway==<br />
<br />
'''gCube Infrastructure Gateway enabling technology (Liferay)''' comes with a default database called HSQL or "hypersonic." '''This is not meant for production use!'''' <br />
You should switch to a real database to use the gCube Infrastructure Gateway. For further information see Section Configuring gCube Infrastructure Gateway database section in the following of this page.<br />
<br />
However, if you want to startup gCube Infrastructure Gateway using the default database just open a shell and enter the following command: <br />
<pre><br />
source ~/.bashrc<br />
</pre><br />
and start tomcat<br />
<pre><br />
$CATALINA_HOME/bin/startup.sh<br />
</pre> <br />
<br />
'''gCube Infrastructure Gateway take less than a minute to initialize''', in the meanwhile you might want to '''check the portal logs''', to do so, enter the command: <br />
<pre><br />
tail -f $CATALINA_HOME/logs/catalina.out<br />
</pre><br />
open your browser and go to http://localhost:8080<br />
if you performed all of the above operations correctly, you should be presented something like the following <br />
[[File:Gcube4.0-bundle-1.png | screenshot]]<br />
<br />
all you need to do now is '''follow the instructions on the screen''', a wizard application will guide you through the [https://wiki.gcube-system.org/gcube/GCube_Portal_Installation#Staging_procedure Staging procedure]. '''Thank you for using [http://www.gcube-system.org gCube]'''.<br />
<br />
== Starting/Stopping a gCube Infrastructure Gateway ==<br />
In order to start / stop tomcat, you should execute the following commands:<br />
* source the configured ~/.bashrc<br />
* $CATALINA_HOME/bin/startup.sh<br />
* $CATALINA_HOME/bin/shutdown -force<br />
<br />
'''please note''': If you change gCore's configuration or Tomcat's settings or Liferay portal.ext.properties file, you should restart tomcat for the changes to be applied.<br />
<br />
==Additional Information==<br />
<br />
=== Setting the Infrastructure ===<br />
<br />
By Default the bundle is configured to run on the gCube Development infrastructure. However if you want to change infrastructure please edit this file under $CATALINA_HOME/conf/infrastructure.properties<br />
<br />
=== Changing Company web id ===<br />
<br />
By default, gCube Infrastructure Gateway starts with "liferay.com" as default company web id, this means, among other things, that at login time portal will suggest email addresses ending with @liferay.com. If you want to configure the portal to use your own Company web id just add the following property in portal.ext.properties file located under $CATALINA_HOME/webapps/ROOT/WEB-INF/classes. <br />
<br />
Replace <MY_COMPANY_WEB_ID> with your own company web id, e.g. google.com<br />
<br />
<pre><br />
#company web id name<br />
company.default.web.id=<MY_COMPANY_WEB_ID><br />
</pre><br />
<br />
'''please note 1''': This change must be performed '''before starting up your portal for the first time'''. <br />
<br />
'''please note 2''': This change '''will not work''' if you use the default HSQL DB (Hypersonic)<br />
<br />
=== Configuring gCube Infrastructure Gateway database ===<br />
<br />
If you want to configure the portal to use a different database than the default HSQL DB one (and '''you are encouraged to do so''') you should add some properties in portal.ext.properties file located under $CATALINA_HOME/webapps/ROOT/WEB-INF/classes.<br />
Please refer to the Liferay [http://www.liferay.com/community/wiki/-/wiki/Main/Database+Portal+Properties Database Portal Properties] to know which type of database is supported.<br />
<br />
Configuration examples: (replace <YOUR_DB_NAME>, <YOUR_DB_USER>, <YOUR_DB_PASSWORD> with actual values<br />
* '''PostgreSQL'''<br />
<pre><br />
#<br />
# PostgreSQL<br />
#<br />
jdbc.default.driverClassName=org.postgresql.Driver<br />
jdbc.default.url=jdbc\:postgresql\://localhost\:5432/<YOUR_DB_NAME><br />
jdbc.default.username=<YOUR_DB_USER><br />
jdbc.default.password=<YOUR_DB_PASSWORD> <br />
</pre><br />
* '''MySQL'''<br />
<pre><br />
#<br />
# MySQL<br />
#<br />
jdbc.default.driverClassName=com.mysql.jdbc.Driver<br />
jdbc.default.url=jdbc:mysql://localhost/<YOUR_DB_NAME>?useUnicode=true&characterEncoding=UTF-8&useFastDateParsing=false<br />
jdbc.default.username=<YOUR_DB_USER><br />
jdbc.default.password=<YOUR_DB_PASSWORD><br />
</pre><br />
<br />
The above configuration are known to be working with gCube Infrastructure Gateway.<br />
<br />
== Troubleshooting ==<br />
<br />
* If you get something like the following exception at portal startup, just cleanup your browser cookies.<br />
<br />
<pre><br />
ERROR [CompanyThreadLocal:54] com.liferay.portal.NoSuchCompanyException: No Company exists with the primary key 1<br />
com.liferay.portal.NoSuchCompanyException: No Company exists with the primary key 1<br />
at com.liferay.portal.service.persistence.CompanyPersistenceImpl.findByPrimaryKey(CompanyPersistenceImpl.java:436)<br />
at com.liferay.portal.service.base.CompanyLocalServiceBaseImpl.getCompany(CompanyLocalServiceBaseImpl.java:325)<br />
at sun.reflect.GeneratedMethodAccessor515.invoke(Unknown Source)<br />
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)<br />
at java.lang.reflect.Method.invoke(Method.java:597)<br />
.....<br />
</pre><br />
<br />
* If the Workspace Portlet shipped within this Bundle popups an error on startup you need to Install the Encryption key and add the Encryption key in your classpath to access runtime resources.<br />
<br />
== Staging procedure ==<br />
<br />
'''Staging is a revised feature bundled with gCube Infrastructure Gateway 4.0''' When you install the portal for the first time a Wizard Application will guide you through the installation configuration and will setup everything's needed for you.<br />
<br />
Follow the standard [https://dev.liferay.com/discover/deployment/-/knowledge_base/6-2/using-liferays-setup-wizard Liferay Wizard Procedure] until entering your desired password for your user. After that you will be presented the gCube Portal Staging procedure as displayed in the screenshot below.<br />
<br />
[[File:Gcube4.0-bundle-5.png]]<br />
<br />
Click "Begin Install" button to make the gCube Staging installation procedure start.<br />
<br />
The whole procedure consist of 3 steps, as shown in the figure below:<br />
* Infrastructure Selection<br />
* Virtual Research Environments check-up<br />
* Final step<br />
<br />
Provide the information requested, if your not sure about the settings you should read more about gCube on this [http://wiki.gcube-system.org wiki]. The next steps are quite handy and we think no further documentation is needed, however feel free to ask for support if needed.<br />
<br />
[[File:Gcube4.0-bundle-staging.png]]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=OAuth2.0&diff=33275OAuth2.02020-03-06T15:31:05Z<p>Massimiliano.assante: /* Step 2 - Require an authorization code */</p>
<hr />
<div>__TOC__<br />
<br />
= OAuth 2.0 gCube Support =<br />
By means of the OAuth 2.0 protocol (authorised) third party applications can operate on user's behalf over the D4Science infrastructure (while protecting the member's credentials).<br />
For more information about the OAuth authorization framework please visit the official [https://oauth.net/2/ OAuth site]. For technical details you could also check the [https://tools.ietf.org/html/rfc6749| OAuth 2.0 RFC]. In the following, the needed steps to authorize third party applications to operate on user's behalf and the D4Science infrastructure are explained. <br />
<br />
== Step 1 - Configure your application ==<br />
Authorized third party applications must be registered on the Infrastructure. Up to now the application registration is not automatic but you need to open a ticket at https://support.d4science.org and select 3rd Party App Registration.<br />
<br />
The ticket needs to have '''3rd Party Application Support''' as tracker. The application name as well as the redirect uri need to be provided (https is strongly suggested). This url must be absolute, and without arguments. Once the application is registered, it will have associated:<br />
<br />
* a unique client identifier (client_id);<br />
* a client secret which must be kept secret, as the name itself says (client_secret).<br />
<br />
<source lang="xml"><br />
client_id: ab9d32c9-9425-42f1-99d7-50d68e651e9c<br />
client_secret: y9r93zDv4D<br />
</source><br />
<br />
Both are needed to proceed with the OAuth 2.0 workflow.<br />
<br />
== Step 2 - Require an authorization code ==<br />
Once the application is registered and configured, an authorization code can be requested. This code is not the final one needed to perform operations on user behalf, but is the one that will be exchanged for that in the next step. To require this code that has a very short time to live, the application must invoke the authorization service, which is gateway dependant.<br />
<br />
=== Retrieving the authorization endpoint location===<br />
<br />
The previous authorization endpoint location is portal's gateway dependant and '''can be found in Table 1''' or retrieved dinamically by using the registry service REST-API hosted at <br />
<br />
''https://registry.d4science.org/icproxy/gcube/service/'' <br />
<br />
'''Get authorization endpoint location in the following Table 1'''<br />
<br />
{| class="wikitable"<br />
|+Table 1: authorization endpoints location depending on the D4Science Gateway<br />
! style="font-weight: bold;" | D4Science Gateway name<br />
! style="font-weight: bold;" | Authorization endpoint<br />
|-<br />
| ARIADNEplus Gateway<br />
| https://ariadne.d4science.org/group/ariadneplus-gateway/authorization<br />
|-<br />
| iMarine Gateway<br />
| https://i-marine.d4science.org/group/imarine-gateway/authorization<br />
|-<br />
| AGINFRA+ Gateway<br />
| https://aginfra.d4science.org/group/aginfra-gateway/authorization<br />
|-<br />
| Blue-Cloud Gateway<br />
| https://blue-cloud.d4science.org/group/bluecloud-gateway/authorization<br />
|-<br />
| D4Science.org Gateway<br />
| https://services.d4science.org/group/d4science-services-gateway/authorization<br />
|-<br />
| DESCRAMBLE Gateway<br />
| https://descramble.d4science.org/group/descramble-gateway/authorization<br />
|-<br />
| GEMex Gateway<br />
| https://gemex.d4science.org/group/gemex-gateway/authorization<br />
|-<br />
| OpenAIRE-Connect Gateway<br />
| https://openaire-connect.d4science.org/group/openaire-connect-gateway/authorization<br />
|-<br />
| PARTHENOS Gateway<br />
| https://parthenos.d4science.org/group/parthenos-gateway/authorization<br />
|-<br />
| SoBigData Gateway<br />
| https://sobigdata.d4science.org/group/sobigdata-gateway/authorization<br />
|}<br />
<br />
'''Retrieve the authorization endpoint location dynamically (ADVANCED MODE)'''<br />
<br />
Registry Swagger REST-API avaiable at: https://dev.d4science.org/swagger/registry/<br />
<br />
Which requires an [https://dev.d4science.org/authorization/ authorization token]. The returned profile is something like<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<serviceEndpoints><br />
<Resource version="0.4.x"><br />
<ID>b1c5851f-1d47-4c60-8906-941ab859a334</ID><br />
<Type>RuntimeResource</Type><br />
<Scopes><br />
<Scope>/gcube</Scope><br />
</Scopes><br />
...<br />
<AccessPoint><br />
<Description>AuthorisationServer where to authenticate external SmartGears services oAuth like</Description><br />
<Interface><br />
<Endpoint EntryName="AuthorisationServer">https://next.d4science.org/group/next/authorization</Endpoint><br />
</Interface><br />
</AccessPoint><br />
</Profile><br />
</Resource><br />
</serviceEndpoints><br />
</source><br />
<br />
=== Invoking the authorization service===<br />
<br />
Once you got the AuthorisationServer endpoint and the call is executed, two things can happen:<br />
* the user is requested to perform the login, thus accepting the application operation and granting it;<br />
* if there is a valid existing permission grant for the application, the login operation is by-passed and the redirect_uri is followed.<br />
<br />
The request is an '''HTTP GET''' call, that needs the following parameters to be provided<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| client_id<br />
| the client identifier associated to the application<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the redirect uri provided at registration phase<br />
| yes<br />
|-<br />
| state<br />
| a unique string value hard to guess, needed to prevent CRSF attacks (e.g. AAedBEeF545sdfKef897)<br />
| yes<br />
|-<br />
| scope<br />
| this field is optional. It can be specified to request (with the next step) an access token valid for a given<br />
infrastructure scope. If missing, the scope will be choose by the user on credentials insertion.<br />
| no<br />
|}<br />
<br />
This is an example in development environment<br />
<pre><br />
https://next.d4science.org/group/next/authorization?client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&scope=%2Fgcube&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&state=7d12bf13-111c-4f46-ab06-9e9e08ad377b<br />
</pre><br />
<br />
==== End user Experience ====<br />
<br />
Once redirected, the user will be presented with D4Science's authentication dialog box. This identifies your application as well as outlines the particular member permissions that your application requests. The logo and application name can be configurable, in the example the application name is '''Application X'''<br />
<br />
[[File:Oauth-ux.jpg]]<br />
<br />
==== Application is Accepted ====<br />
<br />
If the user grants permissions to the application, a redirection to the redirect_uri is performed. Attached to the redirect_uri, two important url arguments are provided<br />
<br />
* code: which is the authorization code. It expires in 10 seconds;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
==== Application is Rejected ====<br />
<br />
If, for any reason, the request fails, the redirect_uri is still followed but the url has attached the following parameters<br />
* error: a code that states the type of error;<br />
* error_description: a textual description of the actual error;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
== Step 3 - Exchange Authorization Code for an Access Token ==<br />
This is the step in which the previously obtained code can be exchanged to obtain a real access token. <br />
This time, an '''HTTP POST''' method needs to be performed, with a '''Content-Type''' of type '''application/x-www-form-urlencoded'''.<br />
<br />
The client_id and secret must be passed as the basic auth header Base64 encoded, see the example in Java below:<br />
<br />
e.g.<br />
<source lang="java"><br />
...<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
..<br />
</source><br />
<br />
The request-body must contain the following parameters:<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| grant_type<br />
| The value of this field should always be authorization_code<br />
| yes<br />
|-<br />
| code<br />
| the authorization code you received from the previous step.<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the same 'redirect_uri' value that you<br />
passed in the previous step.<br />
| yes<br />
|}<br />
<br />
For instance, in development environment<br />
<br />
<source lang="xml"><br />
POST /gcube-oauth/v2/access-token HTTP/1.1<br />
Host: socialnetworking-d-d4s.d4science.org<br />
Content-Type: application/x-www-form-urlencoded<br />
<br />
grant_type=authorization_code&code=...&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&client_secret=...<br />
</source><br />
<br />
In case of success, the response will contain a JSON object, having these properties:<br />
<br />
* access_token: the obtained access token to use to perform operations on user's behalf;<br />
* expires_in: the time to live of the token expressed in seconds;<br />
* scope: in which scope the token is valid;<br />
* token_type: that is "Bearer".<br />
<br />
In case of failures, instead, the returned object will contain:<br />
<br />
* error : an error string according the oauth 2.0 RFC.<br />
* error_description : a text description for the error.<br />
<br />
=== D4Science oAuth2 endpoint location === <br />
In the '''production''' environment, its value is<br />
<source lang="xml"><br />
https://api.d4science.org/oauth/v2/access-token<br />
</source><br />
<br />
'''HTTP POST Example in JAVA'''<br />
<br />
<source lang="java"><br />
String ACCESS_TOKEN_ATTR = "access_token";<br />
String SCOPE_ATTR = "scope";<br />
String fullPath2oAuthService = "https://api.d4science.org/oauth/v2/access-token";<br />
try {<br />
CloseableHttpClient httpClient = HttpClientBuilder.create().setRedirectStrategy(new LaxRedirectStrategy()).build();<br />
HttpPost httpPostRequest = new HttpPost(fullPath2oAuthService);<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
httpPostRequest.addHeader("Content-Type", "application/x-www-form-urlencoded");<br />
List<NameValuePair> params = new ArrayList<NameValuePair>();<br />
params.add(new BasicNameValuePair("grant_type", "authorization_code"));<br />
params.add(new BasicNameValuePair("code", tempCode));<br />
params.add(new BasicNameValuePair("redirect_uri", "https://www.gcube-system.org/oauth-authentication-example/example-app"));<br />
httpPostRequest.setEntity(new UrlEncodedFormEntity(params));<br />
<br />
CloseableHttpResponse httpresponse = httpClient.execute(httpPostRequest);<br />
int responseCode = httpresponse.getStatusLine().getStatusCode();<br />
if (responseCode < 200 || responseCode >= 300) {<br />
System.out.println("Error while getting authorization code, response status line from "<br />
+ fullPath2oAuthService + " was: " + responseCode); <br />
}<br />
JSONParser parser = new JSONParser();<br />
JSONObject json = (JSONObject) parser.parse(EntityUtils.toString(httpresponse.getEntity()));<br />
accessToken = json.get(ACCESS_TOKEN_ATTR).toString();<br />
scope = json.get(SCOPE_ATTR).toString();<br />
} catch(Exception e){<br />
System.out.println("Failed to perform request");<br />
e.printStackTrace();<br />
<br />
}<br />
</source><br />
<br />
== Step 4 - Get member data ==<br />
Having an access token, your application can perform some operations on behalf of the user. For instance, it can retrieve user's information profile by exploiting the [https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface social networking web service].<br />
<br />
Its current production endpoint is: https://api.d4science.org/rest/<br />
<br />
The subservice to call, once the social networking service endpoint has been retrieved, is this one<br />
<br />
<source lang="xml"><br />
2/users/get-oauth-profile<br />
</source><br />
<br />
It requires a ''gcube-token'' as query parameter, whose value is exactly the access token retrieved in the previous step. The result is something like this<br />
<br />
<source lang="ruby"><br />
{<br />
id: ".....",<br />
roles: [ ], // roles in the given-token context, e.g. VRE-Manager<br />
picture: "....",<br />
name: "....",<br />
middle_name: "....",<br />
male: as boolean ,<br />
location_industry: "....",<br />
given_name: "....",<br />
email: "....",<br />
job_title: "....",<br />
family_name: "....",<br />
verified_email: as boolean <br />
}<br />
</source><br />
<br />
== See Also ==<br />
<br />
[https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface Social Networking Service]<br />
<br />
[https://dev.d4science.org/swagger/social-networking/ Social Networking Service Docs SWAGGER]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=OAuth2.0&diff=33274OAuth2.02020-03-06T15:18:23Z<p>Massimiliano.assante: /* Step 1 - Configure your application */</p>
<hr />
<div>__TOC__<br />
<br />
= OAuth 2.0 gCube Support =<br />
By means of the OAuth 2.0 protocol (authorised) third party applications can operate on user's behalf over the D4Science infrastructure (while protecting the member's credentials).<br />
For more information about the OAuth authorization framework please visit the official [https://oauth.net/2/ OAuth site]. For technical details you could also check the [https://tools.ietf.org/html/rfc6749| OAuth 2.0 RFC]. In the following, the needed steps to authorize third party applications to operate on user's behalf and the D4Science infrastructure are explained. <br />
<br />
== Step 1 - Configure your application ==<br />
Authorized third party applications must be registered on the Infrastructure. Up to now the application registration is not automatic but you need to open a ticket at https://support.d4science.org and select 3rd Party App Registration.<br />
<br />
The ticket needs to have '''3rd Party Application Support''' as tracker. The application name as well as the redirect uri need to be provided (https is strongly suggested). This url must be absolute, and without arguments. Once the application is registered, it will have associated:<br />
<br />
* a unique client identifier (client_id);<br />
* a client secret which must be kept secret, as the name itself says (client_secret).<br />
<br />
<source lang="xml"><br />
client_id: ab9d32c9-9425-42f1-99d7-50d68e651e9c<br />
client_secret: y9r93zDv4D<br />
</source><br />
<br />
Both are needed to proceed with the OAuth 2.0 workflow.<br />
<br />
== Step 2 - Require an authorization code ==<br />
Once the application is registered and configured, an authorization code can be requested. This code is not the final one needed to perform operations on user behalf, but is the one that will be exchanged for that in the next step. To require this code that has a very short time to live, the application must invoke the authorization service, which is gateway dependant.<br />
<br />
=== Retrieving the authorization endpoint location===<br />
<br />
The previous authorization endpoint location is portal's gateway dependant and '''can be found in Table 1''' or retrieved dinamically by using the registry service REST-API hosted at <br />
<br />
''https://registry.d4science.org/icproxy/gcube/service/'' <br />
<br />
'''Get authorization endpoint location in the following Table 1'''<br />
<br />
{| class="wikitable"<br />
|+Table 1: authorization endpoints location depending on the D4Science Gateway<br />
! style="font-weight: bold;" | D4Science Gateway name<br />
! style="font-weight: bold;" | Authorization endpoint<br />
|-<br />
| ARIADNEplus Gateway<br />
| https://ariadne.d4science.org/group/ariadneplus-gateway/authorization<br />
|-<br />
| iMarine Gateway<br />
| https://i-marine.d4science.org/group/imarine-gateway/authorization<br />
|-<br />
| AGINFRA+ Gateway<br />
| https://aginfra.d4science.org/group/aginfra-gateway/authorization<br />
|-<br />
| BlueBRIDGE Gateway<br />
| https://bluebridge.d4science.org/group/bluebridge-gateway/authorization<br />
|-<br />
| D4Science.org Gateway<br />
| https://services.d4science.org/group/d4science-services-gateway/authorization<br />
|-<br />
| DESCRAMBLE Gateway<br />
| https://descramble.d4science.org/group/descramble-gateway/authorization<br />
|-<br />
| GEMex Gateway<br />
| https://gemex.d4science.org/group/gemex-gateway/authorization<br />
|-<br />
| OpenAIRE-Connect Gateway<br />
| https://openaire-connect.d4science.org/group/openaire-connect-gateway/authorization<br />
|-<br />
| PARTHENOS Gateway<br />
| https://parthenos.d4science.org/group/parthenos-gateway/authorization<br />
|-<br />
| SoBigData Gateway<br />
| https://sobigdata.d4science.org/group/sobigdata-gateway/authorization<br />
|}<br />
<br />
'''Retrieve the authorization endpoint location dynamically (ADVANCED MODE)'''<br />
<br />
Registry Swagger REST-API avaiable at: https://dev.d4science.org/swagger/registry/<br />
<br />
Which requires an [https://dev.d4science.org/authorization/ authorization token]. The returned profile is something like<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<serviceEndpoints><br />
<Resource version="0.4.x"><br />
<ID>b1c5851f-1d47-4c60-8906-941ab859a334</ID><br />
<Type>RuntimeResource</Type><br />
<Scopes><br />
<Scope>/gcube</Scope><br />
</Scopes><br />
...<br />
<AccessPoint><br />
<Description>AuthorisationServer where to authenticate external SmartGears services oAuth like</Description><br />
<Interface><br />
<Endpoint EntryName="AuthorisationServer">https://next.d4science.org/group/next/authorization</Endpoint><br />
</Interface><br />
</AccessPoint><br />
</Profile><br />
</Resource><br />
</serviceEndpoints><br />
</source><br />
<br />
=== Invoking the authorization service===<br />
<br />
Once you got the AuthorisationServer endpoint and the call is executed, two things can happen:<br />
* the user is requested to perform the login, thus accepting the application operation and granting it;<br />
* if there is a valid existing permission grant for the application, the login operation is by-passed and the redirect_uri is followed.<br />
<br />
The request is an '''HTTP GET''' call, that needs the following parameters to be provided<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| client_id<br />
| the client identifier associated to the application<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the redirect uri provided at registration phase<br />
| yes<br />
|-<br />
| state<br />
| a unique string value hard to guess, needed to prevent CRSF attacks (e.g. AAedBEeF545sdfKef897)<br />
| yes<br />
|-<br />
| scope<br />
| this field is optional. It can be specified to request (with the next step) an access token valid for a given<br />
infrastructure scope. If missing, the scope will be choose by the user on credentials insertion.<br />
| no<br />
|}<br />
<br />
This is an example in development environment<br />
<pre><br />
https://next.d4science.org/group/next/authorization?client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&scope=%2Fgcube&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&state=7d12bf13-111c-4f46-ab06-9e9e08ad377b<br />
</pre><br />
<br />
==== End user Experience ====<br />
<br />
Once redirected, the user will be presented with D4Science's authentication dialog box. This identifies your application as well as outlines the particular member permissions that your application requests. The logo and application name can be configurable, in the example the application name is '''Application X'''<br />
<br />
[[File:Oauth-ux.jpg]]<br />
<br />
==== Application is Accepted ====<br />
<br />
If the user grants permissions to the application, a redirection to the redirect_uri is performed. Attached to the redirect_uri, two important url arguments are provided<br />
<br />
* code: which is the authorization code. It expires in 10 seconds;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
==== Application is Rejected ====<br />
<br />
If, for any reason, the request fails, the redirect_uri is still followed but the url has attached the following parameters<br />
* error: a code that states the type of error;<br />
* error_description: a textual description of the actual error;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
== Step 3 - Exchange Authorization Code for an Access Token ==<br />
This is the step in which the previously obtained code can be exchanged to obtain a real access token. <br />
This time, an '''HTTP POST''' method needs to be performed, with a '''Content-Type''' of type '''application/x-www-form-urlencoded'''.<br />
<br />
The client_id and secret must be passed as the basic auth header Base64 encoded, see the example in Java below:<br />
<br />
e.g.<br />
<source lang="java"><br />
...<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
..<br />
</source><br />
<br />
The request-body must contain the following parameters:<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| grant_type<br />
| The value of this field should always be authorization_code<br />
| yes<br />
|-<br />
| code<br />
| the authorization code you received from the previous step.<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the same 'redirect_uri' value that you<br />
passed in the previous step.<br />
| yes<br />
|}<br />
<br />
For instance, in development environment<br />
<br />
<source lang="xml"><br />
POST /gcube-oauth/v2/access-token HTTP/1.1<br />
Host: socialnetworking-d-d4s.d4science.org<br />
Content-Type: application/x-www-form-urlencoded<br />
<br />
grant_type=authorization_code&code=...&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&client_secret=...<br />
</source><br />
<br />
In case of success, the response will contain a JSON object, having these properties:<br />
<br />
* access_token: the obtained access token to use to perform operations on user's behalf;<br />
* expires_in: the time to live of the token expressed in seconds;<br />
* scope: in which scope the token is valid;<br />
* token_type: that is "Bearer".<br />
<br />
In case of failures, instead, the returned object will contain:<br />
<br />
* error : an error string according the oauth 2.0 RFC.<br />
* error_description : a text description for the error.<br />
<br />
=== D4Science oAuth2 endpoint location === <br />
In the '''production''' environment, its value is<br />
<source lang="xml"><br />
https://api.d4science.org/oauth/v2/access-token<br />
</source><br />
<br />
'''HTTP POST Example in JAVA'''<br />
<br />
<source lang="java"><br />
String ACCESS_TOKEN_ATTR = "access_token";<br />
String SCOPE_ATTR = "scope";<br />
String fullPath2oAuthService = "https://api.d4science.org/oauth/v2/access-token";<br />
try {<br />
CloseableHttpClient httpClient = HttpClientBuilder.create().setRedirectStrategy(new LaxRedirectStrategy()).build();<br />
HttpPost httpPostRequest = new HttpPost(fullPath2oAuthService);<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
httpPostRequest.addHeader("Content-Type", "application/x-www-form-urlencoded");<br />
List<NameValuePair> params = new ArrayList<NameValuePair>();<br />
params.add(new BasicNameValuePair("grant_type", "authorization_code"));<br />
params.add(new BasicNameValuePair("code", tempCode));<br />
params.add(new BasicNameValuePair("redirect_uri", "https://www.gcube-system.org/oauth-authentication-example/example-app"));<br />
httpPostRequest.setEntity(new UrlEncodedFormEntity(params));<br />
<br />
CloseableHttpResponse httpresponse = httpClient.execute(httpPostRequest);<br />
int responseCode = httpresponse.getStatusLine().getStatusCode();<br />
if (responseCode < 200 || responseCode >= 300) {<br />
System.out.println("Error while getting authorization code, response status line from "<br />
+ fullPath2oAuthService + " was: " + responseCode); <br />
}<br />
JSONParser parser = new JSONParser();<br />
JSONObject json = (JSONObject) parser.parse(EntityUtils.toString(httpresponse.getEntity()));<br />
accessToken = json.get(ACCESS_TOKEN_ATTR).toString();<br />
scope = json.get(SCOPE_ATTR).toString();<br />
} catch(Exception e){<br />
System.out.println("Failed to perform request");<br />
e.printStackTrace();<br />
<br />
}<br />
</source><br />
<br />
== Step 4 - Get member data ==<br />
Having an access token, your application can perform some operations on behalf of the user. For instance, it can retrieve user's information profile by exploiting the [https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface social networking web service].<br />
<br />
Its current production endpoint is: https://api.d4science.org/rest/<br />
<br />
The subservice to call, once the social networking service endpoint has been retrieved, is this one<br />
<br />
<source lang="xml"><br />
2/users/get-oauth-profile<br />
</source><br />
<br />
It requires a ''gcube-token'' as query parameter, whose value is exactly the access token retrieved in the previous step. The result is something like this<br />
<br />
<source lang="ruby"><br />
{<br />
id: ".....",<br />
roles: [ ], // roles in the given-token context, e.g. VRE-Manager<br />
picture: "....",<br />
name: "....",<br />
middle_name: "....",<br />
male: as boolean ,<br />
location_industry: "....",<br />
given_name: "....",<br />
email: "....",<br />
job_title: "....",<br />
family_name: "....",<br />
verified_email: as boolean <br />
}<br />
</source><br />
<br />
== See Also ==<br />
<br />
[https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface Social Networking Service]<br />
<br />
[https://dev.d4science.org/swagger/social-networking/ Social Networking Service Docs SWAGGER]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=GCube_Portal_Installation&diff=33270GCube Portal Installation2020-02-12T14:40:06Z<p>Massimiliano.assante: /* Install the gCube Infrastructure Gateway Bundle */</p>
<hr />
<div>[[Category:Administrator's Guide]]<br />
gCube adopts Liferay Portal as enabling portal technology for its Infrastructure Gateway. gCube portal is built over [https://sourceforge.net/projects/lportal/files/Liferay%20Portal/6.2.5%20GA6/ Liferay Portal 6.2 CE GA6 (Newton / Build 6205 / January 6, 2016)]<br />
<br />
=gCube Infrastructure Gateway 4.7.0-SNAPSHOT (February 2020)=<br />
<br />
Starting from 3.0 release, the gCube Infrastructure Gateway has been mavenized, this make installing it '''a very simple process''' and takes '''less than five minutes''' to complete. We've been '''working hard''' to make this process as '''easiest and quicker as possible'''. <br />
<br />
Supported Operating Systems are Linux and MacOSX. With Java8 installed.<br />
<br />
[[File:Gcube4.0-gateway.png]]<br />
<br />
<br />
==Install the gCube Infrastructure Gateway Bundle==<br />
'''[https://code-repo.d4science.org/gCubeSystem/gcube-portal-bundle gCube Infrastructure Gateway]''' is bundled with [https://tomcat.apache.org/tomcat-7.0-doc/index.html Apache Tomcat 7] and the latest stable relase of [https://gcube.wiki.gcube-system.org/gcube/index.php/Featherweight_Stack gCube Featherweight Stack Distribution], [http://gcube.wiki.gcube-system.org/gcube/index.php/Social_Networking_Library gCube Social Networking Library] and [http://gcube.wiki.gcube-system.org/gcube/index.php/Home_Library gCube Home Library]<br />
<br />
All you need to do is the following:<br />
* Download the latest version of the Portal-Bundle from maven, make sure you get the latest snapshot from our Nexus repo: [http://maven.research-infrastructures.eu/nexus/index.html#nexus-search;quick~gcube-portal-bundle gcube-portal-bundle SNAPSHOT]<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>gcube-portal-bundle</artifactId><br />
<version>LATEST</version><br />
<type>tar.gz</type><br />
</dependency><br />
</source><br />
* Extract the gCube Infrastructure Gateway bundle tar.gz into a folder on your machine<br />
* Add to the ~/.bashrc file the following:<br />
** clean CLASSPATH: "export CLASSPATH="<br />
** add CATALINA_HOME env. variable (the home directory of tomcat)<br />
** add JAVA_HOME env. variable (the home directory of java)<br />
** add JRE_HOME env. variable (JAVA_HOME/jre)<br />
** add CATALINA_OPTS env. variable "export CATALINA_OPTS="-Dexternal-properties=portal-developer.properties -Xmx1000m -Xms1000m -Dlog4j.ignoreTCL=true -Dlog4j.configuration=file://$CATALINA_HOME/portal-log4j.properties -Dlogback.configurationFile=file://$CATALINA_HOME/portal-logback.xml"<br />
** add CATALINA_PID env. variable "export CATALINA_PID=~/pid.txt"<br />
<br />
''' IMPORTANT '''<br />
The CATALINA_OPTS env variable: '''-Dexternal-properties=portal-developer.properties''' optimizes Liferay Portal for development by removing all caches and thus making sure that any change made by the developer is visible in the website as soon as technically possible. One of the final goals is to avoid rebooting the server as much as possible. Further Information at: [https://web.liferay.com/community/wiki/-/wiki/Main/Liferay+Developer+Mode LiferayDeveloper Mode Doc]<br />
<br />
After that, say you created a "Portal-Bundle" folder on your user's home(~) and extracted the org.gcube.portal.portal-bundle-2+ in there, the final part of your ~/.bashrc file may look like the following:<br />
<source lang="xml"><br />
export JAVA_HOME=/usr/lib/jvm/java-7-oracle/<br />
export JRE_HOME=$JAVA_HOME/jre<br />
export CATALINA_HOME=~/Portal-Bundle/tomcat-7.0.62<br />
export CATALINA_BASE=$CATALINA_HOME<br />
export CATALINA_TMPDIR=$CATALINA_HOME/temp<br />
<br />
export CLASSPATH=<br />
export CATALINA_OPTS="-Dexternal-properties=portal-developer.properties -Xmx1000m -Xms1000m -Djava.protocol.handler.pkgs=com.sun.net.ssl.internal.www.protocol -Dlog4j.ignoreTCL=true -Dlog4j.configuration=file://$CATALINA_HOME/portal-log4j.properties"<br />
<br />
export CATALINA_PID=~/pid.txt<br />
</source><br />
<br />
==First startup of gCube Infrastructure Gateway==<br />
<br />
'''gCube Infrastructure Gateway enabling technology (Liferay)''' comes with a default database called HSQL or "hypersonic." '''This is not meant for production use!'''' <br />
You should switch to a real database to use the gCube Infrastructure Gateway. For further information see Section Configuring gCube Infrastructure Gateway database section in the following of this page.<br />
<br />
However, if you want to startup gCube Infrastructure Gateway using the default database just open a shell and enter the following command: <br />
<pre><br />
source ~/.bashrc<br />
</pre><br />
and start tomcat<br />
<pre><br />
$CATALINA_HOME/bin/startup.sh<br />
</pre> <br />
<br />
'''gCube Infrastructure Gateway take less than a minute to initialize''', in the meanwhile you might want to '''check the portal logs''', to do so, enter the command: <br />
<pre><br />
tail -f $CATALINA_HOME/logs/catalina.out<br />
</pre><br />
open your browser and go to http://localhost:8080<br />
if you performed all of the above operations correctly, you should be presented something like the following <br />
[[File:Gcube4.0-bundle-1.png | screenshot]]<br />
<br />
all you need to do now is '''follow the instructions on the screen''', a wizard application will guide you through the [https://wiki.gcube-system.org/gcube/GCube_Portal_Installation#Staging_procedure Staging procedure]. '''Thank you for using [http://www.gcube-system.org gCube]'''.<br />
<br />
== Starting/Stopping a gCube Infrastructure Gateway ==<br />
In order to start / stop tomcat, you should execute the following commands:<br />
* source the configured ~/.bashrc<br />
* $CATALINA_HOME/bin/startup.sh<br />
* $CATALINA_HOME/bin/shutdown -force<br />
<br />
'''please note''': If you change gCore's configuration or Tomcat's settings or Liferay portal.ext.properties file, you should restart tomcat for the changes to be applied.<br />
<br />
==Additional Information==<br />
<br />
=== Setting the Infrastructure ===<br />
<br />
By Default the bundle is configured to run on the gCube Development infrastructure. However if you want to change infrastructure please edit this file under $CATALINA_HOME/conf/infrastructure.properties<br />
<br />
=== Changing Company web id ===<br />
<br />
By default, gCube Infrastructure Gateway starts with "liferay.com" as default company web id, this means, among other things, that at login time portal will suggest email addresses ending with @liferay.com. If you want to configure the portal to use your own Company web id just add the following property in portal.ext.properties file located under $CATALINA_HOME/webapps/ROOT/WEB-INF/classes. <br />
<br />
Replace <MY_COMPANY_WEB_ID> with your own company web id, e.g. google.com<br />
<br />
<pre><br />
#company web id name<br />
company.default.web.id=<MY_COMPANY_WEB_ID><br />
</pre><br />
<br />
'''please note 1''': This change must be performed '''before starting up your portal for the first time'''. <br />
<br />
'''please note 2''': This change '''will not work''' if you use the default HSQL DB (Hypersonic)<br />
<br />
=== Configuring gCube Infrastructure Gateway database ===<br />
<br />
If you want to configure the portal to use a different database than the default HSQL DB one (and '''you are encouraged to do so''') you should add some properties in portal.ext.properties file located under $CATALINA_HOME/webapps/ROOT/WEB-INF/classes.<br />
Please refer to the Liferay [http://www.liferay.com/community/wiki/-/wiki/Main/Database+Portal+Properties Database Portal Properties] to know which type of database is supported.<br />
<br />
Configuration examples: (replace <YOUR_DB_NAME>, <YOUR_DB_USER>, <YOUR_DB_PASSWORD> with actual values<br />
* '''PostgreSQL'''<br />
<pre><br />
#<br />
# PostgreSQL<br />
#<br />
jdbc.default.driverClassName=org.postgresql.Driver<br />
jdbc.default.url=jdbc\:postgresql\://localhost\:5432/<YOUR_DB_NAME><br />
jdbc.default.username=<YOUR_DB_USER><br />
jdbc.default.password=<YOUR_DB_PASSWORD> <br />
</pre><br />
* '''MySQL'''<br />
<pre><br />
#<br />
# MySQL<br />
#<br />
jdbc.default.driverClassName=com.mysql.jdbc.Driver<br />
jdbc.default.url=jdbc:mysql://localhost/<YOUR_DB_NAME>?useUnicode=true&characterEncoding=UTF-8&useFastDateParsing=false<br />
jdbc.default.username=<YOUR_DB_USER><br />
jdbc.default.password=<YOUR_DB_PASSWORD><br />
</pre><br />
<br />
The above configuration are known to be working with gCube Infrastructure Gateway.<br />
<br />
== Troubleshooting ==<br />
<br />
* If you get something like the following exception at portal startup, just cleanup your browser cookies.<br />
<br />
<pre><br />
ERROR [CompanyThreadLocal:54] com.liferay.portal.NoSuchCompanyException: No Company exists with the primary key 1<br />
com.liferay.portal.NoSuchCompanyException: No Company exists with the primary key 1<br />
at com.liferay.portal.service.persistence.CompanyPersistenceImpl.findByPrimaryKey(CompanyPersistenceImpl.java:436)<br />
at com.liferay.portal.service.base.CompanyLocalServiceBaseImpl.getCompany(CompanyLocalServiceBaseImpl.java:325)<br />
at sun.reflect.GeneratedMethodAccessor515.invoke(Unknown Source)<br />
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)<br />
at java.lang.reflect.Method.invoke(Method.java:597)<br />
.....<br />
</pre><br />
<br />
* If the Workspace Portlet shipped within this Bundle popups an error on startup you need to Install the Encryption key and add the Encryption key in your classpath to access runtime resources.<br />
<br />
== Staging procedure ==<br />
<br />
'''Staging is a revised feature bundled with gCube Infrastructure Gateway 4.0''' When you install the portal for the first time a Wizard Application will guide you through the installation configuration and will setup everything's needed for you.<br />
<br />
Follow the standard [https://dev.liferay.com/discover/deployment/-/knowledge_base/6-2/using-liferays-setup-wizard Liferay Wizard Procedure] until entering your desired password for your user. After that you will be presented the gCube Portal Staging procedure as displayed in the screenshot below.<br />
<br />
[[File:Gcube4.0-bundle-5.png]]<br />
<br />
Click "Begin Install" button to make the gCube Staging installation procedure start.<br />
<br />
The whole procedure consist of 3 steps, as shown in the figure below:<br />
* Infrastructure Selection<br />
* Virtual Research Environments check-up<br />
* Final step<br />
<br />
Provide the information requested, if your not sure about the settings you should read more about gCube on this [http://wiki.gcube-system.org wiki]. The next steps are quite handy and we think no further documentation is needed, however feel free to ask for support if needed.<br />
<br />
[[File:Gcube4.0-bundle-staging.png]]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=GCube_Portal_Installation&diff=33269GCube Portal Installation2020-02-12T14:38:22Z<p>Massimiliano.assante: /* gCube Infrastructure Gateway 4.4.0.1-SNAPSHOT (February 2018) */</p>
<hr />
<div>[[Category:Administrator's Guide]]<br />
gCube adopts Liferay Portal as enabling portal technology for its Infrastructure Gateway. gCube portal is built over [https://sourceforge.net/projects/lportal/files/Liferay%20Portal/6.2.5%20GA6/ Liferay Portal 6.2 CE GA6 (Newton / Build 6205 / January 6, 2016)]<br />
<br />
=gCube Infrastructure Gateway 4.7.0-SNAPSHOT (February 2020)=<br />
<br />
Starting from 3.0 release, the gCube Infrastructure Gateway has been mavenized, this make installing it '''a very simple process''' and takes '''less than five minutes''' to complete. We've been '''working hard''' to make this process as '''easiest and quicker as possible'''. <br />
<br />
Supported Operating Systems are Linux and MacOSX. With Java8 installed.<br />
<br />
[[File:Gcube4.0-gateway.png]]<br />
<br />
<br />
==Install the gCube Infrastructure Gateway Bundle==<br />
'''gCube Infrastructure Gateway 4.7.0_SNAPSHOT''' is bundled with [https://tomcat.apache.org/tomcat-7.0-doc/index.html Apache Tomcat 7] and the latest stable relase of [https://gcube.wiki.gcube-system.org/gcube/index.php/Featherweight_Stack gCube Featherweight Stack Distribution], [http://gcube.wiki.gcube-system.org/gcube/index.php/Social_Networking_Library gCube Social Networking Library] and [http://gcube.wiki.gcube-system.org/gcube/index.php/Home_Library gCube Home Library]<br />
<br />
All you need to do is the following:<br />
* Download the latest version of the Portal-Bundle from maven, make sure you get the latest snapshot from our Nexus repo: [http://maven.research-infrastructures.eu/nexus/index.html#nexus-search;quick~gcube-portal-bundle gcube-portal-bundle SNAPSHOT]<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>gcube-portal-bundle</artifactId><br />
<version>LATEST</version><br />
<type>tar.gz</type><br />
</dependency><br />
</source><br />
* Extract the gCube Infrastructure Gateway bundle tar.gz into a folder on your machine<br />
* Add to the ~/.bashrc file the following:<br />
** clean CLASSPATH: "export CLASSPATH="<br />
** add CATALINA_HOME env. variable (the home directory of tomcat)<br />
** add JAVA_HOME env. variable (the home directory of java)<br />
** add JRE_HOME env. variable (JAVA_HOME/jre)<br />
** add CATALINA_OPTS env. variable "export CATALINA_OPTS="-Dexternal-properties=portal-developer.properties -Xmx1000m -Xms1000m -Dlog4j.ignoreTCL=true -Dlog4j.configuration=file://$CATALINA_HOME/portal-log4j.properties -Dlogback.configurationFile=file://$CATALINA_HOME/portal-logback.xml"<br />
** add CATALINA_PID env. variable "export CATALINA_PID=~/pid.txt"<br />
<br />
''' IMPORTANT '''<br />
The CATALINA_OPTS env variable: '''-Dexternal-properties=portal-developer.properties''' optimizes Liferay Portal for development by removing all caches and thus making sure that any change made by the developer is visible in the website as soon as technically possible. One of the final goals is to avoid rebooting the server as much as possible. Further Information at: [https://web.liferay.com/community/wiki/-/wiki/Main/Liferay+Developer+Mode LiferayDeveloper Mode Doc]<br />
<br />
After that, say you created a "Portal-Bundle" folder on your user's home(~) and extracted the org.gcube.portal.portal-bundle-2+ in there, the final part of your ~/.bashrc file may look like the following:<br />
<source lang="xml"><br />
export JAVA_HOME=/usr/lib/jvm/java-7-oracle/<br />
export JRE_HOME=$JAVA_HOME/jre<br />
export CATALINA_HOME=~/Portal-Bundle/tomcat-7.0.62<br />
export CATALINA_BASE=$CATALINA_HOME<br />
export CATALINA_TMPDIR=$CATALINA_HOME/temp<br />
<br />
export CLASSPATH=<br />
export CATALINA_OPTS="-Dexternal-properties=portal-developer.properties -Xmx1000m -Xms1000m -Djava.protocol.handler.pkgs=com.sun.net.ssl.internal.www.protocol -Dlog4j.ignoreTCL=true -Dlog4j.configuration=file://$CATALINA_HOME/portal-log4j.properties"<br />
<br />
export CATALINA_PID=~/pid.txt<br />
</source><br />
<br />
==First startup of gCube Infrastructure Gateway==<br />
<br />
'''gCube Infrastructure Gateway enabling technology (Liferay)''' comes with a default database called HSQL or "hypersonic." '''This is not meant for production use!'''' <br />
You should switch to a real database to use the gCube Infrastructure Gateway. For further information see Section Configuring gCube Infrastructure Gateway database section in the following of this page.<br />
<br />
However, if you want to startup gCube Infrastructure Gateway using the default database just open a shell and enter the following command: <br />
<pre><br />
source ~/.bashrc<br />
</pre><br />
and start tomcat<br />
<pre><br />
$CATALINA_HOME/bin/startup.sh<br />
</pre> <br />
<br />
'''gCube Infrastructure Gateway take less than a minute to initialize''', in the meanwhile you might want to '''check the portal logs''', to do so, enter the command: <br />
<pre><br />
tail -f $CATALINA_HOME/logs/catalina.out<br />
</pre><br />
open your browser and go to http://localhost:8080<br />
if you performed all of the above operations correctly, you should be presented something like the following <br />
[[File:Gcube4.0-bundle-1.png | screenshot]]<br />
<br />
all you need to do now is '''follow the instructions on the screen''', a wizard application will guide you through the [https://wiki.gcube-system.org/gcube/GCube_Portal_Installation#Staging_procedure Staging procedure]. '''Thank you for using [http://www.gcube-system.org gCube]'''.<br />
<br />
== Starting/Stopping a gCube Infrastructure Gateway ==<br />
In order to start / stop tomcat, you should execute the following commands:<br />
* source the configured ~/.bashrc<br />
* $CATALINA_HOME/bin/startup.sh<br />
* $CATALINA_HOME/bin/shutdown -force<br />
<br />
'''please note''': If you change gCore's configuration or Tomcat's settings or Liferay portal.ext.properties file, you should restart tomcat for the changes to be applied.<br />
<br />
==Additional Information==<br />
<br />
=== Setting the Infrastructure ===<br />
<br />
By Default the bundle is configured to run on the gCube Development infrastructure. However if you want to change infrastructure please edit this file under $CATALINA_HOME/conf/infrastructure.properties<br />
<br />
=== Changing Company web id ===<br />
<br />
By default, gCube Infrastructure Gateway starts with "liferay.com" as default company web id, this means, among other things, that at login time portal will suggest email addresses ending with @liferay.com. If you want to configure the portal to use your own Company web id just add the following property in portal.ext.properties file located under $CATALINA_HOME/webapps/ROOT/WEB-INF/classes. <br />
<br />
Replace <MY_COMPANY_WEB_ID> with your own company web id, e.g. google.com<br />
<br />
<pre><br />
#company web id name<br />
company.default.web.id=<MY_COMPANY_WEB_ID><br />
</pre><br />
<br />
'''please note 1''': This change must be performed '''before starting up your portal for the first time'''. <br />
<br />
'''please note 2''': This change '''will not work''' if you use the default HSQL DB (Hypersonic)<br />
<br />
=== Configuring gCube Infrastructure Gateway database ===<br />
<br />
If you want to configure the portal to use a different database than the default HSQL DB one (and '''you are encouraged to do so''') you should add some properties in portal.ext.properties file located under $CATALINA_HOME/webapps/ROOT/WEB-INF/classes.<br />
Please refer to the Liferay [http://www.liferay.com/community/wiki/-/wiki/Main/Database+Portal+Properties Database Portal Properties] to know which type of database is supported.<br />
<br />
Configuration examples: (replace <YOUR_DB_NAME>, <YOUR_DB_USER>, <YOUR_DB_PASSWORD> with actual values<br />
* '''PostgreSQL'''<br />
<pre><br />
#<br />
# PostgreSQL<br />
#<br />
jdbc.default.driverClassName=org.postgresql.Driver<br />
jdbc.default.url=jdbc\:postgresql\://localhost\:5432/<YOUR_DB_NAME><br />
jdbc.default.username=<YOUR_DB_USER><br />
jdbc.default.password=<YOUR_DB_PASSWORD> <br />
</pre><br />
* '''MySQL'''<br />
<pre><br />
#<br />
# MySQL<br />
#<br />
jdbc.default.driverClassName=com.mysql.jdbc.Driver<br />
jdbc.default.url=jdbc:mysql://localhost/<YOUR_DB_NAME>?useUnicode=true&characterEncoding=UTF-8&useFastDateParsing=false<br />
jdbc.default.username=<YOUR_DB_USER><br />
jdbc.default.password=<YOUR_DB_PASSWORD><br />
</pre><br />
<br />
The above configuration are known to be working with gCube Infrastructure Gateway.<br />
<br />
== Troubleshooting ==<br />
<br />
* If you get something like the following exception at portal startup, just cleanup your browser cookies.<br />
<br />
<pre><br />
ERROR [CompanyThreadLocal:54] com.liferay.portal.NoSuchCompanyException: No Company exists with the primary key 1<br />
com.liferay.portal.NoSuchCompanyException: No Company exists with the primary key 1<br />
at com.liferay.portal.service.persistence.CompanyPersistenceImpl.findByPrimaryKey(CompanyPersistenceImpl.java:436)<br />
at com.liferay.portal.service.base.CompanyLocalServiceBaseImpl.getCompany(CompanyLocalServiceBaseImpl.java:325)<br />
at sun.reflect.GeneratedMethodAccessor515.invoke(Unknown Source)<br />
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)<br />
at java.lang.reflect.Method.invoke(Method.java:597)<br />
.....<br />
</pre><br />
<br />
* If the Workspace Portlet shipped within this Bundle popups an error on startup you need to Install the Encryption key and add the Encryption key in your classpath to access runtime resources.<br />
<br />
== Staging procedure ==<br />
<br />
'''Staging is a revised feature bundled with gCube Infrastructure Gateway 4.0''' When you install the portal for the first time a Wizard Application will guide you through the installation configuration and will setup everything's needed for you.<br />
<br />
Follow the standard [https://dev.liferay.com/discover/deployment/-/knowledge_base/6-2/using-liferays-setup-wizard Liferay Wizard Procedure] until entering your desired password for your user. After that you will be presented the gCube Portal Staging procedure as displayed in the screenshot below.<br />
<br />
[[File:Gcube4.0-bundle-5.png]]<br />
<br />
Click "Begin Install" button to make the gCube Staging installation procedure start.<br />
<br />
The whole procedure consist of 3 steps, as shown in the figure below:<br />
* Infrastructure Selection<br />
* Virtual Research Environments check-up<br />
* Final step<br />
<br />
Provide the information requested, if your not sure about the settings you should read more about gCube on this [http://wiki.gcube-system.org wiki]. The next steps are quite handy and we think no further documentation is needed, however feel free to ask for support if needed.<br />
<br />
[[File:Gcube4.0-bundle-staging.png]]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=UserManagement_Core&diff=33267UserManagement Core2020-02-05T11:39:26Z<p>Massimiliano.assante: /* Maven Dependency */</p>
<hr />
<div>==User Management==<br />
=== Introduction ===<br />
User Management is a library that sits between the Liferay portal services and gCube portlets. The main functionality of the Library is to provide management of users, roles and organizations. It provides a layer of abstraction above the liferay services, hiding the configuration complexities of roles and organizations. The gcube requirement of roles in the scope of organization is achieved by creating a role with the organization name suffixed to it. The addition/removal of organization names to the role names are handled by the Library and are transparent to the application and portlets using the library. <br />
<br />
=== Interfaces ===<br />
The library exposes three interfaces:<br />
* User Manager<br />
* Role and Team* Manager<br />
* Group Manager<br />
<br />
(*)The notion of a Team is somewhat similar to a Role but a Role is a portal wide entity (Role exists in any VRE) while a Team is restricted to a particular VRE/VO.<br />
<br />
These interfaces provide the functionalities necessary for CRUD operation on Users, Roles and Groups and additional functionalities that organise users into Teams and assigning/dismissal of roles.<br />
<br />
=== Maven Dependency and Code repository===<br />
<br />
Add the following dependency to your maven project to use the library.<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.dvos</groupId><br />
<artifactId>usermanagement-core</artifactId><br />
<version>[2.0.0-SNAPSHOT, 3.0.0-SNAPSHOT)</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
'''Please find this Java Library Open Source code''' at [https://code-repo.d4science.org/gCubeSystem/usermanagement-core https://code-repo.d4science.org/gCubeSystem/usermanagement-core]<br />
<br />
=== UserManager APIs ===<br />
<br />
The UserManager interface allows to create, retrieve and delete users. In order to perform such operations you need to instanciate the proper manager. See the below example.<br />
<br />
<source lang="java"><br />
// Instanciate the manager<br />
UserManager userManager = new LiferayUserManager();<br />
<br />
//you want to retrieve information about an existing user<br />
GCubeUser userByMail = userManager.getUserByEmail(validUserMail);<br />
<br />
</source><br />
<br />
The UserManager offers a lot of other utilities. They are reported below:<br />
<br />
* get user information such as email, id, username, professional information, fullname and much more;<br />
* list all the users (also by range: int start, int end);<br />
* list users belonging to a given group (also by range: int start, int end);;<br />
* remove users;<br />
* remove users from groups;<br />
* manage users' registrations and membership requests.<br />
<br />
==== Listing Users by groupId and range ====<br />
<br />
It is possible to define a range of all the users. Useful when paginating results and avoid to fetch them all.<br />
<br />
<source lang="java"><br />
<br />
LiferayUserManager#listUsersByGroup(long groupId, int start, int end) // start is the lower bound of the range of users, while end is the upper bound of the range of users (not inclusive). Returns the range of users<br />
<br />
LiferayUserManager#getGroupUsersCount(long groupId) //returns the total number of users in the given group / VRE<br />
<br />
</source><br />
<br />
==== Search Users by groupId and keywords ====<br />
<br />
It is possible to search all the users of a given groupId by the keywords (space separated), which may occur in the user's first name, middle name, last name, username, or email address. Useful when you are building an interface where a number of user accounts have to be selected (e.g. workspace share folder with).<br />
<br />
<source lang="java"><br />
LiferayUserManager#searchUsersByGroup(String keywords, long groupId) //all the users who match the keywords ordered by username (Ascendent order)<br />
</source><br />
<br />
==== Reading User Custom Attributes ====<br />
<br />
It is possible to define User's custom attributes and to Read and Update them. To do so the following methods should be used:<br />
<source lang="java"><br />
<br />
LiferayUserManager#readCustomAttr(long userId, String attributeKey) //which returns a java.io.Serializable<br />
<br />
LiferayUserManager#saveCustomAttr(long userId, String attributeKey, Serializable value)<br />
<br />
</source><br />
<br />
=== GroupManager APIs===<br />
<br />
The GroupManager interface allows to manage VRE, VO and ROOT VO of the gCube Infrastructure that are mapped on the Liferay's Site objects. It offers <br />
the APIs to create, delete and modify them. A gCube group is represented by a GCubeGroup class object. In order to perform such operations you need to instanciate the proper manager. See the below example. <br />
<br />
<source lang="java"><br />
// Instanciate the manager<br />
GroupManager groupManager = new LiferayGroupManager();<br />
<br />
// you can now retrieve the list of VREs, VOs or get the Root VO<br />
GCubeGroup rootVO = groupManager.getRootVO();<br />
<br />
// you can create a new VO (child of the root VO)<br />
GCubeGroup newVO = groupManager.createVO("newVoName", rootVO.getGroupId(), "VO description goes here");<br />
<br />
// you can now create a VRE (child of the just created VO)<br />
<br />
....<br />
<br />
// retrieve the groups to whom a given user belongs (given the user identifier)<br />
List<GCubeGroup> listOfGroups = groupManager.listGroupsByUser(userId);<br />
<br />
</source><br />
<br />
Among the other available functions, you can:<br />
* check if a given group is a VRE, VO or root VO;<br />
* retrieve the scope of the group;<br />
* retrieve the infrastructure scope;<br />
* manage virtual groups (i.e., a collection of groups).<br />
<br />
=== RoleManager APIs===<br />
The RoleManager interface defines the manager that manipulates Roles and Teams in the gCube Portals. Both Roles and Teams are assigned to people, but a Role is inter-VREs (so it needs to be created once but assigned to users on any VRE you wish that role to be used), whereas a Team is restricted to a VRE or VO. <br />
<br />
In other words the notion of a Team is somewhat similar to a Role but a Role is a portal wide entity (Role exists in any Site/VRE or VO) while a Team is restricted to a particular Site/VRE or VO. <br />
<br />
The library uses two different classes to abstract them: '''GCubeRole''' and '''GCubeTeam''' respectively. <br />
<br />
<br />
See the below example.<br />
<br />
<source lang="java"><br />
// instanciate the interface<br />
RoleManager roleManager = new LiferayRoleManager();<br />
<br />
// create a role<br />
boolean created = roleManager.createRole("role name", "role description");<br />
<br />
// retrieve the role's id by name<br />
long roleId = roleManager.getRoleIdByName("role name");<br />
<br />
// retrieve the object<br />
GCubeRole role = roleManager.getRole(roleId);<br />
<br />
// now you can assign this role to someone in a group (the group is necessary because that role must be assigned somewhere)<br />
roleManager.assignRoleToUser(userId, groupId, roleId);<br />
<br />
// list all roles<br />
List<GCubeRole> roles = roleManager.listAllRoles();<br />
<br />
// list roles by group (i.e., roles assigned to someone that belongs to that group)<br />
List<GCubeRoles> rolesByGroup = roleManager.listAllRolesByGroup(groupId);<br />
<br />
// list teams<br />
List<GCubeTeam> teams = roleManager.listAllTeams();<br />
<br />
// list teams by group<br />
List<GCubeTeam> teamsByGroup = roleManager.listAllTeamsByGroup(groupId);<br />
<br />
// assign a team to someone. There is no groupId here, because the teamId directly determines the group in which it was defined<br />
roleManager.assignTeamToUser(userId, teamId);<br />
<br />
</source><br />
<br />
You can also:<br />
* delete teams/roles;<br />
* remove a certain team/role to a user in a group;<br />
* update team/role definitions.</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=Statistical_Algorithms_Importer&diff=33243Statistical Algorithms Importer2020-01-20T11:11:33Z<p>Massimiliano.assante: /* Overview */</p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
In this guide we describe the Statistical Algorithms Importer (also known as '''Methods Importer''') web interface.<br />
<br />
== Overview ==<br />
This Methods Importer (SAI) is a tool to import algorithms in the D4Science e-Infrastructure. Currently, it supports different types of software integration. SAI separates software development from its deployment in the infrastructure in a very flexible way. After the first deployment, made in collaboration with the e-Infrastructure team, script developers can modify and update their scripts by themselves, without the intervention of the e-Infrastructure team.<br />
<br />
In order to transform an algorithm, three main passages are required: <br />
<br />
1 - Indicate Input, Output and types of a main script orchestrating the process<br />
<br />
2 - Create the Software: this operation creates the interface from the e-Infrastructure service to the script and should be used each time either the interface (I/O) or the name of the algorithm or the required additional packages change<br />
<br />
3 - Publish the Software: this operation communicates to the infrastructure that a newly create software should be put online<br />
<br />
Additionally, the Repackage function can be used when only the internal code of the orchestrating script changes and the algorithms has already been published.<br />
The pages in this Wiki explain the details of these operations. <br />
<br />
[[Image:StatisticalAlgorithmsImporter1.png|thumb|center|800px|Statistical Algorithms Importer (SAI), portlet. Main interface.]]<br />
<br />
== F.A.Q. ==<br />
Please, read our best practices first: [[Statistical Algorithms Importer: FAQ|F.A.Q.]]<br />
<br />
== Demonstration ==<br />
A demonstration video is available [http://data.d4science.org/SkhVR3AyTUNLaStCV2tNdUhsL2VIQ3AwMG1tTjVka3dHbWJQNStIS0N6Yz0 here].<br />
<br />
== Main Steps==<br />
<br />
# [[Statistical Algorithms Importer: Create Project|Creating a new Project]]<br />
# [[Statistical Algorithms Importer: Publish Algorithms|Publishing Algorithms for deployment]]<br />
# [[Statistical Algorithms Importer: Repackage| Repackaging a script]]<br />
# [[Advanced Input| Advanced Input ]]<br />
# [[Statistical Algorithms Importer: StatusUpdate| Updating the status of a computation]]<br />
# [[Statistical Algorithms Importer: FAQ|F.A.Q.]]<br />
# [[Statistical_Algorithms_Importer:_R_Project#Import_Resources_From_GitHub | Import projects from GitHub ]]<br />
<br />
==Related Links==<br />
* [[DataMiner_Manager|DataMiner Manager]]<br />
<br />
<!--<br />
==References==<br />
{{Reflist}} <br />
[[Template:Statistical Algorithms Importer]] <br />
--><br />
<br />
[[Category:Statistical Algorithms Importer]]<br />
[[Category:DataMiner]]<br />
[[Category:User's Guide]]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=Statistical_Algorithms_Importer&diff=33242Statistical Algorithms Importer2020-01-20T11:10:46Z<p>Massimiliano.assante: </p>
<hr />
<div>{| align="right"<br />
||__TOC__<br />
|}<br />
<br />
In this guide we describe the Statistical Algorithms Importer (also known as '''Methods Importer''') web interface.<br />
<br />
== Overview ==<br />
Statistical Algorithms Importer (SAI) is a tool to import algorithms in the D4Science e-Infrastructure. Currently, it supports different types of software integration. SAI separates software development from its deployment in the infrastructure in a very flexible way. After the first deployment, made in collaboration with the e-Infrastructure team, script developers can modify and update their scripts by themselves, without the intervention of the e-Infrastructure team.<br />
<br />
In order to transform an algorithm, three main passages are required: <br />
<br />
1 - Indicate Input, Output and types of a main script orchestrating the process<br />
<br />
2 - Create the Software: this operation creates the interface from the e-Infrastructure service to the script and should be used each time either the interface (I/O) or the name of the algorithm or the required additional packages change<br />
<br />
3 - Publish the Software: this operation communicates to the infrastructure that a newly create software should be put online<br />
<br />
Additionally, the Repackage function can be used when only the internal code of the orchestrating script changes and the algorithms has already been published.<br />
The pages in this Wiki explain the details of these operations. <br />
<br />
[[Image:StatisticalAlgorithmsImporter1.png|thumb|center|800px|Statistical Algorithms Importer (SAI), portlet. Main interface.]]<br />
<br />
== F.A.Q. ==<br />
Please, read our best practices first: [[Statistical Algorithms Importer: FAQ|F.A.Q.]]<br />
<br />
== Demonstration ==<br />
A demonstration video is available [http://data.d4science.org/SkhVR3AyTUNLaStCV2tNdUhsL2VIQ3AwMG1tTjVka3dHbWJQNStIS0N6Yz0 here].<br />
<br />
== Main Steps==<br />
<br />
# [[Statistical Algorithms Importer: Create Project|Creating a new Project]]<br />
# [[Statistical Algorithms Importer: Publish Algorithms|Publishing Algorithms for deployment]]<br />
# [[Statistical Algorithms Importer: Repackage| Repackaging a script]]<br />
# [[Advanced Input| Advanced Input ]]<br />
# [[Statistical Algorithms Importer: StatusUpdate| Updating the status of a computation]]<br />
# [[Statistical Algorithms Importer: FAQ|F.A.Q.]]<br />
# [[Statistical_Algorithms_Importer:_R_Project#Import_Resources_From_GitHub | Import projects from GitHub ]]<br />
<br />
==Related Links==<br />
* [[DataMiner_Manager|DataMiner Manager]]<br />
<br />
<!--<br />
==References==<br />
{{Reflist}} <br />
[[Template:Statistical Algorithms Importer]] <br />
--><br />
<br />
[[Category:Statistical Algorithms Importer]]<br />
[[Category:DataMiner]]<br />
[[Category:User's Guide]]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=Social_Networking_Service&diff=33093Social Networking Service2019-12-07T15:06:43Z<p>Massimiliano.assante: /* Discover the Web Service */</p>
<hr />
<div>{| align="right"<br />
| __TOC__<br />
|}<br />
<br />
The Social Networking Service is a RESTful application which exposes operations coming from both the [https://wiki.gcube-system.org/gcube/Social_Networking_Library gCube Social-Networking-Library] and the [https://wiki.gcube-system.org/gcube/UserManagement_Core User Management] components. The former is the bridge between the Cassandra Cluster (on which social data are stored) and the social applications, whereas the latter is the abstraction layer over the enabling technology of the Infrastructure portals, [https://www.liferay.com/ Liferay], in terms of: roles, users and organizations and how they are mapped to gCube concepts (VREs, Teams, and so on).<br />
<br />
The web services exposes a subset of such functionalities over '''https''' in a standard, reliable and secure way. <br />
<br />
See the vailable REST-API at a [https://dev.d4science.org/swagger/social-networking/ glance here.]<br />
= Discover the Web Service =<br />
For gCube releases older than 4.3, a Service Endpoint that has<br />
<br />
<source lang="java"><br />
Name = SocialNetworking<br />
Category = Portal<br />
</source><br />
<br />
mantains the base path of the service. Starting from gCube 4.3 version, the service runs on SmartGears, thus a gCore Endpoint needs to be searched having<br />
<br />
<source lang="java"><br />
Service Name = SocialNetworking<br />
Service Class = Portal<br />
</source><br />
<br />
The base path is the entry point of the web service. In the '''development''' environment, its current value is<br />
<source lang="xml"><br />
https://socialnetworking-d-d4s.d4science.org/social-networking-library-ws/rest<br />
</source><br />
<br />
In the '''production''' environment, its current value is<br />
<source lang="xml"><br />
https://api.d4sciende.org/rest<br />
</source><br />
<br />
This url is referred as ''BASE_URL''.<br />
<br />
= Service versions =<br />
Starting from gCube 4.3, the service exposes its methods using a more standard naming approach. Moreover, they accept (in case of http POST/PUT methods) JSON objects. The result of all methods is always a JSON object of this kind<br />
<br />
<source lang="java"><br />
{"success": false/true,<br />
"message": ...,<br />
"result": ...,<br />
}<br />
</source><br />
<br />
Where<br />
* success reports if the request succeeded or failed;<br />
* message is a status/error message that can be checked in case of errors (success equals false);<br />
* result is the current result object (it can be a list, a single object and so on depending on the invoked method). <br />
<br />
Inputs are automatically validated before the request is served.<br />
<br />
The older version (Version 1) instead has some limitation thus you are kindly suggested to no longer use it. However, in the following, its methods are going to be reported as well. The older version will be removed starting from the next gCube releases.<br />
<br />
= Authorization =<br />
No matter the version number of the service, every call to one of its method require a gcube-token. See [https://wiki.gcube-system.org/gcube/Authorization_Framework here] for more information about this token. This secret token needs to be valid in the current context. It can be put as query parameter of the http request or within the header. Both approaches are valid. If the token is missing, the operation will be rejected and an http forbidden code will be returned.<br />
<br />
= Usage =<br />
The methods of the Web Service can be called by writing your own REST client application or by using (a lot of) already existent rest client plugins. In the following, we will use the RESTClient plugin for Firefox for POST methods. Your browser can be used for GET methods. The return value of each method is an HTTP Response object that has an HTTP CODE (200, 201, 400, 500, ...) and, optionally, a JSON or text object that represents the created/retrieved resource or better specifies the occurred error. In case of errors, please refer to the [[#Troubleshooting|Troubleshooting]] section for more details.<br />
<br />
= Service methods (Version 2) =<br />
The list of sub-services available in the new version is going to be reported next. The name convention is different. For instance, to read user's posts, the method to be called is<br />
<br />
<source lang="java"><br />
BASE_PATH/2/posts/get-posts-user/<br />
</source><br />
<br />
The real result of the method, if the request succeeded, is contained into the 'result' field of the json object<br />
<br />
<source lang="java"><br />
{"success": ..., "message": ..., "result": "[{first post}, {second post}, ...}<br />
</source><br />
<br />
In the following, the list of available sub-services available under the newest version, its methods and the parameters they need in order to be called are reported. Again, you can use your preferred REST client so examples won't be reported.<br />
<br />
== List of Available Sub-Services and methods==<br />
=== Tokens ===<br />
====Generate application token====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/tokens/generate-application-token<br />
</source><br />
<br />
First of all, you need to have a ''user gcube-token'' for authentication purposes. You can get it using the Token Generator portlet available on the VRE you are interesting the app will post in.<br />
Then, to get an ''application gcube-token'' you can invoke the method. It is an HTTP/POST method that accepts as parameter:<br />
<br />
* a json object of this kind {"app_id": "your-app-id-goes-here"}.<br />
<br />
=== User profile and roles ===<br />
<br />
To retrieve user's profile user profile info (and roles) the subservice to call, once the social networking service endpoint has been retrieved, is this one:<br />
<br />
<code><br />
/2/people/profile<br />
</code><br />
<br />
further details: [https://wiki.gcube-system.org/gcube/OAuth2.0#Step_4_-_Get_member_data get member information]<br />
<br />
=== Full text search ===<br />
==== Search by query ====<br />
Social data are indexed into an ElasticSearch engine and are querable using a full-text approach. You can retrieve more information about this index [https://wiki.gcube-system.org/index.php?title=Social_Networking_Data_Discovery|here].<br />
The link to append to '''Base_URL''' is the following<br />
<source lang="java"><br />
2/full-text-search/search-by-query<br />
</source><br />
<br />
The query can be submitted also to the service, by using the following parameters<br />
<br />
* query: the query to perform against the index [mandatory];<br />
* from : the result set should be returned by starting from the n-th result [default is zero, optional];<br />
* quantity: how many enhanced posts need to be returned [default is 10, optional].<br />
<br />
=== Hashtags ===<br />
==== Get hashtags and occurrences in a context====<br />
The link to append to '''Base_URL''' is the following<br />
<source lang="java"><br />
2/hashtags/get-hashtags-and-occurrences<br />
</source><br />
<br />
The result is of the kind<br />
{"success": true,<br />
"message": null,<br />
"result":{{"hashtag1" : x}, {"hashtag2" : y}, ...}<br />
}<br />
=== Posts ===<br />
==== Write application post====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/write-post-app<br />
</source><br />
<br />
By means of the application token requested with the above method, the app can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters in a JSON object<br />
<br />
* text: text of the post; <br />
* params: additional parameters if your application supports the direct opening of of this update's object e.g. id=12345&type=foo [optional];<br />
* preview_title: a title for the preview of the post [optional];<br />
* preview_host: a host for the preview of the post [optional];<br />
* preview_description: a description for the preview of the post [optional];<br />
* preview_url: the preview url [optional];<br />
* image_url: if you want to put an image, pass its url [optional];<br />
* enable_notification: a boolean value to enable notifications [optional, default is false].<br />
<br />
====Write user post====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/write-post-user<br />
</source><br />
<br />
By means of a valid user token, he/she can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters in a JSON object:<br />
<br />
* text: text of the post; <br />
* preview_title: a title for the preview of the post [optional];<br />
* preview_description: a description for the preview of the post [optional];<br />
* preview_host: a host for the preview of the post [optional];<br />
* preview_url: the preview url [optional];<br />
* image_url: if you want to put an image, pass its url [optional];<br />
* enable_notification: a boolean value to enable notifications [optional].<br />
<br />
====Retrieve all application posts==== <br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-app<br />
</source><br />
<br />
In order to retrieve all the posts the application wrote (in every VRE it is allowed to write in) you can invoke this HTTP/GET method.<br />
====Retrieve all user posts====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user<br />
</source><br />
<br />
====Retrieve all user posts since a given date====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user-since<br />
</source><br />
<br />
If you are interested in retrieving user's posts whose timestamps are greater than a given date you can invoke this method.<br />
This HTTP/GET method accepts as parameter:<br />
<br />
* time: the time of the oldest post to retrieve converted in milliseconds.<br />
<br />
====Retrieve all user posts up to a given quantity====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-user-quantity<br />
</source><br />
<br />
If you are interested in retrieving user's posts, starting from the newest up to a given quantity you can invoke this method.<br />
This HTTP/GET method accepts as parameter:<br />
<br />
* quantity: a value greater than zero (the default value is 10).<br />
<br />
====Retrieve the identifiers of the posts a user liked====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-id-liked-posts<br />
</source><br />
<br />
By means of this method a user can retrieve the identifiers of the posts he/she liked.<br />
====Retrieve the posts a user liked====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-liked-posts<br />
</source><br />
<br />
By means of this method a user can retrieve the posts he/she liked.<br />
====Retrieve all vre posts====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-vre<br />
</source><br />
<br />
By means of this method a user that has a token valid for a certain VRE, can retrieve all the posts of that VRE.<br />
====Get posts by hashtag====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/posts/get-posts-by-hashtag<br />
</source><br />
The method requires as parameter:<br />
* hashtag: the hashtag to be contained in the returned posts[mandatory].<br />
<br />
=== Comments ===<br />
====Retrieve comments by user====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/comments/get-comments-user<br />
</source><br />
It is an http GET method.<br />
====Retrieve comments by user and time====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/comments/get-comments-user-by-time<br />
</source><br />
It is an http GET method. It requires:<br />
* time: the time (in milliseconds) since when the comments need to be retrieved [default is zero, optional];<br />
=== Notifications ===<br />
====Retrieve user's notifications in a given range====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/notifications/get-range-notifications<br />
</source><br />
<br />
By means of this method a user can retrieve his/her last notifications in the range [a, b]. The method requires:<br />
<br />
* from: an integer greater than zero;<br />
* quantity: an integer equal or greater than zero.<br />
<br />
The final range is evaluated as [from, from + quantity].<br />
====Notify job status====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
2/notifications/notify-job-status<br />
</source><br />
<br />
By means of this method a user can be notified about the status of a job in the infrastructure. Jobs are, for instance, SmartExecutor's plugins or DataMiner's algorithms. The method requires the following parameters in a JSON object via HTTP POST<br />
<br />
* recipient: the username of the user to be alerted [mandatory];<br />
* job_id: the identifier of the job (e.g, a uuid) [mandatory];<br />
* job_name: the name of the running job (e.g., Social-data-indexer-plugin)[mandatory];<br />
* service_name: the name of the service that launched the job (e.g., SmartExecutor) [mandatory];<br />
* status: the status reached by the job, that is any among CANCELLED, CANCELLING, DELETED, DELETING, EXECUTING, FAILED, SUCCEEDED, SUBMITTED, TIMED_OUT, WAITING [mandatory];<br />
* status_message: a detailed message about the status [optional].<br />
<br />
=== Messages ===<br />
====Send a message====<br />
It is an HTTP POST method.<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/write-message<br />
</source><br />
<br />
By default, the sender of the message is the owner of the gcube-token. According to the recipients' notifications preferences, they could receive a portal notification, an email notification and so on.<br />
<br />
Parameters, to be put in a JSON object are:<br />
<br />
* subject: the subject of the message to send;<br />
* body: the body of the message;<br />
* recipients: a list of valid comma separated users' ids (e.g., andrea.rossi, mario.rossi) or users' emails (e.g., andrea.rossi@isti.cnr.it, mario.rossi@isti.cnr.it);<br />
<br />
An example of the body of the request is as follows<br />
<source lang="xml"><br />
{"subject": "....", "body": ".....", "recipients":[{"id":"userid1"},{"id":"userid2"}]}<br />
</source><br />
<br />
or, if you want to use valid mails (i.e. recognized in the infrastructure)<br />
<source lang="xml"><br />
{"subject": "....", "body": ".....", "recipients":[{"id":"mail1"},{"id":"mail2"}]}<br />
</source><br />
<br />
====Get received messages====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/get-received-messages<br />
</source><br />
It is an http GET method.<br />
<br />
====Get sent messages====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/messages/get-sent-messages<br />
</source><br />
It is an http GET method.<br />
=== Users ===<br />
====User exists====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/user-exists<br />
</source><br />
<br />
The method requires<br />
<br />
* username: the username of the user which has to be checked.<br />
<br />
'''Note''' The method can be only invoked with a root-context application token.<br />
====Read user's custom fields====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-custom-attribute<br />
</source><br />
<br />
The method requires<br />
<br />
* attribute: that is the name of a valid attribute to be read.<br />
====Get user's email====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-email<br />
</source><br />
<br />
The method returns the email associated to the owner of the security token used.<br />
====Get user's profile====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-profile<br />
</source><br />
<br />
The method returns the profile of the owner of the security token used.<br />
====Get user's fullname====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-fullname<br />
</source><br />
<br />
The method returns the fullname associated to the owner of the security token used.<br />
====Get all users' usernames in VRE====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames<br />
</source><br />
<br />
The method returns the usernames of the context associated to the security token.<br />
<br />
====Get usernames by global role====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames-by-global-role<br />
</source><br />
<br />
The method requires<br />
<br />
* role-name: that is the name of the role to consider.<br />
<br />
'''''This method can be only invoked with an Infrastructure Token.'''''<br />
<br />
<br />
====Get usernames by role====<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
2/users/get-usernames-by-role<br />
</source><br />
<br />
The method requires<br />
<br />
* role-name: that is the name of the role to consider.<br />
<br />
=== VREs ===<br />
====Get User VREs====<br />
This method allows a user to retrieve the list of VREs she belongs to<br />
<br />
<source lang="java"><br />
2/vres/get-my-vres<br />
</source><br />
<br />
The method accepts an option parameter:<br />
* getManagers : if set to true, reports also the VRE Managers of the VRE<br />
<br />
== Troubleshooting ==<br />
<br />
There could be many reasons for a failed request. Please always check the HTTP CODE and the message within the response. <br />
<br />
=== HTTP Error Code ===<br />
This table tries to help you:<br />
<br />
{| border="1" cellpadding="20" cellspacing="0"<br />
!HTTP CODE<br />
!Meaning<br />
|-<br />
|400 - BAD_REQUEST<br />
|Please check the parameters you are passing<br />
|-<br />
|403 - FORBIDDEN<br />
|Is the token valid? / Does it belong to an application? [In case of application request]<br />
|-<br />
|404 - NOT_FOUND<br />
|The object(s) you are asking for is(are) not available<br />
|-<br />
|500 - INTERNAL_SERVER_ERROR<br />
|There was an error while serving your request (retry later)<br />
|-<br />
|}<br />
<br />
<br />
=== Always manage HTTP Redirects on POST requests (i.e. 30X codes) ===<br />
The service accepts both incoming http and https client requests, which are automatically redirect to https protocol by the reverse proxy nginx. This redirect mechanism must be taken into account when POST calls are performed, because it is not (always) supported and transparent by clients. In order to be sure, please check if you get a 30X error and extract the redirection urls within the response header. The following is a working java code that extract such information. It uses the apache-client library to perform http calls.<br />
<br />
<source lang="java"><br />
<br />
// build the client and the request<br />
String path = ... ; <br />
HttpClientBuilder clientBuilder = HttpClientBuilder.create();<br />
CloseableHttpClient httpClient = clientBuilder.build();<br />
HttpPost request = new HttpPost(path);<br />
JSONObject obj = new JSONObject();<br />
<br />
... // build the json object<br />
<br />
// set header properties<br />
request.addHeader("gcube-token", ...);<br />
request.addHeader("Content-type", ContentType.APPLICATION_JSON.toString());<br />
StringEntity params = new StringEntity(obj.toJSONString(), ContentType.APPLICATION_JSON);<br />
request.setEntity(params);<br />
HttpResponse response = httpClient.execute(request);<br />
logger.info(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
<br />
int status = response.getStatusLine().getStatusCode();<br />
<br />
// check the response status and look if it was a redirect problem<br />
if (status != HttpURLConnection.HTTP_OK && (status == HttpURLConnection.HTTP_MOVED_TEMP ||<br />
status == HttpURLConnection.HTTP_MOVED_PERM ||<br />
status == HttpURLConnection.HTTP_SEE_OTHER)) {<br />
<br />
// redirect -> fetch new location<br />
Header[] locations = response.getHeaders("Location");<br />
Header lastLocation = locations[locations.length - 1];<br />
String realLocation = lastLocation.getValue();<br />
logger.debug("New location is " + realLocation);<br />
<br />
// perform again the same request<br />
request = new HttpPost(realLocation);<br />
request.addHeader("gcube-token", ...);<br />
request.addHeader("Content-type", ContentType.APPLICATION_JSON.toString());<br />
params = new StringEntity(obj.toJSONString(), ContentType.APPLICATION_JSON);<br />
request.setEntity(params);<br />
response = httpClient.execute(request);<br />
logger.info(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
} else<br />
logger.debug(" " + response.getStatusLine().getStatusCode() + " and response message is " + response.getStatusLine().getReasonPhrase());<br />
</source><br />
<br />
= Service methods (Version 1) - DEPRECATED =<br />
In the following, the list of available sub-services available under the first version, its methods and the parameters they need in order to be called are reported. The gcube-token is always required so it won't be reported as parameter. <br />
<br />
'''This version has been replaced by version 2, please do not use this version and update your code asap.'''<br />
<br />
== List of available Sub-Services and methods ==<br />
=== Tokens ===<br />
==== Generate application token ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
tokens/generateApplicationToken<br />
</source><br />
<br />
First of all, you need to have a ''user gcube-token'' for authentication purposes. You can get it using the Token Generator portlet available on the VRE you are interesting the app will post in.<br />
Now, to get an ''application gcube-token'' you can invoke the method. It is an HTTP/POST method that accepts as parameters:<br />
<br />
* appid: the identifier of the application as reported in the Application Profile.<br />
<br />
For example, using the RESTClient plugin for Firefox you will have something like this for requiring a new application token:<br />
<br />
[[File:GenerateApplicationToken.png]]<br />
<br />
Push on the SEND button and if all it's ok, your application gcube-token will be generated and returned as ''plain text'' in the response within the HTTP response having code 201.<br />
=== Posts ===<br />
==== Write application post ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/writePostApplication<br />
</source><br />
<br />
By means of the application token requested with the above method, the app can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters:<br />
<br />
* text: text of the post; <br />
* params: additional parameters if your application supports the direct opening of of this update's object e.g. id=12345&type=foo [optional];<br />
* previewtitle: a title for the preview of the post [optional];<br />
* previewdescription: a description for the preview of the post [optional];<br />
* httpimageurl: if you want to put an image, pass its url [optional];<br />
* enablenotification: a boolean value to enable notifications [optional].<br />
<br />
For example:<br />
<br />
[[File:WriteApplicationPost.png]]<br />
<br />
Push on SEND and check the code associated to the returned response (201 in case of success).<br />
<br />
==== Retrieve all application posts ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByApp<br />
</source><br />
<br />
In order to retrieve all the posts the application wrote (in every VRE it is allowed to write in) you can invoke this HTTP/GET method.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByApp?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByUser<br />
</source><br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts since a given date ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getRecentPostsByUserAndDate<br />
</source><br />
<br />
If you are interested in retrieving user's posts whose timestamps are greater than a given date you can invoke this method.<br />
This HTTP/GET method accepts this parameter:<br />
<br />
* time: the time of the oldest post to retrieve converted in milliseconds.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getRecentPostsByUserAndDate?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&time=xxxxx<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Retrieve all user posts up to a given quantity ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getRecentPostsByUser<br />
</source><br />
<br />
If you are interested in retrieving user's posts, starting from the newest up to a given quantity you can invoke this method.<br />
This HTTP/GET method accepts two parameter:<br />
<br />
* quantity: a value greater than zero (the default value is 10).<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getRecentPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&quantity=20<br />
</source><br />
<br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
==== Write user post ====<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/writePostUser<br />
</source><br />
<br />
By means of a valid user token, he/she can write on the VRE for which the token has been previously requested. Just invoke this method, that accepts these parameters:<br />
<br />
* text: text of the post; <br />
* previewtitle: a title for the preview of the post [optional];<br />
* previewdescription: a description for the preview of the post [optional];<br />
* previewHost: [optional];<br />
* previewUrl: [optional];<br />
* httpimageurl: if you want to put an image, pass its url [optional];<br />
* enablenotification: a boolean value to enable notifications [optional];<br />
<br />
For example:<br />
<br />
[[File:Userpost.png]]<br />
<br />
Push on SEND and check the code associated to the returned response (201 in case of success).<br />
<br />
==== Retrieve the identifiers of the posts a user liked ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllLikedPostIdsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve the identifiers of the posts he/she liked.<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllLikedPostIdsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the identifiers of the posts are returned in JSON format.<br />
<br />
==== Retrieve the posts a user liked ====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllLikedPostsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve the posts he/she liked. <br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllLikedPostsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
====Retrieve all vre posts====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
posts/getAllPostsByVRE<br />
</source><br />
<br />
By means of this method a user that has a token valid for a certain VRE, can retrieve all the posts of that VRE.<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/posts/getAllPostsByVRE?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
<br />
=== Notifications ===<br />
====Retrieve user's notifications in a given range====<br />
<br />
The link to append to '''Base_URL''' is the following:<br />
<br />
<source lang="java"><br />
notifications/getRangeNotificationsByUser<br />
</source><br />
<br />
By means of this method a user can retrieve his/her last notifications in the range [a, b]. The method requires:<br />
<br />
* from: an integer greater than zero;<br />
* quantity: an integer equal or greater than zero.<br />
<br />
The final range is evaluated as [from, from + quantity]<br />
<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/notifications/getRangeNotificationsByUser?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&from=from&quantity=quantity<br />
</source><br />
<br />
If all goes ok, the response object has status code 200 and the posts are returned in JSON format.<br />
=== Messages ===<br />
<br />
====Send a message====<br />
You can send a message via the web service by using the following POST method:<br />
<br />
* writeMessageToUsers<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
messages/writeMessageToUsers<br />
</source><br />
<br />
By default, the sender of the message is the owner of the gcube-token. If you need to specify a different "sender", you can do that by specifying the parameter sender. Furthermore, according to the recipients' notifications preferences, they could receive a portal notification, a mail notification and so on.<br />
<br />
Parameters are:<br />
<br />
* subject: the subject of the message to send;<br />
* body: the body of the message;<br />
* recipients: a list of valid comma separated users' ids (e.g., andrea.rossi, mario.rossi).<br />
* sender: a valid sender's id [optional]<br />
<br />
An example usage of the method is reported in the picture below (without specifying the sender):<br />
<br />
[[File:Send_message_example.png]]<br />
=== Users ===<br />
====Read user's custom fields====<br />
You can read user's custom attributes using the following method<br />
<br />
*readCustomAttr<br />
<br />
The link to append to BASE_URL is<br />
<br />
<source lang="java"><br />
users/readCustomAttr<br />
</source><br />
<br />
The ''read'' method requires<br />
<br />
* attribute: that is the name of a valid attribute to be read.<br />
For example, using your browser:<br />
<br />
<source lang="java"><br />
BASE_URL/users/readCustomAttr?gcube-token=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyy&attribute=attributeX<br />
</source><br />
<br />
= See Also =<br />
<br />
== D4Science Developers website ==<br />
[https://dev.d4science.org D4Science Developers website]<br />
<br />
== SWAGGER documentation==<br />
[https://dev.d4science.org/swagger/social-networking/ SWAGGER documentation of the Social-Networking RESTful interface]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=Import_from_SVN&diff=32816Import from SVN2019-08-21T07:17:39Z<p>Massimiliano.assante: /* Clone the SVN Repository */</p>
<hr />
<div><br />
This guide explains how to migrate a project from SVN to Git.<br />
<br />
= Configure your system to allow Migration Facilities to properly work =<br />
<br />
== Download migration utilities ==<br />
For an easy reference in this guide, the migration scripts is downloaded in the home folder:<br />
<source lang="bash"><br />
cd $HOME && wget https://code-repo.d4science.org/gCubeSystem/Configs/raw/branch/master/SVN/svn-migration-scripts.jar <br />
</source><br />
<br />
== Install git-svn (Ubuntu) ==<br />
<br />
On Ubuntu, if you you get this output:<br />
<br />
<source lang="bash"><br />
$ java -jar ./svn-migration-scripts.jar verify <br />
svn-migration-scripts: using version 0.1.56bbc7f <br />
Git: using version 2.17.1 <br />
Subversion: using version 1.9.7<br />
git: 'svn' is not a git command. See 'git --help'.<br />
The most similar commands are<br />
fsck<br />
mv<br />
show<br />
git-svn: ERROR: Unable to determine version.<br />
</source><br />
<br />
You must install the git-svn package by running:<br />
<source lang="bash"><br />
$ sudo apt install git-svn<br />
</source><br />
<br />
Now, you will get something like this:<br />
<br />
<source lang="bash"><br />
$ java -jar ./svn-migration-scripts.jar verify<br />
svn-migration-scripts: using version 0.1.56bbc7f<br />
Git: using version 2.17.1<br />
Subversion: using version 1.9.7<br />
git-svn: using version 2.17.1<br />
</source><br />
<br />
== Mount a case-sensitive disk image (for Mac OS) ==<br />
Check if this step is needed by running:<br />
<source lang="bash"><br />
java -jar ~/svn-migration-scripts.jar verify<br />
<br />
svn-migration-scripts: using version 0.1.56bbc7f<br />
Git: using version 2.11.0<br />
Subversion: using version 1.9.4<br />
git-svn: using version 2.11.0<br />
You appear to be running on a case-insensitive file-system. This is unsupported, and can result in data loss.<br />
</source><br />
<br />
Following the warning, we create a disk image dedicated to the migration activities:<br />
<source lang="bash"><br />
java -jar ~/svn-migration-scripts.jar create-disk-image 5 GitMigration<br />
<br />
created: /Users/manuelesimi/GitMigration.sparseimage<br />
/dev/disk2 GUID_partition_scheme <br />
/dev/disk2s1 EFI <br />
/dev/disk2s2 Apple_HFS /Users/manuelesimi/GitMigration<br />
The disk image was created successfully and mounted as: /Users/manuelesimi/GitMigration<br />
</source><br />
<br />
= Extract the author(s) information=<br />
<br />
SVN uses the username to associate the commit, instead, Git uses the email. The author file is required to properly associate the history of commits to the right person.<br />
<br />
It is possible to get the author list for the current repository only or use the global author list (extracted for your convenience).<br />
<br />
== Get authors info from the desired repository ==<br />
<br />
<source lang="bash"><br />
cd ~/GitMigration<br />
java -jar ~/svn-migration-scripts.jar authors http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/gxREST > authors.txt<br />
About to create the authors file.<br />
</source><br />
<br />
<br />
Alternative (pure SVN):<br />
<source lang="bash"><br />
svn co https://svn.d4science.research-infrastructures.eu/gcube/trunk/Common/gxREST<br />
cd gxREST<br />
svn log -q | awk -F '|' '/^r/ {sub("^ ", "", $2); sub(" $", "", $2); print $2" = "$2" <"$2">"}' | sort -u > authors.txt<br />
</source><br />
<br />
=== Edit the authors file ===<br />
You need to edit the file and add the correct email address (i.e. the address configured in the Git service) for each listed author:<br />
<source lang="bash"><br />
$ cat authors.txt<br />
luca.frosini = luca.frosini <luca.frosini@mycompany.com><br />
lucio.lelii = lucio.lelii <lucio.lelii@mycompany.com><br />
manuele.simi = manuele.simi <manuele.simi@mycompany.com<br />
$ vi authors.txt<br />
<br />
$ cat authors.txt<br />
luca.frosini = Luca Frosini <luca.frosini@isti.cnr.it><br />
lucio.lelii = Lucio Lelii <lucio.lelii@isti.cnr.it><br />
manuele.simi = Manuele Simi <manuele.simi@isti.cnr.it><br />
</source><br />
<br />
You can also do that with just one bash command<br />
<br />
<source lang="bash"><br />
$ sed -i s/@mycompany.com/@isti.cnr.it/g authors.txt<br />
</source><br />
<br />
== Use the global authors list ==<br />
<br />
You can get the global authors mapping file as following<br />
<br />
<source lang="bash"><br />
$ wget -O authors.txt https://code-repo.d4science.org/gCubeSystem/Configs/raw/branch/master/SVN/all-svn-authors.txt<br />
</source><br />
<br />
<br />
'''BEFORE USE THE FILE PLEASE DOUBLE CHECK IF YOUR INFORMATION ARE CORRECT. IF YOU NOTICE SOME ERRORS FOR ANY AUTHOR PLEASE CORRECT THEM ON THE SOURCE FILE'''<br />
<br />
= SVN Repository Layouts=<br />
Depending on the structure of your SVN repo, the git-svn (see below) command needs to be configured differently. There are two possible layouts.<br />
<br />
== Standard Layout ==<br />
The SVN project uses the standard /trunk, /branches, and /tags directory layout. This is the recommended way to organize a repository: <br />
* a ''trunk'' directory to hold the “main line” of development, <br />
* a ''branches'' directory to contain branch copies, <br />
* and a ''tags'' directory to contain tag copies. <br />
<br />
In the standard layout, these are top-level directories.<br />
<br />
[[File:SNVStardardLayout.png|400px]]<br />
<br />
== Non-Standard Layout ==<br />
The SVN project uses a custom layout.<br />
<br />
[[File:SVNNonStandardLayout.png|400px]]<br />
<br />
= Clone the SVN Repository =<br />
The git svn clone command transforms the trunk, branches, and tags in your SVN repository into a new Git repository. Depending on the structure of the SVN repo, the command needs to be configured differently.<br />
<br />
<source lang="bash"><br />
> git svn clone --authors-file=authors.txt --follow-parent http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/gxREST \<br />
--username manuele.simi gxRest<br />
<br />
Note: --follow-parent makes it slower, but it’s needed if the SVN folder has been moved around.<br />
<br />
Initialized empty Git repository in /Users/manuelesimi/GitMigration/gxRest/.git/<br />
<br />
This may take a while on large repositories<br />
Checked through r173000<br />
<br />
Checked Ahrough .classpath<br />
<br />
A pom.xml<br />
A gxJRS/.classpath<br />
A gxJRS/.project<br />
A gxJRS/distro/profile.xml<br />
A gxJRS/distro/LICENSE<br />
A gxJRS/distro/changelog.xml<br />
A gxJRS/distro/README<br />
A gxJRS/src/test/java/org/gcube/common/gxrest/request/GXWebTargetAdapterRequestTest.java<br />
A gxJRS/src/test/java/org/gcube/common/gxrest/request/GXHTTPStringRequestTest.java<br />
A gxJRS/src/test/resources/logback-test.xml<br />
A gxJRS/src/main/java/org/gcube/common/gxrest/methods/package-info.java<br />
A gxJRS/src/main/java/org/gcube/common/gxrest/request/GXHTTPStreamRequest.java<br />
A gxJRS/src/main/java/org/gcube/common/gxrest/request/package-info.java<br />
A gxJRS/src/main/java/org/gcube/common/gxrest/request/GXWebTargetAdapterRequest.java<br />
<br />
[omitted output]<br />
<br />
A gxJRS/src/main/java/org/gcube/common/gxrest/response/entity/SerializableErrorEntityTextWriter.java<br />
A gxJRS/src/main/java/org/gcube/common/gxrest/response/entity/SerializableErrorEntityTextReader.java<br />
M gxJRS/src/main/java/org/gcube/common/gxrest/response/entity/SerializableErrorEntity.java<br />
r178787 = ac04855b00de818f2095d0784eb68c51a6ec9f77 (refs/remotes/git-svn)<br />
Checked out HEAD:<br />
https://svn.d4science.research-infrastructures.eu/gcube/trunk/Common/gxREST r178787<br />
creating empty directory: gxHTTP/src/main/resources<br />
creating empty directory: gxJRS/src/main/resources<br />
creating empty directory: gxJRS/src/test/java/org/gcube/common/gxrest/response<br />
</source><br />
<br />
Do note that the command above DO NOT automatically import the SVN branches because the repo does not have the standard SVN [[Import_from_SVN#SVN_Repository_Layouts|layout]] .<br />
<br />
If the SVN repository doesn’t have a standard layout and you want to import everything, you need to provide the locations of your trunk, branches, and tags using the --trunk, --branches, and --tags command line options. See [https://git-scm.com/docs/git-svn git-svn] for further options.<br />
<br />
= Change dir in the cloned folder = <br />
The last parameter of the "git svn clone" command is the name of the folder ("gxRest" in the previous example) where the SVN repository is cloned and converted to Git. All the git commands in the next sections are executed in the cloned folder.<br />
So you need to:<br />
<source lang="bash"><br />
> cd <cloned repo><br />
</source><br />
<br />
= Rename “trunk” branch to “master” (if needed) = <br />
<source lang="bash"><br />
> git branch<br />
* trunk<br />
> git branch -m trunk master<br />
> git branch<br />
* master<br />
</source><br />
<br />
= Check the imported history =<br />
<source lang="bash"><br />
> git log -10<br />
commit ac04855b00de818f2095d0784eb68c51a6ec9f77<br />
Author: manuele.simi <manuele.simi@isti.cnr.it><br />
Date: Sun Mar 31 03:39:06 2019 +0000<br />
<br />
Add JAX-RS MessageBodyWriter/Reader responsible for converting SerializableErrorEntity to/from a stream.<br />
<br />
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/Common/gxREST@178787 82a268e6-3cf1-43bd-a215-b396298e98cf<br />
<br />
commit 6eb3f608dbe31c50578e46c53e16c469a0cc7f0c<br />
Author: manuele.simi <manuele.simi@isti.cnr.it><br />
Date: Sat Mar 30 19:58:21 2019 +0000<br />
<br />
Tweak some javadoc.<br />
<br />
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/Common/gxREST@178786 82a268e6-3cf1-43bd-a215-b396298e98cf<br />
</source><br />
<br />
= Add code-repo’s Git repository as new remote = <br />
<br />
Do note that, before running this step, you need to create a [[Git_Repositories |new Git repository]]. <br />
<br />
<source lang="bash"><br />
> git remote add origin https://code-repo.d4science.org/manuele.simi/gxRest.git<br />
> git remote -v<br />
origin https://code-repo.d4science.org/manuele.simi/gxRest.git (fetch)<br />
origin https://code-repo.d4science.org/manuele.simi/gxRest.git (push)<br />
</source><br />
<br />
= Push the local repository to the new remote =<br />
<source lang="bash"><br />
> git push --set-upstream --force origin master<br />
Counting objects: 168, done.<br />
Delta compression using up to 4 threads.<br />
Compressing objects: 100% (125/125), done.<br />
Writing objects: 100% (168/168), 39.47 KiB | 0 bytes/s, done.<br />
Total 168 (delta 44), reused 0 (delta 0)<br />
remote: Resolving deltas: 100% (44/44), done.<br />
To https://code-repo.d4science.org/manuele.simi/gxRest.git<br />
+ 4ec6b48...ac04855 master -> master (forced update)<br />
Branch master set up to track remote branch master from origin.<br />
</source><br />
<br />
= Clone the repository and create a working copy =<br />
Change dir to a selected (or workspace) folder and clone the new Git repository:<br />
<source lang="bash"><br />
> cd ~/Work/CNR/Projects/Git/<br />
> git clone https://code-repo.d4science.org/manuele.simi/gxRest.git<br />
Cloning into 'gxRest'...<br />
remote: Counting objects: 168, done.<br />
remote: Compressing objects: 100% (81/81), done.<br />
remote: Total 168 (delta 44), reused 168 (delta 44)<br />
Receiving objects: 100% (168/168), 39.47 KiB | 0 bytes/s, done.<br />
Resolving deltas: 100% (44/44), done.<br />
</source><br />
<br />
= Configure authors info = <br />
<br />
== Single repo configuration ==<br />
In the root folder of the repository: <br />
<source lang="bash"><br />
> git config user.email "manuele.simi@isti.cnr.it"<br />
> git config user.name "Manuele Simi"<br />
</source><br />
<br />
== Global configuration (all repos on this computer) ==<br />
In any folder: <br />
<source lang="bash"><br />
> git config --global user.email "manuele.simi@isti.cnr.it"<br />
> git config --global user.name "Manuele Simi"<br />
</source><br />
<br />
= Change the SCM Connection =<br />
The SCM section of the POM must be changed to reflect that we are now working with a Git repository.<br />
<br />
For instance, the following section:<br />
<pre><br />
<scm><br />
<connection>scm:svn:http://svn.d4science.research-infrastructures.eu/gcube/trunk/distributions/${project.artifactId}</connection><br />
<developerConnection>scm:svn:https://svn.d4science.research-infrastructures.eu/gcube/trunk/distributions/${project.artifactId}</developerConnection><br />
<url>http://svn.d4science.research-infrastructures.eu/gcube/trunk/distributions/${project.artifactId}</url><br />
</scm><br />
</pre><br />
<br />
must be changed to:<br />
<pre><br />
<scm><br />
<connection>scm:git:https://code-repo.d4science.org/gCubeSystem/gcube-bom.git</connection><br />
<developerConnection>scm:git:https://code-repo.d4science.org/gCubeSystem/gcube-bom.git</developerConnection><br />
<url>https://code-repo.d4science.org/gCubeSystem/gcube-bom</url><br />
</scm><br />
</pre><br />
<br />
''Back to the [[Continuous_Integration_procedure_(2019) | CI guide]].''<br />
<br />
[[Category:Continuous_Integration]]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=Import_from_SVN&diff=32815Import from SVN2019-08-21T07:13:05Z<p>Massimiliano.assante: /* Get authors info from the desired repository */</p>
<hr />
<div><br />
This guide explains how to migrate a project from SVN to Git.<br />
<br />
= Configure your system to allow Migration Facilities to properly work =<br />
<br />
== Download migration utilities ==<br />
For an easy reference in this guide, the migration scripts is downloaded in the home folder:<br />
<source lang="bash"><br />
cd $HOME && wget https://code-repo.d4science.org/gCubeSystem/Configs/raw/branch/master/SVN/svn-migration-scripts.jar <br />
</source><br />
<br />
== Install git-svn (Ubuntu) ==<br />
<br />
On Ubuntu, if you you get this output:<br />
<br />
<source lang="bash"><br />
$ java -jar ./svn-migration-scripts.jar verify <br />
svn-migration-scripts: using version 0.1.56bbc7f <br />
Git: using version 2.17.1 <br />
Subversion: using version 1.9.7<br />
git: 'svn' is not a git command. See 'git --help'.<br />
The most similar commands are<br />
fsck<br />
mv<br />
show<br />
git-svn: ERROR: Unable to determine version.<br />
</source><br />
<br />
You must install the git-svn package by running:<br />
<source lang="bash"><br />
$ sudo apt install git-svn<br />
</source><br />
<br />
Now, you will get something like this:<br />
<br />
<source lang="bash"><br />
$ java -jar ./svn-migration-scripts.jar verify<br />
svn-migration-scripts: using version 0.1.56bbc7f<br />
Git: using version 2.17.1<br />
Subversion: using version 1.9.7<br />
git-svn: using version 2.17.1<br />
</source><br />
<br />
== Mount a case-sensitive disk image (for Mac OS) ==<br />
Check if this step is needed by running:<br />
<source lang="bash"><br />
java -jar ~/svn-migration-scripts.jar verify<br />
<br />
svn-migration-scripts: using version 0.1.56bbc7f<br />
Git: using version 2.11.0<br />
Subversion: using version 1.9.4<br />
git-svn: using version 2.11.0<br />
You appear to be running on a case-insensitive file-system. This is unsupported, and can result in data loss.<br />
</source><br />
<br />
Following the warning, we create a disk image dedicated to the migration activities:<br />
<source lang="bash"><br />
java -jar ~/svn-migration-scripts.jar create-disk-image 5 GitMigration<br />
<br />
created: /Users/manuelesimi/GitMigration.sparseimage<br />
/dev/disk2 GUID_partition_scheme <br />
/dev/disk2s1 EFI <br />
/dev/disk2s2 Apple_HFS /Users/manuelesimi/GitMigration<br />
The disk image was created successfully and mounted as: /Users/manuelesimi/GitMigration<br />
</source><br />
<br />
= Extract the author(s) information=<br />
<br />
SVN uses the username to associate the commit, instead, Git uses the email. The author file is required to properly associate the history of commits to the right person.<br />
<br />
It is possible to get the author list for the current repository only or use the global author list (extracted for your convenience).<br />
<br />
== Get authors info from the desired repository ==<br />
<br />
<source lang="bash"><br />
cd ~/GitMigration<br />
java -jar ~/svn-migration-scripts.jar authors http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/gxREST > authors.txt<br />
About to create the authors file.<br />
</source><br />
<br />
<br />
Alternative (pure SVN):<br />
<source lang="bash"><br />
svn co https://svn.d4science.research-infrastructures.eu/gcube/trunk/Common/gxREST<br />
cd gxREST<br />
svn log -q | awk -F '|' '/^r/ {sub("^ ", "", $2); sub(" $", "", $2); print $2" = "$2" <"$2">"}' | sort -u > authors.txt<br />
</source><br />
<br />
=== Edit the authors file ===<br />
You need to edit the file and add the correct email address (i.e. the address configured in the Git service) for each listed author:<br />
<source lang="bash"><br />
$ cat authors.txt<br />
luca.frosini = luca.frosini <luca.frosini@mycompany.com><br />
lucio.lelii = lucio.lelii <lucio.lelii@mycompany.com><br />
manuele.simi = manuele.simi <manuele.simi@mycompany.com<br />
$ vi authors.txt<br />
<br />
$ cat authors.txt<br />
luca.frosini = Luca Frosini <luca.frosini@isti.cnr.it><br />
lucio.lelii = Lucio Lelii <lucio.lelii@isti.cnr.it><br />
manuele.simi = Manuele Simi <manuele.simi@isti.cnr.it><br />
</source><br />
<br />
You can also do that with just one bash command<br />
<br />
<source lang="bash"><br />
$ sed -i s/@mycompany.com/@isti.cnr.it/g authors.txt<br />
</source><br />
<br />
== Use the global authors list ==<br />
<br />
You can get the global authors mapping file as following<br />
<br />
<source lang="bash"><br />
$ wget -O authors.txt https://code-repo.d4science.org/gCubeSystem/Configs/raw/branch/master/SVN/all-svn-authors.txt<br />
</source><br />
<br />
<br />
'''BEFORE USE THE FILE PLEASE DOUBLE CHECK IF YOUR INFORMATION ARE CORRECT. IF YOU NOTICE SOME ERRORS FOR ANY AUTHOR PLEASE CORRECT THEM ON THE SOURCE FILE'''<br />
<br />
= SVN Repository Layouts=<br />
Depending on the structure of your SVN repo, the git-svn (see below) command needs to be configured differently. There are two possible layouts.<br />
<br />
== Standard Layout ==<br />
The SVN project uses the standard /trunk, /branches, and /tags directory layout. This is the recommended way to organize a repository: <br />
* a ''trunk'' directory to hold the “main line” of development, <br />
* a ''branches'' directory to contain branch copies, <br />
* and a ''tags'' directory to contain tag copies. <br />
<br />
In the standard layout, these are top-level directories.<br />
<br />
[[File:SNVStardardLayout.png|400px]]<br />
<br />
== Non-Standard Layout ==<br />
The SVN project uses a custom layout.<br />
<br />
[[File:SVNNonStandardLayout.png|400px]]<br />
<br />
= Clone the SVN Repository =<br />
The git svn clone command transforms the trunk, branches, and tags in your SVN repository into a new Git repository. Depending on the structure of the SVN repo, the command needs to be configured differently.<br />
<br />
<source lang="bash"><br />
> git svn clone --authors-file=authors.txt --follow-parent https://svn.d4science.research-infrastructures.eu/gcube/trunk/Common/gxREST \<br />
--username manuele.simi gxRest<br />
<br />
Note: --follow-parent makes it slower, but it’s needed if the SVN folder has been moved around.<br />
<br />
Initialized empty Git repository in /Users/manuelesimi/GitMigration/gxRest/.git/<br />
<br />
This may take a while on large repositories<br />
Checked through r173000<br />
<br />
Checked Ahrough .classpath<br />
<br />
A pom.xml<br />
A gxJRS/.classpath<br />
A gxJRS/.project<br />
A gxJRS/distro/profile.xml<br />
A gxJRS/distro/LICENSE<br />
A gxJRS/distro/changelog.xml<br />
A gxJRS/distro/README<br />
A gxJRS/src/test/java/org/gcube/common/gxrest/request/GXWebTargetAdapterRequestTest.java<br />
A gxJRS/src/test/java/org/gcube/common/gxrest/request/GXHTTPStringRequestTest.java<br />
A gxJRS/src/test/resources/logback-test.xml<br />
A gxJRS/src/main/java/org/gcube/common/gxrest/methods/package-info.java<br />
A gxJRS/src/main/java/org/gcube/common/gxrest/request/GXHTTPStreamRequest.java<br />
A gxJRS/src/main/java/org/gcube/common/gxrest/request/package-info.java<br />
A gxJRS/src/main/java/org/gcube/common/gxrest/request/GXWebTargetAdapterRequest.java<br />
<br />
[omitted output]<br />
<br />
A gxJRS/src/main/java/org/gcube/common/gxrest/response/entity/SerializableErrorEntityTextWriter.java<br />
A gxJRS/src/main/java/org/gcube/common/gxrest/response/entity/SerializableErrorEntityTextReader.java<br />
M gxJRS/src/main/java/org/gcube/common/gxrest/response/entity/SerializableErrorEntity.java<br />
r178787 = ac04855b00de818f2095d0784eb68c51a6ec9f77 (refs/remotes/git-svn)<br />
Checked out HEAD:<br />
https://svn.d4science.research-infrastructures.eu/gcube/trunk/Common/gxREST r178787<br />
creating empty directory: gxHTTP/src/main/resources<br />
creating empty directory: gxJRS/src/main/resources<br />
creating empty directory: gxJRS/src/test/java/org/gcube/common/gxrest/response<br />
</source><br />
<br />
Do note that the command above DO NOT automatically import the SVN branches because the repo does not have the standard SVN [[Import_from_SVN#SVN_Repository_Layouts|layout]] .<br />
<br />
If the SVN repository doesn’t have a standard layout and you want to import everything, you need to provide the locations of your trunk, branches, and tags using the --trunk, --branches, and --tags command line options. See [https://git-scm.com/docs/git-svn git-svn] for further options.<br />
<br />
= Change dir in the cloned folder = <br />
The last parameter of the "git svn clone" command is the name of the folder ("gxRest" in the previous example) where the SVN repository is cloned and converted to Git. All the git commands in the next sections are executed in the cloned folder.<br />
So you need to:<br />
<source lang="bash"><br />
> cd <cloned repo><br />
</source><br />
<br />
= Rename “trunk” branch to “master” (if needed) = <br />
<source lang="bash"><br />
> git branch<br />
* trunk<br />
> git branch -m trunk master<br />
> git branch<br />
* master<br />
</source><br />
<br />
= Check the imported history =<br />
<source lang="bash"><br />
> git log -10<br />
commit ac04855b00de818f2095d0784eb68c51a6ec9f77<br />
Author: manuele.simi <manuele.simi@isti.cnr.it><br />
Date: Sun Mar 31 03:39:06 2019 +0000<br />
<br />
Add JAX-RS MessageBodyWriter/Reader responsible for converting SerializableErrorEntity to/from a stream.<br />
<br />
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/Common/gxREST@178787 82a268e6-3cf1-43bd-a215-b396298e98cf<br />
<br />
commit 6eb3f608dbe31c50578e46c53e16c469a0cc7f0c<br />
Author: manuele.simi <manuele.simi@isti.cnr.it><br />
Date: Sat Mar 30 19:58:21 2019 +0000<br />
<br />
Tweak some javadoc.<br />
<br />
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/Common/gxREST@178786 82a268e6-3cf1-43bd-a215-b396298e98cf<br />
</source><br />
<br />
= Add code-repo’s Git repository as new remote = <br />
<br />
Do note that, before running this step, you need to create a [[Git_Repositories |new Git repository]]. <br />
<br />
<source lang="bash"><br />
> git remote add origin https://code-repo.d4science.org/manuele.simi/gxRest.git<br />
> git remote -v<br />
origin https://code-repo.d4science.org/manuele.simi/gxRest.git (fetch)<br />
origin https://code-repo.d4science.org/manuele.simi/gxRest.git (push)<br />
</source><br />
<br />
= Push the local repository to the new remote =<br />
<source lang="bash"><br />
> git push --set-upstream --force origin master<br />
Counting objects: 168, done.<br />
Delta compression using up to 4 threads.<br />
Compressing objects: 100% (125/125), done.<br />
Writing objects: 100% (168/168), 39.47 KiB | 0 bytes/s, done.<br />
Total 168 (delta 44), reused 0 (delta 0)<br />
remote: Resolving deltas: 100% (44/44), done.<br />
To https://code-repo.d4science.org/manuele.simi/gxRest.git<br />
+ 4ec6b48...ac04855 master -> master (forced update)<br />
Branch master set up to track remote branch master from origin.<br />
</source><br />
<br />
= Clone the repository and create a working copy =<br />
Change dir to a selected (or workspace) folder and clone the new Git repository:<br />
<source lang="bash"><br />
> cd ~/Work/CNR/Projects/Git/<br />
> git clone https://code-repo.d4science.org/manuele.simi/gxRest.git<br />
Cloning into 'gxRest'...<br />
remote: Counting objects: 168, done.<br />
remote: Compressing objects: 100% (81/81), done.<br />
remote: Total 168 (delta 44), reused 168 (delta 44)<br />
Receiving objects: 100% (168/168), 39.47 KiB | 0 bytes/s, done.<br />
Resolving deltas: 100% (44/44), done.<br />
</source><br />
<br />
= Configure authors info = <br />
<br />
== Single repo configuration ==<br />
In the root folder of the repository: <br />
<source lang="bash"><br />
> git config user.email "manuele.simi@isti.cnr.it"<br />
> git config user.name "Manuele Simi"<br />
</source><br />
<br />
== Global configuration (all repos on this computer) ==<br />
In any folder: <br />
<source lang="bash"><br />
> git config --global user.email "manuele.simi@isti.cnr.it"<br />
> git config --global user.name "Manuele Simi"<br />
</source><br />
<br />
= Change the SCM Connection =<br />
The SCM section of the POM must be changed to reflect that we are now working with a Git repository.<br />
<br />
For instance, the following section:<br />
<pre><br />
<scm><br />
<connection>scm:svn:http://svn.d4science.research-infrastructures.eu/gcube/trunk/distributions/${project.artifactId}</connection><br />
<developerConnection>scm:svn:https://svn.d4science.research-infrastructures.eu/gcube/trunk/distributions/${project.artifactId}</developerConnection><br />
<url>http://svn.d4science.research-infrastructures.eu/gcube/trunk/distributions/${project.artifactId}</url><br />
</scm><br />
</pre><br />
<br />
must be changed to:<br />
<pre><br />
<scm><br />
<connection>scm:git:https://code-repo.d4science.org/gCubeSystem/gcube-bom.git</connection><br />
<developerConnection>scm:git:https://code-repo.d4science.org/gCubeSystem/gcube-bom.git</developerConnection><br />
<url>https://code-repo.d4science.org/gCubeSystem/gcube-bom</url><br />
</scm><br />
</pre><br />
<br />
''Back to the [[Continuous_Integration_procedure_(2019) | CI guide]].''<br />
<br />
[[Category:Continuous_Integration]]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=OAuth2.0&diff=32811OAuth2.02019-07-26T13:53:16Z<p>Massimiliano.assante: /* Step 4 - Get member data */</p>
<hr />
<div>__TOC__<br />
<br />
= OAuth 2.0 gCube Support =<br />
By means of the OAuth 2.0 protocol (authorised) third party applications can operate on user's behalf over the D4Science infrastructure (while protecting the member's credentials).<br />
For more information about the OAuth authorization framework please visit the official [https://oauth.net/2/ OAuth site]. For technical details you could also check the [https://tools.ietf.org/html/rfc6749| OAuth 2.0 RFC]. In the following, the needed steps to authorize third party applications to operate on user's behalf and the D4Science infrastructure are explained. <br />
<br />
== Step 1 - Configure your application ==<br />
Authorized third party applications must be registered on the Infrastructure. Up to now the application registration is not automatic but you need to open a ticket at https://support.d4science.org/projects/d4science.<br />
<br />
The ticket needs to have '''3rd Party Application Support''' as tracker. The application name as well as the redirect uri need to be provided (https is strongly suggested). This url must be absolute, and without arguments. Once the application is registered, it will have associated:<br />
<br />
* a unique client identifier (client_id);<br />
* a client secret which must be kept secret, as the name itself says (client_secret).<br />
<br />
<source lang="xml"><br />
client_id: ab9d32c9-9425-42f1-99d7-50d68e651e9c<br />
client_secret: y9r93zDv4D<br />
</source><br />
<br />
Both are needed to proceed with the OAuth 2.0 workflow.<br />
<br />
== Step 2 - Require an authorization code ==<br />
Once the application is registered and configured, an authorization code can be requested. This code is not the final one needed to perform operations on user behalf, but is the one that will be exchanged for that in the next step. To require this code that has a very short time to live, the application must invoke the authorization service, which is gateway dependant.<br />
<br />
=== Retrieving the authorization endpoint location===<br />
<br />
The previous authorization endpoint location is portal's gateway dependant and '''can be found in Table 1''' or retrieved dinamically by using the registry service REST-API hosted at <br />
<br />
''https://registry.d4science.org/icproxy/gcube/service/'' <br />
<br />
'''Get authorization endpoint location in the following Table 1'''<br />
<br />
{| class="wikitable"<br />
|+Table 1: authorization endpoints location depending on the D4Science Gateway<br />
! style="font-weight: bold;" | D4Science Gateway name<br />
! style="font-weight: bold;" | Authorization endpoint<br />
|-<br />
| ARIADNEplus Gateway<br />
| https://ariadne.d4science.org/group/ariadneplus-gateway/authorization<br />
|-<br />
| iMarine Gateway<br />
| https://i-marine.d4science.org/group/imarine-gateway/authorization<br />
|-<br />
| AGINFRA+ Gateway<br />
| https://aginfra.d4science.org/group/aginfra-gateway/authorization<br />
|-<br />
| BlueBRIDGE Gateway<br />
| https://bluebridge.d4science.org/group/bluebridge-gateway/authorization<br />
|-<br />
| D4Science.org Gateway<br />
| https://services.d4science.org/group/d4science-services-gateway/authorization<br />
|-<br />
| DESCRAMBLE Gateway<br />
| https://descramble.d4science.org/group/descramble-gateway/authorization<br />
|-<br />
| GEMex Gateway<br />
| https://gemex.d4science.org/group/gemex-gateway/authorization<br />
|-<br />
| OpenAIRE-Connect Gateway<br />
| https://openaire-connect.d4science.org/group/openaire-connect-gateway/authorization<br />
|-<br />
| PARTHENOS Gateway<br />
| https://parthenos.d4science.org/group/parthenos-gateway/authorization<br />
|-<br />
| SoBigData Gateway<br />
| https://sobigdata.d4science.org/group/sobigdata-gateway/authorization<br />
|}<br />
<br />
'''Retrieve the authorization endpoint location dynamically (ADVANCED MODE)'''<br />
<br />
Registry Swagger REST-API avaiable at: https://dev.d4science.org/swagger/registry/<br />
<br />
Which requires an [https://dev.d4science.org/authorization/ authorization token]. The returned profile is something like<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<serviceEndpoints><br />
<Resource version="0.4.x"><br />
<ID>b1c5851f-1d47-4c60-8906-941ab859a334</ID><br />
<Type>RuntimeResource</Type><br />
<Scopes><br />
<Scope>/gcube</Scope><br />
</Scopes><br />
...<br />
<AccessPoint><br />
<Description>AuthorisationServer where to authenticate external SmartGears services oAuth like</Description><br />
<Interface><br />
<Endpoint EntryName="AuthorisationServer">https://next.d4science.org/group/next/authorization</Endpoint><br />
</Interface><br />
</AccessPoint><br />
</Profile><br />
</Resource><br />
</serviceEndpoints><br />
</source><br />
<br />
=== Invoking the authorization service===<br />
<br />
Once you got the AuthorisationServer endpoint and the call is executed, two things can happen:<br />
* the user is requested to perform the login, thus accepting the application operation and granting it;<br />
* if there is a valid existing permission grant for the application, the login operation is by-passed and the redirect_uri is followed.<br />
<br />
The request is an '''HTTP GET''' call, that needs the following parameters to be provided<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| client_id<br />
| the client identifier associated to the application<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the redirect uri provided at registration phase<br />
| yes<br />
|-<br />
| state<br />
| a unique string value hard to guess, needed to prevent CRSF attacks (e.g. AAedBEeF545sdfKef897)<br />
| yes<br />
|-<br />
| scope<br />
| this field is optional. It can be specified to request (with the next step) an access token valid for a given<br />
infrastructure scope. If missing, the scope will be choose by the user on credentials insertion.<br />
| no<br />
|}<br />
<br />
This is an example in development environment<br />
<pre><br />
https://next.d4science.org/group/next/authorization?client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&scope=%2Fgcube&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&state=7d12bf13-111c-4f46-ab06-9e9e08ad377b<br />
</pre><br />
<br />
==== End user Experience ====<br />
<br />
Once redirected, the user will be presented with D4Science's authentication dialog box. This identifies your application as well as outlines the particular member permissions that your application requests. The logo and application name can be configurable, in the example the application name is '''Application X'''<br />
<br />
[[File:Oauth-ux.jpg]]<br />
<br />
==== Application is Accepted ====<br />
<br />
If the user grants permissions to the application, a redirection to the redirect_uri is performed. Attached to the redirect_uri, two important url arguments are provided<br />
<br />
* code: which is the authorization code. It expires in 10 seconds;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
==== Application is Rejected ====<br />
<br />
If, for any reason, the request fails, the redirect_uri is still followed but the url has attached the following parameters<br />
* error: a code that states the type of error;<br />
* error_description: a textual description of the actual error;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
== Step 3 - Exchange Authorization Code for an Access Token ==<br />
This is the step in which the previously obtained code can be exchanged to obtain a real access token. <br />
This time, an '''HTTP POST''' method needs to be performed, with a '''Content-Type''' of type '''application/x-www-form-urlencoded'''.<br />
<br />
The client_id and secret must be passed as the basic auth header Base64 encoded, see the example in Java below:<br />
<br />
e.g.<br />
<source lang="java"><br />
...<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
..<br />
</source><br />
<br />
The request-body must contain the following parameters:<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| grant_type<br />
| The value of this field should always be authorization_code<br />
| yes<br />
|-<br />
| code<br />
| the authorization code you received from the previous step.<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the same 'redirect_uri' value that you<br />
passed in the previous step.<br />
| yes<br />
|}<br />
<br />
For instance, in development environment<br />
<br />
<source lang="xml"><br />
POST /gcube-oauth/v2/access-token HTTP/1.1<br />
Host: socialnetworking-d-d4s.d4science.org<br />
Content-Type: application/x-www-form-urlencoded<br />
<br />
grant_type=authorization_code&code=...&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&client_secret=...<br />
</source><br />
<br />
In case of success, the response will contain a JSON object, having these properties:<br />
<br />
* access_token: the obtained access token to use to perform operations on user's behalf;<br />
* expires_in: the time to live of the token expressed in seconds;<br />
* scope: in which scope the token is valid;<br />
* token_type: that is "Bearer".<br />
<br />
In case of failures, instead, the returned object will contain:<br />
<br />
* error : an error string according the oauth 2.0 RFC.<br />
* error_description : a text description for the error.<br />
<br />
=== D4Science oAuth2 endpoint location === <br />
In the '''production''' environment, its value is<br />
<source lang="xml"><br />
https://api.d4science.org/oauth/v2/access-token<br />
</source><br />
<br />
'''HTTP POST Example in JAVA'''<br />
<br />
<source lang="java"><br />
String ACCESS_TOKEN_ATTR = "access_token";<br />
String SCOPE_ATTR = "scope";<br />
String fullPath2oAuthService = "https://api.d4science.org/oauth/v2/access-token";<br />
try {<br />
CloseableHttpClient httpClient = HttpClientBuilder.create().setRedirectStrategy(new LaxRedirectStrategy()).build();<br />
HttpPost httpPostRequest = new HttpPost(fullPath2oAuthService);<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
httpPostRequest.addHeader("Content-Type", "application/x-www-form-urlencoded");<br />
List<NameValuePair> params = new ArrayList<NameValuePair>();<br />
params.add(new BasicNameValuePair("grant_type", "authorization_code"));<br />
params.add(new BasicNameValuePair("code", tempCode));<br />
params.add(new BasicNameValuePair("redirect_uri", "https://www.gcube-system.org/oauth-authentication-example/example-app"));<br />
httpPostRequest.setEntity(new UrlEncodedFormEntity(params));<br />
<br />
CloseableHttpResponse httpresponse = httpClient.execute(httpPostRequest);<br />
int responseCode = httpresponse.getStatusLine().getStatusCode();<br />
if (responseCode < 200 || responseCode >= 300) {<br />
System.out.println("Error while getting authorization code, response status line from "<br />
+ fullPath2oAuthService + " was: " + responseCode); <br />
}<br />
JSONParser parser = new JSONParser();<br />
JSONObject json = (JSONObject) parser.parse(EntityUtils.toString(httpresponse.getEntity()));<br />
accessToken = json.get(ACCESS_TOKEN_ATTR).toString();<br />
scope = json.get(SCOPE_ATTR).toString();<br />
} catch(Exception e){<br />
System.out.println("Failed to perform request");<br />
e.printStackTrace();<br />
<br />
}<br />
</source><br />
<br />
== Step 4 - Get member data ==<br />
Having an access token, your application can perform some operations on behalf of the user. For instance, it can retrieve user's information profile by exploiting the [https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface social networking web service].<br />
<br />
Its current production endpoint is: https://api.d4science.org/rest/<br />
<br />
The subservice to call, once the social networking service endpoint has been retrieved, is this one<br />
<br />
<source lang="xml"><br />
2/users/get-oauth-profile<br />
</source><br />
<br />
It requires a ''gcube-token'' as query parameter, whose value is exactly the access token retrieved in the previous step. The result is something like this<br />
<br />
<source lang="ruby"><br />
{<br />
id: ".....",<br />
roles: [ ], // roles in the given-token context, e.g. VRE-Manager<br />
picture: "....",<br />
name: "....",<br />
middle_name: "....",<br />
male: as boolean ,<br />
location_industry: "....",<br />
given_name: "....",<br />
email: "....",<br />
job_title: "....",<br />
family_name: "....",<br />
verified_email: as boolean <br />
}<br />
</source><br />
<br />
== See Also ==<br />
<br />
[https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface Social Networking Service]<br />
<br />
[https://dev.d4science.org/swagger/social-networking/ Social Networking Service Docs SWAGGER]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=OAuth2.0&diff=32810OAuth2.02019-07-26T13:52:49Z<p>Massimiliano.assante: /* Step 4 - Get member data */</p>
<hr />
<div>__TOC__<br />
<br />
= OAuth 2.0 gCube Support =<br />
By means of the OAuth 2.0 protocol (authorised) third party applications can operate on user's behalf over the D4Science infrastructure (while protecting the member's credentials).<br />
For more information about the OAuth authorization framework please visit the official [https://oauth.net/2/ OAuth site]. For technical details you could also check the [https://tools.ietf.org/html/rfc6749| OAuth 2.0 RFC]. In the following, the needed steps to authorize third party applications to operate on user's behalf and the D4Science infrastructure are explained. <br />
<br />
== Step 1 - Configure your application ==<br />
Authorized third party applications must be registered on the Infrastructure. Up to now the application registration is not automatic but you need to open a ticket at https://support.d4science.org/projects/d4science.<br />
<br />
The ticket needs to have '''3rd Party Application Support''' as tracker. The application name as well as the redirect uri need to be provided (https is strongly suggested). This url must be absolute, and without arguments. Once the application is registered, it will have associated:<br />
<br />
* a unique client identifier (client_id);<br />
* a client secret which must be kept secret, as the name itself says (client_secret).<br />
<br />
<source lang="xml"><br />
client_id: ab9d32c9-9425-42f1-99d7-50d68e651e9c<br />
client_secret: y9r93zDv4D<br />
</source><br />
<br />
Both are needed to proceed with the OAuth 2.0 workflow.<br />
<br />
== Step 2 - Require an authorization code ==<br />
Once the application is registered and configured, an authorization code can be requested. This code is not the final one needed to perform operations on user behalf, but is the one that will be exchanged for that in the next step. To require this code that has a very short time to live, the application must invoke the authorization service, which is gateway dependant.<br />
<br />
=== Retrieving the authorization endpoint location===<br />
<br />
The previous authorization endpoint location is portal's gateway dependant and '''can be found in Table 1''' or retrieved dinamically by using the registry service REST-API hosted at <br />
<br />
''https://registry.d4science.org/icproxy/gcube/service/'' <br />
<br />
'''Get authorization endpoint location in the following Table 1'''<br />
<br />
{| class="wikitable"<br />
|+Table 1: authorization endpoints location depending on the D4Science Gateway<br />
! style="font-weight: bold;" | D4Science Gateway name<br />
! style="font-weight: bold;" | Authorization endpoint<br />
|-<br />
| ARIADNEplus Gateway<br />
| https://ariadne.d4science.org/group/ariadneplus-gateway/authorization<br />
|-<br />
| iMarine Gateway<br />
| https://i-marine.d4science.org/group/imarine-gateway/authorization<br />
|-<br />
| AGINFRA+ Gateway<br />
| https://aginfra.d4science.org/group/aginfra-gateway/authorization<br />
|-<br />
| BlueBRIDGE Gateway<br />
| https://bluebridge.d4science.org/group/bluebridge-gateway/authorization<br />
|-<br />
| D4Science.org Gateway<br />
| https://services.d4science.org/group/d4science-services-gateway/authorization<br />
|-<br />
| DESCRAMBLE Gateway<br />
| https://descramble.d4science.org/group/descramble-gateway/authorization<br />
|-<br />
| GEMex Gateway<br />
| https://gemex.d4science.org/group/gemex-gateway/authorization<br />
|-<br />
| OpenAIRE-Connect Gateway<br />
| https://openaire-connect.d4science.org/group/openaire-connect-gateway/authorization<br />
|-<br />
| PARTHENOS Gateway<br />
| https://parthenos.d4science.org/group/parthenos-gateway/authorization<br />
|-<br />
| SoBigData Gateway<br />
| https://sobigdata.d4science.org/group/sobigdata-gateway/authorization<br />
|}<br />
<br />
'''Retrieve the authorization endpoint location dynamically (ADVANCED MODE)'''<br />
<br />
Registry Swagger REST-API avaiable at: https://dev.d4science.org/swagger/registry/<br />
<br />
Which requires an [https://dev.d4science.org/authorization/ authorization token]. The returned profile is something like<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<serviceEndpoints><br />
<Resource version="0.4.x"><br />
<ID>b1c5851f-1d47-4c60-8906-941ab859a334</ID><br />
<Type>RuntimeResource</Type><br />
<Scopes><br />
<Scope>/gcube</Scope><br />
</Scopes><br />
...<br />
<AccessPoint><br />
<Description>AuthorisationServer where to authenticate external SmartGears services oAuth like</Description><br />
<Interface><br />
<Endpoint EntryName="AuthorisationServer">https://next.d4science.org/group/next/authorization</Endpoint><br />
</Interface><br />
</AccessPoint><br />
</Profile><br />
</Resource><br />
</serviceEndpoints><br />
</source><br />
<br />
=== Invoking the authorization service===<br />
<br />
Once you got the AuthorisationServer endpoint and the call is executed, two things can happen:<br />
* the user is requested to perform the login, thus accepting the application operation and granting it;<br />
* if there is a valid existing permission grant for the application, the login operation is by-passed and the redirect_uri is followed.<br />
<br />
The request is an '''HTTP GET''' call, that needs the following parameters to be provided<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| client_id<br />
| the client identifier associated to the application<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the redirect uri provided at registration phase<br />
| yes<br />
|-<br />
| state<br />
| a unique string value hard to guess, needed to prevent CRSF attacks (e.g. AAedBEeF545sdfKef897)<br />
| yes<br />
|-<br />
| scope<br />
| this field is optional. It can be specified to request (with the next step) an access token valid for a given<br />
infrastructure scope. If missing, the scope will be choose by the user on credentials insertion.<br />
| no<br />
|}<br />
<br />
This is an example in development environment<br />
<pre><br />
https://next.d4science.org/group/next/authorization?client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&scope=%2Fgcube&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&state=7d12bf13-111c-4f46-ab06-9e9e08ad377b<br />
</pre><br />
<br />
==== End user Experience ====<br />
<br />
Once redirected, the user will be presented with D4Science's authentication dialog box. This identifies your application as well as outlines the particular member permissions that your application requests. The logo and application name can be configurable, in the example the application name is '''Application X'''<br />
<br />
[[File:Oauth-ux.jpg]]<br />
<br />
==== Application is Accepted ====<br />
<br />
If the user grants permissions to the application, a redirection to the redirect_uri is performed. Attached to the redirect_uri, two important url arguments are provided<br />
<br />
* code: which is the authorization code. It expires in 10 seconds;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
==== Application is Rejected ====<br />
<br />
If, for any reason, the request fails, the redirect_uri is still followed but the url has attached the following parameters<br />
* error: a code that states the type of error;<br />
* error_description: a textual description of the actual error;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
== Step 3 - Exchange Authorization Code for an Access Token ==<br />
This is the step in which the previously obtained code can be exchanged to obtain a real access token. <br />
This time, an '''HTTP POST''' method needs to be performed, with a '''Content-Type''' of type '''application/x-www-form-urlencoded'''.<br />
<br />
The client_id and secret must be passed as the basic auth header Base64 encoded, see the example in Java below:<br />
<br />
e.g.<br />
<source lang="java"><br />
...<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
..<br />
</source><br />
<br />
The request-body must contain the following parameters:<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| grant_type<br />
| The value of this field should always be authorization_code<br />
| yes<br />
|-<br />
| code<br />
| the authorization code you received from the previous step.<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the same 'redirect_uri' value that you<br />
passed in the previous step.<br />
| yes<br />
|}<br />
<br />
For instance, in development environment<br />
<br />
<source lang="xml"><br />
POST /gcube-oauth/v2/access-token HTTP/1.1<br />
Host: socialnetworking-d-d4s.d4science.org<br />
Content-Type: application/x-www-form-urlencoded<br />
<br />
grant_type=authorization_code&code=...&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&client_secret=...<br />
</source><br />
<br />
In case of success, the response will contain a JSON object, having these properties:<br />
<br />
* access_token: the obtained access token to use to perform operations on user's behalf;<br />
* expires_in: the time to live of the token expressed in seconds;<br />
* scope: in which scope the token is valid;<br />
* token_type: that is "Bearer".<br />
<br />
In case of failures, instead, the returned object will contain:<br />
<br />
* error : an error string according the oauth 2.0 RFC.<br />
* error_description : a text description for the error.<br />
<br />
=== D4Science oAuth2 endpoint location === <br />
In the '''production''' environment, its value is<br />
<source lang="xml"><br />
https://api.d4science.org/oauth/v2/access-token<br />
</source><br />
<br />
'''HTTP POST Example in JAVA'''<br />
<br />
<source lang="java"><br />
String ACCESS_TOKEN_ATTR = "access_token";<br />
String SCOPE_ATTR = "scope";<br />
String fullPath2oAuthService = "https://api.d4science.org/oauth/v2/access-token";<br />
try {<br />
CloseableHttpClient httpClient = HttpClientBuilder.create().setRedirectStrategy(new LaxRedirectStrategy()).build();<br />
HttpPost httpPostRequest = new HttpPost(fullPath2oAuthService);<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
httpPostRequest.addHeader("Content-Type", "application/x-www-form-urlencoded");<br />
List<NameValuePair> params = new ArrayList<NameValuePair>();<br />
params.add(new BasicNameValuePair("grant_type", "authorization_code"));<br />
params.add(new BasicNameValuePair("code", tempCode));<br />
params.add(new BasicNameValuePair("redirect_uri", "https://www.gcube-system.org/oauth-authentication-example/example-app"));<br />
httpPostRequest.setEntity(new UrlEncodedFormEntity(params));<br />
<br />
CloseableHttpResponse httpresponse = httpClient.execute(httpPostRequest);<br />
int responseCode = httpresponse.getStatusLine().getStatusCode();<br />
if (responseCode < 200 || responseCode >= 300) {<br />
System.out.println("Error while getting authorization code, response status line from "<br />
+ fullPath2oAuthService + " was: " + responseCode); <br />
}<br />
JSONParser parser = new JSONParser();<br />
JSONObject json = (JSONObject) parser.parse(EntityUtils.toString(httpresponse.getEntity()));<br />
accessToken = json.get(ACCESS_TOKEN_ATTR).toString();<br />
scope = json.get(SCOPE_ATTR).toString();<br />
} catch(Exception e){<br />
System.out.println("Failed to perform request");<br />
e.printStackTrace();<br />
<br />
}<br />
</source><br />
<br />
== Step 4 - Get member data ==<br />
Having an access token, your application can perform some operations on behalf of the user. For instance, it can retrieve user's information profile by exploiting the [https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface social networking web service].<br />
<br />
Its current production endpoint is: https://api/rest/<br />
<br />
The subservice to call, once the social networking service endpoint has been retrieved, is this one<br />
<br />
<source lang="xml"><br />
2/users/get-oauth-profile<br />
</source><br />
<br />
It requires a ''gcube-token'' as query parameter, whose value is exactly the access token retrieved in the previous step. The result is something like this<br />
<br />
<source lang="ruby"><br />
{<br />
id: ".....",<br />
roles: [ ], // roles in the given-token context, e.g. VRE-Manager<br />
picture: "....",<br />
name: "....",<br />
middle_name: "....",<br />
male: as boolean ,<br />
location_industry: "....",<br />
given_name: "....",<br />
email: "....",<br />
job_title: "....",<br />
family_name: "....",<br />
verified_email: as boolean <br />
}<br />
</source><br />
<br />
== See Also ==<br />
<br />
[https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface Social Networking Service]<br />
<br />
[https://dev.d4science.org/swagger/social-networking/ Social Networking Service Docs SWAGGER]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=OAuth2.0&diff=32809OAuth2.02019-07-26T13:51:38Z<p>Massimiliano.assante: /* Get the gcube-oauth endpoint location */</p>
<hr />
<div>__TOC__<br />
<br />
= OAuth 2.0 gCube Support =<br />
By means of the OAuth 2.0 protocol (authorised) third party applications can operate on user's behalf over the D4Science infrastructure (while protecting the member's credentials).<br />
For more information about the OAuth authorization framework please visit the official [https://oauth.net/2/ OAuth site]. For technical details you could also check the [https://tools.ietf.org/html/rfc6749| OAuth 2.0 RFC]. In the following, the needed steps to authorize third party applications to operate on user's behalf and the D4Science infrastructure are explained. <br />
<br />
== Step 1 - Configure your application ==<br />
Authorized third party applications must be registered on the Infrastructure. Up to now the application registration is not automatic but you need to open a ticket at https://support.d4science.org/projects/d4science.<br />
<br />
The ticket needs to have '''3rd Party Application Support''' as tracker. The application name as well as the redirect uri need to be provided (https is strongly suggested). This url must be absolute, and without arguments. Once the application is registered, it will have associated:<br />
<br />
* a unique client identifier (client_id);<br />
* a client secret which must be kept secret, as the name itself says (client_secret).<br />
<br />
<source lang="xml"><br />
client_id: ab9d32c9-9425-42f1-99d7-50d68e651e9c<br />
client_secret: y9r93zDv4D<br />
</source><br />
<br />
Both are needed to proceed with the OAuth 2.0 workflow.<br />
<br />
== Step 2 - Require an authorization code ==<br />
Once the application is registered and configured, an authorization code can be requested. This code is not the final one needed to perform operations on user behalf, but is the one that will be exchanged for that in the next step. To require this code that has a very short time to live, the application must invoke the authorization service, which is gateway dependant.<br />
<br />
=== Retrieving the authorization endpoint location===<br />
<br />
The previous authorization endpoint location is portal's gateway dependant and '''can be found in Table 1''' or retrieved dinamically by using the registry service REST-API hosted at <br />
<br />
''https://registry.d4science.org/icproxy/gcube/service/'' <br />
<br />
'''Get authorization endpoint location in the following Table 1'''<br />
<br />
{| class="wikitable"<br />
|+Table 1: authorization endpoints location depending on the D4Science Gateway<br />
! style="font-weight: bold;" | D4Science Gateway name<br />
! style="font-weight: bold;" | Authorization endpoint<br />
|-<br />
| ARIADNEplus Gateway<br />
| https://ariadne.d4science.org/group/ariadneplus-gateway/authorization<br />
|-<br />
| iMarine Gateway<br />
| https://i-marine.d4science.org/group/imarine-gateway/authorization<br />
|-<br />
| AGINFRA+ Gateway<br />
| https://aginfra.d4science.org/group/aginfra-gateway/authorization<br />
|-<br />
| BlueBRIDGE Gateway<br />
| https://bluebridge.d4science.org/group/bluebridge-gateway/authorization<br />
|-<br />
| D4Science.org Gateway<br />
| https://services.d4science.org/group/d4science-services-gateway/authorization<br />
|-<br />
| DESCRAMBLE Gateway<br />
| https://descramble.d4science.org/group/descramble-gateway/authorization<br />
|-<br />
| GEMex Gateway<br />
| https://gemex.d4science.org/group/gemex-gateway/authorization<br />
|-<br />
| OpenAIRE-Connect Gateway<br />
| https://openaire-connect.d4science.org/group/openaire-connect-gateway/authorization<br />
|-<br />
| PARTHENOS Gateway<br />
| https://parthenos.d4science.org/group/parthenos-gateway/authorization<br />
|-<br />
| SoBigData Gateway<br />
| https://sobigdata.d4science.org/group/sobigdata-gateway/authorization<br />
|}<br />
<br />
'''Retrieve the authorization endpoint location dynamically (ADVANCED MODE)'''<br />
<br />
Registry Swagger REST-API avaiable at: https://dev.d4science.org/swagger/registry/<br />
<br />
Which requires an [https://dev.d4science.org/authorization/ authorization token]. The returned profile is something like<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<serviceEndpoints><br />
<Resource version="0.4.x"><br />
<ID>b1c5851f-1d47-4c60-8906-941ab859a334</ID><br />
<Type>RuntimeResource</Type><br />
<Scopes><br />
<Scope>/gcube</Scope><br />
</Scopes><br />
...<br />
<AccessPoint><br />
<Description>AuthorisationServer where to authenticate external SmartGears services oAuth like</Description><br />
<Interface><br />
<Endpoint EntryName="AuthorisationServer">https://next.d4science.org/group/next/authorization</Endpoint><br />
</Interface><br />
</AccessPoint><br />
</Profile><br />
</Resource><br />
</serviceEndpoints><br />
</source><br />
<br />
=== Invoking the authorization service===<br />
<br />
Once you got the AuthorisationServer endpoint and the call is executed, two things can happen:<br />
* the user is requested to perform the login, thus accepting the application operation and granting it;<br />
* if there is a valid existing permission grant for the application, the login operation is by-passed and the redirect_uri is followed.<br />
<br />
The request is an '''HTTP GET''' call, that needs the following parameters to be provided<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| client_id<br />
| the client identifier associated to the application<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the redirect uri provided at registration phase<br />
| yes<br />
|-<br />
| state<br />
| a unique string value hard to guess, needed to prevent CRSF attacks (e.g. AAedBEeF545sdfKef897)<br />
| yes<br />
|-<br />
| scope<br />
| this field is optional. It can be specified to request (with the next step) an access token valid for a given<br />
infrastructure scope. If missing, the scope will be choose by the user on credentials insertion.<br />
| no<br />
|}<br />
<br />
This is an example in development environment<br />
<pre><br />
https://next.d4science.org/group/next/authorization?client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&scope=%2Fgcube&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&state=7d12bf13-111c-4f46-ab06-9e9e08ad377b<br />
</pre><br />
<br />
==== End user Experience ====<br />
<br />
Once redirected, the user will be presented with D4Science's authentication dialog box. This identifies your application as well as outlines the particular member permissions that your application requests. The logo and application name can be configurable, in the example the application name is '''Application X'''<br />
<br />
[[File:Oauth-ux.jpg]]<br />
<br />
==== Application is Accepted ====<br />
<br />
If the user grants permissions to the application, a redirection to the redirect_uri is performed. Attached to the redirect_uri, two important url arguments are provided<br />
<br />
* code: which is the authorization code. It expires in 10 seconds;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
==== Application is Rejected ====<br />
<br />
If, for any reason, the request fails, the redirect_uri is still followed but the url has attached the following parameters<br />
* error: a code that states the type of error;<br />
* error_description: a textual description of the actual error;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
== Step 3 - Exchange Authorization Code for an Access Token ==<br />
This is the step in which the previously obtained code can be exchanged to obtain a real access token. <br />
This time, an '''HTTP POST''' method needs to be performed, with a '''Content-Type''' of type '''application/x-www-form-urlencoded'''.<br />
<br />
The client_id and secret must be passed as the basic auth header Base64 encoded, see the example in Java below:<br />
<br />
e.g.<br />
<source lang="java"><br />
...<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
..<br />
</source><br />
<br />
The request-body must contain the following parameters:<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| grant_type<br />
| The value of this field should always be authorization_code<br />
| yes<br />
|-<br />
| code<br />
| the authorization code you received from the previous step.<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the same 'redirect_uri' value that you<br />
passed in the previous step.<br />
| yes<br />
|}<br />
<br />
For instance, in development environment<br />
<br />
<source lang="xml"><br />
POST /gcube-oauth/v2/access-token HTTP/1.1<br />
Host: socialnetworking-d-d4s.d4science.org<br />
Content-Type: application/x-www-form-urlencoded<br />
<br />
grant_type=authorization_code&code=...&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&client_secret=...<br />
</source><br />
<br />
In case of success, the response will contain a JSON object, having these properties:<br />
<br />
* access_token: the obtained access token to use to perform operations on user's behalf;<br />
* expires_in: the time to live of the token expressed in seconds;<br />
* scope: in which scope the token is valid;<br />
* token_type: that is "Bearer".<br />
<br />
In case of failures, instead, the returned object will contain:<br />
<br />
* error : an error string according the oauth 2.0 RFC.<br />
* error_description : a text description for the error.<br />
<br />
=== D4Science oAuth2 endpoint location === <br />
In the '''production''' environment, its value is<br />
<source lang="xml"><br />
https://api.d4science.org/oauth/v2/access-token<br />
</source><br />
<br />
'''HTTP POST Example in JAVA'''<br />
<br />
<source lang="java"><br />
String ACCESS_TOKEN_ATTR = "access_token";<br />
String SCOPE_ATTR = "scope";<br />
String fullPath2oAuthService = "https://api.d4science.org/oauth/v2/access-token";<br />
try {<br />
CloseableHttpClient httpClient = HttpClientBuilder.create().setRedirectStrategy(new LaxRedirectStrategy()).build();<br />
HttpPost httpPostRequest = new HttpPost(fullPath2oAuthService);<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
httpPostRequest.addHeader("Content-Type", "application/x-www-form-urlencoded");<br />
List<NameValuePair> params = new ArrayList<NameValuePair>();<br />
params.add(new BasicNameValuePair("grant_type", "authorization_code"));<br />
params.add(new BasicNameValuePair("code", tempCode));<br />
params.add(new BasicNameValuePair("redirect_uri", "https://www.gcube-system.org/oauth-authentication-example/example-app"));<br />
httpPostRequest.setEntity(new UrlEncodedFormEntity(params));<br />
<br />
CloseableHttpResponse httpresponse = httpClient.execute(httpPostRequest);<br />
int responseCode = httpresponse.getStatusLine().getStatusCode();<br />
if (responseCode < 200 || responseCode >= 300) {<br />
System.out.println("Error while getting authorization code, response status line from "<br />
+ fullPath2oAuthService + " was: " + responseCode); <br />
}<br />
JSONParser parser = new JSONParser();<br />
JSONObject json = (JSONObject) parser.parse(EntityUtils.toString(httpresponse.getEntity()));<br />
accessToken = json.get(ACCESS_TOKEN_ATTR).toString();<br />
scope = json.get(SCOPE_ATTR).toString();<br />
} catch(Exception e){<br />
System.out.println("Failed to perform request");<br />
e.printStackTrace();<br />
<br />
}<br />
</source><br />
<br />
== Step 4 - Get member data ==<br />
Having an access token, your application can perform some operations on behalf of the user. For instance, it can retrieve user's information profile by exploiting the [https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface social networking web service].<br />
<br />
Its current production endpoint is: https://socialnetworking1.d4science.org/social-networking-library-ws/rest/<br />
<br />
The subservice to call, once the social networking service endpoint has been retrieved, is this one<br />
<br />
<source lang="xml"><br />
2/users/get-oauth-profile<br />
</source><br />
<br />
It requires a ''gcube-token'' as query parameter, whose value is exactly the access token retrieved in the previous step. The result is something like this<br />
<br />
<source lang="ruby"><br />
{<br />
id: ".....",<br />
roles: [ ], // roles in the given-token context, e.g. VRE-Manager<br />
picture: "....",<br />
name: "....",<br />
middle_name: "....",<br />
male: as boolean ,<br />
location_industry: "....",<br />
given_name: "....",<br />
email: "....",<br />
job_title: "....",<br />
family_name: "....",<br />
verified_email: as boolean <br />
}<br />
</source><br />
<br />
== See Also ==<br />
<br />
[https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface Social Networking Service]<br />
<br />
[https://dev.d4science.org/swagger/social-networking/ Social Networking Service Docs SWAGGER]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=OAuth2.0&diff=32808OAuth2.02019-07-26T13:45:25Z<p>Massimiliano.assante: /* Get the gcube-oauth endpoint location */</p>
<hr />
<div>__TOC__<br />
<br />
= OAuth 2.0 gCube Support =<br />
By means of the OAuth 2.0 protocol (authorised) third party applications can operate on user's behalf over the D4Science infrastructure (while protecting the member's credentials).<br />
For more information about the OAuth authorization framework please visit the official [https://oauth.net/2/ OAuth site]. For technical details you could also check the [https://tools.ietf.org/html/rfc6749| OAuth 2.0 RFC]. In the following, the needed steps to authorize third party applications to operate on user's behalf and the D4Science infrastructure are explained. <br />
<br />
== Step 1 - Configure your application ==<br />
Authorized third party applications must be registered on the Infrastructure. Up to now the application registration is not automatic but you need to open a ticket at https://support.d4science.org/projects/d4science.<br />
<br />
The ticket needs to have '''3rd Party Application Support''' as tracker. The application name as well as the redirect uri need to be provided (https is strongly suggested). This url must be absolute, and without arguments. Once the application is registered, it will have associated:<br />
<br />
* a unique client identifier (client_id);<br />
* a client secret which must be kept secret, as the name itself says (client_secret).<br />
<br />
<source lang="xml"><br />
client_id: ab9d32c9-9425-42f1-99d7-50d68e651e9c<br />
client_secret: y9r93zDv4D<br />
</source><br />
<br />
Both are needed to proceed with the OAuth 2.0 workflow.<br />
<br />
== Step 2 - Require an authorization code ==<br />
Once the application is registered and configured, an authorization code can be requested. This code is not the final one needed to perform operations on user behalf, but is the one that will be exchanged for that in the next step. To require this code that has a very short time to live, the application must invoke the authorization service, which is gateway dependant.<br />
<br />
=== Retrieving the authorization endpoint location===<br />
<br />
The previous authorization endpoint location is portal's gateway dependant and '''can be found in Table 1''' or retrieved dinamically by using the registry service REST-API hosted at <br />
<br />
''https://registry.d4science.org/icproxy/gcube/service/'' <br />
<br />
'''Get authorization endpoint location in the following Table 1'''<br />
<br />
{| class="wikitable"<br />
|+Table 1: authorization endpoints location depending on the D4Science Gateway<br />
! style="font-weight: bold;" | D4Science Gateway name<br />
! style="font-weight: bold;" | Authorization endpoint<br />
|-<br />
| ARIADNEplus Gateway<br />
| https://ariadne.d4science.org/group/ariadneplus-gateway/authorization<br />
|-<br />
| iMarine Gateway<br />
| https://i-marine.d4science.org/group/imarine-gateway/authorization<br />
|-<br />
| AGINFRA+ Gateway<br />
| https://aginfra.d4science.org/group/aginfra-gateway/authorization<br />
|-<br />
| BlueBRIDGE Gateway<br />
| https://bluebridge.d4science.org/group/bluebridge-gateway/authorization<br />
|-<br />
| D4Science.org Gateway<br />
| https://services.d4science.org/group/d4science-services-gateway/authorization<br />
|-<br />
| DESCRAMBLE Gateway<br />
| https://descramble.d4science.org/group/descramble-gateway/authorization<br />
|-<br />
| GEMex Gateway<br />
| https://gemex.d4science.org/group/gemex-gateway/authorization<br />
|-<br />
| OpenAIRE-Connect Gateway<br />
| https://openaire-connect.d4science.org/group/openaire-connect-gateway/authorization<br />
|-<br />
| PARTHENOS Gateway<br />
| https://parthenos.d4science.org/group/parthenos-gateway/authorization<br />
|-<br />
| SoBigData Gateway<br />
| https://sobigdata.d4science.org/group/sobigdata-gateway/authorization<br />
|}<br />
<br />
'''Retrieve the authorization endpoint location dynamically (ADVANCED MODE)'''<br />
<br />
Registry Swagger REST-API avaiable at: https://dev.d4science.org/swagger/registry/<br />
<br />
Which requires an [https://dev.d4science.org/authorization/ authorization token]. The returned profile is something like<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<serviceEndpoints><br />
<Resource version="0.4.x"><br />
<ID>b1c5851f-1d47-4c60-8906-941ab859a334</ID><br />
<Type>RuntimeResource</Type><br />
<Scopes><br />
<Scope>/gcube</Scope><br />
</Scopes><br />
...<br />
<AccessPoint><br />
<Description>AuthorisationServer where to authenticate external SmartGears services oAuth like</Description><br />
<Interface><br />
<Endpoint EntryName="AuthorisationServer">https://next.d4science.org/group/next/authorization</Endpoint><br />
</Interface><br />
</AccessPoint><br />
</Profile><br />
</Resource><br />
</serviceEndpoints><br />
</source><br />
<br />
=== Invoking the authorization service===<br />
<br />
Once you got the AuthorisationServer endpoint and the call is executed, two things can happen:<br />
* the user is requested to perform the login, thus accepting the application operation and granting it;<br />
* if there is a valid existing permission grant for the application, the login operation is by-passed and the redirect_uri is followed.<br />
<br />
The request is an '''HTTP GET''' call, that needs the following parameters to be provided<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| client_id<br />
| the client identifier associated to the application<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the redirect uri provided at registration phase<br />
| yes<br />
|-<br />
| state<br />
| a unique string value hard to guess, needed to prevent CRSF attacks (e.g. AAedBEeF545sdfKef897)<br />
| yes<br />
|-<br />
| scope<br />
| this field is optional. It can be specified to request (with the next step) an access token valid for a given<br />
infrastructure scope. If missing, the scope will be choose by the user on credentials insertion.<br />
| no<br />
|}<br />
<br />
This is an example in development environment<br />
<pre><br />
https://next.d4science.org/group/next/authorization?client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&scope=%2Fgcube&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&state=7d12bf13-111c-4f46-ab06-9e9e08ad377b<br />
</pre><br />
<br />
==== End user Experience ====<br />
<br />
Once redirected, the user will be presented with D4Science's authentication dialog box. This identifies your application as well as outlines the particular member permissions that your application requests. The logo and application name can be configurable, in the example the application name is '''Application X'''<br />
<br />
[[File:Oauth-ux.jpg]]<br />
<br />
==== Application is Accepted ====<br />
<br />
If the user grants permissions to the application, a redirection to the redirect_uri is performed. Attached to the redirect_uri, two important url arguments are provided<br />
<br />
* code: which is the authorization code. It expires in 10 seconds;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
==== Application is Rejected ====<br />
<br />
If, for any reason, the request fails, the redirect_uri is still followed but the url has attached the following parameters<br />
* error: a code that states the type of error;<br />
* error_description: a textual description of the actual error;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
== Step 3 - Exchange Authorization Code for an Access Token ==<br />
This is the step in which the previously obtained code can be exchanged to obtain a real access token. <br />
This time, an '''HTTP POST''' method needs to be performed, with a '''Content-Type''' of type '''application/x-www-form-urlencoded'''.<br />
<br />
The client_id and secret must be passed as the basic auth header Base64 encoded, see the example in Java below:<br />
<br />
e.g.<br />
<source lang="java"><br />
...<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
..<br />
</source><br />
<br />
The request-body must contain the following parameters:<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| grant_type<br />
| The value of this field should always be authorization_code<br />
| yes<br />
|-<br />
| code<br />
| the authorization code you received from the previous step.<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the same 'redirect_uri' value that you<br />
passed in the previous step.<br />
| yes<br />
|}<br />
<br />
For instance, in development environment<br />
<br />
<source lang="xml"><br />
POST /gcube-oauth/v2/access-token HTTP/1.1<br />
Host: socialnetworking-d-d4s.d4science.org<br />
Content-Type: application/x-www-form-urlencoded<br />
<br />
grant_type=authorization_code&code=...&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&client_secret=...<br />
</source><br />
<br />
In case of success, the response will contain a JSON object, having these properties:<br />
<br />
* access_token: the obtained access token to use to perform operations on user's behalf;<br />
* expires_in: the time to live of the token expressed in seconds;<br />
* scope: in which scope the token is valid;<br />
* token_type: that is "Bearer".<br />
<br />
In case of failures, instead, the returned object will contain:<br />
<br />
* error : an error string according the oauth 2.0 RFC.<br />
* error_description : a text description for the error.<br />
<br />
=== Get the gcube-oauth endpoint location === <br />
The host to contact to perform this call can be again discovered via the icproxy service. This time the url will be something like<br />
<br />
<source lang="xml"><br />
http://node10-d-d4s.d4science.org/icproxy/gcube/service/GCoreEndpoint/Portal/oauth?gcube-token=35eaf849-43d3-439c-a3bc-2f7c355c0f67<br />
</source> <br />
<br />
A resource profile is returned which contains the property ''jersey-servlet'' to be retrieved.<br />
<br />
In '''development''' environment its current value is<br />
<br />
<source lang="xml"><br />
http://socialnetworking-d-d4s.d4science.org/gcube-oauth/v2/access-token<br />
</source><br />
<br />
In the '''production''' environment, its value is<br />
<source lang="xml"><br />
https://api.d4science.org/oauth/v2/access-token<br />
</source><br />
<br />
'''HTTP POST Example in JAVA'''<br />
<br />
<source lang="java"><br />
String ACCESS_TOKEN_ATTR = "access_token";<br />
String SCOPE_ATTR = "scope";<br />
String fullPath2oAuthService = "https://socialnetworking1.d4science.org/gcube-oauth/v2/access-token";<br />
try {<br />
CloseableHttpClient httpClient = HttpClientBuilder.create().setRedirectStrategy(new LaxRedirectStrategy()).build();<br />
HttpPost httpPostRequest = new HttpPost(fullPath2oAuthService);<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
httpPostRequest.addHeader("Content-Type", "application/x-www-form-urlencoded");<br />
List<NameValuePair> params = new ArrayList<NameValuePair>();<br />
params.add(new BasicNameValuePair("grant_type", "authorization_code"));<br />
params.add(new BasicNameValuePair("code", tempCode));<br />
params.add(new BasicNameValuePair("redirect_uri", "https://www.gcube-system.org/oauth-authentication-example/example-app"));<br />
httpPostRequest.setEntity(new UrlEncodedFormEntity(params));<br />
<br />
CloseableHttpResponse httpresponse = httpClient.execute(httpPostRequest);<br />
int responseCode = httpresponse.getStatusLine().getStatusCode();<br />
if (responseCode < 200 || responseCode >= 300) {<br />
System.out.println("Error while getting authorization code, response status line from "<br />
+ fullPath2oAuthService + " was: " + responseCode); <br />
}<br />
JSONParser parser = new JSONParser();<br />
JSONObject json = (JSONObject) parser.parse(EntityUtils.toString(httpresponse.getEntity()));<br />
accessToken = json.get(ACCESS_TOKEN_ATTR).toString();<br />
scope = json.get(SCOPE_ATTR).toString();<br />
} catch(Exception e){<br />
System.out.println("Failed to perform request");<br />
e.printStackTrace();<br />
<br />
}<br />
</source><br />
<br />
== Step 4 - Get member data ==<br />
Having an access token, your application can perform some operations on behalf of the user. For instance, it can retrieve user's information profile by exploiting the [https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface social networking web service].<br />
<br />
Its current production endpoint is: https://socialnetworking1.d4science.org/social-networking-library-ws/rest/<br />
<br />
The subservice to call, once the social networking service endpoint has been retrieved, is this one<br />
<br />
<source lang="xml"><br />
2/users/get-oauth-profile<br />
</source><br />
<br />
It requires a ''gcube-token'' as query parameter, whose value is exactly the access token retrieved in the previous step. The result is something like this<br />
<br />
<source lang="ruby"><br />
{<br />
id: ".....",<br />
roles: [ ], // roles in the given-token context, e.g. VRE-Manager<br />
picture: "....",<br />
name: "....",<br />
middle_name: "....",<br />
male: as boolean ,<br />
location_industry: "....",<br />
given_name: "....",<br />
email: "....",<br />
job_title: "....",<br />
family_name: "....",<br />
verified_email: as boolean <br />
}<br />
</source><br />
<br />
== See Also ==<br />
<br />
[https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface Social Networking Service]<br />
<br />
[https://dev.d4science.org/swagger/social-networking/ Social Networking Service Docs SWAGGER]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=User%27s_Guide&diff=32187User's Guide2019-04-29T08:20:02Z<p>Massimiliano.assante: /* Data Management Facilities */</p>
<hr />
<div>[[Category:User's Guide]]<br />
<br />
The User's guide is targeted towards the end-user of a ''Virtual Research Environment'' (VRE). <br />
It focuses on the community portals that provide access to the VRE capabilities for content management and collaboration. <br />
<br />
<br />
=== Basic Facilities ===<br />
: [[Log in User Guide | How to Log in]]<br />
: [[Workspace | How to organize your Workspace]]<br />
: [[Sharing Posts and using News feed | How to use the social communication tools]]<br />
: [[Token generator | Retrieve/Generate a security token ]]<br />
<br />
=== Data Management Facilities ===<br />
: [[Tabular Data Manager | How to managing tabular resources such as very large datasets, codelist and generic tables ]]<br />
: [[How to add my data to the gCube Spatial Data Infrastructure]]<br />
<br />
=== Data Consumption Facilities ===<br />
: [[GeoExplorer | How to access and visualising Geospatial Data]]<br />
: [[Statistical Manager Tutorial | How to execute Data Analytics tasks]]<br />
: [[Statistical Algorithms Importer | How to add a new algorithm to the Data Analytics platform]]<br />
: [[AquaMaps Suite | How to produce AquaMaps Objects]]<br />
:: [[Species Explorer | How to browse AquaMaps Objects]]<br />
<!-- [[Search User Guide | How to Search and Browse Information Objects]] --><br />
: [[Databases Manager Tutorial | How to inspect the structure of databases hosted by the infrastructure]]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=GCube_Portal_Engine&diff=32171GCube Portal Engine2019-04-08T11:56:44Z<p>Massimiliano.assante: /* Main Technologies */</p>
<hr />
<div><!-- CATEGORIES --><br />
[[Category:Developer's Guide]]<br />
<!-- END CATEGORIES --><br />
== Introduction ==<br />
To provide the end user with the full functionality of the gCube system, a presentation application, based on the ASL libraries and the featherweight ones, has been implemented.<br />
<br />
In gCube the [Liferay portal] is employed as the portlet-hosting platform. Liferay Portal offers a complete platform for building web apps, mobile apps, and web services quickly, using features and frameworks designed for rapid development, good performance, and ease of use. It runs on all major application servers and servlets containers and it is JSR 168 and JSR 286 compliant. <br />
<br />
Portlets are using JSR 286 and several other technologies such as Java Server Pages for dynamically generation of HTML/XML documents in response to a client’s request, and the most popular Front-End frameworks such as Angular, React or Vue.<br />
<br />
All these technologies are hosted under the Liferay Portal engine.<br />
<br />
== Main Technologies ==<br />
=== JSR 286 ===<br />
JSR (Java Specification Request) 286 establishes a standard API for creating portlets. It constitutes the integration component between applications and portals, and it enables delivery of an application through a portal. Without this standard, each version of an application needed its own portlet API, and each of the various portals required that these portlets should be specifically tailor-made for that portal. Inter-Portlet Communication through events, public render parameters<br />
and dynamically generated resources directly through portlets are some of the API's advantages<br />
<br />
=== Angular ===<br />
Angular is a platform that makes it easy to build applications with the web. Angular combines declarative templates, dependency injection, end to end tooling, and integrated best practices to solve development challenges. Angular empowers developers to build applications that live on the web, mobile, or the desktop.<br />
<br />
<br />
=== JSP ===<br />
Java Server Pages (JSP) is a Java technology that allows developers to dynamically generate HTML, XML or other types of documents in response to a Web client request. The technology allows Java code and certain pre-defined actions to be embedded into static content.<br />
The JSP syntax adds additional XML-like tags, called JSP actions, to be used to invoke built-in functionality. Additionally, the technology allows for the creation of JSP tag libraries that act as extensions to the standard HTML or XML tags. Tag libraries provide a platform independent way of extending the capabilities of a Web server.<br />
JSPs are compiled into Java Servlets by a JSP compiler. A JSP compiler may generate a servlet in Java code that is then compiled by the Java compiler, or it may generate byte code for the servlet directly. JSPs can also be interpreted on-the-fly reducing the time taken to reload changes.<br />
<br />
== Other Technologies ==<br />
=== GWT-OpenLayers ===<br />
GWT-OpenLayers is a Java wrapper for the OpenLayers JavaScript API. It allows GWT projects to use the OpenLayers JavaScript API. OpenLayers makes it easy to put a dynamic map in any web page. It can display map tiles and markers loaded from any source. OpenLayers has been developed to further the use of geographic information of all kinds. OpenLayers is completely free, Open Source JavaScript, released under the 2-clause BSD.<br />
<br />
=== jQuery ===<br />
jQuery is a fast, small, and feature-rich JavaScript library. It makes things like HTML document traversal and manipulation, event handling, animation, and Ajax much simpler with an easy-to-use API that works across a multitude of browsers.</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=GCube_Portal_Engine&diff=32170GCube Portal Engine2019-04-08T11:55:53Z<p>Massimiliano.assante: /* Introduction */</p>
<hr />
<div><!-- CATEGORIES --><br />
[[Category:Developer's Guide]]<br />
<!-- END CATEGORIES --><br />
== Introduction ==<br />
To provide the end user with the full functionality of the gCube system, a presentation application, based on the ASL libraries and the featherweight ones, has been implemented.<br />
<br />
In gCube the [Liferay portal] is employed as the portlet-hosting platform. Liferay Portal offers a complete platform for building web apps, mobile apps, and web services quickly, using features and frameworks designed for rapid development, good performance, and ease of use. It runs on all major application servers and servlets containers and it is JSR 168 and JSR 286 compliant. <br />
<br />
Portlets are using JSR 286 and several other technologies such as Java Server Pages for dynamically generation of HTML/XML documents in response to a client’s request, and the most popular Front-End frameworks such as Angular, React or Vue.<br />
<br />
All these technologies are hosted under the Liferay Portal engine.<br />
<br />
== Main Technologies ==<br />
=== JSR 286 ===<br />
JSR (Java Specification Request) 286 establishes a standard API for creating portlets. It constitutes the integration component between applications and portals, and it enables delivery of an application through a portal. Without this standard, each version of an application needed its own portlet API, and each of the various portals required that these portlets should be specifically tailor-made for that portal. Inter-Portlet Communication through events, public render parameters<br />
and dynamically generated resources directly through portlets are some of the API's advantages<br />
<br />
=== GWT ===<br />
Google Web Toolkit (GWT) is an open source Java software development framework that allows web developers to create Ajax applications in Java. It is licensed under the Apache License version 2.0.<br />
GWT emphasizes reusable, efficient solutions to recurring Ajax challenges, namely asynchronous remote procedure calls, history management, bookmarking, and cross-browser portability. For all these reasons most portlets use the GWT for a better and more efficient Ajax implementation.<br />
<br />
=== Sencha GXT ===<br />
Sencha GXT is the fastest, most powerful way to create rich web-based applications using Java.<br>Sencha GXT uses the Google Web Toolkit (GWT) compiler that allows you to write your applications in Java and compile your code into highly optimized cross-browser HTML5 and JavaScript. Sencha GXT takes GWT to the next level, giving you high-performance widgets, feature-rich templates and layouts, advanced charting, data loaders and stores, and much more.<br />
<br />
=== JSP ===<br />
Java Server Pages (JSP) is a Java technology that allows developers to dynamically generate HTML, XML or other types of documents in response to a Web client request. The technology allows Java code and certain pre-defined actions to be embedded into static content.<br />
The JSP syntax adds additional XML-like tags, called JSP actions, to be used to invoke built-in functionality. Additionally, the technology allows for the creation of JSP tag libraries that act as extensions to the standard HTML or XML tags. Tag libraries provide a platform independent way of extending the capabilities of a Web server.<br />
JSPs are compiled into Java Servlets by a JSP compiler. A JSP compiler may generate a servlet in Java code that is then compiled by the Java compiler, or it may generate byte code for the servlet directly. JSPs can also be interpreted on-the-fly reducing the time taken to reload changes.<br />
<br />
== Other Technologies ==<br />
=== GWT-OpenLayers ===<br />
GWT-OpenLayers is a Java wrapper for the OpenLayers JavaScript API. It allows GWT projects to use the OpenLayers JavaScript API. OpenLayers makes it easy to put a dynamic map in any web page. It can display map tiles and markers loaded from any source. OpenLayers has been developed to further the use of geographic information of all kinds. OpenLayers is completely free, Open Source JavaScript, released under the 2-clause BSD.<br />
<br />
=== jQuery ===<br />
jQuery is a fast, small, and feature-rich JavaScript library. It makes things like HTML document traversal and manipulation, event handling, animation, and Ajax much simpler with an easy-to-use API that works across a multitude of browsers.</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=OAuth2.0&diff=32161OAuth2.02019-04-05T10:28:06Z<p>Massimiliano.assante: /* Step 4 - Get member data */</p>
<hr />
<div>__TOC__<br />
<br />
= OAuth 2.0 gCube Support =<br />
By means of the OAuth 2.0 protocol (authorised) third party applications can operate on user's behalf over the D4Science infrastructure (while protecting the member's credentials).<br />
For more information about the OAuth authorization framework please visit the official [https://oauth.net/2/ OAuth site]. For technical details you could also check the [https://tools.ietf.org/html/rfc6749| OAuth 2.0 RFC]. In the following, the needed steps to authorize third party applications to operate on user's behalf and the D4Science infrastructure are explained. <br />
<br />
== Step 1 - Configure your application ==<br />
Authorized third party applications must be registered on the Infrastructure. Up to now the application registration is not automatic but you need to open a ticket at https://support.d4science.org/projects/d4science.<br />
<br />
The ticket needs to have '''3rd Party Application Support''' as tracker. The application name as well as the redirect uri need to be provided (https is strongly suggested). This url must be absolute, and without arguments. Once the application is registered, it will have associated:<br />
<br />
* a unique client identifier (client_id);<br />
* a client secret which must be kept secret, as the name itself says (client_secret).<br />
<br />
<source lang="xml"><br />
client_id: ab9d32c9-9425-42f1-99d7-50d68e651e9c<br />
client_secret: y9r93zDv4D<br />
</source><br />
<br />
Both are needed to proceed with the OAuth 2.0 workflow.<br />
<br />
== Step 2 - Require an authorization code ==<br />
Once the application is registered and configured, an authorization code can be requested. This code is not the final one needed to perform operations on user behalf, but is the one that will be exchanged for that in the next step. To require this code that has a very short time to live, the application must invoke the authorization service, which is gateway dependant.<br />
<br />
=== Retrieving the authorization endpoint location===<br />
<br />
The previous authorization endpoint location is portal's gateway dependant and '''can be found in Table 1''' or retrieved dinamically by using the registry service REST-API hosted at <br />
<br />
''https://registry.d4science.org/icproxy/gcube/service/'' <br />
<br />
'''Get authorization endpoint location in the following Table 1'''<br />
<br />
{| class="wikitable"<br />
|+Table 1: authorization endpoints location depending on the D4Science Gateway<br />
! style="font-weight: bold;" | D4Science Gateway name<br />
! style="font-weight: bold;" | Authorization endpoint<br />
|-<br />
| ARIADNEplus Gateway<br />
| https://ariadne.d4science.org/group/ariadneplus-gateway/authorization<br />
|-<br />
| iMarine Gateway<br />
| https://i-marine.d4science.org/group/imarine-gateway/authorization<br />
|-<br />
| AGINFRA+ Gateway<br />
| https://aginfra.d4science.org/group/aginfra-gateway/authorization<br />
|-<br />
| BlueBRIDGE Gateway<br />
| https://bluebridge.d4science.org/group/bluebridge-gateway/authorization<br />
|-<br />
| D4Science.org Gateway<br />
| https://services.d4science.org/group/d4science-services-gateway/authorization<br />
|-<br />
| DESCRAMBLE Gateway<br />
| https://descramble.d4science.org/group/descramble-gateway/authorization<br />
|-<br />
| GEMex Gateway<br />
| https://gemex.d4science.org/group/gemex-gateway/authorization<br />
|-<br />
| OpenAIRE-Connect Gateway<br />
| https://openaire-connect.d4science.org/group/openaire-connect-gateway/authorization<br />
|-<br />
| PARTHENOS Gateway<br />
| https://parthenos.d4science.org/group/parthenos-gateway/authorization<br />
|-<br />
| SoBigData Gateway<br />
| https://sobigdata.d4science.org/group/sobigdata-gateway/authorization<br />
|}<br />
<br />
'''Retrieve the authorization endpoint location dynamically (ADVANCED MODE)'''<br />
<br />
Registry Swagger REST-API avaiable at: https://dev.d4science.org/swagger/registry/<br />
<br />
Which requires an [https://dev.d4science.org/authorization/ authorization token]. The returned profile is something like<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<serviceEndpoints><br />
<Resource version="0.4.x"><br />
<ID>b1c5851f-1d47-4c60-8906-941ab859a334</ID><br />
<Type>RuntimeResource</Type><br />
<Scopes><br />
<Scope>/gcube</Scope><br />
</Scopes><br />
...<br />
<AccessPoint><br />
<Description>AuthorisationServer where to authenticate external SmartGears services oAuth like</Description><br />
<Interface><br />
<Endpoint EntryName="AuthorisationServer">https://next.d4science.org/group/next/authorization</Endpoint><br />
</Interface><br />
</AccessPoint><br />
</Profile><br />
</Resource><br />
</serviceEndpoints><br />
</source><br />
<br />
=== Invoking the authorization service===<br />
<br />
Once you got the AuthorisationServer endpoint and the call is executed, two things can happen:<br />
* the user is requested to perform the login, thus accepting the application operation and granting it;<br />
* if there is a valid existing permission grant for the application, the login operation is by-passed and the redirect_uri is followed.<br />
<br />
The request is an '''HTTP GET''' call, that needs the following parameters to be provided<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| client_id<br />
| the client identifier associated to the application<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the redirect uri provided at registration phase<br />
| yes<br />
|-<br />
| state<br />
| a unique string value hard to guess, needed to prevent CRSF attacks (e.g. AAedBEeF545sdfKef897)<br />
| yes<br />
|-<br />
| scope<br />
| this field is optional. It can be specified to request (with the next step) an access token valid for a given<br />
infrastructure scope. If missing, the scope will be choose by the user on credentials insertion.<br />
| no<br />
|}<br />
<br />
This is an example in development environment<br />
<pre><br />
https://next.d4science.org/group/next/authorization?client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&scope=%2Fgcube&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&state=7d12bf13-111c-4f46-ab06-9e9e08ad377b<br />
</pre><br />
<br />
==== End user Experience ====<br />
<br />
Once redirected, the user will be presented with D4Science's authentication dialog box. This identifies your application as well as outlines the particular member permissions that your application requests. The logo and application name can be configurable, in the example the application name is '''Application X'''<br />
<br />
[[File:Oauth-ux.jpg]]<br />
<br />
==== Application is Accepted ====<br />
<br />
If the user grants permissions to the application, a redirection to the redirect_uri is performed. Attached to the redirect_uri, two important url arguments are provided<br />
<br />
* code: which is the authorization code. It expires in 10 seconds;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
==== Application is Rejected ====<br />
<br />
If, for any reason, the request fails, the redirect_uri is still followed but the url has attached the following parameters<br />
* error: a code that states the type of error;<br />
* error_description: a textual description of the actual error;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
== Step 3 - Exchange Authorization Code for an Access Token ==<br />
This is the step in which the previously obtained code can be exchanged to obtain a real access token. <br />
This time, an '''HTTP POST''' method needs to be performed, with a '''Content-Type''' of type '''application/x-www-form-urlencoded'''.<br />
<br />
The client_id and secret must be passed as the basic auth header Base64 encoded, see the example in Java below:<br />
<br />
e.g.<br />
<source lang="java"><br />
...<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
..<br />
</source><br />
<br />
The request-body must contain the following parameters:<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| grant_type<br />
| The value of this field should always be authorization_code<br />
| yes<br />
|-<br />
| code<br />
| the authorization code you received from the previous step.<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the same 'redirect_uri' value that you<br />
passed in the previous step.<br />
| yes<br />
|}<br />
<br />
For instance, in development environment<br />
<br />
<source lang="xml"><br />
POST /gcube-oauth/v2/access-token HTTP/1.1<br />
Host: socialnetworking-d-d4s.d4science.org<br />
Content-Type: application/x-www-form-urlencoded<br />
<br />
grant_type=authorization_code&code=...&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&client_secret=...<br />
</source><br />
<br />
In case of success, the response will contain a JSON object, having these properties:<br />
<br />
* access_token: the obtained access token to use to perform operations on user's behalf;<br />
* expires_in: the time to live of the token expressed in seconds;<br />
* scope: in which scope the token is valid;<br />
* token_type: that is "Bearer".<br />
<br />
In case of failures, instead, the returned object will contain:<br />
<br />
* error : an error string according the oauth 2.0 RFC.<br />
* error_description : a text description for the error.<br />
<br />
=== Get the gcube-oauth endpoint location === <br />
The host to contact to perform this call can be again discovered via the icproxy service. This time the url will be something like<br />
<br />
<source lang="xml"><br />
http://node10-d-d4s.d4science.org/icproxy/gcube/service/GCoreEndpoint/Portal/oauth?gcube-token=35eaf849-43d3-439c-a3bc-2f7c355c0f67<br />
</source> <br />
<br />
A resource profile is returned which contains the property ''jersey-servlet'' to be retrieved.<br />
<br />
In '''development''' environment its current value is<br />
<br />
<source lang="xml"><br />
http://socialnetworking-d-d4s.d4science.org/gcube-oauth/v2/access-token<br />
</source><br />
<br />
In the '''production''' environment, its current value is<br />
<source lang="xml"><br />
https://socialnetworking1.d4science.org/gcube-oauth/v2/access-token<br />
</source><br />
<br />
'''HTTP POST Example in JAVA'''<br />
<br />
<source lang="java"><br />
String ACCESS_TOKEN_ATTR = "access_token";<br />
String SCOPE_ATTR = "scope";<br />
String fullPath2oAuthService = "https://socialnetworking1.d4science.org/gcube-oauth/v2/access-token";<br />
try {<br />
CloseableHttpClient httpClient = HttpClientBuilder.create().setRedirectStrategy(new LaxRedirectStrategy()).build();<br />
HttpPost httpPostRequest = new HttpPost(fullPath2oAuthService);<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
httpPostRequest.addHeader("Content-Type", "application/x-www-form-urlencoded");<br />
List<NameValuePair> params = new ArrayList<NameValuePair>();<br />
params.add(new BasicNameValuePair("grant_type", "authorization_code"));<br />
params.add(new BasicNameValuePair("code", tempCode));<br />
params.add(new BasicNameValuePair("redirect_uri", "https://www.gcube-system.org/oauth-authentication-example/example-app"));<br />
httpPostRequest.setEntity(new UrlEncodedFormEntity(params));<br />
<br />
CloseableHttpResponse httpresponse = httpClient.execute(httpPostRequest);<br />
int responseCode = httpresponse.getStatusLine().getStatusCode();<br />
if (responseCode < 200 || responseCode >= 300) {<br />
System.out.println("Error while getting authorization code, response status line from "<br />
+ fullPath2oAuthService + " was: " + responseCode); <br />
}<br />
JSONParser parser = new JSONParser();<br />
JSONObject json = (JSONObject) parser.parse(EntityUtils.toString(httpresponse.getEntity()));<br />
accessToken = json.get(ACCESS_TOKEN_ATTR).toString();<br />
scope = json.get(SCOPE_ATTR).toString();<br />
} catch(Exception e){<br />
System.out.println("Failed to perform request");<br />
e.printStackTrace();<br />
<br />
}<br />
</source><br />
<br />
== Step 4 - Get member data ==<br />
Having an access token, your application can perform some operations on behalf of the user. For instance, it can retrieve user's information profile by exploiting the [https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface social networking web service].<br />
<br />
Its current production endpoint is: https://socialnetworking1.d4science.org/social-networking-library-ws/rest/<br />
<br />
The subservice to call, once the social networking service endpoint has been retrieved, is this one<br />
<br />
<source lang="xml"><br />
2/users/get-oauth-profile<br />
</source><br />
<br />
It requires a ''gcube-token'' as query parameter, whose value is exactly the access token retrieved in the previous step. The result is something like this<br />
<br />
<source lang="ruby"><br />
{<br />
id: ".....",<br />
roles: [ ], // roles in the given-token context, e.g. VRE-Manager<br />
picture: "....",<br />
name: "....",<br />
middle_name: "....",<br />
male: as boolean ,<br />
location_industry: "....",<br />
given_name: "....",<br />
email: "....",<br />
job_title: "....",<br />
family_name: "....",<br />
verified_email: as boolean <br />
}<br />
</source><br />
<br />
== See Also ==<br />
<br />
[https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface Social Networking Service]<br />
<br />
[https://dev.d4science.org/swagger/social-networking/ Social Networking Service Docs SWAGGER]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=OAuth2.0&diff=32160OAuth2.02019-04-05T10:25:43Z<p>Massimiliano.assante: /* Step 4 - Get member data */</p>
<hr />
<div>__TOC__<br />
<br />
= OAuth 2.0 gCube Support =<br />
By means of the OAuth 2.0 protocol (authorised) third party applications can operate on user's behalf over the D4Science infrastructure (while protecting the member's credentials).<br />
For more information about the OAuth authorization framework please visit the official [https://oauth.net/2/ OAuth site]. For technical details you could also check the [https://tools.ietf.org/html/rfc6749| OAuth 2.0 RFC]. In the following, the needed steps to authorize third party applications to operate on user's behalf and the D4Science infrastructure are explained. <br />
<br />
== Step 1 - Configure your application ==<br />
Authorized third party applications must be registered on the Infrastructure. Up to now the application registration is not automatic but you need to open a ticket at https://support.d4science.org/projects/d4science.<br />
<br />
The ticket needs to have '''3rd Party Application Support''' as tracker. The application name as well as the redirect uri need to be provided (https is strongly suggested). This url must be absolute, and without arguments. Once the application is registered, it will have associated:<br />
<br />
* a unique client identifier (client_id);<br />
* a client secret which must be kept secret, as the name itself says (client_secret).<br />
<br />
<source lang="xml"><br />
client_id: ab9d32c9-9425-42f1-99d7-50d68e651e9c<br />
client_secret: y9r93zDv4D<br />
</source><br />
<br />
Both are needed to proceed with the OAuth 2.0 workflow.<br />
<br />
== Step 2 - Require an authorization code ==<br />
Once the application is registered and configured, an authorization code can be requested. This code is not the final one needed to perform operations on user behalf, but is the one that will be exchanged for that in the next step. To require this code that has a very short time to live, the application must invoke the authorization service, which is gateway dependant.<br />
<br />
=== Retrieving the authorization endpoint location===<br />
<br />
The previous authorization endpoint location is portal's gateway dependant and '''can be found in Table 1''' or retrieved dinamically by using the registry service REST-API hosted at <br />
<br />
''https://registry.d4science.org/icproxy/gcube/service/'' <br />
<br />
'''Get authorization endpoint location in the following Table 1'''<br />
<br />
{| class="wikitable"<br />
|+Table 1: authorization endpoints location depending on the D4Science Gateway<br />
! style="font-weight: bold;" | D4Science Gateway name<br />
! style="font-weight: bold;" | Authorization endpoint<br />
|-<br />
| ARIADNEplus Gateway<br />
| https://ariadne.d4science.org/group/ariadneplus-gateway/authorization<br />
|-<br />
| iMarine Gateway<br />
| https://i-marine.d4science.org/group/imarine-gateway/authorization<br />
|-<br />
| AGINFRA+ Gateway<br />
| https://aginfra.d4science.org/group/aginfra-gateway/authorization<br />
|-<br />
| BlueBRIDGE Gateway<br />
| https://bluebridge.d4science.org/group/bluebridge-gateway/authorization<br />
|-<br />
| D4Science.org Gateway<br />
| https://services.d4science.org/group/d4science-services-gateway/authorization<br />
|-<br />
| DESCRAMBLE Gateway<br />
| https://descramble.d4science.org/group/descramble-gateway/authorization<br />
|-<br />
| GEMex Gateway<br />
| https://gemex.d4science.org/group/gemex-gateway/authorization<br />
|-<br />
| OpenAIRE-Connect Gateway<br />
| https://openaire-connect.d4science.org/group/openaire-connect-gateway/authorization<br />
|-<br />
| PARTHENOS Gateway<br />
| https://parthenos.d4science.org/group/parthenos-gateway/authorization<br />
|-<br />
| SoBigData Gateway<br />
| https://sobigdata.d4science.org/group/sobigdata-gateway/authorization<br />
|}<br />
<br />
'''Retrieve the authorization endpoint location dynamically (ADVANCED MODE)'''<br />
<br />
Registry Swagger REST-API avaiable at: https://dev.d4science.org/swagger/registry/<br />
<br />
Which requires an [https://dev.d4science.org/authorization/ authorization token]. The returned profile is something like<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<serviceEndpoints><br />
<Resource version="0.4.x"><br />
<ID>b1c5851f-1d47-4c60-8906-941ab859a334</ID><br />
<Type>RuntimeResource</Type><br />
<Scopes><br />
<Scope>/gcube</Scope><br />
</Scopes><br />
...<br />
<AccessPoint><br />
<Description>AuthorisationServer where to authenticate external SmartGears services oAuth like</Description><br />
<Interface><br />
<Endpoint EntryName="AuthorisationServer">https://next.d4science.org/group/next/authorization</Endpoint><br />
</Interface><br />
</AccessPoint><br />
</Profile><br />
</Resource><br />
</serviceEndpoints><br />
</source><br />
<br />
=== Invoking the authorization service===<br />
<br />
Once you got the AuthorisationServer endpoint and the call is executed, two things can happen:<br />
* the user is requested to perform the login, thus accepting the application operation and granting it;<br />
* if there is a valid existing permission grant for the application, the login operation is by-passed and the redirect_uri is followed.<br />
<br />
The request is an '''HTTP GET''' call, that needs the following parameters to be provided<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| client_id<br />
| the client identifier associated to the application<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the redirect uri provided at registration phase<br />
| yes<br />
|-<br />
| state<br />
| a unique string value hard to guess, needed to prevent CRSF attacks (e.g. AAedBEeF545sdfKef897)<br />
| yes<br />
|-<br />
| scope<br />
| this field is optional. It can be specified to request (with the next step) an access token valid for a given<br />
infrastructure scope. If missing, the scope will be choose by the user on credentials insertion.<br />
| no<br />
|}<br />
<br />
This is an example in development environment<br />
<pre><br />
https://next.d4science.org/group/next/authorization?client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&scope=%2Fgcube&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&state=7d12bf13-111c-4f46-ab06-9e9e08ad377b<br />
</pre><br />
<br />
==== End user Experience ====<br />
<br />
Once redirected, the user will be presented with D4Science's authentication dialog box. This identifies your application as well as outlines the particular member permissions that your application requests. The logo and application name can be configurable, in the example the application name is '''Application X'''<br />
<br />
[[File:Oauth-ux.jpg]]<br />
<br />
==== Application is Accepted ====<br />
<br />
If the user grants permissions to the application, a redirection to the redirect_uri is performed. Attached to the redirect_uri, two important url arguments are provided<br />
<br />
* code: which is the authorization code. It expires in 10 seconds;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
==== Application is Rejected ====<br />
<br />
If, for any reason, the request fails, the redirect_uri is still followed but the url has attached the following parameters<br />
* error: a code that states the type of error;<br />
* error_description: a textual description of the actual error;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
== Step 3 - Exchange Authorization Code for an Access Token ==<br />
This is the step in which the previously obtained code can be exchanged to obtain a real access token. <br />
This time, an '''HTTP POST''' method needs to be performed, with a '''Content-Type''' of type '''application/x-www-form-urlencoded'''.<br />
<br />
The client_id and secret must be passed as the basic auth header Base64 encoded, see the example in Java below:<br />
<br />
e.g.<br />
<source lang="java"><br />
...<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
..<br />
</source><br />
<br />
The request-body must contain the following parameters:<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| grant_type<br />
| The value of this field should always be authorization_code<br />
| yes<br />
|-<br />
| code<br />
| the authorization code you received from the previous step.<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the same 'redirect_uri' value that you<br />
passed in the previous step.<br />
| yes<br />
|}<br />
<br />
For instance, in development environment<br />
<br />
<source lang="xml"><br />
POST /gcube-oauth/v2/access-token HTTP/1.1<br />
Host: socialnetworking-d-d4s.d4science.org<br />
Content-Type: application/x-www-form-urlencoded<br />
<br />
grant_type=authorization_code&code=...&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&client_secret=...<br />
</source><br />
<br />
In case of success, the response will contain a JSON object, having these properties:<br />
<br />
* access_token: the obtained access token to use to perform operations on user's behalf;<br />
* expires_in: the time to live of the token expressed in seconds;<br />
* scope: in which scope the token is valid;<br />
* token_type: that is "Bearer".<br />
<br />
In case of failures, instead, the returned object will contain:<br />
<br />
* error : an error string according the oauth 2.0 RFC.<br />
* error_description : a text description for the error.<br />
<br />
=== Get the gcube-oauth endpoint location === <br />
The host to contact to perform this call can be again discovered via the icproxy service. This time the url will be something like<br />
<br />
<source lang="xml"><br />
http://node10-d-d4s.d4science.org/icproxy/gcube/service/GCoreEndpoint/Portal/oauth?gcube-token=35eaf849-43d3-439c-a3bc-2f7c355c0f67<br />
</source> <br />
<br />
A resource profile is returned which contains the property ''jersey-servlet'' to be retrieved.<br />
<br />
In '''development''' environment its current value is<br />
<br />
<source lang="xml"><br />
http://socialnetworking-d-d4s.d4science.org/gcube-oauth/v2/access-token<br />
</source><br />
<br />
In the '''production''' environment, its current value is<br />
<source lang="xml"><br />
https://socialnetworking1.d4science.org/gcube-oauth/v2/access-token<br />
</source><br />
<br />
'''HTTP POST Example in JAVA'''<br />
<br />
<source lang="java"><br />
String ACCESS_TOKEN_ATTR = "access_token";<br />
String SCOPE_ATTR = "scope";<br />
String fullPath2oAuthService = "https://socialnetworking1.d4science.org/gcube-oauth/v2/access-token";<br />
try {<br />
CloseableHttpClient httpClient = HttpClientBuilder.create().setRedirectStrategy(new LaxRedirectStrategy()).build();<br />
HttpPost httpPostRequest = new HttpPost(fullPath2oAuthService);<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
httpPostRequest.addHeader("Content-Type", "application/x-www-form-urlencoded");<br />
List<NameValuePair> params = new ArrayList<NameValuePair>();<br />
params.add(new BasicNameValuePair("grant_type", "authorization_code"));<br />
params.add(new BasicNameValuePair("code", tempCode));<br />
params.add(new BasicNameValuePair("redirect_uri", "https://www.gcube-system.org/oauth-authentication-example/example-app"));<br />
httpPostRequest.setEntity(new UrlEncodedFormEntity(params));<br />
<br />
CloseableHttpResponse httpresponse = httpClient.execute(httpPostRequest);<br />
int responseCode = httpresponse.getStatusLine().getStatusCode();<br />
if (responseCode < 200 || responseCode >= 300) {<br />
System.out.println("Error while getting authorization code, response status line from "<br />
+ fullPath2oAuthService + " was: " + responseCode); <br />
}<br />
JSONParser parser = new JSONParser();<br />
JSONObject json = (JSONObject) parser.parse(EntityUtils.toString(httpresponse.getEntity()));<br />
accessToken = json.get(ACCESS_TOKEN_ATTR).toString();<br />
scope = json.get(SCOPE_ATTR).toString();<br />
} catch(Exception e){<br />
System.out.println("Failed to perform request");<br />
e.printStackTrace();<br />
<br />
}<br />
</source><br />
<br />
== Step 4 - Get member data ==<br />
Having an access token, your application can perform some operations on behalf of the user. For instance, it can retrieve user's information profile by exploiting the [https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface social networking web service].<br />
<br />
Its current production endpoint is: https://socialnetworking1.d4science.org/social-networking-library-ws/rest/<br />
<br />
The subservice to call, once the social networking service endpoint has been retrieved, is this one<br />
<br />
<source lang="xml"><br />
2/users/get-oauth-profile<br />
</source><br />
<br />
It requires a ''gcube-token'' as query parameter, whose value is exactly the access token retrieved in the previous step. The result is something like this<br />
<br />
<source lang="ruby"><br />
{<br />
id: ".....",<br />
roles: [ ],<br />
picture: "....",<br />
name: "....",<br />
middle_name: "....",<br />
male: as boolean ,<br />
location_industry: "....",<br />
given_name: "....",<br />
email: "....",<br />
job_title: "....",<br />
family_name: "....",<br />
verified_email: as boolean <br />
}<br />
</source><br />
<br />
== See Also ==<br />
<br />
[https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface Social Networking Service]<br />
<br />
[https://dev.d4science.org/swagger/social-networking/ Social Networking Service Docs SWAGGER]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=OAuth2.0&diff=32159OAuth2.02019-04-05T10:25:31Z<p>Massimiliano.assante: /* Step 4 - Get member data */</p>
<hr />
<div>__TOC__<br />
<br />
= OAuth 2.0 gCube Support =<br />
By means of the OAuth 2.0 protocol (authorised) third party applications can operate on user's behalf over the D4Science infrastructure (while protecting the member's credentials).<br />
For more information about the OAuth authorization framework please visit the official [https://oauth.net/2/ OAuth site]. For technical details you could also check the [https://tools.ietf.org/html/rfc6749| OAuth 2.0 RFC]. In the following, the needed steps to authorize third party applications to operate on user's behalf and the D4Science infrastructure are explained. <br />
<br />
== Step 1 - Configure your application ==<br />
Authorized third party applications must be registered on the Infrastructure. Up to now the application registration is not automatic but you need to open a ticket at https://support.d4science.org/projects/d4science.<br />
<br />
The ticket needs to have '''3rd Party Application Support''' as tracker. The application name as well as the redirect uri need to be provided (https is strongly suggested). This url must be absolute, and without arguments. Once the application is registered, it will have associated:<br />
<br />
* a unique client identifier (client_id);<br />
* a client secret which must be kept secret, as the name itself says (client_secret).<br />
<br />
<source lang="xml"><br />
client_id: ab9d32c9-9425-42f1-99d7-50d68e651e9c<br />
client_secret: y9r93zDv4D<br />
</source><br />
<br />
Both are needed to proceed with the OAuth 2.0 workflow.<br />
<br />
== Step 2 - Require an authorization code ==<br />
Once the application is registered and configured, an authorization code can be requested. This code is not the final one needed to perform operations on user behalf, but is the one that will be exchanged for that in the next step. To require this code that has a very short time to live, the application must invoke the authorization service, which is gateway dependant.<br />
<br />
=== Retrieving the authorization endpoint location===<br />
<br />
The previous authorization endpoint location is portal's gateway dependant and '''can be found in Table 1''' or retrieved dinamically by using the registry service REST-API hosted at <br />
<br />
''https://registry.d4science.org/icproxy/gcube/service/'' <br />
<br />
'''Get authorization endpoint location in the following Table 1'''<br />
<br />
{| class="wikitable"<br />
|+Table 1: authorization endpoints location depending on the D4Science Gateway<br />
! style="font-weight: bold;" | D4Science Gateway name<br />
! style="font-weight: bold;" | Authorization endpoint<br />
|-<br />
| ARIADNEplus Gateway<br />
| https://ariadne.d4science.org/group/ariadneplus-gateway/authorization<br />
|-<br />
| iMarine Gateway<br />
| https://i-marine.d4science.org/group/imarine-gateway/authorization<br />
|-<br />
| AGINFRA+ Gateway<br />
| https://aginfra.d4science.org/group/aginfra-gateway/authorization<br />
|-<br />
| BlueBRIDGE Gateway<br />
| https://bluebridge.d4science.org/group/bluebridge-gateway/authorization<br />
|-<br />
| D4Science.org Gateway<br />
| https://services.d4science.org/group/d4science-services-gateway/authorization<br />
|-<br />
| DESCRAMBLE Gateway<br />
| https://descramble.d4science.org/group/descramble-gateway/authorization<br />
|-<br />
| GEMex Gateway<br />
| https://gemex.d4science.org/group/gemex-gateway/authorization<br />
|-<br />
| OpenAIRE-Connect Gateway<br />
| https://openaire-connect.d4science.org/group/openaire-connect-gateway/authorization<br />
|-<br />
| PARTHENOS Gateway<br />
| https://parthenos.d4science.org/group/parthenos-gateway/authorization<br />
|-<br />
| SoBigData Gateway<br />
| https://sobigdata.d4science.org/group/sobigdata-gateway/authorization<br />
|}<br />
<br />
'''Retrieve the authorization endpoint location dynamically (ADVANCED MODE)'''<br />
<br />
Registry Swagger REST-API avaiable at: https://dev.d4science.org/swagger/registry/<br />
<br />
Which requires an [https://dev.d4science.org/authorization/ authorization token]. The returned profile is something like<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<serviceEndpoints><br />
<Resource version="0.4.x"><br />
<ID>b1c5851f-1d47-4c60-8906-941ab859a334</ID><br />
<Type>RuntimeResource</Type><br />
<Scopes><br />
<Scope>/gcube</Scope><br />
</Scopes><br />
...<br />
<AccessPoint><br />
<Description>AuthorisationServer where to authenticate external SmartGears services oAuth like</Description><br />
<Interface><br />
<Endpoint EntryName="AuthorisationServer">https://next.d4science.org/group/next/authorization</Endpoint><br />
</Interface><br />
</AccessPoint><br />
</Profile><br />
</Resource><br />
</serviceEndpoints><br />
</source><br />
<br />
=== Invoking the authorization service===<br />
<br />
Once you got the AuthorisationServer endpoint and the call is executed, two things can happen:<br />
* the user is requested to perform the login, thus accepting the application operation and granting it;<br />
* if there is a valid existing permission grant for the application, the login operation is by-passed and the redirect_uri is followed.<br />
<br />
The request is an '''HTTP GET''' call, that needs the following parameters to be provided<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| client_id<br />
| the client identifier associated to the application<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the redirect uri provided at registration phase<br />
| yes<br />
|-<br />
| state<br />
| a unique string value hard to guess, needed to prevent CRSF attacks (e.g. AAedBEeF545sdfKef897)<br />
| yes<br />
|-<br />
| scope<br />
| this field is optional. It can be specified to request (with the next step) an access token valid for a given<br />
infrastructure scope. If missing, the scope will be choose by the user on credentials insertion.<br />
| no<br />
|}<br />
<br />
This is an example in development environment<br />
<pre><br />
https://next.d4science.org/group/next/authorization?client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&scope=%2Fgcube&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&state=7d12bf13-111c-4f46-ab06-9e9e08ad377b<br />
</pre><br />
<br />
==== End user Experience ====<br />
<br />
Once redirected, the user will be presented with D4Science's authentication dialog box. This identifies your application as well as outlines the particular member permissions that your application requests. The logo and application name can be configurable, in the example the application name is '''Application X'''<br />
<br />
[[File:Oauth-ux.jpg]]<br />
<br />
==== Application is Accepted ====<br />
<br />
If the user grants permissions to the application, a redirection to the redirect_uri is performed. Attached to the redirect_uri, two important url arguments are provided<br />
<br />
* code: which is the authorization code. It expires in 10 seconds;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
==== Application is Rejected ====<br />
<br />
If, for any reason, the request fails, the redirect_uri is still followed but the url has attached the following parameters<br />
* error: a code that states the type of error;<br />
* error_description: a textual description of the actual error;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
== Step 3 - Exchange Authorization Code for an Access Token ==<br />
This is the step in which the previously obtained code can be exchanged to obtain a real access token. <br />
This time, an '''HTTP POST''' method needs to be performed, with a '''Content-Type''' of type '''application/x-www-form-urlencoded'''.<br />
<br />
The client_id and secret must be passed as the basic auth header Base64 encoded, see the example in Java below:<br />
<br />
e.g.<br />
<source lang="java"><br />
...<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
..<br />
</source><br />
<br />
The request-body must contain the following parameters:<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| grant_type<br />
| The value of this field should always be authorization_code<br />
| yes<br />
|-<br />
| code<br />
| the authorization code you received from the previous step.<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the same 'redirect_uri' value that you<br />
passed in the previous step.<br />
| yes<br />
|}<br />
<br />
For instance, in development environment<br />
<br />
<source lang="xml"><br />
POST /gcube-oauth/v2/access-token HTTP/1.1<br />
Host: socialnetworking-d-d4s.d4science.org<br />
Content-Type: application/x-www-form-urlencoded<br />
<br />
grant_type=authorization_code&code=...&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&client_secret=...<br />
</source><br />
<br />
In case of success, the response will contain a JSON object, having these properties:<br />
<br />
* access_token: the obtained access token to use to perform operations on user's behalf;<br />
* expires_in: the time to live of the token expressed in seconds;<br />
* scope: in which scope the token is valid;<br />
* token_type: that is "Bearer".<br />
<br />
In case of failures, instead, the returned object will contain:<br />
<br />
* error : an error string according the oauth 2.0 RFC.<br />
* error_description : a text description for the error.<br />
<br />
=== Get the gcube-oauth endpoint location === <br />
The host to contact to perform this call can be again discovered via the icproxy service. This time the url will be something like<br />
<br />
<source lang="xml"><br />
http://node10-d-d4s.d4science.org/icproxy/gcube/service/GCoreEndpoint/Portal/oauth?gcube-token=35eaf849-43d3-439c-a3bc-2f7c355c0f67<br />
</source> <br />
<br />
A resource profile is returned which contains the property ''jersey-servlet'' to be retrieved.<br />
<br />
In '''development''' environment its current value is<br />
<br />
<source lang="xml"><br />
http://socialnetworking-d-d4s.d4science.org/gcube-oauth/v2/access-token<br />
</source><br />
<br />
In the '''production''' environment, its current value is<br />
<source lang="xml"><br />
https://socialnetworking1.d4science.org/gcube-oauth/v2/access-token<br />
</source><br />
<br />
'''HTTP POST Example in JAVA'''<br />
<br />
<source lang="java"><br />
String ACCESS_TOKEN_ATTR = "access_token";<br />
String SCOPE_ATTR = "scope";<br />
String fullPath2oAuthService = "https://socialnetworking1.d4science.org/gcube-oauth/v2/access-token";<br />
try {<br />
CloseableHttpClient httpClient = HttpClientBuilder.create().setRedirectStrategy(new LaxRedirectStrategy()).build();<br />
HttpPost httpPostRequest = new HttpPost(fullPath2oAuthService);<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
httpPostRequest.addHeader("Content-Type", "application/x-www-form-urlencoded");<br />
List<NameValuePair> params = new ArrayList<NameValuePair>();<br />
params.add(new BasicNameValuePair("grant_type", "authorization_code"));<br />
params.add(new BasicNameValuePair("code", tempCode));<br />
params.add(new BasicNameValuePair("redirect_uri", "https://www.gcube-system.org/oauth-authentication-example/example-app"));<br />
httpPostRequest.setEntity(new UrlEncodedFormEntity(params));<br />
<br />
CloseableHttpResponse httpresponse = httpClient.execute(httpPostRequest);<br />
int responseCode = httpresponse.getStatusLine().getStatusCode();<br />
if (responseCode < 200 || responseCode >= 300) {<br />
System.out.println("Error while getting authorization code, response status line from "<br />
+ fullPath2oAuthService + " was: " + responseCode); <br />
}<br />
JSONParser parser = new JSONParser();<br />
JSONObject json = (JSONObject) parser.parse(EntityUtils.toString(httpresponse.getEntity()));<br />
accessToken = json.get(ACCESS_TOKEN_ATTR).toString();<br />
scope = json.get(SCOPE_ATTR).toString();<br />
} catch(Exception e){<br />
System.out.println("Failed to perform request");<br />
e.printStackTrace();<br />
<br />
}<br />
</source><br />
<br />
== Step 4 - Get member data ==<br />
Having an access token, your application can perform some operations on behalf of the user. For instance, it can retrieve user's information profile by exploiting the [https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface social networking web service].<br />
<br />
Its current production endpoint is: https://socialnetworking1.d4science.org/social-networking-library-ws/rest/<br />
<br />
The subservice to call, once the social networking service endpoint has been retrieved, is this one<br />
<br />
<source lang="xml"><br />
2/users/get-oauth-profile<br />
</source><br />
<br />
It requires a ''gcube-token'' as query parameter, whose value is exactly the access token retrieved in the previous step. The result is something like this<br />
<br />
<source lang="xml"><br />
{<br />
id: ".....",<br />
roles: [ ],<br />
picture: "....",<br />
name: "....",<br />
middle_name: "....",<br />
male: as boolean ,<br />
location_industry: "....",<br />
given_name: "....",<br />
email: "....",<br />
job_title: "....",<br />
family_name: "....",<br />
verified_email: as boolean <br />
}<br />
</source><br />
<br />
== See Also ==<br />
<br />
[https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface Social Networking Service]<br />
<br />
[https://dev.d4science.org/swagger/social-networking/ Social Networking Service Docs SWAGGER]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=OAuth2.0&diff=32158OAuth2.02019-04-05T10:25:20Z<p>Massimiliano.assante: /* Step 4 - Get member data */</p>
<hr />
<div>__TOC__<br />
<br />
= OAuth 2.0 gCube Support =<br />
By means of the OAuth 2.0 protocol (authorised) third party applications can operate on user's behalf over the D4Science infrastructure (while protecting the member's credentials).<br />
For more information about the OAuth authorization framework please visit the official [https://oauth.net/2/ OAuth site]. For technical details you could also check the [https://tools.ietf.org/html/rfc6749| OAuth 2.0 RFC]. In the following, the needed steps to authorize third party applications to operate on user's behalf and the D4Science infrastructure are explained. <br />
<br />
== Step 1 - Configure your application ==<br />
Authorized third party applications must be registered on the Infrastructure. Up to now the application registration is not automatic but you need to open a ticket at https://support.d4science.org/projects/d4science.<br />
<br />
The ticket needs to have '''3rd Party Application Support''' as tracker. The application name as well as the redirect uri need to be provided (https is strongly suggested). This url must be absolute, and without arguments. Once the application is registered, it will have associated:<br />
<br />
* a unique client identifier (client_id);<br />
* a client secret which must be kept secret, as the name itself says (client_secret).<br />
<br />
<source lang="xml"><br />
client_id: ab9d32c9-9425-42f1-99d7-50d68e651e9c<br />
client_secret: y9r93zDv4D<br />
</source><br />
<br />
Both are needed to proceed with the OAuth 2.0 workflow.<br />
<br />
== Step 2 - Require an authorization code ==<br />
Once the application is registered and configured, an authorization code can be requested. This code is not the final one needed to perform operations on user behalf, but is the one that will be exchanged for that in the next step. To require this code that has a very short time to live, the application must invoke the authorization service, which is gateway dependant.<br />
<br />
=== Retrieving the authorization endpoint location===<br />
<br />
The previous authorization endpoint location is portal's gateway dependant and '''can be found in Table 1''' or retrieved dinamically by using the registry service REST-API hosted at <br />
<br />
''https://registry.d4science.org/icproxy/gcube/service/'' <br />
<br />
'''Get authorization endpoint location in the following Table 1'''<br />
<br />
{| class="wikitable"<br />
|+Table 1: authorization endpoints location depending on the D4Science Gateway<br />
! style="font-weight: bold;" | D4Science Gateway name<br />
! style="font-weight: bold;" | Authorization endpoint<br />
|-<br />
| ARIADNEplus Gateway<br />
| https://ariadne.d4science.org/group/ariadneplus-gateway/authorization<br />
|-<br />
| iMarine Gateway<br />
| https://i-marine.d4science.org/group/imarine-gateway/authorization<br />
|-<br />
| AGINFRA+ Gateway<br />
| https://aginfra.d4science.org/group/aginfra-gateway/authorization<br />
|-<br />
| BlueBRIDGE Gateway<br />
| https://bluebridge.d4science.org/group/bluebridge-gateway/authorization<br />
|-<br />
| D4Science.org Gateway<br />
| https://services.d4science.org/group/d4science-services-gateway/authorization<br />
|-<br />
| DESCRAMBLE Gateway<br />
| https://descramble.d4science.org/group/descramble-gateway/authorization<br />
|-<br />
| GEMex Gateway<br />
| https://gemex.d4science.org/group/gemex-gateway/authorization<br />
|-<br />
| OpenAIRE-Connect Gateway<br />
| https://openaire-connect.d4science.org/group/openaire-connect-gateway/authorization<br />
|-<br />
| PARTHENOS Gateway<br />
| https://parthenos.d4science.org/group/parthenos-gateway/authorization<br />
|-<br />
| SoBigData Gateway<br />
| https://sobigdata.d4science.org/group/sobigdata-gateway/authorization<br />
|}<br />
<br />
'''Retrieve the authorization endpoint location dynamically (ADVANCED MODE)'''<br />
<br />
Registry Swagger REST-API avaiable at: https://dev.d4science.org/swagger/registry/<br />
<br />
Which requires an [https://dev.d4science.org/authorization/ authorization token]. The returned profile is something like<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<serviceEndpoints><br />
<Resource version="0.4.x"><br />
<ID>b1c5851f-1d47-4c60-8906-941ab859a334</ID><br />
<Type>RuntimeResource</Type><br />
<Scopes><br />
<Scope>/gcube</Scope><br />
</Scopes><br />
...<br />
<AccessPoint><br />
<Description>AuthorisationServer where to authenticate external SmartGears services oAuth like</Description><br />
<Interface><br />
<Endpoint EntryName="AuthorisationServer">https://next.d4science.org/group/next/authorization</Endpoint><br />
</Interface><br />
</AccessPoint><br />
</Profile><br />
</Resource><br />
</serviceEndpoints><br />
</source><br />
<br />
=== Invoking the authorization service===<br />
<br />
Once you got the AuthorisationServer endpoint and the call is executed, two things can happen:<br />
* the user is requested to perform the login, thus accepting the application operation and granting it;<br />
* if there is a valid existing permission grant for the application, the login operation is by-passed and the redirect_uri is followed.<br />
<br />
The request is an '''HTTP GET''' call, that needs the following parameters to be provided<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| client_id<br />
| the client identifier associated to the application<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the redirect uri provided at registration phase<br />
| yes<br />
|-<br />
| state<br />
| a unique string value hard to guess, needed to prevent CRSF attacks (e.g. AAedBEeF545sdfKef897)<br />
| yes<br />
|-<br />
| scope<br />
| this field is optional. It can be specified to request (with the next step) an access token valid for a given<br />
infrastructure scope. If missing, the scope will be choose by the user on credentials insertion.<br />
| no<br />
|}<br />
<br />
This is an example in development environment<br />
<pre><br />
https://next.d4science.org/group/next/authorization?client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&scope=%2Fgcube&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&state=7d12bf13-111c-4f46-ab06-9e9e08ad377b<br />
</pre><br />
<br />
==== End user Experience ====<br />
<br />
Once redirected, the user will be presented with D4Science's authentication dialog box. This identifies your application as well as outlines the particular member permissions that your application requests. The logo and application name can be configurable, in the example the application name is '''Application X'''<br />
<br />
[[File:Oauth-ux.jpg]]<br />
<br />
==== Application is Accepted ====<br />
<br />
If the user grants permissions to the application, a redirection to the redirect_uri is performed. Attached to the redirect_uri, two important url arguments are provided<br />
<br />
* code: which is the authorization code. It expires in 10 seconds;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
==== Application is Rejected ====<br />
<br />
If, for any reason, the request fails, the redirect_uri is still followed but the url has attached the following parameters<br />
* error: a code that states the type of error;<br />
* error_description: a textual description of the actual error;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
== Step 3 - Exchange Authorization Code for an Access Token ==<br />
This is the step in which the previously obtained code can be exchanged to obtain a real access token. <br />
This time, an '''HTTP POST''' method needs to be performed, with a '''Content-Type''' of type '''application/x-www-form-urlencoded'''.<br />
<br />
The client_id and secret must be passed as the basic auth header Base64 encoded, see the example in Java below:<br />
<br />
e.g.<br />
<source lang="java"><br />
...<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
..<br />
</source><br />
<br />
The request-body must contain the following parameters:<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| grant_type<br />
| The value of this field should always be authorization_code<br />
| yes<br />
|-<br />
| code<br />
| the authorization code you received from the previous step.<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the same 'redirect_uri' value that you<br />
passed in the previous step.<br />
| yes<br />
|}<br />
<br />
For instance, in development environment<br />
<br />
<source lang="xml"><br />
POST /gcube-oauth/v2/access-token HTTP/1.1<br />
Host: socialnetworking-d-d4s.d4science.org<br />
Content-Type: application/x-www-form-urlencoded<br />
<br />
grant_type=authorization_code&code=...&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&client_secret=...<br />
</source><br />
<br />
In case of success, the response will contain a JSON object, having these properties:<br />
<br />
* access_token: the obtained access token to use to perform operations on user's behalf;<br />
* expires_in: the time to live of the token expressed in seconds;<br />
* scope: in which scope the token is valid;<br />
* token_type: that is "Bearer".<br />
<br />
In case of failures, instead, the returned object will contain:<br />
<br />
* error : an error string according the oauth 2.0 RFC.<br />
* error_description : a text description for the error.<br />
<br />
=== Get the gcube-oauth endpoint location === <br />
The host to contact to perform this call can be again discovered via the icproxy service. This time the url will be something like<br />
<br />
<source lang="xml"><br />
http://node10-d-d4s.d4science.org/icproxy/gcube/service/GCoreEndpoint/Portal/oauth?gcube-token=35eaf849-43d3-439c-a3bc-2f7c355c0f67<br />
</source> <br />
<br />
A resource profile is returned which contains the property ''jersey-servlet'' to be retrieved.<br />
<br />
In '''development''' environment its current value is<br />
<br />
<source lang="xml"><br />
http://socialnetworking-d-d4s.d4science.org/gcube-oauth/v2/access-token<br />
</source><br />
<br />
In the '''production''' environment, its current value is<br />
<source lang="xml"><br />
https://socialnetworking1.d4science.org/gcube-oauth/v2/access-token<br />
</source><br />
<br />
'''HTTP POST Example in JAVA'''<br />
<br />
<source lang="java"><br />
String ACCESS_TOKEN_ATTR = "access_token";<br />
String SCOPE_ATTR = "scope";<br />
String fullPath2oAuthService = "https://socialnetworking1.d4science.org/gcube-oauth/v2/access-token";<br />
try {<br />
CloseableHttpClient httpClient = HttpClientBuilder.create().setRedirectStrategy(new LaxRedirectStrategy()).build();<br />
HttpPost httpPostRequest = new HttpPost(fullPath2oAuthService);<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
httpPostRequest.addHeader("Content-Type", "application/x-www-form-urlencoded");<br />
List<NameValuePair> params = new ArrayList<NameValuePair>();<br />
params.add(new BasicNameValuePair("grant_type", "authorization_code"));<br />
params.add(new BasicNameValuePair("code", tempCode));<br />
params.add(new BasicNameValuePair("redirect_uri", "https://www.gcube-system.org/oauth-authentication-example/example-app"));<br />
httpPostRequest.setEntity(new UrlEncodedFormEntity(params));<br />
<br />
CloseableHttpResponse httpresponse = httpClient.execute(httpPostRequest);<br />
int responseCode = httpresponse.getStatusLine().getStatusCode();<br />
if (responseCode < 200 || responseCode >= 300) {<br />
System.out.println("Error while getting authorization code, response status line from "<br />
+ fullPath2oAuthService + " was: " + responseCode); <br />
}<br />
JSONParser parser = new JSONParser();<br />
JSONObject json = (JSONObject) parser.parse(EntityUtils.toString(httpresponse.getEntity()));<br />
accessToken = json.get(ACCESS_TOKEN_ATTR).toString();<br />
scope = json.get(SCOPE_ATTR).toString();<br />
} catch(Exception e){<br />
System.out.println("Failed to perform request");<br />
e.printStackTrace();<br />
<br />
}<br />
</source><br />
<br />
== Step 4 - Get member data ==<br />
Having an access token, your application can perform some operations on behalf of the user. For instance, it can retrieve user's information profile by exploiting the [https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface social networking web service].<br />
<br />
Its current production endpoint is: https://socialnetworking1.d4science.org/social-networking-library-ws/rest/<br />
<br />
The subservice to call, once the social networking service endpoint has been retrieved, is this one<br />
<br />
<source lang="xml"><br />
2/users/get-oauth-profile<br />
</source><br />
<br />
It requires a ''gcube-token'' as query parameter, whose value is exactly the access token retrieved in the previous step. The result is something like this<br />
<br />
<source lang="json"><br />
{<br />
id: ".....",<br />
roles: [ ],<br />
picture: "....",<br />
name: "....",<br />
middle_name: "....",<br />
male: as boolean ,<br />
location_industry: "....",<br />
given_name: "....",<br />
email: "....",<br />
job_title: "....",<br />
family_name: "....",<br />
verified_email: as boolean <br />
}<br />
</source><br />
<br />
== See Also ==<br />
<br />
[https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface Social Networking Service]<br />
<br />
[https://dev.d4science.org/swagger/social-networking/ Social Networking Service Docs SWAGGER]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=OAuth2.0&diff=32157OAuth2.02019-04-05T10:25:06Z<p>Massimiliano.assante: /* Step 4 - Get member data */</p>
<hr />
<div>__TOC__<br />
<br />
= OAuth 2.0 gCube Support =<br />
By means of the OAuth 2.0 protocol (authorised) third party applications can operate on user's behalf over the D4Science infrastructure (while protecting the member's credentials).<br />
For more information about the OAuth authorization framework please visit the official [https://oauth.net/2/ OAuth site]. For technical details you could also check the [https://tools.ietf.org/html/rfc6749| OAuth 2.0 RFC]. In the following, the needed steps to authorize third party applications to operate on user's behalf and the D4Science infrastructure are explained. <br />
<br />
== Step 1 - Configure your application ==<br />
Authorized third party applications must be registered on the Infrastructure. Up to now the application registration is not automatic but you need to open a ticket at https://support.d4science.org/projects/d4science.<br />
<br />
The ticket needs to have '''3rd Party Application Support''' as tracker. The application name as well as the redirect uri need to be provided (https is strongly suggested). This url must be absolute, and without arguments. Once the application is registered, it will have associated:<br />
<br />
* a unique client identifier (client_id);<br />
* a client secret which must be kept secret, as the name itself says (client_secret).<br />
<br />
<source lang="xml"><br />
client_id: ab9d32c9-9425-42f1-99d7-50d68e651e9c<br />
client_secret: y9r93zDv4D<br />
</source><br />
<br />
Both are needed to proceed with the OAuth 2.0 workflow.<br />
<br />
== Step 2 - Require an authorization code ==<br />
Once the application is registered and configured, an authorization code can be requested. This code is not the final one needed to perform operations on user behalf, but is the one that will be exchanged for that in the next step. To require this code that has a very short time to live, the application must invoke the authorization service, which is gateway dependant.<br />
<br />
=== Retrieving the authorization endpoint location===<br />
<br />
The previous authorization endpoint location is portal's gateway dependant and '''can be found in Table 1''' or retrieved dinamically by using the registry service REST-API hosted at <br />
<br />
''https://registry.d4science.org/icproxy/gcube/service/'' <br />
<br />
'''Get authorization endpoint location in the following Table 1'''<br />
<br />
{| class="wikitable"<br />
|+Table 1: authorization endpoints location depending on the D4Science Gateway<br />
! style="font-weight: bold;" | D4Science Gateway name<br />
! style="font-weight: bold;" | Authorization endpoint<br />
|-<br />
| ARIADNEplus Gateway<br />
| https://ariadne.d4science.org/group/ariadneplus-gateway/authorization<br />
|-<br />
| iMarine Gateway<br />
| https://i-marine.d4science.org/group/imarine-gateway/authorization<br />
|-<br />
| AGINFRA+ Gateway<br />
| https://aginfra.d4science.org/group/aginfra-gateway/authorization<br />
|-<br />
| BlueBRIDGE Gateway<br />
| https://bluebridge.d4science.org/group/bluebridge-gateway/authorization<br />
|-<br />
| D4Science.org Gateway<br />
| https://services.d4science.org/group/d4science-services-gateway/authorization<br />
|-<br />
| DESCRAMBLE Gateway<br />
| https://descramble.d4science.org/group/descramble-gateway/authorization<br />
|-<br />
| GEMex Gateway<br />
| https://gemex.d4science.org/group/gemex-gateway/authorization<br />
|-<br />
| OpenAIRE-Connect Gateway<br />
| https://openaire-connect.d4science.org/group/openaire-connect-gateway/authorization<br />
|-<br />
| PARTHENOS Gateway<br />
| https://parthenos.d4science.org/group/parthenos-gateway/authorization<br />
|-<br />
| SoBigData Gateway<br />
| https://sobigdata.d4science.org/group/sobigdata-gateway/authorization<br />
|}<br />
<br />
'''Retrieve the authorization endpoint location dynamically (ADVANCED MODE)'''<br />
<br />
Registry Swagger REST-API avaiable at: https://dev.d4science.org/swagger/registry/<br />
<br />
Which requires an [https://dev.d4science.org/authorization/ authorization token]. The returned profile is something like<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<serviceEndpoints><br />
<Resource version="0.4.x"><br />
<ID>b1c5851f-1d47-4c60-8906-941ab859a334</ID><br />
<Type>RuntimeResource</Type><br />
<Scopes><br />
<Scope>/gcube</Scope><br />
</Scopes><br />
...<br />
<AccessPoint><br />
<Description>AuthorisationServer where to authenticate external SmartGears services oAuth like</Description><br />
<Interface><br />
<Endpoint EntryName="AuthorisationServer">https://next.d4science.org/group/next/authorization</Endpoint><br />
</Interface><br />
</AccessPoint><br />
</Profile><br />
</Resource><br />
</serviceEndpoints><br />
</source><br />
<br />
=== Invoking the authorization service===<br />
<br />
Once you got the AuthorisationServer endpoint and the call is executed, two things can happen:<br />
* the user is requested to perform the login, thus accepting the application operation and granting it;<br />
* if there is a valid existing permission grant for the application, the login operation is by-passed and the redirect_uri is followed.<br />
<br />
The request is an '''HTTP GET''' call, that needs the following parameters to be provided<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| client_id<br />
| the client identifier associated to the application<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the redirect uri provided at registration phase<br />
| yes<br />
|-<br />
| state<br />
| a unique string value hard to guess, needed to prevent CRSF attacks (e.g. AAedBEeF545sdfKef897)<br />
| yes<br />
|-<br />
| scope<br />
| this field is optional. It can be specified to request (with the next step) an access token valid for a given<br />
infrastructure scope. If missing, the scope will be choose by the user on credentials insertion.<br />
| no<br />
|}<br />
<br />
This is an example in development environment<br />
<pre><br />
https://next.d4science.org/group/next/authorization?client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&scope=%2Fgcube&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&state=7d12bf13-111c-4f46-ab06-9e9e08ad377b<br />
</pre><br />
<br />
==== End user Experience ====<br />
<br />
Once redirected, the user will be presented with D4Science's authentication dialog box. This identifies your application as well as outlines the particular member permissions that your application requests. The logo and application name can be configurable, in the example the application name is '''Application X'''<br />
<br />
[[File:Oauth-ux.jpg]]<br />
<br />
==== Application is Accepted ====<br />
<br />
If the user grants permissions to the application, a redirection to the redirect_uri is performed. Attached to the redirect_uri, two important url arguments are provided<br />
<br />
* code: which is the authorization code. It expires in 10 seconds;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
==== Application is Rejected ====<br />
<br />
If, for any reason, the request fails, the redirect_uri is still followed but the url has attached the following parameters<br />
* error: a code that states the type of error;<br />
* error_description: a textual description of the actual error;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
== Step 3 - Exchange Authorization Code for an Access Token ==<br />
This is the step in which the previously obtained code can be exchanged to obtain a real access token. <br />
This time, an '''HTTP POST''' method needs to be performed, with a '''Content-Type''' of type '''application/x-www-form-urlencoded'''.<br />
<br />
The client_id and secret must be passed as the basic auth header Base64 encoded, see the example in Java below:<br />
<br />
e.g.<br />
<source lang="java"><br />
...<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
..<br />
</source><br />
<br />
The request-body must contain the following parameters:<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| grant_type<br />
| The value of this field should always be authorization_code<br />
| yes<br />
|-<br />
| code<br />
| the authorization code you received from the previous step.<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the same 'redirect_uri' value that you<br />
passed in the previous step.<br />
| yes<br />
|}<br />
<br />
For instance, in development environment<br />
<br />
<source lang="xml"><br />
POST /gcube-oauth/v2/access-token HTTP/1.1<br />
Host: socialnetworking-d-d4s.d4science.org<br />
Content-Type: application/x-www-form-urlencoded<br />
<br />
grant_type=authorization_code&code=...&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&client_secret=...<br />
</source><br />
<br />
In case of success, the response will contain a JSON object, having these properties:<br />
<br />
* access_token: the obtained access token to use to perform operations on user's behalf;<br />
* expires_in: the time to live of the token expressed in seconds;<br />
* scope: in which scope the token is valid;<br />
* token_type: that is "Bearer".<br />
<br />
In case of failures, instead, the returned object will contain:<br />
<br />
* error : an error string according the oauth 2.0 RFC.<br />
* error_description : a text description for the error.<br />
<br />
=== Get the gcube-oauth endpoint location === <br />
The host to contact to perform this call can be again discovered via the icproxy service. This time the url will be something like<br />
<br />
<source lang="xml"><br />
http://node10-d-d4s.d4science.org/icproxy/gcube/service/GCoreEndpoint/Portal/oauth?gcube-token=35eaf849-43d3-439c-a3bc-2f7c355c0f67<br />
</source> <br />
<br />
A resource profile is returned which contains the property ''jersey-servlet'' to be retrieved.<br />
<br />
In '''development''' environment its current value is<br />
<br />
<source lang="xml"><br />
http://socialnetworking-d-d4s.d4science.org/gcube-oauth/v2/access-token<br />
</source><br />
<br />
In the '''production''' environment, its current value is<br />
<source lang="xml"><br />
https://socialnetworking1.d4science.org/gcube-oauth/v2/access-token<br />
</source><br />
<br />
'''HTTP POST Example in JAVA'''<br />
<br />
<source lang="java"><br />
String ACCESS_TOKEN_ATTR = "access_token";<br />
String SCOPE_ATTR = "scope";<br />
String fullPath2oAuthService = "https://socialnetworking1.d4science.org/gcube-oauth/v2/access-token";<br />
try {<br />
CloseableHttpClient httpClient = HttpClientBuilder.create().setRedirectStrategy(new LaxRedirectStrategy()).build();<br />
HttpPost httpPostRequest = new HttpPost(fullPath2oAuthService);<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
httpPostRequest.addHeader("Content-Type", "application/x-www-form-urlencoded");<br />
List<NameValuePair> params = new ArrayList<NameValuePair>();<br />
params.add(new BasicNameValuePair("grant_type", "authorization_code"));<br />
params.add(new BasicNameValuePair("code", tempCode));<br />
params.add(new BasicNameValuePair("redirect_uri", "https://www.gcube-system.org/oauth-authentication-example/example-app"));<br />
httpPostRequest.setEntity(new UrlEncodedFormEntity(params));<br />
<br />
CloseableHttpResponse httpresponse = httpClient.execute(httpPostRequest);<br />
int responseCode = httpresponse.getStatusLine().getStatusCode();<br />
if (responseCode < 200 || responseCode >= 300) {<br />
System.out.println("Error while getting authorization code, response status line from "<br />
+ fullPath2oAuthService + " was: " + responseCode); <br />
}<br />
JSONParser parser = new JSONParser();<br />
JSONObject json = (JSONObject) parser.parse(EntityUtils.toString(httpresponse.getEntity()));<br />
accessToken = json.get(ACCESS_TOKEN_ATTR).toString();<br />
scope = json.get(SCOPE_ATTR).toString();<br />
} catch(Exception e){<br />
System.out.println("Failed to perform request");<br />
e.printStackTrace();<br />
<br />
}<br />
</source><br />
<br />
== Step 4 - Get member data ==<br />
Having an access token, your application can perform some operations on behalf of the user. For instance, it can retrieve user's information profile by exploiting the [https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface social networking web service].<br />
<br />
Its current production endpoint is: https://socialnetworking1.d4science.org/social-networking-library-ws/rest/<br />
<br />
The subservice to call, once the social networking service endpoint has been retrieved, is this one<br />
<br />
<source lang="xml"><br />
2/users/get-oauth-profile<br />
</source><br />
<br />
It requires a ''gcube-token'' as query parameter, whose value is exactly the access token retrieved in the previous step. The result is something like this<br />
<br />
<source lang="son"><br />
{<br />
id: ".....",<br />
roles: [ ],<br />
picture: "....",<br />
name: "....",<br />
middle_name: "....",<br />
male: as boolean ,<br />
location_industry: "....",<br />
given_name: "....",<br />
email: "....",<br />
job_title: "....",<br />
family_name: "....",<br />
verified_email: as boolean <br />
}<br />
</source><br />
<br />
== See Also ==<br />
<br />
[https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface Social Networking Service]<br />
<br />
[https://dev.d4science.org/swagger/social-networking/ Social Networking Service Docs SWAGGER]</div>Massimiliano.assantehttps://wiki.gcube-system.org/index.php?title=OAuth2.0&diff=32156OAuth2.02019-04-05T10:24:40Z<p>Massimiliano.assante: /* Step 4 - Get member data */</p>
<hr />
<div>__TOC__<br />
<br />
= OAuth 2.0 gCube Support =<br />
By means of the OAuth 2.0 protocol (authorised) third party applications can operate on user's behalf over the D4Science infrastructure (while protecting the member's credentials).<br />
For more information about the OAuth authorization framework please visit the official [https://oauth.net/2/ OAuth site]. For technical details you could also check the [https://tools.ietf.org/html/rfc6749| OAuth 2.0 RFC]. In the following, the needed steps to authorize third party applications to operate on user's behalf and the D4Science infrastructure are explained. <br />
<br />
== Step 1 - Configure your application ==<br />
Authorized third party applications must be registered on the Infrastructure. Up to now the application registration is not automatic but you need to open a ticket at https://support.d4science.org/projects/d4science.<br />
<br />
The ticket needs to have '''3rd Party Application Support''' as tracker. The application name as well as the redirect uri need to be provided (https is strongly suggested). This url must be absolute, and without arguments. Once the application is registered, it will have associated:<br />
<br />
* a unique client identifier (client_id);<br />
* a client secret which must be kept secret, as the name itself says (client_secret).<br />
<br />
<source lang="xml"><br />
client_id: ab9d32c9-9425-42f1-99d7-50d68e651e9c<br />
client_secret: y9r93zDv4D<br />
</source><br />
<br />
Both are needed to proceed with the OAuth 2.0 workflow.<br />
<br />
== Step 2 - Require an authorization code ==<br />
Once the application is registered and configured, an authorization code can be requested. This code is not the final one needed to perform operations on user behalf, but is the one that will be exchanged for that in the next step. To require this code that has a very short time to live, the application must invoke the authorization service, which is gateway dependant.<br />
<br />
=== Retrieving the authorization endpoint location===<br />
<br />
The previous authorization endpoint location is portal's gateway dependant and '''can be found in Table 1''' or retrieved dinamically by using the registry service REST-API hosted at <br />
<br />
''https://registry.d4science.org/icproxy/gcube/service/'' <br />
<br />
'''Get authorization endpoint location in the following Table 1'''<br />
<br />
{| class="wikitable"<br />
|+Table 1: authorization endpoints location depending on the D4Science Gateway<br />
! style="font-weight: bold;" | D4Science Gateway name<br />
! style="font-weight: bold;" | Authorization endpoint<br />
|-<br />
| ARIADNEplus Gateway<br />
| https://ariadne.d4science.org/group/ariadneplus-gateway/authorization<br />
|-<br />
| iMarine Gateway<br />
| https://i-marine.d4science.org/group/imarine-gateway/authorization<br />
|-<br />
| AGINFRA+ Gateway<br />
| https://aginfra.d4science.org/group/aginfra-gateway/authorization<br />
|-<br />
| BlueBRIDGE Gateway<br />
| https://bluebridge.d4science.org/group/bluebridge-gateway/authorization<br />
|-<br />
| D4Science.org Gateway<br />
| https://services.d4science.org/group/d4science-services-gateway/authorization<br />
|-<br />
| DESCRAMBLE Gateway<br />
| https://descramble.d4science.org/group/descramble-gateway/authorization<br />
|-<br />
| GEMex Gateway<br />
| https://gemex.d4science.org/group/gemex-gateway/authorization<br />
|-<br />
| OpenAIRE-Connect Gateway<br />
| https://openaire-connect.d4science.org/group/openaire-connect-gateway/authorization<br />
|-<br />
| PARTHENOS Gateway<br />
| https://parthenos.d4science.org/group/parthenos-gateway/authorization<br />
|-<br />
| SoBigData Gateway<br />
| https://sobigdata.d4science.org/group/sobigdata-gateway/authorization<br />
|}<br />
<br />
'''Retrieve the authorization endpoint location dynamically (ADVANCED MODE)'''<br />
<br />
Registry Swagger REST-API avaiable at: https://dev.d4science.org/swagger/registry/<br />
<br />
Which requires an [https://dev.d4science.org/authorization/ authorization token]. The returned profile is something like<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<serviceEndpoints><br />
<Resource version="0.4.x"><br />
<ID>b1c5851f-1d47-4c60-8906-941ab859a334</ID><br />
<Type>RuntimeResource</Type><br />
<Scopes><br />
<Scope>/gcube</Scope><br />
</Scopes><br />
...<br />
<AccessPoint><br />
<Description>AuthorisationServer where to authenticate external SmartGears services oAuth like</Description><br />
<Interface><br />
<Endpoint EntryName="AuthorisationServer">https://next.d4science.org/group/next/authorization</Endpoint><br />
</Interface><br />
</AccessPoint><br />
</Profile><br />
</Resource><br />
</serviceEndpoints><br />
</source><br />
<br />
=== Invoking the authorization service===<br />
<br />
Once you got the AuthorisationServer endpoint and the call is executed, two things can happen:<br />
* the user is requested to perform the login, thus accepting the application operation and granting it;<br />
* if there is a valid existing permission grant for the application, the login operation is by-passed and the redirect_uri is followed.<br />
<br />
The request is an '''HTTP GET''' call, that needs the following parameters to be provided<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| client_id<br />
| the client identifier associated to the application<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the redirect uri provided at registration phase<br />
| yes<br />
|-<br />
| state<br />
| a unique string value hard to guess, needed to prevent CRSF attacks (e.g. AAedBEeF545sdfKef897)<br />
| yes<br />
|-<br />
| scope<br />
| this field is optional. It can be specified to request (with the next step) an access token valid for a given<br />
infrastructure scope. If missing, the scope will be choose by the user on credentials insertion.<br />
| no<br />
|}<br />
<br />
This is an example in development environment<br />
<pre><br />
https://next.d4science.org/group/next/authorization?client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&scope=%2Fgcube&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&state=7d12bf13-111c-4f46-ab06-9e9e08ad377b<br />
</pre><br />
<br />
==== End user Experience ====<br />
<br />
Once redirected, the user will be presented with D4Science's authentication dialog box. This identifies your application as well as outlines the particular member permissions that your application requests. The logo and application name can be configurable, in the example the application name is '''Application X'''<br />
<br />
[[File:Oauth-ux.jpg]]<br />
<br />
==== Application is Accepted ====<br />
<br />
If the user grants permissions to the application, a redirection to the redirect_uri is performed. Attached to the redirect_uri, two important url arguments are provided<br />
<br />
* code: which is the authorization code. It expires in 10 seconds;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
==== Application is Rejected ====<br />
<br />
If, for any reason, the request fails, the redirect_uri is still followed but the url has attached the following parameters<br />
* error: a code that states the type of error;<br />
* error_description: a textual description of the actual error;<br />
* state: which is the same value provided by the application, which is in charge of checking that it is the same.<br />
<br />
== Step 3 - Exchange Authorization Code for an Access Token ==<br />
This is the step in which the previously obtained code can be exchanged to obtain a real access token. <br />
This time, an '''HTTP POST''' method needs to be performed, with a '''Content-Type''' of type '''application/x-www-form-urlencoded'''.<br />
<br />
The client_id and secret must be passed as the basic auth header Base64 encoded, see the example in Java below:<br />
<br />
e.g.<br />
<source lang="java"><br />
...<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
..<br />
</source><br />
<br />
The request-body must contain the following parameters:<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Parameter<br />
! style="font-weight: bold;" | Description<br />
! style="font-weight: bold;" | Required<br />
|-<br />
| grant_type<br />
| The value of this field should always be authorization_code<br />
| yes<br />
|-<br />
| code<br />
| the authorization code you received from the previous step.<br />
| yes<br />
|-<br />
| redirect_uri<br />
| the same 'redirect_uri' value that you<br />
passed in the previous step.<br />
| yes<br />
|}<br />
<br />
For instance, in development environment<br />
<br />
<source lang="xml"><br />
POST /gcube-oauth/v2/access-token HTTP/1.1<br />
Host: socialnetworking-d-d4s.d4science.org<br />
Content-Type: application/x-www-form-urlencoded<br />
<br />
grant_type=authorization_code&code=...&redirect_uri=https%3A%2F%2Fwww.gcube-system.org%2Fdevelopers&client_id=ab9d32c9-9425-42f1-99d7-50d68e651e9c&client_secret=...<br />
</source><br />
<br />
In case of success, the response will contain a JSON object, having these properties:<br />
<br />
* access_token: the obtained access token to use to perform operations on user's behalf;<br />
* expires_in: the time to live of the token expressed in seconds;<br />
* scope: in which scope the token is valid;<br />
* token_type: that is "Bearer".<br />
<br />
In case of failures, instead, the returned object will contain:<br />
<br />
* error : an error string according the oauth 2.0 RFC.<br />
* error_description : a text description for the error.<br />
<br />
=== Get the gcube-oauth endpoint location === <br />
The host to contact to perform this call can be again discovered via the icproxy service. This time the url will be something like<br />
<br />
<source lang="xml"><br />
http://node10-d-d4s.d4science.org/icproxy/gcube/service/GCoreEndpoint/Portal/oauth?gcube-token=35eaf849-43d3-439c-a3bc-2f7c355c0f67<br />
</source> <br />
<br />
A resource profile is returned which contains the property ''jersey-servlet'' to be retrieved.<br />
<br />
In '''development''' environment its current value is<br />
<br />
<source lang="xml"><br />
http://socialnetworking-d-d4s.d4science.org/gcube-oauth/v2/access-token<br />
</source><br />
<br />
In the '''production''' environment, its current value is<br />
<source lang="xml"><br />
https://socialnetworking1.d4science.org/gcube-oauth/v2/access-token<br />
</source><br />
<br />
'''HTTP POST Example in JAVA'''<br />
<br />
<source lang="java"><br />
String ACCESS_TOKEN_ATTR = "access_token";<br />
String SCOPE_ATTR = "scope";<br />
String fullPath2oAuthService = "https://socialnetworking1.d4science.org/gcube-oauth/v2/access-token";<br />
try {<br />
CloseableHttpClient httpClient = HttpClientBuilder.create().setRedirectStrategy(new LaxRedirectStrategy()).build();<br />
HttpPost httpPostRequest = new HttpPost(fullPath2oAuthService);<br />
String idAndSecret = new StringBuilder(client_id).append(":").append(client_secret).toString();<br />
String encodingIdAndSecret = Base64.getEncoder().encodeToString(idAndSecret.getBytes(StandardCharsets.UTF_8));<br />
httpPostRequest.setHeader("Authorization", "Basic " + encodingIdAndSecret);<br />
httpPostRequest.addHeader("Content-Type", "application/x-www-form-urlencoded");<br />
List<NameValuePair> params = new ArrayList<NameValuePair>();<br />
params.add(new BasicNameValuePair("grant_type", "authorization_code"));<br />
params.add(new BasicNameValuePair("code", tempCode));<br />
params.add(new BasicNameValuePair("redirect_uri", "https://www.gcube-system.org/oauth-authentication-example/example-app"));<br />
httpPostRequest.setEntity(new UrlEncodedFormEntity(params));<br />
<br />
CloseableHttpResponse httpresponse = httpClient.execute(httpPostRequest);<br />
int responseCode = httpresponse.getStatusLine().getStatusCode();<br />
if (responseCode < 200 || responseCode >= 300) {<br />
System.out.println("Error while getting authorization code, response status line from "<br />
+ fullPath2oAuthService + " was: " + responseCode); <br />
}<br />
JSONParser parser = new JSONParser();<br />
JSONObject json = (JSONObject) parser.parse(EntityUtils.toString(httpresponse.getEntity()));<br />
accessToken = json.get(ACCESS_TOKEN_ATTR).toString();<br />
scope = json.get(SCOPE_ATTR).toString();<br />
} catch(Exception e){<br />
System.out.println("Failed to perform request");<br />
e.printStackTrace();<br />
<br />
}<br />
</source><br />
<br />
== Step 4 - Get member data ==<br />
Having an access token, your application can perform some operations on behalf of the user. For instance, it can retrieve user's information profile by exploiting the [https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface social networking web service].<br />
<br />
Its current production endpoint is: https://socialnetworking1.d4science.org/social-networking-library-ws/rest/<br />
<br />
The subservice to call, once the social networking service endpoint has been retrieved, is this one<br />
<br />
<source lang="xml"><br />
2/users/get-oauth-profile<br />
</source><br />
<br />
It requires a ''gcube-token'' as query parameter, whose value is exactly the access token retrieved in the previous step. The result is something like this<br />
<br />
{<br />
id: ".....",<br />
roles: [ ],<br />
picture: "....",<br />
name: "....",<br />
middle_name: "....",<br />
male: as boolean ,<br />
location_industry: "....",<br />
given_name: "....",<br />
email: "....",<br />
job_title: "....",<br />
family_name: "....",<br />
verified_email: as boolean <br />
}<br />
</source><br />
<br />
== See Also ==<br />
<br />
[https://wiki.gcube-system.org/gcube/Social_Networking_Library_Web_Service_Interface Social Networking Service]<br />
<br />
[https://dev.d4science.org/swagger/social-networking/ Social Networking Service Docs SWAGGER]</div>Massimiliano.assante