Difference between revisions of "Data Retrieval APIs"
From Gcube Wiki
| Line 73: | Line 73: | ||
:* ''transparent detection and utilization of Data Sources'' | :* ''transparent detection and utilization of Data Sources'' | ||
:* ''results streaming exposition through gRS2 endpoint'' | :* ''results streaming exposition through gRS2 endpoint'' | ||
| + | |||
| + | ==OpenSearch Library== | ||
| + | * '''Description''': The OpenSearch Core Library conforms to the latest OpenSearch specification and provides general OpenSearch-related functionality to any component which needs to query OpenS earch providers. | ||
| + | * '''Type''': CL | ||
| + | * '''Protocol''': Java | ||
| + | * '''Framework''': none | ||
| + | * [[OpenSearch_Library|'''Documentation''']] | ||
| + | * '''Key Features''' | ||
| + | :* ''access to the information hosted by external Providers'' | ||
| + | :* ''extensions provide functionality not only for standard OpenSearch parameters are supported'' | ||
| + | :* ''the role of the OpenSearch Operator is to provide support for querying and retrieval of search results via OpenSearch from providers which expose an OpenSearch description document. '' | ||
| + | :* ''results are returned wrapped in a ResultSet'' | ||
| + | :* ''support of data transformation by the operator'' | ||
| + | :* ''both direct and brokered result processing is supported'' | ||
| + | :* ''support of a set of fixed parameters, which override the user-provided parameters only at the level of the top provider'' | ||
| + | :* ''MIME type of the results that are to be obtained is abstracted away and treated as low-level information which can be exploited by the way OpenSearch Resources are structured'' | ||
| + | :* ''the operator accepts a set of query terms and a set of query parameters'' | ||
| + | :* ''OpenSearch Operator expects to receive all query parameters. Any free-text parameter value should be URL-encoded'' | ||
| + | |||
| + | ==OpenSearchService== | ||
| + | * '''Description''': The OpenSearch Service is a stateful web service responsible for the invocation of the OpenSearch Operator in the context of the provider to be queried. | ||
| + | * '''Type''': WS | ||
| + | * '''Protocol''': SOAP | ||
| + | * '''Framework''': [https://gcore.wiki.gcube-system.org/gCube gCore] | ||
| + | * [[The_OpenSearch_Service|'''Documentation''']] | ||
| + | * '''Key Features''' | ||
| + | :* ''uses a WS-Resource for each OpenSearch provider in order to bind the Service to the Metadata Collection corresponding to the provider.'' | ||
| + | :* ''maintains a cache per provider WS-Resource for performance and reliability reasons'' | ||
| + | :* ''he cache can be refreshed either on demand or periodically, based on a configurable time interval'' | ||
Revision as of 10:19, 27 August 2012
Data Retrieval APIs provide access to to Information Retrieval facilites over large heterogeneous environments, offering a declarative approach for querying the hosted information and allowing for easy integration of external data sources of information.
This page outlines the design rationale for those APIs.
Contents
Overview
| Name | Description | Type | Protocol | Framework |
| gCQLParser | The gCube parser for CQL language (extended with gCube elements) | Java | Local | none |
| Index Service | Provides indexing functionality for the resources | WS | SOAP | gCore |
| OperatorLibrary | The search operators cover the basic functionality that could be encountered in a typical search operation | Java | Local | none |
| Search System Service | Web service that offers information retrieval functionality in a gCube infrastructure. | WS | SOAP | gCore |
| OpenSearch Library | The OpenSearch Core Library conforms to the latest OpenSearch specification and provides general OpenSearch-related functionality to any component which needs to query OpenS earch providers. | CL | Local | none |
| OpenSearchService | TThe OpenSearch Service is a stateful web service responsible for the invocation of the OpenSearch Operator in the context of the provider to be queried. | WS | SOAP | gCore |
gCQLParser API
- Description: Provides functionality that helps presentation layer to use gCube services.
- Type: Local
- Protocol: Java
- Framework: none
- Documentation
- Key Features
- submission of gCQL queries as plain string
- support for gCube extensions in CQL language for presentable information
Index Service API
- Description: Provides indexing functionality for the resources.
- Type: WS
- Protocol: SOAP
- Framework: gCore
- Documentation
- Key Features
- Full Text Index receives CQL queries and transforms them into Lucene queries
- 2 linguistics components are available; the language identifier module, and the lemmatizer module
- database partitioning is supported
- geo-spatial Index Lookup supports one custom CQL relation
- the "geosearch" relation has a number of modifiers that specify parameters such as collection, language, query inclusion type, refiner and ranker
- support of customs refiners and rankers
- the content to be fed into a Geo Index, must be served as a ResultSet containing XML documents conforming to the GeoROWSET schemafield presense and handling by the Geo Index is specified through a GeoIndexType; an XML document conforming to the GeoIndexType schemathe content to be fed into a Geo Index, must be served as a ResultSet containing XML documents conforming to the GeoROWSET schema
OperatorLibrary
- Description: The search operators cover the basic functionality that could be encountered in a typical search operation.
- Type: CL
- Protocol: Java
- Framework: none
- Documentation
- Key Features
- results are transfered via ResultSet
- the comparing parts are either literals (date, string, integer, double literals are supported) or aggregate functions on the results of a search service execution
Search System Service
- Description: Web service that offers information retrieval functionality in a gCube infrastructure.
- Type: WS
- Protocol: SOAP
- Framework: gCore
- Documentation
- Key Features
- queries expressed in Contextual Query Language
- orchestration of search plan production and execution
- transparent detection and utilization of Data Sources
- results streaming exposition through gRS2 endpoint
OpenSearch Library
- Description: The OpenSearch Core Library conforms to the latest OpenSearch specification and provides general OpenSearch-related functionality to any component which needs to query OpenS earch providers.
- Type: CL
- Protocol: Java
- Framework: none
- Documentation
- Key Features
- access to the information hosted by external Providers
- extensions provide functionality not only for standard OpenSearch parameters are supported
- the role of the OpenSearch Operator is to provide support for querying and retrieval of search results via OpenSearch from providers which expose an OpenSearch description document.
- results are returned wrapped in a ResultSet
- support of data transformation by the operator
- both direct and brokered result processing is supported
- support of a set of fixed parameters, which override the user-provided parameters only at the level of the top provider
- MIME type of the results that are to be obtained is abstracted away and treated as low-level information which can be exploited by the way OpenSearch Resources are structured
- the operator accepts a set of query terms and a set of query parameters
- OpenSearch Operator expects to receive all query parameters. Any free-text parameter value should be URL-encoded
OpenSearchService
- Description: The OpenSearch Service is a stateful web service responsible for the invocation of the OpenSearch Operator in the context of the provider to be queried.
- Type: WS
- Protocol: SOAP
- Framework: gCore
- Documentation
- Key Features
- uses a WS-Resource for each OpenSearch provider in order to bind the Service to the Metadata Collection corresponding to the provider.
- maintains a cache per provider WS-Resource for performance and reliability reasons
- he cache can be refreshed either on demand or periodically, based on a configurable time interval
