Difference between revisions of "Data Retrieval APIs"

Revision as of 10:20, 27 August 2012

Data Retrieval APIs provide access to to Information Retrieval facilites over large heterogeneous environments, offering a declarative approach for querying the hosted information and allowing for easy integration of external data sources of information.

This page outlines the design rationale for those APIs.

Overview

gCQLParser API

• Description: Provides functionality that helps presentation layer to use gCube services.
• Type: Local
• Protocol: Java
• Framework: none
• Documentation
• Key Features

• submission of gCQL queries as plain string
• support for gCube extensions in CQL language for presentable information

Index Service API

• Description: Provides indexing functionality for the resources.
• Type: WS
• Protocol: SOAP
• Framework: gCore
• Documentation
• Key Features

• Full Text Index receives CQL queries and transforms them into Lucene queries
• 2 linguistics components are available; the language identifier module, and the lemmatizer module
• database partitioning is supported
• geo-spatial Index Lookup supports one custom CQL relation
• the "geosearch" relation has a number of modifiers that specify parameters such as collection, language, query inclusion type, refiner and ranker
• support of customs refiners and rankers
• the content to be fed into a Geo Index, must be served as a ResultSet containing XML documents conforming to the GeoROWSET schemafield presense and handling by the Geo Index is specified through a GeoIndexType; an XML document conforming to the GeoIndexType schemathe content to be fed into a Geo Index, must be served as a ResultSet containing XML documents conforming to the GeoROWSET schema

OperatorLibrary

• Description: The search operators cover the basic functionality that could be encountered in a typical search operation.
• Type: CL
• Protocol: Java
• Framework: none
• Documentation
• Key Features

• results are transfered via ResultSet
• the comparing parts are either literals (date, string, integer, double literals are supported) or aggregate functions on the results of a search service execution

Search System Service

• Description: Web service that offers information retrieval functionality in a gCube infrastructure.
• Type: WS
• Protocol: SOAP
• Framework: gCore
• Documentation
• Key Features

• queries expressed in Contextual Query Language
• orchestration of search plan production and execution
• transparent detection and utilization of Data Sources
• results streaming exposition through gRS2 endpoint

OpenSearch Library

• Description: The OpenSearch Core Library conforms to the latest OpenSearch specification and provides general OpenSearch-related functionality to any component which needs to query OpenS earch providers.
• Type: CL
• Protocol: Java
• Framework: none
• Documentation
• Key Features

• access to the information hosted by external Providers
• extensions provide functionality not only for standard OpenSearch parameters are supported
• the role of the OpenSearch Operator is to provide support for querying and retrieval of search results via OpenSearch from providers which expose an OpenSearch description document.
• results are returned wrapped in a ResultSet
• support of data transformation by the operator
• both direct and brokered result processing is supported
• support of a set of fixed parameters, which override the user-provided parameters only at the level of the top provider
• MIME type of the results that are to be obtained is abstracted away and treated as low-level information which can be exploited by the way OpenSearch Resources are structured
• the operator accepts a set of query terms and a set of query parameters
• OpenSearch Operator expects to receive all query parameters. Any free-text parameter value should be URL-encoded

OpenSearchService

• Description: The OpenSearch Service is a stateful web service responsible for the invocation of the OpenSearch Operator in the context of the provider to be queried.
• Type: WS
• Protocol: SOAP
• Framework: gCore
• Documentation
• Key Features

• uses a WS-Resource for each OpenSearch provider in order to bind the Service to the Metadata Collection corresponding to the provider.
• maintains a cache per provider WS-Resource for performance and reliability reasons
• he cache can be refreshed either on demand or periodically, based on a configurable time interval