Difference between revisions of "Document Store Framework"

Revision as of 14:56, 9 May 2016

Document Store Lib Provides facilities to store document in a Document Store NoSQL DB. It provides Aggregation, Bufferization, Storing and Fallback & Retry capabilities.

Library

The library has been tough to provide a common way to store Records in Document Store NoSQL DB. In particular is responsible for:

• Buffering
• Aggregating
• Storing Record
• Providing Fallback & Retry Capability

The library is designed to separate the above mentioned capabilities from the logic needed to really store the Records on DB.

Buffering & Aggregating Record

The library provide buffering capabilities. For each new requests to store a record the library tries to aggregate the received record with the ones has already buffered. The aggregation made in this phase is lossless. In other words, the Record aggregated are only the ones having the same values for all properties except the ones where the aggregation has been made (e.g. operationCount).

Storing Record

The library discovers dynamically the Document Store Lib Persistence (i.e. using Java Service Loader). This library is responsible to connect to DS-NoSQL DB and to really store the Record. The configuration parameters are also discovered dynamically.

If no library is discovered the Records are persisted using a Fallback Persistence (on filesystem).

Fallback & Retry

For any reason why a Record cannot be persistend on a Document Store DB (e.g. a network failure, no persistence found or configured correctly), the Record is persisted on filesystem. The Java class responsible to do it is FallbackPersistence.

A recurrent task periodically (i.e. 10 min) check if there are Records stored from Fallback Persistence (A Document Store Lib Persistence must be discovered in advance to activate the recurrent task because otherwise the task has no reason to exist).

Base Model

To set the various properties of a records two different methods are available by setting

• Each Single Property through a generic Key-Value method
  • key : String
  • value : ? extends Serializable
• Multiple Properties at once passing a Map<String,? extends Serializable>.

The build blocks of this library are the interfaces:

• Record
• AggregatedRecord

The first interface (Record) must be implemented to create a class compliant with the library model, so that the library can provide it own facilities. The second interface must be implemented if the model needs the aggregation facilities.

The library provides also a basic implementation AbstractRecord that the developer can use as building block fro its own model. By extending this abstract class the library also provide properties:

• Validations
• Harmonizations

Development

The library is identified by the following maven coordinates:

<groupId>org.gcube.data-publishing</groupId>

<artifactId>document-store-lib</artifactId>

Please set the scope of the library to PROVIDED, if your component runs on a container gHN or vHN.

Available Document Store Lib Persistence

The current implementations of persistence for this library are:

• CouchDB

<groupId>org.gcube.data-publishing</groupId>

<artifactId>document-store-lib-couchdb</artifactId>

• CouchBase

<groupId>org.gcube.data-publishing</groupId>

<artifactId>document-store-lib-couchbase</artifactId>

• MongoDB

<groupId>org.gcube.data-publishing</groupId>

<artifactId>document-store-lib-mongodb</artifactId>

• InfluxDB (under investigation)

The developer MUST not depend from this libraries.

Deployment

Document Store Lib and the Document Store Lib CouchDB are already available on gCore and smartgears bundles (gHN and wHN).

Document Store Lib in the Accounting Architecture

The following image evidences the use of Document Store Lib in Accounting Architecture: