CMEMS Client

The CMEMS Client is a Java library to support search and browse datasets published on the CMEMS online catalogue . The library is part of the CMEMS Dataset Importer.

Overview

This library is based on a non-documented service behind the CMEMS online catalogue. Such service is used to retrieve the full list of all CMEMS product. Although it might expose other operations to search metadata, they're nor known at the time of implementing this client. The service exposes a json array with all products. The json is available here: http://cmems-resources.cls.fr/index.php?option=com_csw&task=advancedsearch

Getting the list from the CMEMS portal, although meant for interactive usage, requires some time (~10"). Considering that the such list changes slowly (daily?) a caching mechanism has been implemented in the client to speed-up execution. The default cache TTL is 8 hours; however the cache is local to the client object, thus creating a new client also means invalidating the cache.

The CMEMS catalogue is built around the concept of product and datasets where the latter contain only some variables of the product and/or with different aggregations (e.g. daily means, monthly means).

No authentication is required to interact with the CMEMS online catalogue.

Usage

Configure your project

Maven coordinates:

  <dependency>
    <groupId>org.gcube.dataanalysis</groupId>
    <artifactId>cmems-client</artifactId>
    <version>[1.0.0, 2.0.0)</version>
  </dependency>

The current version of the library is 1.0.0

Create a client

As a first step, create a CMEMSClient with no arguments. This points to the current endpoint of the CMEMS online catalogue.

  CMEMSClient client = new CMEMSClient();

However, you can provide a different endpoint in the constructor.

  CMEMSClient client = new CMEMSClient("http://some/other/endpoint");

Download all products

You can download metadata for all products and dataset with:

  Collection<CmemsProduct> products = client.downloadProducts();

Each CmemsProduct holds metadata about a single product, including pointers to the services exposing datasets. A full listing of all metadata is given below:

    private String abztract;
    private String id;
    private String display_priority;
    private String external_shortname;    private String version;
    private String short_description;
    private String creation_date;
    private String update_date;
    private String quicklooks;
    private String feature_type;
    private String forecast_length;
    private String all_keywords;
    private String geographical_area;
    private String mission_type;
    private String model_assimilation;
    private String ocean_variables;
    private String ocean_keys;
    private String processing_level;
    private String west;
    private String east;
    private String south;
    private String north;
    private String spatial_resolution_row;
    private String spatial_resolution_row_unit;
    private String spatial_resolution_column;
    private String spatial_resolution_column_unit;
    private String coord_ref_sys;
    private String vertical_min;
    private String vertical_max;
    private String vertical_crs;
    private String vertical_levels;
    private String temporal_resolution;
    private String vertical_coverage;
    private String temporal_begin;
    private String temporal_end;
    private String temporal_extent;
    private String temporal_scale;
    private String update_frequency;
    private String update_schedule;
    private String production_unit;
    private String credit;
    private String documentation;
    private String mail;
    private String ftp;
    private String dgf;
    private String tds;
    private String wms;
    private String service_desk;
    private String upstream_production;
    private String upstream_validation;
    private String downstream_validation;
    private String downstream_production;
    private String MYO_OK;
    private String _version;

Natively, products are described with a set of single-value String mappings (e.g. temporal_resolution=monthly-mean, daily-mean, hourly-mean). For multi-value properties, the CmemsProduct class exposes methods to get values as list, as in the following example:

  String[] oceanKeys = product.getOceanKeys();
  String[] tdsDatasets = product.getTdsDatasets();

Support for query form

The library provides support to build GUI form to search for datasets. It has methods returing acceptable values and ranges for some of the product metadata. The currently implemented methods are those needed to mimic the CMEMS online catalog form.

  // get all regional domains.
  Collection<CodedString> domains = client.getRegionalDomains();
 
  // get the set of keys in datasets.
  Collection<CodedString> getOceanKeys();
 
  // get the lowest timestamp in datasets.
  Calendar getMinTime();
 
  // get the highest timestamp in datasets.
  Calendar getMaxTime();

Searching for products

The class SearchOptions is meant to model keys for a basic search. No remote search is actually performed; rather the client finds a match with datasets in the full list of products retrieved from the server.

  SearchOptions options = new SearchOptions();
 
  // a key that must appear at least once in name, description, abstract, geographic area, keywords, variables and dataset endpoints.
  options.addMatchKey("biology");
  options.addMatchKey("chemistry");
 
  // a key that must NOT be present in the above-mentionned fields.
  options.addUnMatchedKey("thickness");
 
  // a key to appear in the geographic_area field
  options.addRegionalDomain("mediterranean");
 
  // a key to appear among variables
  options.addVariable("surface_height");
 
  // the resulting products/dataset must have data after this timestamp
  Calendar from = ...;
  options.setFrom(from);
 
  // the resulting products/dataset must have data before this timestamp
  Calendar to = ...;
  options.setTo(to);
 
  // when set to true, the search results will only show products containing the whole selected time range. Default is false.
  options.setWholeTimeRange(true);
 
  // ignore case in text matches. Default is true.
  options.setIgnoreCase(false);

The matching set of products can be obtained with:

  Collection<CmemsProduct> products = client.searchProducts(options);