How to use the Data Miner Pool Manager

Data Miner Pool Manager

DMPM is a REST service able to rationalize and automatize the current process for publishing SAI algorithms on DataMiner nodes and keep DataMiner cluster updated.

Overview

The service may accept an algorithm descriptor, including its dependencies (either os, r and custom packages), queries the IS for dataminers in the current scope, generates (via templating) ansible playbook, inventory and roles for relevant stuff (algorithm installer, algorithms, dependencies), executes ansible playbook on a DataMiner. In such sense, the service accepts as input, among the others, the url of an algorithm package (including jar, and metadata), install the script and return asynchronously execution outcome to the caller.

Usage

DMPM is a SmartGear compliant service. An instance has already been deployed and configured at Development level.

http://node2-d-d4s.d4science.org:8080

In order to allow Ansible to access the DataMiner host it is necessary that the SSH key of the host where the Service is deployed is correctly configured at DataMiner host level.

API

Currently the service exposes the following REST methods:

  @GET
  @Path("/hosts/add")
  @Produces("text/plain")
  public String addAlgorithmToHost(
      @QueryParam("algorithm") String algorithm, 
      @QueryParam("hostname") String hostname,
      @QueryParam("name") String name,
      @QueryParam("description") String description,
      @QueryParam("category") String category,
      @QueryParam("algorithmType") String algorithmType,
      @QueryParam("skipJava") String skipJava) throws IOException, InterruptedException {
    Algorithm algo= this.getAlgorithm(algorithm, null, hostname, name, description, category, algorithmType, skipJava);
    //service.addAlgToIs(algo);
    return service.addAlgorithmToHost(algo, hostname);
  }

Mandatory params Options params

Return the id

An example of the usage is the following: <sourc lang ="text"> http://node2-d-d4s.d4science.org:8080/dataminer-pool-manager-1.0.0-SNAPSHOT/rest/hosts/add?gcube-token=TOKEN_ID&algorithm=URL_TP_ALGORITHM&hostname=TARGET_DATAMINER </source>

getLogById: returning asynchronously the detail of the execution

@GET @Path("/log") @Produces("text/plain") public String getLogById(@QueryParam("logUrl") String logUrl) throws IOException { // TODO Auto-generated method stub LOGGER.debug("Returning Log =" + logUrl); return service.getScriptFromURL(service.getURLfromWorkerLog(logUrl)); }

http://node2-d-d4s.d4science.org:8080/dataminer-pool-manager-1.0.0-SNAPSHOT/rest/log?gcube-token=**********************************&logUrl=log_id

parameters can be the same of the metadata or sovrascritti

-- "addAlgorithmToVRE" returning immediatly the ID of the log

skipjava ...

-- "getLogById" returning asynchronously the detail of the execution