Difference between revisions of "DataMiner Installation"

Revision as of 15:23, 27 July 2015

Introduction

DataMiner is an e-Infrastructure service providing state-of-the art DataMining algorithms and ecological modelling approaches under the Web Processing Service (WPS) standard.

In this guide, we show how administrators and site-managers can install DataMiner on top of SmartGears service installations.

Prerequisites

See the SmartGears GHN requirements

In order to manage a request load of 20,000 computations per month with a maximum allowed concurrency of 10 requests we recommend the following machine hardware:

• Ubuntu 12.04.5 LTS
• 6 GB of RAM
• 10 virtual CPUs, e.g. Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz
• 10 GB of HD space

Installation

Passages required to build a fully working development-environment DataMiner installation from scratch:

1 - Install a SmartGears-enabled tomcat service, possibly on the 80 port or with a redirect to the 80 port. Use devsec as starting scope.

2 - Download the official 52 WAR application from the following link and put it under webapps:

 http://build.dev.52north.org/jenkins/view/Processing/job/52n-wps_GIT-DEV_nightly_builds/lastSuccessfulBuild/org.n52.wps$52n-wps-webapp/artifact/org.n52.wps/52n-wps-webapp/3.3.2-20150523.010021-9/52n-wps-webapp-3.3.2-20150523.010021-9.war

3 - Substitute the 52n-wps-server-3.3.2-X.jar with the corresponding jar on our Maven gcube-externals repository:

 (Repository: "gCube Externals")
 <dependency>
   <groupId>rapidminer-custom</groupId>
   <artifactId>52n-wps-server</artifactId>
   <version>3.3.2</version>
 </dependency>

4 - add the following maven library along with its dependencies to the wps/WEB-INF/lib/ folder of the wps application:

 (Repository: "gCube Snapshots")
 <dependency>
   <groupId>org.gcube.dataanalysis</groupId>
   <artifactId>dataminer</artifactId>
   <version>1.0.0-SNAPSHOT</version>
 </dependency>

5 - create a folder named "persistence" under wsp/

6 - create a folder named "ecocfg" under wsp/

7 - copy all the files available at this SVN link into the ecocfg folder:

 https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/DataMinerConfiguration/cfg

8 - copy the PARALLEL_PROCESSING folder at this SVN link into the ecocfg folder (thus creating the PARALLEL_PROCESSING folder under ecocfg):

 https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/DataMinerConfiguration/PARALLEL_PROCESSING

9 - copy the following xml file into the wps/config folder:

 https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/DataMinerConfiguration/wpscfg/wps_config.xml

10 - substitute the hostname and the port inside the following tag of the previous xml file, with the correct indication of the hostname of the machine and of the port (80) of the tomcat:

 <Server protocol="http" hostname="localhost" hostport="8080" includeDataInputsInResponse="false" computationTimeoutMilliSeconds="3600000" cacheCapabilites="false" webappPath="wps" repoReloadInterval="0.0" minPoolSize="10" maxPoolSize="20" keepAliveSeconds="1000" maxQueuedTasks="100">

11 - a reference example of configured and working wps application can be found at this link:

 http://goo.gl/rtbHpW

Related Links

• Description of the Parallel Processing approach used by DataMiner

• Description of the Cloud Computing mechanisms used by DataMiner

• Implementing new algorithms

• list of Algorithms potentially available on the service

• Discussion and instructions on the integration of DataMiner with QuantumGis

• Discussion on signal processing and forecasting of catch statistics

Related Experiments

• Predicting the habitat of a marine species

• Identifying the habitat of a rare species

• Calculating the similarity between two maps

• Signal processing and forecasting of fisheries data

• Application of clustering analysis to identify species commonness