Difference between revisions of "Software Gateway"

From Gcube Wiki
Jump to: navigation, search
(gCube Software Registration)
(Installation requirements)
 
(38 intermediate revisions by one other user not shown)
Line 43: Line 43:
  
 
=== Removal ===  
 
=== Removal ===  
[[Image:SoftwareRemoval2.png|frame|center|Software Removal]]
+
[[Image:SGRemoval.png|frame|center|Software Removal]]
  
 
=== Support to Deployment ===  
 
=== Support to Deployment ===  
Line 95: Line 95:
 
<source lang="xml">
 
<source lang="xml">
 
<Packages>
 
<Packages>
<Package>
+
<Package>
<groupID>org.gcube.common</groupID>
+
    <groupID>org.gcube.common</groupID>
<artifactID>rapidminer-custom</artifactID>
+
    <artifactID>rapidminer-custom</artifactID>
<version>1.0.0</version>
+
    <version>1.0.0</version>
<ID>d3217750-9b6a-11e1-be35-f7d8862056ea</ID>
+
    <ID>d3217750-9b6a-11e1-be35-f7d8862056ea</ID>
<URL>http://maven.research-infrastructures.eu/nexus/service/local/artifact/maven/redirect?r=gcube-staging&g=org.gcube.common&a=rapidminer-custom&v=1.0.0&e=jar</URL>
+
    <URL>http://maven.research-infrastructures.eu/nexus/service/local/artifact/maven/redirect?r=gcube-staging&g=org.gcube.common&a=rapidminer-custom&v=1.0.0&e=jar</URL>
<javadoc>http://maven.research-infrastructures.eu/nexus/service/local/artifact/maven/redirect?r=gcube-staging&g=org.gcube.common&a=rapidminer-custom&v=1.0.0&e=javadoc</javadoc>
+
    <javadoc>http://maven.research-infrastructures.eu/nexus/service/local/artifact/maven/redirect?r=gcube-staging&g=org.gcube.common&a=rapidminer-custom&v=1.0.0&c=javadoc</javadoc>
<Status>SUCCESS</Status>
+
    <Status>SUCCESS</Status>
<Operation>NEW</Operation>
+
    <Operation>NEW</Operation>
<Timestamp>1336741892744</Timestamp>
+
    <Timestamp>1336741892744</Timestamp>
</Package>
+
</Package>
 
</Packages>
 
</Packages>
 
</source>
 
</source>
Line 195: Line 195:
  
 
== SGClient ==
 
== SGClient ==
 +
 +
A client library that implements a high-level facade to the remote APIs of the Software Gateway service
  
 
=== Interface ===
 
=== Interface ===
 +
 +
Each method invokes the corresponding method in the SoftwareGateway service. These methods must be called by console.
 +
Each method accepts a set of standard parameters that are: <code>scope </code>and <code>epr</code> . Furthermore, the methods accepts the parameter specified below:
 +
 +
* <code>DepSolverClient</code>, accepts a set of gcube package coordinates and return an XML that contains the dependencies resolved and missed
 +
* <code>GetLocationClient</code>, accepts a set of gcube package coordinates and returns the url of the package associated
 +
* <code>GetPackagesClient</code>, accepts  a set of gcube service coordinates and returns a list o package coordinates
 +
* <code>GetPluginsClient</code>, accepts a set of gcube service coordinates and returns a list of plugin's coordinated that match with the service
 +
* <code>GetSALocationClient</code>, returning the ServiceArchive location for a inptu set of gcube package coordinates
 +
* <code>RegisterProfileClient</code>, accepts a profile and register it on the information system service
 +
* <code>RemoveProfileClient</code>, unregister a profile from information system service, identifies by the input coordinates
  
 
=== Sample Usage ===
 
=== Sample Usage ===
 +
 +
The following example show a call from console to the  RegisterProfileClient method, for register a profile present in the file profilexm.xml
 +
 +
<source lang="bash">
 +
java -classpath softwaregateway-stubs-1.0.0-SNAPSHOT.jar:SGClient.jar RegisterProfileClient
 +
http://node4.d.d4science.research-infrastructures.eu:8080/wsrf/services/gcube/vremanagement/softwaregateway/Registration 
 +
/gcube/devsec /home/xxx/Documenti/SoftwareGatewayDocs/yyy/profilevm.xml
 +
</source>
 +
 +
where:
 +
<source lang="text">
 +
epr              = http://node4.d.d4science.research-infrastructures.eu:8080/wsrf/services/gcube/vremanagement/softwaregateway/Registration
 +
scope            = /gcube/devsec
 +
path to the file = /home/xxx/Documenti/SoftwareGatewayDocs/yyy/profilevm.xml
 +
</source>
 +
 +
 +
 +
 +
The following example show a call from console to the  UnregisterProfileClient method.
 +
 +
<source lang="bash">
 +
java -classpath softwaregateway-stubs-1.0.0-SNAPSHOT.jar:SGClient.jar  org.gcube.vremanagement.softwaregateway.client.RemoveProfileClient
 +
http://node4.d.d4science.research-infrastructures.eu:8080/wsrf/services/gcube/vremanagement/softwaregateway/Registration  /gcube/devsec
 +
ContentManagement ViewManager 1.0.0 ViewManager-service 1.1.1
 +
</source>
 +
 +
 +
where:
 +
<source lang="text">
 +
Epr = http://node4.d.d4science.research-infrastructures.eu:8080/wsrf/services/gcube/vremanagement/softwaregateway/Registration
 +
Scope = /gcube/devsec
 +
ServiceClass  =  ContentManagement
 +
ServiceName    =  ViewManager
 +
ServiceVersion =  1.0.0
 +
PackageName    =  ViewManager-service
 +
PackageVersion =  1.1.1
 +
</source>

Latest revision as of 12:57, 6 September 2012

Role

Primary Roles:

  • the new service is a gateway over N Maven Repositories for the enabling layer
  • the new service will certifies which software is deployable withing the infrastructure

More specifically:

  • it represents a single point of configuration of a cluster of Maven Repositories in the whole infrastructure (hence, will avoid to configure each gHN for accessing to the recognized Maven repositories)
  • enabling services aiming at communicating with the registered Maven Repositories must talk with the Software Gateway
  • external services may communicate directly with the Maven Repositories
  • it minimizes the impact of the new physical storage (Maven) on the other enabling layer services
  • it serves as mediator from the gCube coordinates (service name, class, version, package name, package version) to the Maven coordinates (groudID,artifactID,version)
  • the service is responsible for publishing service profiles in the Information System
  • the service does not deal with the packages storage
  • the service is scope-agnostic (all the software available in the maven repository must be visible to all the clients)

What it is not be:

  • a gateway over N Maven Repositories for the rest of the world (only for the enabling layer)
  • a wrapper around the Maven API

Packaging Model

As a result of the gCube mavenization, gCube adopted the following packaging model:

About gCube components, a build of component generates 2 artifacts:

  1. the main artifact (JAR/WAR/GAR);
  2. a second artifact for deployment and distribution purposes (Software Archive) with the descriptors of the artifact (README, LICENSE, INSTALL, MAINTAINERS), the pom.xml, profile.xml and all the other files belonging the package (such as installation script, reboot scripts and so on), including the main artifact. It will be a tar.gz archive.

2) may contain more than one artifact, depending on the nature of the gCube component (Maven-based or Ant-based).

About external components:

  • usually there is 1 artifact (the main one) per component (hosted in a non-gCube Maven repository)
  • optionally, there exists a Software Archive artifact per component (hosted in a gCube Maven repository) in case that custom deployment actions or special needs are needed.

How SG deals with them? Regarding the main artifacts, the SG "simply" interfaces the configured Maven repositories to look for them and deals only with their main binary artifact.

Regarding the SA artifacts, the SG needs to do some extra-management actions when they are multi-packages.

Scenarios

gCube Software Registration

gCube Software Registration

Removal

Software Removal

Support to Deployment

Software Deployment Interactions

Service Interface

Registration portType:

  • String register(profile): accepts a service profile with a list of packages inside, return the ID assigned. The profile must include the Maven coordinates for each package
  • void unregister(ServiceName, ServiceClass, ServiceVersion, PackageName, PackageVersion)

Access portType:

  • URL getLocation(ServiceName, ServiceClass, ServiceVersion, PackageName, PackageVersion): returns the HTTP location from which the package can be downloaded
  • URL getSALocation(ServiceName, ServiceClass, ServiceVersion, PackageName, PackageVersion): returns the HTTP location of the Service Archive connected to the package, if any. If a SA is returned, the Deployer executes the instructions inside as pre/post deployment and undeployment operations
  • List<package> getDependencies(ServiceName, ServiceClass, ServiceVersion, PackageName, PackageVersion): returns an XML document reporting the actual resolved dependencies and the missed dependencies (unresolved). PackageName and PackageVersion are optional parameters: if they are not set, the operation must look for the package declared as "service" and for the last version
  • List<package> getPackages(ServiceName, ServiceClass, ServiceVersion): return the gCube packages belonging that service.
  • List<package> getPlugins(ServiceName, ServiceClass, ServiceVersion, PackageName, PackageVersion): returns the list of plugins targeting the package or an empty list

All the above operations throw a ServiceNotAvailableFault if the requested package is not available in any repository.

Implementation

Conversions

Software Gateway deals with two types of coordinates: gCube coordinates and Maven coordinates.

GCube coordinates are expressed as quintuples: <ServiceClass, ServiceName, ServiceVersion, PackageName, PackageVersion>.

Maven coordinates are expressed as triples: <groupId, artifactId, version>.

The Software Gateway often needs to convert gCube coordinates in Maven coordinates and vice versa. In such cases, the following conversions are applied:

  • Conversion gCubeToMaven
  • groupId = service class
  • artifactid = package name
  • version = package version
  • Conversion MavenTogCube
  • service class = groupId
  • service name = artifactId
  • service version = 1.0.0
  • package name = artifactId
  • package version = version

Registration PortType

Registering a New Software

The service receives in input a service profile and registers it in all the scopes it is configured to run. The service returns a report including information about the newly registered gCube packages.

Here is an example of such a report:

<Packages>
 <Package>
    <groupID>org.gcube.common</groupID>
    <artifactID>rapidminer-custom</artifactID>
    <version>1.0.0</version>
    <ID>d3217750-9b6a-11e1-be35-f7d8862056ea</ID>
    <URL>http://maven.research-infrastructures.eu/nexus/service/local/artifact/maven/redirect?r=gcube-staging&g=org.gcube.common&a=rapidminer-custom&v=1.0.0&e=jar</URL>
    <javadoc>http://maven.research-infrastructures.eu/nexus/service/local/artifact/maven/redirect?r=gcube-staging&g=org.gcube.common&a=rapidminer-custom&v=1.0.0&c=javadoc</javadoc>
    <Status>SUCCESS</Status>
    <Operation>NEW</Operation>
    <Timestamp>1336741892744</Timestamp>
 </Package>
</Packages>

Unregistering a Software

The service receives in input the gCube coordinates that identify a package. Then, it performs the following steps:

  • query the IS in order to harvest the profile matching such gCube coordinates
  • if the profile reports only that package, the whole profile is deleted from the IS (all scopes)
  • if the profile reports other packages, the package is withdrawn from the list of packages and the modified profile is updated in the IS (all scopes)

Access PortType

Getting a Software Location

The service receives in input the gCube coordinates that identify a package. Then, it performs the following steps:

  • query the IS in order to harvest the profile matching such gCube coordinates
  • if a profile is returned, then extract the Maven coordinates from the profile
  • if not, the conversion gCubeToMaven is applied
  • having the Maven coordinates, query the configured Maven repositories until a main artifact with such coordinates is found
  • find the URL of the artifact (with the REST interface exposed by Maven)
  • return the URL from which the package can be downloaded

Getting Dependencies

The service receives in input the gCube coordinates that identify a package. Then, it performs the following steps:

  • query the IS in order to harvest the profile matching such gCube coordinates
  • if a profile is returned, then extract the Maven coordinates from the profile
  • if not, the conversion gCubeToMaven is applied
  • having the Maven coordinates, query the configured Maven repositories and prepare the following 2 lists:
  • resolved dependencies
  • missing dependencies
  • elements in the list are expressed in Maven coordinates that must be converted in gCube coordinates
  • for each element in the lists, query the IS in order to find the service profile with such Maven coordinates
  • if a profile is returned, the gCube package's coordinates are extracted from the profile
  • if not, the conversion MavenTogCube is applied
  • finally, prepare the XML report to return to the caller

Here's an example of the report returned:

<DependencyResolutionReport>
	<ResolvedDependencies>
	 	<Dependency>
	 		<Service>
	 			<Class>Portal</Class>
				<Name>ThumbnailService</Name>
				<Version>1.00.00</Version>
 			</Service>
 			<Package>Thumbnailer-service</Package>
 			<Version>1.00.00</Version>
 			<Scope level="GHN"/>
 		</Dependency>
 		<Dependency>
 			<Service>
				<Class>Portal</Class>
 				<Name>ThumbnailService</Name>
				<Version>1.00.00</Version>
 			</Service>
 			<Package>Thumbnailer-stubs</Package>
			<Version>1.00.00</Version>
			<Scope level="GHN"/>
 		</Dependency>
 	</ResolvedDependencies>
 	<MissingDependencies>
 		<MissingDependency>
	 		 	<Service>
	 		 		<Class>ExternalSoftware</Class>
	 		 		<Name>imageJ</Name>
	 		 		<Version>1.40.00</Version>
	 		 	</Service>
	 		 <Package>imageJ</Package>
	 		 <Version>1.40.00</Version>
	 		 <Scope level="GHN"/>
 		</MissingDependency>
	</MissingDependencies>
</DependencyResolutionReport>

Getting a Software Archive

The service receives in input the gCube coordinates that identify a package. Then, it performs the following steps:

  • query the IS in order to harvest the profile matching such gCube coordinates
  • if a profile is returned, then extract the Maven coordinates from the profile
  • if not, the conversion gCubeToMaven is applied
  • having the Maven coordinates, query the configured Maven repositories to find the secondary artifact (we will see later on how it is characterized) related to the package
  • find the URL of the artifact (with the REST interface exposed by Maven)
  • download locally the tarball and uncompress it
  • if it contains more than one package, create a local SA archive with only the package requested
  • re-tar the SA and put in a local (cachable folder)
  • return the URL from which the local SA can be downloaded

SGClient

A client library that implements a high-level facade to the remote APIs of the Software Gateway service

Interface

Each method invokes the corresponding method in the SoftwareGateway service. These methods must be called by console. Each method accepts a set of standard parameters that are: scope and epr . Furthermore, the methods accepts the parameter specified below:

  • DepSolverClient, accepts a set of gcube package coordinates and return an XML that contains the dependencies resolved and missed
  • GetLocationClient, accepts a set of gcube package coordinates and returns the url of the package associated
  • GetPackagesClient, accepts a set of gcube service coordinates and returns a list o package coordinates
  • GetPluginsClient, accepts a set of gcube service coordinates and returns a list of plugin's coordinated that match with the service
  • GetSALocationClient, returning the ServiceArchive location for a inptu set of gcube package coordinates
  • RegisterProfileClient, accepts a profile and register it on the information system service
  • RemoveProfileClient, unregister a profile from information system service, identifies by the input coordinates

Sample Usage

The following example show a call from console to the RegisterProfileClient method, for register a profile present in the file profilexm.xml

java -classpath softwaregateway-stubs-1.0.0-SNAPSHOT.jar:SGClient.jar RegisterProfileClient 
http://node4.d.d4science.research-infrastructures.eu:8080/wsrf/services/gcube/vremanagement/softwaregateway/Registration  
/gcube/devsec /home/xxx/Documenti/SoftwareGatewayDocs/yyy/profilevm.xml

where:

epr              = http://node4.d.d4science.research-infrastructures.eu:8080/wsrf/services/gcube/vremanagement/softwaregateway/Registration
scope            = /gcube/devsec
path to the file = /home/xxx/Documenti/SoftwareGatewayDocs/yyy/profilevm.xml



The following example show a call from console to the UnregisterProfileClient method.

 java -classpath softwaregateway-stubs-1.0.0-SNAPSHOT.jar:SGClient.jar  org.gcube.vremanagement.softwaregateway.client.RemoveProfileClient
 http://node4.d.d4science.research-infrastructures.eu:8080/wsrf/services/gcube/vremanagement/softwaregateway/Registration  /gcube/devsec 
 ContentManagement ViewManager 1.0.0 ViewManager-service 1.1.1


where:

Epr = http://node4.d.d4science.research-infrastructures.eu:8080/wsrf/services/gcube/vremanagement/softwaregateway/Registration
Scope = /gcube/devsec
ServiceClass   =  ContentManagement 
ServiceName    =  ViewManager 
ServiceVersion =  1.0.0 
PackageName    =  ViewManager-service 
PackageVersion =  1.1.1