Difference between revisions of "Software Repository"

From Gcube Wiki
Jump to: navigation, search
(→‎Software Repository)
(→‎Dependencies)
 
(45 intermediate revisions by 3 users not shown)
Line 3: Line 3:
 
=== Introduction ===
 
=== Introduction ===
  
−
The Software Repository validates, stores, and manages packages. It checks packages dependencies and giving both access to packages relations and access to the stored packages supports the dynamic packages deployment.  
+
The Software Repository validates Software Archive, and stores and manages Packages. It checks packages dependencies and giving both access to packages relations and access to the stored packages supports the dynamic packages deployment.  
  
 
This repository accepts registration requests coming from the Sofware Archive Registration Portlet that is part of the VRE Management
 
This repository accepts registration requests coming from the Sofware Archive Registration Portlet that is part of the VRE Management
Line 13: Line 13:
 
The Software Repository is a WSRF service, implemented as specified by the Singleton Pattern.
 
The Software Repository is a WSRF service, implemented as specified by the Singleton Pattern.
  
−
The service includes a Validation phase that checks software archives based on service profiles declarations. It control also that service profiles are valid against the service profile schema. Controls performed includes checking that all declared files are in the right place into the service archive structure. After validation phase (if successfull) it store in a pending state all packages declared in service profile. When the Software Archive is approved the Software Repository finally store all packages in the repository for dynamic deployment
+
The service includes a Validation phase that checks software archives based on service profiles declarations. It control also that service profiles are valid against the service profile schema. Controls performed includes checking that all declared files are in the right place into the service archive structure. After validation phase (if successful) it store in a pending state all packages declared in service profile. When the Software Archive is approved the Software Repository finally store all packages in the repository for dynamic deployment.
 +
 
 +
To submit a Software Archive to the Software Repository some clients are avalaiable. More information are available [[Software_Repository_Clients | here]].
  
  
Line 20: Line 22:
 
The signatures of the actual implemented methods are presented below. These operations allow uploading, deleting, listing, and getting package. As well as other advanced operations that operate over packages dependencies.  
 
The signatures of the actual implemented methods are presented below. These operations allow uploading, deleting, listing, and getting package. As well as other advanced operations that operate over packages dependencies.  
  
−
 
+
===== store =====
−
:1. '''store'''
+
 
<pre>
 
<pre>
−
store(StoreMessage) -> String
+
store(Store) -> String
 
</pre>
 
</pre>
  
Line 29: Line 30:
 
The main issue of this method is the validation. You can find more informations about the controls performed on [[ Software_Archive_Specification | Software Archives Specification]]
 
The main issue of this method is the validation. You can find more informations about the controls performed on [[ Software_Archive_Specification | Software Archives Specification]]
  
 +
 +
''Store'' is an array of ''StoreMessage'' type.
  
 
''StoreMessage'' type contains Service Class, Service Name and Version of the service, the URL from where the software archive can be downloaded, and the service description. Further contains a String[] with scopes where the Service should be published.
 
''StoreMessage'' type contains Service Class, Service Name and Version of the service, the URL from where the software archive can be downloaded, and the service description. Further contains a String[] with scopes where the Service should be published.
  
−
''Return'' String with the result description NEW (if is the service is not already stored), UPDATE if the service is already present on Software Repsoitory and will be updated.
+
''Return'' String containing an XML with the information about the validation of Software Archive.
  
 +
An example of this XML String is:
  
−
:2. '''approve'''
+
<pre>
 +
<Services>
 +
<Service>
 +
<URL>http://grids16.eng.it/repository_recent_builds/org.gcube.HEAD/BUILD_187/org.gcube.vo-management.credential-renewal-servicearchive.HEAD/org.gcube.vo-management.credential-renewal-servicearchive-0.0.0-0.tar.gz</URL>
 +
<ID>082677d0-5cf3-11dd-a000-e94252afcc8c</ID>
 +
<ServiceClass>DVOS</ServiceClass>
 +
<ServiceName>CredentialsRenewalService</ServiceName>
 +
<ServiceVersion>0.0.0</ServiceVersion>
 +
<Status>ERROR</Status>
 +
<Operation>NEW</Operation>
 +
<Timestamp>1217283477453</Timestamp>
 +
<ReportURL>http://dlib05.isti.cnr.it:80/report/29-07-2008/082677d0-5cf3-11dd-a000-e94252afcc8c_1217283477453.xml</ReportURL>
 +
</Service>
 +
<Service>
 +
<URL>http://grids16.eng.it/repository_recent_builds/org.gcube.HEAD/BUILD_187/org.gcube.vre-management.deployer-servicearchive.HEAD/org.gcube.vre-management.deployer-servicearchive-0.0.0-0.tar.gz</URL>
 +
<ID>d253bca0-5903-11dd-9bc1-c9ec09a1eecb</ID>
 +
<ServiceClass>VREManagement</ServiceClass>
 +
<ServiceName>Deployer</ServiceName>
 +
<ServiceVersion>0.0.0</ServiceVersion>
 +
<Status>SUCCESS</Status>
 +
<Operation>UPDATE</Operation>
 +
<Timestamp>1217283478524</Timestamp>
 +
<ReportURL>http://dlib05.isti.cnr.it:80/report/29-07-2008/d253bca0-5903-11dd-9bc1-c9ec09a1eecb_1217283478524.xml</ReportURL>
 +
</Service>
 +
<Service>
 +
<URL>http://grids16.eng.it/repository_recent_builds/org.gcube.HEAD/BUILD_187/org.gcube.vre-management.software-repository-servicearchive.HEAD/org.gcube.vre-management.software-repository-servicearchive-0.0.0-0.tar.gz</URL>
 +
<ID>d31c63d0-5903-11dd-9bc1-c9ec09a1eecb</ID>
 +
<ServiceClass>VREManagement</ServiceClass>
 +
<ServiceName>SoftwareRepository</ServiceName>
 +
<ServiceVersion>0.0.0</ServiceVersion>
 +
<Status>SUCCESS</Status>
 +
<Operation>UPDATE</Operation>
 +
<Timestamp>1217283480125</Timestamp>
 +
<ReportURL>http://dlib05.isti.cnr.it:80/report/29-07-2008/d31c63d0-5903-11dd-9bc1-c9ec09a1eecb_1217283480125.xml</ReportURL>
 +
</Service>
 +
</Services>
 +
</pre>
 +
 
 +
 
 +
During this phase a Report about the validation is generated. You can find informations about the [[ Report_Generation | Report Generation]]
 +
 
 +
===== listPending =====
 +
<pre>
 +
listPending( ) -> String
 +
</pre>
 +
 
 +
''Return'' This operation returns an XML containing all stored service archives with status pending. The XML structure is as follow:
 +
 
 +
<pre>
 +
<Services>
 +
<Service>
 +
  <Id>d253bca0-5903-11dd-9bc1-c9ec09a1eecb</Id>
 +
  <Class>VREManagement</Class>
 +
  <Name>Deployer</Name>
 +
  <Version>0.0.0</Version>
 +
</Service>
 +
</Services>
 +
</pre>
 +
 
 +
===== approve =====
 
<pre>
 
<pre>
 
approve (String) -> String
 
approve (String) -> String
Line 46: Line 109:
 
''Return'' It returns OK if operation success, an error message otherwise.
 
''Return'' It returns OK if operation success, an error message otherwise.
  
−
:3. '''delete'''
+
 
 +
'''ATTENTION''' The Software Repository Running Instance (RI) have to be available in all scopes it have to publish the service profile.
 +
 
 +
 
 +
===== delete =====
 
<pre>
 
<pre>
 
delete (String) -> String
 
delete (String) -> String
 
</pre>
 
</pre>
  
−
This operation deletes a stored pending software archive from the Software Repository with the service ID passed as argument if exists.  
+
This operation deletes a stored pending Software Archive from the Software Repository with the service ID passed as argument if exists.  
  
 
''String'' Service ID
 
''String'' Service ID
Line 57: Line 124:
 
''Return'' It returns OK if operation success, an error message otherwise.
 
''Return'' It returns OK if operation success, an error message otherwise.
  
−
:4. '''get'''
+
===== get =====
 
<pre>
 
<pre>
−
get (ServiceIDPackage) -> String
+
get(GetMessage) -> String
 
</pre>
 
</pre>
  
−
To identify an unique package on the repository two values must be provided: (i) DILIGENT unique ID and (ii) package name.
+
 
 +
''GetMessage'' type contains Service Class, Service Name, Service Version, Package Name and Package Version of the requested package
  
 
''Return'' The method returns the URI from where the requested package can be downloaded.
 
''Return'' The method returns the URI from where the requested package can be downloaded.
  
−
:5. '''getServiceProfile'''
+
===== isDeployable =====
 
<pre>
 
<pre>
−
getServiceProfile (String) -> String
+
isDeployable(String) -> boolean
 
</pre>
 
</pre>
  
−
''Return'' From a given unique ID this method return the corresponding Service Profile if exists, null otherwise.
+
With this functionality it is possible to know if a service is deployable. This means all dependences are deployable too.
  
−
:6. '''getServiceProfileID'''
+
''String'' Service ID
−
<pre>
+
−
getServiceProfileID (ClsssName) -> String
+
−
</pre>
+
  
−
''Return'' From a given class and name parameter this method return the corresponding DILIGENT unique ID if exists, null otherwise.
+
''Return'' true if and only if the dependency resolution has success and there are not missing dependency, false otherwise
  
−
:7. '''listPending'''
+
===== listScopedPackagesByServiceData =====
 
<pre>
 
<pre>
−
listPending ( ) -> String
+
listScopedPackagesByServiceData(ListScopedPackagesByServiceDataMessage) -> String
 
</pre>
 
</pre>
  
−
''Return'' This operation returns an XML containing all stored service archives with status pending. The XML structure is as follow:
+
This operation lists all dependencies with a certain scope level given as argument.
−
<pre>
+
−
<Resultset>
+
−
  <Result>
+
−
    <UniqueID>String</UniqueID>
+
−
    <Class>String</Class>
+
−
    <Name>String</Name>
+
−
    <PackageName>String</PackageName>
+
−
  </Result>
+
−
</Resultset>
+
−
</pre>
+
  
−
:8. '''listApproved'''
+
''ListScopedPackagesByServiceDataMessage'' type contains Service Class, Service Name, Service Version and scope(GHN, VRE, VO)
−
<pre>
+
−
listApproved ( ) -> String
+
−
</pre>
+
  
−
''Return'' This operation returns an XML containing all stored service archives with status approved. The XML structure is as follow:
+
''Return'' XML string with solved and missing (if any) dependency
−
<pre>
+
−
<Resultset>
+
−
  <Result>
+
−
    <UniqueID>String</UniqueID>
+
−
    <Class>String</Class>
+
−
    <Name>String</Name>
+
−
    <PackageName>String</PackageName>
+
−
  </Result>
+
−
</Resultset>
+
−
</pre>
+
  
−
:9. '''listNonDeployableServices'''
 
 
<pre>
 
<pre>
−
listNonDeployableServices ( ) -> String
+
<DependencyResolutionReport>
 +
<ResolvedDependencies>
 +
<Dependency>
 +
<Service>
 +
<Class>Portal</Class>
 +
<Name>ThumbnailService</Name>
 +
<Version>1.00.00</Version>
 +
</Service>
 +
<Package>Thumbnailer-service</Package>
 +
<Version>1.00.00</Version>
 +
<Scope level="GHN"/>
 +
</Dependency>
 +
<Dependency>
 +
<Service>
 +
<Class>Portal</Class>
 +
<Name>ThumbnailService</Name>
 +
<Version>1.00.00</Version>
 +
</Service>
 +
<Package>Thumbnailer-stubs</Package>
 +
<Version>1.00.00</Version>
 +
<Scope level="GHN"/>
 +
</Dependency>
 +
</ResolvedDependencies>
 +
<MissingDependencies>
 +
<MissingDependency>
 +
<Service>
 +
<Class>ExternalSoftware</Class>
 +
<Name>imageJ</Name>
 +
<Version>1.40.00</Version>
 +
</Service>
 +
<Package>imageJ</Package>
 +
<Version>1.40.00</Version>
 +
<Scope level="GHN"/>
 +
</MissingDependency>
 +
</MissingDependencies>
 +
</DependencyResolutionReport>
 
</pre>
 
</pre>
  
−
With this functionality it is possible to know which stored packages are not deployable due missing dependencies.
+
===== listScopedPackages =====
−
 
+
−
''Return'' a XML with the following structure:
+
 
<pre>
 
<pre>
−
<Resultset>
+
listScopedPackages(ListScopedPackagesMessage) -> String
−
  <Result>
+
−
    <Class>String</Class>
+
−
    <Name>String</Name>
+
−
    <PackageName>String</PackageName>
+
−
  </Result>
+
−
</Resultset>
+
 
</pre>
 
</pre>
  
−
:10. '''listAllDependenciesChain'''
+
This operation lists all dependencies with a certain scope level given as argument.  
−
<pre>
+
−
listAllDependenciesChain (String) -> ListPackageArray
+
−
</pre>
+
  
−
This operation lists all dependencies declared on the WSRF package declared on the service profile of a given unique ID.
+
''ListScopedPackagesMessage'' type contains serviceID and scope(GHN, VRE, VO)
  
−
''Return'' Returned object type includes unique ID, package name, and type.
+
''Return'' XML string with solved and missing (if any) dependency
  
−
:11. '''listSameDHNPackages'''
 
 
<pre>
 
<pre>
−
listSameDHNPackages (String) -> ListPackageArray
+
<DependencyResolutionReport>
 +
<ResolvedDependencies>
 +
<Dependency>
 +
<Service>
 +
<Class>Portal</Class>
 +
<Name>ThumbnailService</Name>
 +
<Version>1.00.00</Version>
 +
</Service>
 +
<Package>Thumbnailer-service</Package>
 +
<Version>1.00.00</Version>
 +
<Scope level="GHN"/>
 +
</Dependency>
 +
<Dependency>
 +
<Service>
 +
<Class>Portal</Class>
 +
<Name>ThumbnailService</Name>
 +
<Version>1.00.00</Version>
 +
</Service>
 +
<Package>Thumbnailer-stubs</Package>
 +
<Version>1.00.00</Version>
 +
<Scope level="GHN"/>
 +
</Dependency>
 +
</ResolvedDependencies>
 +
<MissingDependencies>
 +
<MissingDependency>
 +
<Service>
 +
<Class>ExternalSoftware</Class>
 +
<Name>imageJ</Name>
 +
<Version>1.40.00</Version>
 +
</Service>
 +
<Package>imageJ</Package>
 +
<Version>1.40.00</Version>
 +
<Scope level="GHN"/>
 +
</MissingDependency>
 +
</MissingDependencies>
 +
</DependencyResolutionReport>
 
</pre>
 
</pre>
  
−
This operation lists all same DHN dependencies declared on the WSRF package declared on the service profile of a given unique ID.
+
===== listServicePackages =====
−
 
+
−
''Return'' Returned object type includes unique ID, package name, and type.
+
−
 
+
−
:12. '''listSameDLPackages'''
+
 
<pre>
 
<pre>
−
listSameDLPackages (String) -> ListPackageArray
+
listServicePackages(ListServicePackagesMessage) -> StringArray
 
</pre>
 
</pre>
  
−
This operation lists all same DL dependencies declared on the WSRF package declared on the service profile of a given unique ID.  
+
This operation lists all packages belonging the Service.  
  
−
''Return'' Returned object type includes unique ID, package name, and type.  
+
''ListServicePackagesMessage'' type contains Service Class, Service Name, Service Version
 +
 
 +
''Return'' StringArray each element contains an XML string with all information about a package. The XML structure is as follow:
  
−
:13. '''listSameVOPackages'''
 
 
<pre>
 
<pre>
−
listSameVOPackages (String) -> ListPackageArray
+
<Package>
 +
<ServiceClass>VREManagement</ServiceClass>
 +
<ServiceName>Deployer</ServiceName>
 +
<ServiceVersion>1.00.00</ServiceVersion>
 +
<PackageName>Deployer-service</PackageName>
 +
<PackageVersion>1.00.02</PackageVersion>
 +
<PackageDescription></PackageDescription>
 +
<URL>http://dlib25.isti.cnr.it:80/maven/VREManagement/Deployer/1/00/00/Deployer-service/1.00.02/Deployer-service-1.00.02.tar.gz</URL>
 +
<DocURL>http://dlib25.isti.cnr.it:80/maven/VREManagement/Deployer/1/00/00/Deployer-service/1.00.02/doc/api/index.html</DocURL>
 +
<SVN>http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/vre-management/Deployer</SVN>
 +
</Package>
 
</pre>
 
</pre>
  
−
This operation lists all same VO dependencies declared on the WSRF package declared on the service profile of a given unique ID.
+
===== listPluginsByServiceData =====
  
−
''Return'' Returned object type includes unique ID, package name, and type.
+
This operation lists all plugins belonging a gCube Service starting from its Class, Name and Version.  
  
−
=== Dependencies ===
+
This is a sample way to invoke the operation:
  
−
The dependencies of the Package Repository Service are:
+
<source lang="Java">
−
*DHN: to host service.
+
−
*NAL: to access to HNM.
+
−
*Keeper common: for type definition.
+
−
*Grid Storage: to store DILIGENT packages on the Grid.
+
−
*Profile Manager: to parse Service Profile
+
−
*Commons-io from Jakarta: to File system functionality.
+
−
*Commons-compress from Jakarta: to File compress functionality.
+
  
−
=== Usage Example ===
+
import org.gcube.vremanagement.softwarerepository.stubs.ListPluginsByServiceDataMessage;
 +
import org.gcube.vremanagement.softwarerepository.stubs.SoftwareRepositoryPortType;
 +
import org.gcube.vremanagement.softwarerepository.stubs.service.SoftwareRepositoryServiceAddressingLocator;
 +
...
  
−
==== Package Repository client ====
+
String serviceURI = "http://node8.d.d4science.research-infrastructures.eu:8080/wsrf/services/gcube/vremanagement/softwarerepository/SoftwareRepository";
 +
String serviceClass = "VREManagement";
 +
String serviceName = "Executor";
 +
String serviceVersion = "1.00.00";
 +
try {
 +
EndpointReferenceType epr = new EndpointReferenceType();
 +
epr.setAddress(new AttributedURI(serviceURI));
 +
GCUBEScope gCubeScope = GCUBEScope.getScope("/CNRPrivate");
 +
SoftwareRepositoryServiceAddressingLocator locator = new SoftwareRepositoryServiceAddressingLocator();
 +
SoftwareRepositoryPortType portType = locator.getSoftwareRepositoryPortTypePort(epr);
 +
portType = GCUBERemotePortTypeContext.getProxy(portType,gCubeScope);
 +
ListPluginsByServiceDataMessage message = new ListPluginsByServiceDataMessage();
 +
message.setServiceClass(serviceClass);
 +
message.setServiceName(serviceName);
 +
message.setServiceVersion(serviceVersion);
 +
String ret = portType.listPluginsByServiceData(message);
 +
System.out.println(ret);
 +
} catch (Exception e) {
 +
return;
 +
}
 +
...
 +
</source>
  
−
A Package Repository client is provided. The client uses all PR functionality in different ways, i.e. by calling directly the PR operations and by pre-processing the client input to generate the appropriate series of PR service calls.
+
And this is a sample response one may get from such a call:
  
−
===== Dependencies =====
+
<source lang="xml">
−
* Package Repository service
+
<ServicePlugins>
−
* Keeper common
+
<Service>
 +
<Class>Samples</Class>
 +
<Name>SamplePlugin</Name>
 +
<Version>1.00.00</Version>
 +
</Service>
 +
<Service>
 +
<Class>Samples</Class>
 +
<Name>SamplePluginN2</Name>
 +
<Version>1.00.00</Version>
 +
</Service>
 +
</ServicePlugins>
 +
</source>
  
−
===== Usage =====
+
==== Underlying DB ====
−
<pre>
+
−
java ClientPackageRepository pkgRepositoryURI store diligentID serviceArchiveURI description
+
−
 
+
−
java ClientPackageRepository pkgRepositoryURI storeList storeListFile
+
−
 
+
−
java ClientPackageRepository pkgRepositoryURI storeAndApproveList storeListFile
+
−
 
+
−
java ClientPackageRepository pkgRepositoryURI approve diligentID
+
−
 
+
−
java ClientPackageRepository pkgRepositoryURI approveList storeListFile
+
−
 
+
−
java ClientPackageRepository pkgRepositoryURI get diligentID packageName
+
−
 
+
−
java ClientPackageRepository pkgRepositoryURI delete diligentID
+
−
 
+
−
java ClientPackageRepository pkgRepositoryURI getServProfile diligentID
+
−
 
+
−
java ClientPackageRepository pkgRepositoryURI getServProfileID class name
+
−
 
+
−
java ClientPackageRepository pkgRepositoryURI listPending
+
−
 
+
−
java ClientPackageRepository pkgRepositoryURI listApproved
+
−
 
+
−
java ClientPackageRepository pkgRepositoryURI listNonDeployable
+
−
 
+
−
java ClientPackageRepository pkgRepositoryURI listAllDependenciesChain serviceID
+
−
 
+
−
java ClientPackageRepository pkgRepositoryURI listSameDHNPackages serviceID
+
−
 
+
−
java ClientPackageRepository pkgRepositoryURI listSameDLPackages serviceID
+
−
 
+
−
java ClientPackageRepository pkgRepositoryURI listSameVOPackages serviceID
+
−
</pre>
+
−
 
+
−
Where:
+
−
* pkgRepositoryURI: Package Repository address.
+
−
* diligentID: The unique ID of a Service Archive.
+
−
* serviceArchiveURI: The address from where the Service Archive can be downloaded.
+
−
* description: The Service Archive description.
+
−
* storeListFile: a list with Service Archives to store {diligentID, URI, description}.
+
−
* class: The Service Class.
+
−
* name: The Service Name.
+
−
* packageName: The package Name.
+
−
 
+
−
==== DILIGENT infrastructure initialization ====
+
−
 
+
−
In order to speed up the setting up of a DILIGENT infrastructure, and align available DILIGENT packages with current development process, an automated client that initialize the core DCL services (DIS and PR) is provided with the Package Repository component.
+
  
−
The procedure consists on a Thread that run the initialization method described below:
+
A Software Repository instance relies on an embedded instance of [http://hsqldb.org HLSQL DB]. In the default configuration, the instance is stored under the following folder:
 
<pre>
 
<pre>
−
1. Download from ETICS list of last SA built
+
~/.gcore/persisted/<host>-<port>/SoftwareRepository/dbFolder
−
2. Delete from DIS all SPs
+
−
3. Delete from PR all packages
+
−
4. For each SA built
+
−
    4.1. create a new resource into the DIS
+
−
    4.2. if successful store SA on PR
+
−
            4.2.1. if successful log success
+
−
                        else remove resource from DIS and log failure
+
−
            else log failure
+
 
</pre>
 
</pre>
  
−
The Infrastructure Initialization Thread can be run by typing:
+
In order to interact with the database instance in an off line manner (i.e. without the mediation of the SR), the [http://hsqldb.org/doc/guide/ch08.html SqlTool] can be used. Run
 
<pre>
 
<pre>
−
java org.diligentproject.keeperservice.packagerepositoryclients.InfrastructureInitialization  -initInfra
+
java -jar $GLOBUS_LOCATION/lib/hsqldb.jar --inlineRc url=jdbc:hsqldb:file:~/.gcore/persisted/<host>-<port>/SoftwareRepository/dbFolder/db,user=sa,password= --driver org.hsqldb.jdbcDriver
 
</pre>
 
</pre>
 +
for an interactive use of SqlTool.
  
−
The same client can be used to download togheter with the SA the related Source Packages and Javadoc Packages,
+
=== Dependencies ===
−
and create a local storage of source/bynaries/javadoc that can be stored on CVS/SVN repository.
+
−
To store a SA on the PR:
+
−
<pre>
+
−
java org.diligentproject.keeperservice.packagerepositoryclients.InfrastructureInitialization  -insert -module:<SERVICE_ARCHIVE_MODULE_NAME>
+
−
</pre>
+
−
To update a previously stored SA on the PR ( and the related Service Profile on the DIS):
+
−
<pre>
+
−
java org.diligentproject.keeperservice.packagerepositoryclients.InfrastructureInitialization  -update -module:<SERVICE_ARCHIVE_MODULE_NAME>
+
−
-id:<SA_ID>
+
−
</pre>
+
−
To Remove a SA from the PR (and the related ServiceProfile from the DIS)
+
−
<pre>
+
−
java org.diligentproject.keeperservice.packagerepositoryclients.InfrastructureInitialization  -delete -id:<SA_ID>
+
−
</pre>
+
−
To Download a SA binary package togheter with the related source packages and javadoc packages:
+
−
<pre>
+
−
java org.diligentproject.keeperservice.packagerepositoryclients.InfrastructureInitialization  -get -module:<SERVICE_ARCHIVE_MODULE_NAME>
+
−
</pre>
+
  
 +
The dependencies of the Package Repository Service are:
 +
*GHN: to host service.
 +
*Commons-compress from Jakarta: File compression functionality.
 +
*hsqldb: to use the DB
 +
*MavenCustom: to use maven repository functionality
  
  
−
===== Dependencies =====
+
=== Test-Suite ===
−
* activation.jar
+
−
* commons-compress-20061201.jar
+
−
* jaxb-api.jar
+
−
* jaxb-impl.jar
+
−
* jaxb-xjc.jar
+
−
* jaxb1-impl.jar
+
−
* jsr173_api.jar
+
−
* org_diligentproject_common_profilemanager.jar
+
−
* org_diligentproject_keeperservice_packagerepository_stubs.jar
+
−
* org_diligentproject_informationservice_disregistry_stubs.jar
+
−
 
+
−
===== Usage =====
+
−
 
+
−
Update appropriately the required parameters on the properties file
+
−
<pre>
+
−
org/diligentproject/keeperservice/packagerepositoryclients/InfrastructureInitialization.properties
+
−
</pre>
+
−
to refer to a completely deployed infrastructure:
+
−
* VERSION the gCube SA Version (i.e. 0.3.0)
+
−
* REPOSITORY_DIR the local dir where a SVN/CVS Repository is defined
+
−
* SLEEP_TIME: Thread sleep time in minutes
+
−
* TMP_DIR: directory where all Service Archives from build repository will be downloaded and parsed
+
−
* ETICS_SA_LIST_URL: build repository URI
+
−
* VO: The VO to Initialize
+
−
* PR_EPR: PR to update address
+
−
* SECURITY_ENABLED true/false if the security is enabled on the infra
+
−
* UPDATE_PR true/false if the Initialization thread has to update the PR
+
−
* UPDATE_DIS true/false if the Initialization thread has to update the DIS
+
−
* PROXY_FILE the local proxy file
+
−
* BASE_HTTP_ULR teh base HTTP Url where Etics SA are stored
+
−
* LOG4JFILE a custom log4j
+
  
−
Run the program from a machine with the CLASSPATH set appropriately.
+
[TBP]
  
 
=== Know Bugs and Limitations ===
 
=== Know Bugs and Limitations ===
  
−
In current implementation the proxy to access to the Grid need to be created manually.
+
[[Category:VRE Management]]

Latest revision as of 18:41, 17 December 2009

Software Repository

Introduction

The Software Repository validates Software Archive, and stores and manages Packages. It checks packages dependencies and giving both access to packages relations and access to the stored packages supports the dynamic packages deployment.

This repository accepts registration requests coming from the Sofware Archive Registration Portlet that is part of the VRE Management

Implentation Overview

This component is responsible for the validation, storage, and maintenance of the Software Archives, each software archive contains all files declared on the Service Profile . It checks packages dependencies and by ensuring access to them, it supports the dynamic packages deployment.

The Software Repository is a WSRF service, implemented as specified by the Singleton Pattern.

The service includes a Validation phase that checks software archives based on service profiles declarations. It control also that service profiles are valid against the service profile schema. Controls performed includes checking that all declared files are in the right place into the service archive structure. After validation phase (if successful) it store in a pending state all packages declared in service profile. When the Software Archive is approved the Software Repository finally store all packages in the repository for dynamic deployment.

To submit a Software Archive to the Software Repository some clients are avalaiable. More information are available here.


Functionality

The signatures of the actual implemented methods are presented below. These operations allow uploading, deleting, listing, and getting package. As well as other advanced operations that operate over packages dependencies.

store
store(Store) -> String

This operation stores a software archive into the Software Repository in pending state. Pending status means that software archive is valid, and receive an unique ID that it was not previously used. The main issue of this method is the validation. You can find more informations about the controls performed on Software Archives Specification


Store is an array of StoreMessage type.

StoreMessage type contains Service Class, Service Name and Version of the service, the URL from where the software archive can be downloaded, and the service description. Further contains a String[] with scopes where the Service should be published.

Return String containing an XML with the information about the validation of Software Archive.

An example of this XML String is: 

<Services>
	<Service>
		<URL>http://grids16.eng.it/repository_recent_builds/org.gcube.HEAD/BUILD_187/org.gcube.vo-management.credential-renewal-servicearchive.HEAD/org.gcube.vo-management.credential-renewal-servicearchive-0.0.0-0.tar.gz</URL>
		<ID>082677d0-5cf3-11dd-a000-e94252afcc8c</ID>
		<ServiceClass>DVOS</ServiceClass>
		<ServiceName>CredentialsRenewalService</ServiceName>
		<ServiceVersion>0.0.0</ServiceVersion>
		<Status>ERROR</Status>
		<Operation>NEW</Operation>
		<Timestamp>1217283477453</Timestamp>
		<ReportURL>http://dlib05.isti.cnr.it:80/report/29-07-2008/082677d0-5cf3-11dd-a000-e94252afcc8c_1217283477453.xml</ReportURL>
	</Service>
	<Service>
		<URL>http://grids16.eng.it/repository_recent_builds/org.gcube.HEAD/BUILD_187/org.gcube.vre-management.deployer-servicearchive.HEAD/org.gcube.vre-management.deployer-servicearchive-0.0.0-0.tar.gz</URL>
		<ID>d253bca0-5903-11dd-9bc1-c9ec09a1eecb</ID>
		<ServiceClass>VREManagement</ServiceClass>
		<ServiceName>Deployer</ServiceName>
		<ServiceVersion>0.0.0</ServiceVersion>
		<Status>SUCCESS</Status>
		<Operation>UPDATE</Operation>
		<Timestamp>1217283478524</Timestamp>
		<ReportURL>http://dlib05.isti.cnr.it:80/report/29-07-2008/d253bca0-5903-11dd-9bc1-c9ec09a1eecb_1217283478524.xml</ReportURL>
	</Service>
	<Service>
		<URL>http://grids16.eng.it/repository_recent_builds/org.gcube.HEAD/BUILD_187/org.gcube.vre-management.software-repository-servicearchive.HEAD/org.gcube.vre-management.software-repository-servicearchive-0.0.0-0.tar.gz</URL>
		<ID>d31c63d0-5903-11dd-9bc1-c9ec09a1eecb</ID>
		<ServiceClass>VREManagement</ServiceClass>
		<ServiceName>SoftwareRepository</ServiceName>
		<ServiceVersion>0.0.0</ServiceVersion>
		<Status>SUCCESS</Status>
		<Operation>UPDATE</Operation>
		<Timestamp>1217283480125</Timestamp>
		<ReportURL>http://dlib05.isti.cnr.it:80/report/29-07-2008/d31c63d0-5903-11dd-9bc1-c9ec09a1eecb_1217283480125.xml</ReportURL>
	</Service>
</Services>


During this phase a Report about the validation is generated. You can find informations about the Report Generation

listPending
listPending( ) -> String

Return This operation returns an XML containing all stored service archives with status pending. The XML structure is as follow: 

<Services>
 <Service>
  <Id>d253bca0-5903-11dd-9bc1-c9ec09a1eecb</Id>
  <Class>VREManagement</Class>
  <Name>Deployer</Name>
  <Version>0.0.0</Version>
 </Service>
</Services>
approve
approve (String) -> String

Approve an already stored service archive with the service ID passed as argument if exists.

String Service ID

Return It returns OK if operation success, an error message otherwise.


ATTENTION The Software Repository Running Instance (RI) have to be available in all scopes it have to publish the service profile.


delete
delete (String) -> String

This operation deletes a stored pending Software Archive from the Software Repository with the service ID passed as argument if exists.

String Service ID

Return It returns OK if operation success, an error message otherwise.

get
get(GetMessage) -> String


GetMessage type contains Service Class, Service Name, Service Version, Package Name and Package Version of the requested package

Return The method returns the URI from where the requested package can be downloaded.

isDeployable
isDeployable(String) -> boolean

With this functionality it is possible to know if a service is deployable. This means all dependences are deployable too.

String Service ID

Return true if and only if the dependency resolution has success and there are not missing dependency, false otherwise

listScopedPackagesByServiceData
listScopedPackagesByServiceData(ListScopedPackagesByServiceDataMessage) -> String

This operation lists all dependencies with a certain scope level given as argument.

ListScopedPackagesByServiceDataMessage type contains Service Class, Service Name, Service Version and scope(GHN, VRE, VO)

Return XML string with solved and missing (if any) dependency 

<DependencyResolutionReport>
	<ResolvedDependencies>
		<Dependency>
			<Service>
				<Class>Portal</Class>
				<Name>ThumbnailService</Name>
				<Version>1.00.00</Version>
			</Service>
			<Package>Thumbnailer-service</Package>
			<Version>1.00.00</Version>
			<Scope level="GHN"/>
		</Dependency>
		<Dependency>
			<Service>
				<Class>Portal</Class>
				<Name>ThumbnailService</Name>
				<Version>1.00.00</Version>
			</Service>
			<Package>Thumbnailer-stubs</Package>
			<Version>1.00.00</Version>
			<Scope level="GHN"/>
		</Dependency>
	</ResolvedDependencies>
	<MissingDependencies>
		<MissingDependency>
			<Service>
				<Class>ExternalSoftware</Class>
				<Name>imageJ</Name>
				<Version>1.40.00</Version>
			</Service>
			<Package>imageJ</Package>
			<Version>1.40.00</Version>
			<Scope level="GHN"/>
		</MissingDependency>
	</MissingDependencies>
</DependencyResolutionReport>
listScopedPackages
listScopedPackages(ListScopedPackagesMessage) -> String

This operation lists all dependencies with a certain scope level given as argument.

ListScopedPackagesMessage type contains serviceID and scope(GHN, VRE, VO)

Return XML string with solved and missing (if any) dependency 

<DependencyResolutionReport>
	<ResolvedDependencies>
		<Dependency>
			<Service>
				<Class>Portal</Class>
				<Name>ThumbnailService</Name>
				<Version>1.00.00</Version>
			</Service>
			<Package>Thumbnailer-service</Package>
			<Version>1.00.00</Version>
			<Scope level="GHN"/>
		</Dependency>
		<Dependency>
			<Service>
				<Class>Portal</Class>
				<Name>ThumbnailService</Name>
				<Version>1.00.00</Version>
			</Service>
			<Package>Thumbnailer-stubs</Package>
			<Version>1.00.00</Version>
			<Scope level="GHN"/>
		</Dependency>
	</ResolvedDependencies>
	<MissingDependencies>
		<MissingDependency>
			<Service>
				<Class>ExternalSoftware</Class>
				<Name>imageJ</Name>
				<Version>1.40.00</Version>
			</Service>
			<Package>imageJ</Package>
			<Version>1.40.00</Version>
			<Scope level="GHN"/>
		</MissingDependency>
	</MissingDependencies>
</DependencyResolutionReport>
listServicePackages
listServicePackages(ListServicePackagesMessage) -> StringArray

This operation lists all packages belonging the Service.

ListServicePackagesMessage type contains Service Class, Service Name, Service Version

Return StringArray each element contains an XML string with all information about a package. The XML structure is as follow: 

<Package>
	<ServiceClass>VREManagement</ServiceClass>
	<ServiceName>Deployer</ServiceName>
	<ServiceVersion>1.00.00</ServiceVersion>
	<PackageName>Deployer-service</PackageName>
	<PackageVersion>1.00.02</PackageVersion>
	<PackageDescription></PackageDescription>
	<URL>http://dlib25.isti.cnr.it:80/maven/VREManagement/Deployer/1/00/00/Deployer-service/1.00.02/Deployer-service-1.00.02.tar.gz</URL>
	<DocURL>http://dlib25.isti.cnr.it:80/maven/VREManagement/Deployer/1/00/00/Deployer-service/1.00.02/doc/api/index.html</DocURL>
	<SVN>http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/vre-management/Deployer</SVN>
</Package>
listPluginsByServiceData

This operation lists all plugins belonging a gCube Service starting from its Class, Name and Version.

This is a sample way to invoke the operation: 

import org.gcube.vremanagement.softwarerepository.stubs.ListPluginsByServiceDataMessage;
import org.gcube.vremanagement.softwarerepository.stubs.SoftwareRepositoryPortType;
import org.gcube.vremanagement.softwarerepository.stubs.service.SoftwareRepositoryServiceAddressingLocator;
...
 
	String serviceURI = "http://node8.d.d4science.research-infrastructures.eu:8080/wsrf/services/gcube/vremanagement/softwarerepository/SoftwareRepository";		
	String serviceClass = "VREManagement";
	String serviceName = "Executor";
	String serviceVersion = "1.00.00";
	try {
		EndpointReferenceType epr = new EndpointReferenceType();
		epr.setAddress(new AttributedURI(serviceURI));
		GCUBEScope gCubeScope = GCUBEScope.getScope("/CNRPrivate");			
		SoftwareRepositoryServiceAddressingLocator locator = new SoftwareRepositoryServiceAddressingLocator();
		SoftwareRepositoryPortType portType = locator.getSoftwareRepositoryPortTypePort(epr);
		portType = GCUBERemotePortTypeContext.getProxy(portType,gCubeScope);
		ListPluginsByServiceDataMessage message = new ListPluginsByServiceDataMessage();
		message.setServiceClass(serviceClass);
		message.setServiceName(serviceName);
		message.setServiceVersion(serviceVersion);			
		String ret = portType.listPluginsByServiceData(message);
		System.out.println(ret);
	} catch (Exception e) {
		return;
	}
...

And this is a sample response one may get from such a call: 

<ServicePlugins>
	<Service>
		<Class>Samples</Class>
		<Name>SamplePlugin</Name>
		<Version>1.00.00</Version>
	</Service>
	<Service>
		<Class>Samples</Class>
		<Name>SamplePluginN2</Name>
		<Version>1.00.00</Version>
	</Service>
</ServicePlugins>

Underlying DB

A Software Repository instance relies on an embedded instance of HLSQL DB. In the default configuration, the instance is stored under the following folder: 

~/.gcore/persisted/<host>-<port>/SoftwareRepository/dbFolder

In order to interact with the database instance in an off line manner (i.e. without the mediation of the SR), the SqlTool can be used. Run 

java -jar $GLOBUS_LOCATION/lib/hsqldb.jar --inlineRc url=jdbc:hsqldb:file:~/.gcore/persisted/<host>-<port>/SoftwareRepository/dbFolder/db,user=sa,password= --driver org.hsqldb.jdbcDriver

for an interactive use of SqlTool.

Dependencies

The dependencies of the Package Repository Service are:

  • GHN: to host service.
  • Commons-compress from Jakarta: File compression functionality.
  • hsqldb: to use the DB
  • MavenCustom: to use maven repository functionality


Test-Suite

[TBP]

Know Bugs and Limitations

Retrieved from "https://wiki.gcube-system.org/index.php?title=Software_Repository&oldid=8220"