Difference between revisions of "Package Repository"

Revision as of 10:19, 23 January 2008

Package Repository

Introduction

The Package Repository validates, stores, and manages DILIGENT packages. It checks packages dependencies and giving both access to packages relations and access to the stored packages supports the dynamic packages deployment.

This repository accepts registration requests coming from the Service Archive Registration Protlet that is part of the VDL service, whilst accepts access requests from the DL service and the HNM service.

Implentation Overview

This component is responsible for the validation, storage, and maintenance of the DILIGENT service archives, each service archive contains all files declared on the Service Profile . It checks packages dependencies and by ensuring access to them, it supports the dynamic packages deployment.

The Package Repository is a WSRF service, implemented as specified by the Singleton Pattern . Accordingly, it splits the implementation into three sub-components: (i) the resource itself that contains Package Repository state, (ii) the resource home that manages the resource, and (iii) the web service that gives access to components functionality.

The resource state contains information about the stored service archives. The resource home is used by the web service to create and find the resource.

The web service provides to the clients, e.g. HNM, access to the Package Repository functionality ( see below ).

This service is internally composed by a (i) Parser component and by a (ii) Grid access interface.

Parser component includes a Validation phase that checks service archives based on service profiles declarations. It assumes that service profiles are valid against the service profile schema. Controls performed includes checking that all declared files are in the right place into the service archive structure, that all service profiles includes at least a WSRF service, that the declared stubs library point to the corresponding package, etc. After this phase was completed PR create a tar gz file for each declared pakage that is provided on request to the HNM.

Grid Access inteface is in charge of store all PR data structures and packages into the Grid, as well as get from the Grid packages when requested. Package Repository stores all its data structures and packages into a local cache and into the Grid.

Functionality

The signatures of the actual implemented methods are presented below. These operations allow uploading, deleting, list, and getting a DILIGENT package. As well as other advanced operations that operate over packages dependencies.

1. store

store (ServiceIDURLDescription) -> String

This operation stores a service archive into the Package Repository and assigns status pending to it. Pending status means that service archive is valid, and unique ID was not previously used but cannot be deployed.

ServiceIDURLDescription type contains the unique ID assigned to the service, the URL from where the service archive can be downloaded, and the service description.

Return String contains unique ID if service archive is valid, null otherwise.

2. approve

approve (String) -> String

Approve an already stored service achive, if unique ID passed exists with pending status.

Return It returns approved if operation success, an error message otherwise.

3. delete

delete (String) -> String

This operation deletes a stored DILIGENT service archive from the Package Repository if unique ID passed exists. This implies the removal of all software packages from the Grid and from the local cache.

Return A message reporting success of failure is returned.

4. get

get (ServiceIDPackage) -> String

To identify an unique package on the repository two values must be provided: (i) DILIGENT unique ID and (ii) package name.

Return The method returns the URI from where the requested package can be downloaded.

5. getServiceProfile

getServiceProfile (String) -> String

Return From a given unique ID this method return the corresponding Service Profile if exists, null otherwise.

6. getServiceProfileID

getServiceProfileID (ClsssName) -> String

Return From a given class and name parameter this method return the corresponding DILIGENT unique ID if exists, null otherwise.

7. listPending

listPending ( ) -> String

Return This operation returns an XML containing all stored service archives with status pending. The XML structure is as follow:

<Resultset>
  <Result>
    <UniqueID>String</UniqueID>
    <Class>String</Class>
    <Name>String</Name>
    <PackageName>String</PackageName>
  </Result>
</Resultset>

8. listApproved

listApproved ( ) -> String

Return This operation returns an XML containing all stored service archives with status approved. The XML structure is as follow:

<Resultset>
  <Result>
    <UniqueID>String</UniqueID>
    <Class>String</Class>
    <Name>String</Name>
    <PackageName>String</PackageName>
  </Result>
</Resultset>

9. listNonDeployableServices

listNonDeployableServices ( ) -> String

With this functionality it is possible to know which stored packages are not deployable due missing dependencies.

Return a XML with the following structure:

<Resultset>
  <Result>
    <Class>String</Class>
    <Name>String</Name>
    <PackageName>String</PackageName>
  </Result>
</Resultset>

10. listAllDependenciesChain

listAllDependenciesChain (String) -> ListPackageArray

This operation lists all dependencies declared on the WSRF package declared on the service profile of a given unique ID.

Return Returned object type includes unique ID, package name, and type.

11. listSameDHNPackages

listSameDHNPackages (String) -> ListPackageArray

This operation lists all same DHN dependencies declared on the WSRF package declared on the service profile of a given unique ID.

Return Returned object type includes unique ID, package name, and type.

12. listSameDLPackages

listSameDLPackages (String) -> ListPackageArray

This operation lists all same DL dependencies declared on the WSRF package declared on the service profile of a given unique ID.

Return Returned object type includes unique ID, package name, and type.

13. listSameVOPackages

listSameVOPackages (String) -> ListPackageArray

This operation lists all same VO dependencies declared on the WSRF package declared on the service profile of a given unique ID.

Return Returned object type includes unique ID, package name, and type.

Dependencies

The dependencies of the Package Repository Service are:

• DHN: to host service.
• NAL: to access to HNM.
• Keeper common: for type definition.
• Grid Storage: to store DILIGENT packages on the Grid.
• Profile Manager: to parse Service Profile
• Commons-io from Jakarta: to File system functionality.
• Commons-compress from Jakarta: to File compress functionality.

Usage Example

Package Repository client

A Package Repository client is provided. The client uses all PR functionality in different ways, i.e. by calling directly the PR operations and by pre-processing the client input to generate the appropriate series of PR service calls.

Dependencies

• Package Repository service
• Keeper common

Usage

java ClientPackageRepository pkgRepositoryURI store diligentID serviceArchiveURI description

java ClientPackageRepository pkgRepositoryURI storeList storeListFile

java ClientPackageRepository pkgRepositoryURI storeAndApproveList storeListFile

java ClientPackageRepository pkgRepositoryURI approve diligentID

java ClientPackageRepository pkgRepositoryURI approveList storeListFile

java ClientPackageRepository pkgRepositoryURI get diligentID packageName

java ClientPackageRepository pkgRepositoryURI delete diligentID

java ClientPackageRepository pkgRepositoryURI getServProfile diligentID

java ClientPackageRepository pkgRepositoryURI getServProfileID class name

java ClientPackageRepository pkgRepositoryURI listPending

java ClientPackageRepository pkgRepositoryURI listApproved

java ClientPackageRepository pkgRepositoryURI listNonDeployable

java ClientPackageRepository pkgRepositoryURI listAllDependenciesChain serviceID

java ClientPackageRepository pkgRepositoryURI listSameDHNPackages serviceID

java ClientPackageRepository pkgRepositoryURI listSameDLPackages serviceID

java ClientPackageRepository pkgRepositoryURI listSameVOPackages serviceID

Where:

• pkgRepositoryURI: Package Repository address.
• diligentID: The unique ID of a Service Archive.
• serviceArchiveURI: The address from where the Service Archive can be downloaded.
• description: The Service Archive description.
• storeListFile: a list with Service Archives to store {diligentID, URI, description}.
• class: The Service Class.
• name: The Service Name.
• packageName: The package Name.

DILIGENT infrastructure initialization

In order to speed up the setting up of a DILIGENT infrastructure, and align available DILIGENT packages with current development process, an automated client that initialize the core DCL services (DIS and PR) is provided with the Package Repository component.

The procedure consists on a Thread that run the initialization method described below:

1. Download from ETICS list of last SA built
2. Delete from DIS all SPs
3. Delete from PR all packages
4. For each SA built
    4.1. create a new resource into the DIS
    4.2. if successful store SA on PR
            4.2.1. if successful log success
                        else remove resource from DIS and log failure
            else log failure

The Infrastructure Initialization Thread can be run by typing:

java org.diligentproject.keeperservice.packagerepositoryclients.InfrastructureInitialization  -initInfra

The same client can be used to download togheter with the SA the related Source Packages and Javadoc Packages, and create a local storage of source/bynaries/javadoc that can be stored on CVS/SVN repository. To store a SA on the PR:

		
java org.diligentproject.keeperservice.packagerepositoryclients.InfrastructureInitialization  -insert -module:<SERVICE_ARCHIVE_MODULE_NAME>

To update a previously stored SA on the PR ( and the related Service Profile on the DIS):

java org.diligentproject.keeperservice.packagerepositoryclients.InfrastructureInitialization  -update -module:<SERVICE_ARCHIVE_MODULE_NAME>
 -id:<SA_ID> 

To Remove a SA from the PR (and the related ServiceProfile from the DIS)

java org.diligentproject.keeperservice.packagerepositoryclients.InfrastructureInitialization  -delete -id:<SA_ID>

To Download a SA binary package togheter with the related source packages and javadoc packages:

java org.diligentproject.keeperservice.packagerepositoryclients.InfrastructureInitialization  -get -module:<SERVICE_ARCHIVE_MODULE_NAME> 

Dependencies

• activation.jar
• commons-compress-20061201.jar
• jaxb-api.jar
• jaxb-impl.jar
• jaxb-xjc.jar
• jaxb1-impl.jar
• jsr173_api.jar
• org_diligentproject_common_profilemanager.jar
• org_diligentproject_keeperservice_packagerepository_stubs.jar
• org_diligentproject_informationservice_disregistry_stubs.jar

Usage

Update appropriately the required parameters on the properties file

org/diligentproject/keeperservice/packagerepositoryclients/InfrastructureInitialization.properties

to refer to a completely deployed infrastructure:

• VERSION the gCube SA Version (i.e. 0.3.0)
• REPOSITORY_DIR the local dir where a SVN/CVS Repository is defined
• SLEEP_TIME: Thread sleep time in minutes
• TMP_DIR: directory where all Service Archives from build repository will be downloaded and parsed
• ETICS_SA_LIST_URL: build repository URI
• VO: The VO to Initialize
• PR_EPR: PR to update address
• SECURITY_ENABLED true/false if the security is enabled on the infra
• UPDATE_PR true/false if the Initialization thread has to update the PR
• UPDATE_DIS true/false if the Initialization thread has to update the DIS
• PROXY_FILE the local proxy file
• BASE_HTTP_ULR teh base HTTP Url where Etics SA are stored
• LOG4JFILE a custom log4j

Run the program from a machine with the CLASSPATH set appropriately.

Know Bugs and Limitations

In current implementation the proxy to access to the Grid need to be created manually.