Software Archive Specification

From Gcube Wiki
Revision as of 15:48, 2 September 2008 by Luca.frosini (Talk | contribs) (→‎Dependencies on libraries)

Jump to: navigation, search

Short background

A 'Service' in gCube is a software system that delivers functionalities and it is composed by a set of related 'Packages'.

In the gCube context, a 'Package' is a 'piece of software' that can be deployed in a GHN. Packages are single tarballs, compliant with the Package Model, that contain the files to be installed, along with rules describing software/packages dependencies, deployment instructions, etc.

Each service is described by a 'Profile' document, named 'Service Profile'.

For each Service Profile a corresponding 'Software Archive' should be delivered.

Software Archive Structure

A Software Archive is a single TAR GZ file, which contains all the files declared on the Service Profile and has the following structure: 


|-profile.xml
|
|
|--<PackageName>(Main)
|  |--<GARArchive>
|  |--svnpath.txt
|  |--<additional files>
|  |--doc
|     \--api
|        |- index.html
|        \- others javadoc files
|
|
|--<PackageName>
|  |--<LibraryFile>
|  |--svnpath.txt
|  |--<additional files>
|  |--doc
|     \--api
|        |-- index.html
|        \-- others javadoc files
|
|
|....
|
|
\--<PackageName>
   |--<LibraryFile>
   |--svnpath.txt
   |--<additional files>
   |--doc
      \--api
         |-- index.html
         \-- others javadoc files



/.....

where:

  • at the root level, the archive contains the Service Profile of the service itself. All the file names included in the archive have to be reported with a relative path starting from their <PackageName> folder (not included) *as defined on the Service Profile*.
  • for every Package declared inside the Service Profile, the archive contains a directory (called with the same name of the PackageName field of the profile) including the corresponding jar/gar file:
    • for a WSRF service the name reported into the GARArchive field;
    • for a library/stubs the name reported into the LibraryFile field.

and any other additional file declared in the service profile (e.g. installation or reboot scripts)

In Main packages the 'doc/api' folder with javadoc is mandatory, optional for the others packages.

The 'svnpath.txt' file is mandatory for each packages and contains the link to the svn source code.

Creating Software Archives

Manually

An easy way to create a Service Archive is to do it manually.

  1. Create a folder structure as described in previous section according to the corresponding Service Profile.
  2. Create a TAR GZ file of the created folder structure.


Using ETICS

Create a new component in ETICS for each service profile you already defined in order to create the related Service Archive as artefact. Let see now how you should do this in ETICS.

  1. name the component as <ServiceName>-servicearchive
  2. declare a dynamic dependency against each ETICS component that produces a package to be included in the Service Archive. In this way, the packages forming the whole service are available and they are built immediately before they are packaged. Link this configuration with the corresponding parent configuration.
  3. use the CVS Commands to download additional files related to the whole service, but not included in the packages (like the Service Profile)
  4. use the Build Commands available in ETICS in the following way:
INIT:
The init command has to be used to create the ${prefix} directory and all the <PackageName> subdirs

INSTALL:
The install command has to be used to move the GAR/JAR files and other additional files from their ${moduleDir} directory to the appropriate <PackageName> directory previously created.
Copy the javadoc to the appropriate <PackageName> directory.
Furthermore echo command with vcsroot link have to be redirected to a file 'svnpath.txt' in the appropriate <PackageName> directory.

PACKAGING:
Leave this command blank. At the end of the build process the default ETICS package command will create a
tar.gz (the ServiceArchive!) of the ${prefix} folder

To better understand the final result of the process above, you can have a look to the *-servicearchive components already created in ETICS.

Example

To better understand the example below check the Service Profile specification.

Service Profile Example

The following Profile describes a service and its stubs as two packages 

<?xml version="1.0" encoding="UTF-8"?>
<Resource xsi:noNamespaceSchemaLocation="service.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <ID></ID>
  <Type>Service</Type>
  <Profile>
    <Description>Description</Description>
    <Class>Class</Class>
    <Name>Name</Name>
    <Version>1.0.0</Version>
    <Configuration>
      <Static>
        <Configs>
          <Config>
            <File>aux.txt</File>
            <Description>aux</Description>
            <Label>aux</Label>
          </Config>
          <Config>
            <File>aux2.txt</File>
            <Description>aux2</Description>
            <Label>aux2</Label>
          </Config>
        </Configs>
      </Static>
      <Dynamic/>
    </Configuration>
    <Dependencies>
      <Dependency>
        <Class>DepClass</Class>
        <Name>DepName</Name>
        <Version>1.0.0</Version>
      </Dependency>
    </Dependencies>
    <Packages>
      <Main>
        <Description>Main Package Description</Description>
        <Name>MainPackage</Name>
        <Version>1.1.0</Version>
        <MultiVersion value="true"/>
        <Mandatory level="GHN"/>
        <Shareable level="VO"/>
        <InstallScripts>
          <File>maininstall.sh</File>
        </InstallScripts>
        <UninstallScripts>
          <File>mainuninstall.sh</File>
        </UninstallScripts>
        <RebootScripts>
          <File>mainreboot.sh</File>
        </RebootScripts>
        <Dependencies>
          <Dependency>
            <Service>
              <Class>MainDepClass</Class>
              <Name>MainDepName</Name>
              <Version>1.0.0</Version>						
            </Service>
            <Package>DepPackage</Package>
            <Version>1.0.0</Version>
            <Scope level="GHN"/>
            <Optional>true</Optional>
          </Dependency>
        </Dependencies>
        <SpecificData>text</SpecificData>
        <GARArchive>main.gar</GARArchive>
        <ServiceEquivalenceFunctions>...</ServiceEquivalenceFunctions>
        <PortType>...</PortType>
      </Main>
      <Software>
        <Description>Software Description</Description>
        <Name>SoftwareName</Name>
        <Version>1.0.0</Version>
        <MultiVersion value="true"/>
        <Mandatory level="GHN"/>
        <Shareable level="VO"/>
        <GHNRequirements>....</GHNRequirements>
        <InstallScripts>
          <File>softwareinstall.sh</File>
        </InstallScripts>
        <UninstallScripts>
          <File>softwareuninstall.sh</File>
        </UninstallScripts>
        <RebootScripts>
          <File>softwarereboot.sh</File>
        </RebootScripts>
        <Dependencies>
          <Dependency>
            <Service>
              <Class>SoftwareClass</Class>
              <Name>SoftwareName</Name>
              <Version>1.0.0</Version>
            </Service>
            <Package>SoftwarePackage</Package>
            <Version>1.0.0</Version>
            <Scope level="VO"/>
            <Optional>false</Optional>
          </Dependency>
        </Dependencies>
        <SpecificData>text</SpecificData>
        <Type>library</Type>
        <Files>
          <File>stubs.jar</File>
        </Files>
      </Software>
    </Packages>
    <SpecificData>text</SpecificData>
  </Profile>
</Resource>

This service has to produce as ServiceArchive a tar.gz file with the following structure: 

.
|-- MainPackage
|   |-- aux.txt
|   |-- aux2.txt
|   |-- main.gar
|   |-- maininstall.sh
|   |-- mainreboot.sh
|   |-- mainuninstall.sh
|   |-- svnpath.txt
|   |--doc
|     \--api
|        |--index.html
|        \--others javadoc files
|
|-- SoftwareName
|   |-- stubs.jar
|   |-- softwareinstall.sh
|   |-- softwarereboot.sh
|   |-- softwareuninstall.sh
|   \-- svnpath.txt
|
\-- profile.xml

Software Archive Validation Process

The Software Archive validation process relies on the information contained in the following sections of the Service Profile to validate the actual constituents of the Software Archive.

Description Tag

The description supplied in the store(StoreMessage) method has priority with respect to the one supplied with the profile. If no description is supplied with the method invocation the one filled in the profile will be used. If no one of this choice are used to supply the description the validation fails.

Class & Name Tag

During validation the Class and Name must be the same to the one passed as argument to store(StoreMessage) API.

Version Tag

This tag is always replaced with the version supplied with the store(StoreMessage) method. This choice have been thought to simplify the life of the developer to avoid to change everytime profile.

Configuration Section

<Configuration>
 <Static>
  <Configs>
   <Config>
    <File>aux.txt</File>
    <Description>aux</Description>
    <Label>aux</Label>
   </Config>
   <Config>
    <File>aux2.txt</File>
    <Description>aux2</Description>
    <Label>aux2</Label>
   </Config>
  </Configs>
 </Static>
 <Dynamic/>
</Configuration>



.
|-- MainPackage
|   |-- '''aux.txt'''
|   |-- '''aux2.txt'''
|   |-- main.gar
|   |-- maininstall.sh
|   |-- mainreboot.sh
|   |-- mainuninstall.sh
|   |-- svnpath.txt
|   |--doc
|     \--api
|        |--index.html
|        \--others javadoc files
|
|-- SoftwareName
|   |-- stubs.jar
|   |-- softwareinstall.sh
|   |-- softwarereboot.sh
|   |-- softwareuninstall.sh
|   \-- svnpath.txt
|
\-- profile.xml

This section will be moved soon on Main section. For the time being you have just to remind that the files declared in this section must go inside the directory with the name of the Main Package on Software Archive.

InstallScripts/UninstallScripts/RebootScripts Tag

<Main>
 ...
 <Name>MainPackage</Name>
 ...
 <InstallScripts>
  <File>maininstall.sh</File>
 </InstallScripts>
 <UninstallScripts>
  <File>mainuninstall.sh</File>
 </UninstallScripts>
 <RebootScripts>
  <File>mainreboot.sh</File>
 </RebootScripts>
 ...
</Main>



<Software>
 ...
 <Name>SoftwareName</Name>
 ...
 <InstallScripts>
  <File>softwareinstall.sh</File>
 </InstallScripts>
 <UninstallScripts>
  <File>softwareuninstall.sh</File>
 </UninstallScripts>
 <RebootScripts>
  <File>softwarereboot.sh</File>
 </RebootScripts>
 ...
</Software>



.
|-- MainPackage
|   |-- aux.txt
|   |-- aux2.txt
|   |-- main.gar
|   |-- '''maininstall.sh'''
|   |-- '''mainreboot.sh'''
|   `-- '''mainuninstall.sh'''
|   |-- svnpath.txt
|   |--doc
|     \--api
|        |--index.html
|        \--others javadoc files
|
|-- SoftwareName
|   |-- stubs.jar
|   |-- '''softwareinstall.sh'''
|   |-- '''softwarereboot.sh'''
|   `-- '''softwareuninstall.sh'''
|   \-- svnpath.txt
|
\-- profile.xml


Just remind the general rule for this section: The files must be go inside the directory with the name of the Package on Software Archive.

Dependencies Section

<Main>
...
<Name>MainPackage</Name>
...
<Dependencies>
  <Dependency>
   <Service>
    <Class>MainDepClass</Class>
    <Name>MainDepName</Name>
    <Version>1.0.0</Version>						
   </Service>
   <Package>DepPackage</Package>
   <Version>1.0.0</Version>
   <Scope level="GHN"/>
   <Optional>true</Optional>
  </Dependency>
 </Dependencies>
 ...
</Main>



<Software>
 ...
 <Name>SoftwareName</Name>
 ...
 <Dependencies>
  <Dependency>
   <Service>
    <Class>SoftwareClass</Class>
    <Name>SoftwareName</Name>
    <Version>1.0.0</Version>
   </Service>
   <Package>SoftwarePackage</Package>
   <Version>1.0.0</Version>
   <Scope level="VO"/>
   <Optional>false</Optional>
  </Dependency>
 </Dependencies>
 ...
</Software>

Each Dependency is in term of package. Each package is identified by the Service information and PackageName and PackageVersion. Instead of specifying the Package Version you can specify a range of version. The syntax for this can be find at [1] (please do not consider <qualifier> that is for internal use only).

Please consider this mapping between ETICS versioning [2] and MAVEN versioning:

ETICS version:

[major-number].[minor-number].[patch-number]-[age]

MAVEN version:

<major>.<minor>.<revision>-<build>


The version declaration is always in terms of xx.xx.xx The format xx.xx is not valid.


The Software Repository cannot check correctness of the dependencies since packages are not upload all at the same time. You must be very careful and make sure the dependency really exists. You must check the service name/class/version and the package name/version. Like in RPMs, your package is of no value if the dependency is incorrectly specified. You can check the correct values looking on latest ETICS release build in the file distribution.xml that can be found on top right of the page.


The Scope tag is used to specify the Scope Level for dependency deploying. Allowed value are GHN,VRE,VO. This information is used by the deployer.


The Optional tag is not mandatory and its absence corresponds to false value. This allow the developer to specify if a dependency can be potentially optional during the deployment phase. For instance, consider a service A designed to use service B when it is present or do by itself if service B is absent. In this case you can specify B as an Optional dependency.


Dependencies on libraries

For external libraries in D4Science you have to provide a Software Archive where the packages section contains only Software and NO Main. Preparing the Software Archive, you should follow the following convection:

Service Class: ExternalSoftware

Service Name: Library Name

Service Version: Library version

Package Name: Library Name

Package Version: Library Version


E.g: httpClient v.3.0.1

<?xml version="1.0" encoding="UTF-8"?> <Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> 

       <ID></ID>
       <Type>Service</Type>
       <Profile>
               <Description>Commons HTTP Client library</Description>
               <Class>ExternalSoftware</Class>
               <Name>commons-httpclient</Name>
               <Version>3.0.1</Version>
               <Packages>
                       <Software deployable="true">
                               <Description></Description>
                               <Name>commons-httpclient</Name>
                               <Version>3.0.1</Version>
                               <MultiVersion value="true"/>
                               <Mandatory level="VO"/>
                               <Shareable level="VO"/>
                               <Type>library</Type>
                               <Files>
                                       <File>

commons-httpclient-3.0.1.jar 

                                       </File>
                               </Files>
                       </Software>
               </Packages>
               <SpecificData>

</SpecificData> 

       </Profile>

</Resource>


As for D4Science services, the Software Archive has to be created by ETICS. for this reason you have to upload on svn the profile.xml with the library.


The same behaviour should be followed for D4Science library. In this case the Service Class should be the one of the area the library belong.

E.g: CMS library

Service Class: ContentManagement

Service Name: ContentManagement-Library

Service Version: xx.xx.xx

Package Name: ContentManagement-Library

Package Version: xx.xx.xx

Specific Main Package Sections

GARArchive Tag

<Main>
 ...
 <Name>MainPackage</Name>
 ...
 <GARArchive>main.gar</GARArchive>
 ...
<Main>

.
|-- MainPackage
|   |-- aux.txt
|   |-- aux2.txt
|   |-- '''main.gar'''
|   |-- maininstall.sh
|   |-- mainreboot.sh
|   |-- mainuninstall.sh
|   |-- svnpath.txt
|   |--doc
|     \--api
|        |--index.html
|        \--others javadoc files
|
|-- SoftwareName
|   |-- stubs.jar
|   |-- softwareinstall.sh
|   |-- softwarereboot.sh
|   |-- softwareuninstall.sh
|   \-- svnpath.txt
|
\-- profile.xml

The gar archive must go under the Main Package folder on Software Archive

Specific Software Package Sections

Files Section

<Software>
 ...
 <Name>SoftwareName</Name>
 ...
 <Files>
  <File>stubs.jar</File>
 </Files>
 ...
</Software>

.
|-- MainPackage
|   |-- aux.txt
|   |-- aux2.txt
|   |-- main.gar
|   |-- maininstall.sh
|   |-- mainreboot.sh
|   |-- mainuninstall.sh
|   |-- svnpath.txt
|   |--doc
|     \--api
|        |--index.html
|        \--others javadoc files
|
|-- SoftwareName
|   |-- '''stubs.jar'''
|   |-- softwareinstall.sh
|   |-- softwarereboot.sh
|   |-- softwareuninstall.sh
|   \-- svnpath.txt
|
\-- profile.xml

The files must go under the folder with the name of the Software Package on Software Archive

URI Tag

URI tag can be used instead of a Files tag whenever the actual content is stored remotely. This case is not showed in the example above. 

<URI>http://dlib25.isti.cnr.it/maven/Service3.tar.gz<URI>

During the validation phase Software Repository tries to download the file. If it is not downloadable the validation fails. If you specify this tag you have to make sure this file will be available for a long period in the future. If not it would be better to include it directly on the Software Archive

Example

The following Profile describes a service and its stubs as two packages 

<?xml version="1.0" encoding="UTF-8"?>
<Resource>
    <ID>gfgdf80-131b-11db-90fgf98-dgfgfgf66ed3f</ID>
    <Type>Service</Type>
    <Profile>
        <Class>InformationSystem</Class>
        <Name>IS-Registry</Name>
        <Version>1.0.0</Version>
        <Packages>
            <Main>
		<Name>IS-Registry-service</Name>
		<Version>1.0.0</Version>
		<Mandatory level="VO"/>
		<Shareable level="VO"/>
		<GHNRequirements>
			<Requirement category="Site" requirement="string" value="java1.5" operator="ge"/>
		</GHNRequirements>
		<Dependencies>
			<Dependency>
				<Service>
				   <Class>InformationSystem</Class>
		 		   <Name>IS-Registry</Name>
		 		   <Version>1.0.0</Version>
				</Service>
				<Package>IS-Registry-stubs</Package>
				<Version>1.0.0</Version>
				<Scope level="GHN"/>
				<Optional>false</Optional>
			</Dependency>
		</Dependencies>
		<GARArchive>org.gcube.informationsystem.registry.gar</GARArchive>
            	<PortType>
                	<Name>gcube/informationsystem/registry/registryFactory</Name>
                    	<Security/>			
			<WSDL/>
            	</PortType>
            	<PortType>
                	<Name>gcube/informationsystem/registry/registry</Name>
                    	<Security/>			
			<WSDL/>
            	</PortType>
            </Main>
            <Software>
            	<Name>IS-Registry-stubs</Name>
		<Version>1.0.0</Version>
		<MultiVersion value="true"/>
		<Mandatory level="GHN"/>
		<Shareable level="VO"/>
		<GHNRequirements>
			<Requirement category="Site" operator="ge" requirement="string" value="java1.5"/>
		</GHNRequirements>
		<SpecificData/>
		<Type>library</Type>
		<Files>
			<File>org.gcube.informationsystem.registry.stubs.jar</File>
		</Files>
	   </Software>           
        </Packages>
    </Profile>
</Resource>

This service has to produce as ServiceArchive a tar.gz file with the following structure: 

|--profile.xml
|
|
|--IS-Registry-service
|  |--org.gcube.informationsystem.registry.gar
|  |--svnpath.txt
|  \--doc
|     \--api
|        |--index.html
|        \--others javadoc files
|
|
\--IS-Registry-stubs
   |--org.gcube.informationsystem.registry.stubs.jar
   \--svnpath.txt


For this service archive the ETICS configuration is 

INIT: 
mkdir -p ${prefix};
mkdir -p ${prefix}/IS-Registry-service;
mkdir -p ${prefix}/IS-Registry-stubs;


INSTALL:
cp ${org.gcube.information-system.registry.moduleDir}/etc/profile.xml ${prefix};
cp ${org.gcube.information-system.registry-stubs.moduleDir}/lib/org.gcube.informationsystem.registry.stubs.jar ${prefix}/IS-Registry-stubs;
cp ${org.gcube.information-system.registry.moduleDir}/lib/*.gar ${prefix}/IS-Registry-service;
cp -r ${org.gcube.information-system.registry.moduleDir}/doc ${prefix}/IS-Registry-service;
echo ${org.gcube.information-system.registry.vcsroot}/${org.gcube.information-system.registry.tag} > ${prefix}/IS-Registry-service/svnpath.txt;
echo ${org.gcube.information-system.registry-stubs.vcsroot}/${org.gcube.information-system.registry-stubs.tag} > ${prefix}/IS-Registry-stubs/svnpath.txt;
Retrieved from "https://wiki.gcube-system.org/index.php?title=Software_Archive_Specification&oldid=4341"