Difference between revisions of "SAM Tools"

Latest revision as of 15:23, 8 April 2011

SAM : Service Availability Monitoring

The goal of the service availability monitoring is to provide a reliable tool to continuously verify the status of the infrastructure service (i.e. if the infrastructure functionality works as expected). This tool is based on gCube Executor service and on tailored plug-ins that verify concrete infrastructure functionality.

SAM Plugins

Being gCube a SOA infrastructure, the natural way to perform active tests on the production services was to implement a new Service responsible for the availability testing. The service was going to be designed as highly configurable with the possibility to dynamic inject and configure new tests. But looking on what was already available on the gCube framework, we decided to exploit the plugin mechanism offered by the Executor Service.

Concerning the plug-ins, these can be seen as specific tests focused on concrete functional areas of the infrastructure. They recreate a number of invocations to a set of services that provide a given functionality. These invocations represent the standard usage of the infrastructure can be executed periodically. Each plug-in invocation implies the execution of a number of tests, whose results are stored on a remote DB using the facilities offered by the SAM Reports library. The following plug-ins are planned to be implemented:

Search tester

The Search tester plugin has already been released as part of gCube 1.8. The Plugin is able to test the gCube Search Framework as exposed by the ASL HTTP API. This means that Search invocations are not directly performed to gCube Search Service, but the ASL framework Search HTTP API are invoked instead.

The ASL HTTP API gives also the possibility to gather information on gCube Collection and MCollection for each scope and extract the related schema,language, browsable, sortable and searchable fields, which are used as input for the tests:

• Quick Search test
• Combined Search test
• Browse test
• Simple Search Test

Configuration

In order to access the information on VREs, Collection and Metadata Collection and to perform operation trough the ASLHTTP API, the ASLHTTP need to be configured to allow anonymous access. In order to configure anonymous access the file anonymous_access.config need to be placed inside $GLOBUS_LOCATION/shared/d4s folder, with the following content:

ShowCollectionInfos = true
ShowContentInfos = true
GetContent = false
GetMetadata = true
ShowCollections = true
Search = true
GetThumbnails = false
ShowVREs = true

Plugin Parameters

The plugin accepts the following input parameters:

• HTTPAPIURL: the HTTP API instance to contact
• tests : the list of tests to run
• vres : the list of VREs where run the test (in the context of the Executor service that runs at VO level)
• interval : the scheduling interval of the tests. ( 0 means run only once)

Task Manager Json Input

The plugin can be executed trough the TASK Manager portlet.

Here is an example of the task inputs in JSON format:

{
"HTTPAPIURL":"http://portal.d4science.research-infrastructures.eu/applicationSupportLayerHttp/",
"interval":"86400",
"vres":[ "/d4science.research-infrastructures.eu/Arts-Humanities/gMan" ],
"tests":[ "org.gcube.tools.searchtester.plugin.tests.QuickSearchTest","org.gcube.tools.searchtester.plugin.tests.SimpleSearchTest","org.gcube.tools.searchtester.plugin.tests.CombinedSearchTest","org.gcube.tools.searchtester.plugin.tests.BrowseTest" ]
}

IS tester

The IS Tester plugin is the responsible of the testing of the Information System deployed in the VO where the Executor Service is acting. As any other information system, the gCube IS exposes interfaces for Information publication and querying. The two operations are then implemented by 2 different services behind ( IS-Registry and IS-Collector respectively). The gCube infrastructure hides services deployment details ( distribution and replication) by offering libraries to client for publication and querying ( IS-Publisher and IS-Client). Since the scope of the availability test is to check the status of any instance of IS services running on a given VO, the IS tester plugin contacts directly every possible instance of the IS services running the following tests:

• GCUBE Profile publication/removal/update
• Query test ( sequential/parallel)
• WS-resource publication/removal
• GCUBE Notification test

Plugin parameters

• scope: the scope to use to perform the tests
• tests: the list of test to run
• interval : the scheduling interval of the tests. ( 0 means run only once)

Task Manager Json Input

The plugin can be executed trough the TASK Manager portlet.

Here is an example of the task inputs in JSON format:

{
"scope":"/d4science.research-infrastructures.eu/Arts-Humanities",
"interval":"86400",
"tests":[ "org.gcube.tools.sam.istester.plugin.tests.ProfileOperationTest","org.gcube.tools.sam.istester.plugin.tests.QueryOperationTest"]
}

Index Lookup tester

The gCube Search functionality is based ( apart from gCube Search services) also on the availability of gCube Index Lookup Services. gCube Index Lookups are splitted into 3 different implementations:

• Full Text Index Lookups ( responsible for full text search )
• Fwd Index Lookups ( responsible for collection browsing )
• Geo Index Lookups ( responsible for Geo referential search )

The Index Lookup tester is responsible for the availability testing of all of those class of services. It relies on gCube IS to understand what are the available collections and the corresponding Indexes to be tested.

The tests performed are :

• Lookup Index availability ( checks for each collection if at least one Lookup Index is available)
• Lookup Index Query performance

CM tester

The new CM service ( still under integration within gCube) is the new service ( replacement of the Content Managemnt stack of services currently integrated in gCube) which offers homogeneous access to content stored in arbitrarily located and typed repositories. It is designed to adapt multiple back types to the gDoc access type [1] using an open architecture of type-specific plugins. The available plugins implemented so far are:

• SMS Plugin
• OAI-PMH Plugin

The CM Tester is responsible for the availability testing of the CM service, which means performing I/O tests over the available CM instances in the D4science infrastructure:

• Read operations
  • For each collection in the system both single/bulk read operations are tested
  • For each operation the related performance is calculated
• Write operations
  • a collection of document is created and single/bulk documents imports are tested
  • For each operation the related performance is calculated

SAM Reports Library

The SAM Reports libraries hides SAM plugin and SAM portlet from the test report storage facilities offered by the SAM Infrastructure. The current implementation makes use of the Hibernate framework to store and query for test reports. The latest javadoc can be found at TODO Actually the type of reports modeled by the SAM reports Library are:

• BrowseReport
• CombinedSearchReport
• QuickSearchReport
• SimpleSearchReport

all of them filled by SearchTester plugin. New types of Reports will be modeled to follow the the IS tester and Index Lookup tester plugins needs

SAM Portlet

Deployment