Difference between revisions of "Functional Testing"

From Gcube Wiki
Jump to: navigation, search
(→‎Functional Test (FT) Procedure)
(→‎Software Testing Plan)
 
(136 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
The objective of the Functional Testing activity in gCube is to ensure that all the components:
+
Go to [https://wiki.gcube-system.org/gcube/Test_Plan Test_Plan] section.
−
* gCube services and public interfaces, are conform to their specification
+
<br>
 +
= Acronyms List =
 +
 
 +
{| border="0" cellpadding="4" cellspacing="0"
 +
|-
 +
| '''CRT''' || Component Release Ticket
 +
|-
 +
| '''Dev''' || [[Role Developer|Developer]] role
 +
|-
 +
| '''EPC''' || [[ETICS|ETICS]] Project Configuration (e.g. org.gcube.1-6-0)
 +
|-
 +
| '''ESC''' || [[ETICS|ETICS]] Subsystem Configuration (e.g. org.gcube.annotation-management.1-1-0)
 +
|-
 +
| '''ECC''' || [[ETICS|ETICS]] Component Configuration (e.g. org.gcube.annotation-management.abe.1-1-0)
 +
|-
 +
| '''PRT''' || Project Release Ticket
 +
|-
 +
| '''RMan''' || [[Role Release Manager|Release Manager]] role
 +
|-
 +
| '''SMan''' || [[Role Subsystem Manager|Subsystem Manager]] role
 +
|-
 +
| '''SRT''' || Subsystem Release Ticket
 +
|-
 +
| '''SW Test''' || Software Test Ticket
 +
|-
 +
| '''TTeam''' || the set of [[Role Tester|Testers]]
 +
|}
 +
 
 +
= Introduction =
 +
<BR>
 +
The objective of the Functional Testing activity in gCube is to ensure that all the components gCube services and public interfaces:
 +
* are conform to their specification
 
* are free from erroneous or malfunction
 
* are free from erroneous or malfunction
 
* can be run in the defined system requirement
 
* can be run in the defined system requirement
 
* are free from code anomalies and coding errors
 
* are free from code anomalies and coding errors
  
−
The goal of functional testing is to assure the appropriate quality of the gCube software. Testing, in general, is not a procedure to find bugs, but to check that the software works in accordance with the specification, architectural and detailed design documents. In addition the testing activity has to ensure that the software is easy to use and gives appropriate response in the case of wrong usage or defects that cannot entirely be eliminated. Testing is planned according to these goals.
+
The goal of functional testing is to assure the appropriate quality of the gCube software. Testing, in general, is not a procedure to find bugs, but to check that the software works in accordance with the specification, architectural and detailed design documents. In addition, the testing activity has to ensure that the software is easy to use and gives appropriate response in the case of wrong usage or defects that cannot entirely be eliminated. Testing is planned according to these goals.
  
 
The gCube software is composed by three types of components: services, libraries, and portlets. While deployment of services and libraries are tested with the [[Deployment Testing|Deployment tests]], the functionality of portlets is tested through a specific procedure. Since portlets rely on services and libraries to perform their functions, testing the portlets functionalities implicitly tests also the functionalities of services and libraries.
 
The gCube software is composed by three types of components: services, libraries, and portlets. While deployment of services and libraries are tested with the [[Deployment Testing|Deployment tests]], the functionality of portlets is tested through a specific procedure. Since portlets rely on services and libraries to perform their functions, testing the portlets functionalities implicitly tests also the functionalities of services and libraries.
  
−
= Portlet Functional Testing =
+
= Portlet Functional Testing =
−
The Portlets composing gCube have been required to pass a functional testing procedure in order to be made available for end users. The actors involved in the procedure are: the ''portal manager'', the ''infrastructure manager'', and the ''portlet application domain experts''. Portlet application domain experts are expert users of a specific application. The domain expert knows very well all the possible interactions between the end user and the user interface. In some specific cases, they could be application developers or application owners.
+
The Portlets composing gCube have been required to pass a functional testing procedure in order to be made available for end users. The actors involved in the procedure are: the ''portal manager'', the ''infrastructure manager'', and the ''owner'' (generally , portlet application domain experts or
 +
the developer of the Portlet or the responsible of the application) . The owners (portlet application domain experts) are expert users of a specific application. The owner (domain expert) knows very well all the possible interactions between the end user and the user interface. In some specific cases, they could be application developers or application owners.
  
 
The procedure starts by analysing the software artefacts included in the release along with their software dependencies. This task is carried out manually by the portal manager. Each portlet artefact which is released must be part of the functional testing procedure. Also, if anything in the dependency chain of a Portlet X is released, then X has to be functionally tested too.
 
The procedure starts by analysing the software artefacts included in the release along with their software dependencies. This task is carried out manually by the portal manager. Each portlet artefact which is released must be part of the functional testing procedure. Also, if anything in the dependency chain of a Portlet X is released, then X has to be functionally tested too.
Line 18: Line 50:
 
# '''Back-end Service check''':  The portal manager and the infrastructure manager make sure that the back-end services of the Portlet application are ready and fully functional in the testing / pre-production infrastructure.
 
# '''Back-end Service check''':  The portal manager and the infrastructure manager make sure that the back-end services of the Portlet application are ready and fully functional in the testing / pre-production infrastructure.
 
# '''Rendering check''': every Portlet web archive passing stage no. 1 is then actually deployed into the pre-production portal to check if it renders correctly. In this stage there are few things that could make the Portlet not render correctly. For instance, the GWT compilation could fail during Web Archive creation, or the developer could have used some CSS classes that clash with those of the iMarine Gateway. If the check is not passed the Portlet developer is notified through a new ticket of type integration issue;
 
# '''Rendering check''': every Portlet web archive passing stage no. 1 is then actually deployed into the pre-production portal to check if it renders correctly. In this stage there are few things that could make the Portlet not render correctly. For instance, the GWT compilation could fail during Web Archive creation, or the developer could have used some CSS classes that clash with those of the iMarine Gateway. If the check is not passed the Portlet developer is notified through a new ticket of type integration issue;
−
# '''Functional Test''': when stage 1 and 2 and 3 are completed the functional test is assigned to the testing team which performs it by following the Software Testing Plan. An online table '''[[Functional Test Master Table Template|Functional Test Master Table Template]]''' is created by the portal manager and shared with each actor involved in the functional testing procedure. Each row contains a Portlet artefact having to pass the functional testing procedure[[https://wiki.gcube-system.org/index.php?title=Functional_Testing&action=edit&section=1 Functional Test (FT) Procedure]].  Below the description of the table's fields:
+
# '''Functional Test''': When stage 1 and 2 and 3 are completed the functional test is assigned to the testing team which performs it by following the Software Testing Plan. An updating of the Functional test procedure is discussed to use the the Redmine tool to track the functional testing activities and dismiss the [https://wiki.gcube-system.org/gcube/Functional_Test_Master_Table_Template Functional Master Table]. The new Software Test procedure is provided in the [[Functional Testing#Functional Test (FT) Procedure |Functional Test (FT) Procedure]] section.
−
#* Component Name: Portlet unique identifier;
+
−
#* Owner: generally the developer of the Portlet or the responsible of the application;
+
−
#* Tester Name: name of the tester;
+
−
#* Scope: the infrastructure scope where the Portlet has to be functionally tested;
+
−
#* Web Archive validity: indicates if the software package produced by the build system can be correctly deployed in the iMarine Gateway. The portal manager compiles this part;
+
−
#* Renders OK: indicates whether the Portlet displays correctly in the iMarine Gateway. The portal manager compiles this part;
+
−
#* Service Deployed: indicates whether the infrastructure services composing the back-end of the Portlet application are ready and fully functional. The infrastructure manager compiles this part;
+
−
#* Functional Test: compiled by the tester user, indicates if the functional test was performed;
+
−
#* Link Test Plan: the [https://goo.gl/kQGSf0 general XLS portlet test plan] link;
+
−
#* Ticket: the [https://support.d4science.org/projects/gcube Redmine ticket] associated to the functional test.
+
  
−
 
+
=== Software Testing Plan ===
−
 
+
The Software Testing Plan [https://support.d4science.org/issues/1413# #1413] is created, from the Portal Manager, in the [https://i-marine.d4science.org/group/data-e-infrastructure-gateway/workspace VRE Folders > gCube > Portlet Testing Plans ] containing two folders: '''Material''' and '''Releases'''.
−
== Functional Test (FT) Procedure ==
+
−
<br>
+
−
The Software Testing Plan [https://support.d4science.org/issues/1413#change-6072 #1413] is created in the [https://i-marine.d4science.org/group/data-e-infrastructure-gateway/workspace BlueCommons VRE Folder] containing two folders: '''Material''' and '''Releases'''.
+
 
<br>
 
<br>
−
'''Material''' will contain:
+
The '''Material''' directory provides the following information:
−
#the [https://goo.gl/kQGSf0 general XLS template] to be instantiated by each portlet developer for compiling the Testing Plan and
+
#the [https://goo.gl/BDjVeA general XLS template] to be instantiated by each portlet developer for compiling the Testing Plan and
 
#the portlet folder (created by '''Portal Manager''') for each portlet to be functionality tested. The syntax used to create the portlet folder
 
#the portlet folder (created by '''Portal Manager''') for each portlet to be functionality tested. The syntax used to create the portlet folder
−
will be: '''''$portlet_name_folder'''''. Every developer will create the actual testing plan into '''''Material/$portlet_name_folder''''' including by adding additional files required for the test (e.g. cvs_files). To describe the functional tests, the developer will use the [https://goo.gl/kQGSf0 Portlet Testing Plan Template]; this file will be pasted and completed into the Release folders by the testers.
+
will be: '''''$portlet_name_folder'''''. Every developer will create the actual testing plan into '''''Material/$portlet_name_folder''''' including by adding additional files required for the test (e.g. cvs_files). To describe the functional tests, the developer will use the [https://goo.gl/BDjVeA Portlet Testing Plan Template]; this file will be pasted and completed into the Release folders by the testers.
 
A Task ([https://support.d4science.org/projects/gcube Redmine Ticket]) will be assigned, by the '''Release Manager''', to each portlet developer.
 
A Task ([https://support.d4science.org/projects/gcube Redmine Ticket]) will be assigned, by the '''Release Manager''', to each portlet developer.
 
<br>
 
<br>
−
'''Releases''' will contain a folder for each gCube release containing the tests to be executed.
+
The '''Releases''' directory contains a folder for each gCube release containing the tests to be executed.
 +
The Portal Manager creates (in this directory) the gCube release directory and the portlets directories related.
 
<br>
 
<br>
−
FT procedure's steps are describe below:
+
 
−
*The '''pre-production infrastructure''', [https://pre.d4science.org hosted at CNR], will be used for the FT testing.
+
== Functional Test (FT) Procedure ==
−
*The Release Manager will ask to '''every partners''' to suggest the persons that will form the testing team.
+
The Software Testing Plan is created in the BlueCommons VRE Folder containing two folders: Material and Releases.
−
*The following '''recommendations''' should be '''mandatory''':
+
*'''Material''' provides:
−
#The '''effort''' should be based not per application but per '''number of tests to be executed''' [https://support.d4science.org/issues/1413#change-6072].  
+
**the general XLS template to be instantiated by each portlet developer for compiling the Testing Plan and
−
#The tester should '''not''' be the '''same owner of the portlet'''[https://support.d4science.org/issues/1413#change-6072].
+
**the portlet folder (created by Portal Manager) for each portlet to be functionality tested. The syntax used to create the portlet folder will be '''$portlet_name_folder'''.
−
#For every release a wiki page will be created: '''$functional_test_master_table_Org_gCube_#Release'''(e.g. [https://wiki.gcube-system.org/gcube/FunctionalTestMasterTablegCube3.10.0 Functional Test Master Table gCube Release 3.10.0] or [https://wiki.gcube-system.org/gcube/FTMasterTablegCube4.0.0 Functional Test Master Table gCube Release 4.0.0]). The '''[[Functional Test Master Table Template|Functional Test Master Table Template]]''' will be fill by the testers with the FT results for the portlets applications in the pre-production infrastructure [https://wiki.gcube-system.org/index.php?title=Functional_Testing&action=edit&section=4]
+
**Every developer will create the actual testing plan into '''Material/$portlet_name_folder''' including by adding additional files required for the test (e.g. cvs_files).  A Task (Redmine Ticket) will be assigned, by the Release Manager,to each portlet developer.  
 +
*'''Releases''' provides a folder for each gCube release containing the tests to be executed.
 +
 
 +
The Release Manager will ask to every partners to suggest the persons that will form the testing team.
 +
 
 +
==Mandatory Recommendations and Rules==
 +
*The effort should be based not per application but per number of tests to be executed .
 +
*The tester should '''NOT''' be the '''same owner of the portlet'''.
 +
*The functional tests will be executed using the Portlet Testing Plan Template.  
 +
*The Portlets applications are executed in the [https://pre.d4science.org/home pre-production infrastructure].
 +
* All testers use Redmine Ticket to track their testing activities.
 +
*If  a new portlets or tester is added, the Developer or Subsystem Manager must communicate it by email to the Test Manager, Infrastructure Manager, Deployer Manager and Subsystem Manager.
 
<br>
 
<br>
 +
 +
== SoftWare Test Flow ==
 +
The Tester Manager creates the Master and the SW Test Ticket component (the procedure is inspired to the [https://wiki.gcube-system.org/gcube/Major/Minor_Release_Cycle_procedure#Release_Preparation Configuration Release Ticket]).
 +
<br>
 +
The fields to fill for the SW Test ticket are: 
 +
*Project: gCube 
 +
*Tracker: SW Test (software test) 
 +
*Status : 
 +
*#'''New''' :when is created by Tester Manager;
 +
*#'''Available''' : the  Deployer Manager change the status from New to Available when the portlets service is deployed in the pre-production.The tester know can start with the test; 
 +
*#'''Under Testing''': the Tester change the status from Available to Under Testing when start with the test and change the %Done field to track the progress of the test;
 +
*#'''Test Issue''':  when the portlets not pass the test,  the Tester change the status from Under Testing to Test Issue and assign the ticket to the developer. When the issue is solved the developer changes the status from Test Issue to Available so the Tester can re-started with the test;
 +
*#'''Tested on Pre-prod''' : Test is passed. The Tester changes the status from Under Testing to Tested on Pre-prod. And add as watcher Tester Manager. In this way the Tester Manager (if exist the release ticket) changes the status from Deployed on Pre-prod to Tester-on-preprod. 
 +
*#'''Assignee''': the SW test ticket will assigned always to the Testers except when the Status of the portlets is Test Issue (in this  case the ticket is assigned to the  developer to solve the issue). 
 +
*#'''Description''': the Tester Manager adds the "VRE Workspace link" of the Test Plan (if exist) . 
 +
*#'''%Done''': the tester (or developer) should use it to track the work in progress. 
 +
*#'''Watcher''': Tester Manager, Infrastructure Manager or Deployer Manager. 
 +
 +
The flow is showed in the following figure:
 +
 +
[[File:SWTestWorkflow.png| center]]
 +
 +
The SW Test Portlet Tickets will be created copying the Master Ticket (it's considered the template of the SW Test Portlet). The Master Ticket and its sub-tickets can be used as template with Progress field at "0" and Status field at "New" to create the other tickets.
 +
 +
For the testing of each release the Tester Manager copies from that template. In this way, only the sprint is changed (in one action thanks to the multi-selection).
 +
 +
The SW Test Portlet are available after the services are deployed on pre-production and after the Deployer Manager following the procedure as reported in the ([https://wiki.gcube-system.org/gcube/Major/Minor_Release_Cycle_procedure#Release_Integration Releases Integration]).
 +
 +
The SoftWare Test ticket will be opened by the Tester or Portal Manager and assigned to the testers (the procedure is similar as for the release tickets).
 +
 +
#The TesterManager creates the ticket: <code>{Project: gCube; Tracker: SW Test;Subject: name of the portlet to test with release version;Assignee: name of Tester;Status: New;Description: Test Plan URL;%Done : 0;Watcher: Developer, Subsystem Manager, Deployer Manager,Infrastructure or Portal Manager;Reference : of the Release Ticket , if exist.}</code>.
 +
#After the deployement of the service or portlet , the Deployer Manager updates: the SW Test ticket <code>{status: New -> Available; Assignee:Tester}</code>  ( and the related Release Ticket from <code>{status:Under Integration to Deployed on Pre-prod}</code> ). The Tester knows that he/she can start with the test.
 +
#The Tester can start with the test updating the SW Test status: <code>{status: Available -> Under Testing}</code>.
 +
#If an issue occurred during the test , the Tester changes the SW Test Ticket status<code>{status: Under Testing -> Test Issue; Assignee:Developer responsible of the portlet}</code>.
 +
#When the issue is solved , the developer updates the Software Test Ticket <code>{status:Test Issue ->New; Assignee: Tester}</code> if the portlet must be deployed otherwise <code>{status: Test issue -> Available}</code>. The Tester repeats the test.
 +
#If the test passes the Tester changes the status from <code>{status: Under Testing -> Tested on pre-prod}</code> and copies the TestPlan, with the results of the test, in the [https://goo.gl/p5o75b Releases] directory.For example: the tester tests the invites-friends.1-1-0 portlets. The Tester copies the TestPlan for this portlets into Releases/org.gcube.4.0.0/invites-friends.1-1-0.
 +
#When all portlet are in status Tested on pre-prod , the Tester Manager can be changed the status of the Release ticket from <code>{status: Deployed on pre-prod -> Tested on pre-prod}</code>.
 +
 +
<pre>
 +
NOTE:
 +
- Test issue  = The test is fails.
 +
- Tested on Preprod : indicated that the developer and/or the portal manager or the preprod infrastructure manager
 +
  are confident that the component behave as expected and can be released as is.
 +
</pre>
 +
 +
The '''pre-production infrastructure''' is provided for the functional testing [https://pre.d4science.org hosted at CNR].
 +
 +
= Test Log=
 +
The information and statistics for the Functional Test Portlet (for all release involved in it) is reported in the: [https://wiki.gcube-system.org/gcube/TestLog Test Log] section.
  
 
= Roles =
 
= Roles =
  
 
The functionality testing activity foresees the involvement of the following roles:
 
The functionality testing activity foresees the involvement of the following roles:
 +
* Tester
 +
* Developer
  
−
* [[Role Tester|Testers]]:
+
== Tester  ==
−
** responsible for the execution of the functionality tests and to submit functionality test bugs
+
−
** responsible to fill the dedicated [https://goo.gl/kQGSf0 Portlet Testing Plan Template] and the [[Functional Test Master Table Template|Functional Test Master Table Template]] as described in the [https://wiki.gcube-system.org/index.php?title=Functional_Testing&action=edit&section=2 previous section].
+
−
* [[Role Developer|Developers]]:
+
−
** responsible for fixing functionality test bugs
+
−
** carried out by JRA members
+
−
** responsible to fill the dedicated [https://goo.gl/kQGSf0 Portlet Testing Plan Template] as described in the [https://wiki.gcube-system.org/index.php?title=Functional_Testing&action=edit&section=2 previous section].
+
  
−
= Infrastructure =
 
  
−
The execution of functionality tests requires a gCube testing infrastructure fully functional.  
+
The TesterManager is responsible:
−
A detailed description of the testing infrastructure can be found [[Testing Infrastructure|Testing Infrastructure]]
+
* to create the new Software Test Ticket (master ticket and its sub-tickets);
 +
* to create the query;
 +
* to clone the master ticket acting as template for the portlet testing.
 +
 
 +
The Tester is responsible:
 +
* for the execution of the functionality tests and to submit functionality test bugs
 +
* to fill the dedicated [https://goo.gl/p5o75b Portlet Testing Plan Template] and the creation of the SW Test ticket.
 +
* to track the Test Issue through the use of the track system issue tool (ref. D4Science support).
 +
<br>
 +
The details about the assignment of the testers (for each partners) are hosted on the [[Role Tester|Role Assignment]] paragraph.
 +
<br>
 +
 
 +
==Developer  ==
 +
The details about the role of the developers are hosted on  [[Role Developer|Developers]] wiki page.
 +
The developer is:
 +
* responsible for fixing (functionality and no) test bugs
 +
* carried out by JRA members
 +
* responsible to fill the dedicated [https://goo.gl/BDjVeA Portlet Testing Plan Template] as described in the [[Functional Testing#Functional Test (FT) Procedure| previous section]].
 +
 
 +
= Infrastructure =
 +
The execution of functionality tests requires a gCube testing infrastructure fully functional. A detailed description of the testing infrastructure can be found [[Testing Infrastructure|Testing Infrastructure]]

Latest revision as of 10:17, 1 March 2018

Go to Test_Plan section.

Acronyms List

CRT Component Release Ticket
Dev Developer role
EPC ETICS Project Configuration (e.g. org.gcube.1-6-0)
ESC ETICS Subsystem Configuration (e.g. org.gcube.annotation-management.1-1-0)
ECC ETICS Component Configuration (e.g. org.gcube.annotation-management.abe.1-1-0)
PRT Project Release Ticket
RMan Release Manager role
SMan Subsystem Manager role
SRT Subsystem Release Ticket
SW Test Software Test Ticket
TTeam the set of Testers

Introduction


The objective of the Functional Testing activity in gCube is to ensure that all the components gCube services and public interfaces:

  • are conform to their specification
  • are free from erroneous or malfunction
  • can be run in the defined system requirement
  • are free from code anomalies and coding errors

The goal of functional testing is to assure the appropriate quality of the gCube software. Testing, in general, is not a procedure to find bugs, but to check that the software works in accordance with the specification, architectural and detailed design documents. In addition, the testing activity has to ensure that the software is easy to use and gives appropriate response in the case of wrong usage or defects that cannot entirely be eliminated. Testing is planned according to these goals.

The gCube software is composed by three types of components: services, libraries, and portlets. While deployment of services and libraries are tested with the Deployment tests, the functionality of portlets is tested through a specific procedure. Since portlets rely on services and libraries to perform their functions, testing the portlets functionalities implicitly tests also the functionalities of services and libraries.

Portlet Functional Testing

The Portlets composing gCube have been required to pass a functional testing procedure in order to be made available for end users. The actors involved in the procedure are: the portal manager, the infrastructure manager, and the owner (generally , portlet application domain experts or the developer of the Portlet or the responsible of the application) . The owners (portlet application domain experts) are expert users of a specific application. The owner (domain expert) knows very well all the possible interactions between the end user and the user interface. In some specific cases, they could be application developers or application owners.

The procedure starts by analysing the software artefacts included in the release along with their software dependencies. This task is carried out manually by the portal manager. Each portlet artefact which is released must be part of the functional testing procedure. Also, if anything in the dependency chain of a Portlet X is released, then X has to be functionally tested too.

The Portlets Functional Testing procedure consists of 4 stages:

  1. Web Archive check: every Portlet web archive released is verified to contain all the necessary files needed for its correct deployment. If the check is not passed the Portlet developer is notified through a new ticket of type integration issue;
  2. Back-end Service check: The portal manager and the infrastructure manager make sure that the back-end services of the Portlet application are ready and fully functional in the testing / pre-production infrastructure.
  3. Rendering check: every Portlet web archive passing stage no. 1 is then actually deployed into the pre-production portal to check if it renders correctly. In this stage there are few things that could make the Portlet not render correctly. For instance, the GWT compilation could fail during Web Archive creation, or the developer could have used some CSS classes that clash with those of the iMarine Gateway. If the check is not passed the Portlet developer is notified through a new ticket of type integration issue;
  4. Functional Test: When stage 1 and 2 and 3 are completed the functional test is assigned to the testing team which performs it by following the Software Testing Plan. An updating of the Functional test procedure is discussed to use the the Redmine tool to track the functional testing activities and dismiss the Functional Master Table. The new Software Test procedure is provided in the Functional Test (FT) Procedure section.

Software Testing Plan

The Software Testing Plan #1413 is created, from the Portal Manager, in the VRE Folders > gCube > Portlet Testing Plans containing two folders: Material and Releases.
The Material directory provides the following information:

  1. the general XLS template to be instantiated by each portlet developer for compiling the Testing Plan and
  2. the portlet folder (created by Portal Manager) for each portlet to be functionality tested. The syntax used to create the portlet folder

will be: $portlet_name_folder. Every developer will create the actual testing plan into Material/$portlet_name_folder including by adding additional files required for the test (e.g. cvs_files). To describe the functional tests, the developer will use the Portlet Testing Plan Template; this file will be pasted and completed into the Release folders by the testers. A Task (Redmine Ticket) will be assigned, by the Release Manager, to each portlet developer.
The Releases directory contains a folder for each gCube release containing the tests to be executed. The Portal Manager creates (in this directory) the gCube release directory and the portlets directories related.

Functional Test (FT) Procedure

The Software Testing Plan is created in the BlueCommons VRE Folder containing two folders: Material and Releases.

  • Material provides:
    • the general XLS template to be instantiated by each portlet developer for compiling the Testing Plan and
    • the portlet folder (created by Portal Manager) for each portlet to be functionality tested. The syntax used to create the portlet folder will be $portlet_name_folder.
    • Every developer will create the actual testing plan into Material/$portlet_name_folder including by adding additional files required for the test (e.g. cvs_files). A Task (Redmine Ticket) will be assigned, by the Release Manager,to each portlet developer.
  • Releases provides a folder for each gCube release containing the tests to be executed.

The Release Manager will ask to every partners to suggest the persons that will form the testing team.

Mandatory Recommendations and Rules

  • The effort should be based not per application but per number of tests to be executed .
  • The tester should NOT be the same owner of the portlet.
  • The functional tests will be executed using the Portlet Testing Plan Template.
  • The Portlets applications are executed in the pre-production infrastructure.
  • All testers use Redmine Ticket to track their testing activities.
  • If a new portlets or tester is added, the Developer or Subsystem Manager must communicate it by email to the Test Manager, Infrastructure Manager, Deployer Manager and Subsystem Manager.


SoftWare Test Flow

The Tester Manager creates the Master and the SW Test Ticket component (the procedure is inspired to the Configuration Release Ticket).
The fields to fill for the SW Test ticket are:

  • Project: gCube
  • Tracker: SW Test (software test)
  • Status :
    1. New :when is created by Tester Manager;
    2. Available : the Deployer Manager change the status from New to Available when the portlets service is deployed in the pre-production.The tester know can start with the test;
    3. Under Testing: the Tester change the status from Available to Under Testing when start with the test and change the %Done field to track the progress of the test;
    4. Test Issue: when the portlets not pass the test, the Tester change the status from Under Testing to Test Issue and assign the ticket to the developer. When the issue is solved the developer changes the status from Test Issue to Available so the Tester can re-started with the test;
    5. Tested on Pre-prod : Test is passed. The Tester changes the status from Under Testing to Tested on Pre-prod. And add as watcher Tester Manager. In this way the Tester Manager (if exist the release ticket) changes the status from Deployed on Pre-prod to Tester-on-preprod.
    6. Assignee: the SW test ticket will assigned always to the Testers except when the Status of the portlets is Test Issue (in this case the ticket is assigned to the developer to solve the issue).
    7. Description: the Tester Manager adds the "VRE Workspace link" of the Test Plan (if exist) .
    8. %Done: the tester (or developer) should use it to track the work in progress.
    9. Watcher: Tester Manager, Infrastructure Manager or Deployer Manager.

The flow is showed in the following figure:

SWTestWorkflow.png

The SW Test Portlet Tickets will be created copying the Master Ticket (it's considered the template of the SW Test Portlet). The Master Ticket and its sub-tickets can be used as template with Progress field at "0" and Status field at "New" to create the other tickets.

For the testing of each release the Tester Manager copies from that template. In this way, only the sprint is changed (in one action thanks to the multi-selection).

The SW Test Portlet are available after the services are deployed on pre-production and after the Deployer Manager following the procedure as reported in the (Releases Integration).

The SoftWare Test ticket will be opened by the Tester or Portal Manager and assigned to the testers (the procedure is similar as for the release tickets).

  1. The TesterManager creates the ticket: {Project: gCube; Tracker: SW Test;Subject: name of the portlet to test with release version;Assignee: name of Tester;Status: New;Description: Test Plan URL;%Done : 0;Watcher: Developer, Subsystem Manager, Deployer Manager,Infrastructure or Portal Manager;Reference : of the Release Ticket , if exist.}.
  2. After the deployement of the service or portlet , the Deployer Manager updates: the SW Test ticket {status: New -> Available; Assignee:Tester} ( and the related Release Ticket from {status:Under Integration to Deployed on Pre-prod} ). The Tester knows that he/she can start with the test.
  3. The Tester can start with the test updating the SW Test status: {status: Available -> Under Testing}.
  4. If an issue occurred during the test , the Tester changes the SW Test Ticket status{status: Under Testing -> Test Issue; Assignee:Developer responsible of the portlet}.
  5. When the issue is solved , the developer updates the Software Test Ticket {status:Test Issue ->New; Assignee: Tester} if the portlet must be deployed otherwise {status: Test issue -> Available}. The Tester repeats the test.
  6. If the test passes the Tester changes the status from {status: Under Testing -> Tested on pre-prod} and copies the TestPlan, with the results of the test, in the Releases directory.For example: the tester tests the invites-friends.1-1-0 portlets. The Tester copies the TestPlan for this portlets into Releases/org.gcube.4.0.0/invites-friends.1-1-0.
  7. When all portlet are in status Tested on pre-prod , the Tester Manager can be changed the status of the Release ticket from {status: Deployed on pre-prod -> Tested on pre-prod}.
NOTE:
 - Test issue  = The test is fails.
 - Tested on Preprod : indicated that the developer and/or the portal manager or the preprod infrastructure manager 
   are confident that the component behave as expected and can be released as is.

The pre-production infrastructure is provided for the functional testing hosted at CNR.

Test Log

The information and statistics for the Functional Test Portlet (for all release involved in it) is reported in the: Test Log section.

Roles

The functionality testing activity foresees the involvement of the following roles:

  • Tester
  • Developer

Tester

The TesterManager is responsible:

  • to create the new Software Test Ticket (master ticket and its sub-tickets);
  • to create the query;
  • to clone the master ticket acting as template for the portlet testing.

The Tester is responsible:

  • for the execution of the functionality tests and to submit functionality test bugs
  • to fill the dedicated Portlet Testing Plan Template and the creation of the SW Test ticket.
  • to track the Test Issue through the use of the track system issue tool (ref. D4Science support).


The details about the assignment of the testers (for each partners) are hosted on the Role Assignment paragraph.

Developer

The details about the role of the developers are hosted on Developers wiki page. The developer is:

Infrastructure

The execution of functionality tests requires a gCube testing infrastructure fully functional. A detailed description of the testing infrastructure can be found Testing Infrastructure

Retrieved from "https://wiki.gcube-system.org/index.php?title=Functional_Testing&oldid=31013"