Difference between revisions of "Index Management"

From Gcube Wiki
Jump to: navigation, search
(Creating a Full Text Index Node)
(Full Text Index)
 
(3 intermediate revisions by one other user not shown)
Line 16: Line 16:
 
A big green folder represents a collection which has one or more related indices.
 
A big green folder represents a collection which has one or more related indices.
 
A big blue folder represents a collection which does not have any related index.
 
A big blue folder represents a collection which does not have any related index.
The single blue folder marked with a pencil and labeled as "Fake Collections" is the only icon which is always present even if no collections exists. This icon represents all collection IDs which are related to one or more indices, but which can not be mapped to a current available collection.
+
The single blue folder marked with a pencil and labeled as "Empty Managers" is the only icon which is always present even if no collections exists. This icon represents all collection IDs which are related to one or more indices, but which can not be mapped to a current available collection. The New Index Node icon under the "Empty Managers" folder is used in order to
 +
create a new Index (see [Index_Management#Creating_a_Full_Text_Index_Node here] for details).
  
 
If you select an Index Node item in the Index Explorer, the Index Detail panel will show a tab panel with information related to the selected resource as you can see on the picture above.
 
If you select an Index Node item in the Index Explorer, the Index Detail panel will show a tab panel with information related to the selected resource as you can see on the picture above.
Line 46: Line 47:
 
This node is empty and can be later used for any of the available collections.
 
This node is empty and can be later used for any of the available collections.
  
[[File:Index-new-index-node.png]
+
[[File:Index-new-index-node.png]]
 
+
==== Feeding a Full Text Index ====
+
The information to be indexed is fed to the Index through a [[ResultSet Framework|ResultSet]]. In order to index the ResultSet, select the Index item to be used in the Index Explorer, and choose the "Feed" tab. Enter a valid RSLocator string in the textbox as shown below (the RSLocator string in the example is a bit strange one since it points to a local ResultSet):
+
[[Image:FullTextPortlet9.png|frame|none|Generic Editor]]
+
 
+
Push the "Update" button to start the indexing of the provided ResultSet. A green label will indicate that the update has been successfully started:
+
[[Image:FullTextPortlet10.png|frame|none|Generic Editor]]
+
 
+
==== Testing/Querying a Full Text Index ====
+
In order to test (by querying) an Index, an Index Node for the Index in question is needed. If no Index Node is available, it has to be [[Common_Functionality#Creating a Full Text Index Lookup|created]]. If an Index Node is available for the Index to be queried, select the Index Node in the Index Explorer and select the "Query" tab in the Index Detail panel. This tab allows you to enter a query string as shown below:
+
[[Image:FullTextPortlet11.png|frame|none|Generic Editor]]
+
 
+
Pushing the Query button will perform a query against the Index Node. Upon success, the first page of the results of the query will be shown in its raw XML format bellow the query text box:
+
[[Image:FullTextPortlet12.png|frame|none|Generic Editor]]
+
 
+
==== Removing a Full Text Index or Index Node ====
+
 
+
In order to remove an Index or an Index Node, select the item to be removed in the Index Explorer and choose the "Remove" tab in the Index Detail. Clicking the remove button (marked [<span style="color:red">1</span>] below), will cause a warning dialog (marked [<span style="color:red">2</span>]) to be shown. Push the "OK" button to remove the resource (and keep in mind that removing an Index resource will automatically remove all Index Node resources connected to it):
+
[[Image:FullTextPortletDetail5.png|frame|none|Generic Editor]]
+
 
+
Upon success, a green confirmation message will be shown:
+
[[Image:FullTextPortletDetail6.png|frame|none|Generic Editor]]
+
 
+
Upon failure, a red error message is shown. This message is usually due to the fact that the resource just removed is still being present in the Index Explorer due to delays in the underlying infrastructure.
+
[[Image:FullTextPortletDetail7.png|frame|none|Generic Editor]]
+
 
+
=== Full Text Index ===
+
 
+
After opening the the FullTextManagementPortlet, you see a screen with an Index Explorer tree on the left side, and an (empty) Index Detail panel on the right. The IndexDetail Panel is empty as no index resource has currently been selected in the Index Explorer tree. The Index Explorer panel, however, always displays one or more folder icons when the portlet is ready for use:
+
[[Image:FullTextPortlet1.png|frame|none|Generic Editor]]
+
 
+
The folder icons represent metadata collections available in the VO.
+
A big green folder (marked with a [<span style="color:red">1</span>] above) represents a collection name which has one or more related indices.
+
A big blue folder (marked [<span style="color:red">2</span>]) represents a collection name which does not have any related index.
+
The single blue folder marked with a pencil and labeled as "Fake Collections" (marked [<span style="color:red">3</span>]) is the only icon which is always present even if no collections exists. This icon represents all collection IDs which are related to one or more indices, but which can not be mapped to a true metadata collections. This is a usual scenario for test indices, in order to avoid other resources harvesting the epr of the test index from DIS when looking for valid indices of a specific collection.
+
 
+
 
+
A collection name is not necessarily unique, as many collections are allowed to have the same name though not the same ID. Actual metadata collections are represented by a smaller folder icon, along with its collection ID. Collections which have been indexed, are represented by a green folder (marked [<span style="color:red">4</span>]), while collections which have not been indexed are represented by a grey folder (marked [<span style="color:red">5</span>]).
+
 
+
==== Full Text Index Information ====
+
If you expand the green collection tree items (by clicking on the [+] beside them), one or more Index icons (marked [<span style="color:red">1</span>] below) appear. These represent Index Managers. If some Index Lookups are connected to the Index Manager, the Index item can be expanded to reveal one or more Index Lookup icons (marked[<span style="color:red">2</span>]):
+
 
+
[[Image:FullTextPortlet2.png|frame|none|Generic Editor]]
+
 
+
If you select either an Index item or an Index Lookup item in the Index Explorer, the Index Detail panel will show a tab panel with information related to the selected resource (WS ResourceProperty values). If an Index item is chosen, the IndexDetail will look similar to the following illustration:
+
 
+
[[Image:FullTextPortletDetail1.png|frame|none|Generic Editor]]
+
 
+
And if an Index Lookup item is chosen, it will look like the following:
+
[[Image:FullTextPortletDetail2.png|frame|none|Generic Editor]]
+
 
+
==== Editing the IndexType ====
+
 
+
In order to edit the Index Type of an Index Manager, make sure the correct Index item is selected in the Index Explorer, then select the "Edit" tab in the Index Detail tab panel. This will reveal Index Edit panel:
+
[[Image:FullTextPortletDetail3.png|frame|none|Generic Editor]]
+
The chosen Index items IndexType is displayed as a list of Index Type Fields with editable checkboxes. The save button saves the IndexType currently shown into the selected Index Management resource.
+
 
+
If a Index Lookup item has been selected, choosing the "Edit" tab will reveal the Index Lookup Edit panel:
+
[[Image:FullTextPortletDetail4.png|frame|none|Generic Editor]]
+
This panel is identical to the Index Edit panel, however only the "Return" checkboxes are enabled. This is due to the fact that the Return Element is the only part of an IndexLookup's index type which is editable. Any other change has to be performed towards the Index Management (using the Index Edit panel)
+
 
+
==== Creating a Full Text Index ====
+
Creating a new Index can be accomplished by either selecting the "New Index" item (marked [<span style="color:red">1</span>] below) or the "New Fake Collection" item (marked [<span style="color:red">2</span>])in the Index Explorer:
+
[[Image:FullTextPortlet5.png|frame|none|Generic Editor]]
+
Upon selecting one of the two items mentioned above, the Index Detail panel allows you to specify the properties of the new Index. The IndexID property textbox should usually be left empty, in which case the IndexService itself creates a unique ID for the Index. If the "New Fake Collection" item has been chosen, you also need to specify a collection ID (fake - e.g., "myTestCollection1" or something like this), however if the "New Index" item has been chosen, the collection ID will be retrieved from the parent collection item and will not be editable (as shown in the image above).
+
 
+
After specifying the properties of the new Index, push the "Create" button in order to create the new Index. Upon success, the new Index Item appears as the active element in the Index Explorer. The "Info" tab of the new Index is then shown in the Index Detail panel:
+
[[Image:FullTextPortlet6.png|frame|none|Generic Editor]]
+
As shown in the image above, the Index Detail panel is usually unable to retrieve information about the newly created Index. This is due to the fact that there is a slight lag between the creation of an Index and the availability of information about it. Clicking the newly created Index item once more will update the Index Detail panel with valid data.
+
 
+
==== Creating a Full Text Index Lookup ====
+
In order to query an Index, an Index Lookup resource is needed. An Index Lookup can be created at any point in the life time of an Index, and will automatically be updated when new information is indexed by its Index Manager. In order to create a new Index Lookup resource for an Index, select the "New Lookup" item under the Index item in the Index Explorer:
+
[[Image:FullTextPortlet7.png|frame|none|Generic Editor]]
+
The Index Detail panel shows a very simple interface, allowing you to specify the Lemmatizer plugin the new Index Lookup should use. Usually, this text box should be left empty, in which case the default Lemmatizer plugin is used. Pushing the "Create" button  triggers the creation of the new Index Lookup. Upon success, an item with a new icon appears as the active element in the Index Explorer. This item represents the newly created Index Lookup. The "Info" tab of the Index Lookup is then shown in the Index Detail panel:
+
[[Image:FullTextPortlet8.png|frame|none|Generic Editor]]
+
 
+
As it occurs in the creation of an Index, the Index Detail is usually not able to get information about the Index Lookup right after its creation, in which case you have to select the Lookup item once more if you need to see valid information about the new item.
+
 
+
==== Feeding a Full Text Index ====
+
The information to be indexed is fed to the Index through a [[ResultSet Framework|ResultSet]]. In order to index the ResultSet, select the Index item to be used in the Index Explorer, and choose the "Feed" tab. Enter a valid RSLocator string in the textbox as shown below (the RSLocator string in the example is a bit strange one since it points to a local ResultSet):
+
[[Image:FullTextPortlet9.png|frame|none|Generic Editor]]
+
 
+
Push the "Update" button to start the indexing of the provided ResultSet. A green label will indicate that the update has been successfully started:
+
[[Image:FullTextPortlet10.png|frame|none|Generic Editor]]
+
 
+
==== Testing/Querying a Full Text Index ====
+
In order to test (by querying) an Index, an Index Lookup for the Index in question is needed. If no Index Lookup is available, it has to be [[Common_Functionality#Creating a Full Text Index Lookup|created]]. If an Index Lookup is available for the Index to be queried, select the Index Lookup in the Index Explorer and select the "Query" tab in the Index Detail panel. This tab allows you to enter a query string as shown below:
+
[[Image:FullTextPortlet11.png|frame|none|Generic Editor]]
+
 
+
Pushing the Query button will perform a query against the Index Lookup. Upon success, the first page of the results of the query will be shown in its raw XML format bellow the query text box:
+
[[Image:FullTextPortlet12.png|frame|none|Generic Editor]]
+
 
+
==== Removing a Full Text Index or Index Lookup ====
+
 
+
In order to remove an Index or an Index Lookup, select the item to be removed in the Index Explorer and choose the "Remove" tab in the Index Detail. Clicking the remove button (marked [<span style="color:red">1</span>] below), will cause a warning dialog (marked [<span style="color:red">2</span>]) to be shown. Push the "OK" button to remove the resource (and keep in mind that removing an Index resource will automatically remove all Index Lookup and Index Updater resources connected to it):
+
[[Image:FullTextPortletDetail5.png|frame|none|Generic Editor]]
+
 
+
Upon success, a green confirmation message will be shown:
+
[[Image:FullTextPortletDetail6.png|frame|none|Generic Editor]]
+
 
+
Upon failure, a red error message is shown. This message is usually due to the fact that the resource just removed is still being present in the Index Explorer due to delays in the underlying infrastructure.
+
[[Image:FullTextPortletDetail7.png|frame|none|Generic Editor]]
+

Latest revision as of 15:43, 20 December 2013

Index Management

For each different type of index a dedicated portlet is available to manage the available indexes, create new index nodes and manage the IndexType generic resources that are used when creating new indexes. Currently 2 portlets are available:

  • FullTextIndexManagement portlet
  • ForwardIndexManagement portlet

Both portlets have the same User Interface and offer the same functionality. Thus we will present here one of the 2 portlets

Full Text Index

After opening the FullTextManagementPortlet, you see a screen with an Index Explorer tree on the left side, and an (empty) Index Detail panel on the right. The IndexDetail Panel is empty as no index resource has currently been selected in the Index Explorer tree. The Index Explorer panel, however, always displays one or more folder icons when the portlet is ready for use:

Generic Editor

The folder icons represent the collections available in the VO. A big green folder represents a collection which has one or more related indices. A big blue folder represents a collection which does not have any related index. The single blue folder marked with a pencil and labeled as "Empty Managers" is the only icon which is always present even if no collections exists. This icon represents all collection IDs which are related to one or more indices, but which can not be mapped to a current available collection. The New Index Node icon under the "Empty Managers" folder is used in order to create a new Index (see [Index_Management#Creating_a_Full_Text_Index_Node here] for details).

If you select an Index Node item in the Index Explorer, the Index Detail panel will show a tab panel with information related to the selected resource as you can see on the picture above. For each index node you can see:

  • The cluster it belongs to
  • When it was created
  • When it was last modified
  • The host of this index

Moreover you can:

  • Feed the index by providing a ResultSet locator
  • Query the index given a CQL query
  • Remove the index. If the index contains more than one collections, only the data related to the specific collection is removed


Creating - Editing - Removing IndexType Resources

In the second tab of the portlet you can create, edit and remove the available index type resources:
Index-index-types.png

The chosen Index items IndexType is displayed as a list of Index Type Fields with editable checkboxes. The save button saves the IndexType currently shown into the selected Index Management resource.

Creating a Full Text Index Node

A new index node can be created using this portlet. In order to do this click on the New Index Node under the Empty Managers tree. For each new node you should provide:

  • The cluster ID it will join, if any
  • The Running instance of the index service

This node is empty and can be later used for any of the available collections.

Index-new-index-node.png