Difference between revisions of "Creating Indices at the VO Level"
From Gcube Wiki
(→Creating a Geo-Spatial Index for a collection with ES payload) |
(→DataTransformation Programs) |
||
| Line 24: | Line 24: | ||
=== DataTransformation Programs === | === DataTransformation Programs === | ||
| − | + | ====FtsRowset_Transformer==== | |
| + | The following transformation program is called for fulltext rowset creation. Transformation unit with id="6" takes multiple XSLTs and applies final XSLT at the end. | ||
<gDTSTransformationProgram> | <gDTSTransformationProgram> | ||
<Transformer> | <Transformer> | ||
| Line 70: | Line 71: | ||
</gDTSTransformationProgram> | </gDTSTransformationProgram> | ||
| − | + | ====FwRowset_Transformer==== | |
| + | The following transformation program is called for forward rowset creation. Transformation unit with id="1" takes multiple XSLTs and applies final XSLT at the end. | ||
<gDTSTransformationProgram> | <gDTSTransformationProgram> | ||
<Transformer> | <Transformer> | ||
Revision as of 17:29, 19 November 2013
Contents
Indexing Procedure
The Indexing procedure refers to the creation of indices for the collections imported in a Virtual Organization. It consists of three steps:
- Creation of the Rowset XSLT generic resources, that transform collection data into data that can be fed to an Index.
- Creation of the Index type generic resources, that define the Index configuration.
- Definition of an IRBootstrapper job that will perform the steps required to create the Indices.
In the first two steps we create generic resources for the Rowset XSLTs and Index Types through the Resource Management portlet . You can find detailed descriptions for the Rowset data (the output of the Rowset XSLT transformation) in the following sections:
You can find detailed descriptions for the Index Type definition here:
For the third step, a definition of an IRBootstrapper job is required. You can find the details for defining such a job in the IR Bootstrapper section. To complete the Index creation, the administrator must go to the IRBootstrapper and run the job. The two examples that follow will clarify the three steps.
Creating a Full Text and a Forward Index for a OAI-DC collection
DataTransformation Programs
FtsRowset_Transformer
The following transformation program is called for fulltext rowset creation. Transformation unit with id="6" takes multiple XSLTs and applies final XSLT at the end.
<gDTSTransformationProgram>
<Transformer>
<Class>org.gcube.datatransformation.datatransformationlibrary.programs.metadata.indexfeed.FtsRowset_Transformer</Class>
<GlobalProgramParameters />
</Transformer>
<TransformationUnits>
<TransformationUnit id="6" isComposite="false">
<Sources>
<Source>
<Input id="TRInput0" />
<ContentType>
<Mimetype>text/xml</Mimetype>
<Parameters>
<Parameter isOptional="false" name="schemaURI" value="-" />
</Parameters>
</ContentType>
</Source>
</Sources>
<ProgramParameters>
<Parameter isOptional="false" name="finalftsxslt" value="-" />
<Parameter isOptional="false" name="indexType" value="-" />
<Parameter isOptional="true" name="xslt" value="-" />
<Parameter isOptional="true" name="xslt:1" value="-" />
<Parameter isOptional="true" name="xslt:2" value="-" />
<Parameter isOptional="true" name="xslt:3" value="-" />
<Parameter isOptional="true" name="xslt:4" value="-" />
<Parameter isOptional="true" name="xslt:5" value="-" />
<Parameter isOptional="true" name="xslt:6" value="-" />
<Parameter isOptional="true" name="xslt:7" value="-" />
<Parameter isOptional="true" name="xslt:8" value="-" />
<Parameter isOptional="true" name="xslt:9" value="-" />
</ProgramParameters>
<Target>
<Output id="TROutput" />
<ContentType>
<Mimetype>text/xml</Mimetype>
<Parameters>
<Parameter isOptional="false" name="schemaURI" value="http://ftrowset.xsd" />
</Parameters>
</ContentType>
</Target>
</TransformationUnit>
</TransformationUnits>
</gDTSTransformationProgram>
FwRowset_Transformer
The following transformation program is called for forward rowset creation. Transformation unit with id="1" takes multiple XSLTs and applies final XSLT at the end.
<gDTSTransformationProgram>
<Transformer>
<Class>org.gcube.datatransformation.datatransformationlibrary.programs.metadata.indexfeed.FwRowset_Transformer</Class>
<GlobalProgramParameters />
</Transformer>
<TransformationUnits>
<TransformationUnit id="1" isComposite="false">
<Sources>
<Source>
<Input id="TRInput0" />
<ContentType>
<Mimetype>text/xml</Mimetype>
<Parameters />
</ContentType>
</Source>
</Sources>
<ProgramParameters>
<Parameter isOptional="false" name="finalfwdxslt" value="-" />
<Parameter isOptional="true" name="xslt" value="-" />
<Parameter isOptional="true" name="xslt:1" value="-" />
<Parameter isOptional="true" name="xslt:2" value="-" />
<Parameter isOptional="true" name="xslt:3" value="-" />
<Parameter isOptional="true" name="xslt:4" value="-" />
<Parameter isOptional="true" name="xslt:5" value="-" />
<Parameter isOptional="true" name="xslt:6" value="-" />
<Parameter isOptional="true" name="xslt:7" value="-" />
<Parameter isOptional="true" name="xslt:8" value="-" />
<Parameter isOptional="true" name="xslt:9" value="-" />
</ProgramParameters>
<Target>
<Output id="TROutput" />
<ContentType>
<Mimetype>text/xml</Mimetype>
<Parameters>
<Parameter isOptional="false" name="schemaURI" value="http://fwrowset.xsd" />
</Parameters>
</ContentType>
</Target>
</TransformationUnit>
</TransformationUnits>
</gDTSTransformationProgram>
Index Types
In this section we present the required IndexTypes for both FullText and Forward Indices.
FullTextIndexType
In order to extract the fields from the OAI-DC payload and build the FullText Index the following FullTextIndexType is required:
<Name>IndexType_ft_oai_dc_1.0</Name> <SecondaryType>FullTextIndexType</SecondaryType> <Description>Definition of the fulltext index type for the 'oai dc' schema</Description> <Body> <index-type name="default"> <field-list sort-xnear-stop-word-threshold="2E8"> <field name="contributor"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="coverage"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="creator"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="date"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="description"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="format"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="identifier"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="language"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="publisher"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="relation"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="rights"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="source"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="subject"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="title"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="type"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="ObjectID"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="gDocCollectionID"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="gDocCollectionLang"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="S"> <index>no</index> <store>yes</store> <return>yes</return> <tokenize>no</tokenize> <sort>no</sort> <boost>1.0</boost> </field> </field-list> </index-type> </Body>
ForwardIndexType
In OAI-DC many fields, such as "title" and "creator", have string values, so we just need to create a ForwardIndexType for string-string key-value pairs in order to be able to create the Forward Index:
<SecondaryType>ForwardIndexType</SecondaryType> <Name>IndexType_fwd_string_string</Name> <Description>Definition of the index type 'string_string' for the forward index</Description> <Body> <field-list> <field name="key"> <type>string</type> <sort>ascending</sort> </field> <field name="value"> <type>string</type> </field> </field-list> </Body>
Note that, in contrast to the FullTextIndexType in ForwardIndexType there is no field-datatype mapping but just declaration of the datatypes supported in the index.
Bootstrapper Configuration
The IRBootstrapper portlet requires a Generic Resource to be available on the IS with name: IRBootstrapperConfiguration and secondary type: IRBootstrapperConfig. For more information please refer to this section IRBoostrapperConfiguration Generic Resource
An example of the configuration is the following:
<BootstrapInfo> <types> <type class="org.gcube.portlets.admin.irbootstrapperportlet.gwt.server.types.data.TreeManagerCollectionDataType" name="TreeManagerCollection" /> <type class="org.gcube.portlets.admin.irbootstrapperportlet.gwt.server.types.data.OpenSearchDataType" name="OpenSearch" /> <type class="org.gcube.portlets.admin.irbootstrapperportlet.gwt.server.types.data.FullTextIndexNodeDataType" name="FullTextIndexNode" /> <type class="org.gcube.portlets.admin.irbootstrapperportlet.gwt.server.types.data.ForwardIndexNodeDataType" name="ForwardIndexNode" /> <type class="org.gcube.portlets.admin.irbootstrapperportlet.gwt.server.types.data.GCUBECollectionDataType" name="GCUBECollection" /> <tasktype class="org.gcube.portlets.admin.irbootstrapperportlet.gwt.server.types.task.OpenSearchGenerationTaskType" name="OpenSearchGenerationTaskType"> <input type="GCUBECollection" /> <output type="OpenSearch" /> <run>true</run> </tasktype> <tasktype class="org.gcube.portlets.admin.irbootstrapperportlet.gwt.server.types.task.ForwardIndexNodeGenerationTaskType" name="ForwardIndexNodeGenerationTask"> <input type="TreeManagerCollection" /> <output type="ForwardIndexNode" /> <run>true</run> </tasktype> <tasktype class="org.gcube.portlets.admin.irbootstrapperportlet.gwt.server.types.task.FullTextIndexNodeGenerationTaskType" name="FullTextIndexNodeGenerationTask"> <input type="TreeManagerCollection" /> <output type="FullTextIndexNode" /> <run>true</run> </tasktype> <jobtype description="Creates the required fulltext indices for a collection." name="FTIndexNodeCollection"> <input type="TreeManagerCollection" /> <jobDefinition> <parallel> <sequential> <assign to="%Create_ft_node_index.input" value="%FTIndexNodeCollection.input" /> <assign to="%Create_ft_node_index.output.IndexedCollectionID" value="%Create_ft_node_index.input.ColID" /> <task name="Create_ft_node_index" tasktype="FullTextIndexNodeGenerationTask" /> </sequential> </parallel> </jobDefinition> </jobtype> <jobtype description="Creates the required forward indices for a collection." name="FWDIndexNodeCollection"> <input type="TreeManagerCollection" /> <jobDefinition> <parallel> <sequential> <assign to="%Create_fwd_node_index.input" value="%FWDIndexNodeCollection.input" /> <assign to="%Create_fwd_node_index.output.IndexedCollectionID" value="%Create_fwd_node_index.input.ColID" /> <task name="Create_fwd_node_index" tasktype="ForwardIndexNodeGenerationTask" /> </sequential> </parallel> </jobDefinition> </jobtype> <jobtype description="Creates the open search resource for an open search collection." name="CreateOpenSearchCollectionResource"> <input type="GCUBECollection" /> <jobDefinition> <parallel> <sequential> <assign to="%Create_OSR.input" value="%CreateOpenSearchCollectionResource.input" /> <assign to="%Create_OSR.output.OpenSearchCollectionID" value="%Create_OSR.input.ColID" /> <task name="Create_OSR" tasktype="OpenSearchGenerationTaskType" /> </sequential> </parallel> </jobDefinition> </jobtype> </types> <jobs> <job jobtype="FTIndexNodeCollection" name="FullText Index SPD Tree Collections"> <initialization> <assign to="%FTIndexNodeCollection.input.Type" value="ns5:SPD" /> <assign to="%Create_ft_node_index.FullTextIndexNodeGenerationTask.IndexTypeID" value="ft_SPD_1.0" /> <assign to="%Create_ft_node_index.FullTextIndexNodeGenerationTask.TransformationXSLTID" value="$BrokerXSLT_wrapperFT" /> <assign to="%Create_ft_node_index.FullTextIndexNodeGenerationTask.XsltsIDs" value="[ $BrokerXSLT_PROVENANCE_anylanguage_to_ftRowset_anylanguage, $BrokerXSLT_DwC_anylanguage_to_ftRowset_anylanguage, $BrokerXSLT_Properties_anylanguage_to_ftRowset_anylanguage ]" /> <assign to="%Create_ft_node_index.FullTextIndexNodeGenerationTask.IdOfIndexManagerToAppend" userInputLabel="ID of index node to append" value="%userInput" /> </initialization> </job> <job jobtype="FTIndexNodeCollection" name="FullText Index OAI Tree Collections"> <initialization> <assign to="%FTIndexNodeCollection.input.Type" value="ns5:OAI" /> <assign to="%Create_ft_node_index.FullTextIndexNodeGenerationTask.IndexTypeID" value="ft_oai_dc_1.0" /> <assign to="%Create_ft_node_index.FullTextIndexNodeGenerationTask.TransformationXSLTID" value="$BrokerXSLT_wrapperFT" /> <assign to="%Create_ft_node_index.FullTextIndexNodeGenerationTask.XsltsIDs" value="[ $BrokerXSLT_FARM_dc_anylanguage_to_ftRowset_anylanguage ]" /> <assign to="%Create_ft_node_index.FullTextIndexNodeGenerationTask.IdOfIndexManagerToAppend" userInputLabel="ID of index node to append" value="%userInput" /> </initialization> </job> <job jobtype="FTIndexNodeCollection" name="FullText Index FIGIS Tree Collections"> <initialization> <assign to="%FTIndexNodeCollection.input.Type" value="ns5:FIGIS" /> <assign to="%Create_ft_node_index.FullTextIndexNodeGenerationTask.IndexTypeID" value="ft_FIGIS_1.0" /> <assign to="%Create_ft_node_index.FullTextIndexNodeGenerationTask.TransformationXSLTID" value="$BrokerXSLT_wrapperFT" /> <assign to="%Create_ft_node_index.FullTextIndexNodeGenerationTask.XsltsIDs" value="[ $BrokerXSLT_FIGIS_anylanguage_to_ftRowset_anylanguage ]" /> <assign to="%Create_ft_node_index.FullTextIndexNodeGenerationTask.IdOfIndexManagerToAppend" userInputLabel="ID of index node to append" value="%userInput" /> </initialization> </job> <job jobtype="FWDIndexNodeCollection" name="Forward Index SPD Tree Collections"> <initialization> <assign to="%FWDIndexNodeCollection.input.Type" value="ns5:SPD" /> <assign to="%Create_fwd_node_index.ForwardIndexNodeGenerationTask.TransformationXSLTID" value="$BrokerXSLT_wrapperFWD" /> <assign to="%Create_fwd_node_index.ForwardIndexNodeGenerationTask.IndexedKeyNames" value="[ ObjectID, gDocCollectionID, gDocCollectionLang, scientificName, scientificNameAuthorship, genus, phylum, kingdom, order, family, specificEpithet ]" /> <assign to="%Create_fwd_node_index.ForwardIndexNodeGenerationTask.IndexedKeyTypes" value="[ fwd_string_string, fwd_string_string, fwd_string_string, fwd_string_string, fwd_string_string, fwd_string_string, fwd_string_string, fwd_string_string, fwd_string_string, fwd_string_string, fwd_string_string ]" /> <assign to="%Create_fwd_node_index.ForwardIndexNodeGenerationTask.XsltsIDs" value="[ $BrokerXSLT_DwC_anylanguage_to_fwRowset_anylanguage ]" /> <assign to="%Create_fwd_node_index.ForwardIndexNodeGenerationTask.IdOfIndexManagerToAppend" userInputLabel="ID of index node to append" value="%userInput" /> </initialization> </job> <job jobtype="FWDIndexNodeCollection" name="Forward Index FIGIS Tree Collections"> <initialization> <assign to="%FWDIndexNodeCollection.input.Type" value="ns5:FIGIS" /> <assign to="%Create_fwd_node_index.ForwardIndexNodeGenerationTask.TransformationXSLTID" value="$BrokerXSLT_wrapperFWD" /> <assign to="%Create_fwd_node_index.ForwardIndexNodeGenerationTask.IndexedKeyNames" value="[ ObjectID, gDocCollectionID, gDocCollectionLang, scientific_name, family, personal_author ]" /> <assign to="%Create_fwd_node_index.ForwardIndexNodeGenerationTask.IndexedKeyTypes" value="[ fwd_string_string, fwd_string_string, fwd_string_string, fwd_string_string, fwd_string_string, fwd_string_string ]" /> <assign to="%Create_fwd_node_index.ForwardIndexNodeGenerationTask.XsltsIDs" value="[ $BrokerXSLT_FIGIS_anylanguage_to_fwRowset_anylanguage ]" /> <assign to="%Create_fwd_node_index.ForwardIndexNodeGenerationTask.IdOfIndexManagerToAppend" userInputLabel="ID of index node to append" value="%userInput" /> </initialization> </job> <job jobtype="FWDIndexNodeCollection" name="Forward Index OAI Tree Collections"> <initialization> <assign to="%FWDIndexNodeCollection.input.Type" value="ns5:OAI" /> <assign to="%Create_fwd_node_index.ForwardIndexNodeGenerationTask.TransformationXSLTID" value="$BrokerXSLT_wrapperFWD" /> <assign to="%Create_fwd_node_index.ForwardIndexNodeGenerationTask.IndexedKeyNames" value="[ ObjectID, gDocCollectionID, gDocCollectionLang, title, creator, subject, coverage ]" /> <assign to="%Create_fwd_node_index.ForwardIndexNodeGenerationTask.IndexedKeyTypes" value="[ fwd_string_string, fwd_string_string, fwd_string_string, fwd_string_string, fwd_string_string, fwd_string_string, fwd_string_string ]" /> <assign to="%Create_fwd_node_index.ForwardIndexNodeGenerationTask.XsltsIDs" value="[ $BrokerXSLT_dc_anylanguage_to_fwRowset_anylanguage_title_creator_subject_coverage ]" /> <assign to="%Create_fwd_node_index.ForwardIndexNodeGenerationTask.IdOfIndexManagerToAppend" userInputLabel="ID of index node to append" value="%userInput" /> </initialization> </job> <job jobtype="CreateOpenSearchCollectionResource" name="CreateOSResourceForDRIVERCollection"> <initialization> <assign to="%CreateOpenSearchCollectionResource.input.ColName" value="DRIVER" /> <assign to="%Create_OSR.OpenSearchGenerationTask.FieldParameters" value="[ en:s:allIndexes, en:p:title, en:p:creator, en:p:pubDate ]" /> <assign to="%Create_OSR.OpenSearchGenerationTask.OpenSearchResourceID" value="30602b90-603f-11e0-90e5-a7c4e0a7bbf8" /> </initialization> </job> <job jobtype="CreateOpenSearchCollectionResource" name="CreateOSResourceForEcoscopeCollection"> <initialization> <assign to="%CreateOpenSearchCollectionResource.input.ColName" value="Ecoscope" /> <assign to="%Create_OSR.OpenSearchGenerationTask.FieldParameters" value="[ en:s:allIndexes, en:p:title, en:p:link, en:p:description, en:p:S ]" /> <assign to="%Create_OSR.OpenSearchGenerationTask.OpenSearchResourceID" value="ae0210b0-dcd2-11e2-be24-9415b2540510" /> </initialization> </job> <job jobtype="CreateOpenSearchCollectionResource" name="CreateOSResourceForBING"> <initialization> <assign to="%CreateOpenSearchCollectionResource.input.ColName" value="Bing" /> <assign to="%Create_OSR.OpenSearchGenerationTask.FieldParameters" value="[ en:s:allIndexes, en:p:title, en:p:link, en:p:description, en:p:S, en:p:pubDate ]" /> <assign to="%Create_OSR.OpenSearchGenerationTask.FixedParameters" value="[ http%3A%2F%2Fa9.com%2F-%2Fspec%2Fopensearch%2F1.1%2F:count="25", config:numOfResults="200" ]" /> <assign to="%Create_OSR.OpenSearchGenerationTask.OpenSearchResourceID" value="bcd216d0-dce6-11e2-89e1-9415b2540510" /> </initialization> </job> <job jobtype="CreateOpenSearchCollectionResource" name="CreateOSResourceForINSPIRECollection"> <initialization> <assign to="%CreateOpenSearchCollectionResource.input.ColName" value="INSPIRE" /> <assign to="%Create_OSR.OpenSearchGenerationTask.FieldParameters" value="[ en:s:allIndexes, en:p:title, en:p:creator, en:p:pubDate ]" /> <assign to="%Create_OSR.OpenSearchGenerationTask.OpenSearchResourceID" value="30602b90-603f-11e0-90e5-a7c4e0a7bbf8" /> </initialization> </job> <job extends="FullText Index OAI Tree Collections" jobtype="FTIndexNodeCollection" name="Indexing"> <initialization> <assign to="%FTIndexNodeCollection.input.ColID" value="FIGIS" /> </initialization> </job> </jobs> </BootstrapInfo>
Metadata Broker XSLT
- BrokerXSLT_oai_dc_anylanguage_to_ftRowset_anylanguage
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> <xsl:output indent="yes" method="xml" omit-xml-declaration="yes" /> <xsl:template match="/"> <ROWSET> <ROW> <xsl:for-each select="//*[local-name()='title']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='creator']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='subject']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='description']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='publisher']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='contributor']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='date']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='type']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='format']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='identifier']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='source']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='language']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='relation']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='coverage']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='rights']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='alternative']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='tableOfContents']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='abstract']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='created']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='valid']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='available']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='issued']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='modified']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='dateAccepted']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='dateCopyrighted']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='dateSubmitted']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='extend']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='medium']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='isVersionOf']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='hasVersion']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='isReplacedBy']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='replaces']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='isRequiredBy']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='requires']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='isPartOf']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='hasPart']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='isReferencedBy']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='references']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='isFormatOf']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='hasFormat']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='conformsTo']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='spatial']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='temporal']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='audience']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='accrualMethod']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='accrualPeriodicity']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='accrualPolicy']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='instructionalMethod']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='provenance']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='rightsHolder']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='mediator']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='educationLevel']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='accessRights']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='license']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> <xsl:for-each select="//*[local-name()='bibliographicCitation']"> <xsl:if test="normalize-space(.)"> <FIELD name="{local-name()}"> <xsl:value-of select="normalize-space(.)" /> </FIELD> </xsl:if> </xsl:for-each> </ROW> </ROWSET> </xsl:template> </xsl:stylesheet>
- BrokerXSLT_wrapperFT
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> <xsl:output indent="yes" method="xml" omit-xml-declaration="yes" /> <xsl:template match="/"> <ROWSET> <ROW> <xsl:for-each select="//ROWSET/ROW/FIELD"> <xsl:copy-of select="self::node()[text() and not(@name = preceding::FIELD/@name and text() = preceding::FIELD/text())]" /> </xsl:for-each> </ROW> </ROWSET> </xsl:template> </xsl:stylesheet>
- BrokerXSLT_dc_anylanguage_to_fwRowset_anylanguage_title_creator_subject_coverage
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> <xsl:output indent="yes" method="xml" omit-xml-declaration="yes" /> <xsl:variable name="keys"> <key> <keyName>title</keyName> <keyXPath>//*[local-name()='title']</keyXPath> </key> <key> <keyName>creator</keyName> <keyXPath>//*[local-name()='creator']</keyXPath> </key> <key> <keyName>subject</keyName> <keyXPath>//*[local-name()='subject']</keyXPath> </key> <key> <keyName>coverage</keyName> <keyXPath>//*[local-name()='coverage']</keyXPath> </key> </xsl:variable> <xsl:template match="/"> <xsl:element name="ROWSET"> <xsl:element name="INSERT"> <xsl:element name="TUPLE"> <xsl:element name="VALUE"> <xsl:for-each select="//*[local-name()='title']"> <FIELD name="title"> <xsl:value-of select="." /> </FIELD> </xsl:for-each> <xsl:for-each select="//*[local-name()='creator']"> <FIELD name="creator"> <xsl:value-of select="." /> </FIELD> </xsl:for-each> <xsl:for-each select="//*[local-name()='subject']"> <FIELD name="subject"> <xsl:value-of select="." /> </FIELD> </xsl:for-each> <xsl:for-each select="//*[local-name()='coverage']"> <FIELD name="coverage"> <xsl:value-of select="." /> </FIELD> </xsl:for-each> </xsl:element> </xsl:element> </xsl:element> </xsl:element> </xsl:template> </xsl:stylesheet>
- BrokerXSLT_wrapperFWD
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> <xsl:output indent="yes" method="xml" omit-xml-declaration="yes" /> <xsl:template match="/"> <ROWSET> <INSERT> <TUPLE> <xsl:for-each select="//ROWSET/INSERT/TUPLE/KEY"> <xsl:copy-of select="self::node()[not(KEYNAME = preceding::KEY/KEYNAME and KEYVALUE = preceding::KEY/KEYVALUE)]" /> </xsl:for-each> <VALUE> <xsl:for-each select="//ROWSET/INSERT/TUPLE/VALUE/FIELD"> <xsl:copy-of select="self::node()[text() and not(@name = preceding::FIELD/@name and text() = preceding::FIELD/text())]" /> </xsl:for-each> </VALUE> </TUPLE> </INSERT> </ROWSET> </xsl:template> </xsl:stylesheet>
