.. _schemaPlugins: Schema Plugins ============== A schema in GeoNetwork is a directory with stylesheets, XML schema descriptions (XSDs) and other information necessary for GeoNetwork to index, view and possibly edit content from XML metadata records. To be used in GeoNetwork, a schema directory can be manually placed in the ``config/schema_plugins`` sub directory of the geonetwork data directory. The default geonetwork data directory location is ``INSTALL_DIR/web/geonetwork/WEB-INF/data``. The contents of these schemas are parsed during GeoNetwork initialization. If valid, they will be available for use when GeoNetwork starts up. Schemas can also added to GeoNetwork dynamically if a zip archive of the schema directory is created and then uploaded to GeoNetwork in one of following ways using functions in the Administration menu: #. Server file path (specified using file chooser) #. HTTP URL (eg. http://somehost/somedirectory/iso19139.mcp.zip) #. As an online resource attached to an ISO19115/19139 metadata record Uploaded schemas are also stored in the ``config/schema_plugins`` sub directory of the geonetwork data directory. Contents of a GeoNetwork schema ``````````````````````````````` When installed, a GeoNetwork schema is a directory. The following subdirectories can be present: - **convert**: (*Mandatory*) Directory of XSLTs to convert metadata from or to this schema. This could be to convert metadata to other schemas or to convert metadata from other schemas and formats to this schema. Eg. ``convert/oai_dc.xsl`` - **loc**: (*Optional*) Directory of localized information: labels, codelists or schema specific strings. Eg. ``loc/eng/codelists.xml`` - **present**: (*Mandatory*) contains XSLTs for presenting metadata in the viewer/editor and in response to CSW requests for brief, summary and full records. - **process**: (*Optional*) contains XSLTs for processing metadata elements by metadata suggestions mechanism (see **suggest.xsl** below). - **sample-data**: (*Mandatory*) Sample metadata for this schema. The metadata samples are in MEF format so that samples can have thumbnails or browse graphics as well as online resources. - **schema**: (*Optional*) Directory containing the official XSDs of the metadata schema. If the schema is described by a DTD then this directory is optional. Note that schemas described by a DTD cannot be edited by GeoNetwork. - **templates**: (*Optional*) Directory containing template and subtemplate metadata records for this schema. Template metadata records are usually metadata records with the set of elements (and content) that will be used for a specific purpose. Eg. iso19139.mcp schema has a 'Minimum Element' template that has the mandatory elements for the schema and a example of the content that is expected. The following stylesheets can be present: - **extract-date-modified.xsl**: (*Mandatory*) Extract the date of modification from the metadata record. - **extract-gml.xsl**: (*Mandatory*) Extract the spatial extent from the metadata record as a GML GeometryCollection element. - **extract-thumbnails.xsl**: (*Optional*) Extract the browse graphic/thumbnail from the metadata record. - **extract-uuid.xsl**: (*Mandatory*) Extract the UUID of the metadata record. - **index-fields.xsl**: (*Mandatory*) Index the metadata record content in Lucene. Creates the Lucene document used by GeoNetwork to index the metadata record content. - **schematron-rules-*.xsl**: (*Optional*) XSLT created from schematron rules when building the schema plugin (see schematrons directory). - **set-thumbnail.xsl**: (*Optional*) Set the browse graphic/thumbnail in the metadata record. - **set-uuid.xsl**: (*Optional*) Set the UUID of the metadata record. - **suggest.xsl**: (*Optional*) XSLT run by metadata suggestions service. The XSLT contains processes that can be registered and run on different elements of a metadata record. eg. expand keyword field with comma separated content into multiple keyword fields. See http://trac.osgeo.org/geonetwork/wiki/proposals/MetadataEditorSuggestion for more info. - **unset-thumbnail.xsl**: (*Optional*) Remove the browse graphic/thumbnail from the metadata record. - **update-child-from-parent-info.xsl**: (*Optional*) XSLT to specify which elements in a child record are updated from a parent record. Used to manage hierarchical relationships between metadata records. - **update-fixed-info.xsl**: (*Optional*) XSLT to update 'fixed' content in metadata records. The following configuration files can be present: - **oasis-catalog.xml**: (*Optional*) An oasis catalog describing any mappings that should be used for this schema eg. mapping URLs to local copies such as schemaLocations eg. http://www.isotc211.org/2005/gmd/gmd.xsd is mapped to ``schema/gmd/gmd.xsd``. Path names used in the oasis catalog are relative to the location of this file which is the schema directory. - **schema.xsd**: (*Optional*) XML schema directory file that includes the XSDs used by this metadata schema. If the schema uses a DTD then this file should not be present. Metadata records from schemas that use DTDs cannot be edited in GeoNetwork. - **schema-conversions.xml**: (*Optional*) XML file that describes the converters that can be applied to records belonging to this schema. This information is used to show these conversions as options for the user to choose when a metadata record belonging to this schema is shown in the search results. - **schema-ident.xml**: (*Mandatory*) XML file that contains the schema name, identifier, version number and details on how to recognise metadata records that belong to this schema. This file has an XML schema definition in ``INSTALL_DIR/web/geonetwork/xml/validation/schemaPlugins/schema-ident.xsd`` which is used to validate it when the schema is loaded. - **schema-substitutes.xml**: (*Optional*) XML file that redefines the set of elements that can be used as substitutes for a specific element. - **schema-suggestions.xml**: (*Optional*) XML file that tells the editor which child elements of a complex element to automatically expand in the editor. To help in understanding what each of these components is and what is required, we will now give a step-by-step example of how to build a schemaPlugin for GeoNetwork. .. _preparation: Preparation ``````````` In order to create a schema plugin for GeoNetwork, you should check out the schemaPlugins directory from the GeoNetwork sourceforge subversion repository. You can do this by installing subversion on your workstation and then executing the following command: :: svn co https://github.com/geonetwork/schema-plugins/branches/2.8.x schemaPlugins This will create a directory called schemaPlugins with some GeoNetwork schema plugins in it. To work with the example shown here, you should create your new schema plugin in a subdirectory of this directory. Example - ISO19115/19139 Marine Community Profile (MCP) ``````````````````````````````````````````````````````` The Marine Community Profile (MCP) is a profile of ISO19115/19139 developed for and with the Marine Community. The profile extends the ISO19115 metadata standard and is implemented using an extension of the XML implementation of ISO19115 described in ISO19139. Both the ISO19115 metadata standard and its XML implementation, ISO19139, are available through ISO distribution channels. The documentation for the Marine Community Profile can be found at http://www.aodc.gov.au/files/MarineCommunityProfilev1.4.pdf. The implementation of the Marine Community Profile as XML schema definitions is based on the approach described at https://www.seegrid.csiro.au/wiki/AppSchemas/MetadataProfiles. The XML schema definitions (XSDs) are available at the URL http://bluenet3.antcrc.utas.edu.au/mcp-1.4. Looking at the XML schema definitions, the profile adds a few new elements to the base ISO19139 standard. So the basic idea in defining a plugin Marine Community Profile schema for GeoNetwork is to use as much of the basic ISO19139 schema definition supplied with GeoNetwork as possible. We'll now describe in basic steps how to create each of the components of a plugin schema for GeoNetwork that implements the MCP. Creating the schema-ident.xml file ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Now we need to provide the information necessary to identify the schema and metadata records that belong to the schema. The schema-ident.xml file for the MCP is as follows: :: iso19139.mcp 19c9a2b2-dddb-11df-9df4-001c2346de4c 1.5 http://bluenet3.antcrc.utas.edu.au/mcp http://bluenet3.antcrc.utas.edu.au/mcp-1.5-experimental/schema.xsd http://www.isotc211.org/2005/gmd http://www.isotc211.org/2005/gmd/gmd.xsd http://www.isotc211.org/2005/srv http://schemas.opengis.net/iso/19139/20060504/srv/srv.xsd Australian Marine Community Profile of ISO 19115:2005/19139| Marine Community Profile of ISO 19115:2005/19139 1.5-experimental| MCP:BlueNet V1.5-experimental| MCP:BlueNet V1.5 Each of the elements is as follows: - **name** - the name by which the schema will be known in GeoNetwork. If the schema is a profile of a base schema already added to GeoNetwork then the convention is to call the schema .. - **id** - a unique identifier for the schema. - **version** - the version number of the schema. Multiple versions of the schema can be present in GeoNetwork. - **schemaLocation** - a set of pairs, where the first member of the pair is a namespace URI and the second member is the official URL of the XSD. The contents of this element will be added to the root element of any metadata record displayed by GeoNetwork as a schemaLocation/noNamespaceSchemaLocation attribute, if such as attribute does not already exist. It will also be used whenever an official schemaLocation/noNamespaceSchemaLocation is required (eg. in response to a ListMetadataFormats OAI request). - **autodetect** - contains elements or attributes (with content) that must be present in any metadata record that belongs to this schema. This is used during schema detection whenever GeoNetwork receives a metadata record of unknown schema. After creating this file you can validate it manually using the XML schema definition (XSD) in ``INSTALL_DIR/web/geonetwork/xml/validation/schemaPlugins/schema-ident.xsd``. This XSD is also used to validate this file when the schema is loaded. If schema-ident.xml failes validation, the schema will not be loaded. ~~~~~~~~~~~~~~~~~~ More on autodetect ~~~~~~~~~~~~~~~~~~ The autodetect section of schema-ident.xml is used when GeoNetwork needs to identify which metadata schema a record belongs to. The five rules that can be used in this section in order of evaluation are: **1. Attributes** - Find one or more attributes and/or namespaces in the document. An example use case is a profile of ISO19115/19139 that adds optional elements under a new namespace to gmd:identificationInfo/gmd:MD_DataIdentification. To detect records that belong to this profile the autodetect section in the schema-ident.xml file could look something like the following: :: Some other points about attributes autodetect: - multiple attributes can be specified - all must be match for the record to be recognized as belonging to this schema. - if the attributes have a namespace then the namespace should be specified on the autodetect element or somewhere in the schema-ident.xml document. **2. Elements** - Find one or more elements in the document. An example use case is the one shown in the example schema-ident.xml file earlier: :: Australian Marine Community Profile of ISO 19115:2005/19139| Marine Community Profile of ISO 19115:2005/19139 1.5-experimental| MCP:BlueNet V1.5-experimental| MCP:BlueNet V1.5 Some other points about elements autodetect: - multiple elements can be specified - eg. as in the above, both metadataStandardName and metadataStandardVersion have been specified - all must be match for the record to be recognized as belonging to this schema. - multiple values for the elements can be specified. eg. as in the above, a match for gmd:metadataStandardVersion will be found for ``1.5-experimental`` OR ``MCP:BlueNet V1.5-experimental`` OR ``MCP:BlueNet V1.5`` - the vertical line or pipe character '|' is used to separate the options here. - if the elements have a namespace then the namespace(s) should be specified on the autodetect element or somewhere in the schema-ident.xml document before the element in which they are used - eg. in the above there are there namespace declarations on the autodetect element so as not to clutter the content. **3. Root element** - root element of the document must match. An example use case is the one used for the eml-gbif schema. Documents belonging to this schema always have root element of eml:eml so the autodetect section for this schema is: :: Some other points about root element autodetect: - multiple elements can be specified - any element in the set that matches the root element of the record will trigger a match. - if the elements have a namespace then the namespace(s) should be specified on the autodetect element or somewhere in the schema-ident.xml document before the element that uses them - eg. as in the above there is a namespace declaration on the autodetect element for clarity. **4. Namespaces** - Find one or more namespaces in the document. An example use case is the one used for the csw:Record schema. Records belonging to the csw:Record schema can have three possible root elements: csw:Record, csw:SummaryRecord and csw:BriefRecord, but instead of using a multiple element root autodetect, we could use the common csw namespace for autodetect as follows: :: Some other points about namespaces autodetect: - multiple namespaces can be specified - all must be present for the record to be recognized as belonging to this schema. - the prefix is ignored. A namespace match occurs if the namespace URI found in the record matches the namespace URI specified in the namespaces autodetect element. **5. Default schema** - This is the fail-safe provision for records that don't match any of the installed schemas. The value for the default schema is specified in the appHandler configuration of the ``INSTALL_DIR/web/geonetwork/WEB-INF/config.xml`` config file or it could be a default specified by the operation calling autodetect (eg. a value parsed from a user bulk loading some metadata records). For flexibility and accuracy reasons it is preferable that records be detected using the autodetect information of an installed schema. The default schema is just a 'catch all' method of assigning records to a specific schema. The config element in ``INSTALL_DIR/web/geonetwork/WEB-INF/config.xml`` looks like the following: :: ..... ..... ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ More on autodetect evaluation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The rules for autodetect are evaluated as follows: :: for-each autodetect rule type in ( 'attributes/namespaces', 'elements', 'namespaces', 'root element' ) for-each schema if schema has autodetect rule type then check rule for a match if match add to list of previous matches end if end for-each if more than one match throw 'SCHEMA RULE CONFLICT EXCEPTION' if one match then set matched = first match and break loop end for-each if no match then if namespaces of record and default schema overlap then set match = default schema else throw 'NO SCHEMA MATCHES EXCEPTION' end if return matched schema As an example, suppose we have three schemas iso19139.mcp, iso19139.mcp-1.4 and iso19139.mcp-cmar with the following autodetect elements: **iso19139.mcp-1.4:** :: Australian Marine Community Profile of ISO 19115:2005/19139 MCP:BlueNet V1.4 **iso19139.mcp-cmar:** :: **iso19139.mcp:** :: A record going through autodetect processing (eg. on import) would be checked against: - iso19139.mcp-cmar first as it has an 'attributes' rule - then iso19139.mcp-1.4 as it has an 'elements' rules - then finally against iso19139.mcp, as it has a 'root element' rule. The idea behind this processing algorithm is that base schemas will use a 'root element' rule (or the more difficult to control 'namespaces' rule) and profiles will use a finer or more specific rule such as 'attributes' or 'elements'. After setting up schema-ident.xml, our new GeoNetwork plugin schema for MCP contains: :: schema-ident.xml .. _schema_conversions: Creating the schema-conversions.xml file ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This file describes the converters that can be applied to metadata records that belong to the schema. Each converter must be manually defined as a GeoNetwork (Jeeves) service that can be called to transform a particular metadata record to a different schema. The schema-conversions.xml file for the MCP is as follows: :: Each converter has the following attributes: - **name** - the name of the converter. This is the service name of the GeoNetwork (Jeeves) service and should be unique (prefixing the service name with xml_ is a good way to make this name unique). - **nsUri** - the primary namespace of the schema produced by the converter. eg. xml_iso19139.mcpTorifcs transforms metadata records from iso19139.mcp to the RIFCS schema. Metadata records in the RIFCS metadata schema have primary namespace URI of http://ands.org.au/standards/rif-cs/registryObjects. - **schemaLocation** - the location (URL) of the XML schema definition (XSD) corresponding to the nsURI. - **xslt** - the name of the XSLT that actually carries out the transformation. This XSLT should be located in the convert subdirectory of the schema plugin. After setting up schema-conversions.xml, our new GeoNetwork plugin schema for MCP contains: :: schema-conversions.xml schema-ident.xml .. _schema_and_schema_xsd: Creating the schema directory and schema.xsd file ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The schema and schema.xsd components are used by the GeoNetwork editor and validation functions. GeoNetwork's editor uses the XSDs to build a form that will not only order the elements in a metadata document correctly but also offer options to create any elements that are not in the metadata document. The idea behind this approach is twofold. Firstly, the editor can use the XML schema definition rules to help the user avoid creating a document that is structurally incorrect eg. missing mandatory elements or elements not ordered correctly. Secondly, the same editor code can be used on any XML metadata document with a defined XSD. If you are defining your own metadata schema then you can create an XML schema document using the XSD language. The elements of the language can be found online at http://www.w3schools.com/schema/ or you can refer to a textbook such as Priscilla Walmsley's Definitive XML Schema (Prentice Hall, 2002). GeoNetwork's XML schema parsing code understands almost all of the XSD language with the exception of redefine, any and anyAttribute (although the last two can be handled under special circumstances). In the case of the Marine Commuity Profile, we are basically defining a number of extensions to the base standard ISO19115/19139. These extensions are defined using the XSD extension mechanism on the types defined in ISO19139. The following snippet shows how the Marine Community Profile extends the gmd:MD_Metadata element to add a new element called revisionDate:: .... Extends the metadata element to include revisionDate In short, we have defined a new element mcp:MD_Metadata with type mcp:MD_Metadata_Type, which is an extension of gmd:MD_Metadata_Type. By extension, we mean that the new type includes all of the elements of the old type plus one new element, mcp:revisionDate. A mandatory attribute (gco:isoType) is also attached to mcp:MD_Metadata with a fixed value set to the name of the element that we extended (gmd:MD_Metadata). By defining the profile in this way, it is not necessary to modify the underlying ISO19139 schemas. So the schema directory for the MCP essentially consists of the extensions plus the base ISO19139 schemas. One possible directory structure is as follows: :: extensions gco gmd gml gmx gsr gss gts resources srv xlink The extensions directory contains a single file mcpExtensions.xsd, which imports the gmd namespace. The remaining directories are the ISO19139 base schemas. The schema.xsd file, which is the file GeoNetwork looks for, will import the mcpExtensions.xsd file and any other namespaces not imported as part of the base ISO19139 schema. It is shown as follows: :: At this stage, our new GeoNetwork plugin schema for MCP contains: :: schema-conversions.xml schema-ident.xml schema.xsd schema Creating the extract-... XSLTs ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ GeoNetwork needs to extract certain information from a metadata record and translate it into a common, simplified XML structure that is independent of the metadata schema. Rather than do this with Java coded XPaths, XSLTs are used to process the XML and return the common, simplified XML structure. The three xslts we'll create are: - **extract-date-modified.xsl** - this XSLT processes the metadata record and extracts the date the metadata record was last modified. For the MCP, this information is held in the mcp:revisionDate element which is a child of mcp:MD_Metadata. The easiest way to create this for MCP is to copy extract-date-modified.xsl from the iso19139 schema and modify it to suit the MCP namespace and to use mcp:revisionDate in place of gmd:dateStamp. - **extract-gml.xsl** - this XSLT processes the metadata record and extracts the spatial extent as a gml GeometryCollection element. The gml is passed to geotools for insertion into the spatial index (either a shapefile or a spatial database). For ISO19115/19139 and profiles, this task is quite easy because spatial extents (apart from the bounding box) are encoded as gml in the metadata record. Again, the easiest way to create this for the MCP is to copy extract-gml.xsd from the iso19139 schema ad modify it to suit the MCP namespace. An example bounding box fragment from an MCP metadata record is: :: 112.9 153.64 -43.8 -9.0 Running extract-gml.xsl on the metadata record that contains this XML will produce: :: 112.9,-9.0, 153.64,-9.0, 153.64,-43.8, 112.9,-43.8, 112.9,-9.0 If there is more than one extent in the metadata record, then they should also appear in this gml:GeometryCollection element. To find out more about gml, see Lake, Burggraf, Trninic and Rae, "GML Geography Mark-Up Language, Foundation for the Geo-Web", Wiley, 2004. Finally, a note on projections. It is possible to have bounding polygons in an MCP record in a projection other than EPSG:4326. GeoNetwork transforms all projections known to GeoTools (and encoded in a form that GeoTools understands) to EPSG:4326 when writing the spatial extents to the shapefile or spatial database. - **extract-uuid.xsl** - this XSLT processes the metadata record and extracts the identifier for the record. For the MCP and base ISO standard, this information is held in the gmd:fileIdentifier element which is a child of mcp:MD_Metadata. These xslts can be tested by running them on a metadata record from the schema. You should use the saxon xslt processor. For example: :: java -jar INSTALL_DIR/web/geonetwork/WEB-INF/lib/saxon-9.1.0.8b-patch.jar -s testmcp.xml -o output.xml extract-gml.xsl At this stage, our new GeoNetwork plugin schema for MCP contains: :: extract-date-modified.xsl extract-gml.xsd extract-uuid.xsl schema-conversions.xml schema-ident.xml schema.xsd schema Creating the localized strings in the loc directory ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The loc directory contains localized strings specific to this schema, arranged by language abbreviation in sub-directories. You should provide localized strings in whatever languages you expect your schema to be used in. Localized strings for this schema can be used in the presentation xslts and schematron error messages. For the presentation xslts: - codelists for controlled vocabulary fields should be in loc//codelists.xml eg. ``loc/eng/codelists.xml`` - label strings that replace XML element names with more intelligible/alternative phrases and rollover help strings should be in loc//labels.xml eg. ``loc/eng/labels.xml``. - all other localized strings should be in loc//strings.xml eg. ``loc/eng/strings.xml`` Note that because the MCP is a profile of ISO19115/19139 and we have followed the GeoNetwork naming convention for profiles, we need only include the labels and codelists that are specific to the MCP or that we want to override. Other labels and codelists will be retrieved from the base schema iso19139. ~~~~~~~~~~~~~~~~~~~~~ More on codelists.xml ~~~~~~~~~~~~~~~~~~~~~ Typically codelists are generated from enumerated lists in the metadata schema XSDs such as the following from http://www.isotc211.org/2005/gmd/identification.xsd for gmd:MD_TopicCategoryCode in the iso19139 schema: :: The following is part of the codelists.xml entry manually created for this element: :: farming Rearing of animals and/or cultivation of plants. Examples: agriculture, irrigation, aquaculture, plantations, herding, pests and diseases affecting crops and livestock biota Flora and/or fauna in natural environment. Examples: wildlife, vegetation, biological sciences, ecology, wilderness, sealife, wetlands, habitat boundaries Legal land descriptions. Examples: political and administrative boundaries ..... The codelists.xml file maps the enumerated values from the XSD to a localized label and a description via the code element. A localized copy of codelists.xml is made available on an XPath to the presentation XSLTs eg. /root/gui/schemas/iso19139/codelist for the iso19139 schema. The XSLT metadata.xsl which contains templates used by all metadata schema presentation XSLTs, handles the creation of a select list/drop down menu in the editor and display of the code and description in the metadata viewer. The iso19139 schema has additional codelists that are managed external to the XSDs in catalog/vocabulary files such as http://www.isotc211.org/2005/resources/Codelist/gmxCodelists.xml These have also been added to the codelists.xml file so that they can be localized, overridden in profiles and include an extended description to provide more useful information when viewing the metadata record. The iso19139 schema has additional templates in its presentation xslts to handlese codelists because they are specific to that schema. These are discussed in the section on presentation XSLTs later in this manual. ~~~~~~~~~~~~~~~~~~ More on labels.xml ~~~~~~~~~~~~~~~~~~ A localized copy of labels.xml is made available on an XPath to the presentation XSLTs eg. /root/gui/schemas/iso19139/labels for the iso19139 schema. The ``labels.xml`` file can also be used to provide helper values in the form of a drop down/select list for free text fields. As an example: :: Recognition of those who contributed to the resource(s) This would result in the Editor (through the XSLT metadata.xsl) displaying the credit field with these helper options listed beside it in a drop down/select menu something like the following: .. figure:: Editor-Helpers.png ~~~~~~~~~~~~~~~~~~~ More on strings.xml ~~~~~~~~~~~~~~~~~~~ A localized copy of ``strings.xml`` is made available on an XPath to the presentation XSLTs eg. /root/gui/schemas/iso19139/strings for the iso19139 schema. After adding the localized strings, our new GeoNetwork plugin schema for MCP contains: :: extract-date-modified.xsl extract-gml.xsd extract-uuid.xsl loc present schema-conversions.xml schema-ident.xml schema.xsd schema Creating the presentations XSLTs in the present directory ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each metadata schema should contain XSLTs that display and possibly edit metadata records that belong to the schema. These XSLTs are held in the ``present`` directory. To be be used in the XSLT include/import hierarchy these XSLTs must follow a naming convention: metadata-.xsl. So for example, the presentation xslt for the iso19139 schema is ``metadata-iso19139.xsl``. For the MCP, since our schema name is iso19139.mcp, the presentation XSLT would be called ``metadata-iso19193.mcp.xsl``. Any XSLTs included by the presentation XSLT should also be in the present directory (this is a convention for clarity - it is not mandatory as include/import URLs can be mapped in the oasis-catalog.xml for the schema to other locations). There are certain XSLT templates that the presentation XSLT must have: - the **main** template, which must be called: metadata-. For the MCP profile of iso19139 the main template would look like the following (taken from metadata-iso19139.mcp.xsl): :: Analyzing this template: #. The name="metadata-iso19139.mcp" is used by the main element processing template in metadata.xsl: elementEP. The main metadata services, show and edit, end up calling metadata-show.xsl and metadata-edit.xsl respectively with the metadata record passed from the Java service. Both these XSLTs, process the metadata record by applying the elementEP template from metadata.xsl to the root element. The elementEP template calls this main schema template using the schema name iso19139.mcp. #. The job of this main template is set to process all the elements of the metadata record using templates declared with a mode name that matches the schema name or the name of the base schema (in this case iso19139). This modal processing is to ensure that only templates intended to process metadata elements from this schema or the base schema are applied. The reason for this is that almost all profiles change or add a small number of elements to those in the base schema. So most of the metadata elements in a profile can be processed in the mode of the base schema. We'll see later in this section how to override processing of an element in the base schema. - a **completeTab** template, which must be called: CompleteTab. This template will display all tabs, apart from the 'default' (or simple mode) and the 'XML View' tabs, in the left hand frame of the editor/viewer screen. Here is an example for the MCP: :: ...... (same as for iso19139CompleteTab in GEONETWORK_DATA_DIR/schema_plugins/iso19139/present/ metadata-iso19139.xsl) ...... This template is called by the template named "tab" (which also adds the "default" and "XML View" tabs) in ``INSTALL_DIR/web/geonetwork/xsl/metadata-tab-utils.xsl`` using the schema name. That XSLT also has the code for the "displayTab" template. 'mcpMinimum', 'mcpCore', 'complete' etc are the names of the tabs. The name of the current or active tab is stored in the global variable "currTab" available to all presentation XSLTs. Logic to decide what to display when a particular tab is active should be contained in the root element processing tab. - a **root element** processing tab. This tab should match on the root element of the metadata record. For example, for the iso19139 schema: :: ......... This template is basically a very long "choose" statement with "when" clauses that test the value of the currently defined tab (in global variable currTab). Each "when" clause will display the set of metadata elements that correspond to the tab definition using "elementEP" directly (as in the "when" clause for the 'identification' tab above) or via a named template (as in the 'metadata' tab above). For the MCP our template is similar to the one above for iso19139, except that the match would be on "mcp:MD_Metadata" (and the processing mode may differ - see the section 'An alternative XSLT design for profiles' below for more details). - a **brief** template, which must be called: Brief. This template processes the metadata record and extracts from it a format neutral summary of the metadata for purposes such as displaying the search results. Here is an example for the eml-gbif schema (because it is fairly short!): :: <xsl:value-of select="normalize-space(dataset/title[1])"/> Analyzing this template: #. The template matches on an element eml-gbifBrief, created by the mode="brief" template in metadata-utils.xsl. The metadata record will be the first child in the /metadata XPath. #. Then process metadata elements to produce a flat XML structure that is used by search-results-xhtml.xsl to display a summary of the metadata record found by a search. Once again, for profiles of an existing schema, it makes sense to use a slighlty different approach so that the profile need not duplicate templates. Here is an example from metadata-iso19139.mcp.xsl: :: This template splits the processing between the base iso19139 schema and a brief template that handles elements specific to the profile. This assumes that: #. The base schema has separated the element from the remainder of its brief processing so that it can be called by profiles #. The profile includes links to equivalent elements that can be used by the base schema to process common elements eg. for ISO19139, elements in the profile have gco:isoType attributes that give the name of the base element and can be used in XPath matches such as "gmd:MD_DataIdentification|*[@gco:isoType='gmd:MD_DataIdentification']". - templates that match on elements specific to the schema. Here is an example from the eml-gbif schema: :: , ( ) Analyzing this template: #. In view mode the individual keywords from the set are concatenated into a comma separated string with the name of the thesaurus in brackets at the end. #. In edit mode, the keywordSet is handled as a complex element ie. the user can add individual keyword elements with content and a single thesaurus name. #. This is an example of the type of processing that can be done on an element in a metadata record. For profiles, templates for elements can be defined in the same way except that the template will process in the mode of the base schema. Here is an example showing the first few lines of a template for processing the mcp:revisionDate element: :: ...... If a template for a profile is intended to override a template in the base schema, then the template can be defined in the presentation XSLT for the profile with a priority attribute set to a high number and an XPath condition that ensures the template is processed for the profile only. For example in the MCP, we can override the handling of gmd:EX_GeographicBoundingBox in metadata-iso19139.xsl by defining a template in metadata-iso19139.mcp.xsl as follows: :: ...... Finally, a profile may also extend some of the existing codelists in the base schema. These extended codelists should be held in a localized codelists.xml. As an example, in iso19139 these codelists are often attached to elements like the following: :: custodian Templates for handling these elements are in the iso19139 presentation XSLT ``GEONETWORK_DATA_DIR/schema_plugins/iso19139/present/metadata-iso19139.xsl``. These templates use the name of the element (eg. gmd:CI_RoleCode) and the codelist XPath (eg. /root/gui/schemas/iso19139/codelists) to build select list/drop down menus when editing and to display a full description when viewing. See templates near the template named 'iso19139Codelist'. These templates can handle the extended codelists for any profile because they: - match on any element that has a child element with attribute codeList - use the schema name in the codelists XPath - fall back to the base iso19139 schema if the profile codelist doesn't have the required codelist However, if you don't need localized codelists, it is often easier and more direct to extract codelists directly from the ``gmxCodelists.xml`` file. This is in fact the solution that has been adopted for the MCP. The ``gmxCodelists.xml`` file is included in the presentation xslt for the MCP using a statement like: :: Check the codelist handling templates in ``metadata-iso19139.mcp.xsl`` to see how this works. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An alternative XSLT design for profiles ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In all powerful languages there will be more than one way to achieve a particular goal. This alternative XSLT design is for processing profiles. The idea behind the alternative is based on the following observations about the GeoNetwork XSLTs: #. All elements are initially processed by apply-templates in mode "elementEP". #. The template "elementEP" (see ``INSTALL_DIR/web/geonetwork/xsl/metadata.xsl``) eventually calls the **main** template of the schema/profile. #. The main template can initially process the element in a mode particular to the profile and if this is not successful (ie. no template matches and thus no HTML elements are returned), process the element in the mode of the base schema. The advantage of this design is that overriding a template for an element in the base schema does not need the priority attribute or an XPath condition check on the schema name. Here is an example for the MCP (iso19139.mcp) with base schema iso19139: - the **main** template, which must be called: metadata-iso19139.mcp.xsl: :: Analyzing this template: #. The name="metadata-iso19139.mcp" is used by the main element processing template in metadata.xsl: elementEP. The main metadata services, show and edit, end up calling metadata-show.xsl and metadata-edit.xsl respectively with the metadata record passed from the Java service. Both these XSLTs, process the metadata record by applying the elementEP template from metadata.xsl to the root element. elementEP calls the appropriate main schema template using the schema name. #. The job of this main template is set to process all the elements of the metadata profile. The processing takes place in one of two modes. Firstly, the element is processed in the profile mode (iso19139.mcp). If a match is found then HTML elements will be returned and copied to the output document. If no HTML elements are returned then the element is processed in the base schema mode, iso19139. - templates that match on elements specific to the profile have mode iso19139.mcp: :: ..... - templates that override elements in the base schema are processed in the profile mode iso19139.mcp :: ..... Notice that the template header of the profile has a simpler design than that used for the original design? Neither the priority attribute or the schema XPath condition is required because we are using a different mode to the base schema. - To support processing in two modes we need to add a null template to the profile mode iso19139.mcp as follows: :: This template will match all elements that we don't have a specific template for in the profile mode iso19139.mcp. These elements will be processed in the base schema mode iso19139 instead because the null template returns nothing (see the main template discussion above). The remainder of the discussion in the original design relating to tabs etc applies to the alternative design and is not repeated here. ~~~~~~~~~~~~~~~~~~~~~~ CSW Presentation XSLTs ~~~~~~~~~~~~~~~~~~~~~~ The CSW server can be asked to provide records in a number of output schemas. The two supported by GeoNetwork are: - **ogc** - http://www.opengis.net/cat/csw/2.0.2 - a dublin core derivative - **iso** - http://www.isotc211.org/2005/gmd - ISO19115/19139 From each of these output schemas, a **brief**, **summary** or **full** element set can be requested. These output schemas and element sets are implemented in GeoNetwork as XSLTs and they are stored in the 'csw' subdirectory of the 'present' directory. The ogc output schema XSLTs are implemented as ogc-brief.xsl, ogc-summary and ogc-full.xsl. The iso output schema XSLTs are implemented as iso-brief.xsl, iso-summary.xsl and iso-full.xsl. To create these XSLTs for the MCP, the best option is to copy and modify the csw presentation XSLTs from the base schema iso19139. After creating the presentation XSLTs, our new GeoNetwork plugin schema for MCP contains: :: extract-date-modified.xsl extract-gml.xsd extract-uuid.xsl loc present schema-conversions.xml schema-ident.xml schema.xsd schema Creating the index-fields.xsl to index content from the metadata record ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This XSLT indexes the content of elements in the metadata record. The essence of this XSLT is to select elements from the metadata record and map them to lucene index field names. The lucene index field names used in GeoNetwork are as follows: =========================== =========================================================================== Lucene Index Field Name Description =========================== =========================================================================== abstract Metadata abstract any Content from all metadata elements (for free text) changeDate Date that the metadata record was modified createDate Date that the metadata record was created denominator Scale denominator in data resolution download Does the metadata record have a downloadable resource attached? (0 or 1) digital Is the metadata record distributed/available in a digital format? (0 or 1) eastBL East bounding box longitude keyword Metadata keywords metadataStandardName Metadata standard name northBL North bounding box latitude operatesOn Uuid of metadata record describing dataset that is operated on by a service orgName Name of organisation listed in point-of-contact information parentUuid Uuid of parent metadata record paper Is the metadata record distributed/available in a paper format? (0 or 1) protocol On line resource access protocol publicationDate Date resource was published southBL South bounding box latitude spatialRepresentationType vector, raster, etc tempExtentBegin Beginning of temporal extent range tempExtentEnd End of temporal extent range title Metadata title topicCat Metadata topic category type Metadata hierarchy level (should be dataset if unknown) westBL West bounding box longitude =========================== =========================================================================== For example, here is the mapping created between the metadata element mcp:revisionDate and the lucene index field changeDate: :: Notice that we are creating a new XML document. The Field elements in this document are read by GeoNetwork to create a Lucene document object for indexing (see the SearchManager class in the GeoNetwork source). Once again, because the MCP is a profile of ISO19115/19139, it is probably best to modify ``index-fields.xsl`` from the schema iso19139 to handle the namespaces and additional elements of the MCP. At this stage, our new GeoNetwork plugin schema for MCP contains: :: extract-date-modified.xsl extract-gml.xsd extract-uuid.xsl index-fields.xsl loc present schema-conversions.xml schema-ident.xml schema.xsd schema Creating the sample-data directory ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This is a simple directory. Put MEF files with sample metadata in this directory. Make sure they have a ``.mef`` suffix. A MEF file is a zip archive with the metadata, thumbnails, file based online resources and an info file describing the contents. The contents of a MEF file are discussed in more detail in the next section of this manual. Sample data in this directory can be added to the catalog using the Administration menu. At this stage, our new GeoNetwork plugin schema for MCP contains: :: extract-date-modified.xsl extract-gml.xsd extract-uuid.xsl index-fields.xsl loc present sample-data schema-ident.xml schema.xsd schema Creating schematrons to describe MCP conditions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Schematrons are rules that are used to check conditions and content in the metadata record as part of the two stage validation process used by GeoNetwork. Schematron rules are created in the schematrons directory that you checked out earlier - see :ref:`preparation` above. An example rule is: :: $loc/strings/M30 $loc/strings/alert.M30 $loc/strings/report.M30 As for most of GeoNetwork, the output of this rule can be localized to different languages. The corresponding localized strings are: :: ..... [ISOFTDS19139:2005-TableA1-Row15] - Check point description required if available .....
'checkPointDescription' is mandatory if 'checkPointAvailability' = 1 or true.
 ..... Check point description documented. ..... Procedure for adding schematron rules, working within the schematrons directory: #. Place your schematron rules in 'rules'. Naming convetion is 'schematron-rules-.sch' eg. ``schematron-rules-iso-mcp.sch``. Place localized strings for the rule assertions into 'rules/loc/'. #. Edit ``build.xml``. #. Add a "clean-schema-dir" target for your plugin schema directory. This target will remove the schematron rules from plugin schema directory (basically removes all files with pattern schematron-rules-\*.xsl). #. Add a "compile-schematron" target for your rules - value attribute is the suffix used in the rules name. eg. for ``schematron-rules-iso-mcp.sch`` the value attribute should be "iso-mcp". This target will turn the .sch schematron rules into an XSLT using the saxon XSLT engine and 'resources/iso_svrl_for_xslt2.xsl'. #. Add a "publish-schematron" target. This target copies the compiled rules (in XSLT form) into the plugin schema directory. #. Run 'ant' to create the schematron XSLTs. At this stage, our new GeoNetwork plugin schema for MCP contains: :: extract-date-modified.xsl extract-gml.xsd extract-uuid.xsl index-fields.xsl loc present sample-data schema-conversions.xml schema-ident.xml schema.xsd schema schematron-rules-iso-mcp.xsl Adding the components necessary to create and edit MCP metadata ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ So far we have added all the components necessary for GeoNetwork to identify, view and validate MCP metadata records. Now we will add the remaining components necessary to create and edit MCP metadata records. We'll start with the XSLTs that set the content of various elements in the MCP metadata records. ~~~~~~~~~~~~~~~~~~~~~ Creating set-uuid.xsl ~~~~~~~~~~~~~~~~~~~~~ - **set-uuid.xsl** - this XSLT takes as a parameter the UUID of the metadata record and writes it into the appropriate element of the metadata record. For the MCP this element is the same as the base ISO schema (called iso19139 in GeoNetwork), namely gmd:fileIdentifier. However, because the MCP uses a different namespace on the root element, this XSLT needs to be modified. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Creating the extract, set and unset thumbnail XSLTs ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If your metadata record can have a thumbnail or browse graphic link, then you will want to add XSLTs that extract, set and unset this information so that you can use the GeoNetwork thumbnail editing interface. The three XSLTs that support this interface are: - **extract-thumbnails.xsl** - this XSLT extracts the thumbnails/browse graphics from the metadata record, turning it into generic XML that is the same for all metadata schemas. The elements need to have content that GeoNetwork understands. The following is an example of what the thumbnail interface for iso19139 expects (we'll duplicate this requirement for MCP): :: bluenet_s.png thumbnail png bluenet.png large_thumbnail png When ``extract-thumbnails.xsl`` is run, it creates a small XML hierarchy from this information which looks something like the following: :: bluenet.png bluenet_s.png - **set-thumbnail.xsl** - this XSLT does the opposite of extract-thumbnails.xsl. It takes the simplified, common XML structure used by GeoNetwork to describe the large and small thumbnails and creates the elements of the metadata record that are needed to represent them. This is a slightly more complex XSLT than extract-thumbnails.xsl because the existing elements in the metadata record need to be retained and the new elements need to be created in their correct places. - **unset-thumbnail.xsl** - this XSLT targets and removes elements of the metadata record that describe a particular thumbnail. The remaining elements of the metadata record are retained. Because the MCP is a profile of ISO19115/19139, the easiest path to creating these XSLTs is to copy them from the iso19139 schema and modify them for the changes in namespace required by the MCP. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Creating the update-... XSLTs ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - **update-child-from-parent-info.xsl** - this XSLT is run when a child record needs to have content copied into it from a parent record. It is an XSLT that changes the content of a few elements and leaves the remaining elements untouched. The behaviour of this XSLT would depend on which elements of the parent record will be used to update elements of the child record. - **update-fixed-info.xsl** - this XSLT is run after editing to fix certain elements and content in the metadata record. For the MCP there are a number of actions we would like to take to 'hard-wire' certain elements and content. To do this the XSLT the following processing logic: :: if the element is one that we want to process then add a template with a match condition for that element and process it else copy the element to output Because the MCP is a profile of ISO19115/19139, the easiest path to creating this XSLT is to copy update-fixed-info.xsl from the iso19139 schema and modify it for the changes in namespace required by the MCP and then to include the processing we want. A simple example of MCP processing is to make sure that the gmd:metadataStandardName and gmd:metadataStandardVersion elements have the content needed to ensure that the record is recognized as MCP. To do this we can add two templates as follows: :: Australian Marine Community Profile of ISO 19115:2005/19139 MCP:BlueNet V1.5 Processing by ``update-fixed-info.xsl`` can be enabled/disabled using the *Automatic Fixes* check box in the System Configuration menu. By default, it is enabled. Some important tasks handled in ``upgrade-fixed-info.xsl``: - creating URLs for metadata with attached files (eg. onlineResources with 'File for download' in iso19139) - setting date stamp/revision date - setting codelist URLs to point to online ISO codelist catalogs - adding default spatial reference system attributes to spatial extents A specific task required for the MCP ``update-fixed-info.xsl`` was to automatically create an online resource with a URL pointing to the metadata.show service with parameter set to the metadata uuid. This required some changes to the update-fixed-info.xsl supplied with iso19139. In particular: - the parent elements may not be present in the metadata record - processing of the online resource elements for the metadata point of truth URL should not interfere with other processing of online resource elements Rather than describe the individual steps required to implement this and the decisions required in the XSLT language, take a look at the ``update-fixed-info.xsl`` already present for the MCP schema in the iso19139.mcp directory and refer to the dot points above. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Creating the templates directory ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This is a simple directory. Put XML metadata files to be used as templates in this directory. Make sure they have a ``.xml`` suffix. Templates in this directory can be added to the catalog using the Administration menu. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Editor behaviour: Adding schema-suggestions.xml and schema-substitutes.xml ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - **schema-suggestions.xml** - The default behaviour of the GeoNetwork advanced editor when building the editor forms is to show elements that are not in the metadata record as unexpanded elements. To add these elements to the record, the user will have to click on the '+' icon next to the element name. This can be tedious especially as some metadata standards have elements nested in others (ie. complex elements). The schema-suggestions.xml file allows you to specify elements that should be automatically expanded by the editor. An example of this is the online resource information in the ISO19115/19139 standard. If the following XML was added to the ``schema-suggestions.xml`` file: :: The effect of this would be that when an online resource element was expanded, then input fields for the protocol (a drop down/select list), name and description would automatically appear in the editor. Once again, a good place to start when building a ``schema-suggestions.xml`` file for the MCP is the ``schema-suggestions.xml`` file for the iso19139 schema. - **schema-substitutes.xml** - Recall from the earlier :ref:`schema_and_schema_xsd` section, that the method we used to extend the base ISO19115/19139 schemas is to extend the base type, define a new element with the extended base type and allow the new element to substitute for the base element. So for example, in the MCP, we want to add a new resource constraint element that holds Creative Commons and other commons type licensing information. This requires that the MD_Constraints type be extended and a new mcp:MD_Commons element be defined which can substitute for gmd:MD_Constraints. This is shown in the following snippet of XSD: :: Add MD_Commons as an extension of gmd:MD_Constraints_Type For MCP records, the GeoNetwork editor will show a choice of elements from the substitution group for gmd:MD_Constraints when adding 'Resource Constraints' to the metadata document. This will now include mcp:MD_Commons. .. figure:: Editor-Constraints-Choices.png Note that by similar process, two other elements, now deprecated in favour of MD_Commons, were also added as substitutes for MD_Constraints. If it was necessary to constrain the choices shown in this menu, say to remove the deprecated elements and limit the choices to just legal, security and commons, then this can be done by the following piece of XML in the schema-substitutes.xml file: :: The result of this change is shown below. .. figure:: Editor-Constraints-Choices-Modified.png Once again, a good place to start when building a schema-substitutes.xml file for the MCP is the schema-substitutes.xml file for the iso19139 schema. Adding components to support conversion of metadata records to other schemas ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Creating the convert directory ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the new GeoNetwork plugin schema is to support on the fly translation of metadata records to other schemas, then the convert directory should be created and populated with appropriate XSLTs. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Supporting OAIPMH conversions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The OAIPMH server in GeoNetwork can deliver metadata records from any of the schemas known to GeoNetwork. It can also be configured to deliver schemas not known to GeoNetwork if an XSLT exists to convert a metadata record to that schema. The file ``INSTALL_DIR/web/geonetwork/WEB-INF/config-oai-prefixes.xml`` describes the schemas (known as prefixes in OAI speak) that can be produced by an XSLT. A simple example of the content of this file is shown below: :: In the case of the prefix oai_dc shown above, if a schema converter with prefix *oai_dc* exists in the ``schema-conversions.xml`` file of a GeoNetwork schema, then records that belong to this schema will be transformed and included in OAIPMH requests for the *oai_dc* prefix. See :ref:`schema_conversions` for more info. To add oai_dc support for the MCP, the easiest method is to copy oai_dc.xsl from the convert directory of the iso19139 schema, modify it to cope with the different namespaces and additional elements of the MCP and add it to the ``schema-conversions.xml`` file for the MCP.