Blog post about the use of Yaml2Text to encode ModSpec instances

_posts/2024-04-04-ogc-modspec-in-yaml.adoc

@@ -0,0 +1,348 @@
+---
+layout: post
+title: "Encode OGC ModSpec using `yaml2text` templates"
+date: 2024-04-04
+categories: documentation
+
+authors:
+  - name: Manuel Fuenmayor
+    email: [email protected]
+    use_picture: assets
+    social_links:
+    - https://github.com/manuelfuenmayor
+
+excerpt: >-
+  Take advantage of the `yaml2text` template capabilities to encode large
+  amounts of ModSpec instances in fewer lines of markup.
+---
+
+== Purpose
+
+OGC standards use ModSpec to encode requirements, and sometimes there are a lot
+of them. ISO/TC 211 has also begun to encode requirements in the OGC ModSpec
+fashion.
+
+`yaml2text` is a Metanorma plugin that allows you to encode large amounts of
+data that share the same structure in a reduced number of lines via pre-defined
+template. The data is to be arranged in YAML format, and the template, written
+in Liquid.
+
+The main goal of this article is to introduce you on the application of
+`yaml2text` to encode ModSpec instances.
+
+To read this article you have to be familiar with the encoding basics of
+`yaml2text` and ModSpec instances in Metanorma. For that matter, it is
+recommended to first read these before continuing:
+
+* link:/author/topics/blocks/requirements-modspec/[OGC ModSpec]
+* link:/author/topics/automation/yaml_to_text/[`yaml2text` plugin]
+
+
+== Encoding requirements with `yaml2text`
+
+In order to ensure an expeditious process in the use of `yaml2text`,
+and to avoid code repetition, follow the next steps:
+
+. Place and arrange all the requirements data into a YAML file.
+
+. Write the template in Liquid and save it in a separate `.liquid` file.
+
+. Create the `yaml2text` block in the Metanorma document specifying
+the corresponding YAML file, and including the liquid template using
+the `include::` directive.
+
+. Compile the document to test the correct rendering of the requirements;
+debug if necessary.
+
+Now, let's look at two examples: a simple one and larger one.
+
+
+=== Encoding a simple requirement
+
+link:/author/topics/blocks/requirements-modspec/[OGC ModSpec instances] are
+typically encoded as a definition list.
+
+NOTE: There are two methods to encode requirements: as a definition list or as
+attributes. We adopt the recommended practice of the definition list here.
+
+In this example, we want to encode the following **Requirement** using `yaml2text`.
+
+[[simple-req]]
+.Sample requirement to be encoded
+|===
+2+^|*Requirement 1*
+
+|Identifier |`/req/relief/classes`
+|Statement | For each UML class defined or referenced in the Relief Package:
+|A | The Implementation Specification SHALL contain an element which represents the
+same concept as that defined for the UML class.
+|B | The Implementation Specification SHALL represent associations with the same
+source, target, direction, roles, and multiplicities as those of the UML class.
+|===
+
+First, we define the *data file*.
+The data file represents the requirement using a fixed structure in YAML.
+Let's call it `data.yaml`.
+
+.YAML file `data.yaml` representing the <<simple-req>>
+[source,yaml]
+----
+---
+identifier: /req/relief/classes
+statement: "For each UML class defined or referenced in the Relief Package:"
+parts:
+- The Implementation Specification SHALL contain an element which represents the
+same concept as that defined for the UML class.
+- The Implementation Specification SHALL represent associations with the same
+source, target, direction, roles, and multiplicities as those of the UML class.
+----
+
+In YAML, data is represented using key-value pairs. Also note that we used
+array representation for the `parts` field. This is how is done when we have
+several elements mapped to a single field.
+
+Once we have our data properly structured in YAML, we proceed to write the
+template in Liquid.
+
+We could write our liquid template directly in the `yaml2text` block,
+but it is good practice to do so in a separate file, the *template file*.
+Let's call this file `template.liquid`.
+
+`yaml2text` requires naming a `context` variable that will represent the
+totality of the data saved in the YAML file. Let's call this variable `context`.
+
+Having all set, the template is defined as follows:
+
+.Template file `template.liquid` for rendering the <<simple-req>>
+[source,liquid]
+----
+[requirement]
+====
+[%metadata]
+identifier:: {{ context.identifier }}
+statement:: {{ context.statement }}
+
+{% for part in context.parts %}
+part:: {{ part }}
+{% endfor %}
+====
+----
+
+[NOTE]
+====
+In Liquid, arrays are typically handled with _for_ loops:
+
+[source%unnumbered,liquid]
+----
+{% for element in elements %}
+//... content ...
+{% endfor %}
+----
+====
+
+With the *data file* and the *template file*, we proceed to create the
+`yaml2text` block in our Metanorma document:
+
+[[simple-req-yaml2text]]
+.Definition of the `yaml2text` block encoding the <<simple-req>>
+[source,asciidoc]
+----
+[yaml2text,data.yaml,context] <1>
+--
+\include::template.liquid[] <2>
+--
+----
+<1> The *data file* `data.yaml` is passed into the block.
+<2> The *template file* `template.liquid` receives the `context` variable
+from the block.
+
+Here, we have assumed that `data.yaml` and `template.liquid` are in the same
+location as the Metanorma document. Remember that the path to these files is
+calculated based on relative location.
+
+At this point, we can compile the document to check if the requirement
+renders correctly. Note that for such small template, we could place the code right
+inside of the `yaml2text` block without the need for the `include` directive.
+But we do this mainly to avoid code repetition in subsequent blocks.
+
+Once Metanorma processes the Liquid template, the `yaml2text` block
+will result in this content:
+
+.Output of the `yaml2text` processed block
+[source,asciidoc]
+----
+[requirement]
+====
+[%metadata]
+identifier:: /req/relief/classes
+statement:: For each UML class defined or referenced in the Relief Package:
+part:: The Implementation Specification SHALL contain an element which represents the
+same concept as that defined for the UML class.
+part:: The Implementation Specification SHALL represent associations with the same
+source, target, direction, roles, and multiplicities as those of the UML class.
+====
+----
+
+That's it! The process to encode a requirement using `yaml2text` is that simple.
+
+Now, let's investigate a more complex example.
+
+
+=== Encoding a Conformance class with embedded Conformance tests
+
+In ModSpec, **Conformance class**es contains **Conformance test**s.
+
+The challenge in managing them is that the Conformance class links to individual
+Conformance tests, the individual Conformance tests also have to link back to
+the Conformance class. Hence we opt to encode all of them a single YAML file.
+
+Let's encode a Conformance class that is already defined by this YAML markup.
+
+NOTE: This is a real example from the source files of
+the published https://www.iso.org/standard/80874.html[ISO 19115-3:2023].
+
+[[cc-ex-yaml]]
+.Data file `data.yaml` of a Conformance class instance arranged in YAML format
+[source,yaml]
+----
+---
+conformance_classes:
+- name: Validation of XML instance for metadata basic information
+  identifier: https://standards.isotc211.org/19115/-1/1/conf/metadata-xml/basic
+  target: https://standards.isotc211.org/19115/-1/1/req/metadata-xml/basic
+  dependencies:
+  - https://standards.isotc211.org/19115/-1/1/conf/metadata-minimal-xml
+  - https://standards.isotc211.org/19115/-1/1/conf/metadata-xml/common
+  - https://standards.isotc211.org/19115/-1/1/conf/metadata-xml/multilingual
+  tests:
+  - name: Validate with XSD
+    identifier: https://standards.isotc211.org/19115/-1/1/conf/metadata-xml/basic/schema-valid
+    targets:
+    - https://standards.isotc211.org/19115/-1/1/req/metadata-xml/basic/valid
+    method: Validate with metadataBase.xsd
+  - name: Verify presence of identification information
+    identifier: https://standards.isotc211.org/19115/-1/1/conf/metadata-xml/basic/identification
+    targets:
+    - https://standards.isotc211.org/19115/-1/1/req/metadata-xml/basic/identification
+    method: |
+      Inspection to determine that the element populating the "identification"
+      property is defined in the substitution group for
+      Abstract_ResourceDescription.
+----
+
+In this arrangement, the `conformance_classes` field is meant to bundle several
+Conformance classes. Here only one Conformance class is shown.
+
+Each Conformance class has the following components:
+
+* `name`
+* `identifier`
+* `target`
+* several `dependencies` (array)
+* several `tests` (array)
+
+Under `tests`, each Conformance test is composed of:
+
+* `name`
+* `identifier`
+* `target` (array)
+* `method`
+
+Once the structure of the data is well-understood, we can proceed to write the
+Liquid template.
+
+As above, we define `context` as the context variable.
+
+[[cc-ex-liquid]]
+.Template file `template.liquid` that renders the Conformance class and Conformance tests
+[source,liquid]
+----
+{% for scope in context.conformance_classes %}
+
+.{{scope.name}}
+[conformance_class]
+====
+[%metadata]
+identifier:: {{scope.identifier}}
+target:: {{scope.target}}
+
+{% for depend in {{scope.dependencies}} %}
+inherit:: {{depend}}
+{% endfor %}
+
+{% for test in {{scope.tests}} %}
+conformance-test:: {{test.identifier}}
+{% endfor %}
+====
+
+{% for test in {{scope.tests}} %}
+{% if {{test.name}} %}
+.{{test.name}}
+{% endif %}
+[conformance_test]
+====
+[%metadata]
+identifier:: {{test.identifier}}
+
+{% for target in {{test.targets}} %}
+target:: {{target}}
+
+{% endfor %}
+
+{% for depend in {{test.dependencies}} %}
+inherit:: {{depend}}
+{% endfor %}
+
+{% if {{test.method}} %}
+test-method::
++
+--
+{{test.method}}
+--
+{% endif %}
+====
+
+{% endfor %}
+
+{% endfor %}
+----
+
+Multiple _if_ statements are used to verify the presence of data in fields.
+This is necessary when dealing with multiple requirement instances.
+
+This template, assumed to be saved as `template.liquid` file at the same
+location as the Metanorma file, is to be included in a `yaml2text` block inside
+the Metanorma document.
+
+.`yaml2text` block that encodes Conformance classes and Conformance tests
+[source,asciidoc]
+----
+[yaml2text,data.yaml,context]
+--
+\include::template.liquid[]
+--
+----
+
+From here, we can compile the document to verify its correct rendering,
+and debug if necessary.
+
+This process is equally applicable to any other ModSpec instances, including
+Recommendations and Permissions.
+
+
+== External resources
+
+Thanks to https://www.ogc.org[OGC],
+the https://www.ogc.org/standard/geopose/[OGC GeoPose] document
+(https://github.com/metanorma/ogc-GeoPose[GitHub]) is an open-source, fully
+fledged example of this approach in encoding Requirements and Conformance
+classes.
+
+Since it is a real-life example, the templates provided there are more generic
+and comprehensive (aka longer) than what we have explained here. The
+fundamentals, however, are the same as what is explained in this post.
+
+* https://github.com/metanorma/ogc-GeoPose/tree/main/standard/standard/modspec[OGC GeoPose ModSpec templates]
+
+Feel free to use them directly, or as a guide to design your own templates
+according to your needs!