Merge pull request #2694 from stfc/2690_value_verification

2690 value verification

.github/workflows/lfric_test.yml

@@ -73,7 +73,6 @@ jobs:
         # than the latest release from pypi.
         # pip install external/fparser
         pip install .[test]
-        pip install jinja2
 
     # PSyclone, compile and run MetOffice gungho_model on GPU
     - name: LFRic GungHo with OpenMP offload

changelog

@@ -256,9 +256,12 @@
 
 	89) PR #2712 for #2711. Adds support for user-supplied Kernels that
         operate on dofs (in the LFRic API).
-	
+
 	99) PR #2685 for #2027. Improves datatype and shape inference.
 
+	100) PR #2694 for #2690. Extends the PSyData NaN-checking tooling to
+	perform value verification.
+
 release 2.5.0 14th of February 2024
 
 	1) PR #2199 for #2189. Fix bugs with missing maps in enter data

config/psyclone.cfg

@@ -46,7 +46,7 @@ REPRODUCIBLE_REDUCTIONS = false
 # Amount to pad the local summation array when REPRODUCIBLE_REDUCTIONS is true
 REPROD_PAD_SIZE = 8
 PSYIR_ROOT_NAME = psyir_tmp
-VALID_PSY_DATA_PREFIXES = profile, extract, read_only_verify, nan_test
+VALID_PSY_DATA_PREFIXES = profile, extract, read_only_verify, value_range_check
 
 # Specify number of OpenCL devices per node. When combining OpenCL with MPI,
 # the mpirun/mpiexec ranks_per_node parameter must match this number.

doc/developer_guide/system_specific_setup.rst

@@ -180,19 +180,3 @@ Verifying the pylint standards is done with::
 
 
 OK, you're all set up.
-
-Installing Tools for PSyData Wrapper Libraries
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-If you intend to compile the PSyData wrapper libraries or develop new libraries,
-you might need to install Jinja2 (most wrapper libraries require Jinja2 though
-some, like the NVIDIA GPU profiling wrapper, do not need it). You can install
-the necessary dependencies to create all PSyData wrapper libraries with::
-
-    > pip install psyclone[psydata]
-
-or when using the git version::
-
-    > pip install .[psydata]
-
-
-Check :ref:`psy_data` and especially the section :ref:`jinja` for more details.

doc/user_guide/examples.rst

@@ -315,33 +315,53 @@ read-only variables:
     New value:         123.00000000000000     
     --------------------------------------
 
-.. _gocean_example_nan:
+.. _gocean_example_value_range_check:
 
-Example 5.4: Valid Number Verification (NaN Test)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Example 5.4: Value Range Check
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 This example shows the use of valid number verification with PSyclone.
 It instruments each of the two invokes in the example program
-with the PSyData-based NaN-verification code.
-It uses the dl_esm_inf-specific nan_test library
-(``lib/nan_test/dl_esm_inf/``).
+with the PSyData-based Value-Range-Check code.
+It uses the dl_esm_inf-specific value range check library
+(``lib/value_range_check/dl_esm_inf/``).
 
 .. note:: The ``update_field_mod`` subroutine contains code
     that will trigger a division by 0 to create NaNs. If
-    the compiler should add floating point exception handling
-    code, this will take effect before the NaN testing is done
-    by the PSyData-based verification code.
+    the compiler happens to add code that handles floating point
+    exceptions , this will take effect before the value testing
+    is done by the PSyData-based verification code.
 
 The ``Makefile`` in this example will link with the compiled
-nan_test library. You can execute the created
-binary and it will print five warnings about invalid numbers
-at the indices  1 1, ..., 5 5:
+value_range_check library. You can then execute the binary
+and enable the value range check by setting environments
+(see :ref:`value range check<psydata_value_range_check>` for
+details).
 
-.. code-block:: none
+.. code-block:: shell
 
-    PSyData: Variable a_fld has the invalid value
-                     Infinity  at index/indices            1           1
-    mainupdate
+    PSYVERIFY__main__init__b_fld=2:3 ./value_range_check
+    ...
+    PSyData: Variable b_fld has the value 0.0000000000000000 at index/indices 6 1 in module 'main', region 'init', which is not between '2.0000000000000000' and '3.0000000000000000'.
     ...
+    PSyData: Variable a_fld has the invalid value 'Inf' at index/indices 1 1 in module 'main', region 'update'.
+
+As indicated in :ref:`value range check<psydata_value_range_check>`, you can
+also check a variable in all kernels of a module, or in any instrumented
+code region (since the example has only one module, both settings below
+will create the same warnings):
+
+.. code-block:: shell
+
+    PSYVERIFY__main__b_fld=2:3 ./value_range_check
+    PSYVERIFY__b_fld=2:3 ./value_range_check
+    ...
+    PSyData: Variable b_fld has the value 0.0000000000000000 at index/indices 6 1 in module 'main', region 'init', which is not between '2.0000000000000000' and '3.0000000000000000'.
+    ...
+    PSyData: Variable b_fld has the value 0.0000000000000000 at index/indices 6 1 in module 'main', region 'update', which is not between '2.0000000000000000' and '3.0000000000000000'.
+
+Notice that now a warning is created for both kernels: ``init`` and ``update``.
+
+Support for checking arbitrary Fortran code is tracked as issue #2741.
 
 
 Example 6: PSy-layer Code Creation using PSyIR

doc/user_guide/libraries.rst

@@ -108,17 +108,18 @@ the ``lib/read_only`` `directory
 For detailed instructions on how to build and use these libraries
 please refer to their specific ``README.md`` documentation.
 
-NAN Test
-^^^^^^^^
+Value Range Check
+^^^^^^^^^^^^^^^^^
 
-These libraries test all input and output parameters of a kernel to
-make sure they are not ``NaN`` or infinite.  More information can be
-found in the :ref:`NAN Test <psydata_nan_test>` section.
+These libraries can test if user-defined variables are within a
+specified range. Additionally, they also verify that they are
+not ``NaN`` or infinite.  More information can be
+found in the :ref:`Value Range Check <psydata_value_range_check>` section.
 
 The libraries for :ref:`LFRic <lfric-api>` and
 :ref:`GOcean <gocean-api>` APIs are included with PSyclone in
-the ``lib/nan_test`` `directory
-<https://github.com/stfc/PSyclone/tree/master/lib/nan_test>`__.
+the ``lib/value_range_check`` `directory
+<https://github.com/stfc/PSyclone/tree/master/lib/value_range_check>`__.
 For detailed instructions on how to build and use these libraries
 please refer to their specific ``README.md`` documentation.
 
@@ -137,9 +138,9 @@ The majority of wrapper libraries use `Jinja
 classes (please refer to :ref:`dev_guide:psy_data` and :ref:`dev_guide:jinja`
 for full details about the PSyData API).
 
-Compilation of ``extract``, ``nan_test``, ``read_only`` and some of the
+Compilation of ``extract``, ``value_range_check``, ``read_only`` and some of the
 profiling wrapper libraries depends on infrastructure libraries relevant
-to the API they are used for. :ref:`LFRic API <lfric-api>` uses the
+to the API they are used for. The :ref:`LFRic API <lfric-api>` uses the
 LFRic infrastructure and :ref:`GOcean <gocean-api>` uses the
 dl_esm_inf library. The LFRic infrastructure can be obtained from the
 LFRic `code repository <https://code.metoffice.gov.uk/trac/lfric/browser>`_,

doc/user_guide/psy_data.rst

@@ -72,10 +72,12 @@ Access Verification:
   to a field (and scalar values). See :ref:`psydata_read_verification`
   for details.
 
-NAN Test:
+Value Range Check:
   The callbacks can be used to make sure that all floating point input
-  and output parameters of a kernel are not a ``NaN`` (not-a-number) or
-  infinite. See :ref:`psydata_nan_test` for the full description.
+  and output parameters of a kernel are within a user-specified range.
+  Additionally, it will also verify that the values are not a ``NaN``
+  (not-a-number) or infinite. See :ref:`psydata_value_range_check` for
+  the full description.
 
 In-situ Visualisation:
   By giving access to output fields of a kernel, an in-situ visualisation
@@ -99,7 +101,7 @@ Read-Only Verification
 ----------------------
 
 The PSyData interface is being used to verify that read-only variables
-in a kernel are not overwritten. The ``ReadOnlyVerifyTrans`` (in 
+in a kernel are not overwritten. The ``ReadOnlyVerifyTrans`` (in
 ``psyir.transformations.read_only_verify_trans``, or the
 :ref_guide:`Transformation Reference Guide psyclone.psyir.transformations.html#classes`) uses the dependency
 analysis to determine all read-only variables (i.e. arguments declared
@@ -145,11 +147,12 @@ Both libraries support the environment variable ``PSYDATA_VERBOSE``.
 This can be used to control how much output is generated
 by the read-only-verification library at runtime. If the
 variable is not specified or has the value '0', warnings will only
-be printed if checksums change. If it is set to '1', a message will be 
+be printed if checksums change. If it is set to '1', a message will be
 printed before and after each kernel call that is checked. If the
 variable is set to '2', it will additionally print the name of each
 variable that is checked.
 
+
 Read-Only Verification Library for LFRic
 ++++++++++++++++++++++++++++++++++++++++
 
@@ -185,6 +188,13 @@ the required variables:
 
 This will create a library called ``lib_read_only.a``.
 
+An executable example for using the LFRic read-only-verification library is
+included in ``tutorial/practicals/LFRic/building_code/4_psydata`` directory,
+see `this link for more information
+<https://github.com/stfc/PSyclone/tree/master/tutorial/practicals/LFRic/building_code/4_psydata>`_.
+
+
+
 Read-Only-Verification Library for GOcean
 +++++++++++++++++++++++++++++++++++++++++
 
@@ -224,39 +234,87 @@ An executable example for using the GOcean read-only-verification
 library is included in ``examples/gocean/eg5/readonly``, see
 :ref:`gocean_example_readonly`.
 
-.. _psydata_nan_test:
+.. _psydata_value_range_check:
 
-NAN Test
---------
+Value Range Check
+-----------------
 
 This transformation can be used for both LFRic and GOcean APIs. It will
-test all input and output parameters of a kernel to make sure they are not
-``NaN`` or infinite. If they are, an error message like the following
-is printed, but the program is not aborted::
+test all input and output parameters of a kernel to make sure they are
+within a user-specified range. Additionally, it will also verify that floating
+point values are not ``NaN`` or infinite.
+
+At runtime, environment variables must be specified to indicate which variables
+are within what expected range, and optionally also at which location.
+The range is specified as a ``:`` separated tuple::
+
+    1.1:3.3   A value between 1.1 and 3.3 (inclusive).
+    :3.3      A value less than or equal to 3.3
+    1.1:      A value greater than or equal to 1.1
+
+The syntax for the environment variable is one of:
+
+``PSYVERIFY__module__kernel__variable``
+    The specified variable is tested when calling the specified kernel in the
+    specified module.
+
+``PSYVERIFY__module__variable``
+    The specified variable name is tested in all kernel calls of the
+    specified module that are instrumented with the ValueRangeCheckTrans
+    transformation.
+
+``PSYVERIFY__variable``
+    The specified variable name is tested in any instrumented code region.
 
-     PSyData: Variable a_fld has the invalid value Inf at index/indices 1 1 in module 'main' region 'update'.
+If the module name or kernel name contains a `-` (which can be inserted
+by PSyclone, e.g. `invoke_compute-r1`), it needs to be replaced with an
+underscore character in the environment variable (`_`)
 
-Is uses the function ``IEEE_IS_FINITE`` from the ieee_arithmetic module
-for this test. Note that only floating point numbers will be tested.
-Integer numbers do not have a bit pattern for 'infinity' or ``NaN``.
+An example taken from the LFric tutorial (note that values greater than
+4000 are actually valid, the upper limit was just chosen to show
+a few warnings raised by the value range checker)::
+
+    PSYVERIFY__time_evolution__invoke_initialise_perturbation__perturbation_data=0.0:4000
+    PSYVERIFY__time_evolution__perturbation_data=0.0:4000
+    PSYVERIFY__perturbation_data=0.0:4000
+    
+.. warning:: Note that while the field variable is called `perturbation`, PSyclone will
+             append `_data` when the LFRic domain is used, so the name becomes
+             `perturbation_data`. You have to use
+             this name in LFRic in order to trigger the value range check. To verify
+             that the tests are done as expected, set the environment variable
+             `PSYDATA_VERBOSE` to 1, which will print which data is taken from the
+             environment variables:
+
+             .. code-block:: bash
+
+                 PSyData: checking 'time_evolution' region 'invoke_initialise_perturbation' :   0.0000000000000000       <= perturbation_data <=    4000.0000000000000
+
+
+If values outside the specified range are found, appropriate warnings are printed,
+but the program is not aborted::
+
+    PSyData: Variable 'perturbation_data' has the value 4227.3587826606408 at index/indices 27051 in module 'time_evolution', region 'invoke_initialise_perturbation', which is not between '0.0000000000000000' and '4000.0000000000000'.
+
+
+The library uses the function ``IEEE_IS_FINITE`` from the ieee_arithmetic module
+for additionally verifying that values are not ``NAN`` or ``infinity``
+for any floating point variable, even if no ``PSY_VERIFY...`` environment
+variable is set for this variable. Integer numbers do not have a bit pattern
+for 'infinity' or ``NaN``, so they will only be tested for valid range
+if a corresponding environment variable is specified.
 
 The runtime libraries for GOcean and LFRic are based on a jinja-template
-contained in the directory ``<PSYCLONEHOME>/lib/nan_test``.
+contained in the directory ``<PSYCLONEHOME>/lib/value_range_check``.
 The respective API-specific libraries map the internal field structures
 to Fortran basic types and call the functions from the base class to
 handle those.
 
 The relevant libraries for the LFRic and GOcean APIs are contained in
-the ``lib/nan_test/lfric`` and``lib/nan_test/dl_esm_inf`` subdirectories,
+the ``lib/value_range_check/lfric`` and ``lib/value_range_check/dl_esm_inf`` subdirectories,
 respectively. For more information on how to build and link these libraries,
 please refer to the relevant ``README.md`` files.
 
-An executable example for using the LFRic read-only-verification library is
-included in ``tutorial/practicals/LFRic/building_code/4_psydata`` directory,
-see `this link for more information
-<https://github.com/stfc/PSyclone/tree/master/tutorial/practicals/LFRic/building_code/4_psydata>`_.
-
-
 .. _integrating_psy_data_lfric:
 
 Integrating PSyData Libraries into the LFRic Build Environment

examples/gocean/README.md

@@ -30,22 +30,29 @@ Examples of the application of kernel transforms to kernels that access
 data and/or routines from other Fortran modules. Note that this is not
 yet fully supported and is the subject of Issue #342.
 
-## Example 5a
+## Example 5a (profile)
 
 Illustrates the use of the profiling support in PSyclone. The resulting
 code may be compiled and executed.
 
-## Example 5b
+## Example 5b (extract)
 
 Illustrates the use of the kernel-data extraction support in PSyclone. The
 resulting code may be compiled and executed (requires a netcdf installation).
 
-## Example 5c
+## Example 5c (readonly)
 
 Illustrates the use of the read-only verification in PSyclone. The
 resulting code may be compiled and executed to show warnings printed
 by the read-only verification.
 
+## Example 5d (value_range_check)
+
+Illustrates the use of the value range check in PSyclone. The
+resulting code may be compiled and executed to show warnings printed.
+Note that certain environment variables need to be defined to enable
+the value range check, see the README.md in that directory for details.
+
 ## Example 6
 
 Informs the development of the code generation of PSy-layer code using the

examples/gocean/eg5/Makefile

@@ -33,6 +33,6 @@
 # ------------------------------------------------------------------------------
 # Author: J. Henrichs, Bureau of Meteorology
 
-EXAMPLES=extract nan profile readonly
+EXAMPLES=extract value_range_check profile readonly
 
 include ../../top_level.mk

examples/gocean/eg5/value_range_check/.gitignore

@@ -0,0 +1,3 @@
+alg.f90
+psy.f90
+value_range_check