Developing Regression Tests

Developing Regression Tests

See Adding New Tests for more information.

Regression tests for the python RAVEN framework are found in raven/tests/framework. There is a hierarchy of folders with tests collected by similar testing. In order to add a new test, the following node must be included in the test file within the <Simulation> block:

<Simulation>
  ...
  <TestInfo>
    <name>framework/path/to/test/label</name>
    <author>AuthorGitHubTag</author>
    <created>YYYY-MM-DD</created>
    <classesTested>Module.Class, Module.Class</classesTested>
    <description>
        Paragraph describing workflows, modules, classes, entities, et cetera, how they are tested, and any other notes
    </description>
    <requirements>RequirementsLabel</requirements>
    <analytic>paragraph description of analytic test</analytic>
    ...
  </TestInfo>
  ...
</Simulation>

The <requirements> and <analytic> nodes are optional, for those tests who satisfy an NQA design requirement and/or have an analytic solution documented in the analytic tests document. Other notes on block contents:

• name: this is the test framework path, as well as the name/label assigned in the tests file block. This is the path and name that show up when running the tests using the MOOSE testing harness (run_tests).
• author: this is the GitHub tag of the author who constructed this test originally, i.e. alfoa for @alfoa.
• created: this is the date on which the test was originally created, in year-month-day YYYY-MM-DD XSD date format.
• classesTested: a list of the classes tested in the python framework, listed as Entity.Class, i.e. Samplers.MonteCarlo.
• description: general notes about what workflows or other methods are tested.
• requirements: (optional) lists the NQA requirement that this test satisfies.
• analytic: (optional) describes the analytic nature of this test and how it is documented in the analytic tests documentation.

An additional node is optionally available to demonstrate significant revisions to a test:

<Simulation>
  ...
  <TestInfo>
    ...
    <revisions>
      <revision author="AuthorGitHubTag" date="YYYY-MM-DD">paragraph description of revision</revision>
    </revisions>
    ...
  </TestInfo>
  ...
</Simulation>