Skip to content

Latest commit

 

History

History
45 lines (30 loc) · 2.49 KB

README.md

File metadata and controls

45 lines (30 loc) · 2.49 KB
Raw

Build Status Results

Pints Change Point Testing

This repository is used to code that will become a change point detection module, used to see if the statistical properties of a piece of software change over time. It will be used to make sure PINTS functions reliably.

The information here is out of date. This will be rewritten at some point: #38

Old test results can be viewed here (or by clicking the blue badge above).

Installation

  • Functional testing requires Python 3.4 or later
  • When cloning, make sure to add the --recursive switch
  • To install, use python3 -m pip install -r requirements.txt. This makes sure you have all the dependencies you know.
git clone [email protected]:pints-team/functional-testing.git --recursive

Running tests

  • To see a list of available tests, use ./funk list.
  • Use ./funk run test_name to run a test.
    • This will store results in a database (default ./results.db).
  • Use ./funk run --next to run the next test in line. This is determined by looking at the result files and seeing which test hasn't been run for the longest time.
  • Use ./funk run test_name --show to show the resulting plot as well

Adding tests

  • Add a test to the tests module, and then add it to tests.py so that the framework can find it.
  • Write any results (int, floats, strings, or numpy arrays) to the result object.
  • Write any pints logs to the given log path (e.g. opt.set_log_to_file(log_path)).

Some details:

  • The internal pints repo is reloaded before every test is run, and the version number and commit hash are automatically added to the results object.
  • The numpy random generator is seeded before every tests is run, and the seed is automatically added to the results object.
  • If the test passes successfully, please set status=passed in the results object.

Creating plots, running analysis

  • Use ./funk plot test_name to run a plot, or ./funk plot --all to run all plots
  • Use ./funk analyse test_name to check if a test passed or failed, or ./funk analyse --all to check all tests