Skip to content

Latest commit

 

History

History
114 lines (91 loc) · 4.34 KB

CONTRIBUTING.md

File metadata and controls

114 lines (91 loc) · 4.34 KB
Raw

Contributing to Alcotest

Setting up your working environment

To clone the project's sources and install its various dependencies, run:

git clone https://github.com/mirage/alcotest.git    # Get the repository
cd alcotest

opam switch create ./ ocaml-base-compiler.5.0.0    # OPTIONAL: install a project-local Opam switch
opam install -t --deps-only .                      # Install regular and test dependencies

Optionally, install js_of_ocaml-compiler and nodejs to be able run the testsuite using js_of_ocaml.

opam install -t js_of_ocaml-compiler

Now you can work on the project using the standard Dune commands:

dune build              # Build all libraries, executables and tests
dune test               # Run the test-suite
dune build @runtest-js  # Run the test-suite on nodejs (using js_of_ocaml)
dune clean              # Delete all artefacts

Project structure

The high-level project structure is as follows:

├── src/
│   ├── alcotest-engine/
│   │   ├── core.ml           # The main test runner, parameterised over...
│   │   ├── monad.ml          # ... a concurrency monad
│   │   ├── platform.ml       # ... a platform implementation / OS bindings
│   │   │
│   │   ├── cli.ml            # Extends `core.ml` with a command-line API
│   │   │
│   │   ├── pp.ml             # Pretty-printers for Alcotest output
│   │   ├── test.ml           # Combinators for test assertions (with `Alcotest.check`)
│   │   └── utils.ml          # Standard-library extension
│   │
│   │                         # Specific backends:
│   ├── alcotest/             # - Unix-specific API (compatible with js_of_ocaml)
│   ├── alcotest-async/       # - Extends `alcotest/` for test-suites using Async concurrency
│   ├── alcotest-lwt/         # - Extends `alcotest/` for test-suites using Lwt concurrency
│   └── alcotest-mirage/      # - MirageOS-specific API
│
├── test/                     # Project tests (see '§ Expect tests' below)
│
├── dune-project              # Project metadata (opam files are generated from
├── alcotest-async.opam       # data in the `dune-project` file by running
├── alcotest-lwt.opam         # `dune build @install`).
├── alcotest-mirage.opam
├── alcotest.opam
│
├── alcotest-help.txt         # Manpage output (generated by `dune test --auto-promote`)
├── dune
│
├── CHANGES.md                # All user-facing changes get an entry here
└── README.md

Expect tests

The repository contains a number of 'expect' tests in https://github.com/mirage/alcotest/tree/main/test/e2e. These are short, self-contained usages of the Alcotest API, with the corresponding output snapshotted in an .expected file:

test/e2e/alcotest/passing/
├── basic.ml          # The test case
├── basic.expected    # The expected output of the test case
└── basic.opts        # OPTIONAL: command-line options to pass to the test

If you change the Alcotest output in some way, you will likely need to update the expect test snapshots:

  1. run dune test to get a list of diffs to the tests,
  2. check that the diffs are as expected,
  3. run dune promote to update the snapshots in the repository.

To add a new test, create a new foo.{ml,expected} pair in one of the test directories and run dune test --auto-promote to automatically generate the Dune rules necessary to run your test.

Random output in tests

Some sections of the Alcotest output are not portable / reproducible under test, for instance:

  • the time taken to run a particular test;
  • randomly-generated unique identifiers;
  • the full path to the repository on disk;
  • the executable file extension.

When running an expect test, these portions of the output are sanitized by strip_randomness.ml before being compared to the .expected file, resulting in files with portable placeholders:

The full test results are available in `<build-context>/_build/_tests/<uuid>`.
Test Successful in <test-duration>s. 4 tests run.

If your change adds non-determinism to the Alcotest output, you may need to alter strip_randomness.ml to get it past the CI.