You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Improve the testing in the pipeline, and automate deployment to the testnet.
Checklist
Yarn tests: we can stick with the current yarn tests. However, we should also add a t=3,n=5 yarn test to match the testnet, and make these parameters configurable with a command-line flag if possible to run multiple configurations
Stress tests: we can take advantage of (stress) testing-harness-v1, and spin up a series of tests to stress test. This has insofar been instrumental in finding bugs, and can help rule out bugs introduced in future PRs.
Staging phase: automated launching to the testnet ("staging") after the pipeline is green (upon manual approval). Make the testnet pass before even considering merging to master (unless, of course, the PR does not touch the DKG code). This phase will require a manual approval, but once approved, it will send the PR to the testnet.
The text was updated successfully, but these errors were encountered:
The checklist here @tbraun96 should be more thorough, it should contain amongst other things all todo items for each section. Such as
Yarn tests
Add a test for t=3,n=5 to match the testnet
Make the yarn tests configurable from a CLI.
Stress Tests
Stress test 1
Stress test 2
...
Staging Phase
Automate the deployment of staging releases to testnet.
etc.
My take is that we don't actually need so much integration tests if we just also hook up things to the runtime as closely as possible too. We can leverage the test externalities environment of Substrate to drive tests that things are working through. This would cover everything but the gossip issues (but to the extent that the harness exposes those we'd be good).
Unit testing improvements
Integrate testing harness directly with the runtime environment.
Create simulations that drive the testing harness through the TestExternalities structure.
Reduce our dependency on integration tests (potentially the yarn tests, etc.)
Overview
Improve the testing in the pipeline, and automate deployment to the testnet.
Checklist
Yarn tests: we can stick with the current yarn tests. However, we should also add a t=3,n=5 yarn test to match the testnet, and make these parameters configurable with a command-line flag if possible to run multiple configurations
Stress tests: we can take advantage of (stress) testing-harness-v1, and spin up a series of tests to stress test. This has insofar been instrumental in finding bugs, and can help rule out bugs introduced in future PRs.
Staging phase: automated launching to the testnet ("staging") after the pipeline is green (upon manual approval). Make the testnet pass before even considering merging to master (unless, of course, the PR does not touch the DKG code). This phase will require a manual approval, but once approved, it will send the PR to the testnet.
The text was updated successfully, but these errors were encountered: