-
Notifications
You must be signed in to change notification settings - Fork 167
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
memory and runtime benchmark/testing infrastructure #8858
Comments
Comment by Ned Molter on JIRA: A PR in stcal adds a context manager that makes it easier to compare the peak memory usage of a code block against an expected usage. A single unit test to the median calculator is implemented that uses the context manager, and this test should be used as an example of how to implement a test where the expected memory usage can be calculated straightforwardly based on the input size. The new context manager can be imported into roman and jwst. PR link here: spacetelescope/stcal#299 edit: this part of the work is complete |
Issue JP-3775 was created on JIRA by Ned Molter:
Add and implement tools to ensure that pull requests do not inadvertently cause significant regressions in memory usage or runtime.
At a hack day discussion, we targeted initial implementation and testing in stcal, since it's a smaller repository but still has non-trivial memory- and runtime-hungry functions (e.g., outlier detection median calculation). Goal is very quick adoption by JWST, and adoption by Roman once other outstanding regression test issues have been resolved.
The preferred workflow would be to have decorators that could be added to tests to flag them as memory- and/or runtime-checked. At time of writing, codspeed is being considered for runtime, and pytest-memray is being considered for memory.
Task list for now:
The text was updated successfully, but these errors were encountered: