Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates performance baselines #4491

Merged
merged 34 commits into from
Oct 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
698b373
Removes default coverage reporting and offloads to actions
jasonb5 Sep 15, 2023
e144fea
Adds test_coverage to ignore list
jasonb5 Sep 15, 2023
2245e74
Adds similar line as TPUTCOMP to inform user of difference
jasonb5 Sep 20, 2023
a6b8d8f
Updates how performance baselines are written and compared to tests
jasonb5 Sep 20, 2023
a367ab9
Updates MEMCOMP and TPUTCOMP to DIFF if they're checked and fail
jasonb5 Sep 20, 2023
2cc8352
Refactors _compare_throughput and _compare_memory methods
jasonb5 Sep 21, 2023
5ac9e1e
Refactors compare_throughput and compare_memory functions
jasonb5 Sep 22, 2023
9bab639
Adds --tput-only and --mem-only
jasonb5 Sep 23, 2023
b808e80
Merge branch 'master' into performance-baselines
jasonb5 Sep 29, 2023
52c8d7b
Fixes failing test
jasonb5 Sep 29, 2023
6dc623c
Refactors and updates bless_test_results
jasonb5 Oct 3, 2023
9450256
Refactors base config class
jasonb5 Oct 5, 2023
d948986
Adds feature for coupler to define custom read/compare functions
jasonb5 Oct 6, 2023
ec661ca
Fixes failing tests
jasonb5 Oct 6, 2023
4bf6605
Fixes black formatting
jasonb5 Oct 6, 2023
44215c4
Merge branch 'master' into performance-baselines
jasonb5 Oct 9, 2023
e61d641
Updates permissions
jasonb5 Oct 9, 2023
b353d1d
Removes --pull
jasonb5 Oct 9, 2023
7e8f77b
Merge branch 'master' into performance-baselines
jasonb5 Oct 11, 2023
ecab154
Fixes BASECMP_CASE name
jasonb5 Oct 11, 2023
a786b10
Cleans up baselines
jasonb5 Oct 12, 2023
1c6d7a1
Fixes handling non performance metrics
jasonb5 Oct 12, 2023
3596b55
Updates _check_for_memleak to be customized by the coupler
jasonb5 Oct 12, 2023
7890021
Renames methods
jasonb5 Oct 13, 2023
d76f407
Fixes more naming
jasonb5 Oct 13, 2023
13e0e8b
Fixes striping newlines when reading baseline files
jasonb5 Oct 13, 2023
7780b1f
Fixes passing correct paths to memory/throughput bless functions
jasonb5 Oct 13, 2023
52ca0a1
Renames memory leak functions
jasonb5 Oct 13, 2023
69f7e36
Fixes black formatting
jasonb5 Oct 13, 2023
ccb3b4e
Fixes baseline module organization
jasonb5 Oct 13, 2023
b856aa1
Fixes returning raw string from baseline
jasonb5 Oct 14, 2023
192eb32
Updates baseline documentation
jasonb5 Oct 14, 2023
3071dc3
Adds missing logging
jasonb5 Oct 14, 2023
7e30703
Fixes commenting preview that was never pushed
jasonb5 Oct 14, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,9 @@ jobs:
user_name: 'github-actions[bot]'
user_email: 'github-actions[bot]@users.noreply.github.com'
- name: Comment about previewing documentation
if: ${{ github.event_name == 'pull_request' }}
if: |
github.event_name == 'pull_request' &&
github.event.pull_request.head.repo.full_name == github.repository
uses: actions/github-script@v6
with:
script: |
Expand Down
7 changes: 3 additions & 4 deletions .github/workflows/testing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ concurrency:

permissions:
contents: read # to fetch code (actions/checkout)
packages: read

jobs:
pre-commit:
Expand Down Expand Up @@ -58,7 +59,6 @@ jobs:
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
options: '--pull=always'
strategy:
matrix:
python-version: ['3.8', '3.9', '3.10']
Expand Down Expand Up @@ -87,7 +87,7 @@ jobs:

init_cime

pytest -vvv --machine docker --no-fortran-run CIME/tests/test_unit*
pytest -vvv --cov=CIME --machine docker --no-fortran-run CIME/tests/test_unit*

# Run system tests
system-testing:
Expand All @@ -98,7 +98,6 @@ jobs:
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
options: '--pull=always'
strategy:
matrix:
model: ["e3sm", "cesm"]
Expand Down Expand Up @@ -144,7 +143,7 @@ jobs:

conda activate base

pytest -vvv --machine docker --no-fortran-run --no-teardown CIME/tests/test_sys*
pytest -vvv --cov=CIME --machine docker --no-fortran-run --no-teardown CIME/tests/test_sys*
- name: Create testing log archive
if: ${{ failure() }}
shell: bash
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,4 @@ scripts/Tools/JENKINS*
components
libraries
share
test_coverage/**
Loading
Loading