Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates performance baselines #4491

Merged
merged 34 commits into from
Oct 16, 2023
Merged

Conversation

jasonb5
Copy link
Collaborator

@jasonb5 jasonb5 commented Sep 23, 2023

Updates how performance baselines are created and compared.

Before

  • Baseline generation stores cpl.log
  • Comparison parses baseline and testcase cpl.log and compares throughput/memory
  • --check-throughput and --check-memory will PASS even if MEMCOMP or TPUTCOMP fail
  • bless_test_results --hist-only will update the baseline cpl.log

After

  • Baseline generation stores throughput in cpl-tput.log and memory usage in cpl-mem.log
  • Comparison parses testcase cpl.log and compares the baselines respective files
  • --check-throughput and --check-memory will DIFF if MEMCOMP or TPUTCOMP fail
  • bless_test_results now has --tput-only and --mem-only to bless individual performance DIFF
  • Fixes logging throughput/memory comparison results in TestStatus.log
  • Allows generation/comparison to be customized by the coupler

Test suite: pytest -vvv CIME/tests
Test baseline: n/a
Test namelist changes: n/a
Test status: n/a

Fixes E3SM-Project/E3SM#5885

User interface changes?: N
Update gh-pages html (Y/N)?: N

@jasonb5
Copy link
Collaborator Author

jasonb5 commented Sep 23, 2023

Leaving this as a draft until we determine if additional updates are needed from E3SM-Project/E3SM#5937 discussion.

@jasonb5 jasonb5 marked this pull request as ready for review October 14, 2023 01:21
Copy link
Contributor

@jgfouca jgfouca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

@jgfouca jgfouca merged commit 88255d5 into ESMCI:master Oct 16, 2023
9 checks passed
@jasonb5 jasonb5 deleted the performance-baselines branch October 16, 2023 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Need better tracking of performance baselines
3 participants