Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add flexible controls of GPU configuration and remove old GPU configuration #97

Merged
merged 27 commits into from
Aug 9, 2023
Merged

Add flexible controls of GPU configuration and remove old GPU configuration #97

merged 27 commits into from
Aug 9, 2023

Conversation

sjsprecious
Copy link
Collaborator

@sjsprecious sjsprecious commented Apr 17, 2023 •
edited
Loading

This PR:

  • updates the configurations for the GPU offload based on the new XML variables in CMEPS (Add new XML variables for flexible controls of GPU configuration ESCOMP/CMEPS#363).
  • removes the old approaches for GPU configurations.
  • updates the module versions and batch script template on Derecho.
  • removes #PBS -V option as it incorrectly populate the NGPUS env variable from the login node to the compute node on Derecho.

@sjsprecious
Add Jim's changes for Gust's modules from jedwards4b@87b9539
9beb292 
Use nvhpc/22.11, cray-mpich/8.1.21, and esmf/8.4.1.b02

	modified:   machines/cmake_macros/gust.cmake
	modified:   machines/config_machines.xml
@sjsprecious
Add Jim's gpu_improvement branch at https://github.com/jedwards4b/ccs…
4134af3 
…_config_cesm/compare/52c06b3..fbc05d6

Update settings on Gust and Casper

	modified:   Depends.nvhpc
	deleted:    Depends.nvhpc-gpu
	deleted:    cmake_macros/nvhpc-gpu.cmake
	deleted:    cmake_macros/nvhpc-gpu_casper.cmake
	modified:   cmake_macros/nvhpc.cmake
	modified:   cmake_macros/nvhpc_casper.cmake
	deleted:    cmake_macros/pgi-gpu.cmake
	deleted:    cmake_macros/pgi-gpu_casper.cmake
	modified:   config_batch.xml
	modified:   config_machines.xml
	deleted:    mpi_run_gpu.casper
@sjsprecious
Add max_cputasks_per_gpu_node, gpu_type and gpu_offload options for C…
3b7a182 
…asper and Gust

	modified:   machines/config_machines.xml
@sjsprecious
update batch script template for Gust GPU node
ee726b5 
	modified:   config_batch.xml
@sjsprecious
typo fix for mismatched tags for GPU options
1f5eb30 
	modified:   machines/config_machines.xml
@sjsprecious
Add missing "MAX_GPUS_PER_NODE"
8649a95 
	modified:   machines/config_machines.xml
@sjsprecious
downgrade netcdf-mpi fpr nvhpc/22.11
e7bfbe1 
	modified:   machines/config_machines.xml
@sjsprecious
bug fix of batch script template with gpu_enabled option
bf14967 
	modified:   machines/config_batch.xml
@sjsprecious
remove -target=zen3 flag on gust
14f605e 
load cuda module on Gust for a GPU run

	deleted:    cmake_macros/nvhpc_gust.cmake
	modified:   config_machines.xml
@sjsprecious
update to nvhpc/23.1 and related modules
59086af 
	modified:   machines/config_machines.xml
@sjsprecious
turn off MPS on Gust
918e3a8 
	modified:   config_batch.xml
@sjsprecious
Update module versions for intel compiler
d0093a5 
	modified:   machines/config_batch.xml
	modified:   machines/config_machines.xml
@sjsprecious
Merge branch 'main' into add_gpu_gust
4023d67
@sjsprecious sjsprecious added the enhancement New feature or request label Apr 17, 2023
@sjsprecious sjsprecious mentioned this pull request Apr 17, 2023
Merged
@sjsprecious
load ncarenv and ncarcompilers manually
7224a37 
	modified:   config_machines.xml
@sjsprecious
update the ncarenv and cesmdev module on Gust
822f7b7 
	modified:   machines/config_machines.xml
@sjsprecious
Copy link
Collaborator Author

@jedwards4b will issue a separate PR to introduce these changes.

@sjsprecious sjsprecious reopened this Aug 8, 2023
@sjsprecious sjsprecious mentioned this pull request Aug 8, 2023
Merged
@sjsprecious
Copy link
Collaborator Author

I updated the nvhpc compiler version to 23.5 with a newer version of ESMF8.6.0b03. The GPU run finished successfully on Derecho.

@jedwards4b jedwards4b merged commit fe65a9c into ESMCI:main Aug 9, 2023
@sjsprecious sjsprecious deleted the add_gpu_gust branch August 9, 2023 19:24
@sjsprecious sjsprecious mentioned this pull request Sep 18, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jedwards4b jedwards4b jedwards4b approved these changes

Assignees

@jedwards4b jedwards4b

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sjsprecious @jedwards4b