Add experimental ability to restart from lat/lon A-grid horizontal winds #34
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I ran similar test cases at C48 resolution to get a sense for how resolution influences things, and also added some diagnostics to see how much transforming winds from the D-grid to A-grid back to the D-grid degrades them. The differences we see upon restarting the model with A-grid winds are consistent with the degradation experienced in this "roundtrip" exercise. Running at finer resolution also improves things a little. See this notebook.
This all makes me more confident that this is working, so I'll proceed to polishing this PR up for review. |
|
An important question, and the ultimate motivation behind this experimental functionality, is how smoothly does a coarse simulation initialized from coarsened restart files with A-grid winds spin up compared to simulations initialized with coarsened restart files with winds coarsened using other strategies? This notebook explores this question. Ultimately we find that the "complicated" method of D-grid wind coarse-graining implemented in this notebook for this set of restart files still performs the best by the global mean absolute value of surface pressure tendency metric, but A-grid coarse-graining and initialization still beats our current method (pressure-level D-grid wind coarse-graining along coarse cell edges). We will have to ponder the tradeoff between complexity and initialization performance here. Similar to the alternate D-grid wind coarsening method, most of the benefit is over the ocean. See this plot of the surface pressure tendency in the first timestep in each case.
|
|
Ok I think this should be ready for review. This PR adds a couple namelist parameters to control this functionality (all in
For reference, the |
spencerkclark
commented
May 6, 2020
rheacangeo
approved these changes
May 7, 2020
oliverwm1
requested changes
May 7, 2020
oliverwm1
approved these changes
May 8, 2020
There was a problem hiding this comment.
LGTM. Thanks for the changes. Nice to be able to specify which coarse-grained winds to output. Had one comment about describing some of the parameters.
Maybe add an issue to say we've copied cubed_a2d and that we might want to refactor both instances to a common place at some point.
frodre
pushed a commit
that referenced
this pull request
Jul 12, 2020
nbren12
pushed a commit
that referenced
this pull request
Apr 11, 2022
nbren12
pushed a commit
that referenced
this pull request
Apr 11, 2022
Major changes: - `Partitioner` class added to handle domain decomposition and inter-rank communication - filesystem module added to use fsspec to handle remote locations - Added logic to manage model properties into fv3util. Dynamics and physics properties are stored at the package level in JSON files, while tracer properties are registered externally. This allows support for different microphysics packages and field tables, and allows the wrapper to automatically register the tracers being used by the present model configuration. - Added `open_restart` routine to load restart files. One rank per tile loads the file, and then distributes the data over MPI to each other process. Can choose to load only a subset of the variables to reduce data transfer when the directory is remote. - Added `apply_nudging` and `get_nudging_tendencies` functions to nudge a model state to a reference state according to given timescales and a timestep. - Added `ZarrMonitor` object which stores a sequence of model states in a Zarr store. Data is chunked such that each process is responsible for one chunk (with the last process being responsible for two chunks, in the case of interface-level variables). - Significant additions and reworks to documentation. Removed outdated build instructions and added docker-centric build instructions. Added developer notes about fv3util. - Added docs-docker make target to build docs inside docker - Removed old, unused `get_output_array` routine in the `_wrapper.so`. Minor changes: - Nudging example runfile added - Updated target fortran sources, requiring small tweaks to docker build instructions - Added docstrings to any routines missing them in the API docs - Removed recommended installation targets for apt-get and reduced the number of RUN commands in docker to reduce disk space requirements for intermediate images - Added new dependencies for zarr writing and documentation building to docker image - Changed documentation theme to readthedocs theme - Added IDE directory to gitignore Commit history: * remove reference to old repo * add reference to fv3atm repo * first running docker build with fv3atm * use ARG for root of fv3gfs build root in fortran image * update permissions of fv3config cache directory to allow writes * ignore DS_Store files * add back legacy restart tests, disabled because not passing * Add regression tests which use fv3gfs-fortran reference files * update fv3config and fv3gfs-fortran to latest master * update fv3config to v0.2.0 * only run tagged build on master * bind-mount docker credentials for all tests * fixed label for smallest version increment from bugfix to patch * move image tests outside of in-docker test script * remove redundant tagged-build workflow * install python test dependencies on machine * update pip and setuptools * install fv3config for tests * use local fv3config instead of git commit * initial move of routines into fv3util * move routines from restart into fv3util * run fv3util tests * fix build process to install new package and use its json files * require fv3util to have same version as fv3gfs-python * move some ghost cell logic back to fv3gfs, remove legacy restart logic to be replaced later * add xarray and numpy dependencies * include package data files when installing * copy properties json files from correct location in dockerfile * use properties dictionaries in fv3util to fill templates * various fixes to get tests non-failing * comment out restart tests, to be re-enabled in next PR with nudging * update fv3gfs-fortran * enable python 2 for image upload on circle ci * use master for pushing images, including unversioned * set fv3util version to match python wrapper * run pure python tests before building model * don't repeat python tests outside of docker in build step * add fv3util tests inside of built image * use correct requirements.txt file for dependency cache checksum * add back pytest installation for image tests * added nudging code with passing tests * initial work on open_restart * completed untested code for reading restart * Added working tests for several legacy restart utility functions * also set units metadata * make restart load codes work for gcsfs paths * Add public nudging functions to top-level namespace * Added initial version of ZarrMonitor * updating docs * add directives for building docs in container * updated docstrings * initial version of zarr monitor and domain decomposition * zarr monitor working on multiple processors with basic tests * update Partitioner signature in docs * add docstrings and type hints * updated documentation with docs-docker build instructions * recursively update submodules on circleci * add --recursive to submodule update command in README * combine init and update recursive submodule commands in git README instructions * fixed dummy comm object to correctly broadcast multiple values * also submodule init recursively on circleci * store image test output as artifacts on circleci * delete grb files from correct folder in circleci cleanup * add json files to manifest in fv3util * Added scatter tests including interface variable tests * Added python package caching to circleci * Update external/fv3util/fv3util/domain.py Co-Authored-By: Oliver Watt-Meyer <[email protected]> * fixes per oliwm's review * updates from reviewer comments from nbren and oliwm * remove nz from partitioner, represent using dim_lengths dict in ArrayMetadata * test xr.open_zarr can open zarr store * add dask to requirements.txt * add toolz to requirements.txt * Add xr.open_zarr dependencies to dockerfile * fix typo in exception message * fix newlines * add history changes for #34 Co-authored-by: Oliver Watt-Meyer <[email protected]>
nbren12
pushed a commit
that referenced
this pull request
Apr 11, 2022
Major changes: - `Partitioner` class added to handle domain decomposition and inter-rank communication - filesystem module added to use fsspec to handle remote locations - Added logic to manage model properties into fv3util. Dynamics and physics properties are stored at the package level in JSON files, while tracer properties are registered externally. This allows support for different microphysics packages and field tables, and allows the wrapper to automatically register the tracers being used by the present model configuration. - Added `open_restart` routine to load restart files. One rank per tile loads the file, and then distributes the data over MPI to each other process. Can choose to load only a subset of the variables to reduce data transfer when the directory is remote. - Added `apply_nudging` and `get_nudging_tendencies` functions to nudge a model state to a reference state according to given timescales and a timestep. - Added `ZarrMonitor` object which stores a sequence of model states in a Zarr store. Data is chunked such that each process is responsible for one chunk (with the last process being responsible for two chunks, in the case of interface-level variables). - Significant additions and reworks to documentation. Removed outdated build instructions and added docker-centric build instructions. Added developer notes about fv3util. - Added docs-docker make target to build docs inside docker - Removed old, unused `get_output_array` routine in the `_wrapper.so`. Minor changes: - Nudging example runfile added - Updated target fortran sources, requiring small tweaks to docker build instructions - Added docstrings to any routines missing them in the API docs - Removed recommended installation targets for apt-get and reduced the number of RUN commands in docker to reduce disk space requirements for intermediate images - Added new dependencies for zarr writing and documentation building to docker image - Changed documentation theme to readthedocs theme - Added IDE directory to gitignore Commit history: * remove reference to old repo * add reference to fv3atm repo * first running docker build with fv3atm * use ARG for root of fv3gfs build root in fortran image * update permissions of fv3config cache directory to allow writes * ignore DS_Store files * add back legacy restart tests, disabled because not passing * Add regression tests which use fv3gfs-fortran reference files * update fv3config and fv3gfs-fortran to latest master * update fv3config to v0.2.0 * only run tagged build on master * bind-mount docker credentials for all tests * fixed label for smallest version increment from bugfix to patch * move image tests outside of in-docker test script * remove redundant tagged-build workflow * install python test dependencies on machine * update pip and setuptools * install fv3config for tests * use local fv3config instead of git commit * initial move of routines into fv3util * move routines from restart into fv3util * run fv3util tests * fix build process to install new package and use its json files * require fv3util to have same version as fv3gfs-python * move some ghost cell logic back to fv3gfs, remove legacy restart logic to be replaced later * add xarray and numpy dependencies * include package data files when installing * copy properties json files from correct location in dockerfile * use properties dictionaries in fv3util to fill templates * various fixes to get tests non-failing * comment out restart tests, to be re-enabled in next PR with nudging * update fv3gfs-fortran * enable python 2 for image upload on circle ci * use master for pushing images, including unversioned * set fv3util version to match python wrapper * run pure python tests before building model * don't repeat python tests outside of docker in build step * add fv3util tests inside of built image * use correct requirements.txt file for dependency cache checksum * add back pytest installation for image tests * added nudging code with passing tests * initial work on open_restart * completed untested code for reading restart * Added working tests for several legacy restart utility functions * also set units metadata * make restart load codes work for gcsfs paths * Add public nudging functions to top-level namespace * Added initial version of ZarrMonitor * updating docs * add directives for building docs in container * updated docstrings * initial version of zarr monitor and domain decomposition * zarr monitor working on multiple processors with basic tests * update Partitioner signature in docs * add docstrings and type hints * updated documentation with docs-docker build instructions * recursively update submodules on circleci * add --recursive to submodule update command in README * combine init and update recursive submodule commands in git README instructions * fixed dummy comm object to correctly broadcast multiple values * also submodule init recursively on circleci * store image test output as artifacts on circleci * delete grb files from correct folder in circleci cleanup * add json files to manifest in fv3util * Added scatter tests including interface variable tests * Added python package caching to circleci * Update external/fv3util/fv3util/domain.py Co-Authored-By: Oliver Watt-Meyer <[email protected]> * fixes per oliwm's review * updates from reviewer comments from nbren and oliwm * remove nz from partitioner, represent using dim_lengths dict in ArrayMetadata * test xr.open_zarr can open zarr store * add dask to requirements.txt * add toolz to requirements.txt * Add xr.open_zarr dependencies to dockerfile * fix typo in exception message * fix newlines * add history changes for #34 Co-authored-by: Oliver Watt-Meyer <[email protected]>
nbren12
pushed a commit
that referenced
this pull request
Apr 11, 2022
Major changes: - Added Quantity object to replace DataArray in the state dictionary. Quantity.view[:] provides something akin to DataArray.values, except that it may be a cupy or numpy array. Quantity.values is a read-only numpy array. - Partitioner has been replaced by TilePartitioner - Added TilePartitioner and CubedSpherePartitioner objects for domain decomposition. - Added TileCommunicator and CubedSphereCommunicator objects for inter-rank communication. - Tile scattering is now in `TileCommunicator.scatter`. - Added `to_dataset(state)` which turns a state dictionary into an xarray Dataset object. - Added constants `X_DIM, X_INTERFACE_DIM, Y_DIM, Y_INTERFACE_DIM, Z_DIM, Z_INTERFACE_DIM, X_DIMS, Y_DIMS, HORIZONTAL_DIMS`, and `INTERFACE_DIMS` - updated ZarrMonitor, `get_state`, `set_state`, `open_restart`, `apply_nudging`, `get_nudging_tendencies`, `read_state`, `write_state` and tests to work with a Quantity-based state. - updated `open_restart` to work with the new communicator objects - Removed `without_ghost_cells` and `with_ghost_cells`, as a ghost-cell independent view is now available as `Quantity.view`. - examples are updated for the new code and now run on CircleCI Minor changes: - Added C12 regression test for `open_restart` - added scipy and pytest-subtests to requirements and docker container - fixed docker container so that `dev_docker.sh` no longer overwrites ESMF and FMS installations - Removed the old "integration" tests, as their functionality (at least what was enabled) is covered entirely by the newer "image" tests which are compared against the results from the pure-fortran repo. When we add these back for more complex integration tests, we should do it in a way that leverages `fv3run` (similar to how examples are being treated). Commit history (a little polluted by merges): * remove reference to old repo * add reference to fv3atm repo * first running docker build with fv3atm * use ARG for root of fv3gfs build root in fortran image * update permissions of fv3config cache directory to allow writes * ignore DS_Store files * add back legacy restart tests, disabled because not passing * Add regression tests which use fv3gfs-fortran reference files * update fv3config and fv3gfs-fortran to latest master * update fv3config to v0.2.0 * only run tagged build on master * bind-mount docker credentials for all tests * fixed label for smallest version increment from bugfix to patch * move image tests outside of in-docker test script * remove redundant tagged-build workflow * install python test dependencies on machine * update pip and setuptools * install fv3config for tests * use local fv3config instead of git commit * initial move of routines into fv3util * move routines from restart into fv3util * run fv3util tests * fix build process to install new package and use its json files * require fv3util to have same version as fv3gfs-python * move some ghost cell logic back to fv3gfs, remove legacy restart logic to be replaced later * add xarray and numpy dependencies * include package data files when installing * copy properties json files from correct location in dockerfile * use properties dictionaries in fv3util to fill templates * various fixes to get tests non-failing * comment out restart tests, to be re-enabled in next PR with nudging * update fv3gfs-fortran * enable python 2 for image upload on circle ci * use master for pushing images, including unversioned * set fv3util version to match python wrapper * run pure python tests before building model * don't repeat python tests outside of docker in build step * add fv3util tests inside of built image * use correct requirements.txt file for dependency cache checksum * add back pytest installation for image tests * added nudging code with passing tests * initial work on open_restart * completed untested code for reading restart * Added working tests for several legacy restart utility functions * also set units metadata * make restart load codes work for gcsfs paths * Add public nudging functions to top-level namespace * Added initial version of ZarrMonitor * updating docs * add directives for building docs in container * updated docstrings * initial version of zarr monitor and domain decomposition * zarr monitor working on multiple processors with basic tests * update Partitioner signature in docs * add docstrings and type hints * updated documentation with docs-docker build instructions * recursively update submodules on circleci * add --recursive to submodule update command in README * combine init and update recursive submodule commands in git README instructions * fixed dummy comm object to correctly broadcast multiple values * also submodule init recursively on circleci * store image test output as artifacts on circleci * delete grb files from correct folder in circleci cleanup * add json files to manifest in fv3util * Added scatter tests including interface variable tests * Added python package caching to circleci * added Quantity class and started replacing xarray usage with quantity * Update external/fv3util/fv3util/domain.py Co-Authored-By: Oliver Watt-Meyer <[email protected]> * fixes per oliwm's review * Add edge and corner boundary routines with tests * refactor communications routines out of partitioner into new objects * test updates for new quantity * added boundary_data method to quantity * added placeholder test files" * updates from reviewer comments from nbren and oliwm * remove nz from partitioner, represent using dim_lengths dict in ArrayMetadata * working zeros transfer for non-vector halo update * added full zeros and boundary distance tests for halo updates also fixed ZarrMonitor tests, added tests on 1-rank tiles that the correct ranks are sending/receiving to one another in halo updates, added sel() convenience function for Quantity which might be removed. * cleanups while writing spec in paper * Add PR #34 changes to HISTORY.md * undo history changes meant for another branch * fix bug where rotated data was not correctly copied if n_rot=3 * move boundary slice responsibility from Quantity to Boundary * fixed style error * remove nx and ny from HorizontalGridSpec, use Quantity shape instead * remove old ghost cell routines, now obsolete due to how Quantity functions * replace get_tile_number with get_tile_index, update __init__ files, add coverage make target * use Quantity in getters instead of DataArray * update setters to use Quantity instead of DataArray * adding docstrings * fix zero-dimension array indexing * remove unused code, add more type hints and docstrings, add to_dataset for state * removed HorizontalGridSpec * add more boundary tests * add boundary methods for tile, use for part of cubed sphere boundary methods * integrate more of tile.boundary into cube.boundary, fix caching of mutable objects * update documentation with new quantity and communication features * add pytest-subtests to requirements * rename TileCommunicator.tile_scatter to TileCommunicator.scatter * add constants and UnitsError to top-level namespace * fix fv3gfs tests * add restart data for new restart example test, perform fixes to tests from merge * fix bug in zarr monitor tests * update submodules * added scipy to requirements * some docs tidying * fixed cross-reference link in docs * added some type hinting in quantity * added more docstrings * import ModuleType from the right place * move DummyComm into fv3util.testing * add required packages to fv3util * started updating examples to work with new code, integrated examples into CI tests * fix circleci config * attempt to fix online code example * remove units check from nudging code * fix online code example, add gs cache to docker image * add nudging to cases tested on CI * update set_state docstring * move broadcast_state to TileCommunicator.scatter_state * remove integration_scripts tests, as they are covered entirely by the image tests (test that ptyhon and fortran run and are equal) * remove large and unnecessary forcings from cache * remove also unused cache files from the old-style cache * only run examples that use public data on circleci * remove dead code * update docstring * remove empty test files * response to reviewer comments * add history entries for #41 to HISTORY.md Co-authored-by: Oliver Watt-Meyer <[email protected]>
nbren12
pushed a commit
that referenced
this pull request
Apr 11, 2022
Major changes: - Added Quantity object to replace DataArray in the state dictionary. Quantity.view[:] provides something akin to DataArray.values, except that it may be a cupy or numpy array. Quantity.values is a read-only numpy array. - Partitioner has been replaced by TilePartitioner - Added TilePartitioner and CubedSpherePartitioner objects for domain decomposition. - Added TileCommunicator and CubedSphereCommunicator objects for inter-rank communication. - Tile scattering is now in `TileCommunicator.scatter`. - Added `to_dataset(state)` which turns a state dictionary into an xarray Dataset object. - Added constants `X_DIM, X_INTERFACE_DIM, Y_DIM, Y_INTERFACE_DIM, Z_DIM, Z_INTERFACE_DIM, X_DIMS, Y_DIMS, HORIZONTAL_DIMS`, and `INTERFACE_DIMS` - updated ZarrMonitor, `get_state`, `set_state`, `open_restart`, `apply_nudging`, `get_nudging_tendencies`, `read_state`, `write_state` and tests to work with a Quantity-based state. - updated `open_restart` to work with the new communicator objects - Removed `without_ghost_cells` and `with_ghost_cells`, as a ghost-cell independent view is now available as `Quantity.view`. - examples are updated for the new code and now run on CircleCI Minor changes: - Added C12 regression test for `open_restart` - added scipy and pytest-subtests to requirements and docker container - fixed docker container so that `dev_docker.sh` no longer overwrites ESMF and FMS installations - Removed the old "integration" tests, as their functionality (at least what was enabled) is covered entirely by the newer "image" tests which are compared against the results from the pure-fortran repo. When we add these back for more complex integration tests, we should do it in a way that leverages `fv3run` (similar to how examples are being treated). Commit history (a little polluted by merges): * remove reference to old repo * add reference to fv3atm repo * first running docker build with fv3atm * use ARG for root of fv3gfs build root in fortran image * update permissions of fv3config cache directory to allow writes * ignore DS_Store files * add back legacy restart tests, disabled because not passing * Add regression tests which use fv3gfs-fortran reference files * update fv3config and fv3gfs-fortran to latest master * update fv3config to v0.2.0 * only run tagged build on master * bind-mount docker credentials for all tests * fixed label for smallest version increment from bugfix to patch * move image tests outside of in-docker test script * remove redundant tagged-build workflow * install python test dependencies on machine * update pip and setuptools * install fv3config for tests * use local fv3config instead of git commit * initial move of routines into fv3util * move routines from restart into fv3util * run fv3util tests * fix build process to install new package and use its json files * require fv3util to have same version as fv3gfs-python * move some ghost cell logic back to fv3gfs, remove legacy restart logic to be replaced later * add xarray and numpy dependencies * include package data files when installing * copy properties json files from correct location in dockerfile * use properties dictionaries in fv3util to fill templates * various fixes to get tests non-failing * comment out restart tests, to be re-enabled in next PR with nudging * update fv3gfs-fortran * enable python 2 for image upload on circle ci * use master for pushing images, including unversioned * set fv3util version to match python wrapper * run pure python tests before building model * don't repeat python tests outside of docker in build step * add fv3util tests inside of built image * use correct requirements.txt file for dependency cache checksum * add back pytest installation for image tests * added nudging code with passing tests * initial work on open_restart * completed untested code for reading restart * Added working tests for several legacy restart utility functions * also set units metadata * make restart load codes work for gcsfs paths * Add public nudging functions to top-level namespace * Added initial version of ZarrMonitor * updating docs * add directives for building docs in container * updated docstrings * initial version of zarr monitor and domain decomposition * zarr monitor working on multiple processors with basic tests * update Partitioner signature in docs * add docstrings and type hints * updated documentation with docs-docker build instructions * recursively update submodules on circleci * add --recursive to submodule update command in README * combine init and update recursive submodule commands in git README instructions * fixed dummy comm object to correctly broadcast multiple values * also submodule init recursively on circleci * store image test output as artifacts on circleci * delete grb files from correct folder in circleci cleanup * add json files to manifest in fv3util * Added scatter tests including interface variable tests * Added python package caching to circleci * added Quantity class and started replacing xarray usage with quantity * Update external/fv3util/fv3util/domain.py Co-Authored-By: Oliver Watt-Meyer <[email protected]> * fixes per oliwm's review * Add edge and corner boundary routines with tests * refactor communications routines out of partitioner into new objects * test updates for new quantity * added boundary_data method to quantity * added placeholder test files" * updates from reviewer comments from nbren and oliwm * remove nz from partitioner, represent using dim_lengths dict in ArrayMetadata * working zeros transfer for non-vector halo update * added full zeros and boundary distance tests for halo updates also fixed ZarrMonitor tests, added tests on 1-rank tiles that the correct ranks are sending/receiving to one another in halo updates, added sel() convenience function for Quantity which might be removed. * cleanups while writing spec in paper * Add PR #34 changes to HISTORY.md * undo history changes meant for another branch * fix bug where rotated data was not correctly copied if n_rot=3 * move boundary slice responsibility from Quantity to Boundary * fixed style error * remove nx and ny from HorizontalGridSpec, use Quantity shape instead * remove old ghost cell routines, now obsolete due to how Quantity functions * replace get_tile_number with get_tile_index, update __init__ files, add coverage make target * use Quantity in getters instead of DataArray * update setters to use Quantity instead of DataArray * adding docstrings * fix zero-dimension array indexing * remove unused code, add more type hints and docstrings, add to_dataset for state * removed HorizontalGridSpec * add more boundary tests * add boundary methods for tile, use for part of cubed sphere boundary methods * integrate more of tile.boundary into cube.boundary, fix caching of mutable objects * update documentation with new quantity and communication features * add pytest-subtests to requirements * rename TileCommunicator.tile_scatter to TileCommunicator.scatter * add constants and UnitsError to top-level namespace * fix fv3gfs tests * add restart data for new restart example test, perform fixes to tests from merge * fix bug in zarr monitor tests * update submodules * added scipy to requirements * some docs tidying * fixed cross-reference link in docs * added some type hinting in quantity * added more docstrings * import ModuleType from the right place * move DummyComm into fv3util.testing * add required packages to fv3util * started updating examples to work with new code, integrated examples into CI tests * fix circleci config * attempt to fix online code example * remove units check from nudging code * fix online code example, add gs cache to docker image * add nudging to cases tested on CI * update set_state docstring * move broadcast_state to TileCommunicator.scatter_state * remove integration_scripts tests, as they are covered entirely by the image tests (test that ptyhon and fortran run and are equal) * remove large and unnecessary forcings from cache * remove also unused cache files from the old-style cache * only run examples that use public data on circleci * remove dead code * update docstring * remove empty test files * response to reviewer comments * add history entries for #41 to HISTORY.md Co-authored-by: Oliver Watt-Meyer <[email protected]>
nbren12
pushed a commit
that referenced
this pull request
Apr 11, 2022
* remove reference to old repo * add reference to fv3atm repo * first running docker build with fv3atm * use ARG for root of fv3gfs build root in fortran image * update permissions of fv3config cache directory to allow writes * ignore DS_Store files * add back legacy restart tests, disabled because not passing * Add regression tests which use fv3gfs-fortran reference files * update fv3config and fv3gfs-fortran to latest master * update fv3config to v0.2.0 * only run tagged build on master * bind-mount docker credentials for all tests * fixed label for smallest version increment from bugfix to patch * move image tests outside of in-docker test script * remove redundant tagged-build workflow * install python test dependencies on machine * update pip and setuptools * install fv3config for tests * use local fv3config instead of git commit * initial move of routines into fv3util * move routines from restart into fv3util * run fv3util tests * fix build process to install new package and use its json files * require fv3util to have same version as fv3gfs-python * move some ghost cell logic back to fv3gfs, remove legacy restart logic to be replaced later * add xarray and numpy dependencies * include package data files when installing * copy properties json files from correct location in dockerfile * use properties dictionaries in fv3util to fill templates * various fixes to get tests non-failing * comment out restart tests, to be re-enabled in next PR with nudging * update fv3gfs-fortran * enable python 2 for image upload on circle ci * use master for pushing images, including unversioned * set fv3util version to match python wrapper * run pure python tests before building model * don't repeat python tests outside of docker in build step * add fv3util tests inside of built image * use correct requirements.txt file for dependency cache checksum * add back pytest installation for image tests * added nudging code with passing tests * initial work on open_restart * completed untested code for reading restart * Added working tests for several legacy restart utility functions * also set units metadata * make restart load codes work for gcsfs paths * Add public nudging functions to top-level namespace * Added initial version of ZarrMonitor * updating docs * add directives for building docs in container * updated docstrings * initial version of zarr monitor and domain decomposition * zarr monitor working on multiple processors with basic tests * update Partitioner signature in docs * add docstrings and type hints * updated documentation with docs-docker build instructions * recursively update submodules on circleci * add --recursive to submodule update command in README * combine init and update recursive submodule commands in git README instructions * fixed dummy comm object to correctly broadcast multiple values * also submodule init recursively on circleci * store image test output as artifacts on circleci * delete grb files from correct folder in circleci cleanup * add json files to manifest in fv3util * Added scatter tests including interface variable tests * Added python package caching to circleci * added Quantity class and started replacing xarray usage with quantity * Update external/fv3util/fv3util/domain.py Co-Authored-By: Oliver Watt-Meyer <[email protected]> * fixes per oliwm's review * Add edge and corner boundary routines with tests * refactor communications routines out of partitioner into new objects * test updates for new quantity * added boundary_data method to quantity * added placeholder test files" * updates from reviewer comments from nbren and oliwm * remove nz from partitioner, represent using dim_lengths dict in ArrayMetadata * working zeros transfer for non-vector halo update * added full zeros and boundary distance tests for halo updates also fixed ZarrMonitor tests, added tests on 1-rank tiles that the correct ranks are sending/receiving to one another in halo updates, added sel() convenience function for Quantity which might be removed. * cleanups while writing spec in paper * Add PR #34 changes to HISTORY.md * undo history changes meant for another branch * fix bug where rotated data was not correctly copied if n_rot=3 * move boundary slice responsibility from Quantity to Boundary * fixed style error * remove nx and ny from HorizontalGridSpec, use Quantity shape instead * remove old ghost cell routines, now obsolete due to how Quantity functions * replace get_tile_number with get_tile_index, update __init__ files, add coverage make target * use Quantity in getters instead of DataArray * update setters to use Quantity instead of DataArray * adding docstrings * fix zero-dimension array indexing * remove unused code, add more type hints and docstrings, add to_dataset for state * removed HorizontalGridSpec * add more boundary tests * add boundary methods for tile, use for part of cubed sphere boundary methods * integrate more of tile.boundary into cube.boundary, fix caching of mutable objects * update documentation with new quantity and communication features * add pytest-subtests to requirements * rename TileCommunicator.tile_scatter to TileCommunicator.scatter * add constants and UnitsError to top-level namespace * fix fv3gfs tests * add restart data for new restart example test, perform fixes to tests from merge * fix bug in zarr monitor tests * update submodules * added scipy to requirements * some docs tidying * fixed cross-reference link in docs * added some type hinting in quantity * added more docstrings * import ModuleType from the right place * move DummyComm into fv3util.testing * add gt4py to docker image * integrate gt4py into quantity, run pytest tests using gt4py * remove from_storage method, duplicates functionality of normal init * remove dawn from docker image for now * remove empty files * Update docker/Dockerfile Co-Authored-By: Tobias Wicky <[email protected]> * Update docker/Dockerfile * add history from #44 Co-authored-by: Oliver Watt-Meyer <[email protected]> Co-authored-by: Tobias Wicky <[email protected]>
nbren12
pushed a commit
that referenced
this pull request
Apr 11, 2022
* remove reference to old repo * add reference to fv3atm repo * first running docker build with fv3atm * use ARG for root of fv3gfs build root in fortran image * update permissions of fv3config cache directory to allow writes * ignore DS_Store files * add back legacy restart tests, disabled because not passing * Add regression tests which use fv3gfs-fortran reference files * update fv3config and fv3gfs-fortran to latest master * update fv3config to v0.2.0 * only run tagged build on master * bind-mount docker credentials for all tests * fixed label for smallest version increment from bugfix to patch * move image tests outside of in-docker test script * remove redundant tagged-build workflow * install python test dependencies on machine * update pip and setuptools * install fv3config for tests * use local fv3config instead of git commit * initial move of routines into fv3util * move routines from restart into fv3util * run fv3util tests * fix build process to install new package and use its json files * require fv3util to have same version as fv3gfs-python * move some ghost cell logic back to fv3gfs, remove legacy restart logic to be replaced later * add xarray and numpy dependencies * include package data files when installing * copy properties json files from correct location in dockerfile * use properties dictionaries in fv3util to fill templates * various fixes to get tests non-failing * comment out restart tests, to be re-enabled in next PR with nudging * update fv3gfs-fortran * enable python 2 for image upload on circle ci * use master for pushing images, including unversioned * set fv3util version to match python wrapper * run pure python tests before building model * don't repeat python tests outside of docker in build step * add fv3util tests inside of built image * use correct requirements.txt file for dependency cache checksum * add back pytest installation for image tests * added nudging code with passing tests * initial work on open_restart * completed untested code for reading restart * Added working tests for several legacy restart utility functions * also set units metadata * make restart load codes work for gcsfs paths * Add public nudging functions to top-level namespace * Added initial version of ZarrMonitor * updating docs * add directives for building docs in container * updated docstrings * initial version of zarr monitor and domain decomposition * zarr monitor working on multiple processors with basic tests * update Partitioner signature in docs * add docstrings and type hints * updated documentation with docs-docker build instructions * recursively update submodules on circleci * add --recursive to submodule update command in README * combine init and update recursive submodule commands in git README instructions * fixed dummy comm object to correctly broadcast multiple values * also submodule init recursively on circleci * store image test output as artifacts on circleci * delete grb files from correct folder in circleci cleanup * add json files to manifest in fv3util * Added scatter tests including interface variable tests * Added python package caching to circleci * added Quantity class and started replacing xarray usage with quantity * Update external/fv3util/fv3util/domain.py Co-Authored-By: Oliver Watt-Meyer <[email protected]> * fixes per oliwm's review * Add edge and corner boundary routines with tests * refactor communications routines out of partitioner into new objects * test updates for new quantity * added boundary_data method to quantity * added placeholder test files" * updates from reviewer comments from nbren and oliwm * remove nz from partitioner, represent using dim_lengths dict in ArrayMetadata * working zeros transfer for non-vector halo update * added full zeros and boundary distance tests for halo updates also fixed ZarrMonitor tests, added tests on 1-rank tiles that the correct ranks are sending/receiving to one another in halo updates, added sel() convenience function for Quantity which might be removed. * cleanups while writing spec in paper * Add PR #34 changes to HISTORY.md * undo history changes meant for another branch * fix bug where rotated data was not correctly copied if n_rot=3 * move boundary slice responsibility from Quantity to Boundary * fixed style error * remove nx and ny from HorizontalGridSpec, use Quantity shape instead * remove old ghost cell routines, now obsolete due to how Quantity functions * replace get_tile_number with get_tile_index, update __init__ files, add coverage make target * use Quantity in getters instead of DataArray * update setters to use Quantity instead of DataArray * adding docstrings * fix zero-dimension array indexing * remove unused code, add more type hints and docstrings, add to_dataset for state * removed HorizontalGridSpec * add more boundary tests * add boundary methods for tile, use for part of cubed sphere boundary methods * integrate more of tile.boundary into cube.boundary, fix caching of mutable objects * update documentation with new quantity and communication features * add pytest-subtests to requirements * rename TileCommunicator.tile_scatter to TileCommunicator.scatter * add constants and UnitsError to top-level namespace * fix fv3gfs tests * add restart data for new restart example test, perform fixes to tests from merge * fix bug in zarr monitor tests * update submodules * added scipy to requirements * some docs tidying * fixed cross-reference link in docs * added some type hinting in quantity * added more docstrings * import ModuleType from the right place * move DummyComm into fv3util.testing * add gt4py to docker image * integrate gt4py into quantity, run pytest tests using gt4py * remove from_storage method, duplicates functionality of normal init * remove dawn from docker image for now * remove empty files * Update docker/Dockerfile Co-Authored-By: Tobias Wicky <[email protected]> * Update docker/Dockerfile * add history from #44 Co-authored-by: Oliver Watt-Meyer <[email protected]> Co-authored-by: Tobias Wicky <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds some experimental functionality for restarting the model using A-grid horizontal winds. The changes required for doing this are small; the diff is mainly comprised of the
cubed_a2dsubroutine I copied fromexternal_ic.F90tofv_io.90to prevent a circular dependency.Results from an initial test at C12 resolution can be found in this notebook, comparing the time series of
UGRD500andVGRD500in a one hour continuous simulation, and in a one hour simulation that was split in half, with the second half restarted from A-grid winds. I think things are working; I don't have a great idea of how well I would expect this to work, particularly at such coarse resolution where the interpolation distances are large.Once we're confident about the way this works though, I will probably add another namelist flag to optionally write out D-grid winds to restart files from a simulation that was restarted with A-grid winds. This would allow us to seamlessly restart a simulation with D-grid winds whose first segment was initialized from restart files with A-grid winds.