Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1pt] PR: Upgrade base Docker image to GDAL v3.8.0 #1058

Merged
merged 32 commits into from
Jan 12, 2024

Conversation

mluck
Copy link
Contributor

@mluck mluck commented Dec 22, 2023

Upgrades base Docker image to GDAL v3.8.0. In order to upgrade past GDAL v.3.4.3 (see #1029), TauDEM's aread8 was replaced with a module from the pyflwdir Python package. Supersedes #1038 and undoes #1047 in favor of upgrading the base Docker image that contains JDK 17 rather than installing it into a base container that uses JDK 11 in order to resolve #1029.

Additions

  • src/accumulate_headwaters.py: Uses pyflwdir to accumulate headwaters and threshold and create stream pixels.

Changes

  • Dockerfile: Upgrade GDAL from v.3.4.3 to v.3.8.0; remove JDK 17 and TauDEM aread8 and threshold.
  • Pipfile and Pipfile.lock: Add pyflwdir and upgrade Python version.
  • src/delineate_hydros_and_produce_HAND.sh: Add src/accumulate_headwaters.py and remove TauDEM aread8 and threshold

Testing

Ran fim_pipeline.sh and synthesize_test_cases.py on HUCs 12040101. fim_pipeline.sh completed without any error, and CSI and MCC for 100yr and 500yr BLE magnitudes remained identical to v.4.4.8.2.

Issuer Checklist (For developer use)

You may update this checklist before and/or after creating the PR. If you're unsure about any of them, please ask, we're here to help! These items are what we are going to look for before merging your code.

  • Informative and human-readable title, using the format: [_pt] PR: <description>
  • Links are provided if this PR resolves an issue, or depends on another other PR
  • If submitting a PR to the dev branch (the default branch), you have a descriptive Feature Branch name using the format: dev-<description-of-change> (e.g. dev-revise-levee-masking)
  • Changes are limited to a single goal (no scope creep)
  • The feature branch you're submitting as a PR is up to date (merged) with the latest dev branch
  • pre-commit hooks were run locally
  • Any change in functionality is tested
  • Passes all unit tests locally (inside interactive Docker container, at /foss_fim/, run: pytest unit_tests/)
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future todos are captured in comments
  • CHANGELOG updated with template version number, e.g. 4.x.x.x
  • Reviewers requested
  • Add yourself as an assignee in the PR as well as the FIM Technical Lead

Merge Checklist (For Technical Lead use only)

  • Update CHANGELOG with latest version number and merge date
  • Update the Citation.cff file to reflect the latest version number in the CHANGELOG
  • If applicable, update README with major alterations

@mluck mluck added refactoring Refacting code to obtain the same result High Priority dependencies Pull requests that update a dependency file FIM4 labels Dec 22, 2023
@mluck mluck requested a review from robgpita December 22, 2023 19:31
@mluck mluck changed the title [1pt] PR: Upgrade base image to GDAL v3.8.0 [1pt] PR: Upgrade base Docker image to GDAL v3.8.0 Dec 22, 2023
@mluck mluck requested review from RobHanna-NOAA and removed request for robgpita December 26, 2023 16:55
@RobHanna-NOAA
Copy link
Contributor

I did a bunch of testing on the code and it was fine. We won't official approve this until we can test it with Nikki next week to see if it passes the security tests. A huc was run against the old code base, then the new one and a fair bit of comparisons were done including looking carefully of key output geotiff files. Also ran the pre and post set through alpha testing and it was fine as well. So.. code is approved, just need the security test.

RobHanna-NOAA
RobHanna-NOAA previously approved these changes Jan 10, 2024
Copy link
Contributor

@RobHanna-NOAA RobHanna-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@CarsonPruitt-NOAA CarsonPruitt-NOAA merged commit e7a6b79 into dev Jan 12, 2024
1 check passed
@CarsonPruitt-NOAA CarsonPruitt-NOAA deleted the dev-gdal-3.8.0-pyflwdir branch January 12, 2024 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file FIM4 High Priority refactoring Refacting code to obtain the same result
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[21pt] Upgrade jdk or our base Docker image
3 participants