Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1ptt] PR: Fix error and warning scan bug #1320

Merged
merged 5 commits into from
Oct 25, 2024
Merged

Conversation

RobHanna-NOAA
Copy link
Contributor

Note: This is built on top of 1318, Fix merge conflict bug, branch dev-4.5.11.0-merge-conflict. It can be tested but can not be merged until 1318 is merged to dev.

The fix: During the post processing scan for the word "error" or "warning", it was only finding records which had either of those two words as stand alone words and not part of bigger phrases. ie); "error" was found, but not "fielderror". Added wildcards and it is now fixed.

Note: it is finding a good handful more errors and warnings that were being missed in earlier code versions.

Changes

fim_post_processing.sh: fix as described.

Testing

You can run pipeline on any huc such as 05030104 (the smallest), then manually add a file in the logs folder which has different permutations of the words error and warning. Some with those words alone other with character front and/or back. Then run post-processing again.

Deployment Plan (For developer use)

How does the changes affect the product?

  • Code only?
  • If applicable, has a deployment plan be created with the deployment person/team?
  • Require new or adjusted data inputs? Does it have start, end and duration code (in UTC)?
  • If new or updated data sets, has the FIM code been updated and tested with the new/adjusted data (subset is fine, but must be a subset of the new data)?
  • Require new pre-clip set?
  • Has new or updated python packages?

Issuer Checklist (For developer use)

You may update this checklist before and/or after creating the PR. If you're unsure about any of them, please ask, we're here to help! These items are what we are going to look for before merging your code.

  • Informative and human-readable title, using the format: [_pt] PR: <description>
  • [ x Links are provided if this PR resolves an issue, or depends on another other PR
  • If submitting a PR to the dev branch (the default branch), you have a descriptive Feature Branch name using the format: dev-<description-of-change> (e.g. dev-revise-levee-masking)
  • Changes are limited to a single goal (no scope creep)
  • The feature branch you're submitting as a PR is up to date (merged) with the latest dev branch
  • pre-commit hooks were run locally
  • Any change in functionality is tested
  • n/a - New functions are documented (with a description, list of inputs, and expected output)
  • n/a Placeholder code is flagged / future todos are captured in comments
  • CHANGELOG updated with template version number, e.g. 4.x.x.x
  • Add yourself as an assignee in the PR as well as the FIM Technical Lead

Merge Checklist (For Technical Lead use only)

  • Update CHANGELOG with latest version number and merge date
  • Update the Citation.cff file to reflect the latest version number in the CHANGELOG
  • If applicable, update README with major alterations

@RobHanna-NOAA RobHanna-NOAA self-assigned this Oct 17, 2024
@RobHanna-NOAA RobHanna-NOAA added bug Something isn't working FIM4 labels Oct 17, 2024
@RobHanna-NOAA RobHanna-NOAA linked an issue Oct 17, 2024 that may be closed by this pull request
@RobHanna-NOAA RobHanna-NOAA requested review from AliForghani-NOAA and hhs732 and removed request for AliForghani-NOAA October 17, 2024 23:57
hhs732
hhs732 previously approved these changes Oct 23, 2024
Copy link
Contributor

@hhs732 hhs732 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR was tested by running the fim_pipeline for two HUCs of 05030101 and 12030109 and creating a error file (test_error.log) in the log folder. Different words with error in them (including errortest, error1, failerror, .error., Error, ...) were added to test_error.log and the code could successfully detect them.

@CarsonPruitt-NOAA CarsonPruitt-NOAA merged commit 889a785 into dev Oct 25, 2024
1 check passed
@CarsonPruitt-NOAA CarsonPruitt-NOAA deleted the dev-error-scan branch October 25, 2024 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working FIM4
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[3pt] Post Processing error scan might not be working
3 participants