Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a default suffix to ELP #1378

Merged
merged 11 commits into from
Aug 23, 2024
Merged

Add a default suffix to ELP #1378

merged 11 commits into from
Aug 23, 2024

Conversation

nden
Copy link
Collaborator

@nden nden commented Aug 22, 2024

Resolves RCAL-898

Closes #1379

This PR addresses an issue in OPS where files generated from the ELP do not have the expected suffix.
With this change passing --sufix= to the strun command generates output with the specified suffix. If no suffix is passed, the default suffix is cal.

Checklist

  • added entry in CHANGES.rst under the corresponding subsection
  • updated relevant tests
  • updated relevant documentation
  • updated relevant milestone(s)
  • added relevant label(s)
  • ran regression tests, post a link to the Jenkins job below. How to run regression tests on a PR

@nden nden requested a review from a team as a code owner August 22, 2024 19:25
@nden nden marked this pull request as draft August 22, 2024 19:28
@nden
Copy link
Collaborator Author

nden commented Aug 22, 2024

Marking it as Draft because it needs a test and until it's confirmed this is what SDP needs.

@braingram

This comment was marked as resolved.

@ddavis-stsci
Copy link
Collaborator

With this fix in place I get
for run_elpp.csh
[stpipe.ExposurePipeline.source_catalog] Saved model in ```
r0000101001001001001_01101_0002_WFI10_sourcecatalog.asdf
2024-08-22 15:32:58,667 - stpipe.ExposurePipeline.source_catalog - INFO - Step source_catalog done
[stpipe.ExposurePipeline.source_catalog] Step source_catalog done
[ root] Number of models to tweakreg: (1, 1)
[ root] teakreg STATUS: <romancal.tweakreg.tweakreg_step.TweakRegStep object at 0x16ceef4d0>
[ root] suffix sourcedetection
2024-08-22 15:32:58,716 - stpipe.ExposurePipeline.tweakreg - INFO - Step tweakreg running with args ([<roman_datamodels.datamodels._datamodels.ImageModel object at 0x16d82b710>],).
[stpipe.ExposurePipeline.tweakreg] Step tweakreg running with args ([<roman_datamodels.datamodels._datamodels.ImageModel object at 0x16d82b710>],).
2024-08-22 15:32:58,716 - stpipe.ExposurePipeline.tweakreg - INFO - Step skipped.
[stpipe.ExposurePipeline.tweakreg] Step skipped.
[ root] Roman exposure calibration pipeline ending...
2024-08-22 15:32:59,486 - stpipe.ExposurePipeline - INFO - Saved model in r0000101001001001001_01101_0002_WFI10_sourcedetection.asdf



running tweakreg
I get the expected files (this is one example)

2024-08-22 15:40:49,833 - stpipe.TweakRegStep - INFO - Saved model in r0000101001001001001_01101_0002_WFI10_cal.asdf
2024-08-22 15:40:49,833 - stpipe.TweakRegStep - INFO - Step TweakRegStep done

So it appears that it is now running as I expect. 

@braingram
Copy link
Collaborator

braingram commented Aug 22, 2024

Regtests running at: https://github.com/spacetelescope/RegressionTests/actions/runs/10514634684

EDIT: restarted these after the addition of the new regtest

Copy link
Collaborator

@ddavis-stsci ddavis-stsci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Local tests in the comments

Copy link

codecov bot commented Aug 22, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.61%. Comparing base (b24429c) to head (51b49de).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1378      +/-   ##
==========================================
+ Coverage   78.58%   78.61%   +0.03%     
==========================================
  Files         117      117              
  Lines        7862     7856       -6     
==========================================
- Hits         6178     6176       -2     
+ Misses       1684     1680       -4     
Flag Coverage Δ *Carryforward flag
nightly 62.24% <ø> (-0.03%) ⬇️ Carriedforward from 855da88

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@braingram braingram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! LGTM.

@nden nden marked this pull request as ready for review August 22, 2024 21:13
assert model.meta.cal_step.flat_field == "COMPLETE"
assert model.meta.cal_step.photom == "COMPLETE"
assert model.meta.cal_step.source_detection == "COMPLETE"
assert model.meta.cal_step.tweakreg == "INCOMPLETE"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch.

There's an open issue already for why this isn't SKIPPED: spacetelescope/roman_datamodels#343

@nden nden merged commit c314057 into spacetelescope:main Aug 23, 2024
29 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ELP generating products with conflicting names
3 participants