Update rocoto task names #2981

WalterKolczynski-NOAA · 2024-10-04T01:07:57Z

Description

Updates the GEFS task names to include the $RUN so they are consistent with the task names in the GFS system. This is one step towards abstracting out common elements.

Also adds an underscore between the RUN and task name for all tasks to make them more readable.

Type of change

Bug fix (fixes something broken)
New feature (adds functionality)
Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

Is this a breaking change (a change in existing functionality)? NO
Does this change require a documentation update? NO
Does this change require an update to any of the following submodules? NO

How has this been tested?

Cycled test on Hercules
GEFS test on Hercules

Checklist

Any dependent changes have been merged and published
My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have documented my code, including function, input, and output descriptions
My changes generate no new warnings
New and existing tests pass with my changes
This change is covered by an existing CI test or a new one has been added
I have made corresponding changes to the system documentation if necessary

KateFriedman-NOAA

Changes look good, nice QOL updates, thanks @WalterKolczynski-NOAA !

DavidHuber-NOAA

Changes look good.

One feature request from this would be to have more logical ensemble metatask names. Since the task names include a number, it seems easy enough to just name the metatasks after the root name of the task (see comments). Would this be an easy add to this PR? If not, I'd be happy to open a follow-on issue.

DavidHuber-NOAA · 2024-10-04T12:17:18Z

workflow/rocoto/gfs_tasks.py

@@ -2680,7 +2680,7 @@ def _get_ecengroups():
 'maxtries': '&MAXTRIES;'
 }

- metatask_dict = {'task_name': f'{self.run}ecmn',
+ metatask_dict = {'task_name': f'{self.run}_ecmn',


Not something necessarily for this PR, but it would be nice to have simpler metatask names, like f'{self.run}_ecen'.

Improving the task names is planned as a next step.

workflow/rocoto/gfs_tasks.py

aerorahul

What @DavidHuber-NOAA said.
An alternative/extension, e.g. eamn -> earcmn to go with earc#grp#

WalterKolczynski-NOAA · 2024-10-04T15:32:52Z

When I do come to updating the task names, it's going to be to real words like recenter or archive.

emcbot · 2024-10-05T06:02:28Z

Experiment C96_atm3DVar FAILED on Hera in Build# 1 in
/scratch1/NCEPDEV/global/CI/2981/RUNTESTS/EXPDIR/C96_atm3DVar_10f5e5b2

emcbot · 2024-10-05T06:02:28Z

Experiment C96C48_hybatmDA FAILED on Hera in Build# 1 in
/scratch1/NCEPDEV/global/CI/2981/RUNTESTS/EXPDIR/C96C48_hybatmDA_10f5e5b2

emcbot · 2024-10-05T06:02:28Z

Experiment C96C48_hybatmaerosnowDA FAILED on Hera in Build# 1 in
/scratch1/NCEPDEV/global/CI/2981/RUNTESTS/EXPDIR/C96C48_hybatmaerosnowDA_10f5e5b2

emcbot · 2024-10-05T06:02:28Z

Experiment C48mx500_3DVarAOWCDA FAILED on Hera in Build# 1 with error logs:

/scratch1/NCEPDEV/global/CI/2981/RUNTESTS/COMROOT/C48mx500_3DVarAOWCDA_10f5e5b2/logs/2021032418/gdas_arch.log

Follow link here to view the contents of the above file(s): (link)

emcbot · 2024-10-05T06:02:29Z

Experiment C96C48_ufs_hybatmDA FAILED on Hera in Build# 1 in
/scratch1/NCEPDEV/global/CI/2981/RUNTESTS/EXPDIR/C96C48_ufs_hybatmDA_10f5e5b2

emcbot · 2024-10-05T06:02:36Z

Experiment C48mx500_3DVarAOWCDA FAILED on Hera in Build# 1 in
/scratch1/NCEPDEV/global/CI/2981/RUNTESTS/EXPDIR/C48mx500_3DVarAOWCDA_10f5e5b2

emcbot · 2024-10-05T10:05:33Z

CI Failed on Hera in Build# 1
Built and ran in directory /scratch1/NCEPDEV/global/CI/2981


Experiment C48_ATM_10f5e5b2 Completed 1 Cycles: *SUCCESS* at Sat Oct  5 04:52:36 UTC 2024
Experiment C96_atm3DVar_10f5e5b2 Terminated with 0
FAIL
FAIL tasks failed and 1 dead at Sat Oct  5 05:29:21 UTC 2024
Experiment C96_atm3DVar_10f5e5b2 Terminated: *FAIL*
Error logs:
/scratch1/NCEPDEV/global/CI/2981/RUNTESTS/COMROOT/C96_atm3DVar_10f5e5b2/logs/2021122100/gdas_arch.log
Experiment C96C48_hybatmaerosnowDA_10f5e5b2 Terminated with 0
FAIL
FAIL tasks failed and 1 dead at Sat Oct  5 05:31:35 UTC 2024
Experiment C96C48_hybatmaerosnowDA_10f5e5b2 Terminated: *FAIL*
Error logs:
/scratch1/NCEPDEV/global/CI/2981/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_10f5e5b2/logs/2021122018/enkfgdas_earc00.log
Experiment C48mx500_3DVarAOWCDA_10f5e5b2 Terminated with 0
FAIL
FAIL tasks failed and 1 dead at Sat Oct  5 05:34:24 UTC 2024
Experiment C48mx500_3DVarAOWCDA_10f5e5b2 Terminated: *FAIL*
Error logs:
/scratch1/NCEPDEV/global/CI/2981/RUNTESTS/COMROOT/C48mx500_3DVarAOWCDA_10f5e5b2/logs/2021032418/gdas_arch.log
Experiment C96C48_ufs_hybatmDA_10f5e5b2 Terminated with 0
FAIL
FAIL tasks failed and 1 dead at Sat Oct  5 05:37:40 UTC 2024
Experiment C96C48_ufs_hybatmDA_10f5e5b2 Terminated: *FAIL*
Experiment C96C48_hybatmDA_10f5e5b2 Terminated with 0
FAIL
FAIL tasks failed and 1 dead at Sat Oct  5 05:37:40 UTC 2024
Experiment C96C48_hybatmDA_10f5e5b2 Terminated: *FAIL*
Error logs:
/scratch1/NCEPDEV/global/CI/2981/RUNTESTS/COMROOT/C96C48_ufs_hybatmDA_10f5e5b2/logs/2024022400/gdas_arch.log
Error logs:
/scratch1/NCEPDEV/global/CI/2981/RUNTESTS/COMROOT/C96C48_hybatmDA_10f5e5b2/logs/2021122100/gdas_arch.log
Experiment C48_S2SW_10f5e5b2 Completed 1 Cycles: *SUCCESS* at Sat Oct  5 08:58:14 UTC 2024
Experiment C48_S2SWA_gefs_10f5e5b2 Completed 1 Cycles: *SUCCESS* at Sat Oct  5 10:05:22 UTC 2024

WalterKolczynski-NOAA · 2024-10-07T16:26:22Z

CI failed because the gdas archive job saves the log files, so those had to be updated for the new task names.

emcbot · 2024-10-09T00:43:07Z

CI Passed on Hercules in Build# 5
Built and ran in directory /work2/noaa/stmp/CI/HERCULES/2981


Experiment C48_ATM_4750db15 Completed 1 Cycles: *SUCCESS* at Tue Oct  8 18:11:55 CDT 2024
Experiment C48_S2SW_4750db15 Completed 1 Cycles: *SUCCESS* at Tue Oct  8 19:12:28 CDT 2024
Experiment C48_S2SWA_gefs_4750db15 Completed 1 Cycles: *SUCCESS* at Tue Oct  8 19:31:31 CDT 2024
Experiment C96_atm3DVar_4750db15 Completed 3 Cycles: *SUCCESS* at Tue Oct  8 19:42:43 CDT 2024
Experiment C96C48_hybatmDA_4750db15 Completed 3 Cycles: *SUCCESS* at Tue Oct  8 19:42:59 CDT 2024

emcbot · 2024-10-09T13:50:48Z

CI Update on Wcoss2 at 10/09/24 01:50:47 PM
============================================
Cloning and Building global-workflow PR: 2981
with PID: 156597 on host: clogin04

emcbot · 2024-10-09T14:35:12Z

Automated global-workflow Testing Results:

Machine: Wcoss2
Start: Wed Oct  9 13:57:22 UTC 2024 on clogin04
---------------------------------------------------
Build: Completed at 10/09/24 02:34:50 PM
Case setup: Completed for experiment C48_ATM_4750db15
Case setup: Skipped for experiment C48mx500_3DVarAOWCDA_4750db15
Case setup: Skipped for experiment C48_S2SWA_gefs_4750db15
Case setup: Completed for experiment C48_S2SW_4750db15
Case setup: Completed for experiment C96_atm3DVar_extended_4750db15
Case setup: Skipped for experiment C96_atm3DVar_4750db15
Case setup: Completed for experiment C96C48_hybatmaerosnowDA_4750db15
Case setup: Completed for experiment C96C48_hybatmDA_4750db15
Case setup: Completed for experiment C96C48_ufs_hybatmDA_4750db15

Updates the GEFS task names to include the `$RUN` so they are consistent with the task names in the GFS system. This is one step towards unifying the systems where possible. Also adds an underscore between the RUN and task name for *all* tasks to make them more readable.

emcbot · 2024-10-10T02:28:14Z

All CI Test Cases Passed on Wcoss2:

Experiment C48_ATM_4750db15 *** SUCCESS *** at 10/09/24 03:56:07 PM
Experiment C48_S2SW_4750db15 *** SUCCESS *** at 10/09/24 04:14:16 PM
Experiment C96C48_hybatmDA_4750db15 *** SUCCESS *** at 10/09/24 04:56:27 PM
Experiment C96C48_hybatmaerosnowDA_4750db15 *** SUCCESS *** at 10/09/24 05:42:24 PM
Experiment C96C48_ufs_hybatmDA_4750db15 *** SUCCESS *** at 10/09/24 06:49:20 PM
Experiment C96_atm3DVar_extended_4750db15 *** SUCCESS *** at 10/10/24 02:21:43 AM

DavidHuber-NOAA

Looks good!

KateFriedman-NOAA

Looks good!

…orkflow into feature/ci_tool * 'feature/ci_tool' of github.com:davidhuber-noaa/global-workflow: Update memory settings in config.resources for job postsnd (NOAA-EMC#3003) Update rocoto task names (NOAA-EMC#2981)

WalterKolczynski-NOAA requested review from aerorahul, KateFriedman-NOAA and DavidHuber-NOAA October 4, 2024 01:07

WalterKolczynski-NOAA added the CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera label Oct 4, 2024

KateFriedman-NOAA previously approved these changes Oct 4, 2024

View reviewed changes

DavidHuber-NOAA previously approved these changes Oct 4, 2024

View reviewed changes

aerorahul previously approved these changes Oct 4, 2024

View reviewed changes

emcbot added CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Oct 5, 2024

emcbot added CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Oct 5, 2024

WalterKolczynski-NOAA dismissed stale reviews from aerorahul, DavidHuber-NOAA, and KateFriedman-NOAA via 7ec3055 October 7, 2024 16:25

WalterKolczynski-NOAA force-pushed the feature/task_names branch from 7ec3055 to 4750db1 Compare October 7, 2024 16:25

WalterKolczynski-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Oct 7, 2024

WalterKolczynski-NOAA added CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules CI-Wcoss2-Ready **CM use only** PR is ready for CI testing on WCOSS labels Oct 8, 2024

emcbot added CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS and removed CI-Wcoss2-Ready **CM use only** PR is ready for CI testing on WCOSS labels Oct 9, 2024

emcbot added CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress and removed CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS labels Oct 9, 2024

WalterKolczynski-NOAA added 2 commits October 9, 2024 15:04

Update archive for new task names

f3493ac

WalterKolczynski-NOAA force-pushed the feature/task_names branch from 4750db1 to f3493ac Compare October 9, 2024 20:04

WalterKolczynski-NOAA requested review from aerorahul, DavidHuber-NOAA and KateFriedman-NOAA October 9, 2024 21:06

emcbot added CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully and removed CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress labels Oct 10, 2024

DavidHuber-NOAA approved these changes Oct 10, 2024

View reviewed changes

KateFriedman-NOAA approved these changes Oct 10, 2024

View reviewed changes

DavidHuber-NOAA merged commit a0771f8 into NOAA-EMC:develop Oct 10, 2024
5 checks passed

AndrewEichmann-NOAA mentioned this pull request Oct 10, 2024

Change in rocoto task names in g-w breaks gw-ci NOAA-EMC/GDASApp#1319

Closed

EricSinsky-NOAA mentioned this pull request Oct 10, 2024

Add more ocean variables for post-processing in GEFS #2995

Open

21 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update rocoto task names #2981

Update rocoto task names #2981

WalterKolczynski-NOAA commented Oct 4, 2024 •

edited

Loading

KateFriedman-NOAA left a comment

DavidHuber-NOAA left a comment

DavidHuber-NOAA Oct 4, 2024

WalterKolczynski-NOAA Oct 4, 2024

aerorahul left a comment

WalterKolczynski-NOAA commented Oct 4, 2024

emcbot commented Oct 5, 2024

emcbot commented Oct 5, 2024

emcbot commented Oct 5, 2024

emcbot commented Oct 5, 2024

emcbot commented Oct 5, 2024

emcbot commented Oct 5, 2024

emcbot commented Oct 5, 2024

WalterKolczynski-NOAA commented Oct 7, 2024

emcbot commented Oct 9, 2024

emcbot commented Oct 9, 2024

emcbot commented Oct 9, 2024

emcbot commented Oct 10, 2024

DavidHuber-NOAA left a comment

KateFriedman-NOAA left a comment

Update rocoto task names #2981

Update rocoto task names #2981

Conversation

WalterKolczynski-NOAA commented Oct 4, 2024 • edited Loading

Description

Type of change

Change characteristics

How has this been tested?

Checklist

KateFriedman-NOAA left a comment

Choose a reason for hiding this comment

DavidHuber-NOAA left a comment

Choose a reason for hiding this comment

DavidHuber-NOAA Oct 4, 2024

Choose a reason for hiding this comment

WalterKolczynski-NOAA Oct 4, 2024

Choose a reason for hiding this comment

aerorahul left a comment

Choose a reason for hiding this comment

WalterKolczynski-NOAA commented Oct 4, 2024

emcbot commented Oct 5, 2024

emcbot commented Oct 5, 2024

emcbot commented Oct 5, 2024

emcbot commented Oct 5, 2024

emcbot commented Oct 5, 2024

emcbot commented Oct 5, 2024

emcbot commented Oct 5, 2024

WalterKolczynski-NOAA commented Oct 7, 2024

emcbot commented Oct 9, 2024

emcbot commented Oct 9, 2024

emcbot commented Oct 9, 2024

emcbot commented Oct 10, 2024

DavidHuber-NOAA left a comment

Choose a reason for hiding this comment

KateFriedman-NOAA left a comment

Choose a reason for hiding this comment

WalterKolczynski-NOAA commented Oct 4, 2024 •

edited

Loading