Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move the variational scripts to this repository #2920

Merged
merged 37 commits into from
Sep 26, 2024
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
c9e41b1
...
guillaumevernieres Aug 28, 2024
53a4de8
var init ok, run needs tlc
guillaumevernieres Sep 4, 2024
ba78b00
updated checkpoint task
guillaumevernieres Sep 9, 2024
0dd87f0
Introduce BASE_DATA variable
KateFriedman-NOAA Sep 11, 2024
4509b0c
working finalize
guillaumevernieres Sep 11, 2024
2b667f2
Merge branch 'develop' into feature/marinevar
guillaumevernieres Sep 11, 2024
eb717a1
Merge branch 'NOAA-EMC:develop' into feature/experimental_obs_path
KateFriedman-NOAA Sep 12, 2024
317e351
...
guillaumevernieres Sep 13, 2024
ea6ae91
removed envar test yaml
guillaumevernieres Sep 13, 2024
5c7eb42
norms, try 1
guillaumevernieres Sep 13, 2024
b4d46aa
Merge branch 'develop' into feature/marinevar
guillaumevernieres Sep 13, 2024
3b15dc6
norms, try 2
guillaumevernieres Sep 13, 2024
90ad1d3
norms, try 3
guillaumevernieres Sep 13, 2024
f1d7f8b
addressed review comments
guillaumevernieres Sep 16, 2024
f84ae82
Merge branch 'develop' into feature/marinevar
guillaumevernieres Sep 16, 2024
3357a0d
Merge remote-tracking branch 'origin/develop' into feature/experiment…
KateFriedman-NOAA Sep 17, 2024
8981d9f
Merge branch 'NOAA-EMC:develop' into feature/experimental_obs_path
KateFriedman-NOAA Sep 18, 2024
9e8bbad
module tidy
guillaumevernieres Sep 18, 2024
a286b1e
Merge branch 'develop' into feature/marinevar
guillaumevernieres Sep 18, 2024
a4d8391
updated gdas and jcb-gdas hashes
guillaumevernieres Sep 18, 2024
2a5e9bf
fixed logging
guillaumevernieres Sep 19, 2024
511daab
Merge branch 'develop' into feature/marinevar
guillaumevernieres Sep 19, 2024
a2eb39e
missing module
guillaumevernieres Sep 20, 2024
259df53
Merge branch 'develop' into feature/marinevar
aerorahul Sep 20, 2024
77f3ddc
Update jobs/JGLOBAL_MARINE_ANALYSIS_FINALIZE
guillaumevernieres Sep 20, 2024
98ace06
Update jobs/JGLOBAL_MARINE_ANALYSIS_FINALIZE
guillaumevernieres Sep 20, 2024
d565d07
Merge branch 'develop' into feature/marinevar
guillaumevernieres Sep 23, 2024
2fa46b0
addressed reviewer's comments
guillaumevernieres Sep 23, 2024
6cc7f8a
Merge branch 'NOAA-EMC:develop' into feature/experimental_obs_path
KateFriedman-NOAA Sep 23, 2024
1fbb19b
Merge branch 'NOAA-EMC:develop' into feature/experimental_obs_path
KateFriedman-NOAA Sep 23, 2024
bd54d24
Update JGLOBAL_MARINE_ANALYSIS_FINALIZE
guillaumevernieres Sep 23, 2024
66c09f9
Merge branch 'develop' into feature/marinevar
guillaumevernieres Sep 23, 2024
0d4456e
Merge remote-tracking branch 'origin/feature/experimental_obs_path' i…
KateFriedman-NOAA Sep 23, 2024
771efe4
Merge pull request #8 from KateFriedman-NOAA/feature/marinevar
guillaumevernieres Sep 24, 2024
131e5eb
addressed reviews
guillaumevernieres Sep 24, 2024
8002a06
Merge branch 'feature/marinevar' of https://github.com/guillaumeverni…
guillaumevernieres Sep 24, 2024
2b43678
Merge branch 'develop' into feature/marinevar
aerorahul Sep 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ci/cases/gfsv17/ocnanal.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ base:
FHMAX_GFS: 240
ACCOUNT: {{ 'HPC_ACCOUNT' | getenv }}

ocnanal:
marineanl:
guillaumevernieres marked this conversation as resolved.
Show resolved Hide resolved
SOCA_INPUT_FIX_DIR: {{ HOMEgfs }}/fix/gdas/soca/1440x1080x75/soca
SOCA_OBS_LIST: {{ HOMEgfs }}/sorc/gdas.cd/parm/soca/obs/obs_list.yaml
SOCA_NINNER: 100
Expand Down
4 changes: 2 additions & 2 deletions env/CONTAINER.env
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ ulimit -s unlimited
ulimit -a


if [ "${step}" = "ocnanalrun" ]; then
if [ "${step}" = "marineanlvar" ]; then
export NTHREADS_OCNANAL=1
export APRUN_OCNANAL="${launcher} -n 2"
export APRUN_MARINEANLVAR="${launcher} -n 2"
fi
10 changes: 4 additions & 6 deletions env/HERA.env
Original file line number Diff line number Diff line change
Expand Up @@ -138,17 +138,15 @@ elif [[ "${step}" = "marinebmat" ]]; then
export APRUNCFP="${launcher} -n \$ncmd --multi-prog"
export APRUN_MARINEBMAT="${APRUN_default}"

elif [[ "${step}" = "ocnanalrun" ]]; then
elif [[ "${step}" = "marineanlvar" ]]; then

export APRUNCFP="${launcher} -n \$ncmd --multi-prog"
export APRUN_MARINEANLVAR="${APRUN_default}"

export APRUN_OCNANAL="${APRUN_default}"

elif [[ "${step}" = "ocnanalchkpt" ]]; then
elif [[ "${step}" = "marineanlchkpt" ]]; then

export APRUNCFP="${launcher} -n \$ncmd --multi-prog"

export APRUN_OCNANAL="${APRUN_default}"
export APRUN_MARINEANLCHKPT="${APRUN_default}"

elif [[ "${step}" = "ocnanalecen" ]]; then

Expand Down
8 changes: 4 additions & 4 deletions env/HERCULES.env
Original file line number Diff line number Diff line change
Expand Up @@ -133,10 +133,10 @@ case ${step} in
export APRUNCFP="${launcher} -n \$ncmd ${mpmd_opt}"
export APRUN_MARINEBMAT="${APRUN_default}"
;;
"ocnanalrun")
"marineanlvar")

export APRUNCFP="${launcher} -n \$ncmd ${mpmd_opt}"
export APRUN_OCNANAL="${APRUN_default}"
export APRUN_MARINEANLVAR="${APRUN_default}"
;;
"ocnanalecen")

Expand All @@ -148,12 +148,12 @@ case ${step} in
[[ ${NTHREADS_OCNANALECEN} -gt ${max_threads_per_task} ]] && export NTHREADS_OCNANALECEN=${max_threads_per_task}
export APRUN_OCNANALECEN="${launcher} -n ${ntasks_ocnanalecen} --cpus-per-task=${NTHREADS_OCNANALECEN}"
;;
"ocnanalchkpt")
"marineanlchkpt")

export APRUNCFP="${launcher} -n \$ncmd ${mpmd_opt}"

export NTHREADS_OCNANAL=${NTHREADSmax}
export APRUN_OCNANAL="${APRUN_default} --cpus-per-task=${NTHREADS_OCNANAL}"
export APRUN_MARINEANLCHKPT="${APRUN_default} --cpus-per-task=${NTHREADS_OCNANAL}"
;;
"anal" | "analcalc")

Expand Down
6 changes: 3 additions & 3 deletions env/JET.env
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ elif [[ "${step}" = "atmensanlsol" ]]; then

export NTHREADS_ATMENSANLSOL=${NTHREADSmax}
export APRUN_ATMENSANLSOL="${APRUN_default}"

elif [[ "${step}" = "atmensanlletkf" ]]; then

export NTHREADS_ATMENSANLLETKF=${NTHREADSmax}
Expand Down Expand Up @@ -121,10 +121,10 @@ elif [[ "${step}" = "marinebmat" ]]; then
export APRUNCFP="${launcher} -n \$ncmd ${mpmd_opt}"
export APRUN_MARINEBMAT="${APRUN_default}"

elif [[ "${step}" = "ocnanalrun" ]]; then
elif [[ "${step}" = "marineanlvar" ]]; then

export APRUNCFP="${launcher} -n \$ncmd ${mpmd_opt}"
export APRUN_OCNANAL="${APRUN_default}"
export APRUN_MARINEANLVAR="${APRUN_default}"

elif [[ "${step}" = "anal" ]] || [[ "${step}" = "analcalc" ]]; then

Expand Down
9 changes: 5 additions & 4 deletions env/ORION.env
Original file line number Diff line number Diff line change
Expand Up @@ -130,18 +130,19 @@ elif [[ "${step}" = "marinebmat" ]]; then
export NTHREADS_MARINEBMAT=${NTHREADSmax}
export APRUN_MARINEBMAT="${APRUN_default}"

elif [[ "${step}" = "ocnanalrun" ]]; then
elif [[ "${step}" = "marineanlvar" ]]; then

export APRUNCFP="${launcher} -n \$ncmd ${mpmd_opt}"

export APRUN_OCNANAL="${APRUN_default}"
export APRUN_MARINEANLVAR="${APRUN_default}"

elif [[ "${step}" = "ocnanalchkpt" ]]; then
elif [[ "${step}" = "marineanlchkpt" ]]; then

export APRUNCFP="${launcher} -n \$ncmd ${mpmd_opt}"

export NTHREADS_OCNANAL=${NTHREADSmax}
export APRUN_OCNANAL="${APRUN_default} --cpus-per-task=${NTHREADS_OCNANAL}"

export APRUN_MARINEANLCHKPT="${APRUN} --cpus-per-task=${NTHREADS_OCNANAL}"

elif [[ "${step}" = "ocnanalecen" ]]; then

Expand Down
58 changes: 0 additions & 58 deletions jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT

This file was deleted.

49 changes: 0 additions & 49 deletions jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_POST

This file was deleted.

2 changes: 1 addition & 1 deletion jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/bash
source "${HOMEgfs}/ush/preamble.sh"
source "${HOMEgfs}/ush/jjob_header.sh" -e "ocnanalprep" -c "base ocnanal ocnanalprep"
source "${HOMEgfs}/ush/jjob_header.sh" -e "marineanlinit" -c "base ocnanal marineanlinit"


##############################################
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
#!/bin/bash
source "${HOMEgfs}/ush/preamble.sh"
export WIPE_DATA="NO"
export DATA="${DATAROOT}/${RUN}ocnanal_${cyc}"
source "${HOMEgfs}/ush/jjob_header.sh" -e "ocnanalrun" -c "base ocnanal ocnanalrun"
export DATAjob="${DATAROOT}/${RUN}marineanalysis.${PDY:-}${cyc}"
guillaumevernieres marked this conversation as resolved.
Show resolved Hide resolved
export DATA="${DATAjob}/marinevariational"
source "${HOMEgfs}/ush/jjob_header.sh" -e "marineanlchkpt" -c "base marineanl marineanlchkpt"


##############################################
Expand All @@ -13,11 +14,10 @@ source "${HOMEgfs}/ush/jjob_header.sh" -e "ocnanalrun" -c "base ocnanal ocnanalr
# Begin JOB SPECIFIC work
##############################################


###############################################################
# Run relevant script

EXSCRIPT=${GDASOCNRUNSH:-${HOMEgfs}/sorc/gdas.cd/scripts/exgdas_global_marine_analysis_run.sh}
EXSCRIPT=${GDASMARINEANALYSIS:-${SCRgfs}/exglobal_marine_analysis_checkpoint.py}
${EXSCRIPT}
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"
Expand All @@ -33,9 +33,4 @@ if [[ -e "${pgmout}" ]] ; then
cat "${pgmout}"
fi

##########################################
# Do not remove the Temporary working directory (do this in POST)
##########################################
cd "${DATAROOT}" || exit 1

exit 0
43 changes: 43 additions & 0 deletions jobs/JGLOBAL_MARINE_ANALYSIS_FINALIZE
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
#!/bin/bash
source "${HOMEgfs}/ush/preamble.sh"
export WIPE_DATA="NO"
export DATAjob="${DATAROOT}/${RUN}marineanalysis.${PDY:-}${cyc}"
guillaumevernieres marked this conversation as resolved.
Show resolved Hide resolved
export DATA="${DATAjob}/marinevariational"
source "${HOMEgfs}/ush/jjob_header.sh" -e "marineanlfinal" -c "base marineanl marineanlfinal"

##############################################
# Set variables used in the script
##############################################

##############################################
# Begin JOB SPECIFIC work
##############################################

# Generate COM variables from templates
YMD=${PDY} HH=${cyc} declare_from_tmpl -rx COMIN_OBS:COM_OBS_TMPL

YMD=${PDY} HH=${cyc} declare_from_tmpl -rx \
COMOUT_OCEAN_ANALYSIS:COM_OCEAN_ANALYSIS_TMPL \
COMOUT_ICE_ANALYSIS:COM_ICE_ANALYSIS_TMPL \
COMOUT_ICE_RESTART:COM_ICE_RESTART_TMPL

mkdir -p "${COMOUT_OCEAN_ANALYSIS}"
guillaumevernieres marked this conversation as resolved.
Show resolved Hide resolved
mkdir -p "${COMOUT_ICE_ANALYSIS}"
mkdir -p "${COMOUT_ICE_RESTART}"

###############################################################
# Run relevant script
###############################################################

EXSCRIPT=${GDASMARINEANALYSIS:-${SCRgfs}/exglobal_marine_analysis_finalize.py}
${EXSCRIPT}
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"

##########################################
# Remove the Temporary working directory
##########################################
cd "${DATAROOT}" || exit 1
[[ "${KEEPDATA}" = "NO" ]] && rm -rf "${DATA}"

exit 0
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
#!/bin/bash

source "${HOMEgfs}/ush/preamble.sh"
export DATA="${DATAROOT}/${RUN}ocnanal_${cyc}"
source "${HOMEgfs}/ush/jjob_header.sh" -e "ocnanalprep" -c "base ocnanal ocnanalprep"
export DATAjob="${DATAROOT}/${RUN}marineanalysis.${PDY:-}${cyc}"
export DATA="${DATAjob}/marinevariational"
source "${HOMEgfs}/ush/jjob_header.sh" -e "marineanlinit" -c "base marineanl marineanlinit"


##############################################
Expand All @@ -10,42 +12,30 @@ source "${HOMEgfs}/ush/jjob_header.sh" -e "ocnanalprep" -c "base ocnanal ocnanal
# Ignore possible spelling error (nothing is misspelled)
# shellcheck disable=SC2153
GDATE=$(date --utc +%Y%m%d%H -d "${PDY} ${cyc} - ${assim_freq} hours")
export GDATE
export gPDY=${GDATE:0:8}
export gcyc=${GDATE:8:2}
gPDY=${GDATE:0:8}
gcyc=${GDATE:8:2}
export GDUMP=${GDUMP:-"gdas"}

export OPREFIX="${RUN}.t${cyc}z."
export GPREFIX="${GDUMP}.t${gcyc}z."
export APREFIX="${RUN}.t${cyc}z."
##############################################
# Begin JOB SPECIFIC work
##############################################

# Generate COM variables from templates
YMD=${PDY} HH=${cyc} declare_from_tmpl -rx COM_OBS

RUN=${GDUMP} YMD=${gPDY} HH=${gcyc} declare_from_tmpl -rx \
COM_OCEAN_HISTORY_PREV:COM_OCEAN_HISTORY_TMPL \
COM_ICE_HISTORY_PREV:COM_ICE_HISTORY_TMPL \
COM_ICE_RESTART_PREV:COM_ICE_RESTART_TMPL
COMIN_OCEAN_HISTORY_PREV:COM_OCEAN_HISTORY_TMPL \
COMIN_ICE_HISTORY_PREV:COM_ICE_HISTORY_TMPL \
COMIN_ICE_RESTART_PREV:COM_ICE_RESTART_TMPL

YMD=${PDY} HH=${cyc} declare_from_tmpl -rx \
COMIN_OCEAN_BMATRIX:COM_OCEAN_BMATRIX_TMPL \
COMIN_ICE_BMATRIX:COM_ICE_BMATRIX_TMPL

##############################################
# Begin JOB SPECIFIC work
##############################################

# Add UFSDA to PYTHONPATH
ufsdaPATH="${HOMEgfs}/sorc/gdas.cd/ush/"
# shellcheck disable=SC2311
pyiodaPATH="${HOMEgfs}/sorc/gdas.cd/build/lib/python$(detect_py_ver)/"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${ufsdaPATH}:${pyiodaPATH}"
export PYTHONPATH

###############################################################
# Run relevant script

EXSCRIPT=${GDASOCNPREPPY:-${HOMEgfs}/sorc/gdas.cd/scripts/exgdas_global_marine_analysis_prep.py}
EXSCRIPT=${GDASMARINEANALYSIS:-${SCRgfs}/exglobal_marine_analysis_initialize.py}
${EXSCRIPT}
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"
Expand Down
Loading
Loading