Skip to content

Update Regionalization to Run as Module Instead of Script #70

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mxkpp merged 4 commits into from
May 13, 2026
Merged

Update Regionalization to Run as Module Instead of Script #70

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
449513c
Update run_region.sh for new way of calling run_regionalization.py as…
mxkpp May 12, 2026
4960bca
run_region.sh: extend container's existing PYTHONPATH
mxkpp May 12, 2026
2cce4f9
change permission of runtime folder
May 13, 2026
3391673
move chmod to be after ensure_dir in run_region.sh
May 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion bin_mounted/ngen_rte/run_regionalization.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ def main(config_dir: str, parreg: bool, formreg: bool, ngen: bool, run_eval: boo
parser.add_argument("--eval", dest="run_eval", action="store_true")
args = parser.parse_args()

assert os.path.exists(args.config_dir)
if not os.path.isdir(args.config_dir):
raise NotADirectoryError(args.config_dir)

main(**vars(args))
25 changes: 16 additions & 9 deletions run_region.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,7 @@ require_dir "${forcing_config_dir}"
# Create run-time temporary directory with timestamp to avoid conflicts between simultaneous runs
RUNTIME_DIR_TMP=$(mktemp -d "${WORK_DIR}/run_time_XXXXXXXX")
ensure_dir "$RUNTIME_DIR_TMP"
chmod a+rx "${RUNTIME_DIR_TMP}"
echo "Created run-time temporary directory: ${RUNTIME_DIR_TMP}."

# create runtime directories for forcing
Expand All @@ -233,8 +234,10 @@ if $DELETE_RUNTIME_DIR; then
trap cleanup EXIT
fi

# script to run with docker run
SCRIPT="${REPOS_COMMON_ROOT__HOST}/nwm-rte/bin_mounted/run_regionalization.py"
# Python module to run with docker run
RUN_REGION_MODULE="ngen_rte.run_regionalization"
# Parent dir of where the ngen_rte module is mounted inside the container (for setting PYTHONPATH)
CONTAINER_PYTHONPATH_ENTRY="${REPOS_COMMON_ROOT__HOST}/nwm-rte/bin_mounted"

# docker image to use
TARGET_IMAGE_NAME="ghcr.io/ngwpc/nwm-rte:${IMAGE_TAG}"
Expand All @@ -248,13 +251,17 @@ if $PULL_IMAGE || ! docker image inspect "${TARGET_IMAGE_NAME}" >/dev/null 2>&1;
docker pull "${TARGET_IMAGE_NAME}"
fi

# Docker run function
# To run ngen-forcing with debug log, add below: -e "FORCING_LOGLEVEL=DEBUG" \
echo "Getting existing PYTHONPATH from the container..."
CONTAINER_PYTHONPATH_EXISTING="$(docker run --rm --entrypoint sh ${TARGET_IMAGE_NAME} -lc 'printf "%s" "$PYTHONPATH"')"
echo "Existing PYTHONPATH from the container: ${CONTAINER_PYTHONPATH_EXISTING}"
CONTAINER_PYTHONPATH_COMBINED="${CONTAINER_PYTHONPATH_EXISTING}:${CONTAINER_PYTHONPATH_ENTRY}"

function docker_run {
docker run \
--entrypoint python \
--user "$(id -u):$(id -g)" \
-e WORK_DIR="${WORK_DIR}" \
-e PYTHONPATH="${CONTAINER_PYTHONPATH_COMBINED}" \
-e REPOS_COMMON_ROOT__HOST="${REPOS_COMMON_ROOT__HOST}" \
-e HOME="${RUNTIME_DIR_TMP}/home" \
-w "${WORK_DIR}" \
Expand All @@ -265,14 +272,14 @@ function docker_run {
-v "${CONFIG_DIR}:${CONFIG_DIR}" \
-v "${RUNTIME_DIR_TMP}/run_ngen/data:/ngencerf-app/runtime_data" \
-v "${RUNTIME_DIR_TMP}/docker_logs/run:/ngencerf/data/run-logs" \
--rm ${TARGET_IMAGE_NAME} "$@"
--rm ${TARGET_IMAGE_NAME} -um "$@"
}

# Run requested workflow steps
$parreg && docker_run "$SCRIPT" -c "$CONFIG_DIR" --parreg
$formreg && docker_run "$SCRIPT" -c "$CONFIG_DIR" --formreg
$ngen && docker_run "$SCRIPT" -c "$CONFIG_DIR" --ngen
$eval && docker_run "$SCRIPT" -c "$CONFIG_DIR" --eval
$parreg && docker_run "$RUN_REGION_MODULE" -c "$CONFIG_DIR" --parreg
$formreg && docker_run "$RUN_REGION_MODULE" -c "$CONFIG_DIR" --formreg
$ngen && docker_run "$RUN_REGION_MODULE" -c "$CONFIG_DIR" --ngen
$eval && docker_run "$RUN_REGION_MODULE" -c "$CONFIG_DIR" --eval

echo "All requested workflows (${selected_workflows[*]}) completed successfully."
exit 0
Loading