Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates to --list_jobs and batch settings #31

Merged
merged 4 commits into from
Nov 27, 2023
Merged

Updates to --list_jobs and batch settings #31

merged 4 commits into from
Nov 27, 2023

Conversation

nweires
Copy link
Collaborator

@nweires nweires commented Nov 21, 2023

  • Update --list_jobs option: Include postprocessing and only show a single matching batch job, since we no longer create unique job IDs automatically. (Maybe we should rename this flag?)
  • Update timeout for simulation tasks: Give what should be plenty of time, but ensure they do timeout eventually, so they don't hang forever if something goes wrong.
  • Update retry policy: Retry if a job is preempted, but not if it fails in other ways. (Got this instructions for this here.)

Example --list_jobs output:

INFO:2023-11-21 15:59:23:buildstockbatch.gcp.gcp:Batch job
INFO:2023-11-21 15:59:23:buildstockbatch.gcp.gcp:  Name: projects/buildstockbatch-dev/locations/us-central1/jobs/list-jobs
INFO:2023-11-21 15:59:23:buildstockbatch.gcp.gcp:  UID: list-jobs-18995f63-a46b-446f-92c9-e200
INFO:2023-11-21 15:59:23:buildstockbatch.gcp.gcp:  Status: SUCCEEDED
INFO:2023-11-21 15:59:23:buildstockbatch.gcp.gcp:  Task statuses: {'SUCCEEDED': 2}
INFO:2023-11-21 15:59:23:buildstockbatch.gcp.gcp:Post-processing Cloud Run job
INFO:2023-11-21 15:59:23:buildstockbatch.gcp.gcp:  Name: list-jobs-pp-jgfrq
INFO:2023-11-21 15:59:23:buildstockbatch.gcp.gcp:  Status of latest run (list-jobs-pp-jgfrq): Completed

And when there are no matching jobs (e.g. if they've been cleaned up):

INFO:2023-11-21 16:34:30:buildstockbatch.gcp.gcp:No existing Batch jobs match: projects/buildstockbatch-dev/locations/us-central1/jobs/test-job-name
INFO:2023-11-21 16:34:30:buildstockbatch.gcp.gcp:See all Batch jobs at https://console.cloud.google.com/batch/jobs?project=buildstockbatch-dev
INFO:2023-11-21 16:34:31:buildstockbatch.gcp.gcp:No existing Cloud Run jobs match projects/buildstockbatch-dev/locations/us-central1/jobs/test-job-name-pp
INFO:2023-11-21 16:34:31:buildstockbatch.gcp.gcp:See all Cloud Run jobs at https://console.cloud.google.com/run/jobs?project=buildstockbatch-dev

@@ -22,7 +22,7 @@ Running a project file is straightforward. Call the ``buildstock_local`` command

Eagle
~~~~~
After you have :ref:`activated the appropriate conda environment on Eagle <eagle_install>`,
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing whitespace was cleaned up by the pre-commit checks.

Copy link

github-actions bot commented Nov 21, 2023

File Coverage
All files 86%
base.py 90%
eagle.py 77%
exc.py 57%
local.py 70%
postprocessing.py 84%
utils.py 91%
cloud/docker_base.py 93%
sampler/base.py 79%
sampler/downselect.py 33%
sampler/precomputed.py 93%
sampler/residential_quota.py 61%
test/test_docker.py 33%
test/test_validation.py 97%
workflow_generator/base.py 90%
workflow_generator/commercial.py 53%
workflow_generator/residential_hpxml.py 86%

Minimum allowed coverage is 33%

Generated by 🐒 cobertura-action against ac67e71

@nweires nweires requested a review from lathanh November 21, 2023 16:47
buildstockbatch/gcp/gcp.py Show resolved Hide resolved
buildstockbatch/gcp/gcp.py Outdated Show resolved Hide resolved
buildstockbatch/gcp/gcp.py Outdated Show resolved Hide resolved
buildstockbatch/gcp/gcp.py Outdated Show resolved Hide resolved
docs/run_sims.rst Outdated Show resolved Hide resolved
@nweires nweires merged commit e034567 into gcp Nov 27, 2023
6 checks passed
@nweires nweires deleted the natalie/list_jobs branch November 27, 2023 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants