[BUG]: junifer reset fails as it tries to delete junifer_jobs directory

[synchon]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

1. Queue a job (do not submit)
2. Run junifer reset

You see this:

OSError: [Errno 39] Directory not empty: '/home/fraimondo/dev/scratch/neurodc/junifer_jobs'

Expected Behavior

No error, and that directory is not deleted.

Steps To Reproduce

1. Install latest (--pre) from pip
2. Queue any yaml

Environment

junifer:
  version: 0.0.5.dev27
python:
  version: 3.11.8
  implementation: CPython
dependencies:
  click: 8.1.7
  numpy: 1.26.4
  scipy: 1.11.4
  datalad: 0.19.6
  pandas: 2.1.4
  nibabel: 5.2.1
  nilearn: 0.10.2
  sqlalchemy: 2.0.29
  ruamel.yaml: 0.17.40
  httpx: 0.26.0
  tqdm: 4.66.1
  templateflow: 24.2.0
  looseversion: None
system:
  platform: Linux-6.6.13+bpo-amd64-x86_64-with-glibc2.36
environment:
  LC_CTYPE: en_US.UTF-8
  PATH: 
    /home/fraimondo/miniconda3/envs/neurodc/bin:/home/fraimondo/miniconda3/condabin:/home/fraimondo/.dotfiles/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin:/usr/local/games:/usr/games

Relevant log output

2024-04-12 22:35:59,336 - JUNIFER - INFO - ===== Lib Versions =====
2024-04-12 22:35:59,336 - JUNIFER - INFO - numpy: 1.26.4
2024-04-12 22:35:59,336 - JUNIFER - INFO - scipy: 1.11.4
2024-04-12 22:35:59,336 - JUNIFER - INFO - pandas: 2.1.4
2024-04-12 22:35:59,336 - JUNIFER - INFO - nilearn: 0.10.2
2024-04-12 22:35:59,336 - JUNIFER - INFO - nibabel: 5.2.1
2024-04-12 22:35:59,336 - JUNIFER - INFO - junifer: 0.0.5.dev27
2024-04-12 22:35:59,336 - JUNIFER - INFO - ========================
2024-04-12 22:35:59,336 - JUNIFER - INFO - Parsing yaml file: /home/fraimondo/dev/scratch/neurodc/aomic_fc.yaml
2024-04-12 22:35:59,345 - JUNIFER - INFO - Deleting /data/group/riseml/fraimondo/ds003097_FC/ds003097_FC.hdf5
2024-04-12 22:35:59,345 - JUNIFER - INFO - Deleting job directory at /home/fraimondo/dev/scratch/neurodc/junifer_jobs/aomic_fc
Traceback (most recent call last):
  File "/home/fraimondo/miniconda3/envs/neurodc/bin/junifer", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/fraimondo/miniconda3/envs/neurodc/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniconda3/envs/neurodc/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniconda3/envs/neurodc/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniconda3/envs/neurodc/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniconda3/envs/neurodc/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniconda3/envs/neurodc/lib/python3.11/site-packages/junifer/api/cli.py", line 451, in reset
    api_reset(config)
  File "/home/fraimondo/miniconda3/envs/neurodc/lib/python3.11/site-packages/junifer/api/functions.py", line 363, in reset
    job_dir.parent.rmdir()
  File "/home/fraimondo/miniconda3/envs/neurodc/lib/python3.11/pathlib.py", line 1156, in rmdir
    os.rmdir(self)
OSError: [Errno 39] Directory not empty: '/home/fraimondo/dev/scratch/neurodc/junifer_jobs'

Anything else?

No response

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (8f96159) to head (b7f6b89).

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #332   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            1         1           
  Lines            1         1           
=========================================
  Hits             1         1           

Flag	Coverage Δ
docs	100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

[github-actions]

PR Preview Action v1.4.7
Preview removed because the pull request was closed.
2024-04-30 11:42 UTC