Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

nipype.interfaces.cat12.preprocess.CAT12Segment does not accept compressed .nii.gz files #3653

Open
JohannesWiesner opened this issue Jun 6, 2024 · 9 comments · May be fixed by #3683
Open

nipype.interfaces.cat12.preprocess.CAT12Segment does not accept compressed .nii.gz files #3653

JohannesWiesner opened this issue Jun 6, 2024 · 9 comments · May be fixed by #3683

Comments

@JohannesWiesner
Copy link

Summary

The Cat12Segment interface does not accept compressed .nii.gz files. Everything works fine when one inputs an .nii file.

Actual behavior

This is the error I get:

TraitError: Each element of the 'in_files' trait of a CAT12SegmentInputSpec instance must be a pathlike object or string representing an existing file, but a value of '/pandora/home/johannes.wiesner/work/testing/debug_cat12/T1w.nii.gz' <class 'str'> was specified.

which by the way is a little bit misleading because the file exists but the file extension if the problem (but this is another topic I guess).

Expected behavior

Run Segmentation on a .nii.gz file should work because newer versions of CAT12 are able to deal with .nii.gz files. My CAT12 version is CAT12.9 (r2560) inside an spm12/9.14 environment module.

See also: https://www.neuro.uni-jena.de/cat12/CAT12-Manual-old.pdf

The segmentation module now supports the input of nii.gz files (not
possible for the longitudinal pipeline).

How to replicate the behavior

Here's an example script that should reproduce the error:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Debug CAT12Segment
"""

import subprocess
from nipype.interfaces.cat12.preprocess import CAT12Segment
from nipype import Node
import os 

###############################################################################
# Preparation: Download an anatomical files from a public available AWS-bucket 
###############################################################################

# download T1w image
command = """
curl \
--create-dirs https://s3.amazonaws.com/openneuro.org/ds004302/sub-02/anat/sub-02_T1w.nii.gz?versionId=93eQ.AwPcMeccJAT3sO9otYv4A_WH3Bj  \
-o ./T1w.nii.gz
"""
subprocess.run(command,shell=True)

# gunzip image to get from .nii.gz to .nii
subprocess.run("gunzip -c ./T1w.nii.gz > T1w.nii",shell=True)

# get the full path to the directory where this script is being executed
script_dir = os.path.dirname(os.path.realpath(__file__))

# get full path to both images (nipype always needs absolut paths to inputs)
image_nii = os.path.join(script_dir,'T1w.nii')
image_nii_gz = os.path.join(script_dir,'T1w.nii.gz')

###############################################################################
# Run CAT12Segment inside a node on one image
###############################################################################

# create node
cat12segment = Node(CAT12Segment(surface_and_thickness_estimation=0,
                                 surface_measures=0),
                    name='cat12segment')

# input image
cat12segment.inputs.in_files = image_nii_gz

# set working directory of node to script directory
cat12segment.base_dir = script_dir

# run the node
cat12segment.run()

Platform details:

{'commit_hash': '%h',
 'commit_source': 'archive substitution',
 'networkx_version': '3.2.1',
 'nibabel_version': '5.2.0',
 'nipype_version': '1.8.6',
 'numpy_version': '1.26.3',
 'pkg_path': '/csp-tools/anaconda3/envs/csp_wiesner_johannes/lib/python3.9/site-packages/nipype',
 'scipy_version': '1.12.0',
 'sys_executable': '/csp-tools/anaconda3/envs/csp_wiesner_johannes/bin/python',
 'sys_platform': 'linux',
 'sys_version': '3.9.18 | packaged by conda-forge | (main, Dec 23 2023, '
                '16:33:10) \n'
                '[GCC 12.3.0]',
 'traits_version': '6.3.2'}

Execution environment

Choose one

  • My python environment outside container
@JohannesWiesner JohannesWiesner changed the title ipype.interfaces.cat12.preprocess.CAT12Segment does not accept compressed .nii.gz files nipype.interfaces.cat12.preprocess.CAT12Segment does not accept compressed .nii.gz files Jun 6, 2024
@JohannesWiesner
Copy link
Author

JohannesWiesner commented Jun 6, 2024 •
edited
Loading

Maybe @mfmachado has an idea?

@effigies
Copy link
Member

effigies commented Jun 6, 2024

This is an explicit helper to remind people to put gunzip interfaces if they need:

nipype/nipype/interfaces/spm/base.py

Lines 617 to 631 in 4d1352a

class ImageFileSPM(ImageFile):
"""Defines a trait whose value must be a NIfTI file."""
def __init__(
self, value=NoDefaultSpecified, exists=False, resolve=False, **metadata
):
"""Create an ImageFileSPM trait."""
super().__init__(
value=value,
exists=exists,
types=["nifti1", "nifti2"],
allow_compressed=False,
resolve=resolve,
**metadata,
)

If SPM now allows compressed NIfTI, we could change it to something like:

allow_compressed=LooseVersion(Info.version()) >= LooseVersion('9.14'),

Would probably take some fiddling to get right, and would be good to know what version they added compression support in.

@JohannesWiesner
Copy link
Author

@effigies : It might be more complicated than that, because CAT12 is a toolbox inside SPM12. CAT12 for sure supports .nii.gz files in newer versions but I am not sure if SPM12 does (or at least every version of it). What I can confirm is, that if I run the CAT12 version that my nipype package is currently adressing in it's GUI version, I have no problems with the .nii.gz extension. In other words, the problem apparently lies on the nipype side.

@effigies
Copy link
Member

effigies commented Jun 6, 2024

Did earlier versions of CAT12 not support compression? If it has always supported compression, then you can just change ImageFileSPM to ImageFile in the input spec of the CAT12 interfaces, for example, here:

nipype/nipype/interfaces/cat12/preprocess.py

Lines 25 to 31 in 4d1352a

in_files = InputMultiPath(
ImageFileSPM(exists=True),
field="data",
desc="file to segment",
mandatory=True,
copyfile=False,
)

If it might be different, then we could add a cat12.Info class to detect the version and make an ImageFileCAT12 similar to the ImageFileSPM above that compares the CAT12 version. OTOH, that might be too much work, given that people are unlikely to use new nipype and old CAT12, so you could just drop the ImageFileSPM stuff.

@JohannesWiesner
Copy link
Author

Pinging the maintainer @ChristianGaser here, he might be able to answer which versions of CAT12 support .nii.gz files and which do not.

@JohannesWiesner
Copy link
Author

Did earlier versions of CAT12 not support compression? If it has always supported compression, then you can just change ImageFileSPM to ImageFile in the input spec of the CAT12 interfaces, for example, here:

nipype/nipype/interfaces/cat12/preprocess.py

Lines 25 to 31 in 4d1352a

in_files = InputMultiPath(
ImageFileSPM(exists=True),
field="data",
desc="file to segment",
mandatory=True,
copyfile=False,
)

If it might be different, then we could add a cat12.Info class to detect the version and make an ImageFileCAT12 similar to the ImageFileSPM above that compares the CAT12 version. OTOH, that might be too much work, given that people are unlikely to use new nipype and old CAT12, so you could just drop the ImageFileSPM stuff.

Sounds good to me! I had a brief try and just replace ImageFileSPM with ImageFile and it seems to work. Ending up with issue #3654 but that's another topic of course.

My suggestion:

Replace ImageFileSPM with ImageFile in nipype.interfaces.cat12.preprocess AFTER Christian Gaser has responded from which CAT12 version on compressed files are supported. I can send a PR then. But maybe then the minimum version of CAT12 that supports compressed files should also be added to the nipype docs so folks are aware that they cannot use older versions than this minium version.

@ChristianGaser
Copy link

ChristianGaser commented Jun 6, 2024 via email

@ChristianGaser ChristianGaser mentioned this issue Jun 7, 2024
Open
@JailanOweda
Copy link

JailanOweda commented Jun 21, 2024 •
edited
Loading

@effigies when we replace ImageFileSPM with ImageFile we get the following error:

SPM version: SPM12 Release: 7771
        SPM path: /apps/lib/matlab/spm12/spm.m
        Item 'Volumes', field 'val': Number of matching files (0) less than required (1).

        Standard error:
        MATLAB code threw an exception:
        No executable modules, but still unresolved dependencies or incomplete module inputs.
        File:/apps/lib/matlab/spm12/spm_jobman.m
        Name:/apps/lib/matlab/spm12/spm_jobman.m
        Line:47
        File:./pyscript_cat12segment.m
        Name:fill_run_job
        Line:115
        File:pm_jobman
        Name:pyscript_cat12segment
        Line:472
        File:÷
        Name:>
        Line:
        Return code: 0

We don't really understand where the error is coming from, since there is no such trait as 'Volumes' for the CAT12 interface. Maybe it's refering to the input image? That it couldn't be read?

I don't know if this information helps you, but we are currently setting all atlases for ROI processing (neuromorphometrics, etc.) to False, as a workaround for the issue #3654.

Here's the full pyscript_cat12segment.m file:

jobs{1}.spm.tools.cat.estwrite.nproc = 32;
jobs{1}.spm.tools.cat.estwrite.opts.affreg = 'mni';
jobs{1}.spm.tools.cat.estwrite.opts.biasacc = 0.5;
jobs{1}.spm.tools.cat.estwrite.extopts.APP = 1070;
jobs{1}.spm.tools.cat.estwrite.extopts.LASstr = 0.5;
jobs{1}.spm.tools.cat.estwrite.extopts.gcutstr = 2.0;
jobs{1}.spm.tools.cat.estwrite.extopts.WMHC = 2;
jobs{1}.spm.tools.cat.estwrite.extopts.vox = 1.5;
jobs{1}.spm.tools.cat.estwrite.extopts.restypes.optimal(1) = 1.0;
jobs{1}.spm.tools.cat.estwrite.extopts.restypes.optimal(2) = 0.1;
jobs{1}.spm.tools.cat.estwrite.extopts.ignoreErrors = 1;
jobs{1}.spm.tools.cat.estwrite.output.surface = 0;
jobs{1}.spm.tools.cat.estwrite.output.ROImenu.atlases.neuromorphometrics = 0;
jobs{1}.spm.tools.cat.estwrite.output.ROImenu.atlases.lpba40 = 0;
jobs{1}.spm.tools.cat.estwrite.output.ROImenu.atlases.hammers = 0;
jobs{1}.spm.tools.cat.estwrite.output.ROImenu.atlases.cobra = 0;
jobs{1}.spm.tools.cat.estwrite.output.ROImenu.atlases.thalamus = 0;
jobs{1}.spm.tools.cat.estwrite.output.ROImenu.atlases.thalamic_nuclei = 0;
jobs{1}.spm.tools.cat.estwrite.output.ROImenu.atlases.suit = 0;
jobs{1}.spm.tools.cat.estwrite.output.GM.native = 0;
jobs{1}.spm.tools.cat.estwrite.output.GM.mod = 1;
jobs{1}.spm.tools.cat.estwrite.output.GM.dartel = 0;
jobs{1}.spm.tools.cat.estwrite.output.WM.native = 0;
jobs{1}.spm.tools.cat.estwrite.output.WM.mod = 1;
jobs{1}.spm.tools.cat.estwrite.output.WM.dartel = 0;
jobs{1}.spm.tools.cat.estwrite.output.CSF.native = 0;
jobs{1}.spm.tools.cat.estwrite.output.CSF.mod = 0;
jobs{1}.spm.tools.cat.estwrite.output.CSF.dartel = 0;
jobs{1}.spm.tools.cat.estwrite.output.label.native = 1;
jobs{1}.spm.tools.cat.estwrite.output.label.warped = 0;
jobs{1}.spm.tools.cat.estwrite.output.label.dartel = 0;
jobs{1}.spm.tools.cat.estwrite.output.labelnative = 1;
jobs{1}.spm.tools.cat.estwrite.output.bias.warped = 1;
jobs{1}.spm.tools.cat.estwrite.output.las.native = 0;
jobs{1}.spm.tools.cat.estwrite.output.las.warped = 0;
jobs{1}.spm.tools.cat.estwrite.output.las.dartel = 0;
jobs{1}.spm.tools.cat.estwrite.output.jacobianwarped = 0;
jobs{1}.spm.tools.cat.estwrite.output.warps(1) = 1;
jobs{1}.spm.tools.cat.estwrite.output.warps(2) = 0;

@JailanOweda
Copy link

JailanOweda commented Jun 24, 2024 •
edited
Loading

@effigies the problem has something to do with the fact that for some reason when I choose the zipped file as an input it is written like this in the matlab script:

jobs{1}.spm.tools.cat.estwrite.data = {...
'./testing/test_cat12/cat12segment/T1w.nii.gz,1';...
};

So it's looking for the first slice in the zipped file (.gz,1), although if I try choosing the same file in the GUI slices are not displayed for the zipped files, you just take the zipped file as a whole and the script looks as follows:

matlabbatch{1}.spm.tools.cat.estwrite.data = {'./testing/test_cat12/T1w.nii.gz'};

I ran the same script generated by Nipype directly in Matlab before and after removing ,1 from the input path. After removing it, there were no more errors, so the problem is really there.
Any ideas on what is causing Nipype to search for slices in the zipped file and how I can avoid that?

JailanOweda pushed a commit to JailanOweda/nipype that referenced this issue Jun 24, 2024
FIX nipy#3653: added option for using compressed files as inputs in C…
d91f6d2 
…AT12 interface
@JailanOweda JailanOweda linked a pull request Sep 25, 2024 that will close this issue
Open
effigies pushed a commit to JailanOweda/nipype that referenced this issue Oct 6, 2024
@effigies
FIX nipy#3653: added option for using compressed files as inputs in C…
035bf8d 
…AT12 interface
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix nipy#3653 and nipy#3654 in the CAT12 Interface JailanOweda/nipype
4 participants
@effigies @ChristianGaser @JohannesWiesner @JailanOweda