BUG: Segfault in smoothing length calculation on Mac #4961
Comments
|
update: I tried to run the same thing on the university linux cluster, and after many frustrations getting Details of the linux setup:
|
|
Interestingly, the following doesn't crash (at least on my system) import yt
yt.load_sample("TipsyGalaxy")logger output A seemingly crucial difference I see with your report is that the kdtree isn't loaded, so let's force it: import yt
ds = yt.load_sample("TipsyGalaxy")
ds.index.kdtreelogger output: still no crash. I made several attempts with numpy 2.0.1 and 1.26.4 You're using |
|
I got the same error earlier without the loaded KDTree. I can try removing it, but I'm pretty sure |
|
Do you get a crash every time or does is it flaky ? |
|
It seems to crash every time in this setup. Also, I am not running this from the yt directory or its subdirectories, but from a separate directory with a few test scripts. I just tried the following setup with I initially got the same error: >>> import yt
>>> sampledatadir = "/Users/nastasha/code/ytdev_misc/sample_data/"
>>> datasetn = sampledatadir + "TipsyGalaxy/galaxy.00300"
>>> ds = yt.load(datasetn)yielded but after deleting the previously generated kdtree file, it did run ok: TLDR; using |
|
I certainly didn't mean to imply that conda wasn't supported, I was just wondering why I wasn't able to reproduce the problem. So, it seems we've established that this is a build-time issue with something other than yt itself (since you're compiling/installing it with pip in both scenarios). The most likely suspect would be numpy, and I think we can deduce that the issue is that conda distributions are built against some broken C++ dependency, while wheels from PyPI are not. We're making progress but there is still a lot of guessing. What we really need at this stage is a much simpler reproducer script, ideally one that doesn't use yt at all. This is typically a lot of work; I cannot promise that I would even attempt it (I'd like to avoid installing conda if I can avoid it). |
|
Oh geez. Well, for the record, here are the installed packages and versions in venv + pipusing conda + pipusing This also gets |
|
finally got around to trying on my mac (which already had a miniconda installation) and I can't reproduce the failure. Tried a couple different numpy versions, went back a bit in the yt development branch too. Since I do have conda, @neutrinoceros or @nastasha-w let me know if you've got any ideas for something else to try. I'm on a M2 mac running Sonoma 14.6.1 . Installed everything into a fresh conda environment with Python 3.12.4 and using pip. |
|
I suspect the real bug is in openblas or highway (both C/C++ dependencies to NumPy). I don't know how they are linked to numpy in conda-world so it might be worth exploring if this bug is reproducible with downgraded versions of those; if numpy is installed from PyPI I think it includes its own copies. |
Bug report
Bug summary
I get a segfault when loading the Tipsy 00300 galaxy into
yt. From the output, this seems to occur whenyttries to calculate smoothing lengths.The KDTree construction before the smoothing length computation seems to work. This issue occurred on my Mac both with numpy 1.26, and with the bleeding-edge numpy version. The error reminded me of #4953, but that issue was marked resolved after an upstream fix in numpy.
Code for reproduction
This issue was triggered when analyzing the Tipsy galaxy 00300 from the yt hub.
Actual outcome
Expected outcome
A loaded
ytdataset, with smoothing lengths for the SPH particles.Version Information
conda create -n ytdev python=3.12 pip conda activate ytdev python -m pip install git+https://github.com/numpy/numpy.git python -m pip install -e ".[doc]" python -m pip install h5py pandasThe text was updated successfully, but these errors were encountered: