Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ModuleNotFoundError when importing skops after import prefect (prefect 3) #15739

Open
protagonyist opened this issue Oct 17, 2024 · 0 comments · May be fixed by #15741
Open

ModuleNotFoundError when importing skops after import prefect (prefect 3) #15739

protagonyist opened this issue Oct 17, 2024 · 0 comments · May be fixed by #15741
Labels
bug Something isn't working

Comments

@protagonyist
Copy link

protagonyist commented Oct 17, 2024

Bug summary

Recently, I was working on upgrading project to prefect 3. That project uses skops for safe storing and loading ML models. Following imports (prefect==3.0.10, skops==0.10):

import prefect
from skops.io import load as skops_load

Cause exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/ppaa/prefect-sandbox/venv/lib/python3.12/site-packages/skops/io/__init__.py", line 1, in <module>
    from ._persist import dump, dumps, get_untrusted_types, load, loads
  File "/home/ppaa/prefect-sandbox/venv/lib/python3.12/site-packages/skops/io/_persist.py", line 22, in <module>
    module = importlib.import_module(module_name, package="skops.io")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ppaa/prefect-sandbox/venv/lib/python3.12/site-packages/skops/io/_general.py", line 16, in <module>
    from ._trusted_types import (
  File "/home/ppaa/prefect-sandbox/venv/lib/python3.12/site-packages/skops/io/_trusted_types.py", line 23, in <module>
    SCIPY_UFUNC_TYPE_NAMES = get_public_type_names(
                             ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ppaa/prefect-sandbox/venv/lib/python3.12/site-packages/skops/io/_utils.py", line 238, in get_public_type_names
    and (type_name := get_type_name(obj)).startswith(module_name)
                      ^^^^^^^^^^^^^^^^^^
  File "/home/ppaa/prefect-sandbox/venv/lib/python3.12/site-packages/skops/io/_utils.py", line 183, in get_type_name
    return f"{get_module(t)}.{t.__name__}"
              ^^^^^^^^^^^^^
  File "/home/ppaa/prefect-sandbox/venv/lib/python3.12/site-packages/skops/io/_utils.py", line 90, in get_module
    return whichmodule(obj, obj.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ppaa/prefect-sandbox/venv/lib/python3.12/site-packages/skops/io/_utils.py", line 53, in whichmodule
    if _getattribute(module, name)[0] is obj:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ppaa/prefect-sandbox/venv/lib/python3.12/site-packages/skops/io/_utils.py", line 25, in _getattribute
    obj = getattr(obj, subpath)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ppaa/prefect-sandbox/venv/lib/python3.12/site-packages/prefect/__init__.py", line 108, in __getattr__
    return importlib.import_module(f".{attr_name}", package=__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'prefect.agm'

code to reproduce:

pip3 install prefect==3.0.10 skops==0.10 && python3 -c 'import prefect;from skops.io import load as skops_load'

After some investigation, I figured that __getattr__ function (that was added for lazy imports in #14190) throws ModuleNotFoundErorr in case when regular python modules throw AttributeError (and because of that, skops does not catch it while inspecting imported modules):
2024-10-17_19-22
I already prepared fix (here), but contribution guideline suggests that I create issue first, so here it is
P. S. I still feel a bit insecure writing in English, sorry about any mistakes

Version info (prefect version output)

Version:             3.0.10
API version:         0.8.4
Python version:      3.12.6
Git commit:          3aa2d893
Built:               Tue, Oct 15, 2024 1:31 PM
OS/Arch:             linux/x86_64
Profile:             local
Server type:         unconfigured
Pydantic version:    2.9.2

Additional context

No response

@protagonyist protagonyist added the bug Something isn't working label Oct 17, 2024
protagonyist pushed a commit to protagonyist/prefect that referenced this issue Oct 17, 2024
protagonyist added a commit to protagonyist/prefect that referenced this issue Oct 17, 2024
@protagonyist protagonyist linked a pull request Oct 17, 2024 that will close this issue
4 tasks
@desertaxle desertaxle linked a pull request Oct 17, 2024 that will close this issue
4 tasks
protagonyist added a commit to protagonyist/prefect that referenced this issue Oct 18, 2024
protagonyist added a commit to protagonyist/prefect that referenced this issue Oct 18, 2024
protagonyist added a commit to protagonyist/prefect that referenced this issue Oct 20, 2024
protagonyist added a commit to protagonyist/prefect that referenced this issue Oct 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant