Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bugfix: workspace dir when no GPU is available #579

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

abcdabcd987
Copy link
Member

When GPU is not available (e.g., CUDA_VISIBLE_DEVICES=-1), _get_cuda_arch_flags() raises IndexError.

/opt/venv/lib/python3.10/site-packages/flashinfer/__init__.py:17: in <module>
    from .activation import (
/opt/venv/lib/python3.10/site-packages/flashinfer/activation.py:21: in <module>
    from .jit import (
/opt/venv/lib/python3.10/site-packages/flashinfer/jit/__init__.py:27: in <module>
    from .activation import (
/opt/venv/lib/python3.10/site-packages/flashinfer/jit/activation.py:19: in <module>
    from .env import FLASHINFER_GEN_SRC_DIR
/opt/venv/lib/python3.10/site-packages/flashinfer/jit/env.py:31: in <module>
    FLASHINFER_WORKSPACE_DIR = _get_workspace_dir_name()
/opt/venv/lib/python3.10/site-packages/flashinfer/jit/env.py:24: in _get_workspace_dir_name
    flags = _get_cuda_arch_flags()
/opt/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1984: in _get_cuda_arch_flags
    arch_list[-1] += '+PTX'
E   IndexError: list index out of range

Although FlashInfer is not useful in this case, we still don't want to crash user program when importing flashinfer. This PR fixes this issue.

Another change is to hide the warning of TORCH_CUDA_ARCH_LIST not set when importing flashinfer. This can be annoying when using AOT wheels.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant