Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

trt10.5 pytorch-quantization has compile bug #4197

Open
lix19937 opened this issue Oct 13, 2024 · 1 comment
Open

trt10.5 pytorch-quantization has compile bug #4197

lix19937 opened this issue Oct 13, 2024 · 1 comment
Labels
triaged Issue has been triaged by maintainers

Comments

@lix19937
Copy link

Description

trt10.5 pytorch-quantization has compile bug.

https://github.com/NVIDIA/TensorRT/blob/release/10.5/tools/pytorch-quantization/src/tensor_quant_gpu.cu#L28-L37
define two macro AT_DISPATCH_CASE_FLOATING_TYPES and AT_DISPATCH_FLOATING_TYPES

#define AT_DISPATCH_CASE_FLOATING_TYPES(...)   \
  AT_DISPATCH_CASE(at::ScalarType::Double, __VA_ARGS__)  \
  AT_DISPATCH_CASE(at::ScalarType::Float, __VA_ARGS__)  \
  AT_DISPATCH_CASE(at::ScalarType::Half, __VA_ARGS__)   \
  AT_DISPATCH_CASE(at::ScalarType::BFloat16, __VA_ARGS__)

#define AT_DISPATCH_FLOATING_TYPES(TYPE, NAME, ...) \
  AT_DISPATCH_SWITCH(                                        \
      TYPE, NAME, AT_DISPATCH_CASE_FLOATING_TYPES(__VA_ARGS__))

but in https://github.com/NVIDIA/TensorRT/blob/release/10.5/tools/pytorch-quantization/src/tensor_quant_gpu.cu#L18
#include <ATen/ATen.h> --> #include <ATen/Dispatch.h> --> has already defined these two macros.
I check torch1.13 and torch2.4.1, both the same case.

two macros duplicate definition. @moraxu
need use

#undef AT_DISPATCH_CASE_FLOATING_TYPES(...)  
#undef AT_DISPATCH_FLOATING_TYPES(TYPE, NAME, ...)  

before #define in tensor_quant_gpu.cu

Environment

TensorRT Version:10.5

NVIDIA GPU:rtx2000

NVIDIA Driver Version:

CUDA Version:11.8

CUDNN Version:9.1

Operating System:

Python Version (if applicable):3.8

PyTorch Version (if applicable):1.13 or 2.4.1

@yuanyao-nv yuanyao-nv added the triaged Issue has been triaged by maintainers label Oct 15, 2024
@yuanyao-nv
Copy link
Collaborator

pytorch-quantization development has been discontinued in favor of Model Optimizer since TRT 10.2. Please try that if possible.
cc @nzmora-nvidia for opinion on the raised issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
triaged Issue has been triaged by maintainers
Projects
None yet
Development

No branches or pull requests

2 participants