meet problem when running demo for video

[unbeliveyu]

when i run the commend as follows：

python demo.py tracking,ddd --load_model ../models/nuScenes_3Dtracking.pth --dataset nuscenes --pre_hm --track_thresh 0.1 --demo ../videos/nuscenes_mini.mp4 --test_focal_length 633

i met this

` D:\anaconda\envs\DCNv2\lib\site-packages\sklearn\utils\linear_assignment_.py:22: FutureWarning: The linear_assignment_ module is deprecated in 0.21 and will be removed from 0.23. Use scipy.optimize.linear_sum_assignment instead.
FutureWarning)
Running tracking
Using tracking threshold for out threshold! 0.1
Fix size testing.
training chunk_sizes: [32]
input h w: 448 800
heads {'hm': 10, 'reg': 2, 'wh': 2, 'tracking': 2, 'dep': 1, 'rot': 8, 'dim': 3, 'amodel_offset': 2}
weights {'hm': 1, 'reg': 1, 'wh': 0.1, 'tracking': 1, 'dep': 1, 'rot': 1, 'dim': 1, 'amodel_offset': 1}
head conv {'hm': [256], 'reg': [256], 'wh': [256], 'tracking': [256], 'dep': [256], 'rot': [256], 'dim': [256], 'amodel_offset': [256]}
Creating model...
Using node type: (<class 'model.networks.dla.DeformConv'>, <class 'model.networks.dla.DeformConv'>)
Warning: No ImageNet pretrain!!
loaded ../models/nuScenes_3Dtracking.pth, epoch 70
out_name nuscenes_mini.mp4
Initialize tracking!
error in modulated_deformable_im2col_cuda: no kernel image is available for execution on the device
error in modulated_deformable_im2col_cuda: no kernel image is available for execution on the device
error in modulated_deformable_im2col_cuda: no kernel image is available for execution on the device
error in modulated_deformable_im2col_cuda: no kernel image is available for execution on the device
error in modulated_deformable_im2col_cuda: no kernel image is available for execution on the device
Traceback (most recent call last):
File "demo.py", line 128, in
demo(opt)
File "demo.py", line 83, in demo
ret = detector.run(img)
File "E:\github\CenterTrack\src\lib\detector.py", line 119, in run
images, self.pre_images, pre_hms, pre_inds, return_time=True)
File "E:\github\CenterTrack\src\lib\detector.py", line 339, in process
output = self.model(images, pre_images, pre_hms)[-1]
File "D:\anaconda\envs\DCNv2\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "E:\github\CenterTrack\src\lib\model\networks\base_model.py", line 75, in forward
feats = self.imgpre2feats(x, pre_img, pre_hm)
File "E:\github\CenterTrack\src\lib\model\networks\dla.py", line 633, in imgpre2feats
x = self.dla_up(x)
File "D:\anaconda\envs\DCNv2\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "E:\github\CenterTrack\src\lib\model\networks\dla.py", line 572, in forward
ida(layers, len(layers) -i - 2, len(layers))
File "D:\anaconda\envs\DCNv2\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "E:\github\CenterTrack\src\lib\model\networks\dla.py", line 543, in forward
layers[i] = upsample(project(layers[i]))
File "D:\anaconda\envs\DCNv2\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "D:\anaconda\envs\DCNv2\lib\site-packages\torch\nn\modules\conv.py", line 929, in forward
output_padding, self.groups, self.dilation)
RuntimeError: cuDNN error: CUDNN_STATUS_INTERNAL_ERROR
You can try to repro this exception using the following code snippet. If that doesn't trigger the error, please include your original repro script when reporting this issue.

import torch
torch.backends.cuda.matmul.allow_tf32 = True
torch.backends.cudnn.benchmark = False
torch.backends.cudnn.deterministic = False
torch.backends.cudnn.allow_tf32 = True
data = torch.randn([1, 128, 56, 100], dtype=torch.float, device='cuda', requires_grad=True)
net = torch.nn.Conv2d(128, 128, kernel_size=[4, 4], padding=[1, 1], stride=[2, 2], dilation=[1, 1], groups=128)
net = net.cuda().float()
out = net(data)
out.backward(torch.randn_like(out))
torch.cuda.synchronize()

ConvolutionParams
data_type = CUDNN_DATA_FLOAT
padding = [1, 1, 0]
stride = [2, 2, 0]
dilation = [1, 1, 0]
groups = 128
deterministic = false
allow_tf32 = true
input: TensorDescriptor 000002941E9B2530
type = CUDNN_DATA_FLOAT
nbDims = 4
dimA = 1, 128, 56, 100,
strideA = 716800, 5600, 100, 1,
output: TensorDescriptor 000002941E9B49F0
type = CUDNN_DATA_FLOAT
nbDims = 4
dimA = 1, 128, 28, 50,
strideA = 179200, 1400, 50, 1,
weight: FilterDescriptor 000002941715AD70
type = CUDNN_DATA_FLOAT
tensor_format = CUDNN_TENSOR_NCHW
nbDims = 4
dimA = 128, 1, 4, 4,
Pointer addresses:
input: 0000000745200000
output: 000000070C024000
weight: 00000007091EE400
Additional pointer addresses:
grad_output: 000000070C024000
grad_input: 0000000745200000
Backward data algorithm: 1

`