Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't reimplement the result of nuscenes 3d detection result #274

Open
Wang-zipeng opened this issue Aug 10, 2022 · 0 comments
Open

Can't reimplement the result of nuscenes 3d detection result #274

Wang-zipeng opened this issue Aug 10, 2022 · 0 comments

Comments

@Wang-zipeng
Copy link

I used the script nuScenes_3Ddetection_e140.sh in the CenterTrack/experiments, but I can't reimplement the result in the model zoo with a mAP about 0.30~.When I run the command "python test.py ddd --exp_id nuScenes_3Ddetection_e140 --dataset nuscenes --resume" to evaluate the model, I got this error:
Accumulating metric data...
Traceback (most recent call last):
File "/home/wangzp/git_projects/CenterTrack/src/tools/nuscenes-devkit/python-sdk/nuscenes/eval/detection/evaluate.py", line 302, in
nusc_eval.main(plot_examples=plot_examples_, render_curves=render_curves_)
File "/home/wangzp/git_projects/CenterTrack/src/tools/nuscenes-devkit/python-sdk/nuscenes/eval/detection/evaluate.py", line 204, in main
metrics, metric_data_list = self.evaluate()
File "/home/wangzp/git_projects/CenterTrack/src/tools/nuscenes-devkit/python-sdk/nuscenes/eval/detection/evaluate.py", line 116, in evaluate
md = accumulate(self.gt_boxes, self.pred_boxes, class_name, self.cfg.dist_fcn_callable, dist_th)
File "/home/wangzp/miniconda3/envs/mmdet3d/lib/python3.9/site-packages/nuscenes/eval/detection/algo.py", line 103, in accumulate
match_data['scale_err'].append(1 - scale_iou(gt_box_match, pred_box))
File "/home/wangzp/miniconda3/envs/mmdet3d/lib/python3.9/site-packages/nuscenes/eval/common/utils.py", line 99, in scale_iou
assert all(sr_size > 0), 'Error: sample_result sizes must be >0.'
AssertionError: Error: sample_result sizes must be >0.

I think there is any class in any distance has an empty result result in this error. And my model's loss is bigger than the model provided in the model zoo.
My trained model's loss is:
hm 2.8014 |wh 6.9593 |reg 0.2466 |dep 8.8626 |dim 0.7961 |rot 2.1047 |amodel_offset 2.9173
The loss of provided model in the model zoo is:
hm 1.0984 |wh 1.5407 |reg 0.2202 |dep 1.6663 |dim 0.2449 |rot 1.6251 |amodel_offset 0.9897
It's much smaller than my trained model.
ps: i got your loss by load the model to train some step,and set the lr 2.5e-6. And i can reimplement the validation result with model provided in the model zoo.

@xingyizhou
My question is if i can reimplement the model with the script nuScenes_3Ddetection_e140.sh, If i can't could you please share the script?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant