Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to convert Yolov4x-mish/csp model into onnx &do inferencing? #253

Open
akashAD98 opened this issue Aug 19, 2021 · 14 comments
Open

How to convert Yolov4x-mish/csp model into onnx &do inferencing? #253

akashAD98 opened this issue Aug 19, 2021 · 14 comments
Labels

Comments

@akashAD98
Copy link

I followed this tutorial https://github.com/linghu8812/tensorrt_inference
I was able to do the conversion, but how can i do inferencing or testing of this converted yolov4x-mish.onnx model?

@stevenlix
Copy link

stevenlix commented Sep 30, 2021

You may use OnnxRuntime for onnx mode inferencing. https://github.com/microsoft/onnxruntime. It provides various execution providers such as CPU, CUDA and TensorRT.

@akashAD98
Copy link
Author

@stevenlix i was able to do inference using this script, but how should i draw bbox & visualize results?
this is code

import onnxruntime
import cv2
import numpy as np

model = onnxruntime.InferenceSession(r"K:\company_work\MODEL_WEIGHTS\trt_onnx\yolov4x-mish-custom-electronics-416x416.onnx",None)
print("model Loaded")
img = np.zeros([3,416,416],dtype = np.float32)

#def onnx_inf(model,img):
img = img[np.newaxis,...].astype(np.float32)
input_name = model.get_inputs()[0].name
onnx_output_name = [out.name for out in model.get_outputs()]
result = model.run(onnx_output_name, {input_name: img})
print(results)

reference:
https://github.com/jkjung-avt/tensorrt_demos/blob/master/modnet/test_onnx.py

@akashAD98
Copy link
Author

using this scipt i was able to solve inferense , this is yolov4.onxx which is provided by repo
https://github.com/onnx/models/blob/master/vision/object_detection_segmentation/yolov4/model/yolov4.onnx
image

image

But when im trying to do same with my custom data im not able to get proper output shapes

image

i tried this repo for converting yolo models to onnx & tensorrt
& i im getting results on this converted model using tensorrt
https://github.com/jkjung-avt/tensorrt_demos

@stevenlix
Copy link

stevenlix commented Nov 19, 2021

the error message indicates your input shape doesn't match model's input. Please check your input data on 000_net

@akashAD98

This comment has been minimized.

@akashAD98
Copy link
Author

@stevenlix i solved this problem using np.rollaxis , my model is reciving NHWC as input,so i converted it into NCHW ,but again im getting this issue in postprocessing.

image

i solved this issue by conv_raw_dxdy = pred[:, :, :, :, 0:2] to this commenting one conv_raw_dxdy = pred[:, :, :, 0:2]

again im getting this issue

image

@jkjung-avt
Copy link

@akashAD98 Could you provide the full python script you're working with?

Otherwise, what are the output tensor shapes when you inference your custom model with onnxruntime?

......
result = model.run(onnx_output_name, {input_name: img})
print(results)  # What are the shapes?

@akashAD98
Copy link
Author

@jkjung-avt im using this scipt for onnx inference

https://github.com/onnx/models/blob/master/vision/object_detection_segmentation/yolov4/dependencies/inference.ipynb

import onnxruntime
import cv2
import numpy as np

model = onnxruntime.InferenceSession(r"K:\company_work\MODEL_WEIGHTS\trt_onnx\yolov4x-mish-custom-electronics-416x416.onnx",None)
print("model Loaded")
img = np.zeros([3,416,416],dtype = np.float32)

#def onnx_inf(model,img):
img = img[np.newaxis,...].astype(np.float32)
input_name = model.get_inputs()[0].name
onnx_output_name = [out.name for out in model.get_outputs()]
result = model.run(onnx_output_name, {input_name: img})

#print(len(output))
print(result)

--->
[array([[[[5.43156087e-01, 5.38730502e-01, 4.62164581e-01, ...,
5.12988985e-01, 5.09564400e-01, 5.12043834e-01],
[5.16158760e-01, 4.61559117e-01, 4.31563795e-01, ...,
5.43736160e-01, 5.67586184e-01, 5.66980243e-01],
[5.12543499e-01, 4.32360113e-01, 4.53552306e-01, ...,
5.67302525e-01, 5.82414269e-01, 5.57022810e-01],
...,
[5.04360497e-01, 4.82946813e-01, 4.90960002e-01, ...,
5.76641083e-01, 6.24482334e-01, 5.19942403e-01],
[4.92966831e-01, 5.32791436e-01, 4.49620664e-01, ...,
6.05896771e-01, 5.61023772e-01, 5.09344757e-01],
[4.99246687e-01, 5.43988764e-01, 4.45850015e-01, ...,
5.49508035e-01, 5.43327272e-01, 5.04986584e-01]],

    [[5.36974788e-01, 5.24553537e-01, 4.67143923e-01, ...,
      3.71252656e-01, 3.98430407e-01, 4.51538593e-01],
     [5.16066372e-01, 4.74151522e-01, 4.49761689e-01, ...,
      4.63975430e-01, 5.23927808e-01, 5.52703500e-01],
     [4.67342854e-01, 4.12634462e-01, 4.03690666e-01, ...,
      4.13400263e-01, 4.38304871e-01, 4.64697480e-01],
     ...,
     [5.75694323e-01, 6.34198070e-01, 6.52417362e-01, ...,
      6.21224701e-01, 6.01580858e-01, 5.75538635e-01],
     [4.92898732e-01, 5.56907117e-01, 6.32649481e-01, ...,
      5.97918928e-01, 5.69094121e-01, 5.40199101e-01],
     [4.87957597e-01, 4.86043215e-01, 4.91812527e-01, ...,
      4.99950886e-01, 5.12410343e-01, 4.86961126e-01]],

    [[5.23443937e-01, 7.79292107e-01, 9.07118022e-01, ...,
      8.78608942e-01, 7.61271536e-01, 4.38795388e-01],
     [4.55329448e-01, 7.08072305e-01, 8.29850137e-01, ...,
      8.56963336e-01, 7.07964420e-01, 3.79904032e-01],
     [4.47295010e-01, 6.74613118e-01, 7.91363716e-01, ...,
      8.19179177e-01, 6.79931641e-01, 4.00378555e-01],
     ...,
     [4.41179067e-01, 7.08656669e-01, 8.15967798e-01, ...,
      8.01772654e-01, 6.60377979e-01, 4.46031868e-01],
     [4.39138174e-01, 7.59865761e-01, 8.72310162e-01, ...,
      8.63716364e-01, 7.28884459e-01, 4.47639048e-01],
     [4.14665610e-01, 7.16006160e-01, 8.40928793e-01, ...,
      8.36795390e-01, 6.88124239e-01, 4.22244400e-01]],

    ...,

    [[1.85467005e-02, 8.53380561e-03, 3.49757075e-03, ...,
      8.82121921e-03, 1.54880881e-02, 3.84887159e-02],
     [1.15795434e-02, 6.30369782e-03, 4.77072597e-03, ...,
      2.43663788e-03, 6.25890493e-03, 3.95016968e-02],
     [1.07738376e-02, 7.96377659e-03, 8.93741846e-03, ...,
      5.26088476e-03, 7.91543722e-03, 3.22411358e-02],
     ...,
     [6.91694021e-03, 8.35761428e-03, 9.16910172e-03, ...,
      1.30720437e-02, 1.53514445e-02, 2.85124183e-02],
     [7.45928288e-03, 1.36717856e-02, 1.55269504e-02, ...,
      2.31153369e-02, 1.57741308e-02, 2.37288475e-02],
     [8.14956427e-03, 1.13956034e-02, 1.10798478e-02, ...,
      1.75575018e-02, 1.23745799e-02, 1.25564337e-02]],

    [[4.33595479e-02, 2.30600834e-02, 1.73615813e-02, ...,
      1.64348781e-02, 2.26296484e-02, 5.99259436e-02],
     [4.48815823e-02, 1.34840310e-02, 1.30189359e-02, ...,
      5.93450665e-03, 1.35666728e-02, 6.22644126e-02],
     [5.61770797e-02, 2.33932436e-02, 7.28278458e-02, ...,
      1.11199915e-02, 1.72501802e-02, 4.85971570e-02],
     ...,
     [1.01473629e-02, 4.24060225e-03, 4.53561544e-03, ...,
      7.10424781e-03, 1.17792785e-02, 2.84530818e-02],
     [1.72387362e-02, 1.10016763e-02, 1.15490556e-02, ...,
      1.37041807e-02, 1.38654709e-02, 2.51607597e-02],
     [3.68901789e-02, 2.81743705e-02, 2.78824568e-02, ...,
      1.65835321e-02, 1.88879073e-02, 2.82895267e-02]],

    [[1.32935941e-02, 2.34824419e-03, 1.26335025e-03, ...,
      5.09813428e-03, 5.77861071e-03, 2.42372155e-02],
     [4.84403968e-03, 1.01804733e-03, 6.72370195e-04, ...,
      9.83953476e-03, 6.44868612e-03, 1.99303627e-02],
     [8.70105624e-03, 2.13134289e-03, 1.93604827e-03, ...,
      1.75871849e-02, 1.35977864e-02, 2.57759392e-02],
     ...,
     [6.23479486e-03, 1.55609846e-03, 1.16497278e-03, ...,
      8.16076994e-03, 8.66988301e-03, 1.53889656e-02],
     [6.87518716e-03, 1.66732073e-03, 1.13087893e-03, ...,
      7.97024369e-03, 5.68249822e-03, 1.54123902e-02],
     [2.71277130e-02, 1.03957355e-02, 7.73647428e-03, ...,
      1.70073509e-02, 1.32489800e-02, 1.97783113e-02]]]],
  dtype=float32), array([[[[6.82633579e-01, 5.92198133e-01, 5.36219060e-01, ...,
      5.22313654e-01, 5.33153296e-01, 4.08586204e-01],
     [5.63020945e-01, 4.97934908e-01, 4.85931903e-01, ...,
      5.14598966e-01, 5.69518089e-01, 4.36063617e-01],
     [5.41509748e-01, 4.83195364e-01, 5.04000783e-01, ...,
      4.88779098e-01, 5.51217556e-01, 4.52640295e-01],
     ...,
     [5.38222015e-01, 4.33686465e-01, 4.58312184e-01, ...,
      5.19559264e-01, 5.18851757e-01, 5.14279842e-01],
     [5.81782997e-01, 4.60810632e-01, 5.05498111e-01, ...,
      4.84434098e-01, 4.95694876e-01, 4.41713005e-01],
     [6.11413419e-01, 4.79556262e-01, 5.27434230e-01, ...,
      4.62450296e-01, 4.83112365e-01, 4.03794616e-01]],

    [[6.37833953e-01, 6.42928839e-01, 6.27475202e-01, ...,
      6.16223693e-01, 6.30499005e-01, 6.48454309e-01],
     [4.97826725e-01, 4.56331700e-01, 4.40702945e-01, ...,
      4.61066753e-01, 4.56165671e-01, 4.99953568e-01],
     [4.74054486e-01, 4.78003621e-01, 4.57944244e-01, ...,
      5.32258451e-01, 5.19274056e-01, 5.04699171e-01],
     ...,
     [4.30407256e-01, 4.30192322e-01, 4.67941225e-01, ...,
      5.03148675e-01, 4.92921591e-01, 4.81659621e-01],
     [4.17766780e-01, 4.18412983e-01, 4.43399787e-01, ...,
      4.79176104e-01, 4.43045139e-01, 4.04151917e-01],
     [3.07517707e-01, 3.47622275e-01, 3.89939308e-01, ...,
      3.81535292e-01, 3.78883064e-01, 3.79344821e-01]],

    [[4.57153320e-01, 6.56591892e-01, 7.65714765e-01, ...,
      7.75973320e-01, 5.85696876e-01, 4.44520652e-01],
     [3.88055593e-01, 5.95386744e-01, 7.94610620e-01, ...,
      7.05632269e-01, 5.53930521e-01, 3.66048396e-01],
     [3.76973093e-01, 6.13675117e-01, 7.69203782e-01, ...,
      7.30606318e-01, 5.62068701e-01, 3.43667805e-01],
     ...,
     [3.83884132e-01, 5.85814714e-01, 7.69175887e-01, ...,
      7.41853058e-01, 5.92259824e-01, 3.28055650e-01],
     [3.84677619e-01, 5.80844343e-01, 7.98523188e-01, ...,
      7.78843999e-01, 6.17938697e-01, 3.80232394e-01],
     [4.02339399e-01, 5.67262769e-01, 7.13167489e-01, ...,
      7.63966739e-01, 6.26933932e-01, 4.51630592e-01]],

    ...,

    [[1.53054297e-02, 2.84454226e-03, 7.10904598e-04, ...,
      2.52485275e-03, 3.63647938e-03, 1.90643668e-02],
     [3.91662121e-03, 1.00281835e-03, 1.96963549e-04, ...,
      1.93920732e-03, 3.61639261e-03, 2.13303268e-02],
     [5.59258461e-03, 1.60396099e-03, 1.05446577e-03, ...,
      4.41458821e-03, 5.30707836e-03, 1.41301751e-02],
     ...,
     [3.07147801e-02, 2.93711424e-02, 1.66025162e-02, ...,
      5.99618852e-02, 2.56006122e-02, 1.48141384e-02],
     [2.08319128e-02, 1.41707957e-02, 5.74657321e-03, ...,
      1.72817111e-02, 6.36804104e-03, 7.03477859e-03],
     [2.44514346e-02, 4.44053113e-02, 4.16858792e-02, ...,
      2.06050873e-02, 1.64674819e-02, 1.65671110e-02]],

    [[4.03031707e-03, 8.62270594e-04, 1.64061785e-04, ...,
      1.55246258e-03, 2.49183178e-03, 1.24293268e-02],
     [3.61201167e-03, 4.31030989e-04, 5.04553318e-05, ...,
      1.04942918e-03, 2.28783488e-03, 1.18854046e-02],
     [1.50660276e-02, 1.65867805e-03, 3.09884548e-04, ...,
      3.61818075e-03, 1.11929178e-02, 2.72503793e-02],
     ...,
     [4.83229756e-03, 2.75895000e-03, 6.92218542e-04, ...,
      2.49090791e-03, 7.38501549e-03, 1.05685294e-02],
     [2.18084455e-03, 7.71880150e-04, 1.03861094e-04, ...,
      4.57286835e-04, 2.17649341e-03, 6.27920032e-03],
     [7.27033615e-03, 3.02991271e-03, 1.21340156e-03, ...,
      1.92365050e-03, 5.83842397e-03, 1.63541436e-02]],

    [[1.51494443e-02, 2.53537297e-03, 7.02977180e-04, ...,
      7.22974539e-03, 1.03940070e-02, 2.41469443e-02],
     [2.86045671e-03, 4.72068787e-04, 1.08987093e-04, ...,
      2.15992332e-03, 4.55725193e-03, 1.23973191e-02],
     [3.32146883e-03, 7.33464956e-04, 4.30107117e-04, ...,
      1.79910660e-03, 3.71208787e-03, 1.16176009e-02],
     ...,
     [2.11039782e-02, 3.10261548e-02, 7.01978505e-02, ...,
      8.55357349e-02, 3.97977829e-02, 2.49013603e-02],
     [8.98000598e-03, 7.03552365e-03, 8.68326426e-03, ...,
      6.59251213e-03, 5.32424450e-03, 8.27720761e-03],
     [1.41623318e-02, 1.30357146e-02, 1.09080374e-02, ...,
      7.37300515e-03, 7.38766789e-03, 1.39478445e-02]]]],
  dtype=float32), array([[[[0.6213708 , 0.5661254 , 0.4481128 , ..., 0.47417   ,
      0.39457536, 0.31794304],
     [0.60866475, 0.5381434 , 0.4955645 , ..., 0.5112848 ,
      0.4531557 , 0.32479328],
     [0.5456804 , 0.49108282, 0.43284062, ..., 0.46667838,
      0.4916706 , 0.34456927],
     ...,
     [0.5514385 , 0.49755314, 0.46719247, ..., 0.53557044,
      0.48335603, 0.39584845],
     [0.5848158 , 0.5570413 , 0.4873504 , ..., 0.5354574 ,
      0.4694582 , 0.40961075],
     [0.56729645, 0.4964865 , 0.49604815, ..., 0.49212763,
      0.46994185, 0.37628034]],

    [[0.5797265 , 0.55703634, 0.57070756, ..., 0.53550434,
      0.5818699 , 0.6344744 ],
     [0.48733407, 0.43768787, 0.43407708, ..., 0.41947973,
      0.470441  , 0.5378491 ],
     [0.47731167, 0.47195023, 0.5722215 , ..., 0.5269695 ,
      0.50486314, 0.5404542 ],
     ...,
     [0.47561967, 0.5077128 , 0.48406363, ..., 0.43139488,
      0.43169814, 0.4254319 ],
     [0.41377634, 0.4478855 , 0.48052   , ..., 0.503476  ,
      0.49483192, 0.45689604],
     [0.32941583, 0.3828954 , 0.42593485, ..., 0.44739684,
      0.39706135, 0.35836905]],

    [[0.37236294, 0.46066505, 0.54077566, ..., 0.5734195 ,
      0.4744218 , 0.36087704],
     [0.27793106, 0.38961384, 0.49518955, ..., 0.49869967,
      0.4144217 , 0.29593486],
     [0.2490434 , 0.37769425, 0.47748053, ..., 0.4834631 ,
      0.38290572, 0.28281468],
     ...,
     [0.26597092, 0.39243084, 0.49143824, ..., 0.51652086,
      0.43144616, 0.28304714],
     [0.28111193, 0.4223315 , 0.5530058 , ..., 0.55130154,
      0.43678355, 0.28008476],
     [0.33966106, 0.45716453, 0.5357013 , ..., 0.6242875 ,
      0.4995057 , 0.34788686]],

    ...,

    [[0.09541142, 0.0106169 , 0.00353444, ..., 0.01782465,
      0.08237103, 0.26649067],
     [0.06177682, 0.00348103, 0.00145179, ..., 0.02080318,
      0.09804979, 0.1838795 ],
     [0.0975647 , 0.00991598, 0.0057207 , ..., 0.04245368,
      0.11341035, 0.1739361 ],
     ...,
     [0.06827524, 0.03735083, 0.02400085, ..., 0.02770317,
      0.05273968, 0.07968196],
     [0.04161656, 0.00982705, 0.00560099, ..., 0.02726749,
      0.0604848 , 0.10021621],
     [0.06214532, 0.01442632, 0.0111495 , ..., 0.03796089,
      0.09984532, 0.17455974]],

    [[0.08270353, 0.01272744, 0.00861493, ..., 0.01821861,
      0.05675682, 0.16674715],
     [0.095061  , 0.01328394, 0.00812387, ..., 0.02328688,
      0.06117198, 0.17921776],
     [0.12598729, 0.02685288, 0.01897079, ..., 0.04337043,
      0.1049211 , 0.1742737 ],
     ...,
     [0.07125282, 0.01545295, 0.01271072, ..., 0.03880051,
      0.08900312, 0.11251709],
     [0.03476515, 0.00483862, 0.00450209, ..., 0.02490637,
      0.04450244, 0.08635229],
     [0.06149161, 0.01381046, 0.01042584, ..., 0.01981547,
      0.05496967, 0.15815589]],

    [[0.04172212, 0.00397569, 0.00199655, ..., 0.00866416,
      0.02224022, 0.10098869],
     [0.04964879, 0.00369138, 0.00213677, ..., 0.01223791,
      0.02142304, 0.08171579],
     [0.07312593, 0.01034167, 0.00652793, ..., 0.02509719,
      0.04538339, 0.09740078],
     ...,
     [0.06944206, 0.02742767, 0.016478  , ..., 0.05350739,
      0.08730069, 0.08299339],
     [0.0652068 , 0.00961456, 0.00706685, ..., 0.03660542,
      0.04542011, 0.054407  ],
     [0.12979382, 0.02721074, 0.017088  , ..., 0.07418898,
      0.12004897, 0.09969553]]]], dtype=float32)]

@jkjung-avt
Copy link

What are the "shapes" of the output tensors? Are they (1, 52, 52, 3, 8), (1, 26, 26, 3, 8) and (1, 13, 13, 3, 8)?

@akashAD98
Copy link
Author

yes its Output shape: [(1, 51, 52, 52), (1, 51, 26, 26), (1, 51, 13, 13)]
image

@jkjung-avt
Copy link

It looks like the postprocess_boxes() code in inference.ipynb handles channel_last tensors. But the outputs of "your yolov4-mish-custom-electronics-416x416.onnx" are channel_first.

You need to either rewrite/modify the postprocessing code or shuffle the outputs (converting channel_first to channel last)...

@akashAD98
Copy link
Author

akashAD98 commented Nov 22, 2021

image

so i changed NHWC to NCHW using
image_data=np.rollaxis(image_data, 3,0) at the time of loading image

then i got output shape which i have mentioned here #253 (comment)

now i ran last script & got this issue
image

now im confused ,where & which line should i need to do conversion of Channnel first to channel last ? in
postprocess_boxes() can you please help me here

@akashAD98
Copy link
Author

def postprocess_boxes(pred_bbox, org_img_shape, input_size, score_threshold):
'''remove boundary boxs with a low detection probability'''

valid_scale=[0, np.inf]
pred_bbox = np.array(pred_bbox)
pred_xywh = pred_bbox[:, 0:4]
pred_conf = pred_bbox[:, 4]
pred_prob = pred_bbox[:, 5:]

# # (1) (x, y, w, h) --> (xmin, ymin, xmax, ymax)
pred_coor = np.concatenate([pred_xywh[:, :2] - pred_xywh[:, 2:] * 0.5,
                            pred_xywh[:, :2] + pred_xywh[:, 2:] * 0.5], axis=-1)
# # (2) (xmin, ymin, xmax, ymax) -> (xmin_org, ymin_org, xmax_org, ymax_org)
org_h, org_w = org_img_shape
resize_ratio = min(input_size / org_w, input_size / org_h)

dw = (input_size - resize_ratio * org_w) / 2
dh = (input_size - resize_ratio * org_h) / 2

pred_coor[:, 0::2] = 1.0 * (pred_coor[:, 0::2] - dw) / resize_ratio
pred_coor[:, 1::2] = 1.0 * (pred_coor[:, 1::2] - dh) / resize_ratio

# # (3) clip some boxes that are out of range
pred_coor = np.concatenate([np.maximum(pred_coor[:, :2], [0, 0]),
                            np.minimum(pred_coor[:, 2:], [org_w - 1, org_h - 1])], axis=-1)
invalid_mask = np.logical_or((pred_coor[:, 0] > pred_coor[:, 2]), (pred_coor[:, 1] > pred_coor[:, 3]))
pred_coor[invalid_mask] = 0

# # (4) discard some invalid boxes
bboxes_scale = np.sqrt(np.multiply.reduce(pred_coor[:, 2:4] - pred_coor[:, 0:2], axis=-1))
scale_mask = np.logical_and((valid_scale[0] < bboxes_scale), (bboxes_scale < valid_scale[1]))

# # (5) discard some boxes with low scores
classes = np.argmax(pred_prob, axis=-1)
scores = pred_conf * pred_prob[np.arange(len(pred_coor)), classes]
score_mask = scores > score_threshold
mask = np.logical_and(scale_mask, score_mask)
coors, scores, classes = pred_coor[mask], scores[mask], classes[mask]

return np.concatenate([coors, scores[:, np.newaxis], classes[:, np.newaxis]], axis=-1)

@jkjung-avt
Copy link

@akashAD98 This post might be helpful to you: jkjung-avt/tensorrt_demos#510 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants