How to Convert CO-DETR Model? (original) (raw)

October 31, 2024, 9:41am 1

Description

I’m trying to convert a CO-DETR model to TensorRT and encountering the following error:

[10/30/2024-15:34:42] [E] [TRT] ModelImporter.cpp:726: While parsing node number 6009 [TopK -> "/transformer/TopK_output_cast_0"]:
[10/30/2024-15:34:42] [E] [TRT] ModelImporter.cpp:727: --- Begin node ---
[10/30/2024-15:34:42] [E] [TRT] ModelImporter.cpp:728: input: "/transformer/TopK_input_cast_0"
input: "/transformer/Reshape_26_output_0"
output: "/transformer/TopK_output_cast_0"
output: "/transformer/TopK_output_1"
name: "/transformer/TopK"
op_type: "TopK"
attribute {
  name: "axis"
  i: 1
  type: INT
}
attribute {
  name: "largest"
  i: 1
  type: INT
}
attribute {
  name: "sorted"
  i: 1
  type: INT
}
[10/30/2024-15:34:42] [E] [TRT] ModelImporter.cpp:729: --- End node ---
[10/30/2024-15:34:42] [E] [TRT] ModelImporter.cpp:732: ERROR: ModelImporter.cpp:168 In function parseGraph:
[6] Invalid Node - /transformer/TopK
This version of TensorRT only supports input K as an initializer. Try applying constant folding on the model using Polygraphy: https://github.com/NVIDIA/TensorRT/tree/master/tools/Polygraphy/examples/cli/surgeon/02_folding_constants
[10/30/2024-15:34:42] [E] Failed to parse onnx file
[10/30/2024-15:34:42] [I] Finish parsing network model
[10/30/2024-15:34:42] [E] Parsing model failed
[10/30/2024-15:34:42] [E] Failed to create engine from model or file.
[10/30/2024-15:34:42] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8501] # trtexec --onnx=end2end.onnx --saveEngine=end2end.trt --workspace=65535 --verbose --plugins=../mmdeploy/lib/libmmdeploy_tensorrt_ops.so

I’m using the following Co-DETR model:

CO-DETR Model

The steps are:

1. Run tools/deploy.py from MMDeploy to output the ONNX file:

python mmdeploy/tools/deploy.py \  
        mmdeploy/configs/mmdet/detection/detection_onnxruntime-fp16_dynamic.py \  
        mmdetection/projects/CO-DETR/configs/codino/co_dino_5scale_swin_l_16xb1_16e_o365tococo.py \  
        mmdetection/checkpoints/co_dino_5scale_swin_large_16e_o365tococo-614254c9.pth \  
        mmdetection/demo/demo.jpg \  
        --work-dir mmdetection/checkpoints \  
        --device cuda:0  
        --show  

1. Convert the output ONNX file using trtexec

trtexec --onnx=end2end.onnx --saveEngine=end2end.trt --workspace=65535 --verbose --plugins=../mmdeploy/lib/libmmdeploy_tensorrt_ops.so

1. The above error occurs

Please let me know the cause of this error and how to respond to it.

Environment

TensorRT Version: 8.5.1.7
GPU Type: Tesla V100
Nvidia Driver Version: 525.105.17
CUDA Version: 11.8
CUDNN Version: 8.7.0
Operating System + Version: nvcr.io/nvidia/tensorrt:22.12-py3
Python Version (if applicable): 3.8
TensorFlow Version (if applicable): None
PyTorch Version (if applicable): 2.0.0+cu118
Baremetal or Container (if container which image + tag): nvcr.io/nvidia/tensorrt:22.12-py3

Python packages

mmcv: 2.0.0
mmdeploy: 1.3.1
mmdeploy-runtime-gpu: 1.3.1
mmdet: 3.3.0
mmengine: 0.10.5
onnx: 1.17.0
onnxconverter-common: 1.14.0
onnxruntime-gpu: 1.15.1
openmim: 0.3.9

Relevant Files

https://drive.google.com/drive/folders/1ga0HHb3xLr4nm15jetgF6pt_Sv1CVfHc

Hi @masashi.inoue ,

Have you successfully converted to TRT and inferred the TRT model?

Here is a reference, you may take a look: GitHub - DataXujing/Co-DETR-TensorRT: 🔥 全网首发，mmdetection Co-DETR TensorRT端到端推理加速

Hi @edward.nguyen1705 ,

Thank you very much for the references.

I tried to follow up and do other tests based on this article before, but could not get it to work.

Specifically, the ONNX file output during the process was different from what I expected to be replaced by the script in the repository, and other errors occurred.

I am currently working on a different approach to convert Co-DETR to TensorRT.

Thanks for your support.

I managed to compile the mmdetection Co-DETR model from Pytorch to TensorRT (without the ONNX intermediate representation) by using Torch-TensorRT (e.g. torch.compile).
I can export the model to a serialized TensorRT engine file that can be run in in C++.