SolvedTensorRT [TensorRT] ERROR: Network must have at least one output

Hi, I try to convert my .onnx model to .trt model using tensorrt, my script is:

import tensorrt as trt

def ONNX_build_engine(onnx_file_path, engine_file_path):
    G_LOGGER = trt.Logger(trt.Logger.WARNING)
    with trt.Builder(G_LOGGER) as builder, builder.create_network() as network, trt.OnnxParser(network, G_LOGGER) as parser:
        builder.max_batch_size = 16
        builder.max_workspace_size = 1 << 20

        print('Loading ONNX file from path {}...'.format(onnx_file_path))
        with open(onnx_file_path, 'rb') as model:
            print('Beginning ONNX file parsing')
            parser.parse(model.read())
        print('Completed parsing of ONNX file')

        print('Building an engine from file {}; this may take a while...'.format(onnx_file_path))
        engine = builder.build_cuda_engine(network)
        print("Completed creating Engine")

        with open(engine_file_path, "wb") as f:
            f.write(engine.serialize())
        return engine

ONNX_build_engine('./BiSeNet.onnx', './BiSeNet.trt')

I have tested in my two desktops, one configue:

cuda 10.0
pytorch1.3
torch 5.0.6.6

the other configure:

cuda 10.1
pytorch1.3
torch6.0.1.5

Both have the same error. there is an error:

Loading ONNX file from path ./BiSeNet_simplifier.onnx...
Beginning ONNX file parsing
Completed parsing of ONNX file
Building an engine from file ./BiSeNet_simplifier.onnx; this may take a while...
[TensorRT] ERROR: Network must have at least one output
Completed creating Engine
Traceback (most recent call last):
File "onnx2trt.py", line 31, in
ONNX_build_engine('./BiSeNet_simplifier.onnx', './BiSeNet.trt')
File "onnx2trt.py", line 28, in ONNX_build_engine
f.write(engine.serialize())
AttributeError: 'NoneType' object has no attribute 'serialize'

The .onnx file is converted by torch.onnx.export fcn, it seems right.

import torch
from model.build_BiSeNet import BiSeNet

model = BiSeNet(13, 'resnet18').eval()  # .cuda().half().eval()
model.load_state_dict(torch.load('./checkpoints_18_sgd/best_dice_loss_13_class.pth'))
x = torch.ones((1, 3, 720, 960))  # .cuda().half()
output_names = ["output"]
onnx_model = torch.onnx._export(model, x, "BiSeNet.onnx", verbose=False, output_names=output_names, export_params=True) #,  keep_initializers_as_inputs=True)
22 Answers

✔️Accepted Answer

Hi @RizhaoCai

I just had the same issue with TensorRT 7.0. The solution, as mentioned in the documentaiton, was creating the network using the EXPLICIT_BATCH flag:

explicit_batch = 1 << (int)(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH)
network = builder.create_network(explicit_batch)

Other Answers:

just add

last_layer = network.get_layer(network.num_layers - 1)
network.mark_output(last_layer.get_output(0))

it will work. It's a necessary procedure to convert successfully, I'm curious why most tutorials don't mention it.

More Issues: