SolvedTensorRT virtual nvinfer1::ILayer* nvinfer1::Network::getLayer(int) const: Assertion `layerIndex >= 0' failed.

After solving the [TensorRT] ERROR: Network must have at least one output

Another error happened

The code is:

# The Onnx path is used for Onnx models.
def build_engine_onnx(model_file):
  print('debug1')
  with trt.Builder(TRT_LOGGER) as builder, builder.create_network() as network, trt.OnnxParser(network, TRT_LOGGER) as parser:
    print('debug2')
    builder.max_workspace_size = common.GiB(1)
    print('debug3')
    # Load the Onnx model and parse it in order to populate the TensorRT network.
    with open(model_file, 'rb') as model:
      print('debug4')
      parser.parse(model.read())
      print('debug5')
    last_layer = network.get_layer(network.num_layers - 1)
    # Check if last layer recognizes it's output
    if not last_layer.get_output(0):
        # If not, then mark the output using TensorRT API
        network.mark_output(last_layer.get_output(0))
    return builder.build_cuda_engine(network)

The output is:

debug1
debug2
debug3
debug4
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:604] Reading dangerously large protocol message.  If the message turns out to be larger than 2147483647 bytes, parsing will be halted for security reasons.  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:81] The total number of bytes read was 765571023
debug5
python: ../builder/Network.cpp:863: virtual nvinfer1::ILayer* nvinfer1::Network::getLayer(int) const: Assertion `layerIndex >= 0' failed.
Aborted (core dumped)
(tensorrt) nvidia@Dell:~/Desktop/onnx_trt$ 

My pytorch version is 1.3.0.
My tensorrt version is 6.0.1.5.

Which situation lead to this problem?
Thanks.

16 Answers

✔️Accepted Answer

thanks @rmccorm4

the code snippet really helped to debug the issue. following was the error while parsing the onnx model:

In node -1 (importModel): INVALID_VALUE: Assertion failed: !_importer_ctx.network()->hasImplicitBatchDimension() && "This version of the ONNX parser only supports TensorRT INetworkDefinitions with an explicit batch dimension. Please ensure the network was created using the EXPLICIT_BATCH NetworkDefinitionCreationFlag."

i was able resolve this by doing

explicit_batch = 1 << (int)(tensorrt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH)
builder.create_network(explicit_batch)

thanks

Other Answers:

The onnx-simplifier is helpful!
Thank you!

Related Issues:

43
TensorRT [TensorRT] ERROR: Network must have at least one output
Hi @RizhaoCai I just had the same issue with TensorRT 7.0 The solution as mentioned in the documenta...
16
TensorRT virtual nvinfer1::ILayer* nvinfer1::Network::getLayer(int) const: Assertion `layerIndex >= 0' failed.
thanks @rmccorm4 the code snippet really helped to debug the issue After solving the [TensorRT] ERRO...
10
TensorRT ONNX networks can't use INT8 calibration and batching
Hi @gcp Sorry for the delay Description This is due to mutually incompatible changes in the TRT7 rel...
6
TensorRT bilinear IResizeLayer setting align_corners=False makes different result to torch interpolate
and further more Description the result from IResizeLayer(scale_factor=scale_factor ...
926
keras Loading model with custom loss function: ValueError: 'Unknown loss function'
I solved this problem by adding 'custom_bojects' my loss function: I trained and saved a model that ...
633
keras Tensorflow backend - bug in model._make_predict_function(...)
I had this problem when doing inference in a different thread than where I loaded my model ...
516
keras Creating Model variable throws "AttributeError: 'Tensor' object has no attribute '_keras_history'"
@zaid478 oh I assumed you were doing it on coursera Anyway Hello all setup: Keras 2.0 Tensorflow 1.0...
456
tensorflow ImportError: libcublas.so.10.0: cannot open shared object file: No such file or directory
@rhinsall I just found this out myself not sure if it's common knowledge but got around this by doin...
395
keras "<tensor> is not an element of this graph." when loading model.
I had a problem similar to that of @piraka9011 which was solved by calling model._make_predict_funct...
387
pytorch I can't import PyTorch, libomp.dylib can't be loaded.
brew install libomp solves the problem. 🐛 Bug I tried to install PyTorch but I can't use it ...
380
pytorch RuntimeError: cuda runtime error (59) : device-side assert triggered when running transfer_learning_tutorial
@soumith Hi I find the error it is due to my labels contain value -1 which is unacceptable in criter...
356
tensorflow At Runtime : "Error while reading resource variable softmax/kernel from Container: localhost"
I had the same issue in tensorflow 1.13.1 which I have resolved by creating a reference to the sessi...
355
tensorflow Mac + Python 3.6.1: Attempting to download mnist data results in CERTIFICATE_VERIFY_FAILED error
This might have also been because I recently switched to a new machine and it's still only partially...
308
tensorflow UnrecognizedFlagError: Unknown command line flag 'f'
I solved this issue by adding the line: tf.app.flags.DEFINE_string('f' '' 'kernel') what this essent...
298
tensorflow FutureWarning: Deprecated numpy API calls in tf.python.framework.dtypes
@Akeaakar And for the time being pip install numpy<1.17 to revert to numpy version 1.16.4 ...
265
tensorflow ERROR: Cannot uninstall 'wrapt'. during upgrade
This worked for me: pip install wrapt --upgrade --ignore-installed pip install tensorflow ...
254
pytorch ModuleNotFoundError: No module named 'torch._C'
Solution the problem is that you have a folder called torch in the same directory which is being pic...
252
tensorflow Keras application - Tensor is not an element of this graph on eval after train
K.clear_session() did not work for me however what worked was : While predicting use the same graph ...
204
keras Quick Question: can a model be fit for multiple times?
Yes successive calls to fit will incrementally train the model. I'm now having something around 5000...
196
tensorflow InternalError: Blas GEMM launch failed
Make sure you have no other processes using the GPU running Run nvidia-smi to check this. ...
194
pytorch RuntimeError: DataLoader worker is killed by signal: Killed.
I've encountered the same problem recently If you're using the docker to run the PyTorch program wit...
188
tensorflow Is python 3.7.x supported with Tensorflow
I sill get Ive been trying to install Tensorflow on my computer which currently runs python 3.7 howe...
184
tensorflow Tensorflow installation Error in Mac Sierra due to numpy
Also Had to do this with the six dependency as well Hi I want to install TensorFlow on Mac Currently...
183
keras Error when checking model target: expected activation_2 to have shape (None, 10) but got array with shape (3, 1)
Your model has an output of shape (10,) however your outputs have dimension (1,) You probably want t...
180
tensorflow tensorboard: view graph from saved_model.pb file [feature request]
@brandondutra As far as I know you don't need to create any summaries to load the graph into Tensorb...
178
keras Is there a way in Keras to apply different weights to a cost function?
Ok so I had the time to quickly test it Hi there I am trying to implement a classification problem w...
178
tensorflow ImportError: libcudart.so.8.0: cannot open shared object file: No such file or directory
Double check that you have exported the LD_LIBRARY_PATH export LD_LIBRARY_PATH=/usr/local/cuda-8.0/l...
176
pytorch [PyTorch][Feature Request] Label Smoothing for CrossEntropyLoss
Here is my implement Hi guys The type torch.LongTensor of target will hinder the implementation like...
172
caffe libstdc++.so.6: version `GLIBCXX_3.4.21' not found
^ Should fix this error Please ask installation questions only in the Caffe mailing list. ...
172
keras Keras model cannot be loaded if it contains a Lambda layer calling tf.image.resize_images
Try using custom_objects argument in load_model (or model_from_json) function For your case load_mod...
167
164
keras how to use fit_generator with multiple image inputs
Thanks Here is how I solved it following some ideas from issue 3386 maybe someone might find it usef...
154
tensorflow Tensorflow Failed to create Session
maybe out of GPU memory? Try running with export CUDA_VISIBLE_DEVICES='' Hi I tried basic program in...
151
keras Expected 3 dimensions but got array with shape (11, 2)
The problem is that you start with a three dimensional layer but never reduce the dimensionality in ...
148
keras AttributeError: 'ProgbarLogger' object has no attribute 'log_values'
This happens if steps_per_epoch is 0 Make sure that your batch size is not greater than the dataset ...
147
keras For large datasets, which to use: fit or train_on_batch?
I think the best way to fit large data which can not fit into memory is writing a customize generato...
147
keras Split train data into training and validation when using ImageDataGenerator and model.fit_generator
I just found this in the documentation given here Its okay if I am keeping my training and validatio...
146
pytorch optimizer load_state_dict() problem?
@apaszke Ah my bad I forgot to update the line where the optimizer is recreated But otherwise the fo...
145
keras using pre trained VGG16 for another classification task
One way to do this is to not include the fully-connected layers at the top of the network Then add n...
136
tensorflow Error in `python': double free or corruption (!prev)
Can confirm this I ran into this on my CI server and the the following fixed it: ...
127
tensorflow Why this error on tensorflow 1.13.1 with python 2.7 : ImportError: No module named model_utils
I was able to solve this under Python 3.6 on Linux with: System information Have I written custom co...
123
keras "Merge" versus "merge", what is the difference?
Merge is a layer Merge takes layers as input Merge is usually used with Sequential models merge is a...
123
tensorflow TF 1.x: remove the "deprecated" warning messages
On 1.14.0 only this works: I know the functional APIs such as tf.layers.dense will disappear in TF 2...
115
keras load_model() with custom layers, and custom layers in general
For the people landing here by a Google search the code we should use: as can be found on kapre's Gi...
115
tensorflow Couldn't open CUDA library libcupti.so.8.0
PIP has no relation to the environment variable Based on your reply I assume you did not set it ...
112
tensorflow can't rollback to python 3.6.5 due to recursive dependency between sphinx-doc and python
Had the same issues Please make sure that this is a build/installation issue As per our GitHub Polic...
109
tensorflow AttributeError: module 'tensorflow' has no attribute 'app'
Try to use: import tensorflow.compat.v1 as tf Hi I'm new in the field of machine learning ...