CARVIEW |
Select Language
HTTP/2 200
date: Thu, 24 Jul 2025 10:11:04 GMT
content-type: text/html; charset=utf-8
cache-control: max-age=0, private, must-revalidate
content-security-policy: default-src 'none'; base-uri 'self'; child-src github.githubassets.com github.com/assets-cdn/worker/ github.com/assets/ gist.github.com/assets-cdn/worker/; connect-src 'self' uploads.github.com www.githubstatus.com collector.github.com raw.githubusercontent.com api.github.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com *.rel.tunnels.api.visualstudio.com wss://*.rel.tunnels.api.visualstudio.com objects-origin.githubusercontent.com copilot-proxy.githubusercontent.com proxy.individual.githubcopilot.com proxy.business.githubcopilot.com proxy.enterprise.githubcopilot.com *.actions.githubusercontent.com wss://*.actions.githubusercontent.com productionresultssa0.blob.core.windows.net/ productionresultssa1.blob.core.windows.net/ productionresultssa2.blob.core.windows.net/ productionresultssa3.blob.core.windows.net/ productionresultssa4.blob.core.windows.net/ productionresultssa5.blob.core.windows.net/ productionresultssa6.blob.core.windows.net/ productionresultssa7.blob.core.windows.net/ productionresultssa8.blob.core.windows.net/ productionresultssa9.blob.core.windows.net/ productionresultssa10.blob.core.windows.net/ productionresultssa11.blob.core.windows.net/ productionresultssa12.blob.core.windows.net/ productionresultssa13.blob.core.windows.net/ productionresultssa14.blob.core.windows.net/ productionresultssa15.blob.core.windows.net/ productionresultssa16.blob.core.windows.net/ productionresultssa17.blob.core.windows.net/ productionresultssa18.blob.core.windows.net/ productionresultssa19.blob.core.windows.net/ github-production-repository-image-32fea6.s3.amazonaws.com github-production-release-asset-2e65be.s3.amazonaws.com insights.github.com wss://alive.github.com api.githubcopilot.com api.individual.githubcopilot.com api.business.githubcopilot.com api.enterprise.githubcopilot.com; font-src github.githubassets.com; form-action 'self' github.com gist.github.com copilot-workspace.githubnext.com objects-origin.githubusercontent.com; frame-ancestors 'none'; frame-src viewscreen.githubusercontent.com notebooks.githubusercontent.com; img-src 'self' data: blob: github.githubassets.com media.githubusercontent.com camo.githubusercontent.com identicons.github.com avatars.githubusercontent.com private-avatars.githubusercontent.com github-cloud.s3.amazonaws.com objects.githubusercontent.com release-assets.githubusercontent.com secured-user-images.githubusercontent.com/ user-images.githubusercontent.com/ private-user-images.githubusercontent.com opengraph.githubassets.com copilotprodattachments.blob.core.windows.net/github-production-copilot-attachments/ github-production-user-asset-6210df.s3.amazonaws.com customer-stories-feed.github.com spotlights-feed.github.com objects-origin.githubusercontent.com *.githubusercontent.com; manifest-src 'self'; media-src github.com user-images.githubusercontent.com/ secured-user-images.githubusercontent.com/ private-user-images.githubusercontent.com github-production-user-asset-6210df.s3.amazonaws.com gist.github.com; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; upgrade-insecure-requests; worker-src github.githubassets.com github.com/assets-cdn/worker/ github.com/assets/ gist.github.com/assets-cdn/worker/
link: ; rel=preload; as=fetch; crossorigin=use-credentials
referrer-policy: no-referrer-when-downgrade
server-timing: issue_layout-fragment;desc="issue_layout fragment";dur=174.688992,issue_conversation_content-fragment;desc="issue_conversation_content fragment";dur=436.02575,issue_conversation_sidebar-fragment;desc="issue_conversation_sidebar fragment";dur=42.763867,nginx;desc="NGINX";dur=1.215464,glb;desc="GLB";dur=102.88358
strict-transport-security: max-age=31536000; includeSubdomains; preload
vary: X-PJAX, X-PJAX-Container, Turbo-Visit, Turbo-Frame, X-Requested-With, Accept,Accept-Encoding, Accept, X-Requested-With
x-content-type-options: nosniff
x-frame-options: deny
x-voltron-version: fd8fbbc
x-xss-protection: 0
server: github.com
content-encoding: gzip
accept-ranges: bytes
set-cookie: _gh_sess=gcOnqt6%2Byqm2spOmAMhfKuKeyeDci04WqCKouexCHAZNsJVcc0OiJmIvvJv%2Fbl0OzrTyEwFpuJnzrncByLIsVegP2xnu%2BxxFaV0uaOnczcPQygFE5h5rrl45OIuSOFVMI1j0Lqv8t5OHm3gC8jBNTuVU%2B0rkqfsexEliYwhKjyKkQzx4OXNCq%2BhVd61aluTPASTQD6QnWz55dwIXCdBUlWY9pWlifTh4kypHEK9biblf64BjN0zpamkENYhRSiqgYkfW2ldrVikzb1%2B1g1E30Q%3D%3D--TBQ5EqxKlFRu%2FGFq--CrZ9YpjOLUhcYb%2BB12ELIg%3D%3D; Path=/; HttpOnly; Secure; SameSite=Lax
set-cookie: _octo=GH1.1.964398491.1753351863; Path=/; Domain=github.com; Expires=Fri, 24 Jul 2026 10:11:03 GMT; Secure; SameSite=Lax
set-cookie: logged_in=no; Path=/; Domain=github.com; Expires=Fri, 24 Jul 2026 10:11:03 GMT; HttpOnly; Secure; SameSite=Lax
x-github-request-id: 83BC:941DF:187F290:1D410FF:688206B7
Cannot load the onnx model converted from pytorch Β· Issue #16971 Β· opencv/opencv Β· GitHub
No one assignedNo typeNo projectsNo milestoneNone yetNo branches or pull requests
Skip to content
Navigation Menu
{{ message }}
-
-
Notifications
You must be signed in to change notification settings - Fork 56.2k
Closed
Labels
Description
System information (version)
- OpenCV => OpenCV => 4.3.0(e6f32c6)
- Operating System / Platform => Windows 64 Bit
- Compiler => Visual Studio 2017 64 bits
Detailed description
The decoder model converted from this project do not work.
Steps to reproduce
- install pytorch 1.4.0(cpu version)
- clone the project
- Navigate to the folder of the project, create a file called to_onnx.py
from __future__ import absolute_import, division, print_function
import os
import sys
import glob
import numpy as np
import PIL.Image as pil
import matplotlib as mpl
import matplotlib.cm as cm
import torch
from torchvision import transforms, datasets
import networks
from layers import disp_to_depth
from utils import download_model_if_doesnt_exist
device = torch.device("cpu")
model_name = "mono+stereo_640x192"
model_path = os.path.join("models", "mono+stereo_640x192")
print("-> Loading model from ", model_path)
encoder_path = os.path.join(model_path, "encoder.pth")
depth_decoder_path = os.path.join(model_path, "depth.pth")
# LOADING PRETRAINED MODEL
print(" Loading pretrained encoder")
encoder = networks.ResnetEncoder(18, False)
loaded_dict_enc = torch.load(encoder_path, map_location=device)
# extract the height and width of image that this model was trained with
feed_height = loaded_dict_enc['height']
feed_width = loaded_dict_enc['width']
filtered_dict_enc = {k: v for k, v in loaded_dict_enc.items() if k in encoder.state_dict()}
encoder.load_state_dict(filtered_dict_enc)
encoder.to(device)
encoder.eval()
import onnx
import onnxruntime
def test_model_accuracy(export_model_name, raw_output, input):
ort_session = onnxruntime.InferenceSession(export_model_name)
def to_numpy(tensor):
return tensor.detach().cpu().numpy() if tensor.requires_grad else tensor.cpu().numpy()
# compute ONNX Runtime output prediction
ort_inputs = {ort_session.get_inputs()[0].name: to_numpy(input)}
ort_outs = ort_session.run(None, ort_inputs)
# compare ONNX Runtime and PyTorch results
np.testing.assert_allclose(to_numpy(raw_output), ort_outs[0], rtol=1e-03, atol=1e-05)
print("Exported model has been tested with ONNXRuntime, and the result looks good!")
def export_model(model, input, export_model_name):
torch.onnx.export(model, input, export_model_name, verbose=False, export_params=True, opset_version=11)
onnx_model = onnx.load(export_model_name)
onnx.checker.check_model(onnx_model)
graph_output = onnx.helper.printable_graph(onnx_model.graph)
with open("graph_output.txt", mode="w") as fout:
fout.write(graph_output)
export_model_name = "encoder_" + model_name + '.onnx'
input_image = pil.open("assets/runner.jpg").convert('RGB')
input_image = input_image.resize((feed_width, feed_height), pil.LANCZOS)
input_image = transforms.ToTensor()(input_image).unsqueeze(0)
features = encoder(input_image)
export_model(encoder, input_image, export_model_name)
test_model_accuracy(export_model_name, features, input_image)
#print(" Loading pretrained decoder")
depth_decoder = networks.DepthDecoder(
num_ch_enc = encoder.num_ch_enc, scales=range(4))
loaded_dict = torch.load(depth_decoder_path, map_location=device)
depth_decoder.load_state_dict(loaded_dict)
depth_decoder.to(device)
depth_decoder.eval()
decoder_output = depth_decoder(features)
export_model_name = "decoder_" + model_name + '.onnx'
export_model(depth_decoder, features, export_model_name)
test_model_accuracy(export_model_name, decoder_output, features)
- Change the class ResnetEncoder in networks/resnet_encoder.py to
class ResnetEncoder(nn.Module):
"""Pytorch module for a resnet encoder
"""
def __init__(self, num_layers, pretrained, num_input_images=1):
super(ResnetEncoder, self).__init__()
self.num_ch_enc = np.array([64, 64, 128, 256, 512])
resnets = models.resnet18
self.encoder = resnets(pretrained)
def forward(self, input_image):
x = (input_image - 0.45) / 0.225
x = self.encoder.conv1(x)
x = self.encoder.bn1(x)
output1 = self.encoder.relu(x)
output2 = self.encoder.layer1(self.encoder.maxpool(output1))
output3 = self.encoder.layer2(output2)
output4 = self.encoder.layer3(output3)
output5 = self.encoder.layer4(output4)
return torch.cat((output1.flatten(), output2.flatten(), output3.flatten(), output4.flatten(), output5.flatten()))
- Change the class DepthDecoder in networks/depth_decoder.py to
class DepthDecoder(nn.Module):
def __init__(self, num_ch_enc, scales=range(4), num_output_channels=1, use_skips=True):
super(DepthDecoder, self).__init__()
self.num_output_channels = num_output_channels
self.use_skips = use_skips
self.upsample_mode = 'nearest'
self.scales = scales
self.num_ch_enc = num_ch_enc
self.num_ch_dec = np.array([16, 32, 64, 128, 256])
# decoder
self.convs = OrderedDict()
for i in range(4, -1, -1):
# upconv_0
num_ch_in = self.num_ch_enc[-1] if i == 4 else self.num_ch_dec[i + 1]
num_ch_out = self.num_ch_dec[i]
self.convs[("upconv", i, 0)] = ConvBlock(num_ch_in, num_ch_out)
# upconv_1
num_ch_in = self.num_ch_dec[i]
if self.use_skips and i > 0:
num_ch_in += self.num_ch_enc[i - 1]
num_ch_out = self.num_ch_dec[i]
self.convs[("upconv", i, 1)] = ConvBlock(num_ch_in, num_ch_out)
for s in self.scales:
self.convs[("dispconv", s)] = Conv3x3(self.num_ch_dec[s], self.num_output_channels)
self.decoder = nn.ModuleList(list(self.convs.values()))
self.sigmoid = nn.Sigmoid()
def forward(self, input_features):
#simplify the forward functions a lot, only post the codes which cause the issue
#the output processed by self.convs[("upconv", 4, 0)] will caused the error message
return self.convs[("upconv", 4, 0)](input_features[2826240:2887680].reshape(1, 512, 6, 20))
- Load the model by opencv
std::string const model_folder("C:/Users/yyyy/programs/Qt/app_to_simplify_ml/"
"cv_projects/clients/mobile_depth_estimation/monodepth2");
cv::dnn::Net net = cv::dnn::readNet(model_folder + "/decoder_mono+stereo_640x192.onnx");
Both of the decoder and encoder models passed the test.
To save your troubles, you can have the graph,model and codes(to_onnx.py, resnet_encoder.py, depth_decoder.py) from google drive.
Issue submission checklist
Cannot load the encoder model after convert, got error messages
OpenCV: terminate handler is called! The last OpenCV error is:OpenCV(4.3.0-pre) Error: Assertion failed (shapeIt != outShapes.end()) in cv::dnn::dnn4_v20200310::ONNXImporter::populateNet, file C:\Users\yyyy\programs\Qt\3rdLibs\opencv\dev\opencv\modules\dnn\src\onnx\onnx_importer.cpp, line 1116
ashishkrshrivastava and berak
Metadata
Metadata
Assignees
Labels
Type
Projects
Milestone
Relationships
Development
Issue actions
You canβt perform that action at this time.