CARVIEW |
Navigation Menu
-
-
Notifications
You must be signed in to change notification settings - Fork 56.2k
Support FlowNet2 model #16575
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support FlowNet2 model #16575
Conversation
@@ -360,5 +360,67 @@ Ptr<NormalizeBBoxLayer> NormalizeBBoxLayer::create(const LayerParams ¶ms) | |||
return Ptr<NormalizeBBoxLayer>(new NormalizeBBoxLayerImpl(params)); | |||
} | |||
|
|||
|
|||
class ChannelNormLayerImpl CV_FINAL : public ChannelNormLayer |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you think if instead of adding this custom layer we can modify .ptototxt
replacing it to series of layers that give the sample result? In example,
x*x -> reshape (1, 1, C, -1) -> ave pool by H -> multiply by C -> reshape back
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or, which is better, to do the same in the importer
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this case, we need to store outShapes
in importer. Maybe we will have problems with nets without .caffemodel
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we will have problems with nets without .caffemodel.
Why so?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have already tried to calculate outShapes in caffe_importer. Some tests failed in getMemoryShapes
method if we don't use .caffemodel for nets with weighted layers.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I modified channel_norm.prototxt
to a series of layers.
@dkurt Friendly reminder. |
outputs[0][3] = inputs[1][3]; | ||
if ((interpolation == "linear" && (factor_w != 1.0 || factor_h != 1.0)) || | ||
(interpolation == "nearest" && (factor_w < 1.0 || factor_h < 1.0))) | ||
CV_Error(Error::StsNotImplemented, "Unsupported Resize mode"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why they are not supported?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Caffe result with these factors is not equal to OpenCV resize
@dkurt Friendly reminder. |
50af357
to
afc2e4b
Compare
samples/dnn/optical_flow.py
Outdated
|
||
To run model in OpenCV modify Flownetv2.prototxt. | ||
|
||
change ChannelNorm layers to series of layers: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How many ChannelNorm
layers inside? Maybe it make sense to put modified .prototxt
to opencv_extra?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I put modified .prototxt and .caffemodel to cloud.
samples/dnn/optical_flow.py
Outdated
You can download the converted .caffemodel model from https://drive.google.com/open?id=16qvE9VNmU39NttpZwZs81Ga8VYQJDaWZ | ||
or convert .h5 model to .caffemodel yourself. | ||
Download .prototxt from https://drive.google.com/open?id=19bo6SWU2p8ZKvjXqMKiCPdK8mghwDy9b | ||
or modify original Flownetv2.prototxt - change ChannelNorm layers to series of layers as in link above. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please rephrase so user can understand which are required steps and that are optional. Otherwise one can download .caffemodel then "To get original model". I'd like to put everything in one place and say "Download this and that. Otherwise download original model from here".
Changes required in prototxt file I'd like to recommend to write right in the prototxt file as a reference to origin prototxt file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 Well done!
Merge with extra: opencv/opencv_extra#708
relates #16319
Supported layers:
Added tests:
Example
Inputs


Output