[GSoC] Added TF and PyTorch classification conversion cases #17604

LupusSanctus · 2020-06-21T21:19:49Z

GSoC project: "Write a tutorial about training a network in TF or PyTorch and then running it in OpenCV"

Part 1. Classification case.

Provided dnn_model_runner module with the below-listed functions:

evaluation mode: a full pipeline for the evaluation of PyTorch and TF models (conversion, OpenCV Net retrievement, data fetch and preparation, evaluation)
test mode: integrated with samples/dnn/classification.py for inference execution
includes autonomous scripts with step by step TF/PyTorch models => OpenCV Net conversion and inference (described in tutorials)

Provided tutorial series

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under OpenCV (BSD) License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
The PR is proposed to proper branch
There is reference to original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

samples/python/tutorial_code/dnn/dnn_conversion/common/accuracy_eval.py

samples/python/tutorial_code/dnn/dnn_conversion/requirements.txt

samples/python/tutorial_code/dnn/dnn_conversion/tf/classification/py_to_py_resnet50.py

samples/python/tutorial_code/dnn/dnn_conversion/tf/tf_model.py

alalek

Need to clarify licensing terms of used images.

doc/tutorials/dnn/dnn_pytorch_tf_classification/pytorch_cls_model_conversion_tutorial.md

samples/dnn/common.py

samples/dnn/dnn_model_runner/dnn_conversion/requirements.txt

doc/tutorials/dnn/dnn_pytorch_tf_classification/tf_cls_model_conversion_tutorial.md

samples/dnn/dnn_model_runner/dnn_conversion/requirements.txt

asmorkalov · 2020-11-24T11:10:45Z

There is conflict in requirements.txt. Pip reports:

ERROR: Double requirement given: torch>=1.7.0.dev20200727 (from -r ./requirements.txt (line 15)) (already in torch>=1.5.1 (from -r ./requirements.txt (line 5)), name='torch')

asmorkalov · 2020-11-24T11:11:28Z

Please add bock with setup instructions. Let's expect that user already have python3 and pip installed.

doc/tutorials/dnn/dnn_pytorch_tf_classification/pytorch_cls_model_conversion_c_tutorial.md

doc/tutorials/dnn/dnn_pytorch_tf_classification/pytorch_cls_model_conversion_tutorial.md

asmorkalov · 2020-11-24T12:04:51Z

@LupusSanctus Sorry for confusion, please ignore the requirements.txt issue. I used old version of the code. Please remove OpenCV package from requirements.txt and add line to tutorial or just leave OpenCV without version. 4.5.x is not available with pip now.

alalek · 2020-12-04T18:11:48Z

Please rebase commits to resolve merge conflicts.

LupusSanctus · 2020-12-10T22:34:56Z

@asmorkalov, setup instructions were added in tutorial's Requirements section. Also requirements.txt was corrected.

alalek · 2020-12-18T08:36:59Z

Is this PR in WIP status?

alalek

Well done!

Please take a look on minor comments below.

doc/tutorials/dnn/dnn_pytorch_tf_classification/pytorch_cls_model_conversion_c_tutorial.md

doc/tutorials/dnn/dnn_pytorch_tf_classification/tf_cls_model_conversion_tutorial.md

asmorkalov · 2021-01-02T12:12:05Z

samples/dnn/classification.py

+
+def get_args_parser(func_args):
+    backends = (cv.dnn.DNN_BACKEND_DEFAULT, cv.dnn.DNN_BACKEND_HALIDE, cv.dnn.DNN_BACKEND_INFERENCE_ENGINE,
+                cv.dnn.DNN_BACKEND_OPENCV)


samples/dnn/classification.py

asmorkalov · 2021-01-02T12:19:09Z

samples/dnn/dnn_model_runner/dnn_conversion/common/img_utils.py

+    img = img.astype(np.float32)
+    img *= BASE_IMG_SCALE_FACTOR
+    img -= [0.485, 0.456, 0.406]
+    img /= [0.229, 0.224, 0.225]


The same values are in default_preprocess_config.py. Let's reuse the constants.

asmorkalov · 2021-01-18T09:15:28Z

@LupusSanctus Friendly reminder.

alalek

Well done! Thank you 👍

…utorial [GSoC] Added TF and PyTorch classification conversion cases * Added TF and PyTorch classification conversion cases * Modified structure, some processing scripts. Added evaluation pipeline * Minor structure change * Removed extra functions, minor structure change * Modified structure, code corrections * Updated classification code block, added classification tutorials * Added minor modifications of paths * Classification block corrections in accordance with comments

asmorkalov added category: dnn category: documentation Documentation fix or update category: samples GSoC labels Jun 22, 2020

LupusSanctus force-pushed the am/pytorch_tf_cls_tutorial branch from b1747b0 to 1eba64f Compare July 2, 2020 08:28

iamannakogan reviewed Jul 10, 2020

View reviewed changes

LupusSanctus force-pushed the am/pytorch_tf_cls_tutorial branch from dfd9c89 to a772769 Compare July 10, 2020 13:25

LupusSanctus mentioned this pull request Jul 20, 2020

[GSoC] Added TF and PyTorch segmentation conversion cases #17801

Merged

6 tasks

LupusSanctus force-pushed the am/pytorch_tf_cls_tutorial branch from a772769 to 7e6cb9c Compare July 31, 2020 12:57

LupusSanctus force-pushed the am/pytorch_tf_cls_tutorial branch from 7e6cb9c to 3907135 Compare August 30, 2020 20:03

LupusSanctus changed the title ~~WIP: Added TF and PyTorch classification conversion cases~~ WIP: [GSoC] Added TF and PyTorch classification conversion cases Aug 31, 2020

LupusSanctus mentioned this pull request Aug 31, 2020

[GSoC] Added TF and PyTorch detection model conversion cases #18237

Merged

6 tasks

alalek reviewed Aug 31, 2020

View reviewed changes

doc/tutorials/dnn/dnn_pytorch_tf_classification/pytorch_cls_model_conversion_tutorial.md Outdated Show resolved Hide resolved

samples/dnn/common.py Outdated Show resolved Hide resolved

samples/dnn/common.py Outdated Show resolved Hide resolved

asmorkalov requested changes Sep 4, 2020

View reviewed changes

LupusSanctus force-pushed the am/pytorch_tf_cls_tutorial branch 6 times, most recently from e93ee9d to b4e36ea Compare November 24, 2020 10:22

asmorkalov requested changes Nov 24, 2020

View reviewed changes

LupusSanctus force-pushed the am/pytorch_tf_cls_tutorial branch from b4e36ea to 968723b Compare December 10, 2020 22:30

LupusSanctus changed the title ~~WIP: [GSoC] Added TF and PyTorch classification conversion cases~~ [GSoC] Added TF and PyTorch classification conversion cases Dec 18, 2020

alalek reviewed Dec 20, 2020

View reviewed changes

asmorkalov requested changes Jan 2, 2021

View reviewed changes

Anastasia added 8 commits January 24, 2021 00:04

Added TF and PyTorch classification conversion cases

563b947

Modified structure, some processing scripts. Added evaluation pipeline

1843d44

Minor structure change

323d25f

Removed extra functions, minor structure change

6e5e9c8

Modified structure, code corrections

4906cc7

Updated classification code block, added classification tutorials

c8acbd4

Added minor modifications of paths

045e995

Classification block corrections in accordance with comments

1cb5054

LupusSanctus force-pushed the am/pytorch_tf_cls_tutorial branch from 968723b to 1cb5054 Compare January 25, 2021 10:52

asmorkalov approved these changes Jan 26, 2021

View reviewed changes

alalek approved these changes Jan 26, 2021

View reviewed changes

alalek assigned asmorkalov Jan 26, 2021

alalek merged commit 8c22198 into opencv:master Jan 26, 2021

alalek mentioned this pull request Apr 9, 2021

(5.x) Merge 4.x #19885

Merged

asmorkalov mentioned this pull request Jun 4, 2021

Samples and tutorials for the Dnn High Level API #15240

Closed

Uh oh!

[GSoC] Added TF and PyTorch classification conversion cases #17604

[GSoC] Added TF and PyTorch classification conversion cases #17604

Uh oh!

Conversation

LupusSanctus commented Jun 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Readiness Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alalek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

asmorkalov commented Nov 24, 2020

Uh oh!

asmorkalov commented Nov 24, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

asmorkalov commented Nov 24, 2020

Uh oh!

alalek commented Dec 4, 2020

Uh oh!

LupusSanctus commented Dec 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alalek commented Dec 18, 2020

Uh oh!

alalek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

asmorkalov Jan 2, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

asmorkalov Jan 2, 2021

Choose a reason for hiding this comment

Uh oh!

asmorkalov commented Jan 18, 2021

Uh oh!

alalek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

LupusSanctus commented Jun 21, 2020 •

edited

Loading

LupusSanctus commented Dec 10, 2020 •

edited

Loading