[G-API] Enable 2D inputs for GIEBackend #20925

TolyaTalamanov · 2021-10-22T08:39:28Z

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
The PR is proposed to proper branch
There is reference to original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

Build configuration

NV12 color format is missed.
Build configuration
force_builders=Custom,Custom Win,Custom Mac
build_gapi_standalone:Linux x64=ade-0.1.1f
build_gapi_standalone:Win64=ade-0.1.1f
build_gapi_standalone:Mac=ade-0.1.1f
build_gapi_standalone:Linux x64 Debug=ade-0.1.1f
Xbuild_image:Custom=centos:7
Xbuildworker:Custom=linux-1
build_gapi_standalone:Custom=ade-0.1.1f
build_image:Custom=ubuntu-openvino-2021.4.1:20.04
build_image:Custom Win=openvino-2021.4.1
build_image:Custom Mac=openvino-2021.4.1
test_modules:Custom=gapi,python2,python3,java
test_modules:Custom Win=gapi,python2,python3,java
test_modules:Custom Mac=gapi,python2,python3,java
buildworker:Custom=linux-1
# disabled due high memory usage: test_opencl:Custom=ON
test_opencl:Custom=OFF
test_bigdata:Custom=1
test_filter:Custom=*

mpashchenkov · 2021-10-22T11:40:17Z

modules/gapi/src/backends/ie/giebackend.cpp

+
+            // NB: For some reason RGB image is 2D image
+            // (since channel component is not counted here).
+            // In order to disnguish 2D tensor from image, have a look on model


disnguish -- > distinguish

TolyaTalamanov · 2021-10-26T14:58:13Z

This branch based: #20918

TolyaTalamanov · 2021-11-03T07:35:46Z

@alalek These warnings come from ngraph

   572>C:\Program Files (x86)\IntelSWTools\openvino_2021.4.689\deployment_tools\ngraph\include\ngraph/op/abs.hpp(26,57): warning C4100: 'visitor': unreferenced formal parameter (compiling source file C:\build\precommit_custom_windows\opencv\modules\gapi\test\infer\gapi_infer_ie_test.cpp) [C:\build\precommit_custom_windows\build\modules\gapi\opencv_test_gapi.vcxproj]
   572>C:\Program Files (x86)\IntelSWTools\openvino_2021.4.689\deployment_tools\ngraph\include\ngraph/op/acos.hpp(34,57): warning C4100: 'visitor': unreferenced formal parameter (compiling source file C:\build\precommit_custom_windows\opencv\modules\gapi\test\infer\gapi_infer_ie_test.cpp) [C:\build\precommit_custom_windows\build\modules\gapi\opencv_test_gapi.vcxproj]
   572>C:\Program Files (x86)\IntelSWTools\openvino_2021.4.689\deployment_tools\ngraph\include\ngraph/op/acosh.hpp(37,57): warning C4100: 'visitor': unreferenced formal parameter (compiling source file C:\build\precommit_custom_windows\opencv\modules\gapi\test\infer\gapi_infer_ie_test.cpp) [C:\build\precommit_custom_windows\build\modules\gapi\opencv_test_gapi.vcxproj]
   572>C:\Program Files (x86)\IntelSWTools\openvino_2021.4.689\deployment_tools\ngraph\include\ngraph/op/asin.hpp(37,57): warning C4100: 'visitor': unreferenced formal parameter (compiling source file C:\build\precommit_custom_windows\opencv\modules\gapi\test\infer\gapi_infer_ie_test.cpp) [C:\build\precommit_custom_windows\build\modules\gapi\opencv_test_gapi.vcxproj]
   572>C:\Program Files (x86)\IntelSWTools\openvino_2021.4.689\deployment_tools\ngraph\include\ngraph/op/asinh.hpp(37,57): warning C4100: 'visitor': unreferenced formal parameter (compiling source file C:\build\precommit_custom_windows\opencv\modules\gapi\test\infer\gapi_infer_ie_test.cpp) [C:\build\precommit_custom_windows\build\modules\gapi\opencv_test_gapi.vcxproj]
   572>C:\Program Files (x86)\IntelSWTools\openvino_2021.4.689\deployment_tools\ngraph\include\ngraph/op/atan.hpp(38,57): warning C4100: 'visitor': unreferenced formal parameter (compiling source file C:\build\precommit_custom_windows\opencv\modules\gapi\test\infer\gapi_infer_ie_test.cpp) [C:\build\precommit_custom_windows\build\modules\gapi\opencv_test_gapi.vcxproj]
   572>C:\Program Files (x86)\IntelSWTools\openvino_2021.4.689\deployment_tools\ngraph\include\ngraph/op/atanh.hpp(37,57): warning C4100: 'visitor': unreferenced formal parameter (compiling source file C:\build\precommit_custom_windows\opencv\modules\gapi\test\infer\gapi_infer_ie_test.cpp) [C:\build\precommit_custom_windows\build\modules\gapi\opencv_test_gapi.vcxproj]
   572>C:\Program Files (x86)\IntelSWTools\openvino_2021.4.689\deployment_tools\ngraph\include\ngraph/op/ceiling.hpp(28,57): warning C4100: 'visitor': unreferenced formal parameter (compiling source file C:\build\precommit_custom_windows\opencv\modules\gapi\test\infer\gapi_infer_ie_test.cpp) [C:\build\precommit_custom_windows\build\modules\gapi\opencv_test_gapi.vcxproj]
   572>C:\Program Files (x86)\IntelSWTools\openvino_2021.4.689\deployment_tools\ngraph\include\ngraph/op/constant.hpp(177,79): warning C4100: 'inputs': unreferenced formal parameter (compiling source file C:\build\precommit_custom_windows\opencv\modules\gapi\test\infer\gapi_infer_ie_test.cpp) [C:\build\precommit_custom_windows\build\modules\gapi\opencv_test_gapi.vcxproj]
   572>C:\Program Files (x86)\IntelSWTools\openvino_2021.4.689\deployment_tools\ngraph\include\ngraph/op/constant.hpp(177,50): warning C4100: 'outputs': unreferenced formal parameter (compiling source file C:\build\precommit_custom_windows\opencv\modules\gapi\test\infer\gapi_infer_ie_test.cpp) [C:\build\precommit_custom_windows\build\modules\gapi\opencv_test_gapi.vcxproj]
   572>C:\Program Files (x86)\IntelSWTools\openvino_2021.4.689\deployment_tools\ngraph\include\ngraph/op/embedding_segments_sum.hpp(69,65): warning C4100: 'visitor': unreferenced formal parameter (compiling source file C:\build\precommit_custom_windows\opencv\modules\gapi\test\infer\gapi_infer_ie_test.cpp) [C:\build\precommit_custom_windows\build\modules\gapi\opencv_test_gapi.vcxproj]

Should I just suppress them for gapi_infer_ie_test.cpp

TolyaTalamanov · 2021-11-03T08:40:44Z

modules/gapi/CMakeLists.txt

@@ -207,6 +207,10 @@ if(OPENCV_GAPI_INF_ENGINE)
  ocv_target_link_libraries(${the_module} PRIVATE ${INF_ENGINE_TARGET})
 endif()

+if (HAVE_NGRAPH)
+    ocv_target_link_libraries(${the_module} PRIVATE ngraph::ngraph)


@alalek I noticed that InferenceEngine and ngraph are handled different way, do we need to make alias to ngraph::ngraph like NGRAPH_TARGET?

Does it make sense not to use InferenceEngine separately from ngraph by default?

separately

find_package(InferenceEngine) and find_package(ngraph).

Check DNN module integration.

I believe it's so similar to my implementation, isn't it? Except maybe this: https://github.com/opencv/opencv/blob/master/modules/dnn/CMakeLists.txt#L162 in DNN definitions set globally, in G-API we need this only for opencv_test_gapi, so far

Is this question closed now?

@alalek Is it OK?

TolyaTalamanov · 2021-11-03T08:42:31Z

PR based on this: #20918

alalek · 2021-11-03T09:03:07Z

modules/gapi/test/infer/gapi_infer_ie_test.cpp

@@ -22,6 +22,10 @@
 #include "backends/ie/util.hpp"
 #include "backends/ie/giebackend/giewrapper.hpp"

+#ifdef HAVE_NGRAPH
+#include <ngraph/ngraph.hpp>
+#endif


@alalek These warnings come from ngraph

572>C:\Program Files (x86)\IntelSWTools\openvino_2021.4.689\deployment_tools\ngraph\include\ngraph/op/abs.hpp(26,57): warning C4100: 'visitor': unreferenced formal parameter (compiling source file C:\build\precommit_custom_windows\opencv\modules\gapi\test\infer\gapi_infer_ie_test.cpp) [C:\build\precommit_custom_windows\build\modules\gapi\opencv_test_gapi.vcxproj]

Use this prolog code before ngraph include :

opencv/modules/core/src/hal_replacement.hpp

Lines 50 to 59 in 17234f8

#if defined(__clang__) // clang or MSVC clang

#pragma clang diagnostic push

#pragma clang diagnostic ignored "-Wunused-parameter"

#elif defined(_MSC_VER)

#pragma warning(push)

#pragma warning(disable : 4100)

#elif defined(__GNUC__)

#pragma GCC diagnostic push

#pragma GCC diagnostic ignored "-Wunused-parameter"

#endif

and similar prolog code (check referenced file).

/cc @ilya-lavrenov FYI (public headers should not emit warnings)

Added, works fine now, thanks!

@alalek it's fixed in latest master (checked by openvinotoolkit/openvino#8403 where I don't have to fix such issues)

Secondly, you can include IE headers as SYSTEM

as SYSTEM

Even on Windows and MSVC?

in latest master

Upcoming release is 2021.4 LTS update. So we still forced to use/validate that.

dmatveev · 2021-11-10T15:39:12Z

modules/gapi/CMakeLists.txt

@@ -207,6 +207,10 @@ if(OPENCV_GAPI_INF_ENGINE)
  ocv_target_link_libraries(${the_module} PRIVATE ${INF_ENGINE_TARGET})
 endif()

+if (HAVE_NGRAPH)
+    ocv_target_link_libraries(${the_module} PRIVATE ngraph::ngraph)


Is this question closed now?

modules/gapi/src/backends/ie/giebackend.cpp

dmatveev · 2021-11-10T15:47:07Z

modules/gapi/test/infer/gapi_infer_ie_test.cpp

@@ -22,6 +22,20 @@
 #include "backends/ie/util.hpp"
 #include "backends/ie/giebackend/giewrapper.hpp"

+#ifdef HAVE_NGRAPH


Should tests link with ngraph::ngraph too? I don't remember if it is the case for CMake.

That's a good point, but ngraph is enabled by using -DWITH_NGRAPH=ON and this flag isn't mandatory to use IE itself.
So if user just do cmake as same as usual: -DWITH_INF_ENGINE=ON ngraph won't be found.

@ilya-lavrenov Could you help, please? Do you know whether ngraph will be mandatory part of IE in upcoming OpenVINO 2.0 release?

dmatveev

LGTM but please close the linker question

modules/gapi/include/opencv2/gapi/infer/ie.hpp

modules/gapi/test/infer/gapi_infer_ie_test.cpp

mpashchenkov

LGTM

TolyaTalamanov · 2021-11-16T11:25:16Z

@alalek I believe it's ready to be merged

…ts-giebackend [G-API] Enable 2D inputs for GIEBackend * Enable 2D inputs * Fix typo

TolyaTalamanov added the category: g-api / gapi label Oct 22, 2021

TolyaTalamanov added this to the 4.5.5 milestone Oct 22, 2021

TolyaTalamanov requested review from rgarnov and mpashchenkov October 22, 2021 08:39

mpashchenkov reviewed Oct 22, 2021

View reviewed changes

TolyaTalamanov requested a review from mpashchenkov November 3, 2021 08:39

TolyaTalamanov commented Nov 3, 2021

View reviewed changes

alalek reviewed Nov 3, 2021

View reviewed changes

dmatveev self-assigned this Nov 10, 2021

dmatveev reviewed Nov 10, 2021

View reviewed changes

dmatveev approved these changes Nov 12, 2021

View reviewed changes

Enable 2D inputs

0d2531f

TolyaTalamanov force-pushed the at/enable-2d-inputs-giebackend branch from 494aa33 to 0d2531f Compare November 16, 2021 07:03

mpashchenkov reviewed Nov 16, 2021

View reviewed changes

modules/gapi/include/opencv2/gapi/infer/ie.hpp Outdated Show resolved Hide resolved

modules/gapi/test/infer/gapi_infer_ie_test.cpp Show resolved Hide resolved

Fix typo

d01c985

mpashchenkov approved these changes Nov 16, 2021

View reviewed changes

alalek merged commit c7df824 into opencv:4.x Nov 16, 2021

alalek mentioned this pull request Nov 26, 2021

gapi: fix build with MSVS2015 (with IE) #21134

Merged

alalek mentioned this pull request Dec 30, 2021

(5.x) Merge 4.x #21371

Merged

alalek mentioned this pull request Feb 22, 2022

(5.x) Merge 4.x #21651

Merged

a-sajjad72 pushed a commit to a-sajjad72/opencv that referenced this pull request Mar 30, 2023

Merge pull request opencv#20925 from TolyaTalamanov:at/enable-2d-inpu…

908f9b6

…ts-giebackend [G-API] Enable 2D inputs for GIEBackend * Enable 2D inputs * Fix typo

	#if defined(__clang__) // clang or MSVC clang
	#pragma clang diagnostic push
	#pragma clang diagnostic ignored "-Wunused-parameter"
	#elif defined(_MSC_VER)
	#pragma warning(push)
	#pragma warning(disable : 4100)
	#elif defined(__GNUC__)
	#pragma GCC diagnostic push
	#pragma GCC diagnostic ignored "-Wunused-parameter"
	#endif

Uh oh!

[G-API] Enable 2D inputs for GIEBackend #20925

[G-API] Enable 2D inputs for GIEBackend #20925

Uh oh!

Conversation

TolyaTalamanov commented Oct 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Readiness Checklist

Build configuration

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TolyaTalamanov commented Oct 26, 2021

Uh oh!

TolyaTalamanov commented Nov 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TolyaTalamanov commented Nov 3, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmatveev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mpashchenkov left a comment

Choose a reason for hiding this comment

Uh oh!

TolyaTalamanov commented Nov 16, 2021

Uh oh!

Uh oh!

TolyaTalamanov commented Oct 22, 2021 •

edited

Loading

TolyaTalamanov commented Nov 3, 2021 •

edited

Loading