CARVIEW |
Navigation Menu
-
-
Notifications
You must be signed in to change notification settings - Fork 56.2k
core(parallel): plugins support #19470
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
I have a problem with OpenMP backend built as a plugin (Ubuntu 18):
With gdb:
|
@@ -0,0 +1,60 @@ | |||
function(ocv_create_builtin_core_parallel_plugin name target) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This file and plugin_standalone.cmake are similar to videoio/cmake/plugin*.cmake
. Can they be unified somehow? Maybe moved to the root cmake/
directory?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mshabunin Thanks!
Added task to find workaround for cases with unloading crashes
message(STATUS "OpenMP detection requires CMake 3.9+") # OpenMP::OpenMP_CXX target | ||
endif() | ||
|
||
find_package(OpenMP) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
libraryRelease unload /build/lib/libopencv_core_parallel_openmp.so
Segmentation fault
LD_DEBUG=libs
dump is:
[ PASSED ] 1 test.
[ INFO:0] global /home/alalek/projects/opencv/dev/modules/core/src/utils/plugin_loader.impl.hpp (50) libraryRelease unload /home/alalek/projects/opencv/build/opencv/lib/libopencv_core_parallel_openmp.so
232271:
232271: calling fini: /home/alalek/projects/opencv/build/opencv/lib/libopencv_core_parallel_openmp.so [0]
232271:
232271:
232271: calling fini: /lib64/libgomp.so.1 [0]
232271:
Segmentation fault (core dumped)
libgomp.so.1
is de-initialized (finit
call), but its worker threads are still alive (gdb shows that).
So,
- this is a bug of
libgomp.so.1
(no idea why worker threads are not terminated during unload stage) - we should not force OpenMP thread pool termination from OpenCV side, because application can use OpenMP directly
- some workaround is required (probably don't unload parallel plugins)
The similar issue can be observed with TBB
Issue is sporadic with TBB (not 100% reproducible).
GDB:
[ RUN ] Core_Rand.parallel_for_stable_results
[New Thread 0x7fffd46b7640 (LWP 232758)]
[New Thread 0x7fffa8ffa640 (LWP 232760)]
[New Thread 0x7fffb07f8640 (LWP 232759)]
[ OK ] Core_Rand.parallel_for_stable_results (174 ms)
...
[ INFO:0] global /home/alalek/projects/opencv/dev/modules/core/src/utils/plugin_loader.impl.hpp (50) libraryRelease unload /home/alalek/projects/opencv/build/opencv/lib/libopencv_core_parallel_onetbb.so
Thread 34 "opencv_test_cor" received signal SIGSEGV, Segmentation fault.
(gdb) info threads
...
33 Thread 0x7fffd46b7640 (LWP 232758) "opencv_test_cor" 0x00007ffff47ba55d in syscall () from /lib64/libc.so.6
* 34 Thread 0x7fffa8ffa640 (LWP 232760) "opencv_test_cor" 0x00007fffe749ba32 in ?? ()
35 Thread 0x7fffb07f8640 (LWP 232759) "opencv_test_cor" 0x00007ffff47ba55d in syscall () from /lib64/libc.so.6
(232760
is created during the test run)
LD_DEBUG dump:
[ INFO:0] global /home/alalek/projects/opencv/dev/modules/core/src/utils/plugin_loader.impl.hpp (50) libraryRelease unload /home/alalek/projects/opencv/build/opencv/lib/libopencv_core_parallel_onetbb.so
232863:
232863: calling fini: /home/alalek/projects/opencv/build/opencv/lib/libopencv_core_parallel_onetbb.so [0]
232863:
232863:
232863: calling fini: /lib64/libtbb.so.2 [0]
232863:
232863:
232863: calling fini: /lib64/libirml.so.1 [0]
232863:
Segmentation fault (core dumped)
TODO: Will try to check some workarounds.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added code to prevent automatic unloading of parallel backend library (including underlying libraries).
There is no huge impact, because plugins are kept loaded until opencv_core unloading anyway.
# FIXIT: stop using PARENT_SCOPE in dependencies | ||
if(PROJECT_NAME STREQUAL "OpenCV") | ||
macro(add_backend backend_id cond_var) | ||
if(${cond_var}) | ||
include("${CMAKE_CURRENT_LIST_DIR}/detect_${backend_id}.cmake") | ||
endif() | ||
endmacro() | ||
else() | ||
function(add_backend backend_id cond_var) | ||
if(${cond_var}) | ||
include("${CMAKE_CURRENT_LIST_DIR}/detect_${backend_id}.cmake") | ||
endif() | ||
endfunction() | ||
endif() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is inconsistency in scope of dependencies.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps we could switch to target checks instead of HAVE_
variables. Additional information can be stored in custom target properties.
relates #19365
Usage examples are on GitHub PR page.
Hardening option:
Control environment variables:
OPENCV_PARALLEL_PRIORITY_TBB=0
- disableOPENCV_PARALLEL_PRIORITY_LIST=TBB
- preferred backends namesOPENCV_PARALLEL_BACKEND=TBB
- forced backend nameTasks:
setParallelForBackend(const std::string& backendName, ...)
)ParallelForAPI
HAVE_OPENMP
(create define from CMake)OPENCV_PLUGIN_MODULE_PREFIX
plugin_standalone.cmake
videoio/misc/build_plugins.sh