| CARVIEW |
Navigation Menu
-
-
Notifications
You must be signed in to change notification settings - Fork 56.2k
cv::UMat output/input in VideoCapture/VideoWriter (data stays in GPU memory) #19755
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
UMat output/input works in my local testing on VAAPI decoders/encoders (Ubuntu 20.04), D3D11 decoders (Windows 10), MFX decoders/encoders (Ubuntu 20.04 and Windows 10).
|
|
|
||
| namespace cv { namespace ocl { | ||
|
|
||
| //! @addtogroup core_opencl | ||
| //! @{ | ||
|
|
||
| CV_EXPORTS_W bool haveOpenCL(); | ||
| CV_EXPORTS_W bool useOpenCL(); | ||
| CV_EXPORTS_W bool useOpenCL(bool init = true); // TODO: can we change function signature? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@alalek Should be another function, as we can't change signature of existent function?
|
@asmorkalov, this PR has 4 failing tests for vp9 decoding on Windows 10, is it same system setup and probably same reason as issue 19694? |
|
@mikhail-nikolskiy Yes. |
|
jenkins cn please retry a build |
1 similar comment
|
jenkins cn please retry a build |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for great contribution!
I'm sorry for huge review delay (we were busy with release activities).
This is the first review iteration.
| CV_LOG_INFO(NULL, "FFMPEG: Created video acceleration context (av_hwdevice_ctx_create) for " << hw_child_name << " on device " << device_name); | ||
| // if OpenCL context not created yet, create it with binding to video acceleration context | ||
| if (ocl::useOpenCL(false)) { | ||
| if (ocl_context.empty()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ocl::useOpenCL(false)
I believe it is enough to check ocl_context.empty() only.
So adding new "useOpenCL" behavior is not necessary.
Current opencv_test_videoio tests stuck with these messages (due to pre-initialized OpenCL device):
[ RUN ] videoio/videocapture_acceleration.read/55, where GetParam() = (sample_322x242_15frames.yuv420p.libx264.mp4, FFMPEG, VAAPI, true)
...
[ INFO:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (268) hw_check_device FFMPEG: Using vaapi video acceleration on device: Intel iHD driver for Intel(R) Gen Graphics - 20.3.0 ()
[ INFO:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (559) hw_create_device FFMPEG: Created video acceleration context (av_hwdevice_ctx_create) for vaapi on device 'default'
[ INFO:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (571) hw_create_device FFMPEG: Can't bind vaapi video acceleration context to already created OpenCL context
BTW, I would suggest to add explicit "VideoCapture" open parameter which forces creation of new of OpenCL context.
| return AV_HWDEVICE_TYPE_NONE; | ||
| } | ||
|
|
||
| static void hw_init_opencl(AVBufferRef* ctx) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is an regression for case with disabled OpenCL subsystem in OpenCV through OPENCV_OPENCL_RUNTIME=disabled. Both Mat / UMat test cases don't work (skipped) with messages:
[OPENCV:FFMPEG:40] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 20.4.5 (74e2f111).
[OPENCV:FFMPEG:40] Driver not found in known nonstandard list, using standard behaviour.
[ INFO:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (268) hw_check_device FFMPEG: Using vaapi video acceleration on device: Intel iHD driver for Intel(R) Gen Graphics - 20.4.5 (74e2f111)
[ INFO:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (559) hw_create_device FFMPEG: Created video acceleration context (av_hwdevice_ctx_create) for vaapi on device 'default'
[ERROR:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap.cpp (160) open VIDEOIO(FFMPEG): raised OpenCV exception:
OpenCV(4.5.2-dev) /home/alalek/projects/opencv/dev/modules/core/src/opencl/runtime/opencl_core.cpp:378: error: (-220:Unknown error code -220) OpenCL function is not available: [clGetPlatformIDs] in function 'opencl_check_fn'
[ SKIP ] FFMPEG VideoCapture on sample_322x242_15frames.yuv420p.libx264.mp4 not supported with HW acceleration, skipping
| @@ -277,6 +279,8 @@ class CV_EXPORTS Context | |||
| /** @returns cl_context value */ | |||
| void* ptr() const; | |||
|
|
|||
| /** @returns Property specified on context creation */ | |||
| void* getProperty(long propertyId) const; | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As we want to work with OpenCL enums lets use more specific method name getOpenCLContextProperty() and add note to doc string about returned values and passed parameter (extract properties through clGetContextInfo(CL_CONTEXT_PROPERTIES) call)
modules/core/src/ocl.cpp
Outdated
| if (CL_SUCCESS == clGetContextInfo(p->handle, CL_CONTEXT_PROPERTIES, 0, NULL, &size)) { | ||
| std::vector<cl_context_properties> prop(size / sizeof(cl_context_properties)); | ||
| if (CL_SUCCESS == clGetContextInfo(p->handle, CL_CONTEXT_PROPERTIES, size, prop.data(), NULL)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a question: can we cache this properties array? or its contents can be changed in runtime
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OpenCL doesn't allow to change properties (I would really like to have such ability - change media context property after context creation;)
But properties cached by OpenCL user-mode library, so these these two calls should work very fast.
Is it ok, or better cache on OpenCV side?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OpenCL doesn't allow to change properties (I would really like to have such ability - change media context property after context creation;)
But properties cached by OpenCL user-mode library, so these these two calls should work very fast.
Is it ok, or better cache on OpenCV side?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for information. Caching support here is low priority for now.
| @@ -290,6 +294,21 @@ class CV_EXPORTS Context | |||
|
|
|||
| void release(); | |||
|
|
|||
| class CV_EXPORTS UserContext { | |||
| public: | |||
| virtual ~UserContext() {}; | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It makes sense to put destructor implementation in .cpp file.
Otherwise "full header" classes may lead to RTTI issues (e.g. with clang - it creates independent clones in each module)
cmake/templates/cvconfig.h.in
Outdated
| @@ -149,6 +149,9 @@ | |||
| /* Intel VA-API/OpenCL */ | |||
| #cmakedefine HAVE_VA_INTEL | |||
|
|
|||
| /* Intel MediaSDK/oneVPL */ | |||
| #cmakedefine HAVE_MFX | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TODO(alalek): check how to move this to videoio plugins
|
|
||
| // check that current OpenCL context initilized on same media device as codec | ||
| //if (!hw_check_opencl_context(video_st->codec->hw_device_ctx)) | ||
| // return false; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps robust check should check UMat's OpenCL context.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would require code like below?
CV_Assert(u.u && u.u->currAllocator == ocl::getOpenCLAllocator());
CV_Assert(std::static_pointer_cast<ocl::Context>(u.u->allocatorContext)->ptr() == ocl_context.getContext().ptr());
I believe SetKernelArgument will throw exception if wrong context, and this code under try/catch - is it ok or better check context?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure that OpenCL CTS has checks for "wrong context" cases.
I would prefer to raise OpenCV exception instead of crash in OpenCL runtime internals.
|
jenkins cn please retry a build |
|
Thanks for review! Regarding "explicit VideoCapture open parameter which forces creation of new of OpenCL context". or Is it beneficial? |
Looks like "git merge" has been done in the wrong direction. |
Looks like using some handles here is over-complicated for users. I would prefer to have ON/OFF property:
|
mikhail-nikolskiy
force-pushed
the
ffmpeg-umat
branch
from
4b015ec to
1d9e6e6
Compare
I renamed property to |
|
Excluded "merge" and "revert" commits from the PR (no changes in the code). |
…L GPU+CPU platform
|
Thanks for removing mess with revert commits! |
|
On Windows 10 builder I saw test failures on test cases with use_umat=true parameter, but can't find any log messages from test execution (exceptions, errors, etc). Is it possible to get debug log? It would help a lot as can't reproduce on local iGPU and dGPU devices. |
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for contribution!
This patch works well in these configurations on my side (validated):
- Linux + VAAPI + Intel iGPU
- Windows (with FFmpeg build) + D3D11 + Intel iGPU
Remaining CI "errors" are specific to OpenCV / build configurations. I will take a look how to resolve them (added list to PR's description)
|
That's interesting stuff for sure! @mikhail-nikolskiy would you mind adding support for |
I checked RMat::access, it seems to be functionally equal to UMat::getMat, so wrapper from UMat to RMat could be implemented in generic way, not specifically in videoio module? |
- plugins don't support OpenCL/UMat yet - change hanling of *_USE_OPENCL flag
| @@ -80,7 +113,7 @@ std::string getDecoderConfiguration(VideoAccelerationType va_type, AVDictionary | |||
| case VIDEO_ACCELERATION_ANY: return "vaapi.iHD"; | |||
| case VIDEO_ACCELERATION_D3D11: return ""; | |||
| case VIDEO_ACCELERATION_VAAPI: return "vaapi.iHD"; | |||
| case VIDEO_ACCELERATION_MFX: return ""; | |||
| case VIDEO_ACCELERATION_MFX: return "qsv.iHD"; | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is a regression on Linux machine for the test sample_322x242_15frames.yuv420p.libvpx-vp9.mp4, FFMPEG, MFX, false
Test logs are below (OPENCV_FFMPEG_DEBUG=1 OPENCV_LOG_LEVEL=DEBUG):
With patch
[ RUN ] videoio/videocapture_acceleration.read/84, where GetParam() = (sample_322x242_15frames.yuv420p.libvpx-vp9.mp4, FFMPEG, MFX, false)
[OPENCV:FFMPEG:48] Opening '/home/alalek/projects/opencv/extra/testdata/highgui/video/sample_322x242_15frames.yuv420p.libvpx-vp9.mp4' for reading
[OPENCV:FFMPEG:48] Setting default whitelist 'file,crypto,data'
[OPENCV:FFMPEG:56] Probing mov,mp4,m4a,3gp,3g2,mj2 score:100 size:2048
[OPENCV:FFMPEG:56] Probing mp3 score:1 size:2048
[OPENCV:FFMPEG:48] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[OPENCV:FFMPEG:56] type:'ftyp' parent:'root' sz: 28 8 31057
[OPENCV:FFMPEG:48] ISO: File Type Major Brand: isom
[OPENCV:FFMPEG:56] type:'free' parent:'root' sz: 8 36 31057
[OPENCV:FFMPEG:56] type:'mdat' parent:'root' sz: 30210 44 31057
[OPENCV:FFMPEG:56] type:'moov' parent:'root' sz: 811 30254 31057
[OPENCV:FFMPEG:56] type:'mvhd' parent:'moov' sz: 108 8 803
[OPENCV:FFMPEG:56] time scale = 1000
[OPENCV:FFMPEG:56] type:'trak' parent:'moov' sz: 597 116 803
[OPENCV:FFMPEG:56] type:'tkhd' parent:'trak' sz: 92 8 589
[OPENCV:FFMPEG:56] type:'edts' parent:'trak' sz: 36 100 589
[OPENCV:FFMPEG:56] type:'elst' parent:'edts' sz: 28 8 28
[OPENCV:FFMPEG:56] track[0].edit_count = 1
[OPENCV:FFMPEG:56] duration=600 time=0 rate=1.000000
[OPENCV:FFMPEG:56] type:'mdia' parent:'trak' sz: 461 136 589
[OPENCV:FFMPEG:56] type:'mdhd' parent:'mdia' sz: 32 8 453
[OPENCV:FFMPEG:56] type:'hdlr' parent:'mdia' sz: 45 40 453
[OPENCV:FFMPEG:56] ctype=[0][0][0][0]
[OPENCV:FFMPEG:56] stype=vide
[OPENCV:FFMPEG:56] type:'minf' parent:'mdia' sz: 376 85 453
[OPENCV:FFMPEG:56] type:'vmhd' parent:'minf' sz: 20 8 368
[OPENCV:FFMPEG:56] type:'dinf' parent:'minf' sz: 36 28 368
[OPENCV:FFMPEG:56] type:'dref' parent:'dinf' sz: 28 8 28
[OPENCV:FFMPEG:48] Unknown dref type 0x206c7275 size 12
[OPENCV:FFMPEG:56] type:'stbl' parent:'minf' sz: 312 64 368
[OPENCV:FFMPEG:56] type:'stsd' parent:'stbl' sz: 132 8 304
[OPENCV:FFMPEG:56] size=116 4CC=vp09 codec_type=0
[OPENCV:FFMPEG:56] type:'vpcC' parent:'stsd' sz: 20 8 30
[OPENCV:FFMPEG:56] type:'fiel' parent:'stsd' sz: 10 28 30
[OPENCV:FFMPEG:56] type:'stts' parent:'stbl' sz: 24 140 304
[OPENCV:FFMPEG:56] track[0].stts.entries = 1
[OPENCV:FFMPEG:56] sample_count=15, sample_duration=512
[OPENCV:FFMPEG:56] type:'stss' parent:'stbl' sz: 20 164 304
[OPENCV:FFMPEG:56] keyframe_count = 1
[OPENCV:FFMPEG:56] type:'stsc' parent:'stbl' sz: 28 184 304
[OPENCV:FFMPEG:56] track[0].stsc.entries = 1
[OPENCV:FFMPEG:56] type:'stsz' parent:'stbl' sz: 80 212 304
[OPENCV:FFMPEG:56] sample_size = 0 sample_count = 15
[OPENCV:FFMPEG:56] type:'stco' parent:'stbl' sz: 20 292 304
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 0, offset 2c, dts 0, size 5068, distance 0, keyframe 1
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 1, offset 13f8, dts 512, size 1308, distance 1, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 2, offset 1914, dts 1024, size 1265, distance 2, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 3, offset 1e05, dts 1536, size 1507, distance 3, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 4, offset 23e8, dts 2048, size 2089, distance 4, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 5, offset 2c11, dts 2560, size 1956, distance 5, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 6, offset 33b5, dts 3072, size 1795, distance 6, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 7, offset 3ab8, dts 3584, size 2204, distance 7, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 8, offset 4354, dts 4096, size 2267, distance 8, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 9, offset 4c2f, dts 4608, size 1315, distance 9, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 10, offset 5152, dts 5120, size 5251, distance 10, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 11, offset 65d5, dts 5632, size 1670, distance 11, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 12, offset 6c5b, dts 6144, size 632, distance 12, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 13, offset 6ed3, dts 6656, size 612, distance 13, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 14, offset 7137, dts 7168, size 1263, distance 14, keyframe 0
[OPENCV:FFMPEG:48] Processing st: 0, edit list 0 - media time: 0, duration: 7680
[OPENCV:FFMPEG:56] type:'udta' parent:'moov' sz: 98 713 803
[OPENCV:FFMPEG:56] type:'meta' parent:'udta' sz: 90 8 90
[OPENCV:FFMPEG:56] type:'hdlr' parent:'meta' sz: 33 8 78
[OPENCV:FFMPEG:56] ctype=[0][0][0][0]
[OPENCV:FFMPEG:56] stype=mdir
[OPENCV:FFMPEG:56] type:'ilst' parent:'meta' sz: 45 41 78
[OPENCV:FFMPEG:56] type:'[169]too' parent:'ilst' sz: 37 8 37
[OPENCV:FFMPEG:56] on_parse_exit_offset=31057
[OPENCV:FFMPEG:48] Before avformat_find_stream_info() pos: 31057 bytes read:31057 seeks:0 nb_streams:1
[OPENCV:FFMPEG:56] stream 0, sample 0, dts 0
[OPENCV:FFMPEG:48] Format yuv420p chosen by get_format().
[OPENCV:FFMPEG:48] All info found
[OPENCV:FFMPEG:56] stream 0: start_time: 0 duration: 0.6
[OPENCV:FFMPEG:56] format: start_time: 0 duration: 0.6 (estimate from stream) bitrate=414 kb/s
[OPENCV:FFMPEG:48] After avformat_find_stream_info() pos: 5112 bytes read:31057 seeks:0 frames:1
[DEBUG:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_impl.hpp (1022) open FFMPEG: stream[0] is video stream with codecID=167 width=322 height=242
[DEBUG:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (927) HWAccelIterator FFMPEG: allowed acceleration types (mfx): 'qsv.iHD'
[DEBUG:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (945) HWAccelIterator FFMPEG: disabled codecs: 'av1.vaapi,av1_qsv,vp8.vaapi,vp8_qsv'
[DEBUG:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_impl.hpp (1054) open FFMPEG: trying to configure H/W acceleration: 'qsv.iHD'
[OPENCV:FFMPEG:40] Trying to use DRM render node for device 0.
[OPENCV:FFMPEG:40] libva: VA-API version 1.9.0
libva: Trying to open /usr/lib64/dri/iHD_drv_video.so
libva: Found init function __vaDriverInit_1_9
libva: va_openDriver() returns 0
Initialised VAAPI connection: version 1.9
[OPENCV:FFMPEG:48] Format 0x41524742 -> bgra.
[OPENCV:FFMPEG:48] Format 0x42475241 -> argb.
[OPENCV:FFMPEG:48] Format 0x41424752 -> rgba.
[OPENCV:FFMPEG:48] Format 0x52474241 -> abgr.
[OPENCV:FFMPEG:48] Format 0x58524742 -> bgr0.
[OPENCV:FFMPEG:48] Format 0x42475258 -> 0rgb.
[OPENCV:FFMPEG:48] Format 0x58424752 -> rgb0.
[OPENCV:FFMPEG:48] Format 0x52474258 -> 0bgr.
[OPENCV:FFMPEG:48] Format 0x30335241 -> unknown.
[OPENCV:FFMPEG:48] Format 0x30334241 -> unknown.
[OPENCV:FFMPEG:48] Format 0x30335258 -> unknown.
[OPENCV:FFMPEG:48] Format 0x30334258 -> unknown.
[OPENCV:FFMPEG:48] Format 0x36314752 -> unknown.
[OPENCV:FFMPEG:48] Format 0x50424752 -> unknown.
[OPENCV:FFMPEG:48] Format 0x50524742 -> unknown.
[OPENCV:FFMPEG:48] Format 0x56555941 -> unknown.
[OPENCV:FFMPEG:48] Format 0x30303859 -> gray.
[OPENCV:FFMPEG:48] Format 0x3231564e -> nv12.
[OPENCV:FFMPEG:48] Format 0x3132564e -> unknown.
[OPENCV:FFMPEG:48] Format 0x32595559 -> yuyv422.
[OPENCV:FFMPEG:48] Format 0x59565955 -> uyvy422.
[OPENCV:FFMPEG:48] Format 0x32315659 -> yuv420p.
[OPENCV:FFMPEG:48] Format 0x30323449 -> yuv420p.
[OPENCV:FFMPEG:48] Format 0x50313134 -> yuv411p.
[OPENCV:FFMPEG:48] Format 0x48323234 -> yuv422p.
[OPENCV:FFMPEG:48] Format 0x56323234 -> yuv440p.
[OPENCV:FFMPEG:48] Format 0x50343434 -> yuv444p.
[OPENCV:FFMPEG:48] Format 0x33434d49 -> unknown.
[OPENCV:FFMPEG:48] Format 0x30313050 -> p010le.
[OPENCV:FFMPEG:40] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 20.3.0 ().
[OPENCV:FFMPEG:40] Driver not found in known nonstandard list, using standard behaviour.
[ INFO:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (272) hw_check_device FFMPEG: Using MFX video acceleration on device: Intel iHD driver for Intel(R) Gen Graphics - 20.3.0 ()
[ INFO:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (562) hw_create_device FFMPEG: Created video acceleration context (av_hwdevice_ctx_create) for vaapi on device 'default'
[ INFO:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (579) hw_create_device FFMPEG: Can't bind vaapi video acceleration context to already created OpenCL context
[OPENCV:FFMPEG:40] Initialize MFX session: API version is 1.34, implementation version is 1.34
[ INFO:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (305) hw_create_derived_context FFMPEG: Created derived video acceleration context (av_hwdevice_ctx_create_derived) for qsv
VideoCapture FFMPEG:MFX
[DEBUG:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (618) hw_create_frames FFMPEG: avcodec_get_hw_frames_parameters() call failed: -2
[OPENCV:FFMPEG:48] Created surface 0.
[OPENCV:FFMPEG:48] Direct mapping possible.
[OPENCV:FFMPEG:48] Created surface 0x1.
[OPENCV:FFMPEG:48] Created surface 0x2.
[OPENCV:FFMPEG:48] Created surface 0x3.
[OPENCV:FFMPEG:48] Created surface 0x4.
[OPENCV:FFMPEG:48] Created surface 0x5.
[OPENCV:FFMPEG:48] Created surface 0x6.
[OPENCV:FFMPEG:48] Created surface 0x7.
[OPENCV:FFMPEG:48] Created surface 0x8.
[OPENCV:FFMPEG:48] Created surface 0x9.
[OPENCV:FFMPEG:48] Created surface 0xa.
[OPENCV:FFMPEG:48] Created surface 0xb.
[OPENCV:FFMPEG:48] Created surface 0xc.
[OPENCV:FFMPEG:48] Created surface 0xd.
[OPENCV:FFMPEG:48] Created surface 0xe.
[OPENCV:FFMPEG:48] Created surface 0xf.
[OPENCV:FFMPEG:48] Created surface 0x10.
[OPENCV:FFMPEG:48] Created surface 0x11.
[OPENCV:FFMPEG:48] Created surface 0x12.
[OPENCV:FFMPEG:48] Created surface 0x13.
[OPENCV:FFMPEG:48] Created surface 0x14.
[OPENCV:FFMPEG:48] Created surface 0x15.
[OPENCV:FFMPEG:48] Created surface 0x16.
[OPENCV:FFMPEG:48] Created surface 0x17.
[OPENCV:FFMPEG:48] Created surface 0x18.
[OPENCV:FFMPEG:48] Created surface 0x19.
[OPENCV:FFMPEG:48] Created surface 0x1a.
[OPENCV:FFMPEG:48] Created surface 0x1b.
[OPENCV:FFMPEG:48] Created surface 0x1c.
[OPENCV:FFMPEG:48] Created surface 0x1d.
[OPENCV:FFMPEG:48] Created surface 0x1e.
[OPENCV:FFMPEG:48] Created surface 0x1f.
[OPENCV:FFMPEG:48] Format qsv chosen by get_format().
[OPENCV:FFMPEG:40] Decoder: output is video memory surface
[DEBUG:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (618) hw_create_frames FFMPEG: avcodec_get_hw_frames_parameters() call failed: -2
[OPENCV:FFMPEG:48] Created surface 0x20.
[OPENCV:FFMPEG:48] Direct mapping possible.
[OPENCV:FFMPEG:48] Created surface 0x21.
[OPENCV:FFMPEG:48] Created surface 0x22.
[OPENCV:FFMPEG:48] Created surface 0x23.
[OPENCV:FFMPEG:48] Created surface 0x24.
[OPENCV:FFMPEG:48] Created surface 0x25.
[OPENCV:FFMPEG:48] Created surface 0x26.
[OPENCV:FFMPEG:48] Created surface 0x27.
[OPENCV:FFMPEG:48] Created surface 0x28.
[OPENCV:FFMPEG:48] Created surface 0x29.
[OPENCV:FFMPEG:48] Created surface 0x2a.
[OPENCV:FFMPEG:48] Created surface 0x2b.
[OPENCV:FFMPEG:48] Created surface 0x2c.
[OPENCV:FFMPEG:48] Created surface 0x2d.
[OPENCV:FFMPEG:48] Created surface 0x2e.
[OPENCV:FFMPEG:48] Created surface 0x2f.
[OPENCV:FFMPEG:48] Created surface 0x30.
[OPENCV:FFMPEG:48] Created surface 0x31.
[OPENCV:FFMPEG:48] Created surface 0x32.
[OPENCV:FFMPEG:48] Created surface 0x33.
[OPENCV:FFMPEG:48] Created surface 0x34.
[OPENCV:FFMPEG:48] Created surface 0x35.
[OPENCV:FFMPEG:48] Created surface 0x36.
[OPENCV:FFMPEG:48] Created surface 0x37.
[OPENCV:FFMPEG:48] Created surface 0x38.
[OPENCV:FFMPEG:48] Created surface 0x39.
[OPENCV:FFMPEG:48] Created surface 0x3a.
[OPENCV:FFMPEG:48] Created surface 0x3b.
[OPENCV:FFMPEG:48] Created surface 0x3c.
[OPENCV:FFMPEG:48] Created surface 0x3d.
[OPENCV:FFMPEG:48] Created surface 0x3e.
[OPENCV:FFMPEG:48] Created surface 0x3f.
[OPENCV:FFMPEG:48] Format qsv chosen by get_format().
[OPENCV:FFMPEG:40] Decoder: output is video memory surface
[OPENCV:FFMPEG:16] Error initializing the MFX video decoder: unsupported (-3)
/home/alalek/projects/opencv/dev/modules/videoio/test/test_video_io.cpp:743: Failure
Value of: hw_reader.read(frame)
Actual: false
Expected: true
Google Test trace:
/home/alalek/projects/opencv/dev/modules/videoio/test/test_video_io.cpp:732: frame=0
/home/alalek/projects/opencv/dev/modules/videoio/test/test_video_io.cpp:745: Failure
Value of: frame.empty()
Actual: true
Expected: false
Google Test trace:
/home/alalek/projects/opencv/dev/modules/videoio/test/test_video_io.cpp:732: frame=0
[OPENCV:FFMPEG:40] Statistics: 31057 bytes read, 0 seeks
[ FAILED ] videoio/videocapture_acceleration.read/84, where GetParam() = (sample_322x242_15frames.yuv420p.libvpx-vp9.mp4, FFMPEG, MFX, false) (6 ms)
Master branch
[ RUN ] videoio/videocapture_acceleration.read/84, where GetParam() = (sample_322x242_15frames.yuv420p.libvpx-vp9.mp4, FFMPEG, MFX, false)
[OPENCV:FFMPEG:48] Opening '/home/alalek/projects/opencv/extra/testdata/highgui/video/sample_322x242_15frames.yuv420p.libvpx-vp9.mp4' for reading
[OPENCV:FFMPEG:48] Setting default whitelist 'file,crypto,data'
[OPENCV:FFMPEG:56] Probing mov,mp4,m4a,3gp,3g2,mj2 score:100 size:2048
[OPENCV:FFMPEG:56] Probing mp3 score:1 size:2048
[OPENCV:FFMPEG:48] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[OPENCV:FFMPEG:56] type:'ftyp' parent:'root' sz: 28 8 31057
[OPENCV:FFMPEG:48] ISO: File Type Major Brand: isom
[OPENCV:FFMPEG:56] type:'free' parent:'root' sz: 8 36 31057
[OPENCV:FFMPEG:56] type:'mdat' parent:'root' sz: 30210 44 31057
[OPENCV:FFMPEG:56] type:'moov' parent:'root' sz: 811 30254 31057
[OPENCV:FFMPEG:56] type:'mvhd' parent:'moov' sz: 108 8 803
[OPENCV:FFMPEG:56] time scale = 1000
[OPENCV:FFMPEG:56] type:'trak' parent:'moov' sz: 597 116 803
[OPENCV:FFMPEG:56] type:'tkhd' parent:'trak' sz: 92 8 589
[OPENCV:FFMPEG:56] type:'edts' parent:'trak' sz: 36 100 589
[OPENCV:FFMPEG:56] type:'elst' parent:'edts' sz: 28 8 28
[OPENCV:FFMPEG:56] track[0].edit_count = 1
[OPENCV:FFMPEG:56] duration=600 time=0 rate=1.000000
[OPENCV:FFMPEG:56] type:'mdia' parent:'trak' sz: 461 136 589
[OPENCV:FFMPEG:56] type:'mdhd' parent:'mdia' sz: 32 8 453
[OPENCV:FFMPEG:56] type:'hdlr' parent:'mdia' sz: 45 40 453
[OPENCV:FFMPEG:56] ctype=[0][0][0][0]
[OPENCV:FFMPEG:56] stype=vide
[OPENCV:FFMPEG:56] type:'minf' parent:'mdia' sz: 376 85 453
[OPENCV:FFMPEG:56] type:'vmhd' parent:'minf' sz: 20 8 368
[OPENCV:FFMPEG:56] type:'dinf' parent:'minf' sz: 36 28 368
[OPENCV:FFMPEG:56] type:'dref' parent:'dinf' sz: 28 8 28
[OPENCV:FFMPEG:48] Unknown dref type 0x206c7275 size 12
[OPENCV:FFMPEG:56] type:'stbl' parent:'minf' sz: 312 64 368
[OPENCV:FFMPEG:56] type:'stsd' parent:'stbl' sz: 132 8 304
[OPENCV:FFMPEG:56] size=116 4CC=vp09 codec_type=0
[OPENCV:FFMPEG:56] type:'vpcC' parent:'stsd' sz: 20 8 30
[OPENCV:FFMPEG:56] type:'fiel' parent:'stsd' sz: 10 28 30
[OPENCV:FFMPEG:56] type:'stts' parent:'stbl' sz: 24 140 304
[OPENCV:FFMPEG:56] track[0].stts.entries = 1
[OPENCV:FFMPEG:56] sample_count=15, sample_duration=512
[OPENCV:FFMPEG:56] type:'stss' parent:'stbl' sz: 20 164 304
[OPENCV:FFMPEG:56] keyframe_count = 1
[OPENCV:FFMPEG:56] type:'stsc' parent:'stbl' sz: 28 184 304
[OPENCV:FFMPEG:56] track[0].stsc.entries = 1
[OPENCV:FFMPEG:56] type:'stsz' parent:'stbl' sz: 80 212 304
[OPENCV:FFMPEG:56] sample_size = 0 sample_count = 15
[OPENCV:FFMPEG:56] type:'stco' parent:'stbl' sz: 20 292 304
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 0, offset 2c, dts 0, size 5068, distance 0, keyframe 1
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 1, offset 13f8, dts 512, size 1308, distance 1, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 2, offset 1914, dts 1024, size 1265, distance 2, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 3, offset 1e05, dts 1536, size 1507, distance 3, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 4, offset 23e8, dts 2048, size 2089, distance 4, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 5, offset 2c11, dts 2560, size 1956, distance 5, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 6, offset 33b5, dts 3072, size 1795, distance 6, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 7, offset 3ab8, dts 3584, size 2204, distance 7, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 8, offset 4354, dts 4096, size 2267, distance 8, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 9, offset 4c2f, dts 4608, size 1315, distance 9, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 10, offset 5152, dts 5120, size 5251, distance 10, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 11, offset 65d5, dts 5632, size 1670, distance 11, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 12, offset 6c5b, dts 6144, size 632, distance 12, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 13, offset 6ed3, dts 6656, size 612, distance 13, keyframe 0
[OPENCV:FFMPEG:56] AVIndex stream 0, sample 14, offset 7137, dts 7168, size 1263, distance 14, keyframe 0
[OPENCV:FFMPEG:48] Processing st: 0, edit list 0 - media time: 0, duration: 7680
[OPENCV:FFMPEG:56] type:'udta' parent:'moov' sz: 98 713 803
[OPENCV:FFMPEG:56] type:'meta' parent:'udta' sz: 90 8 90
[OPENCV:FFMPEG:56] type:'hdlr' parent:'meta' sz: 33 8 78
[OPENCV:FFMPEG:56] ctype=[0][0][0][0]
[OPENCV:FFMPEG:56] stype=mdir
[OPENCV:FFMPEG:56] type:'ilst' parent:'meta' sz: 45 41 78
[OPENCV:FFMPEG:56] type:'[169]too' parent:'ilst' sz: 37 8 37
[OPENCV:FFMPEG:56] on_parse_exit_offset=31057
[OPENCV:FFMPEG:48] Before avformat_find_stream_info() pos: 31057 bytes read:31057 seeks:0 nb_streams:1
[OPENCV:FFMPEG:56] stream 0, sample 0, dts 0
[OPENCV:FFMPEG:48] Format yuv420p chosen by get_format().
[OPENCV:FFMPEG:48] All info found
[OPENCV:FFMPEG:56] stream 0: start_time: 0 duration: 0.6
[OPENCV:FFMPEG:56] format: start_time: 0 duration: 0.6 (estimate from stream) bitrate=414 kb/s
[OPENCV:FFMPEG:48] After avformat_find_stream_info() pos: 5112 bytes read:31057 seeks:0 frames:1
[DEBUG:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_impl.hpp (1022) open FFMPEG: stream[0] is video stream with codecID=167 width=322 height=242
[DEBUG:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (927) HWAccelIterator FFMPEG: allowed acceleration types (mfx): 'qsv.iHD'
[DEBUG:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (945) HWAccelIterator FFMPEG: disabled codecs: 'av1.vaapi,av1_qsv,vp8.vaapi,vp8_qsv'
[DEBUG:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_impl.hpp (1054) open FFMPEG: trying to configure H/W acceleration: 'qsv.iHD'
[OPENCV:FFMPEG:40] Trying to use DRM render node for device 0.
[OPENCV:FFMPEG:40] libva: VA-API version 1.9.0
libva: Trying to open /usr/lib64/dri/iHD_drv_video.so
libva: Found init function __vaDriverInit_1_9
libva: va_openDriver() returns 0
Initialised VAAPI connection: version 1.9
[OPENCV:FFMPEG:48] Format 0x41524742 -> bgra.
[OPENCV:FFMPEG:48] Format 0x42475241 -> argb.
[OPENCV:FFMPEG:48] Format 0x41424752 -> rgba.
[OPENCV:FFMPEG:48] Format 0x52474241 -> abgr.
[OPENCV:FFMPEG:48] Format 0x58524742 -> bgr0.
[OPENCV:FFMPEG:48] Format 0x42475258 -> 0rgb.
[OPENCV:FFMPEG:48] Format 0x58424752 -> rgb0.
[OPENCV:FFMPEG:48] Format 0x52474258 -> 0bgr.
[OPENCV:FFMPEG:48] Format 0x30335241 -> unknown.
[OPENCV:FFMPEG:48] Format 0x30334241 -> unknown.
[OPENCV:FFMPEG:48] Format 0x30335258 -> unknown.
[OPENCV:FFMPEG:48] Format 0x30334258 -> unknown.
[OPENCV:FFMPEG:48] Format 0x36314752 -> unknown.
[OPENCV:FFMPEG:48] Format 0x50424752 -> unknown.
[OPENCV:FFMPEG:48] Format 0x50524742 -> unknown.
[OPENCV:FFMPEG:48] Format 0x56555941 -> unknown.
[OPENCV:FFMPEG:48] Format 0x30303859 -> gray.
[OPENCV:FFMPEG:48] Format 0x3231564e -> nv12.
[OPENCV:FFMPEG:48] Format 0x3132564e -> unknown.
[OPENCV:FFMPEG:48] Format 0x32595559 -> yuyv422.
[OPENCV:FFMPEG:48] Format 0x59565955 -> uyvy422.
[OPENCV:FFMPEG:48] Format 0x32315659 -> yuv420p.
[OPENCV:FFMPEG:48] Format 0x30323449 -> yuv420p.
[OPENCV:FFMPEG:48] Format 0x50313134 -> yuv411p.
[OPENCV:FFMPEG:48] Format 0x48323234 -> yuv422p.
[OPENCV:FFMPEG:48] Format 0x56323234 -> yuv440p.
[OPENCV:FFMPEG:48] Format 0x50343434 -> yuv444p.
[OPENCV:FFMPEG:48] Format 0x33434d49 -> unknown.
[OPENCV:FFMPEG:48] Format 0x30313050 -> p010le.
[OPENCV:FFMPEG:40] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 20.3.0 ().
[OPENCV:FFMPEG:40] Driver not found in known nonstandard list, using standard behaviour.
[ INFO:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (272) hw_check_device FFMPEG: Using MFX video acceleration on device: Intel iHD driver for Intel(R) Gen Graphics - 20.3.0 ()
[ INFO:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (562) hw_create_device FFMPEG: Created video acceleration context (av_hwdevice_ctx_create) for vaapi on device 'default'
[ INFO:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (579) hw_create_device FFMPEG: Can't bind vaapi video acceleration context to already created OpenCL context
[OPENCV:FFMPEG:40] Initialize MFX session: API version is 1.34, implementation version is 1.34
[ INFO:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (305) hw_create_derived_context FFMPEG: Created derived video acceleration context (av_hwdevice_ctx_create_derived) for qsv
VideoCapture FFMPEG:MFX
[DEBUG:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (618) hw_create_frames FFMPEG: avcodec_get_hw_frames_parameters() call failed: -2
[OPENCV:FFMPEG:48] Created surface 0.
[OPENCV:FFMPEG:48] Direct mapping possible.
[OPENCV:FFMPEG:48] Created surface 0x1.
[OPENCV:FFMPEG:48] Created surface 0x2.
[OPENCV:FFMPEG:48] Created surface 0x3.
[OPENCV:FFMPEG:48] Created surface 0x4.
[OPENCV:FFMPEG:48] Created surface 0x5.
[OPENCV:FFMPEG:48] Created surface 0x6.
[OPENCV:FFMPEG:48] Created surface 0x7.
[OPENCV:FFMPEG:48] Created surface 0x8.
[OPENCV:FFMPEG:48] Created surface 0x9.
[OPENCV:FFMPEG:48] Created surface 0xa.
[OPENCV:FFMPEG:48] Created surface 0xb.
[OPENCV:FFMPEG:48] Created surface 0xc.
[OPENCV:FFMPEG:48] Created surface 0xd.
[OPENCV:FFMPEG:48] Created surface 0xe.
[OPENCV:FFMPEG:48] Created surface 0xf.
[OPENCV:FFMPEG:48] Created surface 0x10.
[OPENCV:FFMPEG:48] Created surface 0x11.
[OPENCV:FFMPEG:48] Created surface 0x12.
[OPENCV:FFMPEG:48] Created surface 0x13.
[OPENCV:FFMPEG:48] Created surface 0x14.
[OPENCV:FFMPEG:48] Created surface 0x15.
[OPENCV:FFMPEG:48] Created surface 0x16.
[OPENCV:FFMPEG:48] Created surface 0x17.
[OPENCV:FFMPEG:48] Created surface 0x18.
[OPENCV:FFMPEG:48] Created surface 0x19.
[OPENCV:FFMPEG:48] Created surface 0x1a.
[OPENCV:FFMPEG:48] Created surface 0x1b.
[OPENCV:FFMPEG:48] Created surface 0x1c.
[OPENCV:FFMPEG:48] Created surface 0x1d.
[OPENCV:FFMPEG:48] Created surface 0x1e.
[OPENCV:FFMPEG:48] Created surface 0x1f.
[OPENCV:FFMPEG:48] Format qsv chosen by get_format().
[OPENCV:FFMPEG:40] Decoder: output is video memory surface
[DEBUG:0] global /home/alalek/projects/opencv/dev/modules/videoio/src/cap_ffmpeg_hw.hpp (618) hw_create_frames FFMPEG: avcodec_get_hw_frames_parameters() call failed: -2
[OPENCV:FFMPEG:48] Created surface 0x20.
[OPENCV:FFMPEG:48] Direct mapping possible.
[OPENCV:FFMPEG:48] Created surface 0x21.
[OPENCV:FFMPEG:48] Created surface 0x22.
[OPENCV:FFMPEG:48] Created surface 0x23.
[OPENCV:FFMPEG:48] Created surface 0x24.
[OPENCV:FFMPEG:48] Created surface 0x25.
[OPENCV:FFMPEG:48] Created surface 0x26.
[OPENCV:FFMPEG:48] Created surface 0x27.
[OPENCV:FFMPEG:48] Created surface 0x28.
[OPENCV:FFMPEG:48] Created surface 0x29.
[OPENCV:FFMPEG:48] Created surface 0x2a.
[OPENCV:FFMPEG:48] Created surface 0x2b.
[OPENCV:FFMPEG:48] Created surface 0x2c.
[OPENCV:FFMPEG:48] Created surface 0x2d.
[OPENCV:FFMPEG:48] Created surface 0x2e.
[OPENCV:FFMPEG:48] Created surface 0x2f.
[OPENCV:FFMPEG:48] Created surface 0x30.
[OPENCV:FFMPEG:48] Created surface 0x31.
[OPENCV:FFMPEG:48] Created surface 0x32.
[OPENCV:FFMPEG:48] Created surface 0x33.
[OPENCV:FFMPEG:48] Created surface 0x34.
[OPENCV:FFMPEG:48] Created surface 0x35.
[OPENCV:FFMPEG:48] Created surface 0x36.
[OPENCV:FFMPEG:48] Created surface 0x37.
[OPENCV:FFMPEG:48] Created surface 0x38.
[OPENCV:FFMPEG:48] Created surface 0x39.
[OPENCV:FFMPEG:48] Created surface 0x3a.
[OPENCV:FFMPEG:48] Created surface 0x3b.
[OPENCV:FFMPEG:48] Created surface 0x3c.
[OPENCV:FFMPEG:48] Created surface 0x3d.
[OPENCV:FFMPEG:48] Created surface 0x3e.
[OPENCV:FFMPEG:48] Created surface 0x3f.
[OPENCV:FFMPEG:48] Format qsv chosen by get_format().
[OPENCV:FFMPEG:40] Decoder: output is video memory surface
[OPENCV:FFMPEG:16] Error initializing the MFX video decoder: unsupported (-3)
/home/alalek/projects/opencv/dev/modules/videoio/test/test_video_io.cpp:743: Failure
Value of: hw_reader.read(frame)
Actual: false
Expected: true
Google Test trace:
/home/alalek/projects/opencv/dev/modules/videoio/test/test_video_io.cpp:732: frame=0
/home/alalek/projects/opencv/dev/modules/videoio/test/test_video_io.cpp:745: Failure
Value of: frame.empty()
Actual: true
Expected: false
Google Test trace:
/home/alalek/projects/opencv/dev/modules/videoio/test/test_video_io.cpp:732: frame=0
[OPENCV:FFMPEG:40] Statistics: 31057 bytes read, 0 seeks
[ FAILED ] videoio/videocapture_acceleration.read/84, where GetParam() = (sample_322x242_15frames.yuv420p.libvpx-vp9.mp4, FFMPEG, MFX, false) (6 ms)
vainfo
$ vainfo
libva info: VA-API version 1.9.0
libva info: Trying to open /usr/lib64/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_9
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.9 (libva 2.9.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 20.3.0 ()
vainfo: Supported profile and entrypoints
VAProfileNone : VAEntrypointVideoProc
VAProfileNone : VAEntrypointStats
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Simple : VAEntrypointEncSlice
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointVLD
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointFEI
VAProfileH264Main : VAEntrypointEncSliceLP
VAProfileH264High : VAEntrypointVLD
VAProfileH264High : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointFEI
VAProfileH264High : VAEntrypointEncSliceLP
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileJPEGBaseline : VAEntrypointVLD
VAProfileJPEGBaseline : VAEntrypointEncPicture
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264ConstrainedBaseline: VAEntrypointFEI
VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
VAProfileVP8Version0_3 : VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointEncSlice
VAProfileHEVCMain : VAEntrypointFEI
Looks like there is no VP9 entry in vainfo:
- Fedora 33
intel-media-driver.x86_64 20.3.0-2.fc33 @rpmfusion-nonfree-updates- CPU: i5-6600 (Skylake)
Similar problem: https://bugs.launchpad.net/ubuntu/+source/intel-vaapi-driver/+bug/1591714
@mikhail-nikolskiy Can we handle/workaround this somehow?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No easy workaround. ffmpeg-qsv decoder designed with delayed initialization, the MFXVideoDECODE_Init called on decoding first frame, it's too late for fallback to SW decoder in current implementation. No such problem in VAAPI and D3D11 decoders as they report all errors on opening stage (av_codec_open2 call).
Disable MFX decode (revert "qsv.iHD" back to empty string "") or keep as known issue in case MFX acceleration explicitly requested?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mikhail-nikolskiy Thank you for the information!
Looks like we need to add the first frame grabbing as part of .open() implementation to handle/workaround this case (I believe we can implement this later).
|
@mshabunin Could you please take a look on CMake scripts changes? Note: "plugins" build mode is not fully supported yet (but we should ensure that compilation is not broken). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great contribution! Thank you đź‘Ť
cv::UMat output/input in VideoCapture/VideoWriter (data stays in GPU memory) * FFMPEG with UMat input/output * OpenCL_D3D* context * fix Linux build * cosmetic changes * fix build if USE_AV_HW_CODECS=0 * simplify how child context pointer stored in parent context * QSV interop with OpenCL on Windows * detect_msdk.cmake via pkg-config * fix av_buffer_ref() usage * revert windows-decode-mfx whitelisting; remove debug msg * address review comments * rename property to HW_ACCELERATION_USE_OPENCL * fix issue with "cl_khr_d3d11_sharing" extension not reported by OpenCL GPU+CPU platform * core(ocl): add OpenCL stubs for configurations without OpenCL * videoio(ffmpeg): update #if guards * Put OpenCL related code under HAVE_OPENCL; simplify reuse of media context from OpenCL context * videoio(test): skip unsupported tests - plugins don't support OpenCL/UMat yet - change handling of *_USE_OPENCL flag * videoio(ffmpeg): OpenCL dependency * videoio(ffmpeg): MediaSDK/oneVPL dependency * cleanup, logging * cmake: fix handling of 3rdparty interface targets Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
The pull request utilizes HW acceleration added recently to FFMPEG backend of videoio module, and core APIs for interoperability with DirectX and VAAPI.
Interoperability requires OpenCL context created with DirectX/VAAPI device passed at context init time. So in case of HW decode/encode and cv::UMat, data can stay in GPU memory if VideoCapture/VideoWriter open() called before any OpenCL-related functions so OpenCL context is empty at open() time, or OpenCL context explicitly initialized in interop mode before open() via calling functions cv::directx::ocl::initializeContextFromD3D*Device/cv::va_intel::ocl::initializeContextFromVA. Otherwise data copied through system memory GPU->CPU->GPU.
if cv::UMat and HW mode
For example, if app creates VideoCapture then VideoWriter instances with parameters PROP_HW_ACCELERATION and PROP_HW_DEVICE before OpenCL calls, the first-opened VideoCapture will try open GPU device specified in PROP_HW_DEVICE, the second-opened VideoWriter will reuse media/OpenCL device instance previously created by VideoCapture.
The pull request also introduces new functions in ocl::Context to unify capabilities of attaching external context to OpenCL context, as it used now in three places directx.cpp (core), va_intel.cpp (core), cap_ffmpeg_hw.hpp (videoio).
relates #19676
TODO (alalek):
HAVE_MFXof cvconfig.hHAVE_MFX/HAVE_OPENCLfor FFmpeg plugin (with include directories)HAVE_VA/HAVE_VA_INTEL/ etc for FFmpeg plugin (with include directories)Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
Patch to opencv_extra has the same branch name.