CARVIEW |
Navigation Menu
-
-
Notifications
You must be signed in to change notification settings - Fork 56.2k
G-API: oneVPL (simplification) unite components in entire VPL source #20773
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it supposed to work at this stage? I believe I've seen parameter/debug handling only but not the actual execution
} catch(const std::exception& ex) { | ||
std::stringstream ss; | ||
ss << ex.what() << ". Unload VPL session: " << mfx_session; | ||
const std::string str = ss.str(); | ||
GAPI_LOG_WARNING(nullptr, str); | ||
MFXClose(mfx_session); | ||
util::throw_error(std::logic_error(str)); | ||
} | ||
} catch (const std::exception& ex) { | ||
std::stringstream ss; | ||
ss << "Cannot create VPL source Impl, error: " << ex.what() << ". Unload MFX handle: " << mfx_handle; | ||
const std::string str = ss.str(); | ||
GAPI_LOG_WARNING(nullptr, str); | ||
util::throw_error(std::logic_error(str)); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we really need this handling here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, throw exception on upper level to invoke dtor to free mfx_handle
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
but if the above code throws already, the upper level dtor will be called anyway - the question is whether we need the catch
block.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why not just use delegating constructor to be sure that destructor will be always called if exception happens in this delegating constructor?
https://en.cppreference.com/w/cpp/language/throw
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@AsyaPronina
'delegating ctor' - it used as well.
@dmatveev
The question is to print out WARN about with all summarized information about failure
But it is possible to get rid of duplicating the error message in exception rethrowing but only in case if WARN severity is always ON
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Catching your own exceptions only makes debugging harder not easier - you wan't get into the place where the original exception has been thrown.
Please remove this handling and if you want some better reporting, produce logs such that when an exception is thrown, the context is clear for the reader.
if (!cfg_param_it->is_major()) { | ||
GAPI_LOG_DEBUG(nullptr, "Skip not major param: " << cfg_param_it->get_name()); | ||
++cfg_param_it; | ||
continue; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
two step logic:
user passed MAJOR & not-MAJOR (optional) params to create as VPL source.
VPL source inside chose several available implementations (by MAJOR params such as ACCEL type, codec and so on) in generic case. So we can got several impls inside. It is possible to select the first one of them but it also possible to introduce not-MAJOR/optional/preferrable params which allow to select preferrable implementation from availavle implementations which provide MAJOR params match already.
Amid optional params VPL version may be present ( in case of use have multiple versions).
So at first step:
- we request VPL available Implementations using MAJOR params only
- we printout available implementations ALL params in
std::string
- serialize string into CFG not-major params
- use max intersection of requested not-major params from user & seriaziled implemnetation params to find the better match
If no not-major params passed then the first VPL implementation would chosen
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So at first step:
we request VPL available Implementations using MAJOR params only
we printout available implementations ALL params in std::string
serialize string into CFG not-major params
use max intersection of requested not-major params from user & seriaziled implemnetation params to find the better match
This behavior sounds complicated, is this what we really have to do here? Is there a way to be a thinner wrapper over this VPL stuff?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If optional params from multiple implementation create equal max intersections with user ones, first one will be used?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This behavior sounds complicated, is this what we really have to do here? Is there a way to be a thinner wrapper over this VPL stuff?
If it looks complicated: it is fully possible just use default CfgParams
creation (with major by default) without any concern about optional params and fine-grained tuning
Optional params implementation is stick on already written code: parse in/from string and mfxVariant - (only operator less
introduced) and comes with cheap implementation cost and functional for fine-grained tuning
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If optional params from multiple implementation create equal max intersections with user ones, first one will be used?
After set major params: vpl dispatcher then draw several implementations which matches requested CfgParams.
For example, if you choose HEVC encoding, then VPL dispatcher returns all implementations which support HEVC encoding: CPU,GPU, FPGA(?) and so on.
So current function just serialize each collected VPL implementation descriptions into string for two purposes:
- to print out this setting in LOG
- to deserialize it back into CfgParams
The latter deserialized CfgParams (of available VPL implementations) puts in intersection with optional params. And entersected ( common params) count are remembered. Each VPL implementation tested in the same way with optional params intersection. Then an implementation with MAX intersected params count would chosen in final and finish off source creation
IF no optional params are passed (95% of regular user scenarios) into VPL source during its creation than the first VPL implementation would be used (simplified case)
But, for example (5% cases), I choose optional VPL param version in my development to test & launch my decode pipeline to switch off my-custom debug and ordinary vpl version without recompilation. and some other debug-like, performance-like purposes or VPL-api testing. because ( in last case it looks not-finalized at the moment and might or might not come with testing & unstable features) Especially when it will turn-on TigerLake support, because at least TWO more version of VPL api would become available at that moment
return {major_val << 16 | minor_val}; | ||
} | ||
|
||
std::ostream& operator<< (std::ostream& out, const mfxImplDescription& idesc) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
where do we need this? is it debugging purposes only?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
serialize to string for printout + for restoring into CfgParams to find intersection
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for restoring into CfgParams to find intersection
Why do we need this at all?
} // namespace wip | ||
} // namespace gapi | ||
} // namespace cv | ||
#endif // HAVE_ONEVPL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this file can be ~2x shorter. :)
it works, but in ~10-100 times worst than VideoCap |
@dmatveev
It takes remembered state for mfx_session and go through "state-machine" State Machine declared in DecodeEngine here
Frame put into queue |
@@ -117,7 +117,7 @@ mfxStatus ReadEncodedStream(mfxBitstream &bs, std::shared_ptr<IDataProvider>& da | |||
return MFX_ERR_NOT_ENOUGH_BUFFER; | |||
} | |||
|
|||
std::copy_n(p0, bs.DataLength, p1); | |||
std::copy_n(p1, bs.DataLength, p0); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this bug was introduced in prev PR fixing
if (!cfg_param_it->is_major()) { | ||
GAPI_LOG_DEBUG(nullptr, "Skip not major param: " << cfg_param_it->get_name()); | ||
++cfg_param_it; | ||
continue; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
two step logic:
user passed MAJOR & not-MAJOR (optional) params to create as VPL source.
VPL source inside chose several available implementations (by MAJOR params such as ACCEL type, codec and so on) in generic case. So we can got several impls inside. It is possible to select the first one of them but it also possible to introduce not-MAJOR/optional/preferrable params which allow to select preferrable implementation from availavle implementations which provide MAJOR params match already.
Amid optional params VPL version may be present ( in case of use have multiple versions).
So at first step:
- we request VPL available Implementations using MAJOR params only
- we printout available implementations ALL params in
std::string
- serialize string into CFG not-major params
- use max intersection of requested not-major params from user & seriaziled implemnetation params to find the better match
If no not-major params passed then the first VPL implementation would chosen
std::vector<CfgParam> matched_params; | ||
std::set_intersection(impl_params.begin(), impl_params.end(), | ||
preferred_params.begin(), preferred_params.end(), | ||
std::back_inserter(matched_params)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
test-out user-defiend preferred params & serialized impl params to find intersercted params
|
||
if (preferred_params.empty()) { | ||
// in case of no preferrance we consider all params are matched | ||
matches_count.emplace(impl_params.size(), i++); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
and remember intersected params count to find better match (by match counts)
} catch(const std::exception& ex) { | ||
std::stringstream ss; | ||
ss << ex.what() << ". Unload VPL session: " << mfx_session; | ||
const std::string str = ss.str(); | ||
GAPI_LOG_WARNING(nullptr, str); | ||
MFXClose(mfx_session); | ||
util::throw_error(std::logic_error(str)); | ||
} | ||
} catch (const std::exception& ex) { | ||
std::stringstream ss; | ||
ss << "Cannot create VPL source Impl, error: " << ex.what() << ". Unload MFX handle: " << mfx_handle; | ||
const std::string str = ss.str(); | ||
GAPI_LOG_WARNING(nullptr, str); | ||
util::throw_error(std::logic_error(str)); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, throw exception on upper level to invoke dtor to free mfx_handle
} | ||
|
||
GSource::Priv::Priv(std::shared_ptr<IDataProvider>, const std::vector<CfgParam>&) : | ||
GSource::Priv::Priv(std::shared_ptr<IDataProvider> provider, const std::vector<CfgParam>& params) : |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why GSource
and not VPLSource
/OneVPLSource
/GVPLSource
/...?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought about that, but cv::gapi::wip::onevpl::**VPL**Source
and something like that with VPL root looks redundant
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, good point!
I think that for user it might be easier to distinguish GVPLSource
and GStreamerSource
rather than cv::gapi::wip::onevpl::GSource
and cv::gapi::wip::gstreamer::GSource
.
So might be can introduce kind of type alias here in a namespace one-level higher to uniform source names later?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good idea
i'll add aliases into source.hpp
if (!cfg_param_it->is_major()) { | ||
GAPI_LOG_DEBUG(nullptr, "Skip not major param: " << cfg_param_it->get_name()); | ||
++cfg_param_it; | ||
continue; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If optional params from multiple implementation create equal max intersections with user ones, first one will be used?
dec.Codecs[codec].Profiles[profile].Profile) << std::endl; | ||
for (int memtype = 0; memtype < dec.Codecs[codec].Profiles[profile].NumMemTypes; memtype++) { | ||
out << "mfxImplDescription.mfxDecoderDescription.decoder.Profiles.MemDesc.MemHandleType: " | ||
<< mfx_resource_type_to_cstr(dec.Codecs[codec].Profiles[profile].MemDesc[memtype].MemHandleType) << std::endl; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems that there are should be intermediate variables for codec
, profile
and memtype
..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i tried to align it with MFX/VPL configurable params name. I mean, it's description can be used to create VPL inner-dependable creation param name. Some of them do not match with object structure
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I am bit confused with the answer
The comment was about that the code repeatedly duplicated accesses to the same fields of dec
variable. Some fields (or field of field...) can be cached in intermediate variable
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I got it. lets' introduce caching variables here later? because there are several comments about MACRO & renaming for entire file and I'm getting on this later in next PR series
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please clear the internal exception handling and we're good to go.
mfx_impl_description(), | ||
mfx_handle_configs(), | ||
cfg_params(), | ||
mfx_session(), | ||
description(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still don't get why do we need this. Shouldn't the plain =nullptr;
just work in the class declaration?
}; | ||
|
||
template<typename ValueType> | ||
std::vector<ValueType> get_params_from_string(const std::string& str) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so do we parse a string here which we generate somewhere previously?
} catch(const std::exception& ex) { | ||
std::stringstream ss; | ||
ss << ex.what() << ". Unload VPL session: " << mfx_session; | ||
const std::string str = ss.str(); | ||
GAPI_LOG_WARNING(nullptr, str); | ||
MFXClose(mfx_session); | ||
util::throw_error(std::logic_error(str)); | ||
} | ||
} catch (const std::exception& ex) { | ||
std::stringstream ss; | ||
ss << "Cannot create VPL source Impl, error: " << ex.what() << ". Unload MFX handle: " << mfx_handle; | ||
const std::string str = ss.str(); | ||
GAPI_LOG_WARNING(nullptr, str); | ||
util::throw_error(std::logic_error(str)); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Catching your own exceptions only makes debugging harder not easier - you wan't get into the place where the original exception has been thrown.
Please remove this handling and if you want some better reporting, produce logs such that when an exception is thrown, the context is clear for the reader.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
@alalek could you merge this PR please? |
G-API: oneVPL (simplification) unite components in entire VPL source * Unify components in VPLSource * Revert back decode WRN & Add compile guard * Address come comments * Add source alias * Apply comment for exception handling
Make VPL source usable for decoding simple demultiplexed video streams.
Limitations:
Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
Patch to opencv_extra has the same branch name.
Build configuration