BTW,
DNN redesign is necessary to resolve problem with storing of stalled unused buffers.

We need 2 entities of DL model at least:

• "imported model"
• "compiled model" for the specific target
• imported model could be released with its buffers after compilation (if user won't to change the target device).
• user infers using compiled model only.

@opencv-alalek, agree. For now something intermediate is used - we clean buffers during the import trying not to store two instances of the weights (protobuf does not let more memory ownership).

If you mean current OpenVINO workflow with model/compiled_model - they have an advantage with offline IR converter. But yes, after the compilation (CompiledModel) a model reference (Model) can be removed.

Not sure if we need introduce two instances in public API, maybe add two private implementations: LayerProto (with original weights, shapes logic, backends initialization) and LayerOCV, which is a separate "default backend" implementation. And need an agreement or option to let us clean original weights (means after OpenCV forward user won't set a different backend).

Gemm, MatMul and Attention layers also have this problem as constant weight needs to be prepacked and stored while leaving the original one unused.