| CARVIEW |
Navigation Menu
-
Notifications
You must be signed in to change notification settings - Fork 5.8k
[Paddle-TRT] custom operator support generating plugin automatically #58976
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
你的PR提交成功，感谢你对开源项目的贡献! |
| @@ -27,6 +27,10 @@ limitations under the License. */ | |||
| #include "paddle/utils/none.h" | |||
| #include "paddle/utils/optional.h" | |||
|
|
|||
| #ifdef PADDLE_WITH_TENSORRT | |||
| #include "NvInfer.h" | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个.h在inference demo仓库编译用户模块的时候，能找到吗
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TensorRT的头文件，用户指定TENSORRT_ROOT后是可以找到的。这个问题会在demo的README.me中补充说明一下。
paddle/phi/api/ext/op_meta_info.h
Outdated
| // NOTE(HongyuJia): Used to be compatible with the 2.0.1 released | ||
| // interface, and will be deprecated in the future | ||
| PD_SPECIALIZE_TrtSupportsFormateCallHelper_FOR_ATTR(const bool&); | ||
| PD_SPECIALIZE_TrtSupportsFormateCallHelper_FOR_ATTR(const int&); | ||
| PD_SPECIALIZE_TrtSupportsFormateCallHelper_FOR_ATTR(const float&); | ||
| PD_SPECIALIZE_TrtSupportsFormateCallHelper_FOR_ATTR(const int64_t&); | ||
|
|
||
| // NOTE(HongyuJia): Used to be compatible with the 2.1 released | ||
| // interface, but not recommended | ||
| PD_SPECIALIZE_TrtSupportsFormateCallHelper_FOR_ATTR(std::string); | ||
| PD_SPECIALIZE_TrtSupportsFormateCallHelper_FOR_ATTR(std::vector<int>); | ||
| PD_SPECIALIZE_TrtSupportsFormateCallHelper_FOR_ATTR(std::vector<float>); | ||
| PD_SPECIALIZE_TrtSupportsFormateCallHelper_FOR_ATTR(std::vector<std::string>); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这些为了兼容的，你这里是否有必要再加上？
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
我认为还是有必要的，不然在存量的自定义算子中，会出现为了使用trt还要把const int的属性改为int。
|
inference demo仓库里也需要提供demo示例 |
|
需要在PR描述里把所有不支持的情况以及TODO暴露一下，比如win上有bug之类的 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM for set_tests_properties(test_inference_gap_setup PROPERTIES TIMEOUT 180)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM for const_cast
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
paddle/phi/api/ext/op_meta_info.h
Outdated
| pos_nbInputs_nbOutputs, in_out, attrs, pargs..., arg); \ | ||
| } catch (paddle::bad_any_cast&) { \ | ||
| PD_THROW( \ | ||
| "Attribute cast error in custom operator TrtSupportsFormate " \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这里换行有点奇怪
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这里断行是和原自定义算子的SetKernelFn, SetInferDtypeFn, SetInferDtypeFn中保持一致写的
paddle/phi/api/ext/op_meta_info.h
Outdated
|
|
||
| // end: base template | ||
| template <typename T> | ||
| struct TrtSupportsFormateCallHelper<TypeTag<T>> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Formate -> format？
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已修改
test/custom_op/inference_gap.cc
Outdated
| return dimsOutput; | ||
| } | ||
|
|
||
| bool supportsFormatCombination( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个函数写起来过于繁琐，增加用户理解成本，可以考虑使用代码生成：
https://github.com/NVIDIA-AI-IOT/tensorrt_plugin_generator/blob/main/tpg/generate.py
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已修改
ming1753
dismissed stale reviews from yuanlehome, Ligoml, zhiqiu, and YuanRisheng
via
fcfc024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM for const_cast
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…addlePaddle#58976) * [Paddle-TRT] custom operator support generating plugin automatically
PR types
Others
PR changes
Others
Description
Pcard-71501
本PR的目的是在写了自定义算子(custom opetator)后，只需提供尽可能少的代码就能使自定义算子进trt，即自动生成对应的trt plugin。
Example：