CARVIEW |
Navigation Menu
-
Notifications
You must be signed in to change notification settings - Fork 5.8k
Adapt python op converters covered in paddle-3-beta2 for TRT 10 #69510
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adapt python op converters covered in paddle-3-beta2 for TRT 10 #69510
Conversation
你的PR提交成功,感谢你对开源项目的贡献! |
Anderson Meng seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
e528354
to
252f83d
Compare
目前CI流水线上缺乏对TRT10版本的监控,PR先暂时这样提上来,可能先不合入,等最近协调后应该会上线带有TRT10的流水线,到时候再合入PR,谢谢 |
@YuanRisheng
|
@anderson101866 对于第一点你的理解应该是对的,这个我们再考虑一下, 想一想是先让你们这边把修复trt10的单测pr提交合入,还是先把trt10的ci弄起来(convert相关单测禁止运行),你们这边提交修复代码后打开单测。对于第二点,你的理解也是对的,我们这边会找人来让CI-APPROVAL通过 |
好的謝謝,另外附註,NV這邊因為環境關係,我們只有beta2的環境,所以除了build engine,op的部分只有打開幾個beta2單測有覆蓋的op。 實際develop分支的op數量應該更多,這個PR未包含 |
了解了,但是我们建议后续使用develop来验证trt10,beta2包含的op数量应该很少,大部分在develop里 |
@YuanRisheng @vivienfanghuagood |
@anderson101866 我们讨论了一下,可以先推进pr的合入,合入后我们这边线下测试一下,辛苦更新PR解决一下conflict |
a2ec045
to
29ed847
Compare
CI-Codestyle-Check存在代码格式的问题,请本地修正后再提交一次,谢谢 |
This commit includes basic migration to TRT 10 API, and also enable those converter who are tested with existing unittest to TRT10 For those 2 converter which is NOT included in this commit: - python/paddle/tensorrt/impls/attribute.py - python/paddle/tensorrt/impls/common.py Need to fix these 2 behavior issues in later commit.
Although `IShapeLayer` supports shape in int64 since TRT10, some paddle native op kernel only implements their input shape tensor (if exists) in int32. Hence, there is a workaround in `trt_shape` to cast the result of TRT `IShapeLayer` back to int32 to be more compatible with other paddle op. (see python/paddle/tensorrt/converter_utils.py) Please remove the workaround when all paddle op supports their shape in int64. Also, since `IShapeLayer` return shape in int64 in TRT10, the "pd_op.shape64" will be seamlessly supported in TRT10 w/o any extra workaround.
Error detail: { (%1) = "pd_op.bilinear_interp" [id:28] (%2, %3, <<NULL VALUE>>, <<NULL VALUE>>) {__l_trt__:true,align_corners:false,align_mode:(Int32)0,data_format:"NCHW",interp_method:"bilinear",out_d:(Int32)-1,out_h:(Int32)12,out_w:(Int32)12,scale:[],stop_gradient:[true]} : (builtin.tensor<-1x3x6x10xf32>, builtin.tensor<2xi32>, <<NULL TYPE>>, <<NULL TYPE>>) -> builtin.tensor<-1x3x12x12xf32> () = "cf.yield" [id:36] (%1) {} : (builtin.tensor<-1x3x12x12xf32>) -> } [TRT] [E] ITensor::getDimensions: Error Code 4: API Usage Error ((Unnamed Layer* 6) [Concatenation]: concat input tensors 0 and 2 have incompatible types Int64 and Int32) [TRT] [E] IBuilder::buildSerializedNetwork: Error Code 4: API Usage Error ((Unnamed Layer* 6) [Concatenation]: concat input tensors 0 and 2 have incompatible types Int64 and Int32) The error happened in "python/paddle/tensorrt/impls/common.py" because IConcatenationLayer requires all input to be same dtype. The (shape)tensor passed from paddle op will be int32, while the TRT IShapeLayer supports int64 shape; thereby, cannot be concatenated with each other. Here, we call `trt_shape` to get shape tensor whose dtype aligned with the dtype from paddle op.
test_converter_math can pass unittest with environment with TRT 10.6
- "pd_op.expand" - "pd_op.expand_as" - "pd_op.slice"
29ed847
to
adb26c6
Compare
OK重解conflict的時候沒有注意到,已上傳 等待check重新驗證 |
(format check已通過) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
PR Category
Inference
PR Types
Bug fixes
Description
細節主要分為:
(1)
build TRT engine相關的改動(PR審核期間,Paddle內部已移除deprecatedbuild_engine
, see commit:d4e1784b) 、以及(2) Layer相關的改動
- 其中(1)為一次性的改動engine建置過程;而 後續python op converter開發者僅需專注(2) 的部分,專注TRT 10的介面變化
(1)的部分請參考
python/paddle/tensorrt/converter.py
的改動,很簡單。可以參考上文deprecated API的表格(2)的部分包括:
TRT10的IShapeLayer從此擴展為INT64的介面,以適應形狀大的Tensor。
shape_tensor
只有int32版本,因此TRT上/下游的paddle op實作INT64之前,只得套用現有的workaround (see:python/paddle/tensorrt/converter_utils.py
統一轉換成較窄的int32,直到paddle正式支援int64 shapeTRT10對於pool layer僅支援nd的介面,移除2d介面 (同樣可參考上文的deprecated API表格)
seepython/paddle/tensorrt/impls/pooling.py
,add_pooling
/stride
/padding
等介面移除然而此PR合併前,Paddle內部已於
pooling.py
中移除2d介面,see commit:6043c7f3其餘改動,僅是驗證現有paddle-3.0.0-beta2有的單元測試有覆蓋的op,將支援版本限制放鬆
trt_version="8.x"
->trt_version="trt_version_ge=8.0"