| CARVIEW |
Navigation Menu
-
Notifications
You must be signed in to change notification settings - Fork 5.8k
[0-size Tensor No.175] Add 0-size Tensor support for group_norm #73152
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
你的PR提交成功，感谢你对开源项目的贡献! |
co63oc
changed the title
Codecov ReportAttention: Patch coverage is
❌ Your patch status has failed because the patch coverage (9.52%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## develop #73152 +/- ##
==========================================
Coverage ? 9.52%
==========================================
Files ? 2
Lines ? 21
Branches ? 0
==========================================
Hits ? 2
Misses ? 19
Partials ? 0 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| dev_ctx.template Alloc<T>(d_x); | ||
| if (d_scale) { | ||
| phi::Full<T, Context>(dev_ctx, | ||
| phi::IntArray(common::vectorize(d_scale->dims())), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
需要验证一下输出存在0-size Tensor时，非0-size Tensor的反向，torch返回的是nan还是0。需要与torch保持一致。而且单测中出了验证grad的shape之外还需要验证非0-size的值是否和预期一样。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已修改单测，比较scale, bias 的grad，torch中bias返回为0, scale 在batch为0时返回0，其他返回NAN
https://docs.pytorch.org/docs/stable/generated/torch.nn.GroupNorm.html
import torch
import torch.nn as nn
input = torch.randn(0, 6, 10, 10)
m = nn.GroupNorm(3, 6)
output = m(input)
torch.sum(output).backward()
print(m.weight.grad)
print(m.bias.grad)
tensor([0., 0., 0., 0., 0., 0.])
tensor([0., 0., 0., 0., 0., 0.])
import torch
import torch.nn as nn
input = torch.randn(1, 6, 0, 10)
m = nn.GroupNorm(3, 6)
output = m(input)
torch.sum(output).backward()
print(m.weight.grad)
print(m.bias.grad)
tensor([nan, nan, nan, nan, nan, nan])
tensor([0., 0., 0., 0., 0., 0.])
| @@ -1178,6 +1178,18 @@ void GroupNormKernel(const Context& dev_ctx, | |||
| DenseTensor* y, | |||
| DenseTensor* mean, | |||
| DenseTensor* var) { | |||
| if (y && y->numel() == 0) { | |||
| dev_ctx.template Alloc<T>(y); | |||
| if (mean) { | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
同理mean var等tensor的默认值是nan还是0，请与torch对比一下并保持一致。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mean, var 是中间变量，torch中没有返回https://docs.pytorch.org/docs/stable/generated/torch.nn.GroupNorm.html
，paddle中python接口也没有返回可以测试
|
Sorry to inform you that a6fb766's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
PR Category
Execute Infrastructure
PR Types
Improvements
Description
待修改
[0-size Tensor No.175] Add 0-size Tensor support for group_norm
修改前向和反向
infermeta没有修改
kernel修改cpu/gpu/xpu
group和channel维度需要一致并大于0
PaddleAPITest 测试通过
