| CARVIEW |
Navigation Menu
-
Notifications
You must be signed in to change notification settings - Fork 5.8k
【Hackathon 6th No.23】为 paddle.nn.functional.embedding/paddle.nn.Embedding 增加参数 max_norm/norm_type/scale_grad_by_freq - part #63130
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
你的PR提交成功，感谢你对开源项目的贡献! |
|
❌ The PR is not created using PR's template. You can refer to this Demo. |
|
@AndPuQing 你好，需要看一下大量CI没通过的原因 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
覆盖率流水线没过，如果这些函数本地测试都跑进了，请提供本地截图
|
Sorry to inform you that 8b803d9's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually. |
|
Sorry to inform you that c9d0326's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually. |
|
@AndPuQing 冲突麻烦修改一下 |
python/paddle/nn/functional/input.py
Outdated
| r""" | ||
| This operator is used to update the embedding weight by renorm. | ||
| """ | ||
| if in_dynamic_mode(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- 这里能否 不新加kernel，直接复用已有kernel，因为embedding renorm kernel里其实也复用的是pnorm（
paddle.linalg.norm） - 如果不能复用norm，是不是因为inplace的问题，那能否直接在原有
paddle.linalg.norm的基础上改造使得其支持，尽可能复用已有的计算逻辑 paddle.nn.functional.embedding_renorm_需要公开出来不，torch有这个没
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1.2. 现在修改为了利用现有API组装这个算子，inplace 使用 paddle.scatter_
- 用户是可以直接调用
torch.embedding_renorm_的，但是没有doc
python/paddle/nn/functional/input.py
Outdated
| @@ -127,10 +129,33 @@ def one_hot( | |||
| return one_hot_out | |||
|
|
|||
|
|
|||
| @inplace_apis_in_dygraph_only | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个还需要吗，因为组合的API都是动静统一的，是因为组合里面的有纯动态图下的API吗？@AndPuQing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
scatter_ 是 inplace_apis_in_dygraph_only
Paddle/python/paddle/tensor/manipulation.py
Line 4078 in 900d27c
| @inplace_apis_in_dygraph_only |
| @@ -17,6 +17,7 @@ | |||
| import numpy as np | |||
|
|
|||
| import paddle | |||
| from paddle.nn.functional.input import embedding_renorm_ | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这里的静态图有测试吗，静态图可以正常运行吗
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
@AndPuQing 这个还少一个参数scale_grad_by_freq，该任务完成了2/3 |
|
@AndPuQing API英文文档没有修改，请提交一个PR。该PR合入后，请在docs里修改API中文文档、API映射文档。 以上所有文档（英文文档、中文文档、映射文档）合入后，该任务算完成。（计做2/3） |
…ding 增加参数 max_norm/norm_type/scale_grad_by_freq - part (PaddlePaddle#63130) * Add embedding_renorm_ function to functional module * Fix embedding_renorm_ function signature * Add max_norm and norm_type parameters to Embedding class * Remove scale_grad_by_freq parameter from embedding function * remove scale_grad_by_freq parameter * add test case * refactor: use fabs and powf functions * refactor: use api to build embedding_renorm * remove embedding_renorm_ function from __all__ * fix * add static test
PR Category
Operator Mechanism
PR Types
New features
Description
为embedding添加了 max_norm, norm_type 参数，并添加了单测
实现方法：分别实现了CPU、GPU上的EmbeddingRenormKernel，并注册了inplace算子