| CARVIEW |
Navigation Menu
-
Notifications
You must be signed in to change notification settings - Fork 5.8k
[XPU] Add rms_norm and rms_norm_grad op #63989
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
你的PR提交成功，感谢你对开源项目的贡献! |
|
Sorry to inform you that 8aade47's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually. |
|
Sorry to inform you that d7554df's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually. |
cqulilujia
force-pushed
the
rmsnorm
branch
8 times, most recently
from
0266647 to
829c231
Compare
cqulilujia
force-pushed
the
rmsnorm
branch
4 times, most recently
from
6c0b398 to
5bdfd9d
Compare
cqulilujia
force-pushed
the
rmsnorm
branch
2 times, most recently
from
f24a5b8 to
e66362f
Compare
cqulilujia
force-pushed
the
rmsnorm
branch
2 times, most recently
from
4618bc6 to
bbaa41a
Compare
| paddle.autograd.backward([out], [out_g], True) | ||
| return out, (x.grad, scale.grad) | ||
|
|
||
| # dtypes = [paddle.float32, paddle.bfloat16, paddle.float16] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bf16数据类型，单测拼的小算子kl2/kl3都有一些小问题，待api更新后再添加进来
| #pragma once | ||
|
|
||
| #include "paddle/phi/core/dense_tensor.h" | ||
| #include "paddle/phi/core/selected_rows.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个头文件应该没用到
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
好的，下个PR我一起删掉
| @@ -13,6 +13,7 @@ See the License for the specific language governing permissions and | |||
| limitations under the License. */ | |||
|
|
|||
| #include "paddle/phi/infermeta/backward.h" | |||
| #include "glog/logging.h" | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个头文件用到了吗？
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
调试时加上的，后续我来删掉
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
从外部使用者的角度来看，新增算子绑定，和现有的走fast_paddle，有啥区别吗？
一方面，这个算子是PaddleNLP中的算子，GPU角度看，这个算子在PaddleNLP中已经被移到legacy文件夹，由PaddleNLP迁向paddle是一个趋势；另一方面，从XPU角度，fast paddle的地位需要逐渐弱化，让用户尽量不修改GPU代码就能使用，能迁移的尽量迁移，对外部使用者来说使用Paddle中的算子更为友好 |
PaddleNLP模型侧的代码修改可以参考 PaddlePaddle/PaddleNLP#8746 |
|
LGTM |
PR Category
Custom Device
PR Types
New features
Description
添加XPU rms_norm和rms_norm_grad融合算子，与GPU保持一致