CARVIEW |
Navigation Menu
-
Notifications
You must be signed in to change notification settings - Fork 5.8k
在KaimingNormal/Uniform中增加mode参数 #73347
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
modified: python/paddle/nn/initializer/kaiming.py modified: test/legacy_test/test_initializer.py
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## develop #73347 +/- ##
===========================================
Coverage ? 100.00%
===========================================
Files ? 1
Lines ? 9
Branches ? 0
===========================================
Hits ? 9
Misses ? 0
Partials ? 0 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
docs: PaddlePaddle/docs#7324 |
) | ||
if self._mode == 'fan_out' and self._fan_in is not None: | ||
raise ValueError( | ||
"The mode of MSRAInitializer is 'fan_out', " |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
KaimingNormal/KaimingUniform
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
嗯,收到,已修改
modified: python/paddle/nn/initializer/kaiming.py
modified: python/paddle/nn/initializer/kaiming.py modified: test/legacy_test/test_initializer.py
@Qin-sx CI可能随机挂了,注意rerun下 |
嗯,收到,已重跑 |
@@ -62,6 +62,7 @@ class MSRAInitializer(Initializer): | |||
seed (int32, optional): random seed. Default is 0. | |||
negative_slope (float, optional): negative_slope (only used with leaky_relu). Default is 0.0. | |||
nonlinearity(str, optional): the non-linear function. Default is relu. | |||
mode(str, optional): the mode of initialization, can be 'fan_in' or 'fan_out'. When set to 'fan_in', the fan_in parameter is used for initialization. When set to 'fan_out', the out_features of trainable Tensor will be used. Default is 'fan_in'. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个逻辑是不是得这么改一下,如果设置了 fan_in,则fan_in生效,mode不生效。如果没设置 fan_in,则mode生效。
fan_in与fan_out不可同时设置,给出明确的报错提示。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
嗯,我想的逻辑是以mode参数为判断标准,mode='fan_in'时,fan_in参数生效。mode='fan_out'时,fan_in参数不生效。感觉以fan_in参数为判断标准,逻辑不太好解释。
if self._mode == 'fan_in':
fan_in = f_in if self._fan_in is None else self._fan_in
if self._mode == 'fan_out':
fan_in = f_out
以mode参数为判断标准的报错逻辑在这里
if self._mode not in ['fan_in', 'fan_out']:
raise ValueError(
"The mode of KaimingNormal/KaimingUniform should be 'fan_in' or 'fan_out', "
f"but received {self._mode}."
)
if self._mode == 'fan_out' and self._fan_in is not None:
raise ValueError(
"The mode of KaimingNormal/KaimingUniform is 'fan_out', "
"but fan_in is set. Please set fan_in to None."
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK,这样更合理
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
PR Category
User Experience
PR Types
Improvements
Description
当前问题
paddle.nn.initializer.KaimingNormal目前只fan_in模式,不支持fan_out模式,fan_out需要自己计算out_feature值后填入fan_in参数
解决方案
paddle.nn.initializer.KaimingNormal中增加一个mode参数: