PR Category

Auto Parallel

PR Types

Improvements

Description

• upgrade matmul_grad inferspmd to support shard optimizer stage3.
  • In sharding stage3 , the parameter is sharded both dp&mp dimensions。If the parameter is the 'y' input of matmul op, it will call c_allgather in dp dimension for execute matmul op. In old version, the matmul grad op will execute in all replicated。 Therefore, we upgrade thre inferspmd function of the matmul_grad op to avoid this situation.
• improve the shard accumulator in stage3.
  • If the paramater not attach mesh attribue in dygraph, we can't shard it in stage3. In this case, we will only shard the moment for this parameter.

Other

Pcard-67164