[fp8]Support fp8 deep gemm in paddle #73092

risemeup1 · 2025-06-04T08:05:54Z

PR Category

Execute Infrastructure

PR Types

Others

Description

支持paddle接入fp8 deepgemm

pcard-67164

paddle-bot · 2025-06-04T08:06:00Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

SigureMo · 2025-06-04T13:22:16Z

paddle/fluid/fp8/deep_gemm/include/deep_gemm/fp8_gemm.cuh

+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// The file has been adapted from DeepSeek DeepEP project


Suggested change

// The file has been adapted from DeepSeek DeepEP project

// The file has been adapted from DeepSeek DeepGEMM project

SigureMo · 2025-06-04T13:22:25Z

paddle/fluid/fp8/deep_gemm/include/deep_gemm/fp8_gemm.cuh

+
+// The file has been adapted from DeepSeek DeepEP project
+// Copyright (c) 2025 DeepSeek
+// Licensed under the MIT License - https://github.com/deepseek-ai/DeepEP/blob/main/LICENSE


Suggested change

// Licensed under the MIT License - https://github.com/deepseek-ai/DeepEP/blob/main/LICENSE

// Licensed under the MIT License - https://github.com/deepseek-ai/DeepGEMM/blob/main/LICENSE

后同

SigureMo · 2025-06-04T13:23:17Z

paddle/fluid/fp8/deep_gemm/include/deep_gemm/fp8_gemm.cuh

+                    // TODO: remove some useless computation for unaligned Ms
+                    #pragma unroll
+                    for (uint32_t local_idx = 0; local_idx < BLOCK_M / WAVE_BLOCK_M; ++ local_idx) {
+                      	auto m_offset = local_idx * WAVE_BLOCK_M;


这里怎么用的是 tab？使用空格替换掉 tab

下同

SigureMo · 2025-06-04T13:31:50Z

python/paddle/incubate/fp8/deep_gemm/__init__.py

为什么 fp8 目录没有 __init__.py？这样 fp8 目录不是一个 module，有潜在的 import 问题

另外，fp8 目录只有 deep_gemm 一个子目录是后续给其他模块预留的么？

SigureMo · 2025-06-04T13:42:22Z

python/paddle/incubate/fp8/deep_gemm/jit_kernels/tuner.py

+        # keys = {k: keys[k] for k in sorted(keys.keys())}
+        # signature = (name, f"{keys}")
+        # if signature in self.tuned:
+        #     return self.tuned[signature]


无用代码删掉

SigureMo · 2025-06-04T13:48:29Z

test/fp8/test_fp8_deep_gemm.py

+    # x_np = np.full((num_groups, m, k), 3)
+    # y_np = np.full((num_groups, n, k), 2)
+    # x=paddle.to_tensor(x_np).astype(paddle.bfloat16)
+    # y=paddle.to_tensor(y_np).astype(paddle.bfloat16)


无用注释删掉

SigureMo · 2025-06-04T13:49:05Z

test/fp8/test_fp8_deep_gemm.py

+    test_gemm()
+    test_m_grouped_gemm_contiguous()
+    test_m_grouped_gemm_masked()


改成 unittest 的形式

后续再改

SigureMo · 2025-06-04T13:52:43Z

python/paddle/incubate/__init__.py

+        'gemm_fp8_fp8_bf16_nt',
+        'm_grouped_gemm_fp8_fp8_bf16_nt_contiguous',
+        'm_grouped_gemm_fp8_fp8_bf16_nt_masked',
+    ]


确认下，这些是公开 API 么？这些如果是公开 API，那就需要为每个 API

完善类型提示

完善文档

另外，如果是公开 API，目前这里也是不可行的，因为编译文档的环境这里应该不满足 cuda_version >= 12.9 and 90 in paddle.version.cuda_archs() 吧？这会导致官网无法渲染相关文档

SigureMo · 2025-06-04T13:54:31Z

python/paddle/incubate/__init__.py

+            m_grouped_gemm_fp8_fp8_bf16_nt_contiguous,
+            m_grouped_gemm_fp8_fp8_bf16_nt_masked,
+        )
+except ImportError:


这个 ImportError 在什么时候会出现？

SigureMo

LGTMeow for subprocess and print usage

Hongqing-work

LGTM for changes in DeepEP.

zhiqiu

LGTM

risemeup1 · 2025-06-05T09:50:05Z

/re-run approval

risemeup1 added 6 commits June 3, 2025 16:44

update cutlass

cedb638

update cutlass to 3.0

1758208

update cutlass

afa1b6a

fix

6c6c18c

fix

cc7e2fb

fix

e2d90d9

risemeup1 requested review from sneaxiy and ForFishes as code owners June 4, 2025 08:05

risemeup1 force-pushed the support_fp8_deep_gemm_in_paddle branch 2 times, most recently from 2dabc5f to a5c4aea Compare June 4, 2025 08:48

risemeup1 changed the title ~~Support fp8 deep gemm in paddle~~ [fp8]Support fp8 deep gemm in paddle Jun 4, 2025

risemeup1 force-pushed the support_fp8_deep_gemm_in_paddle branch from a5c4aea to f618e85 Compare June 4, 2025 13:00

SigureMo reviewed Jun 4, 2025

View reviewed changes

risemeup1 force-pushed the support_fp8_deep_gemm_in_paddle branch 2 times, most recently from cb0cabb to 11654d3 Compare June 4, 2025 18:15

support deep gemm

03c2430

risemeup1 force-pushed the support_fp8_deep_gemm_in_paddle branch from 11654d3 to 03c2430 Compare June 5, 2025 02:17

SigureMo approved these changes Jun 5, 2025

View reviewed changes

Hongqing-work approved these changes Jun 5, 2025

View reviewed changes

zhiqiu approved these changes Jun 5, 2025

View reviewed changes

zhangbo9674 approved these changes Jun 5, 2025

View reviewed changes

risemeup1 added the skip-ci: approval label Jun 5, 2025

risemeup1 merged commit e964106 into PaddlePaddle:develop Jun 5, 2025
65 of 73 checks passed

SigureMo mentioned this pull request Jun 8, 2025

[CodeStyle] Format all .cuh files #73179

Merged

	// The file has been adapted from DeepSeek DeepEP project
	// The file has been adapted from DeepSeek DeepGEMM project

	// Licensed under the MIT License - https://github.com/deepseek-ai/DeepEP/blob/main/LICENSE
	// Licensed under the MIT License - https://github.com/deepseek-ai/DeepGEMM/blob/main/LICENSE

[fp8]Support fp8 deep gemm in paddle #73092

[fp8]Support fp8 deep gemm in paddle #73092

Uh oh!

Conversation

risemeup1 commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Category

PR Types

Description

Uh oh!

paddle-bot bot commented Jun 4, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SigureMo left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Hongqing-work left a comment

Choose a reason for hiding this comment

Uh oh!

zhiqiu left a comment

Choose a reason for hiding this comment

Uh oh!

risemeup1 commented Jun 5, 2025

Uh oh!

Uh oh!

Uh oh!

risemeup1 commented Jun 4, 2025 •

edited

Loading

SigureMo left a comment •

edited

Loading