[SOT][Faster Guard] implement basic guard tree mechanism #70154

zrr1999 · 2024-12-11T15:00:52Z

PR Category

Execute Infrastructure

PR Types

Performance

Description

实现了基本的 guard tree 机制，GuardTree 目前是list[list[node]]，没有实现树结构

目前实现的节点：

ConstantExprNode：常量表达式节点
LocalVarExprNode：局部变量表达式节点
GlobalVarExprNode：全局变量表达式节点
AttributeExprNode：属性访问表达式节点
ItemExprNode：索引访问表达式节点

下面的行为是后续的计划，目前还没有实现
ENV_SOT_ENABLE_STRICT_GUARD_CHECK 记做 S, ENV_SOT_ENABLE_GUARD_TREE 记做C1，ENV_SOT_ENABLE_FASTER_GUARD 记做C2

S	C1	C2	执行结果
True	任意值	True	默认 FasterGuard，同时跑其他 Guard
True	任意值	False	默认 PyGuard，同时跑其他 Guard
False	True	任意值	仅 GuardTree
False	False	False	仅 FasterGuard
False	False	False	仅 PyGuard

paddle-bot · 2024-12-11T15:00:57Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

paddle-ci-bot · 2024-12-20T03:09:50Z

Sorry to inform you that e54a6b3's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

Copilot

Copilot reviewed 5 out of 10 changed files in this pull request and generated no comments.

Files not reviewed (5)

paddle/fluid/pybind/jit.cc: Language not supported
paddle/fluid/pybind/sot/guards.cc: Language not supported
paddle/fluid/pybind/sot/guards.h: Language not supported
python/paddle/jit/sot/opcode_translator/executor/tracker.py: Evaluated as low risk
python/paddle/jit/sot/opcode_translator/executor/variables/base.py: Evaluated as low risk

Comments suppressed due to low confidence (3)

python/paddle/jit/sot/opcode_translator/executor/executor_cache.py:135

[nitpick] The function name 'analyse_guard_global_object' is ambiguous. Consider renaming it to 'analyze_global_objects_in_guard' for clarity.

def analyse_guard_global_object(self, guard_fn):

python/paddle/jit/sot/opcode_translator/executor/variables/basic.py:456

[nitpick] The method name 'make_faster_guard' is ambiguous. It should be renamed to something more descriptive, such as 'create_faster_guard_nodes'.

def make_faster_guard(self) -> list[paddle.framework.core.GuardNode]:

python/paddle/jit/sot/opcode_translator/executor/variables/basic.py:457

The error message 'Only support PIR' is unclear. It should be more descriptive, such as 'PIR API must be enabled to use faster guard nodes.'

assert paddle.framework.use_pir_api(), "Only support PIR"

paddle-ci-bot · 2025-02-20T03:04:40Z

Sorry to inform you that 5a86b32's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

Copilot

Pull Request Overview

This PR implements the basic guard tree mechanism to improve performance in guard checks by introducing faster guard support. Key changes include:

New APIs for creating and checking faster guard trees (e.g. make_faster_guard).
Extensions to tracker, executor, and variable modules to support guard tree expressions.
Updates to integrate the faster guard mechanism into the function graph and executor cache.

Reviewed Changes

Copilot reviewed 8 out of 11 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/sot/test_guard_tree.py	Added tests for the guard tree mechanism with a callback that verifies guard results.
python/paddle/jit/sot/opcode_translator/executor/guard.py	Introduced make_faster_guard and updated guard checks.
python/paddle/jit/sot/opcode_translator/executor/executor_cache.py	Modified cache lookup logic based on ENV_SOT_ENABLE_GUARD_TREE.
python/paddle/jit/sot/opcode_translator/executor/tracker.py	Added guard_tree_expr_node implementations for various trackers and marked several expression nodes as TODO.
python/paddle/jit/sot/opcode_translator/executor/function_graph.py	Updated guard_fn property to use faster guard nodes when enabled.
python/paddle/jit/sot/opcode_translator/executor/variables/basic.py	Added a check_faster_guard-decorated implementation of make_faster_guard.
python/paddle/jit/sot/opcode_translator/executor/opcode_inline_executor.py	Provided guard_tree_expr_node implementations and noted TODO for FunctionClosureExprNode.
python/paddle/jit/sot/opcode_translator/executor/variables/base.py	Extended variable base with a faster guard method decorated for consistency.

Files not reviewed (3)

paddle/fluid/pybind/jit.cc: Language not supported
paddle/fluid/pybind/sot/guards.cc: Language not supported
paddle/fluid/pybind/sot/guards.h: Language not supported

Comments suppressed due to low confidence (1)

python/paddle/jit/sot/opcode_translator/executor/executor_cache.py:161

[nitpick] The conditional check on ENV_SOT_ENABLE_GUARD_TREE may lead to confusion given the accompanying TODO. It is recommended to refactor this logic into a clearer structure once faster guard tree support for error analysis is complete.

elif not ENV_SOT_ENABLE_GUARD_TREE.get():

test/sot/test_guard_tree.py

Copilot · 2025-03-27T08:37:35Z

python/paddle/jit/sot/opcode_translator/executor/tracker.py

+        # TODO(zrr1999): implement BinaryExprNode
+        raise NotImplementedError("BinaryExprNode is not implemented")
+        # left_expr = self.operands[0].tracker.guard_tree_expr_node()
+        # right_expr = self.operands[1].tracker.guard_tree_expr_node()
+        # return paddle.framework.core.BinaryExprNode(
+        #     left_expr,
+        #     right_expr,
+        #     self.get_operator_symbol(),
+        # )
+


BinaryExprNode is not implemented, which will cause runtime errors for binary operations. Implement BinaryExprNode or provide an alternative to handle binary operators.

Suggested change

# TODO(zrr1999): implement BinaryExprNode

raise NotImplementedError("BinaryExprNode is not implemented")

# left_expr = self.operands[0].tracker.guard_tree_expr_node()

# right_expr = self.operands[1].tracker.guard_tree_expr_node()

# return paddle.framework.core.BinaryExprNode(

# left_expr,

# right_expr,

# self.get_operator_symbol(),

# )

left_expr = self.operands[0].tracker.guard_tree_expr_node()

right_expr = self.operands[1].tracker.guard_tree_expr_node()

return paddle.framework.core.BinaryExprNode(

left_expr,

right_expr,

self.get_operator_symbol(),

)

Copilot · 2025-03-27T08:37:35Z

python/paddle/jit/sot/opcode_translator/executor/tracker.py

+        # TODO(zrr1999): implement IterExprNode
+        raise NotImplementedError("IterExprNode is not implemented")
+


IterExprNode is not implemented. Ensure that this is implemented before iterators in the guard tree are used to prevent unexpected runtime errors.

Suggested change

# TODO(zrr1999): implement IterExprNode

raise NotImplementedError("IterExprNode is not implemented")

return IterExprNode(self.iter_source.guard_tree_expr_node())

Copilot · 2025-03-27T08:37:36Z

python/paddle/jit/sot/opcode_translator/executor/tracker.py

+        # TODO(zrr1999): implement LayerExprNode
+        raise NotImplementedError("LayerExprNode is not implemented")
+


LayerExprNode is not implemented, which may lead to runtime errors when processing layer attributes. It is recommended to implement LayerExprNode to properly support layer tracing.

Suggested change

# TODO(zrr1999): implement LayerExprNode

raise NotImplementedError("LayerExprNode is not implemented")

class LayerExprNode(paddle.framework.core.ExprNode):

def __init__(self, layer_class, args, kwargs):

super().__init__()

self.layer_class = layer_class

self.args = args

self.kwargs = kwargs

def __repr__(self):

return f"LayerExprNode(layer_class={self.layer_class}, args={self.args}, kwargs={self.kwargs})"

return LayerExprNode(self.layer_class, self.args, self.kwargs)

and more guard_tree_expr_node

gouzil · 2025-03-28T06:49:13Z

paddle/fluid/pybind/jit.cc

@@ -123,6 +125,81 @@ void BindGuard(pybind11::module *m) {
 #endif
 }

+void BindGuardTree(pybind11::module *m) {
+#if SOT_IS_SUPPORTED


这个 SOT_IS_SUPPORTED 应该要包住整个 BindGuardTree 函数吧

前面几个都没有啊

SigureMo · 2025-03-29T07:33:00Z

paddle/fluid/pybind/sot/guards.cc

  return PyDict_GetItemString(frame->frame->f_locals, var_name_.c_str());
+#else
+  return PyDict_GetItemString(frame->f_locals, var_name_.c_str());


如果比较多的话，可以后续考虑加一个宏 PyFrameProxy_GET_FRAME

目前看起来应该只有这两个

…e#70154) * add GuardTree * add guard_tree_expr_node * add make_faster_guard and more guard_tree_expr_node * add test * format * support 3.11- * fix

paddle-bot bot added the contributor External developers label Dec 11, 2024

zrr1999 changed the title ~~[SOT][Faster Guard] add ENV_SOT_ENABLE_GUARD_TREE~~ [SOT][Faster Guard] implement basic guard tree mechanism Dec 11, 2024

zrr1999 force-pushed the guard_tree branch from e54a6b3 to 7ae36b8 Compare February 3, 2025 10:01

zrr1999 requested a review from Copilot February 3, 2025 10:02

Copilot AI reviewed Feb 3, 2025

View reviewed changes

zrr1999 marked this pull request as ready for review February 4, 2025 01:45

zrr1999 requested review from SigureMo and gouzil as code owners February 4, 2025 01:45

zrr1999 force-pushed the guard_tree branch 2 times, most recently from 89caff2 to 5a86b32 Compare February 5, 2025 10:31

zrr1999 force-pushed the guard_tree branch 2 times, most recently from a551d2e to 49a5ad1 Compare March 27, 2025 08:35

zrr1999 requested a review from Copilot March 27, 2025 08:35

Copilot AI reviewed Mar 27, 2025

View reviewed changes

zrr1999 force-pushed the guard_tree branch from 05f8d02 to d3b922b Compare March 27, 2025 14:07

zrr1999 added 4 commits March 27, 2025 14:08

add GuardTree

22ac62a

add guard_tree_expr_node

7fe83ee

add make_faster_guard

aa9807b

and more guard_tree_expr_node

add test

84756aa

zrr1999 force-pushed the guard_tree branch from d3b922b to 51262e3 Compare March 28, 2025 06:33

format

51262e3

gouzil reviewed Mar 28, 2025

View reviewed changes

zrr1999 force-pushed the guard_tree branch from 95b4bd2 to 51262e3 Compare March 29, 2025 05:33

zrr1999 changed the title ~~[SOT][Faster Guard] implement basic guard tree mechanism~~ [SOT][Faster Guard][3.13] implement basic guard tree mechanism Mar 29, 2025

SigureMo changed the title ~~[SOT][Faster Guard][3.13] implement basic guard tree mechanism~~ [SOT][Faster Guard][3.13] implement basic guard tree mechanism for Python 3.11- Mar 29, 2025

SigureMo changed the title ~~[SOT][Faster Guard][3.13] implement basic guard tree mechanism for Python 3.11-~~ [SOT][Faster Guard][3.13] implement basic guard tree mechanism for Python 3.11+ Mar 29, 2025

zrr1999 force-pushed the guard_tree branch from dd63105 to 7171651 Compare March 29, 2025 06:45

zrr1999 force-pushed the guard_tree branch from 7171651 to 98c366a Compare March 29, 2025 07:25

support 3.11-

98c366a

zrr1999 changed the title ~~[SOT][Faster Guard][3.13] implement basic guard tree mechanism for Python 3.11+~~ [SOT][Faster Guard][3.13] implement basic guard tree mechanism Mar 29, 2025

SigureMo previously approved these changes Mar 29, 2025

View reviewed changes

zrr1999 dismissed SigureMo’s stale review via c27d116 March 29, 2025 07:57

fix

c27d116

zrr1999 requested a review from SigureMo March 31, 2025 07:17

SigureMo approved these changes Mar 31, 2025

View reviewed changes

SigureMo changed the title ~~[SOT][Faster Guard][3.13] implement basic guard tree mechanism~~ [SOT][Faster Guard] implement basic guard tree mechanism Mar 31, 2025

zrr1999 merged commit e275cc4 into PaddlePaddle:develop Mar 31, 2025
34 checks passed

zrr1999 mentioned this pull request Apr 13, 2025

[WeeklyReports] 2025.3.24~2025.4.6 周报汇总 PFCCLab/Camp#459

Open

12 tasks

zrr1999 mentioned this pull request Apr 24, 2025

SOT Guard 机制性能优化 #69264

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[SOT][Faster Guard] implement basic guard tree mechanism #70154

[SOT][Faster Guard] implement basic guard tree mechanism #70154

Uh oh!

zrr1999 commented Dec 11, 2024 •

edited

Loading

Uh oh!

paddle-bot bot commented Dec 11, 2024

Uh oh!

paddle-ci-bot bot commented Dec 20, 2024

Uh oh!

Copilot AI left a comment

Uh oh!

paddle-ci-bot bot commented Feb 20, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI Mar 27, 2025

Uh oh!

Copilot AI Mar 27, 2025

Uh oh!

Copilot AI Mar 27, 2025

Uh oh!

gouzil Mar 28, 2025

Uh oh!

zrr1999 Mar 28, 2025

Uh oh!

SigureMo Mar 29, 2025

Uh oh!

zrr1999 Mar 29, 2025

Uh oh!

Uh oh!

Uh oh!

		# TODO(zrr1999): implement IterExprNode
		raise NotImplementedError("IterExprNode is not implemented")

	# TODO(zrr1999): implement IterExprNode
	raise NotImplementedError("IterExprNode is not implemented")
	return IterExprNode(self.iter_source.guard_tree_expr_node())

		# TODO(zrr1999): implement LayerExprNode
		raise NotImplementedError("LayerExprNode is not implemented")

-        # TODO(zrr1999): implement LayerExprNode
-        raise NotImplementedError("LayerExprNode is not implemented")
+        class LayerExprNode(paddle.framework.core.ExprNode):
+            def __init__(self, layer_class, args, kwargs):
+                super().__init__()
+                self.layer_class = layer_class
+                self.args = args
+                self.kwargs = kwargs
+            def __repr__(self):
+                return f"LayerExprNode(layer_class={self.layer_class}, args={self.args}, kwargs={self.kwargs})"
+        return LayerExprNode(self.layer_class, self.args, self.kwargs)

[SOT][Faster Guard] implement basic guard tree mechanism #70154

[SOT][Faster Guard] implement basic guard tree mechanism #70154

Uh oh!

Conversation

zrr1999 commented Dec 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Category

PR Types

Description

Uh oh!

paddle-bot bot commented Dec 11, 2024

Uh oh!

paddle-ci-bot bot commented Dec 20, 2024

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

paddle-ci-bot bot commented Feb 20, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Copilot AI Mar 27, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 27, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 27, 2025

Choose a reason for hiding this comment

Uh oh!

gouzil Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

zrr1999 Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

SigureMo Mar 29, 2025

Choose a reason for hiding this comment

Uh oh!

zrr1999 Mar 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

zrr1999 commented Dec 11, 2024 •

edited

Loading