[SOT][3.12] Eval frame compat for Python 3.12 #61272

gouzil · 2024-01-29T04:34:10Z

PR types

Others

PR changes

Others

Description

尝试适配 sot Python3.12 （还没在ci打开单测）

SigureMo · 2024-01-29T04:47:06Z

paddle/fluid/pybind/cpython_internals.c

+#if PY_VERSION_HEX >= 0x030c0000
+#else


Suggested change

#if PY_VERSION_HEX >= 0x030c0000

#else

#if PY_VERSION_HEX < 0x030c0000

SigureMo · 2024-01-29T04:48:13Z

paddle/fluid/pybind/eval_frame.c

@@ -176,7 +176,11 @@ inline static PyObject *eval_custom_code_py311_plus(PyThreadState *tstate,
  PyFunctionObject *func =
      (PyFunctionObject *)PyFunction_New((PyObject *)code, frame->f_globals);
  Py_INCREF(func);
-#if PY_VERSION_HEX < 0x030c0000
+#if PY_VERSION_HEX >= 0x030c0000
+  Py_XINCREF(((PyFunctionObject *)frame->f_funcobj)->func_closure);


这里确定要强转才可以么？原来类型不是这个么？

frame->f_funcobj 已经是 PyObject 类型了，或许可以直接写Py_XINCREF(frame->f_funcobj);

SigureMo · 2024-01-29T04:48:45Z

paddle/fluid/pybind/eval_frame.c

+#if PY_VERSION_HEX >= 0x030c0000
+  Py_XINCREF(((PyFunctionObject *)frame->f_funcobj)->func_closure);
+  func->func_closure = ((PyFunctionObject *)frame->f_funcobj)->func_closure;
+  _PyFrame_Initialize(shadow, func, NULL, code, 0);


最后一个参数确定是 0 么？

在初始化 shadow 的时候，希望把所有 localsplus 初始化为 NULL ，所以他是0，_PyFrame_Initialize 会在函数内部根据 code->co_nlocalsplus 的大小逐个赋 NULL

SigureMo

这两个文件下一个 PR 修改

SigureMo · 2024-01-29T04:59:35Z

python/paddle/jit/sot/opcode_translator/executor/pycode_generator.py

@@ -878,7 +878,8 @@ def gen_unpack_sequence(self, count):

    def gen_call_function(self, argc=0):
        if sys.version_info >= (3, 11):
-            self._add_instr("PRECALL", arg=argc, argval=argc)
+            if sys.version_info == (3, 11):


Suggested change

if sys.version_info == (3, 11):

if sys.version_info >= (3, 11) and sys.version_info < (3, 12):

Python 3.11.6 (main, Oct 2 2023, 13:45:54) [Clang 15.0.0 (clang-1500.0.40.1)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.version_info sys.version_info(major=3, minor=11, micro=6, releaselevel='final', serial=0) >>> sys.version_info == (3, 11) False >>> (3, 11) <= sys.version_info < (3, 12) True >>> sys.version_info >= (3, 11) and sys.version_info < (3, 12) True >>>

虽然 (3, 11) <= sys.version_info < (3, 12) 看起来更好些，但静态分析工具不支持这种 pattern

SigureMo · 2024-01-29T05:01:02Z

python/paddle/jit/sot/opcode_translator/executor/opcode_executor.py

+        if sys.version_info >= (3, 12) and instr.arg & 1:
+            attr_name_var = ConstantVariable.wrap_literal(
+                attr_name, self._graph
+            )
+            obj = self.stack.pop()
+
+            method = BuiltinVariable(
+                getattr, graph=self._graph, tracker=DanglingTracker()
+            )(obj, attr_name_var)
+
+            if isinstance(method, MethodVariable) and "__getattr__" not in dir(
+                method.bound_instance.get_py_type()
+            ):
+                # bound method or the class override the __getattr__
+                # push the unbound method and the self
+                self.stack.push(method.fn)
+                self.stack.push(obj)
+            else:
+                # unbound method, push the dummy and the function
+                self.stack.push(NullVariable())
+                self.stack.push(method)
+            return


这是 LOAD_METHOD 是吧，可以看看怎么复用，比如抽出一个新的函数 load_method

SigureMo · 2024-01-29T05:01:39Z

点错了，咋直接 approve 了

SigureMo · 2024-01-29T05:55:07Z

test/dygraph_to_static/test_eval_frame.py

        ):
-            # print("skip test_eval_frame, current only support 3.8 - 3.10")
+            # print("skip test_eval_frame, current only support 3.8 - 3.13")


这行直接删掉

（不然还触发 print 检查）

SigureMo · 2024-01-29T06:05:52Z

test/dygraph_to_static/test_eval_frame.py

@@ -29,9 +29,9 @@ def tearDown(self):

    def test_eval_frame(self):
        if version_info.major != 3 or (
-            version_info.minor <= 8 or version_info.minor >= 12
+            version_info.minor <= 8 or version_info.minor >= 13


原来写的 <= 8 不对吧？

统一用

if not (sys.version_info >= (3, 8) and sys.version_info < (3, 13)):

这种 pattern 吧

SigureMo

LGTMeow

gouzil added 4 commits January 27, 2024 21:51

open eval_frame and clean pycode_generator.py

5c91c62

fix describe

7173317

Merge branch 'develop' of github.com:gouzil/Paddle into eval_frame_py312

5ace8d5

supper cpython_internals, LOAD_ATTR

1a1ea85

gouzil requested review from SigureMo and removed request for SigureMo January 29, 2024 04:37

gouzil assigned SigureMo Jan 29, 2024

add test

1ce3ed8

SigureMo reviewed Jan 29, 2024

View reviewed changes

SigureMo previously approved these changes Jan 29, 2024

View reviewed changes

RollBACK opcode_translator

f2335d7

gouzil dismissed SigureMo’s stale review via f2335d7 January 29, 2024 05:13

fix cpython_internals

70c2a40

SigureMo reviewed Jan 29, 2024

View reviewed changes

gouzil added 2 commits January 29, 2024 13:56

rm print

d9a83e0

fix

de3a29f

SigureMo reviewed Jan 29, 2024

View reviewed changes

fix test_eval_frame

b49dd25

SigureMo changed the title ~~[SOT] try support Python3.12~~ [SOT][3.12] Eval frame compat for Python 3.12 Jan 29, 2024

SigureMo approved these changes Jan 29, 2024

View reviewed changes

gouzil merged commit 72d8acd into PaddlePaddle:develop Jan 29, 2024

SigureMo mentioned this pull request Feb 20, 2024

SOT Python3.12 支持任务汇总 #61173

Closed

gouzil deleted the eval_frame_py312 branch April 23, 2024 11:50

gouzil mentioned this pull request Jun 11, 2024

WAVE SUMMIT+2024上半年飞桨开源之星评选-信息征集 PaddlePaddle/community#892

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[SOT][3.12] Eval frame compat for Python 3.12 #61272

[SOT][3.12] Eval frame compat for Python 3.12 #61272

Uh oh!

gouzil commented Jan 29, 2024 •

edited

Loading

Uh oh!

SigureMo Jan 29, 2024

Uh oh!

SigureMo Jan 29, 2024

Uh oh!

gouzil Jan 29, 2024

Uh oh!

SigureMo Jan 29, 2024

Uh oh!

gouzil Jan 29, 2024 •

edited

Loading

Uh oh!

SigureMo left a comment

Uh oh!

SigureMo Jan 29, 2024

Uh oh!

SigureMo Jan 29, 2024

Uh oh!

SigureMo commented Jan 29, 2024

Uh oh!

SigureMo Jan 29, 2024

Uh oh!

SigureMo Jan 29, 2024

Uh oh!

SigureMo left a comment

Uh oh!

Uh oh!

	#if PY_VERSION_HEX >= 0x030c0000
	#else
	#if PY_VERSION_HEX < 0x030c0000

	if sys.version_info == (3, 11):
	if sys.version_info >= (3, 11) and sys.version_info < (3, 12):

[SOT][3.12] Eval frame compat for Python 3.12 #61272

[SOT][3.12] Eval frame compat for Python 3.12 #61272

Uh oh!

Conversation

gouzil commented Jan 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR types

PR changes

Description

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gouzil Jan 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SigureMo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SigureMo commented Jan 29, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SigureMo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gouzil commented Jan 29, 2024 •

edited

Loading

gouzil Jan 29, 2024 •

edited

Loading