【Hackathon 9th Sprint No.86】feat: implement AgentUnittestGenerator for both paddle/torch run_model.py -part #422

Kozmosa · 2025-12-06T15:47:29Z

Summary

Added built-in decorator AgentUnittestGenerator to both run_model entrypoints (run_model.py, run_model.py) to generate standalone unittest scripts from model.py + meta files.
Use jinja2.Template for codegen templating (placeholders: target device and dummy/numpy flags);
Provided smoke script to exercise the decorator on one CV and one NLP Torch sample (test_agent_unittest_generator.sh); scripts output <sample>_test.py beside each sample.

How to Use

Prepare a decorator config (base64 JSON) pointing to the built-in class:

{
  "decorator_class_name": "AgentUnittestGenerator",
  "decorator_config": {
    "model_path": "/abs/path/to/sample",
    "force_device": "auto",      // cpu|cuda|auto (Torch); cpu|gpu|auto (Paddle)
    "output_path": null,         // default: <sample>/<sample_name>_test.py
    "use_dummy_inputs": false    // Torch only
    // Paddle: use_numpy flag is supported instead
  }
}

Invoke run_model:
- Torch: python -m graph_net.torch.run_model --model-path /abs/path/to/sample --decorator-config <base64>
- Paddle: python -m graph_net.paddle.run_model --model-path /abs/path/to/sample --decorator-config <base64>
The generated unittest is self-contained and runnable: python <sample_name>_test.py.

Validation (manual)

CV (Torch): sample resnet18 → generated resnet18_test.py runs on CPU.
NLP (Torch): sample albert-base-v2 → generated albert-base-v2_test.py runs on CPU.
Paddle side: generation logic mirrors Torch; not explicitly run in this check.

Notes

New dependency: jinja2 (required to render templates).
Backward compatibility: _get_decorator still accepts both argparse.Namespace and dict configs.

Sample Artifact

Generated Torch unittest example: albert-base-v2_test.py (created via the decorator).

(graphnet) kozmosa@dev_kozmosa:~/code/GraphNet$ bash graph_net/test/test_agent
_unittest_generator.sh
[AgentTest] running CV (torchvision/resnet18) sample at /home/kozmosa/code/GraphNet/samples/torchvision/resnet18
model_path='/home/kozmosa/code/GraphNet/samples/torchvision/resnet18'
[Agent] unittest 已生成: /home/kozmosa/code/GraphNet/samples/torchvision/resnet18/resnet18_test.py (device=cuda)
use_dummy_inputs=False
[AgentTest] running NLP (transformers-auto-model/albert-base-v2) sample at /home/kozmosa/code/GraphNet/samples/transformers-auto-model/albert-base-v2
model_path='/home/kozmosa/code/GraphNet/samples/transformers-auto-model/albert-base-v2'
[Agent] unittest 已生成: /home/kozmosa/code/GraphNet/samples/transformers-auto-model/albert-base-v2/albert-base-v2_test.py (device=cuda)
use_dummy_inputs=False
[AgentTest] done. Generated *_test.py files should now exist beside the samples.

(graphnet) kozmosa@dev_kozmosa:~/code/GraphNet$ python /home/kozmosa/code/Graph
Net/samples/torchvision/resnet18/resnet18_test.py
.
----------------------------------------------------------------------
Ran 1 test in 6.119s

OK

(graphnet) kozmosa@dev_kozmosa:~/code/GraphNet$ python /home/kozmosa/code/GraphNet/samples/transformers-auto-model/albert-base-v2/albert-base-v2_test.py
.
----------------------------------------------------------------------
Ran 1 test in 2.871s

OK

resnet18_test.py

import importlib.util
import os
import tempfile
import unittest

import torch
from graph_net.torch import utils


def _load_graph_module(model_path: str, target_device: str):
    source_path = os.path.join(model_path, "model.py")
    with open(source_path, "r", encoding="utf-8") as f:
        code = f.read()

    if target_device != "cuda":
        code = utils.modify_code_by_device(code, target_device)

    tmp_dir = tempfile.mkdtemp(prefix="agent_unittest_")
    tmp_file = os.path.join(tmp_dir, "model_tmp.py")
    with open(tmp_file, "w", encoding="utf-8") as f:
        f.write(code)

    spec = importlib.util.spec_from_file_location("agent_graph_module", tmp_file)
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)
    return module.GraphModule


class AgentGraphTest(unittest.TestCase):
    def setUp(self):
        self.model_path = os.path.dirname(__file__)
        self.target_device = "cuda"
        self.GraphModule = _load_graph_module(self.model_path, self.target_device)
        self.meta = utils.load_converted_from_text(self.model_path)
        self.use_dummy_inputs = False

    def _with_device(self, info):
        cloned = {"info": dict(info["info"]), "data": info.get("data")}
        cloned["info"]["device"] = self.target_device
        return cloned

    def test_forward_runs(self):
        model = self.GraphModule()
        weight_info = self.meta["weight_info"]

        def _build_tensor(val):
            wrapped = self._with_device(val)
            return (
                utils.get_dummy_tensor(wrapped)
                if self.use_dummy_inputs
                else utils.replay_tensor(wrapped)
            )

        state_dict = {k: _build_tensor(v) for k, v in weight_info.items()}
        model.__graph_net_file_path__ = self.model_path
        output = model(**state_dict)
        self.assertIsNotNone(output)


if __name__ == "__main__":
    unittest.main()

albert-base-v2_test.py

import importlib.util
import os
import tempfile
import unittest

import torch
from graph_net.torch import utils


def _load_graph_module(model_path: str, target_device: str):
    source_path = os.path.join(model_path, "model.py")
    with open(source_path, "r", encoding="utf-8") as f:
        code = f.read()

    if target_device != "cuda":
        code = utils.modify_code_by_device(code, target_device)

    tmp_dir = tempfile.mkdtemp(prefix="agent_unittest_")
    tmp_file = os.path.join(tmp_dir, "model_tmp.py")
    with open(tmp_file, "w", encoding="utf-8") as f:
        f.write(code)

    spec = importlib.util.spec_from_file_location("agent_graph_module", tmp_file)
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)
    return module.GraphModule


class AgentGraphTest(unittest.TestCase):
    def setUp(self):
        self.model_path = os.path.dirname(__file__)
        self.target_device = "cuda"
        self.GraphModule = _load_graph_module(self.model_path, self.target_device)
        self.meta = utils.load_converted_from_text(self.model_path)
        self.use_dummy_inputs = False

    def _with_device(self, info):
        cloned = {"info": dict(info["info"]), "data": info.get("data")}
        cloned["info"]["device"] = self.target_device
        return cloned

    def test_forward_runs(self):
        model = self.GraphModule()
        weight_info = self.meta["weight_info"]

        def _build_tensor(val):
            wrapped = self._with_device(val)
            return (
                utils.get_dummy_tensor(wrapped)
                if self.use_dummy_inputs
                else utils.replay_tensor(wrapped)
            )

        state_dict = {k: _build_tensor(v) for k, v in weight_info.items()}
        model.__graph_net_file_path__ = self.model_path
        output = model(**state_dict)
        self.assertIsNotNone(output)


if __name__ == "__main__":
    unittest.main()

CLAassistant · 2025-12-06T15:47:35Z

All committers have signed the CLA.

paddle-bot · 2025-12-06T15:47:36Z

Thanks for your contribution!

…e and torch.

Xreki · 2025-12-07T03:05:02Z

收到您的PR，感谢您的贡献。
在次跟您解释下，这个题目需要生成一个不依赖graph_net环境和样本本身的单测，即这个单测拿到任何一个只有python和torch的环境，都可以直接执行。

JewelRoam · 2025-12-08T03:57:35Z

整体还是很清晰的，PR说明很详细（赞），有几个小改进点：

拆分出独立组件，通过config中"handler_path"/"handler_class_name"(model_path_handler)或"decorator_path"(run_model)指定，尽可能不要修改入口脚本（model_path_handler/run_model）
我们目前逐步过渡到使用graph_net.model_path_handler代替run_model，需要辛苦入口脚本尽可能选用model_path_handler（参数配置类似，修改成本应该不高）
参照上面Xreki提到的，单测中不依赖graph_net，不应该需要import graph_net utils
与子图生成流程对齐，支持在config中指定"output_dir"，设置单测保存位置
使用简洁的英文注释，不要使用中文

graph_net/paddle/run_model.py

lixinqi · 2025-12-12T08:04:57Z

graph_net/paddle/run_model.py

+import unittest
+
+import paddle
+from graph_net.paddle import utils


这是不可以的。单测脚本应该能脱离graph_net单独工作。

已经按照建议进行修改，最新的提交中，生成的单侧脚本不再依赖graph_net

lixinqi · 2025-12-12T08:05:52Z

graph_net/test/test_agent_unittest_generator.sh

+encode_cfg() {
+  MODEL_PATH="$1" python - <<'PY'
+import base64, json, os
+cfg = {
+    "decorator_class_name": "AgentUnittestGenerator",
+    "decorator_config": {
+        "model_path": os.environ["MODEL_PATH"],
+        "force_device": "auto",
+        "output_path": None,
+        "use_dummy_inputs": False,
+    },
+}
+print(base64.b64encode(json.dumps(cfg).encode()).decode())
+PY
+}


可以参考https://github.com/PaddlePaddle/GraphNet/blob/develop/graph_net/bash_templates/model_path_handler_sh.txt 简化这里的逻辑

已经按照建议进行修改了

… AgentUnittestGenerator, adopt jinja2 as render engine

Kozmosa · 2025-12-16T03:27:43Z

使用precommit检查并修复了代码风格问题

Xreki · 2025-12-17T01:30:01Z

graph_net/torch/run_model.py

+    if not decorator_config:
+        return lambda model: model
+
+    class_name = decorator_config.get("decorator_class_name", "RunModelDecorator")


run_model机制要求固定类型名RunModelDecorator和GraphModule，model_path_handler可以指定类型名，建议使用model_path_handler来运行，不要这么修改run_model。

Xreki · 2025-12-17T01:32:57Z

graph_net/torch/sample_passes/agent_unittest_generator.py

+import torch
+
+
+def _get_classes(file_path: str):


实际上我们需要将样本model.py、xxx_meta.py的内容直接写到这个生成的单测文件里面，最后达到的效果是，你把这个单测文件给我我就直接能跑。

paddle-bot bot added the contributor External developers label Dec 6, 2025

feat: add AgentUnittestGenerator class for run_model.py in both paddl…

b2a820e

…e and torch.

Kozmosa force-pushed the develop branch from 076e07c to b2a820e Compare December 6, 2025 15:53

luotao1 mentioned this pull request Dec 8, 2025

【Hackathon 9th】开源贡献个人挑战赛冲刺赛 PaddlePaddle/Paddle#76333

Open

lixinqi reviewed Dec 12, 2025

View reviewed changes

Merge branch 'PaddlePaddle:develop' into develop

675631e

luotao1 changed the title ~~【Hackathon 9th Sprint No.87】feat: implement AgentUnittestGenerator for both paddle/torch run_model.py~~ 【Hackathon 9th Sprint No.86】feat: implement AgentUnittestGenerator for both paddle/torch run_model.py Dec 15, 2025

Kozmosa added 2 commits December 15, 2025 17:32

feat: refactor code to improve style, use a seperate method to inject…

05c4cd3

… AgentUnittestGenerator, adopt jinja2 as render engine

style: fix code style issues to pass pre-commit check

7eb8a16

Xreki reviewed Dec 17, 2025

View reviewed changes

chore: revert run_model.py

ed394e4

Xreki approved these changes Dec 17, 2025

View reviewed changes

Xreki merged commit 3af85cf into PaddlePaddle:develop Dec 17, 2025
3 checks passed

luotao1 changed the title ~~【Hackathon 9th Sprint No.86】feat: implement AgentUnittestGenerator for both paddle/torch run_model.py~~ 【Hackathon 9th Sprint No.86】feat: implement AgentUnittestGenerator for both paddle/torch run_model.py -part Dec 17, 2025

【Hackathon 9th Sprint No.86】feat: implement AgentUnittestGenerator for both paddle/torch run_model.py -part #422

【Hackathon 9th Sprint No.86】feat: implement AgentUnittestGenerator for both paddle/torch run_model.py -part #422

Conversation

Kozmosa commented Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CLAassistant commented Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paddle-bot bot commented Dec 6, 2025

Uh oh!

Xreki commented Dec 7, 2025

Uh oh!

JewelRoam commented Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lixinqi Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Kozmosa Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

lixinqi Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Kozmosa Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Kozmosa commented Dec 16, 2025

Uh oh!

Xreki Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Xreki Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Kozmosa commented Dec 6, 2025 •

edited

Loading

CLAassistant commented Dec 6, 2025 •

edited

Loading

JewelRoam commented Dec 8, 2025 •

edited

Loading