Skip to content

feat(engine)!: execute loop and iteration in engine frames#197

Draft
laipz8200 wants to merge 9 commits into
mainfrom
laipz8200/frame-aware-execution
Draft

feat(engine)!: execute loop and iteration in engine frames#197
laipz8200 wants to merge 9 commits into
mainfrom
laipz8200/frame-aware-execution

Conversation

@laipz8200

@laipz8200 laipz8200 commented Jun 28, 2026

Copy link
Copy Markdown
Member

Important

  1. Make sure you have read our contribution guidelines
  2. Search existing issues and pull requests to confirm this change is not a duplicate
  3. Open or identify the issue this pull request resolves or advances
  4. Use a Conventional Commits title for this pull request, and mark breaking changes with !
  5. Remember that the pull request title will become the squash merge commit message
  6. If CLA Assistant prompts you, sign CLA.md in the pull request conversation

Related Issue

close #198

Summary

  • Executes loop and iteration containers inside GraphEngine frames instead of nested engines.
  • Adds frame-aware ready tasks, event routing, and child frame materialization for container bodies.
  • Makes DSL loading understand loop and iteration container node types.

Breaking Changes

  • LoopNode and IterationNode no longer execute child graphs from their own _run() methods. Their bodies are now interpreted by GraphEngine through frame-aware execution.
  • Runtime state no longer exposes the child-engine execution path used by the old loop and iteration implementation.
  • Engine-internal task routing now carries frame_id with node_id. Custom ready queue, dispatcher, worker, or event-handler integrations must preserve the frame when scheduling or handling node work.
  • Container child graphs are materialized by rebinding the graph node factory to a frame-local GraphRuntimeState. Custom node factories used with loop or iteration graphs must provide with_runtime_state(graph_runtime_state).
  • NodeExecution.execution_id is now required when the domain model is created, and NodeExecution.mark_started() no longer accepts an execution id.
  • Nodes no longer create or restore execution ids through ensure_execution_id; the method has been removed. The worker binds the frame-scoped execution id before calling node.run().

Downstream Adaptation

  • Run complete graphs through GraphEngine.run() or graphon.dsl.loads(...).run() instead of calling LoopNode.run() or IterationNode.run() directly.
  • Treat emitted graph events as the public execution record. Child node events inside containers still surface through the engine event stream and carry in_loop_id or in_iteration_id.
  • If you implement a custom ready queue, store and return ReadyTask(frame_id, node_id) values. Use root for normal top-level work.
  • If you implement a custom node factory and need loop or iteration support, add a with_runtime_state() method that returns an equivalent factory bound to the supplied runtime state.
  • If you instantiate NodeExecution directly, provide execution_id up front. Prefer GraphExecution.get_or_create_node_execution() when possible.
  • Replace node_execution.mark_started(execution_id) with node_execution.mark_started().
  • Replace direct node.ensure_execution_id() usage with worker-style binding: get the frame-scoped node execution, call node.bind_execution_id(node_execution.execution_id), then call node.run().

Checklist

  • This pull request links the issue it resolves or advances
  • This pull request title follows Conventional Commits, and any breaking change is marked with !
  • If CLA Assistant prompted me, I signed CLA.md in the pull request conversation

Move loop and iteration execution into the main graph engine frame model.

Add frame-aware task routing and event handling so container bodies run without creating nested engines.

Extend DSL loading for container node types and add workflow event scenarios for complete graphs.
Rename ReadyQueueProtocol to ReadyQueue and update direct imports.
Add field comments for loop and iteration run contexts.
Keep nested iteration outputs when flatten_output is disabled.
@laipz8200 laipz8200 changed the title refactor(engine)!: execute loop and iteration in engine frames feat(engine)!: execute loop and iteration in engine frames Jun 28, 2026
Route LoopNode and IterationNode through the normal BaseNode.run() generator path. Add explicit start/resume ready tasks, WorkerPool-owned suspended invocation storage, and semantic container await/result models so child-frame execution can complete without occupying a worker thread.
Avoid firing GraphEngineLayer.on_node_run_end when a cooperative container node yields an await request. The hook now runs only when the invocation actually completes after resume.
Protect ContainerExecution bookkeeping with a reentrant lock because worker threads start container awaits while the dispatcher completes child frames. Add a regression test that blocks complete_frame and verifies concurrent start_container_await waits.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature proposal: execute subgraphs as engine frames

1 participant