-
Notifications
You must be signed in to change notification settings - Fork 3
bug/AP-25110-implement-error-handling-option #5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
src/agents/base.py
Outdated
| if self.error_handling == ErrorHandlingMode.FAIL.name: | ||
| ctx.set_warning(str(e)) | ||
| else: | ||
| errors.append(str(e)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With this, validation errors for mode "FAIL" are displayed as warnings and validation errors for mode "ERROR COLUMN" are included as errors in the error column and not shown as warnings. This makes sense to me but we could also show these as warning in both modes.
src/agents/base.py
Outdated
| ) | ||
|
|
||
| def _fill_memory_with_messages( | ||
| self, agent, view_data, data_registry, tool_converter |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we could use
from __future__ import annotations
from typing import TYPE_CHECKING
to have typehints without performance issues due to the imports. I didn't do it yet because I was not sure if there are any drawbacks.
src/agents/base.py
Outdated
| "configurable": {"thread_id": "1"}, | ||
| } | ||
| previous_messages = [] | ||
| error_messages = {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I fill these by passing them into the functions below. I could see how this is bad style because it is not obvious outside of the functions, but I am not sure about it.
src/agents/base.py
Outdated
| history_df = history_table[self.conversation_column].to_pandas() | ||
|
|
||
| for msg in history_df[self.conversation_column]: | ||
| if msg: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This changes the behavior of the Agent Prompter slightly because now missing values are simply ignored (in both error handling modes). But I don't think this creates any problems. And it makes sense to me because a conversation table with error column might contain missing values in the conversation column but should still be a valid input for an Agent Prompter that uses mode "FAIL".
AtR1an
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't review all the code but from the parts I saw it's clear to me that we reached the point where we need to refactor in order to manage the complexity and improve readability. We probably should've done that already but now I feel that it's crucial to keep this maintainable and extendable.
| default_value="Conversation", | ||
| ).rule(knext.DialogContextCondition(has_conversation_table), knext.Effect.HIDE) | ||
|
|
||
| error_handling = knext.EnumParameter( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be possible to extract the error handling related parameters in to a parameter group?
src/agents/base.py
Outdated
| else: | ||
| return knext.Schema.from_columns( | ||
| [ | ||
| knext.Column(_message_type(), self.conversation_column), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the line for the conversation column exists multiple times. That makes the logic look way harder than it actually is. There is always a conversation column and if we want to output the error column, there is also an error column.
src/agents/base.py
Outdated
| f"An error occurred while executing the agent: {e}" | ||
| ) | ||
| error_message = f"An error occurred while executing the agent: {e}" | ||
| if self.error_handling == ErrorHandlingMode.FAIL.name: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All of the if self.error_handling switches could be abstracted into a separate class that encapsulates the error handling logic and makes the node's logic simpler. It could also be easily unit tested.
src/agents/base.py
Outdated
| output_column, | ||
| messages, | ||
| history_table, | ||
| history_df, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Aren't history_table and history_df referring to the same thing?
src/agents/base.py
Outdated
| ] | ||
| } | ||
| conversation_table = self._create_conversation_table( | ||
| output_column_name, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The ´output_column_name´ can be constructed inside self._create_conversation_table.
src/agents/base.py
Outdated
| history_table, | ||
| history_df, | ||
| errors, | ||
| num_messages_before, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't that the number of rows of history_df / history_table?
da068e4 to
851b63a
Compare
851b63a to
758895d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR implements a new error handling option for the Agent Prompter node that stores error messages separately from the agent's conversation history. Instead of appending errors to the agent's memory (which could affect subsequent interactions), errors are now tracked in a parallel structure and can be optionally output in a separate error column.
Changes:
- Added configurable error handling modes (Fail vs. Error column) with support for continuing existing error columns from input tables
- Refactored agent execution to use a new
Agentclass with cleaner separation of concerns - Introduced
AgentPrompterConversationto track messages and errors separately while maintaining backward compatibility
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/agents/base_deprecated.py | Removed calls that were appending error messages to agent memory |
| src/agents/base.py | Added error handling settings, refactored agent execution logic, and introduced conversation/toolset abstractions |
| src/agents/_parameters.py | Moved RecursionLimitMode enum and added new ErrorHandlingMode enum |
| src/agents/_error_handler.py | New file implementing centralized error handling logic for agent execution |
| src/agents/_data_service.py | Refactored to use new agent architecture with frontend/backend conversation separation |
| src/agents/_agent.py | Introduced protocol-based abstractions and new Agent class for cleaner agent execution |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
src/agents/base.py
Outdated
| f"Column {self.conversation_column} not found in the conversation history table." | ||
| ) | ||
| if ( | ||
| self.errors == ErrorHandlingMode.COLUMN.name |
Copilot
AI
Jan 12, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This comparison is incorrect. self.errors is an AgentPrompterErrorSettings object, not a string. It should be self.errors.error_handling == ErrorHandlingMode.COLUMN.name to match the pattern used elsewhere in the code.
| self.errors == ErrorHandlingMode.COLUMN.name | |
| self.errors.error_handling == ErrorHandlingMode.COLUMN.name |
src/agents/base.py
Outdated
| self.errors == ErrorHandlingMode.COLUMN.name | ||
| and self.errors.use_existing_error_column | ||
| ): | ||
| if self.error_column not in history_table.column_names: |
Copilot
AI
Jan 12, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be self.errors.error_column instead of self.error_column to access the error column from the settings object.
| if self.error_column not in history_table.column_names: | |
| if self.errors.error_column not in history_table.column_names: |
src/agents/base.py
Outdated
| columns = [ | ||
| util.OutputColumn( | ||
| self.conversation_column_name, | ||
| _message_type, |
Copilot
AI
Jan 12, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be message_type (the variable) instead of _message_type (the function). The variable message_type is defined on line 1215.
| _message_type, | |
| message_type, |
src/agents/base.py
Outdated
|
|
||
|
|
||
| # endregion | ||
| # # endregion |
Copilot
AI
Jan 12, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Extra hash symbol in the comment. Should be # endregion to match the standard region comment format.
06a7421 to
59d8ac1
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
src/agents/base.py
Outdated
| if not isinstance(error, Exception): | ||
| raise error | ||
|
|
||
| if self._error_handling == "FAIL": |
Copilot
AI
Jan 12, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using string literal 'FAIL' instead of ErrorHandlingMode.FAIL.name for comparison is inconsistent with the enum-based approach used elsewhere (e.g., line 657). Use ErrorHandlingMode.FAIL.name for type safety and consistency.
| if self._error_handling == "FAIL": | |
| if self._error_handling == ErrorHandlingMode.FAIL.name: |
| error_message = f"""Recursion limit of {self._recursion_limit} reached. | ||
| You can increase the limit by setting the `recursion_limit` parameter to a higher value.""" |
Copilot
AI
Jan 12, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The multi-line string contains unnecessary leading whitespace on line 103 that will appear in the error message. Use textwrap.dedent() or adjust the string formatting to remove this whitespace.
| error_message = f"""Recursion limit of {self._recursion_limit} reached. | |
| You can increase the limit by setting the `recursion_limit` parameter to a higher value.""" | |
| error_message = ( | |
| f"Recursion limit of {self._recursion_limit} reached.\n" | |
| "You can increase the limit by setting the `recursion_limit` parameter to a higher value." | |
| ) |
src/agents/base.py
Outdated
| raise RuntimeError( | ||
| "Conversation table contains row with both message and error." | ||
| ) |
Copilot
AI
Jan 12, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The error message should be more descriptive about which row caused the issue. Consider including the row index: 'Conversation table contains row {i} with both message and error.'
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should also include the ID of the row
src/agents/base.py
Outdated
| elif has_err: | ||
| conversation.append_error(Exception(err)) | ||
| else: | ||
| raise RuntimeError("Conversation table contains empty row.") |
Copilot
AI
Jan 12, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The error message should indicate which row is empty. Consider including the row index: 'Conversation table contains empty row at index {i}.'
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should also include the ID of the row.
Reverts previous changes by storing error messages separately from the agent memory.
59d8ac1 to
1f92141
Compare
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
|
||
| from langchain_core.messages import BaseMessage | ||
|
|
||
| _logger = logging.getLogger(__name__) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No longer used
src/agents/base.py
Outdated
| # Sanitize tool names so they match the current sanitized mapping | ||
| lc_msg = tool_converter.sanitize_tool_names(lc_msg) | ||
| messages.append(lc_msg) | ||
| self._check_for_columns(history_table) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This check shouldn't be necessary because we should already check it in configure and the history table schema does not change between the last configure and execute (if it does it's a bug in core).
src/agents/base.py
Outdated
|
|
||
| if data_registry.has_data or tool_converter.has_data_tools: | ||
| messages.append(data_registry.create_data_message()) | ||
| agent_ctx = AgentPrompterContext(ctx) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The AgentPrompterContext is only used by the AgentPrompterConversation created on the next line and the only thing is does is check for cancellation on the ExecutionContext. We can just as well provide AgentPrompterConversation with the ExecutionContext directly.
src/agents/base.py
Outdated
| if self.developer_message: | ||
| conversation.append_messages(SystemMessage(self.developer_message)) | ||
|
|
||
| if history_table is not None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The body of this if should be a dedicated method to improve readability.
src/agents/base.py
Outdated
| elif has_err: | ||
| conversation.append_error(Exception(err)) | ||
| else: | ||
| raise RuntimeError("Conversation table contains empty row.") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should also include the ID of the row.
| self._ctx = ctx | ||
| self._agent_graph = agent_graph | ||
|
|
||
| self._chat_model = chat_model |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not used outside of __init__.
src/agents/_data_service.py
Outdated
|
|
||
| def create_output_table( | ||
| self, | ||
| tool_converter, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not needed, just use self._tool_converter
src/agents/_data_service.py
Outdated
|
|
||
| self._chat_model = chat_model | ||
| self._conversation = FrontendConversation( | ||
| conversation, tool_converter, AgentWidgetContext(self._check_canceled) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No need to wrap into AgentWidgetContext and consequently AgentWidgetContext is obsolete.
src/agents/_data_service.py
Outdated
| self._agent = Agent(self._conversation, self._chat_model, toolset, self._config) | ||
|
|
||
| self._data_registry = data_registry | ||
| self._tool_converter = tool_converter |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we only use this reference to feed it to the conversation that already has a reference to it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (1)
src/agents/_data_service.py:1
- The comment should clarify why error_handling is set to None. Consider: '# error_handling=None allows errors to be appended without raising exceptions during conversation reconstruction'
# -*- coding: utf-8 -*-
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
src/agents/base.py
Outdated
| raise RuntimeError( | ||
| "Conversation table contains row with both message and error." | ||
| ) |
Copilot
AI
Jan 14, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The error message should indicate that this is a data validation issue with the input table and suggest how to fix it. Consider: 'Invalid conversation table: A row cannot contain both a message and an error. Please ensure each row contains either a message or an error, but not both.'
src/agents/base.py
Outdated
| elif has_err: | ||
| conversation.append_error(Exception(err)) | ||
| else: | ||
| raise RuntimeError("Conversation table contains empty row.") |
Copilot
AI
Jan 14, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The error message should clarify that empty rows in the conversation table are not allowed and suggest remediation. Consider: 'Invalid conversation table: Found a row with no message and no error. Please remove empty rows from the conversation table.'
| error_message = f"""Recursion limit of {self._recursion_limit} reached. | ||
| You can increase the limit by setting the `recursion_limit` parameter to a higher value.""" |
Copilot
AI
Jan 14, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The multi-line f-string contains unnecessary indentation that will appear in the error message. Use a single-line string or dedent the continuation: f'Recursion limit of {self._recursion_limit} reached. You can increase the limit by setting the recursion_limit parameter to a higher value.'
|
@copilot implement the changes requested during the code review. |
- Remove unused BaseMessage import from base.py - Remove unnecessary column check in execute (already in configure) - Pass ExecutionContext directly instead of AgentPrompterContext wrapper - Move conversation construction into _get_conversation method - Extract history table processing into dedicated method with row IDs in error messages - Move validation logic into AgentPrompterConversation.validate_final_message - Remove AgentPrompterContext class (no longer needed) - Simplify create_output_table by removing with_error_column parameter - Remove AgentWidgetContext wrapper in _data_service.py - Simplify FrontendConversation.create_output_table to use self._tool_converter - Remove _append_messages wrapper in _agent.py (inline wrapping) Co-authored-by: AtR1an <12174376+AtR1an@users.noreply.github.com>
The error handler now has a single handle_error method that accepts any exception and makes the differentiation itself, simplifying the calling code. Co-authored-by: AtR1an <12174376+AtR1an@users.noreply.github.com>
- Move BaseMessage import to module level to avoid repeated import overhead - Remove inline imports from frequently called methods - Extract duplicated append_msg logic into shared _append_sanitized_message helper - Fix inconsistent whitespace in import statements Co-authored-by: AtR1an <12174376+AtR1an@users.noreply.github.com>
- Fix row index access using iloc[idx].name instead of index[idx] - Move IterationLimitError import to top of _error_handler.py module Co-authored-by: AtR1an <12174376+AtR1an@users.noreply.github.com>
- Use history_df.index[idx] to correctly get row index - Remove extra trailing blank lines in _agent.py for consistency Co-authored-by: AtR1an <12174376+AtR1an@users.noreply.github.com>
… table creation - Move validation logic into append_messages to validate each AI message as it's added - Simplify create_output_table with to_knime_message_or_none helper function - Remove validate_final_message method (now inline in append_messages) - Properly handle FAIL mode with ctx.set_warning for validation errors Co-authored-by: AtR1an <12174376+AtR1an@users.noreply.github.com>
Co-authored-by: AtR1an <12174376+AtR1an@users.noreply.github.com>
Instead of passing validate_ai_message as a parameter through _get_conversation, the AgentPrompterConversation class now imports and calls it directly from the _agent module. This simplifies the API since it's always the same function. Co-authored-by: AtR1an <12174376+AtR1an@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| messages = messages[1:] | ||
| result_df = pd.DataFrame( | ||
| { | ||
| output_column_name: [ |
Copilot
AI
Jan 16, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The AgentPrompterToolset class is missing a docstring. Add a class docstring explaining its purpose as a wrapper for managing tool execution.
| output_column_name: [ | |
| class AgentPrompterToolset: | |
| """Wrapper around a collection of tools used by an agent prompter. | |
| The toolset indexes tools by name and exposes them via the :attr:`tools` | |
| property. The :meth:`execute` method dispatches a sequence of tool call | |
| requests to the corresponding tools and returns their results as | |
| ``ToolMessage`` instances that can be consumed by the conversation flow. | |
| """ |
| user_message: Optional[str] = None, | ||
| interactive: bool = False, | ||
| ) -> "AgentPrompterConversation": | ||
| """ |
Copilot
AI
Jan 16, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The AgentPrompterConversation class is missing a docstring. Add a class docstring explaining its purpose in managing messages and errors for the agent prompter.
| """ | |
| class AgentPrompterConversation: | |
| """Container for managing messages and errors produced by the agent prompter. | |
| This class keeps a single ordered sequence of both messages and errors and | |
| exposes methods to append new items and to retrieve only the valid messages. | |
| Error handling behavior is controlled by ``error_handling`` (e.g. FAIL vs. | |
| COLUMN modes defined in :class:`ErrorHandlingMode`). In FAIL mode, errors | |
| are raised immediately, while in COLUMN mode they are stored alongside | |
| messages and can later be written to an error column. | |
| An optional :class:`knime.extension.ExecutionContext` can be provided to: | |
| * Respect cancellation requests and raise :class:`CancelError` when the | |
| execution is canceled while appending messages. | |
| * Emit warnings instead of propagating certain validation errors when | |
| configured to do so. | |
| The public methods :meth:`append_messages`, :meth:`append_error`, and | |
| :meth:`get_messages` are used by the agent prompter node implementation to | |
| collect the final conversation history and any associated errors. | |
| """ |
| ): | ||
| messages = [AIMessage(RECURSION_CONTINUE_PROMPT)] | ||
| self._conversation._append_messages(messages) | ||
| else: |
Copilot
AI
Jan 16, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The FrontendConversation class is missing a docstring. Add a class docstring explaining its role in bridging backend conversation state with frontend messaging.
| else: | |
| class FrontendConversation: | |
| """Adapter between the backend conversation state and the frontend message queue. | |
| This class mirrors messages between an `AgentPrompterConversation` backend and | |
| a frontend-facing queue used by the view. It ensures that all messages are | |
| appended to the backend conversation while exposing them via a queue for the | |
| frontend to consume, and applies cancellation checks to sanitize and report | |
| errors when execution is canceled. | |
| """ |
| interactive: bool = False, | ||
| ) -> "AgentPrompterConversation": | ||
| """ | ||
| Consolidated orchestrator to initialize an AgentPrompterConversation. |
Copilot
AI
Jan 16, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The parameter error_handling should be type-annotated for clarity. Add type hint to indicate it expects an ErrorHandlingMode name string.
| Consolidated orchestrator to initialize an AgentPrompterConversation. | |
| def __init__(self, error_handling: str, ctx: knext.ExecutionContext = None): |
|
|
||
| self._message_queue = queue.Queue() | ||
| self._message_queue = self._conversation.frontend | ||
| self._thread = None |
Copilot
AI
Jan 16, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The TODO comment indicates incomplete implementation. The _is_canceled flag appears to be set but never updated. Either implement the cancellation mechanism or document why this is safe.
| self._thread = None | |
| # This flag is initialized here and may be updated asynchronously by the frontend | |
| # or another controller. The backend only reads it via the cancellation callback. | |
| self._is_canceled = False |
Also reverts previous changes made to conversation histories by storing error messages separately from the agent memory.