test: improve some tests + retry others

[anakin87]

Related Issues

I analyzed test failures on main from November 1st and I found out that some integration tests are responsible of most failures. So I am trying to improve them.

Proposed Changes:

• OpenAI test_run_async_cancellation_integration: use a faster model to make sure that generation already started when we cancel the task
• OpenAI test_live_run_with_agent_streaming_and_reasoning: define the function used here at the module level, not inside the test function. Otherwise it frequently fails with "Serialization of nested functions is not supported"
• OpenAI test_live_run_with_toolset: relax the requirements for the "city" value: sometimes the model produces "Paris, France" insted of "Paris" only
• Whisper remote, OpenAPI and LinkContentFetcher: mark tests as flaky to retry them. In these cases, failures are often due to network problems.

How did you test it?

CI, multiple local runs

Checklist

• I have read the contributors guidelines and the code of conduct.
• I have updated the related issue with new insights and changes.
• I have added unit tests and updated the docstrings.
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I have documented my code.
• I have added a release note file, following the contributors guidelines.
• I have run pre-commit hooks and fixed any issue.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Preview	Comments	Updated (UTC)
haystack-docs	Ignored			Dec 12, 2025 3:41pm

[sjrl]

Thanks!