fix(llm): pin claude-cli output to a JSON schema when supported (#2076)#2095
Closed
Yyunozor wants to merge 1 commit into
Closed
fix(llm): pin claude-cli output to a JSON schema when supported (#2076)#2095Yyunozor wants to merge 1 commit into
Yyunozor wants to merge 1 commit into
Conversation
…hify-Labs#2076) The claude-cli backend delivers the extraction schema in the user turn and trusts the model to emit raw JSON. Newer Claude Code releases treat that prompt as an agentic task and report the result in prose instead ("Knowledge graph extracted — 21 nodes, 20 edges…"), so the graph parses empty, reads as truncation, and adaptive-retry bisects without ever converging. Pass --json-schema (structured output) when the CLI advertises it — probed once via `claude --help` and cached — so the object shape is constrained regardless of prompt framing. Older CLIs that predate the flag keep the user-turn prompt as a fallback. The `result` envelope still carries the JSON string, so the parse path is unchanged.
safishamsi
added a commit
that referenced
this pull request
Jul 22, 2026
Contributor
Author
wojiucece
added a commit
to wojiucece/graphify
that referenced
this pull request
Jul 23, 2026
上游 0.9.25 主要变更： - relicense MIT -> Apache-2.0（保留 LICENSE-MIT 双 license 兼容） - Graphify-Labs#2112 删 .graphifyinclude 死代码 (detect) - Graphify-Labs#2106 敏感文件过滤误删 topic 文档修复 (detect) - Graphify-Labs#2102 dedup 顺序无关 gap-fill + same-source 属性保留 - Graphify-Labs#2095 claude-cli 结构化输出解析 (llm) - Graphify-Labs#2082 别名 import 解析 calls 边 (extract) - Graphify-Labs#2094 explain 高度节点分组断连 + 确定性 tie-break (cli) - XAML .cs 扫描 hang 修复 (extract) 冲突解决（仅 pyproject.toml 1 处）： - version: 0.9.24+fork.1 -> 0.9.25+fork.1（保留 fork 标识） - license: 跟随上游 Apache-2.0（license-files=LICENSE/LICENSE-MIT/NOTICE） - requires: setuptools>=68 -> >=77（取上游） Auto-merging 验证（0.9.24 经验：自动合并不等于语义正确）： - cli.py 自动合并通过，explain 实测正常(248 connections)，UserPromptSubmit 端点完整 - 8 核心文件 py_compile 全通过 - grep 确认 24 处 fork 定制标记完整（PreToolUse禁用/UserPromptSubmit启用/ OpenCode before+after+run+failopen/hub floor:30/codegraph防抖降级重试） - check-custom.sh EXIT 0 - serve.py import OK（hub 改动后可加载） 注：本次上游未触及 serve.py/install.py/watch.py（fork 三大定制文件安全）， 仅 cli.py 一处 Auto-merging 需语义验证。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
claude-clibackend puts the extraction schema in the user turn and trusts the model to emit raw JSON (thellm.py:1376workaround, "verified against Claude Code 2.1.197"). On newer Claude Code the model treats that prompt as an agentic task and reports the result in prose instead —Knowledge graph extracted — 21 nodes, 20 edges…. That parses to zero nodes,_response_is_hollowreads truncation, and adaptive-retry bisects the chunk without ever converging, so the graph comes out empty (#2076).Why this is version-robust, and how it was checked
We run Claude Code as this backend's harness, so the failing component is directly testable here. On 2.1.207 the current user-turn workaround still returns JSON — the empty-graph break does not reproduce there, so the regression is version-gated, consistent with the 2.1.216 report. But the failure mode is easy to force and to fix on a live CLI: a prompt that invites prose makes
claude -p --output-format jsonput a conversational summary inresult, while adding--json-schemamakes the same call return the object regardless of framing (also surfaced in astructured_outputenvelope field). Structured output is a structural guarantee independent of the model's discretion, so it survives the prompt-behaviour drift that broke the prompt-shape workaround.Fix
Pass
--json-schema— a schema pinning the top-level{nodes, edges}shape graphify consumes, item internals left loose so valid extractions are never rejected — when the CLI advertises the flag, probed once viaclaude --helpand cached. Older CLIs that predate the flag keep the user-turn prompt as a fallback, unchanged. Theresultenvelope still carries the JSON string, so the_parse_llm_jsonpath is untouched.The issue's second suggestion — routing a successful-sounding prose response away from the bisect path — is complementary and left out of scope; with structured output the prose case no longer arises on supported CLIs.
Validation
tests/test_claude_cli_backend.py: 28 passed, 5 new (flag present when supported, absent when not, the--helpcapability probe, safe fallback on probe error). Full suite: 3317 passed on the branch vs 3312 on the parente32c9f4, same environment — delta is exactly these 5 tests. (13 failures in unrelated suites — terraform/ollama/manifest/install-references — are pre-existing, identical on the parent, from optional deps absent locally.)ruff checkclean.