feat(claude): surface per-turn duration, tokens, and cost from ResultMessage#47
feat(claude): surface per-turn duration, tokens, and cost from ResultMessage#47herikwebb wants to merge 7 commits into
Conversation
…Message The Claude Agent SDK ends every turn with a ResultMessage carrying duration_ms, token usage, and total_cost_usd. The receive loop dropped it in its silent else branch, so users had no signal of what a turn cost short of typing /cost. Format it as a small italic footer (e.g. '12.3s · 45.2K in (38.1K cached) / 1.2K out · $0.0842') streamed after the turn's content. Error results produce no footer (the error path already reports), zero cost is omitted (subscription sessions report 0.0 and showing $0.0000 would read as a billing claim), and malformed usage values degrade to whatever segments remain rather than failing the turn.
Automated PR ReviewModel: Changed files: Review result: FindingsMedium —
|
Review follow-up (PR #47): the footer is streamed as another MarkdownData chunk right after the assistant's last text block; without a leading paragraph break it concatenated onto prose that doesn't end in a blank line ('Done.*12.3s ...*'), breaking the emphasis markup. Prefix the footer with a blank line and pin it with a test.
Automated PR ReviewModel: Changed files: Review result: No automated findings were found in the changed code. Residual risks: this depends on the exact |
The footer streamed after every turn is persistent visual noise, and its cost figure comes from the SDK's total_cost_usd, which the CLI prices from public list rates — wrong for subscription logins (marginal cost is $0), enterprise-negotiated rates, and non-first-party endpoints. Add a 'show_turn_usage' claude_setting (default off) surfaced as a Settings-panel toggle, and only emit the footer when it is on. Gate the $ segment on a direct API key being configured so the duration/token counts (always accurate) still show while a dollar amount we can't stand behind is omitted.
Reworked to opt-in (d45cad0)Manual testing surfaced two issues with an always-on footer, addressed here:
Changes:
Manually verified: default → no footer; toggle on + API key → |
Automated PR ReviewModel: Changed files: Review result: Automated PR Review FindingsMedium —
|
The cost gate only checked for a direct API key, but a user with a custom base_url (proxy, gateway, Bedrock/Vertex front, OpenAI-compatible endpoint) plus a key would still see Claude public-list-price costs that don't match their actual billing — the very case the footer's cost suppression is meant to avoid. Extract _should_show_turn_cost, requiring both a direct API key and Anthropic's own endpoint (unset or api.anthropic.com base_url); a custom base_url now suppresses the dollar figure while duration/tokens still render. Add regression tests covering the base_url cases.
Addressed the base_url cost finding (f26b771)Agreed — the gate ignored
The earlier paragraph-break note is already covered on this branch (footer is prefixed with |
Automated PR ReviewModel: Changed files: Review result: FindingsMedium —
|
…st settings The cost gate read api_key/base_url only from claude_settings, but NBI overlays ANTHROPIC_API_KEY/ANTHROPIC_BASE_URL onto the CLI subprocess env merged over the server's os.environ — so a blank setting falls through to an ambient value the CLI actually uses. An env-provided custom base_url would then show Anthropic list-price cost for a proxy/gateway endpoint. Add _resolve_effective_credential mirroring that precedence (setting wins, else env) and route _should_show_turn_cost through it, so the gate honors env-provided keys and endpoints. Add env-resolution regression tests and an autouse fixture keeping the suite hermetic.
Round 2 findings (f1e28d9)Medium — env-resolved credentials/endpoint — fixedGood catch. NBI overlays
Low — stale
|
Automated PR ReviewModel: Changed files: Review result: No automated findings were found. Residual risks: the new usage footer is covered at the formatter/cost-gating helper level, but the actual Claude SDK receive-loop/UI rendering path is not exercised by the added tests, so regressions there would require integration or manual validation. |
Drop the billing caveats from the settings toggle title — those belong in the docs, not the settings UI. Keep a one-line description.
Automated PR ReviewModel: Changed files: Review result: No automated findings were found in the changed code. Residual risks: the usage footer behavior depends on the exact |
Explain the Show usage after each turn toggle and when the cost figure is shown vs omitted (direct API key on the default endpoint only), matching the setting's trimmed UI description.
Automated PR ReviewModel: Changed files: Review result: No automated findings were found. Residual risks include compatibility with the exact |
|
Raised upstream as plmbr#391; closing this fork PR. Review history retained here. |
1 participant
Problem
The Claude Agent SDK ends every turn with a
ResultMessagecarryingduration_ms, token usage, andtotal_cost_usd. The receive loop dropped it in its silentelsebranch, so users had no signal of what a turn cost short of typing/cost.Change
format_result_usagehelper renders a small italic footer streamed after the turn's content, e.g.*12.3s · 45.2K in (38.1K cached) / 1.2K out · $0.0842*.ResultMessageinstances.Testing
Full Python suite passes (1300 tests).