Skip to content

executor, planner: improve plan replayer dump URL output (#69439)#69554

Open
ti-chi-bot wants to merge 2 commits into
pingcap:release-nextgen-202603from
ti-chi-bot:cherry-pick-69439-to-release-nextgen-202603
Open

executor, planner: improve plan replayer dump URL output (#69439)#69554
ti-chi-bot wants to merge 2 commits into
pingcap:release-nextgen-202603from
ti-chi-bot:cherry-pick-69439-to-release-nextgen-202603

Conversation

@ti-chi-bot

@ti-chi-bot ti-chi-bot commented Jul 1, 2026

Copy link
Copy Markdown
Member

This is an automated cherry-pick of #69439

What problem does this PR solve?

Issue Number: close #69438

Problem Summary:

In TiDB Cloud, PLAN REPLAYER DUMP may return a presigned download URL, but the old result only showed it as File_token. Users were not told that the URL can be opened directly, used with curl, or that it expires after 1 hour.

What changed and how does it work?

This PR changes the PLAN REPLAYER DUMP result to a two-column Item / Value output.

When the dump token is an HTTP(S) presigned URL, TiDB returns:

  • the download URL
  • the expiration duration
  • browser usage guidance
  • a curl command
  • a note to rerun PLAN REPLAYER DUMP after expiration

When the token is not a URL, for example local storage, TiDB still returns the generated file token in the new table format. The @@tidb_last_plan_replayer_token session variable keeps its original raw token value.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Unit test commands:

PATH=/usr/local/go/bin:/home/zhouzemin/.local/go/bin:$PATH ./tools/check/failpoint-go-test.sh pkg/executor/test/planreplayer -run 'Test(ExplainExploreReplayer|PlanReplayerDump(PresignedURLOutput|Single|Multiple))$' -count=1
PATH=/usr/local/go/bin:/home/zhouzemin/.local/go/bin:$PATH ./tools/check/failpoint-go-test.sh pkg/server/handler/optimizor -run TestPlanReplayerLoadWithSemicolonInColumnComment -count=1
PATH=/usr/local/go/bin:/home/zhouzemin/.local/bin:/usr/bin:/bin:$PATH GOENV=off GOROOT= make lint

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

Change the `PLAN REPLAYER DUMP` result from a single `File_token` column to a two-column `Item`/`Value` output. When the dump is stored in remote object storage and returns a presigned download URL, the result now shows the download URL, its validity window, and usage guidance. This is a breaking change for all deployments (including on-prem): tooling that reads the dump result by column position or by the `File_token` column name should switch to the `@@tidb_last_plan_replayer_token` session variable, which still returns the raw token unchanged.

Summary by CodeRabbit

  • New Features

    • Enhanced plan replayer dump explain output to distinguish raw file tokens from presigned download URLs, showing download, expiration, browser guidance, and a curl command when applicable.
  • Bug Fixes

    • Updated historical-stats plan replayer output to use an Item/Value layout (instead of a single token column) for clearer results.
  • Tests

    • Expanded and refactored plan replayer tests to validate both file-token and presigned-URL outputs more directly and reliably.

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/planner SIG: Planner size/L Denotes a PR that changes 100-499 lines, ignoring generated files. type/cherry-pick-for-release-nextgen-202603 labels Jul 1, 2026
@ti-chi-bot

Copy link
Copy Markdown
Member Author

@zeminzhou This PR has conflicts, I have hold it.
Please resolve them or ask others to resolve them, then comment /unhold to remove the hold label.

@ti-chi-bot

ti-chi-bot Bot commented Jul 1, 2026

Copy link
Copy Markdown

@ti-chi-bot: ## If you want to know how to resolve it, please read the guide in TiDB Dev Guide.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot

ti-chi-bot Bot commented Jul 1, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign fzzf678, time-and-fate for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Jul 1, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 88ad60f5-828b-4fc9-9b9f-20af7e7cc0f7

📥 Commits

Reviewing files that changed from the base of the PR and between 2162350 and fe797eb.

📒 Files selected for processing (4)
  • pkg/executor/plan_replayer.go
  • pkg/executor/test/planreplayer/BUILD.bazel
  • pkg/executor/test/planreplayer/plan_replayer_test.go
  • pkg/server/handler/optimizor/plan_replayer_test.go
💤 Files with no reviewable changes (2)
  • pkg/executor/plan_replayer.go
  • pkg/executor/test/planreplayer/plan_replayer_test.go

📝 Walkthrough

Walkthrough

This PR adds a named presign expiration constant, reformats PLAN REPLAYER DUMP output to show download guidance when a presigned URL is returned, updates the historical-stats result schema to two columns, and refactors executor and handler tests to use shared file-token extraction helpers.

Changes

Plan Replayer Presigned URL Output

Layer / File(s) Summary
Presign expiration constant
pkg/domain/plan_replayer_dump.go
Adds exported PlanReplayerPresignExpire constant (time.Hour) and uses it in getPresignedURL instead of a hardcoded duration.
Dump output formatting and schema
pkg/executor/plan_replayer.go, pkg/planner/core/planbuilder.go
Adds a helper that detects HTTP(S) presigned URLs and emits download guidance rows (URL, expiry, browser, curl, note) versus a plain "File token" row; changes the historical-stats result schema from one File_token column to Item/Value columns.
Executor test helpers and presigned URL coverage
pkg/executor/test/planreplayer/plan_re_player_test.go, pkg/executor/test/planreplayer/BUILD.bazel
Adds a fake presign storage stub and requirePlanReplayerFileToken helper, updates existing dump tests to use it, adds presigned URL output coverage, and updates Bazel test deps/shard count.
Handler test helper refactors
pkg/server/handler/optimizor/plan_replayer_test.go
Adds requirePlanReplayerFileTokenFromRows, requirePlanReplayerFileTokenFromResult, and requireSingleStringFromRows helpers; refactors multiple tests and data-prep functions to use them instead of manual row scanning/casting.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

  • pingcap/tidb#69439: Modifies the same plan replayer dump output path and related presign-handling behavior.

Suggested reviewers: qw4990, D3Hunter, winoros

Poem

A hop, a URL, a token set free,
No more guessing what the link might be.
Curl it, click it, download with cheer,
One hour ticks — but a new one's near!
🐇✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly matches the main change: improving plan replayer dump output for executor and planner code paths.
Description check ✅ Passed The description follows the template well, includes the issue, problem summary, tests, side effects, and release note.
Linked Issues check ✅ Passed The PR implements the requested clearer Item/Value output, presigned URL guidance, and expiration notice from #69438.
Out of Scope Changes check ✅ Passed The test and BUILD updates appear necessary to support the new dump output and presigned URL handling, with no obvious unrelated changes.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
pkg/server/handler/optimizor/plan_replayer_test.go (1)

87-114: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Assert that the helper consumed the only expected row.

These helpers close *sql.Rows right after the first scan. Since PLAN REPLAYER DUMP now has both one-row and multi-row result shapes, the local-token tests will still pass if that path accidentally starts returning extra rows. Add an EOF check before closing so the helper keeps enforcing the single-row contract.

Suggested change
 func requirePlanReplayerFileTokenFromRows(t *testing.T, rows *sql.Rows) string {
 	require.True(t, rows.Next(), "unexpected data")
 	var item, filename string
 	require.NoError(t, rows.Scan(&item, &filename))
 	require.Equal(t, "File token", item)
 	require.NotEmpty(t, filename)
+	require.False(t, rows.Next(), "unexpected extra row")
+	require.NoError(t, rows.Err())
 	require.NoError(t, rows.Close())
 	return filename
 }
@@
 func requireSingleStringFromRows(t *testing.T, rows *sql.Rows) string {
 	require.True(t, rows.Next(), "unexpected data")
 	var value string
 	require.NoError(t, rows.Scan(&value))
+	require.False(t, rows.Next(), "unexpected extra row")
+	require.NoError(t, rows.Err())
 	require.NoError(t, rows.Close())
 	return value
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/server/handler/optimizor/plan_replayer_test.go` around lines 87 - 114,
The row-checking helpers in requirePlanReplayerFileTokenFromRows,
requirePlanReplayerFileTokenFromResult, and requireSingleStringFromRows only
verify the first scanned row and then close, so they do not enforce the expected
single-row contract. Update these helpers to assert that no additional rows
remain after the first scan by checking for EOF/false on the next row before
closing, keeping the tests strict for PLAN REPLAYER DUMP’s local-token path even
if extra rows are returned.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/executor/plan_replayer.go`:
- Around line 24-28: Resolve the leftover merge-conflict markers in the import
block of plan_replayer.go so the file parses again. Remove the <<<<<<< / =======
/ >>>>>>> markers and keep the intended imports from the PlanReplayer-related
code path, specifically the net/url and os additions if they are required by the
surrounding plan replayer logic. Verify the import section is a valid single
block with no conflict artifacts.

In `@pkg/executor/test/planreplayer/BUILD.bazel`:
- Around line 11-15: Resolve the merge conflict in the go_test target for the
planreplayer BUILD rule by removing the conflict markers and keeping a single
shard_count assignment in the relevant build stanza. Use the existing go_test
target in BUILD.bazel to locate the conflict, choose the intended shard_count
value, and ensure the file is valid Bazel syntax with no leftover HEAD or commit
markers.

In `@pkg/executor/test/planreplayer/plan_replayer_test.go`:
- Around line 250-315: Resolve the unresolved merge conflict in
plan_replayer_test.go by removing the conflict markers and keeping the intended
test additions. Ensure both TestExplainExploreReplayer and
TestPlanReplayerDumpPresignedURLOutput are cleanly integrated into the test file
with no leftover HEAD/branch markers so the package compiles and the new plan
replayer tests run.

---

Nitpick comments:
In `@pkg/server/handler/optimizor/plan_replayer_test.go`:
- Around line 87-114: The row-checking helpers in
requirePlanReplayerFileTokenFromRows, requirePlanReplayerFileTokenFromResult,
and requireSingleStringFromRows only verify the first scanned row and then
close, so they do not enforce the expected single-row contract. Update these
helpers to assert that no additional rows remain after the first scan by
checking for EOF/false on the next row before closing, keeping the tests strict
for PLAN REPLAYER DUMP’s local-token path even if extra rows are returned.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: f783a14a-c0d1-4ebe-9c09-a3e18d9a72ba

📥 Commits

Reviewing files that changed from the base of the PR and between dd1be2d and 2162350.

📒 Files selected for processing (6)
  • pkg/domain/plan_replayer_dump.go
  • pkg/executor/plan_replayer.go
  • pkg/executor/test/planreplayer/BUILD.bazel
  • pkg/executor/test/planreplayer/plan_replayer_test.go
  • pkg/planner/core/planbuilder.go
  • pkg/server/handler/optimizor/plan_replayer_test.go

Comment thread pkg/executor/plan_replayer.go Outdated
Comment thread pkg/executor/test/planreplayer/BUILD.bazel Outdated
Comment on lines +250 to +315
<<<<<<< HEAD
=======
func TestExplainExploreReplayer(t *testing.T) {
ctx := context.Background()
tempDir := t.TempDir()
storage, err := extstore.NewExtStorage(ctx, "file://"+tempDir, "")
require.NoError(t, err)
extstore.SetGlobalExtStorageForTest(storage)
defer func() {
extstore.SetGlobalExtStorageForTest(nil)
storage.Close()
}()

store := testkit.CreateMockStore(t)
tk := testkit.NewTestKit(t, store)
tk.MustExec("use test")
tk.MustExec("create table t_explain_explore_replayer(a int, b int, key(a))")
tk.MustExec("insert into t_explain_explore_replayer values (1, 1), (2, 2), (3, 1)")
tk.MustExec("analyze table t_explain_explore_replayer")
tk.MustExec("create global binding using select * from test.t_explain_explore_replayer where b=1")
res := tk.MustQuery("plan replayer dump explain select * from test.t_explain_explore_replayer where b=1")
fileName := requirePlanReplayerFileToken(t, res.Rows())

loadStore := testkit.CreateMockStore(t)
loadTK := testkit.NewTestKit(t, loadStore)
replayerPath := filepath.Join(tempDir, replayer.GetPlanReplayerDirName(), fileName)
replayerPath = strings.ReplaceAll(replayerPath, "'", "''")
for range 2 {
rows := loadTK.MustQuery(fmt.Sprintf("explain explore replayer '%s'", replayerPath)).Rows()
require.NotEmpty(t, rows)
for _, row := range rows {
require.NotEmpty(t, row[3])
}
}
}

func TestPlanReplayerDumpPresignedURLOutput(t *testing.T) {
ctx := context.Background()
tempDir := t.TempDir()
storage, err := extstore.NewExtStorage(ctx, "file://"+tempDir, "")
require.NoError(t, err)
const presignedURL = "https://example.com/replayer.zip?X-Amz-Expires=3600&X-Amz-Signature=test"
extstore.SetGlobalExtStorageForTest(planReplayerPresignStorage{
Storage: storage,
url: presignedURL,
})
defer func() {
extstore.SetGlobalExtStorageForTest(nil)
storage.Close()
}()

store := testkit.CreateMockStore(t)
tk := testkit.NewTestKit(t, store)
tk.MustExec("use test")
tk.MustExec("create table t_presign(a int)")
tk.MustQuery("plan replayer dump explain select * from t_presign").Check(testkit.RowsWithSep("|",
"Download URL|"+presignedURL,
"Expires in|1h0m0s",
"Browser|Open the Download URL directly before it expires",
"curl|curl -L '"+presignedURL+"' -o plan_replayer.zip",
"Note|If the URL expires, rerun PLAN REPLAYER DUMP to get a new one",
))
require.Equal(t, presignedURL, tk.Session().GetSessionVars().LastPlanReplayerToken)
}

>>>>>>> 693e52cd51b (executor, planner: improve plan replayer dump URL output (#69439))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🔴 Critical | ⚡ Quick win

Remove the merge-conflict block before merging.

Lines 250-315 still contain unresolved conflict markers, so this test file cannot compile and neither of the new test cases will run until the conflict is resolved.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/executor/test/planreplayer/plan_replayer_test.go` around lines 250 - 315,
Resolve the unresolved merge conflict in plan_replayer_test.go by removing the
conflict markers and keeping the intended test additions. Ensure both
TestExplainExploreReplayer and TestPlanReplayerDumpPresignedURLOutput are
cleanly integrated into the test file with no leftover HEAD/branch markers so
the package compiles and the new plan replayer tests run.

@zeminzhou

Copy link
Copy Markdown
Contributor

/unhold

@ti-chi-bot ti-chi-bot Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 1, 2026
@zeminzhou

Copy link
Copy Markdown
Contributor

PTAL~ @D3Hunter @qw4990

@D3Hunter D3Hunter left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why cp to 2603, I thought this is used for starter?

@zeminzhou

Copy link
Copy Markdown
Contributor

why cp to 2603, I thought this is used for starter?

for preminue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/planner SIG: Planner size/L Denotes a PR that changes 100-499 lines, ignoring generated files. type/cherry-pick-for-release-nextgen-202603

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants