test(rest): add fake server for REST scan planning#1494
Conversation
tanmayrauth
left a comment
There was a problem hiding this comment.
Mechanics are solid — this drives the real planning client, not a stub. The gap is coverage: the fake already models the fetch-path 404s but skips a few production error paths it's built to exercise. Details inline.
@tanmayrauth Let me know if any changes needed. |
zeroshade
left a comment
There was a problem hiding this comment.
Review — fake REST scan-planning server
Test-only change (4 files, +2349/-0). Drives the real rest.Catalog scan-planning client (phases 0–5) against an in-process fake. I pulled the branch and ran the full gate locally in addition to CI.
Verification
| Check | Result |
|---|---|
go build ./... |
exit 0 |
go vet ./catalog/rest/... |
clean |
gofmt -l (4 files) |
clean |
golangci-lint v2.8.0 (pinned) on planfake + rest |
0 issues |
go test -race internal/planfake |
PASS |
go test -race -run TestPlanFake ./catalog/rest/ |
11/11 groups PASS |
go test -race ./catalog/rest/... |
PASS (no regressions) |
git diff --check |
clean |
Wire compatibility checks out: endpoint.String() renders "METHOD PATH" byte-identical to the fake's endpoint constants, and endpointFromString round-trips them, so the capability-discovery tests exercise the real parse path (scenario.go:34-37 ≡ endpoints.go:108-111).
What's strong
- Independent wire fixtures. The fake uses hand-authored JSON rather than the production wire structs (
scenario.go:18-20,:30-32), so a struct-tag regression can't silently update both sides of a test.TestPlanFakeIndependentWireFixtureContractasserts the fixture shape on its own. This directly targets the epic's highest-risk concern (silent wire drift). - Opaque-token routing.
parseRoute/pathSegmentssplitEscapedPath()before per-segmenturl.PathUnescape, so plan IDs with encoded slashes (a%2Fb) and unit-separator namespaces (db%1Fanalytics) survive as single decoded segments — matching real server semantics. - Shutdown semantics.
panic(http.ErrAbortHandler)avoids synthesizing phantom200s on close, andClose()shuts the underlyinghttp.Serverbeforehttptest.Server.Close()so a handler blocked inio.ReadAllcan't hang cleanup. Both are covered (TestServerGatedResponseAbortsOnServerClose,TestServerCloseAbortsHandlerBlockedReadingRequestBody). - Concurrency. Single mutex + close/recreate broadcast channel for
WaitForPollCount, with the count incremented before a gated wait so observers can synchronize. A 50-goroutine race test confirms it. - Defensive copying of scenario/requests/headers/bodies throughout, with mutation tests.
- Fixtures document Java
ContentFileParserconventions (typed partition values, hex-encoded binary bounds).
One scope point (non-blocking)
The epic's Phase 6 sketch (#1178) described the fake as "implementing the four endpoints over an in-memory table, running the same local planning code server-side (so parity tests are meaningful)", plus a RUN_INTEGRATION_TESTS=1 suite against the Java iceberg-rest-fixture image.
This PR instead ships a scenario-driven mock (canned JSON), which is arguably a better fit for client conformance — it gives precise control over retries/503 backoff/terminal statuses/expiry/fanout/idempotency headers. Two consequences worth tracking as follow-ups so the epic's acceptance criterion isn't lost:
- The local-vs-remote parity test isn't enabled by this fake (it doesn't actually plan).
- Without the Java-fixture integration suite, nothing automatically guarantees the hand-authored fixtures match real Java output (mitigated here by authoring to Java conventions).
Both are consistent with the PR's stated scope, and #1178 explicitly invites refining the phase breakdown — just want to confirm they're captured as follow-up issues.
Nits (optional)
server_test.go:514-518: thewg.Add(1)/go func(){ defer wg.Done() … }loop could use Go 1.25wg.Go(...)(gopls modernize hint; not flagged by golangci-lint).nextResponseincrements the per-key count even for missing/exhausted keys — intentional and documented ("requests received"), noting only so it isn't mistaken for a bug.
Nicely done — thorough, idiomatic, and well-tested.
zeroshade
left a comment
There was a problem hiding this comment.
LGTM. Test-only change; all CI green (rat/license, build, ubuntu+macos on go1.25/1.26, Spark 3.5 and 4.0 integration). Verified locally with -race tests, golangci-lint v2.8.0 (0 issues), go vet, gofmt, and git diff --check — all clean. Detailed notes in the prior review comment.
Part of #1178.
Summary
Scope
This is test infrastructure only. It does not change
Catalog.PlanFiles, scan-task decoding, planning modes, Java integration, or documentation.Testing
go test ./...go test -race ./catalog/rest/...go vet ./catalog/rest/...golangci-lint run --timeout=10mgit diff --check