chore(codegen): add openapi-to-dart http client generator#1503
Draft
spydon wants to merge 11 commits into
Draft
Conversation
Explores generating the supabase-dart HTTP layer (Storage + Functions) from the shared Smithy models in supabase/sdk#51. Adds a small custom Dart emitter that consumes the committed OpenAPI artifacts and produces http-based clients with streaming upload, streaming response, multipart and per-request header injection. Off-the -shelf generators (dart-dio, dart) were rejected for lacking streaming and requiring build_runner. Part of SDK-1106.
Rename the package from supabase_codegen_spike, move it under packages/, add analysis_options, align README tables and drop em dashes.
- percent-encode path parameters (wildcards keep slashes), mirroring the storage client fix in #1479 - parse numeric response headers tolerantly instead of force-unwrapping - apply operation content-type after caller/default headers so it wins - wire streaming bodies via addStream so source errors are catchable - add tests for each fix - rewrite README to describe the package; drop spike references from code
…ranch The SDK branch (supabase/sdk#51) now includes a DatabaseService model. Generate a DatabaseApi from it to verify the emitter works against every model in the branch. Type query parameters by their schema instead of always String?, and add PostgREST transport coverage.
Replace hand-rolled StringBuffer scaffolding with package:code_builder for
class/field/constructor/method structure and format in-process with
package:dart_style, dropping the Process.runSync('dart format') call and the
manual dart:convert import heuristic. Generated output is equivalent and all
tests pass.
- exclude the tooling package from the SDK capability-matrix scan via the existing .sdk-parse-ignore mechanism (it is not part of the SDK surface) - model HTTP methods as an HttpMethod enum used via .name - spell out abbreviated identifiers in the generator
The generator needs a newer Dart than the oldest supported Flutter (its dart_style/code_builder deps), so bootstrapping it on Flutter 3.22 fails. Exclude it from the melos workspace like examples/launcher, and drop its analysis_options so the repo-wide DCM scan skips it (it is not bootstrapped). Also drop a redundant inferrable type argument in a storage_client test that the current DCM ruleset flags (pre-existing, unrelated to this spike).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Explores whether the mechanical HTTP layer of
supabase-dart(Storage, Functions, PostgREST) can be generated from the shared Smithy models in supabase/sdk#51, producing idiomatic Dart a maintainer would own long-term. Companion to the Swift spike (supabase/supabase-swift#1047).Everything lives in a self-contained
packages/supabase_openapi_generatorpackage so nothing touches the shipped SDK yet. The package README documents how to use it; this description holds the spike analysis.The generator is verified against every OpenAPI artifact currently in the SDK branch: Storage (18 ops), Functions (5), and the newly added PostgREST DatabaseService (7). All generated code formats and analyzes cleanly (
dart format,dart analyze, and DCM) and is covered by tests.Recommendation: adopt a small custom Dart emitter
It is the only option that meets all three hard constraints at once:
Stream<List<int>>, mandatory for TUS and large mobile uploads),build_runnerand nodart:io(so Web/WASM works).The emitter is
bin/generate.dart. It reads the committed OpenAPI artifacts and writeshttp-based clients intolib/src/generated/. It builds the Dart source withpackage:code_builderand formats it in-process withpackage:dart_style(both maintained by the Dart team), so the emitter has no hand-rolled brace/comma bookkeeping and no shelling out todart format. A thin hand-written runtime (ApiClient) owns transport and headers; generated code is pure request-building + response-decoding, mirroring the Swift split.Alternatives evaluated (why not an existing tool)
1. Input model -> Dart client generators
None of the off-the-shelf generators meet streaming-both-ways + no-
build_runner+ web/WASM together.smithy_codegen)smithyruntime is published but "experimental, internal use only";smithy_codegenis unpublished (publish_to: none); heavy toolchain (Java Smithy Docker -> AST -> Dart CLI -> build_runner); every client hard-depends on AWS packages (aws_common,aws_signature_v4). Good as a reference only.stream.toList())dartapi_client.mustacheimportsdart:io)dart-diobuilt_value+dio)swagger_dart_code_generatorretrofit+diotonik(pure-Dart CLI, newest)The two capabilities almost nobody exposes are the two we need most: a
Stream<List<int>>request body (only OpenAPI Generator'sdartdoes it, upload-only, and that generator breaks web viadart:io) and a streaming response (onlyretrofit, which needsbuild_runnerand can't stream requests).package:httpgives both for free viaStreamedRequestandClient.send(), is web/WASM-safe, and needs nobuild_runner— which is why we generate against it.2. Dart code-emission library (how the generator writes Dart)
For emitting the Dart source itself, the standard is
package:code_builder+package:dart_style(used bysource_gen,json_serializable,freezed). This spike uses them:code_builderfor class/field/constructor/method structure and automatic import handling,dart_style'sDartFormatterfor in-process formatting. Procedural method bodies are still supplied as code blocks (code_buildermodels structure, not arbitrary statement ASTs), so it is a hybrid, but the scaffolding is no longer hand-concatenated strings.What's included
Reproduce:
Spike questions
http.StreamedRequest, proven (TUSuploadChunk). Off-the-shelf can't.StreamedApiResponsewrappingresponse.stream, proven.http.MultipartFile(field, stream, length), proven.HeaderProvider, proven with a token that changes between calls.http, nodart:io, no conditional imports.build_runnerrequired?dart run; output committed. Decisive advantage overdart-dio.PostgREST (question 6)
The branch now includes a DatabaseService model, so this was tested directly. The emitter produces a compiling
DatabaseApi, but it confirms the earlier assessment: only the transport benefits, the query builder stays hand-written. Every PostgREST operation models its query semantics as stringly-typed query params (select,order,filters,on_conflict, …) plus a raw octet-stream body. The generatedselectRows(...)is a thin transport method; the logic that builds.select()/.eq()/.order()into those params is exactly what codegen cannot express and must remain hand-written.Model gaps found (for supabase/sdk#51)
@streaming(currently plainBlob->format: byte). The emitter already streamsapplication/octet-streamresponses, but the trait makes intent explicit.octet-streamblobs, so generated methods take a rawStream<List<int>>body rather than a typed row model. Acceptable for a transport layer, but worth noting.ListBucketsResponseContent { items }) generate wrapper objects; the facade unwraps them.Long/Integerboth collapse to OpenAPInumber-> Dartnum. Works for JSON;intwould read better. Not a blocker.Code review
A high-effort review ran on the spike. The following correctness fixes were applied and each has a test:
/), mirroring the storage fix in fix(storage): percent-encode object paths in request URLs #1479.Upload-Offset).addStreamso a source error is catchable.int? limit) instead of alwaysString?— now exercised by PostgREST.DartFormatter(the emitter no longer shells out todart formatwith an unchecked exit code).Out of scope
Auth is not yet in
sdk#51; Realtime is incompatible with REST codegen; wiring the generated clients into the realstorage_client/functions_client/postgrestpackages is deferred to keep the spike self-contained.