feat: checkpoint requests

[stevensJourney]

Overview

This implements the core changes for the Checkpoint Requests proposals:

• https://github.com/orgs/powersync-ja/discussions/324
• https://github.com/orgs/powersync-ja/discussions/317

The proposals refactor write checkpoints into a general Checkpoint Requests methodology: clients
now generate and track checkpoint request ids themselves (previously the PowerSync service
generated write checkpoint ids). The service treats client-generated checkpoint request ids as the
write checkpoint ids it reports to older-protocol clients, so both flows share one id namespace.

For background on the legacy system, see docs/historic-write-checkpoints.md. For a detailed
overview of the new system, see docs/write-checkpoint-requests.md. The powersync_control
command and instruction reference in docs/sync.md has been updated accordingly.

Migrations

Write checkpoint state was previously tracked in a synthetic $local bucket in ps_buckets.
Migration v14 moves this state into dedicated ps_kv keys:

Legacy ($local)	New (ps_kv)
last_applied_op	last_applied_checkpoint_request_id
last_op	last_seen_checkpoint_request_id
target_op	local_target_op

The migration then deletes the $local row (so ps_buckets only contains real sync buckets) and
drops the ps_buckets.target_op column, so older SDKs fail hard rather than silently diverge if
they open a migrated database. A full down migration restores the v13 schema and rebuilds the
$local row from ps_kv; re-upgrading after a downgrade treats the $local row as the source of
truth, including any progress an older SDK made in between. A read/write audit confirmed the mapped
ps_kv usages are semantically identical to the legacy $local bookkeeping.

A new last_requested_checkpoint_request_id key holds the client-side allocation counter. It is
intentionally not seeded from migrated state: SDKs reconcile the counter with the service on every
connect (see below). See the "Migration from $local" section in
docs/write-checkpoint-requests.md for details, including the ambiguous max-op-sentinel case.

Additions

All new functionality is exposed through powersync_control, consistent with the rest of the sync
interface:

• next_checkpoint_request_id — allocates and returns the next checkpoint request id in a
  CheckpointRequestId instruction. Requires an active sync iteration and seeded request state.
• local_target_op — probes, updates, or clears (payload 0) the local target op used to
  block applying downloaded rows while local writes are outstanding. Returns the previous value in
  a LocalTargetOp instruction. Works outside a sync iteration. This is the split that keeps
  explicit checkpoint requests from blocking applies: only write checkpoints update the apply gate.
• seed_checkpoint_request_id — seeds the allocation counter from service state.

Seeding and reconciliation

EstablishSyncStream now carries a last_checkpoint_request_id hint. On every connection attempt,
the SDK reconciles this hint with the service's checkpoint-request state (posting the effective id
to the service) and seeds core with the response. Core stores seeded values verbatim — it does
not enforce monotonicity. The SDK owns id validity and cannot allocate new requests until seeding
completes. This connect-time reconciliation doubles as a re-request, so checkpoint-request waiters
resolve in every new session without durable applied state in core.

Notifying applied checkpoint requests

When a full checkpoint carrying a write_checkpoint is applied locally, core emits a
CheckpointRequestApplied { request_id } instruction. SDKs resolve requestCheckpoint()-style
waiters by watching for an instruction with request_id >= N. Partial (priority) checkpoint
completions never emit this instruction and never advance the seen/applied state.

A general SDK requestCheckpoint flow:

1. Allocate an id: powersync_control('next_checkpoint_request_id', NULL) (in a transaction).
2. Post the id to the service's checkpoint request endpoint.
3. For write checkpoints only, set the apply gate: powersync_control('local_target_op', id).
4. Wait for a CheckpointRequestApplied instruction with request_id >= id.

Local writes set local_target_op to the max-op sentinel and clear the seen/applied high-water
marks, preserving the legacy behavior that only checkpoint request ids observed after a write can
satisfy the apply gate.

Testing

• Dart tests cover the new control commands, seeding semantics (including verbatim/non-monotonic
  stores), apply-gate behavior, and instruction emission for full vs. partial checkpoints.
• Migration tests cover up, down, and re-upgrade-after-downgrade paths, including sentinel target
  ops and schema-equality assertions against fixtures.

Related PRs for the PowerSync service and the initial Swift SDK implementation will be linked here.

Historical context and alternatives considered

Alternatives considered

• Standalone SQLite functions — the initial implementation exposed
  powersync_next_checkpoint_request_id() and powersync_probe_local_target_op() as dedicated
  SQLite functions, e.g.:

  let requestId = try await db.writeTransaction { ctx in
      try ctx.get(sql: "SELECT powersync_next_checkpoint_request_id()", parameters: []) { cursor in
          try cursor.getInt64(index: 0)
      }
  }

  These were replaced with powersync_control commands so the checkpoint request lifecycle lives
  in the same interface (and connection) as the rest of the sync client, and responses flow through
  the shared instruction serialization.

• SyncStatus as the notification stream — an earlier revision emitted a
  last_applied_checkpoint_request_id field on the SyncStatus stream, letting SDKs reuse
  existing status watchers to wait for checkpoint requests. This was replaced with the dedicated
  CheckpointRequestApplied instruction: it keeps an internal high-water mark from being presented
  as meaningful sync progress, and avoids coupling waiters to status serialization.

• max() seeding in core — seeding originally used max(local, service) semantics so the
  counter could never move backwards. This was dropped in favor of storing seeded values verbatim:
  the SDK performs the bidirectional reconciliation with the service and is the only party that can
  judge which value is correct (e.g. after switching users or a service-side reset). The same
  reasoning applies to last_applied_checkpoint_request_id, which is always the id from the last
  applied checkpoint as sent by the service.

• Single JSON value vs. separate ps_kv keys — the migrated state could have been stored as
  one JSON document under a single key. Separate keys were chosen to keep querying simple (both for
  core's apply-gate SQL and for debugging).

• Seeding the request counter from migrated target_op — a concrete legacy $local.target_op
  could seed last_requested_checkpoint_request_id during migration. This was left out as
  redundant: SDKs reconcile the counter with service state on connect before advancing it.

• Keeping the target_op column — the column could have been retained for compatibility.
  Dropping it was chosen deliberately so that older SDK versions fail with a hard SQLite error on
  local writes instead of silently maintaining state the new implementation no longer reads. The
  down migration restores the column for supported downgrades.

---

AI Disclosure: I initially implemented the work by hand without AI. Codex 5.5 assisted with
creating tests and writing docs; Claude Code assisted with a review pass, cleanups, and doc
updates. All AI changes have been manually guided and verified.

[rkistner]

I'm happy with the model overall, but I'll leave review of the technical implementation to @simolus3.

Reading the docs, one part that's not 100% clear to me is seed_checkpoint_request_id:

1. When exactly is this meant to be called? seed typically implies it's during setup only, or perhaps once per connection. Or is that potentially after every request to the server? Describing the SDK implementation like for other operations would help a lot here.
2. Are there concurrency risks here? For example: (1) client sends 3x requests to the server, each incremental last_checkpoint_request_id (2) client receives a response back from the server, calls seed_checkpoint_request_id, which then sets it back to an older value. Current docs mention the client SDK is responsible for reconsiling these, which can include dealing with concurrency issues like that, but the docs should be clear on what the client SDK should do.

On the docs: It's nice to have all the details, and the SDK implementation descriptions are really useful. But some of the other docs are quite verbose and still difficult to follow: It describes a lot of detailed mechanics, but I'm missing the high-level purpose. For example, it took some digging to see what exactly each of the ps_kv fields are used for, and why we need the 4x separate fields.

[rkistner]

Would something like target_checkpoint_request_id be a better description here? target_op originally referred to a concept for buckets that is not really relevant on the client anymore.

[stevensJourney]

1. When exactly is this meant to be called? seed typically implies it's during setup only, or perhaps once per connection. Or is that potentially after every request to the server? Describing the SDK implementation like for other operations would help a lot here.

This currently happens once per connection. This is used for the case where the PowerSync service might have a checkpoint record, but the client doesn't - perhaps after a disconnectAndClear.

Are there concurrency risks here? For example: (1) client sends 3x requests to the server, each incremental last_checkpoint_request_id (2) client receives a response back from the server, calls seed_checkpoint_request_id, which then sets it back to an older value. Current docs mention the client SDK is responsible for reconsiling these, which can include dealing with concurrency issues like that, but the docs should be clear on what the client SDK should do.

We limit the concurrency on the SDK level currently. Essentially, we do 1 request to the service on each connect iteration. We wait for the response from the server before allowing other checkpoint-requests from executing. The client's sequence is then seeded from the server response. Subsequent checkpoint requests then use the client's sequencing.

On the service side, the PowerSync service avoids concurrency issues by only advancing the checkpoint_request_id if it's increasing. Clients then receive the larger (potentially service side state) response - which they wait for.

On the docs: It's nice to have all the details, and the SDK implementation descriptions are really useful. But some of the other docs are quite verbose and still difficult to follow: It describes a lot of detailed mechanics, but I'm missing the high-level purpose. For example, it took some digging to see what exactly each of the ps_kv fields are used for, and why we need the 4x separate fields.

That's fair feedback. I'll use this to improve the docs.

[simolus3]

I agree that more information on checkpoint request seeding and reconciliation would be helpful, the docs say that this is owned by SDKs but having an overview of the process and more details on the "recommended pattern" part would help.

[simolus3]

I wonder if we could inline this into DidCompleteSync?

[simolus3]

CheckpointRequestId and LocalTargetOp aren't really instructions, they're more like a response to a function call, right? Adding them to this enum kind of blurs the types (and this is also visible in the Swift SDK implementation where these two are handled specially). Apart from EstablishSyncStream, which arguably has the same problem, instructions are meant as requests that can appear at any point in the sync process.

This could use some restructuring still, but overall I think the control flow would be cleaner if we could invoke this directly in control() like e.g. this:

"next_checkpoint_request_id" => {
    let client = state.sync_client.borrow();
    let has_sync_iteration = client
        .as_ref()
        .map(|e| e.has_sync_iteration())
        .unwrap_or(false);

    if !has_sync_iteration {
        return Err(PowerSyncError::state_error("No iteration is active"));
    }
    let adapter = state.storage_adapter(db)?;
    if !adapter.has_checkpoint_request_id()? {
        return Err(PowerSyncError::state_error(
            "Checkpoint request state has not been seeded",
        ));
    }

    let request_id = adapter.next_checkpoint_request_id()?;
    ctx.result_int64(request_id);
    return Ok(());
}

I don't think there's a need to tick the sync client here, the subscriptions command is handled in a similar way for example.

(I see how my earlier suggestion to move these into powersync_control calls probably prompted this, sorry for not being clearer there)

[simolus3]

What's the motivation for allowing text values, JavaScript compatibility? I'd hope we could handle that with big ints, but if we want this for safety then a small comment might help :)

[simolus3]

Nit: We don't need comments to describe what code no longer does ;) I also see a few more references to $local, e.g. in collect_bucket_requests and comments in crud_vtab.rs and raw_table.rs. These should probably be removed too?

[simolus3]

I don't really understand this part yet, but IMO if we won't have SDKs seeding with NULL then it shouldn't be supported by the core extension.

[simolus3]

Also, I wonder if it makes sense to move that to the sync status as a hidden field only used internally in SDKs.

For our SDKs using web workers where it's not that trivial to call methods on the sync client directly, I'm kind of hoping the implementation of waitForSync after acquiring a request id in SDKs could be something like:

await for (final snapshot of syncStatus) {
  if (snapshot.internalAppliedRequestId >= requestId) return;
  if (snapshot.error case final error?) throw error;
}

(but there are probably a lot more cases I'm missing here)