improvement(tables): harden pagination for short pages and surface name-validation errors

[TheodoreSpeaks]

Summary

• grid pagination now terminates on an empty page or a covered totalCount instead of rows.length < pageSize, so a short server page can never be misread as end-of-table (prep for byte-bounded pages / larger rows). Termination logic lives in a new pure module hooks/queries/utils/table-rows-pagination.ts shared by getNextPageParam and the use-table drain loops
• offset fallback resumes at the actual loaded-row count (not pages × pageSize); drain loops get a fail-fast no-progress guard
• validateRowSize measures UTF-8 bytes (Buffer.byteLength) instead of UTF-16 code units — a "400KB" CJK row was actually ~1.2MB on disk/wire
• dev-preview TABLE_MAX_PAGE_BYTES env (off by default): server cuts row pages at a byte budget, always keeping ≥1 row. Used to exercise the client termination end-to-end; the production SQL-side cut is a follow-up
• lifted the 10K-char per-string-cell limit — string cells are now bounded by the row byte cap alone, same as json cells (it was an inception-era constant with no hard constraint behind it; sorts/filters/unique checks all handle large values, and large text previously just moved to json columns as a workaround)
• renaming/creating a table with an invalid name now shows the validation message as a toast instead of silently reverting the field

Type of Change

• Improvement

Testing

New pure-fn test suites (pagination termination, byte trim) + rewritten use-table tests incl. first coverage of ensureRowsLoadedUpTo and the short-page-continues regression case. tsc, lint:check, check:api-validation:strict, check:react-query, check:client-boundary all pass. Manually exercised locally against a seeded 60×400KB-row table with the byte cut enabled.

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		Jul 3, 2026 5:29pm

[cursor]

PR Summary

Medium Risk
Pagination and drain-loop behavior affect bulk export, clipboard, and full-table reads; incorrect termination could miss rows or over-fetch, though coverage is strong in new tests.

Overview
Table row pagination no longer treats a page shorter than the requested size as end-of-table. Termination is centralized in table-rows-pagination (hasMoreTableRows / getNextTableRowsPageParam): stop on an empty page or when page-0 totalCount is fully loaded; offset/keyset continuation uses the loaded row count so byte-trimmed pages resume without gaps. use-table drain loops share that logic and fail fast if fetchNextPage resolves without appending a page (e.g. cancelQueries races).

Server (dev-preview): optional TABLE_MAX_PAGE_BYTES trims list responses via trimRowsToByteBudget (always keeps ≥1 row). Validation: row size uses UTF-8 bytes; per-column max string length is removed in favor of the row byte cap.

UX: create/rename table mutations toast the first validation issue (e.g. name pattern) instead of swallowing validation errors on rename/create.

^{Reviewed by Cursor Bugbot for commit 3b412a9. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

This PR hardens table row pagination by replacing the rows.length < pageSize termination heuristic with a count-based approach (hasMoreTableRows), extracting the logic into a shared pure module and adding fail-fast no-progress guards to the drain loops. It also fixes validateRowSize to measure UTF-8 bytes via Buffer.byteLength instead of UTF-16 code units, adds a dev-preview TABLE_MAX_PAGE_BYTES byte-budget cut for exercising client termination, and surfaces table name-validation errors as toasts instead of silently reverting the field.

• Pagination termination: new hasMoreTableRows / getNextTableRowsPageParam utilities in hooks/queries/utils/table-rows-pagination.ts are shared by getNextPageParam and both useTable drain loops; offset fallback now uses actual loaded-row count rather than pages × pageSize, closing a gap on short server pages.
• Byte measurement fix: validateRowSize and trimRowsToByteBudget both use Buffer.byteLength(JSON.stringify(...)) for UTF-8–correct size checks; a CJK row that was previously ~1/3 its real size in the old string.length count is now measured correctly.
• Error surfacing: useCreateTable and useRenameTable onError handlers now call extractValidationIssues(error)[0]?.message ?? error.message so the first validation issue is shown as a toast rather than silently dropped.

Confidence Score: 5/5

Safe to merge — all production paths are additive or corrective fixes with no regressions introduced.

The pagination termination rewrite is well-reasoned: the new count-based rule is strictly more correct than the old page-fullness heuristic, the shared utilities are pure functions with comprehensive unit tests, and the drain loops gain explicit no-progress guards. The UTF-8 byte fix closes a real undercount for multi-byte content, and the validation toast change strictly improves user feedback. The dev-preview byte-cut is disabled by default and isolated behind a null-check.

No files require special attention. The acknowledged stale-low totalCount early-exit in hasMoreTableRows and the per-row envelope exclusion in trimRowsToByteBudget are both explicitly documented and accepted as pre-production trade-offs.

Important Files Changed

Filename	Overview
apps/sim/hooks/queries/utils/table-rows-pagination.ts	New pure module providing countLoadedTableRows, hasMoreTableRows, and getNextTableRowsPageParam; termination logic is well-documented and covered by a dedicated test suite.
apps/sim/app/workspace/[workspaceId]/tables/[tableId]/hooks/use-table.ts	Both drain loops updated to use hasMoreTableRows; fail-fast no-progress guard added; hasMore return value now also accounts for hasMoreTableRows(pages).
apps/sim/hooks/queries/tables.ts	getNextPageParam delegates to getNextTableRowsPageParam; validation error toasts now surface the first issue message via extractValidationIssues for both create and rename mutations.
apps/sim/lib/table/rows/paging.ts	New trimRowsToByteBudget helper; measures only row.data bytes with documented headroom requirement, always keeps ≥1 row for forward progress.
apps/sim/lib/table/validation.ts	validateRowSize corrected from UTF-16 string.length to Buffer.byteLength — fixes undercount for CJK/emoji content.
apps/sim/lib/table/rows/service.ts	queryRows now applies trimRowsToByteBudget when TABLE_MAX_PAGE_BYTES is set; off by default; no change to production path.
apps/sim/lib/table/constants.ts	Added getMaxPageBytes() reading the new TABLE_MAX_PAGE_BYTES env var; returns null when unset or zero.
apps/sim/lib/core/config/env.ts	Added optional TABLE_MAX_PAGE_BYTES numeric env var; correctly optional and integer-validated.

_{Reviews (2): Last reviewed commit: "fix(tables): align getNextPageParam test..." | Re-trigger Greptile}

[TheodoreSpeaks]

Addressed review notes:

• trimRowsToByteBudget TSDoc now states the budget counts data only (envelope excluded — callers leave headroom; the production SQL cut must account for the same overhead)
• hasMoreTableRows TSDoc now documents both staleness directions precisely: stale-high self-corrects via the empty-page rule; stale-low stops the drain early, accepted because the view is already stale and run-stream/interval invalidations refetch

CI failure was hooks/queries/tables.test.ts still asserting the old rows.length < pageSize termination (and passing an empty allPages, which the new implementation reads) — rewritten to the count-based semantics incl. the short-page-continues regression case.

@greptile review