Skip to content

fix(pg_catalog): deterministic, stable OIDs for pooled clients (#389, #390)#391

Closed
kadosh1000 wants to merge 2 commits into
datafusion-contrib:masterfrom
kadosh1000:stable-oids
Closed

fix(pg_catalog): deterministic, stable OIDs for pooled clients (#389, #390)#391
kadosh1000 wants to merge 2 commits into
datafusion-contrib:masterfrom
kadosh1000:stable-oids

Conversation

@kadosh1000

Copy link
Copy Markdown

What

Makes pg_catalog OID assignment deterministic and stable, fixing two problems that break pooled PostgreSQL clients (e.g. DBeaver) against a datafusion-postgres server.

Closes #389 — OIDs unstable across connections.
Closes #390pg_catalog namespace should be the fixed OID 11.

Problem

PgCatalogSchemaProvider assigns catalog/schema/table OIDs from a per-provider AtomicU32 counter, incremented in iteration order. A typical pgwire server creates a fresh SessionContext + setup_pg_catalog per connection, so the same object gets a different OID on different connections. Pooled clients resolve an OID on one connection and reference it on another, so pg_class WHERE relnamespace = <oid> matches nothing and no tables are listed.

Separately, pg_catalog receives a generated OID, but the static catalog rows this crate ships reference the canonical PG_CATALOG_NAMESPACE OID 11 (pg_type.typnamespace = 11, pg_proc.pronamespace = 11, …). No namespace with OID 11 exists in pg_namespace, so joins from a system object to its namespace fail — e.g. DBeaver can't resolve any column's data type (every column shows as unknown).

Fix

Add stable_oid(&OidCacheKey): a deterministic hash (DefaultHasher — fixed keys, so stable across processes) of the object's (catalog, schema, table) name, mapped into the positive int4 range [16384, i32::MAX], with pg_catalog pinned to its fixed OID 11. The seven counter sites in pg_class/pg_namespace/pg_database/pg_attribute now call stable_oid instead of oid_counter.fetch_add.

The same object now resolves to the same OID on every connection, independent of iteration order or which catalog table is queried first.

Notes

pg_catalog OIDs were assigned from a per-connection AtomicU32 counter in
iteration order, so the same object got a different OID on each connection.
Pooled clients (DBeaver, etc.) resolve an OID on one connection and reference it
on another, so lookups like `pg_class WHERE relnamespace = <oid>` returned
nothing and tables/columns didn't list. Separately, pg_catalog itself received a
generated OID while the static catalog rows reference the canonical namespace
OID 11 (pg_type.typnamespace = 11, ...), so type<->namespace joins failed and
clients couldn't resolve any column's data type.

Replace the counter with stable_oid(&OidCacheKey): a deterministic hash of the
object's (catalog, schema, table) name, mapped into the positive int4 user-OID
range, with pg_catalog pinned to its fixed OID 11. The same object now resolves
to the same OID on every connection, independent of iteration order or which
catalog table is queried first.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@sunng87

sunng87 commented Jul 21, 2026

Copy link
Copy Markdown
Member

@kadosh1000 thanks for the quick fix. please check the lint failure.

stable_oid() made the counter unused; drop the field, constructor
param, and pass-throughs, and use a let-chain for the pg_catalog check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01La5HhazCArDHemjcZ7B4z8
@kadosh1000

Copy link
Copy Markdown
Author

@sunng87 no problem :) lint issues fixed

@sunng87

sunng87 commented Jul 21, 2026

Copy link
Copy Markdown
Member

@kadosh1000 Thank you! I'm trying to fix the same issue with another method that can reuse our oid counter mechanism. The root cause is some valid keys in oid_cache may be accidentally dropped. Let me try if I can get it fixed in that way.

@kadosh1000

Copy link
Copy Markdown
Author

Done in #393 #394

@kadosh1000 kadosh1000 closed this Jul 21, 2026
@kadosh1000

Copy link
Copy Markdown
Author

Closing — the maintainer fixed this upstream in #393 (pg_attribute OID eviction) and #394 (pg_catalog pinned to OID 11), released in datafusion-pg-catalog v0.18.2. Verified v0.18.2 resolves the pooled-client (DBeaver) issues on our side. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants